The 3am Phone Call You Don't Need: Quietly Measuring Script Failures in the Wild

There is a particular kind of dread that comes from a monitoring inbox at 3am. Not because something is definitely broken, but because something might be. A third-party widget failed for eleven people in Cornwall. A chunk 404'd for a browser you didn't know existed. An analytics script was blocked by a privacy extension and has been quietly sulking ever since.

At MonkeyLoader we like loaders that behave. But a loader is only as good as your ability to see when it isn't behaving. The good news is that you don't need a six-figure observability platform to find out. You need a small beacon, a bit of sampling and a firm opinion about what deserves a phone call.

Start with the failures you can actually act on

A two-person team cannot chase every error. So don't try. The first job is classification, not collection.

Write down the scripts your product genuinely needs: the checkout loader, the consent wrapper, the map that powers the store locator, the chunk that renders the account area. Those are critical. Everything else — analytics, chat widgets, heatmaps, the tag that exists because someone in marketing once said "attribution" — is secondary. It can fail without waking anyone.

That sounds obvious until you see a dashboard where a blocked social pixel has the same visual weight as a broken payment chunk. It doesn't. One is a Tuesday. The other is a revenue incident.

What counts as a failure around here?

Be precise, because "script error" is not a useful category. For our purposes, a failure is one of these:

• A first-party script or route chunk returns a non-2xx status or never fires load.
• A script loads but throws during initialisation before it exposes its expected API.
• A loader times out waiting for a dependency that should have arrived.
• A critical third-party widget fails while the rest of the page is healthy.

A blocked tracker is not the same thing. It is annoying, but it is often the user doing exactly what they intended. Track it separately if you must, but don't let it page anyone.

A beacon small enough to trust

You don't need to ship a full RUM SDK. A tiny beacon endpoint with a handful of fields will do more than a sprawling dashboard nobody reads.

function reportScriptFailure(detail) {
  const payload = JSON.stringify({
    script: detail.scriptId,
    kind: detail.kind, // 'chunk' | 'vendor' | 'loader'
    critical: detail.critical,
    message: detail.message?.slice(0, 160),
    route: location.pathname,
    conn: navigator.connection?.effectiveType || 'unknown',
    rtt: navigator.connection?.rtt || null,
    sample: window.__mlSample || 0
  });

  if (navigator.sendBeacon) {
    navigator.sendBeacon('/_beacon/script-fail', payload);
  } else {
    fetch('/_beacon/script-fail', { method: 'POST', body: payload, keepalive: true });
  }
}

Keep the payload boring. No user IDs, no free-text URLs with query strings, no stack traces longer than a tweet. You are looking for patterns, not a diary.

Sample like you mean it

Here's the counterintuitive bit: do not report every failure from every session. At even modest traffic, one broken vendor script on a slow mobile network will generate more noise than a supermarket self-checkout at 5pm.

We sample sessions at around 10%, but we always report failures from critical scripts. That gives us a manageable stream for the long tail and full coverage where it matters. If a critical chunk fails, we want to know even if the session wasn't sampled.

Set the sample flag once per session and store it. Don't roll the dice on every beacon, or your denominators will be nonsense.

The RUM fields that earn their keep

You can collect a lot. You should collect almost none of it. The fields that actually help are:

• Script identity — a short stable ID, not a URL.
• Criticality — critical or not. This drives alerting.
• Route — so you can spot a single page or template going wrong.
• Connection type and RTT — to separate a slow network from a broken loader.
• A control beacon — a tiny same-origin request sent at the same time. If the control fails too, the problem is the network or the edge, not the script.

That last one is the secret weapon. Without a control, every mobile failure looks like a script failure. With one, you can see whether the user's connection dropped or your chunk genuinely 404'd.

Blocked, borked or just slow?

A script that never loads might be blocked by an extension, blocked by a corporate proxy, blocked by a content policy, or simply slow enough that your timeout fired. In the browser you rarely get a clean reason. So use correlation.

If a third-party domain fails across a small, privacy-conscious slice of users while first-party scripts are fine, assume blocking. Log it, count it, move on. If a first-party chunk fails on one route, for everyone, at the same time, that's yours. Fix it.

A useful trick is to send a second beacon when the script eventually loads after a timeout. If late loads are common, your timeout is too aggressive. If they never happen, the script really is missing.

A UK mobile report is not a verdict

This is where a lot of small teams go wrong. They see a spike in script failures on a mobile network and conclude the loader is broken. Sometimes it is. Often it's a train going into a tunnel between Leeds and King's Cross, a congested cell in a city centre, or a rural broadband link doing its best impression of a piece of wet string.

Before you blame the loader, compare three things: the same script on Wi-Fi versus mobile, the same network in different regions, and the control beacon failure rate. If the control beacon is also falling over, your script is collateral damage. If the control is healthy and one chunk is failing only on a specific route, you have a real bug.

Also beware the Monday morning report. Weekend traffic is different, and a failure that only appears on Sunday evenings in Cornwall may be a local network issue rather than a code release. Look at the shape of the data, not just the peak.

Alerts for two people, not a NOC

Your alerting rules should be short enough to remember. Ours are:

• Page only when a critical script fails for more than 5% of attempts over 30 minutes, with at least 50 attempts.
• Page if the control beacon is healthy but a critical chunk failure rate jumps by more than 10 percentage points compared with the previous hour.
• Send a daily digest for everything else. Read it with coffee, not adrenaline.

Deduplicate by script, route and connection cohort. One broken chunk on one route should be one alert, not four hundred.

The weekly ritual

Once a week, spend fifteen minutes looking at the failures you didn't alert on. Not to fix them all — just to spot the ones that are becoming normal. A vendor script that fails for 2% of users every day is not an emergency, but it is a decision. Either accept it, replace it, or remove it.

That's the whole point of watching the loader. Not to build a cathedral of dashboards, but to know the difference between a noisy network and a broken promise. Your inbox will thank you. So will the person who isn't woken up at 3am.