Segmenting Third-Party Cost by Device Class

An averaged vendor cost hides the cohort that actually pays it. Segment by device capability so the number you budget against is the number your slowest quartile experiences.

Your per-vendor cost table reports a tag at 40 ms of main-thread time, the number looks acceptable against a 100 ms budget, and Core Web Vitals still reports a failing Interaction to Next Paint that nobody can attribute.

Triage: Look at the Distribution, Not the Mean

  1. Pull the raw per-vendor durations for one week rather than the aggregate your dashboard shows.
  2. Compute the median, the 75th and the 95th percentile for a single vendor.
  3. Compare the 75th against the mean. A 75th percentile several times the mean indicates a heavy upper range the average is hiding.
  4. Group the same data by navigator.hardwareConcurrency or by device memory and compare the groups.
One vendor, four device groups The same vendor script measured across four device groups. On high-end devices it costs about 18 milliseconds of main-thread time; on mid-range about 45; on low-end about 130; and on the slowest group about 240. The overall mean of 40 milliseconds sits close to the mid-range figure and describes none of the groups well, least of all the one that determines the seventy-fifth percentile metric. high-end 18 ms mid-range 45 ms low-end 130 ms slowest quartile 240 ms the mean you are budgeting against The mean describes no group. The metric you are scored on describes the two on the right.

Root Cause: Cost Is a Function of Device, and the Mix Is Not Uniform

Main-thread cost scales with device capability far more sharply than transfer size does. A script that parses and executes in 18 ms on a recent laptop can take ten times that on a mid-range Android device several years old, because parse, compile and execution are all CPU-bound and the gap between the fastest and slowest devices in real traffic is wider than most engineers’ intuition allows.

Averaging across that distribution produces a number that describes nobody. Worse, it is a number that moves with your traffic mix rather than with your code: a marketing campaign that brings more mobile visitors will raise your “vendor cost” without anything about the vendor changing, and a desktop-heavy week will lower it. Teams then chase a metric that is reporting demographics.

The Core Web Vitals thresholds are defined at the 75th percentile precisely to avoid this, which creates an unhelpful asymmetry: you are scored on a percentile and budgeting against a mean. Aligning the two — measuring, budgeting and alerting at p75, segmented by device class — is the whole intervention, and it usually reveals that one or two vendors account for most of the failing cohort’s cost.

Why an average moves without anything changing Two weeks with identical vendor code. In the first, sixty per cent of traffic is desktop and the averaged vendor cost reads thirty-two milliseconds. In the second, a campaign shifts the mix to seventy per cent mobile and the same code averages sixty-eight milliseconds. Nothing about the vendor changed; only the population did. A percentile computed within each device group is stable across both weeks. week 1 · desktop-heavy 60% desktop average: 32 ms looks fine week 2 · mobile-heavy 70% mobile average: 68 ms looks like a regression segmented p75 unchanged in both mid-range: 45 ms reports the code, not the mix The middle column will be investigated as a regression. Nothing regressed. Segmenting first removes an entire category of false alarm.

Resolution: Segment at Collection, Aggregate at p75

// Attach a device class to every sample, at collection time.
function deviceClass() {
  const cores = navigator.hardwareConcurrency ?? 0;
  const mem = navigator.deviceMemory ?? 0;          // GB, coarse and Chromium-only
  if (cores >= 8 && mem >= 8) return 'high';
  if (cores >= 4 && mem >= 4) return 'mid';
  if (cores > 0 || mem > 0) return 'low';
  return 'unknown';                                  // never merge this into a class
}

function sample(vendor, duration) {
  return { vendor, duration, device: deviceClass(), ts: Date.now() };
}
-- Aggregate at the percentile you are scored on, within each class.
SELECT vendor,
       device,
       COUNT(*)                                        AS n,
       APPROX_QUANTILES(duration, 100)[OFFSET(75)]     AS p75_ms
FROM   vendor_samples
WHERE  ts > TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY)
GROUP BY vendor, device
HAVING n > 500          -- below this the percentile is noise, not a measurement
ORDER BY p75_ms DESC;

Keeping unknown as its own class rather than folding it into a default is worth the extra row. The hints are Chromium-only, so unknown is largely Safari and Firefox traffic, and merging it into mid quietly attributes an entire browser population to a capability band nobody measured.

The HAVING clause matters as much as the percentile. A p75 computed over forty samples is not a percentile in any useful sense, and a dashboard that shows one per device class per vendor per day will be dominated by cells with too little data — which is how teams conclude that the metric is noisy and stop looking at it.

Verification: The Failing Cohort Becomes Visible

Re-run the aggregation and read the low and unknown rows. They should show materially higher figures than the overall average you started with, and the vendors at the top of that segment should be the ones your INP attribution has been pointing at. If the segments look identical, either the classification is not working or the cost genuinely is device-independent — which is possible for a pure network cost and rare for a script.

What the segmented table should tell you Three readings of the segmented table. A large gap between the high and low device classes confirms the cost is CPU-bound and the segmentation is working. Similar figures across classes suggest either a classification failure or a genuinely network-bound cost. And a low sample count in a class means the percentile is not yet meaningful and should be suppressed rather than displayed. large gap by class CPU-bound cost segmentation working similar across classes network-bound, or classification broken low sample count not a percentile suppress the cell Suppress rather than display thin cells. A misleading number is worse than a blank.

Budgeting Against the Segment That Matters

Once the table exists, the budget should follow it. A single site-wide ceiling per vendor is a compromise between cohorts that experience wildly different costs, and in practice it is set by whoever argues most persuasively rather than by the population it protects.

Set the ceiling against the low class, and treat the high class figure as informational. That inverts the usual dynamic in a useful way: a vendor that is cheap on fast devices and expensive on slow ones now fails the budget, which is the correct outcome, because the visitors on slow devices are the ones whose experience determines your field metrics and, frequently, your conversion rate.

It also changes what optimisation looks like. Against an averaged budget, the obvious lever is reducing bytes; against a low-class budget, the lever is reducing work — deferring execution, splitting a bundle, or moving computation off the main thread — because that is what the constrained cohort is actually short of. The two lead to different changes, and only one of them moves the metric you are scored on.

Common Pitfalls

  • Merging unknown into a class. It is largely a browser population, not a capability band, and merging it distorts both.
  • Segmenting after aggregation. A mean cannot be decomposed into per-segment percentiles afterwards. The device class has to be on the sample.
  • Using screen size as a proxy for capability. A large tablet can be slower than a small phone; use CPU and memory hints where available, and accept unknown where not.
  • Displaying thin cells. A percentile over a few dozen samples looks authoritative and is not.

Frequently Asked Questions

Are hardwareConcurrency and deviceMemory reliable enough?

They are coarse, capped, and available only in Chromium — and they are still far better than nothing for this purpose. deviceMemory is rounded to a small set of values and hardwareConcurrency reports logical cores rather than performance, so neither is a benchmark; what they give you is a stable ordering, which is all a segmentation needs.

For the browsers that expose neither, the honest approach is a separate unknown bucket rather than an inference. Some teams supplement with a short synthetic benchmark at first load, which is more accurate and costs main-thread time on exactly the devices that can least afford it — a trade that is rarely worth making for a monitoring refinement.

How many classes should we use?

Three plus unknown is usually right. Two hides too much — the interesting distinction is between mid-range and genuinely slow devices, and a binary split puts them together — while five or more fragments the sample count until several cells are too thin to read.

Whatever you choose, keep the boundaries fixed. Re-tuning the thresholds changes every historical figure and destroys your ability to compare across time, which is most of the value once the table has been running for a few months.

Should the segmentation include network as well as device?

For transfer cost, yes; for main-thread cost, it adds little. The two are largely independent — a fast phone on a poor connection has a network problem, and a slow phone on fibre has a CPU problem — so a single combined class conflates two different remedies.

Where you have the volume, segmenting by both and reading them as a grid is genuinely informative: it separates the vendors you should defer because they are heavy to download from the ones you should move off the main thread because they are heavy to run. Where you do not, segment by device for execution cost and leave network to your transfer-size budgets.

Does the same segmentation apply to transfer size?

It applies, and it matters much less. Bytes are bytes regardless of the device that receives them, so a vendor’s transfer size is roughly constant across capability classes and the segmented table will show four similar numbers. That is a useful null result: it confirms the classification is working and tells you the cost is network-bound rather than CPU-bound.

Where segmentation does help on the network side is by connection rather than by device. The same payload costs very different amounts of time on a fast fixed line and a congested mobile connection, and that distinction points at different remedies — fewer bytes and better caching for the first, deferral and origin consolidation for the second.

Up: Attributing RUM Data to Third-Party Vendors

More in Performance Monitoring