Configuring Lighthouse CI Budget Assertions in GitHub Actions

Make a GitHub Actions PR build fail when third-party scripts blow the performance budget, with a complete workflow and lighthouserc assertions.

A pull request added an analytics vendor and merged green, and only after deploy did Total Blocking Time and third-party transfer size regress — because nothing in CI was asserting the budget on pull requests.

Triage: Confirm the Regression Is Real and Reproducible Locally

Before touching workflow YAML, prove the regression exists and that Lighthouse can see it. A CI gate is worthless if it fires on noise, so reproduce deterministically first.

  1. Check out the merge commit that introduced the regression and build the site exactly as production does (npm ci && npm run build).
  2. Run Lighthouse CI locally against the built output with the same settings CI will use, so the numbers are comparable:
npx lhci autorun --collect.numberOfRuns=3
  1. Read the console summary. Note the observed total-blocking-time and the third-party-summary main-thread value, and open the uploaded report to see the per-vendor breakdown.
  2. Check out the parent commit (before the vendor was added), rerun, and diff the two. A real regression shows a consistent delta across all three runs, not a single-run spike.

If the delta only appears in one run, it is variance, not a regression — raise numberOfRuns and re-confirm before building a gate around it. Determinism at this stage is non-negotiable: a gate you cannot reproduce on demand will generate flaky red checks, contributors will learn to ignore or re-run it, and its authority evaporates. Pin throttlingMethod: "simulate" rather than "devtools" for CI, because simulated throttling models the network and CPU mathematically instead of relying on the runner’s live conditions, which vary with load on the shared GitHub-hosted machine.

One more triage decision: run Lighthouse against the built preview output, not the dev server. Dev servers ship unminified modules, disable long-term caching, and inject hot-reload clients — all of which inflate transfer size and blocking time and would make the third-party regression harder to isolate from first-party dev noise. Reproduce against npm run build && npm run preview so the bytes Lighthouse weighs are the bytes users would actually download.

Root Cause: No Assertion Gate Runs on Pull Requests

The regression merged because the repository had no job that ran Lighthouse assertions on the pull request. Lighthouse might have been run manually, or in a nightly job that reports but never fails, so a byte-heavy vendor addition had nothing standing in its way at review time.

There is an important distinction between collecting Lighthouse data and asserting on it. Many teams add a Lighthouse job that uploads a report and posts a score, then treat the score as advisory. A report that never returns a non-zero exit code cannot block a merge — it is documentation, not a gate. The lhci assert phase is what converts a measurement into a pass/fail verdict, and only that verdict, surfaced as a required GitHub status check, stops the regression. A nightly report also arrives after the fact: by the time it runs, the vendor is already on main and, often, already deployed. The gate has to run on the pull request, while rejecting the change is still cheap.

The fix is structural: run lhci autorun inside a GitHub Actions workflow triggered on pull_request, with an assert block and a budget.json that returns a non-zero exit code on breach. GitHub marks the check failed, and branch protection can require it before merge. The full budget model — the assert preset, budget.json structure, and the performance-budget audit — is covered in the governing guide, Enforcing Performance Budgets with Lighthouse CI; this page is specifically about wiring that gate into GitHub Actions. Set the actual threshold values from field data captured via RUM attributed to third-party vendors so the gate reflects real user impact rather than a synthetic guess.

The workflow has to solve a problem that is easy to miss in a first attempt: a single Lighthouse run is noisy enough that a naive assertion produces false failures, and a few false failures are all it takes for the check to be marked non-blocking.

One run is not a measurement Total blocking time across five repeated Lighthouse runs on an identical page varies from 150 to 260 milliseconds. A budget of 200 milliseconds would fail two of the five runs despite nothing having changed. The median of the five sits at 190 milliseconds and passes consistently, which is why the number of runs is a correctness setting rather than a speed trade-off. 200 ms 150 255 175 260 190 five runs, identical page, nothing changed median 190 Two of five runs fail a 200 ms budget on an unchanged page. The median passes every time. numberOfRuns is a correctness setting, not a speed one — three is the practical minimum.

Resolution: A Pull-Request Workflow That Fails on Budget Breach

Add two files. First, lighthouserc.json at the repository root, defining how to collect and what to assert:

{
  "ci": {
    "collect": {
      "numberOfRuns": 3,
      "startServerCommand": "npm run preview -- --port=4173",
      "startServerReadyPattern": "Local:",
      "url": ["http://localhost:4173/"],
      "settings": {
        "preset": "desktop",
        "throttlingMethod": "simulate",
        "budgetsPath": "./budget.json"
      }
    },
    "assert": {
      "preset": "lighthouse:no-pwa",
      "assertions": {
        "total-blocking-time": ["error", { "maxNumericValue": 200 }],
        "largest-contentful-paint": ["warn", { "maxNumericValue": 2500 }],
        "third-party-summary": ["error", { "maxNumericValue": 250 }],
        "performance-budget": ["error", { "minScore": 1 }]
      }
    },
    "upload": {
      "target": "temporary-public-storage"
    }
  }
}

Second, budget.json, which caps the third-party payload that regressed:

[
  {
    "path": "/*",
    "resourceSizes": [
      { "resourceType": "third-party", "budget": 200 },
      { "resourceType": "script", "budget": 350 }
    ],
    "resourceCounts": [
      { "resourceType": "third-party", "budget": 12 }
    ]
  }
]

Now the workflow at .github/workflows/lighthouse-ci.yml. This uses treosh/lighthouse-ci-action, which bundles Chrome and reads the same lighthouserc.json:

name: Lighthouse CI

# Run on every pull request targeting main, plus pushes to main
# so the baseline is always measured too.
on:
  pull_request:
    branches: [main]
  push:
    branches: [main]

# Cancel superseded runs on the same PR to save runner minutes.
concurrency:
  group: lhci-${{ github.ref }}
  cancel-in-progress: true

jobs:
  lighthouse:
    name: Assert performance budgets
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Set up Node
        uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm

      - name: Install dependencies
        run: npm ci

      - name: Build site
        run: npm run build

      - name: Run Lighthouse CI and assert budgets
        uses: treosh/lighthouse-ci-action@v12
        with:
          # Reuse the committed config so local and CI runs are identical.
          configPath: ./lighthouserc.json
          # Fail the job (and the PR check) when any assertion breaches.
          uploadArtifacts: true
          temporaryPublicStorage: true

The action inherits the non-zero exit code from lhci assert, so a breached third-party budget fails the lighthouse job. If you would rather not add an action dependency, the equivalent final step is a plain command — functionally identical because both run the same lighthouserc.json:

      - name: Run Lighthouse CI (no action wrapper)
        run: npx --yes @lhci/[email protected] autorun

A note on where Lighthouse collects from. The workflow above builds the site inside the job and lets lhci start its own preview server, which keeps every run hermetic and independent of deploy timing. The alternative is to point collect.url at a deployed preview URL — a Netlify or Vercel deploy preview for the pull request. That measures the real CDN, compression, and edge caching, which is more faithful, but it couples the check to the preview deploy finishing first and requires the URL to be discoverable at job time. For catching third-party byte regressions, the self-hosted preview is simpler and sufficient, because the vendor payload is identical regardless of who serves your first-party HTML.

If you outgrow temporary-public-storage (reports expire after a few days), stand up a Lighthouse CI server and set upload.target to lhci with a serverBaseUrl and token stored in secrets. That gives you historical trend lines so you can see third-party size drifting upward across many pull requests, not just whether a single build breached. Reference the token in the workflow via ${{ secrets.LHCI_TOKEN }} rather than committing it.

Finally, make the check mandatory. In the repository settings under Branches → Branch protection rules for main, enable Require status checks to pass before merging and select Assert performance budgets. Without this, the check can go red and a maintainer can still merge. Pair the required check with the treosh/lighthouse-ci-action GitHub App or a github-token input so the action posts the report link and per-metric deltas as a status directly on the pull request, giving reviewers the numbers inline instead of buried in job logs.

Where the reports are stored determines whether the check can compare anything. Three options exist, and they differ mainly in how much of your own infrastructure they require.

Where the reports live decides what you can compare Three options. Temporary public storage uploads each run and returns a link, needs no infrastructure, but keeps no history so no comparison against a base branch is possible. Build artefacts stored by the continuous-integration provider retain reports for a retention window and allow comparison against recent runs, at the cost of a download step. A self-hosted server stores every run indefinitely, enables full trend history and base-branch comparison, and is the only option that supports ratcheting budgets over time. temporary-public-storage no setup, returns a link no history → no comparison fine for a first week CI build artefacts kept for the retention window compare against recent runs needs a download step in the job LHCI server every run, kept indefinitely trends and base-branch diffs the only one that supports ratcheting Start on the left, but plan for the right: the delta column in the annotation depends on stored history. Third-party regressions are trend-shaped, and a trend needs somewhere to live.

Verification: A Failed Check Annotation on the PR

Confirm the gate works by regressing it on purpose. On a throwaway branch, add a heavy third-party tag (or lower the third-party budget to 1 KB) and open a pull request. Within a minute or two the Assert performance budgets check turns red, and expanding it shows the annotation from lhci assert:

Assertion failed: performance-budget failure
  Expected: minScore >= 1
  Found:    0.5
Assertion failed: third-party-summary failure
  Expected: maxNumericValue <= 250
  Found:    412  (main-thread ms)
Done running Lighthouse!
1 result(s) for http://localhost:4173/ failed

The check summary links to the temporary-public-storage report, where the third-party-summary audit lists the offending vendor by name. Revert the deliberate regression, push, and confirm the same check returns green — proving the gate distinguishes a real breach from a clean build.

What the check should say on the pull request

The annotation is the whole interface for most contributors, and a check that reports only pass or fail teaches nothing. Reporting the delta against the base branch turns the gate into a review aid rather than an obstacle.

Report the delta, not just the verdict A mock pull-request check annotation. It lists three metrics with the base branch value, the pull request value, the change and the budget: total blocking time rising from 180 to 340 milliseconds against a budget of 200, marked as the failure; third-party transfer size rising slightly from 410 to 425 kilobytes against a budget of 500, marked as within budget; and Largest Contentful Paint unchanged. A note records that showing unchanged metrics is what makes the changed one legible. Performance budget · 1 failing metric base this PR change budget ✗ total blocking time 180 ms 340 ms +160 ms 200 ms ✓ third-party bytes 410 KB 425 KB +15 KB 500 KB ✓ LCP 2.1 s 2.1 s — 2.5 s Showing the passing rows is what makes the failing one legible — a lone red line reads as flakiness. The base-branch column requires storing one report per commit on the default branch; it is worth the storage.

Common Pitfalls

  • Running only on push, never on pull_request. A push-only trigger measures main after the merge has already happened — too late to block anything. Trigger on pull_request so the assertion runs against the proposed change while it can still be rejected.
  • Leaving the check optional. A red Lighthouse job that is not a required status check is advisory only; contributors merge past it. Add it to branch protection, and gate hard byte/count budgets with error while leaving genuinely noisy timings like LCP on warn.
  • Asserting against a stale or missing budgetsPath. If settings.budgetsPath is absent, the performance-budget audit has nothing to evaluate and passes trivially, so the third-party size cap is silently unenforced. Confirm the path resolves and that budget.json is a top-level array.
  • Unpinned Chrome or action versions. Floating @latest on either @lhci/cli or treosh/lighthouse-ci-action lets a background Chrome update shift metric values, producing a red check that no code change caused. Pin the action to a major version (@v12) and the CLI to a minor (@lhci/[email protected]) so metric behaviour only changes when you choose to upgrade.
  • Skipping npm ci/npm run build before collect. Running Lighthouse against a dist/ from a previous job, or against an uninstalled tree, measures stale or broken output. Always install and build fresh in the same job so the assertion reflects the pull request’s actual code.

Frequently Asked Questions

Should the check block merges outright, or just warn?

Block, but scope what it blocks on. A required check that fails on any metric moving in the wrong direction will be routed around within weeks; a required check that fails only when a hard ceiling is crossed stays credible because every failure is real. Warn on deltas, block on ceilings.

Give the block an escape hatch that leaves a record — a label on the pull request that skips the gate and posts a comment saying who bypassed it and why. Teams need that exit for genuine emergencies, and making it visible is far better than the alternative, which is someone quietly setting the job to continue-on-error and nobody noticing for a year.

How do we keep the runner from being the source of variance?

Pin what you can and randomise nothing. Use a fixed runner image rather than a floating label, pin the Lighthouse version, and set an explicit throttling configuration instead of relying on the default, which changes between releases. Shared CI runners are noisy neighbours by nature, so the median across runs is doing most of the work here regardless.

Where variance remains unacceptable, move to simulated throttling rather than DevTools throttling: it is less faithful to a real device but far more repeatable, and for a regression gate repeatability matters more than absolute accuracy. The absolute number should come from your field data anyway.

Up: Enforcing Performance Budgets with Lighthouse CI

More in Performance Monitoring