Caching and revalidation

How it works

The web app's caching is declarative: a query records what it consumed (cache tags) as it runs, and invalidation follows the data. There is no central list of URLs to purge anywhere in the codebase — you never map a publish to a page by hand.

Two paths meet at a tag name.

The read path — a query registers its tags

Every content read goes through the model client's query(), which takes a cache descriptor as its third argument. Under cacheComponents the app wraps that client in a use cache boundary (apps/web/src/lib/content.ts): the descriptor's tags become cacheTag() calls and its profile becomes a cacheLife() timespan, so Next caches the computed result at the function boundary. The framework-free model client itself sets no fetch cache hints — the delivery token lives only in that module client and is never a cache-key argument.

The descriptors are not written inline at the call site. They live in one reviewed place — packages/content-app/src/lib/cache.ts — as the CONTENT_CACHE matrix, and each query names the surface it is:

// packages/content-app/src/news-listing/index.tsx
const result = await client.query(
  GetArticleCardsDocument,
  cardQueryVariables({ categoryId, tags }, page),
  CONTENT_CACHE.articleCards, // { tags: ["content:articlePage"], profile: CACHE_PROFILE.editorialListing }
);

A tag is a string in the shape content:{contentType} — the coarse tag for a whole content type — with slug-level precision for detail surfaces (content:articlePage:{slug}). See Tags & TTLs for the full matrix and the reasoning behind slug-level tags.

The invalidation path — a publish expires tags

When an editor publishes in Contentful, a webhook calls POST /api/revalidate. The handler derives cache tags from the publish payload's content type and calls Next's revalidateTag() for each. Any cached read carrying that tag is marked stale and refetched on the next request.

Because both sides speak the same tag vocabulary, the wiring is implicit: a listing query tagged content:articlePage refreshes the moment ANY article is published, without either side knowing about the other. That shared vocabulary is exactly why tags must come from the one CONTENT_CACHE matrix rather than inline literals — an ad-hoc tag on one side that the other never emits silently never invalidates.

Profiles are SWR + a self-healing floor

Each descriptor also carries a profile — a cacheLife { stale, revalidate, expire } timespan (seconds), defined in CACHE_PROFILE. Two things matter:

• stale is serve-stale-while-revalidating — for TIME-based expiry, the next request after the stale window serves the stale copy instantly and rebuilds in the background. (A webhook publish is different: it EXPIRES tags immediately, a hard miss that re-renders on demand — see The webhook.) A cross-replica, single-flight anti-stampede is a separable future enhancement; today the shared handler shares only the tag manifest, and entry storage stays per-instance.
• revalidate/expire are the self-healing floor, not the freshness mechanism — the webhook is. If a webhook is ever lost, the surface still catches up within its window. Treat it as the worst-case staleness bound, and the webhook as how content normally goes live instantly.

A second source: the Competition Engine

Contentful is not the only data behind a page. The basketball data areas read the Competition Engine through @v4/competition-engine, whose fetchers cache through their own use cache boundary (cachedEngineGet), tagged ce:* and lived by TTL classes. The engine's ingester calls the same POST /api/revalidate after a run, with a payload naming what it touched, and the route expires those ce:* tags. Live scores are never cached on the server. See Competition Engine data.

The next pages walk the webhook setup, the tag/TTL matrix, and the multi-replica Redis handler in turn.

The webhook

On-demand invalidation is driven by a single Contentful webhook that calls the app's revalidation endpoint on every publish. This page covers the endpoint, its shared secret, and the one-time Contentful setup.

The endpoint

POST /api/revalidate (apps/web/src/app/api/revalidate/route.ts) does six things:

1. Reads CONTENTFUL_WEBHOOK_SECRET from the environment. If it is unset, the route returns 503 — revalidation is simply not configured, and the app still serves cached content that self-heals on its TTL floor.
2. Verifies the request's x-revalidate-secret header against the secret using an HMAC + timingSafeEqual comparison (constant-time, so it does not leak the secret through response timing). A mismatch returns 403.
3. Derives the tags from the payload. A Contentful publish goes to tagsForPublish (below); a payload with "source": "competition-engine" is the engine ingester's post-run call and goes to tagsForEngineRun (Competition Engine data). When the derived list is empty — a publish with no content type (sys.contentType.sys.id), or an engine payload that names nothing — it returns 400 (no content type in payload).
4. Calls revalidateTag(tag, { expire: 0 }) for each — this clears Next's own cache (the origin).
5. Then awaits purgeEdgeByTags(tags) — this clears the Cloudflare edge, so a publish is fresh at the edge in seconds instead of waiting out s-maxage. It is a no-op unless the environment caches HTML at the edge and CLOUDFLARE_API_TOKEN + CLOUDFLARE_ZONE_ID are set, and it never throws (the origin revalidation already happened, so a failed edge purge must not fail the webhook). See apps/web/src/lib/cloudflare-purge.ts — purge-by-TAG matched to the Cache-Tag header (a shared block whose entry tag is not in that header coarse-purges the edge-cached surfaces). A publish therefore clears BOTH cache layers.
6. Returns the list of tags it revalidated as JSON.

The { expire: 0 } second argument is a Next 16 cache-life profile object that sets the tag's expiry to NOW — immediate expiration → on-demand re-render, which is the "expire now, refetch on demand" a CMS webhook wants. Do NOT use the string "max" here: it marks the tag stale-now but hard-expires it 1 year out (serve-stale + background revalidate), so the FE only refreshes on the surface's own revalidate window (120s) — the intermittent-staleness bug. A profile object also avoids the deprecation warning the bare single-arg form emits.

What tags a publish emits

The route delegates to a pure, unit-tested mapping — tagsForPublish in apps/web/src/lib/revalidation.ts — which composes:

• always the coarse content:{contentType} tag;
• for a chrome type — siteSettings, navigation, navigationItem, and the footer's footerBlock, sponsorGroup, imageLink, linkList and contactDetails — both chrome tags (content:siteSettings and content:navigation) together: a navigation or footer edit must refresh every page, so the two chrome surfaces re-read as a pair; and
• slug-level tags for articles (content:articlePage:{slug}, from the payload's localised slug field) and address tags for content pages (content:contentPage:{path}, from the localised Full Path, leading-slash as the route builds it), matching the precision the detail queries register in CONTENT_CACHE. A content page also keeps a content:contentPage:{slug} tag, which only the edge Cache-Tag of a top-level page carries; and
• the entry-id reference tag content:entry:{sys.id} for the published entry, so every page that embeds it (a content block, a linked article, a media entry, a content page a link leads to) is invalidated — the reverse index. Omitted for SHARED_TYPES (siteSettings, navigation, navigationItem, category), which keep their coarse/chrome tags to bound fan-out (Tags and TTLs has how the reference graph is built).

Because Next tags are exact-match, the slug tag is what evicts an article's OWN page: a publish refreshes both the listings (coarse tag) AND the changed article (slug tag) instantly, rather than leaving the detail page to its TTL floor. The route and the helper share one implementation, so a route-level test asserts the slug tag is emitted — the two cannot drift apart again. The route also logs a per-publish audit line ([revalidate] {tag} → N tag(s)) so a pathological reference fan-out is visible in the logs.

The secret

CONTENTFUL_WEBHOOK_SECRET is the shared secret on both ends — the env var the route reads and the header value Contentful sends. It is documented in SECRETS.md; set it in the app's environment for each deployment. Keep it out of the repo.

Provisioning the webhook — as code

The webhook is managed as code, not clicked together by hand. Its typed definition lives in packages/model/src/webhooks/revalidate-webhook.ts and is applied by an idempotent script:

CONTENTFUL_SPACE_ID=<space> CONTENTFUL_CMA_TOKEN=<cma> \
CONTENTFUL_WEBHOOK_SECRET=<secret> SITE_BASE_URL=https://<deployment> \
pnpm --filter @v4/model sync-webhooks -- --env=<environment>

It upserts by name (Publishing Hub · revalidate · <env>): created if absent, updated in place otherwise — it never touches webhooks it does not own. The definition is:

• URL: <SITE_BASE_URL>/api/revalidate
• Triggers: Entry.publish and Entry.unpublish (an unpublish must expire the same tags a publish does).
• Header: x-revalidate-secret carrying CONTENTFUL_WEBHOOK_SECRET (stored masked, secret: true).
• Filter: sys.environment.sys.id equals <environment> — one webhook per environment, so a develop publish never revalidates master.
• Payload: the default entry payload (the route reads sys.contentType.sys.id and the localised fields.slug and fields.fullPath), so no custom transformation is needed.

sync-webhooks runs at deploy time, not in CI — it mutates a real Contentful space, so it belongs with migrate (the model rule: Actions validate, the deploy mutates). In this site's Flightcontrol stack the web service is a Docker build, and it runs at the end of the service's BUILD_COMMAND (the env var Dockerfile.web runs), alongside sync-preview-urls, non-fatally. Development's:

"BUILD_COMMAND": "pnpm turbo run build --filter=@v4/web && (node packages/model/scripts/sync-webhooks.mjs; node packages/model/scripts/sync-preview-urls.mjs; true)"

Staging prefixes the model migration and its data steps (Deployment → The model deploy pipeline).

So each environment re-provisions its own webhook on every deploy.

Why the build, not postDeployCommand. The original design used postDeployCommand, but the post-deploy shell did not receive the runtime env vars — the same command provisions correctly from a laptop yet no-op'd in post-deploy because CONTENTFUL_CMA_TOKEN was absent, and a post-deploy failure is non-fatal so the deploy stayed green with an empty space. The build has every env var (includeEnvVariablesInBuild: true) and streams its logs, so a provisioning error is visible. node scripts/*.mjs runs directly (no pnpm --filter, which can no-op to exit 0 when a package isn't resolved).

Base URL: origin, not CDN. Both scripts resolve the base URL as LOAD_BALANCER_URL → SITE_BASE_URL → NEXT_PUBLIC_SITE_BASE_URL. The webhook (/api/revalidate) and the preview link (/api/preview) are dynamic route handlers that must reach the origin directly — behind a CDN (CloudFront), set LOAD_BALANCER_URL to the load-balancer host so provisioning targets it, and keep NEXT_PUBLIC_SITE_BASE_URL as the public/canonical URL.

Confirm against the SPACE, never the deploy status. Provisioning is non-fatal, so a deploy where it threw — e.g. with no base URL set, the scripts throw no site base URL on line one — still shows green Success (though the throw now prints in the build log). Verify with a CMA GET /spaces/<space>/webhook_definitions (and /preview_environments), then by publishing an entry and watching the webhook activity log list the JSON revalidated tags.

Doing it by hand instead? Add a webhook in the space with exactly the URL, triggers, header and filter above — the script is just the same definition, repeatable.

Tags & TTLs

Every cached content read names its surface in one reviewed matrix — CONTENT_CACHE in packages/content-app/src/lib/cache.ts. This is the single source of truth for both the tags a query registers and its cacheLife profile (the SWR { stale, revalidate, expire } timespan). A new query references a surface here; it never writes an inline tag literal, because the webhook has to emit the same strings for invalidation to connect.

The matrix

CACHE_PROFILE holds the four profiles: editorialDetail, editorialListing, slowVocab and chrome (site chrome — rare edits the webhook keeps fresh). The Competition Engine's reads have their own classes, outside this matrix — see Competition Engine data.

The chrome surfaces (content:siteSettings, content:navigation) are emitted together by the webhook rather than registered as one matrix row; see The webhook.

Slug-level precision, and why detail pages are the exception

Detail surfaces wear only their slug/path tag, deliberately not the coarse one:

// packages/content-app/src/lib/cache.ts
articleDetail: (slug: string): ModelQueryCache => ({
  tags: [`content:articlePage:${slug}`],
  profile: CACHE_PROFILE.editorialDetail,
}),

The reason is scale. If a detail query also carried the coarse content:articlePage tag, then publishing ANY single article would expire EVERY article's cached detail page at once — the coarse tag would drag them all out. With a slug-only tag, publishing one article expires only that article's page.

The accepted trade is that a detail page's related-content rail (cards for OTHER articles, fetched in the same query) catches up within its profile's revalidate window when one of those related articles changes, instead of instantly. Listings and rails keep the coarse tag and refresh immediately, as they must.

There is a matching rule for the invalidation side: the slug tags the webhook emits must be the same values the detail query registered — the query tags by its request key (slug/path), and the webhook derives the same from the payload. A mismatch is not harmless. A detail query wears no coarse tag, and a NOT-FOUND result holds no entry to carry a reference tag, so nothing ever expires it. Content pages had exactly this mismatch until it was fixed: the webhook emitted the slug while the query registered the Full Path, so a page just moved under a Parent Page kept showing the not-found page at its new address until its cache aged out. The webhook now emits the Full Path (content:contentPage:{path}).

Reference tags — updating shared content refreshes every page that uses it

The matrix above tags a page by its OWN surface. That is not enough when a page embeds another entry — a content block, a linked article, a media entry: the block's content is baked into the page's cache under the page's tag, so changing the block alone would leave those pages stale. So on top of the matrix tags, lib/content.ts's cachedQuery walks each query result and tags the cache entry with content:entry:{id} for every sys.id it embeds (lib/entry-ids.ts), and tagsForPublish emits content:entry:{id} for the published entry. Publishing any entry therefore invalidates every page whose cached data contained it — the reference graph, built automatically at the one cachedQuery chokepoint. A LINK is in that graph only when its target's sys.id is selected: LinkFields selects it for a content page, so a page linking to one follows it when a new Parent Page or Slug moves its Full Path.

Two guard rails ride with it:

• Fan-out bound (SHARED_TYPES). siteSettings, navigation, navigationItem and category are NOT entry-ID tagged — they are widely referenced, so a per-entry fan-out would herd; they keep their coarse/chrome tags instead.
• Fan-out is the current anti-herd lever, not single-flight. A tag invalidation is a hard miss, so a broad publish could stampede rebuilds. Today that is bounded by SHARED_TYPES (above) and the 128-tag cap (below); a cross-replica single-flight lock is a separable future enhancement, not yet implemented — the shared handler shares only the tag manifest.
• The 128-tag ceiling is respected: boundedEntryIds dedups and caps, warning on an over-budget detail page; listings degrade safely to their coarse tag.

Profiles: SWR window + self-healing floor

Each row names a CACHE_PROFILE — a cacheLife { stale, revalidate, expire } triple. stale is the serve-stale-while-revalidating window (a mass eviction serves stale + rebuilds once in the background, never a blocking herd); revalidate/expire are the self-healing floor if a webhook is ever lost — not the freshness mechanism, which is the webhook. So the categories vocabulary, which changes rarely, sits at an hour, while the high-fan-out listing surfaces keep a short stale so an article-publish storm never causes blocking misses. Pick a profile by asking "how bad is staleness if a webhook goes missing, and how big is this surface's fan-out?", not "how fresh do I want it?" — the webhook already answers the second question.

Adding a surface

1. Add a row to CONTENT_CACHE naming the surface, its tags and a CACHE_PROFILE (reuse one, or add a profile if its frequency × fan-out is genuinely different — a high-frequency surface must never share a coarse tag).
2. Reference it from the query (client.query(Doc, vars, CONTENT_CACHE.mySurface)).
3. Make sure the webhook side emits the same tag for that content type (coarse for a whole type; slug/path only for a per-entry detail surface).

Never introduce a tag on one side without the other emitting it — an unmatched tag is a cache entry nothing ever invalidates.

Self-hosting: the shared-tags cache handler

revalidateTag() invalidates only the cache of the instance that runs it. The site runs on Flightcontrol across several app replicas, so a webhook that lands on one instance would leave the others serving stale content until their own time window elapsed. apps/web/use-cache-handler.mjs fixes that by SHARING the tag invalidation manifest across replicas via Redis.

It must be the NEW cacheHandlers API — not the legacy cacheHandler

The app runs cacheComponents (use cache + PPR). Under cacheComponents, Next resolves cache handlers ONLY through the new cacheHandlers (plural) map. The legacy singular cacheHandler option is never consulted by use cache — a Redis handler wired there is dormant, and revalidation silently stays per-instance. next.config.ts therefore registers the handler as cacheHandlers.default.

What the handler does

It keeps entry STORAGE local (fast) and shares only the invalidation SIGNAL:

• get / set / getExpiration delegate to Next's built-in per-instance handler (a real in-memory cache, 50 MB). Cached bytes are NOT shipped between replicas.
• updateTags (called when a tag is revalidated) mirrors the revalidation into a Redis hash (usecache:tagman, field = tag → { stale, expired }).
• refreshTags — which Next calls before EVERY request — pulls that hash back into this process's local tags manifest, the same manifest the built-in get/getExpiration consult to decide staleness.

So one webhook revalidateTag reaches every replica within a request, not just the instance that received it.

The webhook must EXPIRE tags, not mark them stale

Sharing the tag only helps if consuming it actually refetches. revalidateTag(tag, "max") marks a tag stale-now but hard-expires it ~1 year out — serve-stale + background revalidate (SWR), so the FE only refreshes on the surface's own revalidate window (~120s). The webhook therefore calls revalidateTag(tag, { expire: 0 }) (a profile object — no deprecation warning) for IMMEDIATE expiration → on-demand re-render. See The webhook.

Wired on a build-stable gate; Redis URL resolved at runtime

The registration is decided at BUILD, so it gates on a build-stable signal — the build is not on Vercel (process.env.VERCEL unset), i.e. any self-hosted deploy, which is every deploy of this site. A build on Vercel would not be wired (the platform shares the cache natively).

The Redis URL is read at RUNTIME. On self-hosting (Flightcontrol) CACHE_REDIS_URL comes from the cache service via a fromService binding that exists only at runtime — ABSENT at build. Gating the config block on the URL's presence (the earlier design, via clubConfig.hosting.redis) meant the gate was false at build and the handler was never registered — the app ran per-instance and the webhook reached only one replica. With no CACHE_REDIS_URL at runtime (CI, local dev, a single instance) the handler degrades to the built-in per-instance cache.

Each environment has its own Redis, by ruling

Each environment points at its own Redis (Flightcontrol provisions one per environment). Never point the site at a Redis something else uses — one environment's invalidation, or its cached page, must never reach another. The manifest uses a FIXED key (usecache:tagman) rather than a prefix: the separate Redis is the isolation boundary, and a prefix could not be sourced consistently across Next's bundled vs raw-module realms (the raw handler saw a different value than app code, so reads and writes hit different keys).

It fails safe

Every Redis operation is wrapped so any error degrades to LOCAL behaviour (the built-in per-instance cache), never a request failure — a connection blip, a timeout, malformed stored JSON all fall through. A reconnect cooldown avoids hammering Redis during an outage. A cache-store outage costs cross-replica freshness; it never takes the app down.

evictionPolicy is allkeys-lru

The manifest hash is written with no TTL. Under ElastiCache's default volatile-lru nothing is ever evictable, so the node fills and writes start failing. allkeys-lru lets the store behave like a cache.

Draft preview

Revalidation is about published content reaching the page fast. Preview is the other half: letting an editor see an unpublished draft on the real site before it goes live. It reuses the same GraphQL client, flipped by a boolean — no separate preview API host.

How it works

• One extra token. club.config.ts gains content.preview = env.server("CONTENTFUL_PREVIEW_TOKEN"). It defaults to empty, which keeps preview entirely off — lib/content.ts only builds a preview client, and only ever reads Next draft mode, when a real token is present. Without it the site renders exactly as before.
• The preview client (lib/content.ts) is the same Contentful GraphQL endpoint with the preview token and preview: true (drafts included). It is deliberately not wrapped in use cache — drafts must be fresh.
• The switch is per request. When preview is configured, a request in Next draftMode() is served live from the preview client; every other request goes through the cached delivery path. The token never enters a cache key.

The routes

• GET /api/preview — the entry editor's "Open preview" target. It checks a separate secret (CONTENTFUL_PREVIEW_SECRET, constant-time compare), accepts the target only as a same-origin relative path (no open redirect), enables draft mode, rewrites the __prerender_bypass cookie SameSite=None so it survives Contentful's Live-Preview iframe, and redirects to the page.
• GET /api/disable-preview — clears draft mode and returns to the page.

Turning it on for an environment

1. Create a Contentful preview delivery token (the CPA token) and set CONTENTFUL_PREVIEW_TOKEN in the deployment; pick a CONTENTFUL_PREVIEW_SECRET and set it too.

2. Provision the Contentful preview URLs — as code, the same way the revalidation webhook is:

   CONTENTFUL_SPACE_ID=<space> CONTENTFUL_CMA_TOKEN=<cma> \
   CONTENTFUL_PREVIEW_SECRET=<secret> SITE_BASE_URL=https://<deployment> \
   pnpm --filter @v4/model sync-preview-urls
   

   It upserts a preview environment — Publishing Hub preview for develop, Publishing Hub preview · <env> for any other environment (previewEnvironmentName; preview environments are space-level, so one shared name let the last deploy win) — whose per-content-type URLs point the entry editor's "Open preview" at /api/preview with the secret and a redirect to the entry's address: /news/{entry.fields.slug} for an article, {entry.fields.fullPath} for a content page (a page with a Parent Page lives at /parent/child, so its slug is not its address). The paths carry no locale segment; the proxy (apps/web/src/middleware.ts) adds it with a 308. The typed route map lives in packages/model/src/preview/preview-environment.ts, and scripts/sync-preview-urls.mjs mirrors it (a unit test holds the two together); the deploy pipeline runs it on every deploy, so each environment self-configures.

3. cacheComponents prerequisite. Setting CONTENTFUL_PREVIEW_TOKEN makes every content query read draft mode, which makes a route dynamic — so every content route MUST stream inside a Suspense boundary. On this site that is already in place: the home route has apps/web/src/app/[locale]/loading.tsx (re-exporting ./route-loading), and every News route streams inside the generated news/layout.tsx boundary. Do NOT add a loading.tsx under news/ — route-shims.test.ts fails on one (Route factories).

4. The data areas stay cacheable. The basketball routes take the delivery-only client (deliveryClient in routes.json), which never consults draft mode, so switching preview on does not make them request-time. A route that reads nothing an editor previews should do the same.

Nothing above changes an environment that leaves the token unset.

Test plan: caching & revalidation

How we certify that a deployed environment caches correctly and that a Contentful publish actually reaches the page — across both cache layers. The automated half is the /test-caching-revalidation skill (.claude/skills/test-caching-revalidation/run.mjs); this page is the strategy it encodes plus the manual checks that don't automate.

The two layers (what we're actually testing)

1. Next use cache + shared tags. Content reads go through a use cache boundary (lib/content.ts) that stamps each entry with cacheTag(...) and a cacheLife(...) SWR profile (CONTENT_CACHE / CACHE_PROFILE in @v4/content-app). Cached entries live PER-INSTANCE (Next's built-in handler); use-cache-handler.mjs shares the tag invalidation manifest across replicas via the environment's Redis. Invalidation is on-demand: a Contentful publish → the revalidate webhook → POST /api/revalidate → revalidateTag(tag, { expire: 0 }) (immediate expiration; NOT "max", which is SWR). Observable via x-nextjs-cache: HIT|STALE|MISS and cache-control: s-maxage=…, stale-while-revalidate=….
2. Cloudflare (the edge). The public domain is fronted by Cloudflare. It is pass-through by default (cf-cache-status: DYNAMIC), but an environment can opt in to edge HTML caching with a Cache Rule on its hostname — the app emits a Cache-Tag header on every page and, on publish, purges the edge by tag (purgeEdgeByTags, using CLOUDFLARE_API_TOKEN + CLOUDFLARE_ZONE_ID). See Edge caching & purge-on-publish. The test's job at this layer is to prove parity (the edge shows what the origin shows), classify each surface (static-cached vs PPR-bypassed), and — when edge caching is on — confirm a publish reaches the edge in seconds (purge-on-publish active). It no longer treats an edge HIT as a fault: HTML caching is a supported, opt-in mode now that the purge path exists. (Cache HTML without wiring the purge and pages strand for up to s-maxage — but that is the config error the matrix now surfaces, not the default.)

Tag model (what a publish expires)

tagsForPublish (lib/revalidation.ts, pure + unit-tested) maps a publish payload to:

• the coarse content:{type} tag — always. Listings, rails, the home page and the article sitemap register it (CONTENT_CACHE.articleCards / homePage / newsListing), so they refresh immediately on any publish of that type.
• both chrome tags (content:siteSettings + content:navigation) for nav/settings — a nav edit must refresh every page.
• the slug-level content:{type}:{slug} tag for the high-churn article / content-page detail surfaces. Detail queries register only the slug tag (not the coarse one) so one article's publish doesn't evict every article's cached page — the article's OWN page refreshes instantly, its related-content rail within its profile window.

A route that emits only the coarse tag (the pre-tagsForPublish version) leaves each article's own detail page stale until its TTL — the exact regression the slug tag fixes, and the first thing the skill asserts.

Automated coverage (the skill)

Run it against a non-production env (it publishes a throwaway entry):

node .claude/skills/test-caching-revalidation/run.mjs \
  [--origin=https://<lb-origin>] [--edge=https://<public-cdn>] \
  [--read-only] [--keep] [--json] [--iterations=<n>]

The origin defaults to LOAD_BALANCER_URL and the edge to NEXT_PUBLIC_SITE_BASE_URL / SITE_BASE_URL; credentials come from the environment or the root .env (CONTENTFUL_SPACE_ID, CONTENTFUL_ENVIRONMENT, CONTENTFUL_CMA_TOKEN, CONTENTFUL_WEBHOOK_SECRET). The run is organised in these sections:

The runner keys the throwaway articlePage under the space default locale, reuses a real published article's category + image so it is a valid listing card, and always deletes it (finally) unless --keep is passed. --read-only skips the publishing sections; --json emits machine-readable results. /api/health is the stack's health check path, if you need to tell a dead origin from a cache fault.

Manual checks (what doesn't automate)

• Cross-replica fan-out. revalidateTag must reach every Fargate replica, not just the one that received the webhook — that is the whole point of the shared Redis handler. Verify by publishing and confirming the change on repeated requests (which round-robin replicas), or scale to 1 replica to remove the variable.
• Draft preview isolation. In preview (draft mode) the page must show the draft and must not poison the shared cache — a preview request never writes a public cache entry.
• SWR behaviour under load. A mass eviction (e.g. a siteSettings publish) should serve stale and rebuild once, never stampede. Watch x-nextjs-cache go STALE→HIT.
• The lastmod in the sitemap tracks publish time.

Known findings & caveats (read before trusting a green run)

• Frozen-cache class of bug. A route with no content tag that gets fully prerendered (the full-route/PPR cache) and honours only explicit revalidateTag invalidation keeps serving whatever it first cached — a bad body freezes across deploys. This froze a [object Object] into robots.txt, the sitemap.xml index and its sub-sitemaps. The fix is per-route: await connection() opts the route out of the full-route cache (applied to robots.ts, the sitemap index and the sub-sitemaps; the sitemaps keep their data queries cached + tagged, so it stays cheap and still revalidates on publish). The deeper fix — making the handler respect time-based staleness so SWR self-heals — is deliberately not done yet because it touches cache correctness; until then, section 3 of the skill is the guard. Tagged content routes were never affected: they revalidate by tag, which the handler does honour.
• Cloudflare edge caching is opt-in, and its purge is wired. By default Cloudflare is pass-through; edge HTML caching is turned on with a Cache Rule, and the purge path now exists — /api/revalidate purges the edge by tag (purgeEdgeByTags, gated on CLOUDFLARE_API_TOKEN + CLOUDFLARE_ZONE_ID). Only the STATIC surfaces are edge-cached. Which those are has moved: since the 25–26 Oct 2026 performance work the recent articles and the data areas' pages prerender (staticParams), so re-measure cf-cache-status and Cache-Control on a deployed article and team page rather than assuming detail pages bypass the edge. See Edge caching & purge-on-publish.
• Listing cards stream client-side. /news server HTML contains no article-card links, so "did the listing refresh?" is asserted via the article sitemap (server XML, slug-bearing, same coarse tag) — not the /news document.
• CDA collection propagation. A single-entry read (the detail page) reflects a publish in well under a second; collection reads (sitemap, listings) can lag the Contentful delivery API by tens of seconds even after revalidation. The skill allows for this; a slow listing refresh is not necessarily a revalidation failure.
• A brand-new slug renders fresh, not from cache. "Page appears after publish" is a smoke test; the true cache-invalidation proof is the re-publish step (a cached page updating).

Edge caching & purge-on-publish

Two cache layers can sit in front of the site: Next's own cache (the origin — cleared by revalidateTag, covered in the pages before this) and, optionally, the Cloudflare edge in front of it. This page is the edge layer: the Cache-Tag header the app always emits, how to turn on edge HTML caching, which pages actually cache, and how a publish clears the edge so both layers stay fresh.

By default the edge is pass-through (cf-cache-status: DYNAMIC) — Cloudflare caches nothing, every request reaches the origin, and origin revalidation is the whole story. Edge caching is an opt-in performance optimisation (lower latency, origin offload) that you enable per hostname, together with the purge below. Turning it on without the purge re-introduces up to s-maxage of edge staleness on publish — the two go together.

The Cache-Tag header (always emitted)

apps/web/src/middleware.ts sets a Cloudflare Cache-Tag response header on every page, derived from the pathname (apps/web/src/lib/cache-tags.ts) using the same tag vocabulary the webhook uses: the coarse surface tags (content:articlePage, content:homePage, content:newsListingPage, content:category, content:contentPage), the slug/path tags (content:articlePage:{slug}, content:contentPage:{path}), and the two chrome tags (content:siteSettings, content:navigation) on every page (a nav/settings change must clear all). Category listings also carry content:category and content:newsListingPage. The basketball data areas (/teams, /players, /coaches, /games, /standings, /stats) carry the coarse Competition Engine tag ce — the path cannot name the club or player CODE the fine ce:* tags use, so an ingester run clears every data page at the edge — plus content:team and content:player for the entries the pages merge over, and content:season + content:seasonSponsor on games, players and stats (the headers' "Sponsored by").

It lives in middleware because App Router cannot set a response header from a Server Component. That has one consequence: the per-render reference tag (content:entry:{id}) is not in the header — it is only known at render. That gap is closed at purge time (below), not here. The header is inert while the edge is pass-through; once Cloudflare caches the response it consumes the header (and strips it from the client) for tag-based purge — which requires a Cloudflare Enterprise plan.

Enabling edge HTML caching (a Cloudflare Cache Rule)

Cloudflare does not cache HTML by default, regardless of s-maxage — you add a Cache Rule scoped to the site's hostname (a zone can hold other hostnames, so never match the whole zone). The safe shape:

• Match (custom filter expression):

  (http.host eq "<site-hostname>")
    and (not starts_with(http.request.uri.path, "/api/"))
    and (not (http.cookie contains "__prerender_bypass"))
    and (not any(http.request.headers["rsc"][*] == "1"))
    and (not (http.request.uri.query contains "_rsc"))
  

• Then: Eligible for cache; Edge TTL respect origin (uses the origin's s-maxage); Browser TTL respect origin (the origin sends no max-age, so the browser still won't cache the HTML).

Three exclusions are load-bearing:

• /api/* — never edge-cache API routes (the webhook, preview toggles).
• __prerender_bypass cookie — draft-mode (editor preview) requests bypass the cache, so a draft is never cached and served to the public, and editors always see live drafts.
• rsc header / ?_rsc query — React Server Component navigation/prefetch requests carry an rsc marker and a Vary: rsc, … that Cloudflare ignores. Cache them under the same key as the HTML document and client-side navigation breaks (an RSC request gets served HTML). Cache documents only; let RSC/prefetch pass through to the origin (fresh, correct, and cheap).

Which pages actually cache — static vs dynamic

Only a subset of pages are edge-cached, and it decides what a publish must purge:

Article and content detail pages were Partial-Prerendering / dynamic, so they sent no-store and Cloudflare bypassed them: a publish was instant at the edge for those, but they got no edge offload. Only the static surfaces (home, listings) were edge-cached — the only pages a publish was edge-stale on, and the only ones the purge had to clear.

Re-measure before relying on this table. The 25–26 Oct 2026 performance work made the recent articles (news-path, staticParams) and the data areas' pages (staticParams plus the use cache engine reads) prerender, so some detail pages may now send s-maxage and cache at the edge. Check cf-cache-status and Cache-Control on a deployed article and team page; the purge already covers them (the article slug tags, and ce for the data areas).

Purge-on-publish — clearing the edge

revalidateTag clears Next's cache but has no idea Cloudflare exists. So /api/revalidate also awaits purgeEdgeByTags(tags) (apps/web/src/lib/cloudflare-purge.ts) — a purge-by-tag (Enterprise) matched to the Cache-Tag header:

• Direct publishes (article, content page, category, nav) purge their exact surface/slug tags.
• A shared embedded entity (a block whose content type is not a page surface, carrying only content:entry:{id} — which is not in the edge header) coarse-purges every edge-cached content surface (content:homePage, content:articlePage, content:newsListingPage, content:contentPage). No stale, slightly broad — the coarse-on-reference rule. Blocks change rarely, so the over-purge cost is low.
• The content:entry:{id} tag itself is never purged (that precision stays at the origin, where the use cache tags already do it).

Config + fail-safe. Set CLOUDFLARE_API_TOKEN (a token with Zone → Cache Purge) + CLOUDFLARE_ZONE_ID. Unset → the purge is a no-op (an environment not caching at the edge). Any purge error is swallowed — the origin revalidation already happened, so a failed edge purge must never fail the webhook.

So a publish clears both layers: origin (revalidateTag) and edge (purgeEdgeByTags), each in a few seconds — including a shared block (verified: home edge went from ~`s-maxage` to a few seconds once the purge was wired).

Summary

The /test-caching-revalidation skill's propagation matrix (section 4) certifies all of this against a live environment — it publishes across page kinds, measures origin vs edge, and reports whether purge-on-publish is active.

Competition Engine data

The basketball data areas read two sources: Contentful (the editorial twins — team pages, hero slides, player profiles — cached through CONTENT_CACHE like every other content read) and the Competition Engine, through @v4/competition-engine. The engine is not Contentful, so it has its own cache boundary, its own tags and its own invalidation call. The vocabulary lives in packages/competition-engine/src/tags.ts.

The read path — one use cache boundary

Every structural and computed engine GET goes through cachedEngineGet (packages/competition-engine/src/cached-fetch.ts), a use cache boundary keyed on the request URL. The fetchers hand it to openapi-fetch as the per-call fetch (cacheOptions(tags, ttlClass)); it calls cacheTag(...) with the coarse ce tag plus the read's own, and cacheLife(...) with the class's profile.

A Next fetch hint (next: { tags, revalidate }) is NOT a cache under Cache Components: an uncached fetch marks the render request-time, which is how every basketball page used to render per request and go out no-store (perf, 26 Oct 2026). Never read the engine with a raw fetch.

What is cached is the response's RECORD (status, content type, body text), not a Response. A failure is recorded too — an error status, a refused connection or a timeout (ENGINE_TIMEOUT_MS, 10 s) — and lives only FAILED_LIFE ({ stale: 0, revalidate: 30, expire: 300 }): an outage shows the area's unavailable state for half a minute, not for the five minutes a structural read lives.

Tags

TTL classes

Each read names a class. CE_TTL is its revalidate window in seconds; CE_LIFE turns it into a cacheLife profile whose expire holds a stale answer for an hour while a fresh one is fetched (and stays above Next's five-minute floor, under which the read is request-time again):

The TTL is the ceiling, not the freshness mechanism; the ingester's call below is.

Live data is never server-cached

The live endpoints use noStore — cache: "no-store" with the same timeout. The browser polls the app's own GET /api/live/games (apps/web/src/app/api/live/games/route.ts), which forwards to the engine with the visitor's If-None-Match and passes the engine's Cache-Control and ETag straight through, so a 304 costs nothing. /api/* is outside the edge's cache rule.

The invalidation path — the ingester's post-run call

After a run the engine's ingester calls the same POST /api/revalidate the Contentful webhook does, with the same x-revalidate-secret, and a payload naming what it touched:

{
  "source": "competition-engine",
  "seasons": ["E2026"],
  "teams": ["BAR"],
  "players": ["003469"],
  "coaches": ["001234"],
  "games": [{ "season": "E2026", "code": 247 }],
  "tags": ["ce:season:E2026"]
}

"source": "competition-engine" routes it to tagsForEngineRun (apps/web/src/lib/revalidation.ts) instead of tagsForPublish. Every field is optional; tags takes ready-made names and keeps only well-formed ce tags. The coarse ce tag is ALWAYS included: the edge carries only that one on the data area pages, so a run clears every data page at the edge, while the fine tags expire the origin's reads precisely. The route then calls revalidateTag and purgeEdgeByTags for the list exactly as for a publish.

Adding an engine read

1. Add the fetcher in packages/competition-engine/src/server.ts with cacheOptions([...its CE_TAG tags], "<class>") — or noStore if it is live.
2. Pick the class by how the data changes, not how fresh you would like it: the ingester's call is what makes a change visible at once.
3. Tag it with every entity a run could change it for, so the ingester's payload reaches it; a read tagged only ce refreshes on every run.