Dynamic Bot Rendering

Many AI assistants fetch your pages with a plain HTTP request and no JavaScript engine. On a client-rendered site, they may see an empty shell instead of your content. Dynamic Bot Rendering fixes this at your content delivery network (CDN). You add one small routing rule to the CDN you already run. That rule recognizes AI assistant traffic by User-Agent and forwards it to Profound. For the public pages you have opted in, Profound returns a published, pre-rendered copy of the page when one exists. For everything else it passes your origin’s live response straight through. Human visitors, search crawlers, API calls, and assets never leave your existing route.

​

How a request flows

​

Supported AI assistants

Profound identifies the assistant from the original User-Agent on every request, using the same tokens your CDN rule matches on. Matching is a case-insensitive substring test.

​

Search crawler routing

Ordinary Googlebot and Bingbot requests fall outside the list above and stay on your existing route. Test your site’s cache and edge rules to confirm this before rollout. Dynamic Bot Rendering doesn’t cloak: the rendered page is the same page a browser produces after JavaScript runs.

​

How rendering and serving are controlled

Two independent settings decide what an assistant gets for a given URL:

• Policy answers “may Profound serve stored content for this path, and to which assistants?”
• Publication answers “which exact render is live for this path?”

Both have to line up before a stored render is served. A published page under an off policy serves nothing. A serve policy on a path with no publication falls back to your origin. You manage both yourself in the Profound dashboard.

​

Policy modes

off is the default for any path no rule covers. A newly registered domain does nothing until a rule opts a path in, so onboarding is inert until you decide what to render. Use warm to build up candidates for review before you start serving. serve never publishes anything by itself; it only permits an existing publication to be delivered.

​

Rules and precedence

Every rule belongs to one registered domain. A domain default sets the baseline for the whole host, and page rules override it for specific paths. A page rule matches an exact path (optionally pinned to a query string), a path prefix, or a glob such as /blog/*. Prefix and glob rules match on the path without its query string. For each request Profound picks one winning rule:

1. An exact rule for the path and query string.
2. Otherwise, an exact rule for the path.
3. Otherwise, the best matching prefix or glob rule. Higher priority wins, then the more specific pattern.
4. Otherwise, the domain default.
5. With no rule at all, the effective mode is off.

Only the winning rule applies. Any setting it leaves unset is inherited from the domain default, never from a broader page rule that also matched. For example, with a domain default of serve, a /docs/* rule set to warm, and an exact /docs/start rule that only changes its allowed assistants, /docs/start resolves to serve. The /docs/* rule does not contribute. A rule can also be deactivated. An inactive rule is skipped entirely, so the domain default or another matching rule takes over for that path. Deactivating an off rule that excluded a path can therefore turn serving back on. To keep a path on your origin, use an active rule with mode off.

​

Per-rule settings

Each rule can set any of these and inherit the rest:

• Mode. off, warm, or serve, as above.
• Allowed assistants. Restrict which supported assistants receive treatment on the matched paths. Unset inherits the domain’s list, and when nothing sets a list every supported assistant is eligible. A list replaces the inherited list rather than intersecting with it, and an empty list excludes every assistant. An excluded assistant receives your origin’s response with an effective policy of off. The published render itself is shared: you cannot publish different versions for different assistants.
• Automatic rendering. Allow or block background candidate renders independently of mode. serve with automatic rendering off serves only what you publish by hand. off never renders automatically, whatever this setting says.

​

Pausing a domain

Pausing a domain sends every assistant request for that host to your origin while keeping the registration, rules, renders, and publications intact. Resuming restores the saved behavior. Use pause when you want to stop serving quickly without touching your CDN or your rules. While paused, you can still render, review, and publish pages.

​

From render to live page

Every render starts as a candidate: a stored snapshot of one URL, made either in the background by an eligible request or on demand. A URL can accumulate several candidates. None of them is served until you publish one. A published render stays live until you publish a replacement or unpublish it. Profound never replaces a publication automatically, and a render does not stop serving because it has aged. If a stored render can no longer be read, Profound falls back to your origin for that request rather than serving a different render. Page identity is the public host plus the exact path and query string. Publishing /pricing does not publish /pricing?plan=pro, and each variant is rendered, reviewed, and published on its own.

​

Fail-open by design

Fail-open happens at two layers, and neither covers every failure on its own. A bad API key, an unregistered host, or a malformed path is rejected by Profound rather than proxied, so a misconfigured rule cannot leak requests to your origin under Profound’s name. Cloudflare and CloudFront can mask some of these rejections through their CDN failover, which means a working page in a browser does not prove the integration is correct. Inspect the x-concierge-* response headers, as described in Verify and troubleshoot.

​

Scope and limits

• GET only. Other methods, including HEAD, are not part of the contract. Use curl -sD - -o /dev/null, not curl -I, when testing.
• Public, non-personalized pages only. Opt in only reviewed public pages, whether through your policy rules in Profound or through a path allowlist in your CDN rule. An extensionless path is not proof of a page: keep APIs, GraphQL, authentication, administration, framework resources, and session-sensitive routes out. The Cloudflare and CloudFront examples allow only /, /pricing, /docs, and /docs/getting-started; replace these with your own paths.
• Query strings are part of the page identity. /products?id=a and /products?id=b are distinct pages, stored and rendered separately.
• One host per rule. The public host your rule sends must match a domain registered in Profound. Serve several hosts with one rule per host.
• No credentials reach your origin through Profound. Profound does not forward viewer cookies or authorization headers when it fetches your origin, and the example rules keep credential-bearing requests on your existing route. Protected pages may render as a login page or public content, which is a compatibility boundary rather than a leak.
• One render per URL, not per visitor. A stored render is not varied by session, language, geography, or other request headers. If the same URL serves different content by those signals, preview the render and confirm that one version is acceptable for every assistant before adding it to your allowlist.

​

Before you start

Complete these steps in the Profound dashboard before configuring your CDN. The Dynamic Bot Rendering endpoint your CDN will call is https://concierge.tryprofound.com. Also record your existing CDN routes, functions, cache policies, and rollback settings before you change anything, and check that web application firewall (WAF) rules, bot challenges, deployment protection, and origin allowlists let both assistant requests and Profound’s origin fetches through. If your origin needs a way to identify Profound’s fetches, contact Profound support.

​

Choose your CDN

Running something else, such as Akamai, Fastly, or a custom proxy? The underlying contract is three request headers on a GET. Contact Profound support and we’ll help you map it.