> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tryprofound.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Dynamic Bot Rendering
> Serve AI assistants a fully rendered version of your pages from the edge, while other traffic stays on your existing route.
Many AI assistants fetch your pages with a plain HTTP request and no JavaScript engine. On a client-rendered site, they may see an empty shell instead of your content.
Dynamic Bot Rendering fixes this at your content delivery network (CDN). You add one small routing rule to the CDN you already run. That rule recognizes AI assistant traffic by `User-Agent` and forwards it to Profound. For the public pages you have opted in, Profound returns a published, pre-rendered copy of the page when one exists. For everything else it passes your origin's live response straight through. Human visitors, search crawlers, API calls, and assets never leave your existing route.
Dynamic Bot Rendering is served by Profound's Concierge service, so its request and response headers use the prefix `x-concierge-`.
## How a request flows
Your routing rule matches the `User-Agent` against the [supported assistant list](#supported-ai-assistants). Only matching requests go to Profound. The Cloudflare and CloudFront examples also restrict routing to a path allowlist; the Vercel rule forwards all assistant traffic and lets your Profound policy decide.
The rule attaches your API key, your public host, and the original path and query as trusted headers. It sets those values from its own configuration and the parsed request, never from headers a client sent. Profound ignores client-supplied `x-concierge-*` identity headers.
Profound checks the API key, confirms the host is registered to your organization, and resolves the [policy](#how-rendering-and-serving-are-controlled) for the path. If the path has a published render and policy allows serving, Profound returns that render. Otherwise it fetches the page from your origin and returns the live response unchanged. Depending on policy, that miss can also schedule a background render of the page as a new candidate.
A published render is fully rendered HTML: the same page a browser produces after JavaScript runs. Rendering makes your content readable to the assistant. It does not guarantee inclusion or citation in an answer.
## Supported AI assistants
Profound identifies the assistant from the original `User-Agent` on every request, using the same tokens your CDN rule matches on. Matching is a case-insensitive substring test.
| Token | Assistant |
| ---------------------- | --------------------- |
| `chatgpt-user` | ChatGPT browsing |
| `codex` | OpenAI Codex |
| `claude-user` | Claude |
| `claude-code` | Claude Code |
| `perplexity-user` | Perplexity |
| `gemini-deep-research` | Gemini Deep Research |
| `duckassistbot` | DuckDuckGo DuckAssist |
| `amzn-user` | Amazon |
| `mistralai-user` | Mistral |
Keep your CDN rule's list in sync with this table. Profound classifies traffic independently, so a `User-Agent` your rule forwards but Profound doesn't recognize is treated as an unknown agent. It fails open to your origin and adds a round trip without serving or scheduling a render.
## Search crawler routing
Ordinary Googlebot and Bingbot requests fall outside the list above and stay on your existing route. Test your site's cache and edge rules to confirm this before rollout. Dynamic Bot Rendering doesn't cloak: the rendered page is the same page a browser produces after JavaScript runs.
## How rendering and serving are controlled
Two independent settings decide what an assistant gets for a given URL:
* **Policy** answers "may Profound serve stored content for this path, and to which assistants?"
* **Publication** answers "which exact render is live for this path?"
Both have to line up before a stored render is served. A published page under an `off` policy serves nothing. A `serve` policy on a path with no publication falls back to your origin. You manage both yourself in the Profound dashboard.
### Policy modes
| Mode | What the assistant receives | Renders new candidates in the background? |
| ------- | -------------------------------------------------------------- | ------------------------------------------- |
| `off` | Your origin's live response | No |
| `warm` | Your origin's live response, even if a publication exists | Yes, when the request is eligible |
| `serve` | The published render when one is usable, otherwise your origin | Yes, on a miss when the request is eligible |
`off` is the default for any path no rule covers. A newly registered domain does nothing until a rule opts a path in, so onboarding is inert until you decide what to render. Use `warm` to build up candidates for review before you start serving. `serve` never publishes anything by itself; it only permits an existing publication to be delivered.
### Rules and precedence
Every rule belongs to one registered domain. A **domain default** sets the baseline for the whole host, and **page rules** override it for specific paths. A page rule matches an exact path (optionally pinned to a query string), a path prefix, or a glob such as `/blog/*`. Prefix and glob rules match on the path without its query string.
For each request Profound picks one winning rule:
1. An exact rule for the path and query string.
2. Otherwise, an exact rule for the path.
3. Otherwise, the best matching prefix or glob rule. Higher priority wins, then the more specific pattern.
4. Otherwise, the domain default.
5. With no rule at all, the effective mode is `off`.
Only the winning rule applies. Any setting it leaves unset is inherited from the domain default, never from a broader page rule that also matched. For example, with a domain default of `serve`, a `/docs/*` rule set to `warm`, and an exact `/docs/start` rule that only changes its allowed assistants, `/docs/start` resolves to `serve`. The `/docs/*` rule does not contribute.
A rule can also be deactivated. An inactive rule is skipped entirely, so the domain default or another matching rule takes over for that path. Deactivating an `off` rule that excluded a path can therefore turn serving back on. To keep a path on your origin, use an active rule with mode `off`.
### Per-rule settings
Each rule can set any of these and inherit the rest:
* **Mode.** `off`, `warm`, or `serve`, as above.
* **Allowed assistants.** Restrict which supported assistants receive treatment on the matched paths. Unset inherits the domain's list, and when nothing sets a list every supported assistant is eligible. A list replaces the inherited list rather than intersecting with it, and an empty list excludes every assistant. An excluded assistant receives your origin's response with an effective policy of `off`. The published render itself is shared: you cannot publish different versions for different assistants.
* **Automatic rendering.** Allow or block background candidate renders independently of mode. `serve` with automatic rendering off serves only what you publish by hand. `off` never renders automatically, whatever this setting says.
### Pausing a domain
Pausing a domain sends every assistant request for that host to your origin while keeping the registration, rules, renders, and publications intact. Resuming restores the saved behavior. Use pause when you want to stop serving quickly without touching your CDN or your rules. While paused, you can still render, review, and publish pages.
## From render to live page
Every render starts as a **candidate**: a stored snapshot of one URL, made either in the background by an eligible request or on demand. A URL can accumulate several candidates. None of them is served until you publish one.
A candidate that differs sharply from the page currently published for that URL is held as **pending**. That happens when its visible text collapses to a fraction of the live page's, or when both the title and the first heading change. Preview the candidate, then approve or reject it. Approval marks it eligible for publication and nothing more. When a URL has no published page yet, there is nothing to diverge from and the candidate is stored as approved.
Publishing selects one exact render as the live page for that URL. Publishing a pending candidate approves it in the same step. A rejected candidate cannot be published.
Rolling back is publishing an earlier render for the same URL. Unpublishing removes the live page so assistants return to your origin, and keeps the render history. Rejecting a page that is currently published also unpublishes it. To refresh content, render a new candidate, review it, and publish it.
A published render stays live until you publish a replacement or unpublish it. Profound never replaces a publication automatically, and a render does not stop serving because it has aged. If a stored render can no longer be read, Profound falls back to your origin for that request rather than serving a different render.
Page identity is the public host plus the exact path and query string. Publishing `/pricing` does not publish `/pricing?plan=pro`, and each variant is rendered, reviewed, and published on its own.
## Fail-open by design
Fail-open happens at two layers, and neither covers every failure on its own.
The Cloudflare Worker retries your origin for fetch failures, a response-header deadline, and a configured list of statuses. CloudFront uses an origin group with configured failure criteria and timeouts. Vercel's routing rule has no CDN-level failover.
Once a request is authenticated and valid, Profound fetches your origin whenever it cannot serve a stored render: no publication, policy set to `off` or `warm`, an unknown assistant, an unreadable render, or an internal lookup failure. Your origin must still be reachable.
On Vercel, a Profound outage or a hard rejection can reach the assistant as an error, because the external rewrite has no secondary origin to retry. Cloudflare and CloudFront customers get an extra layer of protection from their CDN failover.
A bad API key, an unregistered host, or a malformed path is rejected by Profound rather than proxied, so a misconfigured rule cannot leak requests to your origin under Profound's name. Cloudflare and CloudFront can mask some of these rejections through their CDN failover, which means a working page in a browser does not prove the integration is correct. Inspect the `x-concierge-*` response headers, as described in [Verify and troubleshoot](/dynamic-bot-rendering/verify-and-troubleshoot).
## Scope and limits
* **`GET` only.** Other methods, including `HEAD`, are not part of the contract. Use `curl -sD - -o /dev/null`, not `curl -I`, when testing.
* **Public, non-personalized pages only.** Opt in only reviewed public pages, whether through your policy rules in Profound or through a path allowlist in your CDN rule. An extensionless path is not proof of a page: keep APIs, GraphQL, authentication, administration, framework resources, and session-sensitive routes out. The Cloudflare and CloudFront examples allow only `/`, `/pricing`, `/docs`, and `/docs/getting-started`; replace these with your own paths.
* **Query strings are part of the page identity.** `/products?id=a` and `/products?id=b` are distinct pages, stored and rendered separately.
* **One host per rule.** The public host your rule sends must match a domain registered in Profound. Serve several hosts with one rule per host.
* **No credentials reach your origin through Profound.** Profound does not forward viewer cookies or authorization headers when it fetches your origin, and the example rules keep credential-bearing requests on your existing route. Protected pages may render as a login page or public content, which is a compatibility boundary rather than a leak.
* **One render per URL, not per visitor.** A stored render is not varied by session, language, geography, or other request headers. If the same URL serves different content by those signals, preview the render and confirm that one version is acceptable for every assistant before adding it to your allowlist.
## Before you start
Complete these steps in the Profound dashboard before configuring your CDN. The Dynamic Bot Rendering endpoint your CDN will call is `https://concierge.tryprofound.com`.
Also record your existing CDN routes, functions, cache policies, and rollback settings before you change anything, and check that web application firewall (WAF) rules, bot challenges, deployment protection, and origin allowlists let both assistant requests and Profound's origin fetches through. If your origin needs a way to identify Profound's fetches, contact Profound support.
Open **Dynamic Bot Rendering** in the Profound dashboard and add the exact public host you serve, such as `www.example.com`. The host must resolve publicly and answer over HTTPS. Register the host visitors reach: if your apex domain redirects to `www`, register the `www` host rather than the apex, and never an internal origin hostname. The hostname is fixed once registered.
Create a Dynamic Bot Rendering API key and copy it. The raw value is shown once, so store it in your CDN's secret store immediately.
One key covers every domain registered to your organization, including domains you add later, so most customers need only one. Create additional keys when you want to rotate without downtime, or to manage rotation and revocation separately across properties. Additional keys are still organization-wide, not restricted to one property.
Give the domain a policy that opts at least one public page in with mode `serve`, render a candidate for that page, review it, and publish it. Keep the published URL and its expected HTML for verification.
After configuring your CDN, an assistant request for that URL should return `x-concierge-cache: HIT` and the rendered HTML. Repeated requests never substitute for publication: a `MISS` stays a `MISS` until a render is published.
Revoking a key affects every site using it. If you share one key across properties, rotate deliberately: create the replacement, roll it out to each CDN, then revoke the old one.
## Choose your CDN
One Worker on your zone.
One CloudFront Function, without Lambda\@Edge.
One routing rule in `vercel.json`.
Running something else, such as Akamai, Fastly, or a custom proxy? The underlying contract is three request headers on a `GET`. Contact Profound support and we'll help you map it.