Announcing adaptive browser rendering for web crawlers

Announced on | Rollout starts | Rollout ends
April 30, 2026 | April 9, 2026 | May 4, 2026

We're excited to announce that web crawlers can now automatically detect and handle client-side rendered content. This improvement significantly expands the types of websites the web crawler can successfully index, including sites built with the latest web technologies and frameworks.

The crawler automatically adapts when JavaScript rendering is needed, ensuring comprehensive content capture without any manual configuration.

This announcement includes the following sections:

What is changing?

Why is Zendesk making this change?

What do I need to do?

What is changing?

The web crawler now features adaptive crawling capabilities that automatically optimize content capture based on how your website renders:

Automatic JavaScript detection: The crawler samples a small number of pages from each section of your website and compares results between a standard HTTP fetch and a full browser render. If the browser render captures significantly more content, the crawler automatically switches to browser mode for that section.

Per-section optimization: Different parts of the same website can be crawled in different modes. For example, a static blog section might use standard fetching for speed, while a dynamic application section uses browser rendering for completeness. This happens automatically with no configuration required.

Improved content capture: Pages that require JavaScript to display their content are now indexed and content is generated dynamically as the page loads.

Why is Zendesk making this change?

Modern websites increasingly use JavaScript frameworks and client-side rendering to deliver dynamic user experiences. Without browser rendering capabilities, crawlers miss content that only appears after JavaScript executes, leading to incomplete knowledge bases and gaps in AI agent responses.

Adaptive crawling solves this by automatically detecting when browser rendering is needed, ensuring your AI agents, generative search, and Copilot have access to complete and accurate content from your website, regardless of how it's built.

What do I need to do?

Adaptive crawling is automatically enabled for all web crawlers and no action is required on your part. The crawler will automatically detect and handle client-side rendered content during regular crawl runs.

If you notice improved content coverage from JavaScript-heavy websites or single-page applications, this feature is working as designed. For more information about the web crawler and how it indexes external content, see Using a Web Crawler to Index External Content.

If you have feedback or questions related to this announcement, visit our community forum where we collect and manage customer product feedback. For general assistance with your Zendesk products, contact Zendesk Customer Support.