Post-mortem - November 18, 2025 - All Pods | Support, Guide - Content Delivery Network issues

SUMMARY

On November 18, 2025, between 11:28 UTC and 13:37 UTC, Zendesk customers on all pods experienced login failures and intermittent errors across multiple products. Customers who were already logged in to Zendesk were not affected.

Timeline

November 18, 2025 12:58 UTC | November 18, 2025 04:58 AM PT

Our Web Application Firewall (WAF) vendor is currently experiencing service degradation affecting certain internal systems. Some customers may encounter issues when interacting with WAF challenges. We are actively assessing the situation to fully understand the impact and will provide updates as more information becomes available. We appreciate your patience and understanding during this time.

November 18, 2025 13:38 UTC | November 18, 2025 05:38 AM PT

Our engineers have deployed a fix for the login issue affecting customers. Please attempt to log in again and inform us if the problem persists so we may assist you accordingly.

November 18, 2025 17:12 UTC | November 18, 2025 09:12 AM PT

We are observing successful logins and monitoring the situation, however we are actively still working with our CDN provider to ensure full recovery and stability. Thank you for your patience.

November 18, 2025 18:50 UTC | November 18, 2025 10:50 AM PT

We are happy to report that, working closely with our CDN provider, we have mitigated the impact of preventing access to Zendesk products and all systems have recovered. Thank you for your patience during our investigation.

Root Cause Analysis

The incident was caused by a failure in the bot management system of Zendesk’s content delivery network provider, which impacted service delivery.

Resolution

To resolve the issue, our Content Delivery Network (CDN) provider's verification prompts were temporarily disabled on the login page. Our CDN reverted to a stable configuration and implemented mitigation measures. Following confirmation from our CDN, services were validated as stable, and the disabled bot management rule was safely re-enabled.

Remediation Items

Monitor the complete login flow to ensure it operates smoothly end-to-end.

Continuously monitor the login Web Application Firewall (WAF) to detect and prevent issues.

Investigate how to handle cases when the BotScore is zero across all zones to improve security measures.

Ensure all third-party modules are sourced and downloaded securely from the Artifactory repository.

FOR MORE INFORMATION

For current system status information about Zendesk and specific impacts to your account, visit our system status page. If you have additional questions about this incident, contact Zendesk customer support.