Service Incident June 17th 2016 Search issues for some customers

16:43 UTC | 09:43 PT
All known issues related to this incident should be improving for the affected customers

POST-MORTEM

Our search cluster experienced a sharp growth in resource usage, which caused searches to fail for users. The growth was caused by a script that a customer was running. The solution was to block the source IPs of the script running the queries to allow the cluster to recover. To prevent this happening in the future, we will implement a generous deep paging limit in the search service; search scripts that exceed this limit will be met with the appropriate error code.

FOR MORE INFORMATION

Please subscribe to this article for regular updates until the issue is resolved. If you aren't subscribed to our Twitter feed, we encourage you to do so in order to get the most current information about any service issues. We also record all site outages on our system status page where you can see the past 12 months of service uptime. If you have questions about this issue, please open a ticket with us by sending a note to support@zendesk.com.