Service Incident February 19th 2016

11:25 UTC | 03:25 PT
We are happy to report that the issue's with loading ticket views are now resolved.

10:56 UTC | 02:56 PT
Our operations team work to mitigate ticket view issues affecting some customers. More information shortly.

10:36 UTC | 02:36 PT
We continue to investigate issues with loading ticket views affecting EU data center customers.

POST-MORTEM

On Friday, February 19th, we began receiving reports of views not loading for some customers hosted in our European data centre. This was found to be due to a configuration for one of our services, which was modified a few hours ago, causing a log pile up in the wrong directories of several servers in the EU data centre. Alerts went unnoticed until node partitions filled up, affecting service performance. As this was a service that view rely on, they became inaccessible to customers.

We began work on resolving the affected hosts and within about 30-45 minutes we got disk somewhat cleared and performed rolling restarts of the service. This solved the problem. We're putting log rotation in place for the service to prevent pile ups affecting Zendesk services going forward. 

FOR MORE INFORMATION

Please subscribe to this article for regular updates until the issue is resolved. If you aren't subscribed to our Twitter feed, we encourage you to do so in order to get the most current information about any service issues. We also record all site outages on our system status page where you can see the past 12 months of service uptime. If you have questions about this issue, please open a ticket with us by sending a note to support@zendesk.com.