Service Incident February 9th 2016

13:41 UTC | 05:41 PT
Performance issues have now stabilised. We will continue to monitor closely. Post mortem to follow:

13:09 UTC | 05:09 PT
Our investigation continues but we are seeing improvements to some affected services.

12:49 UTC | 04:49 PT
We continue to investigate performance issues impacting our services in our European Datacenter.

POST-MORTEM

An undetermined upstream network hiccup caused a disconnection between a service in our European data center and it's mediators, causing a massive flood of reconnection attempts. During this time, there was a significant uptick in outbound traffic, in response to the requests, which caused a network circuit in said data center to become over saturated and drop traffic. This affected various Zendesk services for accounts hosted in the EU data center, as well as caused disconnections in Zopim.

Traffic subsided organically after mediator connections were re-established.

FOR MORE INFORMATION

Please subscribe to this article for regular updates until the issue is resolved. If you aren't subscribed to our Twitter feed, we encourage you to do so in order to get the most current information about any service issues. We also record all site outages on our system status page where you can see the past 12 months of service uptime. If you have questions about this issue, please open a ticket with us by sending a note to support@zendesk.com.