Service Incident on September 30th 2015

As of September 30th, 2015 11:00AM GMT+1 / 03:00AM PST, we are working to resolve the following service incident:

We have some reports from customers experiencing issues login with Zopim. Our team is investigating this. More to info to come.

11:40AM GMT+1 / 3:40AM PST POST-MORTEM

For the duration of this incident, some Zopim customers reported not being able to login to the dashboard and the widgets were not loading intermittently. The platform was in a degraded mode for about an hour. This was caused by a deploy that changed the visitor data model, which led to update failure to certain visitor data.

More tests have been written to increase the coverage of the affected endpoints. However, the tests did not cover this specific case and hence resulting in this issue. Server logs should have been monitored right after the release and that could have prevented the delay in fixes deployed which eventually lead to degradation. We reverted the affected deploy to restore full functionality.

FOR MORE INFORMATION

Please subscribe to this article for regular updates until the issue is resolved.  If you aren't subscribed to our Twitter feed, we encourage you to do so in order to get the most current information about any service issues.   We also record all site outages on our system status page where you can see the past 12 months of service uptime. If you have questions about this issue, please open a ticket with us by sending a note to support@zendesk.com.