Service Incident August 9th 2016 Pod 4 8 Performance Issues

11:25 UTC | 04:25 PT
We are happy to report Zendesk services are restored in pod 4 and 8.

10:36 UTC | 03:36 PT
We are seeing improvements as we continue to restore services for customers on Pod 4 and 8.

POST-MORTEM

On Saturday, August 6th 2016 our network operations team made a change to increase the capacity of our Load Balancer Tier for Pods 4 & 8. We added 1 more blade to our 2 chassis in the data center to give us more memory and higher availability. We hit a bug that caused a memory leak in the new blade. To fix the issue, we failed over from the bad chassis to a good chassis. To prevent this from happening in the future we have upgraded the software on our chassis and have added better monitoring to alert our network operations teams.

FOR MORE INFORMATION

For current system status information about your Zendesk, check out our system status page. During an incident, you can also receive status updates by following @ZendeskOps on Twitter. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.