Hacker News·4 min read·medium

Microsoft blames monster email outage on software defect introduced in July fix

B
Bender
Microsoft blames monster email outage on software defect introduced in July fix
✦AI Summary

Microsoft experienced a major email outage caused by a software defect introduced in a July 2026 update. The failure occurred when a surge in load combined with reduced server capacity triggered a resource-exhaustion bug in the network gateway.

Why it matters

It demonstrates the fragility of large-scale cloud infrastructure and the risks associated with cascading failures in complex distributed systems.

✦Dive DeeperCreate a free account to unlock

Summer code change caused a Fall fail – frantic engineers had to patch the patch instead of rolling back

Microsoft has blamed last week's epic outage on a change it made in July that lurked in the network gateway servers until a surge in load and a drop in capacity triggered the defect, giving the Microsoft 365 world a day off from email.

According to a preliminary post incident report published by Microsoft on November 15, the defect was introduced in July 2026. The update looked fine at the time, Microsoft says, having passed load testing and been deployed without any impact.

However, the testing wasn't good enough. Microsoft's report states: "Our load testing tended to focus on weaker hardware with fewer cores." The defect caused "lock contention inside an internal Windows process which caused the machine's resources (CPU & memory) to be exhausted."

Continue reading on Headlinne

Create a free account to read the full article.

Read full article →

Also covering this story

2 other newsrooms covered this event. We read each version separately.

technologybusiness
✦

Get smarter about the news

Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.

Create free account

Already have an account? Sign in