True
Causes
KO
Bahasa Indonesia
Dansk
Deutsch
English
Español
Français
Italiano
Nederlands
Norsk
Polski
Português
Srpski (latinica)
Svenska
Tiếng Việt
Türkçe
Ελληνικά
Русский
Српски (ћирилица)
Українська
中文
日本語
한국어
Why does the same outage keep recurring?
Retries are unbounded, so a slow dependency becomes a flood
a walkthrough with stand-ins — a demonstration, not a finding · sealed
It significantly influences
The same three services cause most of the pages
certified ·
11 yes · 2 no · 1 abstained
The staging environment does not resemble production
inferred · follows from certified links
Judged not to be linked
Rollbacks take longer than the outage they are meant to end
judged not to ·
1 yes · 13 no
Config lives in five places and drifts between them
judged not to ·
2 yes · 12 no
Reliability has no budget line of its own
judged not to ·
1 yes · 13 no
Customers report outages before monitoring does
judged not to ·
1 yes · 12 no · 1 abstained
The incident channel fills with people asking for status instead of giving it
judged not to ·
2 yes · 12 no
Feature flags are never cleaned up, so nobody knows which paths are live
judged not to ·
1 yes · 12 no · 1 abstained
The service map in the wiki is two years old
judged not to ·
3 yes · 11 no
The same three services cause most of the pages
judged not to ·
1 yes · 13 no
Rollbacks take longer than the outage they are meant to end
judged not to ·
0 yes · 14 no
Config lives in five places and drifts between them
judged not to ·
1 yes · 13 no
Reliability has no budget line of its own
judged not to ·
1 yes · 13 no
Customers report outages before monitoring does
judged not to ·
2 yes · 10 no · 2 abstained
The incident channel fills with people asking for status instead of giving it
judged not to ·
1 yes · 12 no · 1 abstained
The staging environment does not resemble production
judged not to ·
2 yes · 12 no
Feature flags are never cleaned up, so nobody knows which paths are live
judged not to ·
0 yes · 14 no
The service map in the wiki is two years old
judged not to ·
0 yes · 14 no
A group’s judgement, not a measurement. Demo room.
On the atlas
.
↗ cite or embed this claim
Link:
https://truecauses.org/c/6708d57/5f15967-retries-unbounded-slow-dependency-flood
As data:
https://truecauses.org/c/6708d57/5f15967.json
<iframe src="https://truecauses.org/embed/6708d57/5f15967" width="100%" height="360" style="border:0" loading="lazy"></iframe>