True Causes
EN
Bahasa IndonesiaDanskDeutschEnglishEspañolFrançaisItalianoNederlandsNorskPolskiPortuguêsSrpski (latinica)SvenskaTiếng ViệtTürkçeΕλληνικάРусскийСрпски (ћирилица)Українська中文日本語한국어
← lobby
7/7 · Actions · viewing Map
30 people · 30 ideas
walkthrough · not counted
obstacles · anyone with the link · listed publicly · federation on
stewarded by its steward (sealed)
30 people · 30 ideas · 2 groups
Passport → · Logic → · record
↗ share this map
On another site, as a live map:
As a picture: https://truecauses.org/d/6708d57/map.svg
As data: every judged pair in this room (CSV). Each claim has its own address, /c/6708d57/<card>, listed under the map.
Credit when you publish: Causal map: True Causes, https://truecauses.org/d/6708d57
THE QUESTION

Why does the same outage keep recurring?

The root causes, deepest first. Disagreements stay on the map.

closed · back to Actions
Viewing as guest. Enter a handle to join and take part.
The map
Root causes on the left, outlined in green. Tap a claim to see its chain.
a link the room certified contested: the room could not settle it
The same three services cause most of the pages → The staging environment does not resemble productionFeature work always outranks reliability work in planning → Post-mortem actions are written up and then never scheduledRollbacks take longer than the outage they are meant to end → Reliability has no budget line of its ownEvery team has its own logging format → Customers report outages before monitoring doesReliability has no budget line of its own → The incident channel fills with people asking for status instead of giReliability has no budget line of its own → The staging environment does not resemble productionPost-mortems assign actions to people who were not in the room → Post-mortem actions are written up and then never scheduledTests that fail intermittently are retried until green → Customers report outages before monitoring doesRetries are unbounded, so a slow dependency becomes a flood → The same three services cause most of the pagesAlerts fire so often that on-call mutes them → Every team has its own logging formatThe on-call rota has the same two people on it most weeks → Post-mortem actions are written up and then never scheduledThe on-call rota has the same two people on it most weeks → Alerts fire so often that on-call mutes themFeature flags are never cleaned up, so nobody knows which paths are li → The same three services cause most of the pagesFeature flags are never cleaned up, so nobody knows which paths are li → Rollbacks take longer than the outage they are meant to endOne engineer knows how the payment path actually works → The on-call rota has the same two people on it most weeksThe service map in the wiki is two years old → Feature flags are never cleaned up, so nobody knows which paths are liFeature work always outranks reliability work in planningFeature work alwaysoutranks reliability work…Post-mortems assign actions to people who were not in the roomPost-mortems assign actionsto people who were not in…Tests that fail intermittently are retried until greenTests that failintermittently are retried…Retries are unbounded, so a slow dependency becomes a floodRetries are unbounded, so aslow dependency becomes a…One engineer knows how the payment path actually worksOne engineer knows how thepayment path actually worksThe service map in the wiki is two years oldThe service map in the wik…clusterCLUSTER 6Feature flags are never cleaned up, so nobody knows which paths are liFeature flags are never…clusterCLUSTER 2The on-call rota has the same two people on it most weeksThe on-call rota has the…clusterCLUSTER 1Rollbacks take longer than the outage they are meant to endRollbacks take longer thanthe outage they are meant…Alerts fire so often that on-call mutes themAlerts fire so often that…clusterCLUSTER 5The same three services cause most of the pagesThe same three services…clusterCLUSTER 1Post-mortem actions are written up and then never scheduledPost-mortem actions are…clusterCLUSTER 2Reliability has no budget line of its ownReliability has no budget…clusterCLUSTER 4Every team has its own logging formatEvery team has its own…clusterCLUSTER 4The incident channel fills with people asking for status instead of giThe incident channel fillswith people asking for…The staging environment does not resemble productionThe staging environmentdoes not resemble…Customers report outages before monitoring doesCustomers report outages…clusterCLUSTER 5
Show what the room would do about each cause
Every claim on this map, with its own address (17)
Tests that fail intermittently are retried until green
Customers report outages before monitoring does
Alerts fire so often that on-call mutes them
Every team has its own logging format
The on-call rota has the same two people on it most weeks
Feature work always outranks reliability work in planning
Post-mortem actions are written up and then never scheduled
One engineer knows how the payment path actually works
Post-mortems assign actions to people who were not in the room
The service map in the wiki is two years old
The staging environment does not resemble production
Rollbacks take longer than the outage they are meant to end
Reliability has no budget line of its own
Retries are unbounded, so a slow dependency becomes a flood
The same three services cause most of the pages
The incident channel fills with people asking for status instead of giving it
Feature flags are never cleaned up, so nobody knows which paths are live
Each page says what the room certified around the claim, with the counts, and can be cited, embedded, or read as JSON.
FOUND BY THIS ROOM · 20
Retries are unbounded, so a slow dependency becomes a flood→The same three services cause most of the pages
→The staging environment does not resemble production
The service map in the wiki is two years old→Feature flags are never cleaned up, so nobody knows which paths are li
→The staging environment does not resemble production
Feature work always outranks reliability work in planning→Post-mortem actions are written up and then never scheduled
One engineer knows how the payment path actually works→The on-call rota has the same two people on it most weeks
Post-mortems assign actions to people who were not in the room→Post-mortem actions are written up and then never scheduled
Tests that fail intermittently are retried until green→Customers report outages before monitoring does
The on-call rota has the same two people on it most weeks→Alerts fire so often that on-call mutes them
→Post-mortem actions are written up and then never scheduled
→Customers report outages before monitoring does
Feature flags are never cleaned up, so nobody knows which paths are li→Rollbacks take longer than the outage they are meant to end
→The same three services cause most of the pages
Alerts fire so often that on-call mutes them→Every team has its own logging format
→Customers report outages before monitoring does
Rollbacks take longer than the outage they are meant to end→Reliability has no budget line of its own
The same three services cause most of the pages→The staging environment does not resemble production
Reliability has no budget line of its own→The incident channel fills with people asking for status instead of gi
→The staging environment does not resemble production
Every team has its own logging format→Customers report outages before monitoring does
what to do about these →