Invalid configuration data breaks enforce-nfs-quota container
| Field | Value |
|---|---|
| Impact Time | Oct 16 at 21:26 to Oct 16 at 21:51 |
| Duration | 24m 23s |
Overview¶
An engineer deployed invalid Python config to earthscope prod Angus
What Happened¶
The enforce-xfs-container failed to properly load and encountered pod restarts Oct 16 at 21:26 to Oct 16 at 21:51
Resolution¶
Revert invalid config, and then push correct config 24m 23s
Where We Got Lucky¶
The engineer was watching the deployment for errors *All times listed in this report are in Edinburgh, London.
Timeline¶
Oct 16, 2025¶
| Time | Event |
|---|---|
| 9:26 PM | Engineer deployed invalid Python config |
| 9:30 PM | Engineer reverted change |
| 9:35 PM | Engineer deployed valid JSON 9:51 PM [FIRING:1] storage-quota-home-nfs pod has restarted earthscope prod (storage-quota- home-nfs-5d64656464-wg58d same day action needed) |