Starting Server of users with 2i2c admin emails from admin panel fails on earthscope
| Field | Value |
|---|---|
| Impact Time | Oct 15 at 10:28 to Oct 15 at 10:45 |
| Duration | 17m 6s |
Overview¶
An authorized admin user on the earthscope hub attempted to Yuvi Panda start a server for 2i2c staff accounts via the JupyterHub admin panel. Because these accounts were set up for Google Auth emails, the server startup failed, triggering an alert for server startup failure. Oct 15 at 10:28 to Oct 15 at 10:45
What Went Well¶
There were no user facing issues
Our automated alerting setup caught it immediately
Action Items¶
Make sure that custom auth hubs don’t have unrelated 2i2c staff engineer accounts added, as this is confusing in the admin panel 2i2c
-org /infrastructure #6945
Timeline¶
Oct 15, 2025¶
| Time | Event |
|---|---|
| 10:26 AM | Authorized admin user logs into the hub, and accesses the admin panel. They try to start server of a 2i2c user who is listed as admin. This server start fails, because the earthscope hub uses auth0 auth but the admin accounts were added as google email addresses 10:28 AM Description:[FIRING:1] Server Startup Failed earthscope prod (same day action needed) (View Message) Starting Server of users with 2i2c admin emails from admin panel fails on earthscope |
| 10:37 AM | 2i2c |
| 10:40 AM | Existing 2i2c staff email based jupyterhub admin accounts are manually removed through the JupyterHub interface 10:45 AM Starting Server of users with 2i2c admin emails from admin panel fails on earthscope |