Protecting our hubs against the CopyFail kernel exploit

The recently disclosed CopyFail Linux kernel zero-day (CVE-2026-31431) opens up a way for code running inside a container to break out onto the underlying node. We took a close look at our hubs to confirm whether they were exposed, confirmed that our hubs are likely not at risk, and added another layer of protection just in case.

Are 2i2c’s hubs at risk? #

No - based on our testing and mitigation efforts, our hubs are not vulnerable to CopyFail.

Why do we think we’re not at risk? #

• We tried to reproduce the exploit on a staging hub by following the public Kubernetes proof-of-concept on both AWS and EKS, and the exploit was unable to break out of the container.
• Existing JupyterHub hardening on Kubernetes from jupyterhub/kubespawner#545 (originally added by Yuvi in 2021 in response to a different security issue) had already significantly reduced our risk exposure, and the exposure of anyone else running Z2JH (the standard way to deploy JupyterHub on Kubernetes).
• As an extra layer of protection, we deployed copyfail-ebpf-k8s across our hubs in 2i2c-org/infrastructure#8227. It blocks the specific kernel feature that CopyFail depends on. See the project’s explanation for how that works.
• We’ve upgraded all GKE clusters to use a patched image in 2i2c-org/infrastructure#8230.

What else did we look into #

• Deckhouse’s mitigation was too platform-specific for us.
• OVHcloud’s modprobe blocking likely won’t work on Amazon Linux 2023, since the relevant module is built into the kernel image.
• AL2023 security advisories - no patched AL2023 image is available yet, so we can’t rely on a kernel-level fix from AWS for now.

Acknowledgements #

• Huge thanks to Georgiana for the deep dive into the exploit and whether we’re exposed here.
• Thanks to Yuvi for the PR that reduces JupyterHub’s exposure to this back in 2021!
• Thanks to iwanhae for the eBPF daemonset we deployed in Kubernetes, and to JupyterHub for the upstream kubespawner hardening that lowered our exposure.
• Thanks to our collaborators at NASA VEDA for the ongoing conversations about hub security.