Celebrating our progress in Q1 2022
Quarter 1 of 2022 just wrapped up, and the 2i2c team has been busy making improvements across our infrastructure, organization, and operations. This is a quick post to celebrate the work we’ve done over the past three months, and to briefly share what we’re working on next.
These are the main highlights from this quarter - if you’d like to check out more of the work that we’ve done, see:
Infrastructure improvements #
This quarter we did a deep dive into a number of core infrastructure improvements for our Managed JupyterHubs Service. Here are a few highlights:
- Infrastructure reliability and efficiency. We improved the resiliency, reliability, and efficiency of our deployment infrastructure. For example, we refactored our hub configuration so that each community is better-able to track it, we implemented validation steps to ensure that we don’t accidentally push incorrect config to the hubs, and we’ve significantly improved our CI/CD pipeline to push deployments out to our hubs more efficiently.
- Automatic deployments to commercial cloud. With the ICESat hackweek as a test-case for AWS, we’ve finished automating the deployment of clusters and hubs to each major commercial cloud. (there’s not a specific issue for this as it has been a multi-month effort over many PRs and issues!)
- CILogon authentication. CILogon is a non-profit organization that provides “single-sign on” authentication services for the same communities that 2i2c serves. We’ve partnered with them to prototype using CILogon for 2i2c’s hubs, which should make it much easier for communities to user their own institutional sign-ons.
Communities we’ve served and lessons learned #
As described in our Managed Hub Services strategy, our goals for this phase of our organization are to balance serving communities of practice and learning where we can improve our infrastructure and practices. With that in mind, here are a few highlights of communities we’ve served, and what we’ve learned from it:
- We grew a hub for the University of Toronto to around 4000 monthly users. This has taught us a lot about where our support and operations can and cannot scale, and where we have gaps in our sustainability / pricing model.
- We deployed CILogon on a hub for a class at Australian National University. This gives us an opportunity to work out any UX issues and improvements to be made before a deeper CILogon integration.
- We deployed a dedicated database per user for a databases course at UT Austin. This is helping us learn more about how to pair slightly more customized per-user infrastructure with our standard hub setups, as well as how our Right to Replicate model could be followed for more complex setups like a database.
- We ran an event hub for the ICESat2 HackWeek at the University of Washington. This helped refine our infrastructure and expertise with AWS, as well as improved our event “ready mode” practices.
- We deployed a new hub for the LEAP project. This has given us an opportunity to prototype new processes for pass-through cloud costs to simplify our deployments.
Organizational improvements
Beyond our technical and community impact work, we’ve made a lot of significant organizational improvements as well.
- We designed a new role in Product and Community Management. We’re excited for this new hire to spearhead efforts in guiding and developing relationships with the communities we serve, as well as guiding and collaborating with our engineering team in developing our services.
- We designed a new Project Manager role. Our engineering team had been operating as a largely autonomous and independent group, but we’ve realized that we would benefit from someone to help coordinate our actions and plans, especially as we balance more operations/support issues in addition to new development. This new role is an experiment at growing this capacity within our team, in the hopes that we can dedicate a team member to it in the future.
What’s next #
We are still working out our major priorities for the oncoming quarter, but have a few major projects in the works that we’re hopeful to make progress on quickly. Here are a few major examples:
- Improve our process and operations around supporting our users. We are discussing first- and second-line support processes to make our team more responsive and effective at resolving incidents.
- Improve our invoicing and contracting process. We are discussing how to reduce toil associated with invoicing in order to make this practice more reliable and efficient, along with our fiscal sponsor Code for Science and Society.
- Improving our reporting and monitoring infrastructure. We’d like to boost our ability to monitor activity on each of our hubs in order to identify when something abnormal is happening and get ahead of any potential problems (e.g., to avoid unintentionally large cloud bills). We’d also like to improve our usage reporting to more create more accurate cloud bills for hubs running on multi-tenant clusters.
There is a lot more planned for 2i2c, and if you’re curious to see what we’re up to, we invite you to check out our team compass to learn about our practices, and watch our activity on GitHub to see our work.
Many thanks to the amazing 2i2c team, and the multiple open source and scholarly communities that we collaborate with to make all of this possible. In addition, we are grateful to our funders for making this possible. We are looking forward to Q2! 🎉
