It’s hard to imagine the open source world without containers, but there was a time not so long ago when no one knew what the technology would look like—and who would popularize it first. In this interview, we chat with Phil Estes, a principal software engineer at AWS, about the race to containerization and how technology titans came together to help standardize the container runtime ecosystem we use today. He also shares about his work on the containerd project and its journey to Cloud Native Computing Foundation (CNCF) adoption.
Listen to the full episode here. This conversation has been edited and condensed for brevity and clarity.
Katherine Druckman: You’re very involved. You’re a maintainer. You’re a CNCF ambassador. Will you tell us what you do?
Phil Estes: I’m a principal engineer at AWS. I was a longtime IBMer before that, and the continuity through both employers has been my work in the container runtime quarter of the ecosystem. I was a Docker maintainer back in the early days of Docker and helped bring containerd to the CNCF. Docker donated it, but a lot of us early maintainers were part of those discussions. Now seven or eight years in, containerd has become a mature project used by Kubernetes and many other projects. Alongside that, I’ve been involved in the Open Container Initiative (OCI), which oversees the specs and standards around what a container is and what a container image is. Over the years, I became a CNCF ambassador and use that as an opportunity to share what I’ve learned about these projects and the OCI at conferences, meetups, and everything in between.
The Evolution of containerd
Katherine Druckman: When I was working at Linux Journal, we published a 2013 article titled, “Containers—Not Virtual Machines—Are the Future Cloud.” That wasn’t that long ago, and yet now containers are almost as ubiquitous as something like Linux. How did we get here?
Phil Estes: Docker has a complicated history. There were detractors of their maturity model that led to some conflict in the open source world, but I always credit them with creating an easy transition to help people understand how to run a container. When they created Docker, the tool, people really got it. Docker used the technologies that had existed a decade before them. Pieces of the container world existed long before Docker, but they created the standard concept of an image and then a repository, Docker Hub, with tons of images that you could use out of the box. Whether you needed Redis, MySQL, or a web server, all these images were there, and you just typed, “Docker, run Redis,” and magic happened. When that tool came out around 2013, you could sense that people were having an aha moment: “I can package my software like this, and it’s going to be super easy and faster than a VM.” From there it just steamrolled. There was a sense that Docker came about and then Kubernetes, but if you look back and search YouTube for talks about Kubernetes, it’s almost like they were growing up together. Kubernetes was right there in 2014. It arrived at the right time when people were looking for better, faster ways to deploy software and develop locally, and it was this same idea of packaging everything the way you need it to be on the server. All the problems people had with configuration drift and SSHing to a server to edit a conf file—containers arrived to solve a lot of these challenges.
Katherine Druckman: Tell us about the relationship between containerd and Docker. How did it evolve into a donated project?
Phil Estes: Docker was a monolithic tool. It was the Docker engine, it was your client, and it was your build tool, which made it super easy for adoption—there was a single binary you could put on a Linux system that allowed you to do all these things. The open source world was unsure who was going to win the container wars, so to speak. rkt came from CoreOS around the same time, and they had an actual spec that they wanted to produce around an application container interface. So fast forward, the OCI was formed to standardize these ideas together with CoreOS, Docker, Red Hat, Google, and all the major players. Out of that came the first version of containerd as a way to break Docker into its component pieces. So Docker could have a client, it could have the engine, and it could have container data that was actually managing these OCI bundles of what containers and images are. All of a sudden, Docker was no longer a monolith—it was multiple projects that combined together to provide the features of building and running containers. That initial version of containerd was really born out of this decomposition of what had been a single piece of software.
As Kubernetes became popular, there was this idea that Docker was too much of a runtime for Kubernetes—it had too many features. It had its own orchestrator at one point. Since containerd had already established a sort of process manager around OCI bundles, we asked, “Why don’t we expand it to be more of a full runtime? Docker can still use it to manage containers on its behalf, but Kubernetes can use it as a container runtime instead of the whole Docker engine.” That’s when containerd grew up from being this simple life-cycle manager to having registry push/pull operations and snapshotters to manage the on-disk image cache. That’s also when it was donated to CNCF, in early 2017.
Katherine Druckman: From your perspective as a maintainer, how does life change when a project becomes part of the CNCF?
Phil Estes: The CNCF has grown up considerably in seven years. Back then, we were one of 13 projects in the CNCF. But the CNCF was definitely valuable because it helped give confirmation to companies who wanted to consume these projects that the projects were fully open and had good governance. containerd had already tried to adopt good core governance, but the CNCF helped ensure the projects had multiple vendors involved and that there wasn’t single-vendor control. We also benefited from the work CNCF did around marketing the project and giving us time at KubeCon to talk about the project. So coming into the CNCF was part of our maturing process. It helped us gain credibility and adoption through what has grown to be a huge ecosystem.
Katherine Druckman: I imagine the challenges you faced in 2017 were very different than those you face today. What do your challenges look like now?
Phil Estes: The CNCF ecosystem is so big, it can be hard to differentiate and attract contributors. I don’t feel like we’ve had as much trouble because containerd has the maturity of people knowing about it and using it with Kubernetes and Docker. We haven’t experienced some of the challenges that a young project just joining the CNCF might have today. But we do feel the challenge of attracting contributors in a world where the container runtime is one very small piece. Now people are talking about AI, ML, and all the exciting, shinier objects in our ecosystem; container runtime is not necessarily the most exciting thing to work on. We feel we have a niche where we tend to attract people who are operating clouds because they’re the ones consuming containerd in their cloud offerings.
What’s Next for containerd
Katherine Druckman: Tell us about what’s on the road map. What are critical issues in the containerd project you want to see being addressed?
Phil Estes: We’re in an exciting time where we just cut our 2.0 beta. In 2.0, we’re excited about the Kubernetes use case especially. We’ve had some contribution around a new sandbox API. Not everyone wants to run just containers—they want to run WebAssembly and isolated containers inside VM isolation. This new sandbox API is a cleaner way for these kinds of projects to live within the containerd ecosystem. We also have a growing use of custom snapshotters for specialty file systems and lazy loading, so AWS created the SOCI Snapshotter for seekable OCI images. As much as the core of containerd is a mature and well-used project, some of these sandbox ideas and the snapshotter expansions are neat ways that we’re seeing people extend containerd in ways that we hoped for and envisioned.
Katherine Druckman: You mentioned competing for contributors. For those who are excited about containerd, where’s a good place to start contributing?
Phil Estes: I’m sure this is a refrain of many open source maintainers, but we’ve struggled to keep documentation up to date and relevant. We have different use cases, and it’s hard for us to have multiple lines of documentation. For Kubernetes users who just want containerd to be the runtime, that’s more like offering a bunch of administration knobs and configs that they want to know about. For people who want to extend containerd for special use cases, that’s a whole different kind of documentation. We’re always looking for reviewers who just want to help the project deal with the influx of contributions, issues, and everything that goes along with the maintenance of a project.
The Value of Mentorship
Katherine Druckman: What else are you excited about right now in the open source world?
Phil Estes: I’d love to review pull requests and contribute more if I had the time, but I’m enjoying my current mentorship role. I just gave a talk at Linux Foundation’s member summit about mentoring the next generation of open source developers. We’ve got a lot of younger college hires coming into AWS and it’s exciting for me to see them get excited about open source contribution. I enjoy the community aspect of it. Everyone in the tech world is developing software in some sense, but seeing them enjoy the community aspect of it has been a really fun part of my job that I really haven’t had before now.
To hear more of this conversation and others, subscribe to the Open at Intel podcast:
- openatintel.podbean.com
- Google Podcasts
- Apple Podcasts
- Spotify
- Amazon Music
- Your favorite podcast player (RSS)
About the Author
Katherine Druckman, Open Source Evangelist, Intel
Katherine Druckman, an Intel open source evangelist, hosts the podcasts Open at Intel, Reality 2.0, and FLOSS Weekly. A security and privacy advocate, software engineer, and former digital director of Linux Journal, she’s a longtime champion of open source and open standards.
Phil Estes, Principal Engineer, AWS
Phil Estes is a principal engineer for Amazon Web Services (AWS), focused on core container technologies that power AWS container offerings like Fargate, EKS, and ECS. Phil is an active contributor and maintainer for the CNCF containerd runtime project and participates in the Open Container Initiative (OCI) as a member of the Technical Oversight Board (TOB). He is also a current member of the 2023 CNCF Ambassador class and enjoys speaking on container technology topics and events worldwide.