At this year’s Open Source Summit from The Linux Foundation, the main event involved a question-and-answer style keynote featuring Linux creator and master maintainer, Linus Torvalds, that included discussion topics curated by VMware‘s Chief Open Source Officer, Dirk Hohndel.
There were a ton of topics covered in the 40 minute discussion, but one topic appeared to stand out the most to many in the Linux community. The problem regards the difficulty that Torvalds and other kernel developers have experienced with bringing new, younger engineers into Linux kernel development and especially maintainership, something that is essential to the overall health of the Linux kernel as many of its current master developers and maintainers are becoming older and won’t be around forever.
Therefore, without an influx of young, intelligent engineers to continue the work, the Linux kernel could be in serious trouble somewhere down the line. When asked if the Linux developers were becoming “grey”, Torvalds responded with:
“The new people are the ones who are often doing the [programming] work. We have managers and retainers who are old and starting to grey, that’s, I think, a completely different issue. But, we do have a generation of people in their 30s who are moving up the ranks of maintainers so that we have that next wave of people to take over eventually. I mean, look, we’ve been doing this for almost 30 years so we need to start thinking the next 20 to 30 years. And so, we need to have that next generation.“
So, it appears that the problem doesn’t lie in finding engineers to program on the Linux kernel, but the hard part is finding kernel maintainers, which is a job that many younger people could consider boring, as it is much more of a manager-esque position instead of specifically writing code. It also requires quite a unique skill set and even more experience with working on the kernel, its many subsystems, and the connections between those subsystems.
It appears that this is how the project tends to work. Someone starts by sending in patches of code that get reviewed, and if they are deemed up to standards, get merged into the mainline kernel. After years of working on the codebase, which is vast and requires several areas of expertise, they may take on a position as a maintainer of a small subsystem in the codebase. Again, after years of trusted work as a maintainer, they might begin moving on to become maintainers of larger and larger subsystems of the kernel, replacing the old maintainers as they leave.
This process takes a considerable amount of time and effort as well as a massive line of trust built between the up-and-coming developer and maintainers. It isn’t exactly an easy process to ensure that the patches that are submitted are necessary and won’t cause regressions in other, seemingly-unrelated parts of the nearly 30 million lines of code that make up the Linux kernel. So, trust is a must…and that can only be built up after years of experience and consistently good work.
One of the biggest barriers to entry for new software engineers interested in contributing to the Linux kernel today is the fact that they use a mostly outdated process for contributing that has been in place since it really took off in the mid-1990s. Namely, the use of mailing lists is the only way to provide your patches to the source code.
With the invention of version control systems like git (also from the mind of Torvalds) and especially the easy to use web integration platforms like GitHub and GitLab, many newer software developers and engineers become familiar with this easier to navigate service early on in their careers. Mailing lists can be chaotic and hard to follow when you come from the world of pull requests and the carefully recorded comments kept nice and tidy in a singular place.
Therefore, if the Linux kernel embraced a more modern way to take contributions from developers–one that is essential to basically any software role today–I think that the number of younger contributors might rise and diversify in a good way. Of course, organizing the near 30 million line codebase is another story, but it could be done.
As for me, I hope to see a shift in the kernel developer’s method of work to encourage contributions from talented people all around the world from all ages and backgrounds. Actually understanding the Linux kernel well enough to make changes to the codebase is hard enough–add in the difficulty of submitting those changes as well as keeping track of reviews and the pot of possible talent once again shrinks.
If you would like to learn more about the Linux kernel itself, you can find a really handy website as an introduction to new or prospective contributors here. Also, the main website for the Linux kernel itself (including the downloadable source code) can be found here.
If you’re curious about the Linus’ talk with Dirk Hohndel at the Open Source Summit, it is linked below for your viewing pleasure:
Super good article! I very much enjoyed reading it and often have thought about the longevity risk of the kernel. While its protection and power are asserted from its open-source nature, Linus, and those like him, are its life.
If Linus ever passes this torch on, I’ve often wondered who would be skilled enough and willing enough to carry it forward in the same spirit. Its almost like asking someone to exchange their life/occupation to spend all of their time focused and working on this one key project.
… This article put me in the mood to watch ‘Searching for Bobby Fischer’.
Join the discussion at forum.tuxdigital.com