Open source software is everywhere today. A 2024 Synopsis report showed that 96% of the commercial code bases they sampled contained open source software, and 77% of the code within those code bases was open source. Similarly, a 2022 Linux Foundation study found that 70-90% of any given software code base is made up of open source components.
Figure 1. Data from two separate reports illustrates the ubiquity of open source software in our global code bases.
Not only do most open source code bases contain many different open source projects, but the total number of projects is rising rapidly. The chart below, which uses data from Node Package Manager, shows more than a threefold increase in project releases between June 2019 and the end of 2022.
Anchore: Open Source is Bigger Than You Can Imagine
Figure 2. This chart from a Node Package Manager (NPM) report shows new project releases increasing from 1 million project releases in June 2019 to 3.6 million releases in 2022!
A software project today most likely also has many dependencies. This visualization from an actual program is good food for thought. It’s a great illustration of what we need to consider as open source developers when adding a new project to our application.
Credit: https://www.gatsbyjs.com/
Figure 3. If this software program dependency graphs gives you a small panic attack, you’re not alone. But it’s critical to understand a project’s many dependencies before incorporating it into your software.
To consume open source software securely, we need to carefully evaluate the potential dependencies we include in our own projects, and then take responsibility for monitoring those projects, with awareness of both direct and indirect or transitive dependencies.
In addition to managing large numbers of dependencies, developers also face more reported vulnerabilities. There’s been a massive rise in the number of common vulnerabilities and exposures (CVEs) reported in all software between 2016 and 2023, but that doesn’t necessarily mean software is becoming more vulnerable. As we’ve seen, there's a lot more software being produced, as well as more awareness of the importance of security, with more security researchers looking for and reporting vulnerabilities.
Figure 4. The chart shows CVE metrics over time from cve.org. In 2016, 6,457 CVEs were reported, with the number rising to 28,961 in 2023.
Here’s a little quiz for you:
True or False?: A project with no CVEs is more secure than a project with many CVEs.
If you answered “True” you might be surprised to learn that the correct answer is “False.” A project with no CVEs is often one that has not been audited for security issues, whereas a project with CVEs is a project that is scrutinized from a security perspective.
How to Evaluate Open Source Software
The widespread use of open source software presents a unique set of challenges to software development because of its openness and distributed nature. Within the vast collection of projects in the open source ecosystem, many projects are well-maintained, secure, and up to date; however, there are also many projects that lack the human resources required to ensure they’re well managed.
When choosing projects to use in your software, careful consideration will save you a lot of work and potential headaches. In this article, I’ll cover what you need to look for when choosing projects and how to evaluate the projects already in your code base.
Who’s Maintaining It?
Anchore: Open Source is Bigger Than You Can Imagine
Figure 5. This graph from a Node Package Manager (NPM) report shows the number of maintainers per package. You’ll notice that the bulk of the projects have a single maintainer. If you’re thinking that’s a recipe for maintainer burnout, it is.
A surprising number of open source projects today have only one or very few maintainers. Maintainers in projects that face this issue are under pressure to keep projects secure, address issues, and push through the challenges that come with dedication to a project—often with little to no pay or support. This situation can lead to maintainer burnout, creating health implications for both the maintainers themselves and the projects they oversee. In Intel’s annual open source community survey, 45% of survey respondents cited maintainer burnout as their top challenge.
For these reasons, it’s important that a project has enough maintainers and it’s a factor to consider when choosing a project. You also want to see maintainers from more than one company or organization because you don't want the kind of siloed development that can happen when a project lacks diversity among maintainers. Maintainers are sometimes, but not always, listed on a project’s GitHub site in a file called MAINTAINERS.md, and you can evaluate maintainer activity by viewing pull requests and commits.
What’s the Activity Level?
In addition to identifying the number of maintainers supporting a project, you’ll want to look at the project’s activity level for signs that it’s being actively maintained. For example, when was the last commit? What does the issue queue look like? Are issues being addressed, and if so, how quickly? A lack of activity can indicate abandonment, inadequate support, or project immaturity. Any of these situations can lead to security risks and result in you having to take on more ownership responsibilities than would be warranted, given what you hope to get from using the project. Project status can change, so even if it has a lot of stars and users, you’ll need to make this check.
What’s the Release Cadence?
Look at the project’s release cadence to see if there has been substantial activity within the last year. Is the release process documented, and are releases occurring regularly? Also, look at versioning. It might be a red flag if the latest release is an alpha or beta release. In a new project, even the first few production releases might not be mature.
Other considerations:
- Are security issues being addressed, and are timely patch releases created?
- Are versions of dependencies being updated?
- Are there automated tests? Any type of automated test coverage is an indicator of some project maintenance, and the more test coverage the better.
How is the Project Governed?
How are decisions made? Larger projects will have well-defined processes and committees, and these should be documented. For smaller projects, read through documentation, contributor guides, and recent pull requests to evaluate decision-making processes within a project. A lack of governance structure can indicate an immature project and be a concern relative to reliability and security in a project of any age.
Ideally, the project will have an OSI-approved license, but at the minimum, make sure that a clearly stated license is included in the repo. A project missing a license presents potential legal risk and is not following best practices, which suggests there may be other cut corners.
How Engaged is the Community?
As a starting point to get a feel for a project’s community engagement, look for a contributor guide. The presence of a contributor guide can be a great indication that a project is being well cared for, while the lack of one can indicate a lack of maturity. A widely used project may have a more engaged community, so look for signs that the community is helping work toward best practices by contributing issues and patches. Also, ensure the project has a documented and secure bug-reporting process.
Project Security Evaluation Tools
Beyond the steps we’ve just described for reviewing projects, many people in the open source community are working on tools to make it easier to evaluate project health and security. I’ll cover three of them: the CVE Binary Tool, the OpenSSF Best Practices Badge, and the OpenSSF Scorecard.
The CVE Binary Tool
The CVE Binary Tool is a free open source tool that helps find vulnerabilities in software. It uses the list of Common Vulnerabilities and Exposures (CVE) from the National Vulnerability Database (NVD) along with vulnerability data from Red Hat, Open Source Vulnerability Database (OSV), Gitlab Advisory Database (GAD), and Curl. It includes a binary scanner to find the packages included in a software code base and other tools to scan known component lists, such as CSV files, SBOM formats, and more.
Credit: https://github.com/intel/cve-bin-tool
Figure 6. The image above shows the steps the CVE Binary Tool takes to create its report.
Using cve-bin-tool can be as simple as manually scanning a directory on the command line as in the commands below. The tool will output results to the console by default but can be configured to output to formats including CSV, PDF, and JSON.
pip install cve-bin-tool
cve-bin-tool <directory /file>
OpenSSF Best Practices Badge
Evaluating a project using the OpenSSF Best Practices Badge can also be useful. This badge is a way for Free/Libre and Open Source Software (FLOSS) projects to demonstrate that they follow best practices. Projects can use this web application to voluntarily self-certify. The OpenSSF Best Practices Badge makes it possible to quickly assess whether or not a FLOSS project is following best practices. At this point, thousands of projects are using this indicator.
For example, the Gramine Library OS with Intel SGX includes the OpenSSF Best Practices badge, as shown at the top of the project’s README.rst file. If you click the OpenSSF Best Practices tag, you’ll see the OpenSSF Best Practices report for the project.
Figure 7. A README.md screenshot showing that a project has been evaluated using the OpenSSF Best Practices Badge.
OpenSSF Scorecard
You might be familiar with the OpenSSF Scorecard. It’s a great tool for managing your repos, and it can also help you evaluate projects. It's a quick and easy project assessment that can help protect you from malicious packages, poorly maintained projects, and, to some extent, compromised build systems. You can use it on the command line or include it in a GitHub action.
Figure 8. A screenshot showing the OpenSSF Scorecard README.md. Read this file for an overview of Scorecard and detailed information about how to configure and use it.
Evaluating your software with Scorecard can be as simple as running it on the command line.
scorecard --repo=<your choice of repo e.g. github.com/intel/cve-bin-tool>
You can also view the Scorecard’s output using the web interface for a long list of regularly scanned projects. For example, here’s the browser output for Python Fire, a Google project:
Figure 9. This screenshot shows OpenSSF Scorecard browser output for Python Fire, a Google project. The project got an overall score of 5.5 out of 10 with a prioritized list of issues to tackle ranging from low to critical.
You can see that Python Fire had an overall score of 5.5 out of a possible perfect 10, combining scores in multiple categories—neither an exceptionally high nor low score. However, the concept of what is and isn't a good score is a complicated conversation. How you view the report depends on your project’s needs. For example, this report shows that branch protection is turned off. That’s an easy fix that the team will want to address.
Note that the report shows the criticality of each of these checks. In this example, you see that having dangerous workflows is a lot more of a risk than not including the license. From a security perspective, dangerous workflows should be dealt with urgently, as opposed to the license, which has a lower impact on the overall score.
You Ingest It, You Own It
If you include a project in your code base, you need to keep tabs on it. Evaluate whether the community and the maintainers are doing everything they can to be secure. As a beneficiary of the project, and in your role as a responsible open source citizen, do your best to contribute back to the project, for example, by reporting and fixing issues.
Let's all work toward creating and consuming better and more secure open source software!
Learn More
- Visit the Open Ecosystem Evangelism GitHub page for Intel and OpenSSF tools, guides, and community resources, and links to articles and source materials.
- Subscribe to the Open at Intel podcast for critical conversations about security, artificial intelligence, cloud computing, Linux, and more, bringing together some of the best minds from Intel and the open source community.
- Connect with the Intel Open Ecosystem Community & Evangelism team on LinkedIn and X.
- Read Navigating the Maze of Open Source Licenses for advice on how to choose the right open source license for your project.
About the Author
Katherine Druckman is an open source security evangelist at Intel, and she enjoys sharing her passion for a variety of open source topics. She is a long-time open source advocate, developer, and podcaster, and is currently the host of Open at Intel and co-host of the FLOSS Weekly and Reality 2.0 podcasts. Previously, Katherine spent over a decade at Linux Journal. A passionate Drupalist since she first downloaded a tarball in 2005, she has also been a Drupal contributor and engineer.