Trust Stores, Part 1: On Trusting Trust
- Jason R. Miller
- 3 minutes ago
- 6 min read
Introduction
This article is the first in a series about trust stores. Before we can get into the technical details, we need to understand, philosophically, what we are trusting.
41 years ago, Ken Thompson delivered a lecture titled, “Reflections on trusting trust” (DOI 10.1145/358198.358210) upon being awarded the Association for Computing Machinery (ACM) A.M. Turing Award. In this lecture, he dives into the software supply chain (though it wasn’t known as such at the time) and examines what, or whom, we are actually trusting when we trust a piece of software.
To that end, I’d like to talk about the state of trust today. Why do we have trust stores? Who decides what goes in them? And, what are you trusting, right now, as you read this article?
Join me on this journey through the current state of trusting trust!
Today’s trust: certificates
Today, the most ubiquitous method of establishing trust in computing is with X.509 certificates. The name comes from the ITU Recommendation that describes the standard format. (I won’t delve into the technical details here, but you can read the Wikipedia article if you are interested in learning more.)
These certificates are used to prove identity (e.g., am I connecting to the right website?) and authorship (e.g., did this company actually publish this software?).
Certificates are part of a broader system of Public Key Infrastructure (PKI). In a nutshell, PKI ensures confidentiality, integrity, and authenticity. Today, we are focusing on authenticity—specifically, how digital certificates themselves are trusted.
Certificates are signed by Certificate Authorities. A Certificate Authority (CA) verifies that a certificate requester controls/owns the certificate Subject and, if so, “signs” the certificate. Our systems trust this CA and consider the certificates that it signs to be valid and accurate.
There are two categories of CAs: root and intermediate. An entity's certificate is typically signed by an intermediate, or subordinate, CA. Root CAs typically only verify and sign intermediate CA certificates. The following graphic illustrates this.

For all of this to work, we have to decide which CAs we should trust. On the public Internet, this decision is typically made for us. A list of trusted CAs is included in our operating system and other software.
But where does this list come from?
What am I trusting right now?

There has to be some baseline that we start with when we trust. We can’t reasonably travel to every CA and hand-enter their public certificate into our system. Let's take a look at where this trust baseline comes from and how we consume it.
CCADB
Our first stop is the Common CA Database (CCADB). Funded by The Linux Foundation, “the CCADB is a repository of information about CAs whose root and intermediate certificates are included within the products and services of several Root Store Operators.”
Root store operators include organizations like Apple, Google, Microsoft, Mozilla, Opera, and Cisco. So, when you use a product from one of these organizations, you are also using the set of CAs that they have included, which is typically derived from the CCADB.
CAB Forum
Our next stop is the CA/Browser (CAB) Forum. The CAB Forum maintains Baseline Requirements (BR), or minimum standards, for issuing certificates, verifying identity, and improving the security of CAs.
The main difference between the CCADB and the CAB Forum is that the CAB forum represents the interests of certificate consumers (i.e., how should public trust work?), while the CCADB represents the interests of root store operators (i.e., which CAs should be trusted?). These interests tend to overlap, so many organizations participate in both.
Mozilla’s CA Certificate Program
Finally, Mozilla’s CA Certificate Program governs inclusion of root certificates in Network Security Services (NSS), a set of open source libraries designed to support cross-platform development of security-enabled client and server applications.
These libraries are used in some open source software, including all Mozilla products.
What does it take to be a root CA?
Now that we know a bit about who manages root CAs, what does it take to be a trusted root CA?
Well, it's easy. Just complete this form.
Just kidding. A lot goes into being trusted as a root CA.
There is a set of required disclosures and rigorous independent audits that an issuer must go through. If you’re curious, you can view the self-assessment spreadsheets. (When I looked, there were over 500 controls on the Basic Requirements tab alone!)
To get a feel for what the life of a root CA is like, I highly recommend checking out this Ars Technica article about Symantec’s certificate vault. Some key takeaways:
Just getting into the vault requires two sets of two employees using two sets of identification.
The hardware security modules (HSMs) containing the root key require three different people to plug in their physical keys and enter a PIN before the root key can be decrypted.
A "key ceremony" can take as long as 12 hours. "Every keystroke, mouse click and utterance" is recorded for posterity.
Suffice it to say, this is not taken lightly!
How are these delivered to me?
So, how do you actually get a trust store, so you can start trusting public websites? You can download one, of course, but how do you trust the site that’s hosting it?
In short, your trust store comes with your operating system (OS). Your OS vendor is likely one of the Root Store Operators that we talked about earlier, and they're likely a member of the CAB Forum.
Whom are we trusting?
To quote Ken Thompson, “you can't trust code that you did not totally create yourself.” Unfortunately, we have to, or we’d never get anywhere. So, whom or what are we trusting?
The people who manage the Common CA Database.
By proxy, the people and processes that manage Certificate Authorities.
The people and processes putting the trust store in the operating environment. The operating environment can be an operating system, a container image, an orchestration platform, or software with a bundled trust store.
The people who wrote the code that trusts the trust store.
The people who wrote the compiler that compiled the code that trusts the trust store, and so on. Basically, the software supply chain.
Anything we add to the trust store.
The organizations we talked about previously help us implicitly trust some of the above, removing our need to go down the rabbit hole.
Important points
At this point, I’d like to bring up some important points about the limitations and scope of certificates and trust stores.
When it comes to trusting a data source, certificates only verify the identity of the entity you're communicating with. That is, in and of themselves, they only verify the last link in the chain.
Certificates do not verify anything about the data being exchanged. This means they do not address concerns such as:
provenance (origin and chain of custody),
intent (are the data malicious?),
reliability (are the data accurate?), or
trustworthiness (have the data been vetted?).
A secured website can provide incorrect information. A signed software package can contain malicious code. Certificates don't necessarily help with this.
A certificate verifies only that an entity can prove its identity. It does not verify that an entity is trustworthy in the broad sense.
Some common examples of risks that can occur even with a properly-signed certificate serving the content include:
Other tools can help mitigate and identify these risks. But remember that, ultimately, you are responsible for vetting the source!
What’s next?
I hope this article provided some insight into the state of trust today.
We rely on PKI and X.509 certificates to establish trust.
Organizations such as the CCADB and CAB Forum help us establish a trust baseline.
Root store operators decide who gets listed in trust stores.
OS vendors provide those trust stores to us.
There are important limitations to certificates and trust stores in reducing our overall risk.
In upcoming articles, we'll discuss how we actually use trust stores, including some common formats. We'll discuss how to add custom CAs to our trust store. And we'll discuss some language- and runtime-specific configurations and best practices.
See you there!
