Understand the Notary service architecture
Estimated reading time: 15 minutesOn this page, you get an overview of the Notary service architecture.
Brief overview of TUF keys and roles
This document assumes familiarity with The Update Framework, but here is a brief recap of the TUF roles and corresponding key hierarchy:
-
The root key is the root of all trust. It signs the root metadata file, which lists the IDs of the root, targets, snapshot, and timestamp public keys. Clients use these public keys to verify the signatures on all the metadata files in the repository. This key is held by a collection owner, and should be kept offline and safe, more so than any other key.
-
The snapshot key signs the snapshot metadata file, which enumerates the filenames, sizes, and hashes of the root, targets, and delegation metadata files for the collection. This file is used to verify the integrity of the other metadata files. The snapshot key is held by either a collection owner/administrator, or held by the Notary service to facilitate signing by multiple collaborators via delegation roles.
-
The timestamp key signs the timestamp metadata file, which provides freshness guarantees for the collection by having the shortest expiry time of any particular piece of metadata and by specifying the filename, size, and hash of the most recent snapshot for the collection. It is used to verify the integrity of the snapshot file. The timestamp key is held by the Notary service so the timestamp can be automatically re-generated when it is requested from the server, rather than require that a collection owner come online before each timestamp expiry.
-
The targets key signs the targets metadata file, which lists filenames in the collection, and their sizes and respective hashes. This file is used to verify the integrity of some or all of the actual contents of the repository. It is also used to delegate trust to other collaborators via delegation roles. The targets key is held by the collection owner or administrator.
-
Delegation keys sign delegation metadata files, which lists filenames in the collection, and their sizes and respective hashes. These files are used to verify the integrity of some or all of the actual contents of the repository. They are also used to [delegate trust to other collaborators via lower level delegation roles. Delegation keys are held by anyone from the collection owner or administrator to collection collaborators.
Architecture and components
Notary clients pull metadata from one or more (remote) Notary services. Some Notary clients push metadata to one or more Notary services.
A Notary service consists of a Notary server, which stores and updates the signed TUF metadata files for multiple trusted collections in an associated database, and a Notary signer, which stores private keys for and signs metadata for the Notary server. The following diagram illustrates this architecture:
Root, targets, and (sometimes) snapshot metadata are generated and signed by clients, who upload the metadata to the Notary server. The server is responsible for:
- ensuring that any uploaded metadata is valid, signed, and self-consistent
- generating the timestamp (and sometimes snapshot) metadata
- storing and serving to clients the latest valid metadata for any trusted collection
The Notary signer is responsible for:
- storing the private signing keys wrapped and encrypted using Javascript Object Signing and Encryption in a database separate from the Notary server database
- performing signing operations with these keys whenever the Notary server requests
Example client-server-signer interaction
The following diagram illustrates the interactions between the Notary client, server, and signer:
-
Notary server optionally supports authentication from clients using JWT tokens. This requires an authorization server that manages access controls, and a cert bundle from this authorization server containing the public key it uses to sign tokens.
If token authentication is enabled on Notary server, then any connecting client that does not have a token is redirected to the authorization server.
See the docs for Docker Registry v2 authentication for more information.
-
The client logs in to the authorization server via basic auth over HTTPS, obtain a bearer token, and then present the token to Notary server on future requests.
-
When clients uploads new metadata files, Notary server checks them against any previous versions for conflicts, and verifies the signatures, checksums, and validity of the uploaded metadata.
-
Once all the uploaded metadata has been validated, Notary server generates the timestamp (and maybe snapshot) metadata. It sends this generated metadata to the Notary signer to be signed.
-
Notary signer retrieves the necessary encrypted private keys from its database if available, decrypts the keys, and uses them to sign the metadata. If successful, it sends the signatures back to Notary server.
-
Notary server is the source of truth for the state of a trusted collection of data, storing both client-uploaded and server-generated metadata in the TUF database. The generated timestamp and snapshot metadata certify that the metadata files the client uploaded are the most recent for that trusted collection.
Finally, Notary server notifies the client that their upload was successful.
-
The client can now immediately download the latest metadata from the server, using the still-valid bearer token to connect. Notary server only needs to obtain the metadata from the database, since none of the metadata has expired.
In the case that the timestamp has expired, Notary server would go through the entire sequence where it generates a new timestamp, request Notary signer for a signature, stores the newly signed timestamp in the database. It then sends this new timestamp, along with the rest of the stored metadata, to the requesting client.
Threat model
Both the server and the signer are potential attack vectors against all users of the Notary service. Client keys are also a potential attack vector, but not necessarily against all collections at a time. This section discusses how our architecture is designed to deal with compromises.
Notary server compromise
In the event of a Notary server compromise, an attacker would have direct access to the metadata stored in the database as well as access to the credentials used to communicate with Notary signer, and therefore, access to arbitrary signing operations with any key the Signer holds.
-
Denial of Service - An attacker could reject client requests and corrupt or delete metadata from the database, thus preventing clients from the ability to download or upload metadata.
-
Malicious Content - An attacker can create, store, and serve arbitrary metadata content for one or more trusted collections. However, the attacker has no access to any client-side keys, such as root, targets, and potentially the snapshot keys for the existing trusted collections.
A client can only be tricked into downloading and trusting the malicious content for these trusted collections if it has never seen the trusted collections and does not have any form of pinned trust.
If a client has previously interacted with any trusted collection or has its trust pinned to a specific certificate for the collections, the client immediately detects that the content is malicious and doesn’t trust any root, targets, or (maybe) snapshot metadata for these collections.
-
Rollback, Freeze, Mix and Match - The attacker can request that the Notary signer sign any arbitrary timestamp (and maybe snapshot) metadata they want. Attackers can launch a freeze attack, and, depending on whether the snapshot key is available, a mix-and-match attack up to the expiration of the targets file.
Clients both with and without pinned trust would be vulnerable to these attacks, so long as the attacker ensures that the version number of their malicious metadata is higher than the version number of the most recent good metadata that any client may have.
Note: the timestamp and snapshot keys cannot be compromised in a server-only compromise, so a key rotation would not be necessary. Once the Server compromise is mitigated, an attacker cannot generate valid timestamp or snapshot metadata and serve them on a malicious mirror, for example.
Notary signer compromise
In the event of a Notary signer compromise, an attacker would have access to all the (timestamp and snapshot) private keys stored in a database. If the keys are stored in an HSM, they would have the ability to sign arbitrary content with, and to delete, the keys in the HSM, but not to exfiltrate the private material.
-
Denial of Service - An attacker could reject all Notary server requests and corrupt or delete keys from the database (or even delete keys from an HSM), and thus prevent Notary servers from signing generated timestamps or snapshots.
-
Key Compromise - If the Notary signer uses a database as its backend, an attacker can exfiltrate all the (timestamp and snapshot) private material. The capabilities of an attacker are the same as of a Notary server compromise in terms of signing arbitrary metadata, with the important detail that in this particular case key rotations are necessary to recover from the attack.
Notary client keys and credentials compromise
The security of keys held and administered by users depends on measures taken by the users. If the Notary Client CLI was used to create them, then they are password protected and the Notary CLI does not provide options to export them in plaintext.
It is up to the user to choose an appropriate password, and to protect their key from offline brute-force attacks.
The severity of the compromise of a trust collection owner/administrator’s decrypted key depends on the type and combination of keys compromised. For example, were the snapshot and targets key both compromised, or just the targets key?
Possible attacks given the credentials compromised:
Decrypted Delegation Key, only
| Keys compromised | Malicious Content | Rollback, Freeze, Mix and Match | Denial of Service |
|---|---|---|---|
| Delegation key | no | no | no |
Decrypted Delegation Key + Notary Service write-capable credentials
| Keys compromised | Malicious Content | Rollback, Freeze, Mix and Match | Denial of Service |
|---|---|---|---|
| Delegation key | limited, maybe* | limited, maybe* | limited, maybe* |
If the Notary Service holds the snapshot key and the attacker has Notary Service write credentials, then they have effective access to the snapshot and timestamp keys because the server generates and signs the snapshot and timestamp for them.
An attacker can add malicious content, remove legitimate content from a collection, and
mix up the targets in a collection, but only within the particular delegation
roles that the key can sign for. Depending on the restrictions on that role,
they may be restricted in what type of content they can modify. They may also
add or remove the capabilities of other delegation keys below it on the key hierarchy
For example, if DelegationKey2 in the above key hierarchy is compromised, the
compromised key could
only modify the capabilities of DelegationKey4 and DelegationKey5.
Decrypted Delegation Key + Decrypted Snapshot Key, only
| Keys compromised | Malicious Content | Rollback, Freeze, Mix and Match | Denial of Service |
|---|---|---|---|
| Delegation key, Snapshot key | no | no | no |
The attacker does not have access to the timestamp key, which is always held by the Notary Service, and cannot set up a malicious mirror.
Decrypted Delegation Key + Decrypted Snapshot Key + Notary Service write-capable credentials
| Keys compromised | Malicious Content | Rollback, Freeze, Mix and Match | Denial of Service |
|---|---|---|---|
| Delegation key, Snapshot key | limited | limited | limited |
The Notary Service always holds the timestamp key. If the attacker has Notary Service write credentials, then they have effective access to the timestamp key because the server generates and signs the timestamp for them.
An attacker can add malicious content, remove legitimate content from a collection, and
mix up the targets in a collection, but only within the particular delegation
roles that the key can sign for. Depending on the restrictions on that role,
they may be restricted in what type of content they can modify. A key may also
add or remove the capabilities of other delegation keys below it on the key hierarchy
For example, if DelegationKey2 in the above key hierarchy is compromised, it can
only modify the capabilities of DelegationKey4 and DelegationKey5.
Decrypted Targets Key, only
| Keys compromised | Malicious Content | Rollback, Freeze, Mix and Match | Denial of Service |
|---|---|---|---|
| Targets key | no | no | no |
Decrypted Targets Key + Notary Service write-capable credentials
| Keys compromised | Malicious Content | Rollback, Freeze, Mix and Match | Denial of Service |
|---|---|---|---|
| Targets key | maybe* | maybe* | limited, maybe* |
If the Notary Service holds the snapshot key and the attacker has Notary Service write credentials, then they have effective access to the snapshot and timestamp keys because the server generates and signs the snapshot and timestamp for them.
An attacker can add any malicious content, remove any legitimate content from a
collection, and mix up the targets in a collection. They may also add or remove
the capabilities of any top level delegation key or role, such as Delegation1,
Delegation2, and Delegation3 in the key hierarchy diagram. If they remove
the roles entirely, they break the trust chain to the lower delegation roles,
such as Delegation4 or Delegation5.
Decrypted Targets Key + Decrypted Snapshot Key, only
| Keys compromised | Malicious Content | Rollback, Freeze, Mix and Match | Denial of Service |
|---|---|---|---|
| Targets key, Snapshot key | no | no | no |
The attacker does not have access to the timestamp key, which is always held by the Notary Service, and cannot set up a malicious mirror.
Decrypted Targets Key + Decrypted Snapshot Key + Notary Service write-capable credentials
| Keys compromised | Malicious Content | Rollback, Freeze, Mix and Match | Denial of Service |
|---|---|---|---|
| Targets key, Snapshot key | yes | yes | limited |
The Notary Service always holds the timestamp key. If the attacker has Notary Service write credentials, then they have effective access to the timestamp key because the server generates and signs the timestamp for them.
An attacker can add any malicious content, remove any legitimate content from a
collection, and mix up the targets in a collection. They may also add or remove
the capabilities of any top level delegation key or role, for example, Delegation1,
Delegation2, and Delegation3 in the key hierarchy diagram. If they remove
the roles entirely, they’d break the trust chain to the lower delegation roles,
such as Delegation4 or Delegation5.
Decrypted Root Key + none or any combination of decrypted keys, only
| Keys compromised | Malicious Content | Rollback, Freeze, Mix and Match | Denial of Service |
|---|---|---|---|
| All keys | yes | yes | yes |
No other keys are needed, since the attacker can just any rotate or all of them to ones that they generate. With these keys, they can set up a mirror to serve malicious data - any malicious data at all, given that they have access to all the keys.
Decrypted Root Key + none or any combination of decrypted keys + Notary Service write-capable credentials
| Keys compromised | Malicious Content | Rollback, Freeze, Mix and Match | Denial of Service |
|---|---|---|---|
| All keys | yes | yes | yes |
If the Notary Service holds the snapshot key and the attacker has Notary Service write credentials, then they don’t even need to rotate the snapshot and timestamp keys because the server generates and signs the snapshot and timestamp for them.
Mitigations
If a root key compromise is detected, the root key holder should contact whomever runs the notary service to manually reverse any malicious changes to the repository, and immediately rotate the root key. This creates a fork of the repository history, and thus break existing clients who have downloaded any of the malicious changes.
If a targets key compromise is detected, the root key holder must rotate the compromised key and push a clean set of targets using the new key.
If a delegations key compromise is detected, a higher level key holder must rotate the compromised key, and push a clean set of targets using the new key.
If a Notary Service credential compromise is detected, the credentials should be changed immediately.