Secrets, Key Management and Zero Trust

What a secret is. A password, an API key, a database credential, a signing key or a certificate private key: anything that, if copied, lets someone act as you. Config like a timeout or a feature flag is not secret but often shares the same delivery system, so a good design separates them and applies stricter rules to secrets.

Where secrets must not be. Not in source code, not in a container image, not in a log line, not in a chat message. They are fetched at startup or on demand from a secret store, with access limited to the one service that needs each secret and an audit log of every read. Prefer short-lived credentials issued per workload over long-lived passwords, so a leak expires by itself.

Rotation with overlap. Changing a secret in one instant breaks every client that has not updated yet. The safe sequence is: create the new version, accept both the new and the previous version for an overlap window, move clients to the new one, then retire the old. If a rotation goes wrong, the old version still works, so rotation is routine and not an outage.

valid = {'v1': 'old-secret'}                  # version -> secret the server will accept

def accepts(version, secret):
    return valid.get(version) == secret

def rotate(new_version, new_secret):          # step 1: add the new one, keep the old
    valid[new_version] = new_secret

def retire(version):                          # step 3: drop the old one once clients moved
    valid.pop(version, None)

rotate('v2', 'new-secret')
print(accepts('v1', 'old-secret'), accepts('v2', 'new-secret'))   # True True   overlap window
retire('v1')
print(accepts('v1', 'old-secret'), accepts('v2', 'new-secret'))   # False True   old key is dead

Envelope encryption. Encrypting everything directly with one master key is slow and risky. Instead each object (or table, or tenant) is encrypted with its own data key (DEK), and the DEK is stored next to the data in encrypted form, wrapped by a key-encryption key (KEK) that never leaves a key management service or hardware security module. To read, the service asks the KMS to unwrap the DEK. Rotating the KEK means re-wrapping the small DEKs, not re-encrypting terabytes of data.

Key ids and versions. Store the key id and version with every ciphertext so decryption knows which key to ask for, and so old data stays readable after rotation. Separate duties: the people who can use a key to decrypt are not the people who can delete or export it, and every use is logged.

End-to-end encryption. For a password manager or a private messenger, the server must never be able to read the content. The key is derived on the device (from the master password through a deliberately slow key-derivation function) and the server stores only ciphertext. The price is that the server cannot recover a lost password, and features like server-side search are limited.

Zero trust. The old model trusted anything inside the corporate network. Zero trust assumes the network is hostile: every request is authenticated (who is the user, and is the device healthy), authorized against policy (may this person reach this app right now), and encrypted, whether it comes from the office or a coffee shop. An identity-aware access proxy sits in front of each internal app, checks the identity and device, issues short-lived access, and logs the decision. Between services, mutual TLS gives each side a verified identity.

Least privilege everywhere. Each service gets only the secrets and roles it needs, tokens are scoped and short-lived, and break-glass access is rare, approved and logged. The test of a design is the blast radius: if this one credential leaks, how much can an attacker reach, and how fast can you revoke it?