The secret sharing problem: A developer's guide to secure collaboration that actually makes sense

The problem: Your data, but not just yours

Picture this: You've built a great app where users can store their sensitive documents such as financial records, medical information, legal contracts. Being a responsible developer, you encrypt everything. Problem solved, right?

Not quite.

One day, a user asks: "Can I share this document with my accountant or lawyer? Oh, and I'll need to revoke the lawyer's access next month."

Suddenly, you're stuck. How do you let multiple people decrypt the same data without:

• Giving everyone the same password? (Security nightmare)
• Re-encrypting the data for each person? (Performance nightmare)
• Losing the ability to revoke access? (Compliance nightmare)

Welcome to the world of envelope encryption—an elegant solution that sounds complicated but is actually beautifully simple.

The "shared safe" analogy

Think about how a bank safety deposit with Dual-Key access might work. You want to share access with your spouse and your lawyer.

How it works in real life

• Your valuables are in the safety deposit box
• Each person who requires access has a unique customer key
• The bank guard has a key (the guard key)
• Both keys are required to open the box

Why this system exists

• You can't open the box without the bank present
• The bank employee can't open your box without you
• Every access is logged with both parties present

Revoking access

• If you want to remove someone from your authorised list, the bank stops letting them in
• Your box stays the same
• Your key stays the same
• The guard key stays the same
• The bank updates their records of who's allowed

Adding access

• Want to add your spouse? They get their own customer key
• The guard key doesn't change
• Your box doesn't move
• A new customer key is issued
• The bank updates their records with your spouses details

That's envelope encryption in a nutshell.

How envelope encryption actually works

Let's break it down with a real example: A doctor's office managing patient records.

Step 1: The doctor creates a record

Dr. Smith creates a new patient file:
"Patient John Doe, diagnosis: broken arm, prescription: ibuprofen..."

Dr. Smith's app:
1. Generates a random encryption key (let's call it the "Master Key")
2. Encrypts the patient data with this Master Key
3. Encrypts the Master Key with Dr. Smith's personal password
4. Stores both in the database

The patient data is now safely encrypted. Only Dr. Smith can decrypt it because only she knows her password.

Step 2: Sharing with a nurse

Dr. Smith needs Nurse Johnson to update the patient's vitals. Here's the magic:

Dr. Smith's app:
1. Takes the SAME Master Key from Step 1
2. Encrypts it again—but this time with Nurse Johnson's password
3. Stores this second encrypted copy in the database

Now the database has:
- The encrypted patient data (encrypted with Master Key)
- Dr. Smith's encrypted Master Key (encrypted with Dr. Smith's password)
- Nurse Johnson's encrypted Master Key (encrypted with Nurse Johnson's password)

Notice: The patient data was NOT re-encrypted. We just made another locked box for the same key.

Step 3: The nurse accesses the data

When Nurse Johnson logs in:
1. She enters her password
2. The app uses her password to unlock her encrypted Master Key
3. Now she has the Master Key
4. She uses the Master Key to decrypt the patient data

Result: She sees the same data as Dr. Smith!

Step 4: Removing access

Three months later, Nurse Johnson transfers to another department:

Dr. Smith's app:
1. Deletes Nurse Johnson's encrypted Master Key from the database

That's it. Done.

- The patient data stays encrypted
- Dr. Smith still has access (her encrypted Master Key is still there)
- Nurse Johnson can't decrypt anymore (her encrypted Master Key is gone)
- No re-encryption needed!

Master Key

The master key should be stored securly using a secure enclave such as a Hardware Security Module (AWS offers this as a service), a self hosted Hashicorp Vault or AWS Secrets Manager.

Secure Processing

For confidential workloads the use of Intel Software Guard Extentions (SGX) or AWS Nitro Enclaves can be used within AWS.

These extentions allow a programmer to creates encrypted, isolated "enclaves" in CPU memory for processing sensitive data (e.g., healthcare, financial) in untrusted cloud environments, protecting data from the cloud provider.

The benefits of envelope encryption

1. Performance - Share in milliseconds

Let's say you have a 50MB medical imaging file and you need to share this with 10 people in the medical team while maintaining encryption.

Without envelope encryption:

• Re-encrypt 50MB * ten users is 450mb of duplicated data
• Storage and costs only ever increase
• decryption, copy and re encryption per user is slow

With envelope encryption:

• Encrypt the Master Key (32 bytes) ten times
• Time: ~1 millisecond per doctor
• Total: ~10 milliseconds
• No data duplication. Only store 50mb once
• Very fast

The difference becomes huge at scale.

2. Security - Isolation between users

Each person has their own password/key:

• Compromise one user's password → Only their access is affected
• Revoke access → Just delete their encrypted Master Key
• Rotate passwords → Only re-encrypt the Master Key, not the data

3. Auditability - Know who accessed what

Audit log:
- 2025-01-15 09:30: Dr. Smith created record (encrypted with Master Key ABC)
- 2025-01-15 09:31: Dr. Smith shared Master Key ABC with Nurse Johnson
- 2025-01-15 14:20: Nurse Johnson accessed record (decrypted with Master Key ABC)
- 2025-03-20 10:00: Dr. Smith revoked Nurse Johnson's access to Master Key ABC

This level of detail makes compliance officers very happy.

Real-world use case: A password manager

Let's look at how 1Password or LastPass Teams might work:

Individual Vault

You: Create vault, encrypt with Master Key #1
Your data: Encrypted with Master Key #1
Master Key #1: Encrypted with YOUR master password

Shared Team Vault

Team lead: Creates shared vault, generates Master Key #2
Company credentials: Encrypted with Master Key #2

Master Key #2 is encrypted THREE times:
- Once with team lead's password → stored
- Once with developer's password → stored  
- Once with designer's password → stored

When developer leaves:
- Delete their encrypted copy of Master Key #2
- They can no longer decrypt company credentials
- No need to change all the passwords!

The code (simplified)

Here's what this looks like in practice (simplified for clarity):

// 1. Create encrypted data
func CreateDocument(content []byte, ownerPassword string) {
    // Generate a random Master Key
    masterKey := generateRandomKey()
    
    // Encrypt the document with the Master Key
    encryptedDocument := encrypt(content, masterKey)
    
    // Encrypt the Master Key with owner's password
    ownerEncryptedKey := encrypt(masterKey, ownerPassword)
    
    // Store both
    database.Save(encryptedDocument, ownerEncryptedKey)
}

// 2. Share with another user
func ShareWithUser(documentID, userPassword string) {
    // Get the Master Key (decrypt with owner's password)
    masterKey := getMasterKeyAsOwner(documentID)
    
    // Encrypt the SAME Master Key with user's password
    userEncryptedKey := encrypt(masterKey, userPassword)
    
    // Store user's encrypted key
    database.SaveSharedKey(documentID, userID, userEncryptedKey)
}

// 3. User accesses the document
func ReadDocument(documentID, userPassword string) []byte {
    // Get user's encrypted Master Key
    userEncryptedKey := database.GetUserKey(documentID, userID)
    
    // Decrypt to get the Master Key
    masterKey := decrypt(userEncryptedKey, userPassword)
    
    // Use Master Key to decrypt document
    encryptedDocument := database.GetDocument(documentID)
    content := decrypt(encryptedDocument, masterKey)
    
    return content
}

// 4. Revoke access
func RevokeAccess(documentID, userID string) {
    // Just delete their encrypted key
    database.DeleteUserKey(documentID, userID)
    
    // They can no longer decrypt the Master Key
    // Therefore can't decrypt the document
}

Common mistakes working with envelope encryption (and how to avoid them)

Mistake #1: Deriving different keys

❌ WRONG:
ownerKey = masterKey
userKey = deriveNewKey(masterKey)  // Creates a DIFFERENT key!

// User can't decrypt because they have a different key
decrypt(data, userKey)  // FAILS!

✅ RIGHT:
ownerKey = masterKey
userKey = masterKey  // The SAME key, just encrypted differently

// User CAN decrypt because they have the same key
decrypt(data, userKey)  // WORKS!

This was actually MY mistake when I first designed this. The tests caught it!

Mistake #2: Storing the master password

❌ WRONG:
database.users {
    id: 1,
    master_password: "password123"  // Never store passwords!
}

✅ RIGHT:
database.users {
    id: 1,
    encrypted_master_key: [encrypted bytes]  // Store encrypted keys
}

// User types password → decrypt their key → use key to decrypt data

Mistake #3: Forgetting to rotate keys

Like changing locks after someone loses their keys:

// Rotate the Master Key periodically
func RotateMasterKey(documentID string) {
    // 1. Get old Master Key and decrypt all data
    oldMasterKey := getCurrentMasterKey(documentID)
    data := decryptAll(oldMasterKey)
    
    // 2. Generate new Master Key
    newMasterKey := generateRandomKey()
    
    // 3. Re-encrypt data with new key
    encryptedData := encrypt(data, newMasterKey)
    
    // 4. Re-encrypt new Master Key for all users who had access
    for user in getAllUsersWithAccess(documentID) {
        userEncryptedKey := encrypt(newMasterKey, user.password)
        database.UpdateUserKey(documentID, user.id, userEncryptedKey)
    }
    
    // 5. Update document
    database.UpdateDocument(documentID, encryptedData)
}

When should you use envelope encryption?

✅ Perfect for:

• Collaborative platforms: Google Docs, Notion, Slack
• File sharing: Dropbox, Box, OneDrive
• Team tools: Password managers, secret stores
• Healthcare: EMR systems with doctor/patient sharing
• Finance: Documents shared with accountants, advisors
• Legal: Case files shared with clients, co-counsel

❌ Not needed for:

• Single-user apps: If data is never shared, simpler encryption works
• Public data: If anyone can read it, why encrypt?
• Zero-knowledge systems: If the server should NEVER see plaintext, you need client-side encryption instead

The database schema

Here's what you store:

-- Each user has their own encrypted Master Key
CREATE TABLE encryption_keys (
    owner_id UUID PRIMARY KEY,
    encrypted_master_key BYTEA,  -- Master Key encrypted with owner's password
    created_at TIMESTAMP
);

-- Shared access: Each collaborator gets their own encrypted copy
CREATE TABLE shared_access (
    id UUID PRIMARY KEY,
    owner_id UUID,               -- Who owns the data
    user_id UUID,                -- Who has been granted access
    encrypted_master_key BYTEA,  -- SAME Master Key, encrypted with user's password
    granted_at TIMESTAMP,
    revoked_at TIMESTAMP         -- NULL if still active
);

-- The actual encrypted data
CREATE TABLE encrypted_documents (
    id UUID PRIMARY KEY,
    owner_id UUID,
    encrypted_content BYTEA,     -- Encrypted with Master Key
    created_at TIMESTAMP
);

Security best practices

1. Use a Key Management Service (KMS)

Don't store the "password" that encrypts the Master Keys in your code:

❌ BAD:
const MASTER_PASSWORD = "super-secret-key"  // Hard-coded!

✅ GOOD:
// Fetch from AWS KMS, Azure Key Vault, HashiCorp Vault
kek := fetchFromKMS("production-key-id")

2. Audit everything

func LogAccess(action, userID, documentID string) {
    auditLog.Write({
        timestamp: time.Now(),
        action: action,           // "created", "shared", "accessed", "revoked"
        user_id: userID,
        document_id: documentID,
        ip_address: request.IP,
    })
}

3. Rate limit decryption attempts

Prevent brute force attacks:

func DecryptDocument(userID, documentID string) {
    // Check rate limit
    if rateLimiter.IsOverLimit(userID, "decrypt") {
        return errors.New("too many attempts, try again later")
    }
    
    // Proceed with decryption...
}

4. Use strong encryption

✅ Use: AES-256-GCM
❌ Avoid: AES-ECB, DES, RC4 (broken!)

Performance Tips working with envelope encryption

Cache decrypted master keys (carefully) - advanced topic

Always consider and reason about caching stratergies.

...they are powerful and can reduce load when used in the right place

// Store in memory for a short time
cache := NewTTLCache(15 * time.Minute)

func GetMasterKey(userID, documentID string) []byte {
    // Check cache first
    cacheKey := fmt.Sprintf("%s:%s", userID, documentID)
    if cached := cache.Get(cacheKey); cached != nil {
        return cached
    }
    
    // Not in cache, decrypt it
    encryptedKey := database.GetUserKey(documentID, userID)
    masterKey := decrypt(encryptedKey, getUserPassword())
    
    // Cache for next time
    cache.Set(cacheKey, masterKey)
    
    return masterKey
}

Warning: Only cache in memory, never on disk. Clear on logout.

Compliance benefits of envelope encryption

This pattern helps you comply with:

GDPR (European privacy law)

• Right to erasure: Delete user's encrypted Master Key → they can't access data anymore
• Access control: Prove exactly who can access what
• Data portability: Decrypt and export data for specific users

HIPAA (US healthcare)

• Minimum necessary: Only share with people who need access
• Audit trails: Log every access, share, and revocation
• Access revocation: Remove access immediately when employment ends

SOX (Financial controls)

• Segregation of duties: Different people have different access
• Audit requirements: Complete log of who accessed what when

Real example: Building a healthcare app

Let's say you're building a telemedicine platform:

// Patient creates a record
patientMasterKey := generateKey()
encryptedRecord := encrypt(patientData, patientMasterKey)
patientEncryptedKey := encrypt(patientMasterKey, patientPassword)
database.Save(encryptedRecord, patientEncryptedKey)

// Patient books appointment with Dr. Smith
drSmithEncryptedKey := encrypt(patientMasterKey, drSmithPassword)
database.ShareWithDoctor(patientID, drSmithID, drSmithEncryptedKey)

// Dr. Smith can now see the record
drSmithKey := decrypt(drSmithEncryptedKey, drSmithPassword)
record := decrypt(encryptedRecord, drSmithKey)

// Appointment ends
database.RevokeAccess(patientID, drSmithID)

// Dr. Smith can no longer access the record
// Patient's data stays encrypted
// No performance hit on revocation

Testing your implementation

Here's how to verify it works:

func TestSharing(t *testing.T) {
    // 1. Owner creates data
    ownerKey := generateKey()
    data := []byte("secret document")
    encrypted := encrypt(data, ownerKey)
    
    // 2. Share with user
    userEncryptedKey := encrypt(ownerKey, userPassword)
    
    // 3. User decrypts
    userKey := decrypt(userEncryptedKey, userPassword)
    decrypted := decrypt(encrypted, userKey)
    
    // 4. Verify
    if string(decrypted) != string(data) {
        t.Error("Sharing failed!")
    }
    
    // 5. Verify keys match
    if !bytes.Equal(ownerKey, userKey) {
        t.Error("Keys don't match!")
    }
}

Wrapping up

Envelope encryption solves a problem every collaborative app faces: How do you securely share encrypted data?

The answer is elegant:

1. Encrypt data once with a Master Key
2. Encrypt the Master Key separately for each user
3. Share by giving encrypted copies of the same key
4. Revoke by deleting the user's encrypted key

It's fast, secure, auditable and scales beautifully.

Next time you're building a feature where users need to collaborate on sensitive data, remember: you're not just encrypting data, you're encrypting keys. And that makes all the difference.

---

Further reading

• NIST Key Management Guidelines
• AWS Encryption SDK Documentation
• OWASP Cryptographic Storage Cheat Sheet
• AWS Cloud HSM
• Intel SGX
• AWS Nitro Enclave

---

Appendix: Envelope encryption in the wild

You might be surprised to learn that envelope encryption isn't some exotic technique—it's everywhere. Let's look at how major tech companies and systems use this exact pattern to protect your data.

Apple's iOS & macOS: The gold standard

Apple's implementation is one of the most elegant examples of envelope encryption at multiple layers.

The Secure Enclave

Every modern iPhone has a dedicated chip called the Secure Enclave—a separate processor that handles encryption keys and never lets them leave.

How it works:

Your iPhone storage:
├─ User files (photos, messages, documents)
│  └─ Encrypted with: File Keys (one per file)
├─ File Keys
│  └─ Encrypted with: Class Keys (based on protection class)
└─ Class Keys
   └─ Encrypted with: UID Key + Passcode (in Secure Enclave)

The layers:

1. UID (Unique ID Key): Burned into the Secure Enclave during manufacturing. Cannot be extracted. Ever.

2. Your Passcode: Combined with UID to create a key that encrypts Class Keys

3. Class Keys: Four different types (we'll focus on "Complete Protection"):

   • NSFileProtectionComplete: File accessible only when unlocked
   • Encrypted with: UID + Passcode

4. File Keys: Individual random key for each file

   • Encrypted with: Class Key

5. Your actual file: Photo, message, document

   • Encrypted with: File Key

Example: Taking a photo

Step 1: iPhone generates random File Key
File Key = [random 256-bit key]

Step 2: Encrypt the photo with File Key
Encrypted Photo = AES-256(Photo, File Key)

Step 3: Encrypt File Key with Class Key
Encrypted File Key = AES(File Key, Class Key)

Step 4: Store both
Storage: Encrypted Photo + Encrypted File Key

When you lock your iPhone:

Secure Enclave: *erases Class Keys from memory*

Now:
- Your photos are still encrypted on disk
- The File Keys are still encrypted on disk
- But the Class Keys needed to decrypt File Keys are gone
- Without your passcode, nobody can reconstruct the Class Keys
- Therefore: Nobody can decrypt the File Keys
- Therefore: Nobody can decrypt your photos

This is why FBI couldn't crack the San Bernardino iPhone!

iCloud Keychain

Your passwords in iCloud Keychain use envelope encryption too:

Password for Netflix → Encrypted with Item Key
Item Key → Encrypted with Keychain Key
Keychain Key → Encrypted with iCloud Security Code
iCloud Security Code → Derived from device passcode + iCloud account

When you add a new device:
- New device gets its own encrypted copy of Keychain Key
- Same keychain, multiple encrypted copies
- Remove device = delete its encrypted key copy

Android's File-Based Encryption (FBE)

Android uses a similar but slightly different approach:

Direct Boot Mode (before unlock):
├─ DE Keys (Device Encrypted)
│  └─ Available even when locked
│  └─ Used for: Alarms, phone calls
└─ CE Keys (Credential Encrypted)
   └─ Available only after unlock
   └─ Used for: Apps, photos, messages

Both encrypted with hardware-backed keys in Trustzone

AWS Key Management Service (KMS)

AWS KMS is built around envelope encryption.

Your file: company-secrets.pdf (50 MB)

Step 1: Generate Data Key from KMS
Request: "KMS, give me a data key for customer-master-key-123"
Response: {
  Plaintext: [random 256-bit key],
  CiphertextBlob: [that key, encrypted by master key]
}

Step 2: Use plaintext key to encrypt file
Encrypted File = AES-256(company-secrets.pdf, Plaintext Key)

Step 3: Store encrypted file + encrypted key
S3 stores:
- company-secrets.pdf.encrypted (50 MB)
- Encrypted Data Key (256 bytes) ← stored as metadata

Step 4: Delete plaintext key from memory
The plaintext key is immediately wiped

To decrypt later:
1. Get encrypted key from S3 metadata
2. Send to KMS: "Decrypt this key using customer-master-key-123"
3. KMS returns plaintext key
4. Use plaintext key to decrypt file
5. Wipe plaintext key from memory again

Why this is brilliant:

Without envelope encryption:
- 50 MB file needs to be sent to KMS for encryption
- Network transfer: slow, expensive
- KMS has to process 50 MB: $$$

With envelope encryption:
- Only 256 bytes sent to KMS
- File encrypted locally: fast, cheap
- KMS just manages tiny keys: $

Corporate scenario:

Employee's laptop:
FVEK encrypted with:
1. Employee's TPM
2. Company's recovery password (stored in Active Directory)

Employee leaves company:
→ Company can still decrypt drive using recovery password
→ Company rotates FVEK
→ Re-encrypts drive with new FVEK
→ New FVEK only encrypted with company recovery password
→ Employee's TPM key useless

Employee's data on their personal devices: Unaffected

Signal messenger: Sealed sender

Signal uses envelope encryption for metadata protection:

Your message to Bob: "Meet at 3pm"

Step 1: Encrypt message content
Message Key = generate_random()
Encrypted Message = encrypt("Meet at 3pm", Message Key)

Step 2: Encrypt Message Key for recipient
Bob's Identity Key = [Bob's public key]
Encrypted Message Key = encrypt(Message Key, Bob's Identity Key)

Step 3: Seal the sender (envelope encryption strikes again!)
Sender Key = generate_random()
Sealed Envelope = encrypt({
  from: "You",
  encrypted_message: Encrypted Message,
  encrypted_key: Encrypted Message Key
}, Sender Key)

Step 4: Encrypt Sender Key for recipient
Encrypted Sender Key = encrypt(Sender Key, Bob's Identity Key)

Signal server sees:
- Sealed blob (no idea who sent it)
- Destination: Bob
- Cannot read contents
- Cannot see who sent it

Bob receives:
1. Decrypts Sender Key
2. Uses Sender Key to unseal envelope (learns it's from you)
3. Decrypts Message Key
4. Decrypts message content

HashiCorp Vault: Dynamic secrets

Vault uses envelope encryption for its transit secrets engine:

Application needs to encrypt credit card:
1. App: "Vault, encrypt this credit card with key-name-prod"
2. Vault: Generates data key from master key
3. Vault: Encrypts credit card with data key
4. Vault: Returns encrypted credit card + encrypted data key
5. App: Stores both in database

Later, to decrypt:
1. App: "Vault, decrypt this using key-name-prod"
2. App: Sends encrypted data key + encrypted credit card
3. Vault: Decrypts data key using master key
4. Vault: Decrypts credit card using data key
5. Vault: Returns plaintext credit card
6. App: Uses it, then immediately discards

Key rotation:
- New version of master key created
- Old encrypted data keys can still be decrypted
- New encryptions use new master key version
- Background job re-encrypts data keys with new master key

Modern web browsers: Cookie encryption

Chrome and Edge encrypt cookies using OS-level envelope encryption:

Windows:

Cookie value: session_token=abc123

Step 1: Generate random cookie key
Step 2: Encrypt cookie with cookie key
Step 3: Encrypt cookie key with DPAPI
       (Data Protection API - uses Windows login password)
Step 4: Store encrypted cookie + encrypted key

Result:
- Cookies encrypted on disk
- Tied to your Windows user account
- Another Windows user can't decrypt your cookies
- Cookie stealing malware can't easily extract them

macOS:

Cookie value: session_token=abc123

Step 1: Generate random cookie key  
Step 2: Encrypt cookie with cookie key
Step 3: Store cookie key in Keychain
       (Encrypted with keychain password)
Step 4: Store encrypted cookie in browser DB

Chrome process:
1. Asks macOS Keychain for cookie key
2. User prompted for password if needed
3. Decrypts cookie
4. Uses cookie for web request

WhatsApp multi-device

WhatsApp's multi-device support is envelope encryption magic:

Your messages: Encrypted with Message Keys

Message Keys encrypted separately:
├─ For your phone (Encrypted with Phone Key)
├─ For your laptop (Encrypted with Laptop Key)
└─ For your tablet (Encrypted with Tablet Key)

Add new device (laptop):
1. Scan QR code from phone
2. Establish secure channel
3. Phone sends: All Message Keys encrypted with Laptop Key
4. Laptop can now decrypt all messages

Remove device:
→ Delete device's encrypted Message Keys
→ Future messages: Only encrypt for remaining devices

The pattern emerges

Notice the common pattern across all these systems:

1. Data encrypted with fast, random symmetric key (DEK)
2. DEK encrypted multiple times for different users/devices/contexts
3. Access control = managing encrypted DEK copies
4. Revocation = delete encrypted DEK copy
5. No need to re-encrypt the actual data

This is the power of envelope encryption—it's not just a technique, it's the fundamental pattern for secure, scalable, multi-party access to encrypted data.

Why is everyone using envelope encryption?

Apple, Google, Microsoft, AWS, Signal, and every major security system uses envelope encryption because:

1. Performance: Encrypt data once, share by encrypting tiny keys
2. Scale: Millions of users can share access to the same data
3. Security: Each user has independent credentials
4. Auditability: Clear trail of who can access what
5. Flexibility: Easy to add/remove access without touching data
6. Key Rotation: Can rotate encryption keys without massive re-encryption

It's not just a good idea—it's the way to do secure data sharing at scale.

---

Got questions or found this helpful? Get in touch if you have encryptions needs, Mechanical Rocks' encryption experts would love to help, but we need the keys first ;)