CockroachDB - Fun with Encryption
In this post, Dan shows you how to create multiple store directories for CockroachDB nodes and encrypt them using different encryption keys. Different tables can then be stored and encrypted using different keys.
Problem we are trying to Solve
Migrating from a single-tenant architecture to a multi-tenant architecture where each tenant can use “Bring Your Own Key” (BYOK) to securely encrypt their data at rest in the shared platform.
The success criteria for the specified use case
- Tables can be stored in a specified store
- Multiple stores on a node, each with independent encryption
- Cockroach start - https://www.cockroachlabs.com/docs/v20.2/cockroach-start.html
- Encryption - https://www.cockroachlabs.com/docs/v20.2/encryption.html
- Enterprise features: https://www.cockroachlabs.com/docs/v20.2/enterprise-licensing.html
- Create Database and apply enterprise licence
- Rolling restart with encrypted stores
- Assign tables to an encrypted store
- Verify data and stores is encrypted
Step by Step Process
- Create DB
Create the cockroach DB, initialise and apply the enterprise licence as Documentation
- Rolling restart including the definition of the encrypted store and the associated keys
Working through each node in the cluster in turn, perform a graceful shutdown and restart the node with the following options for each encrypted store required:
--store=path=<path to store encrypted data>,attrs=<label> --enterprise-encryption=path=<path to store encrypted data>,key=<path to key file>,old-key=plain
In this example, the database is being started with 3 stores in the /home/cockroach/DB directory:
For the purpose of this example, the keys were generated via the cli using:
cockroach gen encryption-key -s 128 /home/cockroach/DB/keys/<keyname>.key
Once the node has been successfully added to the cluster, this process is then repeated on each of the remaining nodes in turn until all nodes have been restarted with the encrypted stores.
- Create tables and assign to an encrypted store
Once all nodes have restarted, the tables which will hold data can be assigned to the respective stores:
alter table <schema>.<tablename> configure zone using constraints’[+<label>]’;
For example the following command would assign the table ‘crypt1’ to the store that was labelled as “encrypt1”, which for this example is the store held at “/home/cockroach/DB/crypt_rep1” and encrypted with the “home/cockroach/DB/keys/aes-128-rep1.key” key.
alter table enc_demo.crypt1 configure zone using constraints=’[+encrypt1]’;
Populate data in tables
Data can now be added to the tables as any other table in the database.
Verify data is encrypted at rest
Checking the contents of the .sst files within the encrypted store now shows all data being encrypted:
The encryption status of the various stores can been seen through the web console at (http://
Be aware that cockroach requires an old-key value for the --encryption-enterprise option to allow for key rotation. This case used ‘plain’ as this was the first time encryption was applied. When rotating keys this needs to be set to the old key so Cockroach can decrypt the data and re-encrypt with the new key. See the documentation on “Encryption” in the Documents section.