CockroachDB - Fun with Encryption

Problem we are trying to Solve

Migrating from a single-tenant architecture to a multi-tenant architecture where each tenant can use “Bring Your Own Key” (BYOK) to securely encrypt their data at rest in the shared platform.

The Result

The success criteria for the specified use case

• Tables can be stored in a specified store
• Multiple stores on a node, each with independent encryption

Documents

• Cockroach start - https://www.cockroachlabs.com/docs/v20.2/cockroach-start.html
• Encryption - https://www.cockroachlabs.com/docs/v20.2/encryption.html
• Enterprise features: https://www.cockroachlabs.com/docs/v20.2/enterprise-licensing.html

Process Overview:

1. Create Database and apply enterprise licence
2. Rolling restart with encrypted stores
3. Assign tables to an encrypted store
4. Verify data and stores is encrypted

Step by Step Process

• Create DB
  Create the cockroach DB, initialise and apply the enterprise licence as Documentation
• Rolling restart including the definition of the encrypted store and the associated keys
  Working through each node in the cluster in turn, perform a graceful shutdown and restart the node with the following options for each encrypted store required:

--store=path=<path to store encrypted data>,attrs=<label>

--enterprise-encryption=path=<path to store encrypted data>,key=<path to key file>,old-key=plain

In this example, the database is being started with 3 stores in the /home/cockroach/DB directory:

For the purpose of this example, the keys were generated via the cli using:

cockroach gen encryption-key -s 128 /home/cockroach/DB/keys/<keyname>.key

Once the node has been successfully added to the cluster, this process is then repeated on each of the remaining nodes in turn until all nodes have been restarted with the encrypted stores.

• Create tables and assign to an encrypted store
  Once all nodes have restarted, the tables which will hold data can be assigned to the respective stores:

alter table <schema>.<tablename> configure zone using constraints’[+<label>]’;

For example the following command would assign the table ‘crypt1’ to the store that was labelled as “encrypt1”, which for this example is the store held at “/home/cockroach/DB/crypt_rep1” and encrypted with the “home/cockroach/DB/keys/aes-128-rep1.key” key.

alter table enc_demo.crypt1 configure zone using constraints=’[+encrypt1]’;

• Populate data in tables
  Data can now be added to the tables as any other table in the database.

• Verify data is encrypted at rest
  Checking the contents of the .sst files within the encrypted store now shows all data being encrypted:
  

The encryption status of the various stores can been seen through the web console at (http://:/#/reports/stores/local) as shown below:

Be aware that cockroach requires an old-key value for the --encryption-enterprise option to allow for key rotation. This case used ‘plain’ as this was the first time encryption was applied. When rotating keys this needs to be set to the old key so Cockroach can decrypt the data and re-encrypt with the new key. See the documentation on “Encryption” in the Documents section.