CockroachDB - Fun with Encryption

Problem we are trying to Solve

Migrating from a single-tenant architecture to a multi-tenant architecture where each tenant can use “Bring Your Own Key” (BYOK) to securely encrypt their data at rest in the shared platform.

The Result

The success criteria for the specified use case

  • Tables can be stored in a specified store
  • Multiple stores on a node, each with independent encryption

Documents

Process Overview:

  1. Create Database and apply enterprise licence
  2. Rolling restart with encrypted stores
  3. Assign tables to an encrypted store
  4. Verify data and stores is encrypted

Step by Step Process

  • Create DB
    Create the cockroach DB, initialise and apply the enterprise licence as Documentation
  • Rolling restart including the definition of the encrypted store and the associated keys
    Working through each node in the cluster in turn, perform a graceful shutdown and restart the node with the following options for each encrypted store required:
--store=path=<path to store encrypted data>,attrs=<label>

--enterprise-encryption=path=<path to store encrypted data>,key=<path to key file>,old-key=plain

In this example, the database is being started with 3 stores in the /home/cockroach/DB directory:

Store Label Encryption key
.../repl1 None None
.../crypt_repl1 encrypt1 .../keys/aes-128-rep1.key
.../crypt_repl2 encrypt2 .../keys/aes-128-rep2.key

For the purpose of this example, the keys were generated via the cli using:

cockroach gen encryption-key -s 128 /home/cockroach/DB/keys/<keyname>.key

Once the node has been successfully added to the cluster, this process is then repeated on each of the remaining nodes in turn until all nodes have been restarted with the encrypted stores.

  • Create tables and assign to an encrypted store
    Once all nodes have restarted, the tables which will hold data can be assigned to the respective stores:
alter table <schema>.<tablename> configure zone using constraints’[+<label>]’;

For example the following command would assign the table ‘crypt1’ to the store that was labelled as “encrypt1”, which for this example is the store held at “/home/cockroach/DB/crypt_rep1” and encrypted with the “home/cockroach/DB/keys/aes-128-rep1.key” key.

alter table enc_demo.crypt1 configure zone using constraints=’[+encrypt1]’;
  • Populate data in tables
    Data can now be added to the tables as any other table in the database.

  • Verify data is encrypted at rest
    Checking the contents of the .sst files within the encrypted store now shows all data being encrypted:

The encryption status of the various stores can been seen through the web console at (http://:/#/reports/stores/local) as shown below:

Be aware that cockroach requires an old-key value for the --encryption-enterprise option to allow for key rotation. This case used ‘plain’ as this was the first time encryption was applied. When rotating keys this needs to be set to the old key so Cockroach can decrypt the data and re-encrypt with the new key. See the documentation on “Encryption” in the Documents section.