Common need wants a pattern: encrypting database fields

[fdr]

and with key rotation, which seems to be the place where most existing approaches fall down. @deafbybeheading was compelled to write new software because of this noteworthy defect in most or all others: https://github.com/deafbybeheading/attr_vault. Not sure how much of it is general and how much was get-the-job-done. Also, bugs.

[msakrejda]

+1. I think attr_vault is not mature enough yet, but we're battle-testing it, and I think with some work (and docs), it could be a great fit (though of course I'm a little biased).

[fdr]

Shrug. I looked around for a library with a crisp rotation angle. Null.

[brandur]

@deafbybeheading Hah, attr_vault sounds like a much better solution than what we could've materialized out of thin air.

Great idea guys! It would be amazing to start demonstrating some internal (/external) consistency in this area.

[pedro]

Hey folks,

I've been writing some Pliny docs (about time!). Check it here:
https://github.com/interagent/pliny/wiki

I think encryption would make it for a great topic for the "Extending" section, if anyone wants to take a stab.

It's probably also worth mentioning Fernet – which makes me wonder, why doesn't attr_vault just let Fernet handle the actual encryption?

[msakrejda]

@pedro awesome! And attr_vault basically does use the same crypto as Fernet: it just omits the ttl (which is generally pretty useless when actually storing data), and the Base64 encoding (which is unnecessary since Postgres has perfectly good support for binary data--though watch out for this outstanding pg issue which has been fixed and will be in the upcoming 0.18.3). I didn't maintain Fernet compatibility since I felt it was not that important. I support "migrating" from a plaintext column already, and plan to support migrating from a Fernet-encoded column in the future. Also, given the design of attr_vault, one could actually add Fernet compatibility pretty seamlessly (essentially, you could add encryption mechanism metadata to the keys in the attr_vault keyring, and then have the library choose an encryption / decryption mechanism based on that).

But in any case, I think real progress on this issue is blocked on me actually writing some docs for attr_vault.

[ys]

I am using attr_vault in Limerick.
I would change one thing only, the config vars using json is something I would update.
Otherwise it is a great one for this.

[msakrejda]

@ys update how? The metadata in the keyring is important. I guess I can wrap the whole in Base64 maybe? But please file an issue on that project and propose alternatives there. In fact, here: I made one for you msakrejda/attr_vault#14

[ys]

Let me take a look and come with an idea:)
This is my only issue with it.

[pedro]

@uhoh-itsmaciek got it! attr_vault looks super promising then. I take you folks are already using it in prod?

If you want to write some docs in the repo I can take a stab at the Pliny guide, lemme know!

[msakrejda]

@pedro yeah, we're using it in Shogun (and possibly other places) and other than the ruby-pg issue mentioned above and my lack of docs, it's been pretty seamless. I'm doing some cleanup and docs now.

[msakrejda]

@pedro btw, any tips for documenting a Ruby library like this? My first pass will just be going over usage in the README, but wondering if you know of libraries with clear, simple, idiomatic Ruby docs. I guess in my experience, I usually go for the README of whatever gem I'm using--maybe that's enough here?

[pedro]

@uhoh-itsmaciek can't claim to know much about this, but excon and dotenv come to mind when I think of good docs I used recently-ish!

Most of the gems we use are unsurprisingly well documented – probably some correlation there. But one example that comes to mind on bad docs is rr: they changed it to become a mix of readme and files inside the repo and that confuses the hell out of me.

So yeah, big +1 to just keep it all under README.md. Just went for a wiki in Pliny because I was thinking of it in terms of articles that would explain the different patterns and point people to more docs on them. Still gotta consider how is this going to fit with the repo readme, as it's certainly no replacement for it.

[bjeanes]

But one example that comes to mind on bad docs is rr

Yeah and the first result in google is the wrong repo, which always gets me too. So bad >.<