Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(crypto): Initial pass at crypto implementation. #63

Merged
merged 2 commits into from
Dec 6, 2023
Merged

Conversation

wbrown
Copy link
Collaborator

@wbrown wbrown commented Nov 3, 2023

Tensor encryption and decryption.

Some design decisions and notes:

  • We only encrypt the actual tensor payload itself. This helps us avoid a format change. There's a case to encrypting the metadata header as well, but that requires a format change so I'm punting that.
  • We leverage the existing hash fields to contain the salt, nonce, and block_size for each tensor.
  • Encryption is easy. Just provide passphrase= argument to the Deserializer and Serializer.
  • We follow good practices, such as: random nonce's, ensuring that there's a new nonce each and every time, salting the passphrase with a random salt.
  • By design, we don't encrypt the entire tensor payload in one go -- instead we use a streaming block cipher, xsalsa20 with poly HMAC -- this allows us to decrypt concurrently with reads rather than waiting for the entire thing.

@wbrown wbrown requested a review from Eta0 November 3, 2023 17:51
@wbrown wbrown merged commit 15cb4a1 into main Dec 6, 2023
2 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant