Security Considerations and Encryption with Kettle

Abstract: Kettle is used more and more in enterprises where the standard obfuscation of credentials is not sufficient enough. There are requirements to use strong encryption methods and even to store internal data encrypted (covered in PDI-6168 and PDI-6170). The above use cases inspired me to create some simple transformations to test and play around with encryption.

Note: Kettle already has job entries to encrypt and decrypt files with PGP.

Authors: Jens Bleuel

License: LGPL

Kettle versions: 4.1 and later

Update since Kettle 4.2: There are two steps in the experimental section: Secret key generator, Symmetric Cryptography that cover this use case.

Attachments

• cryptography.zip - Transformations & sample files

Attention: When you want to use the given examples in production please be aware that they are made as a blue print and a proof of concept and need optimization.

Background

Transformation cryptographyCreateSecretKey is used to create a secret key.
Use the transformations cryptographyEncrypt and cryptographyDecrypt for testing.

The named parameters and default values are:

• cryptographyKeySize = 128
• cryptographyMethod = AES

The transformations were tested with AES and 128 bit: Advanced Encryption Standard (AES) is a symmetric-key encryption standard, see also http://java.sun.com/developer/technicalArticles/Security/AES/AES_v1.html

To use AES with 192- and 256-bit key sizes, you need unlimited strength cryptography.
Due to import-control restrictions imposed by some countries, the jurisdiction policy files shipped
with the Java 2 SDK, v 1.4 only permit strong cryptography to be used. An unlimited strength version of these files
(that is, with no restrictions on cryptographic strength) is available for download, however.

Instead of storing the decrypted data to a file there are a lof of other options, e.g.:

• use the decrypted data as credentials in subsequent steps or transformations
• put the decrypted data into variables visible in a limited scope (e.g. parent job) and use them as credentials for databases, repository etc. (see PDI-6168)
• and many more options

We may consider:

• Symmetric-key algorithm vs. asymmetric key algorithms (public-key cryptography)
• Diffie-Hellman key exchange is a specific method of exchanging keys.
• Ensure integrity e.g. by hash-codes
• Key file handling could be optimized in different ways.
• Please keep in mind that unencrypted data is in RAM (see PDI-6170 for a circumvention to prevent heap dumps)
• Beneath the binary or indexed storage type, an encrypted storage type may be possible in Kettle core.

For screen shots, some further background information and a test run, please see http://kettle.bleuel.com/2011/06/07/security-considerations-and-encryption-with-kettle/