AES CBC stream encryption in ruby?

Question

I used a pretty standard example (which is severely broken for my purposes) of cbc encryption in ruby:

def aes(m,k,t) (aes = OpenSSL::Cipher::Cipher.new('aes-256-cbc').send(m)).key = Digest::SHA256.digest(k) aes.update(t) << aes.final end def encrypt(key, text) aes(:encrypt, key, text) end def decrypt(key, text) aes(:decrypt, key, text) end 

This works as an acceptable starting point, but I need to be able to encrypt large streams of data without loading them into one huge chunk of memory. I want to download mega-at a time, update the state of the encryption stream, and then move on to the next block. Looking at the OpenSSL Cipher docs (which are award-winning), I expect that the call for upgrade should just continue the flow of data. However, a simple test tells me that there is something very wrong:

 Length = 256 newaes = OpenSSL::Cipher::Cipher.new('aes-256-cbc') newaes.encrypt newaes.key= Digest::SHA256.digest("foo") puts Base64.encode64(newaes.update("a"*Length)) puts Base64.encode64(newaes.update("a"*Length)) puts Base64.encode64(newaes.final) 

Running this with different values ​​for Length should not give me different threads. However, after completing the first update, there is always a problem. Streams diverge. I assumed that the problem is that for some inexplicable reason, the terminating null character ('\ 0') at the end of the line was encrypted. In the end, each call to update returns a string that is ((string.length / 16) + 1) * 16 bytes, implying that it encrypts an additional byte with each update.

How can I get OpenSSL encryption and decryption to work in a mode in which I can pass in data blocks and get the same result back, regardless of the size of the pieces that I break into data?

EDIT:

The problem does not depend on base64 encoding. Below are 3 different cyphertext results:

 require 'digest/sha2' require 'base64' require 'openssl' def base64(data) Base64.encode64(data).chomp end def crypt_test(blocksize) newaes = OpenSSL::Cipher::Cipher.new('aes-256-cbc') newaes.encrypt newaes.key= Digest::SHA256.digest("foo") plaintext = "" cyphertext = "" File.open("black_bar.jpg") do |fd| while not fd.eof data = fd.read(blocksize) cyphertext += data cyphertext += newaes.update(data) end end cyphertext += newaes.final puts base64(Digest::SHA256.digest(plaintext)) puts base64(Digest::SHA256.digest(cyphertext)) puts end crypt_test(1024) crypt_test(512) crypt_test(2048) 

Answer 1

I have about zero knowledge of Ruby. However, your problem looks like a supplement issue.

AES / CBC encrypts data in blocks of 16 bytes, at least. Padding is the addition of several bytes, which are:

• the filled length is a multiple of 16;
• when decrypting, extra bytes can be unambiguously deleted.

The second condition means that there can be no “indentation of zero length” (at least without resorting to dark tricks, such as “encrypting ciphertext”). There must be at least one additional padding byte. Otherwise, the decoder would not know if the end of the received data is really some addition or the actual message that ends in some bytes that look like an addition.

A very common filling scheme is the one specified in PKCS # 5 (see section 6.1.1): for blocks of length n (n = 16 for AES), at least 1 and no more than n bytes are added; if k bytes are added, then they all have a numerical value k. After decryption, you just need to look at the numerical value of the last byte to see how many bytes added were added. PKCS # 5 padding implies the behavior you observe: encrypting m bytes gives n * ((m / n) +1) output bytes.

If your calls really add the PKCS # 5 add-on on each update , you can restore it by removing the last 16 bytes from what they return. You will also need to reset IV for the next update call so that you can simply add the next update call. Speaking of this, I don't see anything in your IV code, and this is suspicious. CBC mode requires a new random IV (selected with a strong enough generator) for each message; IV must be transmitted along with the encrypted message (the one who decrypts the data will need it, IV can be sent "in clarity").

The paragraph above should be clearer if you know how CBC works. Wikipedia has good schemes.

Answer 2

Here is your problem:

 Length = 256 newaes = OpenSSL::Cipher::Cipher.new('aes-256-cbc') newaes.encrypt newaes.key= Digest::SHA256.digest("foo") s1 = newaes.update("a"*Length) s2 = newaes.update("a"*Length) s3 = newaes.final puts Base64.encode64(s1 + s2 + s3) 

Now the same base64 will be displayed, as if you squished two updates in one.

You are using base64 encoded alignment problem. Base64 encoding takes 3 bytes at a time and converts them to 4 bytes. If you give it the number of bytes, which is not a multiple of 3, it imposes “ = ” signs on things.

This means that if you have two consecutive coding runs that are not a multiple of 3 bytes, and then encode the same sequence of bytes in just one coding run, you will get another base64 output. The second coding run does not align as if this data were part of the first coding run. Here are some examples:

Here the data is a multiple of 3. In two runs of the encoder, base64 sequences are produced that can be combined together to create more or less the same sequence as one encoder run on concatenated strings.

 > Base64.encode64('abc') => "YWJj\n" > Base64.encode64('def') => "ZGVm\n" > Base64.encode64('abcdef') => "YWJjZGVm\n" 

Here the data is divided into sequences of 4 bytes, and 4 is not a multiple of 3. The concatenation of two encoder runs does not match the encoding of two strings concatenated.

 > Base64.encode64('abcd') => "YWJjZA==\n" > Base64.encode64('efgh') => "ZWZnaA==\n" > Base64.encode64('abcdefgh') => "YWJjZGVmZ2g=\n"