Partial encryption and decryption of AES ciphers

Photo by Mauro Sbicego on Unsplash

Not a Chat-JiPiTi article, I put some effort to it, so comments and feedback are much regarded :)

Note: In this article we’re gonna take a look only at aes-ctr-* and aes-gcm-* ciphers, this method of partially encrypting/decrypting messages may not apply to other AES ciphers.

What’s the use-case?

We are building a cloud storage where users can upload/download files via RESTful HTTPS protocol, among many other features. When a user is downloading a file, they may specify what portion of the file exactly they want to get, this is done via the Range HTTP headers. Implementing the partial download is a quite simple task, but NOT when those files are encrypted with AES ciphers.

What is AES encryption and how it works in simple words?

I’m no expert in cybersecurity or cryptography, so figuring things out was not an easy challange, that’s why i’ll make my best to explain everything in the most simple way.

AES ciphers are symmetric encryption algorithms. Symmetric encryption is the one that uses the same secret key for both encryption and decryption. For AES algorithms that key should be 32 bytes long.

Algorithms aes-ctr-* and aes-gcm-* alongside the encryption key also require an Initialisation Vector (IV) - a random integer from 12 to 96 bytes long depending on the algorithm and use-case (for CTR strictly 16 bytes, but for GCM it can be anything from 12 to 96 bytes). Think of them as nonces or salts. They are not required to be kept secret, but you should never encrypt more than one message with the same IV, as that will potentially lead to unauthenticated decryption of messages.

So the final set of required inputs for encrypting a given data is:

• The secret key;
• An Initialization Vector (IV);
• And the data itself;

AES ciphers are block based, meaning that they do not encode the given data as a whole, but instead they split the data into small chunks of a certain size called blocks (in this case 16 bytes each chunk) and encrypt them using the given secret key and IV.

Now the key part for understanding how to partially decrypt an AES encrypted message is the IV that you randomly generated. Your random generated IV will be used as it is only to decrypt the first block, meaning only for the first 16 bytes of the given data. And if you are encrypting more than 16 bytes of data, then the algorithm is going to automatically generate a new IV from your initial one for each of the next blocks of data.

How the IV is generated for each of the next blocks?

As I stated above you generate your IV as a unique integer from 12 to 96 bytes long. To encrypt the second block the AES algorithm will add 1 to the IV of the first block. And to encrypt the third block it’ll add one to the previous, the second block. And so on for each of the next blocks.

So, lets take an example. Imagine you have 128 bytes of data that you are going to encrypt:

I saw a neighbor talking to her cat, it was hilarious that she thought her cat could understand her. I went home and told my dog

​And also you have your secret key, and a 12 bytes long initialization vector (for simplicity i’ll show it in hex encoding):

0x00_00_00_00_00_00_00_00_00_00_00_00

​Now the algorithm will take your message, “brake” it down to block each 16 bytes long. Our message is 128 bytes, so it’ll perfectly fit into 128 / 16 = 8 blocks. Then the algorithm will encrypt each block sequentially, each time adding 1 to our initial IV before using it to encrypt the block. For simplicity in this example lets assume that our randomly generated IV came out to be 0, or 0x00_00_00_00_00_00_00_00_00_00_00_00 in hexadecimal encoding:

Block 1: "I saw a neighbor",
IV for block 1: 0x00_00_00_00_00_00_00_00_00_00_00_00 + 0 = 0x00_00_00_00_00_00_00_00_00_00_00_00
Block 2: " talking to her ",
IV for block 2: 0x00_00_00_00_00_00_00_00_00_00_00_00 + 1 = 0x00_00_00_00_00_00_00_00_00_00_00_01
Block 3: "cat, it was hila",
IV for block 3: 0x00_00_00_00_00_00_00_00_00_00_00_00 + 2 = 0x00_00_00_00_00_00_00_00_00_00_00_02
Block 4: "rious that she t",
IV for block 4: 0x00_00_00_00_00_00_00_00_00_00_00_00 + 3 = 0x00_00_00_00_00_00_00_00_00_00_00_03
Block 5: "hought her cat c",
IV for block 5: 0x00_00_00_00_00_00_00_00_00_00_00_00 + 4 = 0x00_00_00_00_00_00_00_00_00_00_00_04
Block 6: "ould understand ",
IV for block 6: 0x00_00_00_00_00_00_00_00_00_00_00_00 + 5 = 0x00_00_00_00_00_00_00_00_00_00_00_05
Block 7: "her. I went home",
IV for block 7: 0x00_00_00_00_00_00_00_00_00_00_00_00 + 6 = 0x00_00_00_00_00_00_00_00_00_00_00_06
Block 8: " and told my dog",
IV for block 8: 0x00_00_00_00_00_00_00_00_00_00_00_00 + 7 = 0x00_00_00_00_00_00_00_00_00_00_00_07

​Decryption is done in a similar manner: algorithm takes the decrypted message, breaks it down to 16 byte blocks, and decrypts them using the same secret key and the same IV that were used at the encryption.

So how do I decrypt partially?

Lets say we want to decrypt from 20th to 50th byte of our 128byte long message, the part that says lking to her cat, it was hilar. Remember that AES uses block encryption/decryption? That means, that in order to decrypt the message from 20th byte to 50th, we first need to get the entire blocks that contain our desired range of bytes. The blocks that contain our desired portion are 2nd, 3-rd and 4th:

Block 2: " talking to her ",
              ____________ <- the underscored part is the one that we need.
Block 3: "cat, it was hila",
          ________________ <- the underscored part is the one that we need.
Block 4: "rious that she t",
          __               <- the underscored part is the one that we need.

​To compute the offsets of those blocks you just have to do some simple math:

const initializationVector = '0x000000000000000000000000'; // our initial random IV, which is 0
const blockSize = 16; // each block has 16 bytes of data
const rangeStart = 20;
const rangeEnd = 50;
const rangeStartBlockOffset = rangeStart - (rangeStart % blockSize); // = 16
const rangeEndBlockOffset = rangeEnd - (rangeEnd % blockSize) + blockSize; // = 64

​Good! Now we have the offsets of the encrypted blocks that contain our data. Now we can read them from the storage that contains our encrypted message:

const encryptedMessage: Buffer = ***; // imagine we got the encrypted messages somehow
const blocksThatContainOnlyTheNeededPart = encryptedMessage.slice(rangeStartBlockOffset, rangeEndBlockOffset + 1);

Then we will need the initialization vector for the block from which starts the needed part of the data, that is the 2nd block:

const rangeStartBlockIndex = Math.floor(rangeStart / blockSize); // = 1
const initializationVectorForRangeStartBlock = '0x' + (BigInt(initializationVector) + BigInt(rangeStartBlockIndex)).toString(16).padStart(24, '0'); // '0x000000000000000000000001'

​Then we pass all of this into the decryption function:

const decryptedBlocksContainingNeededPart: Buffer = crypto.createDecipheriv(
	'aes-256-gcm',
  secretKey,
  Buffer.from(initializationVectorForRangeStartBlock, 'hex'),
).update(blocksThatContainOnlyTheNeededPart);

Trim the beginning and ending of the buffer if needed to keep not the whole blocks, but only the data that we need:

const neededPartStart = rangeStart - blockOffset;
const neededPartSize = (rangeStart - rangeEnd);
const neededPartEnd = neededPartStart + neededPartSize;
const onlyThePartThatWeNeed: Buffer = decryptedBlocksContainingNeededPart.subarray(neededPartStart, neededPartEnd);

We got it:

console.log(onlyThePartThatWeNeed.toString('utf-8'))
// Output: "lking to her cat, it was hilar"

Furthermore…

… when doing partial/non-partial encryption/decryption in real-world applications you most surely want to do it in a stream. The streaming part is not covered in this article, but when you understand the basics of how the partial encryption/decryption works doing it in streams is not a hard task. Moreover we had to implement this in both Node.js and Golang due to some specific requirements of our product.

My contacts:
Telegram: t.me/edgar_p_yan
Github: github.com/Edgar-P-yan
Twitter/X: x.com/edgar_p_yan