AES query...

[DoubleDutch]

From this link:

https://www.purebasic.com/documentation ... coder.html

it appears that AES has to be at least 16 bytes long, then after that, the length does not need to be padded to the next 16-byte boundary.

But the RFC document says it should be padded.

Does PD auto pad this out? If so, why is the output size not a multiple of 16, and how can the decoder decrypt data that's not a multiple of 16?

If it is padded, what is it padded with? So the output can be used with systems other than PB.

[Saki]

Hi
You have to arrange a padding or leave it out.
Take a look at this, change the value 20

Code: Select all

a$=Space(8)          ; Key
b$=Space(16)         ; 32 Bytes
*b=AllocateMemory(8) ; 24+8=32
CopyMemory(*b, @b$+24, 8)
AESEncoder(@b$, @b$, 20, @a$, 128, 0, #PB_Cipher_ECB)
ShowMemoryViewer(@b$, 32)

[DoubleDutch]

Yes, I left it out - I was just wondering how it was padded. Here are some of the results of my experiments, with a key of zero, ecb mode...

Code: Select all

49 bytes, all zero, notice 16 byte sequence repeats, except bytes 32 to 47 - if the length is an exact multiple of 16 then the sequence repeats every 16 bytes.
[23:04:37] 00000000000000000000000000000000 00000000000000000000000000000000 00000000000000000000000000000000 00
[23:04:37] 66E94BD4EF8A2C3B884CFA59CA342B2E 66E94BD4EF8A2C3B884CFA59CA342B2E BFF16C79E641F17539A7DFA3C5133E00 66

49 bytes, all zero except byte 0 - set to 1
[23:05:15] 01000000000000000000000000000000 00000000000000000000000000000000 00000000000000000000000000000000 00
[23:05:15] 47711816E91D6FF059BBBF2BF58E0FD3 66E94BD4EF8A2C3B884CFA59CA342B2E BFF16C79E641F17539A7DFA3C5133E00 66

49 bytes, all zero except byte 0 - set to 2
[23:05:40] 02000000000000000000000000000000 00000000000000000000000000000000 00000000000000000000000000000000 00
[23:05:40] BCF176A7EAAD8085EBACEA362462A281 66E94BD4EF8A2C3B884CFA59CA342B2E BFF16C79E641F17539A7DFA3C5133E00 66

49 bytes, all zero except byte 0 - set to 3
[23:06:38] 03000000000000000000000000000000 00000000000000000000000000000000 00000000000000000000000000000000 00
[23:06:38] 4FFC69772ED5A336F4615B4503C34814 66E94BD4EF8A2C3B884CFA59CA342B2E BFF16C79E641F17539A7DFA3C5133E00 66

49 bytes, all zero except byte 16 - set to 1
[23:08:05] 00000000000000000000000000000000 01000000000000000000000000000000 00000000000000000000000000000000 00
[23:08:05] 66E94BD4EF8A2C3B884CFA59CA342B2E 47711816E91D6FF059BBBF2BF58E0FD3 BFF16C79E641F17539A7DFA3C5133E00 66

49 bytes, all zero except byte 32 - set to 1 (notice that although the 16 to 31 bytes change, byte 48 has changed too)
[23:08:59] 00000000000000000000000000000000 00000000000000000000000000000000 01000000000000000000000000000000 00
[23:08:59] 66E94BD4EF8A2C3B884CFA59CA342B2E 66E94BD4EF8A2C3B884CFA59CA342B2E 9CE27629B3E92B542D2D53A9CB24FD0B 47

49 bytes, all zero except byte 32 - set to 2 (notice that although the 16 to 31 bytes change, byte 48 has changed again)
[23:10:16] 00000000000000000000000000000000 00000000000000000000000000000000 02000000000000000000000000000000 00
[23:10:16] 66E94BD4EF8A2C3B884CFA59CA342B2E 66E94BD4EF8A2C3B884CFA59CA342B2E 7BC3A48659B53CFCC62A11A58241DB0F BC

63 bytes, all zero except byte 32 - set to 1 (notice that although the 16 to 31 bytes change, 48 to 53 has changed too)
[23:12:45] 00000000000000000000000000000000 00000000000000000000000000000000 01000000000000000000000000000000 000000000000000000000000000000
[23:12:45] 66E94BD4EF8A2C3B884CFA59CA342B2E 66E94BD4EF8A2C3B884CFA59CA342B2E FFBE6FD12A5D8BB7A0CCCA252787ACCA 47711816E91D6FF059BBBF2BF58E0F

63 bytes, all zero except byte 32 - set to 1 (notice that although the 16 to 31 bytes change, 48 to 53 has changed too)
[23:15:30] 00000000000000000000000000000000 00000000000000000000000000000000 02000000000000000000000000000000 000000000000000000000000000000
[23:15:30] 66E94BD4EF8A2C3B884CFA59CA342B2E 66E94BD4EF8A2C3B884CFA59CA342B2E E08DD2A51D3A6488BD2104B6C5A51C09 BCF176A7EAAD8085EBACEA362462A2

Any idea how the padding is worked out if the data isn't a size that's a multiple of 16 ? the RFC says it should be padded, purebasic docs say only the 1st 16 bytes need padding.

[Saki]

Hi, that was also the first thing we noticed.
We didn't look any further and always used a zero padding according to the recommendation.
There is no explainable reason why the first block handling should be different from the second.

[DoubleDutch]

I think the first block has to be 16 bytes because it's somehow used as the filler if the other blocks are not 16 bytes. Maybe if they are decoded first then it's overlayed somehow in the shorter block?

Whatever it is, it works - it removes the need for 16 byte boundary padding (after the first 16 bytes), I've confirmed that there is no issue decrypting non 16-byte boundary data in Purebasic, it's just other aes libs have an issue with it as it's non-standard. It would be very useful if @Fred would open up this bit of the code - so Purebasic aes encrypted data could also be decrypted with other languages.

[Saki]

Hi,
the blocks are completely independent,
so also the first block, so you can only move the pointer.
You can always use the first one, it doesn't make any difference.
CBC is just an addition, you can add that as sample also yourself.
There are many ways to use AES.

[DoubleDutch]

They are not IF the size is not on a 16 byte boundary. If this is the case then the previous block to the last partial block is different than it would have been, the partial block is partially 'normal' however (see my data findings above).

If you change a byte in the previous block to the last partial block, both that previous block and the partial block are changed.

So when the size is not a multiple of 16, different blocks are affected (the last one and the previous one).

[Saki]

Make everything divisible by 16, then it is ever OK and compatible.
It is a 16 byte block cipher.
If one block influences the next one it is an error.

I assume that to avoid padding bytes of the previous block are used to fill up the incomplete block,
that is then quite OK.

This explains why the first block must have 16 bytes.

[DoubleDutch]

Having it not fixed to a 16 byte length is useful - it is one more thing that doesn't let people guess what type of encryption you are using.

[Saki]

Unfortunately, this is not so true.
Since an encryption is either secure or insecure, the length does not matter at all.
But you often change the length on purpose to not reveal the real length of the file to the attacker.

[DoubleDutch]

Either way, it would be very useful to know what is different about the Purebasic aes lib, so the data can be decrypted/encrypted with another language.

[Saki]

You just have to stick strictly to the standard.

There are countless possible variations,
good and not good.

Primarily, implementation errors or special features are the cause of many problems with AES.

I don't know now how it works exactly at PB,
but for example you can encrypt 20 bytes without padding.

Code: Select all

iv$=Space(8)
key$=Space(8)
test$="ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
AESEncoder(@test$, @test$, 16, @key$, 128, @iv, #PB_Cipher_CBC)
AESEncoder(@test$+4, @test$+4, 16, @key$, 128, @iv, #PB_Cipher_CBC)
AESDecoder(@test$+4, @test$+4, 16, @key$, 128, @iv, #PB_Cipher_CBC)
AESDecoder(@test$, @test$, 16, @key$, 128, @iv, #PB_Cipher_CBC)
Debug test$

[DoubleDutch]

Yes, I know (that it works) and there are multiple ways to do it .

That's why it would be good to know how PB does it, so it can possibly be reproduced in other languages (so they can read pd encrypted data)...

[Saki]

Hi,
you can use the small code above to try it out, maybe you will find out.
There should not be many variants here.
If you apply it to files, you will quickly reach the point that a file is smaller than 16 bytes, then you have to use padding anyway.
Therefore it is advisable to always work with padding.

[DoubleDutch]

I do that already for string data, I add 15 then and with $fffffff0 when allocating memory for the string to be copied to before the encrypt. I was just thinking it would be good to know the method used for padding.