Bad results of HASH generators

Hi, I have a problem with HASH generators. In most cases the results are incorrect. In the past it works, but I reinstalled OS (now I have Windows 10 and Arduino IDE 1.6.10) and problems began. I tried it on Arduino UNO and Arduino Pro Mini with the same problem.

I tried several libraries: Cryptographic suite for Arduino, ArduinoLibs Cryptographic library, CryptoC, none of them work.

I need to generate HMAC SHA256. In the past I was able to do that. Maybe there is some bug in the IDE? Can somebody help me, please?

Here is output of the test example sha256test from ArduinoLibs Cryptographic library:

Test: FIPS 180-2 B.1
Expect:ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad
Result:ca978112ca1bbdcafac231b39a23dc4da786eff8147c4e72b9807785afee48bb
 Hash took : 41080 micros

Test: FIPS 180-2 B.2
Expect:248d6a61d20638b8e5c026930c3e6039a33ce45964ff2167f6ecedd419db06c1
Result:ca978112ca1bbdcafac231b39a23dc4da786eff8147c4e72b9807785afee48bb
 Hash took : 41080 micros

Test: RFC4231 4.2
Expect:b0344c61d8db38535ca8afceaf0bf12b881dc200c9833da726e9376c2e32cff7
Result:017c04f38fdc5e8b6a754fd3875680925d11dd325a04c55c1ac1468bed4fa1af

Test: RFC4231 4.3
Expect:5bdcc146bf60754e6a042426089575c75a003f089d2739839dec58b964ec3843
Result:2ad5636de0f08f7b0ecdec6db61026636820bb79bba11a13f30ea122f14ae4e2
 Hash took : 53452 micros

Test: RFC4231 4.4
Expect:773ea91e36800e46854db8ebd09181a72959098b3ef8c122d9635514ced565fe
Result:773ea91e36800e46854db8ebd09181a72959098b3ef8c122d9635514ced565fe
 Hash took : 53552 micros

Test: RFC4231 4.5
Expect:82558a389a443c0ea4cc819899f2083a85f0faa3e578f8077a2e3ff46729665b
Result:82558a389a443c0ea4cc819899f2083a85f0faa3e578f8077a2e3ff46729665b
 Hash took : 53552 micros

Test: RFC4231 4.6
Expect:a3b6167473100ee06e0c796c2955552b-------------------------------
Result:b5d822588417b0fac1b7d99ba44772842f95818d418aca4a7ba739be7e445ac5
 Hash took : 53452 micros

Test: RFC4231 4.7
Expect:60e431591ee0b67f0d8a26aacbf5b77f8e0bc6213728c5140546040f0ee37f54
Result:504635cd34096118a1548accaec3c388d8b0c22faf9d362e8ed19fe03d436864
 Hash took : 87664 micros

Test: RFC4231 4.8
Expect:9b09ffa71b942fcb27635fbcd5b0e944bfdc63644f0713938a7f51535c3a35e2
Result:504635cd34096118a1548accaec3c388d8b0c22faf9d362e8ed19fe03d436864
 Hash took : 87704 micros

I haven't looked into the source code(and probably wouldn't understand it anyway) but I encountered a similar issue with the tzikis/ArduinoMD5 library a while back when I updated from Arduino IDE 1.06 to 1.5.8. I found that someone had already fixed the issue: Example deallocation by per1234 · Pull Request #7 · tzikis/ArduinoMD5 · GitHub. After that change it always generates the correct hash regardless of IDE version, including 1.6.10.

Hi,

Hi, I have a problem with HASH generators. In most cases the results are incorrect. In the past it works, but I reinstalled OS (now I have Windows 10 and Arduino IDE 1.6.10) and problems began. I tried it on Arduino UNO and Arduino Pro Mini with the same problem.

Install the earlier version of the IDE that worked, you can have all the versions on your PC at the same time.
Just select the one you want to use.

Tom.... :slight_smile:

Thanks both of you.

I will try to fix some library (probably CryptoC, because it is smallest one of the mentioned libraries). And if I fail to do that, I will have to downgrade to IDE 1.0.6.

If someone else is able to advise, I’ll be glad if he do it.

I'd think that not the IDE is responsible for the wrong results. Instead I'd guess that some too aggressive optimization will make the code fail.

I don't know how this can be done, but I'd try to turn compiler optimizations off and try again. Does there exist a #pragma to turn optimizations off?

Question to the mod: what's the best forum section for this compiler problem?

In the case of the ArduinoMD5 library it was a clear bug in the library that just happened to work with the older compiler or compiler settings so I wouldn’t rule that out.

If the code actually is good and the compiler is at fault then it’s more difficult to identify the exact cause with such a huge leap in versions. Arduino AVR Boards 1.6.12(bundled with Arduino IDE 1.6.10) upgraded to avr-gcc 4.9.2-atmel3.5.3-arduino2 and also added LTO compiler flags to platform.txt. I’d guess there were one or two other upgrades between Arduino IDE 1.0.6 and Arduino AVR Boards 1.6.11. If you want to investigate that possibility you can edit platform.txt to change the compiler flags. The compiler changes in Arduino AVR Boards versions were: 1.6.2: 4.8.1-arduino2, 1.6.3: 4.8.1-arduino3, 1.6.4:4.8.1-arduino5, 1.6.12:4.9.2-atmel3.5.3-arduino2. It looks like 1.0.6 had avr-gcc 4.3.2 and there were probably another change or two between there and 1.6.2. So you could methodically work back or forwards through the compiler versions to find where it broke.

I tried "Cryptographic suite for Arduino" on various versions...

Arduino 1.0.6 - OK
Arduino 1.6.0 - OK
Arduino 1.6.1 - OK
Arduino 1.6.2 - OK
Arduino 1.6.3 - OK
Arduino 1.6.4 - OK
Arduino 1.6.5r5 - OK
Arduino 1.6.6 - Bad results
Arduino 1.6.7 - Bad results
Arduino 1.6.8 - Bad results
Arduino 1.6.9 - Bad results
Arduino 1.6.10 - Bad results (optimizations set to -Os or -O0, it doesn't matter)

Any suggestions?

Switch optimizations off. If the error persists, it's not caused by the optimizer.

Now I tried turn off optimizations with Arduino 1.6.10 (in the file platform.txt) and bad results persist.

Then it smells like a compiler or system library bug, or incorrect use of a some previously existing compiler “feature”.

The hash library itself shouldn’t be Arduino specific. Can somebody test the library with a different compiler, or for a different target?

I successfully tested it even with Arduino IDE 1.6.5. The results are OK. So the results are incorrect starting with version 1.6.6.

There was a big change in the Arduino IDE between 1.6.5-r5 and 1.6.6 due to the switch to using arduino-builder. This did cause a lot of breakage but every report I saw was a compile error due to function prototype generation failure rather than a change in program behavior.

I found parameters -std=gnu11 and -std=gnu++11 in the file platform.txt from version 1.6.6, but if I remove them, the bad results still reproduce.

Another difference in the file platform.txt is this piece of code. Could it have some effect on the mentioned behavior? But personally I think, probably not.

## Create archives
# archive_file_path is needed for backwards compatibility with IDE 1.6.5 or older, IDE 1.6.6 or newer overrides this value
archive_file_path={build.path}/{archive_file}

LucasCZE:
Another difference in the file platform.txt is this piece of code. Could it have some effect on the mentioned behavior? But personally I think, probably not.

I actually added that one! It shouldn't be the cause of the issue but the only reason it's there is so to make new versions of Arduino AVR Boards compatible with old versions of the Arduino IDE. So as long as you're not trying to use that platform.txt with an IDE version 1.6.5-r5 or earlier commenting it out by adding a # at the start of the line just to make sure it's not the cause of your problem won't break anything.

I finally found the problem. 8) Chuck Todd modified the file Print.cpp (3 August 2015) and then problems began.

He changed functions:
size_t Print::write(const uint8_t *buffer, size_t size)
size_t Print::print(const __FlashStringHelper *ifsh)
size_t Print::printNumber(unsigned long n, uint8_t base)
size_t Print::printFloat(double number, uint8_t digits)

The bad results are caused by function Print::write().

Here is old implementation (which works):

size_t Print::write(const uint8_t *buffer, size_t size) {
  size_t n = 0;
  while (size--) {
    n += write(*buffer++);
  }
  return n;
}

And here is new implementation:

size_t Print::write(const uint8_t *buffer, size_t size) {
  size_t n = 0;
  while (size--) {
    if (write(*buffer++)) n++;
    else break;
  }
  return n;
}

Could someone look at it and fix it, please?

Info from Chuck Todd... Print not aborting when Write(char) returns Error #3614

Oh, my bad. It is not wrong implementation in Print.cpp, but in the library. Bug is in the functions size_t Sha256Class::write(uint8_t data) and size_t Sha1Class::write(uint8_t data).

Old code:

size_t Sha256Class::write(uint8_t data) {
  ++byteCount;
  addUncounted(data);
  return 0;
}

The return value have to be 1. After this change everything is OK. :slight_smile:

Thanks all of you to help, the issue is solved. :slight_smile: