Benchmark STM32 vs ATMega328 (nano) vs SAM3X8E (due) vs MK20DX256 (teensy 3.2)

I've not had a chance to try the Teensy 3.5 I got from Paul, but your bench test looks like an easy thing to try (thanks for posting it).

Teensy 3.5 (MK64FX512 Cortex-M4 120 MHz)
INT_LOOP(30000) bench...= 765 microseconds 39.22MIPS
LONG_LOOP(30000) bench...= 757 microseconds 39.63MIPS
FLOAT_DIV(30000) bench...= 3762 microseconds 9.98MFLOPS
DOUBLE_DIV(30000) bench...= 26316 microseconds 1.17MFLOPS
FLOAT_MUL(30000) bench...= 1257 microseconds 60.00MFLOPS
DOUBLE_MUL(30000) bench...= 10534 microseconds 3.07MFLOPS

NOTE: used Arduino IDE 1.6.12 with Teensyduino 1.31 beta1

EDIT: benchmark with v1.01:
ESP8266 (Wemos D1 mini PRO) @ 80 MHz

INT_LOOP(30000) bench...= 1504 microseconds 19.95MIPS
LONG_LOOP(30000) bench...= 1500 microseconds 20.00MIPS
FLOAT_DIV(30000) bench...= 15001 microseconds 2.22MFLOPS
DOUBLE_DIV(30000) bench...= 16126 microseconds 2.05MFLOPS
FLOAT_MUL(30000) bench...= 19876 microseconds 1.63MFLOPS
DOUBLE_MUL(30000) bench...= 21376 microseconds 1.51MFLOPS

ESP8266 (Wemos D1 mini PRO) @ 160 MHz

INT_LOOP(30000) bench...= 752 microseconds 39.89MIPS
LONG_LOOP(30000) bench...= 750 microseconds 40.00MIPS
FLOAT_DIV(30000) bench...= 7500 microseconds 4.44MFLOPS
DOUBLE_DIV(30000) bench...= 8063 microseconds 4.10MFLOPS
FLOAT_MUL(30000) bench...= 9938 microseconds 3.27MFLOPS
DOUBLE_MUL(30000) bench...= 10688 microseconds 3.02MFLOPS

If anyone got an ESP32 please post your benchmarks! :slight_smile:

Arduino Due (Atmel SAM3X8E 84 MHz Cortex-M3)
INT_LOOP(30000) bench...= 5268 microseconds 5.69MIPS
LONG_LOOP(30000) bench...= 6712 microseconds 4.47MIPS

Teensy 3.2 (MK20DX256 Cortex-M4 96 MHz)
INT_LOOP(30000) bench...= 956 microseconds 31.38MIPS
LONG_LOOP(30000) bench...= 947 microseconds 31.68MIPS

Those are very strange results for the Due...

@rduino-PRO 1284 (ATmega1284P 24MHz)
INT_LOOP(30000) bench...= 9354 microseconds 3.21MIPS
LONG_LOOP(30000) bench...= 14240 microseconds 2.11MIPS
FLOAT_DIV(30000) bench...= 103296 microseconds 0.34MFLOPS
DOUBLE_DIV(30000) bench...= 103307 microseconds 0.34MFLOPS
FLOAT_MUL(30000) bench...= 104565 microseconds 0.33MFLOPS
DOUBLE_MUL(30000) bench...= 104576 microseconds 0.33MFLOPS

Nice to have some more benchmarking tests but the test will give wrong values for the loop times as your printing several lines of text before calculating the loop duration.
I think the elapsed=micros()-elapsed; should come directly after the for loop.

westfw:
Those are very strange results for the Due...

I agree it seens quite slower. The specific loop for integers we are using is

"
for (ic=ie; ic<(ie+30000); ic++)
{

}
"
The specific syntax avoids compiler simplifications (at least with the default compiler) flag as "ie" cannot be evaluated prior execution. It should be implemented as an integer increment and a comparison, however the Cortex-M4 has a more extended instruction set, and the M4 support saturation arithmetic, so probably the different speed is simply due to a different code generated. Hence, as usual the due is maybe not fast in doing this specific task. However, the specific iteration used is quite common in coding.

Any ideas why the DOUBLE_DIV() test is slow on Teensy 3.5

Due, slightly modified; seems more reasonable.

Time (ms)...= 12083 ms
INT_LOOP(30000) bench...= 1151 microseconds 26.06MIPS
LONG_LOOP(30000) bench...= 1131 microseconds 26.53MIPS
FLOAT_DIV(30000) bench...= 28098 microseconds 1.11MFLOPS
DOUBLE_DIV(30000) bench...= 36951 microseconds 0.84MFLOPS
FLOAT_MUL(30000) bench...= 19788 microseconds 1.61MFLOPS
DOUBLE_MUL(30000) bench...= 24436 microseconds 1.29MFLOPS

It turns out that Due compiles with optimization flag "-Os", while Teensy3 compiles with just "-O"
On AVR, -Os seems to incorporate nearly all of the useful optimizations from -O, but that doesn't seem to be the case for ARM. With -Os, Due produces code like this for the integer loop:

  for (ic=ie; ic<(ie+30000); ic++) //this syntax avoid compiler semplifications
   8018a:    460b          mov    r3, r1
   8018c:    f501 42ea     add.w    r2, r1, #29952    ; 0x7500
   80190:    322f          adds    r2, #47    ; 0x2f
   80192:    429a          cmp    r2, r3
   80194:    db01          blt.n    8019a <loop+0x52>
   80196:    3301          adds    r3, #1
   80198:    e7f8          b.n    8018c <loop+0x44>

Notice that the branch at the end goes back to 8018c (the "add.w" instruction), so there are 5 instructions in the loop.
With -O, it does:

  for (ic=ie; ic<(ie+30000); ic++) //this syntax avoid compiler semplifications
   80186:       6823            ldr     r3, [r4, #0]
   80188:       4aa0            ldr     r2, [pc, #640]  ; (8040c <loop+0x2c4>)
   8018a:       6013            str     r3, [r2, #0]
   8018c:       f503 42ea       add.w   r2, r3, #29952  ; 0x7500
   80190:       3230            adds    r2, #48 ; 0x30
   80192:       4293            cmp     r3, r2
   80194:       da05            bge.n   801a2 <loop+0x5a>
   80196:       f247 5330       movw    r3, #30000      ; 0x7530
   8019a:       3b01            subs    r3, #1
   8019c:       d1fd            bne.n   8019a <loop+0x52>

This reduces the loop to a single instruction that it does an "equivalent" number of times, but it has to check the initial condition separately, so it's a bit bigger. (Why they can't use the same 30000 for the add and the loop counter, I'm not sure...) Sketch uses 28,792 bytes, vs Sketch uses 28,044 with -Os - about 3% larger...

Just for kicks, using -O3 makes for 29,528 bytes, and succeeds in completely optimizing the loops away, giving a speed of up to 769 MIPS :slight_smile:

Riva:
Nice to have some more benchmarking tests but the test will give wrong values for the loop times as your printing several lines of text before calculating the loop duration.
I think the elapsed=micros()-elapsed; should come directly after the for loop.

Thanks Riva,

I will implement your modification.

trycage

@ron_sutherland, @hansibull and @Budvar10, if it is not too much trouble
if you could re-run the latest version of bench (whihc now fixes the compiler options on every platform) I will publish your results at the top of the Post.

Thanks

Trycage

I suggest modifying a volatile variable inside the loop.

volatile byte dosomething;
  :
 for (lc=le; lc<(le+30000); lc++) //this syntax avoid compiler semplifications
  {
      dosomething = 0;
  }

Because null loops are pretty boring. Then you won't need to be so tricky with your loops, either...

Various "long" variables used to hold timestamps should be "unsigned long"

...or put NOP instruction there

{
  __asm__ __volatile__("nop"); // AVR
}

EDIT: Forgot this. Several different processors - I totally missed, it was stupid idea. :slight_smile:

"nop" isn't guaranteed to be the right assembly on all chips. (mind you, you'd have to be out of your mind as a chip designer not to have a "nop" instructions, but it could happen...)

@westfw
Yes, yes, while I realize a mistake, you've posted...
:-*

westfw:
I suggest modifying a volatile variable inside the loop.

volatile byte dosomething;

:
for (lc=le; lc<(le+30000); lc++) //this syntax avoid compiler semplifications
  {
      dosomething = 0;
  }



Because null loops are pretty boring. Then you won't need to be so tricky with your loops, either...

Various "long" variables used to hold timestamps should be "unsigned long"

Thanks westfw, our initial version of the code included some operations in the INT loop, however we reason that in the FOR statement there was already an increment operation. The code use the INT loop to calibrate the speed of the FLOAT loop, and it is probably ok to have a rough comparison between the platforms we got.

Probably I could code a WHILE statement where comparison and increment can appear as different recognizable operation, but I got the feeling that It would not be that different for the compiler.

Thanks a lot for the input.

FYI, here are some more results v1.01 (pragma -O1) on Teensy 3.5/3.6/3.2 and on dragonfly (STM32L4@80MHz, hardware float)

       t3.6 @180mhz
         INT_LOOP(30000) bench...= 500 microseconds 60.00MIPS
         LONG_LOOP(30000) bench...= 502 microseconds 59.76MIPS
         FLOAT_DIV(30000) bench...= 2503 microseconds 14.99MFLOPS
         DOUBLE_DIV(30000) bench...= 9343 microseconds 3.39MFLOPS
         FLOAT_MUL(30000) bench...= 667 microseconds 181.82MFLOPS
         DOUBLE_MUL(30000) bench...= 7008 microseconds 4.61MFLOPS

     t3.6 @120mhz
        INT_LOOP(30000) bench...= 752 microseconds 39.89MIPS
        LONG_LOOP(30000) bench...= 753 microseconds 39.84MIPS
        FLOAT_DIV(30000) bench...= 3756 microseconds 9.99MFLOPS
        DOUBLE_DIV(30000) bench...= 14019 microseconds 2.26MFLOPS
        FLOAT_MUL(30000) bench...= 1001 microseconds 120.97MFLOPS
        DOUBLE_MUL(30000) bench...= 10514 microseconds 3.07MFLOPS

       t3.5@120mhz 
        INT_LOOP(30000) bench...= 752 microseconds 39.89MIPS
        LONG_LOOP(30000) bench...= 755 microseconds 39.74MIPS
        FLOAT_DIV(30000) bench...= 3758 microseconds 9.99MFLOPS
        DOUBLE_DIV(30000) bench...= 18797 microseconds 1.66MFLOPS
        FLOAT_MUL(30000) bench...= 1003 microseconds 120.97MFLOPS
        DOUBLE_MUL(30000) bench...= 10529 microseconds 3.07MFLOPS

      t3.2@120mhz
        INT_LOOP(30000) bench...= 751 microseconds 39.95MIPS
        LONG_LOOP(30000) bench...= 755 microseconds 39.74MIPS
        FLOAT_DIV(30000) bench...= 8784 microseconds 3.74MFLOPS
        DOUBLE_DIV(30000) bench...= 17559 microseconds 1.79MFLOPS
        FLOAT_MUL(30000) bench...= 6771 microseconds 4.99MFLOPS
        DOUBLE_MUL(30000) bench...= 10533 microseconds 3.07MFLOPS

    dragonfly@80MHz       
       INT_LOOP(30000) bench...= 1129 microseconds 26.57MIPS
       LONG_LOOP(30000) bench...= 1129 microseconds 26.57MIPS
       FLOAT_DIV(30000) bench...= 5641 microseconds 6.65MFLOPS
       DOUBLE_DIV(30000) bench...= 21813 microseconds 1.45MFLOPS
       FLOAT_MUL(30000) bench...= 1883 microseconds 39.79MFLOPS
       DOUBLE_MUL(30000) bench...= 16173 microseconds 1.99MFLOPS
1 Like

-Updated
Added Arduino Zero and Arduino Pro 1284 (Thanks Budvar10)

Adafruit Metro M4 Express (samd51 @120MHz) cache on
INT_LOOP(30000) bench...= 752 microseconds 39.89MIPS
LONG_LOOP(30000) bench...= 753 microseconds 39.84MIPS
FLOAT_DIV(30000) bench...= 3756 microseconds 9.99MFLOPS
DOUBLE_DIV(30000) bench...= 14022 microseconds 2.26MFLOPS
FLOAT_MUL(30000) bench...= 1002 microseconds 120.48MFLOPS
DOUBLE_MUL(30000) bench...= 10516 microseconds 3.07MFLOPS

@gdsports Thanks!!!!

Then:

-Update
Added Adafruit Metro M4 Express (Thanks gdsports)

Operations in less time than calibration loop?. Not posible. May be invalid formating of time functions.

Arduino Zero (Atmel ATSAMD21G18 48MHz Cortex-M0+)
INT_LOOP(30000) bench...= 116898 microseconds 11.92MIPS
LONG_LOOP(30000) bench...= 116898 microseconds 11.93MIPS
FLOAT_DIV(30000) bench...= 116898 microseconds 0.38MFLOPS
DOUBLE_DIV(30000) bench...= 113126 microseconds 0.27MFLOPS
FLOAT_MUL(30000) bench...= 92387 microseconds 0.33MFLOPS
DOUBLE_MUL(30000) bench...= 116898 microseconds 0.26MFLOPS

At high speed the results are imprecise:
Teensy 3.6 (Cortex M4@180Mhz). The result of FLOAT_MUL is 181.82 MIPS.
The empty reference loop has the following repetitive high level operations:
1)increment
2)compare
3)jump
And takes 502 microsecond for 30000 iterations, so 59.76Mloops. The high level operations MIPS are: 59.76*3=179.28
How is posible to achieve 181.82 MIPS using FLOAT_MUL?. Without optimizations must be 180 MIPS or 179.28 may be.

Operations are operation and asignement, and may be the asignement time was negligible. The inclusion of asignement to a constant in the LONG calibration loop may be a best approach, as sugested by westfw.

May be interesting to measure the asignement time (ad MIPS) of diferent data types

The attach contains a operations MIPS comparative table, asigning 3 operations to a loop

Arduino Benchmark-v0101.doc (22 KB)