Go Down

Topic: Benchmark STM32 vs ATMega328 (nano) vs SAM3X8E (due) vs MK20DX256 (teensy 3.2)  (Read 20198 times) previous topic - next topic

trycage

I suggest modifying a volatile variable inside the loop.


Code: [Select]
volatile byte dosomething;
  :
 for (lc=le; lc<(le+30000); lc++) //this syntax avoid compiler semplifications
  {
      dosomething = 0;
  }

Because null loops are pretty boring.  Then you won't need to be so tricky with your loops, either...

Various "long" variables used to hold timestamps should be "unsigned long"

Thanks westfw, our initial version of the code included some operations in the INT loop, however we reason that in the FOR statement there was already an increment operation. The code use the INT loop to calibrate the speed of the FLOAT loop, and it is probably ok to have a rough comparison between the platforms we got.

Probably I could code a WHILE statement where comparison and increment can appear as different recognizable operation, but I got the feeling that It would not be that different for the compiler.

Thanks a lot for the input.

mantoui

FYI, here are some more results v1.01 (pragma -O1) on Teensy 3.5/3.6/3.2 and on dragonfly (STM32L4@80MHz, hardware float)
Code: [Select]
       t3.6 @180mhz
         INT_LOOP(30000) bench...= 500 microseconds 60.00MIPS
         LONG_LOOP(30000) bench...= 502 microseconds 59.76MIPS
         FLOAT_DIV(30000) bench...= 2503 microseconds 14.99MFLOPS
         DOUBLE_DIV(30000) bench...= 9343 microseconds 3.39MFLOPS
         FLOAT_MUL(30000) bench...= 667 microseconds 181.82MFLOPS
         DOUBLE_MUL(30000) bench...= 7008 microseconds 4.61MFLOPS

     t3.6 @120mhz
        INT_LOOP(30000) bench...= 752 microseconds 39.89MIPS
        LONG_LOOP(30000) bench...= 753 microseconds 39.84MIPS
        FLOAT_DIV(30000) bench...= 3756 microseconds 9.99MFLOPS
        DOUBLE_DIV(30000) bench...= 14019 microseconds 2.26MFLOPS
        FLOAT_MUL(30000) bench...= 1001 microseconds 120.97MFLOPS
        DOUBLE_MUL(30000) bench...= 10514 microseconds 3.07MFLOPS

       t3.5@120mhz
        INT_LOOP(30000) bench...= 752 microseconds 39.89MIPS
        LONG_LOOP(30000) bench...= 755 microseconds 39.74MIPS
        FLOAT_DIV(30000) bench...= 3758 microseconds 9.99MFLOPS
        DOUBLE_DIV(30000) bench...= 18797 microseconds 1.66MFLOPS
        FLOAT_MUL(30000) bench...= 1003 microseconds 120.97MFLOPS
        DOUBLE_MUL(30000) bench...= 10529 microseconds 3.07MFLOPS

      t3.2@120mhz
        INT_LOOP(30000) bench...= 751 microseconds 39.95MIPS
        LONG_LOOP(30000) bench...= 755 microseconds 39.74MIPS
        FLOAT_DIV(30000) bench...= 8784 microseconds 3.74MFLOPS
        DOUBLE_DIV(30000) bench...= 17559 microseconds 1.79MFLOPS
        FLOAT_MUL(30000) bench...= 6771 microseconds 4.99MFLOPS
        DOUBLE_MUL(30000) bench...= 10533 microseconds 3.07MFLOPS

    dragonfly@80MHz      
       INT_LOOP(30000) bench...= 1129 microseconds 26.57MIPS
       LONG_LOOP(30000) bench...= 1129 microseconds 26.57MIPS
       FLOAT_DIV(30000) bench...= 5641 microseconds 6.65MFLOPS
       DOUBLE_DIV(30000) bench...= 21813 microseconds 1.45MFLOPS
       FLOAT_MUL(30000) bench...= 1883 microseconds 39.79MFLOPS
       DOUBLE_MUL(30000) bench...= 16173 microseconds 1.99MFLOPS


trycage

-Updated
Added Arduino Zero and Arduino Pro 1284 (Thanks Budvar10)

gdsports

Adafruit Metro M4 Express (samd51 @120MHz) cache on
 INT_LOOP(30000) bench...= 752 microseconds 39.89MIPS
 LONG_LOOP(30000) bench...= 753 microseconds 39.84MIPS
 FLOAT_DIV(30000) bench...= 3756 microseconds 9.99MFLOPS
 DOUBLE_DIV(30000) bench...= 14022 microseconds 2.26MFLOPS
 FLOAT_MUL(30000) bench...= 1002 microseconds 120.48MFLOPS
 DOUBLE_MUL(30000) bench...= 10516 microseconds 3.07MFLOPS

trycage

@gdsports Thanks!!!!

Then:

-Update
Added Adafruit Metro M4 Express (Thanks gdsports)

moises1953

Operations in less time than calibration loop?. Not posible. May be invalid formating of time functions.

Arduino Zero (Atmel ATSAMD21G18 48MHz Cortex-M0+)
INT_LOOP(30000) bench...= 116898 microseconds 11.92MIPS
LONG_LOOP(30000) bench...= 116898 microseconds 11.93MIPS
FLOAT_DIV(30000) bench...= 116898 microseconds 0.38MFLOPS
DOUBLE_DIV(30000) bench...= 113126 microseconds 0.27MFLOPS
FLOAT_MUL(30000) bench...= 92387 microseconds 0.33MFLOPS
DOUBLE_MUL(30000) bench...= 116898 microseconds 0.26MFLOPS

At high speed the results are imprecise:
Teensy 3.6 (Cortex M4@180Mhz). The result of FLOAT_MUL is 181.82 MIPS.
The empty reference loop has the following repetitive high level operations:
1)increment
2)compare
3)jump
And takes 502 microsecond for 30000 iterations, so 59.76Mloops. The high level operations MIPS are: 59.76*3=179.28
How is posible to achieve 181.82 MIPS using FLOAT_MUL?. Without optimizations must be 180 MIPS or 179.28 may be.

Operations are operation and asignement, and may be the asignement time was negligible. The inclusion of asignement to a constant in the LONG calibration loop may be a best approach, as sugested by westfw.

May be interesting to measure the asignement time (ad MIPS) of diferent data types

The attach contains a operations MIPS comparative table, asigning 3 operations to a loop

moises1953

This is the code of FDIV loop:
Code: [Select]
fa=(float)random(1,2);
  fb=(float)random(1,1000);
  fb=0; // this line must be suppressed
  fg=0;
  le=random(1,2);
  elapsed=micros();
  for (lc=le; lc<(le+30000); lc++)
  {
    fb=fb/fa;       
  }
  elapsed=micros()-elapsed;


If fb is initialized to 0, then all operations are 0./fa, so this initialization must be suppressed, and also in the DDIV loop.

The use of variables fg and dg is useless, and may be suppressed.

The operation in for is a overcharge.

Proposed code for FDIV:
Code: [Select]
fa=(float)random(1,2);
  fb=(float)random(1,1000);
  le=random(1,2);
  lg=le+30000;
  elapsed=micros();
  for (lc=le; lc<lg); lc++)  //this syntax avoid compiler semplifications?
  {
    fb=fb/fa;       
  }
  elapsed=micros()-elapsed;
// compute MIPS and display


The int loop may be a ISUM
Code: [Select]
  ia=random(1,2);
  ib=random(1,1000);
  le=random(1,2);
  lg=le+30000;
  elapsed=micros();
  for (lc=le; lc<lg; lc++) //this syntax avoid compiler semplifications?
  {
    ib=ib+ia;
  }
  elapsed=micros()-elapsed;
// compute MIPS and display

trycage

Operations in less time than calibration loop?. Not posible. May be invalid formating of time functions.

Arduino Zero (Atmel ATSAMD21G18 48MHz Cortex-M0+)
INT_LOOP(30000) bench...= 116898 microseconds 11.92MIPS
LONG_LOOP(30000) bench...= 116898 microseconds 11.93MIPS
FLOAT_DIV(30000) bench...= 116898 microseconds 0.38MFLOPS
DOUBLE_DIV(30000) bench...= 113126 microseconds 0.27MFLOPS
FLOAT_MUL(30000) bench...= 92387 microseconds 0.33MFLOPS
DOUBLE_MUL(30000) bench...= 116898 microseconds 0.26MFLOPS

At high speed the results are imprecise:
Teensy 3.6 (Cortex M4@180Mhz). The result of FLOAT_MUL is 181.82 MIPS.
The empty reference loop has the following repetitive high level operations:
1)increment
2)compare
3)jump
And takes 502 microsecond for 30000 iterations, so 59.76Mloops. The high level operations MIPS are: 59.76*3=179.28
How is posible to achieve 181.82 MIPS using FLOAT_MUL?. Without optimizations must be 180 MIPS or 179.28 may be.

Operations are operation and asignement, and may be the asignement time was negligible. The inclusion of asignement to a constant in the LONG calibration loop may be a best approach, as sugested by westfw.

May be interesting to measure the asignement time (ad MIPS) of diferent data types

The attach contains a operations MIPS comparative table, asigning 3 operations to a loop
Thanks Moises. I am grateful you took the time to look at the code.

I wrote the code a while ago, (indeed 180MHz microcontrollers were not exactly a target).

, if I recall correctly I tried to make all the loops look similar "in structure" to the calibration loop (so I could remove the loop weight). A float should give about 180MFLOPS in cortex-M4+FPU. I see your points however the accuracy is quite undermined by the use of the function micros (which has a granularity of 8 microseconds) and a loop of 30000 is probably quite insufficient. Actually I think 181.82MFLOPS is quite close, but probably the number of digits is definetely pointless.

The "DUMMY" assignments were made (if I still recall) because they somewhat had an effect in the compiled code. Probably a better programmer would have coded directly in assembler caring to make all the loops exaclty the same (and I am also a lazy programmer most of the time!).

I recall testing the different suggestion (looking at the compiled code), but I did not have time to improve the bench for high speed (without affecting the old results).

:)



Marco



phantom_ts


 ESP32
 INT_LOOP(30000) bench...= 1 microseconds 30000.00MIPS
 LONG_LOOP(30000) bench...= 1 microseconds 30000.00MIPS
 FLOAT_DIV(30000) bench...= 6420 microseconds 4.67MFLOPS
 DOUBLE_DIV(30000) bench...= 5036 microseconds 5.96MFLOPS
 FLOAT_MUL(30000) bench...= 501 microseconds 60.00MFLOPS
 DOUBLE_MUL(30000) bench...= 5544 microseconds 5.41MFLOPS

dsyleixa

I also wrote a benchmark for different MCUs, both AVRs and ARMs.
The benchmark test peforms low- and high-level tests for integers, floats, doubles, bitshift, random, sort, matrix algebra, GPIO r/w, and graphics.
The test will run even without having attached a TFT, you may keep the  #included Adafruit libs or optionally substitute them by proprietary ones.

Update: the test for Raspberry Pi now also has been completed.

As AVRs don't feature 64 bit doubles, the 32bit float test is performed twice, without issueing penalty points though (which admittedly is not fair to the ARM boards ;) )

( ... to be continued ... )

Code: [Select]

test design:
  0   int_Add     50,000,000 int +,- plus counter
  1   int_Mult    10,000,000 int *,/  plus counter
  2   fp32_ops    2,500,000 fp32 mult, transc.  plus counter
  3   fp64_ops    2,500,000 fp64 mult, transc.  plus counter (if N/A: 32bit)
  4   randomize   2,500,000 Mersenne PRNG (+ * & ^ << >>)
  5   matrx_algb  150,000 2D Matrix algebra (mult, det)
  6   arr_sort    1500 shellsort of random array[500]
  7   GPIO toggle 6,000,000 toggle GPIO r/w  plus counter
  8   Graphics    10*8 textlines + 10*8 shapes + 20 clrscr

.


Vergleichswerte (update: auch für Raspi jetzt komplett durchgeführt):

Arduino MEGA + ILI9225 + Karlson UTFT + Arduino GPIO-r/w
  0     90244  int_Add
  1    237402  int_Mult
  2    163613  fp32_ops(float)
  3    163613  fp32_ops(float=double)
  4    158567  randomize
  5     46085  matrx_algb
  6     23052  arr_sort
  7     41569  GPIO toggle
  8     62109  Graphics   
runtime ges.:  986254
benchmark:     51




Arduino MEGA + ILI9225 + Karlson UTFT + Register bitRead/Write
  0     90238  int_Add
  1    237387  int_Mult
  2    163602  fp32_ops (float)
  3    163602  fp32_ops (float=double)
  4    158557  randomize
  5     45396  matrx_algb
  6     23051  arr_sort
  7      4528  GPIO_toggle bit r/w
  8     62106  Graphics   
runtime ges.:  948467
benchmark:     53 


Arduino MEGA + adafruit_ILI9341 Hardware-SPI  Arduino GPIO r/w
  0     90244  int_Add
  1    237401  int_Mult
  2    163612  fp32_ops (float)
  3    163612  fp32_ops (float=double)
  4    158725  randomize
  5     46079  matrx_algb
  6     23051  arr_sort
  7     41947  GPIO toggle
  8      6915  Graphics   
runtime ges.:  931586
benchmark:     54 
 
 



Arduino/Adafruit M0 + adafruit_ILI9341 Hardware-SPI 
  0      7746  int_Add
  1     15795  int_Mult
  2     89054  fp32_ops
  3    199888  fp64_ops(double)
  4     17675  randomize
  5     18650  matrx_algb
  6      6328  arr_sort
  7      9944  GPIO_toggle
  8      6752  Graphics
runtime ges.:  371832
benchmark:     134



Arduino DUE + adafruit_ILI9341 Hardware-SPI 
  0      4111  int_Add
  1      1389  int_Mult
  2     29124  fp32_ops(float)
  3     57225  fp64_ops(double)
  4      3853  randomize
  5      4669  matrx_algb
  6      2832  arr_sort
  7     11859  GPIO_toggle
  8      6142  Graphics   
runtime ges.:  121204
benchmark:     413   



Arduino/Adafruit M4 + adafruit_HX8357 Hardware-SPI 
  0      2253  int_Add
  1       872  int_Mult
  2      2773  fp32_ops (float)
  3     24455  fp64_ops (double)
  4      1680  randomize
  5      1962  matrx_algb
  6      1553  arr_sort
  7      2395  GPIO_toggle
  8      4600  Graphics   
runtime ges.:  39864
benchmark:     1254   



Arduino/Adafruit ESP32 + adafruit_HX8357 Hardware-SPI 
  0      2308  int_Add
  1       592  int_Mult
  2      1318  fp32_ops
  3     14528  fp64_ops
  4       825  randomize
  5      1101  matrx_algb
  6       687  arr_sort
  7       972  GPIO_toggle
  8      3053  Graphics   
runtime ges.:  25384     
benchmark:     1969


Raspberry Pi:

Raspi 2 (v1): 4x 900MHz,  GPU 400MHz, no CPU overclock, full-HD, openVG:
  0     384  int_Add
  1     439  int_Mult
  2     346  fp32_ops(float)
  3     441  fp64_ops(double)
  4     399  randomize
  5     173  matrx_algb
  6     508  arr_sort
  7     823  GPIO_toggle
  8    2632  graphics
runtime ges.: 6145
benchmark: 8137   




edit: updated for
Arduino/Adafruit Feather ESP32 



Go Up