I've undertaken a new project - benchmarking all the boards I have. If you see a board is missing from this project, please comment, and I'll get one!
Benchmarks will follow shortly.
1 Like
interesting idea. Any specific benchmark in mind ?
Benchmarking in what?
For example, vehicles can be compared by load capacity, speed, and cross-country ability. Every rank will have its own champion....
yes, definitely multiple ways to compare. For example you can benchmark boards by testing their analog-to-digital conversion speed and accuracy, measuring the execution time for mathematical floating point operations, assessing PWM signal accuracy and frequency limits, evaluating the maximum achievable baud rate for serial communication, comparing power consumption under different workloads, ...
thx for sharing
how did you measure the loop execution for example?
Serial.println("Test 1: Loop execution test...");
startTime = micros();
for (volatile int i = 0; i < 30000; i++) {
// Empty loop
}
endTime = micros();
duration = endTime - startTime;
Serial.print("Loop execution time: ");
Serial.print(duration);
Serial.println(" µs");
Is the volatile supposed to be preventing the optimizer from ignoring the empty loop?
1 Like
Yep.
The loop does nothing. Without volatile, the compiler might optimise or remove it which would nullify the test
Obviously, but I am not sure the compiler still isn't smart enough to optimize the loop in a way that does not give you the results you need.
I would get the assembler listing for EACH board and make sure that loop code id first of all not optimized such that it doesn't reflect reality and that for each board the code is the same.
I am not 100% sure I understand the metric that empty loop measures, and what column it is on the document you presented.
The loop execution code matches the loop execution column.
I can think of five off the top of my head really quickly, I'm sure there's more.
- Baseline performance benchmark
- Detecting overheads
- Real-time constraint calculations
- Time management calculations
- Clock speed validation
I may be wrong here, but if the compiler had optimised the loop away / removed it, wouldn't we get a reading of zero or close to zero after running it? If it wasn't doing something (ie running through the loop 30,000 times), I would think we wouldn't get a time measurement.
Until you look at the assembler code, you can't speculate like in the last paragraph.
As a former hardware/firmware/OS system tuner, I do not recognize your 5 metrics. You need to prove everything you say when it comes to benchmarking. One of the most important documents to do that is the assembler code as that is what is really running although it can be argued there are layers beneath that but I can't recall ever seeing someone other than a system designer digging that deep since it's beyond your direct control anyway.
If you are doing this as comparative benchmarks between boards (board settings are ???) using high level code then these are fine, although an empty loop makes no sense from an app builders perspective, put some meaningful code inside the loop and now the board to board results have meaning.
It would be safer (but still no guarantee) if i was a global volatile and not something with a scope limited to the for loop.
Absolutly, the assembler code for a global vs a loop var will be very different in my experience. It's been a while since I dove this deep but I still think it's a true statement.
The way i think about the difference
➜ For a loop-local volatile variable, modern compilers may optimize away the volatile behavior if they can prove the variable is isolated, has no external visibility, and optimizing it does not violate the as-if rule, as there are no observable side effects.
➜ For a global volatile variable, the compiler must always enforce volatile semantics, ensuring every read and write occurs as written, because the variable has external visibility and may be accessed or modified by hardware, threads, or other parts of the program.
Sounds like that may be the case, but the assembler code is the final judge.
Indeed
This is all very interesting, but my main reason for this thread is information, not arguments on assembler.
These benchmarks are NOT high level. They are extremely basic, and are meant as as a simple comparison between multiple boards running the same code, and how long each operation takes. If there is one in their that doesn't make sense to you, that's great. It is still staying in there because it means something to me and is still useful
I'm sorry to rant, but it is really annoying when I do something and the entire thread gets taken over to argue incessantly about something that doesn't need to be argued over.
I am NOT digging into the assembler for each board as this benchmark is NOT official or really of that much consequence. More than anything it is a "that's interesting" sort of project, and can still be used to see which board is faster at each operation.
For example, the Uno 4u and the Nano Every use the same chip. Why is one so much quicker than the other?
Slight correction, the Nano Every uses an ATMEGA4809 which is the successor to the AVR128DA28 the Uno 4u has.
It is a similar design not the same chip.
My apologies, I hadn't looked at the chip in a while.