Unit testing considerations and examples

Using Arduino Project Guidance
1

Hi,
I am beginning to work with the unit testing functionality offered by platformIO and Arduino. I am trying to build my intuition on how to design useful unit tests.

I want to start with one of my simplest repositories. Maybe you can follow along and check it out here GitHub - jcsb1994/Buzzer: High level interface for beepers and buzzers **See the examples folder for a quick understanding of how it works, feel free to upload it to a uno with a buzzer connected to pin 7.

This project offers an interface for buzzers, and I am not sure how to test most of its functionalities. The public methods are very high level in my opinion and I cannot seem to find the right way to design unit test methods for them:

  1. void init() returns nothing and initiates private members. I dont see what I can test here
  2. step() will step through a private FSM state and play the specified melody, one note at a time. This is where most of the stuff happens, but everything is wrapped in this single public high level function and from my understanding, unit tests should not be designed for private members, so how can I test my step function, what are the best ways to do so?

Thank you!

2

What's your understanding of "unit tests" with microcontrollers?

Typical unit tests check the value returned from a test program.
How would you make an Arduino sketch return a value?
Which values should be applicable to a buzzer?
How do you determine the value inside the sketch?

1 Like
3

I am not sure if those questions are rhetorical, but using platformIO you do not work on a .ino sketch, but on more traditional .cpp and .h files. There already is support for unit tests within a test folder inside every platformio project folder. All there is to do is to create a test.cpp file(s) that contain proper coding tests. You can consult their doc Redirecting...

I find these questions challenging. Given my step() function is void, I should create a few getters and check the state of the FSM, the melody being played, ... have indeed changed after stepping through the state machine. That would seem like a good approach?

4

I still can't see how emulators or unit tests can provide meaningful results in a real time environment.

Which possible errors do you intend to find how in your code?
In case of a state machine, do you want to check each state transition under all possible circumstances, in all possible sequences of events?

Have you ever found an existing unit test for any state machine?

5

Here are some examples of unit tests in use in official Arduino firmware projects:

6

So in your opinion unit tests are not required in this firmware module? I was expecting to test that the melody plays each note for the right amount of time, completes properly, etc. I thought it was a good thing to write unit tests for all code modules I would design in the future. When would firmware modules be pertinent vs when would they be useless?

I seem to be able to find articles on designing state machine unit tests, so I would guess this practice may be a good thing?

7

IMO such tests are quite useless as soon as complex (controller) hardware is involved.

IMO unit tests are used to detect regression when a known (old) bug occurs in a new software revision again. Such procedures are essential e.g. for compiler evolution. In µC this would mean that you get an error report, write test code and data sets to detect that error in the current code, then fix it and check that the test now succeeds. Let all unit tests run after every new unit version or at least whenever errors are suspected.

From Wikipedia:

Another challenge related to writing the unit tests is the difficulty of setting up realistic and useful tests. It is necessary to create relevant initial conditions so the part of the application being tested behaves like part of the complete system. If these initial conditions are not set correctly, the test will not be exercising the code in a realistic context, which diminishes the value and accuracy of unit test results.

8

What complex hardware? an arduino board or the buzzer? They seem simple to me?

There is a type of automated testing called regression tests testing - Unit Test? Integration Test? Regression Test? Acceptance Test? - Stack Overflow Maybe this is different than unit testing per se.

There is also this article on the pertinence of unit tests in microcontroller code: Unit testing in embedded systems: 3 myths and an automated tip - StarFish Medical

The buzzer library may not be the best example to start with, but I think most modules should be individually tested

9

A µC has a lot of signal processing hardware in it (timers, ADC...) that is already used by the framework (for timing). If a software unit uses part of that hardware that is also used by other units then it's useless to test both units independently, because they probably can not cooperate. Or it's only the computation time of one unit that prevents another unit from working properly. µC coders face such concurrency problems every day, and no single unit test can help them out :frowning:

10

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.