Volatile variables inside function

Projects Programming
1

Hi,

I am using watchdog on my arduino like this:

#include "avr/wdt.h"

ISR(WDT_vect) {

//..here comes my code

}

I use several global variables inside this ISR function and if I dont declare them as "volatile" my code does not work as I expect. So I need to use volatile.

Today I realized the code inside ISR is called many other times on my project so I decided to create a named function to it. So now I want to do this:

#include "avr/wdt.h"

ISR(WDT_vect) {
    
    execute_function();

}

The "execute_function" has several global variables... do I still need to declare them as volatile?

2

Yes. Any variable used in the main program and also in the ISR must be volatile. If the
ISR is re-entrant then every global variable used by it needs to be volatile.

Basically volatile tells the compiler the variable changes behind its back, so it mustn't
do any cleverness with them, just read from memory on every reference, write to memory
on every change.

3

Nice! And, should I declare the function execute_function() as volatile? Is there someway I can declare it as volatile?

Another question: why when I use a variable inside the ISR that is not volatile, arduino does not throw an error telling me that?

4

Because it doesn;t really know. Some reading for you.

And btw, you want to keep the IRS as short as absolutely possible. So jumping to another function isn't really a good idea most of the time. Just save what you want to detect with the IRS and exit it. Deal with the effect in the loop.

5

I got another question: do I need to declare as "volatile" only the variables that are changed inside the interrupt or ISR? If a variable has its value only read and never changes inside the interrupt or ISR, do I still need to declare it as volatile Sir?

6

No, if the ISR doesn't change it then it doesn't have to be volatile.

7

I'm not sure about that. The compiler could keep a global variable in registers for a while. The ISR would read an older value. I think that the variable should still be volatile.

If the Arduino is 8-bits, and the variable is longer (for example a 16-bit integer), then the interrupts have to be disabled if that variable is used in the loop().

8

Delta_G:
No, if the ISR doesn't change it then it doesn't have to be volatile.

Wrong. Any global variable shared by an ISR and the main program, or used just by the ISR, must be volatile
unless it is constant. You are telling the compiler not to do fancy optimizations based on the apparent
structure of the program.

9

I thought we were talking about a constant since he said read only. Rereading the post I see that may not have been a safe assumption.

10

MarkT:
Wrong. Any global variable shared by an ISR and the main program, or used just by the ISR, must be volatile
unless it is constant. You are telling the compiler not to do fancy optimizations based on the apparent
structure of the program.

There are some ways to not require declare the variables volatile, but it requires playing some games with how you access and update things in the foreground code.
You can play some games by masking/un-masking interrupts across functions that play with the globals, knowing that the global will be flushed back to RAM before the function returns.
This also potentially requires placing those foreground functions in separate modules to ensure that no other deeper optimizations are done.

The bigger concern of course is atomicity of the variables, especially if the variables are larger than 8 bit on something like the 8 bit AVR.

--- bill

11

MarkT:
Wrong. Any global variable shared by an ISR and the main program, or used just by the ISR, must be volatile
unless it is constant. You are telling the compiler not to do fancy optimizations based on the apparent
structure of the program.

No, a variable used ONLY in an ISR, or ONLY in "foreground" code, does NOT have to be declared volatile.

Regards,
Ray L.

12

bperrybap:
There are some ways to not require declare the variables volatile, but it requires playing some games with how you access and update things in the foreground code.
You can play some games by masking/un-masking interrupts across functions that play with the globals, knowing that the global will be flushed back to RAM before the function returns.
--- bill

There is NO guarantee the variables will be "flushed back to RAM" by the time you re-enable interrupts, since that will often be done when the stack frame is disposed of AFTER your function code has "returned" or otherwise terminated.

Regards,
Ray L.

13

RayLivingston:
There is NO guarantee the variables will be "flushed back to RAM" by the time you re-enable interrupts, since that will often be done when the stack frame is disposed of AFTER your function code has "returned" or otherwise terminated.

Regards,
Ray L.

Yes there is.
I was very specific in my wording:

masking/un-masking interrupts across functions that play with the globals, knowing that the global will be flushed back to RAM before the function returns.

Of particular importance is "across functions that play with the globals".

This is the sequence of what I was referring to:

NoInterrupts();
mess_with_foo();
Interrupts();

The function mess_with_foo() can mess with the global and there is no chance that the ISR can interrupt it while it is running or before the global is updated.
When mess_with_foo() returns, the global is guaranteed to have been updated at which point you can re-enable interrupts.

--- bill

14

I really appreciate everything you said but I still didnt understand the "veredict". Do I need to declare a variable as volatile if its value is not changed inside the interrupt or ISR routine? My variable is not a constant, it may be changed inside the "loop" routine but inside interrupt or ISR it will surelly not be changed, should I declare it as volatile?

15

batata004:
should I declare it as volatile?

It shouldn't be necessary in this case, as it's unlikely that the compiler will make optimization assumptions and remember a value from one call of the ISR to the next.

It will probably work without a volatile declaration. It will definitely work with it. It's so easy, why not make it volatile and remove the risk?

16

batata004:
I really appreciate everything you said but I still didnt understand the "veredict". Do I need to declare a variable as volatile if its value is not changed inside the interrupt or ISR routine? My variable is not a constant, it may be changed inside the "loop" routine but inside interrupt or ISR it will surelly not be changed, should I declare it as volatile?

Your description is incomplete.
You haven't said what each entity is doing with the variable.
(looking at it or modifying it etc...)

It is best to understand what volatile means so you can determine how and when to use it rather than try to get an answer for a specific case.

volatile is used when you have more than one entity (or thread) that is looking at or using the same variable.
It ensures that each thread is flushing the variable to memory so the other thread can have access to the latest update. This is very important when one thread can interrupt/pre-empt another thread which is what happens when an ISR runs.

If you only have 1 thread, either foreground or ISR, that accessing the variable, there is no contention and so volatile is not needed.

You will also need to understand the concept of atomicity as this is very important to volatile variables since the AVR is an 8 bit RISC processor. This means if you are doing something like incrementing a 16 bit variable the value may be seen as "corrupt" since the increment operation is an interrupt able operation.

volatile and atomicity are distinctly different but go hand and hand in that if you are using volatile variables you can easily have an atomicity issue, particularly on something like the AVR when using larger than 8 bit variables.

You can think of volatile as forbidding variable caching. The compiler is smart and will try to leave a variable in a register as long as possible to make the code more efficient and faster. If you make the variable volatile it disables this and the compiler is told to access and update the variable from its real memory location each and every time the variable is accessed or modified.
This is critical if say foreground code is in a loop and looking at a global variable that an ISR is setting.
If the variable was not declared volatile, the foreground code might not see the actual memory update since it would be looked at a "cached" copy of the variable instead of the actual variable in the memory location.
Same is true on the reverse, in that if you have foreground code in a loop that has modified a global variable that the ISR is looking at. If the variable was not declared volatile, then ISR might not see the updated global variable since the foreground loop may not have written it back to memory yet - and might not ever write it to memory.

But again, if you only have 1 thread that looking at and using the variable, regardless if whether that thread is an ISR, volatile is not needed.

atomicity is something else and issues related to it can be subtle and difficult to track down.
Lack of atomicity can cause apparent variable corruption even when volatile is used.
While volatile tells the compiler not to cache the variable, it does not ensure that the value of a variable is fetched or updated atomically.
Issues with atomicity can happen whenever an update to a variable can be interrupted.
This is true for 16 bit variables on an 8 bit processor like the AVR.

For example if you have a volatile variable that is an int (which is 16 bits on the AVR)
an operation like ++, --, or bit operations |= &= ^= ~= and others can be interrupted during the update for each byte.
If you need to ensure atomicity, for larger than 8 bit values, then you have to ensure that the update operation is not interruptible.
To protect a foreground update from an ISR you have to mask interrupts.

Note that there can be atomicity issues even with 8 bit values on the AVR.
An operation like
foo |= bitmask;

Is atomic if bitmask a constant, but is not atomic if bitmask is not a constant.

--- bill

17

@bperrybap AMAZING. Your answer is the most complete one I have found on the entire internet. I know what atomic variable is cause I am a PHP programmer and I use atomicity with mod_security to protect the data of companies I work for. But your explanation was very precise and very easy to put it with clear words.

I have never thought about the volatile as a cache, your comparson was indeed really good! Congrats man, you are a good teacher :slight_smile:

I read twice everything you said and I understood it very well and I was not aware about the 16 bit (or more) variables inside the parallel thread. Indeed, if the register is updating the variable and the processor interrupts that, the variable will have a corrupt value. Really good to know that, I will optimize my code so it only uses the type byte inside the ISR.

I cant use volatile cause I really already working on the edge of ATMEGA, my code is really optimized and I have only a few bytes spare on arduino memory. Also, my code consumes a lot of the power of arduino so I need to make it as more optimized as possible. Using volatile makes my code about 17% slower. I did a very simple benchmarking and it was clear that using volatile was delaying my routines.

But @bperrybap I have a question: my code has a single thread inside the "loop" routine, a really long code but it does not use any interrupt excepet ISR. Lots of variables are created/changed every "loop" call, and none of them are written inside the ISR, only read. So you think I could not use volatile? I removed any volatile clause from my code and it looks work fine, but I want to make sure I will not have problems in the future! Thank you so much for your attention!

18

@bperrybap #15

Thanks for the overview, this was very understandable even for a beginner like me.

bperrybap:
volatile is used when you have more than one entity (or thread) that is looking at or using the same variable.
It ensures that each thread is flushing the variable to memory so the other thread can have access to the latest update. This is very important when one thread can interrupt/pre-empt another thread which is what happens when an ISR runs.

If you only have 1 thread, either foreground or ISR, that accessing the variable, there is no contention and so volatile is not needed.

Could you explain what entity/thread means in this context? (focusing on an Arduino project written in the official IDE) Entity/thread has anything to do with a function written outside the loop() and called several times during loop?

So for example if I have a bunch of function calls in the main loop, which read/write (common) globals, handle state machines based on some of these globals, calls other functions (e.g. set error variables, which also effect the state machines), etc. than volatile should be used for these globals? (ISR-s are not used)

Thanks in advance.

19

amazed:
Could you explain what entity/thread means in this context? (focusing on an Arduino project written in the official IDE) Entity/thread has anything to do with a function written outside the loop() and called several times during loop?

So for example if I have a bunch of function calls in the main loop, which read/write (common) globals, handle state machines based on some of these globals, calls other functions (e.g. set error variables, which also effect the state machines), etc. than volatile should be used for these globals? (ISR-s are not used)

No. No matter how many calls you make from loop() those are still all one execution thread.
volatile is only needed when there are multiple threads/entities/contexts involved that can interrupt each other or run in parallel.
Anything called from loop() no matter how deep the calling goes it is still 1 thread.
That is all the foreground thread.
Code that is run during the ISR would be another thread, and h/w registers would be another entity.
If you have no ISRs, and only have foreground code, and don't have any pointers to h/w registers,
then there is no need for using volatile on any of the globals as there is only one thread/context/entity involved.

--- bill

Even without the use of volatile the compiler ensures that any global variable modified will be available to any and all called functions.
Usually that means it is forced back to memory, but that isn't required. A compiler may choose to optimize a global by keeping it in a register instead, which is allowed as long as it is available to all called functions. If it can't assure this, then it must write it back to memory so that the called function can "see" (re-fetch) it.

volatile is used an only needed where there is more than one entity/thread using/looking at a something that is being modified.
(this is why volatile isn't needed for constants as they are not ever modified)
An entity/thread/context might be foreground code() vs ISR code, or a piece of code vs a hardware register inside the MCU.

volatile can be thought of in terms of caching, the compiler is allowed to "cache" a variable (make a copy of it) into a faster location like a register.
It can then do things with it faster. While the variable is in the "cache", the memory location for the variable is not reflecting the current value of the variable in the cache.
This is ok as long as the code using it that way puts it back to actual memory before it gets looked at/used/modified by some other entity.
For local variables this is never an issue since nothing outside a function can ever see them so the compile never has to write the back to actual memory.
For globals, the compiler must ensure that the updated variable is available to any called function.

Where things get tricky are cases where the compiler can not tell there is another thread/entity involved looking at and/or modifying the variable/memory in parallel to the code using it inside a function.
This is things like an ISR interrupting or h/w registers.
The compiler has no way of seeing or knowing about this parallel / interrupted activity, and that is why volatile must be used in those cases.
volatile is how you tell the compiler that a variable is special and is subject to being modified in ways the compiler can't know about.
using volatile ensures that the compiler does not optimize things for that variable to prevent the actual memory location from being accessed/updated to entities that interrupt the thread,or that run in parallel like h/w registers.

With things like h/w registers, there are multiple entities involved running in parallel even when only a single s/w thread is involved.
For example, suppose you write a loop looking at a bit in a MCU register.
If you didn't declare the memory location as volatile, then the compiler would generate code that would only read the register once, since it doesn't see anyway that it could be modified by the code doing the looking.
It has no idea that the MCU could update the register completely independently of the code executing.

The same could happen on writes. If you don't use volatile, the compiler might not ever write the data to the actual h/w register as it might keep it cached in a temporary MCU register vs storing in the h/w register.

By using volatile, you tell the compiler, "I'm telling you to access at the actual variable/memory location EVERY single time"

--- bill

20

Thanks for the detailed explanation!

bperrybap:
volatile can be thought of in terms of caching, the compiler is allowed to "cache" a variable (make a copy of it) into a faster location like a register.

You mean non-volatile? For me this sentence is kind of opposite of the other paragraphs.