Should I write to SD while inside a timer interrupt??

Hi there!
I have a pretty large project that involved SD logging, ethernet shield, RFID readers...
Right now I have a software interrupt that gets triggered 4 times a second
This interrupt checks for pending cards to be read, and if there are (still within the interrupt service routine "ISR(TIMER1_COMPA_vect)", writes the read card to a file in the SD.

THis method has worked fine for weeks.

I know that the theory states that I should keep my interrupt service route as short as possible, and mine isn't too short, but it's only being triggered 4 times/second. Also I've read something about not using some functions which depend on timing calculations (maybe SD writting) within an interrupt. But the fact is that this has been working fine...til now.

For some reason now, randomly between the 1st and the 4th..5th card, it all gets locked in the middle of writing to the SD card

Can anyone guess what's going on? Writing to the card OUTSIDE the interrupt service route will be a little hard, I would have to write to a buffer and then in my LOOP part, send the buffer to the card, but I would never know how large the buffer would have to be, and my ram space is limited right now

Thanks!

Most times, writing to the SD card simply copies data to a buffer. When the buffer gets full, though, the buffer has to be committed to the card. That is NOT a fast process.

Writing to the card OUTSIDE the interrupt service route will be a little hard, I would have to write to a buffer and then in my LOOP part, send the buffer to the card,

Sorry, that is nonsense. It is no harder to write to the SD card’s buffer in loop() than it is to write in the ISR. You don’t need to buffer the data, since the SD class already does that. But, if you decide to do it anyway, the same size buffer is needed in loop() as is needed in the ISR.

but I would never know how large the buffer would have to be

It’s interesting that you don’t have that same difficulty doing something unnecessary in the ISR.

(sorry I kept hitting TAB to enter my code and accidentally posted it before it was complete, then I had to wait for the 5 minute delay to post again...)

Thanks for your reply, but I don't know if Im understanding you right...
You say it's exactly the same doing it within or outside the ISR.

When I do it inside the ISR, it would be like this (using kinda pseudocode):
ISR () {
SD.open (file);
Sd.write ("time=");
Sd.write (hour);
sd.write (":");
sd.write (minute);
sd.write (" Got a new event!\n");
sd.close();
}

If I do it outside the ISR it would be like:

ISR () {
buffer.append (Hour)
buffer.append (minute)
etc...
}

Inside Loop:
if (buffer.length()) {
Sd.open(file);
Sd.write (buffer);
bzero (buffer);
}

The ISR may be called 1,2,3.. or 200 times before the loop part dispatches the pending messages (ther eare other functions that involve some seconds in the execution, like connecting to s socket to update stuff), so the buffer may ben holding a bunch of messages to be sent, how do I know if I need a 100b or a 10K buffer?

If I do it outsude

you’d probably get arrested.

My previous post was edited (not sure if you'd get another notification about an edit), if you want to check it out.
Thanks

The process for writing the SD card is the same whether you do it in the ISR or in loop(). I don't see why you think it is, or would be, different.

I agree about that, but it’s about the time when I need to write. If I check if I get a card swipe during my main loop(), I may miss swipes as it may be busy during, for example, a sincronization with the server, which takes some seconds. If the “check card swite” routine is invoked duing a periodic (interrupt driven) routine (4/second), it would work either in the middle of a server syncronization, and won’t miss any swipe.
But when I get a card, I should log it into the SD card right away, in order not to miss any

Do you have any other SPI devices on the bus? If you're in the middle of reading or writing to an SPI device and you get an interrupt, then the first device sill has CS selected and it receives some of the instructions for the SD.

SPI transactions can alleviate this problem but I would not rely on them.

Interrupts are a poor way of compensating for a poorly-written sketch. Don't use delay() waiting for the server. Just send the request and set a timer to know when it will be time to look for the server response. You need to keep loop() running thousands of times per second to deal with inputs as they occur.

Thanks for your comment, but I honestly don't think I have a poorly coded sketch.

I don't have a delay awaiting for the server. I have a loop that time out when no response has been received after 5 seconds. The server may be busy so I give it 5 seconds to reply, if I don't get what I want, I close the conection and try again later. I need to make sure that I don't leave my sketch out hanging waiting for a server reply that never comes.

So, if in the middle of those 5 seconds, I get 2 card swipes (normal at peak time), I have lost the first one.

That's why I use interrupts, so every .25 secs I check to see if there are new swipes in which case I store them in a array of "structs" (date, time, ID, etc), waiting to have several of them to be dumped to the server. And also I write the log to my SD card at that time, which I know is not what I should be doing.

This is the best approach I've been able to come up to so far, again the key priority here is not missing a single swipe, but Im open to hear any other scheme that you may want to share.

Thanks

You want to learn it the hard way, so be it. 8)

edugimeno:
That’s why I use interrupts, so every .25 secs I check to see if there are new swipes

That seems a very poor arrangement.

Is it not possible to get the SWIPE to cause an interrupt so you don’t have to keep checking.

And if there is no alternative to regular checking I can’t imagine that it is necessary to use an interrupt to check something that only happens 4 times per second. I would expect it to be possible to poll a device from loop() 100 or 500 times per second.

Post a link to the datasheet for the RFID reader.

And I certainly would NOT write to an SD Card inside an ISR.

…R

To the OP:

Sadly, you'll find most people here consider whether or not to do significant work in an ISR to be pretty much a religious question. Their answer is 100% black and white. Or, more accurately, right and wrong. For those of us who live in the real world, and have been doing this kind of work since the '70s, whether or not it is ok is not black and white, but rather a function of what you're trying to achieve. If the approach you have is working for you, then go for it. There are many perfectly valid reasons to do it that way, despite what you'll be told (in no uncertain terms) here.

As an example, I have a huge Arduino application - many tens of thousands of lines of code, that cmpiles to nearly 300K of object code, running on a Due. That application is servicing numerous "clients" - simultaneously controlling 3 PID-based DC servos doing precision motion control (complete with S-curve acceleration and deceleration ramps), handling two Ethernet telnet clients, a web server, four clients connected to serial ports, an LCD display, and several other devices and interfaces. That program does the majority of its "heavy lifting" in interrupt handlers, as they need to be done on a reliable, predictable schedule. It can also generate a huge volume of "trace" data that allows me to "see" exactly what is happening in the entire system in great detail, down to being able to capture the PID inputs and outputs, to facilitate tuning. Overall, well over half the CPU cycles are spent in interrupt handlers. Yet, ALL of the many client user interfaces remain totally responsive at all times. So, it can be done, and it can work very nicely, if properly written. The only real risk is if you don't fully understand what you can, and cannot, do within an interrupt handler. For example, writing to Serial ports in the interrupt handler is risky at best, so you have to provide mechanisms for dealing with such things in the foreground code.

Regards,
Ray L.

I know that the theory states that I should keep my interrupt service route as short as possible, and mine isn't too short, but it's only being triggered 4 times/second. Also I've read something about not using some functions which depend on timing calculations (maybe SD writting) within an interrupt. But the fact is that this has been working fine...til now.

For some reason now, randomly between the 1st and the 4th..5th card, it all gets locked in the middle of writing to the SD card

Primus: (presses buttons at random)
Secundus: Hey! Dude! Don't do that - it will spit blue ink in your face.
Primus: (contnues to press buttons) Naah - it's been fine so far
Secundus: So what if it's been fine so far? If you keep doing that, it will spit blue ink in your face.
Primus: Look, I've been doing this for an hour. No blue ink. It is never going to spit blue ink.
Secundus: Man, everyone who does that eventually gets blue ink in the face. I'm telling you.
Primus: (contnues to press buttons) Relax! It's been working fine till now.
-- machine gives Primnus a face full of blue ink --
Primus: (sputters) What? What? Why did that happen? Why now and not before? I don't understand - this is unexpected behaviour - for some reason, it's randomly started doing this. Who would have thunk it?

Don't do time-consuming things in an ISR. If you try doing time-consuming things in an ISR, your sketch will lock up. Have the ISR set some volatile flags, and inspect those flags periodically in the main loop of your sketch in a protected block.

If you try doing time-consuming things in an ISR, your sketch will lock up.

I'm sorry, but that statement is ridiculous, and cannot possibly be backed up by objective fact! There is absolutely NO difference in the reliability of interrupt code vs foreground code, as long as both are properly written. As long the software performs its intended purpose, the percentage of time it spends in interrupt handlers is 100%, totally irrelevent, and does NOT in any way, shape or form predict reliability. I have personally had many applications that spent the majority of their time handling interrupts, yet functioned perfectly for many years.

Regards,
Ray L.

Yes, but you're not doing that inside the Arduino framework which requires interrupts enabled for millis() to work properly.

analogread() also causes problems in interrupts as it's possible to interrupt the middle of an analog reading and if you then try to read a different pin, the original one outside the interrupt gets garbage or worse.

MorganS:
Yes, but you’re not doing that inside the Arduino framework which requires interrupts enabled for millis() to work properly.

analogread() also causes problems in interrupts as it’s possible to interrupt the middle of an analog reading and if you then try to read a different pin, the original one outside the interrupt gets garbage or worse.

Yes, I AM doing it inside the “Arduino framework”, and using Serial, analogRead, EthernetServer, EthernetClient, LiquidCrystal, MANY other libraries, and TONS of dynamic memory allocation (as many as 100 or more allocations per second), using malloc, free, new and delete. As I said (more than once), it all works perfectly, IF the code is properly written. It’s all about knowing what you can, and can’t, do where, and when, and writing to code accordingly. Pure foreground code will also crash if badly written…

Regards,
Ray L.

Ok... at this point I think Im going to find a completly different approach in my sketch
As said, it's not that easy. The main () loop invokes, depending on certain conditions, some functions to send or receive stuff from a server, and this can take some seconds, during which I don't want to miss any card swipe.

Im using a Wiegand protocol RFID reader. As you may know, it uses D0 and D1 pins, both driven by interrupts, to push a 0 or a 1 to the queue, and then after 26 bits, it leaves the resulting card ID stored in a variable. I have a wiegand.availabe() function to check if any ID is available, in which case I have to:

  1. Check if it is in the "allowed" list, in which case a digital output is set ("relay open")
  2. store the "punch-in" into an array buffer (I don't send them right away)
  3. Log it into the SD card

The Wiegand library already uses interrupts to receive the bits fom the device.

Yes, it was working 100% until last week (we're still into the prototyping stage), but after some changes, not sure which ones as I added a ton of new functionalities, if I scanned like 5 cards in 5-10 seconds, the serial logging started to be slowed down until it got completly stuck and I realized the code was completly frozen inside, so I had to reboot.

If I disabled SD logging, everything worked again, so it¡'s clear that it's not liking to write to the SD card during an ISR. I have a method to simulate card swipes manually (in the main loop), which doesn't involve interrupts, and I can add 100 cards with no issues, so it's the ISR...

Thanks all!

edugimeno:
Im using a Wiegand protocol RFID reader. As you may know,

I don't know. That is why I asked you (in Reply #10) to post a link to its datasheet.

...R

Robin2:
I don’t know. That is why I asked you (in Reply #10) to post a link to its datasheet.

…R

I wich I had any documentation about this device…As most of them, coming straight from China, it didn’t even come with any sort of manual, just a sticker in the back stating the pins function.

Wiegand is a pretty standard protocol used mostly by RFID card readers. It basically raises (logically) the D0 line when it needs to send a 0 and raises the D1 line when a 1 has to be sent. After a batch of 26 bits have been sent (some variants use 34bits), a complete card has been scanned. Bit 1 and bit 26 are for parity. Bits 2-25 are the 24bit (3 byte, 1 for facility, 2 for Card_id).

I don’t have any issues with the Wiegand reading itself, Im using a library and it works flawlessly, my issue was with doing too much stuff, especially writting to a file in the SD card, during an ISR.

30 mins ago I removed every call to my logging (to serial and sd) function from within the ISR and found a way to log the pending cards from the main loop. Basically I keep a pointer to my last scanned card in the card buffer, and another pointer to the last “logged” card. Then in the main loop, if (logged_card_ptr) < (last_card_ptr) then I log/report the pending cards into Serial and SD.

So far it seems to be working perfectly.

I appreciate all the help in this group, there’s always a lesson learnt with all this discussion