storing ir remote codes in non-volatile memory

I'm working on making a universal remote control for ir devices around my house. So far I have an arduino device (atmega328p @ 8MHz) and sketch that allow me to monitor an ir receiver and record the on and off times for the signal being received. It then replays the same signal with an ir led. The circuit is an ir led connected to pin 11 and an ir receiver hooked up to pin 2. Here is the script I have now.

#include <avr/io.h>

const unsigned long F_IR = 38000; // IR frequency, in hz
const unsigned int PRESCALE = 1; // match this with the values in TCCR2B CS22, CS21, CS20
const byte Timer2APin = 11; // OC2A PIN = PORT B PIN 3 = ATMEGA328 PIN 17 = ARDUINO PIN 11
volatile unsigned int IRPulseNum = 0; // make a counter for IR pulse length array 
const byte codeLength = 68; // one each for hi and low part of each bit 
// enough to record a start bit, 32 code bits,and a stop bit
// still need to find out if this is long enough for all ir codes...
volatile unsigned int code[codeLength]; // make array to hold IR pulse lengths
volatile unsigned long pulseStart; // this will be used to measure pulses from receiver

void setup(){
  pinMode(2, INPUT); // set interrupt pin as input
  digitalWrite(2, HIGH); // turn on the pullup. is this needed?
  pinMode(Timer2APin, INPUT); // SETS DATA DIRECTION REGISTER
  // OUTPUT WILL NOT SHOW UP WHILE SET TO INPUT
  // OUTPUT WILL SHOW UP WHILE SET TO OUTPUT

    // SET VALUE OF TCCR2A
  // (0 << COM2A1) | (1 << COM2A0)             TOGGLE OC2A PIN ON COMPARE MATCH
  // (0 << COM2B1) | (0 << COM2B0)             OC2B PIN DISCONNECTED
  // (1 << WGM21) | (0 << WGM20)               CTC MODE - WGM22 IN TCCR2B MUST ALSO BE 0
  TCCR2A = ((0 << COM2A1) | (1 << COM2A0) | (0 << COM2B1) | (0 << COM2B0) | (1 << WGM21) | (0 << WGM20));
  // SET VALUE OF TCCR2B
  // (0 << FOC2A) | (0 << FOC2B)               FORCE OUT COMPARE A AND B OFF
  // (0 << WGM22)                              CTC MODE - WGM21 AND WGM20 IN TCCR2A MUST ALSO BE 1 AND 0 RESPECTIVELY
  // (0 << CS22) | (0 << CS21) | (1 << CS20)   CLOCK SELECT - NO PRESCALAR
  TCCR2B = ((0 << FOC2A) | (0 << FOC2B) | (0 << WGM22) | (0 << CS22) | (0 << CS21) | (1 << CS20));

  TCNT2 = 0; // RESET COUNTER2 VALUE
  OCR2A = F_CPU / F_IR / 2 / PRESCALE - 1; // SET TOP COUNTER2 VALUE (FOR CTC)
  // have to set this value AFTER the counter starts or it won't work!
  // F_CPU is already defined in the board.txt file
  // just a rearranged formula from datasheet for ctc
  //                f_clk_io
  // f_OC2A = ---------------------------
  //           2 * prescale * (1 + OCR2A)

  attachInterrupt(0, IRDetectLow, FALLING); // put interrupt on pin 2 to detect ir code start
}

void loop(){
  if (IRPulseNum == codeLength){ // this only happens when the code is done recording
    Serial.begin(9600); // turn on serial
    delay(1000); // wait a little to give serial time to start up and not miss anything
    // print out the array of carrier on and off times that was recorded
    Serial.println("Code:");
    for (byte i = 0; i < codeLength; i++)
    {
      if (i % 2 == 0){
        Serial.print("Code[");
        Serial.print(i);
        Serial.print("]: Signal on for ");
      }
      else{
        Serial.print("Code[");
        Serial.print(i);
        Serial.print("]: No signal for "); 
      }
      Serial.print(code[i]);
      Serial.println(" microseconds");
    }
    Serial.end(); // don't need this anymore
    IRPulseNum++; // this will trigger the next if statement
  }
  if (IRPulseNum > codeLength){ // only happens once code is recorded and sent back to computer
    delay(15000); // wait a while
    for (byte i = 0; i < codeLength; i++){ // loop though the recorded array turning the ir signal
      // on and off for the recorded times
      if(i % 2 == 0){ // even array indexes (carrier on)
        pinMode(Timer2APin, OUTPUT); // turn ir signal on
      }
      else{ // odd array indexes (carrier off)
        pinMode(Timer2APin, INPUT); // turn ir signal off
      }
      delayMicroseconds(code[i]); // wait for the amount of time from the array
    }
    pinMode(Timer2APin, INPUT); // turn ir signal off once whole code is sent
  }
}

void IRDetectLow(){ // this runs when carrier start is detected
  if (IRPulseNum == codeLength - 1){ // if last spot in array
    code[IRPulseNum]= micros() - pulseStart; // calculate how long carrier was off
    IRPulseNum++; // increment array counter. used to signal recording is over
    detachInterrupt(0); // turn off interrupt because recording is done
  }
  else{ // not the last spot in array
    if (IRPulseNum > 0){ // if it's not the first pulse
      code[IRPulseNum]= micros() - pulseStart; // calculate how long carrier was off
      IRPulseNum++; // increment array counter
    }
    attachInterrupt(0, IRDetectHigh, RISING); // switch the interrupt to watch for carrier end
    pulseStart = micros(); // record the time when carrier start was detected
  }
}

void IRDetectHigh(){ // this runs when carrier end is detected
  if (IRPulseNum == codeLength - 1){ // if last spot in array
    code[IRPulseNum]= micros() - pulseStart; // calculate how long carrier was on
    IRPulseNum++; // increment array counter. used to signal recording is over
    detachInterrupt(0); // turn off interrupt because recording is done
  }
  else{ // not the last spot in array
    code[IRPulseNum]= micros() - pulseStart; // calculate how long carrier was on
    IRPulseNum++; // increment array counter
    attachInterrupt(0, IRDetectLow, FALLING); // switch the interrupt to watch for carrier start
    pulseStart = micros(); // record the time when carrier end was detected
  }
}

Eventually, I want to be able to save the code and assign it to a button that will play it back. The way it's saved in the array now takes up quite a bit of space (68 unsigned longs * 4 bytes ea = 272 bytes for one code). I realize there are other ways of saving these codes, but this is the most universal way I can think of. I think it should be basically protocol independent as long as you record a long enough signal. Now I'm trying to figure out the best way to save this data. I want it to keep the settings when it turns off, and I would like to save at least a couple dozen codes. Is saving the data into program memory a good option here? If the program took up no space I'd have room for about 117 codes. I'm not sure how big the rest of the program will be when it's doing everything I want it to. What options do I have here? It looks like eeprom chips are pretty cheap, but I've never used one before. Are they a good option here? Are they simple to use? Is there some other option I'm overlooking? Suggestions, comments, or criticisms welcome.

you can only write program memory when its compiled in your code and uploaded (or in other words you cant do it on the fly) the 328 has 1k of eeprom that you can write any time you want, other than that your looking at ram or external rom

Thanks for the reply. I was thinking that if the bootloader can write to flash, then so could I. However, after some more reading, it seems like it is not something I want to try to tackle. It looks like I will need to find a way to shrink the info I'm storing, or have some kind of extra storage. Any recommendations on what I could use to store the codes? It doesn't need to be particularly fast, as I can read it into memory before sending. Or how I could shrink the codes to fit a reasonable number into eeprom? If I give up some resolution on the times, I can scale them down before storing and then scale them up again after reading back and before transmitting. The biggest delay I've seen in a code is at the end, before the code repeats. It was about 40mS. So I think about 60,000uS is the longest thing I would need to store to be safe. The variation I've seen in times for a remote code pulse is about 25uS. If I divide my times which currently have 1uS resolution by 15 to get 15uS resolution, I think that would still be close enough to work. So doing that, I could get away with storing my times as 12 bit numbers [15 * (2^12 - 1) = 61425] instead of 32 bit numbers. Anyone know how to store 12 bit numbers in an array? Or do i have to combine them somehow and disassemble them again later to store in another data type? But that would still take up 12 * 68 / 8 = 102 bytes per code, so I could only save 10 codes. Any ideas how to shrink it more, without having to know the protocol being used? Or recommendations for some kind of external storage to use?

Have you looked at what others have done? For example, have you looked at this... An Arduino universal remote: record and playback IR signals

Yes, I've looked at that. The thing I wanted to do differently was to be able to save codes and play them back without having to know the protocol. His code has the raw mode if it can't id the protocol, which is similar to what I'm doing, but he is using 50uS resolution for recording codes. I'd need to mess around with more devices to see if that is always enough precision. Also I'm not understanding where he is saving the codes, or in what form. Could somebody explain that? On reading through his code again, it makes me wonder if I really need to store times that are so big. The longest signal pulse I've seen in the start or middle of a code is about 10ms. Anything longer is always at the end, and it's when the carrier is turned off. I suppose I could just always leave a big-ish delay after sending any code and not worry about storing the longer times. Using 50uS resolution and up to just 12mS times in the code can get the times down to 8 bits each, or up to 15 codes in eeprom. I don't think 15 codes would be enough, though. I can't think of a way to shrink the codes more than that without just giving up more resolution? Any ideas? I'm really thinking I'll need external storage of some kind. Any suggestions on that?

It's also nice having something you can make more of and/or modify/fix.

drhoff:
Thanks for the reply. I was thinking that if the bootloader can write to flash, then so could I. However, after some more reading, it seems like it is not something I want to try to tackle.

It's really not that hard even with IDE 0022 and has been streamlined a bit since. I can give you cookbook for 0022 and it will probably work but you might want the newer syntax.


If the 328P flash isn't enough, for truly mass storage you can get an SD module for less than $5 (much less in 3+ qty but don't forget jumpers) and be able to customize your device by changing the SD card and be able to back up your codes as PC files.

These have gone up recently, it's the place and item I bought 3 for $2.60 ea. Must be Christmas...
http://dx.com/p/sd-card-reading-writing-module-for-arduino-deep-blue-142121

The modules work with 5V or 3.3V, I took a leap and didn't get smoke. I used the standard SPI and SD libraries (though IMO the SdFat library allows more options) and got success right away.

Note that my SD card was pre-formatted FAT, as they usually are, it has to be FAT16 or FAT32

I looked up the maker's name from the back (which has changed) to try and get docs and ended up sending them an email with some suggestions to make their site easier. Somehow Mikey, they like me! They will deal direct through PayPal and have cool modules:
http://www.lctech-inc.com/Hardware/

But I don't see a storefront. My contact wrote that anyone may buy from them and I am still waiting for my 1st shipment, should be a week plus Christmas postal system delay. I paid a little for tracking/registered delivery.

Looks like the same thing on eBay for $2.98 and free shipping. That's with a less than 1 minute check.
http://www.ebay.com/itm/New-SD-Card-Module-Slot-Socket-Reader-for-Arduino-ARM-MCU-/280758300478?pt=LH_DefaultDomain_0&hash=item415e7faf3e

Who knows what the price is next year? Could go either way thanks to politics.

You could build your own for less too.

drhoff:
The thing I wanted to do differently was to be able to save codes and play them back without having to know the protocol.

In that case, other than altering the quanta (0.1 millisecond seems to be the smallest quanta you could ever encounter), there isn't much you can do to compress the data. Huffman coding may provide some compression.

but he is using 50uS resolution for recording codes...

As I said above, I suspect 500 uS is adequate but...

...I'd need to mess around with more devices to see if that is always enough precision.

...is certainly worth doing.

I'm really thinking I'll need external storage of some kind. Any suggestions on that?

Flash-drive (as GoForSmoke mentioned), FRAM (CrossRoads' favourite), EEPROM (external; Atmel sells these).

All three are available with SPI communications. The latter two are available with I2C communications. I would look for something with an existing Arduino library.

Those are good suggestions. I had never even heard of F-ram before. Looks very good for some applications (speedy and large storage sizes available), but the chips seem fairly expensive compared to the other options.

It's really not that hard even with IDE 0022 and has been streamlined a bit since..

Can you point me to some more info on how to do this? I think this is my preferred option. Otherwise, I think I'll just have to get some eeprom chips and play with those. Also, for the people that know about eeprom, is there anything I should look for, or avoid specifically in a chip? For example I've seen serial and parallel versions, and I probably don't want to waste more pins than I need to with a parallel chip.

GoForSmoke:
It's really not that hard even with IDE 0022 and has been streamlined a bit since. I can give you cookbook for 0022 and it will probably work but you might want the newer syntax.

Am I correct in interpreting this as you saying there IS a way to save to program memory while the program is running? I can't seem to find anybody else who says this is possible.

Just a thought... Would it be possible to have the bootloader copy data in eeprom into flash on reset? So it would go something like this...

reset
bootloader copies value of eeprom into an area of flash after existing program and deletes eeprom
during program, save something new into eeprom
repeat

Although I'm not sure how you could delete old info you don't want anymore that is now in flash. Does this seem doable? Has anybody already done something like this?

drhoff:

GoForSmoke:
It's really not that hard even with IDE 0022 and has been streamlined a bit since. I can give you cookbook for 0022 and it will probably work but you might want the newer syntax.

Am I correct in interpreting this as you saying there IS a way to save to program memory while the program is running? I can't seem to find anybody else who says this is possible.

Not with Arduino, the bootloader doesn't allow writing to flash at run time.

What you would do is get the codes while connected to a PC and send them to serial monitor or other program and then store them in PROGMEM (the old name for FRAM, it's the flash ram on the 328P that the program is stored on, flash ram/program memory become FRAM/PROGMEM names) for the new version with added codes.

Advantages are speed and not having to buy or wire anything to make it work.

Thanks for the clarification. I would like to make this device not be computer dependent, so I guess program memory is out.

I can't keep up with you so here is 2 out-of-phase replies.

The device can be PC independent in the end-use mode. You might even try an Arduino based 'reader' to gather codes and then a separate end product that uses them.

BTW, check this Teensy with micro-SD adapter:
http://www.pjrc.com/store/sd_adaptor.html

Only problem is they don't sell it assembled and if you solder as poorly as I you can burn little parts up on the SD adapter trying to solder the pins in place then hear a >loud< snap when you power it up. Honest, I first thought someone shot a 22 in through the window!

drhoff:
Those are good suggestions. I had never even heard of F-ram before. Looks very good for some applications (speedy and large storage sizes available), but the chips seem fairly expensive compared to the other options.

It's really not that hard even with IDE 0022 and has been streamlined a bit since..

Can you point me to some more info on how to do this? I think this is my preferred option. Otherwise, I think I'll just have to get some eeprom chips and play with those. Also, for the people that know about eeprom, is there anything I should look for, or avoid specifically in a chip? For example I've seen serial and parallel versions, and I probably don't want to waste more pins than I need to with a parallel chip.

You have 32k of flash on the 328P and 1k of EEPROM which is not nearly enough for your needs.

The SD modules are cheap and easy, pick up some extras just because shipping can take 2-4 weeks.

It took me literally minutes to wire one up but then I did have a 10-pack of female-male jumpers (female ends fit onto the module header pins, male ends plugged right into the Arduino holes, pin map is right on this site). If you have an old floppy cable I can show you how to use that and just plain old breadboard jumpers to make the connects (boy do I like old PC cables!).

Making test code to verify the connections were right also took minutes. Collecting docs on the module took a day (had to be emailed, they're not public domain), trying to understand them and ask questions here took another day, working up the nerve to try it out without being sure the thing could take 5V took about 2 hours buy HEYYYY it worked fine so you don't have to go through that part!

man you make it sound so complicated, first time I messed with it, I just took some resistors and away I went. Course that dosent work with every sd card.. here it is, you need a socket, a level shifter, and a card. most modules are just that with some added caps for filtering.

Yes, it's real hard to connect 8 jumpers and load 2 libraries into a test program. It took literally minutes once I got the nerve to try.

The modules I got have a regulator, 4 resistors (maybe 5) and 2 caps. But if you work with 3.3V you don't need any of that except maybe the caps for high speed?

I've seen instructions to turn a micro-SD adapter into the holder for micro-SD cards, don't miss the link about level-shifting. BTW, Frank does some amazing projects:
http://www.frank-zhao.com/cache/fat_sd.php