Reporgramming sparkfun openlog

Sparkfun OpenLog allows easily logging of serial data coming from Arduino.
But it's based on ATMEGA328, so question is: can I upload to it my own Arduino firmware to log digital/analog pin values rather than serial input, so turning it into a standalone logger?

I haven't looked at their code, but as far as I can tell they are just logging incoming data to an SD card. So if you have an SD card attached one way or another, a combination of reading the pins, plus opening a file on the SD card and writing to it, would do it.

I'd like to know HOW to look at their code! They say it's opensource.... but as far as I know, they only provide the .hex version!
Where could I find the .pde/.cpp?
And how can I upload my firmware to OpenLog?

I don't know. It seems the GitHub page is wrong. Perhaps an email to SparkFun?

But who cares what it does? They say it's based on the SDFat library, which is available.

I do, because it does not do what I need, but I would like to know if I can avoid buying an SD reader/writer shield AND an Arduino (I think I can).
OpenLog has the needed circuitry to drive an SD card, and it also mounts an atmega328: so why do I need ANOTHER atmega328 to drive it?!?

It doesn't do what you need, without modification, right? And an Atmega328 chip on a small board with a SD card attached in some way, could also do that.

I think your questions could usefully be directed at Sparkfun. The links they give are broken. So the claim to be open source is not, right now, true.

If you could get the source, and work out how to reprogram the chip, certainly it would be a nice compact solution.

I left a couple of message in OpenLog page, I'll try by email too.

The links they give are broken. So the claim to be open source is not, right now, true.

Links worked yestewrday for me... but the "source" is a .cpp.hex file (double extension?!?) Today they all result in 404-error.

If you could get the source, and work out how to reprogram the chip, certainly it would be a nice compact solution.

Indeed. :slight_smile:

But.... look at what I found! :slight_smile:

http://www.seeedstudio.com/wiki/SDLogger_-_Open_Hardware_Data_Logger

edit: fixed links

Found original OpenLog source:

/*
12-3-09
Copyright SparkFun Electronics© 2010

Can opensource be copirighted?!?

I didn't notice this line in OpenLog Sparkfun page:
Reprogrammable ATmega328 using the Arduino IDE

So it's ACTUALLY an Arduino glued to an SD card reader. :slight_smile:

jumpjack:
Can opensource be copirighted?!?

In particular:

Authors initially derive a right to grant a license to their work based on the legal theory that upon creation of a work the author owns the copyright in that work. What the author/licensor is granting when they grant a license to copy, modify and redistribute their work is the right to use the author’s copyrights. The author still retains ownership of those copyrights, the licensee simply is allowed to use those rights, as granted in the license, so long as they maintain the obligations of the license.

My understanding is that "as owner of the copyright, I give permission for you to use my software". This permission usually includes a clause which stops the recipient from removing that permission for derived works.

jumpjack:
So it's ACTUALLY an Arduino glued to an SD card reader. :slight_smile:

Arduino and Atmeg328 aren't exactly synonyms, but in principle I was trying to say earlier that rather than adapting their gadget, you may as well make your own (eg. using a Nano and an SD card socket). After all, SD cards communicate via SPI so there isn't that much to the interface. A level shifter, maybe.

I was looking for a compact, cheap, standalone, tested and easily programmable SD logger; OpenLog just lacked last feature... I thought. Now i know it's exactly what I was looking for! :slight_smile:

Found main logging routine (lines 611-740):

//Appends a stream of serial data to a given file
//Assumes the currentDirectory variable has been set before entering the routine
//We use the RX interrupt and a circular buffer of 1024 bytes so that we can capture a full stream of
//data even at 115200bps
//Does not exit until Ctrl+z (ASCII 26) is received
//Returns 0 on error
//Returns 1 on success
uint8_t append_file(char* file_name)
{
  //44051
  //This is the size of the receive buffer. The bigger it is, the less likely we will overflow the buffer while doing a record to SD.
  //But we have limited amounts of RAM (~1100 bytes)
  #define BUFF_LEN 250 //50 works well. Too few and we will call file.write A LOT. Too many and we run out of RAM.
  //#define BUFF_LEN 400 //Fails horribly for some reason. Oh right, readSpot only goes to 255.
  //#define BUFF_LEN 100 //Works well.
  char inputBuffer[BUFF_LEN];
  char escape_chars_received = 0;
  byte readSpot = 0; //Limited to 255
  byte incomingByte;

  // O_CREAT - create the file if it does not exist
  // O_APPEND - seek to the end of the file prior to each write
  // O_WRITE - open for write
  if (!file.open(currentDirectory, file_name, O_CREAT | O_APPEND | O_WRITE)) error("open1");

#if DEBUG
  PgmPrintln("File open");
  PgmPrintln("Recording");
#endif

  Serial.print('<'); //give a different prompt to indicate no echoing
  digitalWrite(statled1, HIGH); //Turn on indicator LED

  //Clear out the serial buffer
  Serial.flush();

  //Start recording incoming characters
  //HardwareSerial.cpp has a buffer tied to the interrupt. We increased this buffer to 512 bytes
  //As characters come in, we read them in and record them to FAT.
  while(1){
    uint16_t timeout_counter = 0;

#if BOOTTIME_TESTING
    boottime = millis(); //Get the time from power on to ready to receive
    PgmPrint("Time until ready to recieve (ms): ");
    Serial.println(boottime);
#endif

#if RAM_TESTING
    PgmPrint("Free RAM receive ready: ");
    Serial.println(FreeRam());
#endif

    //Testing to try to get rid of long delay after the first synch
    //This forces OpenLog to scan the fat table and get the pointers ready before the wave of data starts coming in
    //currentDirectory.sync();
    //file.write("123", 3);
    //file.sync();
    //file.close();
    //if (!file.open(currentDirectory, file_name, O_CREAT | O_APPEND | O_WRITE)) error("open1");

    while(!Serial.available()){ //Wait for characters to come in
      if(timeout_counter++ > 1200){ //If we haven't seen a character for about 3 seconds, then record the buffer, sync the SD, and shutdown
        timeout_counter = 0;

        if(readSpot != 0){ //There is unrecorded stuff sitting in the buffer
          //Record the buffer
          if(file.write((byte*)inputBuffer, readSpot) != readSpot)
            PgmPrintln("error writing to file");
        }
        file.sync(); //Push these new file.writes to the SD card
        Serial.flush(); //Clear out the current serial buffer. This is needed if the buffer gets overrun. OpenLog will fail to read the escape character if
        //the buffer gets borked.
        
        //Reset the points so that we don't record these freshly recorded characters a 2nd time, when the unit receives more characters
        readSpot = 0;

        //Now power down until new characters to arrive
        while(!Serial.available()){
          digitalWrite(statled1, LOW); //Turn off stat LED to save power
          sleep_mode(); //Stop everything and go to sleep. Wake up if serial character received
        }
      }
      delay(1); //Hang out for a ms
    }

    incomingByte = Serial.read(); //Grab new character from hardwareserial.cpp buffer (could be 512 bytes)

    //Scan for escape character
    if(incomingByte == setting_escape_character){
#if DEBUG
      Serial.print("!");
#endif
      if(++escape_chars_received == setting_max_escape_character) break;
    }
    else
      escape_chars_received = 0;

    inputBuffer[readSpot++] = incomingByte; //Record character to the local buffer

    if(readSpot == BUFF_LEN){ //If we've filled the local small buffer, pass it to the sd write function.
      //Record the buffer
      if(file.write((byte*)inputBuffer, BUFF_LEN) != BUFF_LEN){
        PgmPrintln("error writing to file");
        break;
      }
      readSpot = 0; //Wrap the buffer
    }

    STAT1_PORT ^= (1<<STAT1); //Toggle the STAT1 LED each time we receive a character
  } //End while - escape character received or error

  //Upon receiving the escape character, we may still have stuff left in the buffer, record the last of the buffer to memory
  if(readSpot != BUFF_LEN){
    //Record the buffer
    if(file.write((byte*)inputBuffer, readSpot) != readSpot)
      PgmPrintln("error writing to file");
  }

  file.sync();
  file.close(); //Done recording, close out the file
  digitalWrite(statled1, LOW); //Turn off indicator LED

#if DEBUG
  PgmPrintln("Done!");
#endif
  PgmPrint("~"); //Indicate a successful record

  return(1); //Success!
}

Its core:

    while(!Serial.available()){ //Wait for characters to come in
      if(timeout_counter++ > 1200){ //If we haven't seen a character for about 3 seconds, then record the buffer, sync the SD, and shutdown
        timeout_counter = 0;

        if(readSpot != 0){ //There is unrecorded stuff sitting in the buffer
          //Record the buffer
          if(file.write((byte*)inputBuffer, readSpot) != readSpot)
            PgmPrintln("error writing to file");
        }
        file.sync(); //Push these new file.writes to the SD card
        Serial.flush(); //Clear out the current serial buffer. This is needed if the buffer gets overrun. OpenLog will fail to read the escape character if
        //the buffer gets borked.
        
        //Reset the points so that we don't record these freshly recorded characters a 2nd time, when the unit receives more characters
        readSpot = 0;

        //Now power down until new characters to arrive
        while(!Serial.available()){
          digitalWrite(statled1, LOW); //Turn off stat LED to save power
          sleep_mode(); //Stop everything and go to sleep. Wake up if serial character received
        }
      }
      delay(1); //Hang out for a ms
    }

    incomingByte = Serial.read(); //Grab new character from hardwareserial.cpp buffer (could be 512 bytes)

    //Scan for escape character
    if(incomingByte == setting_escape_character){
#if DEBUG
      Serial.print("!");
#endif
      if(++escape_chars_received == setting_max_escape_character) break;
    }
    else
      escape_chars_received = 0;

I should change first line of the "core" to make it sensitive to a switch rather than serial input, so logging will stop upon turning on or off or pressing the switch: this will prevent data loss and file corruption w.r.t. just unplugging the device.

Then I'll have to change this line to properly read sensors and write data:

if(file.write((byte*)inputBuffer, readSpot) != readSpot)

Here's the link for Openlog V3 sources on GitHub:

Old versions also available here:

How do I update the firmware once it's ready? Should I just connect Vcc, GND, TX and RX to Arduino? And which board should I select on IDE?

Looking at the board, I would guess that the pro-mini 328 would be similar to the open-log board. Try that and if it does work try other 328 board choices.

Reprogrammable ATmega328 using the Arduino IDE

Found in google cache:

IDE: “Arduino Pro or Pro Mini 5V/16MHz w/ ATmega328”

Get an FTDI Basic and a crossover board
Download a new copy of the Arduino IDE version 0022 into a different directory than your normal Arduino installation. For example ‘c:\arduino-0022-openlog\’. Note that Arduino v1.0 is not yet supported. Please use v0022.
Download the OpenLog firmware either directly here or by checking out the repository (more advanced and not necessary for more people)
Replace the file located here: C:\arduino-0022-openLog\hardware\arduino\cores\arduino\HardwareSerial.cpp with the HardwareSerial.cpp found in the OpenLog firmware zip file that you just downloaded. More information below.
Open the OpenLog_v2 sketch in this special version of Arduino-OpenLog and compile. (If you experience problems, make sure that Arduino isn’t referencing a newer SdFat library in your home/Arduino/libraries folder)
Connecting the FTDI Basic, then the crossover board, then the OpenLog
** Select the “Arduino Pro or Pro Mini 5V/16MHz w/ ATmega328” under the Arduino Boards tab**
Select the right COM port to connect to the FTDI Basic
Upload the code
High five your nearest neighbor

I also found this:

Version 3 of firmware looks quite different; this appears to be the code part I'll have to modify to get analog pins logged:

  //Start recording incoming characters
  while(escape_chars_received < setting_max_escape_character) {

    uint8_t n = NewSerial.read(localBuffer, sizeof(localBuffer)); //Read characters from global buffer into the local buffer
    if (n > 0) {

      if (localBuffer[0] == setting_escape_character) {
        escape_chars_received++;

        //Scan the local buffer for esacape characters
        for(checkedSpot = 1 ; checkedSpot < n ; checkedSpot++) {
          if(localBuffer[checkedSpot] == setting_escape_character) {
            escape_chars_received++;
            //If n is greater than 3 there's a chance here where we receive three esc chars
            // and then reset the variable: 26 26 26 A T + would not escape correctly
            if(escape_chars_received == setting_max_escape_character) break;
          }
          else
            escape_chars_received = 0; 
        }
      }

      file.write(localBuffer, n); //Record the buffer to the card

      STAT1_PORT ^= (1<<STAT1); //Toggle the STAT1 LED each time we record the buffer

      idleTime = 0; //We have characters so reset the idleTime

      if(timeSinceLastRecord++ > maxLoops) { //This will force a sync approximately every second
        timeSinceLastRecord = 0;
        file.sync(); //Sync the card
      }

    }
    else if(idleTime > MAX_IDLE_TIME_MSEC) { //If we haven't received any characters in 2s, goto sleep
      file.sync(); //Sync the card before we go to sleep

      STAT1_PORT &= ~(1<<STAT1); //Turn off stat LED to save power

      power_timer0_disable(); //Shut down peripherals we don't need
      power_spi_disable();
      sleep_mode(); //Stop everything and go to sleep. Wake up if serial character received

      power_spi_enable(); //After wake up, power up peripherals
      power_timer0_enable();

      escape_chars_received = 0; // Clear the esc flag as it has timed out
      idleTime = 0; //A received character woke us up to reset the idleTime
    }
    else {
      idleTime++;
      delay(1); //Burn 1ms waiting for new characters coming in
    }

  }

I received the device but can't get it working: upon connecting to FTDI and to PC, LEDs blink (in a "random" sequence", the they completely stop blinking; no prompt visible in terminal; I switch off and I extract the card, which put into a card reader shows several log files, all empty, and a log file setting baud rate to 9600 bps, just like my terminal was set.

How can I know the meaning of LEDs sequences? How can I get this thing working?!?

Remember, if you get OpenLog stuck into an unknown baud rate, there is a safety mechanism built-in. Tie the RX pin to ground and power up OpenLog. You should see the LEDs blink back and forth for 2 seconds, then blink in unison. Now power down OpenLog and remove the RX/GND jumper. OpenLog is now reset to 9600bps with an escape character of ‘ctrl+z’ pressed three consecutive times.