Pages: [1] 2   Go Down
Author Topic: Reporgramming sparkfun openlog  (Read 5514 times)
0 Members and 1 Guest are viewing this topic.
Rome
Offline Offline
God Member
*****
Karma: 1
Posts: 987
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Sparkfun OpenLog allows easily logging of serial data coming from Arduino.
But it's based on ATMEGA328, so question is: can I upload to it my own Arduino firmware to log digital/analog pin values rather than serial input, so turning it into a standalone logger?
Logged

Global Moderator
Offline Offline
Brattain Member
*****
Karma: 452
Posts: 18694
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I haven't looked at their code, but as far as I can tell they are just logging incoming data to an SD card. So if you have an SD card attached one way or another, a combination of reading the pins, plus opening a file on the SD card and writing to it, would do it.
Logged

Rome
Offline Offline
God Member
*****
Karma: 1
Posts: 987
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I haven't looked at their code, but as far as I can tell they are just logging incoming data to an SD card. So if you have an SD card attached one way or another, a combination of reading the pins, plus opening a file on the SD card and writing to it, would do it.
I'd like to know HOW to look at their code! They say it's opensource.... but as far as I know, they only provide the .hex version!
Where could I find the .pde/.cpp?
And how can I upload my firmware to OpenLog?
Logged

Global Moderator
Offline Offline
Brattain Member
*****
Karma: 452
Posts: 18694
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I don't know. It seems the GitHub page is wrong. Perhaps an email to SparkFun?

But who cares what it does? They say it's based on the SDFat library, which is available.
Logged

Rome
Offline Offline
God Member
*****
Karma: 1
Posts: 987
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset


But who cares what it does?
I do, because it does not do what I need, but I would like to know if I can avoid buying an SD reader/writer shield AND an Arduino (I think I can).
OpenLog has the needed circuitry to drive an SD card, and it also mounts an atmega328: so why do I need ANOTHER atmega328 to drive it?!?
Logged

Global Moderator
Offline Offline
Brattain Member
*****
Karma: 452
Posts: 18694
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

It doesn't do what you need, without modification, right? And an Atmega328 chip on a small board with a SD card attached in some way, could also do that.

I think your questions could usefully be directed at Sparkfun. The links they give are broken. So the claim to be open source is not, right now, true.

If you could get the source, and work out how to reprogram the chip, certainly it would be a nice compact solution.
Logged

Rome
Offline Offline
God Member
*****
Karma: 1
Posts: 987
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

It doesn't do what you need, without modification, right? And an Atmega328 chip on a small board with a SD card attached in some way, could also do that.

I think your questions could usefully be directed at Sparkfun.
I left a couple of message in OpenLog page, I'll try by email too.

Quote
The links they give are broken. So the claim to be open source is not, right now, true.
Links worked yestewrday  for me... but the "source" is a .cpp.hex file (double extension?!?) Today they all result in 404-error.

Quote
If you could get the source, and work out how to reprogram the chip, certainly it would be a nice compact solution.
Indeed.  smiley


But.... look at what I found!  smiley

http://www.seeedstudio.com/wiki/SDLogger_-_Open_Hardware_Data_Logger
https://github.com/magnuskarlsson/SDLogger

edit: fixed links
« Last Edit: April 04, 2012, 05:05:13 am by jumpjack » Logged

Rome
Offline Offline
God Member
*****
Karma: 1
Posts: 987
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Found original OpenLog source:

https://github.com/magnuskarlsson/SDLogger/blob/27f5458e9e3796532fcbbf82ade138c84d72ca4a/OpenLog_v2/OpenLog_v2.pde

Quote
/*
12-3-09
Copyright SparkFun Electronics© 2010

Can opensource be copirighted?!?
Logged

Rome
Offline Offline
God Member
*****
Karma: 1
Posts: 987
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I didn't notice this line in OpenLog Sparkfun page:
Reprogrammable ATmega328 using the Arduino IDE

So it's ACTUALLY an Arduino glued to an SD card reader. :-)
Logged

Global Moderator
Offline Offline
Brattain Member
*****
Karma: 452
Posts: 18694
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Can opensource be copirighted?!?

http://en.wikipedia.org/wiki/Open-source_software

In particular:

Quote
Authors initially derive a right to grant a license to their work based on the legal theory that upon creation of a work the author owns the copyright in that work. What the author/licensor is granting when they grant a license to copy, modify and redistribute their work is the right to use the author’s copyrights. The author still retains ownership of those copyrights, the licensee simply is allowed to use those rights, as granted in the license, so long as they maintain the obligations of the license.

My understanding is that "as owner of the copyright, I give permission for you to use my software". This permission usually includes a clause which stops the recipient from removing that permission for derived works.
Logged

Global Moderator
Offline Offline
Brattain Member
*****
Karma: 452
Posts: 18694
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

So it's ACTUALLY an Arduino glued to an SD card reader. :-)

Arduino and Atmeg328 aren't exactly synonyms, but in principle I was trying to say earlier that rather than adapting their gadget, you may as well make your own (eg. using a Nano and an SD card socket). After all, SD cards communicate via SPI so there isn't that much to the interface. A level shifter, maybe.
Logged

Rome
Offline Offline
God Member
*****
Karma: 1
Posts: 987
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset


Arduino and Atmeg328 aren't exactly synonyms, but in principle I was trying to say earlier that rather than adapting their gadget, you may as well make your own (eg. using a Nano and an SD card socket). After all, SD cards communicate via SPI so there isn't that much to the interface. A level shifter, maybe.
I was looking for a compact, cheap, standalone, tested and easily programmable SD logger; OpenLog just lacked last feature... I thought. Now i know it's exactly what I was looking for! :-)
Logged

Rome
Offline Offline
God Member
*****
Karma: 1
Posts: 987
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Found main logging routine (lines 611-740):
Code:
//Appends a stream of serial data to a given file
//Assumes the currentDirectory variable has been set before entering the routine
//We use the RX interrupt and a circular buffer of 1024 bytes so that we can capture a full stream of
//data even at 115200bps
//Does not exit until Ctrl+z (ASCII 26) is received
//Returns 0 on error
//Returns 1 on success
uint8_t append_file(char* file_name)
{
  //44051
  //This is the size of the receive buffer. The bigger it is, the less likely we will overflow the buffer while doing a record to SD.
  //But we have limited amounts of RAM (~1100 bytes)
  #define BUFF_LEN 250 //50 works well. Too few and we will call file.write A LOT. Too many and we run out of RAM.
  //#define BUFF_LEN 400 //Fails horribly for some reason. Oh right, readSpot only goes to 255.
  //#define BUFF_LEN 100 //Works well.
  char inputBuffer[BUFF_LEN];
  char escape_chars_received = 0;
  byte readSpot = 0; //Limited to 255
  byte incomingByte;

  // O_CREAT - create the file if it does not exist
  // O_APPEND - seek to the end of the file prior to each write
  // O_WRITE - open for write
  if (!file.open(currentDirectory, file_name, O_CREAT | O_APPEND | O_WRITE)) error("open1");

#if DEBUG
  PgmPrintln("File open");
  PgmPrintln("Recording");
#endif

  Serial.print('<'); //give a different prompt to indicate no echoing
  digitalWrite(statled1, HIGH); //Turn on indicator LED

  //Clear out the serial buffer
  Serial.flush();

  //Start recording incoming characters
  //HardwareSerial.cpp has a buffer tied to the interrupt. We increased this buffer to 512 bytes
  //As characters come in, we read them in and record them to FAT.
  while(1){
    uint16_t timeout_counter = 0;

#if BOOTTIME_TESTING
    boottime = millis(); //Get the time from power on to ready to receive
    PgmPrint("Time until ready to recieve (ms): ");
    Serial.println(boottime);
#endif

#if RAM_TESTING
    PgmPrint("Free RAM receive ready: ");
    Serial.println(FreeRam());
#endif

    //Testing to try to get rid of long delay after the first synch
    //This forces OpenLog to scan the fat table and get the pointers ready before the wave of data starts coming in
    //currentDirectory.sync();
    //file.write("123", 3);
    //file.sync();
    //file.close();
    //if (!file.open(currentDirectory, file_name, O_CREAT | O_APPEND | O_WRITE)) error("open1");

    while(!Serial.available()){ //Wait for characters to come in
      if(timeout_counter++ > 1200){ //If we haven't seen a character for about 3 seconds, then record the buffer, sync the SD, and shutdown
        timeout_counter = 0;

        if(readSpot != 0){ //There is unrecorded stuff sitting in the buffer
          //Record the buffer
          if(file.write((byte*)inputBuffer, readSpot) != readSpot)
            PgmPrintln("error writing to file");
        }
        file.sync(); //Push these new file.writes to the SD card
        Serial.flush(); //Clear out the current serial buffer. This is needed if the buffer gets overrun. OpenLog will fail to read the escape character if
        //the buffer gets borked.
        
        //Reset the points so that we don't record these freshly recorded characters a 2nd time, when the unit receives more characters
        readSpot = 0;

        //Now power down until new characters to arrive
        while(!Serial.available()){
          digitalWrite(statled1, LOW); //Turn off stat LED to save power
          sleep_mode(); //Stop everything and go to sleep. Wake up if serial character received
        }
      }
      delay(1); //Hang out for a ms
    }

    incomingByte = Serial.read(); //Grab new character from hardwareserial.cpp buffer (could be 512 bytes)

    //Scan for escape character
    if(incomingByte == setting_escape_character){
#if DEBUG
      Serial.print("!");
#endif
      if(++escape_chars_received == setting_max_escape_character) break;
    }
    else
      escape_chars_received = 0;

    inputBuffer[readSpot++] = incomingByte; //Record character to the local buffer

    if(readSpot == BUFF_LEN){ //If we've filled the local small buffer, pass it to the sd write function.
      //Record the buffer
      if(file.write((byte*)inputBuffer, BUFF_LEN) != BUFF_LEN){
        PgmPrintln("error writing to file");
        break;
      }
      readSpot = 0; //Wrap the buffer
    }

    STAT1_PORT ^= (1<<STAT1); //Toggle the STAT1 LED each time we receive a character
  } //End while - escape character received or error

  //Upon receiving the escape character, we may still have stuff left in the buffer, record the last of the buffer to memory
  if(readSpot != BUFF_LEN){
    //Record the buffer
    if(file.write((byte*)inputBuffer, readSpot) != readSpot)
      PgmPrintln("error writing to file");
  }

  file.sync();
  file.close(); //Done recording, close out the file
  digitalWrite(statled1, LOW); //Turn off indicator LED

#if DEBUG
  PgmPrintln("Done!");
#endif
  PgmPrint("~"); //Indicate a successful record

  return(1); //Success!
}

Its core:

Code:
   while(!Serial.available()){ //Wait for characters to come in
      if(timeout_counter++ > 1200){ //If we haven't seen a character for about 3 seconds, then record the buffer, sync the SD, and shutdown
        timeout_counter = 0;

        if(readSpot != 0){ //There is unrecorded stuff sitting in the buffer
          //Record the buffer
          if(file.write((byte*)inputBuffer, readSpot) != readSpot)
            PgmPrintln("error writing to file");
        }
        file.sync(); //Push these new file.writes to the SD card
        Serial.flush(); //Clear out the current serial buffer. This is needed if the buffer gets overrun. OpenLog will fail to read the escape character if
        //the buffer gets borked.
        
        //Reset the points so that we don't record these freshly recorded characters a 2nd time, when the unit receives more characters
        readSpot = 0;

        //Now power down until new characters to arrive
        while(!Serial.available()){
          digitalWrite(statled1, LOW); //Turn off stat LED to save power
          sleep_mode(); //Stop everything and go to sleep. Wake up if serial character received
        }
      }
      delay(1); //Hang out for a ms
    }

    incomingByte = Serial.read(); //Grab new character from hardwareserial.cpp buffer (could be 512 bytes)

    //Scan for escape character
    if(incomingByte == setting_escape_character){
#if DEBUG
      Serial.print("!");
#endif
      if(++escape_chars_received == setting_max_escape_character) break;
    }
    else
      escape_chars_received = 0;

I should change first line of the "core" to make it sensitive to a switch rather than serial input, so logging will stop upon turning on or off or pressing the switch: this will prevent data loss and file corruption w.r.t. just unplugging the device.

Then I'll have to change this line to properly read sensors and write data:
Code:
if(file.write((byte*)inputBuffer, readSpot) != readSpot)

Logged

Rome
Offline Offline
God Member
*****
Karma: 1
Posts: 987
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Here's the link for Openlog V3 sources on GitHub:
https://github.com/sparkfun/OpenLog

Old versions also available here:
https://github.com/magnuskarlsson/SDLogger
Logged

Rome
Offline Offline
God Member
*****
Karma: 1
Posts: 987
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

How do I update the firmware once it's ready? Should I just connect Vcc, GND, TX and RX to Arduino? And which board should I select on IDE?
Logged

Pages: [1] 2   Go Up
Jump to: