Yun as a data logger - problems and solutions

I am starting this post because I want to use my Yun as a remote data logger located in a remote region with solar power. This will serve as a document to the development solution and problems. To start I used the example data logger sketch .

PART 1 - Getting yun to write analogue inputs to SD card.

The first issues is a) The data logger ceases to write to the SD if it is not connected to the PC via USB. It does work when connected displaying in serial window the 3 x analogue inputs. I tried adding a blink led routine to show me where the program might be stopping as it compiles fine. When I power the Yun via its own power supply micro usb it does not seem to even run the first code which I have made blink pin 13.

Question is why does it not work ( ie write data to the SD card ) when disconnected and running under its own power and how to solve this ?

As previously stated :

PART 1 - Getting yun in wireless mode to read analogue inputs and then write that data to SD card and serial screen.

Just to plan ahead I laid out a task sheet of what I will need to do :

PART 2 - Yun supplying data to a HTML browser (or equivalent ) page on local network PART 3 - Yun supplying data to a HTML browser (or equivalent ) page to be viewed from a non-local network PART 4 - Accessing Yun from a remote network with Yun supplying data to a HTML browser page (or equivalent ) PART 5 - Yun sending emails of data PART 6 - Controlling the Yun from a remote location PART 7 - Yun power control - keeping the Yun in as little a power mode as possible while measuring data incrementally

PART 8 - Assembling the data logger and installing and testing in field .

The unit is going to be fitted in a remote location and getting internet access to wifi may be an issue yet . If the unit can record on SD card this represents a great backup if this fails. The yun will measure currents and voltages of a solar system to log energy accumulated from both wind and solar . This data will be used to improve the system by knowing what happens when nobody is present on site.

Any ideas welcome.

In order to solve issue 1, double check your sketch does NOT have something like while(!Serial); as that is meant to keep the sketch from running until serial monitor gets opened. That is not your case IIUC

Thanks Federico -
I am ( at this stage ) using the standard data logger sketch in arduino window for bridge ,YUN.

#include <FileIO.h>

void setup() 
{
  // Initialize the Bridge and the Serial
  Bridge.begin();
  Serial.begin(9600);
  FileSystem.begin();

  while (!Serial); // wait for Serial port to connect.
  Serial.println("Filesystem datalogger\n");
}

I thought this was for the serial between the Yun and the linux processor not the Yun and PC . Is this an incorrect assumption ?
Maybe I need to understand in more detail what the serial.begin(): Is it referring to the arduino to linux bridge or the arduino to PC link ?

That was tricky for me too to grasp. Yun has two serial: "Serial" is between yun and your pc (so the bits goes through the usb cable). "Serial1" is used by the Bridge library under the hood and that the one that make the on board linux and arduino talk

So serial.begin() command here means the Arduino to pc @ 9600 ? How is the serial being setup to the Linux side , bridge.begin() command ?

Yes. You can dig the source code of Bridge library (they are in IDE_FOLDER/libraries/Bridge/src, where IDE_FOLDER is where you installed/unpacked the ide)

Thanks Frederico ,

It looks like my first hurdle may be solved by the replacement of serial with the console command . check here - https://www.temboo.com/arduino/yun/serial-to-console I will test this later when I get time and report back.

UPDATE- The following ideas represent an update for solutions to the requirements PART 1 - Getting yun to write analogue inputs to SD card. Just to plan ahead I laid out a task sheet of what I will need to do :

Solution idea - use console

PART 2 - Yun supplying data to a HTML browser page on local network now void , see part 3 PART 3 - Yun supplying data to a HTML browser page to be viewed from a non-local network Solution - both above can use Temboo and Twyla to write data and time to a google spreadsheet on a google drive account . This means it can be processed , graphed and analised from my base which is remote to the place the yun will reside. PART 4 - Accessing Yun from a remote network with Yun supplying data to a HTML browser page now void PART 5 - Yun sending emails of data Solution : now void - This can be absorbed as part of part 2/3 via google sheet PART 6 - Controlling the Yun from a remote location This can be performed by google drive by writing variables into the google sheet and getting the YUN to read those values via wifi . EG interval for reading data : 60 minutes , 30 minutes etc PART 7 - Yun power control - The unit should enter a LOW power mode between data collection to save power drained from solar charged batteries. This may not be crucial if the solar system is working correctly but as the whole point is to work out what is happening while I am not there and cannot monitor the users power habits it will be handy to either have it's own battery power or go into a SLEEP mode . PART 8 - Assembling the data logger and installing and testing on bench . PART9 -Assembling the data logger and installing and testing in field . +++++ Rearranging we now have a clearer requirements list - PART 1 - Getting yun to write analogue inputs to SD card. Just to plan ahead I laid out a task sheet of what I will need to do :

Solution idea - use console

PART 2 - Yun supplying data to a HTML browser page to be viewed from a non-local network Solution - both above can use Temboo and Twyla to write data and time to a google spreadsheet on a google drive account . This means it can be processed , graphed and analised from my base which is remote to the place the yun will reside. PART 3 - Controlling the Yun from a remote location This can be performed by google drive by writing variables into the google sheet and getting the YUN to read those values via wifi . EG interval for reading data : 60 minutes , 30 minutes etc PART 4 - Yun power control - The unit should enter a LOW power mode between data collection to save power drained from solar charged batteries. This may not be crucial is the solar system is working correctly but as the whole point is to work out what is happening while I am not there and cannot monitor the users power habits it will be handy to either have it's own battery power or go into a SLEEP mode . If no Sleep mode exists it may be useful to locate a very low powered sleep module that can turn off the YUNs power. The YUN would have an output trigger once it has measured all data and sent wifi data. The sleep module would then disconnect power from YUN for the specified time . it would also read from the YUN the interval for SLEEP via some communication link. Ideas welcome as to what represents a good board for SLEEP power consumption: ideas PSOC , atmiga chip .. PART 5 - Assembling the data logger and installing and testing on bench . PART 6 -Installing and testing in field .

Here is my updated sketch for the DATA LOGGER
Differences from arduino example
a) It flashes an LED quickly as it starts program
b) It flashes x1 for first pin , x2 for 2nd pin , and x3 for third pin on reading its value
c) it keeps LED on for 1 second after it has finshed sketch and loops again .
d) It works wirelessly without USB connection .
So now we have shown requirement 1 - to read analogue values and to post on serial monitor wirelessly

/*
  SD card datalogger

 This example shows how to log data from three analog sensors
 to an SD card mounted on the Arduino Yún using the Bridge library.

 The circuit:
 * analog sensors on analog pins 0, 1 and 2
 * SD card attached to SD card slot of the Arduino Yún

 Prepare your SD card creating an empty folder in the SD root
 named "arduino". This will ensure that the Yún will create a link
 to the SD to the "/mnt/sd" path.

 You can remove the SD card while the Linux and the
 sketch are running but be careful not to remove it while
 the system is writing to it.

 created  24 Nov 2010
 modified 9 Apr 2012
 by Tom Igoe
 adapted to the Yún Bridge library 20 Jun 2013
 by Federico Vanzati
 modified  21 Jun 2013
 by Tom Igoe

 This example code is in the public domain.

 http://arduino.cc/en/Tutorial/YunDatalogger

 */
// Pin 13 has an LED connected on most Arduino boards.
// give it a name:
int led = 13;

#include <FileIO.h>

void setup() 
{
  // Initialize the Bridge and the Console
  Bridge.begin();
  Console.begin();
  FileSystem.begin();

  // while (!Console); // wait for Console port to connect.
  Console.println("Filesystem datalogger\n");
}


void loop () 
{
  for ( int s = 25 ;s < 35 ; s++) 
  {
    Ledfn(s,s,2) ; // turns on LED on delay ,off delay ,number of loops x
  }
  
  delay(500); // make a string that start with a timestamp for assembling the data to log:
  String dataString;
  dataString += getTimeStamp();
  dataString += " = ";

  // read three sensors and append to the string:
  for (int analogPin = 0; analogPin < 3; analogPin++) 
  {
    Ledfn(250,250,analogPin+1) ; // turns on LED on delay ,off delay ,number of loops x
    delay(1000);
    int sensor = analogRead(analogPin);
    dataString += String(sensor);
    if (analogPin < 2) {
      dataString += ",";  // separate the values with a comma
    }
  }

  // open the file. note that only one file can be open at a time,
  // so you have to close this one before opening another.
  // The FileSystem card is mounted at the following "/mnt/FileSystema1"
  File dataFile = FileSystem.open("/mnt/sd/datalog.txt", FILE_APPEND);

  // if the file is available, write to it:
  if (dataFile) {
    dataFile.println(dataString);
    dataFile.close();
    // print to the Console port too:
    Console.println(dataString);
  }
  // if the file isn't open, pop up an error:
  else {
    Console.println("error opening datalog.txt");
  }
  Ledfn(2000,250,1) ; // turns on LED on delay ,off delay ,number of loops x
  delay(3000);

}

// This function return a string with the time stamp
String getTimeStamp() {
  String result;
  Process time;
  // date is a command line utility to get the date and the time
  // in different formats depending on the additional parameter
  time.begin("date");
  time.addParameter("+%D-%T");  // parameters: D for the complete date mm/dd/yy
  //             T for the time hh:mm:ss
  time.run();  // run the command

  // read the output of the command
  while (time.available() > 0) {
    char c = time.read();
    if (c != '\n')
      result += c;
  }

  return result;
}
//||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
//||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
//||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
// Function to turn LED on OFF with variable ON OFF and DELAY periods

void Ledfn(int ond,int offd, int k ) 
{
  int i = 1 ; 
    for (i=1; i <= k;i++)    
    {
      
      digitalWrite(led, HIGH);   // turn the LED on (HIGH is the voltage level)
      delay(ond);               // wait for a second
      digitalWrite(led, LOW);    // turn the LED off by making the voltage LOW
      delay(offd);               // wait for a second
    }
}

FUTURE CHALLENGES : As you know from above : I am building a data logger on a remote farm which measures voltages and currents for the solar /wind system there. People are not on the farm often ( every 2 weeks ) so there is no way of knowing what goes on with the solar so I thought to monitor it with a YUN./SD card and a wifi ( if possible) . The YUN will run off the 12V power from the solar cell .

I read the YUN doesn't have a real time clock : http://blog.arduino.cc/2013/09/02/arduino-yun-hardware-review/ Federico asked whether it would be useful. Considering the cost I think yes very useful ,2 reasons : .

a) SLEEP and NO wifi = NO time : I need to turn the Yun data logger on only once per hour every hour (or 30 min etc ) . This will save much power if I could send it to sleep completely and wake it for 2 minutes every hour . Ideally, during wake time it will transmit this data collected to the SD card for safe keeping and if a wifi is made available to pass it through temboo and a google spread sheet .

To get around this ( unless somebody has a solution ) I was thinking of using a very low sleep powered chip like (uAmps in sleep state with a real time clock ) to run a relay to turn on the YUN and monitor an output of the YUN to turn back off once data is collected. A real time clock though could be so handy in turning most components off for X amount of time.

b) Accurately logging the data points in date time (if no wifi ) . Without a wifi accurate data logging can not be harnessed.

The second issue I have is that there is no WIFI on site so I need to write my data on SD card so that is a half a solution. The neighbour is about 100 meters away and he has wifi at his house . The area is sparse so I assume very little electrical noise . He ,in principal , has no objection to the yun being on his network and my vision was to use the YUN to write values of data ON SD CARD to a google spread sheet via TEMBOO.

So my challenge is a) trying to get a cheap wifi link to his place at such distance with the YUN . b) and developing some way of knowing when the wifi is connected to dump what has not been sent to google sheet using temboo. eg: Lets say the YUN boots up 3 x over the next 3 hours and for some reason does not get wifi .. it must then on the 4th hour when connection does happen needs to dump the last 3 hours of data . So this case of predictable wifi is not assured maybe not even possible with the distance factor.

IDEAS : I am guessing the only way I may be able to get data to the neighbour is : a) Use some kind of arduino like board at the main site doing the measurements and a transmitter and send data to his place via a receiver and YUN near his house ?

Any ideas welcome :-)

I'm not a hardware guy but 100 meters seems a far distance to me

Regarding sending data not yet synchronized with temboo, that's easy: simply store data on a local database (I use sqlite3) and add a "in_sync" column to the table. You can then schedule a script using crontab. It will select all the rows that are "in_sync = false". As the script the rows to google, it will also set in_sync to true. This way, if the script crashes because of google or because of a connection loss, you'll know what was synchronized and your script would have done "as much as possible" Tip: sync each single row, don't sync all the missing rows in one call as you may end up with duplicates.

Regarding the RTC, I can forward your suggestion, but I'm sure this was taken into consideration during design and I don't know which constraints lead to not having it.

Hi there, I know you started this topic a long time ago but I was wondering if you had any luck with setting your Yun to turn on once an hour? I am starting a similar project where I would like the Yun to turn on only once per day.

I'm very much a beginner to this stuff but any help you can give would be really appreciated!! Thanks!