Datalogging via processing

Hello. I am currently making the Arduino networked lamp as described in getting started with arduino, and all that’s working just as it says on the tin :slight_smile:

However, in the attached processing script, i’d like to somehow enable basic logging of the stats, for example, how many times a certain keyword has appeared, and what the average light reading has been during the day.

the code is as follows:

// Example 08A: arduino networked lamp
// parts of the code are inspired
// by a blog post by Tod E. Kurt (todbot)
//
// Copy and paste this example into an empty Processing sketch

import processing.serial.*;

String feed = "http://hackaday/feed/";

int interval = 10;  // retrieve feed every 60 seconds;
int lastTime;       // the last time we fetched the content

int pc    = 0;
int mac   = 0;
int linux = 0;

int light = 0;  // light level measured by the lamp

Serial port;
color c;
String cs;

String buffer = ""; // Accumulates characters coming from arduino

PFont font;

void setup() {
  size(640,480);
  frameRate(10);    // we don't need fast updates

  font = loadFont("HelveticaNeue-Bold-32.vlw");  
  fill(255);  
  textFont(font, 32);
  // IMPORTANT NOTE:
  // The first serial port retrieved by Serial.list()
  // should be your arduino. If not, uncomment the next
  // line by deleting the // before it, and re-run the
  // sketch to see a list of serial ports. Then, change
  // the 0 in between [ and ] to the number of the port
  // that your arduino is connected to.
  //println(Serial.list());
  String arduinoPort = Serial.list()[0];
  port = new Serial(this, arduinoPort, 9600); // connect to arduino

  lastTime = 0;
  fetchData();
}

void draw() {
  background( c );
  int n = (interval - ((millis()-lastTime)/1000));

  // Build a colour based on the 3 values
  c = color(mac, pc, linux);
  cs = "#" + hex(c,6); // Prepare a string to be sent to arduino

  text("arduino Networked Lamp", 10,40);
  text("Reading feed:", 10, 100);
  text(feed, 10, 140);

  text("Next update in "+ n + " seconds",10,450);
  text("mac" ,10,200);
  text(" " + mac, 130, 200);
  rect(200,172, mac, 28);

  text("pc ",10,240);
  text(" " + pc, 130, 240);
  rect(200,212, pc, 28);

  text("linux ",10,280);
  text(" " + linux, 130, 280);
  rect(200,252, linux, 28);

  // write the colour string to the screen
  text("sending", 10, 340);
  text(cs, 200,340);

  text("light level", 10, 380);
  rect(200, 352,light/10.23,28); // this turns 1023 into 100

  if (n <= 0) {
    fetchData();
    lastTime = millis();
  }

  port.write(cs); // send data to arduino

  if (port.available() > 0) { // check if there is data waiting
    int inByte = port.read(); // read one byte
    if (inByte != 10) { // if byte is not newline
      buffer = buffer + char(inByte); // just add it to the buffer
    }
    else {

      // newline reached, let's process the data
      if (buffer.length() > 1) { // make sure there is enough data

        // chop off the last character, it's a carriage return
        // (a carriage return is the character at the end of a
        // line of text)
        buffer = buffer.substring(0,buffer.length() -1);
 
        // turn the buffer from string into an integer number
        light = int(buffer);

        // clean the buffer for the next read cycle
        buffer = "";

        // We're likely falling behind in taking readings
        // from arduino. So let's clear the backlog of
        // incoming sensor readings so the next reading is
        // up-to-date.
        port.clear(); 
      }
    } 
  }

}

void fetchData() {
  // we use these strings to parse the feed
  String data; 
  String chunk;

  // zero the counters
  pc    = 0;
  mac   = 0;
  linux = 0;
  try {
    URL url = new URL(feed);  // An object to represent the URL
    // prepare a connection   
    URLConnection conn = url.openConnection(); 
    conn.connect(); // now connect to the Website

    // this is a bit of virtual plumbing as we connect
    // the data coming from the connection to a buffered
    // reader that reads the data one line at a time.
    BufferedReader in = new
      BufferedReader(new InputStreamReader(conn.getInputStream()));

    // read each line from the feed
    while ((data = in.readLine()) != null) {

      StringTokenizer st =
        new StringTokenizer(data,"\"<>,.()[] ");// break it down
      while (st.hasMoreTokens()) {
        // each chunk of data is made lowercase
        chunk= st.nextToken().toLowerCase() ;

        if (chunk.indexOf("pc") >= 0 ) // found "pc"?
          pc++;    // increment pc by 1
        if (chunk.indexOf("mac") >= 0)   // found "mac"?
          mac++;   // increment mac by 1
        if (chunk.indexOf("linux") >= 0) // found "linux"?
          linux++; // increment linux by 1
      }
    }

    // Set 64 to be the maximum number of references we care about.
    if (mac > 64)   mac = 64;
    if (pc > 64)    pc = 64;
    if (linux > 64) linux = 64;

    mac = mac * 4;     // multiply by 4 so that the max is 255,
    pc = pc * 4;       // which comes in handy when building a
    linux = linux * 4; // colour that is made of 4 bytes (ARGB)
  } 
  catch (Exception ex) { // If there was an error, stop the sketch
    ex.printStackTrace();
    System.out.println("ERROR: "+ex.getMessage());
  }

}

(links mangled due to spam protection)

Everything else is working just great, and this is an amazing example of the power of the Arduino, i am very glad it’s included in the book :slight_smile:

Copy and paste this example into an empty Processing sketch

Big no-no. Copying and pasting code you do not understand should not be done.

Instead, type the code, after first looking at each statement and understanding what it is doing, why it is being called, and what variables are being created or modified.

If you had done this, you would know what "key" words were being counted, and where in the code that was being done. You would know where the light level was being computed, and how and why.

Then, you would know where to add additional code to track the average light level.

Events are occurring at specific times. It is easy to determine how long it has been since the previous event. If you knew that, you could add the light level and duration to appropriate variables, and compute an average.

Key words are being located. You could keep track of how often they appear. The specific key words now being tracked (i.e. counted) are "pc", "mac" and "linux". The counters could be accumulated instead of discarded.

You haven't defined when you want to log this data. Is it every time the Arduino is sent data? Is it every hour? Once a day?

Processing has functions to create and write to files. Look at the Help + Reference data. There is a section called Output. That's what you want to do. Look at selectOutput and createWriter.

The reason i am c/ping code that i do not fully understand is because it is a means to an end, I am going to use the networked lamp as part of a school project, and i just wanted the ability to see how many times a particular word had been registered.

i've already been messing around with the code, changing various properties