Save data with Processing to .CSV

Hi,

I want to send data from my Arduino to my PC and save the data to a CSV file with processing. But because I’m pretty new, I don’t know how to do this.

I read that I have to send it throught the serial port:

meas[0] = analogRead(0);
meas[1] = analogRead(1);
meas[2] = analogRead(2);
meas[3] = analogRead(3);
meas[4] = analogRead(4);
meas[5] = analogRead(5);

Serial.print(meas[0], DEC);
Serial.print(meas[1], DEC);
Serial.print(meas[2], DEC);
Serial.print(meas[3],,DEC);
Serial.print(meas[4],,DEC);
Serial.println(meas[5],,DEC);

But how do I acces this data in Processing, and how do I create a CSV with it?

Thx!

Out of curiosity only...

Did you look at the examples that come with the version 22 compiler?

Did you look at any of the Processing examples?

Ye, I looked at some examples of Processing.. But I didnt found anything on this

Found nothing on reading serial data? That's hard to believe.

Found nothing on writing to a file? Not as hard to believe, but still...

Found no exact solution? Quite likely.

http://processing.org/reference/libraries/serial/Serial_read_.html http://processing.org/reference/PrintWriter_print_.html

Like I said, I'm new to this, so all I wanted was some help..

Thx anyway

Like I said, I'm new to this, so all I wanted was some help..

We're happy to help. You need to be more specific about what you need help with. Sending data to the PC for Processing to read? Making Processing read serial data? Making Processing create a file? Making Processing write data to the file? Getting the data formatted correctly to write to the file?

The two links I provided provide all the info necessary to write a Processing sketch that will save incoming Serial data to a file.

If you don't understand the info provided by those links, then what you need to do is learn the languages Processing and Arduino use to create sketches (and they are extremely similar, so it's really pretty much learning a single language).

I want my data (collected through the analogRead() command) to be written to a .CSV file using Processing. I have an accelerometer with 3 analog inputs

This is what I have at the moment (but not working) Arduino:

void setup()
{
  Serial.begin(9600); 
}

void loop() 
{
  Serial.print(map(analogRead(0), 0, 1023, 0, 255), BYTE);
  Serial.print(map(analogRead(1), 0, 1023, 0, 255), BYTE);
  Serial.print(map(analogRead(2), 0, 1023, 0, 255), BYTE);
  
  delay(50);
}

Processing:

import processing.serial.*;
PrintWriter output;
Serial myPort;   
float[] serialInArray = new float[3];
int inByte = -1;  
float meas1, meas2, meas3 = 0;
int serialCount = 0; 

void setup() {
  println(Serial.list());
  output = createWriter("accelerometer.csv"); 
  myPort = new Serial(this, Serial.list()[1], 9600);
}

void draw () {              
  while (myPort.available() > 0) {
    int serialByte = myPort.read();
    output.println(map(meas1, 0, 255, 0, 1023) + "," + map(meas2, 0, 255, 0, 1023) + "," + map(meas3, 0, 255, 0, 1023));
  }
}
void keyPressed() {
  output.flush(); 
  output.close();
  exit(); 
}

void processByte( int inByte) {  
  serialInArray[serialCount] = inByte;
  serialCount++;
  if (serialCount > 2 ) {
    meas1 = serialInArray[0];
    meas2 = serialInArray[1];
    meas3 = serialInArray[2];
    serialCount = 0;
  }
}

I only found examples how to write one byte, but here I have 3 bytes. The .csv file only contains zeros.

thx

I don't know anything about Processing, but unless processByte is a reserved name, it's never called, so meas1, meas2 and meas3 will keep their initial value of 0, as you see in your csv file.

Also, try testing your program using the serial monitor so you can see what data the accelerometer is actually producing

Oops, forgot! :) Now, I have values in my .csv file, but they dont match with the ones I get in my serial monitor

import processing.serial.*;

PrintWriter output;
Serial myPort;   
float[] serialInArray = new float[6];
int inByte = -1;  
float meas1, meas2, meas3 = 0;
float kalx, kaly, kalz = 0;
int serialCount = 0; 

void setup() {
  println(Serial.list());
  output = createWriter("accelerometer.csv"); 
  myPort = new Serial(this, Serial.list()[1], 9600);
}

void draw () {              
  while (myPort.available() > 0) {
    int serialByte = myPort.read();
    processByte(serialByte);
    output.println(map(meas1, 0, 255, 0, 1023) + "," + map(meas2, 0, 255, 0, 1023) + "," + map(meas3, 0, 255, 0, 1023));
  }
}
void keyPressed() {
  output.flush(); 
  output.close();
  exit(); 
}

void processByte( int inByte) {  
  serialInArray[serialCount] = inByte;
  serialCount++;
  if (serialCount > 2 ) {
    meas1 = serialInArray[0];
    meas2 = serialInArray[1];
    meas3 = serialInArray[2];
    serialCount = 0;
  }
}

Well one thing I suspect is that the output.println...... line is being called every time a byte is read, rather than every three as you're intending, so only every third line is likely to be correct. But that's just a guess: what do you see on the serial monitor, what do you see in the file and how does it differ from what you expected?

If you output the data from the Arduino in the format you want to save it in, then the Processing sketch can be trivial.

So for .csv, you want: val, val, val
or:

Serial.print(val1);
Serial.print(", ");
Serial.print(val2);
Serial.print(", ");
Serial.println(val3); //println() automatically transmits a crlf

On the Processing side, you then just have to read in each byte, and save that byte directly to your file. Zero additional code required.

wildbill:
Well one thing I suspect is that the output.println… line is being called every time a byte is read, rather than every three as you’re intending, so only every third line is likely to be correct. But that’s just a guess: what do you see on the serial monitor, what do you see in the file and how does it differ from what you expected?

I dont see how I have to change that println command

Here is my serial monitor output:

267 323 524
273 313 516
280 314 514
286 338 517
281 316 520
285 328 521
278 319 515
275 326 516
265 314 512
277 315 522
280 334 511
274 320 516
276 331 519
280 324 515
276 329 515
285 323 513
290 327 521
271 334 523
269 316 517
269 320 514
272 327 513
296 330 515
262 318 515
274 320 510
282 325 510

Here is the CSV output:

268.78824,332.97647,505.48236
268.78824,332.97647,505.48236
268.78824,332.97647,505.48236
280.82355,320.9412,505.48236
280.82355,320.9412,505.48236
280.82355,320.9412,505.48236
264.7765,328.96472,509.4941
264.7765,328.96472,509.4941
264.7765,328.96472,509.4941
284.8353,324.95294,505.48236
284.8353,324.95294,505.48236
284.8353,324.95294,505.48236
292.85883,312.91766,505.48236
292.85883,312.91766,505.48236
292.85883,312.91766,505.48236
272.80002,316.9294,501.47058
272.80002,316.9294,501.47058
272.80002,316.9294,501.47058
280.82355,316.9294,509.4941
280.82355,316.9294,509.4941
280.82355,316.9294,509.4941
272.80002,316.9294,509.4941
272.80002,316.9294,509.4941
272.80002,316.9294,509.4941

The values are good, but it seems that the values from the monitor fluctuate much harder…

You are sending integers and then collecting floats.

Is there a reason?

I think you are seeing "conversion error"...

No reason for doing that, thought it would matter.

But the value is ok, only problem is that the values in the file don't fluctuating the same as the normal analogRead() outputs

Here is a suggestion...

Put a loop in your Arduino program so you only send say 10 values. --- sorry , LINES!

Also include a "packet number/line number field"... say the first value in the record-- just count from 1 upwards...

If you send 10 records -- do you receive the same 10?

If you send lots of records -- do you start "dropping" packets? You can tell by the line number....

If you name the file as a CSV extension, you can clearly read it into Open Office or Excel. I am sure you have one of those -- or could get it.

If you are "dropping Packets" you could speed up or slow down the BAUD rate to see if it makes a difference....

Also you could implement a Protocol Like XON/XOFF -- a lot harder -- unless there is a library...

Or you could pay attention to Hardware Handshake on the PC -- through a standard RS232C port ... Nt that tough -- but you need the interface...

I do this through UDP and Delphi from Arduino -- it has its own set of problems and it does drop packets.

If you turn off the Serial Logging to the Consoles -- then I think the Packet Dropping could disappear if that is the issue.

Regardless -- I think numbering packets is a good idea -- and monitoring it -- so you can judge if the program is behaving and assess if the dropout rate is acceptable.

I think it is good practice to guard against dropout if you are transferring data. Opinions may vary...

Thx for the help and suggestions!

I'll try what you told me.

Forgot this... http://arduiniana.org/libraries/newsoftserial/

It has handshaking if that becomes an issue.