Go Down

Topic: How to save data from Serial Monitor to a csv file (Read 1 time) previous topic - next topic

nadimtahmass

Hi all,

I wanted to know how I could save data from a serial monitor to a csv file without an SD card. I wanted it to collect a certain amount of data entries and then save the file. If possible, for it to restart on a new text file after one complete iteration. I am collecting temperature data and I want to store is straight to a csv file from the serial monitor. Thanks.

Code: [Select]

#include <Time.h>
#include <TimeLib.h>

//TMP36 Pin Variables
int sensorPin = 0; //the analog pin the TMP36's Vout (sense) pin is connected to
                        //the resolution is 10 mV / degree centigrade with a
                        //500 mV offset to allow for negative temperatures
 
/*
 * setup() - this function runs once when you turn your Arduino on
 * We initialize the serial connection with the computer
 */
void setup()
{
  Serial.begin(9600);  //Start the serial connection with the computer
                       //to view the result open the serial monitor
  setTime(1554905950);
}
 
void loop()                     // run over and over again
{
 //getting the voltage reading from the temperature sensor
 int reading = analogRead(sensorPin); 
 
 // converting that reading to voltage, for 3.3v arduino use 3.3
 float voltage = reading * 5.0;
 voltage /= 1024.0;
 
 // print out the voltage
 Serial.print(voltage); Serial.println(" volts");
 
 // now print out the temperature
 float temperatureC = (voltage - 0.5) * 100 ;  //converting from 10 mv per degree wit 500 mV offset
                                               //to degrees ((voltage - 500mV) times 100)
 Serial.print(temperatureC); Serial.println(" degrees C");
 
 // now convert to Fahrenheit
 float temperatureF = (temperatureC * 9.0 / 5.0) + 32.0;
 Serial.print(temperatureF); Serial.println(" degrees F");
 
 Serial.print("Timestamp: ");
 Serial.print(month());
 Serial.print("/");
 Serial.print(day());
 Serial.print("/");
 Serial.print(year());
 Serial.print(" ");
 Serial.print(hour());
 Serial.print(":");
 Serial.print(minute());
 Serial.print(":");
 Serial.println(second());
 
 delay(1000);                                     //waiting a second
}

Nick_Pyner

You might use a proper terminal programme instead, like RealTerm and several others, all of which are free. These enable you to record direct to CSV and you can add timestamp as well, using the PC clock.  I have heard that the serial monitor in the latest IDE allows recording. I don't know if it can timestamp.

I don't know about multiple files, and I think that might be better done with on-board SD. Having said that, I don't think there should be a need anyway. You should be able to provide marker points in a single continuous data stream, and everything gets sorted out in Excel later.

sterretje

If that SD card is connected to the Arduino, you can write the same data (that you send to the serial monitor) to the SD card in your Arduino application.

If that SD card is on the PC, see Nick_Pyner's post above.

If you understand an example, use it.
If you don't understand an example, don't use it.

Electronics engineer by trade, software engineer by profession. Trying to get back into electronics after 15 years absence.

Duck51

Hi
Had a similar issue a little while ago. Had an aging laptop (Windows 7, Office 2007) and no internet connection so had to do it with what was on the PC.
 Sent the data, in my case about 6 columns of numbers and about 30 lines with serial.print, with (",") between each number in the line to put the commas in the right place. Don't forget to use serial.println for the last number on the line to ensure the next group starts a new line. Time stamp can be added by ticking the box in the serial window.
When you have got what you need, copy and paste the whole block of data into notepad and save it as file type CSV with a .csv extension. This can then be inserted into Excel if required, which is what I needed to do.
A bit clunky and the experts out there will no doubt laugh at its crudeness and lack of total automation, and it may not serve all purposes. But it got the job done when I had no capability to use anything that was not already on the computer and as a newbie was struggling a bit with everything else there is to learn about Arduinos and coding.

Regards
Rob

Nick_Pyner

no internet connection so had to do it with what was on the PC.
Absence of internet excuses you, but you would be well-advised to get RealTerm at leisure, and also check out PLXv2!
http://forum.arduino.cc/index.php?topic=437398.0

Duck51

Nick
Is your Realterm suggestion some sort of a joke? I have never tried to use something as difficult or convoluted in my life. The help file might as well be written in Greek. Finally got it to find my coms port and managed to display my CSV data. Had no luck getting a time stamp to work. Could not get it to save a file with a CSV format or extension - seems to force a .log or .trm extension. Didn't really care, as long as Excel could use it. However no matter how hard I tried, could not save the file in any sort of format. It looked like it was saved, but could never find it.
If stuff like this is essential for getting Arduinos to work properly, I think I am totally out of my depth, so might quit while I am still ahead.
Is the other suggestion - the PLXv2 any more user friendly?

Regards
Rob

Nick_Pyner

No. I and about a gazillion others use it OK, and it was all fairly intuitive. The capture tab should reveal all, including the timestamp. You can name the file in the File window, or browse an existing one. This really cannot be so difficult, and you are just not looking hard enough.  If you don't like it, there is a swag of alternatives, all free. Try PuTTY or Terraterm. I found a clear preference for RealTerm, but I can't remember what that was about.

IF your ultimate destination is Excel AND the PC with Excel has direct connect to Arduino, PLX is the obvious way to go. It is essentially a macro for Excel that makes it perform like a terminal. This enables you to have live graphs fed by Arduino.  Time stamping is available and you can do maths  there instead of Arduino.

GijKieken

NetDevil did a great job on plx and even an example file is included.

Duck51

hi - especially to Nick and GijKieken
After your assurance that Realterm is widely and successfully used, had another look. You were right  - I hadn't been looking hard enough and was trying to save the file using the diagnostic file window to the right rather than left hand file window. However, still could not get it to save my file. Used several different extensions - csv, txt and various filenames and directory locations - always looked like it was doing the job but no file ever saved.
Any idea what I could be doing wrong?
         
Decided to give up on that and have a go at PLXv2, as suggested by GijKieken. Looked like a good solution as the only thing I need to do is log the data into Excel. Picked up V2.10. Installed OK, started up OK, connected OK, and was able to see my data arrive in the diagnostic window. Was able to delete the sample data from the spreadsheet. However nothing I could do would load my new data - the spreadsheet remained blank.
What have I missed?
Is PLXv2.10 incompatible with Windows 7 / Excel 2007?

Regards
Rob


Nick_Pyner

- always looked like it was doing the job but no file ever saved.
Any idea what I could be doing wrong?
         
............- the spreadsheet remained blank.
What have I missed?
It is pretty hard to tell from here but, with a failure on both methods, it is starting to look like you aren't sending anything. You might need to be a bit more forthcoming about your code, not to mention what you see on the screen.

If PLX 2.10 wasn't compatible with W7 Office 2007, everybody would know about it, and it would be quickly withdrawn.

Duck51

Hi
The code I using is not on this PC - is at my remote site so cannot send right now. However data did display OK on the serial monitor, could be directly pasted to note pad, saved as a .csv file and then inserted directly into Excel, which was my original (sucessful!) method.

Using Realterm the correct data appeared in the window, just could not save it to a file.

Using PLX, correct data showed in the diagnostic window, but did not flow through to the spreadsheet. There was obviously some communication with the spreadsheet, as initial start up loaded the example data, and the delete function deleted it, except for column titles, as it is expected to.

Will post code up tomorrow when I access the device again. There must be some subtle thing I am missing.

At the end of the day, it may be just an academic exercise - The device I am working with is logging data in support of a research project, and I may need to run the code on my sponsor's computer - for which I don't have admin rights. As I probably wont be able to install additional software, I may be stuck with my original clunky method.

I would still like to get Realterm or PLX working, however, just for my own satisfaction.

Regards
Rob


Nick_Pyner

I'm afraid I'm at a loss here. You already know the right area to work in in RealTerm, and I don't actually use PLX v2. I understand you do get confirmation of data received in PLX. I guess this is something you will solve by blunder - probably quite soon.

Duck51

Nick (and others),

PLX DAQ problem solved. I was guilty of a number of "newbie" errors, probably the worst being my error would have been instantly apparent from the code I didn't post. My only excuse here is that the computer I run this on is in my slightly remote workshop, with no internet. However I probably learnt more by having to work it out on my own - with help from any posts I could find referencing PLX.

I will list in reasonable detail what I found - may be of benefit to other newbies.

The USB connection from PC to Arduino has a number of operational modes
  - a) USB connection to Arduino to upload code
  - b) Serial comms from the Arduino to the serial monitor
  - c) 2 way comms between Arduino and PLX environment
and probably a lot more I dont know about yet.

    When the PLX device is connected, the serial monitor cannot be used, and code can't be uploaded, as the port can't be shared.

    Code that simply prints stuff on the serial monitor actually supplies commands to the PLX environment to make things happen. This is what I missed - The code "Serial.println("LABEL,Time,Volts,etc---"); simply prints these words to the serial monitor, but to PLX, "LABEL" is a VBA command that prints the column labels on the spreadsheet.

I didn't know that a day ago, and didn't want to print the word "LABEL" on the screen or on the spreadsheet, so I simply left the word out. I also left out "Serial.print("DATA,TIME,"); as I didn't want to print those words either. Once again didn't realise they were PLX commands. Pretty stupid, wasn't I?

The direct debug device on PLX is actually just a repeat of the serial monitor and prints anything that the serial monitor would print - variables print as you would expect and anything in quotes prints as is - including the VBA commands. Just because it is on the debug screen does not automatically put it on the spreadsheet (which is what I believed yesterday) - there needs to be a valid VBA command in the code.

Any how it is now working perfectly - a very nice utility which will save a lot of spreadsheet entry work with the project I am doing. Many thanks to Netdevil who I believe did an enormous amount of work on PLX, and to Nick for his patience and guidance.

I also hope that Nadimtahmass, who originated this thread gets something useful out of it.

For now I won't worry about the issue with Realterm, as PLX is probably more suitable in this case.

Regards
Rob

Nick_Pyner

Pretty stupid, wasn't I?
Yes, but we all learn and you explained the debacle in detail, which will be of value to other newbies. I thought it had to be something like that but there was no way that I could put a finger on it, and it was definitely time to go to bed.
I guess the summary is - don't let the serial monitor cloud the issue when the real game is PLX.

Go Up