Adding a new column when starting new measurement

Hi you all,
I am working on a project where I have a two way switch, two sensors and a SD card reader. When I flip the switch one way the data of one of sensor 1 is saved to a .csv file on the SD card. When I flip the switch the other way the data of sensor 2 is saved on a different .csv file on the SD card. So far this works but I would like to add one thing:

When I start a measurement series of sensor 1 and then I stop for a while and later take a second series of measurements of sensor 1 then I would like to start on a second column in the .csv file.

I tried to store the data in rows but that didn't work since then the amount of measurements is limited. That is why I store to data in columns.

This is my code so far.
Thank you for your help.

   SD card attached to SPI bus as follows:
 ** MOSI - pin 11
 ** MISO - pin 12
 ** CLK - pin 13
 ** CS - pin 4 

#include <SPI.h>
#include <SD.h>
#include <FastLED.h>


const int chipSelect = 4;

// Two way switch
int Switch = A0;

int NormalSensor = A1;
int TorsionSensor = A2;


// define the LED amount and the pin to which it is connected
#define LED_PIN     6
#define NUM_LEDS    1
CRGB leds[NUM_LEDS];

void setup() {
  // Open serial communications and wait for port to open:
  Serial.begin(9600);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }

 // Set up the pins
pinMode(Switch, INPUT);
pinMode(NormalSensor, INPUT);
pinMode(TorsionSensor, INPUT);

  // LED set up 
  FastLED.addLeds<WS2812, LED_PIN, GRB>(leds, NUM_LEDS);

 // Check if SD card is present 
  Serial.print("Initializing SD card...");
  if (!SD.begin(chipSelect)) {
    Serial.println("Card failed, or not present");

    // don't do anything more:
    while (1) {
    leds[0] = CRGB(255, 0, 0);
    FastLED.show();
    delay (100);
    leds[0] = CRGB(0, 0, 0);
    FastLED.show();
    delay(100);
    }
  }
  Serial.println("card initialized.");
}

void loop() {

  // Check state of the switch and then set LED color and write sensor data to the SD card file.
  while (analogRead(Switch) < 100) {
    File dataFile = SD.open("Normal.csv", FILE_WRITE);
    leds[0] = CRGB(125, 125, 125);
    FastLED.show();
    Serial.println(analogRead(NormalSensor));
    if (dataFile) { 
     dataFile.println(analogRead(NormalSensor));
     dataFile.close();
    }
  
  }
   // Check state of the switch and then set LED color and write sensor data to the SD card file.
  while (analogRead(Switch) < 900 && analogRead(Switch) > 100) {
    File dataFile = SD.open("Torsion.csv", FILE_WRITE);
    leds[0] = CRGB(0, 0, 255);
    FastLED.show();
    Serial.println(analogRead(TorsionSensor));
    if (dataFile) {
     dataFile.println(analogRead(TorsionSensor));
     dataFile.close();
  }
  }
   // If the switch is in the middle position don't do anything just set the color of the status LED different. 
  while (analogRead(Switch) > 900) {
    leds[0] = CRGB(0, 255, 0);
    FastLED.show();
  }

}

Welcome to the forum

Before making changes can you please explain why you are using analogRead() when the input from the switch is a digital signal, ie either on or off ? The A* inputs can be used as normal GPIO (digital) pins with digitalRead().

Is there anything keeping the switch pin in a known state at all times, such as a pullup or pulldown resistor ?

Thank you Bob,

I used this layout for the switch:

That way I use just one pin on the board.

Are you that short of pins ?

If not then in my opinion you are just complicating things

No I am not, I will consider using the easier option with a simple pull-up resistor

Or using the even easier method using INPUT_PULLUP in pinMode() to turn on the built in pullup resistor for the pin

That aside, I don't understand what data layout in the files that you are aiming for

The obvious layout would be one row for each series with a new row starting when switching inputs.

You mention stopping "for a while". Does that mean that you don't want any measurements taken for a while even though you have not switched inputs ? If so, what will signal that you have stopped ?

you can open file with Sensor1(S1) data and add line for line data from S2 while storing in second file.

currently I get a .csv file in the format of example:

504
502
503

and I would like to have the second measurement be added after the first column like this:

504,362
502,364
503,363

but currently when I start a second measurement then it adds the values at the bottom of the column like this:
504
502
503
362
364
363

I end an measurement by putting the switch in the middle position in the neutral

You can't add a second or subsequent column to the data on the file without a great deal of trouble

To me, adding a second column makes no sense. Why not add a new row of data instead ?

Using your example readings the first line would look like this

504,502,503

Then, after the second reading it would look like this
504,502,503
362,364,363

You will need to read the data from the previous file, add the new measurement, and write to a new file.

Have you considered what to do if you do not have the same number of measurements in each measurement period?

@UKHeliBob I did that at first but then I have a limited amount of data that I can store per row. Per row I would need around 5000 measurements and this didn't work.

@david_2018 I haven't thought of this. In my opinion it doesn't matter too much since I will analyse the data afterwards in python. I will take the data per column and the amount of data points is not too important then I would think

with middle position is only 3-way switch. and this means you must connect second pin of switch to arduino

are you using resistors like in post#3 ?

You just need to be able to handle varying numbers of measurements in each measurement period. For instance, if the 2nd measurement period has fewer measurements than the first, then you still need to append an extra comma to the remaining measurements from the 1st measurement period, otherwise a 3rd measurement period will not align properly. If any measurement period has more measurements than all the previous measurement periods, then you need to generate new lines with empty columns for the previous measurement periods to keep the data aligned. This also means your analysis software needs to be able to handle the empty data points.

Don't try to read the entire line in at once, that is likely where you are getting a problem.

A separate file for each measurement period might be easier, just implement a method of sequentially numbering the files, although the one line per measurement period should be achievable. The python code can do the work of combining all the files into a single csv if you really need that.

That is a really good suggestion! I think that is the easiest and most structured way

How frequently are these readings being received ?

The maximum rate of the Arduino Nano.

I am now trying to solve it like this. So I make a counter that makes a new file every new measurement.

  if (analogRead(Switch) < 100) {
    String normalfile = "Normal" + String(counterNormal) + ".csv";
    counterNormal++;
    Serial.println(normalfile);
  }

  // Check state of the switch and then set LED color and write sensor data to the SD card file.
  while (analogRead(Switch) < 100) {
    File dataFile = SD.open(normalfile, FILE_WRITE);
    leds[0] = CRGB(125, 125, 125);
    FastLED.show();
    Serial.println(analogRead(NormalSensor));
    if (dataFile) { 
     dataFile.println(analogRead(NormalSensor));
     dataFile.close();
    }
  
  }

A new file for each new series of readings sounds like madness to me. Exactly what problem did you have with 5000 readings on a single row ?

Well I want one array with around 5000 values every time I make a measurement. I expect to be doing ±20 measurements per session so in my eyes it would be okay to have 20 separate files.

Which Arduino board are you using ?