SD card read and write simultaniously? (affordable data logger)

Hi guys, some backround about the project: I am building a tyre pyrometer to use for motorsports, and said pyrometer will have, among other things, a data log to SD card function.

Hardware is an ESP32 in conjunction with a 2.8" TFT with SD card slot.
GUI is built using GUISlice.

I currently have the basic GUI up and running, and have written a library for the particular thermistor I am using.

So far things are working nicely.

This would be used for Autocross which is for each race 1 warm up round followed by 2 timed rounds. During a race day we normally have 2 races, which is total 6 rounds. Therefore I would like to have up to 6 logs for up to 3 different cars.

Therefore I would like to have the data logger set up in the following way, you select a car profile (1-3) followed by the log number (1-6)

This would either create or override a file (for example car 1 log 5 creates/overrides a file named 1_5.txt)

You then come to the following screen:

Each of the tyres is a button, and when pressed I should be able to take 3 seperate readings for this tyre (inside, middle, outside) that will simultaniously be displayed in the text field alongside the tyre, and written to the log. The values should remain on screen as long as the page is open. It would be even nicer If I could recall the data (e.g. After saving a log, If I open the log again later the values saved in said log will be displayed on screen, and could then be be overwritten).

To make things simpler the order in which the values are recorderd could be fixed (from front right clockwise for example), then each of the 12 lines of data would always correspond to the same tire.

To achieve all this I require some guidance, how exactly should I go about doing this? Any help and advice would be much appreciated. Its a rather large undertaking for me, but I plan to make this opensource once its finished, just like my arduino based shiftlight. Alot of motorsports equipment is very expensive (a pyromter with datalog for example costs over $300) and im trying to make these kind of things more affordable to the grassroots kind of guys.

Aren't you forgetting about how you built the tyre pyrometer? How does it work?

Paul

I am not sure what your questions are. Could you be a bit more specific where you see issues and like some advice?

Its an NTC thermistor that will be mounted inside a small brass needle mounted in a 3d printed handle. The needle can be inserted into the tire a few mm and will basically read the carcass temperature.

The whole project will include a digital pressure gauge (just a simple honeywell I2C pressure sensor, 0-60 psi) an option to use the pyrometer without the log function, an 8x8 pixel thermal camera (also over I2C) to check surface temps of the tyres/have a quck look, and ofcourse, the data logger.

The pressure sensor is a no-brainer, there is already a library avaliable the gives a pressure output, this can simply be linked to a guage on one page of the GUI, the thermal cam should also be achievable, there is also a library avaliable from adafruit. The temperature probe is currently working, I took a library designed for a different type of thermistor, and with the help of some other forum members rewrote it to make it work a bit better and added the lookup table for my specific thermistor.

The hard part will be the data logger.

The thermistor library gives a floting value between -40 and 150 that represents degrees celcius.

I need help with the following:

I need to write some code that allows the user to select a log (.txt or .csv) file.
A floating number (temperature) to be written to the file 12 times, each time one of the 12 values is written to the text file its also written to the text box next to each tire (starting top right and moving clockwise. (see pic of the GUI page in the OP).

After all twelve values are taken, the text file is saved.

As long as the page is open the values remain shown.

If the page is closed, then the page is reopened with the existing log file, the values are written to the text boxes again.

If the the user chooses to take measurements again when the log file is open, the values will be overwritten.

Is this possible, or is this asking too much of the hardware?

Should I post the code from GUIslice, so you can see how the pages are called and so on?

Anyone have any ideas about how to achieve something like this?

I suggest working on a separate program to create the SD card files you need. Use dummy values. Remember, the data for the files do not exist until you either flush the file or close the file. Both will update the file information table on the SD card. Use your PC to look at the data and confirm it is what you want to create. Then you can add the logic to your final program.

Paul

Some questions,

1.Can I temporarily save the values somehow?
For example, I measure 12 times, each value is temporarily stored, when all 12 values are there they can be written to SD?

  1. is it possible to read a text file from SD line for line? Example, if I open the file I can decide which line is read? Something along the lines of: serial.Print (123.txt (line 3)). Is something like that possible?

mickymik:
1.Can I temporarily save the values somehow?
For example, I measure 12 times, each value is temporarily stored, when all 12 values are there they can be written to SD?

Just create an array of values to store the values in RAM. Then write some function that reads and writes the values to/from SD card.

mickymik:
2. is it possible to read a text file from SD line for line? Example, if I open the file I can decide which line is read? Something along the lines of: serial.Print (123.txt (line 3)). Is something like that possible?

Have a look at the SD library. There are read and write functions. They read bytes or buffer with length.

You could simply store the values line by line, as csv or create a simple protocol in the file.

The library usage is slightly more complicated. You have to open a file and then you can read and write. The library comes with multiple examples.

one option for temporary data is to create a database in SPIFFS on the ESP32.
some calculations of the volume of data collected would be in order.

I am missing how you collect the data. is it in the pits or active on the car ? The intrusive temperature sensing does not seem to lend itself to dynamic sensing like a pyrometer would.

Since 3 different cars are involved it would seem like something external to the car.

one other comment is the display. some heading. car-2, race-2 or some such. you would need to have this for the stored files no matter what. As well as select by a menu before taking readings.

The pyrometer is basically just a thermometer you stick into the tyre tread in the pits to measure the temperature inside the rubber. The value to be stored would be maybe like 5 characters (82.75 for example). And this would be stored 12 times (3 times for each tyre). So basically i pull into the pits, and do a lap of the car and take each reading one after another.

Something similar to this.

Gui would work like this:
Homepage -> datalog -> car no. (1-3) -> log no. (1-6) -> log screen (example shown in OP)

The kind of procedure for it to work (in my head at least) would be something like this:

Temperature readings are somehow fixed to 5 digits/characters (eg. 4.031 or 50.34 or 123.1). in order to make recall easier (more info later).

Select the log (car 1 log 3 for example)

you stick the probe in the tyre, temp is displayed in one of 12 text boxes. Press OK, value is stored to ram, remains displayed in text box.

Next temperature reading is taken, value is displayed in next text box, ok is pressed, value is stored to ram, remains displayed in text box.

so on and so on untill all 12 measurements are taken, and all 12 values are displayed on screen/saved in ram.

press save, text is written to SD card (.txt file with each value on a new line, each line has the same number of digits/bytes).

If car 1 log 3 is later reopened, the value on each line of the .txt file will be displayed in each text box (once again, see example pic in OP)

Once open the 12 values can also be re-written, if desired.

notes: I made a point of saying I would like to limit the value to 5 characters, as it should in theory be easier to recall the values using something like myFile.seek(lineNumer * LineLength);

So the question is, is it possible, and could someone possibly (pretty please with cherries on top) help me implement it?

Hop over to Randomnerd Tutorial and have a look at what they have around SD Cards.
Open your Arduino IDE examples and see what is on there for SD cards.
Try the examples they will give you a base to build on. Everything you want to do is possible and more, for instance you could display the data via bluetooth or wifi.
You can easily have multiple files on the SD card, one for each car.
Using the bluetooth or Wifi options you can display multiple values so compare three runs for each car, build charts and graphs etc.
A nice add on feature might be to also record track temperature and/or air temp and humidity etc.

The key is to break it down into small bites and once you get each bite working add the next....

at the moment the part im a bit stuck on is how to write the values to the screen and "pause" them if you understand what I mean.
I need to somehow store the values on e after another to ram or some such, that they can then be written to sd.

so what code have you tried? post it here so we can offer help.
if you create variables called something like Car1_LF1, Car1_Lr1 etc and write the data from I2C into these variables it will stay there until the ESP32 is powered down or the data is over ridden.

Its a bit difficult to explain exactly whats going on, I am using the GUISlice library, so Im not even sure where to start implementing this.

Here is the .ino

//<App !Start!>
// FILE: [tiretool.ino]
// Created by GUIslice Builder version: [0.16.0]
//
// GUIslice Builder Generated File
//
// For the latest guides, updates and support view:
// https://github.com/ImpulseAdventure/GUIslice
//
//<App !End!>

// ------------------------------------------------
// Headers to include
// ------------------------------------------------
#include "Arduino.h"
#include "tiretool_GSLC.h"
#include <TS-NTC-103_simple.h>

// ------------------------------------------------
// Program Globals
// ------------------------------------------------

// Save some element references for direct access
//<Save_References !Start!>
gslc_tsElemRef* m_pElemRadial1    = NULL;
gslc_tsElemRef* m_pElemXRingGauge1= NULL;
//<Save_References !End!>

// Define debug message function
static int16_t DebugOut(char ch) { if (ch == (char)'\n') Serial.println(""); else Serial.write(ch); return 0; }

// ------------------------------------------------
// Callback Methods
// ------------------------------------------------
// Common Button callback
bool CbBtnCommon(void* pvGui,void *pvElemRef,gslc_teTouch eTouch,int16_t nX,int16_t nY)
{
  // Typecast the parameters to match the GUI and element types
  gslc_tsGui*     pGui     = (gslc_tsGui*)(pvGui);
  gslc_tsElemRef* pElemRef = (gslc_tsElemRef*)(pvElemRef);
  gslc_tsElem*    pElem    = gslc_GetElemFromRef(pGui,pElemRef);

  if ( eTouch == GSLC_TOUCH_UP_IN ) {
    // From the element's ID we can determine which button was pressed.
    switch (pElem->nId) {
//<Button Enums !Start!>
      case E_ELEM_IMAGEBTN3:
        gslc_SetPageCur(&m_gui, E_PG2);
        break;
      case E_ELEM_IMAGEBTN4:
        gslc_SetPageCur(&m_gui, E_PG4);
        break;
      case E_ELEM_IMAGEBTN1:
        gslc_SetPageCur(&m_gui, E_PG3);
        break;
      case E_ELEM_IMAGEBTN2:
        gslc_SetPageCur(&m_gui, E_PG5);
        break;
      case E_ELEM_IMAGEBTN6:
        gslc_SetPageCur(&m_gui, E_PG_MAIN);
        break;
      case E_ELEM_IMAGEBTN7:
        gslc_SetPageCur(&m_gui, E_PG_MAIN);
        break;
      case E_ELEM_IMAGEBTN8:
        gslc_SetPageCur(&m_gui, E_PG_MAIN);
        break;
      case E_ELEM_BTN1:
        break;
      case E_ELEM_BTN2:
        break;
      case E_ELEM_BTN5:
        break;
      case E_ELEM_IMAGEBTN9:
        gslc_SetPageCur(&m_gui, E_PG_MAIN);
        break;
      case E_ELEM_BTN7:
        break;
//<Button Enums !End!>
      default:
        break;
    }
  }
  return true;
}
//<Checkbox Callback !Start!>
//<Checkbox Callback !End!>
//<Keypad Callback !Start!>
//<Keypad Callback !End!>
//<Spinner Callback !Start!>
//<Spinner Callback !End!>
//<Listbox Callback !Start!>
//<Listbox Callback !End!>
//<Draw Callback !Start!>
//<Draw Callback !End!>
//<Slider Callback !Start!>
//<Slider Callback !End!>
//<Tick Callback !Start!>
//<Tick Callback !End!>

void setup()
{
  // ------------------------------------------------
  // Initialize
  // ------------------------------------------------
  Serial.begin(9600);
  // Wait for USB Serial 
  //delay(1000);  // NOTE: Some devices require a delay after Serial.begin() before serial port can be used

  gslc_InitDebug(&DebugOut);

  // ------------------------------------------------
  // Create graphic elements
  // ------------------------------------------------
  InitGUIslice_gen();

}









// -----------------------------------
// Main event loop
// -----------------------------------
void loop()
{

  // ------------------------------------------------
  // Update GUI Elements
  // ------------------------------------------------
  
  //TODO - Add update code for any text, gauges, or sliders
 TS_NTC_103 sensor (0.11,10);
int Temp (sensor.getTemp(analogRead(36)));
  gslc_ElemXRingGaugeSetVal(&m_gui,m_pElemXRingGauge1, Temp);
 

  Serial.println (1234);

  // ------------------------------------------------
  // Periodically call GUIslice update function
  // ------------------------------------------------
  gslc_Update(&m_gui);
    
}

attached is the .h file where the page and element info is stored (does not fit in the post limit, too many characters)
You can see in the loop that I am writing the value from the thermistor to the ring gauge.
That is the same value I will be using for the log( TS_NTC_103 sensor (0.11,10); int Temp (sensor.getTemp(analogRead(36))); )
I would like to write this value to E_ELEM_TEXT20 to E_ELEM_TEXT37 one after another.
Placing different car profiles, logs aside, and SD card aside, at the absolute base level first step, I would like to achieve the following:

  1. When PAGE: E_PG5 is open, sensor.getTemp(analogRead(36)) is written in E_ELEM_TEXT20
  2. A button is pressed, the value shown in E_ELEM_TEXT20 is "frozen" and is stored in ram or some such
  3. sensor.getTemp(analogRead(36)) then automatically shown in next text element on the page (E_ELEM_TEXT21)
  4. the button is pressed again, the value shown in E_ELEM_TEXT21 is frozen, stored, and value is then displayed in the next text element.
  5. proccess repeats untill all 12 text elements have a stored value displayed in them.

Im kind of stuck. not sure where to start.

tiretool_GSLC.h (24 KB)

so write a simple piece of code to read one value into a variable and display it in serial. notice how the variable doesn't change until you take the next reading.

Once you get that working create a second variable and "copy" the first variable to the second (hint use something like like Vari 2 = Vari 1) so vari 1 is where the I2C puts the temp reading, vari 2 is where you store it until it is written to the SD card

I might play around with building basic code to do this part of the operation.

i have renamed the 12 text field element references to DATA_TEXT_1 to DATA_TEXT_12
And i have created an array containing the reference names.

I have added a button DATA_NEXT

here is the code:

 bool CbBtnCommon(void* pvGui,void *pvElemRef,gslc_teTouch eTouch,int16_t nX,int16_t nY)
{
  int DATA = 12; 

gslc_tsElemRef* datalogger[DATA] = {
DATA_TEXT_1,
DATA_TEXT_2,
DATA_TEXT_3,
DATA_TEXT_4,
DATA_TEXT_5,
DATA_TEXT_7,
DATA_TEXT_8,
DATA_TEXT_9,
DATA_TEXT_10,
DATA_TEXT_11,
DATA_TEXT_12,
};
// Typecast the parameters to match the GUI and element types
  gslc_tsGui*     pGui     = (gslc_tsGui*)(pvGui);
  gslc_tsElemRef* pElemRef = (gslc_tsElemRef*)(pvElemRef);
  gslc_tsElem*    pElem    = gslc_GetElemFromRef(pGui,pElemRef);
  char   logTemp[MAX_STR];
  TS_NTC_103 sensor (0.11,10);
  int Temp (sensor.getTemp(analogRead(36)));
  if ( eTouch == GSLC_TOUCH_UP_IN ) {
     
    // From the element's ID we can determine which button was pressed.
    switch (pElem->nId) {
//<Button Enums !Start!>
case NEXT_DATA:
       for ( int i = 0; i < DATA; i+=1){
        snprintf(logTemp, MAX_STR, "%f", Temp);
        gslc_ElemSetTxtStr(&m_gui,(datalogger[i]) ,logTemp);
               break;
       }
     }
  }
  return true;
}

Unfortunately only the first text field takes on the value and the value is wrong.

Can anyone help out, shed some light on the situation? Maybe give a few pointers?

mickymik:
Can anyone help out, shed some light on the situation? Maybe give a few pointers?

Did you try simplifying everything down to moving one value between 2 variables and serial printing?
I really think you would benefit from starting with the basics and once you get that working its easier to expand the system.

I was trying to get something simple happening inside of the GUIslice wire frame.

my plan was to start of writing the temperature to the text field, then to write the temperature to more text fields by changing the element name in the following line.

 gslc_ElemSetTxtStr(&m_gui,**element name*** ,logTemp);

I was however unable to take the next name in the array each time the button was pressed.

Is there any chance you could write a small piece of something to get me kicked off? Im having a kind of mental block, it feels like its more complicated than it needs to be.

It seems as though the text fields also act as temporary storage for the integer, If i could just make the next button take the next reference name in the array each time it is pressed i think it could work.

How could I do that, is it possible inside CASE: NEXT_DATA and BREAK; ?