Working with ESP32 SPIFFS

Greetings! Hope you guys are doing well.

I need to access alot of tabulated numericals from an excel file. So I came across of using SPIFFS on esp32 where I need to create the text file and upload it to esp32. data.zip (275.3 KB) the example of which is attached alongwith.

what I need to do is....I have attached a load cell and a NTC to my arduino. Arduino reads both of them and sends the values to EPS32 via serial. Now, the values from the load cell corresponds to the text file name saved and the values corresponding to the readout of NTC should be fetched from that particular text file. I can see lot of examples on how to open the SPIFFS file and read what I am not able to understand is how do I get to pick only the number that corresponds to the temperature readout.

I have also attached a .ino file which someone else wrote to do the task.esp32_disp_adc_nowifi.ino (4.0 KB)
any example to which I can refer to or any help is appreciated. Thanks in advance

You should use littleFS

look into the example-codes

best regards Stefan

To open a file, a text is required. You have to convert that number to text.
You already use sprintf(), so you can use that.

Here is a smal program that show how to get the filenames
https://www.esp8266.com/viewtopic.php?p=87718#p87718

If you want to know more about the available commands
copy the files "FS.h" and "FS.cpp" to a tempfolder and take a look inside what functions are defined.

Use a copy to be sure if you accidently change something inside the file and save the changes it doesn't matter because you used a copy

If you have questions about the details how to use these functions just post your code as far as you have it writen to try it.

Then a lot of answers will come because you will ask concrete questions.

I went through the link, it has three examples. But I couldn't find what I was exactly looking for. I want to pick only a certain number from the text file after it is opened. How do I do it?

can you please elaborate on this. I can see why that guy used sprintf instead of directly sending the values over UART. what really confuses me is he got those values in ASCI and later converted them in HEX and then used it as a file name to refer to the particular file name. And then he uses the temperature readout received over UART in ASCI, converts it to hex and looks for the corresponding value that intersects horizontally with the load values written vertically, when the match is found only that particular value is returned. How does this works? its really confusing for me. Should I share the excel file as well or the files I already shared are good enough to get a clea view of this?

Where I'll be able to find all these files? I have installed the ESP32 board and got my SPIFFS.

You seem to hope "if I find a code about a load-cell and using SPIFFS"
I can adapt it to my needs.

Yes you can if the code found does not fit 99% of your requierements understanding the code and adapting it is more work than writing your own.

ZIP-files are a potentially danger because they could hide malicious software. That's the reason why most users here won't open zip-files. You should post Txt-files as textfiles. If they are too big to attach make a small version that still shows the principle.

I have looked into your textfiles.
I have done quie a lot of programming and would call myself a somehow advanced coder. I did some basic tests with SPIFFS just out of curiosity but I'm no specialist about it.
My way to write such a code would be to start with basic programs
creating three textfiles each with a few numbers in it
and then starting with a program that retrieves the files names of the three textfiles

If this works adding lines of code that read in from one textfile line by line just echoing this line to the serial monitor

If this works adding lines of code comparing the content of one line to some other variable

best regards Stefan

Got it. But text file is not supported format for file upload, am I right? sorry if I said anything wrong, I'm still getting familiar with the newer version of forum

google is always worth a quick try:

googled with keywords "ESP32 uploading files into SPIFFS"
ESP32+uploading+files+into+SPIFFS

Jackpot! Full Hit!

I start with 4 or even 8 keywords. If the hits that google presents are rubbish I can still reduce the number of keywords or change them.
best regards Stefan

Yeah I followed the same website while getting my SPIFFS setup. What I actually meant was, the text format attachments are not allowed on forum?

Can't you drag the text file into the post?
Or
Copy it into the post as a text?
Not sure what you mean?

Tom... :grinning: :+1: :coffee: :australia:

1 Like

No the forum-software refuses *.txt.
@Moderators: if *.txt is not allowed I simply rename it to *.ino even if it is not a valid sketch. If you find this confusing/annyoing add *.txt-file-support to the forumsoftware

MyTest-Textfile.ino (16 Bytes)

Okay got it.

Another thing can you help me with understanding the .ino file that I have attached with my question? that way I can have a reference while I work up with my own code.

I do not need to understand the entire code just why ASCI to HEX conversion and how the interception of the values done from the text file, would do my work.

I did look into the sketch you have attached.

It is written in a style that is hard to understand.
Too short names to say explicitly what the variable or function is for.
Me personal I prefer selfexplaining names. Rather 20 letter-long variable-names that clearly say what they are good for than just a single or three letters.

So to understand this code would require a lot of detail work.

I will not do that. It would take me 5 hours or maybe even more.

If somebody else can do it in 10 minutes well do it
I will take off my hat and bow to this great skills

As far as I have understood your own requirements
You have a value that comes from a loadcell.
This load-cell-value leads to one of your load-cell-filenames.

Inside the load-cell-files there is a list of values which "correspond" to your NTC-values

In which way?

Could you describe in detail and in a "name-the details-style" an example of all the things that shall happen.

Or even better give an overview about your project.
I suspect using a digital temperature-sensor eliminates the need for all those value-tables. Maybe there are real improtant reasons to use that exact type of NTC but if not I guess to realise the functionality you want to achive can be easier done with a different temperatur-sensor.

best regards Stefan

So I was asked to redesign a project with minimum changes possible in it. So I needed to understand how the existing system works.

So the system goes like this... It has an Arduino UNO (actually ATmega8 chip) which I have to replace with the Arduino UNO now and a NODE MCU communicating with eachother over UART.

The Atmega8 chip is equipped with a loadcell along with HX711 module and a NTC for reading out temperature. The sensor will be dipped in petrol or diesel.

The Atmega8 reads the output of loadcell and output of NTC and sends it to the NODE MCU where all these text files are stored on SPIFFS. each file in the SPIFFS has name according to the weights and each text file has list of some numbers(density of fluid at a particular temperature).

Like a file named 654 is actually the output value of the load cell received over UART. File 654 contains densities of the same liquid written one below the another like....for example...

123
456
256
891
like wise.... each of these values are actually the densities of a fluid at different temperature. Now, after receiving the temperature readout over UART, I need to pick the density value from a particular file (which should be the file corresponding to the value output from loadcell) corresponding to the temperature readout by NTC over UART.

ummm...did I made it clear? :thinking: I think I made it more confusing :frowning:
Im attaching an excel file from where I need to create all these text files. (Sorry excel is not supported so I copied it here)

the values alongwith X axis on the top side corresponds to loadcell values and the values alongside the Y axis on left most side are the temperature values and the values in middle are the density of liquid

650 651 652 653 654 655
1 2 3 4 5 6 7
-10 - - - 628.9 630 631
-9.75 - - - 629.2 630.2 631.2
-9.5 - - - 629.4 630.4 631.5
-9.25 - - - 629.7 630.7 631.7
-9 - - - 629.9 630.9 632
-8.75 - - - 630.2 631.2 632.2
-8.5 - - - 630.4 631.4 632.4
-8.25 - - - 630.6 631.7 632.7
-8 - - - 630.9 631.9 632.9
-7.75 - - - 631.1 632.1 633.2
-7.5 - - - 631.4 632.4 633.4
-7.25 - - - 631.6 632.6 633.6
-7 - - - 631.8 632.9 633.9
-6.75 - - - 632.1 633.1 634.1
-6.5 - - - 632.3 633.4 634.4
-6.25 - - - 632.6 633.6 634.6
-6 - - - 632.8 633.8 634.9
-5.75 - - - 633.1 634.1 635.1
-5.5 - - - 633.3 634.3 635.3
-5.25 - - - 633.5 634.6 635.6
-5 - - - 633.8 634.8 635.8
-4.75 - - - 634 635 636.1
-4.5 - - - 634.3 635.3 636.3
-4.25 - - - 634.5 635.5 636.5

I do an example myself with the new given information f I understand the pinciple:
The example will use numbers of the picture of the excel-sheet you posted

The load-cell connected to the Arduino sends a value "654" over serial to the NodeMCU

The Arduino sends a temperature-value let's say -8,5.
This means the nodeMCU receives a value "654" and "-8.5"

654 means open file named "654.txt"

-8.5 means lookup that line inside file 654.txt where in the first columm is value -8.5 and pick the corresponding density which is "631.4" in this case.

If I look into your textfiles there is only a single columm with density-values.
How shall this work?
Do you want to calculate an offset?
This would mean all *.TXT-files start with -10.0 stepping down with 0.25 steps
so to use the above example again -8.5 is the

10.0 - 8.5 = 1.5 / 0.25 is the 6th line inside your textfile pluse some additional offset from the strange values at the beginning?

I think this would be error-prone.
I would store the densities and the temperature-values in each file.

If this needs too much memory you could create a CSV-file.
A comma-separated-value file just right from your excel-sheet.

°C,650,651,652,653,654,655
0,1,2,3,4,5,6,7
-10,-,-,-,628.9,630,631
...
...
...
-8.5,-,-,-,630.4,631.4,632.4

The commas are the separators

Then you would scan the first line for value 654 which is columm 6

then you would scan the lines until value "-8.5" is found and then take value from columm 6

This all is just read in a line of the file and then do certain string manipulatings.
The library SafeString has a lot of functions like find position of substring
give back value of next columm etc., etc., etc.

So make a decision which solution you want to use
a single file with all load-cell-values
or multiple files for each load-cell-value

and then write a first attempt how you would program reading from your textfile.
For tests I would recommend that you use easy to identify load-cell values
the loadcell values start in columm 2
so for testing use the values 222, 333, 444, 555
and as densities
200.1, 300.1, 400.1, 500.1
200.2, 300.2, 400.2, 500.2
200.3, 300.3, 400.3, 500.3
etc. etc.
So you can see very easily if the right line and the right columm are chosen.

This example has a reduced size compared to your complete data but it is still able to explain the relations and by using concrete values everybody can see how the values relate to each other.

If my description is not wrong than correct it and explain it with the same principle as I have done it.
best regards Stefan

Another approach:
Have you ever tried to find a proximity function with excel?
You have all these values-pairs temperature -densiity.

Excel has tools to calculate the parameters for a proximity-function
linear, polynomial, exponential, logarithmical.

You put in the temperatur-value and the proximity function calculates the density.

f (temperature) = density
f_653 (-10) = 628.9
f_6654(-10)= 630

etc. etc. If this apporach can be used depends on how big the maximum deviation of the proximity-function and the real values is.

This can only be found by using the tools and compare the values.
best regards Stefan

by the way the values in your table increase with temperature

Temperature -10 density 630
..
..
..
..
Temperature -4.25 density 635.5

So these values can't be density with units mass / volume like kg / Liter
or kg / m³ etc.

Usually desnity decreases if temperatur rises
best regards Stefan

Thanks Stefan. First of all thanks for bringing into my notice that the values are increasing with the temperature and hence they are not really the densities. I might have gone through that table alot many times but didn't noticed it.
so, apparently I really have no idea what those values really mean...the guy who gave me says that they are densities. Oh yes! I forgot to mention before looking up the table he asked me to multiply the incoming loadcell value by DEC 10. The reason for this the guy is not going to put weight more than 100gms on the load cell, but he wants it to be interpreted as 1Kg.
would it make any difference?
Speaking about the two approaches, yours approach seems much more cool! and I do not even need to create many files! one question I would like to ask though do I have any limitations on how many values I can feed into this file and later access them as well?
And my other question would be coding to open a file is okay, I can do it by looking at the example sketches but how do I iterate among the rows and columns until the desired match is hit? and also then stored the particular value after I found the match? It seems lil confusing for me as I can't even guess where do I start for this iteration thing :confused:

Thanks Stefan