Data storage on SD card - file organisation, library search.

Hello.

I want to use SD card for storing way point in my project of robot GPS navigation system.

File format (single string):

X A.AAAAAA B.BBBBBB

Where:
X - way point index from 1 to NN
A.AAAAAA - latitude in decimal format
B.BBBBBB - longtitude in decimal format

Who ever solved a similar problem?

Any underwater stones?
May be exist a library for working with structured files?

Thanks.
Sorry my bad English.

I'm not sure why you want "structured files".

A simple solution might be to store your data as fixed length strings and then you could retrieve the Nth string by skipping the appropriate number of bytes.

...R

Robin2:
A simple solution might be to store your data as fixed length strings and then you could retrieve the Nth string by skipping the appropriate number of bytes.

...R

I second that (that is, I support this solution).

Use a standard C function called sprint() to format the numbers to be stored and use sscanf() to read them back.

Tutorials on my blog:

Thanks for answers, but this is not good for me.

Input data - way point file - generated automatically in another software (not my project) .
I can change or recode this files. Only reading :frowning:
So - back to my question - any good solution for working with structured files exist on Arduino?

How is the data going to be put on the SD Card?

I assumed from your original post that the Arduino was going to write the data. If so it is easy to make fixed length records even if they are not received as fixed length records.

Can you give an example of 3 or 4 records showing the range of sizes that you need to deal with.

I think it is unlikely there is a structured file system that works within the very limited memory available on an Arduino Uno. Perhaps you need something like a Yun which could have the file system on the Linino side and the hardware interface on the Arduino side.

...R

Thanks for answers, but this is not good for me.

Why not? The file format is pleasantly regular and thus easy to parse. Either manually or using sscanf. I'm not sure what more a 'structured file library' would buy you.

Data on SD card going from path planing software - closed for me.

My task - read first way point lat. and long. and when i reach this way point - increase index and repeat.
And i need write way point in other file with equivalent format - this is for movement reconstruction...

In one time moment i have in memory of my board MCU - a current way point index, lat. and long. Thats it.

Total not more that 1000 way point records can be in one file.

If you answer the questions in Reply #4 I may be able to help.

...R

Robin2:
Can you give an example of 3 or 4 records showing the range of sizes that you need to deal with.

Ok. I try.

File named WAY.TXT - This is data for reading in Arduino and have record:

1 56.3331239 41.3052903
2 56.333281 41.3052708
3 56.334128 41.3042505
etc up to record with index 1000
Can be less that 1000 but no more.

this file exist in SD card before running.

FILE named PATH_1.TXT - This is data for writing - real path storage. Looks like

1 56.331239 41.3052903
2 56.331252 41.30530
3 56.331305 41.3053205
etc up to record with index
1000
Can be less that 1000 but no more.
If index overrun 1000 - it start from 1, but new file must be open for storage with name PATH_2.TXT

1 56.3331239 41.3052903
2 56.333281 41.3052708

Looking at this data, I have a couple of questions.

Are the line numbers (1 and 2) part of the data?
What characters come between each line? Perhaps Cr (13) and/or Lf (10)?
If you want to find a particular record what criterion do you want to use - for example do you want to find the next line or do you want to find the line with Line Number 23 or what?
Is there a guaranteed minimum line length?

If you don't have fixed length records the only way to find a position in the file is to go through the file byte by byte until you find what you want.

You still haven't told us what software is creating the file? Is it a secret? (I'd tell you but then I'd have to shoot myself? :slight_smile: )

...R

Are the line numbers (1 and 2) part of the data?

Yes.

What characters come between each line? Perhaps Cr (13) and/or Lf (10)?

Cr (13) and Lf(10) in sequence CrLf

Robin2:
Looking at this data, I have a couple of questions.

If you want to find a particular record what criterion do you want to use - for example do you want to find the next line or do you want to find the line with Line Number 23 or what?

I want to have aces to any record by it index. Cause my robot can start randomly on place. I check first record (?1) and see - i m to far from it. So i check record with index 10 and i see - i close to it - ok - start movement. After i reach some distance to this point - i go to next by incrementing index by 1.

Is there a guaranteed minimum line length?

Minimum line length - i don't sure. Cause can be latitude and latitude = integer degrees. With nothing in decimal part. But i don't see this in real...

You still haven't told us what software is creating the file? Is it a secret? (I'd tell you but then I'd have to shoot myself? )

This is software developed on QT framework (not my creation) working with topographic maps with linking to coordinate system. This software use maps with 1 cm equal to 10 m.
I don't have any idea where developer of software take this maps, but it exist for my city.
Main task of project - compare the performance of different algorithms for searching for a way to obtain the map position of the robot from GPS / Glonass

My best advice is that you read and store the data character by character until you reach a CrLf. Then you know you have one line and the first few characters - up to the first space - will be the index number.

Repeat until you get to the index number you want.

You could do it a bit smarter by just reading looking for a CrLf and the reading the next few bytes to check the index number. If you don't want that record and if you know there is a minimum length of record you could possibly speed the process a bit by skipping that number of bytes before starting the search for CrLf. But it's a bit hard to explain exactly how to do that in a few words.

Main task of project - compare the performance of different algorithms for searching for a way to obtain the map position of the robot from GPS / Glonass

I don't think you have told us this before. What is that about? What are you planning to compare?

It's hard to give useful advice when we only see part of the picture.

...R

Robin2:
I don't think you have told us this before. What is that about? What are you planning to compare?

We have theoretical path on software generated by different algorithms , a way point set. And we have robot in real world. So. Robot take decision "which way to go" - based on real word situation - sensor data etc. And we want to compare how much randomly exist factors of real world (cat on way of robot as example :slight_smile: ) - make a distortion of theoretical path.
Sorry but i don't have English skill to explain that....

CrazyIgels:
And we want to compare how much randomly exist factors of real world (cat on way of robot as example :slight_smile: )

Thanks. My mistake - sorry. I thought you might be trying to compare algorithms for finding the record on the SD Card.

...R