Check Data from SD-Card vs. CRC

Using Arduino Programming Questions
1

Hello everyone,

I am going to do a 2 weeks Test Run for my Arduino to check if any Bit Flips will appear in that time.
I am using a Data Logger and a 16GB (to increase the probabilty for a Bit Flip) SD-Card and log to the internal EEPROM of the Arduino UNO R3 Board.
One function should test the SD-Card Data with CRC every 3 hours and should log for every possible Bit Flip which occurs.

My question is how do you check the CRC of the SD-Card? I tried to find out something in the SDFat library, but havent found something helpful. Is there any code which will test SD-Card Data vs CRC?

Also, as soon as the first Bit Flip appears, the function should inform me for every following error which will occur. Will it do if i just save the old CRC for this?

Thanks,
Noob

2

Other post/duplicate DELETED
Please do NOT cross post / duplicate as it wastes peoples time and efforts to have more than one post for a single topic.

Continued cross posting could result in a time out from the forum.

Could you also take a few moments to Learn How To Use The Forum.

Other general help and troubleshooting advice can be found here.
It will help you get the best out of the forum in the future.

3

Cannot quite see what this has to do with CRCs.

If the SD card is full of files that are like this

0X55, 0X55, 0X55, 0X55 etc.

You just need to read every byte in every file, if there is a bit flip you wont read 0x55 for every byte ...............................

However, 16Gbyte is a huge amount of data, have you worked out how many bytes a second you would need to read to check the full 16Gbyte in 3 hours ?

4

You are asking the wrong question.

Calculate the CRC value for a predefined collection of stored data, and again some time later. If the two values don't match, something in the data changed over time, or you had a data read or communications error.

A "bit flip" is not the most likely problem to arise.

5

If you have a CRC for the whole SD-Card you would have to re-calculate the CRC across all 16 GB every time you write data to the SD-Card. That would be impossibly slow.

It is more typical to store a CRC for each of many smaller units of data. In the case of an SD-Card you might store a CRC for each 512-byte 'block'. For a 16 GB card that would be 32M CRC's or 64 MB of 16-bit CRC's. I would reduce the reported size of the SD-Card and put the CRC's at the end.

Disable partial block reads. When a block is written, update the CRC for that block. When the block is read, check the CRC.

1 Like
6

Another option is to write error-correction codes with every record. Not only will that tell you if one or more bits were flipped, it will also tell you which bit(s).

After your two-week test you can go back over the data to check for errors.

7

Hello, thanks for your response! It helped me to understand the problem, that i need to write/read in blocks, but I have problems implementing the read/write of 512 byte blocks...
I tried to do this by writing Data to a .txt file, but it seems that the Data is landing in a random sector of the SD-Card ( I checked with HxD editor). Is it possible to begin writing Data to sector 0 and so on? I really cant help me with SD-libraries. I guess smth like readblock(sector) and writeblock(sector) would help, but I have no idea how to do this. Any suggestions? Im a beginner, so anything is appreciated.

8

Hello everyone,

I want to write Data to my SD-Card using 512 bytes Blocks. Actually i want 510 bytes of Data and 2 bytes for CRC of that block.

I havent found out anything in the SD-libs regarding this. How do you write/read these blocks? I know that you need a buffer and then write that buffer to the sector. But how?

Also is it possible to write/read without a file system? I tried this before to write Data into a .txt file, but this seems confusing, as this Data landed in Sector 8448. I want to begin writing blocks from sector 0 and so on.
Is there any code I can get inspired by? Or any tips? Im a beginner, so anything is appreciated.

Thank you,
Noob

9

I note that you opened a new topic asking about writing blocks of data but I feel that it would be better asked here to keep the context of the question so I have merged the two topics

1 Like
11

What do you mean by 'sectors', its confusing ?

12

hey! by sectors im meaning the blocks which contain 512 bytes each. i was confusing youre right, so sector = 512byte block

13

The Arduin SD reference gives the syntax for writing an array to a SD file.

So create a 512 byte array, fill it with 510 bytes of data, add the CRC in the last two bytes and write the array to the SD file.

14

is this possible without writing to a file? As in i want to write an array directly to sector 0 for example? When im writing to a file it lands somewhere in the SD-Card, but i want to write and later on read beginning from sector 0, then increment the sector number, write or read that sector and so on.

15

Check out the SD reference, in particular the use of the seek() command.

16

what's sector 0 ?
SD cards get formatted (FAT16, FAT32, exFAT...) and that's the foundation for writing files.

are you saying you don't want a file system and just issue low level commands ? You would have to read the SD card physical layer specification and experiment with reading and writing blocks. I'm not aware of a library for doing this.

Maybe an SD card is not really the best storage, external EEPROM or the likes could be an alternative.

17

My suscpicion is that sector 0 is the first 512bytes of the file, sector 1 is the second 512bytes etc.

As in sectors are nothing to do with the physical structure of the SD card itself.

18

It looks like the "Sd2Card.cpp" file in the SD library takes car of writing blocks. Of course, if you write over the FAT partition table and boot record in Block 0 you won't have a formatted SD card and it won't be readable as a file.

19

Ok, so i want to run my Test for 2 weeks now and will check a large txt file over that time.
Can i detect possible Bit Flips when reading from this .txt file? Will Bit Flips also change the txt file, or do these files have some kind of extra protection, so I cant detect errors?

I tried using the readblock-method from "Sd2Card.cpp", but all it does it sending me an error and I cant figure out what the problem is.

This is kinda frustrating, so i thought going the way of making an global array which will be my Data Buffer and like i said read then from the file. Can this work? Can I catch possible Bit Flips which occur over time while reading from a .txt file again and again?

20

If you add CRC (Cyclic Redundancy Check) or ECC (Error Correcting Code) when you write the data and check it when you read the data you should be able to detect bit flips. With ECC you can correct one or more bit flips.

I don't think that text files are 'protected' from bit errors but I am not familiar with whatever low-level coding may be done within the SD Card.

21

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.