Problem with SD card file dump(of BMP file)

I am trying to transfer a bmp image from my sd card(from arduino) to my pc through serial.

I tried using the the sd card file dump(slightly modified to print out file stream in HEX) from the example, the output string i get seem to have some padding problems.

been trying to hours. dont know what is wrong. anyone know y?

the output stream:

424D4E00000003600028000300020001018000001800011B0011B00000000006666CC4966C63E369C0006066F96466CC6466A8000

the correct stream(red coloured an example of such a padding problem:

424D4E0000000[color=red]0000000[/color]36000000280000000300000002000000010018000000000018000000110B0000110B000000000000000000006666CC4966C63E369C0000006066F96466CC6466A8000000

The code:

/*
  SD card file dump

 This example shows how to read a file from the SD card using the
 SD library and send it over the serial port.

 The circuit:
 * SD card attached to SPI bus as follows:
 ** MOSI - pin 11
 ** MISO - pin 12
 ** CLK - pin 13
 ** CS - pin 4 (for MKRZero SD: SDCARD_SS_PIN)

 created  22 December 2010
 by Limor Fried
 modified 9 Apr 2012
 by Tom Igoe

 This example code is in the public domain.

 */

#include <SPI.h>
#include <SD.h>

const int chipSelect = 10;

void setup() {
  // Open serial communications and wait for port to open:
  Serial.begin(115200);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }


  Serial.print("Initializing SD card...");

  // see if the card is present and can be initialized:
  if (!SD.begin(chipSelect)) {
    Serial.println("Card failed, or not present");
    // don't do anything more:
    return;
  }
  Serial.println("card initialized.\n\n");

  // open the file. note that only one file can be open at a time,
  // so you have to close this one before opening another.
  File dataFile = SD.open("frame000.bmp");

  // if the file is available, write to it:
  if (dataFile) {
    while (dataFile.available()) {
      Serial.print(dataFile.read(), HEX);
    }
    dataFile.close();

    Serial.println("\n\nEnd of file.");
  }
  // if the file isn't open, pop up an error:
  else {
    Serial.println("error opening datalog.txt");
  }
}

void loop() {
}

a little more info about my project..

i am trying to send and receive bmp files from the arduino to my android app on my phone.

planning to send a bmp file in the sd breakout of arduino to the phone by sending a string of hex through the serial(bluetooth) to my android app. and the android app will convert from hex to bmp.

when sending from arduino to android app, i keep getting the above error, where there arent sufficient number of 0s between.

anyone got any idea why?

would really appreciate help of any kind

I am trying to transfer a bmp image from my sd card(from arduino) to my pc through serial.

I tried using the the sd card file dump(slightly modified to Serial.print(dataFile.read(), HEX); to print it out in HEX).

the output string i get seem to have half the 0s

the output from arduino on serial monitor: 424D3A 0000000 360002800010001000101800000400011B0011B00000000006066F90

the actual correct output according to online file to hex converter: 424D3A 0000000 0000000 36000000280000000100000001000000010018000000000004000000110B0000110B000000000000000000006066F900

As coloured, the output from arduino on serial monitor seem to only half the number of 0s as the actual correct output. the others are all correct

any idea what is going on?

any idea what is going on?

Yes

Serial.print has leading zero suppression. Use Serial.write instead to transfer the exact byte.

Grumpy_Mike: Yes

Serial.print has leading zero suppression. Use Serial.write instead to transfer the exact byte.

i am trying to send a string of hex to my smartphone app through serial. i will need the byte to be converted to hex before sending. is there any way of doing so?

i cant use sprintf

ive tried this too, but still the same output

String thisString = String(dataFile.read(), HEX);
      Serial.print(thisString);

i cant use sprintf

Why can’t you?

Why can’t you send the binary data in the file in binary? I’m sure that the Android can handle binary data.

PaulS: Why can't you?

Why can't you send the binary data in the file in binary? I'm sure that the Android can handle binary data.

sprintf uses too much memory.

sending binary data is a little too slow for large files, even at 115200 baud rate

Grumpy_Mike: Yes

Serial.print has leading zero suppression. Use Serial.write instead to transfer the exact byte.

I've fixed the problem! Thanks for pointing this out!

Thanks PaulS too!

I've fixed the problem by simply printing/padding an additional 0 infront of each hex value that has the length of 1

if(thisString.length() == 1){
        Serial.print("0"); //Serial.print("|");
      }
      
      Serial.print(thisString);

sending binary data is a little too slow for large files, even at 115200 baud rate

So, you made sending data faster by sending two characters for every byte in the file.

I'm sure that that's much faster. You could speed that up even more by sending "eighty", instead of "80". Course, that will take a bit of fancy coding...

well, i could do base32 and so on, but hex will do for now.

any idea how to transform a string of hex(hex representation in ascii) back to a file?

ive tried it on an online hex to file/bin converter : http://tomeko.net/online_tools/hex_to_file.php?lang=en

and the file retrieved is correct.

ive tried myFile.print(input,BIN); and it just prints the binary 0s and 1s into the .bmp like a txt file :sweat_smile: :sweat_smile:

any idea how to transform a string of hex(hex representation in ascii) back to a file?

The best way is not to send them as an ASCII string in the first place. But as my teacher used to say " it’s your own time you are wasting".

But if you must then:-

 recoverdByte = (convertFromHex(s0) << 4) | (convertFromHex(s1));

//.......

byte convertFromHex(byte ascii){ 
  if(ascii > 0x39) ascii -= 7; // adjust for hex letters upper or lower case
  return(ascii & 0xf);
}

where s0 is the most significant ASCII byte and s1 is the least significant ASCII byte .

Grumpy_Mike:
The best way is not to send them as an ASCII string in the first place. But as my teacher used to say " it’s your own time you are wasting".

But if you must then:-

 recoverdByte = (convertFromHex(s0) << 4) | (convertFromHex(s1));

//…

byte convertFromHex(byte ascii){
  if(ascii > 0x39) ascii -= 7; // adjust for hex letters upper or lower case
  return(ascii & 0xf);
}



where s0 is the most significant ASCII byte and s1 is the least significant ASCII byte .

hi, thanks for your help grumpy_mike, i see that you helped someone asking similar question 3years back. but he does things a little differently than me so i cant rly use the solution in there.

just making sure, in the case of hex, for example, 0x4D.
s0 = 4,
s1 = D,

am i right to assume so? i am kinda confused what is going on haha

edit: oh i just saw your MSB and LSB line. cant seem to get the right conversion :confused: shall try again

Grumpy_Mike:
The best way is not to send them as an ASCII string in the first place. But as my teacher used to say " it’s your own time you are wasting".

But if you must then:-

 recoverdByte = (convertFromHex(s0) << 4) | (convertFromHex(s1));

//…

byte convertFromHex(byte ascii){
  if(ascii > 0x39) ascii -= 7; // adjust for hex letters upper or lower case
  return(ascii & 0xf);
}



where s0 is the most significant ASCII byte and s1 is the least significant ASCII byte .

Hi Grumpy_Mike,
I’ve tried your code, cant seem to produce the correct output. am i doing things right?

byte recoverByte,s0,s1;

Serial.print("Writing HexDump to hexdump.bmp...");
Serial.println("\nReceived HexDump: ");
while(s0 != '|' || s1 != '|'){
    if(Serial.available() > 0) {
        s0=Serial.read();
        s1=Serial.read();
    
        if(s0 == '|' || s1 == '|')break;

        recoverByte = (convertFromHex(s0) << 4) | (convertFromHex(s1));
        Serial.write(recoverByte); 
        myFile.write(recoverByte);
    }
}

am i doing things right?

No. However what you are doing I haven't got a clue. What is all this about?

if(s0 == '|' || s1 == '|')

The variables s0 and s1 are the two bytes you receive as ASCII characters. That code is seriously weird, can you explain what you think it is doing?

Grumpy_Mike:
No.
However what you are doing I haven’t got a clue.
What is all this about?

if(s0 == '|' || s1 == '|')

The variables s0 and s1 are the two bytes you receive as ASCII characters. That code is seriously weird, can you explain what you think it is doing?

YES ITS WORKING. THANKS DUDE!!! :D:D:D:D:D

it appears its bad coding on my side. i did not take into account that the serial buffer could be empty after read() to s0, making s1 sometimes get unknown values(due to buffer empty).

my program works like this

  1. enter a custom command, executes receiveHexDump() function
  2. the receiveHexDump() function waits endlessly for the hex string, until the terminator ‘|’ is received
  3. the function reads the first 2bytes off the buffer, converts it to byte/bin, and writes to the .bmp image
  4. when it reads ‘|’ off the buffer, it means the bmp file write operation is complete. and it gets out of the receiveHexDump() function.

Heres how my code looks like after refining:
it now ensures there is something in the buffer before writing.

byte recoverByte;
char input;
char s0,s1;
char scounter=0;

if (myFile) {
		Serial.print("Writing HexDump to hexdump.bmp...");
		Serial.println("\nReceived HexDump: ");
		while(input != '|'){
			if(Serial.available() > 0) {
				input=Serial.read();
				Serial.print("input: ");Serial.println(input);
				
				if(scounter == 0){
					s0=input;
					scounter=1;
					continue;
				}
				else if(scounter == 1){
					s1=input;
					scounter=0;
					if(s0 == '|' || s1 == '|')break;
					Serial.print("s0: ");Serial.print(s0);
					Serial.print("\t\ts1: ");Serial.println(s1);
					recoverByte = (convertFromHex(s0) << 4) | (convertFromHex(s1));
					Serial.write(recoverByte); 
					myFile.write(recoverByte);
				}
				
			}
			
		}
	// close the file:
		myFile.close();
		Serial.println("\ndone.");
	}

making s1 sometimes get unknown values(due to buffer empty).

There is nothing unknown about the -1 that Serial.read() returns to tell you that you were a moron fro trying to read from an empty buffer.

PaulS: There is nothing unknown about the -1 that Serial.read() returns to tell you that you were a moron fro trying to read from an empty buffer.

prolly with my code or something, but sometimes i get afew ? to the s0 and s1, probably cause i serial.printed in hex the bytes printed into the file.

only from seeing and the subsequent chars that only arrive later, did it make me deduce the buffer was empty.

bad coding on my part

the 'moron' was kinda unnecessary.. but still, thanks for ur help.

the 'moron' was kinda unnecessary.. but still, thanks for ur help.

If you KNOW that you need to read characters in pairs, why not write the code that way? That convoluted code is very hard to follow.

   if(Serial.available() >= 2)
   {
      char s0 = Serial.read();
      char s1 = Serial.read();

      // Now put them back together...

PaulS: If you KNOW that you need to read characters in pairs, why not write the code that way? That convoluted code is very hard to follow.

   if(Serial.available() >= 2)
   {
      char s0 = Serial.read();
      char s1 = Serial.read();

      // Now put them back together...

oh right.. why didnt i think of that LOL.. HAHAHA thanks paul!