Reading Sd file to serial port

Projects Programming
1

Hi,
I'm trying to transmit a txt stored in a Sd to a FTP server through GPRS. Once I connected correctly to the ftp server I have to send the file, to do this I use the next program:

boolean end_file = false;
  byte bytes_read = 0;
  char* buffer_int = (char*) calloc(250,sizeof(char));
  if (sd.begin(chipSelect, SPI_HALF_SPEED)){
    if (myfile.open(filename, O_READ)){

      do{
        end_file = false;	//Starts to send data
        bytes_read = myfile.read (buffer_int, 250);
        if (bytes_read < 250) {
          end_file = true;
          
        }
        
        Serial.print (buffer_int);
        
        
      } while(end_file == false);
    }
  }
}

With this sketch I try to read small pieces of the file and store it in a buffer. Afterwards I send the buffer to the serial port. It works realatively well but between the fragments of the file appears many strange characters in the serial port. This is the outcome in the serial monitor:

3898922,-692388,120828,112858,181,16,0,0
3898909,-692373,121001,202459,194,15,0,0
3898909,-692373,121001,202459,194,15,0,0
3898910,-692373,121001,202500,194,15,0,0
3898910,-692373,121001,202500,194,15,0,0
1111111,-111111,111111,111111,111,11,1,1]?Ý«·¼Ývã;ÝK»÷·ÈOZÿ~?1Å)¦ïTßuiu­þ[}[qÞóæüÝÛ^þsõ©ßc?yo®?u÷#O?üÎ3ñ÷«'}A¿;äó­-?[ýRÿõÿ½?OÌØí³Ï??úïûm»õæTUoþqÝ?¬Zþü?ßz;¿·ï_ð>òÊÞÖhòǯ:ºÚ¶9µý+À??[tjÿ´®GìS^¿üÝþïYÊ¿FsÊÿß-Oþ=ÏÅÞ>wÌÛJÍ?ßK[Ϻ?JF?RïïØwº\ÛÿI]qÚ>ÏÝ篠WÕ?öÙ)?Ïâ#oþ3lçòî'úOùÿ{çJ?諪ûkÛ¾?ýþçð^eïz?õÇÅ-;MûùÄjÖ¦~pifç[Ø÷ÿ¹);ÏÐ?î°?3ÿÓg7u~ÿ&±G£ëÕþÞÿÙØ*?×÷Ãÿw?ó`Z»?ÜF?©vPßâØ?ëõû¯ÿȺ¦eß?ª?©ãÕÇßZþ qû¹Õõ<mE/·^ç?}q?
í«ý,[]Þþôéeût÷g(]ñë¸_üþóÙjÏûî±çÜj{Ï·ßÿÿA¡ÿÛmß?¬Oùöÿ'?ýû	°Ç?¸wÍ
­¶ÝîÐ~ÛyÆ·ßøì?/7õ÷p}I?wù+ü{ö´÷?vïþ¸&ø\ãïÿäô{ÿ³z3áG¯ko-¾ç??ùx\õû"×íÓ×ð³Î'ÕÃ<ýÍûÿ3Bq|ïûÖòí?YÍåYïèßít?ro|Úlza÷®ÜÊ+}ÝŝïØþbûö§øçHUëöÖo?ìµ?ëö®Õ?¾òýÿaÚ¿«öcç°m]Wÿÿ þÓ
3898910,-692373,121001,202501,194,15,0,0
3898910,-692373,121001,202501,194,15,0,0
3898910,-692373,121001,202502,194,15,0,0
3898910,-692373,121001,202502,194,15,0,0
3898910,-692373,121001,202503,194,15,0,0
3898910,-692373,121001,202503,194,15,0]?Ý«·¼Ývã;ÝK»÷·ÈOZÿ~?1Å)¦ïTßuiu­þ[}[qÞóæüÝÛ^þsõ©ßc?yo®?u÷#O?üÎ3ñ÷«'}A¿;äó­-?[ýRÿõÿ½?OÌØí³Ï??úïûm»õæTUoþqÝ?¬Zþü?ßz;¿·ï_ð>òÊÞÖhòǯ:ºÚ¶9µý+À??[tjÿ´®GìS^¿üÝþïYÊ¿FsÊÿß-Oþ=ÏÅÞ>wÌÛJÍ?ßK[Ϻ?JF?RïïØwº\ÛÿI]qÚ>ÏÝ篠WÕ?öÙ)?Ïâ#oþ3lçòî'úOùÿ{çJ?諪ûkÛ¾?ýþçð^eïz?õÇÅ-;MûùÄjÖ¦~pifç[Ø÷ÿ¹);ÏÐ?î°?3ÿÓg7u~ÿ&±G£ëÕþÞÿÙØ*?×÷Ãÿw?ó`Z»?ÜF?©vPßâØ?ëõû¯ÿȺ¦eß?ª?©ãÕÇßZþ qû¹Õõ<mE/·^ç?}q?
í«ý,[]Þþôéeût÷g(]ñë¸_üþóÙjÏûî±çÜj{Ï·ßÿÿA¡ÿÛmß?¬Oùöÿ'?ýû	°Ç?¸wÍ
­¶ÝîÐ~ÛyÆ·ßøì?/7õ÷p}I?wù+ü{ö´÷?vïþ¸&ø\ãïÿäô{ÿ³z3áG¯ko-¾ç??ùx\õû"×íÓ×ð³Î'ÕÃ<ýÍûÿ3Bq|ïûÖòí?YÍåYïèßít?ro|Úlza÷®ÜÊ+}ÝŝïØþbûö§øçHUëöÖo?ìµ?ëö®Õ?¾òýÿaÚ¿«öcç°m]Wÿÿ þÓ,0
3898910,-692373,121001,202504,194,15,0,0
3898910,-692373,121001,202504,194,15,0,0
3898910,-692373,121001,202505,194,15,0,0
3898910,-692373,121001,202505,194,15,0,0
3898910,-692374,121001,202506,194,15,0,0
3898910,-692374,121001,202506,194,15]?Ý«·¼Ývã;ÝK»÷·ÈOZÿ~?1Å)¦ïTßuiu­þ[}[qÞóæüÝÛ^þsõ©ßc?yo®?u÷#O?üÎ3ñ÷«'}A¿;äó­-?[ýRÿõÿ½?OÌØí³Ï??úïûm»õæTUoþqÝ?¬Zþü?ßz;¿·ï_ð>òÊÞÖhòǯ:ºÚ¶9µý+À??[tjÿ´®GìS^¿üÝþïYÊ¿FsÊÿß-Oþ=ÏÅÞ>wÌÛJÍ?ßK[Ϻ?JF?RïïØwº\ÛÿI]qÚ>ÏÝ篠WÕ?öÙ)?Ïâ#oþ3lçòî'úOùÿ{çJ?諪ûkÛ¾?ýþçð^eïz?õÇÅ-;MûùÄjÖ¦~pifç[Ø÷ÿ¹);ÏÐ?î°?3ÿÓg7u~ÿ&±G£ëÕþÞÿÙØ*?×÷Ãÿw?ó`Z»?ÜF?©vPßâØ?ëõû¯ÿȺ¦eß?ª?©ãÕÇßZþ qû¹Õõ<mE/·^ç?}q?
í«ý,[]Þþôéeût÷g(]ñë¸_üþóÙjÏûî±çÜj{Ï·ßÿÿA¡ÿÛmß?¬Oùöÿ'?ýû	°Ç?¸wÍ
­¶ÝîÐ~ÛyÆ·ßøì?/7õ÷p}I?wù+ü{ö´÷?vïþ¸&ø\ãïÿäô{ÿ³z3áG¯ko-¾ç??ùx\õû"×íÓ×ð³Î'ÕÃ<ýÍûÿ3Bq|ïûÖòí?YÍåYïèßít?ro|Úlza÷®ÜÊ+}ÝŝïØþbûö§øçHUëöÖo?ìµ?ëö®Õ?¾òýÿaÚ¿«öcç°m]Wÿÿ þÓ,0,0
3898910,-692374,121001,202507,194,15,0,0
3898910,-692374,121001,202507,194,15,0,0
3898910,-692374,121001,202508,194,15,0,0
3898910,-692374,121001,202508,194,15,0,0
3898910,-692374,121001,202509,194,15,0,0
3898910,-692374,121001,202509,194,]?Ý«·¼Ývã;ÝK»÷·ÈOZÿ~?1Å)¦ïTßuiu­þ[}[qÞóæüÝÛ^þsõ©ßc?yo®?u÷#O?üÎ3ñ÷«'}A¿;äó­-?[ýRÿõÿ½?OÌØí³Ï??úïûm»õæTUoþqÝ?¬Zþü?ßz;¿·ï_ð>òÊÞÖhòǯ:ºÚ¶9µý+À??[tjÿ´®GìS^¿üÝþïYÊ¿FsÊÿß-Oþ=ÏÅÞ>wÌÛJÍ?ßK[Ϻ?JF?RïïØwº\ÛÿI]qÚ>ÏÝ篠WÕ?öÙ)?Ïâ#oþ3lçòî'úOùÿ{çJ?諪ûkÛ¾?ýþçð^eïz?õÇÅ-;MûùÄjÖ¦~pifç[Ø÷ÿ¹);ÏÐ?î°?3ÿÓg7u~ÿ&±G£ëÕþÞÿÙØ*?×÷Ãÿw?ó`Z»?ÜF?©vPßâØ?ëõû¯ÿȺ¦eß?ª?©ãÕÇßZþ qû¹Õõ<mE/·^ç?}q?
í«ý,[]Þþôéeût÷g(]ñë¸_üþóÙjÏûî±çÜj{Ï·ßÿÿA¡ÿÛmß?¬Oùöÿ'?ýû	°Ç?¸wÍ
­¶ÝîÐ~ÛyÆ·ßøì?/7õ÷p}I?wù+ü{ö´÷?vïþ¸&ø\ãïÿäô{ÿ³z3áG¯ko-¾ç??ùx\õû"×íÓ×ð³Î'ÕÃ<ýÍûÿ3Bq|ïûÖòí?YÍåYïèßít?ro|Úlza÷®ÜÊ+}ÝŝïØþbûö§øçHUëöÖo?ìµ?ëö®Õ?¾òýÿaÚ¿«öcç°m]Wÿÿ þÓ15,0,0
3898910,-692374,121001,202510,194,15,0,0
3898910,-692374,121001,202510,194,15,0,0
3898910,-692374,121001,202511,194,15,0,0
3898910,-692374,121001,202511,194,15,0,0
3898911,-692375,121001,202512,194,15,0,0
3898911,-692375,121001,202512,19]?Ý«·¼Ývã;ÝK»÷·ÈOZÿ~?1Å)¦ïTßuiu­þ[}[qÞóæüÝÛ^þsõ©ßc?yo®?u÷#O?üÎ3ñ÷«'}A¿;äó­-?[ýRÿõÿ½?OÌØí³Ï??úïûm»õæTUoþqÝ?¬Zþü?ßz;¿·ï_ð>òÊÞÖhòǯ:ºÚ¶9µý+À??[tjÿ´®GìS^¿üÝþïYÊ¿FsÊÿß-Oþ=ÏÅÞ>wÌÛJÍ?ßK[Ϻ?JF?RïïØwº\ÛÿI]qÚ>ÏÝ篠WÕ?öÙ)?Ïâ#oþ3lçòî'úOùÿ{çJ?諪ûkÛ¾?ýþçð^eïz?õÇÅ-;MûùÄjÖ¦~pifç[Ø÷ÿ¹);ÏÐ?î°?3ÿÓg7u~ÿ&±G£ëÕþÞÿÙØ*?×÷Ãÿw?ó`Z»?ÜF?©vPßâØ?ëõû¯ÿȺ¦eß?ª?©ãÕÇßZþ qû¹Õõ<mE/·^ç?}q?
í«ý,[]Þþôéeût÷g(]ñë¸_üþóÙjÏûî±çÜj{Ï·ßÿÿA¡ÿÛmß?¬Oùöÿ'?ýû	°Ç?¸wÍ
­¶ÝîÐ~ÛyÆ·ßøì?/7õ÷p}I?wù+ü{ö´÷?vïþ¸&ø\ãïÿäô{ÿ³z3áG¯ko-¾ç??ùx\õû"×íÓ×ð³Î'ÕÃ<ýÍûÿ3Bq|ïûÖòí?YÍåYïèßít?ro|Úlza÷®ÜÊ+}ÝŝïØþbûö§øçHUëöÖo?ìµ?ëö®Õ?¾òýÿaÚ¿«öcç°m]Wÿÿ þÓ4,15,0,0
3898911,-692375,121001,202513,194,15,0,0
3898911,-692375,121001,202513,194,15,0,0
3898911,-692375,121001,202514,194,15,0,0
3898911,-692375,121001,202514,194,15,0,0
3898911,-692374,121001,202515,194,15,0,0
3898911,-692374,121001,202515,]?Ý«·¼Ývã;ÝK»÷·ÈOZÿ~?1Å)¦ïTßuiu­þ[}[qÞóæüÝÛ^þsõ©ßc?yo®?u÷#O?üÎ3ñ÷«'}A¿;äó­-?[ýRÿõÿ½?OÌØí³Ï??úïûm»õæTUoþqÝ?¬Zþü?ßz;¿·ï_ð>òÊÞÖhòǯ:ºÚ¶9µý+À??[tjÿ´®GìS^¿üÝþïYÊ¿FsÊÿß-Oþ=ÏÅÞ>wÌÛJÍ?ßK[Ϻ?JF?RïïØwº\ÛÿI]qÚ>ÏÝ篠WÕ?öÙ)?Ïâ#oþ3lçòî'úOùÿ{çJ?諪ûkÛ¾?ýþçð^eïz?õÇÅ-;MûùÄjÖ¦~pifç[Ø÷ÿ¹);ÏÐ?î°?3ÿÓg7u~ÿ&±G£ëÕþÞÿÙØ*?×÷Ãÿw?ó`Z»?ÜF?©vPßâØ?ëõû¯ÿȺ¦eß?ª?©ãÕÇßZþ qû¹Õõ<mE/·^ç?}q?
í«ý,[]Þþôéeût÷g(]ñë¸_üþóÙjÏûî±çÜj{Ï·ßÿÿA¡ÿÛmß?¬Oùöÿ'?ýû	°Ç?¸wÍ
­¶ÝîÐ~ÛyÆ·ßøì?/7õ÷p}I?wù+ü{ö´÷?vïþ¸&ø\ãïÿäô{ÿ³z3áG¯ko-¾ç??ùx\õû"×íÓ×ð³Î'ÕÃ<ýÍûÿ3Bq|ïûÖòí?YÍåYïèßít?ro|Úlza÷®ÜÊ+}ÝŝïØþbûö§øçHUëöÖo?ìµ?ëö®Õ?¾òýÿaÚ¿«öcç°m]Wÿÿ þÓ194,15,0,0
3898911,-692374,121001,202516,194,15,0,0
3898911,-692374,121001,202516,194,15,0,0
3898911,-692374,121001,202517,194,15,0,0
3898911,-692374,121001,202517,194,15,0,0
3898911,-692374,121001,202518,194,15,0,0
3898911,-692374,121001,20251]?Ý«·¼Ývã;ÝK»÷·ÈOZÿ~?1Å)¦ïTßuiu­þ[}[qÞóæüÝÛ^þsõ©ßc?yo®?u÷#O?üÎ3ñ÷«'}A¿;äó­-?[ýRÿõÿ½?OÌØí³Ï??úïûm»õæTUoþqÝ?¬Zþü?ßz;¿·ï_ð>òÊÞÖhòǯ:ºÚ¶9µý+À??[tjÿ´®GìS^¿üÝþïYÊ¿FsÊÿß-Oþ=ÏÅÞ>wÌÛJÍ?ßK[Ϻ?JF?RïïØwº\ÛÿI]qÚ>ÏÝ篠WÕ?öÙ)?Ïâ#oþ3lçòî'úOùÿ{çJ?諪ûkÛ¾?ýþçð^eïz?õÇÅ-;MûùÄjÖ¦~pifç[Ø÷ÿ¹);ÏÐ?î°?3ÿÓg7u~ÿ&±G£ëÕþÞÿÙØ*?×÷Ãÿw?ó`Z»?ÜF?©vPßâØ?ëõû¯ÿȺ¦eß?ª?©ãÕÇßZþ qû¹Õõ<mE/·^ç?}q?
í«ý,[]Þþôéeût÷g(]ñë¸_üþóÙjÏûî±çÜj{Ï·ßÿÿA¡ÿÛmß?¬Oùöÿ'?ýû	°Ç?¸wÍ
­¶ÝîÐ~ÛyÆ·ßøì?/7õ÷p}I?wù+ü{ö´÷?vïþ¸&ø\ãïÿäô{ÿ³z3áG¯ko-¾ç??ùx\õû"×íÓ×ð³Î'ÕÃ<ýÍûÿ3Bq|ïûÖòí?YÍåYïèßít?ro|Úlza÷®ÜÊ+}ÝŝïØþbûö§øçHUëöÖo?ìµ?ëö®Õ?¾òýÿaÚ¿«öcç°m]Wÿÿ þÓ8,194,15,0,0
3898911,-692374,121001,202519,194,15,0,0
3898911,-692374,121001,202519,194,15,0,0
3898911,-692374,121001,202520,194,15,0,0
3898911,-692374,121001,202520,194,15,0,0
3898911,-692374,121001,202521,194,15,0,0
3898911,-692374,121001,202]?Ý«·¼Ývã;ÝK»÷·ÈOZÿ~?1Å)¦ïTßuiu­þ[}[qÞóæüÝÛ^þsõ©ßc?yo®?u÷#O?üÎ3ñ÷«'}A¿;äó­-?[ýRÿõÿ½?OÌØí³Ï??úïûm»õæTUoþqÝ?¬Zþü?ßz;¿·ï_ð>òÊÞÖhòǯ:ºÚ¶9µý+À??[tjÿ´®GìS^¿üÝþïYÊ¿FsÊÿß-Oþ=ÏÅÞ>wÌÛJÍ?ßK[Ϻ?JF?RïïØwº\ÛÿI]qÚ>ÏÝ篠WÕ?öÙ)?Ïâ#oþ3lçòî'úOùÿ{çJ?諪ûkÛ¾?ýþçð^eïz?õÇÅ-;MûùÄjÖ¦~pifç[Ø÷ÿ¹);ÏÐ?î°?3ÿÓg7u~ÿ&±G£ëÕþÞÿÙØ*?×÷Ãÿw?ó`Z»?ÜF?©vPßâØ?ëõû¯ÿȺ¦eß?ª?©ãÕÇßZþ qû¹Õõ<mE/·^ç?}q?
í«ý,[]Þþôéeût÷g(]ñë¸_üþóÙjÏûî±çÜj{Ï·ßÿÿA¡ÿÛmß?¬Oùöÿ'?ýû	°Ç?¸wÍ
­¶ÝîÐ~ÛyÆ·ßøì?/7õ÷p}I?wù+ü{ö´÷?vïþ¸&ø\ãïÿäô{ÿ³z3áG¯ko-¾ç??ùx\õû"×íÓ×ð³Î'ÕÃ<ýÍûÿ3Bq|ïûÖòí?YÍåYïèßít?ro|Úlza÷®ÜÊ+}ÝŝïØþbûö§øçHUëöÖo?ìµ?ëö®Õ?¾òýÿaÚ¿«öcç°m]Wÿÿ þÓ521,194,15,0,0
3898911,-692374,121001,202522,194,15,0,0
3898911,-692374,121001,202522,194,15,0,0
3898911,-692374,121001,202523,194,15,0,0
3898911,-692374,121001,202523,194,15,0,0
3898911,-692374,121001,202524,194,15,0,0

What could be the cause of this strange behaviour?
Thanks in advance for your help

2 
char* buffer_int

Why do you call the character buffer buffer_int?

        Serial.print (buffer_int);

Perhaps you should look at the documentation for Serial.print(). There are several overloads. The one you are using is for strings. buffer_int is NOT a string.

A string is a NULL terminated array of chars. Your array of chars is NOT null terminated. You must use a different form of Serial.print().

3

Hi,
Thanks for your reply Paul.
What kind of buffer should I use byte? My knowledge is very limited and I see in similar application the use of char’s buffers. I took this one from a very similar project that use both char buffer and print (printString). If I use a byte buffer, should I use serial.write to send to the serial port?
Why appears that strange character in the serial monitor just at the end of each block?

4

What kind of buffer should I use byte?

The type is correct. The name is wrong. The buffer does not contain ints, so int should not be part of its name.

There are two versions of the print() method that take arrays of chars or bytes. One expects the array to be NULL terminated, so you do not have to specify the length. The other does not expect the array to be NULL terminated, so you do have to specify the length.

You can either NULL terminate your array or specify its length.

5

What are the two versions of the print function?

If I add buffer [249] ='\0' to add the null character the outcome in the serial monitor is partial, many part of each line was not printed, and it is very slow.

6

If I add buffer [249] ='\0' to add the null character the outcome in the serial monitor is partial, many part of each line was not printed, and it is very slow.

Properly NULL terminating the array should have no negative affect of speed. In fact, it should improve things, since the function knows when the stop sending data.

I'll retract what I said earlier. The write() method, not the print() method allows you to send non-NULL-terminated arrays of chars to the serial port. The print() method does not.

When you added the NULL in position 249, did you change the length in the File::read() call, so that it doesn't place a character in that position?

7

Thanks for your support PaulS!!!
Following your recommendations I modified the sketch and now it send even more quickly (perhaps thanks to the 512 buffer).
The end sketch is:

#include <SdFat.h>
#define BUF_LENGTH 513

char filename[13] ="00112233.txt";
byte success;
boolean power;
boolean status_on;

const int chipSelect = 4;



SdFat sd;
SdFile myfile;

void setup (){
  
  Serial.begin (9600);
  
  
  pinMode(chipSelect, OUTPUT);
  Serial.println ("5 seg");
  delay (5000);
  
  boolean end_file = false;
  int bytes_read = 0;
  char buffer [BUF_LENGTH];
  if (sd.begin(chipSelect, SPI_HALF_SPEED)){
    if (myfile.open(filename, O_READ)){

      do{
        end_file = false;	//Starts to send data
        bytes_read = myfile.read (buffer, 512);
        if (bytes_read == 512){
          buffer [512] = '\0';
        }
        else {
          end_file =true;
        }
        
        Serial.print (buffer);
        
        for (int i=0; i<513; i++){ //Restart buffer
          buffer [i] = 0;
        }
        
      } while(end_file == false);
    }
  }
}

void loop (){}

Now the next step is send the data to a FTP server

8 
        for (int i=0; i<513; i++){ //Restart buffer
          buffer [i] = 0;
        }

This is like putting 513 stop signs on one pole. One is sufficient. If not, 513 aren't, either.

        if (bytes_read == 512){
          buffer [512] = '\0';
        }
        else {
          end_file =true;
        }

So, full packets get a NULL on the end. Partial packets do not get NULL terminated. Why is that?

The NULL should, obviously, go in buffer[bytes_read], regardless of the number of bytes read.

9

With this I try to delete all the characters of the buffer:

for (int i=0; i<513; i++){ //Restart buffer
   buffer [i] = 0;
}

I thought that I should make NULL all the characters to correctly delete all stored characters not only put a stop byte. Without this I get partial packets mixed with the data of the previous full packet. I'll try to include a NULL also in this partial packet but my thought was that partial packet get the null from the end of the SD file.

10

FTP client communication requires two ports. One is the command port, the other is the data port. Here is an example of FTP that I use. It will transfer files either way.

11

but my thought was that partial packet get the null from the end of the SD file.

It is highly unlikely that there is a NULL in the file.

I thought that I should make NULL all the characters to correctly delete all stored characters not only put a stop byte. Without this I get partial packets mixed with the data of the previous full packet.

Then you are not getting the NULL in the right place, or you are overwriting it. On the other hand, the failure to NULL terminate the partial packet would explain this too.