Android to Arduino BLE file transfer

Hi, i am doing a project where on the android, u send a file/image to the Arduino, which stores it in the SD card.

On the android app, Im sending byte by byte of the file, converted to hex, to the arduino.
The arduino receives the hex chars, and convert it back to bytes and write to sdcard(via sdfat library)

The problem is, the transfer speed is way too slow.

500bytes takes 30secs.
60kb takes more than an hour!

Are there any other way to do it quicker?

I know u can send the raw bytes instead, but i need a human readable format so i can debug which part of the datastream went wrong, and why.

You need a protocol for it. Post your code here. I am sure u used delay in your code. You are slowing the data stream yourself for it. There is no short answer to your problem. But the main idea is , arduino can accept and process a curtain amount of data at a time. Due to this limitation, you have to specify a transfer protocol. For example if you want to send a 1 word string that has 1000 character, you cannot send all of the data and record it to sd card. What you do is , send data 20 character at a time by adding 2 character at the beginning and ending of that 20 character. Call this a batch. There will be 50 batch you have to send. Do this in for loop at android side and wait for a signal from arduino. At arduino side, use datareceived event and define a string variable that stores batches. When batch is processed by arduino send a signal to android device and so on. Your problem is overflowing the serial buffer. You need a protocol. And you need to increase baud rate if you are doing the transfer at 9600. We cannot help without the code and time. This is not a rookie job. You will struggle a lot.

I am sure u used delay in your code.

Nope, I dont use delays when sending or receiving the hex string.

I have flags in the android app, when a byte(hex representation) is sent out, android will only send the next byte when the Arduino responses with the same hex string. this is to ensure there are no data loss. about 1 out of 50 messages sent by android over BLE will somehow be lost. by lost, i mean android debug shows it is sent out, but arduino does not receive anything.

For example if you want to send a 1 word string that has 1000 character, you cannot send all of the data and record it to sd card. What you do is , send data 20 character at a time by adding 2 character at the beginning and ending of that 20 character. Call this a batch. There will be 50 batch you have to send. Do this in for loop at android side and wait for a signal from arduino.

yeah, the serial buffer will be full and start discarding.

currently, my Android app sends 1 byte(2 hex character) at a time.
Once Arduino receives it, writes to sd card, and reply Android with the same 2 hex character to indicate it has received it.

perhaps i should increase the message sent to 30characters(half of serial buffer).

is it really necessary to wrap each ‘batch’?

you need to increase baud rate if you are doing the transfer at 9600.

I am using 115200 for my Teensy++ 2.0 board. believe that is the max the BLE module(hm-10, grove BLE) can handle reliably

void writeToFileHex(){
	byte recoverByte;
	char s0,s1;
	sd.remove("hexdump.bmp"); //remove existing file first
	myFile = sd.open("hexdump.bmp", FILE_WRITE);
	Serial.println("\nReceiving HexDump, terminates with '|': ");
	HWSERIAL.print("q");Serial.println("\nSent q to Android."); //tell Android to send first byte
	
	if (myFile) {
		Serial.print("Writing HexDump to hexdump.bmp...");
		Serial.println("\nReceived HexDump: ");
		while( s0 != '|' && s1 != '|' ){
			
			if(HWSERIAL.available() >= 2) {
				
				s0=HWSERIAL.read();
				s1=HWSERIAL.read();
				if(s0 == '|' || s1 == '|'){
					HWSERIAL.print("t"); 
					Serial.print("| received. Sent t. break.");
					break;
				}
				
				HWSERIAL.print(s0);HWSERIAL.print(s1); //send same hex chars back to android
				Serial.print(s0);Serial.print(s1);
			

                                //convert the received hex char back to byte  
				recoverByte = (convertFromHex(s0) << 4) | (convertFromHex(s1));
				myFile.write(recoverByte);
				
			}


//the following is for when message sent by android is lost, tells android to resent previous message
			if(HWSERIAL.available()==0){
				Timer = millis();
				while(HWSERIAL.available() == 0){
					//HWSERIAL.print("w");Serial.print("w");//tell Android to send next byte
					//Serial.println("s");
					//if no response from android by 4 milliseconds, send another message.
					if( millis() - Timer > 250){
						if(s0 == '|' || s1 == '|'){
							Serial.print("millis | break.");
							break;
						}	
						
						HWSERIAL.print("s");Serial.println("\nno response, sent s");//tell Android to send next byte
						Serial.print(s0);Serial.print(s1);
						Timer = millis();
						
					}
				}
			}else{
				Timer = 0;
			}
			

			
		}//while( s0 != '|' && s1 != '|' )

		
	// close the file:
		myFile.close();
		Serial.println("\ndone.");
	} 
	else {
		// if the file didn't open, print an error:
		Serial.println("error opening hexdump.bmp");
	}
}

Android code

public void bytesToHex(byte[] in) {
        final StringBuilder builder = new StringBuilder();
        int i=0;
        sendingData = true;
        Log.d("byteToHex", "sendingData = true, start sending data.");
        sendSerial("w"); //write command
        Log.d("byteToHex", "sending w");
        for(byte b : in) {

            while(!newData){//wait for response from Arduino, newData == true 
                if(resendData == true){ //timer on Arduino up, arduino sent command req for resend of prev hex chars
                    sendSerial(sendByte);
                    Log.d("byteToHex", "resendData = true, resending: " + sendByte);
                    resendData = false; //reset resendData flag
                }
            } 


            i++;
            sendByte = String.format("%02x", b); //send next byte(hex representation) if got response from Arduino
            sendSerial(sendByte);
            newData = false;
            Log.d("byteToHex", "newData = false");
            Log.d("byteToHex", "Sent " + i + " byte");


        }
        sentTerminator = true; //flag to check if terminator is received on mcu
        sendSerial("||"); //terminating command, tell teensy last hex has been sent
        while(sentTerminator == true){ //while terminator not yet received
            if(resendTerminator == true){ //
                sendSerial("||");
                Log.d("byteToHex", "resending terminator");
                resendTerminator = false; //Resend complete. reset resendTerminator flag.
            }
        }
        sendingData = false;
        
    }