Please help - Serial over bluetooth (HC-06)

Hello

First please could I say that after two years of great fun with Arduino's this is my first post here on the forums! Shameful really, but thanks for the help I've had/read over that time.

Project overview:
So, rather than be cryptic about what I'm trying to achieve, I'll start by saying that I'm building a digital scoreboard for my cricket team. The idea is that I will have a shift register controlling an array of LED's for each digit on the scoreboard, and interface with these via an Android app over bluetooth using (HC-06).

Prototype details:
At the moment I have a buggy but working Android app (it's only a prototype, I'll gut it and start a-fresh when I've got things working properly) and a simple, yet also inefficient Arduino sketch for testing.
Because I've not ordered the shift registers yet, or the rest of the materials for creating the digit displays for that matter, I am instead just flashing a set of 6 LED's. For example, if I were to set the score to 100 (for arguments sake, LED 2) then the corresponding LED would simply flash 100 times when it was called in the flash loop.

Problem description:
In step by step form:

  • Turn Arduino on with battery and bluetooth, no wired connection, HC-06 flashes while waiting for connection, LED's flash once each (twice actually, need to figure out why sometime) in a continuous loop
  • Android app connects to HC-06, initializes deviceSocket, HC-06 light becomes constant
  • Android app sends data to Arduino, led loop hangs while the Serial.read() commences
  • Arduino continues without change, no change in number of flashes for any of the LED's

What I'm trying to do is send the following byte array from the app

'{','c','}',pinNumber,pinValue

. pinNumber being the pin that I want to affect, and pinValue being the number of flashes to do (I recognize this means I cannot input a score over 127, please advise on how I could overcome this? :slight_smile: )

What should have happened after the data was transmitted was that pin2 should've flashed 60 times, while all others remained on 1 flash.

I'm finding it hard to track down my issue as it is very difficult to get any kind of idea what bytes are arriving at the Arduino when the app is communicating with it. I am still considering writing a C# windows app on my computer to display the received bytes from the Android app and will do this if we can't figure it out :slight_smile:

Code:

Arduino:

#define NUM_PINS 6
#define PIN_START 2
#define BUFFER_LEN 3
#define BAUD_RATE 9600

class Pin {


private:

	int id;
	float pause = 30;
	float flashTime = 1000;
	int value = 0;

public:

	void assignValue(int val)
	{
		value = val;
	}
	void initialize(int i)
	{
		id = i;
		pinMode(id, OUTPUT);
		activate();
	}
	void activate()
	{
		flash(id, value);
	}
};
class Communicator {

private:
	char buffer[BUFFER_LEN];

public:
	
	int lastBytes[2] = { -1,0 };

	void initializeSerial(int baud) 
	{
		Serial.begin(baud);
	}
	void getPinData()
	{

		if (Serial.available() > 0)
		{
			if (Serial.read() == '{')
			{
				if (Serial.read() == 'c')
				{
					if (Serial.read() == '}')
					{
						int pinNum = Serial.parseInt();
						int pinVal = Serial.parseInt();

						lastBytes[0] = pinNum;
						lastBytes[1] = pinVal;
					}
				}
			}
			else
			{
				while (Serial.peek() != '{')
				{
					Serial.read();
				}
			}
		}

	}
};

Pin *pins = new Pin[6];
Communicator *comm = new Communicator();

void setup()
{
	comm->initializeSerial(BAUD_RATE);

	// Debug LED pin
	pinMode(13, OUTPUT);
	// Setup input pins (not sure this is actually necessary)
	pinMode(0, INPUT);

	for (int i = 0; i < NUM_PINS; i++) {

		int v = i + PIN_START;
		pins[i].assignValue(1);
		pins[i].initialize(v);

	}

}

void loop()
{
	// Get data from serial
	comm->getPinData();

	// If we were assigned pin data for a specific pin, assign it
	if (comm->lastBytes[0] > 0)
	{
		pins[comm->lastBytes[0]].assignValue(comm->lastBytes[1]);
	}

	for (int i = 0; i < NUM_PINS; i++)
	{
		pins[i].activate();
	}

}

void flash(int pin, int numOfFlashes)
{
	for (int i = 0; i <= numOfFlashes; i++) {

		delay(50);
		digitalWrite(pin, HIGH);
		delay(50);
		digitalWrite(pin, LOW);
	}
}

Android (Only relevant transmit thread):

public class sendDataThread implements Runnable{

    OutputStream oStream;
    byte[] buffer;

    public void run(){
        Log.d("Debug", "Thread started with: " + buffer[0] + " and " + buffer[1] + " then " + buffer[2] + " ends with: " + buffer[3]);
        try {

            //TODO: Un-fuck this bit
            oStream.write(buffer);
        } catch (IOException e) {

        }
    }

    public sendDataThread(byte p, byte v, OutputStream os){

        // Construct the buffer to transmit, we start with '{c}' to tell the Arduino to start listening.
        // The buffer therefore looks something like '{c},2,128'
        buffer = new byte[]{'{','c','}',p,v};
        Log.d("GTStudios",buffer.toString());
        oStream = os;
    }
}

Thank you all in advance, sorry if this is a rather long post but I wanted to help you help me as much as possible :slight_smile:

I recognize this means I cannot input a score over 127, please advise on how I could overcome this?

Bytes are unsigned, so they range from 0 to 255.

		if (Serial.available() > 0)
		{
			if (Serial.read() == '{')
			{
				if (Serial.read() == 'c')
				{
					if (Serial.read() == '}')

No! No! No!

If there is one byte available to read, it is NOT okay to read more than one.

Once you read one byte, and find that it is a '{', the next read will fail, and the result will not be a 'c'.

When that happens, you are out of sync.

Sending a start marker, the two values, and an end marker makes more sense than sending three start markers. Sending all ASCII data makes more sense than sending a mix of ASCII and binary data.

Find Robin2's Serial Input Basics thread, and read it, post-haste.

Thanks for the reply Paul,

No! No! No!

If there is one byte available to read, it is NOT okay to read more than one.

Interesting, from my reading I thought that Serial.read() would remove the byte from the buffer, therefore the next read being the next byte - thanks for clearing this up. Please could you provide a very basic (psuedo code is fine) example of how I should structure it?

Find Robin2's Serial Input Basics thread, and read it, post-haste.

I've found it and will read it now, thanks letting me know about this :slight_smile:

I thought that Serial.read() would remove the byte from the buffer, therefore the next read being the next byte

It does.

Suppose you have a drawer full of socks that are not matched up. You pull out a sock. It's red with a blue toe. You pull out another sock. It's green, with a pink toe. No match. What do you do next?

What your code is doing is throwing both socks into the shredder.

Please could you provide a very basic (psuedo code is fine) example of how I should structure it?

Robin2's thread covers the topic better than I could in a couple of paragraphs.

One of my concerns is in how best to send the data. For example if I used < and > to mark the beginning and end of my data, what if the score was 62? This would be a problem because in the ascii table, '>' is also 62, and so my transmission would end early.

Do you have any thoughts on how best to counter this?

Perhaps I could use a fixed size transmission beginning and ending with < and >, store everything received after and including the first '<', then check the last digit was '>' and if so, process the bytes inbetween?

One of my concerns is in how best to send the data. For example if I used < and > to mark the beginning and end of my data, what if the score was 62? This would be a problem because in the ascii table, '>' is also 62, and so my transmission would end early.

Now, suppose that you sent "<13,62>". Could you figure out, from that, which pin to flash and how many times?

From the human eye yes, but would that not be sent as: '60 13 44 62 62'? (In which case, perhaps we missed the first digit and it ended earlier than expected?)

I may be over thinking this, in which case please feel free to say so!

I may be over thinking this, in which case please feel free to say so!

You may be. Or, you may be under thinking it.

Since the only valid characters that make up the packet are '0' to '9' and ',', there is NO way to mistake a value in the packet for a '<' or '>'.

Ok, so I've made some changes and it is running much more reliably (it actually recognizes when a compatible set of bytes have been sent).

However now I'm finding that parseInt is returning 0 every time... (I'm using the serial viewer... wonder if it is sending ASCII values?). I understand that 0 is the default return of parseInt if it doesn't find anything worthwhile, I'm more just curious as to why this is the case?

Edit: When transmitting from the android app the byte array is not recognized as compatible, I guess this is the 'true' representation of what I was trying to achieve with the serial monitor.

Within the android app the byte array is declared as:

 byte[] buffer = new byte[] {'<',(byte)2,',',(byte)3,',',(byte)0,'>'};

Would parseInt() not recognize bytes appearing in the buffer like this?

class Communicator {

private:
	static const char startData = '<';
	static const char endData = '>';
	static const char dividor = ',';

	// The variable that holds our pin instructions
	int instruction[2] = { -1,0 };

	// Flag for whether data was received
	bool receivingData = false;
	bool hasInstruction = false;
	// index as to where we are storing data in buffer
	int bufferIndex = 0;
	// Buffer that stores our received data
	byte buffer[BUFFER_LEN];

	bool confirmData()
	{
		bool result = false;
		if ((char)buffer[0] == startData && (char)buffer[BUFFER_LEN - 1] == endData)
		{
			result = true;
		}
		// Clean up the buffer
		for (int i = 0; i < BUFFER_LEN; i++)
		{
			buffer[i] = 0;
		}
		// Return false, no message received
		resetInstruction(result);

		// DEBUG
		if (result)
		{
			flash(13, 5, 100);
			/*pauseLED(13, 100);
			flash(13, instruction[1], 200);
			pauseLED(13, 100);
			flash(13, getInstruction()[2]);
			pauseLED(13, 100);
			flash(13, getInstruction()[3]);*/
		}


		return result;
	}

	// Method resets or assigns new instruction
	void resetInstruction(bool useNew)
	{
		// Check for hard reset
		if (useNew)
		{
			// Set instruction to value pulled from bytes
			instruction[0] = buffer[1];
			instruction[1] = buffer[2] + buffer[3];
		}
		else
		{
			// Reset instruction to default state
			instruction[0] = -1;
			instruction[1] = 0;
		}
	}

public:

	void initializeSerial(int baud) 
	{
		Serial.begin(baud);
	}
	bool getHasInstruction() {
		return hasInstruction;
	}
	int* getInstruction() 
	{
		hasInstruction = false;
		return instruction;
	}
	void gatherPinData()
	{

		// Loop works out when transmission started
		if (Serial.available() > 0 && !receivingData)
		{
			if ((char)Serial.peek() == startData)
			{
				receivingData = true;
				buffer[bufferIndex] = Serial.read();
				bufferIndex++;
			}
			else
			{
				Serial.read();
			}
		}
		else
		{
			// Loop gathers transmission data
			if (receivingData)
			{
				while (bufferIndex < BUFFER_LEN)
				{

					if (Serial.peek() == dividor || Serial.peek() == endData)
					{
						buffer[bufferIndex] = Serial.read();
					}
					else
					{
						buffer[bufferIndex] = Serial.parseInt();
					}

					bufferIndex++;
				}

				// Now that we've gathered the rest of the transmission, we need to cleanup
				bufferIndex = 0;
				receivingData = false;

				hasInstruction = confirmData();
			}
			else
			{
				Serial.read();
			}
		}
	}

};

I'm more just curious as to why this is the case?

What is the integer value of the string above? What other value would you suggest as the default? Maybe 452624852578?

Why are you sending binary data and expecting ASCII data?

PaulS:
Now, suppose that you sent "<13,62>". Could you figure out, from that, which pin to flash and how many times?

Guy123:
From the human eye yes, but would that not be sent as: '60 13 44 62 62'? (In which case, perhaps we missed the first digit and it ended earlier than expected?)

It depends on how you send this; if it would be pure ascii, the result is 60d 49d 51d 44d 54d 50d 62d (d indicates decimal values) and there will not be any confusion possible. Wait for '<', read up to '>'. Extract the actual pin and pinvalue and convert the numbers.

If your message contains binary data, you need to define another protocol. You can add a length in the message indicating how long the actual data is and you can add a checksum or crc to validate the received data. Based on that you can decide if you have received a full message and if it is valid.

Implementing a timeout is advisable in both scenarios.

You seem to be creating a class and as I do not really do C++ stuff I can not help further.

Thanks for replying sterretje!

So because I've decided to send data via binary I've settled on a fixed length of 7 bytes, which may for example look like:

 '<2,20,0>'

I'm having an issue where Serial.parseInt(); is returning '0' (not finding anything), is this because it is trying to parse bytes and should be looking at chars?

If you send binary, there is no reason to parse (as PaulS already indicated). If the three numbers are indeed numbers (bytes), your code could look something like

void loop()
{
  static byte data[7];
  static byte index = 0;

  if (Serial.available())
  {
    // read data byte
    data[index] = Serial.read();

    // first byte should be start marker
    if (index == 0 && data[index] != '<')
    {
      // bail out
      return;
    }

    // 2nd and 4th byte must be comma
    if ((index == 2 || index == 4) && data[index] != ',')
    {
      // clear the buffer
      memset(data, 0, sizeof(data));
      // reset index
      index = 0;
      // bail out
      return;
    }

    // last byte must be end marker
    if (index == sizeof(data) - 1 && data[index] != '>')
    {
      // clear the buffer
      memset(data, 0, sizeof(data));
      // reset index
      index = 0;
      // bail out
      return;
    }

    // increment index so next data byte goes to next position in array
    index++;
  }

  // if not all bytes received
  if (index != sizeof(data))
  {
    // return and wait for next byte
    return;
  }

  // OK, got 7 bytes, extract the data
  value1 = data[1];
  value2 = data[3];
  value3 = data[5];

  // cleanup
  index = 0;
  memset(data,0,sizeof(data));
}

Not tested.

The above waits for '<' and only starts populating the data array if that is found. You probably want to build a timeout around it so the code does not wait forever if you missed '<' and one of the data bytes actually is '<'.

By the way, commas in your protocol do not really make sense.

Thanks again S,

By the way, commas in your protocol do not really make sense.

Aye I agree, I guess that is the problem with chopping and changing your protocol so frequently. I'll remove them when I come to fine tuning.

Do you do much java? I'd be keen to run my byte array code by you if so, to make sure I'm not being retarded, I'm quite new to Java (ironically, I learned C++ first and spend most of my time in C#).

I'm now getting an issue where the Bluetooth receiver isn't receiving anything from the phone, but I think that was due to some refactoring I did on the app, I'll fix that on my own :slight_smile:

Thanks again

No java. Assembly, C, C# and a little C++; the latter since I started with the Arduino. And a couple of scripting languages.

@sterretje: Assembly! you sadist :grin: (I'm jealous really)

Ok so I got it working :sunglasses: although by my own admittance, I gave up trying to get raw bytes across the bluetooth-serial connection and just settled for letting java deconstruct a string, pass it across the comm and then parse the ints in the arduino. This approach hurts my perfectionism, there should be no need to use Strings to transmit scalar data but there we go (actually makes transmitting numbers over 128 easier but I won't admit it in public).

Here's the working arduino code:

#define NUM_PINS 6
#define PIN_START 2
#define PROTOCOL_LEN 2
#define BAUD_RATE 9600

class Pin {


private:

	int id;
	float pause = 30;
	float flashTime = 1000;
	int value = 0;

public:

	void assignValue(int val)
	{
		value = val;
	}
	void initialize(int i)
	{
		id = i;
		pinMode(id, OUTPUT);
		activate();
	}
	void activate()
	{
		flash(id, value, 200);
		assignValue(0);
	}
};
class Communicator {

private:
	static const char startData = '<';
	static const char endData = '>';

	// The variable that holds our pin instructions
	int instruction[2] = { -1,0 };
	// Holds a temp array for us to shit-check
	int tempParsedInts[PROTOCOL_LEN] = { 0,0 };

	// Flag for whether data was received
	bool receivingData = false;
	bool hasInstruction = false;


	bool confirmData()
	{
		bool result = false;
		if (tempParsedInts[0] > 0)
		{
			result = true;
		}

		// push result to instruction data
		resetInstruction(result);

		return result;
	}

	// Method resets or assigns new instruction
	void resetInstruction(bool useNew)
	{
		// Check for hard reset
		if (useNew)
		{
			// Set instruction to value pulled from bytes
			instruction[0] = tempParsedInts[0];
			instruction[1] = tempParsedInts[1];

			// Notify anyone checking that we have a new instruction.
			hasInstruction = true;
		}
		else
		{
			// Reset instruction to default state
			instruction[0] = -1;
			instruction[1] = 0;

			// Make sure we don't have an instruction queued
			hasInstruction = false;
		}
	}

public:

	void initializeSerial(int baud) 
	{
		Serial.begin(baud);
	}
	bool getHasInstruction() {
		return hasInstruction;
	}
	int* getInstruction() 
	{
		hasInstruction = false;
		return instruction;
	}
	void gatherPinData()
	{
		receivingData = false;
		if (Serial.available() > 0)
		{
			// Check for initiator
			if (Serial.peek() == startData && !receivingData)
			{
				Serial.read();
				receivingData = true;
			}
			else if (!receivingData)
			{
				Serial.read();
			}

			// If initiator was 'initialized' then read
			if(receivingData)
			{
				// Begin parsing ints, we know our protocol will provide three ints followed by a '>'
				for (int i = 0; i < PROTOCOL_LEN; i++)
				{
					tempParsedInts[i] = Serial.parseInt();
				}
				// Only 'check data' if we received a proper transmission.
				if (Serial.read() == endData)
					confirmData();

				// Either way we are done checking
				receivingData = false;
			}
		}
	}

};

Pin *pins = new Pin[6];
Communicator *comm = new Communicator();

void setup()
{
	comm->initializeSerial(BAUD_RATE);

	// Debug LED pin
	pinMode(13, OUTPUT);
	// Setup input pins (not sure this is actually necessary)
	pinMode(0, INPUT);

	for (int i = 0; i < NUM_PINS; i++) {

		int v = i + PIN_START;
		pins[i].assignValue(0);
		pins[i].initialize(v);

	}

}

void loop()
{
	// Get data from serial
	comm->gatherPinData();

	// If we were assigned pin data for a specific pin, assign it
	if (comm->getHasInstruction())
	{
		int* instruction = comm->getInstruction();
		pins[instruction[0]].assignValue(instruction[1]);
	}

	for (int i = 0; i < NUM_PINS; i++)
	{
		pins[i].activate();
	}

}

void flash(int pin, int numOfFlashes)
{
	for (int i = 0; i < numOfFlashes; i++) {

		delay(50);
		digitalWrite(pin, HIGH);
		delay(50);
		digitalWrite(pin, LOW);
	}
}
void flash(int pin, int numOfFlashes, int pause)
{
	for (int i = 0; i < numOfFlashes; i++) {

		delay(pause);
		digitalWrite(pin, HIGH);
		delay(pause);
		digitalWrite(pin, LOW);
	}
}
void pauseLED(int pin, int pause)
{
	digitalWrite(pin, HIGH);
	delay(pause);
	digitalWrite(pin, LOW);
}

It's quite specific to my goals, but if anyone reading this wants to use all or any of the code - feel free!

there should be no need to use Strings to transmit scalar data

There isn't, IF:

  1. the integers are the same size on both ends
  2. the byte order is the same (or you are aware that the byte order is different and you manage that
  3. you can be absolutely certain that no byte ever gets lost.

The last one will kill you, since serial data DOES get lost/mangled.

  1. you can be absolutely certain that no byte ever gets lost.

Yeah I was/still am genuinely concerned about this as the final application of this product will be outside. I feel I've considered the incoming data in such a way that will account for this.

Honestly in hindsight I really do think that a formatted String is the best way of transmitting data :sob: