Streaming binary serial data to arduino

Projects Networking, Protocols, and Devices
1

I'm attempting to use my phone to do some audio analysis, and then stream RGB colours as binary (for some LED ribbon) over bluetooth to my arduino. I send 0xFF as a start byte, and then one byte for each colour - 0x01 to 0xFE

For convienience, the Bluetooth is connected with SoftwareSerial, and the PC to the h/w serial, so I can easily get debug output and load new programs. At the moment in testing I'm just printing straight out from the SoftwareSerial.read to Serial.write (which I can obviously see on my PC).

I have it working, but am struggling with losing data. If I put a delay of 50ms per byte in the phone sending side it works OK, but less than that the arduino starts loosing bytes. i.e. if I send 1-255 (as binary), I get the first half or so back onto my PC, but then only about 50% of the rest.

The arduino code is:

void loop() {
      while (!bluetooth.available() || (bluetooth.available() && (bluetooth.read() != 0xFF)));  // wait till we hit 0xFF
      while (bluetooth.available() < 3); // Wait until we have 3 bytes waiting
      bluetooth.readBytes(buffer,3);
      Serial.write(buffer);
      set(left, buffer);
}

Obviously a buffer somewhere is overflowing, but it seems incredibly slow. At 9600baud (i.e. bits/sec), I should be able to do 1200bytes/sec. But requiring a 50ms delay from the phone, that's only 20 bytes/sec! Is the SoftwareSerial really this slow, or is something else going on?!

2

I have it working, but am struggling with losing data. If I put a delay of 50ms per byte in the phone sending side it works OK, but less than that the arduino starts loosing bytes. i.e. if I send 1-255 (as binary), I get the first half or so back onto my PC, but then only about 50% of the rest.

First: you should post complete code, with just snippets were condemned to make some assumptions which might be correct but often are not.

The first half does mean the first two bytes? The problem with the SoftwareSerial class is that is uses a lot of resources of the processor and is not very reliable with speeds above 4800 baud, especially if other interrupt sources are active (such as the serial interface in your case).

3

Sorry here's the rest of it:

#include <Arduino.h>
#include <SoftwareSerial.h>
#include "lib/Tlc5940/Tlc5940.h"

int bluetoothTx = 6;
int bluetoothRx = 7;

extern HardwareSerial Serial;

SoftwareSerial bluetooth(bluetoothTx, bluetoothRx);
char buffer[3];
int left[] = {13, 14, 12};
int right[] = {29, 30, 28};

void setup()
{
  Tlc.init();
  Tlc.setAll(4095);
  Tlc.update();

  //Setup usb serial connection to computer
  Serial.begin(57600);

  //Setup Bluetooth serial connection to android
  bluetooth.begin(115200);
}

void set(int channel[3], char buffer[3]) {
  Tlc.set(channel[0], 4095);
  Tlc.set(channel[1], 4095);
  Tlc.set(channel[2], 4095 - 16*buffer[2]);
  Tlc.update();
}

void loop() {
      while (!bluetooth.available() || (bluetooth.available() && (bluetooth.read() != 0xFF)));  // wait till we hit 0xFF
      while (bluetooth.available() < 3); // Wait until we have 3 bytes waiting
      bluetooth.readBytes(buffer,3);
      Serial.write(buffer);
      set(left, buffer);
}

And the Java from Android (mostly not mine) is:

package com.example.bluetoothtest;

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Set;
import java.util.UUID;

import android.app.Activity;
import android.bluetooth.BluetoothAdapter;
import android.bluetooth.BluetoothDevice;
import android.bluetooth.BluetoothSocket;
import android.content.Intent;
import android.os.Bundle;
import android.os.Handler;
import android.util.Log;
import android.view.View;
import android.widget.Button;
import android.widget.EditText;
import android.widget.TextView;

public class MainActivity extends Activity
{
    TextView myLabel;
    EditText myTextbox;
    BluetoothAdapter mBluetoothAdapter;
    BluetoothSocket mmSocket;
    BluetoothDevice mmDevice;
    OutputStream mmOutputStream;
    InputStream mmInputStream;
    Thread workerThread;
    byte[] readBuffer;
    int readBufferPosition;
    int counter;
    volatile boolean stopWorker;
    
    @Override
    public void onCreate(Bundle savedInstanceState)
    {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        
        Button openButton = (Button)findViewById(R.id.open);
        Button sendButton = (Button)findViewById(R.id.send);
        Button closeButton = (Button)findViewById(R.id.close);
        myLabel = (TextView)findViewById(R.id.label);
        myTextbox = (EditText)findViewById(R.id.entry);
        
        //Open Button
        openButton.setOnClickListener(new View.OnClickListener()
        {
            public void onClick(View v)
            {
                try 
                {
                    findBT();
                    openBT();
                }
                catch (IOException ex) { }
            }
        });
        
        //Send Button
        sendButton.setOnClickListener(new View.OnClickListener()
        {
            public void onClick(View v)
            {
                try 
                {
                    sendData();
                }
                catch (IOException ex) { } catch (Exception e) {
					// TODO Auto-generated catch block
					e.printStackTrace();
				}
            }
        });
        
        //Close button
        closeButton.setOnClickListener(new View.OnClickListener()
        {
            public void onClick(View v)
            {
                try 
                {
                    closeBT();
                }
                catch (IOException ex) { }
            }
        });
    }
    
    void findBT()
    {
        mBluetoothAdapter = BluetoothAdapter.getDefaultAdapter();
        if(mBluetoothAdapter == null)
        {
            myLabel.setText("No bluetooth adapter available");
        }
        
        if(!mBluetoothAdapter.isEnabled())
        {
            Intent enableBluetooth = new Intent(BluetoothAdapter.ACTION_REQUEST_ENABLE);
            startActivityForResult(enableBluetooth, 0);
        }
        
        Set<BluetoothDevice> pairedDevices = mBluetoothAdapter.getBondedDevices();
        if(pairedDevices.size() > 0)
        {
        	
            for(BluetoothDevice device : pairedDevices)
            {
            	Log.v("BT2", "Device: " + device.getName());
                if(device.getName().equals("BT UART")) 
                {
                    mmDevice = device;
                    break;
                }
            }
        }
        myLabel.setText("Bluetooth Device Found");
    }
    
    void openBT() throws IOException
    {
        UUID uuid = UUID.fromString("00001101-0000-1000-8000-00805f9b34fb"); //Standard SerialPortService ID
        mmSocket = mmDevice.createRfcommSocketToServiceRecord(uuid);        
        mmSocket.connect();
        mmOutputStream = mmSocket.getOutputStream();
        mmInputStream = mmSocket.getInputStream();
        
        beginListenForData();
        
        myLabel.setText("Bluetooth Opened");
    }
    
    void beginListenForData()
    {
        final Handler handler = new Handler(); 
        final byte delimiter = 10; //This is the ASCII code for a newline character
        
        stopWorker = false;
        readBufferPosition = 0;
        readBuffer = new byte[1024];
        workerThread = new Thread(new Runnable()
        {
            public void run()
            {                
               while(!Thread.currentThread().isInterrupted() && !stopWorker)
               {
                    try 
                    {
                        int bytesAvailable = mmInputStream.available();                        
                        if(bytesAvailable > 0)
                        {
                            byte[] packetBytes = new byte[bytesAvailable];
                            mmInputStream.read(packetBytes);
                            for(int i=0;i<bytesAvailable;i++)
                            {
                                byte b = packetBytes[i];
                                if(b == delimiter)
                                {
                                    byte[] encodedBytes = new byte[readBufferPosition];
                                    System.arraycopy(readBuffer, 0, encodedBytes, 0, encodedBytes.length);
                                    final String data = new String(encodedBytes, "US-ASCII");
                                    readBufferPosition = 0;
                                    
                                    handler.post(new Runnable()
                                    {
                                        public void run()
                                        {
                                            myLabel.setText(data);
                                        }
                                    });
                                }
                                else
                                {
                                    readBuffer[readBufferPosition++] = b;
                                }
                            }
                        }
                    } 
                    catch (IOException ex) 
                    {
                        stopWorker = true;
                    }
               }
            }
        });

        workerThread.start();
    }
    
    void sendData() throws IOException, Exception {
    	for (int i = 1; i < 255; i=+3) {
    		mmOutputStream.write(0xFF);
    		mmOutputStream.write(i);
	    	mmOutputStream.write(i+1);
	    	mmOutputStream.write(i+2);
	    	Thread.sleep(50);
    	}
   	
    	
    }
    
    void closeBT() throws IOException
    {
        stopWorker = true;
        mmOutputStream.close();
        mmInputStream.close();
        mmSocket.close();
        myLabel.setText("Bluetooth Closed");
    }
}

Originally I thought it was because I was missing closing null bytes on the transmission (as advised on another forum thread), but that made no difference (I think my arduino code was wrong then), so I've taken them out again. The 1st half (approx half anyway) means the numbers 1-100, after that the numbers start skipping. e.g I'll get 0x64, 0x65, 0x69, 0x72 etc on the serial monitor.

The obviousy problem is an Uno only has one h/w serial, so it's tricky to debug. Maybe I should swap the bluetooth onto the h/w and the PC on the software one and see if it improves...

I've just melted my bluetooth adapter somehow as well(!), so I'll have to wait for another one to arrive now! Smoke is probably a bad sign :frowning:

4

The SoftwareSerial::available() method is not a boolean function. It returns the number of bytes available to be read. Using it as a boolean is poor practice, and makes it harder to understand what your code is doing.

      bluetooth.readBytes(buffer,3);
      Serial.write(buffer);

Read three bytes. Send one. Why?

The array returned by readBytes() is not NULL terminated. When sending that array, you need to define the length, too. Otherwise, only one is sent.

5

I've seen a lot of code that uses serial.available() like that, it's even in the example sketch that comes with the SoftwareSerial library!

Serial.write(buffer) writes the whole buffer - all 3 bytes to the output. It's defined in the start as being 3 bytes - char buffer[3];

I'm pretty sure the problem is in the buffers overflowing coming from the phone, rather than with the output being sent to the PC. It might just be that SoftwareSerial is slower than I thought. But even at 4800 baud, that's still 600bytes/sec, which is still much faster than I'm currently starting to see bytes dropped at.

6

I've seen a lot of code that uses serial.available() like that, it's even in the example sketch that comes with the SoftwareSerial library!

That makes it neither right or preferred.

Serial.write(buffer) writes the whole buffer - all 3 bytes to the output. It's defined in the start as being 3 bytes - char buffer[3];

You've verified that this ALWAYS works, sending exactly 3 bytes - never more, never less - how?

Serial.write(buffer, 3);

Will send exactly 3 bytes every time. Never more, never less. And, it's the proper way to send non-NULL-terminated arrays.

I'm not saying that either issue is the problem, but when I see questionable tactics being used, and it's trivial to fix them and, therefore, rule out those possibilities, it's worth the effort to make the changes.

7

I'm afraid
Serial.write(buffer,3);
doesn't work. When building (on Arduino 1.03) it returns:

main.cpp:50: error: invalid conversion from 'char*' to 'const uint8_t*'
main.cpp:50: error:   initializing argument 1 of 'virtual size_t Print::write(const uint8_t*, size_t)'

Should I do instead:

char buffer[4];
bluetooth.readBytes(buffer,3);
buffer[4] = NULL;
Serial.write(buffer);

Instead?

8

OK, changing it to "byte" from "char" works OK.

i.e.

char buffer[3];
Serial.write(buffer,3);

fails, but:

byte buffer[3];
Serial.write(buffer,3);

works.

I'll change the code and test it when I get my new bluetooth module, (I somewhat melted the old one somehow!).

I'm still not sure why it is so slow though. Even at 4800bps it should not require a 50ms delay from android after each byte?

9

Arghh but now the read is broken!

      char buffer[3];
      Serial.readBytes(buffer,3);
      Serial.write(buffer,3);

gives error:

main.cpp:44: error: invalid conversion from 'char*' to 'const uint8_t*'
main.cpp:44: error:   initializing argument 1 of 'virtual size_t Print::write(const uint8_t*, size_t)''

on the Serial.write() line.

And:

      byte buffer[3];
      Serial.readBytes(buffer,3);
      Serial.write(buffer,3);

gives error:

main.cpp:43: error: invalid conversion from 'byte*' to 'char*'
main.cpp:43: error:   initializing argument 1 of 'size_t Stream::readBytes(char*, size_t)'

It seems there is no way to read and write a fixed size buffer!

Anyone have any ideas?!

10

It seems there is no way to read and write a fixed size buffer!

Sure there is. One function takes a char array. The other takes a byte array. Both a byte and a char are the same size, so if you have one that you want the function that takes the other type to use, lie to the function, telling it that the array IS the right type, using a cast.

      char buffer[3];
      Serial.readBytes(buffer,3);
      Serial.write((const uint8_t *)buffer,3);