Serial Question [SOLVED]

Alright, so I'm working with reading and graphing serial data from the arduino. Everything looks right and I have been able to get the data and graph it. I was appending the serial data to a textarea and then using a Scanner object to break it apart for graphing with JFreeChart.

Here is a video of that
Here is a video of what I'm doing

The problem is that now I'm trying to store the data in an array as it comes in rather than just appending it to a textarea. and it seems like the arduino is sending the serial data in 2 segments for a single Serial.println. The line being sent is

Serial.println(String(ms)+DELIMITER+String(pot));

Serial Event Java Code

/**
	 * Handle an event on the serial port. Read the data and store it.
	 */
	public synchronized void serialEvent(SerialPortEvent oEvent) {
		if (oEvent.getEventType() == SerialPortEvent.DATA_AVAILABLE) {
			try {
				int available = input.available();
				final byte chunk[] = new byte[available];
				input.read(chunk, 0, available);
				System.out.println("SerialEvent - Read : "+new String(chunk));
				//avoid gui synchronization issues on the data area update
				try {
					javax.swing.SwingUtilities.invokeAndWait(new Runnable() {
						public void run() {
							//Append data to textarea on main app window
							ta_dataText.append(new String(chunk)+"|");
							
							//increment position and store data point in array
							i_pos++;
							s_dataStore[i_pos] = new String(chunk);
							//cycle position for next time if maximum data point stores has been met
							if(i_pos == MAX_DATAPOINTS)
								i_pos=0;
						}
					});
				} catch (InterruptedException e) {
					// TODO Auto-generated catch block
					e.printStackTrace();
				} catch (InvocationTargetException e) {
					// TODO Auto-generated catch block
					e.printStackTrace();
				}	
			} catch (Exception e) {
				System.err.println(e.toString());
			}
		}
	}

Console

ArduinoConnection | Connected to Serial
SerialEvent - Read : 7
SerialEvent - Read : 119:174

SerialEvent - Read : 7
SerialEvent - Read : 628:174

SerialEvent - Read : 8
SerialEvent - Read : 137:174

...

I just noticed that if I add an additional println

Serial.println(String(ms)+DELIMITER+String(pot));
 Serial.println("Testing");

it doesn't act like it's really detecting 2 serial events since you only see "Testing" after the data values, not in the middle.

ArduinoConnection | Connected to Serial
SerialEvent - Read : 2
SerialEvent - Read : 068:505
Testing

SerialEvent - Read : 2
SerialEvent - Read : 586:506
Testing

SerialEvent - Read : 3
SerialEvent - Read : 104:505
Testing

...

I have also tried just sending

Serial.println(String(ms));

or

Serial.println(String(pot));

and these values get split up as well - (first char)(the rest), but the String I sent directly (Testing) is not getting divided.

Still trying to figure out what's going on.

If all I send from the arduino Serial is

Serial.println("Testing");

What I get in my SerialEvent function is

SerialEvent - Read : T
SerialEvent - Read : esting

I haven't worked with Serial much before so I think I may just be misunderstanding something here. Is there a reason it breaks off the first character?

Think about how long it takes to send your string, and how long it takes to print results.

Get a character (available == 1), assign it to an array.
The next character hasn't arrived, so no more available.
Print what you've got.
By the time you've finished printing, the rest of the string has arrived.

That's what it looks like to me, but I don't know Processing.

Alright that makes sense and I just tested that.

I changed the Arduino code to this:

 Serial.println("1");
 Serial.println("2");
 Serial.println("3");
 Serial.println("4");
 Serial.println("5");
 Serial.println("6");
 Serial.println("7");
 Serial.println("8");
 Serial.println("9");
 Serial.println("10");

gives me this:

SerialEvent - Read : 1
SerialEvent - Read : 
2
3
4
5
6
7

SerialEvent - Read : 8
9
10

or this (randomly switches between the two, and that timing would explain it, sometimes it processes a little bit quicker

vSerialEvent - Read : 1
SerialEvent - Read : 
2
3
4
5
6
7
8
SerialEvent - Read :
9
10

if I had more "1"s to the first Serial.println("11111"); it moves that middle SerialEvent - Read between 5 and 6. So I guess it takes longer with the first character because it is sending additional serial information on the first read? and then it can only read so many characters before triggering additional serial events to read in the rest.

So I'm thinking I need to send an indicator in my Arduino loop that indicates the end of a transaction after all values are sent (perhaps just a character I don't use, since I'm only sending int values anyway). Would this be the best way to go about this?

Couldn't you use CR and/or LF as your indicator? As a plus, you get those characters when you use Serial.println() instead of Serial.print().

Alright I’ve been having a hell of a time with this. But I finally have it working. Here is the finished class that properly splits the data apart. I would appreciate any constructive criticism as well if anyone has any ideas on how to simplify or improve upon it.

public class ArduinoConnection implements SerialPortEventListener {
	SerialPort serialPort;
        /** The port we're normally going to use. */
	private static final String PORT_NAMES[] = { 
			"/dev/tty.usbserial-A9007UX1", // Mac OS X
			"/dev/ttyUSB0", // Linux
			"COM3", // Windows
			};
	/** the maximum number of previous datapoints to store in memory */
	private static final int MAX_DATAPOINTS = 500;
	/** The delimiter used between milliseconds and units */
	private static final String SERIAL_DELIMITER = ":";
	/** Buffered input stream from the port */
	private InputStream input;
	/** The output stream to the port */
	//private OutputStream output;
	/** Milliseconds to block while waiting for port open */
	private static final int TIME_OUT = 2000;
	/** Default bits per second for COM port. */
	private static final int DATA_RATE = 9600;
	/** Reference to main window's serial data display **/
	private JTextArea ta_dataText;
	
	/** Array to temporarily hold incoming serial data **/
	private byte[] b_tempData;
	/** current position of data being read into b_tempData **/
	private int i_tempDataPos;
	
	/** Array to hold data points **/
	private String[] s_dataStore;
	/** Temporarily hold String Array data for splitting string **/
	private String[] s_temp;
	/** stores the current data point position in the array **/
	private int i_dataPos;
	/** stores the last read data point array position **/
	private int i_lastRead;
	
	public ArduinoConnection(JTextArea ta) {
		ta_dataText = ta;
		
		b_tempData = new byte[50];
		i_tempDataPos = 0;
		
		//initialize position and array size for data points
		s_dataStore = new String[MAX_DATAPOINTS];
		i_dataPos = 0;
		i_lastRead = 0;
	}
	
	public void open() {
		CommPortIdentifier portId = null;
		Enumeration portEnum = CommPortIdentifier.getPortIdentifiers();

		// iterate through, looking for the port
		while (portEnum.hasMoreElements()) {
			CommPortIdentifier currPortId = (CommPortIdentifier) portEnum.nextElement();
			for (String portName : PORT_NAMES) {
				if (currPortId.getName().equals(portName)) {
					portId = currPortId;
					break;
				}
			}
		}

		if (portId == null) {
			System.out.println("Could not find COM port.");
			return;
		}

		try {
			// open serial port, and use class name for the appName.
			serialPort = (SerialPort) portId.open(this.getClass().getName(),
					TIME_OUT);

			// set port parameters
			serialPort.setSerialPortParams(DATA_RATE,
					SerialPort.DATABITS_8,
					SerialPort.STOPBITS_1,
					SerialPort.PARITY_NONE);

			// open the streams
			input = serialPort.getInputStream();
			//output = serialPort.getOutputStream();

			// add event listeners
			serialPort.addEventListener(this);
			serialPort.notifyOnDataAvailable(true);
			
			System.out.println("ArduinoConnection | Connected to Serial");
		} catch (Exception e) {
			System.err.println(e.toString());
		}
	}

	/**
	 * This should be called when you stop using the port.
	 * This will prevent port locking on platforms like Linux.
	 */
	public synchronized void close() {
		if (serialPort != null) {
			serialPort.removeEventListener();
			serialPort.close();
			System.out.println("ArduinoConnection | Disconnected from Serial");
		}
	}

	/**
	 * Handle an event on the serial port. Read the data and store it.
	 */
	public synchronized void serialEvent(SerialPortEvent oEvent) {
		if (oEvent.getEventType() == SerialPortEvent.DATA_AVAILABLE) {
			try {
				int available = input.available();
				final byte chunk[] = new byte[available];
				int numRead = input.read(chunk, 0, available);
				
				//copy this chunk onto the end of temp data store
				System.arraycopy(chunk, 0, this.b_tempData, this.i_tempDataPos, numRead);
				//move temp data position
				this.i_tempDataPos += numRead;
				
				String currentData = new String(b_tempData);
				//If end of line has been reached save this data point and reset to read next point
				if(currentData.contains("\r\n")) {
					s_temp = currentData.split("\r\n");
					
					//store data
					this.s_dataStore[this.i_dataPos] = s_temp[0];
					System.out.println("Data Point Stored = " + this.s_dataStore[this.i_dataPos]);
					
					//increment data point
					this.i_dataPos++;

					//shift any characters after the \r\n in tempData to the beginning and set new position
					if(s_temp.length>1) {
						//remove any whitespace before checking length of overflow chars
						s_temp[1] = s_temp[1].trim();
						if(s_temp[1].length()>0) {
							for(int x = 0; x< s_temp[1].length(); x++) {
								this.b_tempData[x] = (byte)(s_temp[1].charAt(x));
							}
							//set buffer position
							this.i_tempDataPos = s_temp[1].length();							
						} else {
							//reset buffer data
							this.resetBuffer();
						}
					} else {
						//reset buffer
						this.resetBuffer();
					}
				}
				
				//avoid gui synchronization issues on the data area update
				try {
					javax.swing.SwingUtilities.invokeAndWait(new Runnable() {
						public void run() {
							//Append data to textarea on main app window
							ta_dataText.append(new String(chunk)+"|");
						}
					});
				} catch (InterruptedException e) {
					// TODO Auto-generated catch block
					e.printStackTrace();
				} catch (InvocationTargetException e) {
					// TODO Auto-generated catch block
					e.printStackTrace();
				}	
			} catch (Exception e) {
				System.err.println("Err:"+e.toString());
			}
		}
	}
	
	/** reset the b_tempData buffer array **/
	private void resetBuffer() {
		this.i_tempDataPos = 0;
		this.b_tempData = new byte[MAX_DATAPOINTS];
	}