Sending data from Processing to arduino with handshake

I currently have processing code that reads the input from a usb flight stick then turns all the values into integers.( there are 13 variables) My issue comes from getting those 13 values sent to an arduino mega. I have experimented with trying to send a single value at a time, but I'm not sure how to "shake hands" between the two, so that data is sent to the arduino, then the arduino sends a request for the next piece, until all the values are sent. I'm lost when it comes to having the arduino wait till processing sends the second piece of data. Also, I cannot just use the "Delay" function, as I want to values to be sent as fast as possible. Thank you for any help.

what communication method are you using? RS232, WiFi, Bluetooth, etc?
how much data/second?

The communication between processing and the arduino is wired serial. and then the arduino will be using RS-485 for transmitting to other arduinos. As for the data/second goes, the signals are going to be used to control a submarine in real time, so the less lag the better, but there is no real limit to how slow it can be.

have a look at
serial input

are you transmitting the integer values as text strings or in binary, e.g. two bytes for a 16bit integer?
is the data arriving too fast for the mega to process it and it is then loosing data?
could you store the 13 values in an array then process it/

Please provide an example of the data you want to send to the Mega and tell us how often it needs to be sent

Also provide more detail about what the Mega should do with the data.

Post your Mega program.

...R
Serial Input Basics - simple reliable ways to receive data.

Did you see this tutorial :

https://learn.sparkfun.com/tutorials/connecting-arduino-to-processing

Horace-
The data is currently being sent as binary. I am not sure about the speed of the incoming data being to fast for the mega, as i have only been able to read one of the values sent by the processing code, however, I wanted to do what you had mentioned, and send all the integers at once then break that down with the mega. The only issue, is that I am not sure how to break pieces of the string or array apart into separate values. It would be much better to have processing put all the variables into a single string or array then send that, as that is much faster than making the arduino call for data every time it wants the next variable. If you can help me to both better understand how integers can be put into a string then sent, and how to go about breaking that string into the right parts, I would give you my first born child, but I am struggling to learn how on my own.

Robin2-
I have yet to write the code for the arduino other than receiving the values. I have the processing code, and a very basic receive code for the arduino. As far as what the arduino is going to do with the data, it is going to take the data and convert the input values into signals for two esc's for motor controls, as well as send signals for stepper motor controls, and light controls. These signals will be sent through an RS-485 module over about 200" of cable to another arduino that will then send the signals to the right places, as well as send sensor data back to the mega to be processed and displayed. The project is a remotely operated submarine that needs to be controlled through a cable from the surface. Also, I will post the processing code as soon as I get back to my desktop.

I believe my issue is understanding how the integers are sent to the arduino. a set of values that would come from processing would look like this. (127,23,255,235,17,0,1,0,0,1,0,0,6) where the first five numbers are positions of a flight stick on x,y,z, and two sliders, and the rest are buttons except the last value which is a hat switch (0-8). If some one could help with getting a string to work to send the data, breaking it apart is something i think i can do, especially with the tutorial Robin posted. (Thank you for that by the way). This may end up being more of a processing question, so i'm sorry for asking it here, but having minimal experience working with strings, I think I need some help with creating one to be sent.

Also, this is the current processing code, feel free to show me the error in my ways, as this is my first time using processing, and I have very limited understanding of java. (I've taken one C++ course in my life so i know practically nothing about java) Also, I apologize in advance for no commenting in the code, I'm the only one working on it and despite knowing that i should comment it, I haven't.

import net.java.games.input.;
import org.gamecontrolplus.
;
import org.gamecontrolplus.gui.;
import processing.serial.
;

Serial myPort;
ControlIO control;
ControlDevice device;

int Trigger1;
int ButtonTwo;
int ButtonThree;
int ButtonFour;
int ButtonFive;
int ButtonSix;
int ButtonSeven;
int ButtonEight;
int ButtonNine;
int ButtonTen;
int Scroll1;
int Scroll2;
int RButton;

void setup() {
size(800,600);
control = ControlIO.getInstance(this);
device = control.getDevice("Cyborg X");
device = control.getMatchedDevice("Flight Stick");
background(0);
println(Serial.list());
myPort = new Serial(this, Serial.list()[1], 9600);
myPort.clear();

if (device == null) {
println("No suitable device configured");
System.exit(-1); // End the program NOW!
}}

void draw() {
float Y = (device.getSlider("Y axis").getValue());
Y = (Y*127)+127;
int Y1 = (int)Y;

float X = (device.getSlider("X axis").getValue());
X = (X*127)+127;
int X1 = (int)X;

float Z = (device.getSlider("Z Rotate").getValue());
Z = (Z*127)+127;
int Z1 = (int)Z;

float Slider1 = (device.getSlider("Slider1").getValue());
Slider1 = (Slider1*127)+127;
int SliderOne = (int)Slider1;

float Slider2 = (device.getSlider("Slider2").getValue());
Slider2 = (Slider2*127)+127;
int SliderTwo = (int)Slider2;

boolean Trigger = (device.getButton("B0").pressed());
if(Trigger == true)
Trigger1 = 1;
else
Trigger1 = 0;

boolean Button2 = (device.getButton("B1").pressed());
if(Button2 == true)
ButtonTwo = 1;
else
ButtonTwo = 0;

boolean Button3 = (device.getButton("B2").pressed());
if(Button3 == true)
ButtonThree = 1;
else
ButtonThree = 0;

boolean Button4 = (device.getButton("B3").pressed());
if(Button4 == true)
ButtonFour = 1;
else
ButtonFour = 0;

boolean Button5 = (device.getButton("B4").pressed());
if(Button5 == true)
ButtonFive = 1;
else
ButtonFive = 0;

boolean Button6 = (device.getButton("B5").pressed());
if(Button6 == true)
ButtonSix = 1;
else
ButtonSix = 0;

boolean Button7 = (device.getButton("B6").pressed());
if(Button7 == true)
ButtonSeven = 1;
else
ButtonSeven = 0;

boolean Button8 = (device.getButton("B7").pressed());
if(Button8 == true)
ButtonEight = 1;
else
ButtonEight = 0;

boolean Button9 = (device.getButton("B8").pressed());
if(Button9 == true)
ButtonNine = 1;
else
ButtonNine = 0;

boolean Button10 = (device.getButton("B9").pressed());
if(Button10 == true)
ButtonTen = 1;
else
ButtonTen = 0;

boolean ScrollUp = (device.getButton("B10").pressed());
if(ScrollUp == true)
Scroll1 = 1;
else
Scroll1 = 0;

boolean ScrollDown = (device.getButton("B11").pressed());
if(ScrollDown == true)
Scroll2 = 1;
else
Scroll2 = 0;

boolean RedButton = (device.getButton("B13").pressed());
if(RedButton == true)
RButton = 1;
else
RButton = 0;

float AnalogStick = (device.getHat("Analog Stick").getPos());
int AStick = (int)AnalogStick;

println(Y,X,Z,Slider1,Slider2,Trigger,Button2,Button3,Button4,Button5,Button6,Button7,Button8,Button9,Button10,ScrollUp,ScrollDown,RedButton,AnalogStick);

println(Y1,X1,Z1,SliderOne,SliderTwo,Trigger1,ButtonTwo,ButtonThree,ButtonFour,ButtonFive,ButtonSix,ButtonSeven,ButtonEight,ButtonNine,ButtonTen,Scroll1,Scroll2,RButton,AStick);

myPort.write(Y1);
myPort.bufferUntil(1);
myPort.write(X1);
myPort.bufferUntil(2);
}

The values start as floats, and I convert them to integers so they are easier to send to the arduino. If that is unneeded feel free to throw out ideas. I am down to try anything and everything. Also, the println is simple to make sure I know what values should be getting sent out to the arduino, and the last bit of code was for testing, it is essentially meaningless.

does your Java program compile? I though method println()
Java formatting
took an individual value, converts it using toString() and outputs the result as text not binary
have you implemented your own println() ?
I assume you have a Main() method?

what is the numeric range of your data? are you sure the endianness of the two devices is the same?
you could transmit float values as 4 bytes assume the floating point representation is the same on both machines

an example of encoding and decoding 16 bits integers using C is

#include <stdio.h>

// encode 16 bit unsigned integers data into two bytes (chars)characters
void RSencode(unsigned char ch[2], const unsigned int data)
{
     ch[0]= (data>>8) & 0xff;   // top byte of data
     ch[1]=data & 0xff;         // lower 8 bits of data
 }

// decode the above data
void RSdecode(const unsigned char ch[10], unsigned int *data)
{
    *data=(ch[0] << 8) | ch[1];
}

int main()
{
    while(1)
    {
            unsigned int i, x={0},y;
            unsigned char ch[2];
            printf("enter an unsigned int ? ");
            scanf("%d", &x);
            RSencode(ch, x);
            printf("encoded characters ");
            for(i=0;i<2;i++) printf("%x ", ch[i]);
            RSdecode(ch, &y);
            printf("\ndecoded data %d \n", y);
    }
}

a run gives

enter an unsigned int ? 7
encoded characters 0 7
decoded data 7
enter an unsigned int ? 67
encoded characters 0 43
decoded data 67
enter an unsigned int ? 5678
encoded characters 16 2e
decoded data 5678
enter an unsigned int ? 23456
encoded characters 5b a0
decoded data 23456
enter an unsigned int ?

is there any particular reason for using processing?
you could transmit the data using the serial port directly
e.g. this Java program transmits a float to the Arduino

// jSerialComm read float and transmit to COM port

// java -cp D:\Programming\JAVA\JSerialComm\jSerialComm-1.3.11.jar;.  SerialTransmitFloat

import com.fazecast.jSerialComm.*;
import java.util.*;

public class SerialTransmitFloat
{
  public static void main(String[] args) { 
  Scanner console = new Scanner(System.in);
   System.out.println("List COM ports");
   SerialPort comPorts[] = SerialPort.getCommPorts();
       for (int i = 0; i < comPorts.length; i++)   
          System.out.println("comPorts[" + i + "] = " + comPorts[i].getDescriptivePortName());
  int port = 1;      // <<<<<<   assume the arduino port is second COM port (first is COM1:)
  if(!comPorts[port].openPort())
    {  System.out.println("unable to open port " + comPorts[port].getDescriptivePortName() + "\n"); return; };
  comPorts[1].setBaudRate(115200);
  try {
    while (true)
    {
      // transmit integer to serial port
      System.out.print("enter an float ");
      float test = console.nextFloat();                // read float
      //System.out.println("you entered " + test);   // display it
      byte[] writeBuffer=(Float.toString(test) + "\n").getBytes();
      comPorts[port].writeBytes(writeBuffer, writeBuffer.length);
     // read serial port for response and display it
      while (comPorts[port].bytesAvailable() == 0)
         Thread.sleep(20);
      byte[] readBuffer = new byte[comPorts[port].bytesAvailable()];
      int numRead = comPorts[port].readBytes(readBuffer, readBuffer.length);
      System.out.print("Read " + numRead + " bytes from COM port: ");
       for (int i = 0; i < readBuffer.length; i++)   
           System.out.print((char)readBuffer[i]);
       System.out.println();
     }
  } catch (Exception e) { e.printStackTrace(); }
  comPorts[1].closePort();  
}
}

the Arduino code is

// SERIAL: read and echo float value
void setup() {
  // put your setup code here, to run once:
    Serial.begin(115200);
    Serial.setTimeout(60*60*1000ul);
}
void loop() {
  //Serial.print("enter float ? ");
  float data=Serial.parseFloat();
  Serial.print("Arduino received float = ");
  Serial.println(data);
 }

a run gives

D:\temp2>java -cp D:\Programming\JAVA\JSerialComm\jSerialComm-1.3.11.jar;.  SerialTransmitFloat
List COM ports
comPorts[0] = Communications Port (COM1)
comPorts[1] = USB-SERIAL CH340 (COM9)
enter an float 3.14159
Read 31 bytes from COM port: Arduino received float = 3.14

enter an float 567.83
Read 33 bytes from COM port: Arduino received float = 567.83

enter an float -345.78
Read 34 bytes from COM port: Arduino received float = -345.78

enter an float

if you wish to transmit binary you convert the float to a 4 byte array - transmit that and convert it back to a float in the arduino, e.g.

// jSerialComm read float and transmit to COM port in binary as 4 bytes 

// java -cp D:\Programming\JAVA\JSerialComm\jSerialComm-1.3.11.jar;.  SerialTransmitFloat

import com.fazecast.jSerialComm.*;
import java.util.*;
import java.nio.ByteBuffer;

public class SerialTransmitFloat
{
  public static void main(String[] args) { 
  Scanner console = new Scanner(System.in);
   System.out.println("List COM ports");
   SerialPort comPorts[] = SerialPort.getCommPorts();
       for (int i = 0; i < comPorts.length; i++)   
          System.out.println("comPorts[" + i + "] = " + comPorts[i].getDescriptivePortName());
  int port = 1;      // <<<<<<   assume the arduino port is second COM port (first is COM1:)
  if(!comPorts[port].openPort())
    {  System.out.println("unable to open port " + comPorts[port].getDescriptivePortName() + "\n"); return; };
  comPorts[1].setBaudRate(115200);
  try {
    while (true)
    {
      // transmit integer to serial port
      System.out.print("enter an float ");
      float test = console.nextFloat();                // read float
      //System.out.println("you entered " + test);   // display it
      byte[] writeBuffer=ByteBuffer.allocate(4).putFloat(test).array();    // to byte[4] array
      comPorts[port].writeBytes(writeBuffer, writeBuffer.length);
     // read serial port for response and display it
      while (comPorts[port].bytesAvailable() == 0)
         Thread.sleep(20);
      byte[] readBuffer = new byte[comPorts[port].bytesAvailable()];
      int numRead = comPorts[port].readBytes(readBuffer, readBuffer.length);
      System.out.print("Read " + numRead + " bytes from COM port: ");
       for (int i = 0; i < readBuffer.length; i++)   
           System.out.print((char)readBuffer[i]);
       System.out.println();
     }
  } catch (Exception e) { e.printStackTrace(); }
  comPorts[1].closePort();  
}
}

the arduino code

// SERIAL: read and echo float value transmitted as 4 byte binary value

void setup() {
  // put your setup code here, to run once:
    Serial.begin(115200);
    Serial.setTimeout(60*60*1000ul);
}
union FloatToBytes
{
    float data;        // 16 bit integer
    byte bytes[4];       // four bytes
} test;

void loop() {
  Serial.readBytes(test.bytes,4);   // read float as 4 bytes
  // java and Arduino float byte order is revered
  swopBytes(test.bytes);            // swop bytes end for end
  Serial.print("\nfloat as 4 bytes ");
  for(int i=0;i<4;i++)
       { Serial.print(" "); Serial.print(test.bytes[i]); }
  Serial.print("\nArduino received float = ");
  Serial.println(test.data);
 }

// swop array contents end for end
 void swopBytes(byte d[4])
 {
   int i,j;
   for(i=0,j=3;i<2;i++,j--)
     { byte t=d[i]; d[i]=d[j]; d[j]=t; }
 }

note that the byte array has to be swoped end for end - Java hold float in a different order to the Ardi=uino
a run gives

List COM ports
comPorts[0] = Communications Port (COM1)
comPorts[1] = USB-SERIAL CH340 (COM9)
enter an float 3.14159
Read 63 bytes from COM port:
float as 4 bytes  208 15 73 64
Arduino received float = 3.14

enter an float -56789.67
Read 70 bytes from COM port:
float as 4 bytes  172 213 93 199
Arduino received float = -56789.67

enter an float 495.34
Read 67 bytes from COM port:
float as 4 bytes  133 171 247 67
Arduino received float = 495.34

tactfulyeti:
Robin2-
I have yet to write the code for the arduino other than receiving the values. I

Seems like that is all that is necessary to answer my Reply #4

...R

Sorry for being a bit long winded. As for the code though, I have read over some of Robin's posts and the codes posted by horace. and I think my issue has been processing related, I tried to send a regular integer and read it with an uno. I got this to work, as I was pretty stupid, and was trying to send bytes with the print function. Changed it to "write()" and got it to work fine by then reading the serial input as a byte on the uno side. This the issue lies in getting the values from processing to arduino. But there are a few issues here. First, I don't know how to serial monitor the incoming data from processing with the arduino, or if it is even possible. secondly, I'm sure the mega is getting something, the Tx light is on, but i'm not sure what it getting. again, because I can't see the serial monitor of the mega.

My first question is, is it possible to read from processing then write to a different serial port(Serial1,2,or 3) in the same block of code, or is there an issue with having multiple serial ports working in one block?
My second question is, is it possible to have data sent from processing to the mega, then serial monitor it on a pc?

Again, I thank you all for your help with this issue, as I am learning a ton form working on this, and the info you all are giving me is immensely helpful.

with the Mega you can transmit and recieve data on Serial1, Serial2 or Serial3
you will require a USB to TTL serial converter such as the FTDI TTL-232R-5V
FTDI USB serial cables
if you do a search on Ebay for usb-ttl serial you will find plenty of devices

on the PC you use a terminal emulator such as Terterm Pro to display the data

tactfulyeti:
as I was pretty stupid, and was trying to send bytes with the print function. Changed it to "write()" and got it to work fine by then reading the serial input as a byte on the uno side

I do not consider that stupid. My preferred approach is to use the PRINT function to send data in human readable form as it makes debugging very much easier. I would only send data in binary format if it was the only way I could get the performance I needed.

Also, for reliable serial communication is it a very good idea to enclose a message between start- and end-markers and that can be a lot more difficult with binary data because you have to allow for some the data bytes being the same as the markers.

...R

If I wanted to use integers as bytes, could I write a function to exclude 0 and 256 from the possible values being sent, then set those two values as the start-end points? A range of 1-255 is plenty for the motor control i'll be working with.

You cannot send the value 256 as a single byte. The max is 255. A simple suggestion would be to use 254 for the start-marker and 255 for the end-marker and just send values from 0 to 253.

I have never considered using 0 as a start-marker so I don't know if it would work. It probably would.

...R

After a bit of testing and messing around with code, I found an issue. I'm going to run through the entire process to try and make it easier fro someone who knows what im doing wrong to identify where. So, just to test sending data to the arduino, I am just trying to send the int 1 from processing to the arduino. I have the line "myPort.write(1);" I believe this should send a 1 to the arduino. I wrote code for the arduino that goes like this
"
void loop(){
int test;
if(Serial.available() > 0){
test = Serial.read();
lcd.setCursor(0,1);
lcd.print(test);}
}"

I have an lcd to display the int that comes in from processing, however, no number gets displayed. I also tested the code by sending a number from another arduino, and it did work. Therefore, I believe the issue is on the processing side of things. I just don't know where.

tactfulyeti:
I have the line "myPort.write(1);" I believe this should send a 1 to the arduino.

I am not familiar with Processing but it probably sends the binary value 0b00000001 whereas the character '1' is the binary value 0b00110001

If you use Serial.print() in the Arduino program to print the binary value 0b00000001 you won't see anything.

Have you tried using print rather than write in your Processing program?

...R