Arduino, RPi & PHP help! (2-way serial comm)

hey gang-

I am in need of some direction/advice here...

Long story short for project summary:

RPi all set-up (OS, MySQL, Apache, PHP, PhpMyAdmin)
RPi set-up to boot to default (locally hosted web site/page)
RPi set-up to boot to fullscreen (ie: KIOSK mode)
Database created, tables populated, webpage (drink menu) is dynamically created and displayed in browser.

Arduino (UNO) is connected to RPi via USB.

When webpage is submitted.. I take submitted/$_POST'd data... and send it via PHP to the connected Arduino on COM1

Arduino received serial data, parses it.. and does what it needs to do (move some steppers, move some servos...etc..etc. then returns back home)..

Everything up to this point is working as expected, and without issue!! (WOOT!) :slight_smile:

However..... this is where I am stuck at:

When the Arduino is done doing its business.... it then fires off a confirmation that it is done/complete via SERIAL, BACK TO THE RPi!...

....and I am NOT receiving it.

When the webpage submits (to itself).. it shows a 'please wait' message... when the Arduino sends back its 'complete' confirmation to the RPi.. the webpage should re-direct back to the intitial page state (drink menu selection)...

My approach for this is:

HTML/CSS/PHP coded page.. that queries the DB.. and dynamically builds the drink menu based on active/available ingredients to display a front end GUI (images and order button)..

Upon submit ($_POST).. I take this drink recipe/data and send it out to the Arduino.

  • (again) this is working without issue

I then have an AJAX call to an external PHP script that is used to query/listen to the SERIAL port.. with success/error calls back from this AJAX call.

  • My understanding is that the AJAX callbacks will just wait/listen for the external PHP script response.. and execute/do whatever is in the callback function when a response it finally returned from said PHP script

But I am NEVER getting a callback response :frowning:

(if I have webdev tool open and checking the console data.. when I hot the BACK button.. I -do- see the error callback message.. but never a success one, as in I am never receiving the serial data in the php script)

The response time is random/varies.. as there is no telling how long each specific drink will take to make/complete

At this point... I'm not sure if it how the SERIAL port may be configured?

Or how I am doing the AJAX/PHP serial read stuff? (I was under the impression that everything in LINUX is treated like a 'file'.. so I can treat the serial port the same way?... looking for the end of file? not really clear on how to best 'wait/listen'... and also know when the transmission is complete?)

Is there the PHP snippet I am using to:

1.) send the data to the connected Arduino..
2.) Output the AJAX (jQuery) call to the external PHP script
3.) The AJAX callback 'response handlers'

if ($mode == 'submit') {
    
	//grab posted data (save to var)
    $drinkRecipe = $_POST['selectedDrink'];
    
    //set-up com port    
    //exec("mode /dev/ttyACM0 BAUD=9600 PARITY=N data=8 stop=1 xon=off"); //mode not available in Linux?
	//saw this on several RPi posts?  (but not sure of the difference? or why one would be used over the other?)
    //this gets run command line on RPi, not via PHP Script (what are the parms doing even?)
	//stty -F /dev/ttyACM0 cs8 9600 ignbrk -brkint -imaxbel -opost -onlcr -isig -icanon -iexten -echo -echoe -echok -echoctl -echoke noflsh -ixon -crtscts
    
    //open serial port
    $fp = fopen("/dev/ttyACM0", "w+"); //w = write w+ = read/write
        
    //check if open
    if (!$fp) {
        echo "Not open";
        //die();
    } else {
        //if open send data (via PHP) to connected Arduino on serial comm port 1 (ttyACM0)
        fwrite($fp, '<' . $drinkRecipe . '>');		

		?>		
		<div id="waitingContainer">
			<p>Please wait, your brink is being made.</p>
		</div>
		
		<script type="text/JavaScript" language="JavaScript">
			console.log("ajax routine hit");
			$.ajax({
				//async: false,
				//type: "POST",
				url: "serial_listener.php",
				
				//define success handler actions
				success:function(response) {
					//alert("PHP RETURN CHECK: "+response);
					if($.trim(response) == 'complete'){
						console.log("Drink making is complete... return to main menu");
						//do redirect here
					}else{
						console.log("Some value other than 'complete' was returned... look into it!");
						//not sure what to do? (back to main menu anyways?)
					}
				},
				//define error handler actions
				error: function(response) {
					console.log("PHP SERIAL READ FAIL: "+ 'Ready State: '+ response.readyState + ' Status: ' + response.status);
					//do whatever here on failed complete response
				}
			});	
		</script>

		<?		
        //close connection
        //fclose($fp); //needed? //should this go in the external php script instead now?
    }    

}

Here are the contents of the serial_listener.php script:

//open serial port
$fp = fopen("/dev/ttyACM0", "w+"); //w = write w+ = read/write

//check if open
if (!$fp) {
	echo "Not open";
	//die();
	
} else {

	while(!feof($fp)){
		$response = fread($fp, 1000);
	}
	echo $response;
	fclose($fp);

What am I missing or doing wrong here?

All feedback is appreciated.

thanks!

I am not familiar with PHP I tried it years ago and did not like it (polite version :slight_smile: )

Are you sure your PHP program only opens the Serial port once and keeps it open until you are completely finished communicating with the Arduino?

...R

hey Robin2-

HAHA.. I understand.. (its not for everyone!) :slight_smile:

However, it is what I am most comfortable with for server side web programming/development & DB interaction.

Using it for serial communication is a bit of a stretch I'm sure...

however 'writing' is no problem.. and from research 'reading' should not be an issue if not on a Windows platform.

I am not sure if the serial port remains open until specifically 'closed' or not....

(I'm also not clear if I need to 'open' it (again) in the external php script executed by the AJAX call)..

  • which I believe I -am- doing.. (wasnt sure if I need to open it again from the external script or if it just remains open and usable by 'whatever is coming down the pipe' (so to speak)... until closed. I am not 'closing' the port in either script currently.. lines are commented out)..

I'm not really sure where to start or go about trouble shooting here. LOL..

I think the the first is how to 'correctly' read the serial port using PHP.. (and make sure the port is configured correctly as well?)

Since I was told 'RPi/Linux' treats everything like a 'file'.. then !eof (not end of file) should work for the serial port.

  • Ah-Ha moment as I type.....

Do I need to be sending an 'end of file' character back from the Arduino to trigger the PHP while() function to complete?

(off to read up on that more).......

  • perhaps there is a better method to read the port than my approach?

I seem to recall PHP ran as the apache www-data user, so maybe it needs to be added to the dialout group.

hmm..ok (just talking this out more)..

It is DEFINITELY the reading of the serial port that is the issue.

The external .php scrip that the AJAX call executes does the following:

opens com port - check (no error given and it continues on with the else conditional if the port was NOT open)
ie: port is open or that portion of the code in the 'else' snippet wouldnt execute.

I replaced the '

while(!feof($fp)){
        $response = fread($fp, 100);
}    
echo $response;

routine with a simple:

sleep(10);
echo 'complete';
fclose($fp);

I wanted to echo out a static value to see if I could see it being received on the webpage/RPi side of things.. (it is!)..

I added the sleep(10); command in there to test/prove that the AJAX callback stuff will just 'wait' until a response is received (or errors out)..... and doesnt need it to instantly respond..etc. (the A in AJAX! lol)..

So now I know:

  • the port IS open for reading
  • the AJAX call will handled the response whenever it comes back correctly.

I just am not getting anything FROM the serial port to confirm things are complete.. and I can send a legit response.

So that leads me to believe that WHILE state 'is' working.. and I'm just stuck in it.. as it does error out.. (but never succeeds to send any data back either)..

@ron_sutherland

I am not that well versed with Linux/CLI stuff..

I have done this via terminal: (adding www-data user to the dial-out group.. which is for the Apache/PHP communication via serial...no?)

* Add 'www-data' user to dial-out group
# sudo usermod -a -G dialout www-data
# sudo reboot

I'm not sure you want to open the serial port as a file with fopen. I think (fuzzy on this) a file system will try to do block mode transfers, but the serial port is a character mode device (or something like that). I would say try that phpSerial , but I don't like its look.

Update: your CLI commands look right

Update:

seems to be working in some reliable fashion now..

I trimmed the PHP (serial_listener.php) script down to just this

$responseValue = "";
do{
      $incomingChar = fread($fp, 1);
      $responseValue .= $incomingChar;
}while(strlen($responseValue) < 1);
echo $responseValue;

I can now work with single character responses with the above... (and I really only need to send a 1 upon completion instead of 'complete'... and I can send whatever I want back to the AJAX callback to be parsed..

If I want to check for the whole string 'complete' coming from the Arduino (which is not needed)..

I just changed the value to 8 for the conditional before sending..etc..

$responseValue = "";
do{
      $incomingChar = fread($fp, 1);
      $responseValue .= $incomingChar;
}while(strlen($responseValue) < 8);
echo $responseValue;

Update 2:

Lost communication again.

I cant make heads or tails of all this now!

There is something going on with the port opening I think? or a combination of the capacitor I have on the Arduino to NOT reset each time the port is opened perhaps?.. SOMETHING!

I could refresh and/or re-send consistently for a little bit (as I was watching debug output in Firebug, and it matched console.log() data to actions)

I need to do some more tests and log each result to see if any patterns arise I guess. :frowning:

I am NOT seeing any NOT OPEN message displayed anywhere.. so I believe the port IS being opened correctly. Grrrrr.. LOL

xl97:
I am not sure if the serial port remains open until specifically 'closed' or not....

You need to figure that out. My guess is that with PHP it is NOT kept open because (I think) a new PHP program runs every time the server receives a message.

If you can't arrange for all the Arduino communication to take place in the same instance of PHP then I suggest you use a USB-TTL cable to communicate with the Arduino. Just connect it to Tx Rx and GND and then the Arduino will not reset when the Serial posrt is opened by PHP.

...R

I -do- need to open the port in the external .php script executed by the AJAX call.

I have now tried BOTH approaches:

1 - where the initial 'SERIAL WRITE' is in the MAIN .php script.. and the 'SERIAL READ' is in the external PHP script executed by the AJAX call.

1 - where the initial 'SERIAL WRITE' and the 'SERIAL READ' is in the external PHP script executed by the AJAX call.

The stopping/error point seems to still be in the READING..

Its definitely something to do with the PORT talking.. and RESETTING stuff..

I have found some odd examples....where.. if I had rebooted everything RPI included.... (no cap on the RESET/GND pins to prevent resetting)...

the initial SEND doesnt take at first (probably because still resetting)..

if I then ADD the cap... I get an OUTPUT of the value?... but after.. nothing..

I just cant nail down the right process/procedure to make this stable and work correctly! LOL

Wouldnt just adding the capacitor between REST/GND do the same as your USB-TTL programmer approach? (for not resetting)

xl97:
Wouldnt just adding the capacitor between REST/GND do the same as your USB-TTL programmer approach? (for not resetting)

It may. But the USB-TTL cable WILL :slight_smile:

...R

I'll give it a try tonight when I get home then.

So just to make sure I'm following.understanding correctly.

No longer connect my UNO to RPi via USB cable..

but instead use my USB-TTL programmer....

Connect to RX/TX/GND on Arduino.... and use the same USB port on the RPi.

Posting both approaches I have tried in case there is an issue in that area of the project:

*note: First approach is no longer used, and I attempting to use the SECOND approach so I can 'send and receive' all in the same AJAX call.. (to eliminate multiple scripts using the port and multiple 'opening' of said port.

#1 - MAIN page/script:

if ($mode == 'submit') {
    
	//grab posted data (save to var)
    $drinkRecipe = $_POST['selectedDrink'];
    
    //open serial port
    $fp = fopen("/dev/ttyACM0", "w+"); //w = write w+ = read/write
        
    //check if open
    if (!$fp) {
        echo "Not open";
        //die();
    } else {
        //if open send data (via PHP) to connected Arduino on serial comm port 1 (ttyACM0)
        fwrite($fp, '<' . $drinkRecipe . '>');
		
		//output some jQuery/AJAX call to an external PHP script that does the port LISTENING, and respond back once data is received
		?>
		<script type="text/JavaScript" language="JavaScript">
			$.ajax({
				url: "serial_listener.php",
				
				//define success handler actions
				success: function(response) {
					//alert("PHP RETURN CHECK: "+response);
					if($.trim(response) == 'complete'){
						alert("Drink making is complete... return to main menu");
						//do redirect here
					}else{
						alert("Some value other than 'complete' was returned... look into it!");
						alert("Response Received: " + response);
						//not sure what to do? (back to main menu anyways?)
					}
				},
				
				//define error handler actions
				error: function(response) {
					alert("PHP SERIAL READ FAIL: "+ 'Ready State: '+ response.readyState + ' Status: ' + response.status);
					
				}
			});	
		</script>		
		<?		        
        
        //close connection
        //fclose($fp); //needed? //shouldnt be needed.....correct?  You can just leave the port open, right?

    }    

}

#1 - EXTERNAL page/script: (called from AJAX routine)

//open serial port
$fp = fopen("/dev/ttyACM0", "w+"); //w = write w+ = read/write

//check if open
if (!$fp) {
	echo "Not open";
	
} else {
	$responseValue = "";
	do{
	  $incomingChar = fread($fp, 1);
	  $responseValue .= $incomingChar;
	}while(strlen($responseValue) < 1);
	echo $responseValue;

}

Approach #2 is where I am attempting to use the AJAX routine/snippet to do BOTH, send the serial data to the Arduino, as well as READ the serial response from the Arduino once its actions are complete.

#2 - MAIN page/script:

if ($mode == 'submit') {
    
	//grab posted data (save to var)
    $drinkRecipe = $_POST['selectedDrink'];
    		
	//output some jQuery/AJAX call to an external PHP script that does the port LISTENING, and respond back once data is received
	?>
	<script type="text/JavaScript" language="JavaScript">
		var submittedDrink = <?=$drinkRecipe?>;
		
		$.ajax({
			//async: false,
			type: "POST",				
			url: "serial_listener.php",				
			datatype: "text",
			data:{
				"submittedDrink": submittedDrink
			},
			
			//define success handler actions
			success: function(response) {
				//alert("PHP RETURN CHECK: "+response);
				if($.trim(response) == 'complete'){
					alert("Drink making is complete... return to main menu");
					//do redirect here
				}else{
					alert("Some value other than 'complete' was returned... look into it!");
					alert("Response Received: " + response);
					//not sure what to do? (back to main menu anyways?)
				}
			},
			
			//define error handler actions
			error: function(response) {
				alert("PHP SERIAL READ FAIL: "+ 'Ready State: '+ response.readyState + ' Status: ' + response.status);
				
			}
		});	
	</script>		
	<?		        
        
}

#2 - EXTERNAL page/script: (called from AJAX routine)

$drinkData = $_POST['submittedDrink'];
//open serial port
$fp = fopen("/dev/ttyACM0", "w+"); //w = write w+ = read/write

//check if open
if (!$fp) {
	echo "Not open";
	
} else {
	//if open send data (via PHP) to connected Arduino on serial comm port 1 (ttyACM0)
	fwrite($fp, '<' . $drinkData  . '>');
	
	
	//check and wait for incoming serial data form Arduino response
	$responseValue = "";
	do{
		$incomingChar = fread($fp, 1);
		$responseValue .= $incomingChar;
	}while(strlen($responseValue) < 1);
	
	//has data, send to AJAX callback/listener
	echo $responseValue;

}

I believe both of these are viable approaches worth looking into (for now)...

Still need to figure out this fopen() and resetting stuff.... (going to try the USB-TTL on RX/TX IO pins to see if that helps.. or does something the capacitor doesnt do.

Form reading another thread where @Robin2 suggested the same USB-TTL solution..... there is NO reason to ever CLOSE the port.....correct?

If a port is open.. and another 'open' command is sent.. what happens? Nothing? fails silently?

I'm tired and anxious.. as this is the last part of the 'software' (front end) I need to work out.. (then just some cosmetics.. and back to the final Arduino based hardware requirements!)

The R-Pi has a hardware UART. it needs to be enabled and level shifted. My notes:

I guess the R-Pi3 and ZeroW have Bluetooth connected to the good hardware UART, but I suspect the raspi-config tool can swap the good and bad UART around and set it up to be used as a proper serial port. It is even possible to enable nRTS/nCTS hand shaking so nRTS can do the bootload trick (e.g. the thing you don't want). Anyway, you would need to level shift.

xl97:
but instead use my USB-TTL programmer....

That is the second time you have called it a "programmer". I wonder if you are thinking of something different from what I have in mind

...R

sorry..

I dont want to confuse with my lack of proper terminology here.. LOL..

Here is is what I have, and is what I use to upload sketches to Pro-Mini's/custom board without USB..etc

  • couldnt find an exact pic... but this is close enough: (CP2102 based)

I am just settling in at home from work (whew!)..

I'll be setting things up again and see if I can get it running without issues.. (might see some more posts here!) LOL

Question:

if I am using this USB-TTL as a connection from the Arduino RX/TX/GND to the RPi USB port (COM1)..

How should I be powering the Arduino Uno now? (if not from the RPi USB port?) I shouldnt power it via USB right? (Or wont that matter if the OPEN port actions are coming down the RX/TX pins instead?) (not clear how that would work with RX/TX and USB?.. not very well I'd imagine)

I tried what I had.. (didnt work.. couldnt get the port open errors over and over)..

I'm guessing I'd need to install some sort of drivers for it?

I think I might have something more similar to your picture now that I think about it?

I had them before started using the product/item at the link I posted above.

These were just 'cables'.. and either needed to be hacked.. or at best you needed to press reset button manually on the board(s)..

older camera cables or something... (no exposed pcb or options of any kind outside of cutting the end of the cable to expose the wires)

OPEN port actions are coming down the RX/TX pins instead?

When the 16u2 (USB port /dev/ttyACM0) is open it pulls down a pin (lets call it nDTR) and that is coupled to the nRESET pin of the AVR (capacitively). Your extra capacitor added is a hulk and overrides the small one used for the reset trick.

If you don't use the handshake line (nDTR or nRTS) to do the reset trick then the AVR will not reset (hulk not needed).

To repeat: when RX/TX is connected the AVR will not reset when the port is open.

Power is the next matter... when I suggested using the R-Pi serial hardware I totally forgot how dangerous it is since powering one down can damage the other, make sure to put a resistor (like 10k Ohm) between the Pi and the AVR with each line (Pi_RX<-- 10k<--RD_LED<--Uno_TX, Pi_TX -->10k-->Uno_RX) to be safe, and put a jumper on the 16u2's ISP header to hold it in reset.

Power the Uno without going through the R-Pi, sounds like a good idea to me, remember the common ground.

list of serial ports you may need

# hardwar serial on R-Pi
/dev/ttyAMA0
# FTDI and some other usb serial
/dev/ttyUSB0
# modem... what? is the ATmega16u2 a modem. 
/dev/ttyACM0

Update: red LED to level shift

Ahh... (duh)..

I wasnt even thinking I would need to change the PORT name!!

No wonder I wasnt getting anywhere.. and kept getting PORT NOT OPEN errors!! LOL..

I'll have to try and set things up again that way and update the port name!

On a side note...

I have been reading around.. and stumbled upon talk of disabling the REST through the RPi?

using:

stty -F /dev/ttyACM0 -hupcl

How do you RE-ENABLE if it doesnt work? (Im not a huge fan of doing things I cant un-do when I'm very new to all this! Linux jazz!) :slight_smile:

ignore all that stty stuff, that has to do with mapping the serial port (character device) through the pseudo terminal land for the actual terminal (e.g. xTerm, shell or whatever you are using). It is all a waste of time to learn.

if you want to open the port at the command line use picocom, it just makes life much simpler (and doses all the tricks needed for xTerm or a shell terminal).

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install picocom

just short the Pi_TX to Pi_RX to loop back an echo to test it, (sorry but I bet an update was needed)

picocom -b 38400 /dev/ttyAMA0

then try the AVR.

Update: to exit picocom use [C-a] [C-x], sorry should have thought of that.

Update2: I guess you are looking at stty for PHP, and I don't see how that will help, but I also don't have any ideas for PHP.