I'm not able to write a string from console input on the yun

I've got an Arduino Yun and I'm working on a Sparkfun tutorial that is not completely working for me. In this tutorial you use the Yun to send information to a sparkfun datastream, this is mine. Specifically, you are supposed to set a variable called "name" using the console interface. Before I post my code I've got to mention that the code posted on the sparkfun site uses "serial" not "console" intending for the user to connect to the Yun via a USB cable and I am connecting via wifi. After scratching my head a little I found that all I needed to do to get it to work (or at least get to the point I'm at) was replace "Serial" with "Console" in the code... meaning if I saw "Serial.begin" it would be replaced with "Console.begin".

The attached screenshot shows the console interface, and specifically that when I've entered a name that the name is echoed back to me as expected by the "if (Console.available())" section, however that variable gets set to "" when curl sends "name" to the stream. Also worth noting is that the default name does get posted by curl if I do not override it at the console.

Thanks!!

Here is the code:

Phant_Yun.ino
Post data to SparkFun's data stream server system (phant) using
an Arduino Yun
Jim Lindblom @ SparkFun Electronics
Original Creation Date: July 3, 2014

This sketch uses an Arduino Yun to POST sensor readings to 
SparkFun's data logging streams (http://data.sparkfun.com). A post
will be initiated whenever pin 3 is connected to ground.

Make sure you fill in your data stream's public, private, and data
keys before uploading! These are in the global variable section.

Hardware Hookup:
  * These components are connected to the Arduino's I/O pins:
    * D3 - Active-low momentary button (pulled high internally)
    * A0 - Photoresistor (which is combined with a 10k resistor
           to form a voltage divider output to the Arduino).
    * D5 - SPST switch to select either 5V or 0V to this pin.
  * Your Yun should also, somehow, be connected to the Internet.
    You can use Ethernet, or the on-board WiFi module.

Development environment specifics:
    IDE: Arduino 1.5.6-r2
    Hardware Platform: Arduino Yun

This code is beerware; if you see me (or any other SparkFun 
employee) at the local, and you've found our code helpful, please 
buy us a round!

curl example from:
https://github.com/sparkfun/phant/blob/master/examples/sh/curl_post.sh

Distributed as-is; no warranty is given.
*****************************************************************/
// Process.h gives us access to the Process class, which can be
// used to construct Shell commands and read the response.
#include <Process.h>
#include <Console.h>

/////////////////
// Phant Stuff //
/////////////////
// URL to phant server (only change if you're not using data.sparkfun
String phantURL = "http://data.sparkfun.com/input/";
// Public key (the one you see in the URL):
String publicKey = "ZG02gWXDGvcjxvjZ6JMj";
// Private key, which only someone posting to the stream knows
String privateKey = "NOTTHEPRIVATEKEY";
// How many data fields are in your stream?
const int NUM_FIELDS = 3;
// What are the names of your fields?
String fieldName[NUM_FIELDS] = {"light", "switch", "name"};
// We'll use this array later to store our field data
String fieldData[NUM_FIELDS];

////////////////
// Pin Inputs //
////////////////
const int triggerPin = 3;
const int lightPin = A0;
const int switchPin = 5;

String name = "anon"; //this is the default name
boolean newName = true;

void setup() 
{
  // initialize serial communication:
  Bridge.begin();
  Console.begin(); 

  // Setup Input Pins:
  pinMode(triggerPin, INPUT_PULLUP);
  pinMode(switchPin, INPUT_PULLUP);
  pinMode(lightPin, INPUT_PULLUP);
  
  while (!Console){
    ; // wait for Console port to connect.
  }

  Console.println("=========== Ready to Stream ===========");
  Console.println("Press the button (D3) to send an update");
  Console.println("Type your name, followed by '!' to update name");
}

void loop()
{  
  // If the trigger pin (3) goes low, send the data.
  if (!digitalRead(triggerPin))
  {
    // Gather Data
    fieldData[0] = String(analogRead(lightPin));
    fieldData[1] = String(digitalRead(switchPin));
    fieldData[2] = name;

    // Post Data
    Console.println("Posting Data!");
    postData(); // the postData() function does all the work, 
                // see below.
    delay(1000);
  }

  // Check for a new name input:
  if (Console.available())
  {
    char c = Console.read();
    if (c == '!')
    {
      newName = true;
      Console.print("Your name is ");
      Console.println(name);
    }
    else if (newName)
    {
      newName = false;
      name = "";
      name += c;
    }
    else
    {
      name += c;
    }
  }
}

void postData()
{
  Process phant; // Used to send command to Shell, and view response
  String curlCmd; // Where we'll put our curl command
  String curlData[NUM_FIELDS]; // temp variables to store curl data

  // Construct curl data fields
  // Should look like: --data "fieldName=fieldData"
  for (int i=0; i<NUM_FIELDS; i++)
  {
    curlData[i] = "--data \"" + fieldName[i] + "=" + fieldData[i] + "\" ";
  }

  // Construct the curl command:
  curlCmd = "curl ";
  curlCmd += "--header "; // Put our private key in the header.
  curlCmd += "\"Phant-Private-Key: "; // Tells our server the key is coming
  curlCmd += privateKey; 
  curlCmd += "\" "; // Enclose the entire header with quotes.
  for (int i=0; i<NUM_FIELDS; i++)
    curlCmd += curlData[i]; // Add our data fields to the command
  curlCmd += phantURL + publicKey; // Add the server URL, including public key

  // Send the curl command:
  Console.print("Sending command: ");
  Console.println(curlCmd); // Print command for debug
  phant.runShellCommand(curlCmd); // Send command through Shell

  // Read out the response:
  Console.print("Response: ");
  // Use the phant process to read in any response from Linux:
  while (phant.available())
  {
    char c = phant.read();
    Console.write(c);
  }
}

Why did you think it would be better to post this query in the PROGRAMMING section rather than the YUN section of the Forum?

...R

Why did you think it would be better to post this query in the PROGRAMMING section rather than the YUN section of the Forum?

...R

I thought it was an an issue with the PROGRAMMING and not an issue with the YUN.

jessel:

  // Check for a new name input:

if (Console.available())
 {
   char c = Console.read();
   if (c == '!')
   {
     newName = true;
     Console.print("Your name is ");
     Console.println(name);
   }
   else if (newName)
   {
     newName = false;
     name = "";
     name += c;
   }
   else
   {
     name += c;
   }
 }
}

Let's step through your code:

As you enter characters, they become available. You read the character: it is not '!', and newName is false, so the character gets added to name. Finally, you get a '!', so newName becomes true, and the name is printed properly.

Then, you are getting a new line character. It's not '!' so the first IF is skipped, but this time newName is true, so that clause is executed: newName is set false, name is cleared, and the newline character is added to name.

Now, when the data is posted, name is just a newline character, and you see the closing quote of your post string appearing on a new line. Furthermore, the second time you enter a new name, it already starts with a newline character from the first go-around, so when it is echoed it will show up on a new line, just as in your screenshot.

The solution is to not hit enter after typing the '!', change the end of name terminator from '!' to newline, or specifically ignore newline when reading characters.

It looks like the examples in serial input basics would work if you substitute "console" for "serial"

...R

The solution is to not hit enter after typing the '!', change the end of name terminator from '!' to newline, or specifically ignore newline when reading characters.

If you don't hit enter then nothing happens. Perhaps I'm overlooking something, but I tried all the options as far as "no line ending", "newline", "carriage return", and "Both NL and CR". Maybe that behavior would be different with the regular serial interface?

Simply replacing "!" with "\n" as suggested does it though. It makes sense of course, but what I can't figure is how the code was posted, and was never corrected. Perhaps that means that I'm the only person that ran that sketch...

Thanks!

It sounds like you're using the Serial Monitor in the Arduino IDE. It has the curious interface of a separate text box where you enter strings, and then have to hit the Send button to blast it out at once. This adds the newline character.

It would work as written if you used regular terminal emulator that sent characters as you type. Take a look at the "you can also see the Console" portion of this discussion.

Accessing the terminal via:

You can also see the Console by opening a terminal window and typing ssh root@yourYunsName.local 'telnet localhost 6571' then pressing enter.

gives the same performance as the arduino serial monitor. Specifically, nothing is sent until you hit enter, so with the example where you turn the led on and off with "H" and "L" respectively, you actually need to hit enter or nothing happens...

Your original suggestions of either using the newline character to trigger writing the "name" variable, or else filtering out the newline characters are the best solutions.

Thanks for the help!

jessel:
Specifically, nothing is sent until you hit enter, so with the example where you turn the led on and off with "H" and "L" respectively, you actually need to hit enter or nothing happens...

I wonder if this is due to the settings for the program you are using to SSH from?

Some of the terminal programs can certainly be set to send characters as they are typed.

What do you want to send the characters from? Will they be coming from your PC?

...R

jessel:
gives the same performance as the arduino serial monitor. Specifically, nothing is sent until you hit enter

Interesting. I can confirm this with a stripped down version of the sketch that just includes the Console handling portion. I added code to immediately send output on each key press, and nothing shows up until pressing enter, at which point all of the output is blasted out.

I'm using PuTTY as my SSH client, and I know for a fact it is sending out each keystroke as I type it. The Yun's telnet client (running from the SSH shell) must be buffering the lines of data and not sending them to the Console object until there is a line ending. Or maybe the Console object itself is waiting for a line ending before indicating anything is available?

How distressing.

ShapeShifter:
::::SNIP::::

Or maybe the Console object itself is waiting for a line ending before indicating anything is available?

How distressing.

@ShapeShifter,

You may recall from a recent posting
[Documetation] Important Note on Bridge.py
http://forum.arduino.cc/index.php?topic=311959.0

The phrase "Please press Enter to activate this console", should be familiar to the users of Yún Serial Terminal.

I suspect that is the case here. Since as you can see from early in the thread it is stated:

You can also see the Console by opening a terminal window and typing ssh root@yourYunsName.local 'telnet localhost 6571' then pressing enter.

With SSH you can by pass the ALL prompts by using an SSH public/private key.
But as is normal for telnet, you are going to get a prompt

And as the openwrt website shows for [url=http://wiki.openwrt.org/doc/howto/notuci.config#etcinittab]/etc/inittab[/url] you are going to get a "login" prompt. (SEE BELOW)

I suspect the reason you are not getting to see the "Please press Enter to activate this console", is that is being chewed up by some code.

FYI: I'm having vision issues. I'll drop you a note on this.

Jesse

::sysinit:/etc/init.d/rcS S boot
::shutdown:/etc/init.d/rcS K shutdown
tts/0::askfirst:/bin/ash --login
ttyS0::askfirst:/bin/ash --login
tty1::askfirst:/bin/ash --login

Jesse, you're talking about a different type of console. It is the Linux console running on ttyATH0 that prompts for the "press a key to activate the console." And it is a Linux shell that asks for a login.

What this thread is talking about is the Console class that runs as part of the sketch, and is part of the Bridge library. This is a Stream class that can be used in place of the Serial class. The difference is that while the Serial class is connected to the USB port, the Console class is connected to a TCP network socket. This has nothing to do with any sort of Linux console, and the only involvement with Linux is the code to pass this socket along to the network.

The issue is that the Yun doesn't support incoming Telnet connections directly. You must SSH into the Yun first, and then run the Yun's Telnet client to connect to the localhost port to which the Console class is listening. Since I know the SSH connection is passing each character as it is typed, it appears that there is something with the Telnet client that is buffering I/O a line at a time?

Either way, this is strictly a TCP socket connection, it has nothing to do with an actual Linux console.

And of course, the proper solution for this sketch is to use the newline character as the input terminator. I would recommend that even if there were no line level buffering going on. I find it odd that the sketch uses an exclamation point character to signal the end of the name.

ShapeShifter:
::::SNIP::::

And of course, the proper solution for this sketch is to use the newline character as the input terminator. I would recommend that even if there were no line level buffering going on. I find it odd that the sketch uses an exclamation point character to signal the end of the name.

@ShapeShifter,
okay. I misread the OP, and the stated speculation was way off base......

In UNIX programming, you can open a stream in "block mode" and "character mode". "Block mode" is fairly straight forward. Character mode can be confusing because it can be in "BLOCKING" and "NON-BLOCKING" mode.

THIS IS CONFUSING because "BLOCKING" mode IS NOT about "block mode" and has NOTHING to do with "block mode".

To be clear, we are talking about "character mode" when I use the words "BLOCKING" and "NON-BLOCKING"

In this reference, open(3), you want to look at O_NONBLOCK.

When opening a block special or character special file that supports non-blocking opens:

If O_NONBLOCK is set, the open() function shall return without blocking for the device to be ready or available. Subsequent behavior of the device is device-specific.
*
If O_NONBLOCK is clear, the open() function shall block the calling thread until the device is ready or available before returning.

Otherwise, the behavior of O_NONBLOCK is unspecified.

So, tty(s) and console(s) are open as character special devices (or files). And we want the line that says:

If O_NONBLOCK is clear, the open() function shall block the calling thread until the device is ready or available before returning.

And we know psuedo-terminals (or soft-terminals, including ssh and telnet) are not hardware. So, designers concluded that a new-line character (or a carriage return) would the "blocking" mechanism in "character mode" for psuedo-terminals.

What this means is that the TTY was open in "blocking" mode, and that means the stream stalls until it gets a new-line feed (or a carriage return).

The "blocking" mode makes sense because the writers of the shells (bash, ash, sh, csh) expect that if you want a "REAL" stream, you would open a stream without the shell, or programically write a non-blocking stream, or push the data through a socket. (Which, by the way, pipes can stall the same way as is being described - so it is not uncommon to see a or a <^D> (end of file) maker - in code.)

On and I almost forgot, this is NOT a telnet session. It is tty/shell session. This is more like having a terminal, so it is going to behave alot more like a smart shell (bash,csh) than a traditional tty. Which is another topic altogether.

I hope this is clear. If not, please ask questions.
Jesse

jessemonroy650:
On and I almost forgot, this is NOT a telnet session. It is tty/shell session.

No, it's not. It's the Yun's Telnet client talking to a network socket that is then being marshaled over to the Console object in the Sketch.

  • We establish an SSH connection to the Yun. This is talking to a shell. This connection is not line buffered, since the characters are clearly echoed as they are typed. The shell may not actually parse and process the command until there is a line terminator, but that's not the issue here.
  • We then type the command to start the Yun's Telnet client. The shell behind the SSH connection then parses the command string and starts Telnet.
  • The Telnet client is then in control of the connection, the SSH shell is no longer involved until the Telnet client exits.
  • There is now a network socket open between the Telnet client and the Python side of the Bridge.
  • The Python code now marshalls the data across the tty port to the sketch. This is not line buffered, since other means of communication over the Bridge can send individual characters as they are received. (like using the Process class which derives from the same core communications class.)
  • The Console object receives the marshaled data and makes it available to the sketch.

In my mind, it would appear that the Telnet client is running in half duplex mode: as characters are typed, it is locally echoing them, and not sending the data until a complete line is received. This is a reasonable default behavior as it cuts down on network traffic by making the assumption that the shell on the opposite end is really only interested in complete lines. However, this assumption breaks down in this case because the Telnet client is not talking to a line oriented shell, it is talking to our sketch (in a somewhat roundabout way) which is expecting a character oriented stream.

It's much like running a Python program using the Process class. The output of the Python process is piped to the sketch through the bridge. That connection is character oriented, but the default output behavior of Python is line oriented. Therefore the sketch doesn't receive anything until a line terminator is sent. In this case, the line buffering is clearly being done by Python, since running the Python process with the -u option runs it in unbuffered mode, and the characters are now sent immediately and can be received individually by the Process object.

I strongly suspect similar buffering is being done by the Telnet client. There is no shell intercepting the communications at this point, the only shell that was involved is currently dormant waiting for the Telnet client to close.

Looking into this a little further, it appears that the default mode of a standard Linux Telnet client is indeed LINEMODE, and there is a command to change to an unbuffered mode:

character' Disable the TELNET LINEMODE option, or, if the remote side does not understand the LINEMODE option, then enter ''character at a time'' mode.

Unfortunately, the Yun's Telnet client is part of busybox, which does not support any Telnet interactive commands, and supports very few command line options, none of which control the buffering mode:

telnet

telnet [-a] [-l USER] HOST [PORT]

Connect to telnet server

Options:

-a Automatic login with $USER variable
-l USER Automatic login as USER

Maybe there is a different Telnet client that can be installed, all I know is that opkg does not list one on the Yun.

While it would be interesting to enable an unbuffered mode in the Yun's Telnet client, I still think the correct fix to the original problem is changing the line terminator in the code so it is not '!', just as jessel has already done.