Getting client.read characters into a string [SOLVED]

I’m unschooled in C++ so my knowledge of this language (and programming in general) is spotty and lacking the depth required of basic troubleshooting. Serves me right for trying to tackle such a complicated problem, so before I run screaming from my studio, I thought I should put it up on the forum in case there are some simple changes I might make to get me back on track…

What I’ve got:
I want to use an Ethernet Shield Arduino to grab a string from a php script on a web server and display it on a scrolling sign made up of four SURE 0832 dot matrix LED units positioned end-to-end. I have the scrolling sign working to display a string that is coded into the sketch, and I am able to log into the web server. I’m stuck at actually being able to capture the characters from the web server and store them into a string so that they can be displayed.

I’m stuck:
I’m not at all clear about how to get each individual character into a string using the client.read() command. I know that Arduino 1.0 supports some new commands like stream.readBytes(), but I’ve not been able to get them to work. The two problem lines in the code are

char cutup = client.read();
Serial.println(cutup);

I’m also apparently not getting the code hierarchy, because with the code as I have it, I should be able to see the string in the Serial Monitor, but it does not display, only the “connecting… connected” announcement is showing. I’d like to be able to tell Arduino to fetch the online text in setup(), and then use loop() to scroll the text on the sign, but I don’t think I’ve got that right yet, either.

Here is my code:

#include <SPI.h>
#include <Ethernet.h>
#include <font_5x4.h>
#include <HT1632.h>

int wd; 
int i = 1;

byte mac[] = {  
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress server(24,222,117,110); // local

// Initialize the Ethernet client library
EthernetClient client;

void setup() {
  // start the serial library:
  Serial.begin(9600);

  HT1632.begin(9, 8, 7, 6, 5, 4);
  wd = HT1632.getTextWidth("Art can make you rich overnight. . . . . .", 
  FONT_5X4_WIDTH, FONT_5X4_HEIGHT);

  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    for(;;)
      ;
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);
  Serial.println("connecting...");

  // if you get a connection, report back via serial:
  if (client.connect(server, 8888)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET /cutup/1.php");
    client.println();
    delay(500);
    
    // if there are incoming bytes available 
    // from the server, read them and print them:
    if (client.available()) {
      char cutup = client.read();
      Serial.println(cutup);
    }
  } 
  else {
    // if you didn't get a connection to the server:
    Serial.println("connection failed");
  }
  
  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();
  }
}

void loop()
{
  // Select board 1 as the target of subsequent drawing/rendering operations.
  HT1632.drawTarget(BUFFER_BOARD(1));
  HT1632.clear();
  HT1632.drawText("Art can make you rich overnight. . . . . .", 4*OUT_SIZE - i, 0,
  FONT_5X4, FONT_5X4_WIDTH, FONT_5X4_HEIGHT, FONT_5X4_STEP_GLYPH);
  HT1632.render(); // Board 1's contents is updated.

  // Select board 2 as the target of subsequent drawing/rendering operations.
  HT1632.drawTarget(BUFFER_BOARD(2));
  HT1632.clear();
  HT1632.drawText("Art can make you rich overnight. . . . . .", 3*OUT_SIZE - i, 0,
  FONT_5X4, FONT_5X4_WIDTH, FONT_5X4_HEIGHT, FONT_5X4_STEP_GLYPH);
  HT1632.render(); // Board 2's contents is updated.

  // Select board 3 as the target of subsequent drawing/rendering operations.
  HT1632.drawTarget(BUFFER_BOARD(3));
  HT1632.clear();
  HT1632.drawText("Art can make you rich overnight. . . . . .", 2*OUT_SIZE - i, 0,
  FONT_5X4, FONT_5X4_WIDTH, FONT_5X4_HEIGHT, FONT_5X4_STEP_GLYPH);
  HT1632.render(); // Board 3's contents is updated.


  // Select board 4 as the target of subsequent drawing/rendering operations.
  HT1632.drawTarget(BUFFER_BOARD(4));
  HT1632.clear();
  HT1632.drawText("Art can make you rich overnight. . . . . .", OUT_SIZE - i, 0,
  FONT_5X4, FONT_5X4_WIDTH, FONT_5X4_HEIGHT, FONT_5X4_STEP_GLYPH);
  HT1632.render(); // Board 4's contents is updated.
  
  i = (i+1)%(wd + OUT_SIZE * 2); // Make it repeating.
  delay(4);
}
  if (client.connect(server, 8888)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET /cutup/1.php");
    client.println();
    delay(500);
    
    // if there are incoming bytes available 
    // from the server, read them and print them:
    if (client.available()) {
      char cutup = client.read();
      Serial.println(cutup);
    }
  }

Make a request. Wait 1/2 second for the server to respond. If there is a character sent back read it and send it to the serial port. Ignore anything else that the server might be sending.

You need to read the data in a while loop, looking at both client.connected() and client.available(). You need to store the data in an array of characters, or (preferably not) a String.

Start by dumping the code you have in loop. All of it. Concentrate on getting something to display before you worry about how to display it.

Move the code to make the request into loop(). Use a boolean flag to send the request only once. Then use a while loop, checking both client.connected() and client.available(), to read all the data and send it to the serial monitor.

When that works, work on adding code to collect the characters in an array.

When that works, add the code back to display the message on your hardware.

Good advice, thanks. I did as you suggested, and went back a few steps. This amended code works to display the characters from the web server on the Serial Monitor. I didn’t want to complicate things by adding the boolean flag, so this is right now as simple as simple can be (I think):

#include <SPI.h>
#include <Ethernet.h>

byte mac[] = {  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress server(24,222,117,110); // 

EthernetClient client;

void setup() {
  // start the serial library:
  Serial.begin(9600);
  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    for(;;)
      ;
  }

  delay(1000);
  Serial.println("connecting...");

  if (client.connect(server, 8888)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET /cutup/1.php");
    client.println();
  } 
  else {
    Serial.println("connection failed");
  }
}

void loop()
{
  // if there are incoming bytes available 
  // from the server, read them and print them:
  if (client.available()) {
    char c = client.read();
    Serial.print(c);
  }

  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();

    // do nothing forevermore:
    for(;;)
      ;
  }
}

So,

When that works, work on adding code to collect the characters in an array.

Can you point me in the right direction on how to do this? My initial thought would be to put this

char c = client.read();

into a loop that keeps adding a character to an element in an array, but how best to do this: how would you determine the size of the array if it’s sitting on a server, and how would you escape from the loop when it has reached the end?

On each pass through loop(), you read at most one character. That’s fine. You could add another loop to read more than one, but it really isn’t necessary. Move one step at a time.

Now, to collect the data in an array, you need to define an array and an index into that array (where in the array to put the character.

Add this:

char buffer[80];
int index = 0;

before setup.

In loop, after the char c = statement, add:

if(index < 79)
{
   buffer[index] = c;
   index++;
   buffer[index] = '\0';
}

This will collect all the data in an array.

In the block of code that stops listening to the server, after the server disconnects, add:

Serial.print("Server response: ");
Serial.println(buffer);

When that compiles and works, we’ll need to work on this:

I didn’t want to complicate things by adding the boolean flag, so this is right now as simple as simple can be

It really is necessary to “complicate” things with the boolean, so that the loop() function can continue to do other things. As it is right now, the Arduino stops doing anything but that infinite loop when the server closes the connection.

Many thanks PaulS for your help. Got it working after adding the boolean flag. For the benefit of those who were wondering how I got it all to function with the SURE 0832 displays, here’s the code:

/*

 Media Circus functional v01
 by Michael LeBlanc, NSCAD University 2012
 http://generaleccentric.net
 with assistance from PaulS [http://arduino.cc/forum/index.php?action=profile;u=16084] 
 on the Arduino Forum
 
 Ethernet Shield code created 18 Dec 2009 by David A. Mellis
 
 HT1632 library by Gaurav Manek: 
 https://github.com/gauravmm/HT1632-for-Arduino/tree/master/Arduino/HT1632
 
 Notes on this version:
 Fully functional network cutup fetch, display on 0832 scrolling sign.
 
 */

#include <SPI.h>
#include <Ethernet.h>
#include <font_5x4.h>
#include <HT1632.h>

int wd; 
int i = 1;
char buffer[1000];
int index = 0;
boolean fetched = false;

byte mac[] = {  
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress server(nn,nnn,nnn,nnn); // get your own server, mate!

EthernetClient client;

void setup() {

  HT1632.begin(9, 8, 7, 6, 5, 4);

  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    for(;;)
      ;
  }

  delay(1000);

  if (client.connect(server, 8888)) {
    client.println("GET /cutup/cutup_1.php");
    client.println();
  } 
  else {
  }
}

void loop()
{
  if ((client.available()) && (fetched == false)) {
    char c = client.read();
    if(index < 999)
    {
      buffer[index] = c;
      index++;
      buffer[index] = '\0';
    }
  }

  // if the server's disconnected, stop the client:
  if ((!client.connected()) && (fetched == false)) {
    client.stop();
    wd = HT1632.getTextWidth(buffer, FONT_5X4_WIDTH, FONT_5X4_HEIGHT);
    fetched = true;
  }
  if (fetched == true) {

    // Select board 1 as the target of subsequent drawing/rendering operations.
    HT1632.drawTarget(BUFFER_BOARD(1));
    HT1632.clear();

    HT1632.drawText(buffer, 4*OUT_SIZE - i, 0,
    FONT_5X4, FONT_5X4_WIDTH, FONT_5X4_HEIGHT, FONT_5X4_STEP_GLYPH);

    HT1632.render(); // Board 1's contents is updated.

    // Select board 2 as the target of subsequent drawing/rendering operations.
    HT1632.drawTarget(BUFFER_BOARD(2));
    HT1632.clear();

    HT1632.drawText(buffer, 3*OUT_SIZE - i, 0,
    FONT_5X4, FONT_5X4_WIDTH, FONT_5X4_HEIGHT, FONT_5X4_STEP_GLYPH);

    HT1632.render(); // Board 2's contents is updated.

    // Select board 3 as the target of subsequent drawing/rendering operations.
    HT1632.drawTarget(BUFFER_BOARD(3));
    HT1632.clear();

    HT1632.drawText(buffer, 2*OUT_SIZE - i, 0,
    FONT_5X4, FONT_5X4_WIDTH, FONT_5X4_HEIGHT, FONT_5X4_STEP_GLYPH);

    HT1632.render(); // Board 3's contents is updated.


    // Select board 4 as the target of subsequent drawing/rendering operations.
    HT1632.drawTarget(BUFFER_BOARD(4));
    HT1632.clear();

    HT1632.drawText(buffer, OUT_SIZE - i, 0,
    FONT_5X4, FONT_5X4_WIDTH, FONT_5X4_HEIGHT, FONT_5X4_STEP_GLYPH);

    HT1632.render(); // Board 4's contents is updated.

    i = (i+1)%(wd + OUT_SIZE * 2); // Make it repeating.
    delay(4);   
  }
}
char buffer[1000];

That's half of your available memory. Is it really necessary to make the buffer that large?

That's half of your available memory. Is it really necessary to make the buffer that large?

It would be half if i was using an UNO, but I've graduated to using a Mega 1280 because I have up to 1000 bytes of text. I've played with chopping the text into smaller pieces and trying to display them to the scrolling sign progressively, but the display library that I'm using doesn't seem to allow this without resetting the scrolling. I've also looked at saving the text in EEPROM, but I don't know what advantage this would ultimately grant me... I'm open to suggestions, though.

I've graduated to using a Mega 1280 because I have up to 1000 bytes of text.

How many characters can you display at a time? How long does it take to scroll a 1000 character message? Is anyone still paying attention at the end, and remember the first part of the book when the message ends?

The content to be scrolled is headlines separated by several ellipses, so the viewer will be able to read almost the entire "headline" across the display. You can read the current set of headlines by going to http://nscad.dyndns.org:8888/cutup/cutup_1.php.

These are not real headlines--they're 'cutups' in the manner of William Burroughs--from two separate Canadian news organizations. When you read them, they sound somewhat plausible.

The content to be scrolled is headlines separated by several ellipses

So, you only need to collect in memory one headline at a time, not all 1000 bytes.

Yes, and if we could do this without re-starting the scroller, I would do it. The HT1632 library seems to require the entire string in one go, however. I want the next headline to directly follow the previous headline's ellipsis, and this does not seem possible if you feed it one chunk at a time.

The HT1632 library is here: https://github.com/gauravmm/HT1632-for-Arduino

I'm open to exploring other libraries, but the other ones I looked at for the SURE 0832 last year didn't support scrolling.

I'll drop Gaurav Manek an email and let you know what he says...

Gaurav responds by email with some positive news (quote below)... Before I was able to get the project to this point, I had ordered some SPI RAM for this very purpose. I'll be trying this soon and I'll post what transpires.

Hi Michael,

It's certainly possible to do this (and quite easily, too).

I'm making the assumption that the data is being buffered at the server (that means that you receive the headlines in small chunks), and that one headline is sufficient to fill the entire screen (which can be easily changed in the code).

Starting from the code in your latest post, make the following changes:

  1. Instead of having char buffer[1000], have char buffer[2][128], so that you can maintain two separate news items in memory concurrently.
  2. Change int wd; to int wd[2];, one for each buffer.
  3. Add a new int buffer_to_render_first = 0;

Refer to the code here: http://pastebin.com/ycR8NVLR

I haven't actually tested the code yet, so expect to find some bugs. The overall logic, though, should be sound.

The project will appear to viewers as text scrolls in from the right, and momentarily pauses when the currently-displayed news item reaches the leftmost position on the screen. The pause is due to the time taken for the next news item to be loaded.

I would advise against using this method, though, as network traffic is unpredictable at best and the overheads involved are huge. It would be much better to maintain a buffer on the client side, using either internal EEPROM (which is not really a good practice) or a RAM module (see: SpiRAM).

Best of luck for your project! Do let me know how it goes.

Regards,

Gaurav