Poll a server using GET request to retrieve data stream(string), then compare against string2 and turn on output

I have a local server 10.2.0.32 running a power shell script that reads the contents of a database. When the address above is typed in a chrome browser by its ip number:

10.2.0.32 or http://10.2.0.32/mtlqueue/status

It returns or outputs this string in a new browser:

AGE 0UPDATELCDMSG$$MSG$$2:23:25PMEXIT

I'm looking to program a sketch to use Arduino Uno with Ethernet shield to poll this server every minute and read this string in the serial port and have Arduino turn on/off an output pin.
(The string changes from MSG$$ to 'MSG Q size of 2' based on contexts of the what's been loaded into the EPICOR database at any given time)

I've tried using a basic Arduino Client example sketch to connect to the server using client.println("GET http ://10.2.0.32");

When I monitor serial port, the sketch will connect to the server at 10.2.0.32 but once connected it can't retrieve the html.

I don't know how to write that line of code .

Can you help me get started with correct code ?

You already have it.

Post your best attempt and show what output you get and what you wanted/expected.

Use code tags for the code and the serial monitor output.

Here is the code example I am using to setup my ATmega2560 as a client. I would like the arduino to receive a string from a server at 10.2.0.32 and hold it in the serial buffer.
Then I want to display it in the serial monitor, look for the certain characters and do something (turn on led), and display on LCD. I am hung up on the "GET" request format line of code.

By typing this address in my local browser >>>> http://10.2.0.33/mtlqueue/status
A script outputs a blank white webpage with the following string:

AGE 0UPDATELCDMSG$$MSG$$2:23:25PMEXIT

It's this string that reads a database on the server called "mtlqueue". If the database increases in size, the string will change some of its charactera, size and length. For example,

#include <LiquidCrystal.h>
#include <SPI.h>
#include <Ethernet.h>


//setting up LCD
const int rs=12, en=11, d4=5, d5=4, d6=3, d7=2;
LiquidCrystal lcd (rs, en, d4, d5, d6, d7);

// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = { 0x90, 0xA2, 0xDA, 0x0F, 0xAE, 0xD9 };  //alternate mac {0x90,A2,DA,0F,AE,D9};

// if you don't want to use DNS (and reduce your sketch size)
// use the numeric IP instead of the name for the server:
IPAddress server(10,2,0,32);  // numeric IP for your server (no DNS)  
//char server[] = "http://10.2.0.32/mtlqueue/status";    (using DNS)  

// Set the static IP address to use if the DHCP fails to assign
IPAddress ip(10,2,0,25);      //default(192.168.0.177)
IPAddress myDns(10,2,0,1);  //default (192.168.0.1)

// Initialize the Ethernet client library
// with the IP address and port of the server
// that you want to connect to (port 80 is default for HTTP):
EthernetClient client;

// Variables to measure the speed
unsigned long beginMicros, endMicros;
unsigned long byteCount = 0;
bool printWebData = true;  // set to false for better speed measurement

void setup() {
  // You can use Ethernet.init(pin) to configure the CS pin
  //Ethernet.init(10);  // Most Arduino shields
  //Ethernet.init(5);   // MKR ETH shield
  //Ethernet.init(0);   // Teensy 2.0
  //Ethernet.init(20);  // Teensy++ 2.0
  //Ethernet.init(15);  // ESP8266 with Adafruit Featherwing Ethernet
  //Ethernet.init(33);  // ESP32 with Adafruit Featherwing Ethernet

  // Open serial communications and wait for port to open:
  Serial.begin(9600);
  lcd.begin (16,2);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }

  // start the Ethernet connection:
  Serial.println("Initialize Ethernet with DHCP:");
  lcd.print("Initalize ethernet");
    if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // Check for Ethernet hardware present
    if (Ethernet.hardwareStatus() == EthernetNoHardware) {
      Serial.println("Ethernet shield was not found.  Sorry, can't run without hardware. :(");
      while (true) {
        delay(1); // do nothing, no point running without Ethernet hardware
      }
    }
    if (Ethernet.linkStatus() == LinkOFF) {
      Serial.println("Ethernet cable is not connected.");
      lcd.print("cat5 not connected");
    }
    // try to congifure using IP address instead of DHCP:
    Ethernet.begin(mac, ip, myDns);
  } else {
    Serial.print("  DHCP assigned IP ");
    Serial.println(Ethernet.localIP());
    lcd.print(Ethernet.localIP());
    
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);
  Serial.print("connecting to ");    //connecting to server database
  Serial.print(server);              
  Serial.println("...");
  lcd.print(server);   //lcd displays the server
  
// if you get a connection, report back via serial:
  if (client.connect(server, 80)) {  //server=HOST_NAME,HTTP_PORT
    Serial.print("connected to ");  //connected to 10.2.0.32
    Serial.println(client.remoteIP());
    // Make a HTTP request:
    client.println("GET /   /status HTTP/1.1");  //example>> client.println("GET /search?q=mtlqueue/status HTTP/1.1");   
    client.println("http://10.2.0.32/mtlqueue/status");  //example>>  client.println("Host:www.google.com");
    lcd.clear();
    lcd.setCursor(0,0);  //display on top row
    lcd.print("Get request");
    lcd.setCursor(0,1);  //display on botton row
    lcd.print("retrieving data");
    client.println("Connection: close");  //Connection: close
    client.println();  //end HTTTP header
  } else {
    // if you didn't get a connection to the server:
    Serial.println("connection failed");
  }
  beginMicros = micros();
}

 while (client.connected()) {
      if (client.available()) {
        // read an incoming byte from the server and print it to serial monitor:
        char c = client.read();
        Serial.print(c);
      }
    }

  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    endMicros = micros();
    Serial.println();
    Serial.println("disconnecting."); 
    lcd.clear();
    lcd.setCursor(0,0);    
    lcd.print("disconnecting");
    client.stop();
    Serial.print("Received ");
    Serial.print(byteCount);
    Serial.print(" bytes in ");
    float seconds = (float)(endMicros - beginMicros) / 1000000.0;
    Serial.print(seconds, 4);
    float rate = (float)byteCount / seconds / 1000.0;
    Serial.print(", rate = ");
    Serial.print(rate);
    Serial.print(" kbytes/second");
    Serial.println();

    // do nothing forevermore:
    while (true) {
      delay(1);
    }
  }
}
  

I asked you to use code tags when posting output from serial monitor. Please don't post screen captures of serial monitor.

This part does not look right. There is an extra "/" after the "GET" and the "mtlqueue" part is missing.

On the next line, the "Host:" part is missing.

I think there may be other details wrong also. Find an example on-line and play "spot the difference".

Try

client.println("GET /mtlqueue/status HTTP/1.1");
client.println("Host: 10.2.0.32");
client.println("Connection: close");
client.println();

Agreed, I changed/corrected the method syntax and host for the HTTP request.
I've looked at other examples. I got it to work. I received the string!

    // Make a HTTP request:
    client.println("GET http://10.2.0.32/mtlqueue/status HTTP/1.0");      
    client.println("Host: http://10.2.0.32/mtlqueue/status");  

I'm trying to add code to my sketch to turn on a LED if certain characters detected in the data string sent back from the GET request. My new code is not turning on the led.
The serial monitor shows that it's doubling the characters.

AGE 0UPDATELCDMSG $$MSG $$6:13:09 AMEXIT

void loop() {
  // if there's incoming data from the net connection.
  // send it out the serial port.  This is for debugging
  // purposes only:
  if (client.available()) {
    char c = client.read();  //gets one byte from the client
    Serial.write(c);
    lcd.print(c);  //have a serial LCD connected
   Serial.print (c);

//added below code_need to parse the string using different delimiterss //
    command = c;
    command.trim();  //trim the white space off the string

    if (command == "AGE"){
    digitalWrite (blueLed, HIGH);
    digitalWrite (yellowLed, LOW);
     }
    if (command == "Q:2"){
     digitalWrite (yellowLed, HIGH);
      }
//added above code//   
    } 

  // if ten seconds have passed since your last connection,
  // then connect again and send data:
  if (millis() - lastConnectionTime > postingInterval) {
    httpRequest();
  }
}
 

Your code is printing them double:

There's no point trimming white space off what is only ever going to be a single character.

There's no point comparing a string containing 1 character to a string containing 3 characters, it will never match.

Did you copy & paste this code from parts of other people's code without understanding how each part worked? That never succeeds.

I should warn you that if you do the thing I asked you not to do, one more time, I will stop replying.

Thanks for the warning. I didn't expect you to Stop replying if I exceeded your "one more time" critera. You have been a big help which I thankyou for. I meant no harm. I have been running other sketches with a similar execution of what I'm looking to do. I tried to analyze specific portions of code that applied and incorporate it into mine to see how it operates. Yes, I copied and pasted portions of others people code. I'm doing my best to understand why the client.read() was selected as char data type when the incoming data over the ip connection AGE0 UPDATELCDMSG$$MSG $$6:13;09PMEXIT looks like it's a string because of its many characters. This is why I chose to set the variable "command" to a string data type and set it equal to the 'c' variable. It was my attempt to get the data into the serial buffer (if that the correct way) and trigger an action from a few characters.
Can you help me establish some code to turn on a led from these characters 'Q:2AGE:2' if and when they appear in the data stream?
AGE0 UPDATELCDMSG$$MSG $$6:13;09PMEXIT
AGE0 UPDATELCDMSG Q:2 AGE:2 MSG $$6:13;09PMEXIT

Did you read the documentation for this function? If not, do you need help finding it? Usually, a Google search for the "Arduino" and the function name will take to to it.

The function name here is ".read()". It can be used with/on different types of "stream". A stream is an object from which you can receive a stream of characters. The most common example of this is a serial port, which you could read with Serial.read(). In your case, the stream is an HTTP port which is connecting your Arduino to a remote server, which you would read from it with client.read().

The .read() function reads only one character. To read a string, you would need to call this function many times, and concatenate the individual characters into a string.

There are alternative functions available that can read entire strings in a single call.

The page that describes .read() probably has a "see also" section that might allow you to find a function that can read an entire string in a single call, removing the need for you to use .read() many times and concatenate the results.

EDIT: reviewing the Arduino documentation, it is a little confusing for beginners.

Maybe start reading here

void loop() {
  if (client.available()) {
    char c = client.read();  //gets one byte from the client
    Serial.write(c);

Are you saying the read() function as used in above code is reading and storing the data by the client as one character in variable c?

For example, the data which is actually composed of 38 individual characters in length is being read and stored as only one character for every scan cycle in the loop?

AGE0 UPDATELCDMSG$$MSG $$6:13;09PMEXIT

Yes.

In C/C++ language, variables of type char can only hold one character. .read() only reads one character.

If you want a variable that holds a string of characters, there are a couple of options.

You can have a character array, like this:

char myString[30];

or you can use the String class.

String myString;

There are advantages and disadvantages to both.

Okay,
I read that too. But that doesn't mean it's clearly explained for everyone.

This is where I'm having a hard time what to call it. Is it a string or character. I know It's being stored as a char.So technically it should be called a character. It surely don't look like a character when it has 38 characters in it. That looks like a string to me. A string of characters bundled together.
I assumed all along that the piece of data I'm receiving every 10secs was a "string". Am I wrong?

AGE0 UPDATELCDMSG$$MSG $$6:13:09PMEXIT

Are you saying the data is being stored one character at a time or the entire stream as one character ?

What do you call it?

When I try to think through with how the .read() processing works.... I think that it would return 'A' for one loop. Then return 'G' for the next repeat, then 'E' for next repeat, going down the line 38 places ...recreating the entire message:
AGE0 UPDATELCDMSG$$MSG $$6:13:09PMEXIT

Am I correct?

A string is being sent. But your code is reading only 1 character at a time. First it reads the 'A', next time it reads the 'G' and so on. But when it reads the 'G', it has forgotten about the 'A'. The rest of the string is being saved in a buffer until your code reads it. Which it does do, and pretty quickly, faster than the string(s) are being sent. The data is only arriving at 9,600 bits, or 1,200 characters per second, which isn't particularly fast, even for a 16MHz Arduino. EDIT: sorry about that part I deleted, I think I was getting confused with some other forum topic!

So the problem is that your code only ever reads and examines one character at a time. It never sees the whole string at once. To do that, it must either concatenate the individual characters into a complete string, or use a function that reads the entire string in one go instead of one character at a time. In the end, those 2 options are, "under the hood", the same.

Did you take a look at the link I posted? Could you understand it?

Yes, I took a look at some functions: "readString()" came to mind. But they show no examples.

available()
read()
flush()
find()
readBytes()
readBytesUntil()
readString()
readStringUntil()
parseInt()
parseFloat()
setTimeout()

It's so much more helpful to talk to someone.

What you just told me I could not find in the Arduino Stream reference. You have many years of experience under your belt. That was a very helpful howTo what you just explained to me. It's not the way I assumed it was working.
I did know a little about the serial buffer. I learned that on YouTube.

MOVING FORWARD...
So I'm thinking that I want to add and create a variable with type String called (String1) and set it equal to the client.read() char variable 'c' in the code.
Then create a second variable using type String (String2) and set it equal to the actual 38 character text. (This will be used as a reference )
This way I can detect if String1 (actual data string) changes against String2 (reference string).
Then using IF statements I can use the String object functions like compareTo(), equals(), index of() , charAt() , substring to manipulate and turn on an output.

I added couple lines of code that I need to test.
Things to do:

  1. Turn on led if string changes
  2. Break up string and send the substring (34,41) to show on LCD display.
 void loop() {

  if (client.available()) {
    char c = client.read();
    Serial.write(c);
    String readString += c;   //every character read by client add it to readString until it captures entire data stream.
      }
String refString = "AGE0 UPDATELCDMSG Q:2 AGE:2 MSG $$6:13:09PMEXIT" ;  //reference string to check against data string

if (refString.substring(18, 27) == "Q:2 AGE:2"){
digitalWrite (ledPin,HIGH);
}

  // if ten seconds have passed since your last connection,
  // then connect again and send data:

  if (millis() - lastConnectionTime > postingInterval) {

    httpRequest();

       }

}

This won't do what you want, won't do what you put in the comment.

What that line of code will do is create a new, empty string. Then concatenate (add) the character read from the response to the end of the string. On the next line of code there is a } marking the end of the block of code for the if statement. Because the string was declared inside that block, at the end of the block, the string will cease to exist. Next time the if statement is executed, the same thing will happen. So the string will only ever contain one character.

I notice that you enjoy telling me my code won't work. And you yourself offered no actual piece of code to try.

You have a good working knowledge of what I'm trying to achieve because I've been talking to you for some time. You keep telling me your idea of how to solve the Client reading of one character and get it into a string in one call.

So how would you do it?
How would you concatenate the characters into one string?
What function would store the characters into a complete string in one call? Surely you know.

I came very close to dropping off the topic and leaving you to drown when I read that.

Yes, of course I know how to make this work. Yes, I could just give you the answer. That would get you one step further. One step. Then you would be asking for help with the next step and so on. You would quickly become dependant on me, or this forum, and convinced in your own mind that this stuff is beyond your understanding and that the only way to progress with your project is to get others to hand you the solution on a plate, and if they refuse, try to guilt them into it by accusing them of being mean to you. Let's not go to that place.

I'm just trying to give you a nudge in the right direction when I see you going off the path.