Parse data

Hi,
I want to parse an HTTP Get response I receive on the serial port.
Following is a sample response I receive

<?xml version='1.0' encoding='UTF-8'?><feed xmlns='http://www.w3.org/2005/Atom' xmlns:openSearch='http://a9.com/-/spec/opensearchrss/1.0/' xmlns:gs='http://schemas.google.com/spreadsheets/2006' xmlns:batch='http://schemas.google.com/gdata/batch'><id>https://spreadsheets.google.com/feeds/cells/0ApZSAfRKvUfDdElxdmRLZ1FqSEVKejVJTjRoZmRvVEE/od6/public/basic</id><updated>2013-09-22T09:15:09.974Z</updated><category scheme='http://schemas.google.com/spreadsheets/2006' term='http://schemas.google.com/spreadsheets/2006#cell'/><title type='text'>Form Responses</title><link rel='alternate' type='text/html' href='https://spreadsheets.google.com/pub?key=0ApZSAfRKvUfDdElxdmRLZ1FqSEVKejVJTjRoZmRvVEE'/><link rel='http://schemas.google.com/g/2005#feed' type='application/atom+xml' href='https://spreadsheets.google.com/feeds/cells/0ApZSAfRKvUfDdElxdmRLZ1FqSEVKejVJTjRoZmRvVEE/od6/public/basic'/><link rel='http://schemas.google.com/g/2005#batch' type='application/atom+xml' href='https://spreadsheets.google.com/feeds/cells/0ApZSAfRKvUfDdElxdmRLZ1FqSEVKejVJTjRoZmRvVEE/od6/public/basic/batch'/><link rel='self' type='application/atom+xml' href='https://spreadsheets.google.com/feeds/cells/0ApZSAfRKvUfDdElxdmRLZ1FqSEVKejVJTjRoZmRvVEE/od6/public/basic?range=d10'/><author><name>igor.kovba</name><email>igor.kovba@gmail.com</email></author><openSearch:totalResults>1</openSearch:totalResults><openSearch:startIndex>1</openSearch:startIndex><entry><id>https://spreadsheets.google.com/feeds/cells/0ApZvAfRKvUfDdElxdmRLZ1FqSEVKejVJTjRoZmRvVEE/od6/public/basic/R10C4</id><updated>2013-09-22T09:15:09.974Z</updated><category scheme='http://schemas.google.com/spreadsheets/2006' term='http://schemas.google.com/spreadsheets/2006#cell'/><title type='text'>D10</title><content type='text'>456</content><link rel='self' type='application/atom+xml' href='https://spreadsheets.google.com/feeds/cells/0ApZSAfRKvUfDdElxdmRLZ1FqSEVKejVJTjRoZmRvVEE/od6/public/basic/R10C4'/></entry></feed>

I want to read the value in D10 i.e. parse
type='text'>D10456

so that I am able to read the value 456 in the cell D10. Please help me with a code to parse the data

To parse xml files it's usually better to consider the '>' and '/n' as the end of line, so you don't need a larger buffer, since each xml file line are usually very big.

I use something similar to get some weather information. Here is an ideia.

void weather() {
  byte idx = 0;
  char character;
  int timeout = 0;
  char icon[12] = {};
  char response[64] = {};
  if (tcp.connect("site", TCP_PORT) == 1) {
    while (tcp.connected()) {
      while (tcp.available()) {
        timeout = 0;
        character = tcp.read();
        if (idx < sizeof(response) - 1) {
          response[idx++] = character;
        }
        if (character == '\n' || character == '>') {
          if (strstr(response, "temperature") && strstr(response, "value")) {
            sscanf(strstr(response, "value"), "value=\"%hhd\"", &temp);
            //process
            tcp.stop();
          }
        }
        memset(response, 0, sizeof(response));
        idx = 0;
      }
      if (timeout++ >= 1000) {
        tcp.stop();
      }
      delay(1);
    }
  }
  tcp.stop();
}

Could you get the spreadsheet to output the data in CSV format which would be easier to parse?

...R

Can you please help me with your example for the code I have posted. Especially the functions strstr,sscanf and memset. How do I use them in a program for arduino?

Arank:
To parse xml files it's usually better to consider the '>' and '/n' as the end of line, so you don't need a larger buffer, since each xml file line are usually very big.

I use something similar to get some weather information. Here is an ideia.

void weather() {

byte idx = 0;
  char character;
  int timeout = 0;
  char icon[12] = {};
  char response[64] = {};
  if (tcp.connect("site", TCP_PORT) == 1) {
    while (tcp.connected()) {
      while (tcp.available()) {
        timeout = 0;
        character = tcp.read();
        if (idx < sizeof(response) - 1) {
          response[idx++] = character;
        }
        if (character == '\n' || character == '>') {
          if (strstr(response, "temperature") && strstr(response, "value")) {
            sscanf(strstr(response, "value"), "value="%hhd"", &temp);
            //process
            tcp.stop();
          }
        }
        memset(response, 0, sizeof(response));
        idx = 0;
      }
      if (timeout++ >= 1000) {
        tcp.stop();
      }
      delay(1);
    }
  }
  tcp.stop();
}

Is the stuff before the number you are interested in ALWAYS D10

...R

Yes the data will be present between 'text'> and where the value 456 is present in the given example. The HTTP reply will remain same always , however the data can change and can be either of integer or float( signed or unsigned) datatype.

type='text'>D10</title><content type='text'>456</content>

I think you could then use the function strcspn() to find the start of the standard preceding text and find the number by counting forward from that position.

...R

Hi, I am using the web client example for arduino to obtain data from a xml source.
The HTTP response is obtained in the variable char c. I am trying to convert the characters obtained in the response to a string using the variable char a[] using the code given below. But I am unable to do so. Please help if something is wrong with the following piece of code.

 if(client.available()) {
    char c = client.read();
    a[i]=c;
    i=i+1;
    Serial.print(c);
  }
  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");


    i=0;
    client.stop();
// Serial.println(i);

    // do nothing forevermore:
    while (true);
  }

The HTTP response is obtained in the variable char c

One character at a time.

I am trying to convert the characters obtained in the response to a string using the variable char a[] using the code given below.

But, we can't see how a is defined. I'll assume "incorrectly".

But I am unable to do so.

What proof do you have to support that statement?

@ PaulS

1)Yes, response is one char at a time.

komalhm:
variable char a[]

as mentioned in my previous post.

3)The serial monitor does not show any data. When a[] and i are commented it displays the characters.

as mentioned in my previous post.

But there is no code to prove that you have sized the array correctly.

3)The serial monitor does not show any data. When a[] and i are commented it displays the characters.

I stand by my assertion that "incorrectly" is how you have defined the array a.