WiShield & find() function

system · April 24, 2012, 7:25pm

Hi,
i'm having some troubles with extracting values from a website, using a WiShield (asynclabs) and Arduino un0, in order to light a rgb led.

My arduino is able to connect to the webpage where i want to extract my values, and to print on the serial monitor the result: the html code of the pag which is:

(yes Arduino has to be programed as a client, and yes it absolutely has to be connected via wifi)

I now want to extract the values for r,g,b, and parse them into 3 ints: red, green, blue (0-255) on the arduino, so i can light the 3 leds accordingly.
I know textFinder library can do this, but only for Ethernet connections, and I don't know if the new find() function, that appears to have been implemented with Arduino 1.0, works for the WiShield.

The arduino code that i'm using is as follows, and is based on the example for simple client present in the asynclabs wishield library.
If any of you has an idea on how to solve this please tell me!
If anyone knows where i can get more info on the new find() function (besides arduino.cc reference) i would really appreciate it, even if i have to look at the Core libraries of the Arduino.
Thanks,

/*

A simple sketch that uses WiServer to get the hourly weather data from LAX and prints
it via the Serial API
*/

#include <WiServer.h>

#define WIRELESS_MODE_INFRA 1
#define WIRELESS_MODE_ADHOC 2

// Wireless configuration parameters ----------------------------------------
unsigned char local_ip[] = {10,0,1,3}; // IP address of WiShield
unsigned char gateway_ip[] = {10,0,1,1}; // router or gateway IP address
unsigned char subnet_mask[] = {255,255,255,0}; // subnet mask for the local network
const prog_char ssid[] PROGMEM = {"AABBCC"}; // max 32 bytes

unsigned char security_type = 0; // 0 - open; 1 - WEP; 2 - WPA; 3 - WPA2

// WPA/WPA2 passphrase
const prog_char security_passphrase[] PROGMEM = {"aabbcc"}; // max 64 characters

// WEP 128-bit keys
// sample HEX keys
prog_uchar wep_keys[] PROGMEM = { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, // Key 0
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // Key 1
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // Key 2
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 // Key 3
};

// setup the wireless mode
unsigned char wireless_mode = WIRELESS_MODE_INFRA;

unsigned char ssid_len;
unsigned char security_passphrase_len;
// End of wireless configuration parameters ----------------------------------------

void printData(char* data, int len) {
while (len-- > 0) {
Serial.print(*(data++));
}
}

uint8 ip[] = {1,2,3,4};
GETrequest getPage(ip, 80, "www.aabbcc.com", "/abc.html");

int testing;

void setup(){
WiServer.init(NULL);
//WiServer.enableVerboseMode(true);
Serial.begin(9600);
Serial.print("Connexion wifi lancee");
getPage.setReturnFunc(printData);
delay(1000);
}

long updateTime = 0;

void loop(){
if (millis() >= updateTime) {
getPage.submit();
// Get another update one minute from now
updateTime += 1000 * 60 * 01;
}
WiServer.server_task();
}

system · April 24, 2012, 8:08pm

This is the only place where you do anything with the data:

void printData(char* data, int len) {
  while (len-- > 0) {
    Serial.print(*(data++));
  }
}

So, this is where you have to do the parsing. Use the strtok() function, with appropriate delimiters.

system · April 25, 2012, 9:52am

Thanks PaulS for your advice,

so I changed the void printData () function, and added the strtok() in it. So far i've only been able to cut the webpage into pieces of information.
I only want to get the information concerning the color in numeric values (not strings), and put them into r, g and b.
I'm sorry i'm really not good at handling strings or at C, i really need some help.

int r; int g; int b;

void printData(char* data, int len) {
if (len-- > 0) {
char *resultat= NULL;
resultat = strtok(data, "");
if( !resultat){
Serial.println("not working!");
}
while (resultat != NULL){
Serial.println("result is");
Serial.println(resultat );
resultat = strtok(NULL, "");
}
}
}

And the serial monitor gives me:

result is
HTTP
result is
1.1 200 OK
D
result is
te: Wed, 25 A
result is
r 2012 09:48:36 GMT
Server: A
result is
che
L
result is
t-Modified: Wed, 25 A
result is
r 2012 09:46:23 GMT
ET
result is
g: "816e4162-64-4be7dbe893ec6"
Acce
result is
t-R
result is
ge
result is
: byte
result is

Co
result is
te
result is
t-Le
result is
gth: 100
Co
result is
ectio
result is
: clo
result is
e
Co
result is
te
result is
t-Ty
result is
e: text
result is
html

result is
html
result is

result is
body
result is

result is
id="r"
result is
145
result is

result is
id="g"
result is
100
result is

result is
id="b"
result is
54
result is

result is
b
result is
ody
result is

result is
html
result is

Co

system · April 25, 2012, 11:02am

      resultat = strtok(data, "</span>");

This will find a token that ends at '<', '/', 's', 'p', 'a', or 'n', whichever occurs first. Is that what you want? Seems unlikely.

Add some more Serial.print statements. What is in data when the function is called? Perhaps the function is not being called just once for the whole reply.

system · April 25, 2012, 12:43pm

Ok, I understant the '<','/','s','p','a','n' problem, changed it to the 's' letter, (now i understand why the C example only used a '#' tu cut the string);
I also added a Serial.print(data) to see what the arduino is receiving. here it goes (i know it's a bit messy, sorry):

Code:

void printData(char* data, int len) {
  Serial.print(data);
  if (len-- > 0) {
    char *resultat= NULL;
      resultat = strtok(data, "s"); //cut data into pieces when s is found
      if( !resultat){
         // otput debug error if nothing of the sort is found
         Serial.println("not working!");
      }
      while (resultat != NULL){
        Serial.println("result is");
        Serial.println(resultat );
        resultat = strtok(NULL, "s");
      }
  } 
}

result:
Connexion wifi lanceeHTTP/1.1 200 OK
Date: Wed, 25 Apr 2012 12:33:30 GMT
Server: Apache
Last-Modified: Wed, 25 Apr 2012 09:46:23 GMT
ETag: "816e4162-64-4be7dbe893ec6"
Accept-Rangesresult is
HTTP/1.1 200 OK
Date: Wed, 25 Apr 2012 12:33:30 GMT
Server: Apache
La
result is
t-Modified: Wed, 25 Apr 2012 09:46:23 GMT
ETag: "816e4162-64-4be7dbe893ec6"
Accept-Range
: bytes
Content-Length: 100
Connection: close
Content-Type: text/html

145 100 54 </bresult is : byte result is

Content-Length: 100
Connection: clo
result is
e
Content-Type: text/html

< result is pan id="r">145 < result is pan id="g">100 < result is pan id="b">54 Connedy> Connection: cloresult is ody> Connedy> Connection: clo ody> Connedy> Connection: clo

system · April 25, 2012, 12:54pm

You could make that output easier to understand:

void printData(char* data, int len)
{
  Serial.print("WiFi data: [");
  Serial.print(data);
  Serial.println("]");

  if (len-- > 0)
  {
    char *resultat= NULL;
    resultat = strtok(data, "s"); //cut data into pieces when s is found
    if( !resultat)
    {
         // otput debug error if nothing of the sort is found
       Serial.println("not working!");
    }
    while (resultat != NULL)
    {
       Serial.println("resultat: [");
       Serial.print(resultat);
       Serial.print("]");
       resultat = strtok(NULL, "s");
    }
  } 
}

Connexion wifi lanceeHTTP/1.1 200 OK
Date: Wed, 25 Apr 2012 12:33:30 GMT
Server: Apache
Last-Modified: Wed, 25 Apr 2012 09:46:23 GMT
ETag: "816e4162-64-4be7dbe893ec6"
Accept-Ranges

Is the first part of the data received (as near as I can tell).

What part(s) of that do you want to extract?

: bytes
Content-Length: 100
Connection: close
Content-Type: text/html
145 100 54 </b

Appears to be more input data. This looks like what you are interested in.

You are going to need to use strstr() to see if data contains "span". If it does, then get the token that ends with ", and throw it away. Get the token that ends with ", and save it (it will be 'r'). Then, get the token that ends with >, and throw it away. Get the token that ends with < (145) and convert it to an int. Use the saved value to determine where to store it.

system · April 25, 2012, 1:14pm

Indeed, the first part of the data doesn't interest me, only the 150 things.
I'm gonna try to adapt the sketch, with the elements you are telling me.
I'll let you know if i'm successful or not, thanks again

system · May 2, 2012, 12:22pm

Ok after much struggling i was able to find the strstr "span", and get a token that ends with " or < .
However, i have no clue as to how to throw away the first token ending with ", then finding the next one ending with " (should be r ).
I've not been able to parse the data into an int either, though i imagine it has to do something with the atoi function?

oops, forgot to copy/paste the code:

char * resultat=NULL;
char * resultattwo=NULL;

void printData(char* data, int len) {
  
  /*while (len-- > 0) { 
      Serial.println(data);
  }*/
  char * spanstring = "span";
  resultat = strstr(data, spanstring);
  if (resultat != NULL){
    Serial.println("I found strstr");
    Serial.println(resultat);
    char * tokenstringone = "<";
    resultattwo = strtok(resultat, tokenstringone);
    Serial.println("I found strtok");
    Serial.println(resultattwo);
   
    //  char * token = strtok(data, "\"");
  }
}

and the ouput:

Connexion wifi lanceeI found strstr
span id="r">145
100
54
</b
I found strtok
span id="r">145
I found strstr
span id="r">145
I found strtok
span id="r">145
I found strstr
span id="r">145
I found strtok
span id="r">145

why is it repeating itself? Oo

system · May 2, 2012, 1:07pm

    Serial.println("I found strstr");
    Serial.println(resultat);

should be

    Serial.print("resultat: [");
    Serial.print(resultat);
    Serial.println("]");

Ditto for the rest of the print statements.

Once you know that resultat points to:

span id="r">145
100
54
</b

What do you want to extract from this. I thought that you were looking for the value in the "" and the value between the > and the <. The token you are telling strtok to get you is the one ending with <, or "span id="r">145". I have no idea what you expect to get from this token.

All of my code contains output statements (often commented out) that shows when a function is called, and when it ends. Yours should too.

system · May 2, 2012, 1:35pm

You're right i should not be looking for a token ending at < .
And yes i want to get what is inside of "" and > <.

a cleaner code, where i manage to find r :

void printData(char* data, int len) {
  
  /*while (len-- > 0) { 
      Serial.println(data);
  }*/
  
  char * spanstring = "span";
  resultat = strstr(data, spanstring); // find the string "span" within data
  
  if (resultat != NULL){               // if there is a result (resultat)
    Serial.println();
    Serial.print("resultat: [");
    Serial.print(resultat);
    Serial.println("]");
  
    char * tokenstringone = "\"";      // get the token ending at " (resultattwo should be: [span id=])
    resultattwo = strtok(resultat, tokenstringone);
    Serial.print("resultattwo: [");
    Serial.print(resultattwo);
    Serial.println("]");
    
    resultatthree = strtok(NULL, tokenstringone); // get the next token ending at "  (resultatthree should be: [r])
    Serial.print("resultatthree: [");
    Serial.print(resultatthree) ;
    Serial.println("]");
  }
  else if (resultat == NULL){
    Serial.println("no span found");
  }
}

result:

resultat: [span id="r">145
100
54
</b]
resultattwo: [span id=]
resultatthree: [r]

resultat: [span id=]
resultattwo: [span id=]
resultatthree: [É]

I dont understand why resultat changes though

system · May 2, 2012, 1:51pm

I dont understand why resultat changes though

It's how strtok() works. You always make a copy of the original data for parsing, if it is necessary to preserve the original data.

void printData(char* data, int len)
{
Serial.println("printData ==>");
// snipped
Serial.println("printData <==");
}
We need to know when the function is being called!

system · May 2, 2012, 3:14pm

hum, sorry i did not understand your last post

void printData(char* data, int len)
{
Serial.println("printData ==>");
// snipped
Serial.println("printData <==");
}
We need to know when the function is being called!

system · May 2, 2012, 4:05pm

hum, sorry i did not understand your last post

I want you to add the highlighted lines to your function, where I show them.

Topic		Replies	Views
Reading web page over wifi issues Programming Questions	4	1334	May 5, 2021
Help parsing text file through wishield Syntax & Programs	9	4182	May 6, 2021
WiShield + Uno = can't retrieve webpage using sample PDE (code included) Programming Questions	17	6360	May 5, 2021
SimpleClient erro Programming Questions	1	497	May 5, 2021
Wishield 2 with modern IDE (1.8.7) Networking, Protocols, and Devices	6	795	May 6, 2021

WiShield & find() function

Related topics