WiFlySerial and WiFly RN-XV

b) I'm using the wrong RX/TX in WiFlySerial wifi(2, 3);?

Yes, you're using the pins 2 and 3 but your shield is connected to 0 and 1 (Arduino Shield List: DFRobot Xbee Shield). With this shield you should think about using another library that uses the hardware UART instead of the SoftwareSerial library. Because you are using the same library for debugging and only one instance can be actively receiving data.

pylon:
Yes, you're using the pins 2 and 3 but your shield is connected to 0 and 1 (Arduino Shield List: DFRobot Xbee Shield).

ok I did suspected that was the case and had tried 0, 1 but it didn't work for me (what I didn't know was that using softwareserial to debug was causing a conflict). I've commented out all the software serial debugging lines for now and in serial monitor I now see the line:

open http://www.mysite.com/ 80

every three seconds so it looks like it's connecting and the loop is firing. However when I check my php page (which saves to a txt counter file) it's not being called. The full url is (I've changed the domain name here):

http://www.mysite.com/in-progress/arduino-test/counter/index.php?count=1

The php works fine and the txt has write permissions. This is how my query string is being formed:

#define MY_SERVER_GET "http://www.mysite.com"
#define MY_SERVER_GET_URL "in-progress/arduino-test/counter/index.php"
strRequest << F("GET ") << MY_SERVER_GET_URL << F("?count=") << iLoopCounter 
    << F(" HTTP/1.1") << "\n"
    << F("Host: ") << MY_SERVER_GET << "\n"
    << F("Connection: close") << "\n"
    << "\n\n";

Is this being formed correctly? What's the 80 in the serial monitor message, the port or an error number?

pylon:
With this shield you should think about using another library that uses the hardware UART instead of the SoftwareSerial library. Because you are using the same library for debugging and only one instance can be actively receiving data.

I'm looking for an alternative.

Thanks.

Garrett

#define MY_SERVER_GET_URL "in-progress/arduino-test/counter/index.php"

should be

#define MY_SERVER_GET_URL "/in-progress/arduino-test/counter/index.php"

The leading slash is needed.

ah sorry I mistyped it here when I changed the url, the slash is there on the MY_SERVER_GET

#define MY_SERVER_GET "http://www.mysite.com/"
#define MY_SERVER_GET_URL "in-progress/arduino-test/counter/index.php"

the slash is there on the MY_SERVER_GET

But that's not where it needs to be.

I have tried

#define MY_SERVER_GET "http://www.mysite.com/"
#define MY_SERVER_GET_URL "in-progress/arduino-test/counter/index.php"

and this

#define MY_SERVER_GET "http://www.mysite.com"
#define MY_SERVER_GET_URL "/in-progress/arduino-test/counter/index.php"

Still no change, my php page doesnot get called.

This

#define MY_SERVER_GET "www.mysite.com"
#define MY_SERVER_GET_URL "http://www.mysite.com/in-progress/arduino-test/counter/index.php"

without the http:// seems to be doing something. In the Serial Monitor I'm now getting:

$$open www.mysite.com 80
GET http://www.mysite.com/in-progress/arduino-test/counter/index.php?count=237 HTTP/1.1
Host: www.mysite.com
Connection: close

with the count number incrementing up each time. My php counter however still doesn't seem to be called. The full code is below again.

thanks
Garrett

#include <Arduino.h>
#include <Time.h>
#include <SoftwareSerial.h>
#include <Streaming.h>
#include <PString.h>
#include <WiFlySerial.h>


//various buffer sizes
#define REQUEST_BUFFER_SIZE 180
#define TEMP_BUFFER_SIZE 60

//server hosting GET example php script
#define MY_SERVER_GET "www.mysite.com"
#define MY_SERVER_GET_URL "http://www.mysite.com/in-progress/arduino-test/counter/index.php"

//xbee pro shiel uses pins 0, 1
WiFlySerial wifi(0, 1);
char bufRequest[REQUEST_BUFFER_SIZE];
char bufTemp[TEMP_BUFFER_SIZE];

//sotware serial pins for testing
//SoftwareSerial mySerial(4, 5);

//loop counter
int iLoopCounter = 0;

//start time
unsigned long startTime = 0;


void setup()
{
  //set up serial
  //Serial.begin(9600);
  
  //set the data rate for the SoftwareSerial port and send a message to test
  //mySerial.begin(9600);
  //mySerial.println("Software serial working");
  
  //start up WiFly
  wifi.begin();

  //mySerial.println("Joined");
}


void loop()
{
  //calculate the time since last time the cycle was completed
  unsigned long loopTime = millis() - startTime;

  //to test
  //mySerial.println("test 0");

  //if the timer is greater than or equal to 5 seconds (5000 milliseconds)
  if (loopTime >= 5000)
  {
    char bufRequest[REQUEST_BUFFER_SIZE];
    PString strRequest(bufRequest, REQUEST_BUFFER_SIZE);
      
    // Build GET expression
    strRequest << F("GET ") << MY_SERVER_GET_URL << F("?count=") << iLoopCounter 
    << F(" HTTP/1.1") << "\n"
    << F("Host: ") << MY_SERVER_GET << "\n"
    << F("Connection: close") << "\n"
    << "\n\n";
    // send data via request
    // close connection
  
    //to see what the GET request looks like
    //mySerial << F("GET request:")  << strRequest << endl;
      
    //open connection, then send GET Request, and display response.
    if (wifi.openConnection(MY_SERVER_GET))
    {
      wifi <<  (const char*) strRequest << endl;
    
      //to test
      //mySerial.println("test 2");
    
      // Show server response

      //how long to wait before timing out on the GET request
      unsigned long TimeOut = millis() + 3000;

      while (millis()  < TimeOut && wifi.isConnectionOpen())
      {
        if (wifi.available() > 0)
        {
            wifi.read();
        }
      }

      //force-close connection
      wifi.closeConnection();
    }
    else
    {
      //failed to open
      //mySerial.println("Failed to open connection");
    }
  
    //restart timer
    startTime = millis();
  }

  //increment the loop counter
  iLoopCounter++;

  //to prevent spamming serial monitor too much
  delay(250);
}

This is not correct:

GET http://www.mysite.com/in-progress/arduino-test/counter/index.php?count=237 HTTP/1.1

On the GET line you must not supply a URL (with the http:// and the server) but the path (location) on the server. And use HTTP/1.0 as you don't use any HTTP 1.1 feature, some servers are a bit picky about that.

Have you checked if you WiFly gets an IP? And a DNS server? Does your www.mysite.com resolve on that DNS server?

Have tried doing the same request manually (using the telnet command on the command line)?

pylon:
On the GET line you must not supply a URL (with the http:// and the server) but the path (location) on the server.

You mean path client side not server side right?

pylon:
Have you checked if you WiFly gets an IP? And a DNS server? Does your www.mysite.com resolve on that DNS server?

Through my router configuration panel yes. It has the ip 192.168.1.68. A DNS server, for my WiFly or for www.mysite.com? Mysite.com works, I can access it through a browser. If I type it's ip address it also works.

pylon:
Have tried doing the same request manually (using the telnet command on the command line)?

Yes and it works fine.

telnet mysite.com 80
Trying 62.81.51.152...
Connected to mysite.com.

You mean path client side not server side right?

No, I mean server side. A web client requests a resource on the server. That resource has a path (better: location) on the server and that's what you have to supply on the GET line.

I can access it through a browser.

That doesn't mean it's available for the Arduino. Is that www.mysite.com a local or a remote site? Meaning, is it in your LAN or somewhere else in the Internet?
Your browser can also access the site if it's IP is registered in the hosts file or (if you're working on windows) if you gave it a local WINS name. Arduino is not Windows, it just asks the DNS server. I don't think this is a problem, I just wanna check it too.

telnet mysite.com 80
Trying 62.81.51.152...
Connected to mysite.com.

This is just a connection test, you haven't sent the request. After you have the connection send that line with hitting return twice at the end:

GET /in-progress/arduino-test/counter/index.php?count=237 HTTP/1.0

Post the result you get (you can change all IPs and domain names if your security imposes).

pylon:
[No, I mean server side. A web client requests a resource on the server. That resource has a path (better: location) on the server and that's what you have to supply on the GET line.

I think we are talking at cross purposes though. I'm used to programming in things like php and when I mean server-side path I don't mean the tail end of the url.

i.e. not /in-progress/arduino-test/counter/index.php from:
http://www.mysite.com/in-progress/arduino-test/counter/index.php

I mean the server path of that file which might be:
/www/html/in-progress/arduino-test/counter/index.php

You do mean the tail end of the url don't you, as it's visible on the web?

pylon:
That doesn't mean it's available for the Arduino. Is that www.mysite.com a local or a remote site? Meaning, is it in your LAN or somewhere else in the Internet?

It's remote.

pylon:
This is just a connection test, you haven't sent the request. After you have the connection send that line with hitting return twice at the end:

GET /in-progress/arduino-test/counter/index.php?count=237 HTTP/1.0

Post the result you get (you can change all IPs and domain names if your security imposes).

Here's the output, looks fine.

telnet mysite.com 80
Trying 62.81.51.152...
Connected to mysite.com.
Escape character is '^]'.

GET /in-progress/arduino-test/counter/index.php?count=237 HTTP/1.0

HTTP/1.1 200 OK
Date: Tue, 17 Jul 2012 12:40:48 GMT
Server: Apache/2.0.52 (Red Hat)
X-Powered-By: PHP/5.2.2
Content-Length: 0
Connection: close
Content-Type: text/html

I guess your PC is in the same network as the Arduino is. Use WireShark to sniff that network, set the following filter:

host 192.168.1.68

(given the Arduino still has the same address). Then start the Arduino by resetting it or power it up. If you don't get output, there's probably still an error in the setup of the WiFi network. If you get packets, try to identify the HTTP connection, select one of the packets and select "Follow TCP stream" from the "Analyze" menu. Post the result if you don't get the failure reason yourself.

ok, I couldn't filter the hosts, WireShark just kept telling me the syntax was wrong but I did a capture and located the following for UDP:

No.     Time           Source                Destination           Protocol Length Info
    345 5.138944000    192.168.1.68          255.255.255.255       UDP      152    Source port: cisco-sccp  Destination port: 55555

Frame 345: 152 bytes on wire (1216 bits), 152 bytes captured (1216 bits) on interface 0
    Interface id: 0
    WTAP_ENCAP: 1
    Arrival Time: Jul 17, 2012 18:01:55.419436000 BST
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1342544515.419436000 seconds
    [Time delta from previous captured frame: 0.019490000 seconds]
    [Time delta from previous displayed frame: 0.019490000 seconds]
    [Time since reference or first frame: 5.138944000 seconds]
    Frame Number: 345
    Frame Length: 152 bytes (1216 bits)
    Capture Length: 152 bytes (1216 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
    [Protocols in frame: eth:ip:udp:data]
    [Coloring Rule Name: UDP]
    [Coloring Rule String: udp]
Ethernet II, Src: RovingNe_80:1f:f2 (00:06:66:80:1f:f2), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
Internet Protocol Version 4, Src: 192.168.1.68 (192.168.1.68), Dst: 255.255.255.255 (255.255.255.255)
User Datagram Protocol, Src Port: cisco-sccp (2000), Dst Port: 55555 (55555)
Data (110 bytes)

0000  00 18 f6 93 6c cd 01 28 07 d0 00 00 01 42 0c 4a   ....l..(.....B.J
0010  0d 11 54 69 6d 65 20 4e 4f 54 20 53 45 54 00 00   ..Time NOT SET..
0020  57 69 46 6c 79 20 56 65 72 20 32 2e 33 32 2c 20   WiFly Ver 2.32, 
0030  30 32 2d 31 33 2d 32 30 31 32 00 00 57 69 46 6c   02-13-2012..WiFl
0040  79 2d 45 5a 58 00 00 00 00 00 00 00 00 00 00 00   y-EZX...........
0050  00 00 00 00 00 00 00 00 00 00 00 00 b9 00 00 00   ................
0060  00 00 00 00 00 00 00 00 00 00 00 00 00 00         ..............

Follow UDP Stream of that was:

....l..(.....B.J
.Time NOT SET..WiFly Ver 2.32, 02-13-2012..WiFly-EZX.............................................l..(.....K.I
1Time NOT SET..WiFly Ver 2.32, 02-13-2012..WiFly-EZX.........................................

The TCP one is:

No.     Time           Source                Destination           Protocol Length Info
    396 5.855564000    62.81.51.152          192.168.1.68          TCP      54     http > 61394 [ACK] Seq=1 Ack=82 Win=5840 Len=0

Frame 396: 54 bytes on wire (432 bits), 54 bytes captured (432 bits) on interface 0
    Interface id: 0
    WTAP_ENCAP: 1
    Arrival Time: Jul 17, 2012 18:01:56.136056000 BST
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1342544516.136056000 seconds
    [Time delta from previous captured frame: 0.000630000 seconds]
    [Time delta from previous displayed frame: 0.000630000 seconds]
    [Time since reference or first frame: 5.855564000 seconds]
    Frame Number: 396
    Frame Length: 54 bytes (432 bits)
    Capture Length: 54 bytes (432 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
    [Protocols in frame: eth:ip:tcp]
    [Coloring Rule Name: HTTP]
    [Coloring Rule String: http || tcp.port == 80]
Ethernet II, Src: ThomsonT_93:6c:cc (00:18:f6:93:6c:cc), Dst: RovingNe_80:1f:f2 (00:06:66:80:1f:f2)
Internet Protocol Version 4, Src: 62.81.51.152 (62.81.51.152), Dst: 192.168.1.68 (192.168.1.68)
Transmission Control Protocol, Src Port: http (80), Dst Port: 61394 (61394), Seq: 1, Ack: 82, Len: 0

Follow TCP Stream of that was:

HTTP/1.1 200 OK
Date: Tue, 17 Jul 2012 17:01:56 GMT
Server: Apache/2.0.52 (Red Hat)
X-Powered-By: PHP/5.2.2
Content-Length: 0
Connection: close
Content-Type: text/html

There was also a HTTP stream:

No.     Time           Source                Destination           Protocol Length Info
    401 5.906948000    62.81.51.152          192.168.1.68          HTTP     231    HTTP/1.1 200 OK 

Frame 401: 231 bytes on wire (1848 bits), 231 bytes captured (1848 bits) on interface 0
Ethernet II, Src: ThomsonT_93:6c:cc (00:18:f6:93:6c:cc), Dst: RovingNe_80:1f:f2 (00:06:66:80:1f:f2)
Internet Protocol Version 4, Src: 62.81.51.152 (62.81.51.152), Dst: 192.168.1.68 (192.168.1.68)
Transmission Control Protocol, Src Port: http (80), Dst Port: 61394 (61394), Seq: 1, Ack: 118, Len: 177
Hypertext Transfer Protocol

Both TCP and HTTP have my sites correct ip address.

I've gone back into CMD mode for the WiFly to check it's configuration and it looks fine:

show net

SSid=BTHomeHub-2BF6
Chan=1
Assoc=OK
Rate=12, 24Mb
Auth=OK
Mode=WEP
DHCP=OK,renew=64651
Boot=148
Time=FAIL
Links=1
<2.32>

I did a quick test this morning to test calling my php page from something other than the browser. 30 seconds to make in Max/MSP the equivalent of what I'm trying to do with the Arduino:

download http://www.mysite.com/in-progress/arduino-test/counter/index.php?count=2
|
jit.uldl

My computer is wirelessly on the same network as the WiFly and my txt counter on the remote server updates without any issues. So the problem has to be with the Arduino or WiFly.

ok found the issue, this:

    // Build GET expression
    strRequest << F("GET ") << MY_SERVER_GET_URL << F("?count=") << iLoopCounter 
    << F(" HTTP/1.0") << "\n"
    << F("Host: ") << MY_SERVER_GET << "\n"
    << F("Connection: close") << "\n"
    << "\n\n";
    // send data via request
    // close connection

is not a true or correct GET request (I don't know which it is or why and I would quite like to know why). So my php check for the GET request was failing:

if ($_SERVER['REQUEST_METHOD'] == 'GET')

It didn't fail through the browser, command line or my Max/MSP tests as they are all true GET requests. The request string is almost identical to the one in the WebClientGetPost example that comes with WiFly (I just changed the variable and value) so that must be wrong as it's normal to check php side if it's a GET or POST request (amongst some other checks to see if the request originates from the right source).

Thanks for all your help and patience pylon, this I think has been technically the most gruelling thing I've ever done.

Garrett

I found it: "\n" is just a newline, a correct client request must have "\r\n" to end a line.

client.println() does that automatically.

I haven't seen this detail though...

pylon:
I found it: "\n" is just a newline, a correct client request must have "\r\n" to end a line.

mmm I tried changing them all and no difference.

// Build GET expression
    strRequest << F("GET ") << MY_SERVER_GET_URL << F("?count=") << iLoopCounter 
    << F(" HTTP/1.0") << "\r\n"
    << F("Host: ") << MY_SERVER_GET << "\r\n"
    << F("Connection: close") << "\r\n"
    << "\r\n";
    // send data via request
    // close connection

What's strange is when I look at the output in the serial monitor, it looks like this:

$$set u m 1 set u b 9600 vershow conn open www.mysite.com 80GET /in-progress/arduino-test/counter/index.php?count=17 HTTP/1.0
Host: www.mysite.com
Connection: close

The GET is pushed right up against the 80. I tried a \r\n there as well (which would make sense since php doesn't see it as a GET) but it didn't make any difference.

Have you checked your Streaming library if the << operator is doing what you expect it to do? The standard Print class does not support that operator.

The GET is pushed right up against the 80

This is probably correct as "open www.mysite.com 80" is a command for the RN-XV. You don't see the response to it, which is probably CONNECT. Then you send the data for that connection which directly starts with a GET.

pylon:
Have you checked your Streaming library if the << operator is doing what you expect it to do? The standard Print class does not support that operator.

Rather than pick apart the streaming library ( I wouldn't know where to begin) I've tried this:

strRequest = "GET " + String(MY_SERVER_GET_URL) + "?count=" + iLoopCounter + " HTTP/1.0\n" + "Host: " + String(MY_SERVER_GET) + "\n" + "Connection: close\n\n\n";

which looks a lot more familar to me. Works fine when the php GET check isn't there but fails with the check as the other string did.