Ethernet Sheild and GSM router

In your code there are Serial.print() statements, what output do you get, where does it stop?

Do you get an IP address?

Have you tried a simpler example from Ethernet.h examples, like WebClient? Connecting to a host like www.google.com is a much better test than to a service which may be down while you're trying to connect or may have decided to not answer requests from certain networks.

Hi pylon, yes i get an IP and it tells me its tryign to connect but then obviously doesnt for whatever reason and prints "cannot connect, retrying in 2 seconds..." and it keeps doing this, as you can see in the while loop.

I have tried a simple one and it can connect to www.google.com and can print the get request in the serial monitor, so the connection through the GSM router works. But so does the client to server as, if I plug the arduino into my normal internet it connects to the websocket server fine?

I have also wrote a simple script in js that connects to the server and attatched the GSM to laptop (instead of the arduino) and i can connect to the server fine, its only when the arduino tries to connect to the server using the GSM router that i cannot make a connection?

Any ideas?

Thanks
Andy

Have you tried doing the same connection (to the websocket server) on your laptop using the telnet command? I guess it's not an Arduino problem but one on the network, somewhere between your GSM router and the server.

Have you tried doing the same connection (to the websocket server) on your laptop using the telnet command?

Yes and it works fine.

I guess it's not an Arduino problem but one on the network, somewhere between your GSM router and the server.

But i can plug my laptop into the GSM router and connect to the server fine? Unplug laptop, plug in arduino and cant connect.

thanks
Andy

The error is in the library you're using:

    while(_client.available() == 0 && attempts < maxAttempts)
    {
        delay(100);
        attempts++;
    }
    
    while((line = readLine()) != "") {
        handshake += line + '\n';
    }

The code is waiting for up to 30 seconds for one character to arrive but then the whole answer has to be there completely. I'd guess that the GSM modem is splitting the packets into smaller packets and the library code isn't prepared for that.

Ahh ok, ive looked at this code for while thinking this is where the problem (if any) must lie.

I will give it ago despite my C++ not being to good, but i have to ask, do you have any idea how to impliment a workaround to wait for all packets before proceeding?

Thanks
Andy

Although this is an ugly hack, it might be enough for your needs:

    while(_client.available() < 14 && attempts < maxAttempts)
    {
        delay(100);
        attempts++;
    }
    
    while((line = readLine()) != "") {
        handshake += line + '\n';
    }

This way you're waiting for at least the string to be arrived you're testing for afterwards.

BTW: the author of the library did a lot of work to move the constant strings to the flash memory (PROGMEM) but then uses the String class which wastes more memory than he saved with the first action but making the code much more complex.

Why don't you wait for the server to close the connection? That is the signal it is finished sending packets. If it is like the ethernet library, you should be able to use this. It should wait ten seconds for a packet before aborting the connection.

while(_client.connected()) {
  while(_client.available()) {
    // if a packet arrives, reset the attempts counter
    attempts = 0;
    Serial.write(_client.read());
  }
  // increment counter and wait 1ms
  attempts++;
  delay(1);

  // if the counter has reached the timeout value, close the connection
  if(attempts > 10000) _client.stop();
}
// close the connection on your end after the server closes the connection
// This would be the second stop if there is a timeout, 
// but does not affect the ethernet shield.
_client.stop();

My apology in advance for errors or typos.

edit: I had the wrong delay value. It is now corrected.

HI pylon, ive tried doing what you did, but unfortunatly it hasnt wroked as of yet. Where di you pluck the 14 from? (just trying to understand?)

SurferTim is that code a modification of the c++ library? (which is what im trying to do)

Thanks
Andy

Look here. Is this the code you are using?

All those functions are available with that library also. This is pretty standard TCP stuff. The transport should be transparent.

Yes that is what im using, but its not working when used with a GSM router and we think this is becasue the GSM might be splitting up packets and the library therefore isnt interpreting the handshake correctly.

Andy

The code I posted above does not matter how many packets the response is sent in. It could be one packet with a thousand characters, or a thousand packets with one character. It waits for the server to close the connection. That is why the "while(client.connected())" loop.

Ok, but im not sure if the code you posted was a modification to the "websocket library" code or for use in the arduino IDE, if its for arduino IDE then its pointless as i cant event connect to the server, as ive stated in my above posts.

when i plug the arduino into the GSM router that has a internet connect i cannot connect to the server, yet plugging the arduino into a normal intenet router (home internet) it connects fine.

Andy

I would use my code in your sketch. Where are you using your current code now?

Ok, i dont want to rewrite it all as its all in the above posts, my code is, the problem is and what libaray im using is.

As i said, if you read the early posts you will see that your code to use in my sketch is insignificant as im not trying to download bytes using my sketch aim tryign to make a connection to a pusher-app server or my custom websocket server using a GSM router. The code pylon was referring to was the websocket library code.

Andy

@SurferTim: He cannot take your code because the connection is not closed by the server after the reception of the handshake lines. It just reads the first few lines from the server and checks for the connection OK string, then returns a success flag.

@OP: The code I posted is a replacement for the appropriate code in the library, not for use in your sketch. The library is not prepared for a result not being delivered in one packet. The length of 14 is the length of the string the library is testing for plus carriage return and linefeed. If you get more characters: OK but you have to get at least this much to be able to successfully check for a proper connection initialization.

Cheers for the clearup pylon, i was aware you meant the library code and i have tried changing it but to still no avail.

Any other ideas? Im sure the response string is large than 14 bytes?

Thanks
Andy

@pylon: Thanks! I did not see the persistent connection part. I have used those quite successfully with a joystick control from my Linux box to the ethernet shield, but it was a custom server, and not http based. I used a "stop byte" value to insure I got the whole transmission.

@OP: My bad. Is this a http based server, or a custom server? If it is a persistent http server, you may need to parse the response header for "Content-Length: 14". It may not be the same every time.

But you can't even connect, so this part doesn't even matter until you can connect to the server.

Its using the WebSocket protocol and initally i had my own custom server setup running a nodejs websocket but now im using a pusher server (a ready made websocket server with better debugging utilities for testing). Then using this websocket library for arduino you can connect to the websocket server over http which then gets upgraded to a tcp connection. But the connection and handshaking is done over http by sending a receiving headers, but what pylon is saying is the retruned headers that complete the handshake are being split by the GSM router (in order to cope with the load) which the library isnt prepared for and therefore failes to complete a handshake, hence no connection.

Whilst im thinking pylon, could there also be a problem with the GSM splitting the handshake request headers to the server? Does the server need to be ready for split data or shouldnt this matter so much?

Thanks
Andy

@SurferTim: It's a WebSocket server, the connection is persistent. The problem is, that the connect method of his library returns also false if a connection has been established but the initial handshake was not what it expected. The way the library is programmed, it assumes that the whole answer from the server is transferred in one packet and does no additional checking on that assumption.

This is the log of a interactive session doing the same:

 telnet ws.pusherapp.com 80
Trying 50.17.194.145...
Connected to ws.pusherapp.com.
Escape character is '^]'.
GET /app/8bce0224d7dae3e8e75e?client=js&version=1.9.0 HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: ws.pusherapp.com
Origin: ArduinoWebSocketClient

HTTP/1.1 101 Web Socket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
WebSocket-Origin: ArduinoWebSocketClient
WebSocket-Location: ws://ws.pusherapp.com/app/8bce0224d7dae3e8e75e?client=js&version=1.9.0

{"event":"pusher:connection_established","data":"{\"socket_id\":\"13296.159584\"}"}

As you can see, there is no Content-Length header in the response but some JSON to parse. As the connect method just checks the first line till after the response code, I hacked the buffer length in. A correct parser would be the better solution but this needs a more in-depth knowledge of the protocol than I have.