[SOLVED][ESP8266, Arduino] Websockets with AT commands

Using Arduino Networking, Protocols, and Devices
#1

Hi,

I'm learning websockets (trying to make it works without libraries), but I'm having problem with handshake response.

My browser request looks like this:

Provisional headers are shown
Accept-Encoding: gzip, deflate
Accept-Language: pl-PL,pl;q=0.9,en-US;q=0.8,en;q=0.7
Cache-Control: no-cache
Connection: Upgrade
Host: 192.168.0.25
Origin: http://192.168.0.14
Pragma: no-cache
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
Sec-WebSocket-Key: q2vnxYPpBhCBdbMNVNj87A==
Sec-WebSocket-Version: 13
Upgrade: websocket

Arduino/ESP server config via AT commands:

esp8266Data function params are (comman, delay)

String esp8266Data(String command, const int timeout)
{
  String response = "";
  esp8266Serial.print(command);
  long int time = millis();

  while ( (time + timeout) > millis())
  {
    while (esp8266Serial.available())
    {
      char c = esp8266Serial.read();
      response += c;
    }
  }
  
  Serial.println(response);
  
  return response;
}

  esp8266Serial.begin(9600);
  esp8266Data("AT+RST\r\n", 3000);
  esp8266Data("AT+UART_DEF=9600,8,1,0,0\r\n", 3000);
  
  esp8266Data("AT+CWMODE=1\r\n", 3000);
  esp8266Data("AT+CIPMODE=0\r\n", 3000);
  esp8266Data("AT+CWJAP=\"myrouter\",\"pass\"\r\n", 3000);
  esp8266Data("AT+CIPSTA=\"192.168.0.25\",\"192.168.0.1\",\"255.255.255.0\"\r\n", 3000);
  esp8266Data("AT+CIFSR\r\n", 3000);
  esp8266Data("AT+CIPMUX=1\r\n", 3000);
  esp8266Data("AT+CIPSERVER=1,80\r\n", 3000);

then my arduino generates response like this:

esp8266Data("AT+CIPSEND=0,242\r\n", 200);

//then I generate handshake response which looks like this:
HTTP/1.1 101 Switching Protocols
Accept-Encoding: gzip, deflate
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
Sec-WebSocket-Accept: GJgW3dfoFE77yV4JXD8Xds3criU=
Sec-WebSocket-Version: 13

and now I don't know if I should use "AT+CIPCLOSE=0\r\n" to send this response or it should be received automaticaly with CIPSEND?

The problem is that on my browser it is still pending status like nothing is being sent... any idea what may be wrong? I'm sure Sec-WebSocket-Accept is ok, beacuse I've tested it with other tools.

EDIT

My problem was because I counted length of headers wrong, remember that at the and of headers u need 2x "\r\n" so it knows it's end of headers.

#2

this is a complex task which should not be handled with one layer of codding.
if you don’t what to use a websockets library, at least use a library which wraps the AT commands in some usable API, for example the standard Arduino networking API.

#3

My WiFi configurator is fine, at least i was sucessfull sending http 200 ok response and with AT commands ive better control over configuration, like setting static IP, which i need.

I dont agree that its complex, there are some simple rules like key generate with sha1 and base64 which i have done corectly.

Of course, I could split layer my code to my own libraries so the main sketch will looks more abstract and cleaner, but this is not the point.

I just want to understand, why I cant establish websocket connection. Looks like im sending all required headers, but in chrome developer console in message tab, nothing happens and as far as i know it should stay pending status but it should say that handshake was sucessfull.

I think my cipsend is actually not being sent or im missing something.

#4

you can set all the configuration with my WiFiEspAT library.
compare handling of AT command responses in your code and in the library:

#5

Thank you for your link, but looks like your lib is pretty complex for what I need. Studying ur code would cost me a lot of time and I'm not sure if I could find answers there, especially that I've searched for "Sec-WebSocket-Key" header which is core and I could not find it in your code. Also searched for "websocket", "socket" terms and could not find any results, are you sure ur lib handles websockets? I'm searching for low level answers, not beautifull code at that stage that is built on top of other libs. I'd be greatfull if you could answer me few simple questions as you seems to have experience.

  1. Is my server configuration pasted in main post correct to handle websockets connection?
  2. Do I need to close connection, after sending headers with CIPSEND? I think the answer is "no", but I want to be sure.
  3. Do I need to send any other command to my ESP serial after CIPSEND for response to be actually sent to my browser?
  4. I think my ESP serial getting freeze while sending websocket headers, do you have any idea what can cause that behaviour? There is also function which shows how I print to serial.
#6

the WiFiEspAT library implements the standard Arduino networking API over AT commands. The Arduino websocket libraries (for example ArduinoHttpClient library) work usually over the networking API.
if you make it work with AT commands of esp8266 and then you want to port it to one of Ethernet libraries or some Arduino WiFi library, you will have start from the beginning.

your esp8266Data() is nothing more then
Serial.println(command);
Serial.setTimeout(timeout);
String s = Serial.readString();
which will wait 'timeout' after the last character received.

so learn the Arduino APIs and use them. it is much more useful then fighting with AT firmware with naive coding constructs.

#7

Do you mean it is not possible with AT commands + followig websockets docs or you have no knowledge on that level to answer my questions?

Yes, i'm fighting with firmware because I want to learn concepts behind magic and "naive coding" in my opinion is to blindly use libs (well except if making it works is your goal and in my case it is not as I stated before). I prefer to fail trying then win without effort and satisfaction of understanding it deeply.

I wasn't expecting to find asnwer on forum (more likely I was looking for some constructive clue on what may cause my problem), so I'm not suprised by that answers. So thank you for your time sir, I'll keep fighting alone :slight_smile:

#8

riten:
Do you mean it is not possible with AT commands + followig websockets docs or you have no knowledge on that level to answer my questions?

Yes, i'm fighting with firmware because I want to learn concepts behind magic and "naive coding" in my opinion is to blindly use libs (well except if making it works is your goal and in my case it is not as I stated before). I prefer to fail trying then win without effort and satisfaction of understanding it deeply.

I wasn't expecting to find asnwer on forum (more likely I was looking for some constructive clue on what may cause my problem), so I'm not suprised by that answers. So thank you for your time sir, I'll keep fighting alone :slight_smile:

it is possible with AT commands. but you need robust code to handle the communication with AT firmware. it is not like a simple GET request. and the WiFiEspAT library has this code to handle all the cases in the communication with AT firmware.

what understanding do you want? there is nothing special. send commands and receive responses. many different and sometimes unexpected responses. it is only hard work of testing and debugging and incorporating all the strange undocumented behaviors of the esp8266 AT firmware. this know-how is not applicable to anything else. even the esp32 AT firmware doesn't behave the same way,

#9

Websockets actually starts with a simple HTTP GET request (but started from ws/wss protocol) from client to server (with headers I've shown in my first post). Then if handshake is sucessfull, server and client switches to continuous conversation via TCP, if the returned status is 101 and correct key is set in response headers.

"What I want to understand"?

I bascially want to create my own websocket server based on this docs Writing WebSocket servers - Web APIs | MDN

AT firmware maybe won't be usefull in normal enviroment, but the understanding of websockets server connection process is priceless for me and I simply enjoy it. I think I'm pretty close to make it works, so I'll try to find out why response is not being sent. I think ESP serial port is for some reason getting blocked.

#10

the WiFiEspAT library doesn't solve the websocket for you. but it has the standard Arduino networking API which is closer to common TCP socket concept then the AT commands. you still have to do all the HTTP and webskocket protocol comands.

see this HTTP example, which you can find in almost all Arduino networking libraries, WiFi or Ethernet base.

if (client.connect(server, 80)) { // TCP socket connect to IP and port
    client.println("GET /asciilogo.txt HTTP/1.1"); // send HTTP GET line
    client.print("Host: ");
    client.println(server); // HTTP host header
    client.println("Connection: close"); // other header
    client.println(); // end of headers (empty line)
  }

this is the interesting layer. here you learn the application level protocols without fighting with AT firmware

#11

@riten

I have no idea what your version of "spamming" is but it wastes time of the moderators !

Please use the "report to moderator" wisely not gratuitously !

Could you also take a few moments to Learn How To Use The Forum.
Other general help and troubleshooting advice can be found here.
It will help you get the best out of the forum in the future.

#12

@ballscrewbob

I actually think, it is offtopic and advertising library. As the problem in question is clear and response is not connected to problem. I just didn’t want to start that kind of discussion here, instead tried to clean it up and wait for someone who can actually add something to problem :wink:

@Juraj
So thank you for your help and your time, but I don’t want your library, I’m looking for technical answers about AT/websockets and you clearly don’t have any.

#13

you can use the older WiFiEsp library. that is not by me. GitHub - bportaluri/WiFiEsp: Arduino WiFi library for ESP8266 modules
or even better, code directly for the esp8266 with esp8266 Arduino core. it is still the same Arduino networking API.
the point is, not loose time learning the details of AT commands.

#14

1119×61 2.54 KB

Woaah! I've just done it!

Every header I sent was fine, the only problem I had was length given to CIPSEND. Really silly bug, but it works. The most important thing was to send "\r\n" at the end twice so it knows that it is end of header.

#15