Leveraging SoftwareSerial RX Buffer to read an ENTIRE web page

Using Arduino Programming Questions
#1

For some weeks now I’ve been trying to use an ESP8266 module with Arduino Uno and SoftwareSerial to read values from a web API.

After a lot of googling and different code attempts, I had some success with this code, which I found on YouTube. But I was getting part gibberish in the serial monitor, which led to believe I might have had power supply problems. That was, until I found I found the Circuito-io/ESP8266_SoftwareSerial extension.

This code has completely mitigated the gibberish I was getting back in Serial Monitor at 115200 baud, which my ESP8266 requires, and I’m now getting back webpage headers with 100% accuracy/clarity, but only up to the limits of the RX buffer, defined in C:\Program Files (x86)\Arduino\hardware\arduino\avr\libraries\SoftwareSerial\src\SoftwareSerial.h. As advised by the ESP8266_SoftwareSerial README, I increased the buffer from 64 bytes to 256, which really only gave me more of the webpage header.

Sending Request to www.google.com
create tcp - OK
buffer is full!

+IPD,513:HTTP/1.1 400 Bad Request
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
Date: Sun, 21 May 2017 20:55:05 GMT
Connection: close
Content-Length: 334

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE⸮*

My question is how can I modify this code to read a webpage, but keep reading past the initial buffer-load while I search for the content I really want, which in my case is a small amount of JSON text.

#2

Rather than muck about with the SoftwareSerial code have a look at Serial Input Basics. The demo code has the array size set for 32 chars but you can change that to any larger value that you have room for.

...R

#3

Empty the buffer. Read the data out of it and compare to whatever you want to compare to and that reading out of the data will open up room in the buffer.

#4