Arduino Forum

Using Arduino => Programming Questions => Topic started by: DavidRK on Feb 07, 2019, 11:56 am

Title: ESP8266 to Arduino firmware
Post by: DavidRK on Feb 07, 2019, 11:56 am
Hey guys,

I have run into a bit of a roadblock the last few days with connecting up a ESP8266-01 to Arduino Nano.

I have managed to flash the ESP8266 through the Arduino and connect to my WIFI network and also establish a connection with the MQTT server, but for the life of me I can't find a good method or example for the communication between the ESP8266 and the Arduino.

My first thought was to make a serial connection but all the examples I looked at just push the settings from the Arduino Firmware into the ESP8266 and you have to end up using Serial monitor to invoke everything.

Is there an easier solution? I have read a few hundred blog posts and can't find any bilateral solution to this.

This is currently my ESP code, Not sure where to go with the Arduino code:
Code: [Select]

#include <PubSubClient.h>
#include <ESP8266WiFi.h>
#include <WiFiClient.h>

const char* ssid          = "";
const char* password      = "";
const char* mqttServer    = "";
const int   mqttPort      = 1883;
const char* mqttUser      = "";
const char* mqttPassword  = "";
const char* mqttTopic     = "home/kitchen/light";

const char* buttonON      = 0;

WiFiClient espClient;
PubSubClient client(espClient);
long lastMsg = 0;
char msg[50];
int value = 0;

void setup_wifi() {

  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  randomSeed(micros());

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  for (int i = 0; i < length; i++) {
    Serial.print((char)payload[i]);
  }

  // Switch on the LED if an 1 was received as first character
  if ((char)payload[0] == '1') {
    Serial.print("Passed IF");
    digitalWrite(LED_BUILTIN, LOW);
    // but actually the LED is on; this is because
    // it is active low on the ESP-01)
  } else {
    digitalWrite(LED_BUILTIN, HIGH);
  }

}

void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Create a random client ID
    String clientId = "ESP8266Client-";
    clientId += String(random(0xffff), HEX);
    // Attempt to connect
    if (client.connect(clientId.c_str())) {
      Serial.println("connected");
      // Once connected, publish an announcement...
      client.publish(mqttTopic, "hello world");
      // ... and resubscribe
      client.subscribe(mqttTopic);
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

void setup() {
  pinMode(LED_BUILTIN, OUTPUT);     // Initialize the BUILTIN_LED pin as an output
  Serial.begin(115200);
 
  setup_wifi();
  client.setServer(mqttServer, mqttPort);
  client.setCallback(callback);
}

void loop() {

  if (!client.connected()) {
    reconnect();
  }
  client.loop();

}
Title: Re: ESP8266 to Arduino firmware
Post by: gfvalvo on Feb 07, 2019, 01:04 pm
I can't find a good method or example for the communication between the ESP8266 and the Arduino.
What does that mean? What do you really want to do? Describe your project more completely.
Title: Re: ESP8266 to Arduino firmware
Post by: DavidRK on Feb 07, 2019, 11:08 pm
I am building a water tank level sensor, I am using a Nano, an ultrasonic sensor and a temperature/humidity sensor to calculate the level, I also want to add an ESP8266 to transmit the signal to the network.

All the examples I find using the ESP8266 to communicate with an Arduino are using the stock ESP8266 firmware and pushing commands from the Arduino.

Is this the best method?

I wanted for convenience, that all my WIFI and MQTT code was on the ESP8166 and then just passed data through from the Nano, not sure how possible this is.
Title: Re: ESP8266 to Arduino firmware
Post by: UKHeliBob on Feb 07, 2019, 11:10 pm
Quote
I wanted for convenience, that all my WIFI and MQTT code was on the ESP8166 and then just passed data through from the Nano, not sure how possible this is.
A serial link would seem to be the obvious answer
Title: Re: ESP8266 to Arduino firmware
Post by: gfvalvo on Feb 07, 2019, 11:43 pm
The first thing I'd explore is dropping the Nano completely and using the ESP8266 as the only processor programmed via the Arduino IDE. The ESP is a 32-bit, 80 MHz processor. Using it as a support peripheral for an 8-bit, 16 MHz processor always seemed backwards to me.

I'm more familiar with Adafruit's ES8266 Huzzah than ESP8266-01. I see that it's GPIO-limited. But, it's certainly worth seeing if you can connect all your sensors to it.
Title: Re: ESP8266 to Arduino firmware
Post by: DavidRK on Feb 08, 2019, 12:55 am
I'm more familiar with Adafruit's ES8266 Huzzah than ESP8266-01. I see that it's GPIO-limited. But, it's certainly worth seeing if you can connect all your sensors to it.
That is my big issue, the GPIO. I thought about Using the TC/RX pins for a total of 4 IOs but if I add something like a real-time clock (for putting into deep sleep, based on a time/schedule), the then I would run out.

I guess I could use a MCP23016 to expand the IOs but then I might as well use the Nano as I have a ton sitting around.

Open to idea's though, I am pretty new to MCU's and have been on a bit of a rollercoaster this week with the learning, especially just getting the ESP8266 to flash properly lol.
Title: Re: ESP8266 to Arduino firmware
Post by: DavidRK on Feb 08, 2019, 01:02 am
A serial link would seem to be the obvious answer
Yeah, this is what I was thinking, maybe using the GPIO0 & GPIO2 so that I can still use the TX/RX pins for firmware updates.

Just need to find something that's a good example for setting up a serial connection between them.
Title: Re: ESP8266 to Arduino firmware
Post by: gfvalvo on Feb 08, 2019, 01:05 am
Then @UKHeliBob's suggestion of a Serial link seems reasonable. But, you'll have to deal with the supply voltage difference. If you run the Nano's 5V logic into the 3.3V ESP, you blow up the latter. So, now you're adding level shifters. It's getting more complex.

For your consideration, the aforementioned Adafruit Huzzah (https://www.adafruit.com/product/2821) might have enough GPIO to do the job with one board for $17.
Title: Re: ESP8266 to Arduino firmware
Post by: DavidRK on Feb 08, 2019, 01:40 am
Then @UKHeliBob's suggestion of a Serial link seems reasonable. But, you'll have to deal with the supply voltage difference. If you run the Nano's 5V logic into the 3.3V ESP, you blow up the latter. So, now you're adding level shifters. It's getting more complex.

For your consideration, the aforementioned Adafruit Huzzah (https://www.adafruit.com/product/2821) might have enough GPIO to do the job with one board for $17.

Yeah, I think I would have been better just using a NoeMCU or the Huzzah from the beginning.


I am just using a mosfet to step down the 5v from the Nano as the 3.3v does not give enough current.
Title: Re: ESP8266 to Arduino firmware
Post by: Mrtian2 on Feb 08, 2019, 02:32 am
Below was a arduino esp8266 published to mqtt coding. You can refer it.


Code: [Select]

#include <Wire.h>
#include <VL6180X.h>
//#include <WiFiClientSecure.h>
#include <PubSubClient.h>
#include <WiFi.h>
VL6180X sensor;

//------- WiFi Settings -------
const char* ssid = "";
const char* password = "";
const char* mqtt_server = "";

int countDown = 10;  // Countind down 10second
unsigned long lastTick;
long Bot_lasttime;
int Range = 0;   //Set  Range to integer
#define uS_TO_S_FACTOR 1000000  /* Conversion factor for micro seconds to seconds */
#define TIME_TO_SLEEP  60        /* Time ESP32 will go to sleep (in seconds) */
RTC_DATA_ATTR int bootCount = 0;
// SSL client needed for both libraries

String ipAddress = "";

WiFiClient espClient;
PubSubClient client(espClient);
long lastMsg = 0;
char msg[50];

void setup_wifi() {

  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  randomSeed(micros());

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  for (int i = 0; i < length; i++) {
    Serial.print((char)payload[i]);
  }
  Serial.println();
}

void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Create a random client ID
    String clientId = "ESP8266Client-";
    clientId += String(random(0xffff), HEX);
    // Attempt to connect
    if (client.connect(clientId.c_str())) {
      Serial.println("connected");
     // float sensorvalue = sensor.readRangeSingleMillimeters();
     // char msg[30];
     // char val[10];
     // dtostrf(sensorvalue, 8, 5, val);
     // sprintf(msg, "Sensor Data sending: %s", val);
      //Once connected, publish an announcement...
      client.publish("smartdevice", "welcome");
      // ... and resubscribe
      client.subscribe("smartdevice");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

void setup() {
Serial.begin(115200);
Wire.begin();
setup_wifi();
client.setServer(mqtt_server, 1883);
client.setCallback(callback);
sensor.init();
sensor.configureDefault();
sensor.setTimeout(500);
//Increment boot number and print it every reboot
++bootCount;
Serial.println("Boot number: " + String(bootCount));

}
void loop() {
  if (!client.connected()) {
    reconnect();
  }
  client.loop();
  float myfloat;
  float sensorvalue = sensor.readRangeSingleMillimeters();
  char msg[30];
  char val[10];
  dtostrf(sensorvalue, 8, 0, val);
  Serial.println("Wake up Waiting 5 sec for sensor detetcing");
  delay(5000);
  Serial.print("\tRange: ");                                                      //empited
  Serial.println(sensor.readRangeSingleMillimeters());
  unsigned long currentMillis = millis();
  static bool Range_reach = false;
  if(sensor.readRangeSingleMillimeters()>80) {
       delay(1000);
       Serial.println("Checking Empited Condition1");
       delay(2000);       //delay 1 second for checking propose.
       Serial.print("\tRange: "); 
       Serial.println(sensor.readRangeSingleMillimeters());     
       Serial.println("Checking Empited Condition2"); 
       delay(2000);       //delay 1 second for checking propose.
       Serial.print("\tRange: "); 
       Serial.println(sensor.readRangeSingleMillimeters());     
{
            Serial.println("Reading for sending Empty message");
            delay(1000);
            Serial.print("\tRange: ");
            Serial.println(sensor.readRangeSingleMillimeters());
            delay(2000);
            if(sensor.readRangeSingleMillimeters()>80){
                 sprintf(msg, "Distance%s", val);
                 Serial.println("Publish message: ");
                 Serial.println(msg);
                 client.publish("smartdevice", msg);
                 if (sensor.timeoutOccurred()) { Serial.print(" TIMEOUT"); }         
                 Serial.println();
                 esp_sleep_enable_timer_wakeup(TIME_TO_SLEEP * uS_TO_S_FACTOR);
                 Serial.println("Going to sleep now!");
                 delay(3000);
                 esp_deep_sleep_start();
                 Serial.println("This will never be printed"); 

}
}
 
  else if (sensor.readRangeSingleMillimeters()<30) {        //Full
                 delay(1000);
                 Serial.println("Checking Full Condition1");
                 delay(2000);     //delay 30 second for checking propose.
                 Serial.print("\tRange: "); 
                 Serial.println(sensor.readRangeSingleMillimeters());     
                 Serial.println("Checking Full Condition2");
                 delay(2000);     //delay 30 second for checking propose.
                 Serial.print("\tRange: "); 
                 Serial.println(sensor.readRangeSingleMillimeters());     

                      Serial.println("Reading for sending Full message");
                      delay(1000);
                      float myfloat;
                      float sensorvalue = sensor.readRangeSingleMillimeters();
                      char msg[30];
                      char val[10];
                      dtostrf(sensorvalue, 8, 0, val);
                      Serial.print("\tRange: ");
                      Serial.println(sensor.readRangeSingleMillimeters());
                      delay(2000);
                      if(sensor.readRangeSingleMillimeters()<30){
                          sprintf(msg, "Distance%s", val);
                          Serial.print("Publish message: ");
                          Serial.println(msg);
                          client.publish("smartdevice", msg);
                          if (sensor.timeoutOccurred()) { Serial.print(" TIMEOUT"); }
                          Serial.println();
                          esp_sleep_enable_timer_wakeup(TIME_TO_SLEEP * uS_TO_S_FACTOR);
                          Serial.println("Going to sleep now!");
                          delay(3000);
                          esp_deep_sleep_start();
                          Serial.println("This will never be printed");
                       
}
}
  if(currentMillis - lastTick >= 1000){
    countDown--;
    displayCountdownToSerial();
    lastTick += 1000;   
  } 
  }

void displayCountdownToSerial(){
 
  int mins = countDown / 60;
  int secs = countDown % 60;
  Serial.print(" Time Remain To Sleep = ");
  Serial.print(mins);
  Serial.print(" : ");
  Serial.println(secs);

  if(countDown == 0 ){
      esp_sleep_enable_timer_wakeup(TIME_TO_SLEEP * uS_TO_S_FACTOR);
      Serial.println("Going to sleep now!");
      delay(3000);
      esp_deep_sleep_start();
      Serial.println("This will never be printed");         
}
}


//%s
Title: Re: ESP8266 to Arduino firmware
Post by: DavidRK on Feb 08, 2019, 02:55 am
Below was a arduino esp8266 published to mqtt coding. You can refer it.
Thanks, do you know if this required any custom firmware on the ESP8266 or does the Wire library just push through on stock.

Cheers
Title: Re: ESP8266 to Arduino firmware
Post by: Mrtian2 on Feb 08, 2019, 03:00 am
Thanks, do you know if this required any custom firmware on the ESP8266 or does the Wire library just push through on stock.

Cheers
by the way it did not request any firmware , you also can added in yourself but the WiFiClient espClient;
PubSubClient client(espClient);     both  wificlient and pubsubclient must be the same client.
Title: Re: ESP8266 to Arduino firmware
Post by: DavidRK on Feb 08, 2019, 04:16 am
by the way it did not request any firmware , you also can added in yourself but the WiFiClient espClient;
PubSubClient client(espClient);     both  wificlient and pubsubclient must be the same client.
Nice, Yeah that's what I have on my current setup, Ill give it a go now.
Title: Re: ESP8266 to Arduino firmware
Post by: DavidRK on Feb 09, 2019, 07:46 am
Well, Bit of messing around and I have come up with this to chat between the WSP and the Arduino.

There are probably far better ways of doing things and it would be great to hear any idea's.

Arduino sketch:
Code: [Select]
#include <SoftwareSerial.h>

SoftwareSerial ESPserial(2, 3); // RX | TX
 
void setup()
{
    Serial.begin(9600); 
    ESPserial.begin(9600); 
    delay(5000);
}
 
void loop()
{
    String IncommingString = "";
    boolean StringReady = false;
    while (ESPserial.available()){
      IncommingString = ESPserial.readString();
      StringReady = true;     
    }
    if (StringReady){
      Serial.println("Received: " + IncommingString);
    }
    while (Serial.available()){
      ESPserial.write(Serial.read());   
    }
}


And the ESP sketch:
Code: [Select]
#include <PubSubClient.h>
#include <ESP8266WiFi.h>
#include <WiFiClient.h>

const char* ssid          = "";
const char* password      = "";
const char* mqttServer    = "";
const int   mqttPort      = 1883;
const char* mqttUser      = "";
const char* mqttPassword  = "";
const char* mqttTopicIn   = "home/kitchen/light/in";
const char* mqttTopicOut  = "home/kitchen/light/out";

WiFiClient espClient;
PubSubClient client(espClient);

void setup_wifi() {

  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  randomSeed(micros());

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  String msgIN = "";
  for (int i = 0; i < length; i++) {
    msgIN += (char)payload[i];
  }
  String msgString = msgIN;
  Serial.print(msgString);
}

void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Create a random client ID
    String clientId = "ESP8266Client-";
    clientId += String(random(0xffff), HEX);
    // Attempt to connect
    if (client.connect(clientId.c_str(), mqttUser, mqttPassword)) {
      Serial.println("connected");
      // Once connected, publish an announcement...
      client.publish(mqttTopicOut, "hello world");
      // ... and resubscribe
      client.subscribe(mqttTopicIn);
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

void setup() {
  Serial.begin(9600);
  setup_wifi();
  client.setServer(mqttServer, mqttPort);
  client.setCallback(callback);

}

void loop() {
  if (!client.connected()) {
    reconnect();
  }
  while (Serial.available()) {
    sendToClient();
  }
  client.loop();
}

void sendToClient() {
  String payload = "";
  payload = "{";
  payload += "\"state\":";
  payload += Serial.readString(); payload += "}";

  char attributes[100];
  boolean published = false;
  payload.toCharArray( attributes, 100 );
  published = client.publish(mqttTopicOut, attributes);
  if (published) {
    Serial.print("published to MQTT");
  } else {
    Serial.print("publish to MQTT failed");
  }
}


Title: Re: ESP8266 to Arduino firmware
Post by: Deva_Rishi on Feb 09, 2019, 12:12 pm
Quote
I can't find a good method or example for the communication between the ESP8266 and the Arduino.
have a look at Serial Input basics (https://forum.arduino.cc/index.php?topic=288234.0) there are several good methods available.
Quote
My first thought was to make a serial connection but all the examples I looked at just push the settings from the Arduino Firmware into the ESP8266
Actually usually they use the Arduino to command the ESP-firmware. definitely a way, but not The way
Quote
I have come up with this to chat between the WSP and the Arduino.
You are on the right track !
few things :
Code: [Select]
String IncommingString = "";
    boolean StringReady = false;
    while (ESPserial.available()){
      IncommingString = ESPserial.readString();
      StringReady = true;      
    }

First of all, if you use 'Strings' on an Arduino people on the forum will likely blame any mishaps in your programs execution on it, so i would advice to switch over to using char * as a habit.
Second, in this snippet you are likely not to read a complete input (definitely since you are using a low BAUD-rate)  anyway have a look at Robin2's examples and consider once you've gone beyond the debugging stage o use the nano's hwSerial instead for faster communication.
About the ESP-sketch:
Code: [Select]
randomSeed(micros());
although you are not using the random at all, an easier way to get a proper randomSeed
Code: [Select]
randomSeed(analogRead(A0));
that it is not connected to a physical pin doesn't mean it is not there, and it is remarkable how consistent connection times can be.
Code: [Select]
 while (Serial.available()) {
    sendToClient();
  }

 the same issue with the Serial comm. here, you are sending incomplete messages.
Code: [Select]
char attributes[100];
  boolean published = false;
  payload.toCharArray( attributes, 100 );

and if you are going to convert from String to char *,  check the length of the String, add 1 for the '\0' terminator, and declare that as size for the array and as size for the copy. With the ESP it usually is fine to use Strings due to it's much bigger memory size (and different/better String memory managing) and i would actually even recommend it's use over char * at times since there has been a report on strtok() not functioning properly on an ESP.

Title: Re: ESP8266 to Arduino firmware
Post by: Deva_Rishi on Feb 09, 2019, 01:01 pm
Of course if you just want proof of concept,  a small adjustment to the nano sketch
Code: [Select]
while (ESPserial.available()){
      char c =ESPserial.read();
      IncommingString+= c;
      if (!ESPserial.available()) delayMicroseconds(1200); // this waits enough time for at least 1 more character
      StringReady = true;     
    }
and for the ESP sketch
Code: [Select]
String payload = "";
  payload = "{";
  payload += "\"state\":";
  while(Serial.available()) {
     char c=Serial.read();  // of course this could also be just 1 line but now we can test
     payload += c;            // the value of 'c' if we want to
     if (!Serial.available()) delayMicroseconds(1200);
   }
   payload += "}";
would suffice (code without compile test, may have typos)
Title: Re: ESP8266 to Arduino firmware
Post by: DavidRK on Feb 12, 2019, 06:17 am
have a look at Serial Input basics (https://forum.arduino.cc/index.php?topic=288234.0) there are several good methods available. Actually usually they use the Arduino to command the ESP-firmware. definitely a way, but not The wayYou are on the right track !
few things :
Code: [Select]
String IncommingString = "";
    boolean StringReady = false;
    while (ESPserial.available()){
      IncommingString = ESPserial.readString();
      StringReady = true;      
    }

First of all, if you use 'Strings' on an Arduino people on the forum will likely blame any mishaps in your programs execution on it, so i would advice to switch over to using char * as a habit.
Second, in this snippet you are likely not to read a complete input (definitely since you are using a low BAUD-rate)  anyway have a look at Robin2's examples and consider once you've gone beyond the debugging stage o use the nano's hwSerial instead for faster communication.
About the ESP-sketch:
Code: [Select]
randomSeed(micros());
although you are not using the random at all, an easier way to get a proper randomSeed
Code: [Select]
randomSeed(analogRead(A0));
that it is not connected to a physical pin doesn't mean it is not there, and it is remarkable how consistent connection times can be.
Code: [Select]
while (Serial.available()) {
    sendToClient();
  }

 the same issue with the Serial comm. here, you are sending incomplete messages.
Code: [Select]
char attributes[100];
  boolean published = false;
  payload.toCharArray( attributes, 100 );

and if you are going to convert from String to char *,  check the length of the String, add 1 for the '\0' terminator, and declare that as size for the array and as size for the copy. With the ESP it usually is fine to use Strings due to it's much bigger memory size (and different/better String memory managing) and i would actually even recommend it's use over char * at times since there has been a report on strtok() not functioning properly on an ESP.


Awesome, have been amending a couple things the last few days, especially string lol, they came back to bite.


On my journey of try to also find an OTA for the ESP, I stumbled across this https://github.com/jeelabs/esp-link]esp-link (https://github.com/jeelabs/esp-link) Pretty much sorts all the functions I required, and as long as you have the 1Mb ESP-01 you can also do OTA too.
Title: Re: ESP8266 to Arduino firmware
Post by: Deva_Rishi on Feb 12, 2019, 10:28 am
Quote
On my journey of try to also find an OTA for the ESP
I use ArduinoOTA.h and it works like a charm. It wasn't very complex to create a system with multiple users with different permissions.