Using ESP8266 to send sensor data using arduino controller

Hello,

I have been struggling for days on a matter that appears to be harder than i thought it would be.
I'm trying to create a personal air quality sensor manager arduino UNO as well as one or two more sensor and send its values to my domoticz server through an ESP8266-01.

I'm using arduino to get the sensor value. I managed to do the measure part without a single inch of trouble.
To be able to communicate with my domotic server (working on domoticz). I want to send a simple get or post request toward my local API.

To do so, i bought an ESP8266-01. I want to use this only to send the value and not to be programmed itself. (Basically i want to the code to be on my arduino because he has more pins than the esp8266, maybe this is where i'm wrong)

Most website indicate that i should download the esp8266 board and libraries and use the ESP8266WiFi.h library. I already used this library on another project with a standalone esp8266 and a single sensor linked to it without arduino involved, it worked quite nicely.

But as soon as i tried to put the esp8266 as a slave of my arduino, the problems began.

I followed some tutorials to plug the ESP8266
To sum it up, I found two ways where i could interact with my ESP8266

In both cases, I connected alimentation pins and CH_PD as documentations stated and I didn't plug RST pin (since i don't want the code to be on the ESP itself)

  • Schema 1
    RX and TX on arduino where linked to RX and TX on ESP8266
  • Schema 2
    Serial 2 and 3 on arduino where linked to RX and TX on ESP8266

With both schemas I couldn't use the ESP8266WiFi.h libraries since it appears to be only compatible with the ESP8266 processor.

Question 1 - First of all am I doing something wrong or out of sense there?
Question 2 - Is it even possible to use an ESP8266 as a slave emitter for Arduino UNO?
Question 3 - Anybody has an idea of how could ESP8266WiFi.h library in this kind of build?

I never managed to run the simple script BasicHttpClient.ino from ESP8266

#include <ESP8266WiFi.h>
#include <ESP8266WiFiMulti.h>

#include <ESP8266HTTPClient.h>

#include <WiFiClient.h>

ESP8266WiFiMulti WiFiMulti;

void setup() {

  Serial.begin(115200);
  // Serial.setDebugOutput(true);

  Serial.println();
  Serial.println();
  Serial.println();

  for (uint8_t t = 4; t > 0; t--) {
    Serial.printf("[SETUP] WAIT %d...\n", t);
    Serial.flush();
    delay(1000);
  }

  WiFi.mode(WIFI_STA);
  WiFiMulti.addAP("SSID", "PASSWORD");

}

void loop() {
  // wait for WiFi connection
  if ((WiFiMulti.run() == WL_CONNECTED)) {

    WiFiClient client;

    HTTPClient http;

    Serial.print("[HTTP] begin...\n");
    if (http.begin(client, "http://jigsaw.w3.org/HTTP/connection.html")) {  // HTTP


      Serial.print("[HTTP] GET...\n");
      // start connection and send HTTP header
      int httpCode = http.GET();

      // httpCode will be negative on error
      if (httpCode > 0) {
        // HTTP header has been send and Server response header has been handled
        Serial.printf("[HTTP] GET... code: %d\n", httpCode);

        // file found at server
        if (httpCode == HTTP_CODE_OK || httpCode == HTTP_CODE_MOVED_PERMANENTLY) {
          String payload = http.getString();
          Serial.println(payload);
        }
      } else {
        Serial.printf("[HTTP] GET... failed, error: %s\n", http.errorToString(httpCode).c_str());
      }

      http.end();
    } else {
      Serial.printf("[HTTP} Unable to connect\n");
    }
  }

  delay(10000);
}

I managed to send a request through AT commands to the ESP8266-01 thanks to the following script. Sadly, the request i managed to send with my low ESP8266 AT skills was not recognised as a valid request by my domoticz server since it might lack headers.

It is an adaptation of a common serial code for ESP8266 in serial mode.

#include <SoftwareSerial.h>

#define RX 2
#define TX 3

String AP = "wifiSSID";       // CHANGE ME
String PASS = "password"; // CHANGE ME
String HOST = "hostOfDomoticz";
String PORT = "portOfDomoticz";

int countTrueCommand;
int countTimeCommand; 
boolean found = false; 
int valSensor = 1;
SoftwareSerial esp8266(RX,TX); 
 
  
void setup() {
  Serial.begin(9600);
  esp8266.begin(115200);
  sendCommand("AT",5,"OK");
  sendCommand("AT+CWMODE=1",5,"OK");
  sendCommand("AT+CWJAP=\""+ AP +"\",\""+ PASS +"\"",20,"OK");
}

void loop() {
 valSensor = getSensorData();
//This request is a domoticZ request to add a value on a sensor
 String getData = "GET /json.htm?type=command&param=udevice&idx=666&nvalue="+String(valSensor);
 sendCommand("AT+CIPMUX=1",5,"OK");
 sendCommand("AT+CIPSTART=0,\"TCP\",\""+ HOST +"\","+ PORT,15,"OK");
 sendCommand("AT+CIPSEND=0," +String(getData.length()+4),4,">");
 esp8266.println(getData);
 delay(200);
 countTrueCommand++;
 sendCommand("AT+CIPCLOSE=0",5,"OK");
}
int getSensorData(){
  return random(1000); // Replace with 
}
void sendCommand(String command, int maxTime, char readReplay[]) {
  Serial.print(countTrueCommand);
  Serial.print(". at command => ");
  Serial.print(command);
  Serial.print(" ");
  while(countTimeCommand < (maxTime*1))
  {
    esp8266.println(command);//at+cipsend
    if(esp8266.find(readReplay))//ok
    {
      found = true;
      break;
    }
  
    countTimeCommand++;
  }
  
  if(found == true)
  {
    Serial.println("OK");
    countTrueCommand++;
    countTimeCommand = 0;
  }else   {
    Serial.println("Fail");
    countTrueCommand = 0;
    countTimeCommand = 0;
  }
  
  found = false;
 }

This code is working but it seems that the following part

sendCommand("AT+CIPSEND=0," +String(getData.length()+4),4,">");
esp8266.println(getData);

is received by the server but not formed like a GET request that a simple navigator would do.
Which make the whole code not working.

Thank you if anyone as any clue on how i could make it work in any way. And sorry for not giving schemas (i'm a new users and cannot upload file).

Matthieu

you could try my WiFiEspAT library with AT 1.7.5 from SDK 3.0.5 in the esp01.
the library is in Library Manager.
https://github.com/jandrassy/WiFiEspAT#at-17-1

Just tried it without success.
I tried, no success on basic connection/wifi ssid scan :/.

did you do all steps?
https://github.com/jandrassy/WiFiEspAT#getting-started

The very first step of the CheckFirmware example file is malfunctionning:
20:28:23.963 -> Communication with WiFi module failed!

Does this library require a shield because i have none.
I tried with some resistance on TX and RX build like this:

With serial wiring like this (replaced TX and RX by 6/7 pins because of the script)

I tried also the same montage as last but with TX and RX

And i try switching Switch TX and RX cables for each position nothing worked on both 6900baud and 115200 baud nothing worked.

Matthieu

To quote my favorite robot: "This will all end in tears. I just know it."

Using the ESP module to connect an Uno to WiFi is destined to be difficult. The ESP8266 on a Wemos D1 Mini or NodeMCU board is more capable than the Uno, and it already has WiFi. Why do you even need the Uno?

There is a strong hint there! :grin:

How many I/O do you need? The ESP-01 has four digital I/O readily available, but no analog.

To be honest, I'm a web developer and regarding electronics i'm quite new to it.
I don't know much about arduino and NodeCMU.

I'm still taking time and trying.
I happen to have bought some nodeCMU as well to try them up.
I don't see any 5V pin (required for some carbone dioxide sensors or humidity sensors) and seems to my newbies eyes more difficult. I might be wrong on that.

This is why i went on Uno rather than them.

I'm pretty the first sensor out of the 3 i want to put on the arduino is analog which makes it more complicated.

By the way, i already manage to program directly the ESP8266-01 linked to a DHT11. If i had one sensor i would have gone for this solution for sure. For more sensors, i'm not sure the esp8266-01 alone is capable.

that is step 5. what about the steps before that?

After trying hard for one more day, my wife start to be a bit mad at my stuborness :smiley: .

I still manage to send AT commands, but even with trying to form better the request, my domotic box (server with domoticz) cannot recognized the form of this request. (I try adding headers and stuff).

After a night of thought, I tried to kind of simplify my way of thinking. Since i manage to send the request by programming the ESP8266-01 alone and i manage to take measures with arduino, I'm thinking that it might be wiser to think the problem as a micro service problem.

Concreatly, my new idea is to do this:
Program Arduino to measure and send it through 2 serial pins as a simple message like a json,
{"type":"Air Quality", "value"=1145}

Program ESP8266-01 with an USB adapter that allows me to plug it directly on my computer and upload a simple code that

Read the RX entry looking for this json and when it receives a message, check the message, create a request with an ESP8266-01 library and send it to my API. (The same way i did for my DTH11 sensor alone).

For now I didn't manage to read the message i sent from my arduino pin to the ESP8266-01.
I tried to keep it simple and stupid basically i just try to send a texte on arduino with a number and send the request containing the value with ESP and alternating value every 5 seconds.
It didn't work.

The code on ESP8266 is tested and working if i just send some dummy value (domoticz receive it without issue). But IT canno't catch the arduinos message.

On Arduino

#include <SoftwareSerial.h>
#define RX 2
#define TX 3
SoftwareSerial esp8266(RX,TX);
int count=1;
void setup() {
  Serial.begin(9600);
  // put your setup code here, to run once:
  esp8266.begin(19200);
}

void loop() {
  if(count=1){
    esp8266.println("125");
    count=2;
  } else {
    esp8266.println("432");
    count=1;
  }
  delay(5000);
}

On ESP8266

#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>
#include <WiFiClient.h>

WiFiClient wifiClient;

#ifndef STASSID
    #define STASSID "fakeSSID"
    #define STAPSK  "fakeKey"
#endif

const char* ssid = STASSID;
const char* password = STAPSK;
const char* domoticz_host = "http://fakedomoticzhost";
const uint16_t domoticz_port = fakePort;
const bool use_secured_protocol = false;
const bool sensor_domoticz_idx = "420";

void setup() {
    Serial.begin(115200);
    // We start by connecting to a WiFi network
    Serial.print("Connecting to "); 
    Serial.println(ssid);
    /* Explicitly set the ESP8266 to be a WiFi-client, otherwise, it by default,
    would try to act as both a client and an access-point and could cause
    network-issues with your other WiFi-devices on your WiFi-network. */
    WiFi.mode(WIFI_STA);
    WiFi.begin(ssid, password);

    while (WiFi.status() != WL_CONNECTED) {
        delay(500);
        Serial.print(".");
    }

    Serial.println("");
    Serial.println("WiFi connected");
    Serial.println("IP address: ");
    Serial.println(WiFi.localIP());
}


void sendDomoticzDhtMeasureRequest(String measure){
    HTTPClient httpClient;
    //String requestString = (use_secured_protocol) ? "https://" : "http://";
    //For the moment only http works
    String requestString = "";
    requestString += domoticz_host;
    requestString += ":";
    requestString += domoticz_port;
    requestString += "/json.htm?type=command&param=udevice&idx=134&nvalue=";
    requestString += measure;

    httpClient.begin(wifiClient,requestString);
    httpClient.addHeader("Content-Type", "text/plain");
    int httpClientCode= httpClient.POST("Message");
    String payload = httpClient.getString();
    httpClient.end();
}

void loop() {
    if(Serial.available())  {
      String measure = Serial.readString();
      sendDomoticzDhtMeasureRequest(measure, "dht");
    }else {
      Serial.println('Nothing Received');
    }
        
    delay(100); // execute once every 50000 ms, don't flood remote service
}

The montage i used was the same with some resistance to reduce signals arriving on ESP8266 RX to 3.3V. I inverted the SoftwareSerial RX on pin 2 to be linked with ESP8266TX and Software Serial TX on pin 3 to be connected with the RX.

Nothing worked so far.
Maybe i'm a lost cause there

Soulwax62

If your sensors are I2C, then you can have 64 of them on a single ESP8266.

I have more than 75 devices on an IP scan. About half of them are ESP devices, mostly the ESP8266 on a Wemos D1 Mini. Not a single Uno in the mix.

My devices include the router, TV, Alexa devices, and for IOT, lots of sensors and switches.

All of my sensors, with a couple of exceptions, have their own ESP8266. The Wemos D1 Mini are only $5 each, even on Amazon. One processor board per sensor makes the coding and management much easier. And they already have WiFi.

1 Like

Hello everyone,

After having LOTS of trouble to do what i intended to do and some crazy thinking night,
I finally managed to find a way for it to work.

Rather than only programming my arduino, I decided to program both the ESP8266-01 and the arduino.

To put it in a nutshell:

  1. I used the arduino to get the value of my sensor from Analog entries.
  2. I used the ArduinoJson Library to format a nice output to send on my serial.
  3. I transformed my ESP8266-01 Into a gateway that is only doing the following:
  • Listening on its own RX for messages.
  • If the message is a valid json and has the right keys; It construct a request
    and send it to Domoticz.
  1. In the end I just send the json on the arduino board on the serial of arduino TX, The ESP8266-01 grab it on its RX pin and send it to Domoticz.
  2. Because I communication with a Json, i can even send several value from several sensors. The only limit being arduino's code and pins.
    I'm pretty sure this code is compatible with any arduino board.

The json sent from arduino to ESP8266-01 looks like this without the carriage returns and spaces:

{
            "type":"request",
            "sensors": {
                "134":{
                    "type":"air-quality",
                    "value":500
                },
                "125":{
                    "type":"moisture",
                    "value":70
                }
            }
        }

It was an interesting journey. I inspired myself of micro services and now i have a gateway that can gather data and send it to domoticz.

If you are interested in the solution itself, don't hesitate to contact me.

Soulwax62

Hi, i am very interested! If have a working co2 meter on arduino uno. I want to send the data to domoticz with an esp-01 wifi. But i can not get it working. Can you send your solution?
Tx Thomas

@thomassio

I put some code and some information on the public git:

https://github.com/matthieu-antoniol/arduino-esp01-domoticz-gateway

If you don't understand how to plug or make it work, just send me an a private message and an email and i might assist you.

I have some calibration issue on my C02 sensor but this is for another thread.

Soulwax62

Hi Soulwax62,

I bricked my esp so now waiting for another to arrive!

I think i understand your code and wiring, but i probably need some help on the way.

Tx,

Thomas

Hi Soulwax62,

I managed to program the esp and it connects to my network. I have the following questions:

  • My sensor is different then yours and i have some problems in the arduino uno code:
  • I don’t understand the replacing @domoticzSensorIDX@ in the sensors?
  • AirQuality147 is not declared?- What is the script in Domoticz to get the values?

Sorry for being a newby and not understanding some concepts, bur tx for helping!

Thomas

  • For the sensor, you can measure whatever you want as long as it corresponds to one of domoticz sensor and you add a switch case to the ESP if it was not already coded.

If ever you still want your own way of storing value you could create some dummy sensors and deal with it.

https://www.domoticz.com/wiki/Domoticz_API/JSON_URL's

For instance for a PPM sensor you could use this:
image

  • About the @domoticzSensorIDX@ . You can just open the script with the arduino IDE and do CTRL+F and replace all occurences of @domoticzSensorIDX@ in all the script by the id of your sensor in domoticz.
    This is the fastest i found to make multi editing easy.

I just realise while simplifying I did some typos in variable names that i fixed (IE: the 147 and forgetting to rename moisture sensor). Sorry about that, I updated it you can get the new code.

  • To put what you have to do in a nutshell:

    1. Go into Setup -> Hardware and Create a virtual hardware in domoticz (type Dummy)
    2. Generate a new virtual sensor with the hardware previously created (Air quality or something else base on your needs)
    3. Go into Setup -> Device tab and find the idx of the device (first column usually) For the example let's call it IDX 58
    4. Search in the C02 script with the arduino IDE CTRL+F command : @domoticzSensorIDX@
    5. Replace it by 58
    6. Upload the code to your arduino
    7. When reset, you should see the value of your measure changing and the timestamp updating.
  • Disclaimer:

    • Right now i only programmed a few sensors like temp, humidity, moisture or air quality.
    • If you want to improve the code you can add a new case in the sendDomoticzMeasureRequest function of the ESP8266 code. Thanks to the API documentation it is quite easy.

  • About the script that receive the data in Domoticz, it is quite deep in the code and I wouldn't advice to look too much into it. The esp is only creating a request that is using domoticz JSON API to update the value of your sensors.

If ever you want to do something on the value catched you could just create a script in the section Setup -> More Options -> Events.
Take a Lua script and have fun for other things.

See you!

Works perfect. I only changed the softserial to hardware serial because i needed the softserial for my mh z19c co2 sensor. I had some problems with json because of low ram but that was solved by decreasing the size of json to 512.

Tx for the good work!

No problem, glad it worked!

Matthieu