Go Down

Topic: ESP8266 --Internet Weather Datalogger and Dynamic Web Server (Read 3375 times) previous topic - next topic

Techno500

#15
Jun 29, 2017, 01:34 pm Last Edit: Jun 30, 2017, 02:44 pm by Techno500 Reason: Problem uploading image
@ zoomx

Current BME280 on the project board in a China variant; maybe way out of tolerance.  Have requested a replacement; hopefully better, from another source.  

Tested "Observations_SPI_CF.zip"  Works as you coded; see attached file."

I agree on the tolerance of  the Bosch BME280.  Unknown if China BME280 is the "real deal."

William

zoomx

Great!
I don't have any reference barometer, maybe I can use the value provided by the nearby airport.
My Bosh sensor came from China too.

zoomx

software RTClib  seems to work, I have to add NTP request.

Maybe you can change this
Code: [Select]
 //Write Data at 15 minute interval
    if ((((minute) == 0) ||
         ((minute) == 15) ||
         ((minute) == 30) ||
         ((minute) == 45))
        && ((second) == 0))
    {
      lastUpdate = dtStamp;   //store dtstamp for use on dynamic web page
      getWeatherData();
      updateDifference();  //Get Barometric Pressure difference
      logtoSD();   //Output to SD Card  --Log to SD on 15 minute interval.
      delay(10);  //Be sure there is enough SD write time
      speak();
      //lcdDisplay();      //   LCD 1602 Display function --used for 15 minute update
    }
    else
    {
      listen();  //Listen for web client
    }


with this
Code: [Select]
//Write Data at 15 minute interval  
if ((minute) % 15 == 0) {
    if (AcqDone == false) {
      lastUpdate = dtStamp;   //store dtstamp for use on dynamic web page
      getWeatherData();
      updateDifference();  //Get Barometric Pressure difference
      logtoSD();   //Output to SD Card  --Log to SD on 15 minute interval.
      delay(10);  //Be sure there is enough SD write time
      speak();

      AcqDone = true;
    }
  }
  else
  {
    AcqDone = false;
    listen();  //Listen for web client
  }


and add a
Code: [Select]
bool AcqDone = false;
at the beginning.

This way you can change the interval easily. The other advantage is that maybe if you get  a GET request just few seconds before the acquisition you can skip the sending data to ThingSpeak since seconds will be not zero anymore.
The AcqDone flag is used to avoid to send multiple data during the minute.

Techno500

Update:  Web site for project is down; maybe a few days.

Web site Down for maintenance

William

zoomx

I tested successfully the software RTC with NTP but I am in search of a code to change automatically for DayLight Saving Time, USA and Europe.

zoomx

Found the best code, I think: https://github.com/JChristensen/Timezone
it is fully customizable.
But since it uses TimeLib there is a conflict

dayOfWeek is used in TimeLib and RTClib but is a define in TimeLib
Code: [Select]
#define dayOfWeek(_time_) ((( _time_ / SECS_PER_DAY + 4) % DAYS_PER_WEEK)+1) // 1 = Sunday
and a function in RTClib
Code: [Select]
uint8_t dayOfWeek() const;
I added this in the opened issue in TimeLib library but I am thinking to modify one of the libs and put the modified library in the same folder of the main sketch because dayOfWeek is used often and if I change the main library it will break others sketch.
The main candidate is RTClib since it has only two files, TimeLib instead has many files.

This issue is well known since years but never resolved.

Edit: I am wrong because I used an old RTClib, the new one uses dayOfTheWeek

zoomx

Software RTC seems to work well!

Main changes
Hardware RTC removed, it uses now a software RTC that is synchronized every 15 minutes to an NTP server.
RTC is in UTC but Display Clock is automatically adjusted for TimeZone and Daylight Saving Time.

Minor changes
  • Sealevel pressure is now the standard 1013.25, was 1032.4
  • You don't need anymore to modify the Adafruit BME280 library to use a different I2C address, it uses a #define
  • int year, month, date, hour, minute, second were changed to int year1, month1, date1, hour1, minute1, second1 because were in conflict with TimeLib library.


Other possible improvements
  • Check if it is possible to use a cheap ESP-01 module.
  • Move all user defined parts of code in a separate config.h file (if it is possible!).
  • Recalculate .010 to .020 inches of Mecury for file pressure difference in mBar.


Some test required since i wrote it this morning.

Techno500

#22
Jul 04, 2017, 09:25 pm Last Edit: Jul 04, 2017, 09:29 pm by Techno500 Reason: Corrections
@ zoomx

Testing your 08:54 am modifications.  

NTP timestamp is wrong in "LOG.TXT" on Sketch startup, if NTP packet is dropped, or has not been received yet.  Interval that "ThingSpeak.com" uses is affected in that with no fifteen minute interval data is not sent.  No logtoSD function logging occurs when no fifthteen minute interval occurs.

This why I use a real real time clock.  RTC are readily available online at low cost.  

Real Time Clock is needed.

William


zoomx

You're right! Need to improve NTP code or drop it. I have to test much more!

zoomx

Now my brain is online again!
I forgot that RTClib can work with an hardware RTC just changing this
RTC_Millis Clock;
into this
RTC_DS3231 Clock;


But there is a difference. To have Timezone and DST working, RTC must be in UTC.


I found the NTP client problem here

Code: [Select]
sendNTPpacket(timeServerIP); // send an NTP packet to a time server 
// wait to see if a reply is available
  delay(1000);

  int cb = udp.parsePacket();

I need to change the delay into a code that wait for the response with a timeout.

Techno500

#25
Jul 05, 2017, 09:10 pm Last Edit: Jul 08, 2017, 02:25 am by Techno500 Reason: Add link for Graphs
Wemos D1 R2 is back 100%, online now.  Attaching Sketch that the Wemos D1 R2 server is currently running.

zoomx is doing some modifications; at this time, I have not added his modifications.  Top priority, on project is for downloads to be uninterrupted by events such as logging.  Not so much concerned about missing a request to be parsed.  Response time is quick and the client closes when a request finishes.

Try a download, think you will be amazed at the speed, compared to before the "readFile" function that martinayotte of ESP8266 Community Forum created.

June 29, 2017, I submitted this project to "Create Arduino Project Hub." for review and publication.

Project web pages are back online!

Now featuring four graphs from "ThingSpeak.com:"  Graphed Weather Observations  Viewable on one web page; trace along the graphed line with your mouse for date/time stamp and value of graphed point.


William

zoomx

Attached there is a test sketch to evaluate the time taken to get a response from an NTP server. I get very short times using the USA NTP pool time.nist.gov

Using the server without hardware RTC is simpler but I think that I can add another define since the switch between hardware and software RTC is very simple, only one line.

It seems that the sketch fit on a cheap ESP-01 module (leaving 64k for SPIFFS) but I have to test it, I have only compiled.

Techno500

@zoomx

I tested your NTP code[ attached the result.

Do I understand correctly that you have to continuously hit the time server to get the fifteen minute interval?  IMHO, that  is a high demand on a time server.

Loop function is only ran, if on the fifteen minute interval; otherwise, Listen function is listening for a request from a client.

Planning to stay with a Real Time Clock for fifteen minute interval, new day, and day of week; to me, it eliminates high demand on a time server.

William

zoomx

The time request is done only one time every 15 minutes in your sketch modified by me, it can be changed, for example around two AM when in EU there is the  daylight time switch.

Instead the test show that you never get a response from the time server. Maybe UDP traffic is firewalled?
Test ask for a response and, if it don't get a response within one second make another request. This is repeated 4 times. You get continuous request only because this is a test.




Techno500

#29
Jul 17, 2017, 07:58 pm Last Edit: Jul 17, 2017, 08:07 pm by Techno500 Reason: Adding latest version of project changes -as of 7/17/2017.
@ zoomx

Re-ran test; Gateway was set to only pass TCP, my apologies.  Attached find test run.

Willing to try your code and run it for a couple of days.  Are you okay with me doing this?

Using a NTP server is a new experience for me; will need to get up to speed.  Thank you.

Attached latest code changes to project as of 7/17/2017.


William

Go Up