Fastest graph on Android via W5100 shield

I have extensive home automation with several Arduino Mega on Ethernet (W5100) with an SD card on one of them on which graphs (from other) are collected for ~20 different data, mainly temperatures. Each day 1 graph for each data in form of 240 samples (8 bit values), one at every 6 minutes (24h/6min = 240 samples).
Now I want to visualize these data on Android phone as multiple data graph, optionally also on PC. Of course with the ability to choose graphs for days gone by

One of Mega work as 3 phase power meter and I want to built current monitor functionality to visualize current on my Android phone in real time, let say 10 graph per second.

Web server is running on each arduino for remote control with Arduino Total Control (Android app) and UDP data broadcasting.

Which approach would give the best results specifically for current curve from power meter.

There are many options I guess, but from what I understand your best bet would be another wifi arduino(esp32 would probably be best for speed and price) that collects the data from the one with the sd card on demand, with mqtt sever or just asyncwebserver to pass it to mobile device or pc and use an app or pc software to display the data. Search for realtime graph data display apps and software or write your own app/software if you can't find exactly what you want but there must be plenty free graph charting software out there. You did not state what arduinos you are using. However for the sd card one, I wouldn't go with less than esp8266 although if you are using Mega with wifi shield that should be ok. Once you have a webserver that collects and passes the data on demand its just a case of finding a suitable software to request and display the data. Hope that helps.

Also for your power readings of 10 per sec let the esp32 collect the data direct from the Mega with shield when you are requesting that data (common sense I guess).

I imagine one option is to save data to SD on Arduino and download the files into a spreadsheet at leisure. You can send the data to Excel on PC using the PLX macro, where you can make all the relevant graphs to review on the phone later. PLX also gives you live graph on PC.

Tehnicni:
One of Mega work as 3 phase power meter and I want to built current monitor functionality to visualize current on my Android phone in real time, let say 10 graph per second.

This can be done with Bluetooth Graphic Terminal.

Tronicycle:
.... another wifi arduino(esp32 would probably be best for speed and price) that collects the data from the one with the sd card on demand,

I don't understand why the extra Arduino. Already, one Arduino Mega with an SD card is mainly intended for the regular collection of data from the Arduins that control the house. WiFi connection to Android is provided by the router.

Other tips are too general for me, I am 62 years old and have no energy to study protocols, search for applications, also due to my lack of programming and English skills, and especially not enough energy to research.

Nick_Pyner:
...... This can be done with Bluetooth Graphic Terminal.

The Arduino with Power meter is too far from the living room for bluetooth.
Above all, I don't want to use a PC. Occasionally I would like to look at a graph for a given day, maybe 2 years ago. In a few second.

About current graph from power meter I'm thinking about UDP protocol. Especially because it is extremely fast and I know how to do this. I really don't need confirmation of accepted graph. The problem is Android, I doubt there is a suitable app.

The trouble with young guys like you is your your lack of application to the problem and your inability to properly express it, and I fail to see your real need to have a two year old graph on your phone in seconds anyway. That said, it wouldn't surprise me if Android is entirely blameless, and you might be able to do this with Android retrieving the Arduino file over WiFi, or Internet, and displaying the data in a graph using Google Sheets or the like - all going on while you are salmon fishing in Kamchatka.

I only mentioned BTGraphics Terminal because, in the absence of geographic details, it probably fulfillled your clearly stated intention - real-time graphs, not two year old ones. There are similar means, and it may even be possible by WiFi, but who knows if that is in range? Similarly, I mentioned PLX because you said a PC is an option.

I want to visualize these data on Android phone as multiple data graph, optionally also on PC.

Above all, I don't want to use a PC.

The problem is Android

The below server code has a pseudo graph included in the page served. There are probably better examples of serving a graph from a server. If I were interested in a web graph I would put the graph material in an iframe in a static page to get a fast update.

/*
   Copyright (c) 2015, Majenko Technologies
   All rights reserved.

   Redistribution and use in source and binary forms, with or without modification,
   are permitted provided that the following conditions are met:

 * * Redistributions of source code must retain the above copyright notice, this
     list of conditions and the following disclaimer.

 * * Redistributions in binary form must reproduce the above copyright notice, this
     list of conditions and the following disclaimer in the documentation and/or
     other materials provided with the distribution.

 * * Neither the name of Majenko Technologies nor the names of its
     contributors may be used to endorse or promote products derived from
     this software without specific prior written permission.

   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
   ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
   WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
   DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
   ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
   LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
   ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
   SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

#include <ESP8266WiFi.h>
#include <WiFiClient.h>
#include <ESP8266WebServer.h>
#include <ESP8266mDNS.h>

#ifndef STASSID
#define STASSID "ARRIS-D1FA"
#define STAPSK  "xxxxxxxxxx"
#endif

const char *ssid = STASSID;
const char *password = STAPSK;

ESP8266WebServer server(80);

const int led = 13;

void handleRoot() {
  digitalWrite(led, 1);
  char temp[400];
  int sec = millis() / 1000;
  int min = sec / 60;
  int hr = min / 60;

  snprintf(temp, 400,

           "<html>\
  <head>\
    <meta http-equiv='refresh' content='5'/>\
    <title>ESP8266 Demo</title>\
    <style>\
      body { background-color: #cccccc; font-family: Arial, Helvetica, Sans-Serif; Color: #000088; }\
    </style>\
  </head>\
  <body>\
    <h1>Hello from ESP8266!</h1>\
    <p>Uptime: %02d:%02d:%02d</p>\
    <img src=\"/test.svg\" />\
  </body>\
</html>",

           hr, min % 60, sec % 60
          );
  server.send(200, "text/html", temp);
  digitalWrite(led, 0);
}

void handleNotFound() {
  digitalWrite(led, 1);
  String message = "File Not Found\n\n";
  message += "URI: ";
  message += server.uri();
  message += "\nMethod: ";
  message += (server.method() == HTTP_GET) ? "GET" : "POST";
  message += "\nArguments: ";
  message += server.args();
  message += "\n";

  for (uint8_t i = 0; i < server.args(); i++) {
    message += " " + server.argName(i) + ": " + server.arg(i) + "\n";
  }

  server.send(404, "text/plain", message);
  digitalWrite(led, 0);
}

void drawGraph() {
  String out;
  out.reserve(2600);
  char temp[70];
  out += "<svg xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\" width=\"400\" height=\"150\">\n";
  out += "<rect width=\"400\" height=\"150\" fill=\"rgb(250, 230, 210)\" stroke-width=\"1\" stroke=\"rgb(0, 0, 0)\" />\n";
  out += "<g stroke=\"black\">\n";
  int y = rand() % 130;
  for (int x = 10; x < 390; x += 10) {
    int y2 = rand() % 130;
    sprintf(temp, "<line x1=\"%d\" y1=\"%d\" x2=\"%d\" y2=\"%d\" stroke-width=\"1\" />\n", x, 140 - y, x + 10, 140 - y2);
    out += temp;
    y = y2;
  }
  out += "</g>\n</svg>\n";

  server.send(200, "image/svg+xml", out);
}

void setup(void) {
  pinMode(led, OUTPUT);
  digitalWrite(led, 0);
  Serial.begin(115200);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  Serial.println("");

  // Wait for connection
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  Serial.println("");
  Serial.print("Connected to ");
  Serial.println(ssid);
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());

  if (MDNS.begin("esp8266")) {
    Serial.println("MDNS responder started");
  }

  server.on("/", handleRoot);
  server.on("/test.svg", drawGraph);
  server.on("/inline", []() {
    server.send(200, "text/plain", "this works as well");
  });
  server.onNotFound(handleNotFound);
  server.begin();
  Serial.println("HTTP server started");
}

void loop(void) {
  server.handleClient();
  MDNS.update();
}

There is 2 types of graph which I need:
-Real time graph for grid line current curve monitor
-Browsing for old graph from SD card, let say to found which was the coldest day in 2018 and how my solar system work at that day :slight_smile:

Nick_Pyner:
The trouble with young guys like you is your your lack of application to the problem and your inability to properly express it...

:slight_smile: :frowning: And most importantly, I only know the basics of C. And I have no idea how to transfer a file from an SD card on Mega to the phone, probably with ES File explorer, but how, ftp? OK, I can google. And no experience with Google Sheets.

I want simplest working solution, then I make decision, lost few weeks (which means months :D) on PC with pain in the arm, shoulder, back :slight_smile: :frowning:

Tehnicni:
And I have no idea how to transfer a file from an SD card on Mega to the phone

There is a dumpfile example included in the SD examples in your IDE, specifically for this purpose. If you use date as file name, you can simply send the date to Arduino and it returns the file. I do this with Bluetooth to a Bluetooth terminal, and I'm sure you can do the same thing with WiFi to a WiFi terminal.

It might be smart to have a separate long term file just for recording daily statistics. It could save a lot of looking.

I don't know anything about Google Sheets either but, if you are disinclined to use BTGraphic Terminal, I believe it may offer a similar path with WiFi.

Sorry Tehnicni only just revisted these posts.
When I read Nicks reply in post #5 I figured you was getting more helpful info, glancing at all the posts since then It looks like you have quite a selection of options to help you get it sorted.
It sounds like you are using ethernet shield for your arduino. So if thats the case, you are correct in asking why you need another arduino. I thought you also needed a wireless connection to transmit the data to your graph and something with extra speed to manage it all fast. Its very hard to give help when the full details are missing but I'm not much younger than you myself and I sometimes feel the only thing keeping me going is trying to learn the stuff I feel I need to learn, because a free lunch is just a marketing strategy! The good news is your problem seems way easier than I first thought!
I have never written in java script and probably not much better at C than you but I can't afford to pay anyone to do it for me. So I don't see any choices!
I probably need to learn Java Script to try overcoming a problem of two devices communicating incorrectly.
I think you have got some good pointers there from Nick and Zoomcat. Sorry if I was too unhelpful.
Seems a lack of information is a common mistake many of us make "unbeknowingly" if thats how its spelt.
I often take examples and keep playing with them till they fit my requirements, if that helps.
Best of luck to you.
Andy

Tronicycle:
I probably need to learn Java Script to try overcoming a problem of two devices communicating incorrectly.

This is an Arduino Forum. If you are talking about communication between Arduinos and/or their peripherals, I would be surprised if you need to learn Java Script. I'm sure it is the first time I have ever heard of such a thing.

Once again
There is 5 Mega with w5100, which drive my home. I have also 2 Uno with W5100 for communication with Megas. Each Uno have LCD display and IR sensor for remote control of Arduino Mega. Data is broadcasted (UDP) to PC, which I want to replace with Arduino Mega with SD card (replaceing one Uno). Another remote control unit is Android with Arduino Total Control apk.

My idea was to use something like pfod or Blynk, but I think that it's not right for me, pfod because they are to slow when you work with W5100.
On these video 1st you have Power meter display (one of Mega), and then Uno display (standard 20 x 4 LCD with my own big charracters) alternating 3 power and one phase voltage. With IR remote I can switch Uno to show data from another Mega.

So what's the problem?
Is the Mega with SD card replacing the PC, or a Uno, or both? Seeing that you already have five Megas and two Unos in operation, I can't see how hard it would be to add another Mega. The only problem you might have is that the existing gear is under-utilised, but is that really a problem?, probably not, and who would really know?

but I think that it's not right for me, pfod because they are to slow when you work with W5100.

Incoherent, but I can't see how W5100 makes things slow. If you really have a speed problem, maybe you need a faster Arduino, perhaps a NodeMCU doing the job on WiFi.
Anyway, I recall Pfod can work over Wifi, and even other wireless means, and therefore may be a solution for whatever the problem might be.
Blynk is an IoT service. If you are able to use it, or something like it, on your own LAN, it may be able to give you the live display you want. Otherwise, it just does what it normally does, which probably does not suffice.

I think you would do well by properly summarising what the real problem is because, at the moment, you seem to be covering it with red herrings.

Nick I will probably already be in trouble for going slightly (just a tad) off topic. I have already put a post here , but after loads of search, it seems there are still issues with fauxmo 3.1.0 and echo dot version 3's, plus i want to include smart phone control too via webpage there are a lot of technical ways to go but i wanted to try the Java approach of kicking Alexa awake to my code. She will sometimes find 2 or 3 devices never more, oh once she found about 16 when I just had a bunch of test add.devices without functions but when I tried again she ignored me! One morning I put after shave on but that didn't work either. She sees me hobble about in back pain with my stick but still insists i learn more! (almost True but I see no eyes on alexa) Sorry Tehnicni just trying to explain my approx predicament to Nicks post #10 I am using arduino ide with esp8266 but trying to comunicate with an Amazon device, Having been a delivery driver for Amazon I knew there may be a few issues but not this bad. I want to give this thread back to Tehnicni so if anyone wants to discuss this please follow the above link.

@Nick_Pyner I want to replace PC and Uno with Mega&SD card (because of more Flash and RAM)
"The Arduino Ethernet Shield V1 SPI connection/shield is very slow." by these:
https://www.forward.com.au/pfod/pfodDataLogging/highSpeedDataLogging.html
Probable because acknowledgment, so I thing about UDP.

Tehnicni:
visualize current on my Android phone in real time, let say 10 graph per second.

Tehnicni:
"The Arduino Ethernet Shield V1 SPI connection/shield is very slow."

As I said, you really need to summarise what the problem is. I don't know much about Pfod, I don't need to, but I can read, and the next line from your quote tells me that you can do 350 samples a second using SPI.
I only update graphs once per second myself, but I'm sure there are more-qualified people round here who would not actually call 10 graphs a second "fast".

Further, While you might well-justify replacing a Uno with a Mega, no amount of Mega will replace a PC. You either need a PC, or you don't and, although fast is clearly a relative term round here, you may find the PC has the speed, and memory, you want. This is well out of my bailiwick, but I seem to recall some serious operators here who send data to PC because their problem is the Arduino SD card.

"She sees me hobble about in back pain with my stick but still insists i learn more!"

My Alexa says your Alexa is trash talking you behind your back. I think you may have crossed a line with her.

@Nick_Pyner In each graph (current curve from power meter) there is ~1000 samples (curve points) so I want to show ~12000 point per second.

Tehnicni:
I want to show ~12000 point per second.

Now, I wonder why something like that wasn't in the original post?

Nick_Pyner:
Now, I wonder why something like that wasn't in the original post?

Quote from original post:
"One of Mega work as 3 phase power meter and I want to built current monitor functionality to visualize current on my Android phone in real time, let say 10 graph per second."
10 graph of current curve!!