Transfer text from Arduino to a LAMPP SQL database

Hi there,

I have my Arduino Uno, including ethernet shield. It downloads values from a power meter and stores that in MariaDB database (LAMPP), My LAMPP server is on my Ubuntu 22.04 Laptop. I use IDE 2.04. The protocol I use is MODBUS. It works fine but misses sometimes records. This is rather difficult to determine. What I want now is to store text data in another database table, the text data will be a combination of data which I think is important to debug a current problem. E.g.

"ID = 2345 Loop is read data time duration is 23000 millisec."

Some how I don't manage to store text in the database. I have made a small arduino program to test it, but I got stuck.

EthernetClient eth_client;
String message = "Dit is ook een test 34";

void setup() {
  Serial.begin(9600);
  // put your setup code here, to run once:
  Ethernet.begin(mac, ip, dns, gateway);
 // sprintf(message," dit is getal %d ", getal);
//  sscanf("String", "%s", &message);
 Serial.println(message);
  save_debug();  
}

void loop() {
  // put your main code here, to run repeatedly:


}


void save_debug() {
  if (eth_client.connect(serv, 80)) {  //Connecting at the IP address and port we saved before
//    duration_s = millis();
    Serial.println("connected for storage debug");
    eth_client.print("GET /ethernet/debug_message.php?");  //Connecting and Sending values to database
    eth_client.print("message=");
    eth_client.print(message);
    eth_client.print(" HTTP/1.1");
    eth_client.println("");
    delay(5000);
//    duration_s = millis() - duration_s;    
    eth_client.stop();  //Closing the connection
  } else {
    // if you didn't get a connection to the server:
    Serial.println("connection failed");
    delay(1000);

  }
}

The file debug_message

<?php
include ('connection.php');
$sql_insert = "INSERT INTO debug_message (message) 
              VALUES 
                ('".$_GET["message"]."'

					  )";
if(mysqli_query($con,$sql_insert))
{
echo "Done";
mysqli_close($con);
}
else
{
echo "error is ".mysqli_error($con );
}
?>

If I run the following command via my browser
http://localhost/ethernet/debug_message.php?message=”ID 23 duration 5000”

This data is stored correctly in the database, even when I dismiss the double quotes.

In access.log I saw the following log:
<TCP/IP address> [24/Mar/2023:11:25:22 +0100] "GET /ethernet/debug_message.php?message=Dit is" 400 960

It seems that the text is just partly (truncated) there. So the question is how can I transfer fields which contains text to the database. In MariaDB the message field is defined as TEXT and the unicode utf8mb4_general_ci.

The reason to store debug data in the database instead of displaying it on the serial monitor is that the Uno is placed in our Barn (cold and windy) and some meters away from the house, so to have a look on the serial monitor I have to travel to the barn and that is not handy. As far as I know is the serial monitor not available via Ethernet.

That doesn't seem like it's working fine, then. Fix this before doing anything else.

Hi cedar, that is the reason I need more debugging info from that program. With help of data stored in the debug database I should be able to analyse the problem and correct it. With the program works fine I mean it pols every 5 minutes and sometimes once a day or less a record is missing.

So the question is how to store text in a text field of MariaDB via arduino as stated above.

Well, it looks like your HTTP GET request, sent by the Arduino, is badly formatted because you are seeing a 400 error in the log: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400

What value, for example, has serv ?

Maybe print out the entire string that you send to the web server on to the Arduino serial console.
Also check carefully the format of the request with a known working example.
Also, ensure compiler warnings are active when you compile your Arduino program.

Hi 6v6gt, Ok understood, nice to know that code 400 is an error code.
The serv is the TCP / IP address of my Laptop where the LAMP server runs, connection is OK.
The message on the serial monitor is ok as espected: "Dit is ook een test 34" without the double quotes.

The only warning
User configuration file does not exist or is not a regular file, skipping

Kind Regards.

I've no doubt about that. It's value just does not appear in your code snippet.

Anyway, look at the example code here and note at least the "host:" parameter which appears to be missing in your program: Arduino - HTTP Request | Arduino Tutorial

Hi 6v6gt, Regards,
The TCP/IP address is in the message_log, but I did not copy that part.
In my case I use localhost. I think when you omit host then the system assumes localhost, not 100% sure, but I did use the same syntax in another project which does the job ok. See below

[22/Mar/2023:16:18:51 +0100] "GET /ethernet/store_p1_data.php?&actual_usage=711&actual_delivery=0&meter_pw_usage_lt=15609606&meter_pw_delivery_lt=3385912&meter_pw_usage_ht=10435381&meter_pw_delivery_ht=10375997&active_pw_L1=286&active_pw_L2=3&active_pw_L3=422&active_pw_del_L1=0&active_pw_del_L2=0&active_pw_del_L3=0&recorded_on=230322161841 HTTP/1.0" 200 4

But in that case only values are transferred. It must be something with the text, string but what?

maybe try:
String message = "Dit%20is%20ook%20een%20test%2034";

Hi 6v6gt, I tried that but the same message in access_log.

My approach would then be

  1. change the example code/php to send a single integer instead of string just to get something working.
  2. compare the code with known working code and amend ( possible differences: host: , HTTP/1.0 etc. )

Hi I agree. Tomorrow I will try that out. Thanks in advance for the effort helping me and have a nice evening. Greetings Kees

Hi there,

I have tried to store a value into the database, but I did not succeed the way I want. The only option which works is when I omit the HTTP/1.1 at all and write the complete string in one line. As you mentioned the host should be used as well, this is correct for HTTP/1.1, but not for HTTP/1.0 explains maybe why my old program works. But also when I define the "Host : localhost" it did not do the job.

Message
[25/Mar/2023:12:04:35 +0100] "GET /ethernet/debug_message.php?value_1=36 HTTP/1.1" 400 960

OK found the problem missing the following line just before close. I will now check whether a text is possible.

eth_client.println("");
#include <Ethernet.h>

void setup() {
  Serial.begin(9600);
  // put your setup code here, to run once:
  Ethernet.begin(mac, ip, dns, gateway);
 // sprintf(message," dit is getal %d ", getal);
//  sscanf("String", "%s", &message);
 Serial.println(value_1);
  save_debug();  
}

void loop() {
  // put your main code here, to run repeatedly:


}


void save_debug() {
  if (eth_client.connect(serv, 80)) {  //Connecting at the IP address and port we saved before
//    duration_s = millis();
    Serial.println("connected for storage debug");
    Serial.println(value_1);
    eth_client.println("GET /ethernet/debug_message.php?value_1=36");  //Connecting and Sending values to database
  //   eth_client.println("GET /ethernet/debug_message.php?value_1=23");
//     eth_client.print("value_1=");
//     eth_client.print(String(value_1));
//    eth_client.print(" HTTP/1.1");
//    eth_client.println("");
//    eth_client.println("Host: String(HOST_NAME)");

    delay(5000);
//    duration_s = millis() - duration_s;    
    eth_client.stop();  //Closing the connection
  } else {
    // if you didn't get a connection to the server:
    Serial.println("connection failed");
    delay(1000);

  }
}

Tryial with HOST name

oid setup() {
  Serial.begin(9600);
  // put your setup code here, to run once:
  Ethernet.begin(mac, ip, dns, gateway);
 // sprintf(message," dit is getal %d ", getal);
//  sscanf("String", "%s", &message);
 Serial.println(value_1);
  save_debug();  
}

void loop() {
  // put your main code here, to run repeatedly:


}


void save_debug() {
  if (eth_client.connect(serv, 80)) {  //Connecting at the IP address and port we saved before
//    duration_s = millis();
    Serial.println("connected for storage debug");
    Serial.println(value_1);
    eth_client.println("GET /ethernet/debug_message.php?value_1=36 HTTP/1.1");  //Connecting and Sending values to database
  //   eth_client.println("GET /ethernet/debug_message.php?value_1=23");
//     eth_client.print("value_1=");
//     eth_client.print(String(value_1));
//    eth_client.print(" HTTP/1.1");
    eth_client.println("Host: " + String(HOST_NAME));  // serv is the tcp/ip address of my laptop where the database is on
    eth_client.println("");
    delay(5000);
//    duration_s = millis() - duration_s;    
    eth_client.stop();  //Closing the connection
  } else {
    // if you didn't get a connection to the server:
    Serial.println("connection failed");
    delay(1000);

  }
}

Ok finally, thnx to 6v6gt the program is working as I wanted to. What do I need to get thinsg running.

  • The text should not contain spaces, use %20 instead of space

  • When using HTTP/1.1 the Host is mandatory e.g. eth_client.println("Host: " + String(HOST_NAME));, the host in my case is the tcp/ip of the laptop (localhost. HOST_NAME is a char array.

  • Before eth_client.stop;, eth_client.println(""); is mandatory as well

Below the program storing to variables message and value_debug into the database.
Thanks for the fine help. Regards Kees

EthernetClient eth_client;
char message[] = "Dit%20is%20ook%20een%20test%2034";
long value_debug;
unsigned long duration_s;

void setup() {
  Serial.begin(9600);
  // put your setup code here, to run once:
  Ethernet.begin(mac, ip, dns, gateway);
 // sprintf(message," dit is getal %d ", getal);
//  sscanf("String", "%s", &message);
 Serial.println(value_debug);

  save_debug();  
}

void loop() {
  // put your main code here, to run repeatedly:


}


void save_debug() {
  if (eth_client.connect(serv, 80)) {  //Connecting at the IP address and port we saved before
    duration_s = millis();
    delay(2000);
    duration_s = millis() - duration_s;
    value_debug = duration_s;
    Serial.println("connected to storage data message_debug");
    eth_client.print("GET /ethernet/debug_message.php?");  //Connecting and Sending values to database
  //   eth_client.println("GET /ethernet/debug_message.php?value_1=23");
    eth_client.print("message=");
    eth_client.print(message);
    eth_client.print("&value_debug=");
    eth_client.print(value_debug);
    eth_client.println(" HTTP/1.1");

    eth_client.println("Host: " + String(HOST_NAME));  // serv is the tcp/ip address of my laptop where the database is on
    eth_client.println(""); // Mandatory end.
    delay(5000);
   
    eth_client.stop();  //Closing the connection
  } else {
    // if you didn't get a connection to the server:
    Serial.println("connection failed");
    delay(1000);

  }
}

The OP might consider a different scheme.

I have 13 ESP32's and 1 BeagleBone Black located throughout the property doing everything from monitoring the humidity levels in the bathrooms to being able to remotely control my HVAC system.

The ESP32's communicate with a MQTT Broker. On the Broker I run a Python program to do the things, like database storage, image processing, detecting deer to trigger the feeder, to watching the solar array and battery's.

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.