Parsing, Converting, and Storing Data from a Modbus TCP Response

I’m trying to capture data from a Midnite Solar Classic controller and make it accessible on an Arduino Uno with W5100 ethernet shield. I am talking to the controller through TCP and getting an understandable response, but I’m not sure how to parse the response, convert it to an integer, and store it as a variable. The response I am getting is the following:

0:1:0:0:0:5:A:3:2:1:F5

The part I need is the 1:F5 (501 in decimal). I’d like to take 501/10 = 50.1 V, and store it as (int Battery_Bank_Voltage, so I can send the data to the web for remote monitoring.

The data is stored in 16 bit registers on the solar controller, and I am sending the request through Modbus TCP. Right now I am only reading one register. If I can get it working, I would like to read 12 registers and pick out the data that I need.

If anyone can point me in the right direction, I’d really appreciate it. I tried to use the mudbus library, but I don’t understand it very well and couldn’t get it to connect to the controller. Instead, I used the TCPClient example code. My code is below:

#include <SPI.h>
#include <Ethernet.h>

// Enter a MAC address and IP address for your controller below.
// The IP address will be dependent on your local network:
byte mac[] = {  
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //Arduino Uno MAC
//IPAddress ip(MyIP);

byte message[]= {
 0x00, 0x01, 0x00, 0x00, 0x00, 0x06, 0x0A, 0x03, 0x10, 0x12, 0x00, 0x1};  //TCP request to read register 4114 (0x10 0x12)
 
// Enter the IP address of the server you're connecting to:
IPAddress server(controllerIP); // Classic solar controller

// Initialize the Ethernet client library
EthernetClient client;

void setup() {

  // start the Ethernet connection:
  Ethernet.begin(mac);

 // Open serial communications and wait for port to open:
  Serial.begin(9600);

  // give the Ethernet shield a second to initialize:
  delay(1000);
  Serial.println("connecting...");

  // if you get a connection, report back via serial:
  if (client.connect(server, 502)) {
    Serial.println("connected");
  } 
  else {
    // if you didn't get a connection to the server:
    Serial.println("connection failed");
  }
  delay(2000);
}

void loop()
{
  
  //write modbus request
    client.write(message, 12);

  // if there are incoming bytes available 
  // from the server, read them and print them:
  if (client.available()) {
    word c = client.read();
    Serial.print(c, HEX);
    Serial.print(":");
  }

  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();
    // do nothing:
    while(true);
  }
}

Try inserting code like this:

byte readMessage[20];
if (client.read(readMessage, 11) > 10) {
  voltage = (int)readMessage[9] << 8 & readMessage[10];
}

That’s just for that specific case. If you need more help, provide documentation or at least a link to the used product.

Are the strings always that long? If not, strtok() can be used to get the tokens, until you get the correct one. Then strtol() can be used to convert the value to an long.

Any device that uses the Modbus protocol will need to be communicated using the Modbus protocol, regardless if you use TCP or RTU modes for effective data exchange..

I don't see in your code any use of a Modbus protocol library, though you appear to write characters out the Ethernet port that might look like a formed Modbus register request packet (which I haven't checked)?

For Modbus TCP, look at using the Mudbus library, https://gitorious.org/mudbus

For RTU, look at the Simple ModBus Master library. You will find a thread on the later within this forum section, http://forum.arduino.cc/index.php?topic=176142.msg1886727#new

It might be worth while reading up a little on how data is reprsented when using Modbus. For example, data is transferred as 16 bit intergers, then it is up to you how that data is then used, whether it is a bit packed word or whether the data is stored as a float using 2 Modbus registers, hence 4 bytes.

Just to let you know, I'm using Mudbus in my Ether-Mega to communicate Modbus TCP back to my GNU/Linux system for my micro-hydro turbine and solar installation here at home.


Paul

Thanks Paul,

You're right that I'm just sending a Modbus register request packet out the Ethernet port. I tried using the Mudbus library, but I couldn't get it working.

This is my first Arduino project, and I am learning as I go. Often times my code ends up looking like a pile of butchered syntax. I just haven't been able to make sense of the Mudbus library, and I haven't been able to find any Mudbus code examples that read registers. Do you happen to have any examples of the basic Mudbus code to connect to a client and read registers?

Thanks for the response, PaulS.

I think the strings will always be that long, or longer.

Thanks for the suggestion, pylon.

I spent some time trying to store the response in different arrays (byte, char, uint_16, etc) , and then retrieve the values. I couldn't get it to work, which isn't all that surprising since I basically use a monkey-with-a-typewriter approach to coding. I also tried the code you suggested, without any luck.

I'm pretty new to Arduino and networking protocols. I'm learning slowly, but still don't really know my ass from my elbow. I've tried so many things, and gotten myself sufficiently twisted up, that it almost seems futile to try and explain.

Basically, I need more help. It's probably better to just look at the documentation, and ignore my rambling.

Here is a link to the networking documentation and register maps for the controller: http://www.midnitesolar.com/pdfs/classic_register_map_Rev-B3-May-28-2013.pdf

I'd like to read registers 4115 - 4129, and then pick out the values I want.

I can't tell you how much I would appreciate any help you can provide.

Another thing I maybe should have mentioned is that the response I am getting (0:1:0:0:0:5:A:3:2:1:F5) is actually repeated something like 37 times. Not sure if that is relevant or not.

I made a quick example using the Mubus library. It is untested as I don't have a Midnitesolar controller and in my setup, my Arduino is the slave unit, not the master. You will need to set the IP to the correct IP address of the Midnite unit and any other router settings as well.

If you are connecting a computer directly to the Midnite controller, check if you need a cross-over UTP cable or if it will auto-switch the tx and rx lines. Otherwise go through a router.

It is setup for all the registers you asked, but only prints out the first register to the serial port. I looked at the PDF and at their awful mixed up internal register allocation.

You can grab the code from my dropbox https://www.dropbox.com/s/a6g1h4n0au55sd0/Mudbus-example.cpp?dl=0

If you are using the Arduino IDE, you may need to change the file extension from .cpp to .ino I think. ( I don't use the Ardunio IDE)

Do you have further ideas of what you would like to do with this infomation you can get from the solar controller?


Paul

That’s awesome. Thanks, wallaby. I will test the code tomorrow. I’m going through a router.

As of now, I have the Uno controlling a solar water heating system and sending temperature data to emoncms.org for visualization. I’d like to do the same with the data from two Classic charge controllers. If I can manage that, I would also like to connect to two Magnum inverters and grab some data. I looked at the Magnum networking protocols (attached), and one inverter is the master, and the other a slave. They don’t have a register map, and it seems that-the inverters at least-just send out their data every 100ms. I’m not sure how much that will complicate things.

In addition to the two MS-PAE inverters, I have an ME-BMK battery monitor, and an auto gen start (AGS) module - all connected to an ME-RTR router. I’d like to get data for AC voltage/current going in and out of the inverters, DC voltage/current out of the inverts , the SOC of the batteries (ME-BMK), the gen set run time (AGS), and days since last run (AGS).

I’m posting my code for the solar thermal controller below. It is working, but probably ain’t pretty - you might want to put your gumboots on before diving in. If I can figure out the Classic bit, I will try to add that to the solar thermal code. If I make it that far, I’ll try for the Magnum data. It could be a tall order for me.

Thanks again for your help. If you have any other thoughts, I’d be glad to hear 'em.

#include <OneWire.h>
#include <DallasTemperature.h>
#include <SPI.h>
#include <Ethernet.h>
#include <dht.h>
#include <Adafruit_MAX31855.h>

//*****Set thermocouple pins and declare thermocouple breakout board***********
// Thermocouple pins
int thermoDO = 7;
int thermoCS = 6;
int thermoCLK = 5;
Adafruit_MAX31855 thermocouple(thermoCLK, thermoCS, thermoDO);

//*****Set variables for pump and fan operation***********
int pump=0;
int fan=0;

//*****DHT Temp/Humidity sensor setup******
dht DHT;
#define DHT21_PIN 2

// assign a MAC address for the ethernet controller.
// fill in your address here:
byte mac[] = { 
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED};

// fill in an available IP address on your network here,
// for manual configuration:
//IPAddress ip(Myip);

// fill in your Domain Name Server address here:
//IPAddress myDns(1,1,1,1);

// initialize the library instance:
EthernetClient client;

char server[] = "emoncms.org";
String apikey = "Myapi"; //api key

unsigned long lastConnectionTime = 0;          // last time you connected to the server, in milliseconds
boolean lastConnected = false;                 // state of the connection last time through the main loop
const unsigned long postingInterval = 5*1000;  // delay between updates, in milliseconds

//Sets DS18b20 temperature probes to pin 3
#define ONE_WIRE_BUS 3                  
OneWire oneWire(ONE_WIRE_BUS);         
DallasTemperature sensors(&oneWire);
// DS18b20 Addresses where found manually using temperature_search (emontx firmware)
DeviceAddress address_outside  = { 0x28, 0x64, 0x84, 0x04, 0x05, 0x00, 0x00, 0xEE };
DeviceAddress address_tank_bottom = { 0x28, 0x11, 0xBF, 0x56, 0x05, 0x00, 0x00, 0xB6 };
DeviceAddress address_tank_top = { 0x28, 0x05, 0x0F, 0x56, 0x05, 0x00, 0x00, 0x7E };

void setup() {
  // start serial port:
  Serial.begin(9600);
  // give the ethernet module time to boot up:
  delay(1000);
  
  // start the Ethernet connection using a fixed IP address and DNS server:
  Ethernet.begin(mac);
  // print the Ethernet board/shield's IP address:
  Serial.print("My IP address: ");
  Serial.println(Ethernet.localIP());
  
  //initiate temp sensors
  sensors.begin();
  
  // Relay control digital output on pins 8 & 9 
  pinMode(8,OUTPUT); // pump relay
  pinMode(9,OUTPUT); // fan relay
}

void loop() {
  delay(5000);

  // get humidity/temp data (sensor AM2301 (DHT21))
  Serial.print("DHT21, \t");
  int chk = DHT.read21(DHT21_PIN);
  switch (chk)
  {
    case DHTLIB_OK:
//Serial.print("OK,\t");
break;
    case DHTLIB_ERROR_CHECKSUM:
Serial.print("DHT Checksum error,\t");
break;
    case DHTLIB_ERROR_TIMEOUT:
Serial.print("DHT Time out error,\t");
break;
    default:
Serial.print("DHT Unknown error,\t");
break;
  }
int Room_Temp = 1.8*(DHT.temperature) + 32;  
int RH = (DHT.humidity);   


// get DS18b20 temperature probe data
  sensors.requestTemperatures();
  
  int Outside_Temp = sensors.getTempF(address_outside);
  int Tank_Bottom = sensors.getTempF(address_tank_bottom);
  int Tank_Top = sensors.getTempF(address_tank_top);
  int Collector_Temp = thermocouple.readFarenheit();
  int diff = Collector_Temp - Tank_Bottom;
  
  runPump(Collector_Temp, Tank_Bottom, diff);
  
  runFan(RH);
  
  debug(Collector_Temp, Tank_Top, Tank_Bottom, Outside_Temp, Room_Temp, RH, pump, fan, diff);

  // if there's incoming data from the net connection.
  // send it out the serial port.  This is for debugging
  // purposes only:
  if (client.available()) {
   httpDataIn();
  }

  // if there's no net connection, but there was one last time
  // through the loop, then stop the client:
  if (!client.connected() && lastConnected) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();
  }

  // if you're not connected, and ten seconds have passed since
  // your last connection, then connect again and send data:
  if(!client.connected() && (millis() - lastConnectionTime > postingInterval)) {
    httpRequest(Collector_Temp, Tank_Top, Tank_Bottom, Outside_Temp, Room_Temp, RH, pump, fan, diff);
  }
  // store the state of the connection for next time through the loop:
  lastConnected = client.connected();
}

// this method makes a HTTP connection to the server:
void httpRequest(int Collector_Temp, int Tank_Top, int Tank_Bottom,int Outside_Temp, int Room_Temp, int RH, int pump, int fan, int diff) {
   
  
  // if there's a successful connection:
  if (client.connect(server, 80)) {
    Serial.println("connecting...");

    // send the HTTP PUT request:
    client.print("GET /input/post.json?json={outside_temp");
    client.print(":");
    client.print(Outside_Temp);
    client.print(",Tank_Bottom_Temp:");
    client.print(Tank_Bottom);
    client.print(",Tank_Top_Temp:");
    client.print(Tank_Top);
    client.print(",Room_Temp:");
    client.print(Room_Temp);
    client.print(",RH:");
    client.print(RH);
    client.print(",Collector_Temp:");
    client.print(Collector_Temp);
      client.print(",Temp_differential:");
    client.print(diff);
     client.print(",Pump:");
    client.print(pump);
    client.print(",Fan:");
    client.print(fan);
    client.print("}&apikey=");
    client.print(apikey);
    client.println(" HTTP/1.1");
    client.println("Host: emoncms.org");
    client.println("User-Agent: arduino-ethernet");
    client.println("Connection: close");
    client.println();

    // note the time that the connection was made:
    lastConnectionTime = millis();
  } 
  else {
    // if you couldn't make a connection:
    Serial.println("connection failed");
    Serial.println("disconnecting.");
    client.stop();
  }
}

void runPump(int Collector_Temp, int Tank_Bottom, int diff)
{
  // If Collector is 40F above the bottom of the cylinder turn pump on
  if (diff > 40.0) pump = 1;
  
  // If Collector is less than 25F above the bottom of the cylinder turn pump off
  if (diff < 25.0) pump = 0;

  // Check to stop overheating of cylinder 
  if (Tank_Bottom > 130) pump = 0;  
  
  
  if (pump) digitalWrite(8,HIGH); else digitalWrite(8,LOW);
}

void runFan(int RH)
{
   // if humidity > 60% fan turns on
  if (RH > 60) fan = 1;
  //if humidity drops back below 50% fan turns off
  if (RH < 50) fan = 0;
  
  if (fan) digitalWrite(9, HIGH); else digitalWrite(9,LOW);
  
  //print RH value
  Serial.print("Rel. Humidity = ");
  Serial.println(RH);
}

void httpDataIn(){
   while (client.available()) {
   char c = client.read();
    Serial.print(c);}
}

void debug(int Collector_Temp, int Tank_Top, int Tank_Bottom,int Outside_Temp, int Room_Temp, int RH, int pump, int fan, int diff)
{
  //print data to serial bus
  if (isnan(Collector_Temp)) {
    Serial.println("Something wrong with thermocouple!");
   } else {
     Serial.print("Collector Temp = "); 
     Serial.println(Collector_Temp);
   }
  Serial.print("Tank_Bottom Temp = ");
  Serial.println(Tank_Bottom);
  Serial.print("Tank_Top Temp = ");
  Serial.println(Tank_Top);
  Serial.print("Temperature Differential = ");
  Serial.println(diff);
  Serial.print("Outside Temp = ");
  Serial.println(Outside_Temp);
  Serial.print("Room Temp = ");
  Serial.println(Room_Temp);
  Serial.print("Rel. Humidity = ");
  Serial.println(RH);
  Serial.print("Pump state = ");
  Serial.println(pump);
  Serial.print("fan state = ");
  Serial.println(fan);
  Serial.println();
   }

Magnum-Networking-Communications-Protocol-(2009-10-15).pdf (62 KB)

For your HTTP sending, have a look at my suggestion and comment on this thread in this same forum section. http://forum.arduino.cc/index.php?topic=268457.msg1892482#msg1892482

You may be be able to make it more effective and readable by using a such a method, rather than lots of traffic with those client.prints(...). Good luck with the testing.

oh, heck, my gumboots filled up, oh well :D


Paul

Thanks, again. I'll be probably be figuring out the http sending suggestion for the next few days.

I tried to the mudbus example code, and was getting 0 back for all the registers. I set the IPaddress ip() to the ip of the midnite classic, set the other addresses to match my router settings, and added SPI.h to the header.

I'm not even really sure how to troubleshoot and try to figure out the problem.

You can try to conect to the midnite solar controller from your computer and communicate to it with a modbus application that allows you to set exactly the device parameters and then see what comes back.

What sort of setup do you run there, a computer with Linux or Windows or a Mac with OSX? There are a number of utilities for these systems that might offer some additional help.

PS, sent you a PM.


Paul

jwill, I have just noticed something that I don't understand in your post #9.

Why is the code presented in that reply nothing to do with the code you initially presented? It has nothing to do with Modbus at all. Unless something is going wrong with the forum's server database and got its knickers in a knot, I'm lost.

Can you shed any light on it?


Paul

The code in post #9 is just for controlling a solar water heating system, and sending some temperature values to the web. I wanted to add data from the charge controllers and inverters, but didn't really know where to start. I just started tweaking different example code (TCPclient, mudbus, Webserver, etc.) to see if I could figure out how to connect to the Midnite Classic. The only one I made any progress with was a TCPclient example - I got a response from the Classic, but couldn't figure out what to do with it. That is the code that I originally posted. My hope was to figure out how to get the data of the Classic, and then insert that code into my main solar water heating code.

When you asked what I wanted to do with the info from the Classic, I posted the water heating/webClient code (post #9), because that is what I ultimately want to do with the Classic data.

Hope that makes sense. Sorry about the confusion.

I'm running OSX. I will try a modbus application tomorrow, and see if I can figure out what is going wrong.

Did you get your 'ino to talk to your Midnite Solar? I realize this was many moons ago, but here’s one way to parse your string into tokens and convert them to integers:

/*
 * int parsehex(char *s, int reg[])
 *
 * Input
 *   s  a string of hex values separated by the : character. The : separator characters will be
 *   overwritten with NULs.
 * 
 * Ouput
 *   int reg[]  up to NPARSE items
 *
 * Returns
 *   number of values
 */
int parsehex(char *s, int reg[])
{
  char *p, *q;
  p = s;
  q = s;
  int i;
  int lastone = 0;
  for (i = 0; i < MAXPARSE; i++) {
    while (*q != ':' && *q != '\0')
      q++; /* skip until separator or end of string */
    if (*q == ':')
      *q = '\0'; /* change separator into NUL */
    else if (*q == 0) /* end of string */
      lastone = 1;
    reg[i] = (int) strtol(p, NULL, 16); /* convert from hex string to int */
    if (lastone)
      return i+1;
    ++q;
    p = q;
  }
  return MAXPARSE;
}

Here is how to call it and convert item 10, here 0xF5, into the voltage, here 24.5 volts.

#
define MAXPARSE 20
  int reg[MAXPARSE];
  char s[256] = "0:1:0:0:0:5:A:3:2:1:F5";
  int n = parsehex(s, reg);
  if (n >= 11) {
    float volts = reg[10]/10.;
  }