Post data from a serial line on Xively - Not working

Hi All,

We have a cabin up in the Norwegian mountains where we use solar and wind power. A battery bank is always monitored by a Victron BMV-700. Studing the BMV-700 manual I learned that it had a serial line output that push all the interesting data as text (ASCII). My idea was to connect this port to an Arduino which again is connected to internet and send the data to Xively.

As I'm very new to Arduino and programming I experience problems. So far it just does not work. If I remove all of the "BMV-700 SoftSerial" interface and just test it with dummy data I do get the data through to Xively, so some of it work. The most important thing however, does not work. Parsing the serial data and push it to Xively directly. I would be very happy if someone here would have the time to assist me, or at least maybe give me a few clues as to what I'm doing wrong I would really appreciate it. Might also be more people in here that could benefit from this as well?

The code is posted at its current state in the following post due to limitation of characters. Most of the code regarding the SoftSerial and serial parsing is taken from user FNW2013 who did something similar. I've requested help there, but it's silent.

The VE.Direct.Protocol can be found here if needed.

Hope to hear from someone ;D

Hi

Did you solve the issue or does it still exist?

BRs

Hi, sorry for the late reply. I thought I solved it , but was wrong. Still does not work. I'm just not good enough with code I guess.

After a year after my first post I’m picking this up again. Basically I want to read a serial line and post the data to Xively. Ok, so the code is now partially working now. I do get some kind of data to Xively, but it’s random what and when the values are sent. It’s also not all of the values coming over. The serial data is in the following format (ascii) with tabs between Label and Value and New Line (0x0a).

Checksum P
PID 0x203
V 12734
I -3376
P -43
H1 -70344
H2 -1576
H3 0
etc…

My code is as follows. It must be something wrong with how I parse the serial data and the way it is stored and/or forwarded. hope someone out there might have an idea of how this could work.

/*
Energy monitoring data on serial from Vitron BMV700 and upload to xively 
VE.Direct protocol as follows (ASCII)
*/

#include <SPI.h>
#include <Ethernet.h>
#include <HttpClient.h>
#include <Xively.h>
#include <SoftwareSerial.h>		//Added by OSN: Read BMV 700 on software serial

int n = 1;

SoftwareSerial BMS(7, 8);
char cBMSSerialChar;
boolean bPropertyNext	= false;
boolean bValueNext		= false;
String sPropertyBuffer	= "";
char aValueBuffer[10]	= "         ";
char aPropertyBuffer[10]= "         ";
int bValueBufferIndex = 0;		//Added by OSN
int bPropertyBufferIndex = 0;		//Added by OSN
int iValueBufferIndex	= 0;
int nCycles				= 0;
float fVolts			= 0;
float fAmps				= 0;
float fCE				= 0;
float fTTG				= 0;
float fSOC				= 0;
float fTotalAh			= 0;
float fDaysSinceLastSync = 0;

#define API_KEY "############################"	//API key
#define FEEDID #######		//Xively feed ID eg ##########


/////// IP Kongsberg //////
byte mac[] = {0xCA,0xFE,0xBA,0xBE,0x00,0x00};	//Added by OSN: Defines MAC
byte ip[] = { 192, 168, 10, 21 };				//Added by OSN: Defines IP
//byte dns[] = { 192, 168, 10, 1 };				//Added by OSN: Defines DNS
byte gw[] = { 192, 168, 10, 1 };				//Added by OSN: Defines Gateway
byte subnet[] = { 255, 255, 255, 0 };			//Added by OSN: Defines Subnet

unsigned long lastConnectionTime = 0;                // last time we connected to Xively
const unsigned long connectionInterval = 15000;      // delay between connecting to Xively in milliseconds

// Initialize the Xively library

// Define the string for our datastream ID
char Voltage[] = "Voltage";
char Current[] = "Battery_Current";
char Cycles[] = "Charge_Cycles";
char CE[] = "Consumed_Energy";
char SOC[] = "State_of_Charge_Percent";
char TTG[] = "Time_to_Go_Hours";
char DLC[] = "Days_Since_Last_Full_Charge"; // this is actually seconds since last full charge, we'll adjust accordingly later.
char CAH[] = "Cumulative_Energy_from_the_Battery";


XivelyDatastream datastreams[] = 
{
  XivelyDatastream(Voltage, strlen(Voltage), DATASTREAM_FLOAT),
  XivelyDatastream(Current, strlen(Current), DATASTREAM_FLOAT),
  XivelyDatastream(Cycles, strlen(Cycles), DATASTREAM_INT),
  XivelyDatastream(CE, strlen(CE), DATASTREAM_FLOAT),
  XivelyDatastream(SOC, strlen(SOC), DATASTREAM_FLOAT),
  XivelyDatastream(TTG, strlen(TTG), DATASTREAM_FLOAT),
  XivelyDatastream(DLC, strlen(DLC), DATASTREAM_FLOAT),
  XivelyDatastream(CAH, strlen(CAH), DATASTREAM_FLOAT),

 
};

// Wrap the datastream into a feed
XivelyFeed feed(FEEDID, datastreams, 8 /* number of datastreams */);
EthernetClient client;
XivelyClient xivelyclient(client);

void setup() 
{
   
  Serial.begin(19200);
  
  if (Ethernet.begin(mac) != 1);
  {
	Serial.println("Error getting IP address via DHCP, going to static IP:"); 
	Serial.println("ip");
	delay(5000);
	Ethernet.begin(mac, ip);
  }
  Serial.println("Network initialized");
  Serial.println();
}

void readBMSData()
{
	while (BMS.available() > 0)
	{
		// Read data from battery monitoring system
		cBMSSerialChar = BMS.read();
		
		// If carriage return character received
		// and value of a certain property is next
		// data block to expect
		if (cBMSSerialChar == '\r' && bValueNext)
		{
			// Store battery information provided by
			// BMS in floats corresponding to properties
			if (aPropertyBuffer[0] == 'V')
			fVolts = atof(aValueBuffer) / 1000;
			else if (aPropertyBuffer[0] == 'I')
			fAmps = atof(aValueBuffer) / 1000;
			else if (aPropertyBuffer[0] == 'S' && aPropertyBuffer[1] == 'O' && aPropertyBuffer[2] == 'C')
			fSOC = atof(aValueBuffer) / 10;
			else if (aPropertyBuffer[0] == 'C' && aPropertyBuffer[1] == 'E')
			fCE = atof(aValueBuffer) / 1000;
			else if (aPropertyBuffer[0] == 'T' && aPropertyBuffer[1] == 'T' && aPropertyBuffer[2] == 'G')
			fTTG = atof(aValueBuffer) / 60;
			else if (aPropertyBuffer[0] == 'H' && aPropertyBuffer[1] == '4')
			nCycles = atoi(aValueBuffer);
			else if (aPropertyBuffer[0] == 'H' && aPropertyBuffer[1] == '6')
			fTotalAh = atof(aValueBuffer) / 1000;
			else if (aPropertyBuffer[0] == 'H' && aPropertyBuffer[1] == '9')
			fDaysSinceLastSync = atof(aValueBuffer) / 86400;
			
			// Clear buffer values
			// (both buffers are of the same size so it doesn't
			// matter on which to call sizeof)
			for (byte x = 0; x < sizeof(aValueBuffer); x++)
			{
				aValueBuffer[x] = ' ';
				aPropertyBuffer[x] = ' ';
			}
			bValueBufferIndex = 0;
			bPropertyBufferIndex = 0;
		}
		// In case of a newline character
		// next data block will contain a property
		else if (cBMSSerialChar == '\n')
		{
			bPropertyNext = true;
			bValueNext = false;
		}
		// In case of a tabulator character
		// next data block will contain a value
		// (of the property transmitted before)
		else if (cBMSSerialChar == '\t' && bPropertyNext)
		{
			bPropertyNext = false;
			bValueNext = true;
		}
		// Store incoming property characters
		// in corresponding string
		else if (bPropertyNext)
		{
			// Check if property index is still within bounds
			if (bPropertyBufferIndex <= (sizeof(aPropertyBuffer) - 1))
			{
				aPropertyBuffer[bPropertyBufferIndex] = cBMSSerialChar;
				bPropertyBufferIndex++;
			}
		}
		// Store incoming value characters
		// in corresponding char array
		// (in order to be able to transform it
		// to a floating point number by 'atof')
		else if (bValueNext)
		{
			// Check if value index is still within bounds
			if (bValueBufferIndex <= (sizeof(aValueBuffer) - 1))
			{
				aValueBuffer[bValueBufferIndex] = cBMSSerialChar;
				bValueBufferIndex++;
			}
		}
	}
}

void loop() 
{
   if (millis() - lastConnectionTime > connectionInterval) 
	{
	readBMSData();	//Get values from BMV700
    sendData(); 	// send data to xively
    lastConnectionTime = millis(); // update connection time so we wait before connecting again
	}
}

// send the supplied values to Xivily, printing some debug information as we go
void sendData() 
{
long milisec = millis(); // calculate time in milisec
long time=milisec/1000; // convert time to sec

  //Battery Voltage datastream  
  datastreams[0].setFloat(fVolts); // The maths is there to convert mV into V.
  Serial.print("Read Voltage ");
  Serial.println(datastreams[0].getFloat());

  //Battery current datastream
  datastreams[1].setFloat(fAmps);
  Serial.print("Read Current ");
  Serial.println(datastreams[1].getFloat());

  //Consumed energy datastream
  datastreams[2].setFloat(fCE);
  Serial.print("Read Consumed Energy ");
  Serial.println(datastreams[2].getFloat());
  
  //State of charge (expressed as a percantage) datastream
  datastreams[3].setFloat(fSOC);
  Serial.print("Read State of Charge ");
  Serial.println(datastreams[3].getFloat());

  //Time to Go datastream
  datastreams[4].setFloat(fTTG);
  Serial.print("Read Time to Go ");
  Serial.println(datastreams[4].getFloat());
  
  //Number of Cycles
  datastreams[5].setFloat(nCycles);
  Serial.print("Depth of Last Discharge ");
  Serial.println(datastreams[5].getFloat());
  
  //Days since last full charge datastream
  datastreams[6].setFloat(fDaysSinceLastSync); 
  Serial.print("Days Since Last Full Charge ");
  Serial.println(datastreams[6].getFloat());
  
  //Cumulative Energy taken from the Battery
  datastreams[7].setFloat(fTotalAh);
  Serial.print("Cumulative Ah from the Battery ");
  Serial.println(datastreams[7].getFloat());

  Serial.println("Uploading to Xively");
  int ret = xivelyclient.put(feed, API_KEY);
  Serial.print("PUT return code: ");
  Serial.println(ret);

  Serial.println();
}

aValueBuffer is NOT a string. Passing it to a function that expects a string is NOT a good idea.

A string is a NULL terminated array of chars. Your array of chars is NOT NULL terminated. It is trivial to make is a string:

			if (bValueBufferIndex <= (sizeof(aValueBuffer) - 1))
			{
				aValueBuffer[bValueBufferIndex] = cBMSSerialChar;
				bValueBufferIndex++;
                                aValueBuffer[bValueBufferIndex] = '\0'; // NULL terminate the array
			}

What do your Serial.print() statements tell you is happening?

Why don’t you have any?

PaulS,

Thanks for coming back to me. I did have Serial.print() for "aValueBuffer" and more, but removed them (along with lots of notes) due to limitations posting the code. Basically they returned random data as described. Some data came through and some not, but it was not consistently. Voltage eg. never came through, but I did get Power and Ampere if I just waited long enough. I've also recently ported to Arduino Mega so I could rule out Software Serial as factor. Now I use a "real" serial line. Anyway, I'll see if I can get my head around this string/array thing.

As a side note to all this I'll also mention that I do have a "working" solution today involving a Raspberry Pi w/Node-Red sending the data to EmonCMS. It works OK. Still I would prefer an Arduino solution with less power consumption and dead-steady operations as oppose to the RPi with broken SD cards, sudden boot failures etc. (probably due to sudden power losses).

Just solved for this myself.

https://github.com/winginitau/VictronVEDirectArduino