Convert streaming ASCII to Text to read Battery Monitor Data

I am a newbie and would like to get some guidance on how to proceed with a project that I'd like to pursue.

I have a Bogart Engineering TM-2030A Battery Monitor. It has a phone jack connector J2 that allows one to pickup on real time data of constantly repeating ASCII data stream at 2400 baud, 8N1, from pin 4 of that connector.

A link to Bogart Engineering pdf explaining what data is available and how to access it:

https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=54&cad=rja&uact=8&ved=0ahUKEwjY4b76-LXTAhVj7YMKHUX8COA4MhAWCDYwAw&url=https%3A%2F%2Fforum.arduino.cc%2Findex.php%3Faction%3Ddlattach%3Btopic%3D300438.0%3Battach%3D114382&usg=AFQjCNFYb0z-8hAllWHZnMJaySUAHgPrkg&sig2=y4LX3KPU0uLjrsVFi83rOQ

I would like to read in this streaming data to my Arduino Uno and on back to my computer to veiw in Serial Monitor for starters. Eventually I'd like to stream this data into Excel in real time so that I can analyze it, plot it, etc.

One of the pieces of data that is output is Battery % of charge, that the TM-2030 calculates. I will use this value to create a window comparator to trigger an external relay to switch back to Grid power when the battery % of charge goes below a Low set point, then switch back again to Inverter power when the battery % once again returns to a higher set point.

I did a test using the Digital Button sketch:

I replaced the button switch with the interface circuit suggested by Bogart Engineering. It worked, and I got 0's and 1's down column 1 of the Serial Monitor. So it works, but I'm not sure what kind of sketch to use to actually get text to be captured with the Serial Monitor.

I would like a little guidance.

You'll first need to understand the format of the data being set from the controller since the documentation says what is sent but not how it is sent. The best way to start would be a USB to Serial adapter like a FT232 or CH340G available on eBay for $1 to $5, depending upon type. If you have a Chinese UNO, you most likely already have drivers installed for the CH340G so I'd go that direction if that is the case. If you have a genuine Arduino then I'd probably go with the FT232 type.

Get the USB<->serial adapter working at 2400 baud and connect to a terminal program on your PC with the battery monitor attached. Observe the data that arrives and see if broken up into individual strings that are terminated with a CR (carriage return) LF (line feed) pair or just an endless stream of data. This will determine how you setup your UNO - to just capture characters, which you then need to parse into data, or entire strings which can be taken apart a bit easier.

You'll want to look at the Serial.read function to actually handle the incoming data. Your PC can be reading the same string in parallel, that is having the input of the UNO and PC USB adaptor driven from the same serial output of the monitor.

Thank you avr_fred for the quick response. You said,

"the documentation says what is sent but not how it is sent."

I'm confused by your comment because from the documentation, it says,

"The TM-2025 real time data (no logged data) comes out from Pin 4 on J2 as a constantly repeating ASCII data stream at 2400 baud. (The usual: 8 bit, parity none, stop bits=1) For example, the amps and watts show as “A=3.3, W=36.7” The output "high" volts
(+5V) is "mark" or "idle" state. "Start bit" (logic 0), is about 0 volts.)"

Doesn't this statement tell how the data is sent? Since I mentioned that I tested the data stream using the Digital Button sketch, and got a column of 0's and 1's down column 1 of the Serial Monitor and doesn't this also mean that it is already in a serial format (one bit after another)?

I have a genuine Arduino Uno. The new Arduino Uno that I have:

The description says:

The Uno differs from all preceding boards in that it does not use the FTDI USB-to-serial driver chip. Instead, it features the Atmega16U2 (Atmega8U2 up to version R2) programmed as a USB-to-serial converter.

So doesn't this mean that the Arduino Uno already has a USB to Serial adapter?

Your last sentence is puzzling. I'm not sure how to interpret what you mean since there is only data going in one direction, and that is into my computer from the Arduino Uno.

I would like to read in this streaming data to my Arduino Uno and on back to my computer to veiw in Serial Monitor for starters.

The Uno only has one serial port and that is designed to talk to your computer through the USB / serial converter.

So to read the data from your instrument using the software serial libiary using two other pins at 2400baud. Then use the built in serial port at a higher speed, say 9600 baud to send what you read to the computer.

Lots of help on the net, here is one link plucked at random from the net:- http://arduinobasics.blogspot.co.uk/2012/07/arduino-basics-simple-arduino-serial.html

Thank you Grumpy_Mike for your response. As I have written in my response to avr_fred, The TM-2025 real time data (no logged data) comes out from Pin 4 on J2 as a constantly repeating ASCII data stream at 2400 baud. So to try to read the data at any other baud rate, such as 9600 baud, all I get returned is gibberish.

When I originally set up the Digital Read Serial to receive this data stream, I did try other baud rates, other than 2400 baud, just to see what would be displayed. The results were nonsensical characters being displayed horizontally across the Serial Monitor window. When I set the baud rate to 2400 baud, I got one bit per row as per the Line Feed provided by:

Serial.println(buttonState);

So to try to read the data at any other baud rate, such as 9600 baud, all I get returned is gibberish.

I didn't say that. I said send to your computer at 9600 baud and read from software serial at 2400 baud.
You need to write out faster than you read in so that your input buffer doesn't overflow.

I got one bit per row as per the Line Feed provided by:

Which is rubbish.

Do you know about asynchronous data? That is what you are trying to receive. You can only do this on a Uno with software serial. Have you read that link I posted?

2400 baud, 8 data bits, no pairity and "A=3.3, W=36.7" does not define how data is being sent, only a small subset of it. You need to see what comes out of the battery monitor in total before you can start taking it apart, programmatically.

Research (translation - Google) what Mike has said about software serial on the Uno and what I said about a USB to serial converter. Those are two of the key pieces you need in order to do what you've asked.

Here's a good assignment for you - draw a schematic of how you connect the battery monitor, the UNO and your PC together. Nothing fancy, paper and pencil will do. Then post a photo or a scan of what you've drawn.

I suggest you try the second example in Serial Input Basics with a small change. The original uses the newline character '\n' as the end-marker. If you change that to a comma I think it will receive and display your data.

Also modify it so it uses your SoftwareSerial instance rather the Serial - assuming you are following @Grumpy_MIke's advice.

...R

Wow, You all have given me much to chew on, Choke, Choke. I'll get back when I've had a chance to digest a little, maybe tomorrow, or the next. I'll also include a drawing. Thank you all so much.

@fred
I thought th bit that said

(+5V) is "mark" or "idle" state. "Start bit" (logic 0), is about 0 volts.)"

Along with the serial data format told you the signal was an RS232 signal sent at TTL levels.
Not sure what more you need to know.

@avr_Fred, Robin2, Grumpy_Mike

Here's a good assignment for you - draw a schematic of how you connect the battery monitor, the UNO and your PC together. Nothing fancy, paper and pencil will do. Then post a photo or a scan of what you've drawn.

I'm not a programmer but I did go through the exercises in the links provided and set up many of the examples in the tutorials. There is much to be confused about. I'm not sure of how to modify a sketch that I see in an example to suit what I'm trying to do.
One particular confusion point is in many of the built in examples, such as Digital Read Serial, the input pin is defined like:

int pushButton = 2;

and so since this sketch uses pin 2 as an input I used that sketch to experiment with.
In Serial Input Basics, that uses the keyboard as input, so no pins are involved, so no example for me to use.
Robin2 said,

Also modify it so it uses your SoftwareSerial instance rather the Serial - assuming you are following @Grumpy_MIke's advice.

I'm confused by this as well. since there is no defined pin for the input, like Digital Read Serial. You suggested that I use the Example 2, except modify it to recognize a comma, that's easy. But what pin do I put my signal into? I see no pin identified as an Input pin.
Grumpy_Mike said:

I said send to your computer at 9600 baud and read from software serial at 2400 baud.
You need to write out faster than you read in so that your input buffer doesn't overflow.

I know how to set the Arduino Uno to read at 2400 using:

Serial.begin(2400);

but I don't know how to set up the coding to send to my computer at 9600 baud. Every time id set the Serial Monitor to anything other than 2400 baud, I got gibberish. This pin connecting issue is hanging me up.

From the "Serial" reference page:

Serial is used for communication between the Arduino board and a computer or other devices. All Arduino boards have at least one serial port (also known as a UART or USART): Serial. It communicates on digital pins 0 (RX) and 1 (TX) as well as with the computer via USB. Thus, if you use these functions, you cannot also use pins 0 and 1 for digital input or output.

So perhaps part of your confusion is that when you use the Uno's native serial port on digital pins 0 and 1, you are only connecting to your battery monitor, there is no provision to attach your PC. This is why Mike said you need the "software serial" library which would be used to create the second serial port. It would be easiest to use the existing USB connection to your PC, outputting to the monitor window in the Arduino IDE and use software serial for the 2400 baud connection to your battery monitor.

BTW, still no attachments show in your previous posts so still not able to further explain since I don't know how things are connected.

I'm having trouble attaching a 133kb jpeg.

Have you read this? Embedding images in a posting

But what pin do I put my signal into? I see no pin identified as an Input pin.

Any pin on an Arduino can be used as an input pin. With software serial YOU define what pins you want to use.

but I don't know how to set up the coding to send to my computer at 9600 baud.

You use:-

Serial.begin(9600);

You set the SOFTWARE serial rate at 2400 because is that is how you communicate with your device.


1024×745 27.9 KB

avr_fred, I so appreciate your patience with me. I finally got the scan of my drawing uploaded. You said:

"It communicates on digital pins 0 (RX) and 1 (TX) as well as with the computer via USB. Thus, if you use these functions, you cannot also use pins 0 and 1 for digital input or output."

I read that and remember reading that before, but I guess I would have thought that the pin number would have to be called out programatically like they did in 'Digital Read Serial'. It was because of this confusion that I chose the very simplistic sketch that had its input pin identified. I also remember reading that if you use the pin 0 and pin 1, that there is an issue with using the USB connection to the Computer. I need to use USB and minimize issues.

I wanted to make sure I could see that I was receiving a signal, and by using the 'Digital Read Serial', I was able to replace the momentary switch with my bread-boarded input circuit and receive through pin 2, the signal from the battery monitor.

So I guess that actually the 'Digital Read Serial' is an application of "Software Serial", since on that sketch it does identify pin 2 as the Input, and that's how I've had it set up from the beginning. Since that sketch only receives one bit at a time and prints it with a line feed:

Serial.println(buttonState);

What I got is a single column of binary bits. Now to make sense of what I have received and find out how to break it down and deal with it. And yea, Mike called it rubbish, but it helped me see that I was actually able to receive a signal.

So now I'm trying to break it down, and that is where I think that is some software already written to do this, or a slight modification of existing sketch. Can you advise further?

@Grumpy_Mike, as you have probably read above, I think that what I said about why I used the 'Digital Read Serial' sketch hopefully explains where I'm at. You said:

"You set the SOFTWARE serial rate at 2400 because is that is how you communicate with your device."

That is what is in the sketch. I am still at a bit of loss about the difference between setting the communication speed that the Uno reads from the TM2030 to 2400 baud, which I set in the sketch to 2400 baud and in the same sketch tell the Uno to send the data at 9600 baud from the Uno to the computer. If I have the baud rate set in the void setup section, I don't know where the

Serial.begin(9600);, would go.

// digital pin 2 has a input circuit attached to it.
int pushButton = 2;

// the setup routine runs once when you press reset:
void setup() {
  // initialize serial communication at 2400 bits per second:
  Serial.begin(2400);
  // make the pin 2 an input:
  pinMode(pushButton, INPUT);
}

// the loop routine runs over and over again forever:
void loop() {
  // read the input pin:
  int buttonState = digitalRead(pushButton);
  // print out the state of the button:
  Serial.println(buttonState);
  delay(1);        // delay in between reads for stability
}

So now, I hope I am clear, I'm receiving a signal, but its not in the form that I can read, or make sense of. I'm not sure if I'm correct but maybe I can try to modify the example 2 that Robin2 suggested to a Software Serial application, but I sure would like a little guidance on how to do that.

http://forum.arduino.cc/index.php?topic=396450.0

What I got is a single column of binary bits. Now to make sense of what I have received and find out how to break it down and deal with it. And yea, Mike called it rubbish, but it helped me see that I was actually able to receive a signal.

No it didn't. You made a mistake. The mistake was not changing the speed of the serial monitor to match the speed the Arduino was sending. You will get the same display as you did at the magic 2400, with any serial baud rate IF you set the serial monitor to the same baud rate as you send stuff. You are deluding yourself if you think that test told you anything except the baud rates were mismatched. It certainly told you nothing about the the data arriving at pin 2.

OK so that is your diagram. I don't know why you have a diode in the circuit, it does nothing except make the signal more prone to picking up interference.

I am still at a bit of loss about the difference between setting the communication speed that the Uno reads from the TM2030 to 2400 baud, which I set in the sketch to 2400 baud and in the same sketch tell the Uno to send the data at 9600 baud from the Uno to the computer.

Software serial is a libiary, it allows you to communicate serial data over a pair of normal pins.
You would use it like this:-

#include <SoftwareSerial.h>
SoftwareSerial myVoltmeter(2, 3); // RX, TX define the pins to use
void setup(){
Serial.begin(9600); // speed to talk to the computer
myVoltmeter.begin(2400); // speed to read the voltmeter
// other setup stuff here
}

void loop(){
  if (myVoltmeter.available() > 0) {
    char a = myVoltmeter.read();
   Serial.write(a);
  }
}

Now set the serial monitor to 9600 baud and run that code, if ASCII data is being sent at 2400 baud from your voltmeter you will see it on the serial monitor.

@Grumpy_Mike,
Thank you for your response. I pasted your code into the Uno ide and uploaded it. I then started the Serial Monitor and set the baud rate to 9600 in the Serial Monitor and saw that I was receiving text and numbers across the top line. It is moving too fast for me to make out what it is saying, but this is very encouraging.

So now my attention will turn to seeing what I can do to modify your code to read a run of the message, (since it is a repeating message that is updated every 2.5 minutes), then give it a carriage return to repeat the sequence.

Instead of using the Serial Monitor to view what is returned, I used 'Cool Term', which has an automatic word wrap feature, and a 'Connect' and 'Disconnect' buttons. With it, I was able to Connect to fill a screen with data, then click on Disconnect and freeze the screen to what had been printed. Here is a sample:

V=13.3,FV=13.3,V2=00.0,A=00.8,FA=00.5,PW=100,AH=0.01,%=100,W=10.8,DSC=0.04,DSE=5.71,PW=100,V=13.3,FV=13.3,V2=00.0,A=00.8,FA=00.5,PW=100,AH=0.01,%=100,W=10.6,DSC=0.04,DSE=5.71,PW=100,V=13.3,FV=13.3,V2=00.0,A=00.8,FA=00.5,PW=100,AH=0.01,%=100,W=10.4,DSC=0.04,DSE=5.71,PW=100,V=13.3,FV=13.3,V2=00.0,A=00.8,FA=00.5,PW=100,AH=0.01,%=100,W=9.97,DSC=0.04,DSE=5.71,PW=100,V=13.3,FV=13.3,V2=00.0,A=00.8,FA=00.5,PW=100,AH=0.01,%=100,W=9.84,DSC=0.04,DSE=5.71,PW=100,V=13.3,FV=13.3,V2=00.0,A=00.7,FA=00.5,PW=100,AH=0.01,%=100,W=9.97,DSC=0.04,DSE=5.71,PW=100,V=13.3,FV=13.3,V2=00.0,A=00.8,FA=00.5,PW=100,AH=0.01,%=100,W=9.84,DSC=0.04,DSE=5.71,PW=100,V=13.3,FV=13.3,V2=00.0,A=00.7,FA=00.5,PW=100,AH=0.01,%=100,W=9.57,DSC=0.04,DSE=5.71,PW=100,V=13.3,FV=13.3,V2=00.0,A=00.7,FA=00.5,PW=100,AH=0.01,%=100,W=2.90,DSC=0.04,DS

I previewed this and see that the word wrap does not carry over into this window.

Mike, I want you to know that this has made my month. Thank you. Steve

Try replacing the loop function with this:-

int count = 0;
void loop(){
  if (myVoltmeter.available() > 0) {
    char a = myVoltmeter.read();
   Serial.write(a);
   if(a ==",') count++;
   if(count >=12){
     count = 0;
    Serial.println();
     }
  }
}

It will send a new line every 12 commas, which if my counting is correct is the number you get on one line of data.