Hardware serial woes

Hi

I don't know whether this is in the correct section but I can't seem to find an appropriate place.

I built a project for giving me detailed electricity consumption/generation/Export and Import data based on a Pro Mini 16Mhz 5v. It has the following functionality..

A) Reads data from the hardware serial port @ 57600 from a current cost 128 device (basically an xml stream of data). It decodes the data and aggregates the readings into Minute Hour and Day Totals ready to output to a raspberry PI (originally VIA Bluetooth) but now by direct serial connection.

B) Reads an Elster A100C Electricity Generation meter using a Library found here. http://www.rotwang.co.uk/projects/meter.html. Its seems like a 2400 baud serial port reader designed to work on the rising edge of IRDA data. It uses IRQ 0

C) A pulse counter to read My Import electricity meter. This uses IRQ 1 and counts pulses of less than 1 sec in width as W/Hrs also it identifies the meter is in reverse mode when the LED is full on.

D) Listen to my raspberry PI @ 57600 on software serial for a simple set of commands Txxxxxxxx set time Mxxxxxx set meter reading "+" advance Time by 1 second "-" retard time by 1 Second.

E) Output the current Time and Reading data to the Raspberry Pi at 1 minute intervals, Via hardware serial @ 57600. The Pi then stores the info in a Mysql database and I have a 1 page PHP() website on the pi to show me the data. The data comes out of the Arduino in the following format

Type='MIN'&Date='2013-05-18T00:12:00'&GenTot=3876956&GenCnt=0&ImpTot=642&ImpCnt=6.......................

If the PI detects the clock has drifted more than a couple of seconds then the PI sends A + or - character and the Arduino adjusts its clock. I was getting horrible Time drifts before I did this 2 mins per day.

The Pi's Hardware serial Port is connected to the Arduino Hardware TX and Software RX via a suitable bidirectional Level shifter.

Everything Is working fine I added a MEMORY checker to the sketch and there is 800+ bytes ram free.

Now down to the Problem ....

After a day or so the output from the hardware serial port turns into Gibberish strange foreign "A" characters etc just as if I was at the Wrong baud rate. The hardware serial also stops receiving I can tell this because the Led on pin 13 blinks every 6 seconds if there is a sucessfull read, and this stops. I can still send commands to the Arduino from the PI through the software serial. If I send M00000000 to the Arduino It resets the meter counter and sends back an acknowledge which is corrupt but It only acknowledges valid commands. I reduced the occurrence of the problem by adding the following code..

void loop() // run over and over and over and over
{

    if (minute()!=CurrentMin) {
        // dont reset serial whilst data is incomming
        if (!ReadingData)
        {
            // reset hardware serial buffer
            //no need to flush data is only sent by NewMinuite();
            Serial.end();
            Serial.begin(57600);
            Serial.println("#Reset Serial");    
        }

        // reset second serial buffer
        SecondSerial.end();
        SecondSerial.begin(57600);

        NewMinute();

    }

    ....................
}

This resets the hardware serial port every minute just before it spews the data to the PI. To stop it corrupting the xml stream from the Hardware serial port I have a bool ReadingData which gets set as soon as data starts to come into the port and gets reset when it gets to the end of the data. This reduced the crashing from around once per day, to its current condition which has taken 4+ days to crash. I can only assume this latest crash happened whilst an XML packet was being read thus ReadingData was set it never got reset because the stream got corrupt. I knew this was a possibility but I thought resetting the serial port every Minute would stop it getting corupted in the first place.

I am pretty sure its a firmware/arduino problem as :-

1) I have tried two different pro minis from two different sources. which sort of eliminates the hardware. 2)The PI is powered from the same Stepdown Voltage regulator as the Arduino, and it does not crash so I think this eliminates the PSU. The PI is more susceptible to bad power than the Arduino so If I had bad power the PI would have crashed. 3) The program continues to run even after the serial port has crashed so the Pro Mini hasn't Died its just the serial port that is corrupt.

What I intend to do next is get the Pro Mini to reset the serial Port (every minute AND when its not receiving data) OR ( Not Successfully Received data for 30 seconds) Hopefully this will fix it, the CC128 is powered by my project so I can also power this off and on If no data is received for 30 seconds sometimes the CC128 can lose connection with its wireless senders a power off and on fixes this.

I am also going to add a couple of 1000uF caps To the 5V rail to eliminate Power outs.

Is there a known issue with running the Hardware serial port Continuously @ 57600 ?

PS This is the sort of data Info I am getting with this combo. http://86.171.115.160:999

Can you make a minimal sketch that shows the problem?

robtillaart: Can you make a minimal sketch that shows the problem?

Yes I had thought of that. But I think the problem in the Hardware serial may be a combined effect of reading from effectively two software serials and me using another interrupt and using the Hardware serial as well.

I Could do a simple sketch that just echo's back data from the hardware serial like a loop back and the same for software serial.

For now I have modded the software to reset the serial ports if they don't receive data for 40 seconds. The other fix I had thought of was If the pi starts receiving gibberish it could send a RESET command via the software serial to the pro mini. I would then write a routine that saves the current data to the EEPROM and reboots itself. The setup program would retrieve all the data from the eeprom and continue that way i may loose a minutes read but would still have a pretty accurate daily and hourly total. To reset it I thought a 100nf cap between the Reset and an Digital IO pin would do pull the pin low the chip resets and hopefully will come back OC I will hake sure the first thing that Setup does is set it high.

ffimon: Everything Is working fine I added a MEMORY checker to the sketch and there is 800+ bytes ram free.

Since the microcontroller used doesn't have a memory controller, these sketches are estimates at best.

ffimon: I reduced the occurrence of the problem by adding the following code..

Post the rest of the code.

Why do you keep calling Serial.end() followed immediately by Serial.begin()? That seems completely pointless and creates the very real possibility data is coming in that you are going to miss, resulting in junk eventually.

ffimon: I am pretty sure its a firmware/arduino problem as :

It's almost definitely a code problem, so post all of your code. There is no "firmware" running on the Arduino after the bootloader jumps to the code vector.

ffimon: 2)The PI is powered from the same Stepdown Voltage regulator as the Arduino, and it does not crash so I think this eliminates the PSU.

The Pi is a completely different system, so I fail to see the connection. Also you haven't verified the Arduino is "crashing." Again, it is probably problem with your code not the hardware.

ffimon: 3) The program continues to run even after the serial port has crashed

The serial port cannot "crash."

ffimon: I am also going to add a couple of 1000uF caps To the 5V rail to eliminate Power outs.

That's great for low frequency ripple. 1.0uF and 0.1uF will help with high frequency.

ffimon: Is there a known issue with running the Hardware serial port Continuously @ 57600 ?

No, not at all. Plenty of people do this without any issue.

It does not do it “Continuously” Just once per minute, before it sends the mins readings, this Seems to reset the serial port and fixed the corruption. It checks to see if there is an incoming stream before resetting it.

Sorry Firmware was the wrong terminology, the Program is definitely not crashing I can still communicate with the program VIA software serial and get responses that are corrupted via Hardware serial. I suspect its not the buffer that is being corrupted but the port setting or the drivers variables.

3) The program continues to run even after the serial port has crashed The serial port cannot "crash."

A serial port can overflow on the receiving side ...

OK In order to eliminate from the equation the only piece of code I did not write the A100c Meter reader Library I wrote my own.

I took a look at the Library and it uses 100s of bytes to read the meter. It also requires that you service it at regular intervals and also It breaks completely if you have any delay() statements in the main loop.

/*

Elster A100c Meter Reader Software
Reads a 2400 baud stream Stream Starts after MinimumInterBlockGap (uS) 
(c) No one Please feel free to do with as you please. 

*/
#define PulseGap 390  // Approximate Pulse Gap its actually 416uS But 390 allows a little margin for error
#define ReadByteLength 54 // The Number of Bytes to read from the stream we only need to read upto the end of the 
#define MeterReadingDataStart 49 // The Starting Position of the BCD Meter Data
#define MeterReadingDataEnds 53 // The Ending Position of the BCD Meter Data
#define MinimumInterBlockGap 100000 // The Gap Between Frames In (uS)

volatile unsigned long LastPulse; // The micros() of the last Pulse
volatile unsigned long ThisPulseMicrosTemp; // The micros() of the this Pulse
volatile unsigned long MicrosSinceLastPulse; // Gap in uS Between This and the Last Pulse
volatile unsigned long NEWreading_A100C_Reading_1; // Varible To Convert the BCD Reading into 
volatile byte BuffPtr; // Current Character Position in Stream
volatile byte ReadByte; // Current Character 
volatile byte BitPos; // The bit position of the current Pulse 
volatile bool FirstRead; // Flag showing this as the First read I ignore the first meter reading as it seems to be corrupt 1 in 10 times 
//Following reads all seem fine. 


void HandlePulse()
{
	// Recieved an IR Pulse. Let the Games Commence
	// Grab micros soon as the pulse arrives to give the most accurate Inter pulse Gap
	ThisPulseMicrosTemp=micros();
	MicrosSinceLastPulse=(ThisPulseMicrosTemp-LastPulse);
	if (MicrosSinceLastPulse>MinimumInterBlockGap) {
		// Its a Big Gap ... The Frame has just begun
		BuffPtr=0;BitPos=0;ReadByte=255;NEWreading_A100C_Reading_1=0;
	}
	else
	{
		// Process Each bit if we haven't Passed the end of where we want to read to. 
		if (BuffPtr<ReadByteLength)
		{
			// Calculate the number of bits forward the next "0" pulse is 
			BitPos+=(MicrosSinceLastPulse/PulseGap);
			if (BitPos > 8){

				// If the next "0" pulse is after the end of the byte 
				// IE the next start BIT then the current Byte next valid character in the stream

					if (!(BuffPtr<MeterReadingDataStart) && !(BuffPtr>MeterReadingDataEnds))
					{
						// This character is part of the meter reading 
						// Convert the BCD to Unsigned Long
						NEWreading_A100C_Reading_1=(NEWreading_A100C_Reading_1*100);
						NEWreading_A100C_Reading_1+=(ReadByte /16)*10+(ReadByte % 16);
						if ((BuffPtr==MeterReadingDataEnds)) // Its the Last Character Call the Reading Complete with the results 
							if(FirstRead) FirstRead=false;  // If its not the first reading then ignore it.
								else ReadingComplete(NEWreading_A100C_Reading_1);					
					}
					BuffPtr++;ReadByte=255;BitPos=0;
			}
			else
			{
				// Unset this bit as it has a pulse in it and Pulses are 0's
				switch (BitPos){
					case 1:
						ReadByte -=1;break;
					case 2:
						ReadByte -=2;break;
					case 3:
						ReadByte -=4;break;
					case 4:
						ReadByte-=8;break;
					case 5:
						ReadByte-=16;break;
					case 6:
						ReadByte-=32;break;
					case 7:
						ReadByte-=64;break;
					case 8:
						ReadByte-=128;break;
				}
			}
		}
	}
	// Finally Don't Forget to set the last pulse time to this pulse time so we can calculate the next gap.
	LastPulse=ThisPulseMicrosTemp;
}

void StartA100cMeterReader(byte IRQ){
	// This Starts the meter Reader
	FirstRead = true;
	LastPulse=micros();
	attachInterrupt(IRQ,HandlePulse,RISING);
}

void ReadingComplete(unsigned long Reading)
{
	// This is called everytime there is a succesfull meter reading usually around once per second
	// PS Calling Serial.print() in here is probably not a good thing as it is called from an ISR.

	Serial.println(Reading);
}

void setup()
{

  /* add setup code here */

	Serial.begin(57600);
	delay(1000);
	Serial.println("starting");
	StartA100cMeterReader(0);

}

void loop()
{

  /* add main program code here */

}

Here is my simpler solution which doesn’t care less what happens in the main loop, and doesn’t require you call it every mS or so to service it.

My only question relates to the volatile variables I made all the variables volatile because they are in an ISR, just for testing. I don’t think this is needed as they are only used within the routine by the routine itself, which should never be called again (before it completes) whilst its running. As their is 400uS between pulses which is plenty of time to get the job done before the next pulse. I think they could all not be volatile, if so what difference will that make to performance / safety with other ISR’s such as software serial RTC etc.

PS I have integrated the above into my original sketch. The code size has has reduced from 16026 to 13368 and the free memory has jumped from 660 Bytes to 1319 Bytes at the deepest nesting within the program. Cant believe that library used 600+ bytes of ram to read 1 unsigned long. Now just to wait and see if the Pro Mini gets past the 4.5 day mark without a crash.