[SOLVED] Serial data scrambled, then stutters out

I've been trying to get this to work for so long it's ridiculous. I have an Arduino Uno hooked up to my Pi via Serial. At first I tried USB, but then hardwire serial. I'm using a simple voltage divider with 220? on the Arduino side and 1K? on the other going to ground. It gives about 3.1v DC that the Pi accepts fine. ~~I don't want to have to say anything for a third time, so please read both Recieved Data is Scrambled and Not Correct · Issue #297 · serialport/node-serialport · GitHub and Serial data scrambled (solved) - Raspberry Pi Forums

Example of data:

{"data":{"temp":68.00000,"hum":0.00000,"press":35.65534,"wd":0.00000,"ws":0.00000,"light":0.00000},"status":1}
{"data":{"temp":67.66250,"hum":0.00000,"press":35.65207,"wd":0.00000,"ws":0.00000,"light":0.00000},"status":1}
{"data":{"temp":67.88750,"hum":0.00000,"press":35.65749,"wd":0.00000,"ws":0.00000,"light":0.00000},"status":1}
{"data":{"temp":68.00000,"hum":0.00000,"press":35.65800,"wd":0.00000,"ws":0.00000,"light":0.00000},"status":1}
{"data":{"temp":67.88750,"hum":0.00000,"press":35.65688,"wd":0.00000,"ws":0.00000,"light":0.00000},"status":1}
{"data":{"temp":68.00000,"hum":0.00000,"press":35.65836,"wd":0.00000,"ws":0.00000,"light":0.00000},"status":1}
{"data":{"temp":67.88750,"hum":0.00000,"press":35.65681,"wd":0.00000,"":0.00000},"":1}

It screws up the last one and then prints blank lines. When I tried this on my Mega R3, it would get a little farther and really scramble and shorten the last line.

I've come to the conslusion that it's the Arduino. I think it has to do with my I2C Pressure Sensor (SparkFun Altitude/Pressure Sensor Breakout - MPL3115A2 - SEN-11084 - SparkFun Electronics). I think that it cannot provide the information fast enough, even though it says it has the data. I tried it at 9600 baud, and it still didn't work. However, that's as low as I'm willing to go, as this is a real-time application that requires data as the sensor sees it.

My sketch:

#include <Wire.h>
#include <Math.h>
#include <aJSON.h>
#include "MPL3115A2.h"

aJsonStream serial_stream(&Serial);

//Create an instance of the object
MPL3115A2 myPressure;

int oldTemperature;
float oldPressure;
float oldHumidity;
int oldCompareBaroin;
float oldWindDirection;
float oldWindSpeed;
float oldAmbientLight;

void setup() {
  Wire.begin();        // Join i2c bus
  Serial.begin(115200);  // Start serial for output
  
  myPressure.begin(); // Get sensor online
  //Configure the sensor
  //myPressure.setModeAltimeter(); // Measure altitude above sea level in meters
  myPressure.setModeBarometer(); // Measure pressure in Pascals from 20 to 110 kPa
  
  myPressure.setOversampleRate(7); // Set Oversample to the recommended 128
  myPressure.enableEventFlags(); // Enable all three pressure and temp event flags 
}

void loop() {
  float temperature = myPressure.readTempF();
  float humidity = 0;
  float pressure = myPressure.readPressure();
  float windDirection = 0;
  float windSpeed = 0;
  float ambientLight = 0;
  
  
  //References: 
  //Definition of "altimeter setting": http://www.crh.noaa.gov/bou/awebphp/definitions_pressure.php
  //Altimeter setting: http://www.srh.noaa.gov/epz/?n=wxcalc_altimetersetting
  //Altimeter setting: http://www.srh.noaa.gov/images/epz/wxcalc/altimeterSetting.pdf
  //Verified against Boulder, CO readings: http://www.crh.noaa.gov/bou/include/webpres.php?product=webpres.txt
  
  //const int station_elevation_ft = 5374; //Must be obtained with a GPS unit
  //float station_elevation_m = station_elevation_ft * 0.3048; //I'm going to hard code this
  const int station_elevation_m = 1638; //Accurate for the roof on my house
  //1 pascal = 0.01 millibars
  pressure /= 100; //pressure is now in millibars

  float part1 = pressure - 0.3; //Part 1 of formula
  
  const float part2 = 8.42288 / 100000.0;
  float part3 = pow((pressure - 0.3), 0.190284);
  float part4 = (float)station_elevation_m / part3;
  float part5 = (1.0 + (part2 * part4));
  float part6 = pow(part5, (1.0/0.190284));
  float altimeter_setting_pressure_mb = part1 * part6; //Output is now in adjusted millibars
  float baroin = altimeter_setting_pressure_mb * 0.02953;
  
  
  float compareTemperatureFloat = temperature * 1000;
  int compareTemperature = round(compareTemperatureFloat);
  
  float compareBaroinFloat = baroin * 100;
  int compareBaroin = round(compareBaroinFloat);
  
  if (compareTemperature != oldTemperature) {
	sendSerial(temperature, humidity, baroin, windDirection, windSpeed, ambientLight);
  }
  // else if (humidity != oldHumidity) {
	// sendSerial(temperature, humidity, baroin, windDirection, windSpeed, ambientLight);
  // }
  else if (compareBaroin != oldCompareBaroin) {
	sendSerial(temperature, humidity, baroin, windDirection, windSpeed, ambientLight);
  }
  // else if (windDirection != oldWindDirection) {
	// sendSerial(temperature, humidity, baroin, windDirection, windSpeed, ambientLight);
  // }
  // else if (windSpeed != oldWindSpeed) {
	// sendSerial(temperature, humidity, baroin, windDirection, windSpeed, ambientLight);
  // }
  // else if (ambientLight != oldAmbientLight) {
	// sendSerial(temperature, humidity, baroin, windDirection, windSpeed, ambientLight);
  // }
  
  oldTemperature = compareTemperature;
  oldHumidity = humidity;
  oldCompareBaroin = compareBaroin;
  oldWindDirection = windDirection;
  oldWindSpeed = windSpeed;
  oldAmbientLight = ambientLight;
}

void sendSerial(float temp, float hum, float press, float wd, float ws, float light) {
  aJsonObject *root,*data;
  root = aJson.createObject();  
  aJson.addItemToObject(root, "data", data = aJson.createObject());
    aJson.addNumberToObject(data, "temp", temp);
    aJson.addNumberToObject(data, "hum", hum);
    aJson.addNumberToObject(data, "press", press);
    aJson.addNumberToObject (data, "wd", wd);
    aJson.addNumberToObject(data, "ws", ws);
    aJson.addNumberToObject(data, "light", light);
   aJson.addItemToObject(root, "status", aJson.createItem(1));

  // Serial.print("\"data\": { \"temp\": ");
    // Serial.print(temp, 4);
    
    // Serial.print(", \"hum\": ");
    // Serial.print(hum);
    
    // Serial.print(", \"press\": ");
    // Serial.print(press,4);
    
    // Serial.print(", \"wd\": ");
    // Serial.print(wd);
    
    // Serial.print(", \"ws\": ");
    // Serial.print(ws);
    
    // Serial.print(", \"light\": ");
  // Serial.print(light);
  // Serial.print("}");
  aJson.print(root, &serial_stream);
  Serial.print("\n");
}

You'll see that I'm using a library to help with the JSON. I tried this to fix the problem of scrambled data, but it didn't help much. My previous attempt is commented out below.

The libraries:

T

I don't want to have to say anything for a third time, so please read both Recieved Data is Scrambled and Not Correct · Issue #297 · serialport/node-serialport · GitHub and http://www.raspberrypi.org/forums/viewtopic.php?f=63&t=69766&p=516673.

You haven't written that much there so I don't think we have to search all the information on several pages, it's you who have to provide that.

I have an Arduino Uno hooked up to my Pi via Serial.

How did you do that? Schematics? Wiring diagram?

At first I tried USB, but then hardwire serial.

You had the same problem with a USB connection? BTW, you haven't shown the characters that you get. Please also supply the byte you expected.

I think it has to do with my I2C Pressure Sensor

Do you have any reason to think that?

Have you tried using a lower baud rate?

The library you're using is not very high quality. For example:

	Wire.requestFrom(MPL3115A2_ADDRESS, 3); // Request three bytes

	//Wait for data to become available
	counter = 0;
	while(Wire.available() < 3)
	{
		if(counter++ > 100) return(-999); //Error out
		delay(1);
	}

Instead of checking the return code of Wire.requestFrom() the author waits for more than 100ms if anything went wrong in I2C communication. That wait is completely unnecessary because if the data wasn't available after the Wire.requestFrom() call it never will (the Wire library is not asynchronous).
So by using this library and with such high baud rates it may be possible that you loose data on the serial interface.

root = aJson.createObject();

...calls malloc.

aJson.addItemToObject()

...calls malloc.

aJson.createItem(1)

...calls malloc.

And then you abandon root. The problem is simple. You are running out of heap space. At which point your code starts stomping around at "random" locations in memory until the state is so corrupt that unexpected things happen.

pylon, I'm sorry about not giving any information. I'm just really tired of this problem. First I thought it was the Serial library I was using in NodeJS, then the Pi, and now this. I edited my original post to add everything, so please read it again. I think you are right about the library! I find it kind of funny that Sparkfun made this error, but can you provide an example on how to fix it? I would be happy to submit a Pull Request too to contribute to their GitHub project.

Coding Badly, I never thought about clearing it! This could very well be another problem. Is the following correct?
Original:

aJson.print(root, &serial_stream);
Serial.print("\n");

Changed to:

aJson.print(root, &serial_stream);
Serial.print("\n");
free(root);
aJson.deleteItem(root);

Thank you both for the quick replies and helping me with this! I've been at this for several months now.

Change the posted part to

  if (Wire.requestFrom(MPL3115A2_ADDRESS, 3) != 3) {
    return -999;
  }

Although if this happens often you probably have a wiring problem. What length of wires do you have between the Arduino and the sensor? What pullups are you using?

I changed it to this:

//Wait for data to become available
counter = 0;
//while(Wire.available() < 3)
//{
//	if(counter++ > 100) return(-999); //Error out
//	delay(1);
//}
if (Wire.requestFrom(MPL3115A2_ADDRESS, 3) != 3) {
	return -999;
}

But, it didn't change anything. I'm using whatever the built-in pullups are (schematic here: https://dlnmh9ip6v2uc.cloudfront.net/datasheets/Sensors/Pressure/MPL3115A2_breakout.pdf). I'm not using INT1 or INT2. The VCC is hooked up to 3.3v. I'm using the SDA and SCL behind the AREF port. The wires are about 2 inches each, but I'm going to eventually use some CAT6 to get it as far as I can (5 feet?).

free(root);
aJson.deleteItem(root);

Freeing memory to the heap more than once is never correct.

Given the fact that you are going to output the same thing every time and JSON consists of a prefix, a list of tag / value pairs, and a postfix I am having a difficult time understanding why you are bothering with a library. Just use Serial.print[ln].

Serial.print( F("{\"data\":{\"temp\":") );
Serial.print( temp );
Serial.print( F(";\"hum\":") );
Serial.print( hum );
Serial.print( F(";\"press\":") );
...

You might need some pullups, start with 2k7, deactivate the internal pullups and use external ones to 3V3 if the slave device is using that voltage.
I2C busses should not be longer than about half a meter (less than 2 feet) without using bus extenders.

After three months, IT WORKS! Thank you both so much. I started using aJSON to try to fix the problem that I think was in the sensor's library. Both fixes were key in making this work.

If you don't mind pylon, I'm going to make a Pull Request in the GitHub project to fix the error.

My current sketch:

#include <Wire.h>
#include <Math.h>
#include "MPL3115A2.h"

//Create an instance of the object
MPL3115A2 myPressure;

int oldTemperature;
float oldPressure;
float oldHumidity;
int oldCompareBaroin;
float oldWindDirection;
float oldWindSpeed;
float oldAmbientLight;

void setup() {
  Wire.begin();        // Join i2c bus
  Serial.begin(115200);  // Start serial for output
  
  myPressure.begin(); // Get sensor online
  //Configure the sensor
  //myPressure.setModeAltimeter(); // Measure altitude above sea level in meters
  myPressure.setModeBarometer(); // Measure pressure in Pascals from 20 to 110 kPa
  
  myPressure.setOversampleRate(7); // Set Oversample to the recommended 128
  myPressure.enableEventFlags(); // Enable all three pressure and temp event flags 
}

void loop() {
  float temperature = myPressure.readTempF();
  float humidity = 0;
  float pressure = myPressure.readPressure();
  float windDirection = 0;
  float windSpeed = 0;
  float ambientLight = 0;
  
  
  //References: 
  //Definition of "altimeter setting": http://www.crh.noaa.gov/bou/awebphp/definitions_pressure.php
  //Altimeter setting: http://www.srh.noaa.gov/epz/?n=wxcalc_altimetersetting
  //Altimeter setting: http://www.srh.noaa.gov/images/epz/wxcalc/altimeterSetting.pdf
  //Verified against Boulder, CO readings: http://www.crh.noaa.gov/bou/include/webpres.php?product=webpres.txt
  
  //const int station_elevation_ft = 5374; //Must be obtained with a GPS unit
  //float station_elevation_m = station_elevation_ft * 0.3048; //I'm going to hard code this
  const int station_elevation_m = 1638; //Accurate for the roof on my house
  //1 pascal = 0.01 millibars
  pressure /= 100; //pressure is now in millibars

  float part1 = pressure - 0.3; //Part 1 of formula
  
  const float part2 = 8.42288 / 100000.0;
  float part3 = pow((pressure - 0.3), 0.190284);
  float part4 = (float)station_elevation_m / part3;
  float part5 = (1.0 + (part2 * part4));
  float part6 = pow(part5, (1.0/0.190284));
  float altimeter_setting_pressure_mb = part1 * part6; //Output is now in adjusted millibars
  float baroin = altimeter_setting_pressure_mb * 0.02953;
  
  
  float compareTemperatureFloat = temperature * 1000;
  int compareTemperature = round(compareTemperatureFloat);
  
  float compareBaroinFloat = baroin * 100;
  int compareBaroin = round(compareBaroinFloat);
  
  if (compareTemperature != oldTemperature) {
	sendSerial(temperature, humidity, baroin, windDirection, windSpeed, ambientLight);
  }
  // else if (humidity != oldHumidity) {
	// sendSerial(temperature, humidity, baroin, windDirection, windSpeed, ambientLight);
  // }
  else if (compareBaroin != oldCompareBaroin) {
	sendSerial(temperature, humidity, baroin, windDirection, windSpeed, ambientLight);
  }
  // else if (windDirection != oldWindDirection) {
	// sendSerial(temperature, humidity, baroin, windDirection, windSpeed, ambientLight);
  // }
  // else if (windSpeed != oldWindSpeed) {
	// sendSerial(temperature, humidity, baroin, windDirection, windSpeed, ambientLight);
  // }
  // else if (ambientLight != oldAmbientLight) {
	// sendSerial(temperature, humidity, baroin, windDirection, windSpeed, ambientLight);
  // }
  
  oldTemperature = compareTemperature;
  oldHumidity = humidity;
  oldCompareBaroin = compareBaroin;
  oldWindDirection = windDirection;
  oldWindSpeed = windSpeed;
  oldAmbientLight = ambientLight;
}

void sendSerial(float temp, float hum, float press, float wd, float ws, float light) {
  Serial.print( F("{ \"data\": { \"temp\": ") );
  Serial.print( temp );
  Serial.print( F(", \"hum\": ") );
  Serial.print( hum );
  Serial.print( F(", \"press\": ") );
  Serial.print( press );
  Serial.print( F(", \"wd\": ") );
  Serial.print( wd );
  Serial.print( F(", \"ws\": ") );
  Serial.print( ws );
  Serial.print( F(", \"light\": ") );
  Serial.print( light );
  Serial.print( F(" }, \"status\": 1 }") );
  Serial.print("\n");
}

It's also good to note about only going 2 feet. This is going to be in a weather station, but I want to keep the Arduino and Pi indoors for easy management and control. Is it possible I could make my own bus extender? How far would it go while using one?

pylon, is there anything else wrong with the pressure sensor library? Lines 59-64 look to be doing something similar. Same with 108-113, 122-127, 154-159, and 168-173. I was going to make a Pull Request on GitHub myself, but I think there are other things that might need to be fixed too. If you provide a fully redone CPP, I'll upload that for you if you'd like. I just want to contribute to be sure that no one else has this problem.

GitHub project: GitHub - sparkfun/MPL3115A2_Breakout: Example code and board files for MPL3115A2 Breakout board.

In the attached version the wrong calling of the Wire library is fixed.

Maybe other things aren't that fine in the library but to check that I would have to dig more into the datasheet of the chip.

MPL3115A2.cpp (9.43 KB)

The Pull Request can be found here: Fix problems with not using Wire by DaAwesomeP · Pull Request #10 · sparkfun/MPL3115A2_Breakout · GitHub

Thanks pylon!