Pages: [1] 2   Go Down
Author Topic: Advanced programmers advice needed.  (Read 740 times)
0 Members and 1 Guest are viewing this topic.
Belgium
Offline Offline
Edison Member
*
Karma: 68
Posts: 1917
Arduino rocks; but with my plugin it can fly rocking the world ;-)
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi
I have a weird problem with my arduino Yun where I can not get my head around and I would like some input from the smart guys here, to find my problem.
My problem is with a sketch for the arduino Yun (which is a leonardo). The sketch is work in progress and all parts have worked fine.
But I seem to get problems at different locations, then they go away; the sketch works fine for a while and then a problem appears somewhere else.
The problem I currently face is at startup time. It looks like the setup method is not called when the problem appears.
I think so because the serial part (/dev/ttyACM0) is not visible on the linux side and I moved my LCD code to the beginning of the setup code and the LCD does not respond to resets of the leonardo.

I'll start the story at the virtual SerialBridgeCommunicator::setReceivedMessage
SerialBridgeCommunicator is derived from SerialCommunicator and adds "load" and "save" functionality using the bridge to my SerialCommunicator class. To be able to do so I overload the virtual SerialCommunicator::setReceivedMessage but this stopped working.
Using SerialCommunicator instead of SerialBridgeCommunicator works
removing the overloaded setReceivedMessage and it works. Putting the overloaded setReceivedMessage back in and it failed.
I worked around this by copying the code in SerialCommunicator that calls setReceivedMessage to SerialBridgeCommunicator.
Because the code in SerialCommunicator that calls setReceivedMessage is now dead the optimizer removes is and the whole solution is smaller in size  smiley-grin but less easy to maintain  smiley-confuse

Now setup is called but fails very early in the call Bridge.begin(); (this is my mini version of Bridge not the official bridge)
So I add some Serial.print statements to Bridge.begin() to find where it fails.
result setup is no longer called.
I removed the Serial.print statements and the setup works again and Bridge.begin() does not return  smiley-roll
Just like the setReceivedMessage this code worked and I can repeat the problem and I have no clue why.
looking at the compile output I see (when it is working)
Code:
Program:   30194 bytes (92.1% Full)
(.text + .data + .bootloader)

Data:       2037 bytes (79.6% Full)
(.data + .bss + .noinit)
And when it is not working I get
Code:
Program:   30624 bytes (93.5% Full)
(.text + .data + .bootloader)

Data:       2037 bytes (79.6% Full)
(.data + .bss + .noinit)
Before anyone screams the program is to big to fit. As this is the yun I can easily upload without bootloader. I have been running without bootloader on the yun for quite some time now.
I thought maybe size is the reason.
So I removed the print statements in Bridge.begin(); and added
Code:
Serial.println(F("Test 2222222222222222222222222222222222222222222222222222222222222"));
Serial.println(F("Test 2222222222222222222222222222222222222222222222222222222222222"));
Serial.println(F("Test 2222222222222222222222222222222222222222222222222222222222222"));
Serial.println(F("Test 2222222222222222222222222222222222222222222222222222222222222"));
Serial.println(F("Test 2222222222222222222222222222222222222222222222222222222222222"));
Serial.println(F("Test 22222222222222222"));
To make the program the same size and indeed setup is not called.

As I use my eclipse plugin I made a sketch with the same size in the arduino IDE (I had to change boards.txt to tell I removed the bootloader) and the problem did not occur.
In other words there is no general size problem.

Because the setup is not called I suspect the class constructors cause some memory overwrite. However I verified (my library classes) and I don't see anything that may look like "dangerous code". 
This is because in the constructor I copy data to a class variable which is used in the setup call I have in nearly all my classes.
the copy is a pointer copy or a very basic data type.
I also don't use dynamic memory allocation and I refuse to use String (that is part why I modded the bridge).

I use a lot of libraries but all are from good sources ( except mine that is  smiley-lol )
Here is the list (the ones with a * have been written by me; part of them are available at github)
  • AltSoftSerial
  • DataType*
  • gps_library*
  • I2CLiquidCrystal*
  • JanServo*
  • minibridge*
  • SerialBridgeCommunicator*
  • SerialCommunicator*
  • SerialDataInterface*
  • SerialStringReader*
  • Servo
  • Voltmeter*
  • Wire


Any ideas on how I can nail the problem are welcomed.
Best regards
Jantje
Logged

Do not PM me a question unless you are prepared to pay for consultancy.
Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -

Poole, Dorset, UK
Offline Offline
Edison Member
*
Karma: 51
Posts: 2276
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Most like your running out of ram post your code! (blowing the heap and or stack)

Mark
Logged

Belgium
Offline Offline
Edison Member
*
Karma: 68
Posts: 1917
Arduino rocks; but with my plugin it can fly rocking the world ;-)
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

@Holmes4
Thanks for your reply.
I have been suspecting the heap and the stack but I can't get my head around how that would be possible.
I don't use new,alloc,malloc,String so basically that would mean I'm not using the heap.

As to the stack.
Seen the fact I don't reach setup that would mean that the constructors +main() +setup() blow the heap/stack
In my code (that is all the code in the project except for the arduino core code) there is not a single constructor that calls a method (that is apart from a base constructor if it needs a parameter)
Apart from: I'm overlooking a case: How can a stack/heap blow happen under these circumstances before setup()?

As a side note to show I'm aware of the stack usage:
As strings take so much room I also declared a global variable to do temporary string methods.
Code:
char commonlyUsedBuffer[commonlyUsedBuffersize];
An then in the code
Code:
m_LastMessageTimeStamp.ToString(commonlyUsedBuffer,commonlyUsedBuffersize);
This should reduce stack usage.

Now I think about this. In the old days I could provide a stack and heap size. How does this happen on the avr?

Best regards
Jantje
Logged

Do not PM me a question unless you are prepared to pay for consultancy.
Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -

Global Moderator
UK
Offline Offline
Brattain Member
*****
Karma: 301
Posts: 26240
I don't think you connected the grounds, Dave.
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I'd look at the headers for the text orientated libraries and see what sort of size buffers they reserve.
Logged

"Pete, it's a fool looks for logic in the chambers of the human heart." Ulysses Everett McGill.
Do not send technical questions via personal messaging - they will be ignored.

Poole, Dorset, UK
Offline Offline
Edison Member
*
Karma: 51
Posts: 2276
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I still can't see any code!

Mark
Logged

Belgium
Offline Offline
Edison Member
*
Karma: 68
Posts: 1917
Arduino rocks; but with my plugin it can fly rocking the world ;-)
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

@AWOL
I verified the text oriented libraries. As I use 3 serial ports there are quite a lot of buffers. I even found an error saving 12 bytes  smiley-grin
But all these buffers are instantiated as global variables directly or indirectly. So they are counted for in the size statistics.
IMHO The remaining memory is really free for the stack and (not used) heap.

@holmes4
Some of the libraries are available in github. I'll update them and add the others but it seems I need to do a system upgrade first.
https://github.com/jantje/libraries
Due to debugging there is quite some trash in there but here is the main sketch
Code:
#include "ChargerBrains.h"
#include "SerialBridgeCommunicator.h"
//#include "FakeComunicator.h"
#include "LemCurrentSensor.h"
#include "JanServo.h"
#include "I2CLiquidCrystal.h"
#ifdef I_USE_MINI_BRIDGE
#include "MiniBridge.h"
#include "MiniProcess.h"
HardwareSerial &BridgeSerial = Serial1;
#else
#include <Bridge.h>
#include <HttpClient.h>
#endif
#include "gps_Library.h"

#define GPSTX_PIN 13
#define GPSRX_PIN 5
VoltMeter myChargerVoltmeter(A4);
LemCurrentSensor myCurrentSensor1(A0, A1);
LemCurrentSensor myCurrentSensor2(A2, A3);
SerialBridgeCommunicator myCommunicator;
//SerialCommunicator myCommunicator;
//FakeComunicator myCommunicator;
ChargerBrains myBrains;
SERIALTYPE myGPSserial(GPSTX_PIN, GPSRX_PIN);
GPSModule myGPS(9600UL, &myGPSserial);

JanServo myVoltageServo(4);
JanServo myPowerServo(6);

LiquidCrystal myLCD(0x38);
Stream *SerialInput = &Serial;
Stream *SerialOutput = &Serial;

const char mySketchName[] PROGMEM = "BatterijLader";
#define MAXFIELDS 65
FieldData AllFields[MAXFIELDS];

void setup()
{
boolean triedToStartListener = false;
int listenerStartWaitTime = 10000;
delay(4000); //give the LCD slave some time

myLCD.begin();
myLCD.print("batterijlader");
myLCD.setBackground(0, 0, 255);
myLCD.setCursor(0, 1);
myLCD.print(millis());

Serial.begin(115200);
//wait until serial is opened. If it is not opened it means I'm not debugging
//so I can start the serial listener to feed the web site and log reader
while (!Serial)
{

if (millis() > 120000UL) //40 seconds after startup start the serial listener
{
myLCD.setCursor(0, 0);
if (triedToStartListener)
{
myLCD.print(F("listener started 2 times")); //no need to log this to serial as it is not running
//Increase the time to wait for the script to start
listenerStartWaitTime += listenerStartWaitTime;
}
myLCD.print(F("start bridge") );
Bridge.begin();
myLCD.clear();
myLCD.print(F("bridge started"));
triedToStartListener = true;

//start the script on linux that will process the serial commands
runShellCommand(F("/www/cgi-bin/jantje/startReadingArduino.sh"), commonlyUsedBuffer, commonlyUsedBuffersize);
myLCD.clear();
myLCD.print(F("reader started"));
delay(listenerStartWaitTime); //make sure the started program is up and running
//this should also enable serial so a break is not needed
}
delay(100);
myLCD.setCursor(0, 1);
myLCD.print(millis());

}
Serial.println((__FlashStringHelper *) mySketchName);
Bridge.begin();

Serial.println(F("bridge started"));
// Serial.println(F("Test 2222222222222222222222222222222222222222222222222222222222222"));
// //Serial.println(F("Test 2222222222222222222222222222222222222222222222222222222222222"));
// //Serial.println(F("Test 2222222222222222222222222222222222222222222222222222222222222"));
// //Serial.println(F("Test 2222222222222222222222222222222222222222222222222222222222222"));
// //Serial.println(F("Test 2222222222222222222222222222222222222222222222222222222222222"));
// Serial.println(F("Test 22222222222222222"));

myCommunicator.serialRegister(F("Admin"));
myBrains.serialRegister(F("brains"));
myChargerVoltmeter.serialRegister(F("volt.charger"));
myCurrentSensor1.serialRegister(F("current.bat1"));
myCurrentSensor2.serialRegister(F("current.bat2"));
myVoltageServo.serialRegister(F("Servo.volt"));
myPowerServo.serialRegister(F("Servo.Power"));
myGPS.serialRegister(F("GPS"));
Serial.print(F("Current Datas "));
Serial.println(curFieldData);
Serial.println(F(" from "));
Serial.println(MAXFIELDS);
myCommunicator.setup();
myBrains.setup();
myChargerVoltmeter.setup();
myCurrentSensor1.setup();
myCurrentSensor2.setup();
myVoltageServo.setup();
myPowerServo.setup();
myGPS.setup();
Serial.println(F("setup done"));

}

void loop()
{
static uint32_t lastlcdprint = 0;
// myBattery1Voltmeter.loop();
// myBattery2Voltmeter.loop();
myChargerVoltmeter.loop();
myCurrentSensor1.loop();
myCurrentSensor2.loop();
myVoltageServo.loop();
myPowerServo.loop();
myGPS.loop();

myCommunicator.loop();
myBrains.loop();
if (millis() - lastlcdprint >= 2000)
{
snprintf(commonlyUsedBuffer, commonlyUsedBuffersize, "dV: %3i %3i %3i", myChargerVoltmeter.getVoltage() / 100, 0, 0); //myBattery1Voltmeter.getVoltage() / 100, myBattery2Voltmeter.getVoltage() / 100);
myLCD.clear();
myLCD.print(commonlyUsedBuffer);
//Serial.println(lcdBuffer);
snprintf(commonlyUsedBuffer, commonlyUsedBuffersize, "dA:     %3i %3i", myCurrentSensor1.getCurrent(), myCurrentSensor2.getCurrent());
//Serial.println(lcdBuffer);
myLCD.setCursor(0, 1);
myLCD.print(commonlyUsedBuffer);
lastlcdprint = millis();
}

}

As for my question on stack and heap. There is no need to specify sizes as explained here :
http://www.nongnu.org/avr-libc/user-manual/malloc.html

A small update.
I use a GPS module connected to pin 13 and 5. I don't think it is related but it looks as if it causes incomming data on Serial1.
I mean: The code works better without the gps connected (even though it worked 2 days ago  smiley-mr-green  )
But this addapted loop from bridge never finishes when the gps is connected.
Code:
  do {
  Serial.println();
  Serial.print("stream.available()=");
  Serial.println(stream.available());
    dropAll();
    delay(1000);
  } while (stream.available() > 0);


Best regards
Jantje
Logged

Do not PM me a question unless you are prepared to pay for consultancy.
Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -

Global Moderator
UK
Offline Offline
Brattain Member
*****
Karma: 301
Posts: 26240
I don't think you connected the grounds, Dave.
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
As I use 3 serial ports
And GPS and Wire...
Logged

"Pete, it's a fool looks for logic in the chambers of the human heart." Ulysses Everett McGill.
Do not send technical questions via personal messaging - they will be ignored.

Belgium
Offline Offline
Edison Member
*
Karma: 68
Posts: 1917
Arduino rocks; but with my plugin it can fly rocking the world ;-)
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

1 serial port is for GPS 1 for bridge and 1 for serial
And I use wire to communicate with the LCD
There is a reason why I stopped using the bootloader  smiley-twist
yes
Logged

Do not PM me a question unless you are prepared to pay for consultancy.
Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -

Poole, Dorset, UK
Offline Offline
Edison Member
*
Karma: 51
Posts: 2276
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Before setup() is called the main() function calls init()  main() or init() create the instances of hardwareSerial for Serial, Serial1 etc. Each instance of hardwareSerial has 2x64 byte buffers. In addition to that the heap must be created and initialized. My theory is that doing all this is putting you over the 2.5k of sram!.

The compiler can only report static data space used. not anything allocated during startup.

There is a way (playground) to get the amount of  free ram left at run time and it would be very easy to write a stub program to check how well the reported ram usage matches the actual at the beginning of setup().

Mark
Logged

North Queensland, Australia
Offline Offline
Edison Member
*
Karma: 69
Posts: 2166
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

@holmes4

The ( Hardware ) serial data is allocated statically:

Code: (Sketch)
void setup() {
  Serial.begin( 9600 );
  Serial1.begin( 9600 );
  Serial2.begin( 9600 );
}

void loop() { }

Quote from: Static SRAM
00800200 D __data_start
00800210 B __bss_start
00800210 D __data_end
00800210 D _edata
00800210 00000044 B rx_buffer
00800254 00000044 B tx_buffer
00800298 00000044 B rx_buffer1
008002dc 00000044 B tx_buffer1
00800320 00000044 B rx_buffer2
00800364 00000044 B tx_buffer2
008003a8 00000044 B rx_buffer3
008003ec 00000044 B tx_buffer3
00800430 00000022 B Serial
00800452 00000022 B Serial1
00800474 00000022 B Serial2
00800496 00000022 B Serial3
008004b8 00000004 B timer0_overflow_count
008004bc 00000004 B timer0_millis
008004c0 00000001 b timer0_fract
008004c1 B __bss_end

Whats worse, is it appears the data for each Serial port is allocated regardless of what is used ( Serial3 is in SRAM, even though I only used 0,1 & 2 ).

This probably doesn't apply as the Leonardo only supports one serial line and serial over CDC ( statically allocated ). SoftwareSerial or similar is probably being used. Even then those libraries probably have statically allocated buffers too.

@Jantje, you said:
Quote
To be able to do so I overload the virtual SerialCommunicator::setReceivedMessage but this stopped working.

Where did you overload it, in SerialCommunicator or in SerialBridgeCommunicator
Logged


Poole, Dorset, UK
Offline Offline
Edison Member
*
Karma: 51
Posts: 2276
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Well was worth a look.

Mark
Logged

Belgium
Offline Offline
Edison Member
*
Karma: 68
Posts: 1917
Arduino rocks; but with my plugin it can fly rocking the world ;-)
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

@holmes4

The ( Hardware ) serial data is allocated statically:
And so is software serial and altsoftware serial.

The reason is
Code:
class HardwareSerial : public Stream
{
...
    unsigned char _rx_buffer[SERIAL_BUFFER_SIZE];
    unsigned char _tx_buffer[SERIAL_BUFFER_SIZE];
  ...
};
Combined with
Code:
 extern HardwareSerial Serial1;
in hardwareSerial.h and because of that the needed
Code:
HardwareSerial Serial1(&UBRR1H, &UBRR1L, &UCSR1A, &UCSR1B, &UCSR1C, &UDR1, RXEN1, TXEN1, RXCIE1, UDRIE1, U2X1);
In other words the class gets instantiated as a global variable in the arduino core and as such the buffer gets added to the data section.
Remove the last 2 definitions and you save some space but lose the serial functionality.  
That won't really help me as I need them.

@Jantje, you said:
Quote
To be able to do so I overload the virtual SerialCommunicator::setReceivedMessage but this stopped working.

Where did you overload it, in SerialCommunicator or in SerialBridgeCommunicator

[EDIT: changed the code after the remark of Pyro]
Simplified the code looked like
Code:
class SerialCommunicator
{
protected:
virtual void setReceivedMessage(const char* newMessage);
public:
void loop();

};

class SerialBridgeCommunicator: public SerialCommunicator
{
protected:
virtual void setReceivedMessage(const char* newMessage)
                {
                  .......
                  SerialCommunicator::setReceivedMessage(newMessage);
                }
};
I changed it to
Code:
class SerialCommunicator
{
protected:
void setReceivedMessage(const char* newMessage);
public:
void loop();
};

class SerialBridgeCommunicator: public SerialCommunicator
{
protected:
void setReceivedMessage(const char* newMessage)
                {
                 the same code goes here
                }
public:
void loop()
                {
                 code copied from SerialCommunicator.loop
                }

};
By copying the code I changed the namespace and as such the correct setReceivedMessage is called. This was easy as setReceivedMessage is only called once.

In the mean time I have been able to do more code schrinking (I'll be an expert some day  smiley-lol ) and I'm now at
Code:
Program:   29368 bytes (89.6% Full)
(.text + .data + .bootloader)

Data:       1958 bytes (76.5% Full)
(.data + .bss + .noinit)
I'm still experiencing problems. But with these figures (and my coding rules as mentioned above) I think we can rule out stack and heap problems in the pre setup code.


There is a way (playground) to get the amount of  free ram left at run time and it would be very easy to write a stub program to check how well the reported ram usage matches the actual at the beginning of setup().
I see 2 problems here
1) The report in setup does not take into account the top heap/stack usage before the processor came to setup
2) When the problems happen the processor doesn't get to setup.


Best regards
Jantje
« Last Edit: February 14, 2014, 08:38:32 am by Jantje » Logged

Do not PM me a question unless you are prepared to pay for consultancy.
Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -

North Queensland, Australia
Offline Offline
Edison Member
*
Karma: 69
Posts: 2166
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Is the 'serialRegister' function supposed to be named setReceivedMessage. I ask due to the derived class calling 'SerialCommunicator::setReceivedMessage(newMessage);' which is not there, and also means nothing is being overloaded and serialRegister is never defined.

The types of the functions don't match either, you'd need at least a c-style conversion for newMessage.

If the base class virtual functions require a derived implementation, make them pure-virtual ( you do not need the unused code then ).



Logged


Belgium
Offline Offline
Edison Member
*
Karma: 68
Posts: 1917
Arduino rocks; but with my plugin it can fly rocking the world ;-)
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

@pYro_65
My bad I mixed up serialRegister and setReceivedMessage
serialRegister has nothing to do with the overload.
Best regards
Jantje
Logged

Do not PM me a question unless you are prepared to pay for consultancy.
Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -

Poole, Dorset, UK
Offline Offline
Edison Member
*
Karma: 51
Posts: 2276
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quick and worth a go try  reducing the size of the buffers in serial etc, if the program then gets to setup() then you know that you have a stack/heap usage problem during the program start up code. If it still fails to get in to setup() then odds on some kind of coding error in the constructors.

Mark
Logged

Pages: [1] 2   Go Up
Jump to: