Sketch size > 93678 Bytes - digitalRead() does not work anymore

Hi there,

I searched the web and tested a lot now, but it seems I am missing something important. I am working on my project "Watering System" for a year now, but when the size of my sketch grow over 93.678 Bytes recently the entire system went crazy. What I found so far is that past this magic border the function digitalRead (maybe other I/O-functions as well) does not operate as expected.

I use an Arduino Mega R3 with a stacked Ethernet Shield (5100 and 5500) and a RTC (DS1302) attached as well as LCD (20 x 4 chars), my IDE version is 1.8.13

To debounce buttons attached to Arduino I/O-Pins (with internal pullups enabled) I made a small test project where I developed the functionality, then integrated this with my project. After integration to my main project some month ago it worked as expected and the small test project still works fine today on the very same hardware ... but in my main project now with a sketch size over 93.678 Bytes digitalRead returns 0 where it should return 1

I don't know if this is relevant, but to save SRAM I placed some of my static strings in PROGMEM and even in section ".fini7" using this declarations and code:

#define PROGMEM_FAR __attribute__((section(".fini7")))
#define FAR(s) (__extension__({static const char __c[] PROGMEM_FAR = (s); pgm_get_far_address(__c);}))

const char pmfstrText[]    PROGMEM_FAR = "Some text";
char buf1[60];
strncpy_PF(buf1, pgm_get_far_address(pmfstrText), sizeof(buf1));
Serial.println(buf1);
or
strncpy_PF(buf1, FAR("Another string"), sizeof(buf1));
Serial.println(buf1);

In the FAQ section of the avr/pgmspace.h Program Space Utilities at More than 128 KiB of flash, how to make function pointers work? I found the hint that for large sketches the option -mrelax must be given on the compiler command-line ... but I am not able to find the right place to insert and test this option.

Can someone point me into the right direction on my problem or has a hint where compiler options have to be placed ... any help would be greatly appreciated: thx a lot in advance!

Post ALL the code, and forum members will offer a range of helpful suggestions.

In all instances, or just in some occurrence where you expect it? Did you infer it, or did you Serial print the raw reading?

"putting strings in SRAM" doesn't save any flash memory. they must be in flash and are loaded into SRAM at start.
There can be problems with sketch larger than 128 kB, but not with smaller.
I have a working 93k sketch on ATmega 1284p.

I can't post the entire project with a size of 417 kB here, but this is the Debounce-Test-Project which works fine as itself, but not anymore as integrated part of the main project:

/*
  DESCRIPTION
  ====================
  Simple example of how to debounce buttons without an additional library
  For original code source see: https://www.mikrocontroller.net/topic/337404
  Scroll down to post by "Jürgen S." dated 03.07.2014 17:47

  Available click types: SHORTCLICK, DOUBLECLICK, LONGCLICK
*/
    #define readyLedPin    11                                     // Pin used to signal ready/standby state
    #define readyKeyPin     9                                     // Pin used to toggle ready/standby state
    #define internalLed    13
    #define errorLedPin    14                                     // Pin used to signal errors detected (e.g. in water flow measurement, during startup, etc.)
    #define errorKeyPin    15                                     // Pin used to reset last signalled error
    #define masterLedPin   16                                     // Pin used to switch on/off main water pipe/water flow LED
    bool stateError = false;
    bool stateMaster = false;
    bool stateReady = false;

/// For original code source visit: https://www.mikrocontroller.net/topic/337404
/// --> Scroll down to post by "Jürgen S." dated 03.07.2014 17:47

/// Tastenerkennung SHORTCLICK, DOUBLECLICK, LONGCLICK
    #define INPUTMODE INPUT_PULLUP  // INPUT or INPUT_PULLUP
    #define PRELLZEIT 5             // Debounce time in milliseconds
    // #define SHORTCLICKTIME 250   // Längste Zeit für einen SHORTCLICK
    #define DOUBLECLICKTIME 50      // Längste Zeit für den zweiten Klick beim DOUBLECLICK - For use WITHOUT DoubleClicks (reacts immediately to button clicks)
    //#define DOUBLECLICKTIME 150     // Längste Zeit für den zweiten Klick beim DOUBLECLICK - For use WITH FAST DoubleClicks (reacts almost immediately to button clicks)
    //#define DOUBLECLICKTIME 400     // Längste Zeit für den zweiten Klick beim DOUBLECLICK - For use WITH SECURE DoubleClicks (reacts slightly postponed to button clicks)
    #define LONGCLICKTIME 600       // Mindestzeit für einen LONGGLICK
    //byte buttonPins[]={2,3,4,5,6,7,8}; // Arduino Pins
    byte buttonPins[]={readyKeyPin, errorKeyPin};     // Arduino Pins
    #define NUMBUTTONS sizeof(buttonPins)
    byte buttonState[NUMBUTTONS];   // Aktueller Status des Buttons HIGH/LOW
    enum {NONE, FIRSTDOWN, FIRSTUP, SHORTCLICK, DOUBLECLICK, LONGCLICK}; 
    byte buttonResult[NUMBUTTONS];  // Aktueller Klickstatus der Buttons NONE/SHORTCLICK/LONGCLICK

/// Main program entry
    void setup() {

      pinMode(53, OUTPUT);                              // Pin 53 is Slave Select (SS) for the ATmega2560 and a HIGH value prevents it from going into slave mode
      digitalWrite(53, HIGH);                           //    (For more details see https://forum.arduino.cc/t/mega-pin53-where-does-it-go/470986/3

      pinMode(10, OUTPUT);                              // Disable Ethernet chip
      digitalWrite(10, HIGH);

      pinMode(errorLedPin, OUTPUT);                     // Switch off ERROR LED/relay during setup
      digitalWrite(errorLedPin, HIGH);

      pinMode(masterLedPin, OUTPUT);                    // Switch off online LED during setup
      digitalWrite(masterLedPin, HIGH);

      pinMode(readyLedPin, OUTPUT);                     // Switch off online LED during setup
      digitalWrite(readyLedPin, HIGH);

      for (int i = 0; i < NUMBUTTONS; i++)              // Configure button/key pins as an input and enable the internal pull-up resistor
        pinMode(buttonPins[i],INPUTMODE);

      Serial.begin(9600);

      Serial.println(F("=== Debounce Buttons"));
    }

/// Infinite program loop
    void loop() {
      if (mainButtonsDebounce())
      {
        //mainButtonsDebug();   // Just for general debugging
        mainButtonEvents();
      }
      //benchmark();
    }

/// Detects state changes for the configured buttons/keys
///   Return value false ==> debounce time is running, button pins will not get checked
///   Return value true  ==> button pins were checked and state is set
    bool mainButtonsDebounce() {
      static unsigned long lastRunTime;
      static unsigned long buttonDownTime[NUMBUTTONS];
      unsigned long thisTime = millis();
      
      if (thisTime - lastRunTime < PRELLZEIT) 
        return false;                       // Prellzeit läuft noch
        
      lastRunTime = thisTime;
      for (int i = 0; i < NUMBUTTONS; i++) {
        byte curState = digitalRead(buttonPins[i]); 
        if (INPUTMODE == INPUT_PULLUP) 
          curState = !curState;             // Vertauschte Logik bei INPPUT_PULLUP
        if (buttonResult[i] >= SHORTCLICK) 
          buttonResult[i] = NONE;           // Letztes buttonResult löschen
        if (curState != buttonState[i]) {   // Flankenwechsel am Button festgestellt
          if (curState) {                   // Taster wird gedrückt, Zeit merken
            if (buttonResult[i] == FIRSTUP && thisTime-buttonDownTime[i] < DOUBLECLICKTIME)
              buttonResult[i] = DOUBLECLICK;
            else {  
              buttonDownTime[i] = thisTime; 
              buttonResult[i] = FIRSTDOWN;
            } 
          }
          else {                            // Taster wird losgelassen
            if (buttonResult[i] == FIRSTDOWN)
              buttonResult[i] = FIRSTUP;
            if (thisTime - buttonDownTime[i] >= LONGCLICKTIME) 
              buttonResult[i] = LONGCLICK;
          }
        }
        else {                              // kein Flankenwechsel, Up/Down Status ist unverändert
          if (buttonResult[i] == FIRSTUP && thisTime-buttonDownTime[i] > DOUBLECLICKTIME) 
            buttonResult[i] = SHORTCLICK;
        }
        buttonState[i] = curState;
      } // for
      return true;
    }

/// Prints the status of clicked buttons to Serial
    void mainButtonsDebug() {
      for (int i = 0; i < NUMBUTTONS; i++) {
        if (buttonResult[i] >= SHORTCLICK) {  // A button was clicked
          Serial.print(millis() / 1000.0, 3); // Timestamp in milliseconds
          Serial.print("\tPin-");
          Serial.print(buttonPins[i]);
          if (buttonResult[i] == SHORTCLICK) 
            Serial.println(F(" CLICK"));
          else if(buttonResult[i] == DOUBLECLICK)
            Serial.println(F(" DOUBLE CLICK"));
          else if(buttonResult[i] == LONGCLICK) 
            Serial.println(F(" LONG CLICK"));
        }
      }
    }

/// Invokes the actions associated with button clicks
    void mainButtonEvents() {
      if (buttonResult[0] >= SHORTCLICK) {      // Ready/Standby-Button was pressed
        if (buttonResult[0] == SHORTCLICK)
          mainEventReady();
        if (buttonResult[0] == LONGCLICK)
          mainEventReset();
      }
      if (buttonResult[1] >= SHORTCLICK) {      // Error-Button was pressed
        if (buttonResult[1] == SHORTCLICK)
          mainEventError(); 
      }
    }

/// Resets/clears the last error signal/message
    void mainEventError() {
      Serial.println(F("mainEventError()"));
      stateError = !stateError;
      digitalWrite(errorLedPin, !stateError);   // Switch ERROR-LED on/off
    }

/// Toggles the Ready-State (online/standby)
    void mainEventReady() {
      Serial.println(F("mainEventReady()"));
      stateReady = !stateReady;
      digitalWrite(readyLedPin, !stateReady);   // Light green/red LED
    }

/// Intended to reset/reboot the MCU
    void mainEventReset() {
      Serial.println(F("mainEventReset()"));
      stateMaster = !stateMaster;
      digitalWrite(masterLedPin, !stateMaster); // Switch MASTER-LED on/off
    }

/// Measures and prints the average loop execution time to Serial
    void benchmark() {
      static unsigned long counterStartTime;
      static unsigned long counter;
      counter++;
      if (counter>=1000000L) {
        Serial.print(F("Average Time per loop(): "));
        Serial.print((micros()-counterStartTime)/1000000.0);
        Serial.println(F(" microseconds"));
        counter=0;
        counterStartTime=micros();
      }
    }

Couriously not in all instances: when called directly in setup() digitalRead returns expeced results; when called in my function mainButtonsDebounce() being itself called from the loop()-Function digitalRead returns false results.

Because of the size of the sketch I thought about an addressing problem (16 Bit) or linker problem ...

False results, or not the input state that you expect to read at that particular moment in time?

The size of the sketch is suspicious, for the task you've described. Often, novice code can be factored and reduced by a factor of 3 or 10 to 1.

Please post the question on Github, where it belongs.

Thx for your questions and hints - they took me further!

The size of the sketch is because of the functionality of my watering system: I incorporated a web server with GET- & POST-Requests, some classes for web-configurable Sensors and Devices and a web-configurable WorkSet to specify individual watering rules. I am sure that refactoring might lower the resulting sketsch size, but not for 30 % to get below 64 k - and the project is not done yet.

And, to answer your question, in my test I did Serial.print the raw value like

byte r1 = 9; 
r1 = digitalRead(r1);
Serial.println(r1);

In my further tests I figured out that digitalRead finally DOES NOT returns false results. The input pins are configured with internal pullups and when the attached buttons are open I can measure apx 4.85 V at the pin, when I close the buttons to GND the voltage drops to some 110 mV. All this is clearly visible at my oscillocope.

During setup() I can see the voltage drop at the oscilloscope when class WorkSet is instantiated. In the constructor of this class I read the specified config from a SD-File and create new Sensor-, new Device- and new Condition-Objects. The reason for the voltage to drop at a certain moment is when I just insert 3 new Byte-Members in my class declaration (see the 3 lines marked like '<<< MAGIC') ... without any reference somewhere in my code:

class WorkSet {
  
  private:
             bool _isValid = false;                 // Tells whether this workset holds valid data
             bool _isWfmPossible = false;           // Tells whether WFM-data is present und complete to activate WFM-checks
             bool _isSingleActorMode = true;        // Operate only one device at a time, although more than one may meet their switching condition

  public:
    // Member
             char   name[lenFilename + 1] = {0};    // Filename of the WorkSet (including extension .ws)
             char   error[lenError + 1] = {0};      // Last error that has occurred
           time_t   errorTime = 0;                  // Time the last error occurred
          uint8_t   errorState = errCleared;        // The current errorState (see Workset ERROR states above)
             byte   wiredAnalog;    // <<< MAGIC after insert //Number of physically wired analog sensors
             byte   wiredBinary;    // <<< MAGIC after insert //Number of physically wired binary sensors
             byte   wiredDevices;   // <<< MAGIC after insert //Number of physically wired digital devices
             byte   numSensors = 0;                 // Number of Sensors attached
             byte   numDevices = 0;                 // Number of Devices attached
           Sensor **ppSensors;                      // Sensors handled in this WorkSet
           Device **ppDevices;                      // Devices handled in this WorkSet
           int8_t   state = stateUnknown;           // Current state of this WorkSet (see Device-, Sensor- and WorkSet-states above)
           int8_t   stateSaved = stateUnknown;      // Holds the state saved before different actions, e.g. before activating WFM
           time_t   wfmLastOnOff = 0;               // Last time a device has been switched on or off
    unsigned long   wfmLastCount = 0;               // Reading of the water counter at the time the last device was switched on or off
           int8_t   wfmLearnMode = learnOff;        // Shows whether Learning Mode is on
           int8_t   wfmAlertedDeviation = 0;        // Deviation already reported (for this value an alert already has been sent to LCD and log)
           int8_t   wfmAllowedDeviation = 0;        // Threshold for the allowed deviation between counted and calculated pulses within regular operations
                                                    // int8_t allowes a range from -127 to +127 and maxDeviation has to be within this range.
                                                    // Deviation values greater than this threshold causes the Error-LED to be switched on.
          int16_t   wfmCurrentDeviation = 0;        // Records the current deviation
         uint16_t   wfmLostPulses = 0;              // Holds the last reported (to the LCD) and yet unconfirmed number of pulses without active device
             bool   wfmIsPulseCalculated = false;   // Tells wheter WFM pulses were calculated during operational WFM
             bool   wfmIsPulseCounted = false;      // Tells wheter a second WFM pulse was detected after the first WFM start pulse (after the last device was switched on)

    // Constructor
    WorkSet(char *filename);                                // Constructor

    // Functions
       bool checkDevices();                         // Checks whether Devices have to be switched on or off
       bool checkSwitching(Device *d, char *currDay, char *onlyTime); // Checks whether uptimes and condition are met to switch on the device
       bool checkUptimes(Device *d, char *currDay, char *onlyTime);   // Checks uptimes whether the device has to be switched on or off
                                                                      // currDay  = "Mon"-"Son"/"***" (len = 4 including \0)
                                                                      // onlyTime = "0800", "1015"    (len = 5 including \0)
       bool checkValues(Device *d);                 // Checks values of conditions whether the device has to be switched on or off
       void degradeSequence(uint8_t seq);           // Degrades the sequence of all devices ordered past the provided sequence
                                                    // (Used after a seqenced device was switched on in ::checkDevices)
       void dump(byte typeOfDest, byte typeOfLog);  // Dumps the properties of the entire WorkSet to Serial and/or the specified logfiles or to both
     int8_t findActiveDevice();                     // Returns the index of the first device which is not deactivated
     int8_t findDeviceInState(int8_t state);        // Returns the index of the first device found in the the provided state or -1 if none was found
     int8_t findDeviceRunning();                    // Returns the index of the first device found with isOn()==true or -1 if none was found
     int8_t findDeviceToSwitch();                   // Returns the index of the device to switch on next according to sequence
     int8_t findMaxSequence();                      // Returns the the maximum sequence currently found in a device or 0 if no device has set a sequence
     int8_t findMinSequence();                      // Returns the the minimum sequence currently found in a device or 0 if no device has set a sequence
       void getDeviceStates(char *tBuf);            // Fills the provided character buffer with the state of all configured devices: 
        int getSize();                              // Returns the size in Bytes of the entire WorkSet and all included Sensors, Devices, Uptimes and Conditions
       bool isActive();                             // Returns whether this WorkSet is active or not
       bool isValid();                              // Returns true, if the object was successfully created from an WorkSet-file
       bool isWfmPossible();                        // Returns true, if WFM-data is present for all configured devices
       bool isSingleActorMode();                    // Returns true, if only device is allowed to operate at a time, although more than one may meet their switching condition
       bool loadDeviceWfms();                       // Loads the devices .lastAvgWf values from the file 'WorkSet-name'.wf (in case it exists)
       void processCmd(char *);                     // Processes the specified command for the WorkSet
       bool readLine(File, char *, int, int *);     // Reads the next line of the specified file with buffer lenght check
       bool readLine(File, char *);                 // Reads the next line of the specified file, no buffer lenght check
      float readSensorValue(char anDi, byte pin);   // Reads and returns the current value for the specified sensor
       void relocateSequences();                    // Relocates the sequence of all devices for the provided gap so the first sequence starts at 1
       void resequenceDevices(char *currDay, char *onlyTime); // Resequences all unsequenced devices in stateOff having positively completed their check to switch on
       bool saveDeviceWfms();                       // Saves the devices .lastAvgWf values to the file 'WorkSet-name'.wf
       void setState(int8_t);                       // Sets the WorkSet to the specified state-byte
       void wfmCancel();                            // Aborts a possibly ongoing water flow measurement
       void wfmStartNextOrEndWfm();                 // Searches for a device pending for WFM and initiates it (or end the WFM)
};

Tracked effects:

  • With these 3 new lines the voltage changes to LOW during setup() repectively the creation of the WorkSet with its Sensors, Devices, Uptimes, Conditions.
  • Whithout these 3 new lines my scetch works and the voltage stays HIGH
  • When I comment one of these 3 membes leaving just 2 the voltage stays HIGH
  • When I move the 3 lines down to the end of the member declarations right before the declaration of the ctor the voltage stays HIGH
  • When I move them at the very beginning before bool _isValid the voltage changes to LOW

So I changed my config file (different numbers of sensors/devices/conditions) but the effect stays. Only the moment the voltage goes down changes - like it was influenced by the number of Bytes I allocated with my 'new Sensor' & 'new Device' statements.

I can not imagine a reason for this strange behavior ... ?

I think I should close this topic and open a new one with a corrected description!

Please no, the thread is following a normal pattern of investigations and changes. There is no need.

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.