Is there a better way to format text data for display?

Projects Programming
1

Hey, all, I have a question, in two parts, about whether I should do this another way. I've been a computer guy for decades and have done network engineering (MCSE), programming (VB, VB.NET), built and sold computers, blah blah blah. I've never done IC programming which is so very different. I've spent hours trying things and reading these forums trying to achieve even basic functionality towards a goal.

I'm using Arduino Uno and Nano boards.

Where I am right now is I want to have an RTC module do its job and have the Arduino do functions based on time/date. The first step is just reading/writing the time and then displaying the time. The code below is my attempt to do that. It's working now, the code below compiles and operates as I expect (save for padding single digit numbers with a leading 0). However, the memory usage is at 19% for storage space and 25% for dynamic memory.

"Sketch uses 6084 bytes (19%) of program storage space. Maximum is 30720 bytes.
Global variables use 513 bytes (25%) of dynamic memory, leaving 1535 bytes for local variables. Maximum is 2048 bytes."

I suspect this is due to the use of the DS3232.h which, in turn, uses/relies on the TimeLib.h. As I don't have experience with "libraries", C++ (or any C), and limited memory situations, I need guidance, please. I suspect that the entirety of those libraries is loaded into "program storage".

I'm avoiding use of Strings as they seem to be frowned upon for efficiency/memory "leakage". In this code segment the goal is to read and display the time and do so using functions to try to keep the loop() routine as clean as possible for readability/debugging. As I add functionality to this sketch, the loop() will get hard to follow if I don't break code into subs/functions.

So, my uncertainties are: 1) should I attempt anything to reduce program storage usage (possibly editing libraries to remove functions/code I do not need for my project, if that's even possible), 2) should I be doing anything better to format the time information to make it more efficient? For example, I have no need or desire for the alarm functionality on the DS3231; I could remove all code from the DS3232.h file pertaining to the alarms. Hypothetically.

Thank you. Code below:

#include <DS3232RTC.h>      // RTC DS3231 header/functions.  This internally needs TimeLib.h

DS3232RTC myRTC;  //constructor for the RTC class

void setup()
{
    Serial.begin(9600);
    myRTC.begin();    //initializes I2C bus
    setSyncInterval(60);    //synchronizes the Arduino time (TimeLib.h) with the RTC every xx *seconds*, not milliseconds
    setSyncProvider(myRTC.get);   // setSyncProvider from TimeLib.h to sync Arduino time library to the RTC
    if(timeStatus() != timeSet)
        Serial.println("Unable to sync with the RTC");
    else
        Serial.println("RTC has set the system time");
}

void loop()
{
    char DisplayTime[17];      //a string to hold the data we'll display.  16 chars for the display and 1 char for the null terminator \0

    //digitalClockDisplay();  //need to replace this with code which will show time on the LCD
    strcpy(DisplayTime,FormatTime(DisplayTime));
 
    Serial.println(DisplayTime);

    delay(1000);  //remove this as this sketch progresses. internal variable milli counter will be checked
}

char * FormatTime(char DT)
{
    // digital clock display of the time
    char temp[5];       //max 4 chars plus null term

    //strcpy(DT, "Hello");
    itoa(hour(),temp,DEC);
    strcpy(DT, temp);
    strcat(DT,":");
    itoa(minute(),temp,DEC);
    strcat(DT, temp);
    strcat(DT,":");
    itoa(second(),temp,DEC);
    strcat(DT, temp);

    return DT;
}
2

Welcome to the forum

Using libraries will not remove functions from your sketch. What happens is that only functions that you call that are in libraries are incorporated into the sketch behind the scenes during compilation, not the whole library

You might like to look at using the sprintf() and/or snprintf() functions as a way of neatly formatting string data

Given the figures for memory usage that you quote, why do you think that you need to reduce them ?

1 Like
3

I'd use something like:

snprintf(DisplayTime,17,"%02d:%02d:%02d",hour(),minute(),second());

...to protect youself from times like "1:2:3"

See https://cplusplus.com/reference/cstdio/snprintf/

1 Like
4

As an aside, for long literal sequences of characters like this, wrap them in the F() macro:

Serial.println(F("Unable to sync with the RTC"));

This ensures that the text gets stored in program memory rather than taking up valuable runtime memory. It won't take too many examples like that to use up the whopping 2k you have available on a UNO....

There is a bit more explanation here:

5

Thank you for the reply.

I had read about the sprintf() function but got the impression (not saying rightly or wrongly) that it might be preferable to avoid it. I will look into that more.

My concerns re space utilization are: 1) being completely new to this type of programming with such limitations, learning to try to do it better seems wise, and 2) the bigger concern, is that I need to add the ability to calculate sunrise and sunset times and I was afraid that adding further libraries to do that would potentially run me out of memory. After making this post I included a couple of libraries to that end but I have not gotten enough coding done to attempt to compile/load the sketch. However, with what you've said about it not including in the BIN code not used, I'm not as concerned now.

This is a whole new world to me. I spent about 10 hours to get the posted code written and debugged.

--HC

6

Thank you for the reply. Yes, I see how that will save me about 30+ lines of code. That would have been much simpler. I will change my code. Some reading somewhere (I've done a lot of reading just to produce the code I posted) made me feel that snprintf() should maybe be avoided. Can't say where or how authoritative it was or even if I read it correctly. But this still appears to produce a char[] which is preferable in, at least, the Arduino world to Strings.

--HC

7

Thank you for the reply.

I understand what you're saying. 2k is...piddly. :slight_smile:

I read that article briefly. I'll go back and read it again later but I understand enough to then need to ask this question: If using Serial.println("text"); uses up some of the SRAM, when or how is it then freed up when it isn't needed any longer? For printing like this to the computer screen (effectively), that text storage space wouldn't be needed after the transmission is complete. Does it automatically free that storage or is there some explicit release that needs to be performed?

--HC

8

It isn't, that's why using the F() macro was suggested.

9

A slight error in the code. If you change the preferences in the Arduino IDE to show verbose output during compilation, there are several warnings that originate with the following line:

An array of char passed as an argument to a function passes a pointer, not a char, so the line should be:

char * FormatTime(char* DT)

The following strcpy() is not needed. You are passing the DisplayTime array to the FormatTime() function, which directly modifies the array, after which you copy the array onto itself.

As for alternative ways to do the same thing, the following makes minimal use of functions:

char * FormatTime(char* DT)
{
  int i = hour();
  DT[0] = (i / 10) + '0';
  DT[1] = (i % 10) + '0';
  DT[2] = ':';
  i = minute();
  DT[3] = (i / 10) + '0';
  DT[4] = (i % 10) + '0';
  DT[5] = ':';
  i = second();
  DT[6] = (i / 10) + '0';
  DT[7] = (i % 10) + '0';
  DT[8] = '\0';
  return DT;
}

Using delay(1000) to get a 1-second interval will eventually result in skipping a second in the displayed time, because it takes a finite, although small, amount of time to execute the remainder of the code. Additionally, that is 1000ms in which the processor could be doing something useful.
A common technique is to test for when the data changes, in the case of a clock when the seconds change.

void loop()
{
  static byte previousSecond = 0;
  if (previousSecond != second())
  {
    previousSecond = second();
    char DisplayTime[17];      //a string to hold the data we'll display.  16 chars for the display and 1 char for the null terminator \0

    //digitalClockDisplay();  //need to replace this with code which will show time on the LCD
    FormatTime(DisplayTime);

    Serial.println(DisplayTime);
  }
}
1 Like
10

When printing text, the less RAM buffer you use the better.

Program memory (flash aka PROGMEM), you have many times over what you have in RAM, there are many ways to store and print text from PROGMEM to save on precious RAM.

Using stdio.h formatted printing will add a chunk to program memory but worse is the practice of building long complete formatted lines before printing as the RAM buffer space to do so is unnecessary. Using many short code lines to print the text in pieces assembles the line in the Serial output buffer (64 chars) that is already allocated and empties itself.
One thing to do is use a high baud rate for Serial just because that will print out the buffer faster. 9600? Go for 115200!

32K of PROGMEM (ATmega328 boards) can hold a lot of program and still have room for constant data such as text strings and lookup tables, but if you need more then think about using a bigger chip/board.
You have options to get or make such a board or even to use multiple chips/boards. Many times I have seen people jump from Uno or Nano straight to Mega2560 boards with 8K RAM (can add optional external RAM on bus pins... I have a Quadram card for Mega2560 that gives 8 banks of 56K with the onboard RAM used as dedicated stack space --- Rugged Circuits makes the card) and 256K of program space.
Another option is to use an ATmega1284 chip (DIP is what I have) with 16K RAM (has no external RAM bus) and 128K flash. There are boards with/for that chip. The 2560 has 4 serial ports and over 50 IO pins, the 1284 has 2 and 32 and the 328 has 1 and 18.

You can roll your own cheaper than buy if you don't count what you use as a programmer... almost any Arduino can be used as a programmer.
The ATmega2560 only comes in surface mount, I prefer to use DIP AVR's cause my hardware skills are poor.
Nick Gammon's How-To make minimal Duinos with examples 328 and 1284.
What can go on a breadboard can go on protoboard or PCB.

O'Baka Arduino builds a Duino on a bare chip!
It is also possible to wire a socket instead of the chip then plug a chip into that. If you burn a pin, it's easy to replace the chip.

one_chip_arduino
one_chip_arduino640×480 80.4 KB

That is a DIP ATmega328, can be done with a DIP ATmega1284.

AVR family chips are largely self-contained, so much is possible!

1 Like
11

Yeah, printf adds some overhead, but it does provide some nice features.

There's ways to use sprintf with flash/PROGMEM storage:

Here's a modified snprintf_P() example with the added const for more modern c++ compilers:

const char header_1[] PROGMEM = "Dissolved O2";
char buffer[100];

void setup() {
   Serial.begin(57600);
   snprintf_P(buffer,100,PSTR( "The column header is %S") ,header_1);
   Serial.println(buffer);
}
void loop(){}

It might not be relevant to your current needs, but there are a couple rarely-documented functions for formatting doubles into strings, dtostrf() and dtostre():

12

Filling a buffer longer than the serial output buffer and then printing that is a great way to block execution until the output buffer is less than full.

Please THINK.

1 Like
13

Ok, thank you. So, if it isn't freed up, after a certain number of Serial.println("text"); statements, the SRAM will be filled causing errors or a crash?

--HC

14

Thank you, there's a lot here.

I turned on the Compiler Warnings: All now. After posting the code above I continued to bang on the code and had already changed that line to pass a pointer...I don't know how/why I caught it. I do have a number of warnings from the compiler about what I would call "type cast conversions"...whatever they are, with code I've added since the original post here, I'm implicitly making some type conversions which aren't making the compiler happy. Now that I see them, I will try to correct them. It compiles anyway but I don't like compiler warnings.

I have removed the unnecessary strcpy(). I suppose I believed that the function wasn't manipulating the DisplayTime through a pointer. Or I was on crack. Hard to tell. It's manipulating the variable via its pointer so, absolutely you're correct, there's no need to do a strcpy().

I'll have to ask a question here about changing how the formatting is done:

DT[0] this is an array element reference for assignment, I get this
(i / 10) is division by 10 to get the tens place digit (23 / 10 = 2), I get this
" + '0'" I do not understand.

We start with an int (i) and do math (i / 10) and should get an int answer. Then we add a char 0? Or am I misunderstanding the significance of the single quotes? If there is an implicit type conversion to char or a char array, is it then copying the "20" (23 / 10 = 2 + '0') from the start and dropping the rest? Copying "20" to DT[0] copies the "2" then ditches the remainder of the char array as there is no room (single element [0])? That would be slick. Other than saving a function call to snprintf(), would there be an appreciable savings of some kind (clock cycles, memory) to doing it this way? Regardless, it's a cool way to build a char array and pad info with 0's (or any char).

The delay(1000); was part of a code snippet I'd based some of my early code upon. It's already gone in my current code.

if (millis() % 1000 == 0)        //only update once per second
        {
            lastMillis = millis();
            //strcpy(DisplayTime,FormatTime(DisplayTime));
            FormatTime(DisplayTime);
 
            //Serial.println(DisplayTime);        //remark out for production

            // set the cursor to column 0, line 1
            // (note: line 0 is the first row, since counting begins with 0):
            lcd.setCursor(0, 0);
            // print the number of seconds since reset:
            lcd.print(DisplayTime);
        }

I appreciate you pointing it out. The delay() sub parks the whole IC for the duration, not ideal.

Oddly enough, with the modified code above, I would have thought that it would be possible, even likely, that updates to the display would occur multiple times per second (16Mhz clock speed should hit 1,000 milliseconds several times per millisecond I thought) but putting in a Serial.println("Updated"); to identify if that was happening didn't yield the gillion "updated" messages I expected, just one per second. Regardless, I've coded it to check the last second against the current second and only update the display per your example (done while writing this reply and not reflected in the short segment above).

--HC

15

@HazardsMind offered this a while back, it might be of interest:

unsigned long bigNumber  = 0xC0FFEE;

//================================
template<typename TYPE>
inline Print & operator<<(Print &Outport, TYPE arg)
{
  Outport.print(arg);
  return Outport;
}


//=========================================================================================
void setup()
{
  Serial.begin(9600);
}


//=========================================================================================
void loop()
{
  Serial << F("millis() is currently = ") << millis() << F(" bigNumber = ") << bigNumber << "\n";
  delay(1000ul);
}
16

Thank you...there's a fair bit here, too.

Okay, so what I'm getting/understanding about the text for output is this:

Serial.println("The swift brown fox outran the fat, lazy rabbit."); isn't good.

Serial.printf("The ");
Serial.printf("swift ");
Serial.printf("brown ");
...
Serial.println("rabbit.");

is better.

Better to piece it together in the print buffer.

I chose the 9,600 baud because I don't know crap. I didn't understand what the speed was regulating and was thinking it might have something to do with the comms to peripherals like I2C. I understand now that was incorrect. I was afraid a high baud rate might cause lost data or errors. Hey, I'm learning by doing, not getting formal education. I'm embarrassed to admit the baud thing but that's the truth. I will point out that I was surprised to find that setting the baud rates differently from the code to the monitor makes a difference. I shouldn't be surprised but I would have expected the monitor (computer) to adapt maybe. It doesn't. Newbie alert: If you're getting gibberish in the Serial Monitor in the IDE, make sure the code and Serial Monitor baud rates are the same. Yeah, I'm an idiot.

The information on the chips is interesting but I'm overwhelmed trying to learn basic stuff on the small chips. I'm going to stay on the splash pad a bit longer. :wink:

Those micro builds are nuts!

--HC

1 Like
17

You don't need this library at all if all you want is to read timestamps from the DS.

All you need is to:

  1. Send 0x00 byte via I2C to your DS3231. (its address usually is 0x68)
  2. Read 7 bytes from I2C back.

Suppose you got 0x24 0x25 0x22 0x03 0x18 0x09 0x24 back from your DS3231.

These are BCD numbers and are decoded like this:

0x24 0x25 0x22  --> 22:25:24
0x03            --> Third day of the week
0x18 0x09 0x24  --> 18 of September 2024.
2 Likes
18

(i/10) gives the digit in the tens place, which is then added to the numeric value of an ASCII zero. The result will be the ASCII value for the digit in the tens place.

19

Thank you. I did some reading about the snprintf(); function and then learned that the _P variant is for dealing with that data in the PROGMEM. That's cool. I found an article I did a little reading in (I'll read it all after these replies) which explains the value of the snprintf(); with the ability to specify the target size.

I'm not sure it's cool to post links (I forget at the moment) so I'll put this: it's on cpp4arduino and the article page is how-to-format-strings-without-the-string-class.html if anyone cares to look it up.

--HC

20

Thank you but I'm going to have to look at that a few times before I get that. The best I think I can discern from that is the line in the loop() which seems to be a bit stream out.

--HC