Hello,
I'm trying to understand how the Arduino memory works (as in the programming space). For example, lets say I create a very long string in my Arduino sketch. What memory is taken up by this string? Is it the EEPROM or SRAM?
I ask because I'm trying to log data using my Arduino but it seems that there is a max string length which is understandable because it takes memory to keep appending data values to a string.
SRAM
There are THREE kinds of memory on Arduino.
EEPROM (written and read using EEPROM library or the avr eeprom functions)
Flash/Program Memory (stores the sketch)
SRAM (stores data that changes while the program runs, and the contents of all variables not explicitly stored in program memory).
If you have a normal string (c string, or string literal, ie "abcdef"), it is stored as a char array in flash, and then copied to SRAM on startup.
When using constant strings with print statements, you can use the F() macro to store the string in flash.
For other applications, where you need to be doing string comparison and such, you can put the string into flash by declaring the variable const and using the PROGMEM keyword (ex, copied from one of my sketches:
const char AT24R[] PROGMEM = {"AT+24R"}; ) and use the _P versions of the standard c string functions (ie, strcmp_P to compare a string (char array) with one stores in flash, instead of strcmp()).
Strings (with a capitol S, the string class) are always stored in SRAM, and use dynamic memory allocation so that you don't need to know the length of the String at compile time. That sort of thing works great on systems with lots of ram, but the Arduino doesn't have lots of ram, and the dynamic allocation can result in bad behavior caused by running out of memory (and unlike c strings, the compiler can't tell you how much memory is being used by the Strings to warn you of this). You should avoid using Strings if at all possible - c strings (fixed length character arrays) can be made to do all the same things as Strings (albeit sometimes less intuitively), and do not have the problems associated with dynamic memory allocation.
As one example, instead of gluing several strings together and then printing the whole mess to serial, like you might be used to doing on other platforms, with Arduino, it's often better to print each piece with it's own Serial.print() statement, because then you don't need to waste memory storing the combined string.
The max string length is likely adjustable, up to the limit of 2048 bytes less whatever space the rest of your code needs for variables and registers and stack/heap usage for function calls and interrupts and whatnot.
You can of course add it to an array you are storing in EEPROM, that may impact performance as an EEPROM write takes 3.3mS. And you only get so many writes.
Alternately, write to an external SPI interfaced SRAM or FRAM. Will be nearly as fast as using the internal SRAM. Perhaps indistinguishable to a user.
cosmictealeaf:
I'm trying to understand how the Arduino memory works (as in the programming space). For example, lets say I create a very long string in my Arduino sketch. What memory is taken up by this string? Is it the EEPROM or SRAM?
SRAM
The worse your programming style, the more SRAM you will need.
cosmictealeaf:
I ask because I'm trying to log data using my Arduino but it seems that there is a max string length which is understandable because it takes memory to keep appending data values to a string.
Sounds like you create "String" type objects instead of using nullterminated strings (char arrays).
That's the worst of all types you can use in Arduino programs.
Perhaps you better describe what you want to do, then possibly somebody in this forum can explain you how to handle it (limit string length), handle data in chunks of fixed size, send incoming chars to bigger storage (i.e. SD card) if you need long term data storage of very long strings.
Thank you everyone for your info. i understand much better.
I guess it might be helpful to someone if I explain my project a little. I am reading a linear sensor with 512 pixels. I need to record each pixel intensity using the ADS1115 16 bit ADC from Adafruit (as you will see in the code). I used the example on arduino's website (Tutorial on Data logging with Arduino)
The issue is I need to scan the sensor 6 times for each set of data that will be sent to an SD card, and this happens very fast. I need to do this every two minutes as a requirement for my project:
//Libraries included in code
#include <Wire.h>
#include <SPI.h>
#include <RTClib.h>
#include <RTC_DS1307.h>
#include <Adafruit_ADS1015.h>
#include <Ethernet2.h>
#include <EthernetUdp2.h>
//Peripheral components RTC(DS3231) and ADC(ADS1115)
RTC_DS1307 RTC;
Adafruit_ADS1115 ads1115;
//Define several global variables
const int PIXELS = 512; //number of pixels on linear sensor
int counter = 0; //counter to end after six scans
String dataString = "";
byte mac[] = {0x90, 0xA2, 0xDA, 0x10, 0x21, 0xCB}; //MAC address for Ethernet Shield
unsigned int localPort = 8888; // local port to listen for UDP packets
IPAddress timeServer(x,t,y,z); // UArizona NTP server
const int NTP_PACKET_SIZE = 48; // NTP time stamp is in the first 48 bytes of the message
byte packetBuffer[ NTP_PACKET_SIZE]; //buffer to hold incoming and outgoing packets
EthernetUDP Udp;// A UDP instance to let us send and receive packets over UDP
//---------------------------------BEGIN SETUP------------------------------------------------------\\
void setup(){
//Begin Serial at BAUD 1115200
Serial.begin(115200);
//Begin communication with DS3231 RTC
Wire.begin();
RTC.begin();
DateTime now = RTC.now();
DateTime compiled = DateTime(__DATE__, __TIME__);
if (now.unixtime() < compiled.unixtime()) {
// following line sets the RTC to the date & time this sketch was compiled
RTC.adjust(DateTime(__DATE__, __TIME__));
}
//Send Date from RTC to the Serial Port
dataString += String(now.year());
dataString += "-";
dataString += String(now.month());
dataString += "-";
dataString += String(now.day());
dataString += ",";
//Grab UDP Time and print to serial
// start Ethernet and UDP
if(Ethernet.begin(mac) == 0) {
Serial.println("Failed to configure Ethernet using DHCP");
// no point in carrying on, so do nothing forevermore:
for (;;)
;
}
Udp.begin(localPort);
sendNTPpacket(timeServer); // send an NTP packet to a time server
// wait to see if a reply is available
delay(1000);
if (Udp.parsePacket()) {
// We've received a packet, read the data from it
Udp.read(packetBuffer, NTP_PACKET_SIZE); // read the packet into the buffer
// the timestamp starts at byte 40 of the received packet and is four bytes,
// or two words, long. First, extract the two words:
unsigned long highWord = word(packetBuffer[40], packetBuffer[41]);
unsigned long lowWord = word(packetBuffer[42], packetBuffer[43]);
// combine the four bytes (two words) into a long integer
// this is NTP time (seconds since Jan 1 1900):
unsigned long secsSince1900 = highWord << 16 | lowWord;
// Unix time starts on Jan 1 1970. In seconds, that's 2208988800:
const unsigned long seventyYears = 2208988800UL;
// subtract seventy years:
unsigned long epoch = secsSince1900 - seventyYears;
// print the hour, minute and second:
dataString += String((epoch % 86400L)/3600);
dataString += ":";
if (((epoch % 3600) / 60) < 10) {
// In the first 10 minutes of each hour, we'll want a leading '0'
dataString += "0";
}
dataString += String((epoch % 3600) / 60);
dataString += ":";
if ((epoch % 60) < 10) {
// In the first 10 seconds of each minute, we'll want a leading '0'
dataString += "0";
}
dataString += String(epoch % 60);
dataString += ",";
}
//--------------------------------------PIXEL-SCAN--------------------------------------------------------\\
int startTime(int pixel){
int16_t adc;
PORTD |= _BV(3); //start high
PORTD |= _BV(2); //clock high
delayMicroseconds(4); //delay for start pulse width
PORTD &= ~_BV(2); //clock low
PORTD &= ~_BV(3); //start low
for(int i=0;i<pixel;i++) {//clock goes from low to high to
//shuffle through pixels, find the one we want
PORTD |= _BV(2); //clock high
delayMicroseconds(4);
PORTD &= ~_BV(2); //clock low, need to read now
}
//read ADS1115
adc = ads1115.readADC_Differential_0_1();
dataString += String(adc + 32787);
dataString += ",";
for (int i=0;i<=(PIXELS-pixel);i++) {
PORTD |= _BV(2); //clock high
delayMicroseconds(4);
PORTD &= ~_BV(2); //clock low
}
}
void loop(){
counter++; //count each time we scan the sensor
//first value is discarded from each scan
startTime(0);
delay(1);
//scan of pixel array
for (int i=0;i<PIXELS;i++) {
startTime(i);
delay(1);
}
//once sensor is scanned 6 times, wait two minutes, get time stamp and start over
if (counter == 6){
Serial.println(dataString);
delay(120000);
counter = 0;
dataString = ""; //clear dataString begin anew
//create new timestamp
RTC.begin();
DateTime now = RTC.now();
DateTime compiled = DateTime(__DATE__, __TIME__);
if (now.unixtime() < compiled.unixtime()) {
// following line sets the RTC to the date & time this sketch was compiled
RTC.adjust(DateTime(__DATE__, __TIME__));
}
//Send Date from RTC to the Serial Port
dataString += String(now.year());
dataString += "-";
dataString += String(now.month());
dataString += "-";
dataString += String(now.day());
dataString += ",";
//Create new time stamp for next set of measurements
Udp.begin(localPort);
sendNTPpacket(timeServer);
delay(1000);
if (Udp.parsePacket()) {
Udp.read(packetBuffer, NTP_PACKET_SIZE);
unsigned long highWord = word(packetBuffer[40], packetBuffer[41]);
unsigned long lowWord = word(packetBuffer[42], packetBuffer[43]);
unsigned long secsSince1900 = highWord << 16 | lowWord;
const unsigned long seventyYears = 2208988800UL;
unsigned long epoch = secsSince1900 - seventyYears;
// print the hour, minute and second:
dataString += String((epoch % 86400L)/3600);
dataString += ":";
if (((epoch % 3600) / 60) < 10) {
// In the first 10 minutes of each hour, we'll want a leading '0'
dataString += "0";
}
dataString += String((epoch % 3600) / 60);
dataString += ":";
if ((epoch % 60) < 10) {
// In the first 10 seconds of each minute, we'll want a leading '0'
dataString += "0";
}
dataString += String(epoch % 60);
dataString += ",";
//}
}
}
My Arduino is doing a few things in this code. First of all I have an ethernet shield using Wiznet 5500 chip. So I am pulling time from a server using the ethernet. I am also using the ADS1115 from adafruit to read the sensor. I am also using the DS3231 Real Time Clock to take the date. The time and the date are recorded using dataString first. Then I read the linear sensor of 512 pixels 6 times.
I have not included any code on using the SD card to send the data to since I do not have one yet. All I am doing is setting up the code to do that once I have the SD card.
So I make an empty string called dataString. I append to dataString first the date, then the time, then the sensor data.
This is obviously way too much data for the Arduino SRAM to handle, as the String has over 3000 data values.
The reason I am doing this is I eventually want the data to be written in a chunk to an SD card but I don't have time to write one value at a time to the SD card since there are so many values and it would take too long.
So does anyone see any way around this memory/time tradeoff?
Thank you. I appreciate all the help.
What you're doing with that dataString... I'm not at all surprised you're having problems.
You're much better off doing something like:
Serial.print(now.year());
Serial.print(F("-"));
Serial.print(now.month());
Serial.print(F("-"));
Serial.print(now.day());
//and so on, ending with Serial.println() to get the newline.
Otherwise, you're creating and destroying tons of instances of the worst-behaved datatype (String) - this is a very good way to demonstrate why the String class should be avoided. The arduino just doesn't have enough RAM for you to be able to dynamically allocate and reallocate memory just to concatenate strings like that.
If you need a longer string to write to SD card, define a fixed length char array (long enough to fit an entry), and use the c string functions like strcat() to append to it...