Trying to share an array between two libraries to save RAM

My question is about how to let two independent libraries both access a single array.

I am using an Arduino Uno as a datalogger with an SD card reader/writer, and I'm trying to get it to also run a 48x84 pixel display. The problem is that the SD card reader has a 512 byte buffer and the display has a 504 byte buffer, and that is too much memory along with the other needs of the sketch. I have already omitted text message strings or moved them to flash, but I need more space.

I never need to use the two buffers at the same time, so I think it would be OK to have one buffer that both libraries share. I'd fill it with graphics information before I update the display, then clear it, and then fill it with data to write to the SD card before I call the SD write function.

I know it's sinful, but I'm trying to make the SD cache globally accessible so the LCD functions can use the same memory space. I'm thinking to do it that way because I don't have the knowledge to do it what is probably the right way by passing a pointer to all the functions that need this array (and I think there are lots in the SD library.) The variable name is unique and won't be used by chance in some other routine, and I need to get this working so I can put it into service. I'm open to suggestions for a different way to accomplish what I am trying to do, but I am only capable of very simple programming! (And have limited understanding of the different kinds of library files and what their parts do.)

Here's what I've done so far:

in SdFat.h I moved the declaration of cacheBuffer from the private to the public section.

union cache_t {
           /** Used to access cached file data blocks. */
  uint8_t  data[512];
           /** Used to access cached FAT16 entries. */
  uint16_t fat16[256];
           /** Used to access cached FAT32 entries. */
  uint32_t fat32[128];
           /** Used to access cached directory entries. */
  dir_t    dir[16];
           /** Used to access a cached MasterBoot Record. */
  mbr_t    mbr;
           /** Used to access to a cached FAT boot sector. */
  fbs_t    fbs;
};
//------------------------------------------------------------------------------
/**
 * \class SdVolume
 * \brief Access FAT16 and FAT32 volumes on SD and SDHC cards.
 */
class SdVolume {
 public:
  static cache_t cacheBuffer_;        // 512 byte cache for device blocks

now I can access the buffer array from my main sketch like this:

Serial.println(SdVolume::cacheBuffer_.data[10]);

but when I rename the buffer in Adafruit_PCD8544.cpp (which controls the LCD) from "pcd8544_buffer" to "SdVolume::cacheBuffer_.data" it complains that:

Adafruit_PCD8544.cpp: In member function ‘virtual void Adafruit_PCD8544::drawPixel(int16_t, int16_t, uint16_t)’:
Adafruit_PCD8544.cpp:123:5: error: ‘SdVolume’ has not been declared

Is there a way that I can make this variable accessible to Adafruit_PCD8544.cpp?

Again, I realize that making variables global like this is highly frowned upon and a sign of poor character, but it seems like the simplest way to do this and, in my situation not likely to cause mayhem.

Thanks for any advice!

--Abe

The answer is to not have the array in either library, but to have the array in the sketch. You then pass a pointer to the array to both libraries - either in the constructor, or using a member function.

I figured that was the correct way to do it, but the SD library files are so large and confusing to me that I'm not sure I can get the pointer passed to all the right places. Is there a way I can leave it in the library file as it is (and where all the other SD functions can find it), and then make it accessible to the much simpler LCD library?

Thanks,
Abe

Urgh... modifying the SD library... nightmare.

Well, the buffer is just a chunk of memory with a "symbol" pointing to it. That symbol will be in the global linker symbol table, so you should be able to tell your other library about it. For that you use the "extern" keyword.

Define the same variable in the LCD library exactly as it is defined in the SD library (but without any initialization if there is any), and prefix it with "extern". That will link it to that same symbol in the linker table and the memory will be shared between the two instances of the same variable.

Of course... you have to ask yourself "Will this corrupt my SD data?"

Can you be certain that the data in the SD library's buffer is stale at any particular point in time? If it has read a block from the SD card, and then later you write something to that block, and it knows that it read the block earlier (it's in its buffer) will it read it again regardless, or just modify its buffer and write it out again? If you have modified that buffer in between, will you end up corrupting your data nicely?

Thanks, that sounds promising. I’ve updated the LCD library file (Adafruit_PCD8544.cpp), but I guess I did it wrong because I’m getting an error. I tried to define the array exactly like in SdFat.h, like you suggested, but that definition was a union that contained other unique types, so I removed everything but the uint8_t type, so now it’s a union with only one way to access it. The section I added is the part that deals with cache_t right after the line “// THIS IS WHAT I ADDED TO MAKE SD CACHE AVAILABLE IN DISPLAY LIBRARY” Here’s the file, and after I’ll list the error.

//#include <Wire.h>
#include <avr/pgmspace.h>
#if defined(ARDUINO) && ARDUINO >= 100
  #include "Arduino.h"
#else
  #include "WProgram.h"
#endif
#include <util/delay.h>
#include <stdlib.h>

#include <Adafruit_GFX.h>
#include "Adafruit_PCD8544.h"
#include "glcdfont.c"

// a 5x7 font table
const extern uint8_t PROGMEM font[];

// the memory buffer for the LCD

// THIS IS WHAT I ADDED TO MAKE SD CACHE AVAILABLE IN DISPLAY LIBRARY

union cache_t {
           /** Used to access cached file data blocks. */
  uint8_t  data[512];
};

extern cache_t cacheBuffer_;        // 512 byte cache for device blocks

// reduces how much is refreshed, which speeds it up!
// originally derived from Steve Evans/JCW's mod but cleaned up and
// optimized
//#define enablePartialUpdate

#ifdef enablePartialUpdate
static uint8_t xUpdateMin, xUpdateMax, yUpdateMin, yUpdateMax;
#endif



static void updateBoundingBox(uint8_t xmin, uint8_t ymin, uint8_t xmax, uint8_t ymax) {
#ifdef enablePartialUpdate
  if (xmin < xUpdateMin) xUpdateMin = xmin;
  if (xmax > xUpdateMax) xUpdateMax = xmax;
  if (ymin < yUpdateMin) yUpdateMin = ymin;
  if (ymax > yUpdateMax) yUpdateMax = ymax;
#endif
}

Adafruit_PCD8544::Adafruit_PCD8544(int8_t SCLK, int8_t DIN, int8_t DC, int8_t CS, int8_t RST) {
  _din = DIN;
  _sclk = SCLK;
  _dc = DC;
  _rst = RST;
  _cs = CS;

  constructor(LCDWIDTH, LCDHEIGHT);
}

Adafruit_PCD8544::Adafruit_PCD8544(int8_t SCLK, int8_t DIN, int8_t DC, int8_t RST) {
  _din = DIN;
  _sclk = SCLK;
  _dc = DC;
  _rst = RST;
  _cs = -1;

  constructor(LCDWIDTH, LCDHEIGHT);
}


// the most basic function, set a single pixel
void Adafruit_PCD8544::drawPixel(int16_t x, int16_t y, uint16_t color) {
  if ((x < 0) || (x >= LCDWIDTH) || (y < 0) || (y >= LCDHEIGHT))
    return;

  // x is which column
  if (color) 
    cacheBuffer_.data[x+ (y/8)*LCDWIDTH] |= _BV(y%8);  
  else
    cacheBuffer_.data[x+ (y/8)*LCDWIDTH] &= ~_BV(y%8); 

  updateBoundingBox(x,y,x,y);
}


// the most basic function, get a single pixel
uint8_t Adafruit_PCD8544::getPixel(int8_t x, int8_t y) {
  if ((x < 0) || (x >= LCDWIDTH) || (y < 0) || (y >= LCDHEIGHT))
    return 0;

  return (cacheBuffer_.data[x+ (y/8)*LCDWIDTH] >> (y%8)) & 0x1;  
}


void Adafruit_PCD8544::begin(uint8_t contrast) {
  // set pin directions
  pinMode(_din, OUTPUT);
  pinMode(_sclk, OUTPUT);
  pinMode(_dc, OUTPUT);
  if (_rst > 0)
    pinMode(_rst, OUTPUT);
  if (_cs > 0)
    pinMode(_cs, OUTPUT);

  // toggle RST low to reset
  if (_rst > 0) {
    digitalWrite(_rst, LOW);
    _delay_ms(500);
    digitalWrite(_rst, HIGH);
  }

  clkport     = portOutputRegister(digitalPinToPort(_sclk));
  clkpinmask  = digitalPinToBitMask(_sclk);
  mosiport    = portOutputRegister(digitalPinToPort(_din));
  mosipinmask = digitalPinToBitMask(_din);
  csport    = portOutputRegister(digitalPinToPort(_cs));
  cspinmask = digitalPinToBitMask(_cs);
  dcport    = portOutputRegister(digitalPinToPort(_dc));
  dcpinmask = digitalPinToBitMask(_dc);

  // get into the EXTENDED mode!
  command(PCD8544_FUNCTIONSET | PCD8544_EXTENDEDINSTRUCTION );

  // LCD bias select (4 is optimal?)
  command(PCD8544_SETBIAS | 0x4);

  // set VOP
  if (contrast > 0x7f)
    contrast = 0x7f;

  command( PCD8544_SETVOP | contrast); // Experimentally determined


  // normal mode
  command(PCD8544_FUNCTIONSET);

  // Set display to Normal
  command(PCD8544_DISPLAYCONTROL | PCD8544_DISPLAYNORMAL);

  // initial display line
  // set page address
  // set column address
  // write display data

  // set up a bounding box for screen updates

  updateBoundingBox(0, 0, LCDWIDTH-1, LCDHEIGHT-1);
  // Push out cacheBuffer_ to the Display (will show the AFI logo)
  display();
}


inline void Adafruit_PCD8544::fastSPIwrite(uint8_t d) {
  
  for(uint8_t bit = 0x80; bit; bit >>= 1) {
    *clkport &= ~clkpinmask;
    if(d & bit) *mosiport |=  mosipinmask;
    else        *mosiport &= ~mosipinmask;
    *clkport |=  clkpinmask;
  }
}

inline void Adafruit_PCD8544::slowSPIwrite(uint8_t c) {
  shiftOut(_din, _sclk, MSBFIRST, c);
}

void Adafruit_PCD8544::command(uint8_t c) {
  digitalWrite(_dc, LOW);
  if (_cs > 0)
    digitalWrite(_cs, LOW);
  fastSPIwrite(c);
  if (_cs > 0)
    digitalWrite(_cs, HIGH);
}

void Adafruit_PCD8544::data(uint8_t c) {
  digitalWrite(_dc, HIGH);
  if (_cs > 0)
    digitalWrite(_cs, LOW);
  fastSPIwrite(c);
  if (_cs > 0)
    digitalWrite(_cs, HIGH);
}

void Adafruit_PCD8544::setContrast(uint8_t val) {
  if (val > 0x7f) {
    val = 0x7f;
  }
  command(PCD8544_FUNCTIONSET | PCD8544_EXTENDEDINSTRUCTION );
  command( PCD8544_SETVOP | val); 
  command(PCD8544_FUNCTIONSET);
  
 }



void Adafruit_PCD8544::display(void) {
  uint8_t col, maxcol, p;
  
  for(p = 0; p < 6; p++) {
#ifdef enablePartialUpdate
    // check if this page is part of update
    if ( yUpdateMin >= ((p+1)*8) ) {
      continue;   // nope, skip it!
    }
    if (yUpdateMax < p*8) {
      break;
    }d
#endif

    command(PCD8544_SETYADDR | p);


#ifdef enablePartialUpdate
    col = xUpdateMin;
    maxcol = xUpdateMax;
#else
    // start at the beginning of the row
    col = 0;
    maxcol = LCDWIDTH-1;
#endif

    command(PCD8544_SETXADDR | col);

    digitalWrite(_dc, HIGH);
    if (_cs > 0)
      digitalWrite(_cs, LOW);
    for(; col <= maxcol; col++) {
      //uart_putw_dec(col);
      //uart_putchar(' ');
      fastSPIwrite(cacheBuffer_.data[(LCDWIDTH*p)+col]);
    }
    if (_cs > 0)
      digitalWrite(_cs, HIGH);

  }

  command(PCD8544_SETYADDR );  // no idea why this is necessary but it is to finish the last byte?
#ifdef enablePartialUpdate
  xUpdateMin = LCDWIDTH - 1;
  xUpdateMax = 0;
  yUpdateMin = LCDHEIGHT-1;
  yUpdateMax = 0;
#endif

}

// clear everything
void Adafruit_PCD8544::clearDisplay(void) {
  memset(cacheBuffer_.data, 0, LCDWIDTH*LCDHEIGHT/8);
  updateBoundingBox(0, 0, LCDWIDTH-1, LCDHEIGHT-1);
  cursor_y = cursor_x = 0;
}

/*
// this doesnt touch the buffer, just clears the display RAM - might be handy
void Adafruit_PCD8544::clearDisplay(void) {
  
  uint8_t p, c;
  
  for(p = 0; p < 8; p++) {

    st7565_command(CMD_SET_PAGE | p);
    for(c = 0; c < 129; c++) {
      //uart_putw_dec(c);
      //uart_putchar(' ');
      st7565_command(CMD_SET_COLUMN_LOWER | (c & 0xf));
      st7565_command(CMD_SET_COLUMN_UPPER | ((c >> 4) & 0xf));
      st7565_data(0x0);
    }     
    }

}

*/

The error is:

Adafruit_PCD8544_Nokia_5110_LCD/Adafruit_PCD8544.cpp.o: In function `Adafruit_PCD8544::clearDisplay()':
/home/abe/sketchbook/libraries/Adafruit_PCD8544_Nokia_5110_LCD/Adafruit_PCD8544.cpp:306: undefined reference to `cacheBuffer_'

Variables within libraries are totally new to me, so I expect I’m making one or more basic blunders.

Thanks,
Abe

Ah, I foresee a slight problem here (now I have looked at the SD library's code).

cacheBuffer_ is a member variable within the SdVolume class, not a separate variable in its own right.

You won't be able to do it this way.

You would need to get the pointer to that buffer instance's memory space from the volume object you have created in your sketch, then pass that to the LCD library.

OK, I think I'm making progress. I have the pointer to the array and can use in in my sketch to access the buffer space. But when I try to pass it to the library like this:

display.begin(50, SdVolume::cacheBuffer_.data);

I get this error:

In file included from DS18x20_probe_SD_and_RTC.cpp:6:0:
sketchbook/libraries/Adafruit_PCD8544_Nokia_5110_LCD/Adafruit_PCD8544.h:57:8: error: default argument missing for parameter 2 of ‘void Adafruit_PCD8544::begin(uint8_t, uint8_t*)’

here's the part in the .h file:

  void begin(uint8_t contrast = 40, uint8_t* pcd8544_buffer);

I can't find online how to set a default value for a pointer like this. Any advice, or have I made some mistake upstream?

Thanks,
Abe

The SD library performs writes to the buffer, NOT TO THE FILE, when you call the print(), println(), and write() methods. The SD library takes care to transfer the contents of the buffer to the file only as often as needed, because actually writing to the file takes a long time. Actually reading from the file takes a (relatively) long time, too. So, the buffer is what is usually accessed when the read() methods are called.

When the buffer is full, for a file opened for write, or empty for a file opened for read, another read or write operation that actually accesses the file is performed to make room in the buffer/put data in the buffer.

In order to reuse the SD buffer, you'd need to make a copy of the data in the buffer before repurposing the buffer. If you have room to do that, you have room to NOT f**k with the SD buffer.

My advise is to cease and desist in this folly. Buy an Arduino that has more memory.

This is for a datalogger that only writes once every 15 minutes, and I am closing the file between writes so as to not mess up the file in the case of unexpected power loss, someone pulling out the SD card, etc. My understanding is that this writes the buffer to the file. The display updates every 10 seconds or so and has no need to update during the file access period.

To me it looks like the SD functions and the display functions only use the buffer when they are being asked to do something with their associated hardware. Is there a problem with using the array for the display 99% of the time, and then when I need to write to a file I wipe the buffer, open the file, fill the buffer with data, and then write it to the SD file? Unless one of those functions is doing things without being asked I don't see the conflict, and if I close the SD file between writes, I don't think the SD functions will be trying to do anything.

Is there an error in my thinking? I will be making a number of these little guys and would like to stay on the Uno if is is possible.

Thanks,
Abe

There's only 1 place in the library where the buffer is allocated. Change that. Use the same name to hold a pointer and set the pointer to the address of your you-made-sure-it's-safe buffer. Use a cast to get the type right if need be, not a union as it's a 1 time 1 place operation. Be sure to give the new version of the library a descriptive name too.

Other than that, PaulS has a really good point. Think about using a stand-alone AVR for your logger instead of a whole development board.

OK, I've been working on this, trying to follow all the advice (except to get a different board!) Specifically I'm trying to change the LCD library where the buffer is assigned and then from the sketch pass a pointer to it that points to the SD buffer. I think I've found where the allocation happens, but I'm clearly not doing the modification right.

I added a pointer to the line in my sketch where I first invoke the LCD:

Adafruit_PCD8544 display = Adafruit_PCD8544(7, 6, 5, 4, 3, SdVolume::cacheBuffer_.data);

Originally the buffer is defined in the .cpp file as:

uint8_t pcd8544_buffer[LCDWIDTH * LCDHEIGHT / 8] = {
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,

and so on...

I commented that out and modified the first line of the following code to accept the pointer to pcd8544_buffer:

Adafruit_PCD8544::Adafruit_PCD8544(int8_t SCLK, int8_t DIN, int8_t DC, int8_t CS, int8_t RST, uint8_t* pcd8544_buffer) {
  _din = DIN;
  _sclk = SCLK;
  _dc = DC;
  _rst = RST;
  _cs = CS;
  constructor(LCDWIDTH, LCDHEIGHT);
}

Adafruit_PCD8544::Adafruit_PCD8544(int8_t SCLK, int8_t DIN, int8_t DC, int8_t RST) {
  _din = DIN;
  _sclk = SCLK;
  _dc = DC;
  _rst = RST;
  _cs = -1;

  constructor(LCDWIDTH, LCDHEIGHT);
}

then I added the buffer pointer to the third line of the code below in the .h file:

class Adafruit_PCD8544 : public Adafruit_GFX {
 public:
  Adafruit_PCD8544(int8_t SCLK, int8_t DIN, int8_t DC, int8_t CS, int8_t RST, uint8_t* pcd8544_buffer);
  Adafruit_PCD8544(int8_t SCLK, int8_t DIN, int8_t DC, int8_t RST);

but I get this error:

sketchbook/libraries/Adafruit_PCD8544_Nokia_5110_LCD/Adafruit_PCD8544.cpp: In member function ‘virtual void Adafruit_PCD8544::drawPixel(int16_t, int16_t, uint16_t)’:
sketchbook/libraries/Adafruit_PCD8544_Nokia_5110_LCD/Adafruit_PCD8544.cpp:132:5: error: ‘pcd8544_buffer’ was not declared in this scope

It looks like the functions in the library can't find my buffer. Any thoughts?

Thanks,
Abe

Put a pointer in the SD library to the buffer, and one in the LCD library. Read the SD pointer and set the LCD pointer in the project sketch.

It might work and as PaulS noted, it might only work for a while.