Go Down

Topic: virtmem: easily extend your memory (Read 568 times) previous topic - next topic

-Rick-

Hello all,

virtmem is an Arduino library that makes it easy to extend the limited RAM available on microcontrollers with 'virtual memory'. To do so, virtmem provides a simple interface to use external memory sources such as an SD card or SPI RAM chip which resembles working with 'regular' memory.

Some of its main features are:
    * Extend the available memory with kilobytes, megabytes or even gigabytes
    * Supports SPI RAM (23LC series from Microchip), SD cards and RAM from a computer connected through serial
    * Easy C++ interface that closely resembles regular data access
    * Memory page system to speed up access to virtual memory
    * New memory interfaces can be added easily

And a small demonstration that uses an SD card as memory source:
Code: [Select]

#include <Arduino.h>
#include <SdFat.h>
#include <virtmem.h>
#include <alloc/sd_alloc.h>

// Simplify virtmem usage
using namespace virtmem;

// Create virtual a memory allocator that uses SD card (with FAT filesystem) as virtual memory pool
// The default memory pool size (1 MB) is used.
SDVAlloc valloc;

SdFat sd;

struct MyStruct { int x, y; };

void setup()
{
    // Initialize SdFatlib
    if (!sd.begin(9, SPI_FULL_SPEED))
        sd.initErrorHalt();

    valloc.start(); // Always call this to initialize the allocator before using it

    // Allocate a char buffer of 10000 bytes in virtual memory and store the address to a virtual pointer
    VPtr<char, SDVAlloc> str = valloc.alloc<char>(10000);

    // Set the first 1000 bytes to 'A'
    memset(str, 'A', 1000);

    // array access
    str[1] = 'B';

    // Own types (structs/classes) also work.
    VPtr<MyStruct, SDVAlloc> ms = valloc.alloc<MyStruct>(); // alloc call without parameters: use automatic size deduction
    ms->x = 5;
    ms->y = 15;
}

void loop()
{
    // ...
}


The project is hosted on github: https://github.com/rhelmus/virtmem
A detailed manual with more info can be found here: http://rhelmus.github.io/virtmem/index.html

This is the first release, but already pretty feature complete. Nevertheless, any feedback (questions, suggestions, bugs etc) is welcome!

pYro_65

Very nice.

I like how you have overloaded the standard C functions for use (would make it clearer to newbies that you need to be using the namespace to access them).

This looks like a very in depth library. I did a similar (very basic) tool for using my web site as virtual ram. Looing at serial_alloc.h it looks like I could easily move my Ethernet communication into a class using VAlloc. Looks very efficient using CRTP doing away with dynamic polymorphism.

I'm not 100% sure, however I think your lib needs to be the top level folder in the repo to be compatible with the IDE library manager.
Forum Mod anyone?
https://arduino.land/Moduino/

-Rick-

Very nice.

I like how you have overloaded the standard C functions for use (would make it clearer to newbies that you need to be using the namespace to access them).

This looks like a very in depth library. I did a similar (very basic) tool for using my web site as virtual ram. Looing at serial_alloc.h it looks like I could easily move my Ethernet communication into a class using VAlloc. Looks very efficient using CRTP doing away with dynamic polymorphism.

I'm not 100% sure, however I think your lib needs to be the top level folder in the repo to be compatible with the IDE library manager.
Thanks for your feedback. I will think of something to clarify the namespaces. Although hopefully newbies will just stick with the 'using namespace virtmem' line.

The ethernet interface definitely sounds interesting. New allocators should be indeed be fairly easy to setup.

And you were right about the library manager compatibility. I made a dummy repo, hopefully that will work better.

-Rick-

virtmem can now be downloaded from the library manager :)

erathos

Hello Rick (if still here)
Virtmem is an interesting library that looks indeed very well designed. Yet, unless  my own misinterpretation, it uses a lot of memory. On my Mega2560 + Ethernet shield (W5100 + sdcard), compiling your example "sd_simple" in the IDE 1.6.11 uses 3772 bytes of internal RAM (46% of the available 8k). Similarly, the « spiram_simple » example use 3110 bytes (37%)  in RAM. It is therefore a priori the price to pay in internal RAM for the treatment of a single int variable  shifted into virtual memory (hmm !). Surprisingly, when compiling for the UNO board, I get 1180 and 504 bytes respectively (57% vs 24% of the 2k RAM) for both examples. What is the reason of such différence between boards ? Would not this be the expression of a strong limit in the use of this library for long sketches performing multiple tasks, while its main interest should be to expand access to large global variables overcoming limitation in internal RAM?

Go Up
 


Please enter a valid email to subscribe

Confirm your email address

We need to confirm your email address.
To complete the subscription, please click the link in the email we just sent you.

Thank you for subscribing!

Arduino
via Egeo 16
Torino, 10131
Italy