Go Down

Topic: RamDisk - a file system library for RAM devices like 23LCV1024 (Read 4283 times) previous topic - next topic

fat16lib

Mar 29, 2014, 07:04 pm Last Edit: Apr 29, 2014, 07:39 pm by fat16lib Reason: 1
Update: An updated version, 20140429, of RamDisk is attached to this post.

I have attached a very early version of a FAT like file system for external RAM devices.

I wish to thank pito for encouraging me to look into a file library for devices like his 8 MB RAM device.

I have been testing with a 23LCV1024 1Mbit SPI Serial SRAM with Battery Backup.  This 8 pin DIP with a small coin cell provides 128 KB of file storage.

Three example sketches are included with the library.

Please try the library and post any comments.

Here is the readme file.
Quote

This is a very early version of a RamDisk library.  It provides a FAT style
file system on an external RAM device.

RamDisk is incomplete and likely has bugs.  I am releasing this version so
future users can provide comments and input.

Please post any comments in the Arduino forum topic where this file
was posted.

This library was developed using a Microchip 23LCV1024 1 Mbit SPI
serial SRAM with battery backup.

You must have working 23LCV1024 or 23LC1024 hardware or write your own
RAM device library.  You should be able to use other RAM devices by
writing a library with the RamBaseClass API.

Please read the html documentation. Go to the Classes tab and look
at the RamDisk and RamBaseClass.

You must install three libraries to try the examples included with RamDisk.

The RamDisk folder contains file-system code and three examples.

The 23LCV1024 folder contains the raw SRAM driver.

The SdFat folder contains a version of SdFat that can coexist with RamDisk.

There are three examples that demonstrate RamDisk features.

Start with HelloRamDisk.ino.  This example writes two lines of text to
a file and then reads the file and displays the text on the serial monitor.

Here is the output from HelloRamDisk:

Format (Y/N): N
1: Hello RamDisk!
2: Line to test fgets


RamDiskLoggerTest.ino will log 12,000 samples to RAM as fast as possible.
each sample consists of value from five analog pins.  A total of 60,000
values are written to a binary file.  Timing statistics and the first ten
samples in the log file are printed.

Here is output with pins 0, 2, 4 grounded. Pin 1 is 5 V and pin 3 is 3.3 V.

Format (Y/N): Y
clusterCount: 250
Starting test
t: 7.53 sec
minMicros: 60
maxMicros: 216
avgMicros: 66
sync micros: 156
filesize: 120000

pin0,pin1,pin2,pin3,pin4
0,1023,0,676,0
0,1023,0,676,0
0,1023,0,676,0
0,1023,0,676,0
0,1023,0,676,0
0,1023,0,676,0
0,1023,0,676,0
0,1023,0,676,0
0,1023,0,676,0
0,1023,0,676,0
Done


RamDiskToSdFat.ino creates a file with 1000 text lines in RAM.  It then
copies the text file to an SD file.

Here is the output from RamDiskToSdFat:

FreeRam: 1092
Format (Y/N): Y
Writing ramFile
TEST.CSV      2000-01-01 01:00:00 16746
Copying ramFile to sdFile
BENCH.DAT      2000-01-01 01:00:00 4999680
TEST.TXT       2000-01-01 01:00:00 16743
TEST.CSV       2000-01-01 01:00:00 16746
Done

 

pito

#1
Mar 29, 2014, 08:04 pm Last Edit: Mar 29, 2014, 09:22 pm by pito Reason: 1
Here is my mod for 8MB ramdisk (lib for M8MBRDSK11 and HelloRamdisk_8MB), coping with how to define fast IO within the lib.
Enclosed my files, pls be so kind and advice me how to tackle the templated signal pins (and bus port) there in the ram class  :smiley-roll-sweat:
HelloRamdisk_8MB:
Code: [Select]
// With 8MB Ramdisk, atmega1284p
// Fast IO is required with bitbanging here
#include <DigitalIO.h>
DigitalPin<12> NRD(OUTPUT);    // /RD active LOW
DigitalPin<13> NWR(OUTPUT);    // /WR active LOW
DigitalPin<14> NDATA(OUTPUT);   // /Data active LOW
// PortC is the data port

#include <M8MBRDSK11.h>
#include <RamDisk.h>

M8MBRDSK11 ram;
RamDisk file;
char buf[40];
..


Also, I need to increment the *buf while shooting bytes in/out:
Code: [Select]
C:\MyCode\Arduino\libraries\M8MBRDSK11\M8MBRDSK11.cpp: In member function 'virtual bool M8MBRDSK11::read(uint32_t, void*, size_t)':
C:\MyCode\Arduino\libraries\M8MBRDSK11\M8MBRDSK11.cpp:81: error: ISO C++ forbids incrementing a pointer of type 'void*'
C:\MyCode\Arduino\libraries\M8MBRDSK11\M8MBRDSK11.cpp:81: error: 'void*' is not a pointer-to-object type
C:\MyCode\Arduino\libraries\M8MBRDSK11\M8MBRDSK11.cpp: In member function 'virtual bool M8MBRDSK11::write(uint32_t, const void*, size_t)':
C:\MyCode\Arduino\libraries\M8MBRDSK11\M8MBRDSK11.cpp:88: error: ISO C++ forbids incrementing a pointer of type 'const void*'
C:\MyCode\Arduino\libraries\M8MBRDSK11\M8MBRDSK11.cpp:88: error: 'const void*' is not a pointer-to-object type


This is what I need (as discussed offline):
Code: [Select]
..
bool M8MBRDSK11::read(uint32_t address, unsigned char *buf, unsigned int nbytes) {
..
bool M8MBRDSK11::write(uint32_t address, unsigned char *buf, unsigned int nbytes) {

where
address is the starting address of a "block"
*buf is the pointer to buf[nbytes]
nbytes is the number of bytes in the block, ie 512

fat16lib

This compiles.  maybe.zip might be a step forward.

pito

#3
Mar 29, 2014, 09:34 pm Last Edit: Mar 29, 2014, 09:40 pm by pito Reason: 1
Thanks!
I get now (atmega1284p @16MHz):
Code: [Select]

Format (Y/N): Y
1: Hello RamDisk!
2: Line to test fgets


Code: [Select]
Format (Y/N): Y
clusterCount: 250
Starting test
t: 7.32 sec
minMicros: 40
maxMicros: 120
avgMicros: 41
sync micros: 96
filesize: 120000

pin0,pin1,pin2,pin3,pin4
226,208,165,186,153
206,221,206,216,200
213,228,224,230,224
225,237,239,245,244
239,243,242,243,243
238,239,239,240,239
234,234,234,234,234
230,229,227,228,227
224,223,221,218,216
213,211,209,208,207
Done

pito

#4
Mar 29, 2014, 09:48 pm Last Edit: Mar 29, 2014, 09:56 pm by pito Reason: 1
With
Code: [Select]
//for (int j = 0; j < 5; j++) data[j] = analogRead(j);

Code: [Select]

Format (Y/N): Y
clusterCount: 250
Starting test
t: 0.57 sec
minMicros: 40
maxMicros: 124
avgMicros: 42
sync micros: 92
filesize: 120000

pin0,pin1,pin2,pin3,pin4
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
Done


And the same with:
Code: [Select]
if (RamDisk::format(&ram, 16384, 4, 1)) break;

Code: [Select]

Format (Y/N): Y
clusterCount: 16315
Starting test
t: 38.98 sec
minMicros: 40
maxMicros: 124
avgMicros: 42
sync micros: 92
filesize: 8000000

pin0,pin1,pin2,pin3,pin4
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
Done

fat16lib

#5
Mar 29, 2014, 10:01 pm Last Edit: Mar 29, 2014, 10:04 pm by fat16lib Reason: 1
Try running the logger a second time without fomatting.  It will just rewrite the existing file and the maxMicros should be less.

It just needs to  fetch the location of the next cluster, not allocate it.

In the future I will optimize contiguous files.  then all writes will take the same time.

pito

Quote
It will just rewrite the existing file and the maxMicros should be less.

Code: [Select]

Starting test
t: 37.89 sec
minMicros: 40
maxMicros: 80
avgMicros: 41
sync micros: 4
filesize: 8000000

pito

Is it possible to run the "big tests" from SdFat lib?

fat16lib

Just open a file on an SD instead of on the RAM.

Code: [Select]

SdFat sd;
SdFile file;

....

  if (!sd.begin(SD_CS_PIN)) sd.errorHalt();
// rest the same.
  if (!file.open("TEST.BIN", O_CREAT | O_RDWR)) {
    Serial.println(F("open fail"));
    return;
  }

fat16lib

Here are SdFat times.
Quote

Starting test
t: 69.83 sec
minMicros: 40
maxMicros: 61004
avgMicros: 81
sync micros: 2176
filesize: 8000000

minMicros is just the time to copy data to the cache.

fat16lib

SdFat rewrite is bad since you must read a block into the cache update it and write it.
Quote

Starting test
t: 107.11 sec
minMicros: 40
maxMicros: 62024
avgMicros: 127
sync micros: 8
filesize: 8000000

pito

#11
Mar 29, 2014, 10:51 pm Last Edit: Mar 30, 2014, 01:15 am by pito Reason: 1
RamDiskToSdFat_8MB.ino:
Code: [Select]

FreeRam: 15256
Format (Y/N): Y
Writing ramFile
TEST.CSV      2000-01-01 01:00:00 16664
Copying ramFile to sdFile
TEST.CSV       2000-01-01 01:00:00 16664
Done


Test.csv from the sdcard:
Code: [Select]
0,Line,0
336,Line,1
728,Line,2
1128,Line,3
1536,Line,4
..
567696,Line,996
568280,Line,997
568860,Line,998
569444,Line,999


pito

#12
Mar 30, 2014, 01:54 am Last Edit: Mar 30, 2014, 03:11 am by pito Reason: 1
A quick hack of sdfat's bench sketch - bench_8MB.ino:
Code: [Select]
Type any character to start
Free RAM: 15761
File size 5MB
Buffer size 100 bytes
Starting write test.  Please wait up to a minute
Write 745.60 KB/sec
Maximum latency: 212 usec, Minimum Latency: 112 usec, Avg Latency: 129 usec

Starting read test.  Please wait up to a minute
Read 745.41 KB/sec
Maximum latency: 172 usec, Minimum Latency: 120 usec, Avg Latency: 128 usec

Done

Not sure how to interpret the latencies here as they depend on the buffer size..  :~
Code: [Select]
Type any character to start
Free RAM: 11861
File size 5MB
Buffer size 4000 bytes
Starting write test.  Please wait up to a minute
Write 1029.44 KB/sec
Maximum latency: 4120 usec, Minimum Latency: 3816 usec, Avg Latency: 3880 usec

Starting read test.  Please wait up to a minute
Read 992.26 KB/sec
Maximum latency: 4288 usec, Minimum Latency: 3996 usec, Avg Latency: 4025 usec

Done

It seems those are the times for copying to/from buffer, or something like that..

fat16lib

Quote
Not sure how to interpret the latencies here as they depend on the buffer size..


Latency may not be the correct term.  It is just the time to write a buffer so it depends on the buffer size.

These are great times/speeds.  The variation is very small compared to the typical SD in single block mode. 

pito

FYI - this RamDisk filesystem could work nicely with serial SPI FRAM (ie. FM25V10, 256kBytes per chip, 8pin) or serial SPI MRAM (ie. MR25H40, 512kBytes per chip, 8pin) where FRAM and MRAM are nonvolatile.
The performance will be exactly the same as the one with above serial 23LCV1024 SRAM, the driver is basically the same (small change required probably in the addressing sequence).

Go Up