Pages: 1 [2]   Go Down
Author Topic: RamDisk - a file system library for RAM devices like 23LCV1024  (Read 2935 times)
0 Members and 1 Guest are viewing this topic.
Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 212
Posts: 13531
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Congratulations with this impressive work!
(waiting for the RAM-SD shield combination)
Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

0
Offline Offline
Edison Member
*
Karma: 63
Posts: 1604
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
(waiting for the RAM-SD shield combination)
I have a 23LCV1024 on an  Adafruit Data logging shield.  This provides 128 KB of NVRAM by using the shield's CR1220 battery to backup the SRAM. 

The 23LCV1024 requires about 1 uA and the DS1307 on the shield uses about 0.5 uA.

A CR1220 is good for about 35 mah.  At 1.5 uA this is almost 1,000 days.

The 23LCV1024 costs $2.95 from Digi-Key.
Logged

Rapa Nui
Offline Offline
Edison Member
*
Karma: 60
Posts: 2066
Pukao hats cleaning services
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

FYI (digikey, rlx):
Small comparison of "serial" SPI SRAM/FRAM/MRAM prices (single quantity and per MByte).




* memprice.jpg (25.84 KB, 781x128 - viewed 64 times.)
« Last Edit: March 30, 2014, 02:29:55 pm by pito » Logged

Rapa Nui
Offline Offline
Edison Member
*
Karma: 60
Posts: 2066
Pukao hats cleaning services
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Performance (5MB bench test) of the 8MB ramdisk with RamDisk file system - buffer size vs. rd/wr speed vs. "latency - buffer copy time".
Below picture with atmega1284p @16MHz and bitbanging (slow). Will be much better with DUE and its External Memory Bus (8bit static device) - I would guess 11-12MB/sec with DUE is feasible.


* RamDisk driver perf.jpg (37.71 KB, 827x541 - viewed 62 times.)
« Last Edit: April 03, 2014, 03:35:01 am by pito » Logged

Offline Offline
Jr. Member
**
Karma: 0
Posts: 57
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Excellent work!  I got some 23LC1024 on order.  Will give it a try.  The SD Card shield is a perfect place for the ram.

Gerry
Logged

0
Offline Offline
Edison Member
*
Karma: 63
Posts: 1604
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I have attached a new demo release of RamDisk to this reply.  This is a major change since the last version and has a huge amount of new code so I call it a demo, not a beta.

The previous release was very fast for large binary reads/writes but had little gain over an SD for small reads and writes.  Small writes are often required for printing and small reads are required for parsing files.  Also it is nice to have Unix/Linux style character push-back when parsing files.

So:
Quote
RamDisk has been re-factored into several classes.

The key classes are RamVolume, RamBaseFile, RamFile, and RamStream.

The main new features are in the RamStream class which is derived from the RamFile class. The API for this class is based on Unix/Linux stdio streams.

The RamStream class provides an additional level of cache and is based on the Unix/Linux stdio design. The RamStream class is very fast for single byte I/O, formated output, and parsing text input. See the examples for performance tests.

I have also provided the Unix/Linux style character push-back, ungetc, with an definable amount of buffering.

The RamBaseFile or RamFile classes are still fastest for large binary transfers. These classes access the RAM with no intermediate buffering.

Here is the result for single byte IO:
Quote
This example demonstrates the factor of ten speedup provided by RamStream for single byte transfers.

File read/write stats
fileSize: 51000
write millis: 2429
read millis: 2222

Stream getc/putc stats
putc millis: 200
getc millis: 225
streamSize: 51000

Here are the the speedup ratios for formatted output comparing the previous RamFile with the new RamStream class.  The speed-up of 5.6 to 9.5 is due to the new stdio style buffering and use of robtillaart's post based upon the divu10() code from the book Hackers Delight1. (I still need an acknowledgement for  robtillaart in the code.)
Quote
uint8_t 0 to 255, 100 times
fileSize: 116500
print millis: 6776
stdio millis: 711
ratio: 9.53

uint16_t 0 to 18000
fileSize: 114890
print millis: 6226
stdio millis: 749
ratio: 8.31

uint32_t 0 to 18000
fileSize: 114890
print millis: 6221
stdio millis: 852
ratio: 7.30

uint32_t 1000000000 to 1000010000
fileSize: 120000
print millis: 5975
stdio millis: 1061
ratio: 5.63

float nnn.ffff, 10000 times
fileSize: 100000
print millis: 10049
stdio millis: 1723
ratio: 5.83

I plan to make the stdio code generic and port it back into my SD libraries.  It should give a similar speed up for printing and reading files byte-at-a-time.

* RamDiskDemo20140412.zip (451.42 KB - downloaded 25 times.)
Logged

Offline Offline
Jr. Member
**
Karma: 0
Posts: 57
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Regarding latency, what speed is the SPI clock running at?  Can it be increased?
Thanks,
Gerry
Logged

0
Offline Offline
Edison Member
*
Karma: 63
Posts: 1604
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
Regarding latency, what speed is the SPI clock running at?  Can it be increased?

The SPI speed is 8 MHz so it can't be increased.  I am working on reducing latency.  I didn't post the new code yet.

I am rewriting the 23LCV1024 library as a C++ template.  I found that for one or two byte transfers, half of the time is consumed by digitalWrite toggling the chip select pin.  Here is a test sketch that write two byte and then reads them back.
Code:
#include <M23LCV1024.h>
const uint8_t RAM_CS_PIN = 9;

M23LCV1024 ram;
uint16_t wr = 1234;
uint16_t rd = 0;

void setup() {
  Serial.begin(9600);
  ram.begin(RAM_CS_PIN);
 
  uint32_t m = micros();
  ram.write(444, &wr, 2);
  ram.read(444, &rd, 2);
  m = micros() - m;
 
  Serial.print("micros: ");
  Serial.println(m);
  Serial.print("data: ");
  Serial.println(rd);
}
void loop() {}

The current version takes 52 microseconds to transfer 4 bytes.  That's just 77 KB/sec.
Quote
micros: 52
data: 1234

My first cut rewrite runs almost twice as fast.
Quote
micros: 28
data: 1234

I probably won't do much better.  A six bytes transfer on the SPI bus is required to read or write two bytes.  A command byte, three address bytes, and the two data bytes.  So there is actually 12 bytes transferred.

This new driver does not change the basic results for the RamStream class.  Here are the results of the single byte read/write test with the new driver.  I removed code that verified correct read data values for this timing.  This is now a major effect and not part of the read time.
Quote
File read/write stats
fileSize: 51000
write millis: 1871
read millis: 1610

Stream getc/putc stats
streamSize: 51000
putc millis: 187
getc millis: 177
Done!

So RamStream is still about ten times faster.

I am now taking a break from design and implementation of features and trying to improve performance.
Logged

Rapa Nui
Offline Offline
Edison Member
*
Karma: 60
Posts: 2066
Pukao hats cleaning services
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

FYI - these are results with 8MB Ramdisk1.1 (atmega1284p @16MHz, 3.3V), driver bitbanging (see above). Mind the bitbanging is slow with atmega.  The ramdisk performs best as a block device, with larger chunks to be wr/rd. External Memory Bus on Due might be a better choice.
Code:
FreeRam: 15795
Format (Y/N): Y

File read/write stats
fileSize: 51000
write millis: 1680
read millis: 1454

Stream getc/putc stats
putc millis: 147
getc millis: 181
streamSize: 51000
Done!
--------------------------
Type any character to start
uint8_t 0 to 255, 100 times
fileSize: 116500
print millis: 5619
stdio millis: 588
ratio: 9.56

uint16_t 0 to 18000
fileSize: 114890
print millis: 5383
stdio millis: 627
ratio: 8.59

uint32_t 0 to 18000
fileSize: 114890
print millis: 5377
stdio millis: 731
ratio: 7.36

uint32_t 1000000000 to 1000010000
fileSize: 120000
print millis: 5459
stdio millis: 932
ratio: 5.86

float nnn.ffff, 10000 times
fileSize: 100000
print millis: 8857
stdio millis: 1618
ratio: 5.47

Done
------------------------
Format (Y/N): Y
clusterCount: 250
Starting test
t: 7.32 sec
minMicros: 40
maxMicros: 120
avgMicros: 41
sync micros: 100
filesize: 120000

pin0,pin1,pin2,pin3,pin4
0,0,0,0,13
0,0,0,0,1
0,0,0,0,0
0,0,0,0,1
0,0,0,0,1
0,0,0,0,0
0,0,0,0,0
0,0,0,0,13
0,0,0,0,17
0,0,0,0,68
Done
---------------------
FreeRam: 14772
Format (Y/N): Y

Special float values differ:
Arduino -INFINITY, -NAN, -1e11, 1e11, NAN, INFINITY
inf
nan
ovf
ovf
nan
inf
Stdio -INFINITY, -NAN, -1e11, 1e11, NAN, INFINITY
-INF
NaN
-OVF
OVF
NaN
INF

fileSize: 406
files verify
Done![/size]
« Last Edit: April 12, 2014, 06:38:37 pm by pito » Logged

0
Offline Offline
Edison Member
*
Karma: 63
Posts: 1604
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

pito,

Thanks for the info on your 8 MB RAM. 

It is very helpful to see results for a second type of RAM.
Logged

0
Offline Offline
Edison Member
*
Karma: 63
Posts: 1604
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

An updated version, 20140429, of RamDisk has been added to the original post http://forum.arduino.cc/index.php?topic=229485.msg1656674#msg1656674.

See the attached file RamDisk20140429beta.zip.

The new version adds support for MB85RS2MT FRAM and configurations with multiple chips.  Several new test examples have been added and a number of features have been added or changed.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 3
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

@fat16lib

Thanx for this lib. I will try them out at the weekend.

Will you add support for a bcd decoder like HC138 to save some pins?
Logged

0
Offline Offline
Edison Member
*
Karma: 63
Posts: 1604
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
Will you add support for a bcd decoder like HC138 to save some pins?
The RAM driver library is separate from the RamDisk library.  You can either modify one of the sample RAM driver libraries or provide your own library.

Here is the RAM driver interface which has three functions, read, write, and total RAM size as the number of 512 byte blocks.
Code:
class RamBaseDevice {
 public:
  /**
   * Read data from RAM.
   *
   * \param[in] address Location in RAM to be read.
   *
   * \param[out] buf Pointer to the location that will receive the data.
   *
   * \param[in] nbyte Number of bytes to read.
   *
   * \return true for success or false for failure.
   */
  virtual bool read(uint32_t address, void *buf, size_t nbyte) = 0;
  /**
   * \return Total number of 512 byte blocks in the RAM device.
   */
  virtual uint32_t sizeBlocks() = 0;
  /**
   * Write data to RAM.
   *
   * \param[in] address Location in RAM to be written.
   *
   * \param[in] buf Pointer to the location of the data to be written.
   *
   * \param[in] nbyte Number of bytes to write.
   *
   * \return true for success or false for failure.
   */
  virtual bool write(uint32_t address, const void *buf, size_t nbyte) = 0;
};

A typical chip select function in one of the RAM drivers looks like this:
Code:
  void csLow() {
    cli();
    *m_csPort[m_curChip] &= ~m_csBit[m_curChip];
    sei();
  }
You could just send m_curChip to the decoder instead of using it as an array index for the chip select pin's PORT and bit arrays.
Logged

Pages: 1 [2]   Go Up
Jump to: