next topic »

[on:]

Hi,

I need to store for my program large amount of sensor readings ( 3 000 - 4 000) temporarily so I can do some analysis after that. I can't store them in RAM since it's only 1 KB big nor in EEPROM since it's only 512 KB big (I need 7-8 KB). So, flash memory would be appropriate but you can store there only static information (right?) so that doesn't help either. Any ideas? I'm really stuck here (

Thanks

[Reply #1 on:]

but you can store there only static information (right?)

Flash memory is read only, and is populated by the loader when the sketch is compiled, linked, and uploaded. So, no that is not a viable option.

Any ideas?

Store the data on an SD card, or take fewer readings, process those, then take some more. Repeat as needed.

[Reply #2 on:]

Thanks, I suppose I'll have to go with fewer readings then, SD card is not an option

Cheers

[Reply #3 on:]

What is the range of the values of the integers? does it fit in one byte  (0..255) ? would make 50% difference.

It is technically also possible to send the data over wifi / serial / I2C / Xbee etc to a storage server. is tha feasable?

[Reply #4 on:]

Thanks, I suppose I'll have to go with fewer readings then, SD card is not an option

Cheers

Why is an sd card not an option? Speed?
If so have you considered parallel flash memory?
Or maybe a cf card interface?
Are already using all your pins?
if you are already using the hardware spi then you can plug your device and the sd card and the device and use chip select to select between the device and the sd card.
if you are using all your pins and this is not because of already using the hardware spi have you conisdered an i/o expander?

[Reply #5 on:]

How do i read csv from sdcard ? as i know, it read certain pattern only

something look like this ,the modified code from sdfatlib :

Code:

sdin >> f1 >> c1

for 1,2,3,...,N

but it read only 1,2,3,... but in the last number it failed

do i have to read through byte , gain it to char[number of array] then convert from array of char to int

[Reply #6 on:]

Just repeating what robtillaart asked, What is the range of the values of the integers?

______
Rob

[Reply #7 on:]

i'm want to use 16 bit int , number of integers : 256*256 , row*line

The csv is 2d array And want to read only 1 line for calculation , so only 256

[Reply #8 on:]

So that's only 512 bytes then, stick 'em in RAM.

_____
Rob

[Reply #9 on:]

What is the range of the values of the integers? does it fit in one byte  (0..255) ? would make 50% difference.

It is technically also possible to send the data over wifi / serial / I2C / Xbee etc to a storage server. is tha feasable?

it seems we think alike

it doesn't fit but I made all values greater than 255 to be 255 so it does fit now, although it's not perfect solution but it helps

it's possible to send over Xbee but that's simply too slow for me, and I think SD card is also that's why I said it's not an option... I need to have a sensor reading at least (!) every 3-4 ms, is it possible to store data to SD card that fast?

but in the end I think I'll manage to store everything I need with byte array solution and less readings

[Reply #10 on:]

Trunking values above 255 to 255 is a way to byte-ify the dataset. you keep the precission at cost of range

Scaling is another way, you keep the relative range at the cost of precission.

int maxValue = 768; // just an example
long x = readData(); // the long is to big to hold the int but

byte b = (x*255+127)/maxValue;

Please post your code, maybe there are more ideas...

[Reply #11 on:]

Or get a bigger uC:
'1284 has
4K Bytes EEPROM
16K Bytes Internal SRAM

make a big RAM buffer with that.

[Reply #12 on:]

Please post your code, maybe there are more ideas...

my code is still a bit messy, I'll do few changes in following days and inform you about progress and post code if needed

I like this scaling idea though, have to think about it, seems as better solution for me since I have to seperate high sensor readings from lower ones and it seems easier (more accurate) this way, hmmm

what I'm doing is sending bits (128bit key) over optical channel ("1" is light on and "0" is no light, actually better fits "less light" ) and detecting them with arduino

p.s. appreciate all the help

[Reply #13 on:]

Or get a bigger uC:
'1284 has
4K Bytes EEPROM
16K Bytes Internal SRAM

make a big RAM buffer with that.

I'm doing this for my graduate thesis and already have 5 Arduino Unos that I need to use

[Reply #14 on:]

already have 5 Arduino Unos that I need to use

You could make a RAIA (Redundant Array of Inexpensive Arduinos)