storing high-sampling rate data the size of which is grater than MCU's Memory

Official Hardware MKR Family MKR GSM 1400
1

I have a urgent problem and would be so tankful for any help.

The hardware I work with is MKR GSM 1400. I want to send the data I get from ADXL355 (acceleration sensor) to a web-server. each hour, I need to start sampling for one minute continuously. sampling rate is 250 Hz. I thought that maybe the most quickly way is that each time I read data from the sensor, I convert it to a string and concatenate it to a long string variable and then, I can sent the data to the web-server. My problem is that the size of the all samples in one minute is at least 350 KB ; on the other hand the size of the MCU's memory is 256KB. I think I also can not use SD Card because I have seen on the internet at least 200ms is needed for writing data into the SD Card.

Question 1: Do you think writing I2c flash memory or EEPROM can be performed less than 4ms? because I just have 4ms time to store the data before the next one will be ready. I only have I2c pins free to use.

Is there any other solution for this problem ?

2

I believe the 200ms write time is for a 4k block of data so you should have time to write one block whilst the next is being collected.

For a while at least - flash memory has a limited number of write cycles per block so you’d need to use a large sd card and work your way through it.

You might also consider looking for ways to reduce your data size by compressing it in some way.

There are many possible algorithms for on the fly compression but, depending on the nature of your data you might be able to get away with something relatively simple.

Given the nature of your data I suspect run length encoding would be a non-starter but have you considered storing the difference between successive samples rather than the actual raw data?

You could, for example, squash your 3 byte samples down to (almost) 2 by storing each of your x, y and z values as a 4 bit signed offsets and using the remaining 4 bits to indicate occasions where one or more of the offsets is too big and is in its own byte.

If your data doesn’t change too rapidly you might even get away with compressing it down to a single byte by using 2 bit offsets and with the remaining 2 bits used to indicate where the offset is larger and you’re storing the whole value.