Show Posts
Pages: 1 2 3 [4] 5 6 ... 110
46  Development / Suggestions for the Arduino Project / Re: Does Arduino have a future? on: July 14, 2014, 11:48:59 am
While I agree that micontrollers will have a place for quite some time, I'm not as
sure about the 8 bit AVRs.
I started to use Arduino because I wanted an easy way to work with microcontrollers. 

I started using Arduino in 2008 and wrote the Fat16 library for the 168 Arduino with 16 KB flash and 1 KB SRAM.  It was just a toy to play with.

Now I want a microcontroller like Cortex M4 with more power than AVR but not an application processor designed for "hosting a rich OS platform" like the Cortex A series.

I have been developing a new FAT file system using STM32F4 processors.  The difference in capability over AVR is overwhelming.  I tried to love the Arduino Due when it came out but couldn't.

Here are two examples. 

The ADC on AVR Arduinos run at a max of about 10 ksps with analogRead.  You can increase the clock speed and get fair, not full 10-bit, accuracy at 40 ksps.  Low end STM32F4 ADCs can do 2.4 msps, about 60 times faster than AVR with 12-bit accuracy and very flexible DMA.  High end STM32F4 chips can run three ADCs interleaved for 7.2 msps.

I am developing a new file system for high end embedded processors.  The STM32F4 has a good, not great, SDIO controller which can run the 4-bit bus at over 48 MHz.  I have achieved sustained read speeds over 20 MB/sec.  I have sustained write speeds of 12 MB/sec but this is a limited by the SD card I am using.  I have ordered a new Samsung Pro card and hope to write at close to 20 MB/sec.

The AVR can never read or write faster than about 500 KB/sec that's about the best you can do with program I/O on the 8 MHz SPI bus.  In addition SD cards don't present their fast commands on the SPI bus, only on the SDIO bus.  This means you will have very long occasional write latencies with SD cards on the SPI bus.

The Arduino system software is not adequate for these new chips. You really need priority based pre-emptive scheduling to take full advantage of the features of these chips.  Most chip manufactures are now providing a free RTOS solution.  ST and Atmel provide FreeRTOS ports, Freescale has MQX, and there are other great systems like ChibiOS.

I really enjoy development with boards like the  STM32F4 Nucleo boards.  ChibiOS can do about 1.2 million context switches per second on the NUCLEO-F411RE.  I am using a Olimex stm32-H407 and it can do a context switch in 400 ns.  The 407 has 196 KB SRAM and 1 MB flash.
47  Using Arduino / Storage / Re: 0X6 and 0X4 errors in ReadWriteSdFat on: July 13, 2014, 01:17:57 pm
The error probably happened in the close before the re-open for read.  The error is a write error and likely happened when buffers were flushed in the close.
// close the file:
  myFile.close(); // <--------------------Error here

  // re-open the file for reading:
  if (!"test.txt", O_READ)) {
    sd.errorHalt("opening test.txt for read failed");

I didn't add a check for close failure when I modified the Arduino SD.h example.

I use an LC Studio SD shield
This module is very unreliable.  Search the forum for articles about this module.  It can be fixed but it is not worth the time and money.
48  Using Arduino / Storage / Re: fastest data logging option ? on: July 12, 2014, 06:51:32 pm
The 23LCV1024 has a battery backup pin so it can retain data

Another option is FRAM
49  Using Arduino / Storage / Re: Reading 250,000,000 values from sensor and storing it on SD card on: July 09, 2014, 07:05:36 am
I ve also got Raspberry pi if it is easier on that..though i ve never used it till now .. !!

The Pi does not include a hardware analog to digital converter, but a external ADC (such as the MCP3008) can be used along with some bit banged SPI code in python to read external analog devies. Here is a short list of some analog inputs that could be used with this setup.

Raspberry Pi won't come close for your needs.
50  Using Arduino / Storage / Re: Reading 250,000,000 values from sensor and storing it on SD card on: July 08, 2014, 02:20:35 pm
Here is the ($24 from Future Electronics)  STM32F429 board I am using

It's almost trivial to solve your problem with this board if you are a skilled STM32 programmer.

The board has an 8MB RAM device that is accessible by the DMA controller.  You just setup a timer, the ADC, and a DMA channel and acquiring up to 4 million samples is almost automatic.  The board has three ADCs and each ADC can digitize samples at 2.4 million/sec.

The three ADCs can run in interleaved mode to digitize one channel at 7.2 million samples/sec.

Too bad you don't have the skills to use this board.  It kills Due at 180 MHz,  2 MB flash, 256 KB fast RAM, and 8 MB SDRAM.
51  Using Arduino / Storage / Re: can arduino read current size/capacity of one SD card? on: July 08, 2014, 01:56:10 pm
You really can't get the size of a card with SD.h and be able to write to the card.  SD.h is a wrapper for a very old version of SdFat and CardInfo uses the underlying SdFat library directly.  Direct use of the underlying SdFat functions is not compatible with use of SD.h in a single program.

SdFat allows concurrent use of these functions.  You can get both the total number of blocks in the card and the number of free clusters remaining in the volume.  Getting the free space can be slow since the entire File Allocation Table must be scanned.

The SdFat SdInfo example prints the free space and you can see how it is done and how long it takes.  

A pointer to the volume object is available as a member function in the SdFat class.
52  Using Arduino / Storage / Re: Reading 250,000,000 values from sensor and storing it on SD card on: July 08, 2014, 11:40:53 am
ok.. i ll read examples of analog data logger and SdFat library and then code
The SdFat examples won't help the data logger example is only good for about one sample per second.

You want to take data 50,000 times faster.  This is the difference between a cow walking around in a pasture and a Mars or Moon lander at max speed.  I don't think your cow will jump the Moon.  You can't do this with stock Arduino programming.

This article is for the AVR ADC so doesn't apply to Due.  The Due ADC is over 20 times faster than the AVR ADC.   

The problem is that the Arduino Due analogRead function will never be adequate for the time jitter you need.  You need a lot more than reading a few hobby level Arduino programs.

53  Using Arduino / Storage / Re: Reading 250,000,000 values from sensor and storing it on SD card on: July 08, 2014, 09:38:43 am
ohh...Ok.. I ll check that signal and noise issues ..
but why is the outer loop running twice as many times and giving junk values?

Your program still had lots of issues like initializing the SD card in the loop.  You should not be opening and closing the file the way you are.

You have now change loop indices to short int.  short int can only range from  –32,768 to 32,767.  Who knows what this is doing  Use size_t for things like loop indices.

Why are you pursuing this junk program?  The current structure will never be satisfactory.
54  Using Arduino / Storage / Re: Reading 250,000,000 values from sensor and storing it on SD card on: July 08, 2014, 09:12:33 am
I can take 48000 samples in a second through arduino DUE using type using short int..

Of course you are building a very complicated noise generator if you pursue your current program.  You must study signal to noise issues for time sequences taken with an ADC.  At 50,000 samples per second you need time jitter in the nanoseconds.

The formula for maximum time jitter is:
t = 1/(3.14*f*2^(n+1))
Where f is the sampling frequency and n is the number of bits of accuracy for the ADC.

For f = 50,000 and n = 10 bits:

t = 3.11 nanoseconds

Fast data acquisition is not about coding but rather understanding the theory.

You have no idea what things like system tick and SD random delays will do.

i ll need 1 million samples atleast..and i would need same 50,000 samples in one second continuously
You don't need a micro-controller, You need a computer.

This can be done with a modern ARM Cortex M chip but you need need good design and an RTOS would help.  I am experimenting with a fast STM32 chip with a 2.4 MSPS ADC.  I can trigger the ADC using a timer so jitter is less than one cycle for a 168 MHz CPU.  the ADC can be used with a nice multi-buffered  DMA controller and I can write the data to an SD easily in the background without interfering with the ADC.
55  Using Arduino / Storage / Re: Reading 250,000,000 values from sensor and storing it on SD card on: July 08, 2014, 05:30:34 am
  i ll need 1 million samples atleast..and i would need same 50,000 samples in one second continuously
There is no point in logging a time series of one million samples in a program like yours.  The information content of a time series depends as much on the time a point is logged as the value.  There are mathematical treatments of the information content of such a time series but I won't go to that level.  Just think of plotting a two dimensional curve in the x-y plane.  It won't be correct unless you know both x and y for each point.

SD cards have random write delays of up to 250 milliseconds so you may have huge time uncertainty in your data.

Here is a program that triggers the ADC with a timer to eliminate time jitter, reads the ADC in an interrupt routine to avoid data loss, buffers the data for a background process that writes it to an SD

The above logger is for an AVR Arduino.  You need something like this for Due.

I tried writing O_APPEND to continue writing values ..but same result..
remove the O_TRUNC, it truncates the file to be empty.
if(!"pleasework.txt", O_WRITE|O_CREAT|O_TRUNC))

is there any command to directly write the values stored in the array to the card without using the loop i.e writing all value at a time instead of writing them one by one..
Write the entire array in binary and convert the file to text later.  Make your buffer a multiple of 512 bytes, the size of an SD block.  The optimum size is 32KB, the cluster size on a properly formatted SDHC card.
int SdFile::write (const void* buf, size_t nbyte )       
Write data to an open file.

    Data is moved to the cache but may not be written to the storage device until sync() is called.

    [in]   buf   Pointer to the location of the data to be written.
    [in]   nbyte   Number of bytes to write.

Don't use an array of int since your data is under 16-bits, use a 16-bit short array.  This will save memory and reduce the size of the binary file.
56  Using Arduino / Storage / Re: Fast data logger for multiple analog pins on: July 07, 2014, 04:29:10 pm

The AVR ADC can log one pin at 100,000 samples per second but accuracy is reduced to under 8-bits.  It is not possible to log faster.

It is possible to log much faster with modern ARM devices using DMA.  I have been experimenting  with STM32 chips with a 2.4 MSPS ADC.  This ADC can automatically log multiple channels using DMA at (2,400,000/<number-of-channels>). per second.

Edit: Are you sure you are not missing points at 100,000 samples per second?  How are you doing the 100,000 samples per second?
57  Using Arduino / Storage / Re: Reading 250,000,000 values from sensor and storing it on SD card on: July 07, 2014, 10:36:49 am
The times make no sense because the array of 24,000 int uses too much memory.  This program writes over some system memory.

It has a lot of other problems.

Here is a run with z[20000] and a few fixes:
acquisition done
Card Ready
start time=84873
total aquisition time=376344
Done.. PARTY
acquisition done
Card Ready
start time=553122
total aquisition time=400856
Done.. PARTY
acquisition done
Card Ready
start time=1045464
total aquisition time=399503
Done.. PARTY

58  Using Arduino / Storage / Re: Reading 250,000,000 values from sensor and storing it on SD card on: July 07, 2014, 10:04:50 am
The code you just posted compiles with no errors on my Due.

Edit: it even runs after I change the file name to 8.3 format:
acquisition done
Card Ready
start time=
83133total aquisition time=
484628Done.. PARTY
acquisition done
Card Ready
start time=
83647total aquisition time=

The linker must remove the global z[24000].
59  Using Arduino / Storage / Re: Reading 250,000,000 values from sensor and storing it on SD card on: July 07, 2014, 09:01:32 am
C++ is amazing.  

Here is what the braced-init-list does when there is no type before n. (from the ISO C++ 2013 standard).

A braced-init-list may appear on the right-hand side of
β€” an assignment to a scalar, in which case the initializer list shall have at most a single element. The
meaning of x={v}, where T is the scalar type of the expression x, is that of x=T{v}. The meaning of
x={} is x=T{}.
β€” an assignment to an object of class type, in which case the initializer list is passed as the argument to
the assignment operator function selected by overload resolution (13.5.3, 13.3).
[ Example:
complex<double> z;
z = { 1,2 }; // meaning z.operator=({1,2})
z += { 1, 2 }; // meaning z.operator+=({1,2})
int a, b;
a = b = { 1 }; // meaning a=b=1;
a = { 1 } = b; // syntax error
β€”end example ]
So PaulS is correct,
n[24000] = {0};
just zeros the four bytes after n.  A clear error.

Edit: Since the Due has 96 KB of RAM (98,304 bytes)  and this array takes 96,000 bytes, not much memory is left.
n = {0};  // zero whole array.
Of course n, as PaulS said, was already initialized to zero.
60  Using Arduino / Storage / Re: rewindDirectory doesn't? on: July 06, 2014, 05:15:39 pm
I think the arduino SD reference example also omits the file close(); call too.

For years I have been pointing this out but the Arduino company is slow to fix things.
Pages: 1 2 3 [4] 5 6 ... 110