Show Posts
Pages: 1 2 3 [4] 5 6 ... 107
46  Using Arduino / Storage / Re: SDfat slow buffer flush on: June 21, 2014, 07:24:19 am
Quote
I miss capturing data during that period.
The problem is that write latency for SD cards can occasionally be very long.  This is caused by various housekeeping functions in the SD card.

SD cards must erase flash before writing.  SD cards perform wear leveling to increase the life of the SD card.

You must separate data capture from writing to the SD card and buffer data waiting to be written.  

You can capture data in an interrupt routine  like this example http://forum.arduino.cc/index.php?topic=228549.0 .

You can use a small RTOS like the nilSdLogger example in NilRTOS https://code.google.com/p/rtoslibs/downloads/list     NilRTOS20130720.zip.

Your current  program will have so much time jitter that there is no point in logging data this fast.  The information in a time series depends as much on the accuracy of the time of a data point as the accuracy of the value.

Many articles have been published about time jitter noise.  Google for "time jitter noise adc" for articles relevant to logging analog data.

If your data set is small, you could use a RAM disk http://forum.arduino.cc/index.php?topic=229485.0 or http://forum.arduino.cc/index.php?topic=236038.0.  You will still need to solve the time jitter problem.
47  Using Arduino / Storage / Re: SD card doesn't work... most of the time on: June 20, 2014, 02:23:26 pm
Many other users have similar problems with the LC studio module. 

Buy a new SD module or search the forum for instructions on installing level shifters on the LC Studio module.
48  Using Arduino / Storage / Re: SDFat is sooo sloooow !!! How to speed-up? on: June 18, 2014, 06:46:22 pm
Quote
PS : Yes, FRAM looks extremely promising for embedded... But it's hard to replace Flash price and density!

Random access to FRAM can be 1000 times faster than to an SD, microseconds vs millisecond.  Some users are combining FRAM and an SD to achieve far better performance than you will achieve with only an SD.

256 KB could provide much of your tree structure.  A single 256 KB chip is about $7.50.
49  Using Arduino / Storage / Re: SDFat is sooo sloooow !!! How to speed-up? on: June 18, 2014, 10:55:10 am
Here is example code to replace your while loop.

I ran it with an three folders in the root directory, the folders were 1, 4, and 9.

Code:
#include <SdFat.h>
SdFat sd;
uint8_t sdChipSelect = SS;

bool tree_path[10][10];
void example() {
  int pos = 0;
  int level = 0;
  dir_t dir;
  for (pos = 0; pos < 10; pos++) tree_path[level][pos] = false;

//---------------------------------------
  bool level_empty = true;
  sd.vwd()->rewind();
  while (sd.vwd()->read(&dir, sizeof(dir)) == sizeof(dir)) {
    // done if never used
    if (dir.name[0] == DIR_NAME_FREE) break;
    // skip deleted entry and entries for . and  ..
    if (dir.name[0] == DIR_NAME_DELETED || dir.name[0] == '.'
      || !DIR_IS_FILE_OR_SUBDIR(&dir)) continue; 
    pos = dir.name[0] - '0';
    Serial.write(dir.name, sizeof(dir.name));
    Serial.println();
    if (pos >= 0 && pos <= 9) {
      tree_path[level][pos] = true;
      level_empty = false;
    }
  }
  //----------------------------------

  for (pos = 0; pos < 10; pos++) {
    Serial.print(tree_path[0][pos]);
  }
  Serial.println();
}
void setup() {
  Serial.begin(9600);
  Serial.println("Type any char to run example");
  while (!Serial.available());
  if (!sd.begin(sdChipSelect))sd.errorHalt();
  example();
}
void loop() {}
it prints:
Quote
Type any char to run example
1         
4         
9         
0100100001

You may want to replace
Code:
      || !DIR_IS_FILE_OR_SUBDIR(&dir)) continue; 
with
Code:
      || !DIR_IS_SUBDIR(&dir)) continue; 
then only directory files will be checked.
50  Using Arduino / Storage / Re: SDFat is sooo sloooow !!! How to speed-up? on: June 17, 2014, 06:28:27 pm
Quote
BTW, any chance to at least speed-up the listing?
Just open the directory file and read it.  It is just like any other file (read only) with 32 byte dir_t structs.

Quote
PS : the "tree idea which goes back to the beginning of time " is still in use
I know.  Classic data structures will always have there place when the storage device can support them.

I have used FRAM chips on Arduino for random access storage.  These are probably too small, 256 KB, to help you.  FRAM chips are byte addressable and great for linked lists.
51  Using Arduino / Storage / Re: SDFat is sooo sloooow !!! How to speed-up? on: June 17, 2014, 05:32:36 pm
Quote
I know there would not be more than 10 branches at each level: can this be used to create smaller directories?
Is there a formatting option or a new function to write that would help in this process?

The file system on SD cards is designed to match the flash chips used in the SD. Modern SD cards have huge flash erase groups so using smaller clusters won't help much and sometimes makes matters worse.  

When you do a random write to a block on an SD card, the SD flash controller may move up to sixteen existing blocks to a new erased flash block together with your new block.  This take a long time.

When you do sequential writes to an SD, the SD has enough RAM buffer to avoid flash erases and data movement.

In short your tree idea which goes back to the beginning of time is not practical on Arduino with an SD card.
52  Using Arduino / Storage / Re: SDFat is sooo sloooow !!! How to speed-up? on: June 17, 2014, 04:42:12 pm
Quote
Is there any way to speed-up the directory creation?
If  the intermediate directories do not exist this will take a long time.   Creation of a directory on a FAT volume requires zeroing a cluster which is likely to be 64 blocks.   You have about 20 levels so this will require writing about 1200 blocks.

Access to the FAT and writing directory entries takes more time.

Finally the random access pattern to the SD card will require lots of flash erase delays.

PCs, Macs, and phones have lots of buffer and can cache blocks so you don't have this delay, often 50-100 ms.

The Arduino can only cache a single block so it is very slow for file system operations like the above.  Many file system blocks will be read, updated, and written multiple times for the above operation.

Directory operations will also be slow if you have lots of entries in a directory since directories are not indexed.
53  Using Arduino / Storage / Re: SDFat limitation? Trying to copy a file into another, line by line... on: June 16, 2014, 02:16:25 pm
Quote
NOTHING is found in the temporary file!!!
This happens when you don't close the file.  The directory is not updated and clusters allocated to the file are lost.

Quote
So I presume that I burst an internal buffer.
No this program should not cause any buffer problems in SdFat.

Common causes of crashes are writing over memory and using too much memory.  Since you didn't include the entire sketch, I can't look for problems.

I assume that global_sd_buf has a dimension of MAX_SD_LENGTH.

Another possibility is trimwhitespace. I assume it does not use String since String often causes crashes in programs due to dynamic memory use.

You can remove the last '\n' by saving the value returned by fgets like this:
Code:
  size_t n;
  while ((n = list_file.fgets(global_sd_buf,MAX_SD_LENGTH,"\n")) > 0) {
  ...
    if (global_sd_buf[n] == '\n') global_sd_buf[n] = 0;

I suggest you put this line at the end of the while loop:
Code:
  tmp_file.sync()
This will cause data in tmp_file to be saved and you can check what was last written to tmp_file.  See if it is always the same line.
54  Development / Suggestions for the Arduino Project / Re: IDE included libraries on: June 14, 2014, 09:07:35 am
You should not need to download files like io.h.  This file should be included from here by the compiler.

arduino-1.5.6-r2/hardware/tools/avr/avr/include/avr
55  Development / Other Hardware Development / Re: Anyone tried ST Nucleo boards? on: June 14, 2014, 08:53:38 am
I have started using STM32CubeMX to configure Nucleo projects and several other STM32 boards. STM32CubeMX  is a big improvement over previous ST configuration  tools.  http://www.st.com/web/catalog/tools/FM147/CL1794/SC961/SS1743/PF259242

 STM32CubeMX is a graphical configuration tool that automates configuration and generation of STM32 initialization C code.

The STM32CubeMX  pin-out wizard assists pin assignment to avoid conflicts and has a constraints solver.

The clock-tree wizard assigns clocks and performs dynamic validation. The user interface is a diagram of the clock tree and you just edit this diagram.

The peripherals-and-middleware wizard aids configuration to avoid unusable settings. 

The power-consumption wizard estimates the application's power usage.

STM32CubeMX creates a project with the generated start-up code and copies needed library code to the project.  Several people have posted Makefiles templates to build projects generated by STM32CubeMX.  There is also a eclipse plugin.

I also like the STMStudio debug tool http://www.st.com/web/en/catalog/tools/PF251373.  STMStudio reads the elf file generated by the linker to find and displays variables in real-time.

STM Studio reads data through the ST-Link SWD debug port so it does not interfere with the real-time behavior of applications. 
56  Development / Other Hardware Development / Re: Anyone tried ST Nucleo boards? on: June 12, 2014, 08:50:23 am
The Nucleo F401 board arrived.  I spent about two hours playing with it.  So far I like it more than any other ST demo board.

I loaded the USB driver and the board appears as at least three devices. 

Flash can be accessed as a 512 KB mass store device.  I think this is mainly for flash programming by Mbed.

One of the Serial ports shows up as a COM port like Arduino.

An ST-link debug port can be used to load programs, set break points and observe RAM during execution.

Here is how I did my first blink program, very bare metal.  I attached a .zip with the "project files".  I used no fancy tools, just a .bat file to build the hex.  Amazing how simple this was.

Download and installed Gnu Arm tools (gcc-arm-none-eabi-4_8-2014q1-20140314-win32.exe)
https://launchpad.net/gcc-arm-embedded

Download STM32Cube firmware for STM32 F4 and unzipped it in a folder named STM32CubeF4Root.
http://www.st.com/web/catalog/tools/FM147/CL1794/SC961/SS1743/PF259243

Download and installed ST-Link USB driver and ST-Link utility.
http://www.st.com/web/en/catalog/tools/FM146/CL1984/SC720/SS1450/PF251168

Use this Linux article as a guide to make a bare metal blink program on Windows 7.
http://regalis.com.pl/en/arm-cortex-stm32-gnulinux/

Built the hex file with a .bat file, build.bat.

Load the program with the ST-Link utility.

This is true bare metal. The attached zip file has all code for blink.  Just three source files are compiled to .o files and linked.  No other ST firmware files are linked.

The three source files are:

startup_stm32f401xe.s
system.c
main.c
57  Using Arduino / Storage / Re: help pass cardinfo, error card write arduino mini pro on: June 11, 2014, 06:19:02 am
Post your program.

Modifying the card CardInfo program to access files is not a good approach.  You should start with another example.
58  Using Arduino / Storage / Re: Reading analog input at high frequency using arduino mega 2560 and SD card on: June 11, 2014, 06:14:31 am
See this http://forum.arduino.cc/index.php?topic=228549.0.
59  Development / Other Hardware Development / Re: Anyone tried ST Nucleo boards? on: June 10, 2014, 09:13:17 am
Quote
Not to sure about mbed environment though.

ST supports Mbed so users can try STM32 without installing a development environment.

Just use a standard ARM tool chain like the free Sourcery CodeBench Lite Edition including ARM GCC IDE http://www.mentor.com/embedded-software/sourcery-tools/sourcery-codebench/editions/lite-edition/.

I use CodeBench on Windows.

Most Linux distributions have the tools.  Here is how to setup Nucleo on Linux http://regalis.com.pl/en/arm-cortex-stm32-gnulinux/
60  Development / Other Hardware Development / Anyone tried ST Nucleo boards? on: June 10, 2014, 08:29:52 am
STMicroelectronics has produced a series of interesting STM32 boards with Arduino Uno style headers.

Has anyone tried these boards?

This board costs $10.33, has 512 KB flash, 96 KB SRAM and runs at 84 MHz  http://www.digikey.com/product-detail/en/NUCLEO-F401RE/497-14360-ND/4695525.

You only need a min USB cable to use the board since it has an integrated ST-LINK/V2-1 debugger/programmer.

Looks like ST will use this form factor for many more boards.  Here is a brief for the first nine http://www.st.com/st-web-ui/static/active/en/resource/technical/document/data_brief/DM00105918.pdf.

I ordered two of the STM32F401 boards.  Here is the user guide http://www.st.com/st-web-ui/static/active/en/resource/technical/document/user_manual/DM00105823.pdf.
Pages: 1 2 3 [4] 5 6 ... 107