Show Posts
Pages: 1 ... 63 64 [65] 66 67 ... 112
961  Using Arduino / Storage / Re: Unformatted write to SD on: August 20, 2012, 06:22:17 pm
the quotes are by Walt Kester he is the recognized authority on converters.  His book is the standard reference http://www.amazon.com/Data-Conversion-Handbook-Analog-Devices/dp/0750678410.

Actually the 3LSB offset will reduce ENOB.  The reason is that ENOB is based on how well an ADC digitizes a full scale sine wave.  The 3LSB offset error will distort the wave for low values, returning zero when the voltage is not zero.

That's why ENOB is so useful.  Almost any fault in an ADC will lower ENOB.

So I will stick with people like Walter Kester for information on ADCs, not your guesses.

Quote
That is why I suggested that, since you seem to have got a very clean signal, a suitable cyclic perturbation signal would be better than arbitrary, uncharacterised noise.
You don't listen.  I said I don't do oversampling, I get a better converter.  I never suggested a way to inject noise for oversampling.  I just said oversampling won't work because the noise levels are so low.

I played with resistors and pickup by longer wires between the source and ADC to see how sensitive it was.

Adding a cyclic perturbation signal seems like a really bad idea.  You should just spend $5 for a better ADC.

Quote
This is so basic, I don't see why I am having to lay this out for a third time.
I ignore you because what you are saying is at odds with recognized authorities. 

You don't even know what the definition of resolution is for a ADC.  Show me a link to a definition where the it's other than the number of bits output by the converter.

I have spent my career at the finest science labs in the world.  Some of the world's best analog and mixed signal engineers are in these labs.  As a scientist I depended on them and learned from them.

Who are you?  What is the basis of your authority?   You never answered whether you are really an EE.
962  Using Arduino / Storage / Re: Unformatted write to SD on: August 20, 2012, 04:04:23 pm
This a quote from Analog Devices (not mine).
Quote
ENOB specifies the number of bits in the digitized signal above the noise floor, this is accuracy.
Here is another definition, not mine:
Quote
The effective number of bits (ENOB) is a way of quantifying the quality of an analog to digital conversion. A higher ENOB means that voltage levels recorded in an analog to digital conversion are more accurate.
Here is another from Analog Devices.
Quote
Resolution. An N-bit binary converter has N digital data inputs (DAC) or N digital data outputs (ADC). A converter that satisfies this criterion is said to have a resolution of N bits.
Resolution has nothing to do with accuracy.  It's the number of bits an ADC outputs or the number of bits of input to a DAC.

DC accuracy involves these (From Analog Devices), not resolution.
Quote
The static absolute accuracy of a DAC can be described in terms of three fundamental kinds of errors: offset errors, gain errors, and integral nonlinearity.
Another quote from Analog Devices.
Quote
The traditional static specifications such as differential nonlinearity (DNL) and integral nonlinearity (INL) are most certainly reflected in the ac performance.
That's why ENOB is a better measure for quality of signal measurements.  It combines all factors regarding accuracy of the measurement.

For simple DC measurements, non-linearity is the big deal.  Offset errors are easy to calibrate.  Many ADCs like the MCP3421 do it automatically.

From Microchip for the MCP3421
Quote
Self Calibration of Internal Offset and Gain Per Each Conversion.

It's true offset errors may not affect AC performance.  You must compensate for the AVR offset errors for DC measurements.

This is such basic stuff for an EE.  Are you really an EE?  If so when did you go to school?
963  Using Arduino / Storage / Re: Unformatted write to SD on: August 20, 2012, 02:27:12 pm
I missed this.
Quote
I would love to see a theoretical study that shows how adding noise to your signal improves the s/n ratio! I get the gut felling that is violating the second law somewhere on the line
Your funny, wrong but funny.  Time to read the ADC theory.

Quote
Note that this averaging is possible only if the signal contains perfect equally distributed noise (i.e. if the A/D is perfect and the signal's deviation from an A/D result step lies below the threshold, the conversion result will be as inaccurate as if it had been measured by the low-resolution core A/D and the oversampling benefits will not take effect).
The above means adding noise can improve accuracy with enough oversampling.  Too little noise will result in lower resolution.

This is why I never use oversampling, too much can go wrong.  Better to use a more accurate ADC.

Quote
This is why I was cautious about ENOB. It is only a measure of resolution , not accuracy.
Wrong again.

ENOB specifies the number of bits in the digitized signal above the noise floor, this is accuracy.  A 12-bit ADC has 12 bits of resolution but may not be accurate to 12-bits.

Are you really an EE?

Maybe I expect too much.  In physics I work with EEs that design ADCs and other IC parts. 

I worked on the CERN Atlas experiment that discovered the Higgs Boson.  The front end electronics used ASICs (Application Specific Integrated Circuit) designed by CERN engineers.  This is necessary for low noise, high speed, and these parts must be Radiation Hard.

I expect EEs to know theory.
964  Using Arduino / Storage / Re: New fast data logging sketches on: August 20, 2012, 11:58:27 am
Sorry, I get way too many requests to help with code for things like this.
965  Using Arduino / Storage / Re: Unformatted write to SD on: August 20, 2012, 11:10:44 am
I decided To post the DAC/ADC test data.  The DAC is a MCP4921 with 5 V ref. http://ww1.microchip.com/downloads/en/DeviceDoc/22248a.pdf.

I got the noise really clean now as you will see in the attached file.  Only about one reading varied in the set of 64 that I took at each DAC value.

Here is the sketch that generated the data
Code:
#include <McpDac.h>
void setup() {
  Serial.begin(9600);
  mcpDacInit();
  for (uint16_t i = 2000; i < 2024; i++) {
    mcpDacSend(i);
    for (uint16_t j = 0; j < 64; j++) {
      Serial.print(i);
      Serial.write(',');
      Serial.print(j);
      Serial.write(',');
      Serial.println(analogRead(0));
    }
    delay(500);
  }
}
void loop() {}
The sketch loads values from 2000 through 2023 into the DAC.  It then does 64 reads with analogRead() for that DAC value.
There are three columns: DAC value, reading #, ADC value.

The Arduino I used has a large offset error so it reads about 3 counts low.  This is not uncommon for an AVR ADC.  Be sure to calibrate your Arduino http://www.atmel.com/images/doc2559.pdf.

2001 on the DAC should be 500 on the ADC (2001*1023/4095 = 499.9).  The ADC reads 497.

You must be logged in to see the attached file.

Edit: I have now attached files taken with three Arduinos.  Arduino two and three have the fluctuation property since the DAC/ADC values lineup just right.  

This does not mean there is more noise with Arduino two and three.  If you don't understand, read the app notes, you claim to be an engineer so this should be easy.
966  Using Arduino / Storage / Re: Unformatted write to SD on: August 20, 2012, 08:40:42 am
The data looks like 10-bit stair-steps with a little fuzz every once in a while.  It should look like 12-bit stair-steps.  You can't see the fact that the input was from a 12-bit DAC and you can't recover the fact by averaging data.

I'm not going to waste any more of my time.

It's time for you to go back to school and learn what every young EE knows about digital converters.  I work with lots of EE students and they know this stuff.

There are plenty of free sources on the web.

There are newer books but this is a great book and it's free http://www.analog.com/library/analogDialogue/archives/39-06/data_conversion_handbook.html.

Your going to have more problems using a multiplexed AVR ADC to do oversampling on multiple channels.  So learn a little first.

Good luck.
967  Using Arduino / Storage / Re: Unformatted write to SD on: August 19, 2012, 09:24:10 pm
What happens is that three of the steps in the 12-bit DAC result in the same value for all reads with the 10-bit AVR ADC.

The fourth step results in two values but this doesn't mean there is 1 LSB of noise.  Often most of the readings are one of the values.

There is always some voltage where a tiny change will result in the next code.  At this point about half the readings will be n and half will be n+1 even with very low noise.

1LSB for The 10-bit AVR ADC with 5V reference means greater than 5V/1023 of noise.  You really should have more like twice that for oversampling to work.  You need to get more than one code for multiple reads at every voltage.

You really need to read about ADCs, all manufacturers have good app notes on oversampling.

Almost all app notes have charts to illustrate this concept and statements like this.
Quote
In this example, the actual voltage falls between two steps of the 12-bit ADC resolution and there is no
noise riding on the signal. It is easy to see the problem. With no noise on the signal, the ADC result for
each conversion will be the same. Averaging produces no effective gain in resolution.
968  Using Arduino / Storage / Re: Unformatted write to SD on: August 19, 2012, 06:16:22 pm
The tenth bit is the same for 1000 reads with analogRead() for about three out of four steps when I generate a ramp with a 12-bit DAC.

Oversampling just gives the 10-bit values not extra bits.  This is because for oversampling to work you need.
Quote
• The signal-component of interest should not vary significantly during a conversion.
• There should be some noise present in the signal.
• The amplitude of the noise should be at least 1 LSB.
See http://www.atmel.com/Images/doc8003.pdf.

If you power the Arduino with USB or a cheap wall wart there is plenty of noise.

I use a real power supply - one with a three prong grounded plug and low ripple/noise.  This supply is good for lots of amps so it's real overkill.

The DAC is on the Arduino using a shield Limor Fried gave me as a prototype.  She is good with ground planes and filtering.  I even put a big, 100K, resistor between the DAC and ADC and still didn't get noise.  I did get noise when I added about two feet of wire in addition to the resistor.

I would love to see a good case study that shows when ADC Noise Reduction Mode is needed and helps.
969  Using Arduino / Storage / Re: Unformatted write to SD on: August 19, 2012, 01:26:25 pm
What appears sub-optimal?  I just stuck the lines in the loop to test sleep, not as a test of jitter.

I didn't post the Noise Reduction tests.

Appears your C++ is a bit rusty. The first statement is a type declaration so the library will compile.  I could have put it anywhere before the call.
Code:
  int rawAnalogReadWithSleep();
This is the call:
Code:
  rawAnalogReadWithSleep();

The "no-op" ISR is necessary to field the wake-up interrupt.

Most of the time in the loop is spent sleeping.  The person that wrote this function is allowing for wake-up by interrupts other than the ADC.  If the ADC is not done, the function goes back into ADC Noise Reduction Mode.

"no-op" ISRs are not uncommon.  Sometimes they clear a flag or cause other status change. They are very fast since no context needs to be saved.  I use one to clear a timer flag in the 100,000 sample per second logger.

I did more testing on the ADC Noise Reduction Mode.  I used a high resolution DAC to generate a ramp.  The DAC is on a well designed shield on the Arduino I was testing.

I got the noise so low with just analogRead() that I couldn't do oversampling.  For a number of DAC steps the 10-bit Arduino ADC always gives the same value.  I don't need noise reduction, I need noise injection to make oversampling work.
970  Using Arduino / Storage / Re: Unformatted write to SD on: August 19, 2012, 10:21:30 am
I did a test and the good news is reading the ADC in Noise Reduction Mode in the middle of an SD block transfer seems to work OK.

I did not write rawAnalogReadWithSleep() but it seems to use Noise Reduction Mode.

I added the code between the slashes to the write loop (two bytes are sent for each pass to slightly speed the transfer):
Code:
 for (uint16_t i = 0; i < 512; i += 2) {
    while (!(SPSR & (1 << SPIF)));
    //////////////////////////////////////////////////////////////
    int rawAnalogReadWithSleep();
    if (i == 300) {
      rawAnalogReadWithSleep();
      Serial.print('.');
    }
    /////////////////////////////////////////////////////////////
    SPDR = buf[i];
    while (!(SPSR & (1 << SPIF)));
    SPDR = buf[i + 1];
  }
I ran this sketch:
Code:
#include <SdFat.h>
#include <avr/sleep.h>
SdFat sd;
SdFile file;

ISR(ADC_vect) { }

int rawAnalogReadWithSleep() {
  // Generate an interrupt when the conversion is finished
  ADCSRA |= _BV(ADIE);

  // Enable Noise Reduction Sleep Mode
  set_sleep_mode(SLEEP_MODE_ADC);
  sleep_enable();

  // Any interrupt will wake the processor including the millis interrupt so we have to...
  // Loop until the conversion is finished
  do
  {
    // The following line of code is only important on the second pass.  For the first pass it has no effect.
    // Ensure interrupts are enabled before sleeping
    sei();
    // Sleep (MUST be called immediately after sei)
    sleep_cpu();
    // Checking the conversion status has to be done with interrupts disabled to avoid a race condition
    // Disable interrupts so the while below is performed without interruption
    cli();
  }
  // Conversion finished?  If not, loop.
  while( ( (ADCSRA & (1<<ADSC)) != 0 ) );

  // No more sleeping
  sleep_disable();
  // Enable interrupts
  sei();

  // The Arduino core does not expect an interrupt when a conversion completes so turn interrupts off
  ADCSRA &= ~ _BV( ADIE );

  // Return the conversion result
  return( ADC );
}

void setup() {
  Serial.begin(9600);
    // setup ADC
  analogRead(0);
  if (!sd.begin())sd.initErrorHalt();
  if(!file.open("ADC_TEST.TXT", O_RDWR | O_CREAT | O_AT_END)) {
    sd.errorHalt("opening test.txt for write failed");
  }
  for (uint16_t i = 0; i < 50000; i++) {
    file.println(i);
    if (file.writeError) sd.errorHalt("print");
  }
  file.close();
  Serial.println("Done");
}
void loop() {}
Lots of dots get printed and the file has the correct content.  The file has 660 blocks.

The bad news is that I can't find proof on the web that Noise Reduction Mode helps.

I tried various tests comparing analogRead() with rawAnalogReadWithSleep() above.  If there is an improvement it is really small.  Other factors overwhelm the the difference between the two functions.  Clean power to the Arduino makes a huge difference.

One curious result is that the two functions return slightly different results.  You need to average a 1000 measurements to see the difference.

My test setup was very crude so it is not definitive but as until I see proof I won't believe Noise Reduction Mode is worth the pain.

I hope you prove Noise Reduction Mode gives more accuracy.

Edit:
I did some more work and oversampling and Noise Reduction Mode are extremely frustrating.  If I work hard to reduce noise, oversampling will not work since you need noise for oversampling.  I don't trust oversampling with the 10-bit AVR ADC, it's too easy to fool yourself.

If you need more accuracy, an external ADC seem like a far better approach.

971  Development / Other Software Development / Re: Fast digital I/O and software SPI with C++ templates on: August 18, 2012, 09:25:56 pm
The library in this thread didn't work well for use in other bit-bang libraries so I have rewritten it.  One of the main reasons for fast digital I/O is bit-bang for protocols like SPI.

Please see DigitalPinBeta20120804.zip http://code.google.com/p/beta-lib/downloads/list.

Also see this topic http://arduino.cc/forum/index.php/topic,117356.0.html.

I suspect the templates kert describes above will have the same problems.

I still include template classes for simple use in sketches but I base these classes on static functions with constant arguments.

These static functions are easier to use in other bit-bang libraries like a software SPI library which is included as an example in the new library.
972  Using Arduino / Storage / Re: New fast data logging sketches on: August 18, 2012, 04:51:58 pm
If you just want simple hardware SPI access, read section 6.1 of the datasheet http://ww1.microchip.com/downloads/en/DeviceDoc/21290e.pdf.
973  Using Arduino / Storage / Re: Unformatted write to SD on: August 18, 2012, 02:09:31 pm
I don't log access time for read.  I provide a function that allows all time fields of a file to be set with a call like Linux/UNIX touch.

I have never been asked to provide an update to access time for reads.

For write/create I update all fields since it's free.  The directory entry only gets updated when you call sync() or close a file.  That's when time fields get updated in the directory entry.

I think the answer to why Atmel is Arduino.  Arduino give easy software/hardware access to microprocessors.

AVR is old but fun.  It reminds me of my first microprocessor hardware projects with the 6800 and 6502 (the 6502 was used in the Apple II) around 1974.

People use sleep modes with SdFat to save battery power.  They just aren't sleeping in the middle of a block transfer.  I think sleeping between blocks in multi-block mode would work.  The SD card won't know and chip select is high.
974  Using Arduino / Storage / Re: Unformatted write to SD on: August 18, 2012, 12:16:50 pm
Quote
As an engineer "forget the datasheet" is not part of my way of working. It is not a general guide , it's the bible.
As a PhD physics researcher working on some the world's largest experiments, I take datasheets with a grain of salt.  They are a clue but you must evaluate devices in the mode you use them.  Often devices change but not the datasheets.
Quote
I get the impression you are confusing a statistical measurement , ENOB, with the spec for the accuracy on one ADC conversion. The two results are compatible and not contradictory. They are different things.
Accuracy of one conversion is a statistical measurement.  Accuracy is always a statistical measurement.

Did you read the Oskar Leuthold paper on the AVR ADC?  He is an engineer working for GEC Plessey Semiconductors.  He designs fast, 200 Megasample per second  A/D converters.

What is wrong with his testing of the AVR ADC?
Quote
I am logging physical quantities, not audio.
Audio is a physical signal.  Some of the most advanced research in high quality ADCs involves audio.
Quote
Are you able to comment on whether sleeping the CPU would break the SPI streaming or is the protocol robust enough to stand  a circa 100us hiatus?
Using ADC Noise Reduction Mode is likely to cause problems with an SD transfer.  Atmel documents suggest that Idle Mode could work O.K.  The SPI controller is stopped in ADC Noise Reduction Mode and that could cause a glitch.

That's a comment not a proven fact.  My original library allowed interruption of block transfers but I kept having problems so I removed the code.

If you need accuracy at 50 Hz why not us an external ADC?  I use external sigma-delta ADCs frequently.  I wrote a fast Software SPI library that runs at about 2MHz to access external ADCs since it is difficult/impossible to share the hardware SPI bus with the SD.  

At 50 Hz you could even use low cost I2C ADCs.  Here is a family of 18-bit delta-sigma ADCs I like a lot (maybe too slow for you). They have a 0.05% on-board voltage reference and an on-board PGA. http://ww1.microchip.com/downloads/en/devicedoc/22088c.pdf.  It could give you 14 bits at 60 sps.

This may not work for your application but there are many choices that are easy to use.  Why fight with the AVR ADC if it doesn't meet your requirement.

I mainly play with pushing the AVR ADC as a game.  For serious hobby measurements I use external ADCs/sensors.  

I am now mostly using Cortex M4 STM32 processors, not Arduino.  I run ChibiOS/RT as the OS.

Here is an example board https://www.olimex.com/dev/stm32-h407.html it has three really fast 12-bit ADCs that can run in parallel.
975  Using Arduino / Storage / Re: Unformatted write to SD on: August 18, 2012, 09:56:31 am
Quote
Specifically what happens when you go beyond the "management block" size of 128MB or whatever? Do you know how/why you were able to avoid hitting a busy delay?
The busy delay has nothing to do with "management block".  The big delay happens in single block mode because the controller does not plan ahead. For the 12th time I use multi-block streaming mode.
Quote
Yes, but have you tested >5MB to see whether you are still getting no busy time? 
Yes, I designed SdFat for audio recording and other high speed logging.  I have logged for hours in real apps.

Limor Fried asked me to make a version for beginning users.  The Arduino company decided it was too complex and wrapped it with their SD.h API.  That's why you think I designed SdFat for beginning user. 

You can used just the files Sd2Card.h, Sd2Card.cpp and SdInfo.h as a library.  This sketch takes 2120 bytes of flash and will write block zero of an SD.
Code:
#include <Sd2Card.h>
Sd2Card card;
uint8_t buf[512];
void setup() {
  card.init();
  card.writeBlock(0, buf);
}
void loop() {

Quote
ADC noise reduction mode which is required to get (nominal) 10b accuracy from the Atmel chip.  This scheme seems fine for your 8bit sampling but  you have to chose between higher resolution ADC and jitter.
Forget the datasheet, it is a general guide.  Look at the AVR evaluation tests. In the papers that I pointed to, the ADC is triggered by the CPU clock and noise reduction is done using the DIDR.

The result is 7.4 ENOB for the 2MHz rate used at 100,000 samples per second.  For 10-bit sampling 33 ksps gives a ENOB of about 9.3 with a 500kHz ADC clock.   The max ENOB is 9.5 for the AVR ADC in any test.

Here is a paper on SNR due to sampling jitter http://www.analog.com/static/imported-files/tutorials/MT-200.pdf.

Clearly the jitter in the AVR timer compare event is less that a CPU cycle.  I said it was less than a CPU cycle since I don't know the exact number.

ardnut,  If you are over sampling, why write all the data to the SD?  What kind of signal are you recording?  How can you possibly use the AVR ADC for a fast multi-channel signal?
Pages: 1 ... 63 64 [65] 66 67 ... 112