Problems with big array

Hi everybody!
I’m new to this forum and to Arduino, and this is my first post.
Maybe I’m trying to achieve something too high being my first experiment with Arduino, however I’m trying to do so.

I’m realizing a function generator with Arduino uno (sine wave, square wave, etc.) with a sample frequency of 100 KHz using timer interrupts and port manipulation for speed reasons. I need to output 8 digital values in parallel from the digital outputs and then I send them to a R2R DAC.

The problems appears when it comes to store sine wave values in a big array. I have a 10,000 elements byte type array where I store the values for the sine function (from 0 to 255).

I tried to use PROGMEM to store these values as I don’t need to modify them, but just to read them. I figured out that the maximum size allowed for the array in the 32KB memory is around 2,500.
Is it possible to work around this problem?? A possible solution is to split it into several arrays. The problem is that at runtime I should use if statements to choose which array to get the value from, but this is too slow I guess with a 100 KHz sample frequency.
Any other solution?

[edit] I post a sample code (without timer interrupts and port manipulation):

#include <avr/pgmspace.h>

#define numSamples 300

const byte waveformsTable[] PROGMEM = {
... lots of values...
};

static int j = 0;
static byte out = 105;

void setup()
{

  Serial.begin(9600);

    
}


void loop()
{
  if (j==numSamples) j = 0; 
  out = pgm_read_byte_near(waveformsTable + j);
  Serial.println(out);
  j++;
  delay(5);
}

I keep numSamples 300 in order to reset the counter (j), but the size of the array isn’t 300.
It works with 2,450 values, it doesn’t upload to arduino with “avrdude: verification error, first mismatch at byte…” with 2,475 values.

I figured out that the maximum size allowed for the array in the 32KB memory is around 2,500.

I don't understand your reasoning there. A 10000 element byte array would occupy 10000 bytes.

Of course, at 8 bit resolution, a lot of the entries in that table would be identical.

alexei92: 100 KHz using timer interrupts

Timer interrupts at 100 kHz? Perhaps you'd ask Santa Claus for a faster Arduino than you have at the moment!

alexei92: The problems appears when it comes to store sine wave values in a big array. I have a 10,000 elements byte type array where I store the values for the sine function (from 0 to 255).

I tried to use PROGMEM to store these values as I don't need to modify them, but just to read them. I figured out that the maximum size allowed for the array in the 32KB memory is around 2,500.

As you told about "I'm realizing a function generator with Arduino uno" and an UNO has a TOTAL FLASH SIZE of 32KB, there is absolutely no chance to define more than 32KB constants in your program. It would not fit into the flash memory of an UNO (Atmega328).

Try it with as '1284P based board. 128K flash, 16k SRAM. I have written programs with a 14,625 byte array, left it in SRAM so it could be updated by the sketch.

I can't begin to argue with that brilliant code. Nope, can't even try.

AWOL: I don't understand your reasoning there. A 10000 element byte array would occupy 10000 bytes.

Of course, at 8 bit resolution, a lot of the entries in that table would be identical.

Yes, I know. But it doesn't work anyway with 10,000 entries.

jurs: Timer interrupts at 100 kHz? Perhaps you'd ask Santa Claus for a faster Arduino than you have at the moment!

What's the maximum rate I can get?

As you told about "I'm realizing a function generator with Arduino uno" and an UNO has a TOTAL FLASH SIZE of 32KB, there is absolutely no chance to define more than 32KB constants in your program. It would not fit into the flash memory of an UNO (Atmega328).

Ok, but 10,000 entries are less than 10 KB. They're not more than 32KB.

CrossRoads: Try it with as '1284P based board. 128K flash, 16k SRAM. I have written programs with a 14,625 byte array, left it in SRAM so it could be updated by the sketch.

Thank you, but I can't buy other boards at the time. I should use this one.

But it doesn't work anyway with 10,000 entries.

What is "it" "doesn't work" - what does that mean?

AWOL: What is "it" "doesn't work" - what does that mean?

I get the error I mentioned in the first post while uploading the sketch to arduino:

It works with 2,450 values, it doesn't upload to arduino with "avrdude: verification error, first mismatch at byte..." with 2,475 values.

I think it is a memory overflow but I can't understand why as the big array isn't in the RAM memory but in the flash memory.

alexei92: Thank you, but I can't buy other boards at the time. I should use this one.

When you are ready, one step is to put the chip in a breadboard and make your own Arduino.

But the 1284P chip may cost $7 to more than $10 here and some countries charge extra. All I can say is they charge extra for the boards too, make your own could save you the most!

http://www.gammon.com.au/breadboard

One answer to your storage problem is to connect SD. You can buy a shield or adapter or make your own adapter out of the full size SD adapter that comes with most micro-SD chips. It requires soldering. If your Arduino is 5V it will need a number of small parts to keep 5V off the 3.3V card but not too much. Still once you have the card working, you need to learn file operations, a project in itself. What you get for that is enormous data space that can be read and written by a PC and your Arduino.

alexei92: What's the maximum rate I can get?

I have seen a sine wave generator project on the Internet with around 32 kHz timing for 16 MHz Atmega.

But the problem with that is: The output signal becomes only a sine wave while it is sent through a "PWM Output lowpass Filter" built in hardware.

So you cannot have different wave forms on the same output. You could have one output for square wave output (no lowpass filter needed) and another output for sine wave output (lowpass filter needed as shown in the link).

The Arduino UNO has no DAC for generating analogue Voltage output, it only has PWM-Output.

alexei92: Ok, but 10,000 entries are less than 10 KB. They're not more than 32KB.

So what's the problem: Is the problem a "compiling error" or is it an "uploading error"?

Please post the complete code of a simplified sketch that can reproduce the error!

jurs: So what's the problem: Is the problem a "compiling error" or is it an "uploading error"?

Please post the complete code of a simplified sketch that can reproduce the error!

No compiling error, but uploading error. The code I posted before produces the error with the threshold of ~2,500 items.

alexei92: No compiling error, but uploading error. The code I posted before produces the error with the threshold of ~2,500 items.

Maybe you have a controller with defective flash memory.

Did you check that the board setting in the IDE matches exactly the board you are using?

Perhaps go to File->Preferences->Show verbose output during..." and tick the box for "upload".

Then give the upload another try and wait for error messages in the status window.

Then copy the complete output from the status window including all messages to a forums message and show.

Thank you for your reply. I’ve set the verbose output for upload but unfortunately it cuts off the first lines of the output, so this is what I could get:

With a little array (everything ok): http://pastebin.com/DbRs8z1t
With the big array (error occurs): http://pastebin.com/aptSri58

alexei92: With the big array (error occurs): http://pastebin.com/aptSri58

Very strange.

Upload seems to run completely until finished.

But final verify fails with error.

avrdude: verification error, first mismatch at byte 0x09fa 0xff != 0xfa

Is the first mismatch byte always the same number (0x09fa) when you try uploading the same program again and again?

If the number is always the same, I'd try another Arduino board. Maybe the controller is defective.

If the number changes from on try to another try, I'd try another USB cable. Maybe the USB cable is of poor quality.

The number is always the same. It changes for example if I change the length of the array or if I add some instructions in the loop, etc. But it is always the same through different uploads of the same code. Is it normal to have a problem like that? If I try another board and I have the same result? Maybe it's because it is a compatible board?

There is no possible reason why you would need anything like 10,000 , or even 1,000 calculated values for a sine wave.

alexei92: The number is always the same. It changes for example if I change the length of the array or if I add some instructions in the loop, etc. But it is always the same through different uploads of the same code.

I'd say this points to a problem with the flash memory in the controller.

But I did not check out, if it might be also a strange error in the Arduino GCC compiling/uploading toolchain: You never showed any demonstration code that caused the error, you didn't tell about your Arduino and Java version numbers and the type of your operationg system. So might be something else, but I'd try another board for comparison reasons in the first place.

alexei92: Is it normal to have a problem like that?

It's not normal, but it can happen. How many times have you flashed new software into the board? Atmel guarantees for at least 10000 flash cycles for their controllers, I think. But in most cases there will be many more flash cycles possible without any error, so that you hardly will write the flash memory defective by uploading new sketeches too often.

alexei92: If I try another board and I have the same result?

Did you try another board with the same result?

If so, there must be another problem. Don't know what.

alexei92: Maybe it's because it is a compatible board?

In China they create all kind of fakes and cheap electronics. I never heard about fake Atmel controllers, but cheap boards from China may be defective more often than products from quality production. I don't know the reason, but perhaps the board had been run for too long through a too hot soldering. I don't know.

jurs: I'd say this points to a problem with the flash memory in the controller.

But I did not check out, if it might be also a strange error in the Arduino GCC compiling/uploading toolchain: You never showed any demonstration code that caused the error, you didn't tell about your Arduino and Java version numbers and the type of your operationg system. So might be something else, but I'd try another board for comparison reasons in the first place.

In the first post of this topic I showed a demonstration code which, I repeat for the third time, gives the problem once I upload it. I'm new to Arduino, as I said, so I wrote what I thought necessary to say. I expect you would teach a newbie how to move. So if you're asking me other information, I will tell you: PC: HP g62 b29sl - Intel core i3 2GHz/4 GB ram OS: Windows 7 ultimate SP1 Java version: JDK7 installed, however JRE version 7 update 51 (1.7.0_51-b13) Arduino sw: Arduino 1.0.6 Arduino board: DccEle - DCcduino Uno R3 (ATMega328P - USB Controller CH340USB). Driver installed: CH340USB.

In Arduino sw, I selected type "Arduino uno".

It's not normal, but it can happen. How many times have you flashed new software into the board? Atmel guarantees for at least 10000 flash cycles for their controllers, I think. But in most cases there will be many more flash cycles possible without any error, so that you hardly will write the flash memory defective by uploading new sketeches too often.

I flashed no more than 20 sketches. I bought the board less than 1 month ago. When I upload the blink sketch after a failed upload I have no problems.

Did you try another board with the same result?

If so, there must be another problem. Don't know what.

I asked because if I should really try another board without trying some other solutions, I have to buy it. So I'd like not buying another board, but I'd try something other.

alexei92: In the first post of this topic I showed a demonstration code which, I repeat for the third time, gives the problem once I upload it.

You showed some code that even DOES NOT COMPILE.

const byte waveformsTable[] PROGMEM = {
... lots of values...
};

leads to the compiler error:

sketch_jan29a:6: error: expected primary-expression before '...' token

You won't send the full sketch. And I tried my best asking your questions without seeing the full code for several days now. My brain is not able to imagine what you mean by writing "... lots of values..." and testing any code with "... lots of values..." is impossible for me.

Please try another code and find out if it works for you. There are a lot of example programs with Arduino that should compile an upload error-free.

Or try another Arduino (compatible) board.

And now I'm out.

I thought it wasn't necessary to write 2,500 ints. Why do these numbers are so important for this error?

http://pastebin.com/Ps5Vz95s

This compiles and works. I am able to see the right numbers on the serial monitor. When you remove the comment from the last line of values it gives the error.