How to read values from SD card txt into an array

Hi I need some help with my project: I want to read values from a txt file on SD card and store them in an array. The code for reading the values is read but it seems that I cannot set up an array without knowing the lenght/size. So I run the code twice first to count the entries in the txt and then setting up the array with that counted size. but I need to declare the array in the code before with a size just to declare it.

I get my output on my lcd telling me the values and the counter position while reading the txt and I try to store the values using the counter index

myarray[counter] = valuefromtxt;

but in the end the array remains empty.

any idea?

please, I am not an advanced coder - please keep it simple :) thanks a lot!

The easiest thing to do is to declare the array with the maximum expected number of elements. After all, if there are that number of entries in the file then even if you could expand the array or declare it dynamically to accommodate them, then that number of elements would be required.

When reading the file you should ensure that the number of entries put into the array does not exceed the number of elements in the array.

As to your current solution, it is impossible to be sure without seeing the code, but how and where do you declare the array and have you taken its scope into account ?

Please post your code.

Thanks for the fast reply. I will post the code this evening.

Hi

this is the code:

#include <SPI.h>
#include <LiquidCrystal.h>
#include <SD.h>

File myFile;

LiquidCrystal lcd(8, 7, 6, 5, 4, 2);

int dmxmaster_array_size = 0; 
int dmxmaster_array[500];

void setup() {

  lcd.begin(16, 2);
  lcd.print("READY");
  
  readSD();
  
}

void loop () {
  
}

void readSD() {
  
  int dmxmaster_array_size = 0; 
  int arraycount = 0;

  // CHECK
    if (!SD.begin(53)) {
        lcd.setCursor(0, 1);
        lcd.print("NO SD CARD!           ");
        return;
        }
    lcd.setCursor(0, 1);
    lcd.print("SD CARD READY.           ");

delay(500);

// READ FROM CARD

myFile = SD.open("channels.txt");
  if (myFile) {
    lcd.setCursor(0, 1);
    lcd.print("DATA FOUND     ");  
    delay(500);
    lcd.setCursor(0, 1);
    lcd.print("READING DATA FROM SD      ");   
    delay(500);

// COUNT NUMBERS OF ENTRIES

arraycount = 0;

while (myFile.available()) {

  switch ((char)myFile.peek()) {
      case ',': 
      myFile.read();
      break;
      case '\r':
      myFile.read();myFile.read(); 
      break;

      default:
      int newvalue = myFile.parseInt();
      arraycount++;
      break;
      }
}

// SET ARRAY TO SIZE OF ENTRIES
int dmxmaster_array[arraycount];

    lcd.setCursor(0, 1);
    lcd.print("                   ");  
    lcd.setCursor(0, 1);
    lcd.print(arraycount);  
    delay(500);


// READ VALUES TO ARRAY

int arraycount2 = 0;

while (myFile.available()) {

  switch ((char)myFile.peek()) {
    case ',':
    myFile.read();
    break;
    case '\r':
    myFile.read();myFile.read(); 
    break;

    default:

  int newvalue = myFile.parseInt();
  
  // ADD VALUE TO ARRAY AT INDEX ARRAYCOUNT2
  
  dmxmaster_array[arraycount2] = newvalue;
  
  arraycount2++;

break;

  }
}

  myFile.close();
  

// SHOW ENTRIES IN ARRAY

for(int z=0;z<=(sizeof(dmxmaster_array));z++) {

  lcd.setCursor(0, 1);
  lcd.print("          ");   
  lcd.setCursor(0, 1);
  lcd.print(dmxmaster_array[z]); 
  delay(500);
  }
 


  } else {
  lcd.setCursor(0, 1);
  lcd.print("NO DATA FOUND     "); 
  delay(500);
  }

}

Most of it is just to tell me what is happening on the LCD.

the channels.txt contains:

124,10,999,255,0,0,0,0,255,0,0,
125,10,999,0,255,0,0,0,0,255,0,
126,10,999,0,0,255,0,0,0,0,255,
127,10,999,255,255,0,0,0,255,255,0,
128,10,999,255,0,255,0,0,255,0,255,
129,10,999,0,255,255,0,0,0,255,255,
130,10,999,255,255,255,0,0,255,255,255,
131,10,999,155,0,0,0,0,155,0,0,
132,10,999,153,0,255,0,0,153,0,255,
133,10,999,153,102,255,0,0,153,102,255

You seem to have problems with the scope of variables. For instance

    // SET ARRAY TO SIZE OF ENTRIES
    int dmxmaster_array[arraycount];

The comment really ought to say “Create a new array with the same name as the global one with the same name”

    // SHOW ENTRIES IN ARRAY
    for (int z = 0; z <= (sizeof(dmxmaster_array)); z++)

Which of the dmxmaster_array variables does this refer to I wonder ?
By the way, sizeof() returns the number of bytes used by an array not the number of entries in the array unless the array type is byte or char. Yours is an array of ints so the number returned will be half the number of entries.

int dmxmaster_array[500];This array uses 1000 bytes of precious memory and
    int dmxmaster_array[arraycount];uses an unknown number of bytes of memory. On most Arduinos you are close to running out of memory. Whcih board are you using ?

I am using an Arduino Mega

the problem with the dmxmaster_array is, that I have to define it in the beginning - otherwise it will not compile.

You said I should set up the dmxmaster_array with a size more than needed. I tried to redfine it with the right size.

how do I get the numbers of entries in that array? I just need that in the end to get the entries displayed on the lcd for debugging

the channels.txt is just a sample - the original contains more than1300 values

the channels.txt is just a sample - the original contains more than 1300 values

Please explain more about the way channels.txt is organized. Is each line always 11 comma separated values starting with an index and ending with a newline or carriage return?

I just need that in the end to get the entries displayed on the lcd for debugging

Getting the array size to use a for loop on integer values, instead of just reading and displaying each csv text field on the lcd like you do with the serial monitor for your example, seems unnecessary.

Having each value flash by for 500ms does not sound all that great. Perhaps you could use a "read" button which you can press to advance through the file.

Debugging using an lcd display to show one element at a time is tough. Can't you get the sd card onto a computer to read it?

Can you explain more about what you are trying to do?

You said I should set up the dmxmaster_array with a size more than needed.
I tried to redfine it with the right size.

Declare it with a size large enough to hold the maximum number of entries that will ever be used then use it. Do not declare another array with the same name but a different scope later.

As you are using a Mega, memory is less of an issue, but if the maximum value that will be held is 255 then the array would be better declared as byte rather than int to halve memory usage which is good practice.

how do I get the numbers of entries in that array?

You put the entries in the array when you read them from the SD card. Count them as you put them in.

@cattledog: This code is just a fragment of the main program.

What the program does is receiving MIDI events and send out DMX data to control rgb led spots or other DMX controllable devices.

the channels.txt hold the configuration and behaviour what data will be send to the DMX devices.

for Example:

124,10,999,255,0,0,0,0,255,0,0

124 = combination of MIDI channel (1) and note number (24) - if this combination is received the following 6 values are send to the DMX starting at channel 10 - the remaining 3 values are used to contol a rgb neopixel strip for preview of the rgb colour. the 999 is telling the programm to use the MIDI velocity - so in this case the colour RED (255,0,0) is send to the DMX device with the brightness controlled by velocity of the note C1 (note number 24)

I want to store the configuration on the SD card, so I can add or change it without need to change the code.

All the rest is working fine - If I store the values in an array inside the code.

@UKHeliBob

So I should try to use a byte array instead for better memory usage.

but all of this is not solving the problem that the array is not filled with the proper values from the txt file.

Is there an error inside my code? Is it because I declare the array again?

but all of this is not solving the problem that the array is not filled with the proper values from the txt file.

Have you changed the program since it was pointed out that you have two arrays with the same name, each with their own scope ?

I will do so and give feedback - thanks

It works! Thanks a lot for your help!

I am glad that you got it working.

It would be interesting to see your working program as there may be some tweaks that would make it work more efficiently or techniques that might be useful in other programs.

So here’s my little program.

This is part of my MIDI2DMX project which I am building and develop since about 2 years by putting new features to the programm and designing a special MIDI2DMX shield for a Arduino Mega.

Some of you might wonder what this is all about: It#s just a litte device that transform MIDI notes from a keyboard to DMX data for stage lamps. With this device you can play the light on the keybard.
And by switching midi channels you can control 16 individual DMX devices and tell them what to do if you press an note.

Like I said, I am not a real good coder and I just collected some fragments from the web and made it work somehow.
If someone has an idea how to make the program more efficient please feel free to do so.
And if someone is interested in the PBCs please tell me.

#include <DmxSimple.h>
#include <Adafruit_NeoPixel.h>
#include <MIDI.h>
#include <SPI.h>
#include <LiquidCrystal.h>
#include <SD.h>

File myFile;

//MIDI_CREATE_DEFAULT_INSTANCE();
MIDI_CREATE_INSTANCE(HardwareSerial, Serial1, MIDI);

#define PIN 22 // Digital IO pin connected to the NeoPixels.
Adafruit_NeoPixel strip = Adafruit_NeoPixel(8, 22, NEO_GRB + NEO_KHZ800);

// initialize the library with the numbers of the interface pins
LiquidCrystal lcd(8, 7, 6, 5, 4, 2);

byte incomingByte;
byte note;
byte velocity;
byte midichannel;

int startnote = 24;
int action=2; //0 =note off ; 1=note on ; 2= nada
int dmxchannel;
int chan;
int pixelred;
int pixelgreen;
int pixelblue;
int dmxmaster_array[2000];
int arraycount = 0;

// CHANNEL ARRAYS LEGEND
// 
// value 0 = midi channel / midi note number
// value 1 = dmx channel start
// value 2 = dmx channel value        / 999 for Veloctiy / 888 for 255 - Velocity
// value 3 = dmx channel value + 1    / 999 for Veloctiy / 888 for 255 - Velocity
// value 4 = dmx channel value + 2    / 999 for Veloctiy / 888 for 255 - Velocity
// value 5 = dmx channel value + 3    / 999 for Veloctiy / 888 for 255 - Velocity
// value 6 = dmx channel value + 4    / 999 for Veloctiy / 888 for 255 - Velocity
// value 7 = dmx channel value + 5    / 999 for Veloctiy / 888 for 255 - Velocity
// value 7 = neopixel value RED
// value 8 = neopixel value GREEN
// value 9 = neopixel value BLUE


void setup() {
  
  lcd.begin(16, 2);
  lcd.print("midi2dmx!");

// MIDI SETUP
 
  MIDI.setHandleNoteOn(handleNoteOn); // Put only the name of the function
  MIDI.setHandleNoteOff(handleNoteOff);
  MIDI.begin(MIDI_CHANNEL_OMNI);

// NEOPIXEL SETUP
  strip.begin();
  strip.show();
  Serial.begin(31250); 
        
// DMX SETUP
  DmxSimple.usePin(3);
  DmxSimple.maxChannel(90);

// READ SD CARD

readSD();
  
}

void setPixelColor( uint16_t n, uint8_t r, uint8_t g, uint8_t b, uint16_t brightness) {
strip.setPixelColor(n, (brightness*r/255) , (brightness*g/255), (brightness*b/255));
}

void loop () {
  
MIDI.read();

}

void readSD() {

  // CHECK
    if (!SD.begin(53)) {
        lcd.setCursor(0, 1);
        lcd.print("NO SD CARD!           ");
        return;
        }
    lcd.setCursor(0, 1);
    lcd.print("SD CARD READY.           ");

delay(500);

// READ FROM CARD

myFile = SD.open("channels.txt");
  if (myFile) {
    lcd.setCursor(0, 1);
    lcd.print("DATA FOUND       ");  
    delay(500);
    lcd.setCursor(0, 1);
    lcd.print("READING FROM SD        ");   
    delay(500);

  
while (myFile.available()) {

  switch ((char)myFile.peek()) {
      case ',': 
      myFile.read();
      break;
      case '\r':
      myFile.read();myFile.read(); 
      break;

      default:
      int newvalue = myFile.parseInt();
      dmxmaster_array[arraycount] = newvalue;
      arraycount++;
      break;
      }
    }


  myFile.close();
  
  lcd.setCursor(0, 1);
  lcd.print("READY!            "); 

  } 
  // ELSE

}

void handleNoteOn(byte channel, byte note, byte velocity)
{
  // NOTE ON
  // combine midi channel and midi note number
  int channelnote = (channel*100)+note;
          lcd.setCursor(0, 1);
          lcd.print("                       ");
          lcd.setCursor(0, 1);
          lcd.print(channelnote);
  
  // search for value in array
      for (int i=0; i<=arraycount; i=i+11) {
        
          if (channelnote == dmxmaster_array[i]) {
          
          lcd.setCursor(4, 1);
          lcd.print(dmxmaster_array[i]);

          lcd.setCursor(8, 1);
          lcd.print(dmxmaster_array[i+1]);
           
          playNote (           dmxmaster_array[i+1],
                               dmxmaster_array[i+2],
                               dmxmaster_array[i+3],
                               dmxmaster_array[i+4],
                               dmxmaster_array[i+5],
                               dmxmaster_array[i+6],
                               dmxmaster_array[i+7],
                               velocity);
          
          set_neopixel (       channel,
                               dmxmaster_array[i+8],
                               dmxmaster_array[i+9],
                               dmxmaster_array[i+10], 
                               velocity);
          break;
        
      }

    }
      
}

void handleNoteOff(byte channel, byte pitch, byte velocity)
{
  // NOTE OFF
}

void set_neopixel (byte midichannel, byte pixelred, byte pixelgreen, byte pixelblue, byte velocity){

setPixelColor(8-midichannel, pixelred, pixelgreen , pixelblue, 255*velocity/127);
strip.show();

}

/////////////////////////////////////////////////////
// DMX OUTPUT
/////////////////////////////////////////////////////

void playNote(int dmxchannel, int dmx1, int dmx2, int dmx3, int dmx4, int dmx5, int dmx6, int velocity){

    // DMX CHANNEL 1
    
    if (dmx1 == 999) {
      // VELOCITY
      DmxSimple.write(dmxchannel, 255*velocity/127);
      } else if (dmx1 == 888){
      DmxSimple.write(dmxchannel, 255 -(255*velocity/127));
      } else {
      // ARRAY VALUE  
      DmxSimple.write(dmxchannel, dmx1);
    }

    // DMX CHANNEL 2

    if (dmx2 == 999) {
      // VELOCITY
      DmxSimple.write(dmxchannel + 1, 255*velocity/127);
      } else if (dmx2 == 888){
      DmxSimple.write(dmxchannel + 1, 255 -(255*velocity/127));
      } else {
      // ARRAY VALUE  
      DmxSimple.write(dmxchannel + 1, dmx2);
    }

    // DMX CHANNEL 3

    if (dmx3 == 999) {
      // VELOCITY
      DmxSimple.write(dmxchannel + 2, 255*velocity/127);
      } else if (dmx3 == 888){
      DmxSimple.write(dmxchannel + 2, 255 -(255*velocity/127));
      } else {
      // ARRAY VALUE  
      DmxSimple.write(dmxchannel + 2, dmx3);
    }

    // DMX CHANNEL 4

    if (dmx4 == 999) {
      // VELOCITY
      DmxSimple.write(dmxchannel + 3, 255*velocity/127);
      } else if (dmx4 == 888){
      DmxSimple.write(dmxchannel + 3, 255 -(255*velocity/127));
      } else {
      // ARRAY VALUE  
      DmxSimple.write(dmxchannel + 3, dmx4);
    }

    // DMX CHANNEL 5

    if (dmx5 == 999 ) {
      // VELOCITY
      DmxSimple.write(dmxchannel + 4, 255*velocity/127);
      } else if (dmx5 == 888){
      DmxSimple.write(dmxchannel + 4, 255 -(255*velocity/127));
      } else {
      // ARRAY VALUE  
      DmxSimple.write(dmxchannel + 4, dmx5);
    }

    // DMX CHANNEL 6

    if (dmx6 == 999 ) {
      // VELOCITY
      DmxSimple.write(dmxchannel + 5, 255*velocity/127);
      } else if (dmx6 == 888){
      DmxSimple.write(dmxchannel + 5, 255 -(255*velocity/127));
      } else {
      // ARRAY VALUE  
      DmxSimple.write(dmxchannel + 5, dmx6);
    }
}

In the moment the channel.txt looks like this:

124,10,999,255,0,0,0,0,255,0,0,
125,10,999,0,255,0,0,0,0,255,0,
126,10,999,0,0,255,0,0,0,0,255,
127,10,999,255,255,0,0,0,255,255,0,
128,10,999,255,0,255,0,0,255,0,255,
129,10,999,0,255,255,0,0,0,255,255,
130,10,999,255,255,255,0,0,255,255,255,
131,10,999,155,0,0,0,0,155,0,0,
132,10,999,153,0,255,0,0,153,0,255,
133,10,999,153,102,255,0,0,153,102,255,
134,10,999,153,204,255,0,0,153,204,255,
135,10,999,153,102,0,0,0,153,102,0,
136,10,999,204,102,0,0,0,204,102,0,
137,10,999,204,153,0,0,0,204,153,0,
138,10,999,51,51,0,0,0,51,51,0,
139,10,999,0,0,150,0,0,0,0,150,
140,10,999,102,51,51,0,0,102,51,51,
141,10,999,102,153,0,0,0,102,153,0,
142,10,999,153,153,0,0,0,153,153,0,
143,10,999,51,153,204,0,0,51,153,204,
144,10,999,102,102,153,0,0,102,102,153,
145,10,999,255,255,255,0,0,255,255,255,
146,10,999,255,255,255,0,0,255,255,255,
147,10,999,255,255,255,0,0,255,255,255,
148,10,999,0,0,0,0,0,0,0,0,
224,20,999,255,0,0,0,0,255,0,0,
225,20,999,0,255,0,0,0,0,255,0,
226,20,999,0,0,255,0,0,0,0,255,
227,20,999,255,255,0,0,0,255,255,0,
228,20,999,255,0,255,0,0,255,0,255,
229,20,999,0,255,255,0,0,0,255,255,
230,20,999,255,255,255,0,0,255,255,255,
231,20,999,155,0,0,0,0,155,0,0,
232,20,999,153,0,255,0,0,153,0,255,
233,20,999,153,102,255,0,0,153,102,255,
234,20,999,153,204,255,0,0,153,204,255,
235,20,999,153,102,0,0,0,153,102,0,
236,20,999,204,102,0,0,0,204,102,0,
237,20,999,204,153,0,0,0,204,153,0,
238,20,999,51,51,0,0,0,51,51,0,
239,20,999,0,0,150,0,0,0,0,150,
240,20,999,102,51,51,0,0,102,51,51,
241,20,999,102,153,0,0,0,102,153,0,
242,20,999,153,153,0,0,0,153,153,0,
243,20,999,51,153,204,0,0,51,153,204,
244,20,999,102,102,153,0,0,102,102,153,
245,20,999,255,255,255,0,0,255,255,255,
246,20,999,255,255,255,0,0,255,255,255,
247,20,999,255,255,255,0,0,255,255,255,
248,20,999,0,0,0,0,0,0,0,0,
324,30,999,255,0,0,0,0,255,0,0,
325,30,999,0,255,0,0,0,0,255,0,
326,30,999,0,0,255,0,0,0,0,255,
327,30,999,255,255,0,0,0,255,255,0,
328,30,999,255,0,255,0,0,255,0,255,
329,30,999,0,255,255,0,0,0,255,255,
330,30,999,255,255,255,0,0,255,255,255,
331,30,999,155,0,0,0,0,155,0,0,
332,30,999,153,0,255,0,0,153,0,255,
333,30,999,153,102,255,0,0,153,102,255,
334,30,999,153,204,255,0,0,153,204,255,
335,30,999,153,102,0,0,0,153,102,0,
336,30,999,204,102,0,0,0,204,102,0,
337,30,999,204,153,0,0,0,204,153,0,
338,30,999,51,51,0,0,0,51,51,0,
339,30,999,0,0,150,0,0,0,0,150,
340,30,999,102,51,51,0,0,102,51,51,
341,30,999,102,153,0,0,0,102,153,0,
342,30,999,153,153,0,0,0,153,153,0,
343,30,999,51,153,204,0,0,51,153,204,
344,30,999,102,102,153,0,0,102,102,153,
345,30,999,255,255,255,0,0,255,255,255,
346,30,999,255,255,255,0,0,255,255,255,
347,30,999,255,255,255,0,0,255,255,255,
348,30,999,0,0,0,0,0,0,0,0,
424,40,999,255,0,0,0,0,255,0,0,
425,40,999,0,255,0,0,0,0,255,0,
426,40,999,0,0,255,0,0,0,0,255,
427,40,999,255,255,0,0,0,255,255,0,
428,40,999,255,0,255,0,0,255,0,255,
429,40,999,0,255,255,0,0,0,255,255,
430,40,999,255,255,255,0,0,255,255,255,
431,40,999,155,0,0,0,0,155,0,0,
432,40,999,153,0,255,0,0,153,0,255,
433,40,999,153,102,255,0,0,153,102,255,
434,40,999,153,204,255,0,0,153,204,255,
435,40,999,153,102,0,0,0,153,102,0,
436,40,999,204,102,0,0,0,204,102,0,
437,40,999,204,153,0,0,0,204,153,0,
438,40,999,51,51,0,0,0,51,51,0,
439,40,999,0,0,150,0,0,0,0,150,
440,40,999,102,51,51,0,0,102,51,51,
441,40,999,102,153,0,0,0,102,153,0,
442,40,999,153,153,0,0,0,153,153,0,
443,40,999,51,153,204,0,0,51,153,204,
444,40,999,102,102,153,0,0,102,102,153,
445,40,999,255,255,255,0,0,255,255,255,
446,40,999,255,255,255,0,0,255,255,255,
447,40,999,255,255,255,0,0,255,255,255,
448,40,999,0,0,0,0,0,0,0,0,
524,50,0,0,0,0,0,0,255,255,0,
525,50,20,15,999,0,0,0,255,255,0,
526,50,20,25,999,0,0,0,255,255,0,
527,50,20,50,999,0,0,0,255,255,0,
528,50,20,70,999,0,0,0,255,255,0,
529,50,20,90,999,0,0,0,255,255,0,
530,50,20,110,999,0,0,0,255,255,0,
531,50,20,140,999,0,0,0,255,255,0,
532,50,20,160,999,0,0,0,255,255,0,
533,50,20,180,999,0,0,0,255,255,0,
534,50,20,200,999,0,0,0,255,255,0,
535,50,20,220,999,0,0,0,255,255,0,
536,50,20,240,999,0,0,0,255,255,0,
537,50,120,15,999,0,0,0,255,255,0,
538,50,120,25,999,0,0,0,255,255,0,
539,50,120,50,999,0,0,0,255,255,0,
540,50,120,70,999,0,0,0,255,255,0,
541,50,120,90,999,0,0,0,255,255,0,
542,50,120,110,999,0,0,0,255,255,0,
543,50,120,140,999,0,0,0,255,255,0,
544,50,120,160,999,0,0,0,255,255,0,
545,50,120,180,999,0,0,0,255,255,0,
546,50,120,200,999,0,0,0,255,255,0,
547,50,120,220,999,0,0,0,255,255,0,
548,50,120,24,999,0,0,0,255,255,0

I have made an online editor to easly edit the channels.txt