Go Down

Topic: Parsing server data while playing .wav file with Arduino Wave Shield (Read 782 times) previous topic - next topic

newbie99

Dec 04, 2012, 12:43 am Last Edit: Dec 04, 2012, 11:05 pm by newbie99 Reason: 1
So here's what I'm trying to do...

I'm using the ethernet shield to get a large amount of data (around 200 entries) from my server. The code below is simplified. In my actual loop, I break the request out into sections. Each entry will have around 2-13 characters. Here's my code:

Code: [Select]

const char *stars[0] = { };
int countOn = 0;

void loop() {
 if (client.connected()) {
   if (client.available()) {
     char inChar = client.read();
     stars[countOn] = this_char;
     countOn = countOn + 1;
   }
 }
}


So after around 35-ish objects, my data starts to corrupt. So I'm guessing I'm running out of memory.

This is the last possible method I can use... Strings aren't long enough to hold my data, writing/reading file is too memory intensive... And since it's writing to PROGMEM, it's not using RAM.. So why is it freezing up?

Please help!

Thanks.

I've read these articles, btw.
http://www.avrfreaks.net/index.php?name=PNphpBB2&file=viewtopic&t=38003
http://electronics4dogs.blogspot.com/2010/12/simple-way-how-to-use-progmem.html?m=1
http://arduino.cc/en/Reference/PROGMEM

PaulS

Quote
I'm using the ethernet shield to get a large amount of data (around 200 entries) from my server.

That you would then like to store in read-only memory. Nope. It doesn't work that way.

Code: [Select]
const char *stars[0] = { };
The value in the [] is the number of elements in the array. 0? How useful is that?

Quote
So after around 35-ish objects, my data starts to corrupt. So I'm guessing I'm running out of memory.

No. You were stomping all over memory you don't own when the first character arrives. You are not out of memory. You are out of bounds.


dc42

You can't store variable data in PROGMEM, only in RAM or in EEPROM.

What form do the messages take? Do you need to store the entire incoming message, or can you compress or encode it? If you can halve the maximum size you need to store so that it is around 7 bytes maximum, then you can fit 200 messages in 1400 bytes, leaving you around 600 bytes (on a Uno) for other data and the stack.
Formal verification of safety-critical software, software development, and electronic design and prototyping. See http://www.eschertech.com. Please do not ask for unpaid help via PM, use the forum.

newbie99


You can't store variable data in PROGMEM, only in RAM or in EEPROM.

What form do the messages take? Do you need to store the entire incoming message, or can you compress or encode it? If you can halve the maximum size you need to store so that it is around 7 bytes maximum, then you can fit 200 messages in 1400 bytes, leaving you around 600 bytes (on a Uno) for other data and the stack.


Thanks for the reply. I'm relatively new to this... I develop iPhone apps, so I don't have to mess with all this memory stuff. I always have plenty of memory to work with..

But I get it in this format: "500(1467)" Before the parenthesis, I have a delay time. And in the parenthesis, I have the LEDs that I want on. I will then play the matching WAV file using WaveRP and loop through all the data I have saved up. I switch the LEDs according to the sequence values (in the parenthesis) & delay the amount of milliseconds (found before the parenthesis). The number of entries differ, but I would say it's between 200-300 values.

I've tried playing the song, and reading the data all in the main loop(), and it started acting weirdly (I'm guessing it ran out of memory, or was close to it). So I then tried writing all the data to a txt file, and running it later. But then I realized you can't read 2 files off the SD card at once *face palm*. I also tried appending it all to a String and reading it later, but after 40-ish characters the data went corrupt... And again, started acting weirdly.

What would you suggest I do?

dc42

What you need to do is parse the data either as you receive it or whenever you have received a complete message. That way you only need to store the 2 integers in the message, which takes 2 bytes per integer, assuming a 16-bit unsigned range is enough. So 4 bytes per message. If you have 300 of them, that's 1200 bytes, leaving 824 free out of the 2K available on most Arduinos.
Formal verification of safety-critical software, software development, and electronic design and prototyping. See http://www.eschertech.com. Please do not ask for unpaid help via PM, use the forum.

PaulS

Quote
What would you suggest I do?

Read all the replies - especially #1.

newbie99

#6
Dec 04, 2012, 11:04 pm Last Edit: Dec 04, 2012, 11:08 pm by newbie99 Reason: 1

Quote
What would you suggest I do?

Read all the replies - especially #1.


Okay, stupid mistakes... I've tried parsing the data as I receive a complete message:


What you need to do is parse the data either as you receive it or whenever you have received a complete message. That way you only need to store the 2 integers in the message, which takes 2 bytes per integer, assuming a 16-bit unsigned range is enough. So 4 bytes per message. If you have 300 of them, that's 1200 bytes, leaving 824 free out of the 2K available on most Arduinos.


Okay, that somewhat works. However, it stops randomly around halfway through the WAV file. I'm using this function to check the RAM usage...

Code: [Select]

int freeRam(void) {
 extern int  __bss_end;
 extern int  *__brkval;
 int free_memory;
 if((int)__brkval == 0) {
   free_memory = ((int)&free_memory) - ((int)&__bss_end);
 }
 else {
   free_memory = ((int)&free_memory) - ((int)__brkval);
 }
 return free_memory;
}


And I get around "271", so it shouldn't be a memory problem. The WAV file stops playing, but it continue to read data... so am I experiencing a memory problem or what?

dc42


The WAV file stops playing, but it continue to read data... so am I experiencing a memory problem or what?


How can we possibly tell without seeing all of your code?
Formal verification of safety-critical software, software development, and electronic design and prototyping. See http://www.eschertech.com. Please do not ask for unpaid help via PM, use the forum.

newbie99

Code: [Select]

#include <SPI.h>
#include <Ethernet.h>
#include <SdFat.h>
#include <SdFatUtil.h>
#include <WaveRP.h>

byte mac[] = { 0x00, 0xAA, 0xBB, 0xCC, 0xDE, 0x01 };
IPAddress ip(192,168,1,100);
EthernetClient client;
char serverName[] = "justtwobros.com";

int delayOn = 5000;
int delayOff = 900000;
int delaySong = 60000;

boolean readingData = false;
String tempString = "";
String tempoString = "";

#define number_of_74hc595s 1
#define numOfRegisterPins number_of_74hc595s * 8
boolean registers[numOfRegisterPins];
int latchPin = A2;
int clockPin = A5;
int dataPin = A3;

SdFat sd;
SdFile myFile;
WaveRP wave;
#define error(msg) sd.errorHalt_P(PSTR(msg))

int freeRam(void) {
  extern int  __bss_end;
  extern int  *__brkval;
  int free_memory;
  if((int)__brkval == 0) {
    free_memory = ((int)&free_memory) - ((int)&__bss_end);
  }
  else {
    free_memory = ((int)&free_memory) - ((int)__brkval);
  }
  return free_memory;
}

void setup() {
  Serial.begin(9600);
  if (!sd.begin(9, SPI_HALF_SPEED)) sd.initErrorHalt();
  Ethernet.begin(mac, ip);
  Serial.println(Ethernet.localIP());
  connectToServer();
}

void loop() {
  if (client.connected()) {
    if (client.available()) {
      char inChar = client.read();
      if (!readingData) {
        if (inChar == '{') {
          readingData = true;
        }
      } else {
        if (inChar == '}') {
          playfile("2.WAV");
          tempString = "";
        }  else if (inChar == '(') {
          tempoString = tempString;
          tempString = "";
        } else if (inChar == ')') {
          Serial.println(tempString);
          delay(tempoString);
          tempString = "";
        } else if (inChar == ']') {
          client.stop();
          readingData = false;
          tempString = "";
        } else if (inChar == '[') { } else {
          tempString += inChar;
        }
      }
    }
  }
}

void connectToServer() {
  if (client.connect(serverName, 80)) {
      client.print("GET http://justtwobros.com/christmas/scripts/get_sequence.php");
      client.println(" HTTP/1.1");
      client.println("Host: justtwobros.com");
      client.println();
  }
}

void setRegisterPin(int index, int value){ registers[index] = value; }

void writeRegisters(){
  digitalWrite(latchPin, LOW);
  for(int i = numOfRegisterPins - 1; i >=  0; i--){
    digitalWrite(clockPin, LOW);
    int val = registers[i];
    digitalWrite(dataPin, val);
    digitalWrite(clockPin, HIGH);
  }
  digitalWrite(latchPin, HIGH);
}

void playfile(char *name) {
  if (wave.isPlaying()) {// already playing something, so stop it!
    wave.stop(); // stop it
  }
  myFile.close();  // SdFat will not open a file that is already open
  if (!myFile.open(name, O_READ)) {
    PgmPrint("Couldn't open file ");
    Serial.println(name);
    return;
  }
  if (!wave.play(&myFile)) {
    PgmPrint("Not a valid WAV ");   
    Serial.println(name);     
    return;
  }
}



This should be all of it...

dc42

Some suggestions:

1. Don't use the String class, it causes memory fragmentation. Learn to use character arrays instead. You can use the atoi function for extracting and integer form a character array. Be sure to write your code so that you do not overflow the character array if the input does not conform to expectations.

2. If the Ethernet client library supports it, use F strings when printing string literals to it, like this:

Code: [Select]
     client.print(F("GET http://justtwobros.com/christmas/scripts/get_sequence.php"));
     client.println(F(" HTTP/1.1"));
     client.println(F("Host: justtwobros.com"));


This will avoid those strings using RAM.
Formal verification of safety-critical software, software development, and electronic design and prototyping. See http://www.eschertech.com. Please do not ask for unpaid help via PM, use the forum.

newbie99


Some suggestions:

1. Don't use the String class, it causes memory fragmentation. Learn to use character arrays instead. You can use the atoi function for extracting and integer form a character array. Be sure to write your code so that you do not overflow the character array if the input does not conform to expectations.

2. If the Ethernet client library supports it, use F strings when printing string literals to it, like this:

Code: [Select]
     client.print(F("GET http://justtwobros.com/christmas/scripts/get_sequence.php"));
     client.println(F(" HTTP/1.1"));
     client.println(F("Host: justtwobros.com"));


This will avoid those strings using RAM.


Thanks for your response! I tried both, and I still have the same problem. Here's all of my code:

Code: [Select]

#include <SPI.h>
#include <Ethernet.h>
#include <SdFat.h>
#include <SdFatUtil.h>
#include <WaveRP.h>

byte mac[] = { 0x00, 0xAA, 0xBB, 0xCC, 0xDE, 0x01 };
IPAddress ip(192,168,1,100);
EthernetClient client;
char serverName[] = "justtwobros.com";

int delayOn = 5000;
int delayOff = 900000;
int delaySong = 60000;

boolean readingData = false;
char tempString[14];
char tempoString[4];

#define number_of_74hc595s 1
#define numOfRegisterPins number_of_74hc595s * 8
boolean registers[numOfRegisterPins];
int latchPin = A2;
int clockPin = A5;
int dataPin = A3;

SdFat sd;
SdFile myFile;
WaveRP wave;
#define error(msg) sd.errorHalt_P(PSTR(msg))

int freeRam(void)
{
  extern int  __bss_end;
  extern int  *__brkval;
  int free_memory;
  if((int)__brkval == 0) {
    free_memory = ((int)&free_memory) - ((int)&__bss_end);
  }
  else {
    free_memory = ((int)&free_memory) - ((int)__brkval);
  }
  return free_memory;
}

void setup() {
  pinMode(latchPin, OUTPUT);
  pinMode(clockPin, OUTPUT);
  pinMode(dataPin, OUTPUT);
  digitalWrite(9, LOW);
  digitalWrite(10, LOW);
  Serial.begin(9600);
  if (!sd.begin(9, SPI_HALF_SPEED)) sd.initErrorHalt();
  Ethernet.begin(mac, ip);
  connectToServer();
}

void clearTemp() {
  for (int i=0; i<sizeof(tempString); i++) {
    tempString[i] = '\0';
  }
}

void loop() {
  if (client.connected()) {
    if (client.available()) {
      char inChar = client.read();
      if (!readingData) {
        if (inChar == '{') {
          readingData = true;
        }
      } else {
        if (inChar == '}') {
          Serial.print("5. freeram: ");
          Serial.println(freeRam());
          // enableAudio();
          //String songIDWav = tempString;
          //songIDWav += ".WAV";
          //char this_char2[songIDWav.length() + 1];
          //songIDWav.toCharArray(this_char2, sizeof(this_char2));
          //Serial.println(this_char2);
          playfile("2.WAV");
        } else if (inChar == 'n') {
          allOff();
        } else if (inChar == 'o') {
          allOn();
        } else if (inChar == '(') {
          // SAVE TEMPO HERE
          clearTemp();
        } else if (inChar == ')') {
          Serial.println(tempString);
          clearTemp();
          delay(500);
        } else if (inChar == ']') {
          client.stop();
          readingData = false;
          clearTemp();
        } else if (inChar == '[') { } else {
          char str2[2]; str2[0] = inChar; str2[1] = '\0';
          strcat (tempString, str2);
        }
      }
    }
  }
}

void connectToServer() {
  if (client.connect(serverName, 80)) {
      client.print(F("GET http://justtwobros.com/christmas/scripts/get_sequence.php"));
      client.println(F(" HTTP/1.1"));
      client.println(F("Host: justtwobros.com"));
      client.println();
  }
}

void setRegisterPin(int index, int value){ registers[index] = value; }

void writeRegisters(){
  digitalWrite(latchPin, LOW);
  for(int i = numOfRegisterPins - 1; i >=  0; i--){
    digitalWrite(clockPin, LOW);
    int val = registers[i];
    digitalWrite(dataPin, val);
    digitalWrite(clockPin, HIGH);
  }
  digitalWrite(latchPin, HIGH);
}
void allOff(){
  digitalWrite(latchPin, LOW);
  for(int i = numOfRegisterPins - 1; i >=  0; i--){
    digitalWrite(clockPin, LOW);
    digitalWrite(dataPin, LOW);
    digitalWrite(clockPin, HIGH);
  }
  digitalWrite(latchPin, HIGH);
  delay(delayOff);
  resetData();
}

void allOn(){
  digitalWrite(latchPin, LOW);
  for(int i = numOfRegisterPins - 1; i >=  0; i--){
    digitalWrite(clockPin, LOW);
    digitalWrite(dataPin, HIGH);
    digitalWrite(clockPin, HIGH);
  }
  digitalWrite(latchPin, HIGH);
  delay(delayOn);
  resetData();
}

void playfile(char *name) {
  if (wave.isPlaying()) {// already playing something, so stop it!
    wave.stop(); // stop it
  }
  myFile.close();  // SdFat will not open a file that is already open
  if (!myFile.open(name, O_READ)) {
    PgmPrint("Couldn't open file ");
    Serial.println(name);
    return;
  }
  if (!wave.play(&myFile)) {
    PgmPrint("Not a valid WAV ");   
    Serial.println(name);     
    return;
  }
}

void resetData() {
  client.stop();
  readingData = false;
  clearTemp();
  connectToServer();
}

PaulS

Code: [Select]
int delayOff = 900000;
You need a bigger shoehorn. Look at the range of values you can store in an int. 900,000 won't even come close to fitting.

Code: [Select]
int delaySong = 60000;
Nor will 60,000.

Go Up