Pages: [1]   Go Down
Author Topic: save data from HTTP Client sketch to buffer  (Read 254 times)
0 Members and 1 Guest are viewing this topic.
London, UK
Offline Offline
Full Member
***
Karma: 4
Posts: 189
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi I am using the HTTP Client sketch as seen below to load some data from a web page.  However this data can come in different sizes so I don't want to just define a massive array at the beginning and fill it as needed.  I would prefer to define an array of the required size.  I cant use eeprom or progmem because this read occurs once every 10 seconds.  The data ranges between the longest
Code:
8@xx50.00,xxx0.00:xx99.00,xxx0.00
xx50.00,xx50.00:xx50.00,xx50.00
xx50.00,xx50.00:xx50.00,xx50.00
xx50.00,xx50.00:xx50.00,xx50.00
xx50.00,xx50.00:xx50.00,xx50.00
xx50.00,xx50.00:xx50.00,xx50.00
xx50.00,xx50.00:xx50.00,xx50.00
xx50.00,xx50.00:xx50.00,xx50.00
and the shortest
Code:
1@xx50.00,xxx0.00:xx99.00,xxx0.00


Code:
void parseDeviceInfo(){
  delay(100);
  Serial.println("Device Info");
  while (deviceinfoclient.available()) {
    char c = deviceinfoclient.read();
    Serial.print(c);
  }
 
  // if the server's disconnected, stop the client:
  if (!deviceinfoclient.connected()) {
    Serial.println("disconnecting.");
    deviceinfoclient.stop();
    deviceinfoclient.flush();
 
    delay(3000);
 
    deviceinfoconnect();
  }
}
Logged

Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 614
Posts: 49343
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
However this data can come in different sizes so I don't want to just define a massive array at the beginning and fill it as needed.
I suggest that you get over it, then.

Quote
I would prefer to define an array of the required size.
You could use malloc() to allocate the space, but you'd have to free() it before the next packet arrived. So, what's the benefit?

Quote
I cant use eeprom or progmem because this read occurs once every 10 seconds.
That doesn't mean that you can't use EEPROM, but PROGMEM keeps data in program space, which is read-only. The interval between packets is irrelevant.

What are you doing with this data? How long do you need to persist it?
Logged

London, UK
Offline Offline
Full Member
***
Karma: 4
Posts: 189
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Could you provide some example code for the malloc()?
I thought that progmem had a limited amount of read/writes?
The data is then passed into specific buffers that are used for settings and configurations for my device.
Logged

UK
Offline Offline
Shannon Member
****
Karma: 223
Posts: 12630
-
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

If the incoming messages are big enough to potentially exhaust available memory then you should consider whether you need to buffer the whole message at all. You could either buffer and process individual tokens in your message, or process it byte by byte; either way would enable you to process the whole message without ever having to hold a copy of it in memory.

If you know you won't run out of memory but simply choose not to have the memory required to hold the complete message tied up permanently, and if you are willing to have your sketch block (stop) until the whole message has been received and processed, then you could use a local automatic variable as a buffer - this would be allocated on the stack and implicitly released as soon as the containing function returns. That's not a particularly elegant way to structure your sketch but enables you to allocate temporary memory without using heap memory allocation, with all the issues that introduces.
Logged

I only provide help via the forum - please do not contact me for private consultancy.

Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 614
Posts: 49343
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
Could you provide some example code for the malloc()?
I could, but I'm not going to. YOU can use google, or you could answer all my questions, first.

Quote
I thought that progmem had a limited amount of read/writes?
It does. You can only upload sketches so many times. Of course, for most of use that's many years worth of making mistakes.

Quote
The data is then passed into specific buffers that are used for settings and configurations for my device.
You pass data to functions; you store data in buffers. You do not pass data to buffers.
Logged

0
Offline Offline
Tesla Member
***
Karma: 143
Posts: 9601
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

The below textfinder function may be able to pick the data out of an incoming http stream. looks like the largest data stream will contain about 300 bytes, so I would think you could be able to allocate that much memory. There have been other post with code that extracts desired weather data from various web pages. 

http://playground.arduino.cc/Code/TextFinder
Logged

Consider the daffodil. And while you're doing that, I'll be over here, looking through your stuff.   smiley-cool

Pages: [1]   Go Up
Jump to: