Increase Arduino upload speed using the WifFi Shield

Hello,

I have a sketch in which I send the data from the ADC to a computer via the WiFi shield. The problem is that the data transfer takes up to 50 seconds for 1000 pints. I have tried to send the vector as a whole, but it does not work.

Can anyone tell me if there is another way to send data faster?

Best regards,
Jean

I cannot find your sketch in the post? Did you forget to post it? Did you ignore the sticky post at the top of the forum?

Hello,

Sorry about that, here is my code from teh loop:

void loop() {
for (int analogChannel = 0; analogChannel < 10000; analogChannel++) {
v[analogChannel] = analogRead(0);
// delayMicroseconds(10);
}
// listen for incoming clients
WiFiClient client = server.available();
if (client) {
Serial.println(“new client”);
// an http request ends with a blank line
boolean currentLineIsBlank = true;
while (client.connected()) {
if (client.available()) {
char c = client.read();
Serial.write(c);
// if you’ve gotten to the end of the line (received a newline
// character) and the line is blank, the http request has ended,
// so you can send a reply
if (c == ‘\n’ && currentLineIsBlank) {
// send a standard http response header
client.println(“HTTP/1.1 200 OK”);
client.println(“Content-Type: text/html”);
client.println(“Connection: close”); // the connection will be closed after completion of the response
client.println(“Refresh: 60”); // refresh the page automatically every 5 sec
client.println();
client.println("");
client.println("");
// output the value of each analog input pin
// for (int analogChannel = 0; analogChannel < 1000; analogChannel++) {
// v[analogChannel] = analogRead(0);
// delayMicroseconds(10);
/* client.print(“analog input “);
client.print(analogChannel);
client.print(” is “);/
/
client.print(v[analogChannel]);
client.println(”
“);*/
// }
for (int k= 0; k < 1000; k++) {
client.println(v[k]);
client.println(”
“);
}
client.println(””);
break;
}
if (c == ‘\n’) {
// you’re starting a new line
currentLineIsBlank = true;
}
else if (c != ‘\r’) {
// you’ve gotten a character on the current line
currentLineIsBlank = false;
}
}
}
// give the web browser time to receive the data
delay(1);

// close the connection:
client.stop();
Serial.println(“client disonnected”);
}
}

It is a modified example from Arduino. The A/D conversion works fast, but the web page transfer is slow, about 60 seconds for 1000 samples.

You forgot to use code tags, these are important because otherwise the forum system may hide some parts of the code.

And post the complete code not just parts of it because the declaration of variables and the initialization of them may be relevant.

pylon is right. Post all your code, and use the code tags.

The slow upload speed is due to your code sending one byte packets. This flaw makes uploads painfully slow when using the wifi server with a SD card file. This is the way I send 64 byte packets with the wifi and the SD. It gives you about a 4x increase in upload speed.

                  while(myFile.available()) {
                    tBuf[clientCount] = myFile.read();
                    clientCount++;
                    tBuf[clientCount] = 0;

                    if(clientCount > 63) {
                      client.write((byte*)tBuf,64);
                      clientCount = 0;
                    }

                  }
                  if(clientCount > 0) {
                    client.write((byte*)tBuf,clientCount);
                  }

Warning: The wifi library/firmware has a serious bug in the server section. It can cause incorrect or corrupted files if more than one client attempts to connect simultaneously. It also inserts a rather nasty 2 second delay every few seconds during a large file upload. :(

Hello,

Thank you for your reply. Here is the full code:

/*
  WiFi Web Server

 A simple web server that shows the value of the analog input pins.
 using a WiFi shield.

 This example is written for a network using WPA encryption. For
 WEP or WPA, change the Wifi.begin() call accordingly.

 Circuit:
 * WiFi shield attached
 * Analog inputs attached to pins A0 through A5 (optional)

 created 13 July 2010
 by dlf (Metodo2 srl)
 modified 18.03.2014
 by Jean Baptiste

 */
int v[10000];
#include <SPI.h>
#include <WiFi.h>


char ssid[] = "TP_LINK";      // your network SSID (name)
//char pass[] = "secretPassword";   // your network password - no need, no password
int keyIndex = 0;                 // your network key Index number (needed only for WEP) 

int status = WL_IDLE_STATUS;

WiFiServer server(80);

void setup() {
  //Initialize serial and wait for port to open:
  Serial.begin(9600);
  pinMode(22, OUTPUT);  // no wifi whield - needs reset
  pinMode(24, OUTPUT);  // connected wifi - in order to begin Matlab (client) request
  
   digitalWrite(22, LOW); // not yet connected
   digitalWrite(24, LOW); // not yet detected
  while (!Serial) {
    ; // wait for serial port to connect. Needed for Leonardo only
  }
  analogReadResolution(12); // set acquisiiton resolutin to 12 bits
  // check for the presence of the shield:
  if (WiFi.status() == WL_NO_SHIELD) {
    Serial.println("WiFi shield not present");
    digitalWrite(22, HIGH); // highlight no wifi shield
    // don't continue:
    while (true);
  }

  String fv = WiFi.firmwareVersion();
  if ( fv != "1.1.0" )
    Serial.println("Please upgrade the firmware");

  // attempt to connect to Wifi network:
  while ( status != WL_CONNECTED) {
    Serial.print("Attempting to connect to SSID: ");
    Serial.println(ssid);
    // Connect to WPA/WPA2 network. Change this line if using open or WEP network:
    status = WiFi.begin(ssid);

    // wait 10 seconds for connection:
    delay(10000);
  }
  server.begin();
   digitalWrite(24, HIGH); // connected to network
  // you're connected now, so print out the status:
  printWifiStatus();
}


void loop() {   // main loop where I perform acquisition and send data
  for (int analogChannel = 0; analogChannel < 10000; analogChannel++) { // aquire samples
            v[analogChannel] = analogRead(0);
        //    delayMicroseconds(10);
          }
  // listen for incoming clients
  WiFiClient client = server.available();
  if (client) {
    Serial.println("new client");
    // an http request ends with a blank line
    boolean currentLineIsBlank = true;
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();
        Serial.write(c);
        // if you've gotten to the end of the line (received a newline
        // character) and the line is blank, the http request has ended,
        // so you can send a reply
        if (c == '\n' && currentLineIsBlank) {
          // send a standard http response header
          client.println("HTTP/1.1 200 OK");
          client.println("Content-Type: text/html");
          client.println("Connection: close");  // the connection will be closed after completion of the response
          client.println("Refresh: 60");  // refresh the page automatically every 5 sec
          client.println();
          client.println("<!DOCTYPE HTML>");
          client.println("<html>");
          // sendig data ---- here it takes so slow.
        for (int k= 0; k < 1000; k++) {
          client.println(v[k]);
          client.println("
");
          }
          client.println("</html>"); 
          break;
        }
        if (c == '\n') {
          // you're starting a new line
          currentLineIsBlank = true;
        }
        else if (c != '\r') {
          // you've gotten a character on the current line
          currentLineIsBlank = false;
        }
      }
    }
    // give the web browser time to receive the data
    delay(1);

    // close the connection:
    client.stop();
    Serial.println("client disonnected");
  }
}


void printWifiStatus() {
  // print the SSID of the network you're attached to:
  Serial.print("SSID: ");
  Serial.println(WiFi.SSID());

  // print your WiFi shield's IP address:
  IPAddress ip = WiFi.localIP();
  Serial.print("IP Address: ");
  Serial.println(ip);

  // print the received signal strength:
  long rssi = WiFi.RSSI();
  Serial.print("signal strength (RSSI):");
  Serial.print(rssi);
  Serial.println(" dBm");
}

Based on your reply I will modify my program. However, I am not sending data from a SD card, the samples are in µP memory.
I did not know that the amount of data sen,t was so small… where can I find more references?

Best regards,
Jean

int v[10000];

Which Arduino are you using?

Hello,

I am using an Arduino DUE board.

Thank you for your message.

I am using an Arduino DUE board.

Nevertheless, you’re allocate ten times as much memory as you need.

If you use the technique SurferTim suggested, you’ll get much more speed because the print() and println() methods of the Print class call the write method for every character they’re told to transmit. Because the WiFi shield doesn’t have any hints about when the data stream is finished it has no other possibility than to send every single character in a separate packet. If you create a string containing the complete answer and send that using the two argument write() method you’ll get much more throughput. Probably the easiest method to create that string is using the sprintf() function (snprintf - C++ Reference). Be aware that the version of snprintf() supplied with the Arduino IDE does not support the %f placeholder (floats) as far as I know.

Hello,

Thank you for your reply. I have seen that other users are advising for using the "sprintf" function. However, I am kind of new at this Arduino (I am more at ease with Matlab) and I do not understand fully how to use this function.

So I have to avoid the client.println() function and use sprintf(). How does the Arduino know that calling sprintf() will have to send data to the client?

Regarding the length of "v", I am currently trying to see the limits of the data I have to send. The array will contain the signal sampled form the A0 analog input. Then I will pick it up via WiFi in Matlab in order to process it.

Best regards, Jean

So I have to avoid the client.println() function and use sprintf(). How does the Arduino know that calling sprintf() will have to send data to the client?

First of all, I suggest to use the snprintf variant because it takes care of length limitations (see the link in my last post). As I wrote, snprintf() just helps you to fill the string with content. You will then have to send that string using the two argument version of the write() method of the WiFiClient class.

What kind of signal do you want to sample with this sketch?

Hello,

Thank you for your message. It is a small signal form a sensor.

My biggest problem is that I cannot see what it is inside the variables I use in the sketch. Do you know where I can find a c programming interface that will display exactly the contents of my variables?

Second, I have this problem:

char *buf = (char*)calloc(10000 + 1, sizeof(char));
snprintf(buf, 10000, "%c ", v);
for ( int clientCount = 0;  clientCount < 10000;  clientCount++)
     {
                                // buff[clientCount] = 0;

                    if(clientCount > 63) {
                      client.write((byte*)buff,64);
                      clientCount = 0;
                    }

                  }
                  if(clientCount > 0) {
                    client.write((byte*)buff,clientCount);
                  }

I obtain the following error: error: ‘buff’ was not declared in this scope for the client .write() line. Since the buff already has a string, why do I obtain this error.
It is very frustrating that you are not capable of seeing the contents of the variables as the program runs.

Thank you,
Jean

My biggest problem is that I cannot see what it is inside the variables I use in the sketch. Do you know where I can find a c programming interface that will display exactly the contents of my variables?

Do you mean a debugger? Unfortunately that doesn’t exist (it’s not possible) for the Arduino platform.

I obtain the following error: error: ‘buff’ was not declared in this scope for the client .write() line. Since the buff already has a string, why do I obtain this error.

You’ve declared “buf” not “buff”, one character difference in the name is relevant in all programming languages I know of.

BTW: This does not work as you probably intended:

char *buf = (char*)calloc(10000 + 1, sizeof(char));
snprintf(buf, 10000, "%c ", v);

You should read the man page I linked for snprintf. You need to use it like this:

          client.println("HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nConnection: close\r\nRefresh: 60\r\n\r\n<!DOCTYPE HTML>\r\n<html>");// print header
          char buf[1000*(5+6) + 1]; // reserve space for 1000 integer html lines
          int l = 0;
          int i;
          for (i = 0; i < 1000; i++) {
            l += snprintf(&buf[l], 1000-l, "%d
", v[i]) - 1;
          }
          client.write(buf, l);

Hello,

Thank you for your message. Unfortunately, the program did not work, now I do not receive anything on my computer from the Arduino. I have traced the problem to the client.write() command. From what I have understood, this command does exactly the opposite of what I want. I want to send data form the Arduino (server) to a local computer (client) via WiFi. However, the client.write() command writes data from the client to the server the client is connected to…

Based on your code you sent me, I was able to send using client.println() the data:

for (i = 0; i < 1000; i++) {
l += snprintf(&buf[l], 1000-l, "%d
", v*) - 1;*

  • client.println(l);*
  • }*
  • Since I cannot see the content of “l”, I would like to know what is actually inside of “l” at the end of the for loop? Do I have the entire vector of values? can I send it all with one printf command?*
    Also, why it is that sending such a small amount of data is taking so long? There are web pages of files that are send within seconds between PCs over WiFi? Could the UDP protocol be faster?
    Best regards,
    Jean

Use code tags if posting code!! Your code is a sample of code interpreted by the forum system in an unexpected way.

I want to send data form the Arduino (server) to a local computer (client) via WiFi. However, the client.write() command writes data from the client to the server the client is connected to.

The roles client and server are defined by whom is opening the connection. client.write() as well as client.println() sends something from the Arduino to the connecting PC, no matter which role the Arduino currently has.

Unfortunately, the program did not work

Which program? I did only supply a code fragment not a finished program. You are responsible to integrate that into your program. Post the sketch you generated out of it.

Hello,

Thank you for your message and for the clarification between the different commands. Again, where do you find this “extra info” on the commands, because the Arduino web site does not provide enough.

Concerning the program, I just replaced the part made with the println command with your code. Here is the new program:

/*
  WiFi Web Server

 A simple web server that shows the value of the analog input pins.
 using a WiFi shield.

 This example is written for a network using WPA encryption. For
 WEP or WPA, change the Wifi.begin() call accordingly.

 Circuit:
 * WiFi shield attached
 * Analog inputs attached to pins A0 through A5 (optional)

 created 13 July 2010
 by dlf (Metodo2 srl)
 modified 18.03.2014
 by Jean Baptiste

 */
int v[10000];
#include <SPI.h>
#include <WiFi.h>

char data[10000];

char ssid[] = "TP_LINK";  
// your network SSID (name)
// your network password - no need, no password
int keyIndex = 0;                 // your network key Index number (needed only for WEP) 

int status = WL_IDLE_STATUS;

WiFiServer server(80);

void setup() {
  //Initialize serial and wait for port to open:
  Serial.begin(9600);
  pinMode(22, OUTPUT);  // no wifi whield - needs reset
  pinMode(24, OUTPUT);  // connected wifi - in order to begin Matlab (client) request
  
   digitalWrite(22, LOW); // not yet connected
   digitalWrite(24, LOW); // not yet detected
  while (!Serial) {
    ; // wait for serial port to connect. Needed for Leonardo only
  }
  analogReadResolution(12); // set acquisiiton resolutin to 12 bits
  // check for the presence of the shield:
  if (WiFi.status() == WL_NO_SHIELD) {
    Serial.println("WiFi shield not present");
    digitalWrite(22, HIGH); // highlight no wifi shield
    // don't continue:
    while (true);
  }

  String fv = WiFi.firmwareVersion();
  if ( fv != "1.1.0" )
    Serial.println("Please upgrade the firmware");

  // attempt to connect to Wifi network:
  while ( status != WL_CONNECTED) {
    Serial.print("Attempting to connect to SSID: ");
    Serial.println(ssid);
    // Connect to WPA/WPA2 network. Change this line if using open or WEP network:
    status = WiFi.begin(ssid);

    // wait 10 seconds for connection:
    delay(10000);
  }
  server.begin();
   digitalWrite(24, HIGH); // connected to network
  // you're connected now, so print out the status:
  printWifiStatus();
}

void loop() {   // main loop where I perform acquisition and send data
  for (int analogChannel = 0; analogChannel < 10000; analogChannel++) { // aquire samples
            v[analogChannel] = analogRead(0);
        //    delayMicroseconds(10);
          }
         // listen for incoming clients
  WiFiClient client = server.available();
  if (client) {
    Serial.println("new client");
    // an http request ends with a blank line
    boolean currentLineIsBlank = true;
    while (client.connected()) {
       Serial.println("Client connected");
      if (client.available()) {
          char c = client.read();
      //    Serial.write(c);
        //   Serial.write("  ");
        Serial.println("Client available");
        // if you've gotten to the end of the line (received a newline
        // character) and the line is blank, the http request has ended,
        // so you can send a reply
        if (c == '\n' && currentLineIsBlank) {
          // send a standard http response header
        /*  client.println("HTTP/1.1 200 OK");
          client.println("Content-Type: text/html");
          client.println("Connection: close");  // the connection will be closed after completion of the response
          client.println("Refresh: 60");  // refresh the page automatically every 5 sec
          client.println();
          client.println("<!DOCTYPE HTML>");
          client.println("<html>");
          int tr[64];*/
          // sendig data ---- here it takes so slow.
          
          
          Serial.println("Begin writing 2 client");
          client.println("HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nConnection: close\r\nRefresh: 60\r\n\r\n<!DOCTYPE HTML>\r\n<html>");// print header
          char buf[1000*(5+6) + 1]; // reserve space for 1000 integer html lines
          int l = 0;
          int i;
          for (i = 0; i < 1000; i++) {  
            l += snprintf(&buf[l], 1000-l, "%d X
", v[i]) - 1;
                   
        client.write(buf, l);
               }
                break;
        }
          }
    }
    // give the web browser time to receive the data
    delay(0.1);

    // close the connection:
    client.stop();
    Serial.println("client disonnected");
  }
}


void printWifiStatus() {
  // print the SSID of the network you're attached to:
  Serial.print("SSID: ");
  Serial.println(WiFi.SSID());

  // print your WiFi shield's IP address:
  IPAddress ip = WiFi.localIP();
  Serial.print("IP Address: ");
  Serial.println(ip);

  // print the received signal strength:
  long rssi = WiFi.RSSI();
  Serial.print("signal strength (RSSI):");
  Serial.print(rssi);
  Serial.println(" dBm");
}

Following this, here is what I obtain via the serial port when the Arduino connects to the network and after that I try to receive the data in a web page or with Matlab (function urlread(‘http://192.168.1.99’):

P Address: 192.168.1.99
signal strength (RSSI):-42 dBm
new client
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Begin writing 2 client
client disonnected
new client
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Client connected
Client available
Begin writing 2 client
client disonnected

In my opinion the Arduino (server) waits fot the PC (client) to send data, but this data never comes… or something. When switch back ti the println function, the data transfer is possible, but at a slow rate.

Best regards,
Jean

In this code part:

          for (i = 0; i < 1000; i++) {  
            l += snprintf(&buf[l], 1000-l, "%d X
", v[i]) - 1;
                   
        client.write(buf, l);
               }

the client.write() call has to be outside the for loop to work correctly:

          for (i = 0; i < 1000; i++) {  
            l += snprintf(&buf[l], 1000-l, "%d X
", v[i]) - 1;
          }
          client.write(buf, l);

The buffer “buf” gets filled with the integer HTML lines and when it’s full, you should write it to the network and not in every loop while you’re filling it.

Then try to connect using a browser or even better, using the telnet command if you know how to do that. Post what you get.

Hello,

Thank you for your message. Unfortunately, putting the write command outside the for loop did not work. The client (PC) manages to connect, but there is no file transfer. I think that the Arduino expects the PC to send data (the write command).

Best regards, Jean

I had problems sending packets larger than 90 bytes with the wifi shield, so I reduced the packet size to 64 bytes and all works ok. The ethernet shield doesn't have that problem.

Hello

          char buf[1000*(5+6) + 1]; // reserve space for 1000 integer html lines
          int l = 0;
          int i;
          for (i = 0; i < 1000; i++) {  
            l += snprintf(&buf[l], 1000-l, "%d X
", v[i]) - 1;
                   
        client.write(buf, l);
               }
                break;

I don’t think that buf[11001] and then client.write(but) Will work !

I would do something like that :
Char buf[60];
for {int i=0; i<1000;i=i+5} {
sprintf(buf, "%d
%d
%d
%d
%d
",v*,v[i+1] …v[i+4]);*

  • client.write(buf);*
    }
    So you write larger paquets … But not one of 11001 chars !