FTP over WIFI network

Renzocat · October 14, 2013, 9:35pm

Hi @ll,
I'm so darn new with this Arduino....
ok, now my question:
I wanna upload a file from an SD card,in FTP over wifi in my webspace i have
is this possible?
I know: my wifi network data (SSID, PSW) and my FTP data (FTPHost, User, Psw)

Thanks soooo much to anyone can help me
Renzo.

SurferTim · October 15, 2013, 1:48am

Here is my FTP client code for the ethernet shield. Replace the ethernet startup with your wifi startup in the setup routine. When the wifi device is started and connected, all the rest should be the same.
http://playground.arduino.cc/Code/FTP

edit: You must change these also:

// change these
EthernetClient client;
EthernetClient dclient;
// to these
WiFiClient client;
WiFiClient dclient;

system · October 27, 2013, 5:22pm

HI, I'm creating the same system and I keep getting the "Command Connection failed". I did the same steps as Surfer Tim and still no connection. I tried connecting to Core FTP on Windows and to vsftpd on Linux and got the same error. Help, guys.
P.S. I've attached my sketch, maybe you'll find some mistakes in it.

WiFi_FTP.ino (6.82 KB)

SurferTim · October 28, 2013, 9:01am

Have you tried the wifi shield with other protocols? There have been some issues reported concerning the firmware on the wifi shield. If you are using IDE v1.0.5, you probably need to update the firmware on the shield.

If you have access to IDE v1.0.3, you might try that version. Other users report it works with the older firmware.

system · October 28, 2013, 10:02am

The error indicates the server is not responding to the TCP connection request.

The first thing you need to do is check the server is accepting connections on the port you have specified (21).
The easiest way to do that is using telnet, in a command console on your PC or Mac. Note you will need to install the Telnet Client if you are using Windows Vista or later.

Open the MSDOS prompt on Windows or the Terminal on OSX.
Type
telnet 192.168.0.106 21
You should get a response from the sever with a status code ~220
If so, type
bye
To close the connection.

To fault find further, here is a little sketch which simply establishes a ftp connection and disconnects.
Note. I have removed your WiFi password and you will need to insert it into the sketch as indicated.

If you can connect using telnet from your PC but not using my sketch, let me know what output you get from the sketch.

#include <Streaming.h>
#include <SPI.h>
#include <WiFi.h>

char ssid[] = "dlink38";
char pass[] = "<insert your password>";

void setup(void) {
  Serial.begin(9600);
  Serial << "Setup..\r\n";
  
  if (WiFi.status() == WL_NO_SHIELD) {
    Serial << "Can not continue\r\n";
    while(true) delay(500);
  } 
  
  while (WiFi.status() != WL_CONNECTED) {
    Serial << "Try connect to " << ssid << " ...\r\n";
    WiFi.begin(ssid, pass);
  }
  
  Serial << "Joined " << WiFi.SSID()  << " RSSI " << WiFi.RSSI() << " dbm\r\n"; 
  
  Serial << "End setup.\r\n";
}

IPAddress ftpServerIP(192, 168, 0, 106);
const uint16t ftpServerCmdPort = 21;

WiFiClient ftpCmdConnection;
WiFiClient ftpDataConnection;

uint8_t ftpMaxRetry = 3;
bool ftpStateCmdConnected = false;
uint8_t ftpStateRetry = 0;

void loop(void) {

  while (WiFi.status() != WL_CONNECTED) {
    Serial << "Connection lost. Trying " << ssid << " ...\r\n";
    WiFi.begin(ssid, pass);
  }
  
  do {
    if (ftpCmdConnection.connect(ftpServerIP, ftpServerCmdPort)) {
      Serial << "ftp, Try connected to server\r\n";
      ftpStateCmdConnected = true;
    }
    else {
      Serial << "ftp, Failed connect to server\r\n";
    }
  } while ((!ftpStateCmdConnected) && (ftpStateRetry++ < ftpMaxRetry));   
  
  if (ftpStateCmdConnected) {
    Serial << "ftp, Connected.\r\n";
    delay(10000);
    Serial << "ftp, Disconnect\r\n";
    ftpCmdConnection.stop();
  }
    
  Serial << "Loop stop.\r\n";
  while (true) delay(500); 
  
}

system · October 28, 2013, 3:57pm

Hi, MattS-UK.
Thanks for the answer, but I still can't connect to ftp. Using your sketch I successfully connected to my wifi, but I get "ftp, Failed to connect to server" error. I tested the connection via telnet as you said and tried simply typing ftp address in Chrome. Both test were successful: with telnet I got status code 220 and in Chrome I logged in to ftp. Maybe some special server settings are needed to connect Arduino? Maybe it's all Arduino's fault? Please reply if you have any ideas.

system · October 28, 2013, 9:11pm

I did check the sketch works with my own shield.
I am using Arduino IDE version 1.0.5 and WiFi.version() returns 1.1.0
Here is the output I am getting

Setup..
Try connect to wap1 ...
Joined wap1 RSSI -63 dbm
End setup.
ftp, Try connect to server
ftp, Connected.
ftp, Disconnect
Loop stop.

Maybe it's all Arduino's fault?

The fault is either of
Your Arduino board.
Your WiFi Shield.
Your Access Point.
Your FTP Server.

Presuming your Arduino board shows no other sign of failure,
I would be wondering whether the WiFi shield can connect to anything.

Using the same sketch, try changing the following as a test.

//google DNS server
IPAddress ftpServerIP(8.8.8.8);
const uint16_t ftpServerCmdPort = 53;

//google web server
IPAddress ftpServerIP(212.56.71.170);
const uint16_t ftpServerCmdPort = 80;

Also, check your RSSI (signal strength).
Lower than ~ -70dbm, my shield starts to play up.

[edit: sanitisation.]

system · October 29, 2013, 5:04pm

I tried testing the sketch as you suggested and it didn't work. I checked my wifi shield software version and it is lower than yours(1.0.0) so I guess this might be the problem. The RSSI value is -33 dBm which is good so it's not the problem. I've found an article about updatiing wifi software on arduino.cc. And though I've got an official Arduino wifi shield there's no manual on updating my model. And when I followed the link to github repository with firmware for a similar model, I found out that my actual firmware is newer than the one on github. These are the results of the sketch.

Setup..

Try connect to dlink38 ...

Joined dlink38 RSSI -33 dbm

WiFi version 1.0.0
End setup.

ftp ip address 212.56.71.170
ftp, Failed connect to server

ftp, Failed connect to server

ftp, Failed connect to server

ftp, Failed connect to server

Loop stop.

So now the only thing I can do is updating the firmware? Or there are any other options?

SurferTim · October 29, 2013, 6:20pm

Use a known good ftp site ip. This is a movie download site. You should be able to connect here.
194.44.214.3

edit: I always try the connection with a computer before trying it with my Arduino.

system · October 29, 2013, 6:42pm

Your RSSI sounds perfectly fine, so it will be something else. If you can not connect to any site using any port, it is most likely a problem at your end.

The WiFi shield firmware must match the version of the library code you are using. So if you are using Arduino 1.0.5, you will need to upgrade the firmware. You might try installing an earlier version of the Arduino IDE but you will not be able to use UDP, if the need arises. My choice would be to persevere with the upgrade.

The firmware files you ran into on Github are most likely from the old development snapshot, which is known to be problematic and best avoided.

To get the release files. Go to the Arduino IDE Download page, by following the link at the top of this page
and grab the Zip file for the stable version 1.0.5

Within the Zip file you will find a Firmwares folder containing the latest stable firmware for the WiFi shield.
Sorry I can't give you the exact path because I am using a Mac at the moment.
It will be something like ..\Java\hardware\arduino\firmwares\WiFiShield\binary
The ..\binary folder contins firmware flash files.

Upgrading is a bit painful, as it is a two stage process. I will leave you to google the instructions. If you are really struggling, let me know and I will try to find some time to reacquaint myself with the procedure.

system · November 2, 2013, 1:35pm

Hi
I upgraded the firmware on my wifi shield and now I can connect to ftp. The right way to upgrade is located here: not_found. If someone is going to use this link, be careful. You must do EXACTLY the same steps and in EXACTLY the same order.
But though I can connect to my ftp server (connects successfully), when I transfer the file from Arduino, it appears on the server but it's empty. What could cause this?
Here is my sketch.

/*
 
 This sketch connects to a Wifi network. 
 Then it prints the  MAC address of the Wifi shield,
 the IP address obtained, and other network details.
 Then transfer file from SD card on ftp server (if press f)

 Circuit:
 * WiFi shield attached
 
 created 13 July 2010
 by dlf (Metodo2 srl)
 modified 31 May 2012
 by Tom Igoe -- WiFi part
 October 2012 by SurferTim -- ftp part
 October 2013 by Pashkot -- put the parts together (Arduino IDE 1.0.5, Arduino UNO R3)

 */
 #include <WiFi.h>
 #include <SD.h>
 #include <SPI.h>
 // comment out next line to write to SD from FTP server
 #define FTPWRITE

char ssid[] = "dlink38";     //  your network SSID (name) 
char pass[] = "mypassowrd";  // your network password
int status = WL_IDLE_STATUS;     // the Wifi radio's status

// change to your server
IPAddress server( 192, 168, 0, 106 );

WiFiClient client;
WiFiClient dclient;

char outBuf[128];
char outCount;

// change fileName to your file (8.3 format!)
char fileName[13] = "test.txt";

void setup() {
  //Initialize serial and wait for port to open:
  Serial.begin(9600); 
  //while (!Serial) {
    //; // wait for serial port to connect. Needed for Leonardo only
  //}
  
  pinMode(10,OUTPUT);
  digitalWrite(10,HIGH);

  if(SD.begin(4) == 0)
  {
    Serial.println(F("SD init fail"));          
  }
  
  // check for the presence of the shield:
  if (WiFi.status() == WL_NO_SHIELD) {
    Serial.println("WiFi shield not present"); 
    // don't continue:
    while(true);
  } 
  
 // attempt to connect to Wifi network:
  while ( status != WL_CONNECTED) { 
    Serial.print("Attempting to connect to WPA SSID: ");
    Serial.println(ssid);

    // Connect to WPA/WPA2 network:    
    status = WiFi.begin(ssid, pass);

    // wait 10 seconds for connection:
    delay(10000);

    // you're connected now, so print out the data:
    Serial.print("You're connected to the network ");
    printCurrentNet();
    printWifiData();
    Serial.print("FTP server:");
    Serial.println(server);
    Serial.println(F("Ready. Press f or r"));
  }
}

void loop() {
  // check the network connection once every 10 seconds:
  //delay(10000);
  //printCurrentNet();
  
  byte inChar;

  inChar = Serial.read();

  if(inChar == 'f')
  {
    if(doFTP()) Serial.println(F("FTP OK"));
    else Serial.println(F("FTP FAIL"));
  }

  if(inChar == 'r')
  {
    readSD();    
  }

}

File fh;

byte doFTP()
{
#ifdef FTPWRITE
  fh = SD.open(fileName,FILE_READ);
#else
  SD.remove(fileName);
  fh = SD.open(fileName,FILE_WRITE);
#endif

  if(!fh)
  {
    Serial.println(F("SD open fail"));
    return 0;    
  }

#ifndef FTPWRITE  
  if(!fh.seek(0))
  {
    Serial.println(F("Rewind fail"));
    fh.close();
    return 0;    
  }
#endif

  Serial.println(F("SD opened"));

  if (client.connect(server,21)) {
    Serial.println(F("Command connected"));
  } 
  else {
    fh.close();
    Serial.println(F("Command connection failed"));
    return 0;
  }

  if(!eRcv()) return 0;

  client.println(F("USER pi"));

  if(!eRcv()) return 0;

  client.println(F("PASS myftppassword"));

  if(!eRcv()) return 0;

  client.println(F("SYST"));

  if(!eRcv()) return 0;

  client.println(F("PASV"));

  if(!eRcv()) return 0;

  char *tStr = strtok(outBuf,"(,");
  int array_pasv[6];
  for ( int i = 0; i < 6; i++) {
    tStr = strtok(NULL,"(,");
    array_pasv[i] = atoi(tStr);
    if(tStr == NULL)
    {
      Serial.println(F("Bad PASV Answer"));    

    }
  }

  unsigned int hiPort,loPort;

  hiPort = array_pasv[4] << 8;
  loPort = array_pasv[5] & 255;

  Serial.print(F("Data port: "));
  hiPort = hiPort | loPort;
  Serial.println(hiPort);

  if (dclient.connect(server,hiPort)) {
    Serial.println(F("Data connected"));
  } 
  else {
    Serial.println(F("Data connection failed"));
    client.stop();
    fh.close();
    return 0;
  }

#ifdef FTPWRITE 
  client.print(F("STOR "));
  client.println(fileName);
#else
  client.print(F("RETR "));
  client.println(fileName);
#endif

  if(!eRcv())
  {
    dclient.stop();
    return 0;
  }

#ifdef FTPWRITE
  Serial.println(F("Writing"));

  byte clientBuf[64];
  int clientCount = 0;

  while(fh.available())
  {
    clientBuf[clientCount] = fh.read();
    clientCount++;

    if(clientCount > 63)
    {
      dclient.write(clientBuf,64);
      clientCount = 0;
    }
  }

  if(clientCount > 0) dclient.write(clientBuf,clientCount);

#else
  while(dclient.connected())
  {
    while(dclient.available())
    {
      char c = dclient.read();
      fh.write(c);      
      Serial.write(c); 
    }
  }
#endif

  dclient.stop();
  Serial.println(F("Data disconnected"));

  if(!eRcv()) return 0;

  client.println(F("QUIT"));

  if(!eRcv()) return 0;

  client.stop();
  Serial.println(F("Command disconnected"));

  fh.close();
  Serial.println(F("SD closed"));
  return 1;
}

byte eRcv()
{
  byte respCode;
  byte thisByte;

  while(!client.available()) delay(1);

  respCode = client.peek();

  outCount = 0;

  while(client.available())
  {  
    thisByte = client.read();    
    Serial.write(thisByte);

    if(outCount < 127)
    {
      outBuf[outCount] = thisByte;
      outCount++;      
      outBuf[outCount] = 0;
    }
  }

  if(respCode >= '4')
  {
    efail();
    return 0;  
  }

  return 1;
}

void efail()
{
  byte thisByte = 0;

  client.println(F("QUIT"));

  while(!client.available()) delay(1);

  while(client.available())
  {  
    thisByte = client.read();    
    Serial.write(thisByte);
  }

  client.stop();
  Serial.println(F("Command disconnected"));
  fh.close();
  Serial.println(F("SD closed"));
}

void readSD()
{
  fh = SD.open(fileName,FILE_READ);

  if(!fh)
  {
    Serial.println(F("SD open fail"));
    return;    
  }

  while(fh.available())
  {
    Serial.write(fh.read());
  }

  fh.close();
}


void printWifiData() {
  // print your WiFi shield's IP address:
  IPAddress ip = WiFi.localIP();
    Serial.print("IP Address: ");
  Serial.println(ip);
    
  // print your MAC address:
  byte mac[6];  
  WiFi.macAddress(mac);
  Serial.print("MAC address: ");
  Serial.print(mac[5],HEX);
  Serial.print(":");
  Serial.print(mac[4],HEX);
  Serial.print(":");
  Serial.print(mac[3],HEX);
  Serial.print(":");
  Serial.print(mac[2],HEX);
  Serial.print(":");
  Serial.print(mac[1],HEX);
  Serial.print(":");
  Serial.println(mac[0],HEX);
 
}

void printCurrentNet() {
  // print the SSID of the network you're attached to:
  Serial.print("SSID: ");
  Serial.println(WiFi.SSID());

  // print the MAC address of the router you're attached to:
  byte bssid[6];
  WiFi.BSSID(bssid);    
  Serial.print("BSSID: ");
  Serial.print(bssid[5],HEX);
  Serial.print(":");
  Serial.print(bssid[4],HEX);
  Serial.print(":");
  Serial.print(bssid[3],HEX);
  Serial.print(":");
  Serial.print(bssid[2],HEX);
  Serial.print(":");
  Serial.print(bssid[1],HEX);
  Serial.print(":");
  Serial.println(bssid[0],HEX);

  // print the received signal strength:
  long rssi = WiFi.RSSI();
  Serial.print("signal strength (RSSI):");
  Serial.println(rssi);

  // print the encryption type:
  byte encryption = WiFi.encryptionType();
  Serial.print("Encryption Type:");
  Serial.println(encryption,HEX);
  Serial.println();
}

These are the results of the sketch from serial port:

Attempting to connect to WPA SSID: dlink38
You're connected to the network SSID: dlink38
BSSID: F0:7D:68:9F:E:5A
signal strength (RSSI):-24
Encryption Type:4

IP Address: 192.168.0.107
MAC address: 78:C4:E:2:9:98
FTP server:192.168.0.106
Ready. Press f or r
SD opened
Command connected
220 (vsFTPd 2.3.5)
331 Please specify the password.
230 Login successful.
215 UNIX Type: L8
227 Entering Passive Mode (192,168,0,7,30,106).
Data port: 7786
Data connected
150 Ok to send data.
Writing
Data disconnected
421 Data timeout. Reconnect. Sorry.  //after aprox 5 min

And this is the logfile of the server:

pi@raspberrypi ~ $ tail /var/log/vsftpd.log
Thu Oct 31 17:36:08 2013 [pid 2] CONNECT: Client "192.168.0.107"
Thu Oct 31 17:36:08 2013 [pid 1] [pi] OK LOGIN: Client "192.168.0.107"
pi@raspberrypi ~ $

Any ideas,guys?
Btw, MattS-UK thanks for the idea about upgrading the shield, worked out just fine.

SurferTim · November 3, 2013, 1:57am

Have you tried uploading files to the Pi with a FTP client program like CuteFTP or the like? Just to verify it is not the Raspberry PI's FTP setup.

system · November 4, 2013, 8:42am

I've tried using another ftp client from another computer and it worked. So I guess it's not Pi's fault.
Here's the tail of server's logfile:

pi@raspberrypi ~ $ tail /var/log/vsftpd.log
Sun Nov  3 08:54:53 2013 [pid 2] 
CONNECT: Client "192.168.0.101"
Sun Nov  3 08:55:08 2013 [pid 1] [pi] OK 
LOGIN: Client "192.168.0.101"
Sun Nov  3 08:55:24 2013 [pid 3] [pi] OK 
DOWNLOAD: Client "192.168.0.101", "/home/pi/test.txt", 0.00Kbyte/sec
Sun Nov  3 08:56:43 2013 [pid 3] [pi] OK 
UPLOAD: Client "192.168.0.101", "/home/pi/kavkaz_gory_1.jpg", 315851 bytes, 555.67Kbyte/sec
pi@raspberrypi ~ $

Both files were successfully transferred.
So my network is working, my server is working, that means that my Arduino's not working properly. Or are there some other parts of this system that can cause this?

SurferTim · November 4, 2013, 9:26am

I don't have a wifi shield to check the code. It may be something with using two sockets simultaneously that causes the problem with the wifi shield. If it creates the file on the server (command socket), but transfers no data (data socket), that is the indication that the data socket didn't connect to the FTP server. ??

The ethernet shield has no problems with it, but I have diagnostic code for it to check the socket status as I programmed it.

system · November 5, 2013, 7:59pm

Hi, here are some news.
On the previous try I used a test text file that included only one word "123arduino". As I said before the file wasn't uploaded properly. But after I added more text to the file (simply copy-pasted my sketch) the transfer procedure took longer time and the file on the server wasn't empty. The uploaded file was still not "full", it was only 6400 bytes while its actual size is 7558 bytes.
My only guess is that the connection is imperfect so I get a loss in file. If so, why was the small file completely empty?
The results from the console were the same as the previous time, but without disconnecting after 5 minutes. One more thing. The successfully executed sketch should end with "SD closed" in console, but the last line is "Data disconnected".
Any ideas will be appreciated.

SurferTim · November 6, 2013, 5:42am

Are you using my code with the wifi setup()? Insure you didn't leave out the part I commented below.

  byte clientBuf[64];
  int clientCount = 0;

  while(fh.available())
  {
    clientBuf[clientCount] = fh.read();
    clientCount++;

    if(clientCount > 63)
    {
      dclient.write(clientBuf,64);
      clientCount = 0;
    }
  }
  // here is where the extra bytes are sent if the file isn't an exact multiple of 64 bytes.
  if(clientCount > 0) dclient.write(clientBuf,clientCount);

system · November 6, 2013, 6:42pm

Hi SurferTim
Yes, I am using your code from http://playground.todo.arduino.cc/en/Code/FTP# and I have the part you wrote about. So I guess it's not the problem?

SurferTim · November 6, 2013, 6:47pm

The uploaded file was still not "full", it was only 6400 bytes while its actual size is 7558 bytes.

Does there appear to be any logic to the missing characters in the file? Try this code. I reduced the size of the send to 32 characters. Just a test.

  byte clientBuf[32];
  int clientCount = 0;

  while(fh.available())
  {
    clientBuf[clientCount] = fh.read();
    clientCount++;

    if(clientCount > 31)
    {
      dclient.write(clientBuf,32);
      clientCount = 0;
    }
  }

system · November 7, 2013, 5:08pm

I changed the size of the send from 64 to 32 as you said. I've also conducted an experiment and tried running sketch again and again. The results were different from time to time, I mean the size of the file on the server varied from 6560 to 7558 (entire file) bytes. And though I had two attempts where the whole file was transferred, the sketch didn't end with "SD closed" but with "Data disconnected". I have detailed information on all 13 tries from this evening, I can upload it here if you need it. One more thing, the most common size of the recieved file was 7392 bytes (6 of 13). What could cause such results?

bushra · August 7, 2017, 6:22am

can anybody tell me the PINOUT diagram with SD card and esp8266 wemos D1 mini
so that i could run the above program

Topic		Replies	Views
FTP through WiFi shield on the Arduino UNO Networking, Protocols, and Devices	2	1485	May 6, 2021
FTP - I can get it to work over LAN - I want it over Wifi - I'm close - help! Programming	24	8229	May 5, 2021
Can't connect to FTP server Via Wifi Shield Programming	11	3798	May 5, 2021
Translating FTP code for use with Wi-Fi Shield library Programming	14	2017	May 5, 2021
Have promblem with arduino ftp client Networking, Protocols, and Devices	9	1295	May 6, 2021

FTP over WIFI network

Related topics