Arduino Forum

Using Arduino => Networking, Protocols, and Devices => Topic started by: knowhow on Mar 05, 2011, 05:24 am

Title: Automatically download files from internet
Post by: knowhow on Mar 05, 2011, 05:24 am
Hi I'm a newbie looking for help with a school project: I need to automatically download a file from the internet (FTP server) and save it onto a USB drive with the use of an Arduino (and an Ethernet Shield I suppose). Can this be done and if so, can anyone point me in the right direction? Any help would be much appreciated as this project is of great importance to me. Thank you for your time.
Title: Re: Automatically download files from internet
Post by: nickgammon on Mar 05, 2011, 06:10 am
With an Ethernet shield I presume it could be done. A quick search didn't reveal any existing examples for the Arduino, so it sounds like your school project won't be "cut and paste" <grin>.

The TelnetClient example sketch shows how a TCP client can be made to work. Now on top of that I would browse for "how ftp works" or something similar and get the basic protocol.

For example: http://cr.yp.to/ftp/request.html

You will basically have to send/receive various lines of stuff.  Maybe connecting with telnet to an existing ftp server would show the idea.

The broad idea is this:

telnet to (someserver) port 21

The responses are lines with numbers in front, the protocol tells you the difference between success and failure. The numbers give you the machine-readable results, and they are usually followed by human-readable text, like below:

Send/receive:

220 (vsFTPd 2.0.7)
USER knowhow
331 Please specify the password.
PASS swordfish
230 Login successful.
PASV
227 Entering Passive Mode (127,0,0,1,131,206)

Now make a second outgoing connection to the specified address/port.
That is (for this case): 127.0.0.1 port (131 * 256) + 206  (namely 33742, but you need to do the maths on the returned value)

telnet to 127.0.0.1 port 33742

Now, for example, to list the directory do this on the first connection:

LIST
150 Here comes the directory listing.
226 Directory send OK.

The directory comes down the second connection. At this stage it seems the second connection is dropped, so you would need to do PASV again if you wanted to do something else.

To get a file you might do this:

RETR /home/path/foo/test.txt
150 Opening BINARY mode data connection for test.txt (74 bytes).
226 File send OK.

The file appears in the second connection. Now save to disk.

Title: Re: Automatically download files from internet
Post by: CrossRoads on Mar 05, 2011, 06:19 am
To write to a USB card you need to act as a USB master.
Have read in here that arduino is not capable of that. I would love to be proved wrong tho, as being a USB master could come in handy.

Title: Re: Automatically download files from internet
Post by: nickgammon on Mar 05, 2011, 06:40 am
Oh, good point. I skimmed that bit. I read it as "save to an SD card", probably because the Ethernet shield has an SD card reader/writer on it (or one of them does anyway).

I would think that if you download the file, and put it onto an SD card, whilst not strictly a USB drive, you would still get reasonably high marks. If they really mean a USB drive, then this sounds advanced to me for a school project.