Accessing files on SD card without taking the card out

Hi folks, I have this project that I am working for months: an aquarium controller.

It has the Ethernet shield with the micro SD on top of the 2560 Mega. I was able to setup everything and Ethernet works, SD works, it does data log OK, etc. I have created a webserver and works fine. Everything is going to be enclosed and the access to the card is gone.

I am looking for a solution to access the files on the card (datalog.txt) without opening the box. The file can get large after a while.

So far I thought about the following options: - dumpfile in serial monitor and then copy/paste data. Not elegant, possible limited file size. I will have to bring a computer near controller. - create a webpage (at client's request) and print data on it. Not sure how large can go. - wiring to a computer and use serial communication. Want to avoid if other solutions are available.

Is there a way to see the file via network without printing it on the screen. Something similar to "save target as?" Any other solutions?

Thanks, Bogdan

Rather than use the Arduino environment, find a terminal program that can talk USB serial and copy to a file? Depends on the OS of course. It will be slower than plugging the card into a real SD card socket (with native mode transfers).

boga: I am looking for a solution to access the files on the card (datalog.txt) without opening the box. The file can get large after a while.

Since you have an Ethernet interface you can implement a web server, FTP server, or any other server type you like. If your application already has a web interface then adding an administrative page to view/download/flush the log would seem like a sensible solution. Which method is 'better' would depend on where you want the log data to end up - if you want it on the screen, then publishing it within a web page would make sense; if you want it saved as a file then making it available as a download would be more appropriate. As the web site developer, you can provide whichever option(s) you want. You also have the option of publishing the logs using SNMP, Syslog or any other management protocol you want to support. Make sure you consider any authentication/access permission issues before you start - do you want to control who can perform these admin operations, or record who did what?

MarkT: Rather than use the Arduino environment, find a terminal program that can talk USB serial and copy to a file? Depends on the OS of course. It will be slower than plugging the card into a real SD card socket (with native mode transfers).

Thank you Mark. Acknowledged. I will look into it. I would probably need it even in the development stage. I would still try to see how the network options work, as a permanent solution.

PeterH: Since you have an Ethernet interface you can implement a web server, FTP server, or any other server type you like. If your application already has a web interface then adding an administrative page to view/download/flush the log would seem like a sensible solution. Which method is 'better' would depend on where you want the log data to end up - if you want it on the screen, then publishing it within a web page would make sense; if you want it saved as a file then making it available as a download would be more appropriate. As the web site developer, you can provide whichever option(s) you want. You also have the option of publishing the logs using SNMP, Syslog or any other management protocol you want to support. Make sure you consider any authentication/access permission issues before you start - do you want to control who can perform these admin operations, or record who did what?

Thank you Peter. I am planning a webserver where the majority of the webpage would be stored on the card. The file "datalog.txt will be on card, too. There is no need for special security features. Just want to open it from any computer in the local network.

So, Yes, I would like to make it available as a download. Would a simple hyperlink line in the webpage calling the "datalog.txt" work? Is it going to open the "save as" dialog?

I am not a programmer, so please forgive me if my thinking is not coherent enough.

Since you're planning a web server, presumably you're planning to have your sketch receive HTTP requests, extract the path and / or URL parameters, open the corresponding local resource and send the content back in an HTTP response. Handling the log file is not fundamentally any different to handling any other resources, except that if you want to make it downloadable (rather than have it display in the browser) then you would need to specify the content disposition in the HTTP response.

Thank you so much. I got it working :). It opens the file in Notepad or "Save as" on disk. Not sure if connection should be kept alive or closed, or does not matter?

// {.....}
              client.println("HTTP/1.1 200 OK");
              client.println("Content-type: application/octet-stream");
              client.println("Content-disposition: attachment; filename=datalog.txt");
              client.println("Connnection: close");
              client.println();

                        webFile = SD.open("datalog.txt");        // open datalog file
                    }

                    // send file to client
                    if (webFile) {
                        while(webFile.available()) {
                            client.write(webFile.read());
                        }
                        webFile.close();
                    }
// {.....}

It is slow, but working. There is some work done to improve the download speed here http://playground.arduino.cc/Code/WebServerST

Thank you both for help. Bogdan

Hi Bogdan- if you're up for sharing your code, I would appreciate it if you could either show it all here or attach as a file, or email it to me (my email should show on the left). And / or you could post it on the playground...

Jim

JimboZA: Hi Bogdan- if you're up for sharing your code, I would appreciate it if you could either show it all here or attach as a file, or email it to me (my email should show on the left). And / or you could post it on the playground...

Jim

Hi Jim - Are you looking for the entire aquarium controller code? If that is the case, I am far, far from completion.

For the example above, and based on tips from Peter, I modified a file from http://startingelectronics.com/tutorials/arduino/ethernet-shield-web-server-tutorial , part 10. In the code, instead of opening "page 2", I opened my datalog.txt file. I will send you the files by e-mail (sketch, index.htm, datalog.txt)

If you did not see that tutorial so far, I can tell you that is the best I found (for beginners like me).

Thanks Bogdan,

I hadn't seen that tutorial, no- I just had a quick look now. I had used the code from the IDE and SurferTim's Playground code and some code from zoomkat. All of that is superb stuff, but it's not a tutorial, which is really what I need/ed, and I got very confused.

So yep, I'd love the code as you mentioned you would send, but no, I'm not looking for an aquarium controller. (Although, having said that, we do have an aquarium, so who knows.....)

Cheers,

Jim

Jim, Sent you a PM. Your e-mail is hidden to me. Thanks

YGM