Does it really have to be this difficult

I have been searching for some time on an easy means of collecting data and sending it to a central location. I have 3 devices I would like to log events to a location I can see from my PC.

Every wifi solution seems to involve programming on multiple platforms. I purchased a ESP8266 but it fails to connect to my network or the hotspot from my cell phone. I figured I'd eventually build on my knowledge and find a solution, but I've hit a wall right out of the gate.

From my perspective, the best solution would be the ability to simply read the data being collected from a network drive. Is there an easy way to make an arduino share an sd card through a wired or wireless connection. If so, does it make more sense to send data via xbee to a single collection point and write data to the SD card?

Without buying a $100 wifi shield (lets keep the price reasonable) is there a set of librairies that simply allow you to enter your network information and send data to a google doc. I'd like to be able to manipulate the data in a spreadsheet, not just examine some silly graph. All the internet of things videos I've seen (not a lot) seem so useless.

I'll give an example of one project, so you can understand my basic goal. We fill 5 gallon buckets from a 55 gallon drum about once a week. This takes time so sometime the operator walks away and will be left with a spill. I've created a device that controls a valve to shut off the flow when the bucket is full. I'd like to report to a spreadsheet every time a bucket is filled so that I have a tally of how many gallons have been drained from the drum. Hence the need to manipulate the data.

So, if someone can point me to the best gear for my projects, a tutorial that doesn't require I learn 3 more programming languages, or just tell me it's totally hopeless and arduino isn't there yet, I would greatly appreciate it.

First of all, it's not hopeless, but you probably need to do some work.

Simple solutions - like using a WiFi shield - usually cost more than complicated solutions, because someone has already done the hard work for you and now also wants some reward. So, if you can afford the 100 bucks for the WiFi Shield, which is in my opinion not that much for what it offers, you can do everything using the the WiFi Library: http://arduino.cc/en/Tutorial/WiFiWebClientRepeating. Just adapt the server adress you want to send the data to and the HTTP request according to the format that your server expects. For google docs, this is documented here: http://www.instructables.com/id/How-to-use-Google-docs-for-data-logging/

FYI: The ethernet shield is much more reliable than the WiFi shield. The WiFi shield gas some real issues, especially if using any server functions. And you can get one for significantly less money than a WiFi shield.

Adafruit has a different type of Wifi shield, based on a CC3000, that's cheaper. Not ESP8266 cheap, but half the price of the full up Wifi shield. I've not used the shield, but I've used the Breakout Board. I setup a TCP-IP socket with it connected to an Arduino Pro Mini, and transmit data to a Raspberry Pi. BTW, a Raspberry Pi with a WIFI dongle is cheaper than the regular Wifi shield. ESP8266's are cheaper because (as stated above) you've got to do more of the work, and because they tend to be made in quantity and some have problems with quality. Try buying from a more reputable seller like Adafruit. You're more likely to get a working unit, and they have helpful support if you're having problems.

Well, an ethernet shield is pretty cheap. With an arduino uno, ethernetshield, and a rf24 radio you'd have a good starting base. You could host a webserver, and log files to the SD card on the ethernet shield. You could then have a second arduino with another rf24 to send data to the base arduino, which in turn would log the data in a file on the sd card. pointing a browser on your network (or even internet, if you fwd the port) at your webserver could show a file listing of the SD card... This would be a very inexpensive setup arduinox2, ethernet shield, rf24x2, and not really that complex -unless I'm missing something... which is likely ;)

So it sounds like my best solution is ethernet shield or ethernet module connection a single arduino to my network. I can then feed data to this arduino via xbee. Have I interpreted things correctly? Is there something other than an xbee that would do the job. I see these:

Am I going to struggle to get multiple devices to communicate with the single hub with this wireless tranceiver?

I would look into the radio modules which allow multiple transmitters (instead of point-to-point) the JeeNodes system uses RFM12B radios to do exactly what you want: multiple small transmitters going to a central node connected to the network.

Have a look at the examples at JeeLabs.org . I've bought several of his modules and even though he may decide not to make any more, it's open source so you can use a standard Arduino with the same radios.

donek: Does it really have to be this difficult?

Rome was not built in a day - and while the Arduino environment is targetted at amateurs the reality is that it is the prices of Arduino that is the attraction, not the ease of doing anything substantial.

I have spent the last eighteen months building something substantial (which manifests as the Arduino web server application at http://www.2wg.co.nz and elsewhere)- but along the way I did encounter a few seemingly intransigent issues - so yes, there were times when it really was that difficult.

I note in your thread an underlying concern for cost - that is no way to move forward with your objectives. You need to spend a bit of money and a lot of time if you want real progress with Arduino equipment - there are no ready made free solutions.

I also have an interest in your area of interest. I am capturing readings from a variety of devices and would like to have all the data (from multiple implementations of my application) in one central sql database where it can be analysed, including using graphs. My data currently exists as text files on each of my Arduino implementations and each implementation will display (transmit) those files to a remote browser client on request.

So over coming months I intend to develop a Delphi web server application on my PC server to suck the data off the Arduino implementation SD cards in real time and load it into a SQL Server database. I can then build ad hoc data browsing, querying and data graphing into my Delphi web server application. I have an extensive Delphi/SQL Server background so the next part of my development program is mostly going to be a lot of development time on top of my existing expertise and tool set.

What I am trying to say is there are no instant solutions in the Arduino environment - just a lot of options, learning opportunities, frustrations and sometimes success.

Cheers

Catweazle NZ

Am I going to struggle to get multiple devices to communicate with the single hub with this wireless tranceiver?

You probably will until you break your project down into its multiple parts and work on each. Get a 5100 based ethernet shield and get your project working with that before moving a wireless setup.

zoomkat: You probably will until you break your project down into its multiple parts and work on each. Get a 5100 based ethernet shield and get your project working with that before moving a wireless setup.

This sounds like the best course of action. In reality, the cost of a bigger switch and putting together some network cables is probably less work/expense than the headache of the wireless concept.

I used a Raspberry Pi to accomplish something very similar to what your talking about. The Arduino's, or Arduini, collect my weather data and report to the Pi, which then collects all the data, dumps to a DB, and I can view on a plain looking web page, by date. Im not a experienced coder at all, but believe it or not this was very easy to accomplish as the documentation is abundant. Pi cost $35 bucks.

Louie323: I used a Raspberry Pi to accomplish something very similar to what your talking about. The Arduino's, or Arduini, collect my weather data and report to the Pi, which then collects all the data, dumps to a DB, and I can view on a plain looking web page, by date. Im not a experienced coder at all, but believe it or not this was very easy to accomplish as the documentation is abundant. Pi cost $35 bucks.

What device did you use to report to the pi? Does the pi have wifi/ethernet built in?

Louie323: I used a Raspberry Pi to accomplish something very similar to what your talking about. The Arduino's, or Arduini, collect my weather data and report to the Pi, which then collects all the data, dumps to a DB, and I can view on a plain looking web page, by date. Im not a experienced coder at all, but believe it or not this was very easy to accomplish as the documentation is abundant. Pi cost $35 bucks.

So now I'm asking why not connect an Xbee to a PC and just feed the data directly to a PC. Is there any software that is available for doing this. Something that just runs in the background ready to receive data any time it is sent and dump it to a text file or similar.

So now I'm asking why not connect an Xbee to a PC and just feed the data directly to a PC.

Why not? It appears the XBee Pro 63mw unit can transmit a considerable distance. I think I am about to try the same thing except I am going to use a RPi as the PC.

donek: So now I'm asking why not connect an Xbee to a PC and just feed the data directly to a PC. Is there any software that is available for doing this. Something that just runs in the background ready to receive data any time it is sent and dump it to a text file or similar.

I sell a gateway system that uses inexpensive nRF24L01+ modules on Arduinos to connect to a Raspberry Pi that then brokers the TCP/IP service to your LAN and the Internet. The software library I've developed implements TCP over the nRF24L01+ radio connection, allowing to run all the services you would from a WiFi shield or Ethernet shield -- actually more since it also allows for SSL/TLS connections (e.g., having the Arduino nodes connecting to https servers).

Have a look at my website embeddedcoolness.com for details if this is of any interest.

pico: I sell gateway system that uses inexpensive nRF24L01+ modules on Arduinos to connect to a Raspberry Pi that then brokers the TCP/IP service to your LAN and the Internet. The software library I've developed implements TCP over the nRF24L01+ radio connection, allowing to run all the services you would from a WiFi shield or Ethernet shield -- actually more since it also allows for SSL/TSL connections (e.g., having the Arduino nodes connecting to https servers).

Have a look at my website embeddedcoolness.com for details if this is of any interest.

You've got me jumping for joy. I've sent you a message through your web site. Please call me in the morning.