Suggestions for Real Time data evaluation? (w/ Ethernet Shield)

Do any of you have suggestions for pulling real time data from a web server into the Arduino?

My use case: Event on desktop client triggers an update of information in a web server. On eventUpdate, the web server (most likely hosted on Windows Azure) can then either display the result on a queried page or can open a socket to push updates to the Arduino.

I need the Arduino to process these updates in as close to real time as possible.

Any suggestions on ways you've used that proved reliable? Right now I'm looking either at a long polling technique or something similar to this implementation (WebSockets).

Do any of you have suggestions for pulling real time data from a web server into the Arduino?

Typically web client code is used on the arduino to make some type of GET request from a server.

zoomkat: Typically web client code is used on the arduino to make some type of GET request from a server.

Right, I've got that, but I'd prefer not to make a GET request every 1s. It's doable, but I feel like that's the brute force approach. I feel like there's a better implementation.

I feel like there's a better implementation.

How would the server know to keep a specific client connection open? That was never the intent of client/server communications. The intent is that the server has some data that might be useful to clients. The clients request data whenever they need something. No persistent connections are needed.

What else is the Arduino doing? It could act as a server, and get called when the client (the PC) has some need for the server to do something (show this latest data, maybe).

Try this http://arduino.cc/en/Tutorial/UdpNtpClient?action=sourceblock&num=1

This may be a little out in left field, but it might help. Are you familiar with Redis? (http://redis.io) It is an open source data server that is, among other things, really good at moving data from one client to another. Very easy to set up and use.

Bitlash (http://bitlash.net) ships with an Arduino Redis client example that can subscribe to a Redis channel and receive updates over ethernet in near-real-time:

https://github.com/billroy/bitlash/blob/master/examples/BitlashRedisClient/BitlashRedisClient.pde

You would need to use a Redis library for your PC-side language of choice to push updates to the Redis server you set up. It's really quite easy, even compared to pushing around simple HTTP requests.

Free Redis servers can be had at RedisToGo (http://redistogo.com).

I realize it's a different architecture than you're considering, but it's better optimized for the problem you're trying to solve.

Happy hacking,

-br http://bitlash.net

(Edit: fixed the RedisToGo link. -br)

I am on the same page as you, right now. I would definitely go the Websockets way, just like the presentation you posted. Unfortunately, that library only works with W5100 based ethernet shields and mine is a ENC28J60.

Long polling is fine, as long as you don't plan to have too many clients connecting to the same server.