I've found some more info on a Texas Instruments CC2530 system on a chip. In this document they describe the new standard:
However I don't get any clue if this protocol is supported by current modules, or if this is something Digi has to built into a module.
Can someone who has more knowledge about Zigbee shine a light (:)) on this?
It would be interesting to be able to communicatie with the new Philips Hue lights (http://www.meethue.com/) from Arduino directly. Or to build lights with Arduino that can be controlled by this protocol (since Philips, Osram and more manufactures are going to support this).
The Chibi is an arduino with a radio similar to one of your links. It's a nice little board that I bought to set up an XBee sniffer so I could monitor my network of XBees. Akiba, the guy that developed it, did it with the idea of implementing the Zigbee protocol stack and doing stuff exactly like what you describe. Unfortunately, he ran into trouble with the licensing, he has a long article about how he finally gave up after putting months of effort into it.
I took a look at the Zigbee documentation that goes on forever and ever to see what they were doing and it comes down to actually closing a switch, and that part is left to the manufacturer. So, there's a mountain of protocol to get to a manufacturer specific piece that does the actual work. And, you guessed it, each manufacturer does it differently. That's why there are incompatibilities between light switches from different manufacturers. There seems to be some movement recently to adapting to some standards for the final piece, but I'm not tracking that anymore.
This is why Digi only makes gateways and forwarding devices that are certified by the Zigbee folks.
However, all is not lost. You can certainly get one of the devices and set up a Chibi to monitor the conversation and reverse engineer the controls. That will take some work and probably tearing of hair, but it can be done.
I am looking into this too. I have been exploring various cheap technologies for home automation. The Cree Connected Zigbee LightLink bulbs are only $15 (much cheaper than $50 z-wave wall switches). I would love to be able to control Cree Connected bulbs via my arduino and an Xbee.
This guy has gotten some things to work with RPi, does this help?