I did some more testing and actually got it to work ... kind of 
I have the Galaxy S (i9000) with the CyanogenMod 9 (Android version 4.0.4). The setup works if I power the Due over the USB connection and does not work, if I power it via the power jack. Is that by design?
However, the operation is inconsistent, sometimes the enumerates the devices and gets to adk.IsReady() == 1 and most of the time it does not. I've tried to printout the status of the usb_getUsbTaskState function and get different readouts (USB_STATE_RUNNING, USB_STATE_ERROR, USB_DETACHED_SUBSTATE_WAIT_FOR_DEVICE) when plugging the phone in and out.
Sometimes it even goes to USB_STATE_RUNNING and then straight to USB_DETACHED_SUBSTATE_WAIT_FOR_DEVICE. It could be a problem with the phone, since things get better (but not ideal) after the reboot of the phone. Unfortunately I don't have any other device to try it with.