Simple IR to RS232 (w RS232 shield) unstable

kristianskauge:
But when i just power my arduino with a USB A-B cable and a wall outlet, it only seems to listen to IR/ send these commands every now and then.

Smells like you forgot a Gnd connection somewhere.