Is there something code or hardware related that you think would make this impractical? I'm just curious if you have any specific reasons, or if it just fills you with a general sense of unease. 
My biggest worry is how long programming might take. Using a YouTube video on a phone or something, I think it would be tough to get more than 30 bits/second (or whatever the video framerate is) without a much more complicated mechanism. Similarly, even a dedicated app on a desktop PC would be limited by the 60Hz refresh rate of a monitor. The only way to do fast programming would be to use an LED, I would imagine.