Programming ??? Math Question

Okay. But what does happen? What exactly goes wrong?