The longer I think about it, I think it's just a thread-problem, although I don't get any error message, just nothing happens.
So, add another label that you update when/if the SerialPort1_DataReceived function is called. At least you'll know whether there is data coming in.
Now, since the serial data is a stream of characters, I'm not sure why you define message to be an integer. You certainly can't store an array of bytes in an integer variable.