I believe if you look at what pinMode does in the ESP8266, you'll see that it's more than the single assignment your code uses.
And it's just a WAG, but I would not be surprised if the reason your code worked with the internal LED pin is that it's already been set up to be an OUTPUT once, whereas pin 4 hasn't been.