But I have noticed when I write a pin HIGH it turns it off and if I write it LOW it turns it on. It as been the way since I got it. Is there something wrong with it?
Current flows when there is a
difference in voltage. If you connect one end of an LED/resistor to +5V and the other end to an Arduino output, the LED will come on when the ouput goes low (assuming it's wired with the correct polarity

).
If both ends of the LED/resistor are connected to 5V, there is no difference, no current will flow, and the LED won't light-up. It's sort-of like water-flow... If there is no pressure difference, or a difference in height, the water won't flow... And the direction of the water-flow is always from high-to-low.
Similarly, I'm using the internal pull-up resistor on an input with a switch wired to ground. When I turn the switch on, it pulls the input low.
I also noticed that the analog pins read 0 when I put power to them. Is that right?
Something's wrong... If you connect +5V to an analog pin, it should read (about) 1023. The "power" needs to share a common ground with the Arduino. (Again we are looking for a
difference, and you need the ground reference.) And, you can damage the Arduino by applying a negative voltage or a voltage higher than +5V.