However, I have to switch the power to that device via software, so I thought a transistor would help me. I connected the 3.3V to the emitter, one of the arduino pins to the base, and the collector goes to my device.
That is the way to configure a PNP transistor only. These are not the most common type. You always need a resistor between the base and the arduino pin.
Please say what number the transistor is so we can check if you have it wired correctly.