I drive it without any resistors (using short on times) and I light each segment at a time so that all digits are the same brightness and so that I don't exceed the total pin current spec of the 328P.
You are still destroying your Arduino. You need resistors, even if the pins are only on for short durations. The current draw doesn't ramp up slowly.