i'm sorry. then i don't know. there's no extra resistors for these pins on the board http://www.arduino.cc/en/uploads/Main/Arduino-Diecimila-schematic.pdf
what kind of led matrix did you use? did you try to connect just one LED to pin 2 (or 3 ) and GND ? same effect?
maybe the problem is not the provided current but the current sinking on the other side (GND or corresponding pin in your matrix setup). this wouldn't explain anything but help to track down the problem.
i'm sure i've used LEDs on all pins without noticing differences.