Two things come to mind. First, 4xAA is not 6.0v .. it'll probably be over 6.5v. Not a big deal, just be aware and don't make assumptions.
Second, 20mA is the max of most LEDs. Using a little less will prolong life. Now, usually we're not talking lifespans of minutes either way, but still. Are you sure your LEDs were rated for 20mA continuous anyway? That's common, but not guaranteed.