What is the ICSP pins for?

With reference to the AVR chips themselves, they are they way you program the chips in-system. You connect the programmer to these six pins somehow - usually over a ribbon cable with two 6 pin IDC headers but you can just use some jumpers too. The programmer can then send the production file (elf file) to the chip. You can get the production file from the Arduino software or from Atmel Studio.

This guy is going from the 6-pin cable to jumpers to the breadboard. I would have just used a IDC socket. But otherwise, this is how you do it. This bypasses the Arduino for a bare $0.80-$3.00 controller which you can add to any project without having a dangling Arduino, but you have to buy the programmer for $35.