Soldering LED SMD & push buttons directly to attiny84 pin ?

Can I directly solder the LED SMD wires to the attiny pins ? (or the resistance, although i'm a total newbie, it is my understanding I need to connect the red wire of the LED to a resistor and the resistor to the attiny to prevent the LED to fry and last longer).

Yes you can but their are two caveats:-

  1. Soldering something that small requires a very small tipped iron, a good pair of tweezers and some form of magnification. I use a head loupe. Be aware that the surface tension of molten solder is way stronger than gravity on the part. So you have to restrain the parts mechanically while you solder them or they simply stick to the iron.

  2. Once soldered the mechanical strain on the pins could well pull them out and break them, so you would need some form of mechanical backing to fix the parts to.

Can I solder more than one LED SMD light to a pin (so one pin would make 2 LED flash with the same pin)

Yes providing each LED has its own resistor and the combined current it takes is less than the maximum alowable current.

I'm about to create my very first arduino project

It is a big ask for a first project.