many to one wireless communication with arduino

How can one achieve many to one wireless communication without data loss using RF? assuming there are 4 transmitters sending data to a single receiver wont the packets be lost? I'm looking at making a wireless quiz buzzer - the kind you have to be the first to press to answer a question. I want to take this one step forward with 4 switches(assuming 4 options are the maximum in a quiz of this sort) and the user can directly answer a question by pressing a switch.

Here is what i was thinking 4 transmitters with their own encoders and separate addresses 1 receiver with 4 decoders mapped to 4 different addresses OR 4 receivers with 4 decoders mapped to 4 different addresses and an Arduino reading the output from the 4 decoders?

Will this work? is there a way to do this with IR?

You're worried about collisions? I'm not sure there's away around this short of using a different frequency for each buzzer in the same way that RC stuff does. I guess it depends on how critical the outcome is. I suspect for most purposes you could keep the amount of data sent to an absolute minimum. Or perhaps use transcievers in the buzzers and have the buzzers lockout if another is transmitting? Just thinking out loud.

Or perhaps use transcievers

Thats a good idea. But wouldn't it be a over kill to use a tranciever when the communication here is going to be purely one way?