This sound odd since radio waves should travel in air at speeds high enough that you would not notice significant delays over a few meters.
Are you sure there is not another effect coming into play, for example the communication link completely fails and the clock runs on its own oscillator time source ?
Can you describe your installation in more detail including code and a schematic ?