Garbage in Serial1 when 3V battery holder touches circuit board

I am having some issues with my custom 328PB board. Did a lot of testing and eventually found out that the problem is this:

I have an Adafruit ultimate GPS module. The under side has a 3V batter holder (exposed metal for positive). I connected the GPS to 328PB's Serial1 and did a simple pass-through test to pass anything received in Serial1 to Serial so I can see NMEA messages on serial monitor. Everything worked until I touch the battery holder against my custom board. The area they touch has no vias or any exposed electrodes but as soon as the battery holder gets too close, the results are all garbled. If I quickly move the module away from my board, then the program may recover. This never happens if I remove the battery from the holder.

Here is my module. The top half with rectangular area and rounded edges is completely void of vias. There is a trace for D2, which I'm not using in the pass-through code.

Code:

void setup() {
  pinMode(10,OUTPUT);
  Serial.begin(9600);
  Serial1.begin(9600);
}

void loop() {
  while (Serial1.available())
  {
    Serial.write(Serial1.read());
  }
}

Adafruit ultimate GPS: Adafruit Ultimate GPS Breakout - 66 channel w/10 Hz updates [Version 3] : ID 746 : $29.95 : Adafruit Industries, Unique & fun DIY electronics and kits
This picture shows the under side of the GPS with battery holder:

On my board, the area for the GPS module is ground plane. So if I bring a 3V potential within a thin varnish layer from a ground plane, I suppose I formed a leaky capacitor that can change the potential of the ground plane near the battery. Maybe the Serial1 is very sensitive to this. Afterall, the GPS only provides about 2.7V as HIGH.

It seems like the answer is simple. Electromagnetic Interference. You have inputs and outputs from the base board surrounding 3/4 of the way around the perimeter of the GPS shield board. The GPS is implementing RF as well. That doesn't help.
If it is your product which you are producing/customizing, work on the board layout, trace locations, and ground plane. Otherwise, add a layer of shielding between the two and/or mount the two further apart.

Thanks. I just did some reading. It seems that the GPS sensor generates an RF signal to match satellite RF signals to "lock" to the satellite so there IS EMI coming out of the GPS module and it should stay away from my board. I designed the board myself so I can make changes. My first attempt will be to just mount the GPS sufficiently away, about a few millimeters is enough to have no effect on my board. I guess the battery holder was just acting as an antenna so after removing it I had to bring the GPS even closer to my board to get EMI. With the Adafruit Ultimate GPS shield, I've never had a problem. Things are sufficiently far away.

So if I do want to mount the GPS close to my board, or even later reflow the GPS part on my board, should I remove the ground pours on both sides of my board and move my traces away from where the GPS module will be?

Smartphone design must be an art with simulations. So many sources of gigahertz EMI in a tight space yet still everything works. No wonder signal integrity engineers get paid top dollars.