Boards under powered or other faults when trying to operate a servo?

I have an UNO and a Mega and I am trying to set up a simple system to run a servo. I have a trigger box with two contacts that will bridge the connection when both a switch and button are active, I have tested it and it works fine with INPUT_PULLUP. The problem is with the servo, I have been unable to get the servo to run properly as the board keeps shutting down and/or having to restart due to current draw and the voltage dropping too low, even with both USB and a battery attached. The on light doesn’t go off, but the L LED will go off, for a half second, then come back on. If I remove the servo the system runs fine. This isn’t a problem with the servo and it isn’t a massive one with a high amp draw, it is a 9g Micro Servo. This problem is occurring on both boards and I am curious if anyone knows why/has a solution.
If the code is useful:

#include <Servo.h>
Servo ignite;
int gono;   
const int inpin = 4; 

void setup() {
        ignite.attach(9);
        pinMode(inpin, INPUT_PULLUP);
}

void loop() {
  gono = digitalRead(inpin);
  Serial.println(gono);
  if (gono = 0){
    ignite.write(0);
    delay(500);
    ignite.write(180);
  }
  else {
    ignite.write(180);
  }
}

I am reading 4.428v on the 5v pin with the servo attached and idle. It goes down to 3.8 volts when the servo runs with 3.774 being the lowest voltage achieved. The 5v voltage with no load is 4.575. The test was done with a large capacitor to help smooth any current draw spikes but it was of no help.

As you have discovered, you should be using a separate power supply for motors.