LCD is Dimming when Adding 12V via DC Jack

Using Arduino 101. I'm trying to power some servos, but they don't run off of the 5V out from the Arduino. My solution is (going to be) to route 6V (4 AA Batteries) in via the DC jack and out via VIN pin. But, when I connect the 6V to the Arduino, my LCD screen dims dramatically and the data displayed becomes corrupted. LCD is from SparkFun. I assume this is a common problem, but I can't find anyone with similar issues. Am I just being foolish?