Using an ESP8266 attached to a chain of 57 addressable LEDs (WS2812B), I am only able to turn on 20 of them at a time. I can turn on any 20 of them, so there's not a break in the chain, and more of them will turn on when I reset the arduino, so I know it's not a power problem, but I just can't seem to get more than 20 of the leds at once.
How are you wiring the LED strip ?
See Neo Pixel wiring here:
If they get dim it probably is a power problem.... What's the current rating of your power supply? Do you have a multi-meter to make sure the voltage is "holding up"?
"Worst case" should be 60mA (0.06A) per LED, so 20 LEDs is a little more than 1A.
And, there is a voltage drop across the thin built-in conductor and longer distances (more resistance) or more LEDs (more current) means more voltage drop. Adafruit recommends "injecting" power every meter.
They don't get dim. I have a test program that lights them up one by one, and it just stops after 20. I know the program is still running because it outputs the led it's trying to turn on over serial.
I'm powering just over a meter of strip from a usb power bank that claims to output 2.1A, but the problem also occurs if I draw power from a computer usb port.
Will check voltage with a multimeter.
The voltage does drop a bit as more leds come on, from about 5.2 to about 4.9 volts, but that's still within the range specified on the datasheet
The 20th or 21st Pixel might be bad.
I can turn on all the leds individually, just not more than 20 at a time
Looks like it has something to do with the FastLED library. Using the NeoPixel library I can control the whole strip as expected
Well apparently you do, but we do not have your test program so there is no hope of verifying what on earth it does! 
You would need to read the instructions. 