This is not a "problem to solve" post, it's more "please help me understand what I'm not." 
I am working with these modules with an G3MB-202P chip:
These relays work perfectly when I apply a 120V circuit to the switch side of the relay (light bulb.) When I try this simple circuit, 5V with an LED,
The breadboard LED never goes off in the SSR off state.
The relays turn on and off as expected (onboard module LED's light accordingly.) When the relay is in the off state, I can unplug the LED ground and plug it back in (or power down the external supply momentarily) and the LED doesn't come on until the relay is in the ON state.
This tells me (I think) that there is some sort of floating voltage going on or the relay off state is not truly OFF until I unplug the ground. I've tried a few of the approaches used to eliminate floating voltage on digital pins to no avail.
What am I missing?
const int relayPin = 13;
void setup()
{
Serial.begin(9600);
pinMode(relayPin, OUTPUT);
}
void loop()
{
// Turn the relay switch ON
digitalWrite(relayPin, LOW);// set relay pin to low
Serial.println("Relay ON ");
delay(2000);
// Turn the relay switch OFF
digitalWrite(relayPin, HIGH);// set relay pin to HIGH
Serial.println("Relay OFF ");
delay(2000);
}