Yes the base emitter is a diode like any other junction so if you mangle the connections you could get it either conducting current like a normal diode or if it is reverse biased there would be the reverse bias leakage, like a normal diode.
However, it is likely that the diode in this case would go into reverse biased breakdown and allow more current to flow thus lighting any in line LED.