So what you are saying is that your converter output drops to 4.7V as soon as you connect even a minimal load to it. That sounds like a bad converter to me.
So what you really need to do is connect everything as in Case 2 (Arduino connected, nothing else) and then measure the voltage at all points. Is the input to the converter/output from TP405 still ~4V? What is the voltage from Vin to GND? 4.7V? If you're putting only 4.7V into Vin then explains the very low voltages at the 5V pin.
The reason has to be that either the converter is bad or the TP4056 module is bad or being misused. Do you have the 5V (USB or otherwise) connected to the TP4056? It is supposed to be a charger after all. Your pictures don't show it.
Steve