I understand that this setup will work, i want to not use this adapter, and instead make it on board with my attiny85, so in result my board will have in and out port, in -> from charger, out -> to phone.
Note that what ever you do with VUSB, you have USB1.0 speed. Otherwise you'd be just fine. If PD is opported by the cable then you'd have a LDO or stepdown to 5V in the box with the switch.
Referring to your post #12: Does it mean that you want to program (creating and uploading sketch) the ATtiny85 MCU of the Digispark Board using your Android Phone or iPhone?