Hi tomas_albrektas!
Currently I am working on simillar setup (the only difference is I am using Arduino MEGA instead of ESP32). It seems that your circuit is missing very important connection, namely - enable pin (EN) is not connected. According to TMC2209 manual, chip is enabled when EN pin state is LOW and disabled when setup HI. Did you try that approach?
Please, check this thread, as it contains tons of useful informations. I also highly recommend you to use FlexyStepper or SpeedyStepper library.
Best regards!
PS. This is my first post on this Forum, so I am sorry if I made any mistakes 