Using a TRIAC to alter the AC input to a transformer that ends up being rectified isn't the right way to go about this. The output of a TRIAC isn't anything like the output of a variable transformer.
You just want to control, essentially turn on/off, the DC to this motor - by PWM'ing a MOSFET.
There are many examples of that around the Forum (click on "Playground" and Search around a bit.)
I'm PWM'ing a couple of MOSFETs (in parallel) to vary the speed of a 350W motor --