SoftwareSerial changing speed unexpectedly

Or anything other to check?

Seeing your code might help ...

By using SoftwareSerial on a Leonardo you are creating a third serial interface. What are the other two being used for ?