ASCII code 253 (375 in base 8) does not correspond to a key you can type, like A, f, 5, or @. So, you have to define the code as \375, which is not the same as '', '3', '7', '5'.
When the Arduino reads '', '3', '7', and '5' from the serial port, it needs to see that as an escape sequence, and print something different to the speaker chip.
You keep saying "yeah, I know that", but you have not shown any code that understands that \ is the start of an escape sequence, rather than just another character.
Do you have such code that you are unwilling to share, or do you not understand that YOU have to write such code?