By definition, you have. Unless the inputs are 5V tolerant, long term operation of the part is far from guaranteed. Even a single application of overvoltage can cause latent damage.
I was talking about the PCB layout, not the schematic... but you can also modify the module instead. Cut the 5V pin and jumper 5V to 3.3V on the header.
You can't safely run with the solution you've proposed because the module has its own pull up resistors connected to 5V...