TCS34725 color sensor issue

As far as I can tell, it was Adafruit that added the 5V to 3.3V VR on their board. I doubt the native part has that.
I am pretty much convinced you did not toast the board, and also sure that we understand how the 5V vs 3.3V is being done.
So we are back at step 1.
Did you try swapping the SCL and SDA wires?