The problem with your circuit is that the TLS261 is an analog light sensor. The output goes up linearly with light input. (Look at the datasheet for the graph of output voltage vs irradiance.) But you're connecting it to the interrupt input pin which is a digital input. Digital inputs (especially edge triggered inputs like an interrupt) are very unreliable when you drive them with analog signals.
You could add a comparator to your interrupt circuit that compares the output of the TLS261 to a ref voltage set by a pot. This would generate a clean digital signal for the interrupt and would allow you to calibrate the switching point of the sensor.
Edit: Also, I'd power the sensor and comparator from 5V instead of 3.3 so that the sensor circuit can swing in the 5V range.
Good luck!