Have you tried writing a simplified program that just prints the x,y locations of where you touch and then relating that to where you have your buttons placed?
I have a vague memory of one of these touch libraries using a coordinate system that's the opposite (or at least one axis swapped) of the drawing library. However, I've used so many of either that I can't remember exactly which library it was or how it was different.