Even if I don't push the button, it reads HIGH value

Well, THIS is the problem:
image
Here you must connect (with a jumper or another cable) one blue/red rail with the other blue/red (topmost blue one is the culprit in your case) because top and bottom blue/red rails aren't connected from one side to the other.

Besides, for the future I suggest you a better wire color coding.
First of all, ALWAYS use black wires for GND and red for +5V. This is crucial (and sometimes vital for the devices...).

For other wire types make your kinda "standard" choice (e.g. like yellow for analog inputs, blue for digital, green for buttons, and so on), but always get stuck with "your" standard and avoid mixing colors for the same connection type.