You should google I2C communications or look in the Arduino reference.
Basically the I2C communications is a two wire method of communication. One wire sends data the other wire receives data. (thanks RayLivingston)
Updated....sorry I had been working on an SPI issue all day.
Perhaps it is more like a street with the house number being the address. A mailman can drive down the street and deliver at the address. And a letter can be sent from that address to the post office (aka processor).
The address is similar to a phone number, each device has a different address. For your LCD it appears the address is 0x26. This means data sent on the I2C bus using the address 0x26 will go to you LCD instead of another device you might have on the I2C bus. Even though you may only have only one device connected you still need the address for your data to be recognized by your LCD display.
Most displays require some detailed programming to operate correctly. A library is a program or group of programs where some kind soul has written the tedious code and saved it as a library. You then can use this library to control your LCD using only simple commands. The folks who write these libraries should be given kudos for being so kind and helpful.