I have read that UNO chip EEPROM can be written/read around 10,000 times. Is that the same as saying I can only read the EEPROM stored memory only 10,000 times?
And is that referring to each address of the EEPROM as a whole?
First you confusing Flash memory space (where your program is stored) and EEPROM memory space (where user's data can be read and written to).
Also any endurance specification applies to writing and erasing only, there is no limit on reading from either flash or eeprom memory.
For the Atmel avr mega328P chip here are the writing endurance specifications, but note that these are very conservative specifications which are the minimum guaranteed, many have tested such specs and gotten much higher values. The Atmel datasheets are the best place to learn about these kinds of specifications.
Write/Erase Cycles: 10,000 Flash/100,000 EEPROM