I think the vectors can only access the first 64 or 128k
This part turns out to be wrong. The two-word "jump" and "call" instructions can access the full 4MB address space.
However, the indirect jump/call (ijmp/icall) instructions can only access the first 64kbytes, even on CPUs with 128k of memory. Chips with 256k of memory add an "EIJMP/EICALL" that can jump anywhere.
(This sort of "kludge" is why I think 32bit CPUs (like the ARM in Due) will displace 8bit CPUs in applications where code or data exceed 64k.)