A simplification that I think is useful to describe how a computer is 8-bit, 16-bit, 32-bit, or 64-bit is as follows:
Remember when you were in grade school and were told to add 23 + 49? Unless you are a calculation savant, you don't add 23 and 49 in one step. Instead, you add 3 + 9, which gives you 12. You put '2' down as the last digit, and then carry the '1'. You then do 1 + 2 + 4 to give '7', and you put those together as '73'.
Now, an 8-bit computer like the AVR inside of Arduinos (except for the Due) does things in terms of 8-bit buckets. So a 'short' on the AVR is 16-bits, that means it has two 8-bit parts (called bytes). When the computer does an add, it loads the bottom byte from each side, and does an add operation that returns the bottom byte of the resultant add, and the 1 bit carry operation. It stores the bottom byte, and then loads the top bytes, and does an add with carry operation that combines the carry from the previous add along with the two bytes. If you are adding a long which is a 32-bit data type, it would do 1 normal add, and 3 add with carry operations. On the AVR, 'int' is 16 bits (like 'short'), but on an ARM, it is 32 bits.
A 32-bit processor like the ARM in the Arduino Due, would be able to do an add of two longs (or ints) in one instruction. However, in doing two adds of two short values and storing the result in a short value, the ARM would need 3 instructions, because the ARM chip does not have a 16-bit add instruction. Instead the compiler does the calculation in 32-bits, and then needs to convert the 32-bit long to 16-bit short via shift left 16-bits, and shift right 16-bits with sign extension.
Some chips can be 32-bit or 64-bit like modern x86's. There is a mode within the hardware that says whether internally things are done in 32-bits or 64-bits. If you have a new workstation but are running Windows XP on it, you likely are running in 32-bit mode. If you were running Windows 8, most programs would be run in 64-bit mode, but the operating system can recognize when it is asked to run a 32-bit binary, and arranges when that binary is run, the process is run in 32-bit mode (the operating system still runs in 64-bit mode, and when the control goes back to the OS, it switches mode to 64-bit).