This might be redundant for many, but I want to point out that Power is also proportionate to the square of the Voltage. P=V2/R This is something many people learn the hard way after raising the input voltage to a linear regulator. This results in an exponential power dissipation increase just the same as increasing the current flow. What worked coolly with a 9V battery as input, now puts the regulator in shutdown after switching to a 12V supply.
Actually the input to a voltage regulator is a poor example here as linear regulators look like constant current loads (the regulator
keeps the output voltage constant, typically the load then draws a fixed current, thus the input to the regulator is a constant current.)
Increasing the voltage to the linear regulator does not increase the current (unlike the resistor case). Dissipation in regulator is (Vin - Vout) * current.
Also a square law is not exponential - exponential increase is what happens as you increase the voltage across a diode, and makes
a square law look tame!