Interest in micropower / energyharvesting Arduino?

@Udo: It's a shame almost no semiconductor companies spec out all the weird stuff their products do during undervoltage / ramp-up.

There are probably better ways to do it using a collection of FETs, etc., but one way I have used to deal with the "over-the-hump" issue is to use a low-power comparator and voltage reference... mine used an LTC1540 (nanopower w/ builtin reference), the resistor divider on the + input set for the desired turn-on voltage and in the megaohms range. IIRC, this chip can source 40mA, so its output could be used directly if your circuit is not too hungry while active.

On its own it can provide maybe 100mV of hysteresis, but more is probably desirable. For this, add a bit of positive feedback (10M or so) from the output to the + input as well. This will keep the comparator from switching off again during reasonable turn-on voltage dips and keep the size of your input cap more manageable.

Finally, although this chip does not suck excessive current during power-up/undervoltage, it can behave oddly. I found using a 1.5-2V or so open-drain voltage detector to ground the + input during undervoltage solves it. Total current consumption of this setup peaked at 2-3uA @ 1.2V or so, then leveled off to about 0.8uA as the voltage increases to a reasonable level. If the turnon/off thresholds are not critical and the operating voltages are high enough to turn a FET fully on, you can probably skip the comparator and pull the same feedback trick with a voltage detector + FET directly.