Is there any need for any external components (e.g., caps) at all in this setup?
You don't have to have them. But it is better to have them.
A problem easily rectified with tuning.
It actually requires far more than that. RC oscillators suffer a few issues: temperature drift, accuracy and jitter. Accuracy at a given temperature is easy to deal with. Jitter for long-time horizon applications can be dealt with. Temperature drift is difficult to resolve. fungus is right on target on this one.