but in my case I'd be glad to have a simple little config program that sends stuff out via USB.
That is relatively easy. The config data can then be stored in EEPROM, and read when the Arduino starts up.
Opening the serial port resets the Arduino, so any code to get config data could go in setup. When the user starts the Arduino, without the USB connected, there would be a short pause where the sketch looks for serial data. When none is found, it reads the data from EEPROM and starts loop()ing.
When the USB cable IS connected, it reads serial data from the app sending it the data, and overwrites the data in EEPROM. When the end of the data arrives, it starts loop()ing.