The simplest way is to just use the Arduino but no longer the USB interface. China made Arduinos are around US$ 20.
Otherwise it depends on the hardware setup. You have to solder your own hardware add-ons somewhere, haven't you? Just leave a little space on that board for the AVR chip, and 5 or 6 more small parts.
Search for DIY Arduinos or breadboard Arduinos in this forum or the web. Note there there is literally NOTHING in an Arduino- it is a concept rather than lots of hardware.