Okay, read that post again.... whole bunch more issues ;-)
They go between Vcc and Gnd, and between AVcc and Gnd, right next to the microcontroller. I put one on each side of the chip. You need a cap like this on basically any digital integrated circuit, 0.1uF is a good default value - of course always defer to the datasheet if it tells you something different about decoupling. But assume 0.1uF on any digital IC; this is such a routine thing that it's usually mentioned only in passing in the datasheet.
You should immediately discard any guide which omits the decoupling caps. There's a great proliferation of bad guides (the common issues are missing decoupling caps, and conflating ARef and AVcc; the latter must be tied to Vcc, the former must not be) - in the case of errors like this, the system will often pass quick tests of functionality (yet fail under other conditions; note also that some individual parts work better than others in these out-of-spec conditions due to process variation).
You need a third 0.1uf cap if you want do programing via serial (for example with an FTDI adapter) and have the autoreset (it is not needed if programming via ISP. It's not strictly necessary for programming via serial, since you can reset it by carefully timing a press of reset button, but I have like... a 10-20% success rate timing it correctly). One side of this cap goes to reset, the other side to the DTR or RTS pin of your serial adapter, and it automatically resets the chip when serial connection is opened. Google DTR reset for details.
You also need a piece of circuit board material to build it on. I sell some lovely ones that are just the right size for a small atmega328p "perfboarduino" ;-) - my store has a bunch of other sizes and shapes of prototyping board, which is a lot less work than strip-board, and more convenient to work with because it's got copper on both sides, not just one like strip-board
You cannot burn the bootloader with a serial adapter (well, you can in theory, but it's agonizingly slow and nobody does it) - that's what you use to program it after burning the bootloader. You can use another Arduino with the Arduino as ISP sketch, or a USBAsp from ebay/aliexpress ($3 or so) to burn the bootloader.
Under $2 for a supposed FT232RL serial adapter? That's probably counterfeit, and counterfeit FT232's don't work without great difficulty on windows (see "ftdigate"). I recommend against buying any product with an FT232 in it, because you can't be sure when buying from overseas if the FT232 is legit - but at that price, I'll wager money that it's fake.
If you want it cheap, get one of these CH340G based ones (the kind with the black PCB and the switch for voltage - not the black ones with the jumper):
https://www.ebay.com/itm/TTL-USB2-0-To-6Pin-CH340G-Converter-for-STC-Arduino-PRO-Instead-of-CP2102-PL2303/191196018631 (no specific endorsement of that vendor, just shown as an example of the version you want).
There may be other good options made with CP2102 - I think the chip itself is a bit more capable, and the drivers aren't annoying to install on mac like CH340G drivers are - but I haven't seen one with both a nice way to switch to 3.3v logic levels and the 6-pin header you want (for using with pro mini or similar in the future).
0.1" pin header to connect to things with.
I would not classify a reset button as necessary if you have autoreset circuit for programming (or are programming via ISP without a bootloader); I frequently don't include them in my designs (usually it's easy to short the reset pin to ground with the points of a pair of tweezers or needlenose pliers if I really have to)
You're powering it off 18650 batteries and want battery life? You're better off running it at 8mhz, instead of 16mhz - 16 is only in spec above 3.8v or so. In practice, it appears that the overwhelming majority of '328p's will run just fine at 16mhz down to at least 3.3v, but I recommend designing for the parts to be in spec when possible. So get an 8mhz crystal too. You should have 8, 12, and 16mhz crystals on hand imo.
FFS, did I just write that essay? I gotta start charging for forum posts!