I thought that to, don't remember what happened when I set the clock divide, but I changed it back for some reason, also, forgot to mention that the bootloader doesn't seem to be starting up right, it doesn't do its triple flash then stop, it just keeps blinking (fairly quickly too);
EDIT: I find it strange though that I would need to set that bit, as the fuses are labeled as lfuse:0xFF hfuse:0xD8 efuse:0xFD in the boards.txt file, and the clock divide changes lfuse to 0x7F...
EDIT 2: This is very strange, it fixes my issue with not being able to upload with the bootloader and the baud rate issue, but now the timing is funny, delay(1000) seems to delay for about 8 seconds, and delay(100) is closer to a second, would really like to know what is going on with my board...