What about the bootloader?

Well, nevermind.
I was able to use bossa command line tool just fine.
Then I went back to Arduino IDE and was able to upload too. So, maybe it was just bad connection in one of the jumper wires when I was trying before.