Ok, so I used the schematic from the following link as a basis for this. The only change I made to the MAX3232 part of the circuit is the cap from pin 9 on the Max3232 to Reset on the m328P, I read you need to do this to be able to program via serial unless you want to manually time hitting the reset button.
I am using a USB to Serial converter, it shows up as a Prolific USB-to-Serial Comm Port. My schematic is attached. And here is the code I tested with.
Now, I'm clearly getting communication, since I get feedback in the serial window (see attachment). But obviously this is not what I want. So does anyone have a clue as to where I screwed this up? Because at this point I'm lost. I'm getting serial... The only thing I can think of is maybe a timing issue... But considering that the port and the AVR are both set to 9600 baud, I don't see how this could be the issue.
It looks like you're using Eagle. Don't use the blue bus. Just put names on the wires and then two unconnected wires with the same name will be connected on the PCB. This is so much easier to read than having buses cross over the top of components.
This way we can't tell if you've really connected T1in to TX.
Do you know why pins 6 and 4 on the DE-9 plug are connected together? It's probably not necessary and I would like to keep the number of pins that can affect RESET to the absolute minimum.
To clarify the schematic, although I could always redraw the schematic if it would be easier to read, pins 12 and 11 of the 3232 are connected to Rx and tx (12 to Rx, 11 to tx). For debugging purposes I have removed the connection between pins 4 and 6, and pins 7 and 8. The LEDs from tx and Rx have also been removed.
After doing so I'm still getting the same gibberish.
OK, that's progress at least. You've eliminated a lot of the variables.
Have you used this particular USB to Serial converter before? Are you sure that you've got it configured correctly and it talks to other 9600 baud devices OK? I'm thinking that perhaps it has some unusual setting set incorrectly.
Do you have a voltmeter to check the voltage on the MAX232's capacitors? There should be at least +6v and -6V on pin 2 and 6.
Your schematic doesn't show the power connection for the MAX232. The symbol I have for it in my Eagle library has an optional extra piece which is actually not optional. If you don't put that on the schematic and wire it up then your MAX232 gets no power.
Yeah, I think I am using the sparkfun library... I'd have to go check on my desktop (posting from phone for ease...), but I can assure you, power and ground connections are present, and correct.
As for the circuitry, I haven't taken the voltmeter to it. I can check in a bit. But the circuit is laid out, capacitor wise that is, like the datasheet shows.
I checked to make sure my fuse settings were correct, they are. Logic analyzer will tell me more, I'm going to test it tonight. Both the UART and the oscillator (this seems eerily like a timing issue...
Ok, with some quick work on the Saleae Logic analyzer, I have determined that the Serial being sent to the Max3232 is correct (see attachment). So at this point I'm down to one of two things, either the Max3232 is incorrectly wired, or the USB to Serial converter is malfunctioning. And, honestly, I don't know how to progress from here. I have a serial card for my desktop. However, I don't have a cable, I will probably order one tomorrow. And I can't test the USB to Serial cable as the logic from my understanding is 12v, and the Saleae is only good for 3.3-5v. So I don't know which it is that is having the problem. Although I am leaning towards the Mas3232, it is plausible that the USB dongle is not working correctly. Afterall, it was made in china and bought for $2-3 shipped....
I've checked all the settings for the usb dongle, it appears to be set to 9600 baud.... I guess at this point it's time to take a step back and think about it for a while. The only thing I can really say hasn't ever been tested before is the USB Dongle. So that is what I am thinking at this point. My understanding of this circuit is at a minimum it only requires the TX and RX, and GND of course, connected to the RS232 socket. So I think I am going to sever everthing else as a last effort to get it working before trying an actual serial cable to my PCI card. Other than that, I can't think of anything else.
Ok, well with a little bit of googling I happened upon two pages, one of which on the Arduino.cc tutorials site, and I'm going to test the circuits on those pages below:
It appears that they use a different set of pin connections on those two. Ones that on the other pages look to be unused, or used for RTS/CTS. So it appears, as of right now, that it could be an issue with the particular circuit that I used.
I think this is what I get for googling "Arduino + Max3232" instead of "Arduino + Max232", since I don't even remember seeing these two pages pop up. But seeing as how the pinouts of the Max232 and the Max3232 are identicle, I think this might work. Only one way to find out... I'll breadboard it this evening and post the results.