Open serial port problem

It's almost impossible to make sense of a photo like that. Can you draw a diagram of the wiring and post a photo of the drawing?

Can you write a short demo sketch that just sends data through the Max232 without bothering with anything else. Does that data show on your scope? No point trying anything else until you get that to work.

...R