Tobbera:
When tacking upwind, tell tales is basically the only input you have when steering the boat. You set the sails, and steer as high as you can by looking at the tell tales. The tell tales shows the flow around the sail, which essentially is the wind direction. The only reason the tell tales indicates lift or heading, is because of change in apparent wind direction, which I hope I can sense with my wind vane in front of the sail.
I tried to indicate in my earlier post that I understand telltales. Of course they bear some relation to wind direction, but they do not represent the direction of the wind far enough away to be unaffected by the boat. A wind direction sensor that is as close to the sail as the telltales will give the same result as the telltales. One that is clear of the sails will give different data. The telltales are so close to the sail they are almost in the boundary layer.
Placing the wind vane in front of the tell tales will hopefully give enough input to the autopilot to keep the tell tales "flying". At lest better than I as human can. By placing the wind vane lower down in front of the boat, it will replicate what I see on the tell tales, and not be affected as much by side to side movement as you have in the top of the mast (which can oscillate 2 meters from side to side and upset the vane.)
This is where I fear you will be sadly disappointed. As I said earlier I don't believe it will replicate the telltales and I think you are grossly underestimating the computing power of your human brain.
I was never interested in racing but I crewed once or twice with a friend on a 30ft keelboat. He was adjusting the main sheet 1 or 2 inches at a time and it was having a beneficial effect. I don't think an Arduino program can match that.
...R