Took me a while to solve this one
After some time I decided to write complete receive part of code from the beginning and it solved my problem with double sync but this time instead of sending one byte as sync I'm sending complete last received string to arduino for comparison.
If transmitted and received data are not the same it will replay last transmitted data. It keeps data integrity if something goes wrong.
I did try to increase the speed of reading data from eeprom by reading more than 8-9 bites before sending them over serial but it failed almost immediately so I thought it cant be computer side problem because I did define SerialPort1.ReadBufferSize = 2000000... Should be enough right?
So I used Due instead of nano and tried to read eeprom again, it failed again as before so what next?
I started to suspect that maybe arduino serial library causes the problem and decided to rewrite complete code for STM32F446 with atollic studio and HAL drivers and still I got the same result.
After quite a bit of time spent on vb net looking for patterns in the data when it fails I find out that serial receives everything from arduino but it only saves around 30 characters in variable and it triggers serial receive event twice so I should read it twice and keep the data like so.
'serial handler
Public Sub ReceivedText(ByVal [text] As String)
If Me.TextBox1.InvokeRequired Then
Dim x As New SetTextCallBack(AddressOf ReceivedText)
Me.BeginInvoke(x, New Object() {(text)})
ElseIf Not text = "" Then
If Com_port_received_text = "" Then
Com_port_received_text = text 'Read first part
Else
Com_port_received_text1 = text 'Read the rest of data
End If
End If
text = ""
End Sub
And it work as it should so now I can read more than 8 bytes from eeprom and send them over serial.
As sweet spot I decide to read 16 bytes in total it seems like I'm getting the best results when comparing how long it takes to read complete eeprom.
Also I was careful to write code without delays while receiving the data and I'm receiveing data on another thread so basically I did everything I could to get this thing to go as fast as possible. And results are not so great. max transmission rate is 1kB/s.
any way here is edited transmit/receive code
do {
for (int vv = 0; vv < 16; vv++) {
ii = shiftIn(so_pin, sck, MSBFIRST);
readString += ii;
readString = readString + "h";
ii = '\0';
if (i >= address_length - 1) {
vv = 16;
}
i++;
}
Resend:
Serial.print(readString);
char buff[readString.length() + 1];
String readString1 = "";
while (!Serial.available());
if (Serial.available()) {
int length = Serial.readBytes(buff, sizeof(buff) - 1);
buff[length] = 0;
String str(buff);
readString1 = str;
}
if (readString1 != readString) {
delay(500);
Serial.print("NOK");
Serial.flush();
delay(500);
readString1 = "";
goto Resend;
}
readString = "";
if (i == address_length) {
Serial.print("X");
}
if (i >= address_length) {
break;
}
} while (i != address_length);
delay(100);
Serial.print("X");
I didn't bother commenting arduino code, I'm sorry
'read data from eeprom
Dim DataArray As New ArrayList
Dim ReceivedBits As Integer = 0
Retry:
SerialPort1.DiscardOutBuffer() 'Empty output and input buffer of serial
SerialPort1.DiscardInBuffer()
Com_port_received_text = ""
Com_port_received_text1 = "" 'Empty vars where serial receive will store data
Do
If Not Com_port_received_text = "" Then 'if something was received
If Not Com_port_received_text.Contains("NOK") Then 'and it does not contain "NOK"
Dim iCnt As Integer = 0
For Each character As Char In Com_port_received_text
If character = "h" Then iCnt += 1 'Get number of h characters received from serial
Next 'used to count number of bits received from serial
TimerValue = 0
If Not iCnt = 16 And ReceivedBits + 16 <= ReadAddress_Count Then 'if there is less than 16bits received
Do 'wait the rest of the data to be received from serial
If TimerValue = 10 Then
GoTo NoResponse 'if nothing was received in 1second then exit loop
End If
Application.DoEvents()
Loop While Com_port_received_text1 = ""
Com_port_received_text = Com_port_received_text + Com_port_received_text1 'combine received bits
End If
SerialPort1.Write(Com_port_received_text) 'replay received bits to arduino for verification
DataArray.Add(Com_port_received_text) ' save them in array
ReceivedBits = ReceivedBits + 16 ' count how many bits was received
Com_port_received_text = ""
Com_port_received_text1 = ""
Else
DataArray.RemoveAt(DataArray.Count - 1) 'if arduino says we got wrong data remove it from array
Threading.Thread.Sleep(5) 'wait some time for arduino to prepare and send again last bits
GoTo Retry 'go to the begining to get last bits again
End If
Else
NoResponse:
TimerValue = 0
Com_port_received_text = ""
Com_port_received_text1 = ""
Do
If Com_port_received_text1.Contains("NOK") Then 'if arduino replays with NOK
Com_port_received_text1 = ""
End If
If TimerValue > 100 Then 'if arduino stucks after 10 seconds
SerialPort1.Write("1") 'write something to get arduino going
GoTo Retry
End If
Application.DoEvents() 'Be there until arduino responds
Loop While Com_port_received_text1 = "" And Com_port_received_text = ""
End If
Application.DoEvents()
Loop Until Com_port_received_text = "X" 'indicates the end of transmission
Here you can see how arduino and pc handles errors

Arduino serial = RXD, pc serial = TXD
it is not visible on the image but computer didn't respond received data and arduino waits for respond. After 10s computer realizes that arduino waits for respond and it sends random bite.
Then arduino send NOK because random bite is not what was sent last time and arduino replays last bytes, then communication keeps going. on the image you can see that error occurred twice.
At the moment sometimes I'm getting one byte more than I should over serial. By looking at number of positive clock cycles it is clear that arduino sometimes clocks out 8 clocks extra but it happens only if I read more than 100k addresses. Im still fighting with this issue..
It is close to impossible to track with logic analyzer what causes this problem I made simple progress bar to show number of addresses readed. So if it is a glitch in software or hardware I should be able to see on what address it stuck therefore knowing address I can easily compare data between bin file and logic analyzer.
this is how it looks
