Go Down

Topic: SoftwareSerial magic numbers (Read 23614 times)previous topic - next topic

robtillaart

Dec 21, 2012, 11:11 amLast Edit: Dec 21, 2012, 11:13 am by robtillaart Reason: 1
I had a look at the magic numbers in SoftwareSerial today (only the 16Mhz part for now) because I wanted to check non-standard serial baudrates too.

Code: [Select]
`static const DELAY_TABLE PROGMEM table[] = {  //  baud    rxcenter   rxintra    rxstop    tx  { 115200,   1,         17,        17,       12,    },  { 57600,    10,        37,        37,       33,    },  { 38400,    25,        57,        57,       54,    },  { 31250,    31,        70,        70,       68,    },  { 28800,    34,        77,        77,       74,    },  { 19200,    54,        117,       117,      114,   },  { 14400,    74,        156,       156,      153,   },  { 9600,     114,       236,       236,      233,   },  { 4800,     233,       474,       474,      471,   },  { 2400,     471,       950,       950,      947,   },  { 1200,     947,       1902,      1902,     1899,  },  { 300,      3804,      7617,      7617,     7614,  },};`

I came up with the following formulas
Code: [Select]
`rxstop = 16000000L/(7 * baudrate) - 2;rxintra = rxstop;  // an easy one tx = rxstop - 4;   // slight errorrxcenter = rxstop/2 - 5;  // slight error at higher baud rates.`

this generates the following table

Code: [Select]
`baud rxcenter rxintra rxstop tx115200 3 17 17 1457600 13 37 37 3438400 23 57 57 5431250 30 71 71 6828800 33 77 77 7419200 53 117 117 11414400 73 156 156 1539600 113 236 236 2334800 232 474 474 4712400 470 950 950 9471200 946 1902 1902 1899300 3803 7617 7617 7614`

The difference 'matrix'
Code: [Select]
`2 0 0 23 0 0 12 0 0 01 1 1 01 0 0 01 0 0 01 0 0 01 0 0 01 0 0 01 0 0 01 0 0 01 0 0 0`

Not yet tested thoroughly but the error matrix indicates that the tables can be replaced by formulas, allowing non standard baud rates.

(to be continued)

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

robtillaart

#1
Dec 21, 2012, 05:23 pm

8 Mhz tables

Code: [Select]
`static const DELAY_TABLE table[] PROGMEM = {  //  baud    rxcenter    rxintra    rxstop  tx  { 115200,   1,          5,         5,      3,      },  { 57600,    1,          15,        15,     13,     },  { 38400,    2,          25,        26,     23,     },  { 31250,    7,          32,        33,     29,     },  { 28800,    11,         35,        35,     32,     },  { 19200,    20,         55,        55,     52,     },  { 14400,    30,         75,        75,     72,     },  { 9600,     50,         114,       114,    112,    },  { 4800,     110,        233,       233,    230,    },  { 2400,     229,        472,       472,    469,    },  { 1200,     467,        948,       948,    945,    },  { 300,      1895,       3805,      3805,   3802,   },};`

the formulas:
Code: [Select]
`int rxstop = 8000000L/(7 * baudrate) - 4;int rxintra = rxstop;int tx = rxstop - 3;int rxcenter = max(rxstop/2 - 7, 1);`

results of these formulas
Code: [Select]
`baud rxcenter rxintra rxstop tx115200 1 5 5 257600 1 15 15 1238400 5 25 25 2231250 9 32 32 2928800 10 35 35 3219200 20 55 55 5214400 30 75 75 729600 50 115 115 1124800 110 234 234 2312400 229 472 472 4691200 467 948 948 945300 1895 3805 3805 3802`

The difference 'matrix'
Code: [Select]
`0 0 0 10 0 0 13 0 1 12 0 1 01 0 0 00 0 0 00 0 0 00 1 1 00 1 1 10 0 0 00 0 0 00 0 0 0`
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

#2
Dec 21, 2012, 06:58 pm

I think rxcenter is adjusted a bit to try to account for the ISR overhead.  Is that correct?  Have you done something similar (is that the "- 5" in the first formulas)?

robtillaart

#3
Dec 21, 2012, 07:17 pmLast Edit: Dec 21, 2012, 07:55 pm by robtillaart Reason: 1
Quote
(is that the "- 5" in the first formulas)

rxcenter is used to get the middle of the pulse, so when reading the bit stream you're not on a falling or rising edge.

What I actually did was place the existing table in Excel; Calculate the "ideal" table (clock/baud) and adjust the numbers for the differences (subtractions).
For the 8Mhz I got a negative value so I adjusted the formula with an additional max() [pragmatic].
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

robtillaart

#4
Dec 21, 2012, 07:50 pmLast Edit: Dec 21, 2012, 07:55 pm by robtillaart Reason: 1
First test with Arduino 1.0 SoftwareSerial example code and a Serial LCD - 9600 & 19200 baud - it behaves identical.
Mind you: Both the original table version and the new formula version miss a byte once and a while when sending more than ~10 bytes
Adding a delay(1) in the code helps a bit but.....

OK, the changes in SoftwareSerial.cpp are
Code: [Select]
`void SoftwareSerial::begin(long speed){  _rx_delay_centering = _rx_delay_intrabit = _rx_delay_stopbit = _tx_delay = 0;  // added lines  long baud = speed;  _rx_delay_stopbit = 16000000L/(7 * baud) - 2;  _rx_delay_intrabit = _rx_delay_stopbit;  _tx_delay = _rx_delay_stopbit - 4;  _rx_delay_centering = _rx_delay_stopbit/2 - 5;  //  // for (unsigned i=0; i<sizeof(table)/sizeof(table[0]); ++i)  // {    // long baud = pgm_read_dword(&table[i].baud);    // if (baud == speed)    // {      // _rx_delay_centering = pgm_read_word(&table[i].rx_delay_centering);      // _rx_delay_intrabit = pgm_read_word(&table[i].rx_delay_intrabit);      // _rx_delay_stopbit = pgm_read_word(&table[i].rx_delay_stopbit);      // _tx_delay = pgm_read_word(&table[i].tx_delay);      // break;    // }  // }  // Set up RX interrupts, but only if we have a valid RX baud rate  if (_rx_delay_stopbit)  {    if (digitalPinToPCICR(_receivePin))    {      *digitalPinToPCICR(_receivePin) |= _BV(digitalPinToPCICRbit(_receivePin));      *digitalPinToPCMSK(_receivePin) |= _BV(digitalPinToPCMSKbit(_receivePin));    }    tunedDelay(_tx_delay); // if we were low this establishes the end  }#if _DEBUG  pinMode(_DEBUG_PIN1, OUTPUT);  pinMode(_DEBUG_PIN2, OUTPUT);#endif  listen();}`

The memory size of the 2 sketches:
TABLE BASED : 4484
FORMULA BASED: 4302
==> 182 bytes less

every entry in the DELAY_TABLE = 12 bytes x 12 entries = 144 bytes
so the code is also shorter => no search loop and progmem calls)

in fact as we calculate the baudrate we can also leave out the following lines from  SoftwareSerial::begin()
-   _rx_delay_centering = _rx_delay_intrabit = _rx_delay_stopbit = _tx_delay = 0;
- if (_rx_delay_stopbit)
- and remove the local variable baud too

difference is now 192 bytes less!
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

robtillaart

#5
Dec 21, 2012, 07:54 pm
Conclusion:

1) a smaller code base (192 bytes) and
2) freedom of baud rate selection.

The 2nd is quite useful as now one can adapt the baud rate when the Arduino has no 16.000.000 Hz Crystal but a resonator producing less or more cycles / second.

Pity the SoftwareSerial drops some chars once and a while (original version did that too)

Todo:
- check every baud rate (old & new)
- check this code on 1.0.3
- 20Mhz tables => formulas
- Due too?
- test on other Arduino's [MegaTeensy]
- ...
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

robtillaart

#6
Dec 21, 2012, 08:18 pm

first version 20 MHz formulas.

Code: [Select]
`static const DELAY_TABLE PROGMEM table[] ={  //  baud    rxcenter    rxintra    rxstop  tx  { 115200,   3,          21,        21,     18,     },  { 57600,    20,         43,        43,     41,     },  { 38400,    37,         73,        73,     70,     },  { 31250,    45,         89,        89,     88,     },  { 28800,    46,         98,        98,     95,     },  { 19200,    71,         148,       148,    145,    },  { 14400,    96,         197,       197,    194,    },  { 9600,     146,        297,       297,    294,    },  { 4800,     296,        595,       595,    592,    },  { 2400,     592,        1189,      1189,   1186,   },  { 1200,     1187,       2379,      2379,   2376,   },  { 300,      4759,       9523,      9523,   9520,   },};`

formulas
Code: [Select]
`int rxstop = 20000000L/(7 * baudrate) - 1;int rxintra = rxstop;int tx = rxstop - 3;int rxcenter = rxstop/2 - 4;115200 7 23 23 2057600 20 48 48 4538400 32 73 73 7031250 41 90 90 8728800 45 98 98 9519200 69 147 147 14414400 94 197 197 1949600 144 296 296 2934800 293 594 594 5912400 590 1189 1189 11861200 1185 2379 2379 2376300 4757 9522 9522 9519`

The difference 'matrix'
Code: [Select]
`4 2 2 20 5 5 45 0 0 04 1 1 11 0 0 02 1 1 12 0 0 02 1 1 13 1 1 12 0 0 02 0 0 02 1 1 1`
definitely worse than the 16 and 8 Mhz difference matrices, but it's a start
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

robtillaart

#7
Dec 22, 2012, 05:30 pm
Still wondering why there is a rxintra and a rxstop column. The values are identical....
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

Mark G.

#8
Dec 22, 2012, 08:13 pm
@robtillaart,

Noob question, but what's the reason for these values in SoftwareSerial?

Obviously, less memory usage is a good thing.

robtillaart

#9
Jan 02, 2013, 04:40 pm
submitted as proposal - https://github.com/arduino/Arduino/issues/1198 -
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

robtillaart

#10
Jan 02, 2013, 04:54 pmLast Edit: Jan 02, 2013, 06:56 pm by robtillaart Reason: 1
@Marc G

A serial protocol is like a train with wagons and on each wagon there is one bit, 10 bits in total (including start/stop bits). The baud rate represents the speed of the train.

The software serial receiving code is triggered by the edge of the start bit (train). To read a bit properly one wants to read the value of the signal (HIGH/LOW) in the middle of the bit, not at the edges.

rxCenter is the time to the (approx) middle of the first (start) bit,
rxIntra, rxStop are used as timings from the middle of one bit to the middle of the next. The higher the baud rate the lower these numbers.

tx is used for the timing for the transmit.

You can see this in the code of the library - C:\Program Files (x86)\arduino-1.0\libraries\SoftwareSerial - (windows)
and search for this function - void SoftwareSerial::recv() -
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

robtillaart

#11
Jan 02, 2013, 07:19 pmLast Edit: Jan 02, 2013, 07:21 pm by robtillaart Reason: 1
Did a more extensive test with the SoftwareSerial using the formula approach.
I connected two Arduino's - UNO, 16Mhz (resonator) + 2009, 16Mhz (crystal) +  IDE 1.0 - both using SoftwareSerial, one Master and the other Slave (essentially an echo).

The master sent byte 0x55 at baudrate 100 and waits until the slave echos it back.
If the answer is not 0x55, the test fails and master prints a message.
Otherwise it just increases the baud rate with 100 and starts over.

The results are pretty good as it only gets constantly distorted above 190K baud.
Between 90K and 190K it only failed 10 times.

I took 0x55 as test pattern 0x55 == 01010101 ; it helps to see what happened. (see comments after output

Typical output  (multiple runs had comparable output)
Note: started with baud rate 100 in steps of 100...
Code: [Select]
`start...BAUD BYTE97600 F5 FAIL  // = 11110101    ??111200 AA FAIL  // = 10101010   1 bit shifted114400 D5 FAIL  // = 11010101   1 bit failed (interference with start bit ?124600 AA FAIL140600 D5 FAIL145500 AA FAIL149000 D5 FAIL163200 AF FAIL  // = 10101111  ??190500 FF FAIL  // = 11111111  expect sync lost 190600 FF FAIL190700 FF FAIL190800 FF FAIL190900 FF FAIL191000 FF FAIL...`

The master and the slave were kept in sync by starting at the same baud rate an wait for each other.

To repeat the test start the master, then start the slave, and press a char in the serial monitor of the master.

Slave program (essentially echo)
Code: [Select]
`////    FILE: serialSlave (echo)//  AUTHOR: Rob Tillaart//    DATE: 2013-01-02//// PUPROSE: test SW serial with formulas//#include <SoftwareSerial.h>SoftwareSerial mySerial(2, 3);void setup(){  Serial.begin(9600);  Serial.println("start slave...");}unsigned long baud=0;void loop(){  baud += 100;  mySerial.begin(baud);  while (mySerial.available() == 0);  int b = mySerial.read();  mySerial.write(b);  Serial.println(b,DEC);  delay(10);}`

master program
Code: [Select]
`////    FILE: serialMaster//  AUTHOR: Rob Tillaart//    DATE: 2013-01-02//// PUPROSE: test SW serial//#include <SoftwareSerial.h>SoftwareSerial mySerial(2, 3);void setup(){  Serial.begin(9600);  Serial.println("start...");}unsigned long baud=0;void loop(){  if (Serial.available() > 0)  {    Serial.flush();    baud += 100;    mySerial.begin(baud);    mySerial.write(0x55);    while (mySerial.available() == 0);    int b = mySerial.read();    if (b != 0x55)    {      Serial.print(baud);      Serial.print("\t");      Serial.print(b, HEX);      Serial.print("\t");      Serial.println(" FAIL");    }    delay(20);  }}`

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

robtillaart

#12
Jan 02, 2013, 07:49 pm

same test with stepsize 10 gave some more errors (typical run started with baud rate 10, step 10)
Code: [Select]
`start...BAUD BYTE70660 D5 FAIL81950 AD FAIL88870 AF FAIL89570 BD FAIL94410 D5 FAIL95340 AA FAIL96590 D5 FAIL98980 AA FAIL100750 AB FAIL103590 BD FAIL105740 D5 FAIL110600 AA FAIL113260 AF FAIL120200 AA FAIL...`
Up till 70K no failures  ( that are 7000 different baudrates tested !)
between 70K and 115K "only" 13 failures   (13 fail on 4500 baudrates tested ~~ 1/300
above 120K the failures increased, not shown

Conclusion from the tests, SoftwareSerial "by formula" works very good up to 70.000 and reasonable well up to 115.200
Tweaking the formulas further may improve the test results but for now I'm quite satisfied.

This SoftwareSerial "by formula" allows one to build a communication channel in which the baud rate is constantly altered, making it very difficult to eavesdrop - and yes to get in sync
Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

mikalhart

#13
Jan 04, 2013, 04:43 am
@Rob,

I applaud this effort.  Thanks for investigating so thoroughly.  I always thought it might be fun to develop some equations that allow the synthesis of the "table" values on the fly, and now it looks like you are pretty close to doing just that.

It's good that you are getting error-free transmission up to 70K.  Make sure you test not just the single byte round trip, but also lots of bursts.  The values should vary Example:

1. Arduino sends 0x55 as fast as possible to host for one minute.
2. Arduino sends 0xFE as fast as possible to host for one minute.
3. Arduino sends 0x01 as fast as possible to host for one minute.
4. Host sends 0x55 as fast as possible to Arduino for one minute.
5. Host sends 0xFE as fast as possible to Arduino for one minute.
6. Host sends 0x01 as fast as possible to Arduino for one minute.

When constructing the tables, I found several times that I thought the values were good--until I tested the large bursts.

If we want to improve performance at baud rates > 57.6K, I think we're going to have to optimize the timer tick vector.  I studied this for some time with the logic analyzer and discovered that the occasional glitch was due to a timer tick interrupt being processed exactly when a pin change was pending.

Lastly, and you probably already know this, but if your formula is off a bit for the lower baud rates, it shouldn't be a big deal.  They are very tolerant.

Nice!

Mikal

drjiohnsmith

#14
Jan 04, 2013, 09:21 am
robtillaart

like the idea of a formula,
but could be slow to change baud rate could it not.

not certain, and for interest,
did you try two boards connected using the standard software serial code,
did you try two boards using the standard hardware uart.

Go Up

Please enter a valid email to subscribe