next topic »

[on:]

I had a look at the magic numbers in SoftwareSerial today (only the 16Mhz part for now) because I wanted to check non-standard serial baudrates too.

Code:

static const DELAY_TABLE PROGMEM table[] =
{
  //  baud    rxcenter   rxintra    rxstop    tx
  { 115200,   1,         17,        17,       12,    },
  { 57600,    10,        37,        37,       33,    },
  { 38400,    25,        57,        57,       54,    },
  { 31250,    31,        70,        70,       68,    },
  { 28800,    34,        77,        77,       74,    },
  { 19200,    54,        117,       117,      114,   },
  { 14400,    74,        156,       156,      153,   },
  { 9600,     114,       236,       236,      233,   },
  { 4800,     233,       474,       474,      471,   },
  { 2400,     471,       950,       950,      947,   },
  { 1200,     947,       1902,      1902,     1899,  },
  { 300,      3804,      7617,      7617,     7614,  },
};

I came up with the following formulas

Code:

rxstop = 16000000L/(7 * baudrate) - 2;
rxintra = rxstop;  // an easy one
tx = rxstop - 4;   // slight error
rxcenter = rxstop/2 - 5;  // slight error at higher baud rates.

this generates the following table

Code:

baud rxcenter rxintra rxstop tx
115200 3 17 17 14
57600 13 37 37 34
38400 23 57 57 54
31250 30 71 71 68
28800 33 77 77 74
19200 53 117 117 114
14400 73 156 156 153
9600 113 236 236 233
4800 232 474 474 471
2400 470 950 950 947
1200 946 1902 1902 1899
300 3803 7617 7617 7614

The difference 'matrix'

Code:

2 0 0 2
3 0 0 1
2 0 0 0
1 1 1 0
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0

Not yet tested thoroughly but the error matrix indicates that the tables can be replaced by formulas, allowing non standard baud rates.

(to be continued)

[Reply #1 on:]

8 Mhz tables

Code:

static const DELAY_TABLE table[] PROGMEM =
{
  //  baud    rxcenter    rxintra    rxstop  tx
  { 115200,   1,          5,         5,      3,      },
  { 57600,    1,          15,        15,     13,     },
  { 38400,    2,          25,        26,     23,     },
  { 31250,    7,          32,        33,     29,     },
  { 28800,    11,         35,        35,     32,     },
  { 19200,    20,         55,        55,     52,     },
  { 14400,    30,         75,        75,     72,     },
  { 9600,     50,         114,       114,    112,    },
  { 4800,     110,        233,       233,    230,    },
  { 2400,     229,        472,       472,    469,    },
  { 1200,     467,        948,       948,    945,    },
  { 300,      1895,       3805,      3805,   3802,   },
};

the formulas:

Code:

int rxstop = 8000000L/(7 * baudrate) - 4;
int rxintra = rxstop;
int tx = rxstop - 3;
int rxcenter = max(rxstop/2 - 7, 1);

results of these formulas

Code:

baud rxcenter rxintra rxstop tx
115200 1 5 5 2
57600 1 15 15 12
38400 5 25 25 22
31250 9 32 32 29
28800 10 35 35 32
19200 20 55 55 52
14400 30 75 75 72
9600 50 115 115 112
4800 110 234 234 231
2400 229 472 472 469
1200 467 948 948 945
300 1895 3805 3805 3802

The difference 'matrix'

Code:

0 0 0 1
0 0 0 1
3 0 1 1
2 0 1 0
1 0 0 0
0 0 0 0
0 0 0 0
0 1 1 0
0 1 1 1
0 0 0 0
0 0 0 0
0 0 0 0

[Reply #2 on:]

I think rxcenter is adjusted a bit to try to account for the ISR overhead.  Is that correct?  Have you done something similar (is that the "- 5" in the first formulas)?

[Reply #3 on:]

(is that the "- 5" in the first formulas)

rxcenter is used to get the middle of the pulse, so when reading the bit stream you're not on a falling or rising edge.

What I actually did was place the existing table in Excel; Calculate the "ideal" table (clock/baud) and adjust the numbers for the differences (subtractions).
For the 8Mhz I got a negative value so I adjusted the formula with an additional max() [pragmatic].

[Reply #4 on:]

First test with Arduino 1.0 SoftwareSerial example code and a Serial LCD - 9600 & 19200 baud - it behaves identical.
Mind you: Both the original table version and the new formula version miss a byte once and a while when sending more than ~10 bytes
Adding a delay(1) in the code helps a bit but.....

OK, the changes in SoftwareSerial.cpp are

Code:

void SoftwareSerial::begin(long speed)
{
  _rx_delay_centering = _rx_delay_intrabit = _rx_delay_stopbit = _tx_delay = 0;

  // added lines
  long baud = speed;
  _rx_delay_stopbit = 16000000L/(7 * baud) - 2;
  _rx_delay_intrabit = _rx_delay_stopbit;
  _tx_delay = _rx_delay_stopbit - 4;
  _rx_delay_centering = _rx_delay_stopbit/2 - 5;
 
  //
  // for (unsigned i=0; i<sizeof(table)/sizeof(table[0]); ++i)
  // {
    // long baud = pgm_read_dword(&table[i].baud);
    // if (baud == speed)
    // {
      // _rx_delay_centering = pgm_read_word(&table[i].rx_delay_centering);
      // _rx_delay_intrabit = pgm_read_word(&table[i].rx_delay_intrabit);
      // _rx_delay_stopbit = pgm_read_word(&table[i].rx_delay_stopbit);
      // _tx_delay = pgm_read_word(&table[i].tx_delay);
      // break;
    // }
  // }

  // Set up RX interrupts, but only if we have a valid RX baud rate
  if (_rx_delay_stopbit)
  {
    if (digitalPinToPCICR(_receivePin))
    {
      *digitalPinToPCICR(_receivePin) |= _BV(digitalPinToPCICRbit(_receivePin));
      *digitalPinToPCMSK(_receivePin) |= _BV(digitalPinToPCMSKbit(_receivePin));
    }
    tunedDelay(_tx_delay); // if we were low this establishes the end
  }

#if _DEBUG
  pinMode(_DEBUG_PIN1, OUTPUT);
  pinMode(_DEBUG_PIN2, OUTPUT);
#endif

  listen();
}

The memory size of the 2 sketches:
TABLE BASED : 4484
FORMULA BASED: 4302
==> 182 bytes less

every entry in the DELAY_TABLE = 12 bytes x 12 entries = 144 bytes
so the code is also shorter => no search loop and progmem calls)

in fact as we calculate the baudrate we can also leave out the following lines from  SoftwareSerial::begin()
-   _rx_delay_centering = _rx_delay_intrabit = _rx_delay_stopbit = _tx_delay = 0;
- if (_rx_delay_stopbit)
- and remove the local variable baud too

difference is now 192 bytes less!

[Reply #5 on:]

Conclusion:

Not to bad result,
1) a smaller code base (192 bytes) and
2) freedom of baud rate selection.

The 2nd is quite useful as now one can adapt the baud rate when the Arduino has no 16.000.000 Hz Crystal but a resonator producing less or more cycles / second.

Pity the SoftwareSerial drops some chars once and a while (original version did that too)

Todo:
- check every baud rate (old & new)
- check this code on 1.0.3
- 20Mhz tables => formulas
- Due too?
- test on other Arduino's [MegaTeensy]
- ...

[Reply #6 on:]

first version 20 MHz formulas.

Code:

static const DELAY_TABLE PROGMEM table[] =
{
  //  baud    rxcenter    rxintra    rxstop  tx
  { 115200,   3,          21,        21,     18,     },
  { 57600,    20,         43,        43,     41,     },
  { 38400,    37,         73,        73,     70,     },
  { 31250,    45,         89,        89,     88,     },
  { 28800,    46,         98,        98,     95,     },
  { 19200,    71,         148,       148,    145,    },
  { 14400,    96,         197,       197,    194,    },
  { 9600,     146,        297,       297,    294,    },
  { 4800,     296,        595,       595,    592,    },
  { 2400,     592,        1189,      1189,   1186,   },
  { 1200,     1187,       2379,      2379,   2376,   },
  { 300,      4759,       9523,      9523,   9520,   },
};

formulas

Code:

int rxstop = 20000000L/(7 * baudrate) - 1;
int rxintra = rxstop;
int tx = rxstop - 3;
int rxcenter = rxstop/2 - 4;

115200 7 23 23 20
57600 20 48 48 45
38400 32 73 73 70
31250 41 90 90 87
28800 45 98 98 95
19200 69 147 147 144
14400 94 197 197 194
9600 144 296 296 293
4800 293 594 594 591
2400 590 1189 1189 1186
1200 1185 2379 2379 2376
300 4757 9522 9522 9519

The difference 'matrix'

Code:

4 2 2 2
0 5 5 4
5 0 0 0
4 1 1 1
1 0 0 0
2 1 1 1
2 0 0 0
2 1 1 1
3 1 1 1
2 0 0 0
2 0 0 0
2 1 1 1

definitely worse than the 16 and 8 Mhz difference matrices, but it's a start

[Reply #7 on:]

Still wondering why there is a rxintra and a rxstop column. The values are identical....

[Reply #8 on:]

@robtillaart,

Noob question, but what's the reason for these values in SoftwareSerial? 

Obviously, less memory usage is a good thing.

[Reply #9 on:]

submitted as proposal - https://github.com/arduino/Arduino/issues/1198 -

[Reply #10 on:]

@Marc G

A serial protocol is like a train with wagons and on each wagon there is one bit, 10 bits in total (including start/stop bits). The baud rate represents the speed of the train.

The software serial receiving code is triggered by the edge of the start bit (train). To read a bit properly one wants to read the value of the signal (HIGH/LOW) in the middle of the bit, not at the edges.

rxCenter is the time to the (approx) middle of the first (start) bit,
rxIntra, rxStop are used as timings from the middle of one bit to the middle of the next. The higher the baud rate the lower these numbers.

tx is used for the timing for the transmit.

You can see this in the code of the library - C:\Program Files (x86)\arduino-1.0\libraries\SoftwareSerial - (windows)
and search for this function - void SoftwareSerial::recv() -

[Reply #11 on:]

Did a more extensive test with the SoftwareSerial using the formula approach.
I connected two Arduino's - UNO, 16Mhz (resonator) + 2009, 16Mhz (crystal) +  IDE 1.0 - both using SoftwareSerial, one Master and the other Slave (essentially an echo).

The master sent byte 0x55 at baudrate 100 and waits until the slave echos it back.
If the answer is not 0x55, the test fails and master prints a message.
Otherwise it just increases the baud rate with 100 and starts over.

The results are pretty good as it only gets constantly distorted above 190K baud.
Between 90K and 190K it only failed 10 times.

I took 0x55 as test pattern 0x55 == 01010101 ; it helps to see what happened. (see comments after output

Typical output  (multiple runs had comparable output)
Note: started with baud rate 100 in steps of 100...

Code:

start...
BAUD BYTE
97600 F5 FAIL  // = 11110101    ??
111200 AA FAIL  // = 10101010   1 bit shifted
114400 D5 FAIL  // = 11010101   1 bit failed (interference with start bit ?
124600 AA FAIL
140600 D5 FAIL
145500 AA FAIL
149000 D5 FAIL
163200 AF FAIL  // = 10101111  ??
190500 FF FAIL  // = 11111111  expect sync lost
190600 FF FAIL
190700 FF FAIL
190800 FF FAIL
190900 FF FAIL
191000 FF FAIL
...

The master and the slave were kept in sync by starting at the same baud rate an wait for each other.

To repeat the test start the master, then start the slave, and press a char in the serial monitor of the master.


Slave program (essentially echo)

Code:

//
//    FILE: serialSlave (echo)
//  AUTHOR: Rob Tillaart
//    DATE: 2013-01-02
//
// PUPROSE: test SW serial with formulas
//

#include <SoftwareSerial.h>

SoftwareSerial mySerial(2, 3);

void setup()
{
  Serial.begin(9600);
  Serial.println("start slave...");
}

unsigned long baud=0;

void loop()
{
  baud += 100;
  mySerial.begin(baud);
  while (mySerial.available() == 0);
  int b = mySerial.read();
  mySerial.write(b);
  Serial.println(b,DEC);
  delay(10);
}

master program

Code:

//
//    FILE: serialMaster
//  AUTHOR: Rob Tillaart
//    DATE: 2013-01-02
//
// PUPROSE: test SW serial
//

#include <SoftwareSerial.h>

SoftwareSerial mySerial(2, 3);

void setup()
{
  Serial.begin(9600);
  Serial.println("start...");
}

unsigned long baud=0;

void loop()
{
  if (Serial.available() > 0)
  {
    Serial.flush();

    baud += 100;
    mySerial.begin(baud);
    mySerial.write(0x55);
    while (mySerial.available() == 0);
    int b = mySerial.read();
    if (b != 0x55)
    {
      Serial.print(baud);
      Serial.print("\t");
      Serial.print(b, HEX);
      Serial.print("\t");
      Serial.println(" FAIL");
    }
    delay(20);
  }
}

As always comments/remarks are welcome

[Reply #12 on:]

same test with stepsize 10 gave some more errors (typical run started with baud rate 10, step 10)

Code:

start...
BAUD BYTE
70660 D5 FAIL
81950 AD FAIL
88870 AF FAIL
89570 BD FAIL
94410 D5 FAIL
95340 AA FAIL
96590 D5 FAIL
98980 AA FAIL
100750 AB FAIL
103590 BD FAIL
105740 D5 FAIL
110600 AA FAIL
113260 AF FAIL
120200 AA FAIL

...

Up till 70K no failures  ( that are 7000 different baudrates tested !)
between 70K and 115K "only" 13 failures   (13 fail on 4500 baudrates tested ~~ 1/300
above 120K the failures increased, not shown

Conclusion from the tests, SoftwareSerial "by formula" works very good up to 70.000 and reasonable well up to 115.200
Tweaking the formulas further may improve the test results but for now I'm quite satisfied.

This SoftwareSerial "by formula" allows one to build a communication channel in which the baud rate is constantly altered, making it very difficult to eavesdrop - and yes to get in sync

[Reply #13 on:]

@Rob,

I applaud this effort.  Thanks for investigating so thoroughly.  I always thought it might be fun to develop some equations that allow the synthesis of the "table" values on the fly, and now it looks like you are pretty close to doing just that.

It's good that you are getting error-free transmission up to 70K.  Make sure you test not just the single byte round trip, but also lots of bursts.  The values should vary Example:

1. Arduino sends 0x55 as fast as possible to host for one minute.
2. Arduino sends 0xFE as fast as possible to host for one minute.
3. Arduino sends 0x01 as fast as possible to host for one minute.
4. Host sends 0x55 as fast as possible to Arduino for one minute.
5. Host sends 0xFE as fast as possible to Arduino for one minute.
6. Host sends 0x01 as fast as possible to Arduino for one minute.

When constructing the tables, I found several times that I thought the values were good--until I tested the large bursts.

If we want to improve performance at baud rates > 57.6K, I think we're going to have to optimize the timer tick vector.  I studied this for some time with the logic analyzer and discovered that the occasional glitch was due to a timer tick interrupt being processed exactly when a pin change was pending.

Lastly, and you probably already know this, but if your formula is off a bit for the lower baud rates, it shouldn't be a big deal.  They are very tolerant.

Nice!

Mikal

[Reply #14 on:]

robtillaart

like the idea of a formula,
    but could be slow to change baud rate could it not.

not certain, and for interest,
   did you try two boards connected using the standard software serial code,
      did you try two boards using the standard hardware uart.