Determinate Serial Write Time (Buffer?)

In my project, I need to regularly write to my computer's serial port from the arduino, and I need to do that at a constant frequency. However, when I am doing this in a loop, there is about a +/- .01 second variance of how long a constant-length serial message takes to arrive at my terminal. Is there any way to reduce this variance so that it is more of a constant rate? Is there any more information on this subject? Is this some kind of buffering issue? Can this be taken care of? Thanks in advance.

Are you using a real rs-232 serial port (with 9 pins) or a USB port?

I'm using an off-chip FTDI board, so USB.

As I undertsand USB it packetises data and sends when it gets a chance (I may be wrong here). If so maybe you can't get a deterministic result.

Maybe tell us why you need such accuracy. Can you include a time stamp in the data so the PC code can align packets correctly.


Rob

Well, here's the reason for the obnoxious level of accuracy. We are doing some sensor analysis, and we want to do some frequency-domain analysis, so we need a) a constant rate of data output and b) we need to be able to accurately sync that data up with our computer (since some programs running on the computer need to be synced with this data, as well).

I considered doing some kind of a 'check-in' where ~100 or so data samples are saved, as well as a start and end timestamp on the microcontroller's end, and then just dump that entire set of data over serial. It's much messier than the other options, but unless I can fix the USB issues, I can't think of any other way to get it relatively synced up. Any recommendations are very welcome, and thanks so far.

Do you need real time or just relative to say the start of sampling?

If relative then appending the value from millis() to each sample would do the trick. The resultant data could be analysed easily I would think.

NOTE: as I recently found out the mills() value can skip a number every now and then. What is the sampling freq, if it's not that fast this won't matter.

we need to be able to accurately sync that data up with our computer (since some programs running on the computer need to be synced with this data, as well).

I considered doing some kind of a 'check-in' where ~100 or so data samples are saved, as well as a start and end timestamp

It seems that you don't need to actually trigger in real time (probably not possible on a PC anyway), you just need to know when each sample was taken. So adding a timestamp to every sample would give nice atomic data that can be used anywhere in the knowledge that all programs are singing from the same hymn book.


Rob

Yeah, thanks for the tips regarding milli(); I will be sure to keep an eye out for it. That has always been my backup plan, I was just hoping that there was some way to fix the issues with the buffering situation, as it would be more accurate that way. I am still open to ideas on that, but if there are none, I will default for that. Thanks for the help so far.

Actually, now when I look at it again, storing solutions and then printing them doesn't work, either. This is due to the fact that while the device is writing to the serial port, the device isn't collecting any data. I forgot that the program stops at that line and doesn't just run in the background, so every 100 samples there would be a hiccup in the program, and would cease to be periodic. Are there any other ideas on how something like this could be accomplished?

while the device is writing to the serial port, the device isn't collecting any data

Only true if you try to serial.write faster than the characters can be transmitted. The first character (and I think the second because the UART is buffered) will be written immediately, therefore the code doesn't block

You can write the bytes into a circular buffer and organize a method of transmitting them at periods that don't overun the UART hardware.

Also, I think NewSoftSerial is interrupt driven, if so then feeding character to that won't adversly affect your other functions.

Two questions then,

What is the frequency of the data collection? How many bytes per data item? How long does the collection have to run?

Actually that's three quesitons.


Rob

  1. The frequency is about in the 100Hz range
  2. I haven’t calculated the actual bytes, but it’s something on the order of below: (12ish Bytes)
  3. The collection has to run ~indefinitely (Over 30 minutes)

Below is the essence of my code, if it helps to show the problem. The program stalls for an indeterminate amount of time for every Serial.print. Since that’s the case, even if I store values and only write them every X number of samples, those samples are going to have gaps in them when the program is stuck on the serial print (since Serial.print takes so much longer than one sample iteration). The issue is that I’m going to be collecting data as fast as is possible.

long sensorVal = 0;
void setup(){
  Serial.begin(9600);
}

void loop(){
    sensorVal = analogRead(0);
    Serial.print("Sensor1\t");
    Serial.println(sensorVal);
}

Inline comments for improving the on-wire performance…

long sensorVal = 0; // long? Why? analogRead returns an int.

void setup(){
Serial.begin(9600); // 9600? Why? Does 115200 not work?
}

void loop(){
sensorVal = analogRead(0);
Serial.print(“Sensor1\t”); // Why output a constant string? You know data from this Arduino is always going to be “Sensor1”.
Serial.println(sensorVal);
}

Essentially, my problem isn't the performance, it's the unknown amount of time that the serial print takes, even if it's a constant string. My actual program is longer than this (hence the weird sensor data types and printouts), and even the modifications would make it go faster, it still wouldn't use a constant amount of time for each serial transmission.

115200 has some weird jumpiness in the serial transmissions compared to 9600. Faster, but not constant.

Let’s assume a 4-byte timestamp per sample.

12 data + 4 timestamp * 100Hz sample rate = 1600 bytes per second * 10 bits = 16000 kbps data rate.

So in theory the serial port can easily keep up. Some properly written code transmitting 1600 bytes a second should be able to do so with little or no affect on the data sampling unless such sampling itself takes most of the 10mS between samples.

Drop any human-friendly stuff like printing strings, just send the time and data.

void loop(){
    sensorVal = analogRead(0);
    Serial.print(millis());
    Serial.print(sensorVal); // so far a max of 6 bytes, even better than 16
}

constant amount of time for each serial transmission

Doesn’t matter if there is a timestamp with each sample. All that matters is that the samples are taken at the correct time, once this data is in the system you can transmit it next week with no problems :slight_smile:

However you say that there are multiple input channels, how many, that makes a big difference (let’s assume 10 for the moment).

If you packetise each group of samples the formula for the data rate is

((data size * n channels) + 4 ) * 100 * 10 = serial bit rate

or ((2 * 10) + 4) * 100 * 10 or 24000bps.

well within the range of the serial port. So all you have to do is write code that doesn’t just blat these bytes out all in one go because that will block after the second byte.

THINKS: Even if you just blat the bytes out, it’s only 24 bytes every 10mS, that’s 240 bits at about 9uS (115200bps, that should work) per bit or just over 2mS for the lot. So for 10 channels I think you can just blat the bytes out.

byte pins[] = {1,2,3, etc};
int data [10];
for (i = 0; i < 10; 1++) 
   data [i] = analogueRead (pins[i]);
// we now have the data safe in an array 
Serial.print (millis());
for (i = 0; i < 10; 1++) 
  Serial.print (data[i])

So I’m sure it will work (hopefully I haven’t got my decimal points in the wrong place :))


Rob