Frequent timer interrupts and reading Serial data

Projects Programming
1

I'd like to be able to generate a variable waveform based on data I send from my computer over serial. My basic thinking is that I have a Python script that will generate the wave magnitudes then send a byte of data at a time over serial to the Arduino. On the Arduino end of things, I'd read from serial then do direct port manipulation. That part works. The two problems I have are that I need very precise timing and I need to send more data than the Arduino's memory can hold. I think this is solvable by using a queue that will buffer data from my computer until the Arduino's RAM is full, then have a timer routine that pops an item out of the queue and writes it out to the ports. When there is sufficiently free memory, I can send more bytes. I figured out timers and I've verified with an oscilloscope that my timer works. I'd like to update at 16kHz, which I think should be doable if I choose a sufficiently high baud rate. I found a FIFO library (Arduino Playground - QueueArray Library) which seems to work.

The problem I'm having is that when I set my timer to interrupt very frequently (like at every 1/16000th of a second), nothing ever gets written into the array. Setting the timer to interrupt a lot less frequently fixes the issue, so I know it must be something that the interrupt is doing. I guess that makes sense, but is there any way around this? I've posted my code below, thanks!

#include <QueueArray.h>

QueueArray <byte> queue;
int statusCode;

void setup() {
    
  Serial.begin(115200);
  Serial.flush();

  pinMode(30, OUTPUT);
  pinMode(31, OUTPUT);
  pinMode(32, OUTPUT);
  pinMode(33, OUTPUT);
  pinMode(34, OUTPUT);
  pinMode(35, OUTPUT);
  pinMode(36, OUTPUT);
  pinMode(37, OUTPUT);


  noInterrupts();
  // initialize timer 5
  TCCR5A = 0; // clear register A
  TCCR5B = 0; // clear register B
  TCNT5 = 0; // clear the timer counter register

  OCR5A = 1000; // compare match register at 1000 (16MHz/16kHz)
  TCCR5B |= (1 << WGM12); // clear timer on compare match
  TCCR5B |= (1 << CS10); // no prescaler
  TIMSK5 |= (1 << OCIE5A); // enable timer compare interrupt
  interrupts();
}

ISR(TIMER5_COMPA_vect) {  // timer compare interrupt service routine
  if(!queue.isEmpty()) {
    PORTC = queue.pop();
  }
}

void loop() {
  
  if (Serial.available() > 0) {
    if(!queue.isFull()) {
      queue.push(Serial.read()); 
      statusCode = 1; // 1 = success
    }
    else {
     statusCode = 2; // 2 = resend
    }
    Serial.println(statusCode);
  }
}
2

I haven't studied your program, but the way I handle sending "too much" data from pc to Arduino is

void loop() {
  if ( Serial.available()>10 && !XOFFState ) { Serial.write(XOFF) ; XOFFState = true ; }
  if ( Serial.available()<5 && XOFFState) { Serial.write(XON) ; XOFFState = false ; }
:
otherwise handle input and processing as before

The sending software must of course also read what the Arduino sends (here as Processing code)

void draw() {
  if ( PortSMP.available()>0 ) {
    char C = char(PortSMP.read()) ;
    if ( C == XOFF ) XFLOW = false ;
    else  if ( C == XON ) XFLOW = true ; }
:
and then cease to send data if XFLOW is false

The characters XON XOFF can be any charcter that is unique in what you send from the Ardino, but theye are usually ^S and ^Q.

I have found by experimentation that it fails if one first triggers XOFF at - say - 50 characters in the buffer (there is room for 64). The delay of the character oing from the Arduino via the USB and through the whole of Windows/Linux OS and to the sending program takes some time. You may need to tweak the start/stop levels to fit your app.

3

You haven't protected the queue from interrupts in the middle of queue operations. in general, code has to be written very carefully if it uses data structures that are also manipulated at interrupt level, and I doubt that the queueArray code you are using was written that way. (The alternative is just to enclose the manipulation inside interrupts()/noInterrupts() calls.)

Also, it looks as though the queueArray implementation automatically grows the storage used for the queue (at substantial expense!), but since you avoid calling it when the queue is full, I don't think it will ever grow beyond its initial size of 2.

Finally, the Serial code already implements a very similar datastructure, so you're essentially pulling things off of one queue and putting them onto another queue, which is less efficient than it could be. You might be better off figuring out how to write modified Serial code so that your timer interrupt can pull things directly off of its queue.

4

Yeah, I was thinking of writing my own queue. All I need is a circular array of some fixed length. How can I make sure my code is interrupt safe? Thanks.

5

you can define an array of bytes that is volatile. array size depends on what you want the buffer size to be.
you need a head and tail pointer.
the push will add the new value to [head]
the pop will get the value from [tail]
if head==tail, there is no value
make sure head does not overrun tail
increment head/tail after push/pop access, then % to queue size.
wrap the code in loop that push to buffer between cli() and sei()

6

So I've written my own circular buffer library. Now I only need to get flow control going. Running with Msquare's idea, I modified my loop() like so.

void sendXOFF() {
	Serial.print(19); // send XOFF
	XOFFSTATE = true;
}

void sendXON() {
	Serial.print(17);
	XOFFSTATE = false;
}

void loop() {

	int availableBytes = Serial.available();

	if (availableBytes) {
		if ((availableBytes > 50 || q->size > 4000) && !XOFFSTATE) {
			sendXOFF();
		}
		else if((availableBytes < 30 || q->size < 500) && XOFFSTATE){
			sendXON();
		}
		byte b = Serial.read();
		if(q->size != q->memorySize) {
			cli();
			enqueue(q, b);
			sei(); 
		}
	}
}

I'm fairly sure that I'm sending the XON and XOFFs because if I manually flood my Ardunio I can see the XOFF or XONs come back. However, I'm having trouble getting this to work with pySerial. I know that the it takes a parameter xonxoff which should get the library to start doing flow control but I don't think this is working since the program finishes way before it should.

Does anyone have any experience with flow control and Python, Java, C, Objective-C or anything else?