Issues with interrupts and ChibiOS

Hi,

I have struck the wall with my Due and ChibiOS when using interrupt handlers. This might be the wrong forum, maybe some ChibiOS forum is more appropriate - but here it goes.

My application is to read a PPM stream from a RC receiver. The PPM decoder I have implemented works great if using the standard Arduino IDE, but due to the complexity of my system I need to emigrate to a real-time system. Anyway, I initialise my PPM decoder by instansiate my PPM class and call ‘enable’ (prior to this I set pin 2 to input):

PPM::PPM(uint8_t channels, uint16_t minpulse, uint16_t maxpulse) {

	this->channels = channels;
	
	//pmc_enable_periph_clk(TC_INTERFACE_ID + PPM_TC_IF_ID);
	pmc_enable_periph_clk(PPM_PMC_ID);
		
	TC_Configure(ppm_tc, PPM_TC_CHANNEL,
				 TC_CMR_TCCLKS_TIMER_CLOCK1 |  // MCK/2
				 TC_CMR_WAVSEL_UP |			   // Counter running up and reset when overflow
				 TC_CMR_ABETRG |
				 TC_CMR_LDRA_RISING |				
				 TC_CMR_LDRB_RISING);

	TcChannel* pTcCh;
	pTcCh = ppm_tc->TC_CHANNEL + PPM_TC_CHANNEL;
	
	// Interrupt on loading RB
	pTcCh->TC_IER = TC_IER_LDRBS;
	
	// Set default maximum and minimum pulse lengths
	for (uint8_t i = 0; i < channels; i++) {
	
		this->maxpulse[i] = maxpulse;
		this->minpulse[i] = minpulse;
	}
}

void PPM::enable() {

	TC_Start(ppm_tc, PPM_TC_CHANNEL);
	
	NVIC_EnableIRQ(PPM_TC_IRQ);
	
	error_counter = 0;
	
	state = PPM_STATE_UNSYNCED;
}

The code above works great without ChibiOS.

Now to the handlers…

My original “Arduino” handler looks like this:

void TC0_Handler() {

 	TC_GetStatus(ppm_tc, PPM_TC_CHANNEL);
 
 	ppm_flank_buf[ppm_flank_buf_pt] = ppm_tc->TC_CHANNEL[PPM_TC_CHANNEL].TC_RA;
 		
 	ppm_flank_buf_pt = (ppm_flank_buf_pt+1) % PPM_FLANK_BUFFER_LEN;
 		
 	ppm_flank_buf[ppm_flank_buf_pt] = ppm_tc->TC_CHANNEL[PPM_TC_CHANNEL].TC_RB;
 		
 	ppm_flank_buf_pt = (ppm_flank_buf_pt+1) % PPM_FLANK_BUFFER_LEN;
 				
 	return;
 }

As as I said, it works (the actual decoding function is not posted, just the flank collector).

And the one I use for ChibiOS looks like this:

CH_IRQ_HANDLER(pfnTC0_Handler) { // or CH_IRQ_HANDLER(SAM3XA_TC0_HANDLER) ???

	CH_IRQ_PROLOGUE();
	  
	chSysLockFromIsr(); // <--- Don't know if needed
  
	TC_GetStatus(ppm_tc, PPM_TC_CHANNEL);
	
	chSysUnlockFromIsr();

	ppm_flank_buf[ppm_flank_buf_pt] = ppm_tc->TC_CHANNEL[PPM_TC_CHANNEL].TC_RA;

	ppm_flank_buf_pt = (ppm_flank_buf_pt+1) % PPM_FLANK_BUFFER_LEN;

	ppm_flank_buf[ppm_flank_buf_pt] = ppm_tc->TC_CHANNEL[PPM_TC_CHANNEL].TC_RB;

	ppm_flank_buf_pt = (ppm_flank_buf_pt+1) % PPM_FLANK_BUFFER_LEN;

  	CH_IRQ_EPILOGUE();
  	
  	return;
}

What happens is, as soon as I get rising flanks on pin 2, my Due hangs.

Any ideas?

Thx in advance.

The CH_IRQ_HANDLER() macro does nothing for arm.

#define CH_IRQ_HANDLER(id) PORT_IRQ_HANDLER(id)
#define PORT_IRQ_HANDLER(id) void id(void)

You should use this since TC0_Handler is the vector in hardware/arduino/sam/cores/arduino/cortex_handlers.c

CH_IRQ_HANDLER(TC0_Handler)

I suspect the default handler gets executed and the Due “halts”.

static void __halt() {
	// Halts
	while (1)
		;
}
void TC0_Handler        (void) __attribute__ ((weak, alias("__halt")));

You probably don’t need chSysLockFromIsr()/chSysUnlockFromIsr() since you don’t interact with ChibiOS.

CH_IRQ_PROLOGUE() does nothing in arm.

CH_IRQ_EPILOGUE() does a reschedule but you don’t call any ChibiOS functions that would cause a reschedule.

I don’t see a reason to put the ChibiOS stuff in your IRQ handler. You don’t do any thing that will cause a thread to execute.

The ChibiOS forum is a good place for questions about IRQ handlers.

Thank you for the reply.

Unfortunately I experience the same issue i.e. the Due stops execution. This with an empty ISR as well (just clearing by reading TC_RB).

I really don't get this. Maybe I'm just observing symptoms here. Could there be anything else causing this behaviour?

When do you enable the TC0 interrupt? It must be after ChibiOS has started.

I assume by empty ISR you mean this Arduino function with no ChibiOS calls.

void TC0_Handler() {

    TC_GetStatus(ppm_tc, PPM_TC_CHANNEL);

    ppm_flank_buf[ppm_flank_buf_pt] = ppm_tc->TC_CHANNEL[PPM_TC_CHANNEL].TC_RA;

    ppm_flank_buf_pt = (ppm_flank_buf_pt+1) % PPM_FLANK_BUFFER_LEN;

    ppm_flank_buf[ppm_flank_buf_pt] = ppm_tc->TC_CHANNEL[PPM_TC_CHANNEL].TC_RB;

    ppm_flank_buf_pt = (ppm_flank_buf_pt+1) % PPM_FLANK_BUFFER_LEN;

    return;
 }

Here is an example that uses TC0. Quick hack to demo TC0 handler.

It will enable TC0 then print count of interrupts once a second.

// Simple demo of three threads
// LED blink thread, print thread, and main thread
#include <ChibiOS_ARM.h>
// Redefine AVR Flash string macro as nop for ARM
#undef F
#define F(str) str

const uint8_t LED_PIN = 13;

volatile uint32_t count = 0;
//-------------------------------------------------------------------------------
CH_IRQ_HANDLER(TC0_Handler){
  // on ARM CH_IRQ_PROLOGUE is void
  CH_IRQ_PROLOGUE();
  TC_GetStatus(TC0, 0);
  count++;
   chSysLockFromIsr();
  /* Invocation of some I-Class system APIs, never preemptable.*/

  // signal handler task
//  chBSemSignalI(&isrSem);
  chSysUnlockFromIsr(); 
  // Perform rescheduling if required.
  CH_IRQ_EPILOGUE();
}
//-------------------------------------------------------------------------------
//  from another app
void startTimer(Tc *tc, uint32_t channel, IRQn_Type irq, uint32_t frequency) {
  pmc_set_writeprotect(false);
  pmc_enable_periph_clk((uint32_t)irq);
  TC_Configure(tc, channel, TC_CMR_WAVE | TC_CMR_WAVSEL_UP_RC | TC_CMR_TCCLKS_TIMER_CLOCK4);
  uint32_t rc = VARIANT_MCK / 128 / frequency; //128 because we selected TIMER_CLOCK4 above
  TC_SetRA(tc, channel, rc / 2); //50% high, 50% low
  TC_SetRC(tc, channel, rc);
  TC_Start(tc, channel);
  tc->TC_CHANNEL[channel].TC_IER = TC_IER_CPCS;
  tc->TC_CHANNEL[channel].TC_IDR = ~TC_IER_CPCS;
  NVIC_EnableIRQ(irq);
}
//------------------------------------------------------------------------------
// thread 1 - high priority for blinking LED
// 64 byte stack beyond task switch and interrupt needs
static WORKING_AREA(waThread1, 64);

static msg_t Thread1(void *arg) {
  pinMode(LED_PIN, OUTPUT);

  // Flash led every 200 ms.
  while (1) {
    // Turn LED on.
    digitalWrite(LED_PIN, HIGH);

    // Sleep for 50 milliseconds.
    chThdSleepMilliseconds(50);

    // Turn LED off.
    digitalWrite(LED_PIN, LOW);

    // Sleep for 150 milliseconds.
    chThdSleepMilliseconds(150);
  }
  return 0;
}
//------------------------------------------------------------------------------
// thread 2 - print TC0  count every second
// 100 byte stack beyond task switch and interrupt needs
static WORKING_AREA(waThread2, 100);

static msg_t Thread2(void *arg) {
  startTimer(TC0, 0, TC0_IRQn, 4);
  // print count every second
  while (1) {
    // Sleep for one second.
    chThdSleepMilliseconds(1000);

    // Print count for previous second.
    Serial.print("Count: ");
    Serial.print(count);

    // Print unused stack for threads.
    Serial.print(", Unused Stack: ");
    Serial.print(chUnusedStack(waThread1, sizeof(waThread1)));
    Serial.print(' ');
    Serial.print(chUnusedStack(waThread2, sizeof(waThread2)));
    Serial.print(' ');
    Serial.println(chUnusedHeapMain());
  }
}
//------------------------------------------------------------------------------
void setup() {
  Serial.begin(9600);
  // wait for USB Serial
  while (!Serial) {}

  // read any input
  delay(200);
  while (Serial.read() >= 0) {}

  chBegin(mainThread);
  // chBegin never returns, main thread continues with mainThread()
  while(1) {}
}
//------------------------------------------------------------------------------
// main thread runs at NORMALPRIO
void mainThread() {

  // start blink thread
  chThdCreateStatic(waThread1, sizeof(waThread1),
                          NORMALPRIO + 2, Thread1, NULL);

  // start print thread
  chThdCreateStatic(waThread2, sizeof(waThread2),
                          NORMALPRIO + 1, Thread2, NULL);
}
//------------------------------------------------------------------------------
void loop() {
 // not used
}

Here is some output:

ount: 1101, Unused Stack: 132 84 92580
Count: 1105, Unused Stack: 132 84 92580
Count: 1109, Unused Stack: 132 84 92580
Count: 1113, Unused Stack: 132 84 92580
Count: 1117, Unused Stack: 132 84 92580
Count: 1121, Unused Stack: 132 84 92580
Count: 1125, Unused Stack: 132 84 92580
Count: 1129, Unused Stack: 132 84 92580
Count: 1133, Unused Stack: 132 84 92580

Thank you for the example!

Been busy with work, but managed to get time testing what you said earlier to make sure I initiate the interrupt after ChibiOS has started.

I had my call to the timer setup prior the creation of threads, so I did test to put it after:

chBegin(mainThread);

Now the Due avoids the halting. However, instead I never catch the event. So I need to verify my hardware, I will poke around with my scope when I have the time (probably the weekend). And possibly try another input pin, just in case I manage to fry something.

Will of course post the result here.

Cheers!

I still don't know what adding the ChibiOS macros to your TC0 handler accomplishes. You have your own buffering and haven't added any synchronization to the TC0 handler so ChibiOS won't reschedule any threads.

Why are you trying to use ChibiOS?

No no. I removed the ChibiOS macros within the ISR.

The hardware looks fine i.e. I have not burned my input pin. So my problem is in the software domain as far as I can tell.

The reason for using ChibiOS is because I have a quite complex system. I’m building a quadcopter. Multiple control loops. Telemetry and telecommand services. Navigation etc.

So my plan of attack is to take your example (that obviously works), adapt it for catching PPM. And take it from there to see if I can figure out when and where stuff goes wrong. I will also take your advice on debugging.

An interesting observation..

I changed from using the chThdSleepUntil() to chThdSleepMilliseconds() - see below - then the Due would run for near 20-30 seconds, instead of halting after 4-5 seconds (or immediately). Is this just about luck (it seems repeatable), or could it actually be related to what I'm experiencing?

msg_t receiver_thread(void *arg) {

       receiver_setup(); // Calls ppm.enable();
 
       while(1) {

               systime_t time = chTimeNow();
 
               ppm.update();

               //chThdSleepUntil(time+MS2ST(RECEIVER_THREAD_DT));
               chThdSleepMilliseconds(RECEIVER_THREAD_DT);
       }
}

After this I started to think. Could there be parts in ChibiOS that is sensitive to interrupts? I made another observation, if I start the Due with my transmitter ON, then the likelihood of a halt is lower than if I leave the transmitter OFF. I think this could be related to when the transmitter is OFF the receiver sends out a lot of spurious flanks due to noise (old TX/RX RC system) causing a lot of interrupts. And by that increasing the likelihood that an interrupt will occur during some sensitive part?

When I have tested this, the thread above was the only thread besides from a "mainThread" with Serial.println("Alive!") and chThdSleepMilliseconds(1000) inside.

Could there be parts in ChibiOS that is sensitive to interrupts?

Not if you use ChibiOS correctly.

No no. I removed the ChibiOS macros within the ISR.

I suggest you scrap using ChibiOS. Your approach to using an RTOS just won’t work.

There is no point in trying to build a complex system with threads and depend on sleep for scheduling.

ChibiOS does a context switch on Due in about 1 µs. You only get this type performance if you use an RTOS properly.

You must design you system using these features to benefit from an RTOS in a complex system.

Counting semaphores
Binary semaphores
Mutexes (priority inheritance)
Condition variables
Event flags
Synchronous messages
Mailboxes (message queues)
I/O queues
Abstract streams and channels

You should try the ChibiOS forum. Lots of skilled embedded system programmers read that forum and could help.

I would scrap Arduino and use ChibiOS native for a project like yours. Unfortunately Due is not well supported for any RTOS.

Several RC projects use ChibiOS and Giovanni Di Sirio, the author of ChibiOS provided help on his forum.

Tau Labs recently converted from FreeRTOS to ChibiOS and seems happy with the results.

Link to article.

They did a lot of work. See the comparison.

I can't provide close to the help that you will get from the ChibiOS forum.

Thank you for the links I will look it up, plus the ChibiOS forum :)

I have been really happy with ChibiOS, my rate control thread runs smooth at 200 Hz, the angle control thread at 50 Hz. Telemetry threads at 5 Hz and 1 Hz. The receiver thread is supposed to be running at 50 Hz, and simply just decode the flanks I collect in the buffer with the interrupt handler. I need to use semaphores/mutexes for some shared resources, for an instance an I2C bus.

I have as well start looked at moving away from Arduino software. In matter of fact I only use the wire library (which is a mess..). I have replaced the serial library with my own DMA based library (https://github.com/rblilja/DmaSerial).

At the moment I'm setting up a Eclipse environment with the ARM toolchain.

Thanks again!