NilSdLogger

Hello all, I'm hoping to understand the NilSdLogger example a bit better. I've not encountered code before where the reading of the analog sensors occurs in the initialisation section of the code (before the void setup() or void loop()). Would be very grateful if somebody could outline how this works - I thought that void microcontroller loops only in void loop() so don't really understand how the Read ADCs section works - is it reading the ADCs continously? The section of code I am thinking about is posted below, the relevant section is "// Read ADCs"

// Simple ADC data logger that demonstrates NilAnalog, NilFIFO, and NilTimer1.
#include <SdFat.h>
#include <NilRTOS.h>
#include <NilAnalog.h>
#include <NilFIFO.h>
#include <NilTimer1.h>

// Use tiny unbuffered NilRTOS NilSerial library.
#include <NilSerial.h>

// Macro to redefine Serial as NilSerial to save RAM.
// Remove definition to use standard Arduino Serial.
#define Serial NilSerial
//------------------------------------------------------------------------------
// Time between points in microseconds.
// Maximum value is 4,194,304 (2^22) usec.
const uint32_t PERIOD_USEC = 1000;

// Number of ADC channels to log
const uint8_t NADC = 2;

// FIFO buffer size. Adjust so unused idle thread stack is about 100 bytes.
const size_t FIFO_SIZE_BYTES = 950;

// Type for a data record.
// Warning: High 6-bits of adc[0] are used for the overrun count.
//          If you use a higher precision sensor, add a separate overrun field.
struct Record_t {
  uint16_t adc[NADC];
};

// Number of data records in the FIFO.
const size_t FIFO_DIM = FIFO_SIZE_BYTES/sizeof(Record_t);

// Declare FIFO with overrun and minimum free space statistics.
NilStatsFIFO<Record_t, FIFO_DIM> fifo;
//------------------------------------------------------------------------------
// SD card chip select pin.
const uint8_t sdChipSelect = SS;

// SD file system.
SdFat sd;

// Log file.
SdFile file;
//------------------------------------------------------------------------------
// Declare a stack with 16 bytes beyond context switch and interrupt needs.
// The highest priority thread requires less stack than other threads.
NIL_WORKING_AREA(waThread1, 16);

// Declare thread function for thread 1.
NIL_THREAD(Thread1, arg) {

  // Check for very fast reads.
  if (NADC*160 > PERIOD_USEC) {
    // Set ADC prescaler to 64 for very fast reads.
    nilAnalogPrescalar(ADC_PS_64);
  }
  // Initialize ADC by doing first read.
  nilAnalogRead(0);

  // Start timer 1 with a period of PERIOD_USEC.
  nilTimer1Start(PERIOD_USEC);

  // Record data until serial input is available.
  while (!Serial.available()) {

    // Sleep until it's time for next data point.
    if (!nilTimer1Wait()) {
      // Took too long so count an overrun.
      fifo.countOverrun();
      continue;
    }

    // Get a free buffer.
    Record_t* p = fifo.waitFree(TIME_IMMEDIATE);

    // Skip the point if no buffer is available , fifo will count overrun.
    if (!p) continue;

    // Read ADCs
    for (int i = 0; i < NADC; i++) {
      p->adc[i] = nilAnalogRead(i);
    }

    // Save count of overruns since last point in high 6-bits of adc[0].
    uint16_t tmp = fifo.overrunCount();
    p->adc[0] |= tmp > 63 ? (63 << 10) : tmp << 10;

    // Signal SD write thread that new data is ready.
    fifo.signalData();
  }
  // Done, sleep forever.
  nilThdSleep(TIME_INFINITE);
}
//------------------------------------------------------------------------------
/*
 * Threads static table, one entry per thread.  A thread's priority is
 * determined by its position in the table with highest priority first.
 *
 * These threads start with a null argument.  A thread's name is also
 * null to save RAM since the name is currently not used.
 */
NIL_THREADS_TABLE_BEGIN()
NIL_THREADS_TABLE_ENTRY(NULL, Thread1, NULL, waThread1, sizeof(waThread1))
NIL_THREADS_TABLE_END()

So I think I understand a bit more, it is multithreading with the void loop() being the idle thread. In thread 1 (before void setup()) the data is recorded and stored to a buffer?

I did it with two tasks: "The Producer" and "The Consumer" (the loop() is the idle one).
An example:

..
SEMAPHORE_DECL(sdlogfifo, 0);  //  this is the counting semaphore, it starts/stops the sampling
..
/*
 * Thread "SdLogFifo" - The Producer, writes ie. ADC data into the FIFO
 */
// Declare a stack with 128 bytes beyond context switch and interrupt needs
NIL_WORKING_AREA(waSdLogFifo, 196);

// Declare the thread function
NIL_THREAD(SdLogFifo, arg) {

	time_t tnow;
	unsigned int msnow;

	while (TRUE) {

cont:	
		nilSemWait(&sdlogfifo);  // this will start the logging, with a counting semaphore N samples to process

		wakeTime += LOG_SAMPLE_PERIOD; // this sets the sampling period

		// Sleep until time for next data point
		nilThdSleepUntil(wakeTime);

		// Get a free FIFO slot. TIME_IMMEDIATE
		FifoItem_t* p = fifo.waitFree(TIME_IMMEDIATE);

		// Do nothing if no free space.
		if (p == 0) {  
			Serial.println("###########");
			goto cont; 
		}

		// Store the data items into the FIFO (an example).
		p->tnow = now();
		p->msnow = millis() % 1000;		
		p->ads = ads[4]*6.25019076e-5/8.0;

		// Write ADC data
		for (int i = 0; i < NUM_ADC; i++) {
			p->value[i] = adc[i];
		}

		// Signal to sdlog thread the data is available.
		fifo.signalData();
	}
}

/*
 * Thread "SdLog" - The Consumer, writes data from the FIFO into the Sdcard's CSV file
 */
// Declare a stack with 128 bytes beyond context switch and interrupt needs
NIL_WORKING_AREA(waSdLog, 196);

// Declare the thread function
NIL_THREAD(SdLog, arg) {
	uint32_t usecs;
	while (TRUE) {

		// Check for data.  Use TIME_IMMEDIATE to prevent sleep in idle thread.
cont:	
		FifoItem_t* p = fifo.waitData(50);

		// do nothing if no input data
		if (p == 0) { 
			goto cont;
		}

		// Store data items from the FIFO onto the SDcard
		usecs = micros();
		file.print(",");		
		file.printField(p->tnow, ',');
		//file.print(",");
		file.printField(p->msnow, ',');
		//file.print(",");
		file.printField(p->ads, ',', 3);
		//file.print(",");

		// Write ADC data
		for (int i = 0; i < NUM_ADC; i++) {
			file.printField(p->value[i], ',');
			//file.print(",");
		}
		usecs = micros() - usecs;
		file.printField(usecs, '\n');

		// Signal the FIFO slot is free.
	    fifo.signalFree();		
	}
}
..
NIL_THREADS_TABLE_ENTRY("SdLogFifo", SdLogFifo, NULL, waSdLogFifo, sizeof(waSdLogFifo))
NIL_THREADS_TABLE_ENTRY("SdLog", SdLog, NULL, waSdLog, sizeof(waSdLog))
..

The "sdlogfifo" is a "counting semaphore" - you may set it to the number of samples you want to record, ie. 100.000.000 (I changed its type to signed int32, afaik):

nilSemReset(&sdlogfifo, N);

Within the idle loop() you may implement for example a small command line interpreter, which will start/stop your various tasks (via the semaphores).

PS: I had many tasks running in my app, for example this is a task which reads several ADC channels and does an exp. smoothing as well. It runs "full speed" in the background, the "SdLogFifo" thread reads its output (stored in the adc[] array):

/*
 * Thread "ScanAins", scans analog inputs
 */
// Declare a stack with 128 bytes beyond context switch and interrupt needs
NIL_WORKING_AREA(waScanAins, 32);

// Declare the thread function
NIL_THREAD(ScanAins, arg) {

	// Time for next read.
	systime_t wakeTime = nilTimeNow();	

	while (TRUE) {

		wakeTime += ADC_SAMPLE_PERIOD;

		// Sleep until time for next data point
		nilThdSleepUntil(wakeTime);

		// Read ADC data
		for (int i = 0; i < NUM_ADC; i++) {
			// nilAnalogRead() sleeps during ADC conversion.
			adc[i] = 0.02*nilAnalogRead(i) + 0.98*adc[i];
		}
	}
}
..
NIL_THREADS_TABLE_ENTRY("PrScanAins", PrScanAins, NULL, waPrScanAins, sizeof(waPrScanAins))
..

This is a task which reads I2C ADC in background, and the "SdLogFifo" thread reads its output (stored in the ads[] array). Mind the "i2cfree" semaphore is a "mutex", as I used single I2C bus for several I2C devices called from various tasks.

..
SEMAPHORE_DECL(i2cfree, 1);  // Mutex semaphore MUST be initialized to 1
..
/*
 * Thread "ScanADS1", scans analog inputs
 */
// Declare a stack with 128 bytes beyond context switch and interrupt needs
NIL_WORKING_AREA(waScanADS1, 32);

// Declare the thread function
NIL_THREAD(ScanADS1, arg) {

	nilSemWait(&i2cfree);
	conf_ads1110(ads1110_4, 0x8F);
	nilSemSignal(&i2cfree);

	ads[4] = 0.0;

	// Time for next read.
	systime_t wakeTime = nilTimeNow();	

	while (TRUE) {

		wakeTime += ADS1_SAMPLE_PERIOD;

		// Sleep until time for next data point
		nilThdSleepUntil(wakeTime);

		// Read ADC data	
		nilSemWait(&i2cfree);
		ads[4] = read_ads1110(ads1110_4);
		nilSemSignal(&i2cfree);
		
	}
}
..
NIL_THREADS_TABLE_ENTRY("ScanADS1", ScanADS1, NULL, waScanADS1, sizeof(waScanADS1))
..

The above XY_SAMPLE_PERIODs were in a msecs resolution , as the "nilTimeNow()" had 1 msec resolution. For example:

LOG_SAMPLE_PERIOD = 50
ADC_SAMPLE_PERIOD = 5
ADS1_SAMPLE_PERIOD = 100

does log the data 20x per second to the sdcard, while ADC data were averaged each 5msecs, and the I2C ADS was read 10x per second.

The beauty of the (nil)rtos is you may simply add as many "independent" tasks as you "need" (I did 14 with nilrtos for example) where some of them run always in background, while the others are started/stopped from a CLI ran in the idle (in the loop()). From within the CLI you just set/reset the semaphores (thus start/stop the tasks) or the global parameters..
Have a nice rtossing :slight_smile:

Thanks Pito! That is really helpful - I will work through that.