While getting the time from the RTC on the ESP32, there are gaps in milliseconds

Using Arduino Programming Questions
1

Hello,

I am using a timer interrupt on the ESP32 that triggers every 1ms. However, when I read the millisecond time from the RTC, there are gaps between the millisecond values. Instead of the values increasing sequentially (e.g., 0.001, 0.002, 0.003, etc.), they are jumping (e.g., 0.001, 0.004, 0.007, etc.).

How can I solve this issue? I am open to any ideas that might help simplify the solution.

Thank you very much.

#include <Arduino.h>
#include <esp32-hal-timer.h>
#include <sys/time.h>
#include <ESP32Time.h>

ESP32Time rtc(0);

volatile bool timer_flag = false;
hw_timer_t *My_timer = NULL; // Donanım zamanlayıcı oluşturuluyor
volatile unsigned long millisCounter = 0; // Milisaniye sayacı

void IRAM_ATTR onTimer() // Zamanlayıcı kesme fonksiyonu oluşturuluyor
{
  timer_flag = true;
  //millisCounter++; // Milisaniye sayacını her milisaniyede bir arttır
}

void setup() 
{
 

  Serial.begin(115200);
  My_timer = timerBegin(1, 80, true); // Hesaplanan prescaler ile zamanlayıcıyı başlat
  timerAttachInterrupt(My_timer, &onTimer, true);
  timerAlarmWrite(My_timer, 1000, true); // Alarm her 10000 tikte bir (1 milisaniye @ 100 kHz)
  timerAlarmEnable(My_timer); // Zamanlayıcı alarmını etkinleştir

 
 
}

void loop() 
{
  if (timer_flag) 
  {
    timer_flag = false;

    // RTC'den zaman bilgilerini al
    int currentSecond = rtc.getSecond(); // Saniye bilgisini al
    unsigned long currentMillis = rtc.getMicros()/100; 

    // Sadece gerekli bilgileri yazdır
    Serial.print("Saniye: ");
    Serial.print(currentSecond);
    Serial.print(", Milis: ");
    Serial.println(currentMillis); // Sadece milisaniye kısmını yazdır
  }
}
3

I imagine you're quickly filling up the Serial object's buffer causing the printing to revert to blocking mode to flush it out. On average, you can only transfer 11.52 characters (max) per millisecond at 115200 baud.

What are you trying to accomplish anyway?

4

I don't know whether the ESP32 is the same, but on a lot of Arduino's when millis() are calculated, they are actually longer than 1ms (1024Āµs).

In order for one second to be 1000ms, it has to skip a number approximately every 40ms.

5

I haven't got an ESP32, but using an Arduino Uno R3, the following code prints out all the millis() for the first second (approximately), plus the numbers that get skipped.

unsigned long currentMillis;
unsigned long previousMillis;

void setup() {
  Serial.begin(115200);
}

void loop() {
  currentMillis = millis();

  if (currentMillis > 1024) {                   // stop printing after ā‰ˆ1 second
    while (1);
  }

  if (currentMillis - previousMillis > 1) {     // prints numbers that get skipped
    Serial.print("Skipped: ");
    Serial.println(currentMillis - 1);
  }

  if (currentMillis > previousMillis) {         // prints numbers used by millis
    Serial.print(currentMillis);
    Serial.print(", ");
  }
  previousMillis = currentMillis;
}

And here are the results:
(scroll across to the end of the line to show which numbers get skipped)

1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, Skipped: 42
43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, Skipped: 85
86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, Skipped: 127
128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, Skipped: 170
171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, Skipped: 213
214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, Skipped: 255
256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, Skipped: 298
299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339, 340, Skipped: 341
342, 343, 344, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379, 380, 381, 382, Skipped: 383
384, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 425, Skipped: 426
427, 428, 429, 430, 431, 432, 433, 434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461, 462, 463, 464, 465, 466, 467, 468, Skipped: 469
470, 471, 472, 473, 474, 475, 476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503, 504, 505, 506, 507, 508, 509, 510, Skipped: 511
512, 513, 514, 515, 516, 517, 518, 519, 520, 521, 522, 523, 524, 525, 526, 527, 528, 529, 530, 531, 532, 533, 534, 535, 536, 537, 538, 539, 540, 541, 542, 543, 544, 545, 546, 547, 548, 549, 550, 551, 552, 553, Skipped: 554
555, 556, 557, 558, 559, 560, 561, 562, 563, 564, 565, 566, 567, 568, 569, 570, 571, 572, 573, 574, 575, 576, 577, 578, 579, 580, 581, 582, 583, 584, 585, 586, 587, 588, 589, 590, 591, 592, 593, 594, 595, 596, Skipped: 597
598, 599, 600, 601, 602, 603, 604, 605, 606, 607, 608, 609, 610, 611, 612, 613, 614, 615, 616, 617, 618, 619, 620, 621, 622, 623, 624, 625, 626, 627, 628, 629, 630, 631, 632, 633, 634, 635, 636, 637, 638, Skipped: 639
640, 641, 642, 643, 644, 645, 646, 647, 648, 649, 650, 651, 652, 653, 654, 655, 656, 657, 658, 659, 660, 661, 662, 663, 664, 665, 666, 667, 668, 669, 670, 671, 672, 673, 674, 675, 676, 677, 678, 679, 680, 681, Skipped: 682
683, 684, 685, 686, 687, 688, 689, 690, 691, 692, 693, 694, 695, 696, 697, 698, 699, 700, 701, 702, 703, 704, 705, 706, 707, 708, 709, 710, 711, 712, 713, 714, 715, 716, 717, 718, 719, 720, 721, 722, 723, 724, Skipped: 725
726, 727, 728, 729, 730, 731, 732, 733, 734, 735, 736, 737, 738, 739, 740, 741, 742, 743, 744, 745, 746, 747, 748, 749, 750, 751, 752, 753, 754, 755, 756, 757, 758, 759, 760, 761, 762, 763, 764, 765, 766, Skipped: 767
768, 769, 770, 771, 772, 773, 774, 775, 776, 777, 778, 779, 780, 781, 782, 783, 784, 785, 786, 787, 788, 789, 790, 791, 792, 793, 794, 795, 796, 797, 798, 799, 800, 801, 802, 803, 804, 805, 806, 807, 808, 809, Skipped: 810
811, 812, 813, 814, 815, 816, 817, 818, 819, 820, 821, 822, 823, 824, 825, 826, 827, 828, 829, 830, 831, 832, 833, 834, 835, 836, 837, 838, 839, 840, 841, 842, 843, 844, 845, 846, 847, 848, 849, 850, 851, 852, Skipped: 853
854, 855, 856, 857, 858, 859, 860, 861, 862, 863, 864, 865, 866, 867, 868, 869, 870, 871, 872, 873, 874, 875, 876, 877, 878, 879, 880, 881, 882, 883, 884, 885, 886, 887, 888, 889, 890, 891, 892, 893, 894, Skipped: 895
896, 897, 898, 899, 900, 901, 902, 903, 904, 905, 906, 907, 908, 909, 910, 911, 912, 913, 914, 915, 916, 917, 918, 919, 920, 921, 922, 923, 924, 925, 926, 927, 928, 929, 930, 931, 932, 933, 934, 935, 936, 937, Skipped: 938
939, 940, 941, 942, 943, 944, 945, 946, 947, 948, 949, 950, 951, 952, 953, 954, 955, 956, 957, 958, 959, 960, 961, 962, 963, 964, 965, 966, 967, 968, 969, 970, 971, 972, 973, 974, 975, 976, 977, 978, 979, 980, Skipped: 981
982, 983, 984, 985, 986, 987, 988, 989, 990, 991, 992, 993, 994, 995, 996, 997, 998, 999, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008, 1009, 1010, 1011, 1012, 1013, 1014, 1015, 1016, 1017, 1018, 1019, 1020, 1021, 1022, Skipped: 1023
1024,

Does that fit in with what you are seeing?

6

Actually, what I wanted to do was this: I intended to read data from an RTC and write it to an SD card in my own project. However, I noticed gaps in milliseconds when I did this. To understand if this issue was due to the RTC, I tried the code above. When I tried it at 500000 baud rate, I observed a 10ms difference between milliseconds. How can I solve this issue? Because ultimately, milliseconds are not consistently increasing, which is affecting my project. I ultimately want to obtain consecutive milliseconds.
Thank you for your response.
Best regards..

7

Yes, that's exactly what I wanted to convey in my subject title. I'm encountering a similar outcome as yours.

8

What does that mean? As stated, it doesn't make much sense. An RTC doesn't have any interesting data unless it's time stamps associated with events. Be more specific about the project.

It's more likely that they are and that there's inconsistency with your sampling of them.

Finally, please specify which version of the ESP32 Arduino core you are using. There were significant changes in the APIs going from v2.x to v3.x.

9

There was a similar topic to this one at:
https://forum.arduino.cc/t/can-i-use-millis-to-increment-a-data-type-every-millisecond/1255869

10

Here is a code snippet related to the issue I'm facing in my project. What I'm doing in the project is reading data from an accelerometer and writing it to an SD card. I'm also capturing day, month, year, hour, minute, second, and millisecond data from an RTC and writing these timestamp data to the SD card as well. What I actually need is to be able to maintain accelerometer data correlated with time. Therefore, I need measurements corresponding to that specific time. However, due to gaps between millisecond data, my data isn't displayed sequentially. Is there a way to maintain millisecond data without these gaps? How should I proceed? Thank you..

  if((timer_flag==true) && (measurementEnable == 1))
  {  
    //Serial.println("sd karta yazma yeri");
    timer_flag=false;
    //cnt++;

    int16_t x_int,y_int,z_int;
    x_int = LIS.getAccelerationX();   //axis-x
    y_int = LIS.getAccelerationY();  //axis-y
    z_int = LIS.getAccelerationZ(); //axis-z
   
    
    SensorData sensorData;
    sensorData.data.x = static_cast<uint16_t> (x_int);
    sensorData.data.y = static_cast<uint16_t> (y_int);
    sensorData.data.z = static_cast<uint16_t> (z_int);
    sensorData.data.second = static_cast<int8_t>(rtc.getSecond());
    sensorData.data.ms_us_time = static_cast<int8_t>(rtc.getMillis());

    SDfile.write(sensorData.data.x%255);
    SDfile.write(sensorData.data.x/255);
    SDfile.write(sensorData.data.y%255);
    SDfile.write(sensorData.data.y/255);
    SDfile.write(sensorData.data.z%255);
    SDfile.write(sensorData.data.z/255);
    SDfile.write(sensorData.data.second);
    SDfile.write(sensorData.data.ms_us_time%255);
    SDfile.write(sensorData.data.ms_us_time/255);

    //Serial.println(sensorData.data.ms_us_time);
  }
11

I am using Arduino version 2.2.1

12

No, not the Arduino IDE version. We need to know what version of the ESP32 support package you're using. I don't use Arduino IDE v2.x. But in the Board Manager for Arduino IDE 1.x, the ESP32 BSP choices look like this:

image
image797Ɨ455 14.2 KB

You'll need to find the equivalent setting in IDE 2.x.

13

It's not the same in ESP32. Take a look at the source code for millis() on that platform.

14

It may be that your expectations are unrealistic.
How often do you want to sample the data? How long does it take to sample each set of data points? How long does it take to write the data to the SD Card after each set of samples is taken?

Also, do you actually need "day, month, year, ....." for each sample point? Can you work with just total time since data collection started (say in milliseconds or microseconds) and then post process it as required?

15

It shows 3.0.2 selected here. Which one should I choose?
Screenshot_1

16

That's fine as long as you use API calls consistent with that version. Like I said, there were significant changes going from v2.x to 3.x which corresponds with going from Espressif ESP-IDF v4.x to v5.x. Here's a Migration Guide.

1 Like
17

To establish that there are no ā€œgapsā€ in ESP32 timing, hereā€™s a code demonstrating microsecond-level consistency. It sets up a timer interrupt every 1 millisecond. During the ISR, the code it samples the ESP32ā€™s High Resolution Timer which runs from startup with microsecond resolution.

Rather than trying to print the results every millisecond, the code instead gathers statistics over an entire second:

  • Calculates the timer delta (in microseconds) between the current sample and the one that was taken during the previous 1-millisecond interrupt. This delta should nominally be 1000 microseconds.

  • Keeps track of the minimum delta value over the 1-second sampling period.

  • Keeps track of the maximum delta value over the 1-second sampling period.

  • Computes the average delta value over the 1-second sampling period.

The results are printed at the end of every 1-second sampling period. As seen in the first Serial output log, all delta values are consistently 1000 microseconds. This is not surprising as the High Resolution Timer being sampled is also responsible for dispatching the Timer interrupts.

By uncommenting the //#define USE_RTC in the ISR, the code will instead get the timestamps from the ESP32ā€™s RTC. In this case there is a slight variation in the delta values (1 or 2 microseconds). This is shown in the second Serial output log.

Iā€™d guess this small variation is due to the exact timing of when the RTC is updated verse when the High Resolution Timer dispatches the ISR to sample the RTC. However, this microsecond-level variation is three orders of magnitude smaller than what youā€™re reporting from your code. Thus, my conclusion is that any millisecond-level ā€œgapsā€ that you are seeing are due to sampling problems in your code.

BTW, the "ESP32Time.h" library that you used doesn't seem to add much real value. It just provides thin wrapper functions for the standard POSIX time functions natively supported by the ESP32. I prefer to use the latter.

#include "Arduino.h"
#include "driver/gptimer.h"

void samplingTask(void *params);
bool timerCallback(gptimer_handle_t timer, const gptimer_alarm_event_data_t *edata, void *user_data);

void setup() {
	Serial.begin(115200);
	delay(2000);
	Serial.println("Starting");
	auto result { xTaskCreatePinnedToCore(samplingTask, "Blinking", 2000, NULL, 5, NULL, ARDUINO_RUNNING_CORE) };
	assert((result == pdTRUE) && "Failed to Create Task");
}

void loop() {
	vTaskDelete(NULL);
}

struct SampleData {
	int32_t minDelta;
	int32_t maxDelta;
	int32_t averageDelta;
};

void samplingTask(void *params) {
	auto sampleDataQueue = xQueueCreate(5, sizeof(SampleData));
	assert(sampleDataQueue != NULL && "Failed to Create Queue");

	gptimer_handle_t gptimer { NULL };
	const gptimer_config_t timer_config = {
			.clk_src = GPTIMER_CLK_SRC_DEFAULT,
			.direction = GPTIMER_COUNT_UP,
			.resolution_hz = 1000000, // 1MHz, 1 tick=1us
			};
	assert((gptimer_new_timer(&timer_config, &gptimer)==ESP_OK) && "Failed to Create Timer");

	gptimer_event_callbacks_t cbs = {
			.on_alarm = timerCallback,
	};
	assert((gptimer_register_event_callbacks(gptimer, &cbs, static_cast<void*>(sampleDataQueue))==ESP_OK) && "Failed to Register Callback");
	assert((gptimer_enable(gptimer)==ESP_OK) && "Failed to Enable Timer");

	gptimer_alarm_config_t alarm_config = { 1000, 0, true };
	assert((gptimer_set_alarm_action(gptimer, &alarm_config)==ESP_OK) && "Failed to Set Alarm Action");

	assert((gptimer_start(gptimer)==ESP_OK) && "Failed to Start Timer");

	for (;;) {
		SampleData newSampleData;
		xQueueReceive(sampleDataQueue, &newSampleData, portMAX_DELAY);
		Serial.printf("Min Delta: %ld\n", newSampleData.minDelta);
		Serial.printf("Max Delta: %ld\n", newSampleData.maxDelta);
		Serial.printf("Avg Delta: %ld\n\n", newSampleData.averageDelta);
	}
}

bool IRAM_ATTR timerCallback(gptimer_handle_t timer, const gptimer_alarm_event_data_t *edata, void *user_data) {
	SampleData tempSampleData;
	static bool firstTime { true };
	static int64_t maxInterval { -10000000 };
	static int64_t minInterval { 10000000 };
	static int64_t intervalSum { 0 };
	static int64_t lastTimerValue { 0 };
	static uint32_t sampleCount { 0 };

//#define USE_RTC
#ifndef USE_RTC
	int64_t currentTimerValue { esp_timer_get_time() };
#else
	struct timeval tv;
	gettimeofday(&tv, NULL);
	int64_t currentTimerValue = 1000000 * tv.tv_sec + tv.tv_usec;
#endif

	BaseType_t pxHigherPriorityTaskWoken { pdFALSE };

	if (!firstTime) {
		int64_t delta { currentTimerValue - lastTimerValue };

		lastTimerValue = currentTimerValue;
		if (delta > maxInterval) {
			maxInterval = delta;
		}
		if (delta < minInterval) {
			minInterval = delta;
		}
		intervalSum += delta;
		sampleCount++;
		if (sampleCount >= 1000) {
			QueueHandle_t queueHandle { static_cast<QueueHandle_t>(user_data) };
			tempSampleData.maxDelta = maxInterval;
			tempSampleData.minDelta = minInterval;
			tempSampleData.averageDelta = intervalSum / 1000;
			sampleCount = 0;
			intervalSum = 0;
			maxInterval = -10000000;
			minInterval = 10000000;
			xQueueSendToBackFromISR(queueHandle, &tempSampleData, &pxHigherPriorityTaskWoken);
		}
	} else {
		lastTimerValue = currentTimerValue;
		firstTime = false;
	}
	return pxHigherPriorityTaskWoken == pdTRUE;
}

"Delta Value" statistics while sampling ESP32's High Resolution Timer:

Starting
Min Delta: 1000
Max Delta: 1000
Avg Delta: 1000

Min Delta: 1000
Max Delta: 1000
Avg Delta: 1000

Min Delta: 1000
Max Delta: 1000
Avg Delta: 1000

Min Delta: 1000
Max Delta: 1000
Avg Delta: 1000

Min Delta: 1000
Max Delta: 1000
Avg Delta: 1000

Min Delta: 1000
Max Delta: 1000
Avg Delta: 1000

Min Delta: 1000
Max Delta: 1000
Avg Delta: 1000

Min Delta: 1000
Max Delta: 1000
Avg Delta: 1000

Min Delta: 1000
Max Delta: 1000
Avg Delta: 1000

Min Delta: 1000
Max Delta: 1000
Avg Delta: 1000

"Delta Value" statistics while sampling ESP32's RTC:

Starting
Min Delta: 968
Max Delta: 1000
Avg Delta: 999

Min Delta: 999
Max Delta: 1001
Avg Delta: 1000

Min Delta: 999
Max Delta: 1001
Avg Delta: 1000

Min Delta: 999
Max Delta: 1001
Avg Delta: 1000

Min Delta: 999
Max Delta: 1001
Avg Delta: 1000

Min Delta: 999
Max Delta: 1001
Avg Delta: 1000

Min Delta: 999
Max Delta: 1001
Avg Delta: 1000

Min Delta: 999
Max Delta: 1001
Avg Delta: 1000

Min Delta: 999
Max Delta: 1001
Avg Delta: 1000

Min Delta: 999
Max Delta: 1001
Avg Delta: 1000

Min Delta: 999
Max Delta: 1001
Avg Delta: 1000
1 Like