odd problem with PROGMEM? MRE!

I have an odd problem I may have traced to PROGMEM.

I am testing bit reversal strategies. One is to use a table. My test code uses two tables, one in PROGMEM. Use of PROGMEM seems to be interfering with the non-PROGMEM table.

The MRE here shows the matter. I have tinkered with this quite a bit (some hair left, not much) and this is the clearest example.

Arduino UNO, IDE 1.8.7, MacOS Mojave 10.14.5 fresh boot. Yes, desperate.

Both for loops in loop() produce incorrect output. But different.

This MRE is fragile and excitable. Any number of plausible small changes can make the error come and go, including addition or removal of print statements. Late addition: the serial monitor baud rate changes the output…

If I never use the PROGMEM table, the regular table functions correctly. That is, replacing

pgm_read_byte_near(progmemTable + xx)

with

table[xx]

in progmemTableReverse() makes the error go away.

If you have interest, just run this as is (including serial monitor baud!) on an UNO. I won’t be surprised if you have no trouble like I am having, that will just leave me wondering what to do about it.

I hope I have overlooked something entirely obvious. Feel free to flog me with a damp trout as you point that out.

a7

my output:

hello reverse bits timing test world!


0  0  0  0  
1  1  0  80  
2  2  2  40  
 
0 
80 
40 
 
 
0  0  0  0  
1  1  0  80  
2  2  B8  40  
 
0 
80 
40

the MRE

/**/

extern const unsigned char progmemTable[256] PROGMEM;
extern const unsigned char table[256];

void setup() {
	Serial.begin(250000);
	Serial.println("hello reverse bits timing test world!");
	Serial.println("");
}

unsigned char progmemTableReverse(unsigned char xx)
{
	return (pgm_read_byte_near(progmemTable + xx));
}

unsigned char tableReverse(unsigned char xx)
{
	return (table[xx]);
}

void loop() {

	Serial.println("");

	for (int ii = 0; ii < 3; ii++) {
		Serial.print(ii); Serial.print("  "); 
		Serial.print(ii, HEX); Serial.print("  "); 
		Serial.print(table[ii], HEX); Serial.print("  ");
		Serial.print(progmemTableReverse(ii), HEX); Serial.print("  "); 
		
		Serial.println("");
	}

	Serial.println(" ");
	Serial.print(table[0], HEX); Serial.println(" ");
	Serial.print(table[1], HEX); Serial.println(" ");
	Serial.print(table[2], HEX); Serial.println(" ");
	Serial.println(" ");

	Serial.println(" ");

	for (int ii = 0; ii < 3; ii++) {
		Serial.print(ii); Serial.print("  "); 
		Serial.print(ii, HEX); Serial.print("  "); 
		Serial.print(table[ii], HEX); Serial.print("  ");
		Serial.print(progmemTableReverse(ii), HEX); Serial.print("  "); 
		
		Serial.println("");
	}

	Serial.println(" ");
	Serial.print(table[0], HEX); Serial.println(" ");
	Serial.print(table[1], HEX); Serial.println(" ");
	Serial.print(table[2], HEX); Serial.println(" ");
	Serial.println(" ");

	for (; ; );
}

const unsigned char progmemTable[256] PROGMEM = {
        0x00, 0x80, 0x40, 0xc0, 0x20, 0xa0, 0x60, 0xe0,
        0x10, 0x90, 0x50, 0xd0, 0x30, 0xb0, 0x70, 0xf0,
        0x08, 0x88, 0x48, 0xc8, 0x28, 0xa8, 0x68, 0xe8,
        0x18, 0x98, 0x58, 0xd8, 0x38, 0xb8, 0x78, 0xf8,
        0x04, 0x84, 0x44, 0xc4, 0x24, 0xa4, 0x64, 0xe4,
        0x14, 0x94, 0x54, 0xd4, 0x34, 0xb4, 0x74, 0xf4,
        0x0c, 0x8c, 0x4c, 0xcc, 0x2c, 0xac, 0x6c, 0xec,
        0x1c, 0x9c, 0x5c, 0xdc, 0x3c, 0xbc, 0x7c, 0xfc,
        0x02, 0x82, 0x42, 0xc2, 0x22, 0xa2, 0x62, 0xe2,
        0x12, 0x92, 0x52, 0xd2, 0x32, 0xb2, 0x72, 0xf2,
        0x0a, 0x8a, 0x4a, 0xca, 0x2a, 0xaa, 0x6a, 0xea,
        0x1a, 0x9a, 0x5a, 0xda, 0x3a, 0xba, 0x7a, 0xfa,
        0x06, 0x86, 0x46, 0xc6, 0x26, 0xa6, 0x66, 0xe6,
        0x16, 0x96, 0x56, 0xd6, 0x36, 0xb6, 0x76, 0xf6,
        0x0e, 0x8e, 0x4e, 0xce, 0x2e, 0xae, 0x6e, 0xee,
        0x1e, 0x9e, 0x5e, 0xde, 0x3e, 0xbe, 0x7e, 0xfe,
        0x01, 0x81, 0x41, 0xc1, 0x21, 0xa1, 0x61, 0xe1,
        0x11, 0x91, 0x51, 0xd1, 0x31, 0xb1, 0x71, 0xf1,
        0x09, 0x89, 0x49, 0xc9, 0x29, 0xa9, 0x69, 0xe9,
        0x19, 0x99, 0x59, 0xd9, 0x39, 0xb9, 0x79, 0xf9,
        0x05, 0x85, 0x45, 0xc5, 0x25, 0xa5, 0x65, 0xe5,
        0x15, 0x95, 0x55, 0xd5, 0x35, 0xb5, 0x75, 0xf5,
        0x0d, 0x8d, 0x4d, 0xcd, 0x2d, 0xad, 0x6d, 0xed,
        0x1d, 0x9d, 0x5d, 0xdd, 0x3d, 0xbd, 0x7d, 0xfd,
        0x03, 0x83, 0x43, 0xc3, 0x23, 0xa3, 0x63, 0xe3,
        0x13, 0x93, 0x53, 0xd3, 0x33, 0xb3, 0x73, 0xf3,
        0x0b, 0x8b, 0x4b, 0xcb, 0x2b, 0xab, 0x6b, 0xeb,
        0x1b, 0x9b, 0x5b, 0xdb, 0x3b, 0xbb, 0x7b, 0xfb,
        0x07, 0x87, 0x47, 0xc7, 0x27, 0xa7, 0x67, 0xe7,
        0x17, 0x97, 0x57, 0xd7, 0x37, 0xb7, 0x77, 0xf7,
        0x0f, 0x8f, 0x4f, 0xcf, 0x2f, 0xaf, 0x6f, 0xef,
        0x1f, 0x9f, 0x5f, 0xdf, 0x3f, 0xbf, 0x7f, 0xff, 
};

const unsigned char table[256] = {
        0x00, 0x80, 0x40, 0xc0, 0x20, 0xa0, 0x60, 0xe0,
        0x10, 0x90, 0x50, 0xd0, 0x30, 0xb0, 0x70, 0xf0,
        0x08, 0x88, 0x48, 0xc8, 0x28, 0xa8, 0x68, 0xe8,
        0x18, 0x98, 0x58, 0xd8, 0x38, 0xb8, 0x78, 0xf8,
        0x04, 0x84, 0x44, 0xc4, 0x24, 0xa4, 0x64, 0xe4,
        0x14, 0x94, 0x54, 0xd4, 0x34, 0xb4, 0x74, 0xf4,
        0x0c, 0x8c, 0x4c, 0xcc, 0x2c, 0xac, 0x6c, 0xec,
        0x1c, 0x9c, 0x5c, 0xdc, 0x3c, 0xbc, 0x7c, 0xfc,
        0x02, 0x82, 0x42, 0xc2, 0x22, 0xa2, 0x62, 0xe2,
        0x12, 0x92, 0x52, 0xd2, 0x32, 0xb2, 0x72, 0xf2,
        0x0a, 0x8a, 0x4a, 0xca, 0x2a, 0xaa, 0x6a, 0xea,
        0x1a, 0x9a, 0x5a, 0xda, 0x3a, 0xba, 0x7a, 0xfa,
        0x06, 0x86, 0x46, 0xc6, 0x26, 0xa6, 0x66, 0xe6,
        0x16, 0x96, 0x56, 0xd6, 0x36, 0xb6, 0x76, 0xf6,
        0x0e, 0x8e, 0x4e, 0xce, 0x2e, 0xae, 0x6e, 0xee,
        0x1e, 0x9e, 0x5e, 0xde, 0x3e, 0xbe, 0x7e, 0xfe,
        0x01, 0x81, 0x41, 0xc1, 0x21, 0xa1, 0x61, 0xe1,
        0x11, 0x91, 0x51, 0xd1, 0x31, 0xb1, 0x71, 0xf1,
        0x09, 0x89, 0x49, 0xc9, 0x29, 0xa9, 0x69, 0xe9,
        0x19, 0x99, 0x59, 0xd9, 0x39, 0xb9, 0x79, 0xf9,
        0x05, 0x85, 0x45, 0xc5, 0x25, 0xa5, 0x65, 0xe5,
        0x15, 0x95, 0x55, 0xd5, 0x35, 0xb5, 0x75, 0xf5,
        0x0d, 0x8d, 0x4d, 0xcd, 0x2d, 0xad, 0x6d, 0xed,
        0x1d, 0x9d, 0x5d, 0xdd, 0x3d, 0xbd, 0x7d, 0xfd,
        0x03, 0x83, 0x43, 0xc3, 0x23, 0xa3, 0x63, 0xe3,
        0x13, 0x93, 0x53, 0xd3, 0x33, 0xb3, 0x73, 0xf3,
        0x0b, 0x8b, 0x4b, 0xcb, 0x2b, 0xab, 0x6b, 0xeb,
        0x1b, 0x9b, 0x5b, 0xdb, 0x3b, 0xbb, 0x7b, 0xfb,
        0x07, 0x87, 0x47, 0xc7, 0x27, 0xa7, 0x67, 0xe7,
        0x17, 0x97, 0x57, 0xd7, 0x37, 0xb7, 0x77, 0xf7,
        0x0f, 0x8f, 0x4f, 0xcf, 0x2f, 0xaf, 0x6f, 0xef,
        0x1f, 0x9f, 0x5f, 0xdf, 0x3f, 0xbf, 0x7f, 0xff,
};

Hint:

const unsigned char progmemTable[256] PROGMEM = {
  #include "tableContents.csv"
};

const unsigned char table[256] = {
  #include "tableContents.csv"
};

It saves a lot of typing

FWIW, I reproduced your issue. But, don't have an answer.

Something to do with compiler optimizations, it sees the arrays contents are exactly the same so it's optimized somehow.

If you modify slightly an array, for example, table[4] 0xc0 to 0xc1, then it will print the expected result

Edit: when I compile your code (for arduino mega 2560) it uses 2668 bytes of program space. When I modify a single byte of an array, it uses 2924 bytes (exactly 256 bytes more), so I think I'm correct... But don't ask me how to disable this optimization, I have no idea :slight_smile:

This definitely looks like a compiler bug. The optimization that is causing issues is using a single array to store the two identical arrays. It looks like the compiler only compares the contents without checking the attributes.

Simpler example:

#include <Arduino.h>

const uint8_t A[] PROGMEM {
  0x11, 0x91, 0x51, 0xd1, 0x31, 0xb1, 0x71, 0xf1,
  0x09, 0x89, 0x49, 0xc9, 0x29, 0xa9, 0x69, 0xe9,
};
const uint8_t B[] {
  0x11, 0x91, 0x51, 0xd1, 0x31, 0xb1, 0x71, 0xf1,
  0x09, 0x89, 0x49, 0xc9, 0x29, 0xa9, 0x69, 0xe9,
};

volatile int i = 0;

void setup() {
  PORTC = pgm_read_byte(&A[i]);
  PORTB = B[i];
}

void loop() {}

Compiled using: $HOME/.arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino7/bin/avr-g++ -S -Os -Wall -Wextra -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -MMD -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10809 -DARDUINO_AVR_NANO -DARDUINO_ARCH_AVR -I$HOME/.arduino15/packages/arduino/hardware/avr/1.8.3/cores/arduino -I$HOME/.arduino15/packages/arduino/hardware/avr/1.8.3/variants/eightanaloginputs array-dup.ino.cpp -o array-dup.S

Output:

	.file	"array-dup.ino.cpp"
__SP_H__ = 0x3e
__SP_L__ = 0x3d
__SREG__ = 0x3f
__tmp_reg__ = 0
__zero_reg__ = 1
	.text
	.section	.text.setup,"ax",@progbits
.global	setup
	.type	setup, @function
setup:
/* prologue: function */
/* frame size = 0 */
/* stack size = 0 */
.L__stack_usage = 0
	lds r30,i
	lds r31,i+1
	subi r30,lo8(-(_ZL1A))
	sbci r31,hi8(-(_ZL1A))
/* #APP */
 ;  13 "/home/pieter/Arduino/array-dup/array-dup.ino" 1
	lpm r30, Z
	
 ;  0 "" 2
/* #NOAPP */
	out 0x8,r30
	lds r30,i
	lds r31,i+1
	subi r30,lo8(-(_ZL1B))
	sbci r31,hi8(-(_ZL1B))
	ld r24,Z
	out 0x5,r24
/* epilogue start */
	ret
	.size	setup, .-setup
	.section	.text.loop,"ax",@progbits
.global	loop
	.type	loop, @function
loop:
/* prologue: function */
/* frame size = 0 */
/* stack size = 0 */
.L__stack_usage = 0
/* epilogue start */
	ret
	.size	loop, .-loop
.global	i
	.section	.bss.i,"aw",@nobits
	.type	i, @object
	.size	i, 2
i:
	.zero	2
	.section	.progmem.data._ZL1A,"a",@progbits
	.type	_ZL1A, @object
	.size	_ZL1A, 16
_ZL1A:
	.byte	17
	.byte	-111
	.byte	81
	.byte	-47
	.byte	49
	.byte	-79
	.byte	113
	.byte	-15
	.byte	9
	.byte	-119
	.byte	73
	.byte	-55
	.byte	41
	.byte	-87
	.byte	105
	.byte	-23
	.set	_ZL1B,_ZL1A
	.ident	"GCC: (GNU) 7.3.0"
.global __do_clear_bss

As you can see, the compiler emits the array only once, in the .progmem section, and then simply creates an alias for the second array using .set _ZL1B,_ZL1A. That means that the ld r24,Z on line 31 will try to load from an address in PROGMEM (bad). The lpm r30, Z on line 22 will work fine.

If the arrays are different, the compiler correctly places A in the .progmem section (FLASH) and B in the .rodata section (RAM):

	.section	.rodata._ZL1B,"a",@progbits
	.type	_ZL1B, @object
	.size	_ZL1B, 16
_ZL1B:
	.byte	17
	.byte	-111
	.byte	81
	.byte	-47
	.byte	49
	.byte	-79
	.byte	113
	.byte	-15
	.byte	9
	.byte	-119
	.byte	73
	.byte	-55
	.byte	41
	.byte	-87
	.byte	105
	.byte	-22
	.section	.progmem.data._ZL1A,"a",@progbits
	.type	_ZL1A, @object
	.size	_ZL1A, 16
_ZL1A:
	.byte	17
	.byte	-111
	.byte	81
	.byte	-47
	.byte	49
	.byte	-79
	.byte	113
	.byte	-15
	.byte	9
	.byte	-119
	.byte	73
	.byte	-55
	.byte	41
	.byte	-87
	.byte	105
	.byte	-23

As a workaround, simply add a byte to one of the arrays, so they are no longer the same.

Pieter

Really odd bug, compiling for an UNO is clearly not allocating any RAM for the table, the total dynamic memory usage is less than the size of the table.

Just adding an instruction to print the address of the table array brings it into existence.

As a practical matter, if you just allocate space for the table array, then do a memcpy_P from progmemTable to table it seems to work OK, and it would be wasteful to have the compiler store a redundant copy of identical data just to initialize the array.

After seeing Pieter’s post, I tried compiling for an atmega328 using MCUdude’s MiniCore. Disabling LTO (link time optimization) appears to generate the code with the correct RAM usage, so it may be a linker problem.

david_2018:
After seeing Pieter’s post, I tried compiling for an atmega328 using MCUdude’s MiniCore. Disabling LTO (link time optimization) appears to generate the code with the correct RAM usage, so it may be a linker problem.

I didn’t use the linker for my previous post. I simply used GCC to compile the code (without LTO) and output the assembly. The second array is already gone after the compilation stage.

I’m not sure what version of GCC MiniCore uses, but there’s a good chance it’s 5.4.0, and this version seems to swap the two arrays (i.e. place array B in .rodata and make A an alias instead of placing A in .progmem and making B an alias). This results in the correct RAM usage report and will probably work fine (I think reading from RAM using LPM is fine?), but it’s still technically incorrect, because there will be no array A in the .progmem section.

It seems that AVR GCC 9.2.0 still has the same problem, so it might be worth filing a bug report:

Something really strange. I eliminated the use of the progmemTableReverse() funciton, but left the function itself in the code, and the problem remains. Remove the now unused function, or just the line where progmemTable is read, and the code works correctly. Does not make any sense that a function that should be completely optimized away because it is never used would alter the code.

const unsigned char progmemTable[] PROGMEM = {
  0x00, 0x80, 0x40, 0xc0, 0x20, 0xa0, 0x60, 0xe0,
  0x10, 0x90, 0x50, 0xd0, 0x30, 0xb0, 0x70, 0xf0,
  0x08, 0x88, 0x48, 0xc8, 0x28, 0xa8, 0x68, 0xe8,
  0x18, 0x98, 0x58, 0xd8, 0x38, 0xb8, 0x78, 0xf8
};

const unsigned char table[] = {
  0x00, 0x80, 0x40, 0xc0, 0x20, 0xa0, 0x60, 0xe0,
  0x10, 0x90, 0x50, 0xd0, 0x30, 0xb0, 0x70, 0xf0,
  0x08, 0x88, 0x48, 0xc8, 0x28, 0xa8, 0x68, 0xe8,
  0x18, 0x98, 0x58, 0xd8, 0x38, 0xb8, 0x78, 0xf8
};

void setup() {
  Serial.begin(115200);
  Serial.println("hello reverse bits timing test world!");
  Serial.println("");
}

unsigned char progmemTableReverse(unsigned char xx)
{
  return (pgm_read_byte_near(progmemTable + xx)); // << eliminate this line and the code works correctly
  //return random(100);
}

void loop() {

  Serial.println("");

  for (int ii = 0; ii < 3; ii++) {
    Serial.print(ii); Serial.print("  ");
    Serial.print(ii, HEX); Serial.print("  ");
    Serial.print(table[ii], HEX); Serial.print("  ");
    Serial.print(pgm_read_byte_near(progmemTable + ii), HEX); Serial.print("  ");

    Serial.println("");
  }

  Serial.println(" ");
  Serial.print(table[0], HEX); Serial.println(" ");
  Serial.print(table[1], HEX); Serial.println(" ");
  Serial.print(table[2], HEX); Serial.println(" ");
  Serial.println(" ");

  Serial.println(" ");

  for (int ii = 0; ii < 3; ii++) {
    Serial.print(ii); Serial.print("  ");
    Serial.print(ii, HEX); Serial.print("  ");
    Serial.print(table[ii], HEX); Serial.print("  ");
    Serial.print(pgm_read_byte_near(progmemTable + ii), HEX); Serial.print("  ");

    Serial.println("");
  }

  Serial.println(" ");
  Serial.print(table[0], HEX); Serial.println(" ");
  Serial.print(table[1], HEX); Serial.println(" ");
  Serial.print(table[2], HEX); Serial.println(" ");
  Serial.println(" ");

  for (; ; );
}

david_2018:
Does not make any sense that a function that should be completely optimized away because it is never used would alter the code.

It shouldn't be optimized away though, the compiler cannot possibly know that the function isn't used, it might be used in other translation units. If you mark the function static the compiler can be sure that it can only used in the current translation unit and you should see the function and the array being optimized away if it isn't used (probably with a compiler warning about unused functions/variables).

Well no damp trout, so.

Thank you for testing this and going a bit further.

I appreciate your time and effort. Creating the MRE was a process involving some frustration. Tiny changes mattered, so I’d get something all set then add a print statement and destroy it as an example. I think we could add “capricious” to “fragile” and “excitable” when describing the MRE and variants thereto.

I understand your theories, but it is tots odd that printing the array values (after each loop) does not trigger the issue but shows the array contents as correct.

In one attemtp I unrolled the loop and it worked fine. But maybe at that precise moment I had made some other consequential change(s). Argh.

I thought maybe it was the preprocessor, somehow. I get stung by it from time to time.

Not a compiler/assembler/linker/loader problem, what are the odds? I always laugh at subject lines inquiring as to, for example, when did they change the way “if” statements work. Not changed. Not the compiler. Not a mainstream tool in 2021.

I took @PieterP’s link Compiler Explorer but did not make sense of what I could do there as far as filing a bug report is concerned. I did some googling and there are (still?) problems with PROGMEM and the linker, so it may already be on someone’s radar.

I don’t normally have identical arrays stashed variously around my programs. And FWIW, both tables were placed in my code with three keystrokes: One to cut the table from where I found it (on the internet!) and one more each to paste it where it needed to be syntax-wise. Before I got in deep there were oddities and I did take both tables out and beat them around a bit to be sure they were correctly filled in. It did not occur to me though to de-identicalize them.

Part of the genesis of this hole project was to show how to avoid mistakes when creating and using and tables… and it morphed a bit as I wondered if a table was really that much better than about 11 different bit reversal algorithms I have cataloged so far.

Now I can get back to draining the swamp. Look forward to my report (!) on bit reversal. I am thinking I’d like to see the very worst implementation of the dumbest programming idea for bit reversal (yet plausibly coded).

a7

alto777:
I understand your theories, but it is tots odd that printing the array values (after each loop) does not trigger the issue but shows the array contents as correct.

Because the indices are constant. The compiler optimizes away the array access. Instead of doing

load the constant index “0” into a register A (immediate)
load array[A] from RAM into register B
print register B

the compiler does

load the constant value of array[0] into register A (immediate)
print register A

array[0] is constant, so there's no point in reading it from RAM each time, its value is included in the code as a constant argument to a “load immediate” instruction. The main problem with your code is reading from a PROGMEM array using an instruction that can only read from RAM, there are no issues if the compiler optimizes away the RAM access and loads the value immediately.

In one attemtp I unrolled the loop and it worked fine. But maybe at that precise moment I had made some other consequential change(s). Argh.

Same story here: if the loop is unrolled, the indices are known at compile time, and so are the values of the arrays at those indices, so the problematic memory accesses are optimized away.

alto777:
I took @PieterP's link Compiler Explorer but did not make sense of what I could do there as far as filing a bug report is concerned.

The link simply demonstrates the problem with a newer compiler than the one that Arduino uses, so there's a good chance it's not fixed yet. If you look at line 172 of the assembly output, you'll see the same problematic line .set _ZL1B,_ZL1A, ignoring A's section .progmem attribute.

Strange enough, I wasn't able to reproduce this issue on x86 or ARM (but I didn't try that hard).

@PieterP thank you, that makes sense.

The array values I printed were from the non-PROGMEM array, so array[0] being picked up as a constant in the MRE was some optimization.

When I unrolled the loop, I just assigned the loop variable as required before repeating the loop code and left the indices in the array references as variables. I do see that this also could be optimized away.

So I am impressed by the optimizer. Sometimes favorably. I wonder just how bad is non-optimized code when the target is an UNO. And whether turning it off might should be higher on my list of desperate measures. Or on the list at all.

Most of my programs are not particularly heavy on the CPU use. Most of the programs I see here are not. Any program with a delay() or equivalent is obviously not even trying. For heavy lifting there are better choices.

I'll google and learn, but I ask how does one turn off the optimizer?

a7

The compiler is smart enough to see that the two arrays have the same contents... but not smart enough to know that they are in different address spaces. It keeps the first one and uses the PROGMEM address for both arrays. Unfortunately, RAM doesn't contain an array at the PROGMEM address so it grabs whatever garbage it finds.

I changed one byte in the middle of the RAM table to 0xFF and that caused the compiler to keep both tables and use the PROGMEM address for PROGMEM and the RAM address for RAM. You get the same effect if you add a dummy 257th element to one of the arrays.

Here’s a minimal sketch that reproduces the problem

const unsigned char progmemTable[4] PROGMEM = {0x00, 0x80, 0x40, 0xc0};
const unsigned char table[4] = {0x00, 0x80, 0x40, 0xc0};

unsigned char progmemTableReverse(unsigned char xx) {
  return (pgm_read_byte_near(progmemTable + xx));
}
unsigned char tableReverse(unsigned char xx) {
  return (table[xx]);
}

void setup() {
  Serial.begin(250000);
  Serial.println("hello reverse bits timing test world!");
  Serial.println("");
}

void loop() {
  Serial.println(""); // comment out to get B8 in the first test (below) 

  for (int ii = 0; ii < 3; ii++) {
    Serial.print(ii); Serial.print("  ");
    Serial.print(ii, HEX); Serial.print("  ");
    Serial.print(table[ii], HEX); Serial.print("  ");
    Serial.print(progmemTableReverse(ii), HEX); Serial.print("  ");
    Serial.println("");
  }
  Serial.println();

  for (int ii = 0; ii < 3; ii++) {
    Serial.print(ii); Serial.print("  ");
    Serial.print(ii, HEX); Serial.print("  ");
    Serial.print(table[ii], HEX); Serial.print("  ");
    Serial.print(progmemTableReverse(ii), HEX); Serial.print("  ");
    Serial.println("");
  }
  for (; ; );
}

Strange problem … very sensitive. Just changing the baudrate, or renaming from “ii” to “i”, or commenting out a print line changes other data (re: B8). I guess the compiler makes different decisions no matter how small the change might seem.

alto777:
So I am impressed by the optimizer. Sometimes favorably. I wonder just how bad is non-optimized code when the target is an UNO. And whether turning it off might should be higher on my list of desperate measures. Or on the list at all.

Most of my programs are not particularly heavy on the CPU use. Most of the programs I see here are not. Any program with a delay() or equivalent is obviously not even trying. For heavy lifting there are better choices.

Unoptimized code is pretty terrible. Especially on microcontrollers turning off the optimizer is often not feasible (even if you don't care about speed) because of the increased memory usage.

For example, your MRE with an additional 257th byte to get around the issue consumes the following amounts of memory with -Os compared to -O0:

-Os
Sketch uses 2854 bytes (8%) of program storage space. Maximum is 32256 bytes.
Global variables use 484 bytes (23%) of dynamic memory, leaving 1564 bytes for local variables. Maximum is 2048 bytes.

-O0
Sketch uses 10448 bytes (32%) of program storage space. Maximum is 32256 bytes.
Global variables use 610 bytes (29%) of dynamic memory, leaving 1438 bytes for local variables. Maximum is 2048 bytes.

Those are really significant differences.

alto777:
I'll google and learn, but I ask how does one turn off the optimizer?

Edit $HOME/.arduino15/packages/arduino/hardware/avr/1.8.3/platform.txt and replace -Os (optimize for size) with -O0 (no optimization).
I really wouldn't recommend it though (only for testing purposes).

If disabling optimizations fixes your program, your response should never be to disable the optimizer.
In 99.999% of cases, the reason for the problems is your code invoking undefined behavior, and you should absolutely address that first.

If your code is correct, it might be a compiler bug (as in this case), but this is extremely rare. In that case, your reaction should be to file a bug report with the compiler devs, not disabling optimizations.

Noted, @PietreP, THX. BTW you show up quite a bit to disabuse compiler error finder claimants of their fanciful notions. So thanks for that, and usually with enough understandable logic so we can follow along nodding. I certainly would never lead with the idea that I had come across a compiler error - I fully expected this to turn out to be something (stupid) I would have to own. And get over having rattled the cage.

I googled enough to find this

Bug 92606

which I think is this matter already on the radar screen it should be.

And MRE nicely made smaller, thanks @dlloyd. At one point I had chopped a few values off one of the arrays, naturally everything worked, at least that's how I remember it, but it still didn't occur to me that the identical contents was key.

a7

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.