Pointers

Projects Programming
1

Pointers often give beginning programmers problems, probably because of the way they view the data. In the discussion that follows, I'm going to take a few liberties to make things a little more clear. For example, I want you to pretend that the Arduino bootstrap loader is the operating system and is responsible for all of the duties an operating system like Linux or Windows assumes.

When you define a variable in your program with a simple statement like:

   int val;

there are a whole lot of things going on behind your back. First, upon seeing the statement terminator (i.e., the semicolon), the compiler checks the statement for syntax errors. Finding none, it then needs to see if you have already defined a variable named val at the same scope level. To perform this check, it examines an internal data structure it uses called a symbol table. The purpose of a symbol table is to maintain the details about a data item so it can be used in the program. A simplified symbol table might look like:

ID Data Type Scope lvalue …
myData int 0 600
x float 0 610

It says the program has already defined two variables name myData and x. (The ellipsis, …, means there is a lot more information stored than we are showing.) The symbol table also saves information about each variable's type, scope, lvalue, and other data, too.

So, what's an lvalue? Any data item that is defined in a program has two things associated with it: 1) its lvalue, and 2) its rvalue. The

lvalue is the memory address where a data item is stored in memory.

From the symbol table we can see that if your code wants to do something with myData, the compiler must fetch 2 bytes (it's an int) from memory starting at address 600. If you need to access variable x, the compiler needs to fetch 4 bytes from memory starting at address 610. Because your code is trying to define a new variable named val, the compiler searches its symbol table and, not seeing an entry named val at the same scope level, it adds that variable's name (e.g., identifier) to the symbol table:

ID Data Type Scope lvalue …
myData int 0 600
x float 0 610
val int 0 ?

At this point, the lvalue column is not filled in, so we place question marks in that column. Now the compiler sends a request to the bootloader: “Hey, bootloader! My programmer wants 2 bytes of memory for a new int variable. Got 2 free bytes left?” The bootloader checks its free memory bank and says: “Hey, compiler! I found 2 bytes for you at memory address 650. You're welcome.” The symbol table is then changed to:

ID Data Type Scope lvalue …
myData int 0 600
x float 0 610
val int 0 650

Note that the lvalue is now filled in.

We can represent this information about val with a graph:

lvalue and rvalues
The term lvalue evolved from assembly language programming and represented the “location value” of a data item. That is, the lvalue was where the data item was located in memory. If the lvalue column for a variable is empty, that variable is not defined. (This is where so many programmers confuse the terms define and declare. A defined variable has an lvalue, a declared variable may not. They do not mean the same thing.)

The rvalue (i.e., register value) is what it stored in that variable. In other words, because val is an int data type, if we go to memory address 650 and fetch 2 bytes of memory, we will have the current value associated with variable val. Right now, we are showing the rvalue of val as unknown because we have not assigned anything to it.

2

Now suppose your code has this statement:

   val = 10;

What happens now? Actually, a lot! Upon seeing the semicolon, the compiler checks the syntax of the statement. So far, so good. Next, it searches its symbol table to see if you have defined val at this scope level. Again, everything is fine. The compiler then fetches the lvalue for val from the symbol table (i.e., 650), goes to that memory address and moves 2 bytes of data that holds the binary value 10 into memory addresses 650-651. The graph now looks like:

Note that the rvalue is now determined.

Now suppose we find this statement:

   int total;

You should be able to convince yourself that the symbol table changes to (we're assuming the bootloader put total starting at memory address 670):

ID Data Type Scope lvalue …
myData int 0 600
x float 0 610
val int 0 650
total int 0 670

Now you write:

   total = val;

Consider what the compiler has to do. First, it checks the syntax, which is ok. Next, it looks in the symbol table to see if variables val and total are defined. Since the lvalues are filled in, we're ok. The compiler then goes to the lvalue of val (i.e., 650), fetches the 2 bytes of data stored at that address (i.e., 10), and stores that value in a temporary CPU register. It then finds the lvalue of total (i.e., 670) and moves the 2 bytes from the register to the memory address of total. Variable total now has an rvalue of 10.

This leads to a very important conclusion:

  • Normal assignment statements are rvalue to rvalue data movements between variables.*

That is, we located the rvalue of val and moved a copy of those 2 bytes into the rvalue of total. Note that this movement would not be possible if the compiler didn't have the lvalues for both variables.

3

Pointers are Different

Let's define a pointer variable:

   int *ptr;

This definition is different in several ways. First, the int term here does not define ptr as an int. Instead, it defines the type of data that ptr is designed to work with. In our earlier definitions, the int was a data type specifier and is used by the compiler to determine how many bytes of memory to allocate for that variable. For the Arduino family, all pointers are 2 bytes in size regardless of the type of pointer. In a pointer definition, the int is used to determine how pointer arithmetic is going to work. In a pointer definition, the first term (i.e., int) determine the pointer scalar size for pointer arithmetic. The asterisk, or the indirection operator (*), tells the compiler that we are about to define a pointer. The symbol table now looks like (I'm assuming the bootloader gave us 2 bytes at memory address 700):

ID Data Type Scope lvalue …
myData int 0 600
x float 0 610
val int 0 650
total int 0 670
ptr pointer 0 700

Graphically, it looks like any other variable at this point, except its lvalue is 700.

Because we have simply defined the pointer, we assume that its rvalue is whatever random bit pattern existed at memory addresses 700-701 when it was defined.

Note: A valid pointer has a rvalue that can only assume one of two values: 1) void, in which case the pointer contains nothing useful, or 2) the memory address (i.e., the lvalue) of another variable.

Now, let's do something with the pointer. Let's also assume that val still holds the value of 10. Now you write:

   ptr = &val;

First, the compiler checks to see if the syntax is ok and then if ptr and val are in scope. Both are fine. But now something different takes place. As we said earlier, most assignment statement are rvalue-to-rvalue movements of data. However, this statement is different because of the address of operator (&). This says to the compiler:

“Instead of fetching the rvalue of val, I want you to get its lvalue and assign it into ptr.”

The process of moving the lvalue of one variable into the rvalue of a pointer variable is called pointer initialization. The compiler finds the lvalue for val from the symbol table. Graphically, ptr now looks like:

Note that this is now a valid pointer because it has been initialized to hold the memory address (lvalue) of another variable, val in this instance.

Indirection

Now suppose you write:

   *ptr = 20;

The asterisk on the left side of the assignment operator tells the compiler this statement does NOT do the normal rvalue-to-rvalue data movement. Instead, the statement says to the compiler:

“Get ptr's rvalue (i.e., 650), go to that memory address (650), and deposit 2 bytes of
data that hold the binary value 20 starting at that memory address.”

Read that sentence several times, as that is the essence of using pointers. The compiler knows to use 2 bytes of data because you defined the pointer with a scalar size of int (i.e., 2 bytes). If you define a long pointer, it would move 4 bytes of data.

So, what is the rvalue of val now? It's 20 because you indirectly changed its rvalue by using the pointer variable and the process of indirection.

To see if you understand how pointers work, what does this do:

  char name[] = "Jane Smith";
   char *ptr;

   Serial.println(&name[5]);
   
   ptr = &name[5];
   strcpy(ptr, "Jones");
   Serial.print(name);
   ptr++;
   Serial.print(ptr);

After you've formed your answers, load the program into setup() and see if it does what you thought it would do. If you got the answers correct, you're on your way to understanding pointers.

4

Ok...have at it.

5

If you want text to be aligned in columns you should use tables rather than tabs or spaces.

6

johnwasser:
If you want text to be aligned in columns you should use tables rather than tabs or spaces.

Didn't realize there was a table icon!

7

Teletype is also a solution: [tt] [/tt].

8

pretend that the Arduino bootstrap loader is the operating system

The problems with this statement are (at least) threefold:

  1. It is simply wrong. A bootstrap loader is not an OS and "pretending" that it is only serves to confuse whatever issue is at hand.
  2. In the next paragraph, you go straight to a C statement and the compiler with no apparent reason or connection to the bootstrap loader (aka bootloader).
  3. I don't see how the compiler's parsing of the statements is going to clarify the use of pointers - both subjects are tricky to explain to a newbie at best.

the compiler must fetch 2 bytes (it's an int) from memory starting at address 600

The compiler does not do that. It is running on a PC/Mac/Whatever. It generates machine code which will access those locations once that machine code has been uploaded to, and executed on, the Arduino.

Now the compiler sends a request to the bootloader: "Hey, bootloader! My programmer wants 2 bytes of memory for a new int variable. Got 2 free bytes left?" The bootloader checks its free memory bank and says: "Hey, compiler! I found 2 bytes for you at memory address 650. You're welcome."

The compiler does no such thing. It has absolutely no connection with the bootloader (or pretend OS). The compiler itself allocates that address - the bootloader has absolutely nothing to do with it.

The compiler then fetches the lvalue for val from the symbol table (i.e., 650), goes to that memory address and moves 2 bytes of data that holds the binary value 10 into memory addresses 650-651.

No it doesn't. The compiler is not running on the Arduino.
The compiler (on the PC) generates machine code for the Arduino 386 processor (or whichever processor is targeted). After the compilation process has successfully completed, the IDE starts the bootloader process which uploads the machine code into the Arduino's memory. Once all the code and data are uploaded, the bootstrap starts the code executing.

You are confusing the process of generating the machine (assembler) code, which is the compiler's job, with the execution of that code on the Arduino.

Pete

9

P.S. The bootstrap process, in effect, involves two bootloaders. The one on the PC which initiates the process of sending the machine code to the Arduino and the bootloader in the Arduino itself which receives that machine code and writes it into the Arduino's memory.

Pete

10

el_supremo:
P.S. The bootstrap process, in effect, involves two bootloaders. The one on the PC which initiates the process of sending the machine code to the Arduino and the bootloader in the Arduino itself which receives that machine code and writes it into the Arduino's memory.

Pete

Only in the latter, the bootloader in the Arduino, is the word used in its usual meaning. The process on the PC isn't a bootloader. It's a programmer or loader.

11

el_supremo:
You are confusing the process of generating the machine (assembler) code, which is the compiler's job, with the execution of that code on the Arduino.

Pete

In all fairness, this is a tutorial on Pointers, not on boot loader or assembly.

econjack:
For example, I want you to pretend that the Arduino bootstrap loader is the operating system and is responsible for all of the duties an operating system like Linux or Windows assumes.

So, I imagine one should start by pretending (in the case of this example) as @econjack suggests and leave the boot loader semantics aside for the lesson at hand... Pointers.

I remember trying to first learn pointers, and how many examples it took before one finally made it "click" for me. This could well be the one for some people here on the forum!

12

Only in the latter, the bootloader in the Arduino, is the word used in its usual meaning. The process on the PC isn't a bootloader. It's a programmer or loader.

OK, you got me. Yes, that is more precise. But what about the errors in the "tutorial" that I pointed out?

In all fairness, this is a tutorial on Pointers, not on boot loader or assembly.

The compiler does not "fetch 2 bytes (it's an int) from memory starting at address 600". It also does not send "a request to the bootloader".
I fail to see how something containing factual errors about the compiler is going to help someone understand pointers. In fact I don't see why the compiler or bootloader are brought into this at all.

Pete

13

el_supremo:
OK, you got me. Yes, that is more precise. But what about the errors in the "tutorial" that I pointed out?
The compiler does not "fetch 2 bytes (it's an int) from memory starting at address 600". It also does not send "a request to the bootloader".
I fail to see how something containing factual errors about the compiler is going to help someone understand pointers. In fact I don't see why the compiler or bootloader are brought into this at all.

Pete

Well, pretending implies imagination.

It is after all his analogy and he's inclined to use whatever means he wishes, I suppose.

So, a slightly more constructive comment may be something like this perhaps:

Hey, I'm not sure the compiler/bootloader analogy is working for you @econjack.

In which case I'm inclined to agree. For a noob looking to understand pointers, they may get too torqued up in the nomenclature/semantics that don't over the long haul have anything to do with pointers. :wink:

14

@el_supremo: I used the word pretending not for purposes of correctness, but in an attempt to help newbies grasp the concept of pointers. It's like comparing Econ 101 to a Ph.D. econometrics class. At the lowest level, I try to abstract to the level that I think they can understand the concepts I'm trying to get across. Often that means simplifying things to the point where the are not technically correct. I do understand compiler design and my company designed and marketed the DOS Eco-C88 C compiler back in the 1980's. But throwing around terms like an LLR grammars does not serve the beginner all that well. Humanizing an event by making it sound like the compiler is talking with the OS is a learning technique I used for years while teaching in the Computer Technology department at Purdue University, and it served me well. You may have read one of my posts where parameter passing on function calls uses a person's backpack (not the stack) to get the point across. Lately I've used prostitutes to illustrate the dangers of global scope. Clearly, these examples require some pretending, but I think the reader and my past students tend to remember such examples. Forty years of teaching and 18 programming books later, I'm pretty sure that pretending often helps students overcome certain fears as well as learn concepts that are alien to them at first. Once they have a place to firmly stand, then they can look overhead for the details buried in the truth.

15

case study here.

I got lost at the triangle pictures....

I thought pointers were used to point to a memory location that is not necessarily known at compile time.

16

The basic "a pointer is a memory location" isn't all that tricky. The main trouble people have with pointers are:

  • pointer arithmetic - the fact that adding n to a pointer p adds n*sizeof(*p)
  • how pointers relate to arrays
  • declarations of arrays of pointers and pointers to arrays: how int (*a)[80] differs from int *a[80]
  • in C++, references (a reference is a pointer that you are not allowed to do arithmetic on)
17

zhomeslice:
...I'm struggling with knowing i have the concept down.

yes.

just use a Union

union BitField{
  uint32_t allPorts;
  byte port[4];
};

BitField bitField;

void setup()
{
  Serial.begin(9600);
  bitField.port[0] = PORTC;
  bitField.port[1] = PORTB;
  bitField.port[2] = PORTD;
  Serial.println(bitField.allPorts, BIN);
}

void loop()
{
  
}

just figure out the endianness....

18

zhomeslice:
Thanks

OK, but the OP's thread has now been officially hijacked.

Maybe a nice moderator will pull out your sub thread!

:slight_smile:

@econjack... Pointers Rock!

19

No need
I switched from punters to unions and moved on

20

PaulMurrayCbr:
The basic "a pointer is a memory location" isn't all that tricky. The main trouble people have with pointers are:

  • pointer arithmetic - the fact that adding n to a pointer p adds n*sizeof(*p)
  • how pointers relate to arrays
  • declarations of arrays of pointers and pointers to arrays: how int (*a)[80] differs from int *a[80]
  • in C++, references (a reference is a pointer that you are not allowed to do arithmetic on)

My experience is that, once they learn that the rvalue of a pointer is the lvalue of another variable, half the battle is won. That is not at all obvious to a beginner. I was going to do an addon that went into these issues. Indeed, the entire reason for talking about the pointer scalar concept was to lay the groundwork for pointer arithmetic and arrays. From that, it's a small step from the equivalence of ptr = myArray and ptr = &myArray[0]. I developed the Right-Left Rule to help people understand complex definitions:

http://jdurrett.ba.ttu.edu/3345/handouts/RL-rule.html