Pointer vs Variable speed in C++

At a job interview I was asked the question "In C++ how do you access a variable faster, though the normal variable identifier or though a pointer". I must say I did not have a good technical answer to the question so I took a wild guess.

I said that access time will probably be that same as normal variable/identifier is a pointer to the memory address where the value is stored, just like a pointer. In other words, that in terms of speed they both have the same performance, and that pointers are only different because we can specify the memory address we want them to point to.

The interviewer did not seem very convinced/satisfied with my answer (although he did not say anything, just carried on asking something else), therefore I though to come and ask SO'ers wether my answer was accurate, and if not why (from a theory and technical POV).

When you access a "variable", you look up the address, and then fetch the value.

Remember - a pointer IS a variable. So actually, you:

a) look up the address (of the pointer variable),

b) fetch the value (the address stored at that variable)

... and then ...

c) fetch the value at the address pointed to.

So yes, accessing via "pointer" (rather than directly) DOES involve (a bit) of extra work and (slightly) longer time.

Exactly the same thing occurs whether or not it's a pointer variable (C or C++) or a reference variable (C++ only).

But the difference is enormously small.

A variable does not have to live in main memory. Depending on the circumstances, the compiler can store it in a register for all or part of its life, and accessing a register is much faster than accessing RAM.

Let's ignore optimization for a moment, and just think about what the abstract machine has to do to reference a local variable vs. a variable through a (local) pointer. If we have local variables declared as:

int i;
int *p;

when we reference the value of i, the unoptimized code has to go get the value that is (say) at 12 past the current stack pointer and load it into a register so we can work with it. Whereas when we reference *p, the same unoptimized code has to go get the value of p from 16 past the current stack pointer, load it into a register, and then go get the value that the register points to and load it into another register so we can work with it as before. The first part of the work is the same, but the pointer access conceptually involves an additional step that needs to be done before we can work with the value.

That was, I think, the point of the interview question - to see if you understood the fundamental difference between the two types of access. You were thinking that the local variable access involved a kind of lookup, and it does - but the pointer access involves that very same type of lookup to get to the value of the pointer before we can start to go after the thing it is pointing to. In simple, unoptimized terms, the pointer access is going to be slower because of that extra step.

Now with optimization, it may happen that the two times are very close or identical. It is true that if other recent code has already used the value of p to reference another value, you may already find p in a register, so that the lookup of *p via p takes the same time as the lookup of i via the stack pointer. By the same token, though, if you have recently used the value of i, you may already find it in a register. And while the same might be true of the value of *p, the optimizer can only reuse its value from the register if it is sure that p hasn't changed in the mean time. It has no such problem reusing the value of i. In short, while accessing both values may take the same time under optimization, accessing the local variable will almost never be slower (except in really pathological cases), and may very well be faster. That makes it the correct answer to the interviewer's question.

In the presence of memory hierarchies, the difference in time may get even more pronounced. Local variables are going to be located near each other on the stack, which means that you are very likely to find the address you need already in main memory and in the cache the first time you access it (unless it is the very first local variable you access in this routine). There is no such guarantee with the address the pointer points to. Unless it was recently accessed, you may need to wait for a cache miss, or even a page fault, to access the pointed-to address, which could make it slower by orders of magnitude vs. the local variable. No, that won't happen all the time - but it's a potential factor that could make a difference in some cases, and that too is something that could be brought up by a candidate in response to such a question.

Now what about the question other commenters have raised: how much does it matter? It's true, for a single access, the difference is going to be tiny in absolute terms, like a grain of sand. But you put enough grains of sand together and you get a beach. And though (to continue the metaphor) if you are looking for someone who can run quickly down a beach road, you don't want someone who will obsess about sweeping every grain of sand off the road before he or she can start running, you do want someone who will be aware when he or she is running through knee-deep dunes unnecessarily. Profilers won't always rescue you here - in these metaphorical terms, they are much better at recognizing a single big rock that you need to run around than noticing lots of little grains of sand that are bogging you down. So I would want people on my team who understand these issues at a fundamental level, even if they rarely go out of their way to use that knowledge. Don't stop writing clear code in the quest for microoptimization, but be aware of the kinds of things that can cost performance, especially when designing your data structures, and have a sense of whether you are getting good value for the price you are paying. That's why I think this was a reasonable interview question, to explore the candidate's understanding of these issues.

What paulsm4 and LaC said + a little asm:

int y = 0;
mov         dword ptr [y],0

y = x;
mov         eax,dword ptr [x]   ; Fetch x to register
mov         dword ptr [y],eax   ; Store it to y
y = *px;
mov         eax,dword ptr [px]  ; Fetch address of x
mov         ecx,dword ptr [eax] ; Fetch x
mov         dword ptr [y],ecx   ; Store it to y

Not that on the other hand it matters much, also this probably is harder to optimize (fe. you can't keep the value in cpu register, as the pointer just points to some place in memory). So optimized code for y = x; could look like this:

mov dword ptr [y], ebx - if we assume that local var x was stored in ebx

I think the interviewer was looking for you to mention the word register. As in, if you declare a variable as a register variable the compiler will do its utmost to ensure that it is stored in a register on the CPU.

A bit of chat around bus access and negotiation for other types of variables and pointers alike would have helped to frame it.

paulsm4 and LaC has already explained it nicely along with other members. I want to emphasize effect of paging when the pointer is pointing to something in heap which has been paged out.

=> Local variables are available either in the stack or in the register
=> while in case of pointer, the pointer may be pointing to an address which is not in cache and paging will certainly slow down the speed.

A variable holds a value of certain type, and accessing the variable means getting this value, from memory or from a register. When getting the value from memory we need to get it's address from somewhere - most of the time it has to be loaded into a register (sometimes it can be part of the load command itself, but this is quite rare).

A pointer keeps an address of a value; this value has to be in memory, the pointer itself can be in memory or in a register.

I would expect that on average access via a pointer will be slower than accessing the value through a variable.

Your analysis ignores the common scenario in which the pointer itself is a memory variable which must also be accessed.

There are many factors that affect the performance of software, but if you make certain simplifying assumptions about the variables involved (notably that they are not cached in any way), then each level of pointer indirection requires an additional memory access.

int a = 1234; // luggage combination
int *b = &a;
int **c = &b;
...
int e = a; // one memory access
int e = *b; // two memory accesses
int e = **c; // three memory accesses

So the short answer to "which is faster" is: ignoring compiler and processor optimizations which might be occurring, it is faster to access the variable directly.

In a best-case scenario, where this code is executed repeatedly in a tight loop, the pointer value would likely be cached into a CPU register or at worst into the processor's L1 cache. In such a case, it is likely that a first-level pointer indirection is as fast or faster than accessing the variable directly since "directly" probably means via the "stack pointer" register (plus some offset). In both cases you are using a CPU register as a pointer to the value.

There are other scenarios that could affect this analysis, such as for global or static data where the variable's address is hard-coded into the instruction stream. In such a scenario, the answer may depend on the specifics of the processor involved.

I think the key part of the question is "access a variable". To me, if a variable is in scope, why would you create a pointer to it (or a reference) to access it? Using a pointer or a reference would only make sense if the variable was in itself a data structure of some sort or if you were acessing it in some non-standard way (like interpreting an int as a float).

Using a pointer or a reference would be faster only in very specific circumstances. Under general circumstances, it seems to me that you would be trying to second guess the compiler as far as optimization is concerned and my experience tells me that unless you know what you're doing, that's a bad idea.

It would even depend on the keyword. A const keyword might very well mean that the variable is totally optimized out at compile time. That is faster than a pointer. The register keyword does not guarantee that the variable is stored in a register. So how do you know whether its faster or not? I think the answer is that it depends because there is no one size fits all answer.

I think a better answer might be it depends on where the pointer is 'pointing to'. Note, a variable might already be in the cache. However a pointer might incur a fetch penalty. It's similar to a linked list vs Vector performance tradeoff. A Vector is cache friendly because all of your memory is contigious. However a linked list, since it contains pointers, might incur a cache penalty because the memory is potentially scattered all over the place