What does posix_memalign/memalign do

I'm trying to understand what functions memalign() and posix_memalign() do. Reading the available documentation didn't help.

Can someone help me understand how it works and what is it used for? Or, perhaps provide a usage example?

I'm trying to understand how linux memory works, I need to write my own simple memory pool (low-fragmentation heap).

Whereas malloc gives you a chunk of memory that could have any alignment (the only requirement is that it must be aligned for the largest primitive type that the implementation supports), posix_memalign gives you a chunk of memory that is guaranteed to have the requested alignment.

So the result of e.g. posix_memalign(&p, 32, 128) will be a 128-byte chunk of memory whose start address is guaranteed to be a multiple of 32.

This is useful for various low-level operations (such as using SSE instructions, or DMA), that require memory that obeys a particular alignment.

malloc always returns memory that is set to the maximum alignment required by any of the primitive types. This allows malloc'd memory to store any type you may need. My understanding of the description of posix_memalign, is that it returns a memory location who's address will be a multiple of whatever you specify as the alignment.

Im not sure how useful this would be when writing a custom memory pool, but I have had a go at providing an example of how this could be implemented. The difference is with my example, anything allocated with malloc_aligned has to be freed with free_aligned; however, with posix_memalign you can use free.

#include <stdlib.h>
#include <stdio.h>

void *malloc_aligned(size_t alignment, size_t bytes)
{
    // we need to allocate enough storage for the requested bytes, some 
    // book-keeping (to store the location returned by malloc) and some extra
    // padding to allow us to find an aligned byte.  im not entirely sure if 
    // 2 * alignment is enough here, its just a guess.
    const size_t total_size = bytes + (2 * alignment) + sizeof(size_t);
    
    // use malloc to allocate the memory.
    char *data = malloc(sizeof(char) * total_size);
    
    if (data)
    {
        // store the original start of the malloc'd data.
        const void * const data_start = data;
        
        // dedicate enough space to the book-keeping.
        data += sizeof(size_t);
        
        // find a memory location with correct alignment.  the alignment minus 
        // the remainder of this mod operation is how many bytes forward we need 
        // to move to find an aligned byte.
        const size_t offset = alignment - (((size_t)data) % alignment);
        
        // set data to the aligned memory.
        data += offset;
        
        // write the book-keeping.
        size_t *book_keeping = (size_t*)(data - sizeof(size_t));
        *book_keeping = (size_t)data_start;
    }
    
    return data;
}

void free_aligned(void *raw_data)
{
    if (raw_data)
    {
        char *data = raw_data;
        
        // we have to assume this memory was allocated with malloc_aligned.  
        // this means the sizeof(size_t) bytes before data are the book-keeping 
        // which points to the location we need to pass to free.
        data -= sizeof(size_t);
        
        // set data to the location stored in book-keeping.
        data = (char*)(*((size_t*)data));
        
        // free the memory.
        free(data);
    }
}

int main()
{
    char *ptr = malloc_aligned(7, 100);
    
    printf("is 5 byte aligned = %s\n", (((size_t)ptr) % 5) ? "no" : "yes");
    printf("is 7 byte aligned = %s\n", (((size_t)ptr) % 7) ? "no" : "yes");
    
    free_aligned(ptr);
    
    return 0;
}

In addition to Oli's answer I would like to point you to an even more important issue.

On recent x86 architectures a cache-line, which is the smallest amount of data that can fetched from memory to cache, is 64 bytes. Suppose your structure size is 56 bytes, you have a large array of them. When you lookup one element, the CPU will need to issue 2 memory requests (it might issue 2 requests even if it is in the middle of the cacheline). That is bad for performance, as you have to wait for memory, and you use more cache, which ultimately gives a higher cache-miss ratio. In this case it is not enough to just use posix_memalign, but you should pad or compact your structure to be on 64byte boundaries.

Having 40 byte struct is just bad luck :)

How does it work is implementation dependent. The purpose of the function is to give you an n-bytes aligned memory block (the start address of the block is a multiply of n).

As memalign is obsolete (ref: man page), only the difference between malloc() and posix_memalign() will be described here. malloc() is 8-byte aligned (eg, for RHEL 32-bit), but for posix_memalign(), alignment is user-definable. To know the use of this, perhaps one good example is setting memory attribute using mprotect(). To use mprotect(), the memory pointer must be PAGE aligned. And so if you call posix_memalign() with pagesize as the alignment, then the returned pointer can easily submit to mprotect() to set the read-write-executable attributes. (for example, after you copy the data into the memory pointer, you can set it to read-only attribute to protect it from being modified). "malloc()"'s returned pointer cannot be use here.

When you use posix_memalign in GNU C, you should be careful that second parameter should not only be a power of two, but also be a multiple of sizeof (void*). Note that this requirement is different from the one in the memalign function, which requires only power of two.

int
__posix_memalign (void **memptr, size_t alignment, size_t size)
{
  void *mem;

  /* Test whether the SIZE argument is valid.  It must be a power of
     two multiple of sizeof (void *).  */
  if (alignment % sizeof (void *) != 0
      || !powerof2 (alignment / sizeof (void *))
      || alignment == 0)
    return EINVAL;


  void *address = RETURN_ADDRESS (0);
  mem = _mid_memalign (alignment, size, address);

  if (mem != NULL)
    {
      *memptr = mem;
      return 0;
    }

  return ENOMEM;
}
weak_alias (__posix_memalign, posix_memalign) 

When we look at the first if condition in the posix_memalign implementation, it checks whether alignment is multiple of sizeof(void*).