redis bgsave failed because fork Cannot allocate memory

Linux KernelRedisFork

Linux Kernel Problem Overview


all: here is my server memory info with 'free -m'

              total       used       free     shared    buffers     cached
 Mem:         64433       49259      15174          0          3         31
 -/+ buffers/cache:      49224      15209
 Swap:         8197        184       8012

my redis-server has used 46G memory, there is almost 15G memory left free

As my knowledge,fork is copy on write, it should not failed when there has 15G free memory,which is enough to malloc necessary kernel structures .

besides, when redis-server used 42G memory, bgsave is ok and fork is ok too.

Is there any vm parameter I can tune to make fork return success ?

Linux Kernel Solutions


Solution 1 - Linux Kernel

More specifically, from the Redis FAQ

> Redis background saving schema relies on the copy-on-write semantic of fork in modern operating systems: Redis forks (creates a child process) that is an exact copy of the parent. The child process dumps the DB on disk and finally exits. In theory the child should use as much memory as the parent being a copy, but actually thanks to the copy-on-write semantic implemented by most modern operating systems the parent and child process will share the common memory pages. A page will be duplicated only when it changes in the child or in the parent. Since in theory all the pages may change while the child process is saving, Linux can't tell in advance how much memory the child will take, so if the overcommit_memory setting is set to zero fork will fail unless there is as much free RAM as required to really duplicate all the parent memory pages, with the result that if you have a Redis dataset of 3 GB and just 2 GB of free memory it will fail. > > Setting overcommit_memory to 1 says Linux to relax and perform the fork in a more optimistic allocation fashion, and this is indeed what you want for Redis.

Redis doesn't need as much memory as the OS thinks it does to write to disk, so may pre-emptively fail the fork.

Solution 2 - Linux Kernel

Modify /etc/sysctl.conf and add:

vm.overcommit_memory=1

Then restart sysctl with:

On FreeBSD:

sudo /etc/rc.d/sysctl reload

On Linux:

sudo sysctl -p /etc/sysctl.conf

Solution 3 - Linux Kernel

From proc(5) man pages:

> /proc/sys/vm/overcommit_memory > > This file contains the kernel virtual memory accounting mode. Values are: > > 0: heuristic overcommit (this is the default) > > 1: always overcommit, never check > > 2: always check, never overcommit > > In mode 0, calls of mmap(2) with MAP_NORESERVE set are not checked, and the default check is very weak, leading to the risk of getting a process "OOM-killed". Under Linux 2.4 > any non-zero value implies mode 1. In mode 2 (available since Linux 2.6), the total virtual address space on the system is limited to (SS + RAM*(r/100)), where SS is the size > of the swap space, and RAM is the size of the physical memory, and r is the contents of the file /proc/sys/vm/overcommit_ratio. >

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionJim GrayView Question on Stackoverflow
Solution 1 - Linux KernelJeff AtwoodView Answer on Stackoverflow
Solution 2 - Linux KernelGanesh KrishnanView Answer on Stackoverflow
Solution 3 - Linux KernelUtoahView Answer on Stackoverflow