Limiting total allocatable process memory (VSZ) via setting RLIMIT_AS limit is
no longer effective, in order to restrict memory consumption at run time.
We can see from process memory map below, that there are many holes within
the process VA space, which bumps its VSZ to 1.5G. These holes are here by
many reasons and could be explaned at first by the full randomization of
system VA space. Now it is usually enabled in Linux kernels by default. There
are always gaps around the process stack area to trap overflows. Holes before
and after shared libraries could be explained by the fact, that on many
architectures libraries have a 'preferred' address to be loaded at; putting
them elsewhere requires relocation work, and probably some unshared pages.
Repetitive holes of 65380K are most probably correspond to the header that
malloc has to allocate before asked a claimed memory block. This header is
used by malloc to link allocated chunks together and for its internal book
keeping.
$ sudo pmap -x -p `pidof haproxy`
127136: ./haproxy -f /home/haproxy/haproxy/haproxy_h2.cfg
Address Kbytes RSS Dirty Mode Mapping
0000555555554000 388 64 0 r---- /home/haproxy/haproxy/haproxy
00005555555b5000 2608 1216 0 r-x-- /home/haproxy/haproxy/haproxy
0000555555841000 916 64 0 r---- /home/haproxy/haproxy/haproxy
0000555555926000 60 60 60 r---- /home/haproxy/haproxy/haproxy
0000555555935000 116 116 116 rw--- /home/haproxy/haproxy/haproxy
0000555555952000 7872 5236 5236 rw--- [ anon ]
00007fff98000000 156 36 36 rw--- [ anon ]
00007fff98027000 65380 0 0 ----- [ anon ]
00007fffa0000000 156 36 36 rw--- [ anon ]
00007fffa0027000 65380 0 0 ----- [ anon ]
00007fffa4000000 156 36 36 rw--- [ anon ]
00007fffa4027000 65380 0 0 ----- [ anon ]
00007fffa8000000 156 36 36 rw--- [ anon ]
00007fffa8027000 65380 0 0 ----- [ anon ]
00007fffac000000 156 36 36 rw--- [ anon ]
00007fffac027000 65380 0 0 ----- [ anon ]
00007fffb0000000 156 36 36 rw--- [ anon ]
00007fffb0027000 65380 0 0 ----- [ anon ]
...
00007ffff7fce000 4 4 0 r-x-- [ anon ]
00007ffff7fcf000 4 4 0 r---- /usr/lib/x86_64-linux-gnu/ld-2.31.so
00007ffff7fd0000 140 140 0 r-x-- /usr/lib/x86_64-linux-gnu/ld-2.31.so
...
00007ffff7ffe000 4 4 4 rw--- [ anon ]
00007ffffffde000 132 20 20 rw--- [ stack ]
ffffffffff600000 4 0 0 --x-- [ anon ]
---------------- ------- ------- -------
total kB
1499288 75504 72760
This exceeded VSZ makes impossible to start an haproxy process with 200M
memory limit, set at its initialization stage as RLIMIT_AS. We usually
have in this case such cryptic output at stderr:
$ haproxy -m 200 -f haproxy_quic.cfg
(null)(null)(null)(null)(null)(null)
At the same time the process RSS (a memory really used) is only 75,5M.
So to make process memory accounting more realistic let's base the memory
limit, set by -m option, on RSS measurement and let's use RLIMIT_DATA instead
of RLIMIT_AS.
RLIMIT_AS was used before, because earlier versions of haproxy always allocate
memory buffers for new connections, but data were not written there
immediately. So these buffers were not instantly counted in RSS, but were
always counted in VSZ. Now we allocate new buffers only in the case, when we
will write there some data immediately, so using RLIMIT_DATA becomes more
appropriate.
level and verbosity can be optionally specified on each element using ':' as
inner separator with trace name.
- -m <limit> : limit the total allocatable memory to <limit> megabytes across
- all processes. This may cause some connection refusals or some slowdowns
- depending on the amount of memory needed for normal operations. This is
- mostly used to force the processes to work in a constrained resource usage
- scenario. It is important to note that the memory is not shared between
- processes, so in a multi-process scenario, this value is first divided by
- global.nbproc before forking.
+ -m <limit> : limit allocatable memory, which is used to keep process's data,
+ to <limit> megabytes. This may cause some connection refusals or some
+ slowdowns depending on the amount of memory needed for normal operations.
+ This is mostly used to force haproxy process to work in a constrained
+ resource consumption scenario. It is important to note that the memory is
+ not shared between haproxy processes and a child process created via fork()
+ system call inherits its parent's resource limits. So, in a master-worker
+ mode this memory limit is separately applied to the master and its forked
+ worker process.
-n <limit> : limits the per-process connection limit to <limit>. This is
equivalent to the global section's keyword "maxconn". It has precedence
uid_t boot_uid;
gid_t boot_gid;
struct rlimit limit_fd; // RLIMIT_NOFILE
- struct rlimit limit_ram; // RLIMIT_AS or RLIMIT_DATA
+ struct rlimit limit_ram; // RLIMIT_DATA
#if defined(USE_THREAD)
struct {
post_mortem.process.boot_gid = getegid();
getrlimit(RLIMIT_NOFILE, &post_mortem.process.limit_fd);
-#if defined(RLIMIT_AS)
- getrlimit(RLIMIT_AS, &post_mortem.process.limit_ram);
-#elif defined(RLIMIT_DATA)
getrlimit(RLIMIT_DATA, &post_mortem.process.limit_ram);
-#endif
if (strcmp(post_mortem.platform.utsname.sysname, "Linux") == 0)
feed_post_mortem_linux();
if (global.rlimit_memmax) {
limit.rlim_cur = limit.rlim_max =
global.rlimit_memmax * 1048576ULL;
-#ifdef RLIMIT_AS
- if (setrlimit(RLIMIT_AS, &limit) == -1) {
- if (global.tune.options & GTUNE_STRICT_LIMITS) {
- ha_alert("[%s.main()] Cannot fix MEM limit to %d megs.\n",
- argv[0], global.rlimit_memmax);
- exit(1);
- }
- else
- ha_warning("[%s.main()] Cannot fix MEM limit to %d megs.\n",
- argv[0], global.rlimit_memmax);
- }
-#else
if (setrlimit(RLIMIT_DATA, &limit) == -1) {
if (global.tune.options & GTUNE_STRICT_LIMITS) {
ha_alert("[%s.main()] Cannot fix MEM limit to %d megs.\n",
ha_warning("[%s.main()] Cannot fix MEM limit to %d megs.\n",
argv[0], global.rlimit_memmax);
}
-#endif
}
#if defined(USE_LINUX_CAP)
projects / thirdparty / haproxy.git / commitdiff
