---
-title: Memory Pressure Handling
-category: Interfaces
-layout: default
-SPDX-License-Identifier: LGPL-2.1-or-later
+layout: forward
+target: /PRESSURE
---
-
-# Memory Pressure Handling in systemd
-
-When the system is under memory pressure (i.e. some component of the OS
-requires memory allocation but there is only very little or none available),
-it can attempt various things to make more memory available again ("reclaim"):
-
-* The kernel can flush out memory pages backed by files on disk, under the
- knowledge that it can reread them from disk when needed again. Candidate
- pages are the many memory mapped executable files and shared libraries on
- disk, among others.
-
-* The kernel can flush out memory pages not backed by files on disk
- ("anonymous" memory, i.e. memory allocated via `malloc()` and similar calls,
- or `tmpfs` file system contents) if there's swap to write it to.
-
-* Userspace can proactively release memory it allocated but doesn't immediately
- require back to the kernel. This includes allocation caches, and other forms
- of caches that are not required for normal operation to continue.
-
-The latter is what we want to focus on in this document: how to ensure
-userspace process can detect mounting memory pressure early and release memory
-back to the kernel as it happens, relieving the memory pressure before it
-becomes too critical.
-
-The effects of memory pressure during runtime generally are growing latencies
-during operation: when a program requires memory but the system is busy writing
-out memory to (relatively slow) disks in order make some available, this
-generally surfaces in scheduling latencies, and applications and services will
-slow down until memory pressure is relieved. Hence, to ensure stable service
-latencies it is essential to release unneeded memory back to the kernel early
-on.
-
-On Linux the [Pressure Stall Information
-(PSI)](https://docs.kernel.org/accounting/psi.html) Linux kernel interface is
-the primary way to determine the system or a part of it is under memory
-pressure. PSI makes available to userspace a `poll()`-able file descriptor that
-gets notifications whenever memory pressure latencies for the system or a
-control group grow beyond some level.
-
-`systemd` itself makes use of PSI, and helps applications to do so too.
-Specifically:
-
-* Most of systemd's long running components watch for PSI memory pressure
- events, and release allocation caches and other resources once seen.
-
-* systemd's service manager provides a protocol for asking services to monitor
- PSI events and configure the appropriate pressure thresholds.
-
-* systemd's `sd-event` event loop API provides a high-level call
- `sd_event_add_memory_pressure()` enabling programs using it to efficiently
- hook into the PSI memory pressure protocol provided by the service manager,
- with very few lines of code.
-
-## Memory Pressure Service Protocol
-
-If memory pressure handling for a specific service is enabled via
-`MemoryPressureWatch=` the memory pressure service protocol is used to tell the
-service code about this. Specifically two environment variables are set by the
-service manager, and typically consumed by the service:
-
-* The `$MEMORY_PRESSURE_WATCH` environment variable will contain an absolute
- path in the file system to the file to watch for memory pressure events. This
- will usually point to a PSI file such as the `memory.pressure` file of the
- service's cgroup. In order to make debugging easier, and allow later
- extension it is recommended for applications to also allow this path to refer
- to an `AF_UNIX` stream socket in the file system or a FIFO inode in the file
- system. Regardless of which of the three types of inodes this absolute path
- refers to, all three are `poll()`-able for memory pressure events. The
- variable can also be set to the literal string `/dev/null`. If so the service
- code should take this as indication that memory pressure monitoring is not
- desired and should be turned off.
-
-* The `$MEMORY_PRESSURE_WRITE` environment variable is optional. If set by the
- service manager it contains Base64 encoded data (that may contain arbitrary
- binary values, including NUL bytes) that should be written into the path
- provided via `$MEMORY_PRESSURE_WATCH` right after opening it. Typically, if
- talking directly to a PSI kernel file this will contain information about the
- threshold settings configurable in the service manager.
-
-When a service initializes it hence should look for
-`$MEMORY_PRESSURE_WATCH`. If set, it should try to open the specified path. If
-it detects the path to refer to a regular file it should assume it refers to a
-PSI kernel file. If so, it should write the data from `$MEMORY_PRESSURE_WRITE`
-into the file descriptor (after Base64-decoding it, and only if the variable is
-set) and then watch for `POLLPRI` events on it. If it detects the paths refers
-to a FIFO inode, it should open it, write the `$MEMORY_PRESSURE_WRITE` data
-into it (as above) and then watch for `POLLIN` events on it. Whenever `POLLIN`
-is seen it should read and discard any data queued in the FIFO. If the path
-refers to an `AF_UNIX` socket in the file system, the application should
-`connect()` a stream socket to it, write `$MEMORY_PRESSURE_WRITE` into it (as
-above) and watch for `POLLIN`, discarding any data it might receive.
-
-To summarize:
-
-* If `$MEMORY_PRESSURE_WATCH` points to a regular file: open and watch for
- `POLLPRI`, never read from the file descriptor.
-
-* If `$MEMORY_PRESSURE_WATCH` points to a FIFO: open and watch for `POLLIN`,
- read/discard any incoming data.
-
-* If `$MEMORY_PRESSURE_WATCH` points to an `AF_UNIX` socket: connect and watch
- for `POLLIN`, read/discard any incoming data.
-
-* If `$MEMORY_PRESSURE_WATCH` contains the literal string `/dev/null`, turn off
- memory pressure handling.
-
-(And in each case, immediately after opening/connecting to the path, write the
-decoded `$MEMORY_PRESSURE_WRITE` data into it.)
-
-Whenever a `POLLPRI`/`POLLIN` event is seen the service is under memory
-pressure. It should use this as hint to release suitable redundant resources,
-for example:
-
-* glibc's memory allocation cache, via
- [`malloc_trim()`](https://man7.org/linux/man-pages/man3/malloc_trim.3.html). Similar,
- allocation caches implemented in the service itself.
-
-* Any other local caches, such DNS caches, or web caches (in particular if
- service is a web browser).
-
-* Terminate any idle worker threads or processes.
-
-* Run a garbage collection (GC) cycle, if the runtime environment supports it.
-
-* Terminate the process if idle, and can be automatically started when
- needed next.
-
-Which actions precisely to take depends on the service in question. Note that
-the notifications are delivered when memory allocation latency already degraded
-beyond some point. Hence when discussing which resources to keep and which to
-discard, keep in mind it's typically acceptable that latencies incurred
-recovering discarded resources at a later point are acceptable, given that
-latencies *already* are affected negatively.
-
-In case the path supplied via `$MEMORY_PRESSURE_WATCH` points to a PSI kernel
-API file, or to an `AF_UNIX` opening it multiple times is safe and reliable,
-and should deliver notifications to each of the opened file descriptors. This
-is specifically useful for services that consist of multiple processes, and
-where each of them shall be able to release resources on memory pressure.
-
-The `POLLPRI`/`POLLIN` conditions will be triggered every time memory pressure
-is detected, but not continuously. It is thus safe to keep `poll()`-ing on the
-same file descriptor continuously, and executing resource release operations
-whenever the file descriptor triggers without having to expect overloading the
-process.
-
-(Currently, the protocol defined here only allows configuration of a single
-"degree" of memory pressure, there's no distinction made on how strong the
-pressure is. In future, if it becomes apparent that there's clear need to
-extend this we might eventually add different degrees, most likely by adding
-additional environment variables such as `$MEMORY_PRESSURE_WRITE_LOW` and
-`$MEMORY_PRESSURE_WRITE_HIGH` or similar, which may contain different settings
-for lower or higher memory pressure thresholds.)
-
-## Service Manager Settings
-
-The service manager provides two per-service settings that control the memory
-pressure handling:
-
-* The
- [`MemoryPressureWatch=`](https://www.freedesktop.org/software/systemd/man/latest/systemd.resource-control.html#MemoryPressureWatch=)
- setting controls whether to enable the memory pressure protocol for the
- service in question.
-
-* The `MemoryPressureThresholdSec=` setting allows configuring the threshold
- when to signal memory pressure to the services. It takes a time value
- (usually in the millisecond range) that defines a threshold per 1s time
- window: if memory allocation latencies grow beyond this threshold
- notifications are generated towards the service, requesting it to release
- resources.
-
-The `/etc/systemd/system.conf` file provides two settings that may be used to
-select the default values for the above settings. If the threshold isn't
-configured via the per-service nor system-wide option, it defaults to 100ms.
-
-When memory pressure monitoring is enabled for a service via
-`MemoryPressureWatch=` this primarily does three things:
-
-* It enables cgroup memory accounting for the service (this is a requirement
- for per-cgroup PSI)
-
-* It sets the aforementioned two environment variables for processes invoked
- for the service, based on the control group of the service and provided
- settings.
-
-* The `memory.pressure` PSI control group file associated with the service's
- cgroup is delegated to the service (i.e. permissions are relaxed so that
- unprivileged service payload code can open the file for writing).
-
-## Memory Pressure Events in `sd-event`
-
-The
-[`sd-event`](https://www.freedesktop.org/software/systemd/man/latest/sd-event.html)
-event loop library provides two API calls that encapsulate the
-functionality described above:
-
-* The
- [`sd_event_add_memory_pressure()`](https://www.freedesktop.org/software/systemd/man/latest/sd_event_add_memory_pressure.html)
- call implements the service-side of the memory pressure protocol and
- integrates it with an `sd-event` event loop. It reads the two environment
- variables, connects/opens the specified file, writes the specified data to it,
- then watches it for events.
-
-* The `sd_event_trim_memory()` call may be called to trim the calling
- processes' memory. It's a wrapper around glibc's `malloc_trim()`, but first
- releases allocation caches maintained by libsystemd internally. This function
- serves as the default when a NULL callback is supplied to
- `sd_event_add_memory_pressure()`.
-
-When implementing a service using `sd-event`, for automatic memory pressure
-handling, it's typically sufficient to add a line such as:
-
-```c
-(void) sd_event_add_memory_pressure(event, NULL, NULL, NULL);
-```
-
-– right after allocating the event loop object `event`.
-
-## Other APIs
-
-Other programming environments might have native APIs to watch memory
-pressure/low memory events. Most notable is probably GLib's
-[GMemoryMonitor](https://docs.gtk.org/gio/iface.MemoryMonitor.html). As of GLib
-2.86.0, it uses the per-cgroup PSI kernel file to monitor for memory pressure,
-but does not yet read the environment variables recommended above.
-
-In older versions, it used the per-system Linux PSI interface as the backend, but operated
-differently than the above: memory pressure events were picked up by a system
-service, which then propagated this through D-Bus to the applications. This was
-typically less than ideal, since this means each notification event had to
-traverse three processes before being handled. This traversal created
-additional latencies at a time where the system is already experiencing adverse
-latencies. Moreover, it focused on system-wide PSI events, even though
-service-local ones are generally the better approach.
--- /dev/null
+---
+title: Resource Pressure Handling
+category: Interfaces
+layout: default
+SPDX-License-Identifier: LGPL-2.1-or-later
+---
+
+# Resource Pressure Handling in systemd
+
+On Linux the [Pressure Stall Information
+(PSI)](https://docs.kernel.org/accounting/psi.html) Linux kernel interface
+provides a way to monitor resource pressure — situations where tasks are
+stalled waiting for a resource to become available. PSI covers three types of
+resources:
+
+* **Memory pressure**: tasks are stalled because the system is low on memory
+ and the kernel is busy reclaiming it (e.g. writing out pages to swap or
+ flushing file-backed pages).
+
+* **CPU pressure**: tasks are stalled waiting for CPU time because the CPU is
+ oversubscribed.
+
+* **IO pressure**: tasks are stalled waiting for IO operations to complete
+ because the IO subsystem is saturated.
+
+PSI makes available to userspace a `poll()`-able file descriptor that gets
+notifications whenever pressure latencies for the system or a control group
+grow beyond some configured level.
+
+When the system is under memory pressure, userspace can proactively release
+memory it allocated but doesn't immediately require back to the kernel. This
+includes allocation caches, and other forms of caches that are not required for
+normal operation to continue. Similarly, when CPU or IO pressure is detected,
+services can take appropriate action such as reducing parallelism, deferring
+background work, or shedding load.
+
+The effects of resource pressure during runtime generally are growing latencies
+during operation: applications and services slow down until pressure is
+relieved. Hence, to ensure stable service latencies it is essential to detect
+pressure early and respond appropriately.
+
+`systemd` itself makes use of PSI, and helps applications to do so too.
+Specifically:
+
+* Most of systemd's long running components watch for PSI memory pressure
+ events, and release allocation caches and other resources once seen.
+
+* systemd's service manager provides a protocol for asking services to monitor
+ PSI events and configure the appropriate pressure thresholds, for memory, CPU,
+ and IO pressure independently.
+
+* systemd's `sd-event` event loop API provides high-level calls
+ `sd_event_add_memory_pressure()`, `sd_event_add_cpu_pressure()`, and
+ `sd_event_add_io_pressure()` enabling programs using it to efficiently hook
+ into the PSI pressure protocol provided by the service manager, with very few
+ lines of code.
+
+## Pressure Service Protocol
+
+For each resource type, if pressure handling for a specific service is enabled
+via the corresponding `*PressureWatch=` setting (i.e. `MemoryPressureWatch=`,
+`CPUPressureWatch=`, or `IOPressureWatch=`), two environment variables are set
+by the service manager:
+
+* `$MEMORY_PRESSURE_WATCH` / `$CPU_PRESSURE_WATCH` / `$IO_PRESSURE_WATCH` —
+ contains an absolute path in the file system to the file to watch for
+ pressure events. This will usually point to a PSI file such as the
+ `memory.pressure`, `cpu.pressure`, or `io.pressure` file of the service's
+ cgroup. In order to make debugging easier, and allow later extension it is
+ recommended for applications to also allow this path to refer to an `AF_UNIX`
+ stream socket in the file system or a FIFO inode in the file system.
+ Regardless of which of the three types of inodes this absolute path refers
+ to, all three are `poll()`-able for pressure events. The variable can also be
+ set to the literal string `/dev/null`. If so the service code should take this
+ as indication that pressure monitoring for this resource is not desired and
+ should be turned off.
+
+* `$MEMORY_PRESSURE_WRITE` / `$CPU_PRESSURE_WRITE` / `$IO_PRESSURE_WRITE` —
+ optional. If set by the service manager it contains Base64 encoded data (that
+ may contain arbitrary binary values, including NUL bytes) that should be
+ written into the path provided via the corresponding `*_PRESSURE_WATCH`
+ variable right after opening it. Typically, if talking directly to a PSI
+ kernel file this will contain information about the threshold settings
+ configurable in the service manager.
+
+The protocol works the same for all three resource types. The remainder of this
+section uses memory pressure as the example, but the same logic applies to CPU
+and IO pressure with the corresponding environment variable names.
+
+When a service initializes it hence should look for
+`$MEMORY_PRESSURE_WATCH`. If set, it should try to open the specified path. If
+it detects the path to refer to a regular file it should assume it refers to a
+PSI kernel file. If so, it should write the data from `$MEMORY_PRESSURE_WRITE`
+into the file descriptor (after Base64-decoding it, and only if the variable is
+set) and then watch for `POLLPRI` events on it. If it detects the path refers
+to a FIFO inode, it should open it, write the `$MEMORY_PRESSURE_WRITE` data
+into it (as above) and then watch for `POLLIN` events on it. Whenever `POLLIN`
+is seen it should read and discard any data queued in the FIFO. If the path
+refers to an `AF_UNIX` socket in the file system, the application should
+`connect()` a stream socket to it, write `$MEMORY_PRESSURE_WRITE` into it (as
+above) and watch for `POLLIN`, discarding any data it might receive.
+
+To summarize:
+
+* If `$MEMORY_PRESSURE_WATCH` points to a regular file: open and watch for
+ `POLLPRI`, never read from the file descriptor.
+
+* If `$MEMORY_PRESSURE_WATCH` points to a FIFO: open and watch for `POLLIN`,
+ read/discard any incoming data.
+
+* If `$MEMORY_PRESSURE_WATCH` points to an `AF_UNIX` socket: connect and watch
+ for `POLLIN`, read/discard any incoming data.
+
+* If `$MEMORY_PRESSURE_WATCH` contains the literal string `/dev/null`, turn off
+ memory pressure handling.
+
+(And in each case, immediately after opening/connecting to the path, write the
+decoded `$MEMORY_PRESSURE_WRITE` data into it.)
+
+Whenever a `POLLPRI`/`POLLIN` event is seen the service is under pressure. It
+should use this as hint to release suitable redundant resources, for example:
+
+* glibc's memory allocation cache, via
+ [`malloc_trim()`](https://man7.org/linux/man-pages/man3/malloc_trim.3.html). Similarly,
+ allocation caches implemented in the service itself.
+
+* Any other local caches, such as DNS caches, or web caches (in particular if
+ service is a web browser).
+
+* Terminate any idle worker threads or processes.
+
+* Run a garbage collection (GC) cycle, if the runtime environment supports it.
+
+* Terminate the process if idle, and can be automatically started when
+ needed next.
+
+Which actions precisely to take depends on the service in question and the type
+of pressure. Note that the notifications are delivered when resource latency
+already degraded beyond some point. Hence when discussing which resources to
+keep and which to discard, keep in mind it's typically acceptable that latencies
+incurred recovering discarded resources at a later point are acceptable, given
+that latencies *already* are affected negatively.
+
+In case the path supplied via `$MEMORY_PRESSURE_WATCH` points to a PSI kernel
+API file, or to an `AF_UNIX` socket, opening it multiple times is safe and reliable,
+and should deliver notifications to each of the opened file descriptors. This
+is specifically useful for services that consist of multiple processes, and
+where each of them shall be able to release resources on memory pressure.
+
+The `POLLPRI`/`POLLIN` conditions will be triggered every time pressure is
+detected, but not continuously. It is thus safe to keep `poll()`-ing on the
+same file descriptor continuously, and executing resource release operations
+whenever the file descriptor triggers without having to expect overloading the
+process.
+
+(Currently, the protocol defined here only allows configuration of a single
+"degree" of pressure per resource type, there's no distinction made on how
+strong the pressure is. In future, if it becomes apparent that there's clear
+need to extend this we might eventually add different degrees, most likely by
+adding additional environment variables such as `$MEMORY_PRESSURE_WRITE_LOW`
+and `$MEMORY_PRESSURE_WRITE_HIGH` or similar, which may contain different
+settings for lower or higher pressure thresholds.)
+
+## Service Manager Settings
+
+The service manager provides two per-service settings for each resource type
+that control pressure handling:
+
+* `MemoryPressureWatch=` / `CPUPressureWatch=` / `IOPressureWatch=` controls
+ whether to enable the pressure protocol for the respective resource type for
+ the service in question. See
+ [`systemd.resource-control(5)`](https://www.freedesktop.org/software/systemd/man/latest/systemd.resource-control.html#MemoryPressureWatch=)
+ for details.
+
+* `MemoryPressureThresholdSec=` / `CPUPressureThresholdSec=` /
+ `IOPressureThresholdSec=` allows configuring the threshold when to signal
+ pressure to the services. It takes a time value (usually in the millisecond
+ range) that defines a threshold per 1s time window: if resource latencies grow
+ beyond this threshold notifications are generated towards the service,
+ requesting it to release resources.
+
+The `/etc/systemd/system.conf` file provides two settings for each resource
+type that may be used to select the default values for the above settings. If
+the threshold isn't configured via the per-service nor system-wide option, it
+defaults to 100ms.
+
+When pressure monitoring is enabled for a service this primarily does three
+things:
+
+* It enables the corresponding cgroup accounting for the service (this is a
+ requirement for per-cgroup PSI).
+
+* It sets the aforementioned two environment variables for processes invoked
+ for the service, based on the control group of the service and provided
+ settings.
+
+* The corresponding PSI control group file (`memory.pressure`, `cpu.pressure`,
+ or `io.pressure`) associated with the service's cgroup is delegated to the
+ service (i.e. permissions are relaxed so that unprivileged service payload
+ code can open the file for writing).
+
+## Pressure Events in `sd-event`
+
+The
+[`sd-event`](https://www.freedesktop.org/software/systemd/man/latest/sd-event.html)
+event loop library provides API calls that encapsulate the functionality
+described above:
+
+* [`sd_event_add_memory_pressure()`](https://www.freedesktop.org/software/systemd/man/latest/sd_event_add_memory_pressure.html),
+ `sd_event_add_cpu_pressure()`, and `sd_event_add_io_pressure()` implement the
+ service-side of the pressure protocol for each resource type and integrate it
+ with an `sd-event` event loop. Each reads the corresponding two environment
+ variables, connects/opens the specified file, writes the specified data to it,
+ then watches it for events.
+
+* The `sd_event_trim_memory()` call may be called to trim the calling
+ processes' memory. It's a wrapper around glibc's `malloc_trim()`, but first
+ releases allocation caches maintained by libsystemd internally. This function
+ serves as the default when a NULL callback is supplied to
+ `sd_event_add_memory_pressure()`. Note that the default handler for
+ `sd_event_add_cpu_pressure()` and `sd_event_add_io_pressure()` is a no-op;
+ a custom callback should be provided for CPU and IO pressure to take
+ meaningful action.
+
+When implementing a service using `sd-event`, for automatic memory pressure
+handling, it's typically sufficient to add a line such as:
+
+```c
+(void) sd_event_add_memory_pressure(event, NULL, NULL, NULL);
+```
+
+– right after allocating the event loop object `event`. For CPU and IO pressure,
+a custom handler should be provided to take appropriate action:
+
+```c
+(void) sd_event_add_cpu_pressure(event, NULL, my_cpu_pressure_handler, userdata);
+(void) sd_event_add_io_pressure(event, NULL, my_io_pressure_handler, userdata);
+```
+
+## Other APIs
+
+Other programming environments might have native APIs to watch memory
+pressure/low memory events. Most notable is probably GLib's
+[GMemoryMonitor](https://docs.gtk.org/gio/iface.MemoryMonitor.html). As of GLib
+2.86.0, it uses the per-cgroup PSI kernel file to monitor for memory pressure,
+but does not yet read the environment variables recommended above.
+
+In older versions, it used the per-system Linux PSI interface as the backend, but operated
+differently than the above: memory pressure events were picked up by a system
+service, which then propagated this through D-Bus to the applications. This was
+typically less than ideal, since this means each notification event had to
+traverse three processes before being handled. This traversal created
+additional latencies at a time where the system is already experiencing adverse
+latencies. Moreover, it focused on system-wide PSI events, even though
+service-local ones are generally the better approach.
Note that this is a privileged option as, even if it has a timeout, is synchronous and delays the kill,
so use with care.
The typically preferable mechanism to process memory pressure is to do what
- <ulink url="https://systemd.io/MEMORY_PRESSURE/">MEMORY_PRESSURE</ulink> describes which is unprivileged,
+ <ulink url="https://systemd.io/PRESSURE">Resource Pressure Handling</ulink> describes which is unprivileged,
asynchronous and does not delay the kill.
</para>
<constant>LOG_DEBUG</constant> level (with message ID f9b0be465ad540d0850ad32172d57c21) about the memory
pressure operation.</para>
- <para>For further details see <ulink url="https://systemd.io/MEMORY_PRESSURE">Memory Pressure Handling in
+ <para>For further details see <ulink url="https://systemd.io/PRESSURE">Resource Pressure Handling in
systemd</ulink>.</para>
</refsect1>
<term><varname>$MEMORY_PRESSURE_WRITE</varname></term>
<listitem><para>If memory pressure monitoring is enabled for this service unit, the path to watch
- and the data to write into it. See <ulink url="https://systemd.io/MEMORY_PRESSURE">Memory Pressure
+ and the data to write into it. See <ulink url="https://systemd.io/PRESSURE">Resource Pressure
Handling</ulink> for details about these variables and the service protocol data they
convey.</para>
<para>Note that services are free to use the two environment variables, but it is unproblematic if
they ignore them. Memory pressure handling must be implemented individually in each service, and
usually means different things for different software. For further details on memory pressure
- handling see <ulink url="https://systemd.io/MEMORY_PRESSURE">Memory Pressure Handling in
+ handling see <ulink url="https://systemd.io/PRESSURE">Resource Pressure Handling in
systemd</ulink>.</para>
<para>Services implemented using
projects / thirdparty / systemd.git / commitdiff
