Platform devices created with platform_device_alloc() call
platform_device_release() when the last reference to the device's
kobject is dropped. This function calls of_node_put() unconditionally.
This works fine for devices created with platform_device_register_full()
but users of the split approach (platform_device_alloc() +
platform_device_add()) must bump the reference of the of_node they
assign manually. Add the missing call to of_node_get().
Kiran Kumar K [Tue, 2 Jun 2026 04:05:25 +0000 (09:35 +0530)]
octeontx2-af: kpu: Default profile updates
Add support for parsing the following:
1. fabric path header
2. tpids 0x88a8, 0x9100 and 0x9200 parsing for
first pass and second pass packets
3. parse stacked VLANs
4. RoCEv2 header with UDP destination port 4791
5. single SBTAG parsing
====================
selftests: rds: ROCE support follow ups
This is a follow up series to the "Add ROCE support to rds selftests"
series. The first patch renames run.sh to rds_run.sh, which provides
a self-describing name that appears on the netdev CI dashboard.
The second patch addresses a sashiko complaint that I thought was
worth circling back for. In the patch "pin RDS sockets to their
intended transport," sockets are pinned to the specific transport they
are meant to test. By default, socket transports are implicitly
selected based on the network topology, but it is possible that they
can fail back to other transports if the underlying connection could
not be established. So the patch pins them to the intended transport
to avoid false positives.
The third patch "support RDS built as loadable modules," lifts the
CONFIG_MODULES=n requirement, and updates the check_*conf_enabled()
to allow modules set to "=m" and further load the backing modules for
any component set as such. config.sh is updated to match.
The fourth patch converts the rdma-prerequisite checks to return XFAIL
rather than SKIP, since the RDMA datapath is not run in netdev CI.
Questions, comments and feedback appreciated!
====================
selftests: rds: report missing RDMA prereqs as XFAIL
Make the RDMA test return XFAIL rather than skip when RXE is not
available, since the RDMA datapath is not run in netdev CI.
Change the three RDMA-prerequisite checks in check_rdma_conf() and
check_rdma_conf_enabled() to exit with KSFT_XFAIL (2) and tag their
messages [XFAIL] instead of [SKIP].
selftests: rds: support RDS built as loadable modules
Commit 92cc6708f4a2 ("selftests: rds: config: disable modules") set
CONFIG_MODULES=n since run.sh required this kconfig. But disabling
modules also forces every =m option to =n rather than =y, which can
silently drop unrelated features.
This patch removes CONFIG_MODULES=n from the rds selftest config and
updates the check_*conf_enabled() routines to accept a config as
either built-in (=y) or modular (=m). A new probe_module() function
is added to load the backing module when a component is set to be
modular (=m). config.sh no longer forces CONFIG_MODULES=n, so a user
who follows the SKIP message to run config.sh does not silently end
up with modules disabled again.
rds.ko itself is auto-loaded on socket creation, and rds_rdma.ko is
auto-loaded when SO_RDS_TRANSPORT is set with RDS_TRANS_IB, but the
TCP transport (rds_tcp.ko) is not auto-loaded on the bind path, so
the backing modules are loaded explicitly here.
selftests: rds: pin RDS sockets to their intended transport
The RDS selftests create AF_RDS sockets but never selects a transport,
so the transport is chosen implicitly based on network topology when
the socket is bound. If underlying connection establishment fails, RDS
can fall back to another transport (e.g. loopback) and the test still
passes, silently bypassing the intended datapath it is meant to
exercise.
Set SO_RDS_TRANSPORT to the proper RDS_TRANS_IB or RDS_TRANS_TCP before
they are bound, so the test fails loudly if the intended transport is
unavailable rather than passing on a different path.
Akash Sukhavasi [Wed, 3 Jun 2026 20:42:20 +0000 (15:42 -0500)]
dt-bindings: net: dsa: remove obsolete dsa.txt
dsa.txt has been a redirect to dsa.yaml since commit bce58590d1bd
("dt-bindings: net: dsa: Add DSA yaml binding") introduced the .yaml
schema. The .yaml has the same filename in the same directory, making
this redirect unnecessary for discoverability.
Two files still reference dsa.txt, forcing readers through an extra
hop to reach the .yaml. The stub has not been touched since August
2020. Update references in lan9303.txt and
Documentation/networking/dsa/dsa.rst to point directly to dsa.yaml
and remove the stub.
Akash Sukhavasi [Wed, 3 Jun 2026 20:42:18 +0000 (15:42 -0500)]
dt-bindings: net: remove obsolete mdio.txt
mdio.txt has been a single-line redirect to mdio.yaml since
commit 62d77ff7ecbf ("dt-bindings: net: Add a YAML schemas for the
generic MDIO options"), which introduced the .yaml schema and reduced
the .txt to a stub in the same change. The .yaml has the same filename
in the same directory, making this redirect unnecessary for
discoverability.
No files in the tree reference mdio.txt and it has not been touched
since June 2019. Remove the obsolete stub.
Eric Dumazet [Wed, 3 Jun 2026 18:08:31 +0000 (18:08 +0000)]
rtnetlink: use dev_isalive() in rtnl_getlink()
rtnl_getlink() uses an RCU lookup to get the netdevice pointer.
When/If rtnl_lock() is used, we should check if the netdevice is not
being dismantled before potentially perform illegal actions.
Move dev_isalive() out of net/core/net-sysfs.c and make it available
in net/core/dev.h.
Return -ENODEV if rtnl_getlink() finds a device which is currently
being dismantled and RTNL is requested.
Fixes: e896e5c0734b ("rtnetlink: do not acquire RTNL in rtnl_getlink() with RTEXT_FILTER_NAME_ONLY") Signed-off-by: Eric Dumazet <edumazet@google.com> Suggested-by: Jakub Kicinski <kuba@kernel.org> Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev> Link: https://patch.msgid.link/20260603180831.1024716-1-edumazet@google.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Ido Schimmel [Wed, 3 Jun 2026 10:35:22 +0000 (13:35 +0300)]
bridge: mcast: Synchronously shutdown port multicast timers
Currently, while four timers are set up during port multicast context
initialization, only two are synchronously deleted when the context is
de-initialized, just before being deleted.
This is fine because the structure containing the multicast context
(either a bridge port or a VLAN) is only deleted after an RCU grace
period and it will not pass as long as the timers are executing. These
timers are also not supposed to do any work at this stage. They acquire
the bridge multicast lock, see that the multicast context was disabled
and exit.
Make the code more explicit and symmetric and synchronously shutdown all
four timers when the multicast context is de-initialized. Use
timer_shutdown_sync() to guarantee that the timers will not be re-armed
given that the containing structure is being deleted.
Hyunwoo Kim [Tue, 2 Jun 2026 10:21:05 +0000 (19:21 +0900)]
inet: frags: fix use-after-free caused by the fqdir_pre_exit() flush
On netns teardown, fqdir_pre_exit() walks the fqdir rhashtable and
flushes every fragment queue that is not yet complete using
inet_frag_queue_flush(). That helper frees all the skbs queued on the
fragment queue but does not set INET_FRAG_COMPLETE, and leaves
q->fragments_tail and q->last_run_head pointing at the freed skbs.
The queue itself stays in the rhashtable.
fqdir_pre_exit() first lowers high_thresh to 0 to stop new queue lookups,
but it cannot stop a fragment that already obtained the queue through
inet_frag_find() earlier and stalled just before taking the queue lock.
Once that fragment resumes after the flush and takes the queue lock,
it passes the INET_FRAG_COMPLETE check and then dereferences the freed
fragments_tail. inet_frag_queue_insert() reads FRAG_CB() and ->len of
that pointer and, on the append path, writes ->next_frag, causing a
slab use-after-free. IPv6, nf_conntrack_reasm6 and 6lowpan reassembly
share the same flush path and are affected as well.
Reset rb_fragments, fragments_tail and last_run_head in
inet_frag_queue_flush() so a flushed queue no longer points at the
freed skbs. A fragment that resumes after the flush and takes the
queue lock then finds an empty queue and starts a new run instead of
dereferencing the freed fragments_tail. ip_frag_reinit() already
performed this reset after its own flush, so drop the now duplicate
code there.
Cc: stable@vger.kernel.org Fixes: 006a5035b495 ("inet: frags: flush pending skbs in fqdir_pre_exit()") Suggested-by: Eric Dumazet <edumazet@google.com> Signed-off-by: Hyunwoo Kim <imv4bel@gmail.com> Link: https://patch.msgid.link/ah6ukYq5G98LshdA@v4bel Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Iker Pedrosa [Tue, 26 May 2026 14:36:57 +0000 (16:36 +0200)]
i2c: busses: make K1 driver default for SpacemiT platforms
Enable I2C_K1 by default when ARCH_SPACEMIT is configured to ensure SD
card functionality works out-of-the-box.
SpacemiT K1 boards use I2C-controlled PMICs (like the P1 chip) to
provide SD card power supplies. Without the I2C_K1 driver enabled,
regulators cannot be controlled and SD card detection/operation fails.
Dave Airlie [Thu, 4 Jun 2026 23:09:38 +0000 (09:09 +1000)]
Merge tag 'drm-rust-next-2026-06-04' of https://gitlab.freedesktop.org/drm/rust/kernel into drm-next
DRM Rust changes for v7.2-rc1
- Driver Core (shared via signed tag dd-lifetimes-7.2-rc1):
- Introduce Higher-Ranked Lifetime Types (HRT) for Rust device
drivers, allowing driver structs to hold device resources like
pci::Bar and IoMem directly with a lifetime tied to the binding
scope, removing the need for Devres indirection and ARef<Device>.
- Replace drvdata() with scoped registration data on the auxiliary
bus, using the new ForLt trait to thread lifetimes through
registrations. Remove drvdata() and driver_type.
- DRM:
- Add GPUVM immediate mode abstraction for Rust GPU drivers:
- In immediate mode, GPU virtual address space state is updated
during job execution (in the DMA fence signalling critical path),
keeping the GPUVM and the GPU's address space always in sync.
- Provide GpuVm, GpuVa, and GpuVmBo types for managing address
spaces, virtual mappings, and GEM object backing respectively.
- Provide split-merge map/unmap operations that handle partial
overlaps with existing mappings.
- drm_exec integration for dma_resv locking and GEM object
validation based on the external/evicted object lists are not
yet covered and planned as follow-up work.
- Introduce DeviceContext type state for drm::Device, allowing
drivers to restrict operations to contexts where the device is
guaranteed to be registered (or not yet registered) with userspace.
- Add FEAT_RENDER flag to the Driver trait for render node support.
- Nova:
- Hopper/Blackwell enablement:
- Add GPU identification and architecture-based HAL selection for
Hopper (GH100) and Blackwell (GB100, GB202).
- Implement the FSP (Foundation Security Processor) boot path used by
Hopper and Blackwell, including FSP falcon engine support, EMEM
operations, MCTP/NVDM message infrastructure, and FSP Chain of
Trust boot with GSP lockdown release.
- Add support for 32-bit firmware images and auto-detection of
firmware image format.
- GSP boot and unload:
- Refactor the GSP boot process into a chipset-specific HAL,
keeping the SEC2 and FSP boot paths separated cleanly.
- Implement proper driver unload: send UNLOADING_GUEST_DRIVER
command, run Booter Unloader and FWSEC-SB upon unbinding, and run
the unload bundle on Gsp::boot() failure. This removes the need
for a manual GPU reset between driver unbind and re-probe.
- GA100 support:
- Add support for the GA100 GPU, including IFR header detection and
skipping, correct fwsignature selection, conditional FRTS boot,
and documentation of the IFR header layout.
- VBIOS hardening and refactoring:
- Harden VBIOS parsing with checked arithmetic, bounds-checked
accesses, and FromBytes-based structure reads throughout the FWSEC
and Falcon data paths. Simplify the overall VBIOS module
structure.
- HRT adoption:
- Use lifetime-parameterized pci::Bar directly, replacing the
Arc<Devres<Bar0>> indirection. Replace ARef<Device> with &'bound
Device in SysmemFlush and the GSP sequencer. Separate the driver
type from driver data.
- Misc:
- Rename module names to kebab-case (nova-drm, nova-core).
- Require little-endian in Kconfig, making the existing assumption
explicit.
- Tyr:
- Define comprehensive typed register blocks for GPU_CONTROL,
JOB_CONTROL, MMU_CONTROL (including per-address-space registers),
and DOORBELL_BLOCK using the kernel register!() macro. This replaces
manual bit manipulation with typed register and field accessors.
- Add shmem-backed GEM objects and set DMA mask based on GPU physical
address width.
- Adopt HRT: separate driver type from driver data, and use IoMem
directly instead of Devres for register access during probe.
i2c: Use named initializers for arrays of i2c_device_data
While being less compact, using named initializers allows to more easily
see which members of the structs are assigned which value without having
to lookup the declaration of the struct. And it's also more robust
against changes to the struct definition.
The mentioned robustness is relevant for a planned change to struct
i2c_device_id that replaces .driver_data by an anonymous union.
While touching all these arrays, unify usage of whitespace in the list
terminator.
This patch doesn't modify the compiled arrays, only their representation
in source form benefits. The former was confirmed with x86 and arm64
builds.
Dave Jiang [Thu, 4 Jun 2026 18:01:54 +0000 (11:01 -0700)]
cxl/pci: Convert PCIBIOS errors to errno on DVSEC config accesses
PCI config space accessors return positive PCIBIOS_* status codes on
failure that are positive integers. Several DVSEC accesses in the CXL
core propagated these raw values to callers that test for failure against
less than 0. Thus silently misinterpret the return value as success.
Convert the positive error values to negative errno values so the checks
are correct on error paths.
While the chances of a config access failure are low, fix for correctness
and to avoid confusion in the future when more DVSEC accesses are added.
Fixes: 14d788740774 ("cxl/mem: Consolidate CXL DVSEC Range enumeration in the core") Fixes: ce17ad0d5498 ("cxl: Wait Memory_Info_Valid before access memory related info") Reviewed-by: Richard Cheng <icheng@nvidia.com> Reviewed-by: Jonathan Cameron <jic23@kernel.org> Assisted-by: Claude:claude-opus-4-8 Reviewed-by: Alison Schofield <alison.schofield@intel.com> Link: https://patch.msgid.link/20260604180154.1925149-3-dave.jiang@intel.com Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Muhammad Bilal [Sun, 24 May 2026 13:03:19 +0000 (13:03 +0000)]
accel/ethosu: reject DMA commands with uninitialized length
cmd_state_init() initializes the command state with memset(0xff),
leaving dma->len at U64_MAX to signal missing setup. The only setter
is NPU_SET_DMA0_LEN; if userspace omits this command and issues
NPU_OP_DMA_START, dma->len remains U64_MAX.
In dma_length(), a positive stride added to U64_MAX wraps to a small
value. With size0 == 1, check_mul_overflow() does not trigger and
dma_length() returns 0 instead of U64_MAX. The caller's U64_MAX check
then passes, region_size[] stays 0, and the bounds check in
ethosu_job.c is bypassed, allowing hardware to execute DMA with stale
physical addresses.
Fix by checking for U64_MAX at the start of dma_length() before any
arithmetic, consistent with the sentinel value used throughout the
driver to detect uninitialized fields.
Muhammad Bilal [Sun, 24 May 2026 10:37:10 +0000 (10:37 +0000)]
accel/ethosu: fix arithmetic issues in dma_length()
dma_length() derives DMA region usage from command stream values and
updates region_size[]:
len = ((len + stride[0]) * size0 + stride[1]) * size1
region_size[region] = max(..., len + dma->offset)
Several arithmetic issues can corrupt the derived region size:
- signed stride values may underflow when added to len
- intermediate multiplications may overflow
- len + dma->offset may overflow during region_size updates
- dma_length() error returns were not validated by the caller
region_size[] is later used by ethosu_job.c to validate command stream
accesses against GEM buffer sizes. Arithmetic wraparound can therefore
under-report region usage and bypass the bounds validation.
Fix by validating signed additions, using overflow helpers for
multiplications and offset updates, and propagating dma_length()
failures to the caller.
Muhammad Bilal [Sat, 23 May 2026 21:07:53 +0000 (21:07 +0000)]
accel/ethosu: fix wrong weight index in NPU_SET_SCALE1_LENGTH on U85
On non-U65 hardware (e.g. U85), opcode 0x4093 is NPU_SET_WEIGHT2_LENGTH.
The BASE handler for the same opcode correctly assigns to
st.weight[2].base, but the LENGTH handler mistakenly assigns cmds[1]
to st.weight[1].length instead of st.weight[2].length.
This leaves weight[2].length at its initialised sentinel value of
0xffffffff and corrupts weight[1].length with the user-supplied value,
breaking the software bounds-check state for both weight buffers on U85.
Muhammad Bilal [Sat, 23 May 2026 21:07:52 +0000 (21:07 +0000)]
accel/ethosu: reject NPU_OP_RESIZE commands from userspace
NPU_OP_RESIZE is a U85-only command that the driver does not yet
implement. The existing WARN_ON(1) placeholder fires unconditionally
whenever userspace submits this command via DRM_IOCTL_ETHOSU_GEM_CREATE,
causing unbounded kernel log spam.
If panic_on_warn is set the kernel panics, giving any unprivileged user
with access to the DRM device a trivial denial-of-service primitive.
Replace the WARN_ON(1) with an explicit -EINVAL return so the ioctl
rejects the command before it reaches hardware.
Dave Jiang [Thu, 4 Jun 2026 18:01:53 +0000 (11:01 -0700)]
cxl/pci: Fix the incorrect check of pci_read_config_word() return
pci_read_config_word() returns PCIBIOS_* status on error which are
positive values. The check should be for non-zero values to indicate
error. Fix cxl_set_mem_enable() to check for non-zero return value
instead of negative value.
While fixing this, also convert the error to negative errno value when
returning on error path.
Fixes: 34e37b4c432c ("cxl/port: Enable HDM Capability after validating DVSEC Ranges") Reviewed-by: Richard Cheng <icheng@nvidia.com> Reviewed-by: Jonathan Cameron <jic23@kernel.org> Reviewed-by: Alison Schofield <alison.schofield@intel.com> Assisted-by: Claude:claude-opus-4-8 Link: https://patch.msgid.link/20260604180154.1925149-2-dave.jiang@intel.com Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Jakub Kicinski [Wed, 3 Jun 2026 21:08:10 +0000 (14:08 -0700)]
tools: ynl: try to avoid the very slow YAML loader
Turns out Python YAML defaults to a pure Python loader for YAML
files which is a lot slower than the C loader (using libyaml).
Try to use the C one whenever possible.
The avg time to run:
$ tools/net/ynl/pyynl/cli.py --family tc --no-schema
drops from 300+ ms to 115 ms with this change (40 samples).
We could drop the load time further to 85 ms if we "compiled"
the specs to JSON. Slightly tricky parts are that we don't
currently install the specs at all on make install, so it's
unclear where to put the conversion. Also JSON has questionable
support for comments and we need an SPDX line.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com> Reviewed-by: Nicolai Buchwitz <nb@tipi-net.de> Reviewed-by: Donald Hunter <donald.hunter@gmail.com> Link: https://patch.msgid.link/20260603210810.2636193-1-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Muhammad Bilal [Sat, 23 May 2026 19:51:59 +0000 (19:51 +0000)]
accel/ethosu: fix IFM region index out-of-bounds in command stream parser
NPU_SET_IFM_REGION extracts the region index with param & 0x7f, giving
a maximum value of 127. However region_size[] and output_region[] in
struct ethosu_validated_cmdstream_info are both sized to
NPU_BASEP_REGION_MAX (8), giving valid indices [0..7].
Every other region assignment in the same switch uses param & 0x7:
NPU_SET_OFM_REGION: st.ofm.region = param & 0x7;
NPU_SET_IFM2_REGION: st.ifm2.region = param & 0x7;
NPU_SET_WEIGHT_REGION: st.weight[0].region = param & 0x7;
NPU_SET_SCALE_REGION: st.scale[0].region = param & 0x7;
The 0x7f mask on IFM is inconsistent and appears to be a typo.
feat_matrix_length() and calc_sizes() use the region index directly
as an array subscript into the kzalloc'd info struct:
info->region_size[fm->region] = max(...);
A userspace caller supplying NPU_SET_IFM_REGION with param > 7 causes
a write up to 127*8 = 1016 bytes past the start of region_size[],
corrupting adjacent kernel heap data.
Fix by applying the same & 0x7 mask used by all other region
assignments.
Cross-merge networking fixes after downstream PR (net-7.1-rc7).
Silent conflicts:
net/wireless/nl80211.c cb9959ab5f99 ("wifi: cfg80211: enforce HE/EHT cap/oper consistency") a384ae969902 ("wifi: cfg80211: move AP HT/VHT/... operation to beacon info")
https://lore.kernel.org/aiGJDaHV4UlCexIQ@sirena.org.uk
Conflicts:
drivers/net/wireless/intel/iwlwifi/mld/ap.c a342c99cb70d ("wifi: iwlwifi: mld: honor BSS_CHANGED_BEACON_ENABLED") 9bf1b409afc7 ("wifi: iwlwifi: mld: send tx power constraints before link activation")
https://lore.kernel.org/ah2bfedhV45ZxMO8@sirena.org.uk
drivers/net/wireless/intel/iwlwifi/pcie/drv.c 093305d801fa ("wifi: iwlwifi: pcie: simplify the resume flow if fast resume is not used") e2323929a68a ("wifi: iwlwifi: pcie: add debug print for resume flow if powered off")
https://lore.kernel.org/ah2bfedhV45ZxMO8@sirena.org.uk
Ian Bridges [Mon, 1 Jun 2026 18:44:33 +0000 (13:44 -0500)]
ocfs2: fix out-of-bounds write in ocfs2_remove_refcount_extent
[BUG]
Unlinking a refcounted file whose refcount tree has leaf blocks
triggers a fortify panic due to an out-of-bounds write.
[CAUSE]
When the last leaf block is removed from a refcount tree,
ocfs2_remove_refcount_extent() converts the root back to leaf mode
with a bulk memset on &rb->rf_records. rf_records sits in an anonymous
union with rf_list. rf_list.l_tree_depth aliases rf_records.rl_count,
and is 0 for a single-level tree. With rl_count equal to 0, the memset
writes past the 16-byte declared size of rf_records, which the fortify
checker catches.
[FIX]
Replace the bulk memset on &rb->rf_records with a correctly-bounded
memset on rl_recs[] alone, after setting rl_count to the correct value.
Link: https://lore.kernel.org/ah3TESOsEO9j_JLU@dev Fixes: 2f26f58df041 ("ocfs2: annotate flexible array members with __counted_by_le()") Signed-off-by: Ian Bridges <icb@fastmail.org> Reported-by: syzbot+3ef989aae096b30f1663@syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=3ef989aae096b30f1663 Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com> Cc: Mark Fasheh <mark@fasheh.com> Cc: Joel Becker <jlbec@evilplan.org> Cc: Junxiao Bi <junxiao.bi@oracle.com> Cc: Changwei Ge <gechangwei@live.cn> Cc: Jun Piao <piaojun@huawei.com> Cc: Heming Zhao <heming.zhao@suse.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Joseph Qi [Mon, 1 Jun 2026 12:16:18 +0000 (20:16 +0800)]
ocfs2: fix race between ocfs2_control_install_private() and ocfs2_control_release()
Move atomic_inc(&ocfs2_control_opened) and the handshake state update
inside ocfs2_control_lock to close a race window where
ocfs2_control_release() can observe ocfs2_control_opened dropping to zero
(resetting ocfs2_control_this_node and running_proto) while
ocfs2_control_install_private() is about to bump the counter and mark the
connection valid.
Link: https://lore.kernel.org/20260601121618.1263346-1-joseph.qi@linux.alibaba.com Fixes: 3cfd4ab6b6b4 ("ocfs2: Add the local node id to the handshake.") Signed-off-by: Joseph Qi <joseph.qi@linux.alibaba.com> Reported-by: Ginger <ginger.jzllee@gmail.com> Cc: Mark Fasheh <mark@fasheh.com> Cc: Joel Becker <jlbec@evilplan.org> Cc: Junxiao Bi <junxiao.bi@oracle.com> Cc: Changwei Ge <gechangwei@live.cn> Cc: Jun Piao <piaojun@huawei.com> Cc: Heming Zhao <heming.zhao@suse.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Zhang Cen [Sun, 31 May 2026 04:47:14 +0000 (12:47 +0800)]
ocfs2/dlm: require a ref for locking_state debugfs open
debug_lockres_open() copies inode->i_private into struct debug_lockres and
debug_lockres_release() later drops that pointer with dlm_put(). That
only works if open successfully pins the struct dlm_ctxt.
Today open calls dlm_grab(dlm) but ignores its return value. Once the
last domain unregister has removed the context from dlm_domains,
dlm_grab() returns NULL, yet open still stores the raw pointer and returns
success. The later release path is outside the debugfs removal barrier,
so it can call dlm_put() after dlm_free_ctxt_mem() has freed the context.
KASAN reports this as a slab-use-after-free in dlm_put() called from
debug_lockres_release().
Fail the open when dlm_grab() cannot acquire the reference and unwind the
seq_file private state before returning. That keeps locking_state from
handing out a file descriptor whose release path does not own the
dlm_ctxt.
The buggy scenario involves two paths, with each column showing the order
within that path:
locking_state debugfs open: last domain unregister:
1. debug_lockres_open() reads 1. dlm_unregister_domain() calls
inode->i_private. dlm_complete_dlm_shutdown().
2. debug_lockres_open() calls 2. shutdown removes the dlm_ctxt from
dlm_grab(dlm) and gets NULL. dlm_domains.
3. open still stores the raw dlm 3. final teardown reaches
pointer in dl->dl_ctxt and dlm_free_ctxt_mem() and frees it.
returns success.
4. debug_lockres_release() later
calls dlm_put(dl->dl_ctxt).
Validation reproduced this kernel report:
KASAN slab-use-after-free in dlm_put+0x82/0x200
RIP: 0033:0x7f4d349bc9e0
The buggy address belongs to the object at ffff888103a3c000 which belongs
to the cache kmalloc-2k of size 2048
The buggy address is located 816 bytes inside of freed 2048-byte region
[ffff888103a3c000, ffff888103a3c800)
Write of size 4
Call trace:
dump_stack_lvl+0x66/0xa0 (?:?)
print_report+0xd0/0x630 (?:?)
dlm_put+0x82/0x200 (?:?)
srso_alias_return_thunk+0x5/0xfbef5 (?:?)
__virt_addr_valid+0x188/0x2f0 (?:?)
kasan_report+0xe4/0x120 (?:?)
kasan_check_range+0x105/0x1b0 (?:?)
debug_lockres_release+0x53/0x80 (fs/ocfs2/dlm/dlmdebug.c:587)
dlm_put+0x9/0x200 (?:?)
debug_lockres_release+0x5c/0x80 (fs/ocfs2/dlm/dlmdebug.c:587)
full_proxy_release+0x67/0x90 (?:?)
__fput+0x1df/0x4b0 (?:?)
do_raw_spin_lock+0x10f/0x1b0 (?:?)
fput_close_sync+0xd2/0x170 (?:?)
__x64_sys_close+0x55/0x90 (?:?)
do_syscall_64+0x10c/0x640 (arch/x86/entry/syscall_64.c:87)
irqentry_exit+0xac/0x6e0 (?:?)
entry_SYSCALL_64_after_hwframe+0x77/0x7f (?:?)
Freed by task stack:
kasan_save_stack+0x33/0x60 (?:?)
kasan_save_track+0x14/0x30 (?:?)
kasan_save_free_info+0x3b/0x60 (?:?)
__kasan_slab_free+0x5f/0x80 (?:?)
kfree+0x30f/0x580 (?:?)
dlm_put+0x1ce/0x200 (?:?)
dlm_unregister_domain+0xf6/0xb30 (?:?)
o2cb_cluster_disconnect+0x6b/0x90 (?:?)
ocfs2_cluster_disconnect+0x41/0x70 (?:?)
ocfs2_dlm_shutdown+0x1c4/0x220 (?:?)
ocfs2_dismount_volume+0x38a/0x550 (?:?)
generic_shutdown_super+0xc3/0x220 (?:?)
kill_block_super+0x29/0x60 (?:?)
deactivate_locked_super+0x66/0xe0 (?:?)
cleanup_mnt+0x13d/0x210 (?:?)
task_work_run+0xfa/0x170 (?:?)
exit_to_user_mode_loop+0xd6/0x430 (?:?)
do_syscall_64+0x3cb/0x640 (arch/x86/entry/syscall_64.c:87)
entry_SYSCALL_64_after_hwframe+0x77/0x7f (?:?)
Link: https://lore.kernel.org/20260531044714.1640172-1-rollkingzzc@gmail.com Fixes: 4e3d24ed1a12 ("ocfs2/dlm: Dumps the lockres' into a debugfs file") Assisted-by: Codex:gpt-5.5 Signed-off-by: Zhang Cen <rollkingzzc@gmail.com> Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com> Cc: Mark Fasheh <mark@fasheh.com> Cc: Joel Becker <jlbec@evilplan.org> Cc: Junxiao Bi <junxiao.bi@oracle.com> Cc: Changwei Ge <gechangwei@live.cn> Cc: Jun Piao <piaojun@huawei.com> Cc: Heming Zhao <heming.zhao@suse.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Zhang Cen [Thu, 28 May 2026 15:12:47 +0000 (23:12 +0800)]
ocfs2: reject FITRIM ranges shorter than a cluster
ocfs2_trim_mainbm() trims the global bitmap in cluster units, but its
too-short range validation only checks sb->s_blocksize.
On filesystems with a cluster size larger than the block size, a FITRIM
range that is at least one block but shorter than one cluster is accepted
and shifted down to len == 0. The later start + len - 1 and len -= ...
arithmetic then underflows and can drive trimming past the requested
range.
Reject ranges shorter than s_clustersize instead. That preserves the
existing -EINVAL behavior for requests that cannot discard even one
allocation unit and keeps zero-cluster trims out of the group walk.
Link: https://lore.kernel.org/20260528151247.361854-1-rollkingzzc@gmail.com Fixes: aa89762c5480 ("ocfs2: return EINVAL if the given range to discard is less than block size") Assisted-by: Codex:gpt-5.5 Signed-off-by: Zhang Cen <rollkingzzc@gmail.com> Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com> Cc: Mark Fasheh <mark@fasheh.com> Cc: Joel Becker <jlbec@evilplan.org> Cc: Junxiao Bi <junxiao.bi@oracle.com> Cc: Changwei Ge <gechangwei@live.cn> Cc: Jun Piao <piaojun@huawei.com> Cc: Heming Zhao <heming.zhao@suse.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Zhang Cen [Thu, 28 May 2026 15:12:30 +0000 (23:12 +0800)]
ocfs2: validate fast symlink target during inode read
ocfs2_validate_inode_block() already rejects several inconsistent
self-contained dinodes before they are exposed to the rest of the
filesystem. Fast symlinks need the same treatment.
A zero-cluster symlink is treated as a fast symlink and later read through
page_get_link() and ocfs2_fast_symlink_read_folio(). That path uses
strnlen() on the inline payload and then copies len + 1 bytes into the
folio. If a corrupt dinode stores an i_size that does not fit the inline
area or omits the terminating NUL at i_size, that copy reads past the end
of the inode block buffer.
Reject zero-cluster symlink dinodes whose i_size exceeds the inline
fast-symlink capacity or whose inline payload is not NUL-terminated
exactly at i_size when the inode block is validated. This keeps malformed
fast symlinks from reaching the read path.
Joseph Qi [Sun, 31 May 2026 13:16:45 +0000 (21:16 +0800)]
ocfs2: add journal NULL check in ocfs2_checkpoint_inode()
During unmount, ocfs2_journal_shutdown() frees the journal and sets
osb->journal to NULL. Later, when VFS evicts remaining cached inodes,
ocfs2_evict_inode() -> ocfs2_clear_inode() -> ocfs2_checkpoint_inode()
-> ocfs2_ci_fully_checkpointed() dereferences osb->journal, causing a
NULL pointer dereference.
Fix this by adding a NULL check for osb->journal in
ocfs2_checkpoint_inode(). If the journal is NULL, it has already been
fully flushed and destroyed during shutdown, so there is nothing to
checkpoint.
Link: https://lore.kernel.org/20260531131645.3650299-1-joseph.qi@linux.alibaba.com Reported-by: Farhad Alemi <farhad.alemi@berkeley.edu> Fixes: da5e7c87827e ("ocfs2: cleanup journal init and shutdown") Signed-off-by: Joseph Qi <joseph.qi@linux.alibaba.com> Tested-by: Farhad Alemi <farhad.alemi@berkeley.edu> Reviewed-by: Heming Zhao <heming.zhao@suse.com> Cc: Mark Fasheh <mark@fasheh.com> Cc: Joel Becker <jlbec@evilplan.org> Cc: Junxiao Bi <junxiao.bi@oracle.com> Cc: Changwei Ge <gechangwei@live.cn> Cc: Jun Piao <piaojun@huawei.com> Cc: <stable@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Dmitry Antipov [Fri, 29 May 2026 09:41:28 +0000 (12:41 +0300)]
ocfs2: fix buffer head management in ocfs2_read_blocks()
In ocfs2_read_blocks(), caller should't assume that buffer head returned
by 'sb_getblk()' is exclusively owned and so 'put_bh()' always drops
b_count from 1 to 0. If it is not so, buffer head remains on hold and
likely to be returned by the next call to 'sb_getblk()' unchanged - that
is, with BH_Uptodate bit set even if it has failed validation previously,
thus allowing to insert that buffer head into OCFS2 metadata cache and
submit it to upper layers. To avoid such a scenario, BH_Uptodate should
be cleared immediately after 'validate()' callback has detected some data
inconsistency.
Link: https://lore.kernel.org/20260529094128.494293-1-dmantipov@yandex.ru Fixes: cf76c78595ca ("ocfs2: don't put and assigning null to bh allocated outside") Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru> Reported-by: syzbot+caacd220635a9cc3bac9@syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=caacd220635a9cc3bac9 Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com> Cc: Mark Fasheh <mark@fasheh.com> Cc: Joel Becker <jlbec@evilplan.org> Cc: Junxiao Bi <junxiao.bi@oracle.com> Cc: Changwei Ge <gechangwei@live.cn> Cc: Jun Piao <piaojun@huawei.com> Cc: Heming Zhao <heming.zhao@suse.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
raid6_select_algo() allocates 8 pages for buffer that is used
as a scratch area for selection of the best algorithm.
This buffer can be allocated with kmalloc() as there's nothing special
about it to go directly to the page allocator.
kmalloc() provides a better API than ancient __get_free_pages().
kmalloc() does not require ugly casts and kfree() does not need to know the
size of the freed object.
There is no performance difference because kmalloc() redirects allocations
of such size to the page allocator.
Patch series "lib/raid: replace __get_free_pages() call with kmalloc()", v4.
The xor benchmark allocates 4 pages for a scratch buffer that is used
purely as a CPU-only XOR working area.
This buffer can be allocated with kmalloc() as there's nothing special
about it to go directly to the page allocator.
kmalloc() provides a better API than ancient __get_free_pages().
kmalloc() does not require ugly casts and kfree() does not need to know
the size of the freed object.
There is no performance difference because kmalloc() redirects allocations
of such size to the page allocator.
Replace __get_free_pages() call with kmalloc().
This patch (of 2):
The xor benchmark allocates 4 pages for a scratch buffer that is used
purely as a CPU-only XOR working area.
This buffer can be allocated with kmalloc() as there's nothing special
about it to go directly to the page allocator.
kmalloc() provides a better API than ancient __get_free_pages().
kmalloc() does not require ugly casts and kfree() does not need to know the
size of the freed object.
There is no performance difference because kmalloc() redirects allocations
of such size to the page allocator.
Zhang Cen [Sun, 24 May 2026 11:12:48 +0000 (19:12 +0800)]
ocfs2: reject oversized group bitmap descriptors
ocfs2_validate_gd_parent() only bounds bg_bits against the parent
allocator's chain geometry. A malicious descriptor can still claim a
bg_size/bg_bits pair that exceeds the bitmap bytes that physically fit in
the group descriptor block, so later bitmap scans and bit updates can run
past bg_bitmap.
Add a physical-cap check based on ocfs2_group_bitmap_size() for the parent
allocator type and reject descriptors whose bg_size or bg_bits exceed that
capacity. Keep the existing chain geometry check so both the on-disk
bitmap layout and the allocator metadata must agree before the descriptor
is used.
alloc_pages_bulk() is not guaranteed to return all requested pages in a
single call.
Call it repeatedly until all pages have been allocated or no more progress
is being made.
Link: https://lore.kernel.org/20260526-kunit_iov_iter-alloc_bulk-v2-1-24fbcd995c61@weissschuh.net Fixes: 2d71340ff1d4 ("iov_iter: Kunit tests for copying to/from an iterator") Signed-off-by: Thomas Weißschuh <linux@weissschuh.net> Cc: "Christian A. Ehrhardt" <lk@c--e.de> Reviewed-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
The problem here is that IS_ERR() is not inlined and dead code elimination
fails as a consequence.
The err.h helpers all turn into a trivial assignment of a bit mask and
should never result in a function call, so force them to always be inline.
This should generally result in better object code aside from avoiding
the link failure above.
Karl Mehltretter [Tue, 26 May 2026 11:47:15 +0000 (13:47 +0200)]
kcov: use WRITE_ONCE() for selftest mode stores
The KCOV selftest enables coverage by setting current->kcov_mode to
KCOV_MODE_TRACE_PC without installing a coverage area. If an interrupt
records coverage in that window, the access should fault and expose the
bug.
When building for QEMU raspi0 (Raspberry Pi Zero, ARMv6, CONFIG_CPU_V6K=y,
CONFIG_CURRENT_POINTER_IN_TPIDRURO=y) with GCC 13.3.0, the store that
enables the mode is removed. The generated kcov_init() code only stores
zero after the wait loop:
mrc 15, 0, r3, cr13, cr0, {3}
str r4, [r3, #2028]
where r4 is zero. There is no store of KCOV_MODE_TRACE_PC before the
loop, so the selftest reports success without exercising coverage.
Use WRITE_ONCE() for the temporary mode stores. With the same compiler
and config, kcov_init() contains the intended mode store:
Now that the KCOV selftest is actually executed, it may expose KCOV
instrumentation issues depending on the kernel config. That is expected
for a selftest that was intended to catch coverage from interrupt paths.
Link: https://lore.kernel.org/20260526114715.38280-1-kmehltretter@gmail.com Fixes: 6cd0dd934b03 ("kcov: Add interrupt handling self test") Assisted-by: Codex:gpt-5 Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com> Reviewed-by: Alexander Potapenko <glider@google.com> Cc: Andrey Konovalov <andreyknvl@gmail.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Kees Cook <kees@kernel.org> Cc: Marco Elver <elver@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: <stable@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Zhang Cen [Mon, 25 May 2026 04:17:26 +0000 (12:17 +0800)]
ocfs2: rebase copied fsdlm LVB pointers in locking_state
The locking_state debugfs iterator snapshots struct ocfs2_lock_res by
value under ocfs2_dlm_tracking_lock and later formats that copy in
ocfs2_dlm_seq_show(). That is fine for the inline fields, but the
userspace fsdlm stack stores the LVB through lksb_fsdlm.sb_lvbptr. Once
the iterator drops the tracking lock, a copied non-NULL sb_lvbptr still
points into the original lockres owner, so teardown can free that
container before the debugfs dump walks the raw LVB bytes.
Rebase the copied sb_lvbptr to the copied l_lksb before dumping the raw
LVB. The seq snapshot already carries the inline LVB storage reserved in
struct ocfs2_dlm_lksb, so the debugfs reader can dump the copied bytes
without borrowing the original lockres lifetime.
The buggy scenario involves two paths, with each column showing the order
within that path:
locking_state reader: lockres teardown:
1. ocfs2_dlm_seq_start()/next() 1. file release or another owner
copies struct ocfs2_lock_res teardown reaches
2. ocfs2_dlm_seq_show() formats ocfs2_lock_res_free()
the copied row 2. the lockres is removed from the
3. ocfs2_dlm_lvb() follows the tracking list
copied sb_lvbptr 3. the owner frees the original
lockres container
Validation reproduced this kernel report:
KASAN slab-use-after-free in ocfs2_dlm_seq_show+0x1bd/0x430
RIP: 0033:0x7f8ec4b1e29d
The buggy address belongs to the object at ffff88810a1e0800 which belongs
to the cache kmalloc-1k of size 1024
The buggy address is located 368 bytes inside of freed 1024-byte region
[ffff88810a1e0800, ffff88810a1e0c00)
Read of size 1
Call trace:
dump_stack_lvl+0x66/0xa0
print_report+0xce/0x630
ocfs2_dlm_seq_show+0x1bd/0x430 (fs/ocfs2/dlmglue.c:3137)
srso_alias_return_thunk+0x5/0xfbef5
__virt_addr_valid+0x19f/0x330
kasan_report+0xe0/0x110
seq_read_iter+0x29d/0x790
seq_read+0x20a/0x280
find_held_lock+0x2b/0x80
rcu_read_unlock+0x18/0x70
full_proxy_read+0x9e/0xd0
vfs_read+0x12c/0x590
ksys_read+0xd2/0x170
do_user_addr_fault+0x65a/0x890
do_syscall_64+0x115/0x6a0 (arch/x86/entry/syscall_64.c:87)
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Allocated by task stack:
kasan_save_stack+0x33/0x60
kasan_save_track+0x14/0x30
__kasan_kmalloc+0xaa/0xb0
ocfs2_file_open+0x13e/0x300
do_dentry_open+0x233/0x7f0
vfs_open+0x5a/0x1b0
path_openat+0x66d/0x1540
do_file_open+0x186/0x2b0
do_sys_openat2+0xce/0x150
__x64_sys_openat+0xd0/0x140
do_syscall_64+0x115/0x6a0 (arch/x86/entry/syscall_64.c:87)
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Freed by task stack:
kasan_save_stack+0x33/0x60
kasan_save_track+0x14/0x30
kasan_save_free_info+0x3b/0x60
__kasan_slab_free+0x5f/0x80
kfree+0x313/0x590
ocfs2_file_release+0x138/0x260
__fput+0x1df/0x4b0
fput_close_sync+0xd2/0x170
__x64_sys_close+0x55/0x90
do_syscall_64+0x115/0x6a0 (arch/x86/entry/syscall_64.c:87)
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Link: https://lore.kernel.org/20260525041726.4112882-1-rollkingzzc@gmail.com Fixes: cf4d8d75d8ab ("ocfs2: add fsdlm to stackglue") Assisted-by: Codex:gpt-5.5 Signed-off-by: Zhang Cen <rollkingzzc@gmail.com> Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com> Cc: Mark Fasheh <mark@fasheh.com> Cc: Joel Becker <jlbec@evilplan.org> Cc: Junxiao Bi <junxiao.bi@oracle.com> Cc: Changwei Ge <gechangwei@live.cn> Cc: Jun Piao <piaojun@huawei.com> Cc: Heming Zhao <heming.zhao@suse.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Brian Masney [Thu, 28 May 2026 13:45:10 +0000 (09:45 -0400)]
docs: mm: clarify that user_reserve_kbytes has no effect when overcommit_memory is set to 0 or 1
Looking at __vm_enough_memory() in mm/util.c, user_reserve_kbytes has no
effect when overcommit_memory is set to 0 or 1. The documentation for
overcommit_memory already references user_reserve_kbytes when the flag
is set to 2.
Let's go ahead and add a clarification to user_reserve_kbytes in vm.rst
that it has no effect when overcommit_memory is set to 0 or 1.
Link: https://lore.kernel.org/20260528-mm-clarify-docs-v1-1-aa88e83b4bfd@redhat.com Signed-off-by: Brian Masney <bmasney@redhat.com> Cc: David Hildenbrand <david@kernel.org> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Liam R. Howlett <liam@infradead.org> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Shuah Khan <skhan@linuxfoundation.org> Cc: Vlastimil Babka <vbabka@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Brian Masney [Thu, 28 May 2026 13:56:14 +0000 (09:56 -0400)]
MAINTAINERS: add vm.rst to memory management core
The vm.rst file is currently not listed in the MAINTAINERS file, so let's
go ahead and add to the MM core subsystem so that the maintainers are CCed
when changes to the documentation are proposed.
Link: https://lore.kernel.org/20260528-mm-vm-rst-maintainers-file-v1-1-306631c0a610@redhat.com Signed-off-by: Brian Masney <bmasney@redhat.com> Reviewed-by: Lorenzo Stoakes <ljs@kernel.org> Reviewed-by: Oscar Salvador (SUSE) <osalvador@kernel.org> Reviewed-by: Liam R. Howlett (Oracle) <liam@infradead.org> Reviewed-by: SeongJae Park <sj@kernel.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
zram: clear trailing bytes of compressed writeback pages
Patch series "zram: writeback fixes", v2.
Brian (privately) reported a "leak" of writeback bitmap in certain cases,
so that backing device can store less pages; and a theoretical data leak
in the trailing bytes of compressed writeback pages. Both issues are low
risk.
This patch (of 2):
When compressed writeback is available writtenback pages contain "garbage"
in PAGE_SIZE - obj_size trailing bytes. That "garbage" is, basically,
whatever data that page held before we got it for writeback. To get
advantage of it an attacker needs to be able to read from active backing
swap device, which is already catastrophic. Still, just in case, zero out
those trailing bytes before writeback to a backing device so that we only
store swap-ed out data there.
zram_writeback_slots() loop can terminate with valid reserved backing
device blk_idx. The problem is that cleanup code doesn't release that
reserved blk_idx before zram_writeback_slots() returns, which leads to
blk_idx leak (it becomes permanently busy and can not be used for actual
writeback.) This does not lead to any system instabilities, it only means
that we can writeback less pages. The scenario is hard to hit in practice
as it requires writeabck to race with modification (slot-free or
overwrite) of the final post-processing slot.
Release reserved but unused blk_idx before returning from
zram_writeback_slots().
Link: https://lore.kernel.org/20260526022754.2377730-2-senozhatsky@chromium.org Fixes: f405066a1f0db ("zram: introduce writeback bio batching") Signed-off-by: Sergey Senozhatsky <senozhatsky@chromium.org> Suggested-by: Brian Geffon <bgeffon@google.com> Cc: Jens Axboe <axboe@kernel.dk> Cc: Minchan Kim <minchan@kernel.org> Cc: Richard Chang <richardycc@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Shakeel Butt [Tue, 26 May 2026 03:39:31 +0000 (20:39 -0700)]
memcg: multi objcg charge support
Commit 01b9da291c49 ("mm: memcontrol: convert objcg to be per-memcg
per-node type") split a memcg's single obj_cgroup into one per NUMA node
so that reparenting LRU folios can take per-node lru locks. As a side
effect, the per-CPU obj_stock_pcp -- which caches exactly one cached_objcg
-- thrashes on workloads where threads of the same memcg run on different
NUMA nodes. The kernel test robot reported a 67.7% regression on
stress-ng.switch.ops_per_sec from this pattern.
Mirror the multi-slot pattern already used by memcg_stock_pcp: turn
nr_bytes and cached_objcg into NR_OBJ_STOCK-element arrays, scan all slots
on consume/refill/account, prefer empty slots when inserting, and evict a
slot round-robin only when full. With multiple slots a CPU can hold the
per-node objcg variants of one memcg plus a few siblings without ever
forcing a drain.
A single int8_t index records which slot the cached slab stats belong to;
the stats are flushed on slot or pgdat change. With NR_OBJ_STOCK = 5 the
layout (verified with pahole) is:
Shakeel Butt [Tue, 26 May 2026 03:39:30 +0000 (20:39 -0700)]
memcg: int16_t for cached slab stats
Currently struct obj_stock_pcp stores cached slab stats in 'int' which is
4 bytes per counter on 64-bit machines. Switch them to int16_t to shrink
the cached metadata.
The existing PAGE_SIZE flush in __account_obj_stock() bounds *bytes at
PAGE_SIZE on 4KiB and 16KiB page archs, well within int16_t. On 64KiB
pages PAGE_SIZE is well above S16_MAX so that flush never fires, and a
sufficiently long run of accumulations would overflow the cache. Add an
explicit S16_MAX guard before each add: when the next add would push
abs(*bytes) past S16_MAX, fold the cached value into @nr and flush
directly via mod_objcg_mlstate() before the accumulation.
Link: https://lore.kernel.org/20260526033931.1760588-4-shakeel.butt@linux.dev Fixes: 01b9da291c49 ("mm: memcontrol: convert objcg to be per-memcg per-node type") Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev> Tested-by: kernel test robot <oliver.sang@intel.com> Reviewed-by: Harry Yoo (Oracle) <harry@kernel.org> Acked-by: Qi Zheng <qi.zheng@linux.dev> Acked-by: Muchun Song <muchun.song@linux.dev> Cc: Alexandre Ghiti <alex@ghiti.fr> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Joshua Hahn <joshua.hahnjy@gmail.com> Cc: Michal Hocko <mhocko@kernel.org> Cc: Roman Gushchin <roman.gushchin@linux.dev> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Shakeel Butt [Tue, 26 May 2026 03:39:29 +0000 (20:39 -0700)]
memcg: uint16_t for nr_bytes in obj_stock_pcp
Currently struct obj_stock_pcp stores nr_bytes in an 'unsigned int' which
is 4 bytes on 64-bit machines. Switch the field to uint16_t to shrink the
per-CPU cache.
The kernel supports PAGE_SIZE_4KB, _8KB, _16KB, _32KB, _64KB and _256KB
(see HAVE_PAGE_SIZE_* in arch/Kconfig). After the PAGE_SIZE-aligned flush
in __refill_obj_stock(), the sub-page remainder fits in uint16_t up
through 64KiB pages where PAGE_SIZE - 1 == U16_MAX, but on 256KiB pages
PAGE_SIZE - 1 == 0x3FFFF exceeds U16_MAX. The accumulator also needs to
stay within uint16_t between page-aligned flushes on 64KiB pages where
PAGE_SIZE itself is U16_MAX + 1.
Accumulate the new total in an 'unsigned int' local, then on PAGE_SHIFT <=
16 flush whenever the accumulator would hit U16_MAX; together with the
existing allow_uncharge flush at PAGE_SIZE this keeps the uint16_t safe.
On configs with PAGE_SHIFT > 16 (PAGE_SIZE_256KB on hexagon and powerpc
44x, both 32-bit), uint16_t cannot represent the sub-page remainder.
Define obj_stock_bytes_t as 'unsigned int' on those archs so nr_bytes can
hold the full remainder and the normal page-boundary flush in
__refill_obj_stock() and the page extraction in drain_obj_stock() both
work correctly.
The single-cache-line layout target only applies to PAGE_SHIFT <= 16;
those archs are 32-bit embedded and not the optimization target.
Link: https://lore.kernel.org/20260526033931.1760588-3-shakeel.butt@linux.dev Fixes: 01b9da291c49 ("mm: memcontrol: convert objcg to be per-memcg per-node type") Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev> Tested-by: kernel test robot <oliver.sang@intel.com> Reviewed-by: Harry Yoo (Oracle) <harry@kernel.org> Acked-by: Qi Zheng <qi.zheng@linux.dev> Acked-by: Muchun Song <muchun.song@linux.dev> Cc: Alexandre Ghiti <alex@ghiti.fr> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Joshua Hahn <joshua.hahnjy@gmail.com> Cc: Michal Hocko <mhocko@kernel.org> Cc: Roman Gushchin <roman.gushchin@linux.dev> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Shakeel Butt [Tue, 26 May 2026 03:39:28 +0000 (20:39 -0700)]
memcg: store node_id instead of pglist_data pointer
Patch series "memcg: shrink obj_stock_pcp and cache multiple objcgs", v3.
Commit 01b9da291c49 ("mm: memcontrol: convert objcg to be per-memcg
per-node type") split a memcg's single obj_cgroup into one per NUMA node
so that reparenting LRU folios can take per-node lru locks. As a side
effect, the per-CPU obj_stock_pcp -- which caches a single cached_objcg
pointer -- thrashes on workloads where threads of the same memcg run on
different NUMA nodes. The kernel test robot reported a 67.7% regression
on stress-ng.switch.ops_per_sec from this pattern.
Commit d0211878ce06 ("memcg: cache obj_stock by memcg, not by objcg
pointer") landed as a temporary fix by treating sibling per-node objcgs as
equivalent for the cache lookup, intended to be reverted once per-node
kmem accounting is introduced. This series takes a more general approach:
cache multiple objcgs per CPU using the multi-slot pattern memcg_stock_pcp
already uses, so the per-node objcg variants of one memcg can all coexist
in the stock without ever forcing a drain. The temporary fix can then be
reverted.
To avoid increasing the per-CPU cache footprint, the first three patches
shrink the existing single-slot obj_stock_pcp fields. The final patch
converts cached_objcg and nr_bytes into NR_OBJ_STOCK=5 slot arrays and
reorders the struct so the entire consume/refill/account hot path fits
within a single 64-byte cache line on non-debug 64-bit builds (verified
with pahole).
This patch (of 4):
The struct obj_stock_pcp stores a pointer to pglist_data for the slab
stats cached on the cpu. On 64-bit machines, this costs 8 bytes. The
pointer is not strictly required: NODE_DATA() can recover it from the node
id. Replace cached_pgdat with int16_t node_id and use NUMA_NO_NODE as the
"no stats cached" sentinel.
At the moment all the archs limit MAX_NUMNODES to 1024 so int16_t is
plenty; a BUILD_BUG_ON() makes sure we notice if that ever changes.
Link: https://lore.kernel.org/20260526033931.1760588-1-shakeel.butt@linux.dev Link: https://lore.kernel.org/20260526033931.1760588-2-shakeel.butt@linux.dev Fixes: 01b9da291c49 ("mm: memcontrol: convert objcg to be per-memcg per-node type") Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev> Tested-by: kernel test robot <oliver.sang@intel.com> Acked-by: Muchun Song <muchun.song@linux.dev> Reviewed-by: Harry Yoo (Oracle) <harry@kernel.org> Acked-by: Qi Zheng <qi.zheng@linux.dev> Cc: Alexandre Ghiti <alex@ghiti.fr> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Joshua Hahn <joshua.hahnjy@gmail.com> Cc: Michal Hocko <mhocko@kernel.org> Cc: Roman Gushchin <roman.gushchin@linux.dev> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
selftests/memfd: fix -Wmaybe-uninitialized warning in memfd_test
Patch series "selftests/memfd: fix compilation warnings".
This patchset fixes warnings about unused but initialized variables, and
unused dummy buffer passed to pwrite() syscall in the tests.
This patch (of 2):
memfd_test.c: In function 'mfd_fail_grow_write.part.0':
memfd_test.c:685:13: warning: '<unknown>' may be used uninitialized
[-Wmaybe-uninitialized]
685 | l = pwrite(fd, buf, mfd_def_size * 8, 0);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
pwrite() is declared with attribute 'access (read_only, 2, 3)', so GCC
knows it reads from the buffer. malloc() returns uninitialized memory,
hence the warning. Use calloc() to zero-initialize the buffer. The
actual contents don't matter here since the test verifies that pwrite()
fails on a sealed memfd.
Ran Xiaokai [Mon, 25 May 2026 10:27:00 +0000 (10:27 +0000)]
mm: shmem: refactor thpsize_shmem_enabled_show() with helper arrays
Replace the hardcoded if/else chain of test_bit() calls and string
literals in thpsize_shmem_enabled_show() with a loop over
huge_shmem_orders_by_mode[] and huge_shmem_enabled_mode_strings[] arrays.
This makes thpsize_shmem_enabled_show() consistent with
thpsize_shmem_enabled_store() and eliminates duplicated mode name strings.
Link: https://lore.kernel.org/20260525102700.68707-3-ranxiaokai627@163.com Signed-off-by: Ran Xiaokai <ran.xiaokai@zte.com.cn> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Reviewed-by: Barry Song <baohua@kernel.org> Reviewed-by: Breno Leitao <leitao@debian.org> Reviewed-by: Lorenzo Stoakes <ljs@kernel.org> Tested-by: Baolin Wang <baolin.wang@linux.alibaba.com> Tested-by: Lance Yang <lance.yang@linux.dev> Acked-by: David Hildenbrand (arm) <david@kernel.org> Cc: Dev Jain <dev.jain@arm.com> Cc: Hugh Dickins <hughd@google.com> Cc: Liam R. Howlett <liam@infradead.org> Cc: Nico Pache <npache@redhat.com> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Zi Yan <ziy@nvidia.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Ran Xiaokai [Mon, 25 May 2026 10:26:59 +0000 (10:26 +0000)]
mm: shmem: refactor thpsize_shmem_enabled_store() with sysfs_match_string()
Patch series "refactors thpsize_shmem_enabled_store() and
thpsize_shmem_enabled_show()", v4.
This patch (of 2):
Inspired by commit 82d9ff648c6c ("mm: huge_memory: refactor
anon_enabled_store() with set_anon_enabled_mode()"), refactor
thpsize_shmem_enabled_store() using sysfs_match_string(). This eliminates
the duplicated spin_lock/unlock(), set/clear_bit(), calls across all
branches, reducing code duplication.
Behavioral change:
Call start_stop_khugepaged() only when the mode actually changes.
If unchanged, call set_recommended_min_free_kbytes() to preserve
legacy watermark behavior. This avoids unnecessary khugepaged restarts.
Tested with selftests ./run_kselftest.sh -t mm:ksft_thp.sh,
all test cases passed.
Link: https://lore.kernel.org/20260525102700.68707-1-ranxiaokai627@163.com Link: https://lore.kernel.org/20260525102700.68707-2-ranxiaokai627@163.com Signed-off-by: Ran Xiaokai <ran.xiaokai@zte.com.cn> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Reviewed-by: Barry Song <baohua@kernel.org> Tested-by: Baolin Wang <baolin.wang@linux.alibaba.com> Tested-by: Lance Yang <lance.yang@linux.dev> Acked-by: David Hildenbrand (arm) <david@kernel.org> Reviewed-by: Lorenzo Stoakes <ljs@kernel.org> Cc: Breno Leitao <leitao@debian.org> Cc: Dev Jain <dev.jain@arm.com> Cc: Hugh Dickins <hughd@google.com> Cc: Liam R. Howlett <liam@infradead.org> Cc: Nico Pache <npache@redhat.com> Cc: Ran Xiaokai <ran.xiaokai@zte.com.cn> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Zi Yan <ziy@nvidia.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Usama Arif [Mon, 25 May 2026 14:57:51 +0000 (07:57 -0700)]
mm: make mmap_miss accounting symmetric for VM_SEQ_READ
do_sync_mmap_readahead() skips both the mmap_miss increment and the
MMAP_LOTSAMISS check for VM_SEQ_READ mappings, since sequential access is
non-speculative and should always read ahead. The two decrement sites in
do_async_mmap_readahead() and filemap_map_pages() do not mirror this skip,
so concurrent faults on a VM_SEQ_READ mapping can still drive
ra->mmap_miss down to zero through the decrement paths even though nothing
in the sync path ever increments it. The counter itself is per-file
(file->f_ra.mmap_miss), so it can be moved by any VMA mapping the file,
not just the one currently faulting.
Skip the decrement for VM_SEQ_READ in both decrement sites so the counter
only moves for mappings that also participate in the increment side. No
functional change for VM_SEQ_READ users, since the increment-side gate
already prevents the counter from being consulted on their behalf, but it
stops a VM_SEQ_READ mapping from biasing the counter for other mappings of
the same file.
Link: https://lore.kernel.org/20260525145751.2671248-1-usama.arif@linux.dev Signed-off-by: Usama Arif <usama.arif@linux.dev> Closes: https://lore.kernel.org/all/8edc8cd0-f65c-4456-9b3f-362e744c9a96@linux.dev/ Reviewed-by: William Kucharski <william.kucharski@linux.dev> Reviewed-by: Jan Kara <jack@suse.cz> Cc: David Hildenbrand <david@kernel.org> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Matthew Wilcox (Oracle) <willy@infradead.org> Cc: Shakeel Butt <shakeel.butt@linux.dev> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Add tearing tests for /proc/pid/smaps file. New tests reuse the same
logic as with maps file but skipping all the data except for the VMA
addresses, which are the only part relevant for the tearing tests. Skip
PROCMAP_QUERY parts of the tests because smaps does not implement that
ioctl.
Link: https://lore.kernel.org/20260426062718.1238437-4-surenb@google.com Signed-off-by: Suren Baghdasaryan <surenb@google.com> Reviewed-by: Liam R. Howlett <liam@infradead.org> Cc: Jann Horn <jannh@google.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Matthew Wilcox (Oracle) <willy@infradead.org> Cc: "Paul E . McKenney" <paulmck@kernel.org> Cc: Pedro Falcato <pfalcato@suse.de> Cc: Shuah Khan <shuah@kernel.org> Cc: Wei Yang <richard.weiyang@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
selftests/proc: ensure the test is performed at the right page boundary
When running tearing tests we need to ensure the pages we use include VMAs
that were mapped by the child process for this test. Currently we always
use the first two pages, checking VMAs at their boundaries and this works,
however once we add tests for /proc/pid/smaps, the first two pages might
not contain the VMAs that child modifies. Locate the page that contains
the first VMA mapped by the child and use that and the next page for the
test.
Link: https://lore.kernel.org/20260426062718.1238437-3-surenb@google.com Signed-off-by: Suren Baghdasaryan <surenb@google.com> Reviewed-by: Liam R. Howlett <liam@infradead.org> Cc: Jann Horn <jannh@google.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Matthew Wilcox (Oracle) <willy@infradead.org> Cc: "Paul E . McKenney" <paulmck@kernel.org> Cc: Pedro Falcato <pfalcato@suse.de> Cc: Shuah Khan <shuah@kernel.org> Cc: Wei Yang <richard.weiyang@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
fs/proc/task_mmu: read proc/pid/{smaps|numa_maps} under per-vma lock
Patch series "use vma locks for proc/pid/{smaps|numa_maps} reads", v2.
Use per-vma locks when reading /proc/pid/smaps and /proc/pid/numa_maps
similar to /proc/pid/maps to reduce contention on central mmap_lock. One
major difference between maps and smaps/numa_maps reading is that the
latter executes page table walk which can't be done under RCU due to a
possibility of sleeping. Therefore we drop RCU read lock before this walk
while keeping the VMA locked. After the walk we retake RCU read lock,
reset VMA iterator and proceed with the next VMA.
The last two patches extend /proc/pid/maps test to cover /proc/pid/smaps
reading during concurrent address space modification.
This patch (of 3):
proc/pid/{smaps|numa_maps} can be read using the combination of RCU and
VMA read locks, similar to proc/pid/maps. RCU is required to safely
traverse the VMA tree and VMA lock stabilizes the VMA being processed and
the pagetable walk.
Link: https://lore.kernel.org/20260426062718.1238437-1-surenb@google.com Link: https://lore.kernel.org/20260426062718.1238437-2-surenb@google.com Signed-off-by: Suren Baghdasaryan <surenb@google.com> Reviewed-by: Liam R. Howlett <liam@infradead.org> Reviewed-by: Lorenzo Stoakes <ljs@kernel.org> Cc: Jann Horn <jannh@google.com> Cc: Matthew Wilcox (Oracle) <willy@infradead.org> Cc: "Paul E . McKenney" <paulmck@kernel.org> Cc: Pedro Falcato <pfalcato@suse.de> Cc: Shuah Khan <shuah@kernel.org> Cc: Wei Yang <richard.weiyang@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Kairui Song [Mon, 27 Apr 2026 18:07:06 +0000 (02:07 +0800)]
mm/vmscan: unify writeback reclaim statistic and throttling
Currently MGLRU and non-MGLRU handle the reclaim statistic and writeback
handling very differently, especially throttling. Basically MGLRU just
ignored the throttling part.
Let's just unify this part, use a helper to deduplicate the code so both
setups will share the same behavior.
Kairui Song [Mon, 27 Apr 2026 18:07:03 +0000 (02:07 +0800)]
mm/mglru: remove no longer used reclaim argument for folio protection
Now dirty reclaim folios are handled after isolation, not before, since
dirty reactivation must take the folio off LRU first, and that helps to
unify the dirty handling logic.
So this argument is no longer needed. Just remove it.
Link: https://lore.kernel.org/20260428-mglru-reclaim-v7-12-02fabb92dc43@tencent.com Signed-off-by: Kairui Song <kasong@tencent.com> Reviewed-by: Axel Rasmussen <axelrasmussen@google.com> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Barry Song <baohua@kernel.org> Cc: Chen Ridong <chenridong@huaweicloud.com> Cc: Chris Li <chrisl@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: David Stevens <stevensd@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Kalesh Singh <kaleshsingh@google.com> Cc: Leno Hou <lenohou@gmail.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vernon Yang <vernon2gm@gmail.com> Cc: Wei Xu <weixugc@google.com> Cc: Yafang <laoar.shao@gmail.com> Cc: Yuanchu Xie <yuanchu@google.com> Cc: Yu Zhao <yuzhao@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Kairui Song [Mon, 27 Apr 2026 18:07:02 +0000 (02:07 +0800)]
mm/mglru: simplify and improve dirty writeback handling
Right now the flusher wakeup mechanism for MGLRU is less responsive and
unlikely to trigger compared to classical LRU. The classical LRU wakes
the flusher if one batch of folios passed to shrink_folio_list is
unevictable due to under writeback. MGLRU instead check and handle this
after the whole reclaim loop is done.
We previously even saw OOM problems due to passive flusher, which were
fixed but still not perfect [1].
We have just unified the dirty folio counting and activation routine, now
just move the dirty flush into the loop right after shrink_folio_list.
This improves the performance a lot for workloads involving heavy
writeback and prepares for throttling too.
Test with YCSB workloadb showed a major performance improvement:
The performance is a lot better with significantly lower refault. We also
observed similar or higher performance gain for other real-world
workloads.
We were concerned that the dirty flush could cause more wear for SSD: that
should not be the problem here, since the wakeup condition is when the
dirty folios have been pushed to the tail of LRU, which indicates that
memory pressure is so high that writeback is blocking the workload
already.
Link: https://lore.kernel.org/20260428-mglru-reclaim-v7-11-02fabb92dc43@tencent.com Signed-off-by: Kairui Song <kasong@tencent.com> Reviewed-by: Axel Rasmussen <axelrasmussen@google.com> Link: https://lore.kernel.org/linux-mm/20241026115714.1437435-1-jingxiangzeng.cas@gmail.com/ Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Barry Song <baohua@kernel.org> Cc: Chen Ridong <chenridong@huaweicloud.com> Cc: Chris Li <chrisl@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: David Stevens <stevensd@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Kalesh Singh <kaleshsingh@google.com> Cc: Leno Hou <lenohou@gmail.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vernon Yang <vernon2gm@gmail.com> Cc: Wei Xu <weixugc@google.com> Cc: Yafang <laoar.shao@gmail.com> Cc: Yuanchu Xie <yuanchu@google.com> Cc: Yu Zhao <yuzhao@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Kairui Song [Mon, 27 Apr 2026 18:07:01 +0000 (02:07 +0800)]
mm/mglru: use the common routine for dirty/writeback reactivation
Currently MGLRU will move the dirty writeback folios to the second oldest
gen instead of reactivate them like the classical LRU. This might help to
reduce the LRU contention as it skipped the isolation. But as a result we
will see these folios at the LRU tail more frequently leading to
inefficient reclaim.
Besides, the dirty / writeback check after isolation in shrink_folio_list
is more accurate and covers more cases. So instead, just drop the special
handling for dirty writeback, use the common routine and re-activate it
like the classical LRU.
This should in theory improve the scan efficiency. These folios will be
rotated back to LRU tail once writeback is done so there is no risk of
hotness inversion. And now each reclaim loop will have a higher success
rate. This also prepares for unifying the writeback and throttling
mechanism with classical LRU, we keep these folios far from tail so
detecting the tail batch will have a similar pattern with classical LRU.
The micro optimization that avoids LRU contention by skipping the
isolation is gone, which should be fine. Compared to IO and writeback
cost, the isolation overhead is trivial.
And using the common routine also keeps the folio's referenced bits (tier
bits), which could improve metrics in the long term. Also no more need to
clean reclaim bit as the common routine will make use of it.
Note the common routine updates a few throttling and writeback counters,
which are not used, and never have been for the MGLRU case. We will start
making use of these in later commits.
Link: https://lore.kernel.org/20260428-mglru-reclaim-v7-10-02fabb92dc43@tencent.com Signed-off-by: Kairui Song <kasong@tencent.com> Reviewed-by: Axel Rasmussen <axelrasmussen@google.com> Reviewed-by: Barry Song <baohua@kernel.org> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Chen Ridong <chenridong@huaweicloud.com> Cc: Chris Li <chrisl@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: David Stevens <stevensd@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Kalesh Singh <kaleshsingh@google.com> Cc: Leno Hou <lenohou@gmail.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vernon Yang <vernon2gm@gmail.com> Cc: Wei Xu <weixugc@google.com> Cc: Yafang <laoar.shao@gmail.com> Cc: Yuanchu Xie <yuanchu@google.com> Cc: Yu Zhao <yuzhao@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Kairui Song [Mon, 27 Apr 2026 18:07:00 +0000 (02:07 +0800)]
mm/mglru: remove redundant swap constrained check upon isolation
Remove the swap-constrained early reject check upon isolation. This check
is a micro optimization when swap IO is not allowed, so folios are
rejected early. But it is redundant and overly broad since
shrink_folio_list() already handles all these cases with proper
granularity.
Notably, this check wrongly rejected lazyfree folios, and it doesn't cover
all rejection cases. shrink_folio_list() uses may_enter_fs(), which
distinguishes non-SWP_FS_OPS devices from filesystem-backed swap and does
all the checks after folio is locked, so flags like swap cache are stable.
This check also covers dirty file folios, which are not a problem now
since sort_folio() already bumps dirty file folios to the next generation,
but causes trouble for unifying dirty folio writeback handling.
And there should be no performance impact from removing it. We may have
lost a micro optimization, but unblocked lazyfree reclaim for NOIO
contexts, which is not a common case in the first place.
Link: https://lore.kernel.org/20260428-mglru-reclaim-v7-9-02fabb92dc43@tencent.com Signed-off-by: Kairui Song <kasong@tencent.com> Reviewed-by: Axel Rasmussen <axelrasmussen@google.com> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Reviewed-by: Chen Ridong <chenridong@huaweicloud.com> Reviewed-by: Barry Song <baohua@kernel.org> Cc: Chris Li <chrisl@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: David Stevens <stevensd@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Kalesh Singh <kaleshsingh@google.com> Cc: Leno Hou <lenohou@gmail.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vernon Yang <vernon2gm@gmail.com> Cc: Wei Xu <weixugc@google.com> Cc: Yafang <laoar.shao@gmail.com> Cc: Yuanchu Xie <yuanchu@google.com> Cc: Yu Zhao <yuzhao@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Kairui Song [Mon, 27 Apr 2026 18:06:59 +0000 (02:06 +0800)]
mm/mglru: don't abort scan immediately right after aging
Right now, if eviction triggers aging, the reclaimer will abort. This is
not the optimal strategy for several reasons.
Aborting the reclaim early wastes a reclaim cycle when under pressure, and
for concurrent reclaim, if the LRU is under aging, all concurrent
reclaimers might fail. And if the age has just finished, new cold folios
exposed by the aging are not reclaimed until the next reclaim iteration.
What's more, the current aging trigger is quite lenient, having 3 gens
with a reclaim priority lower than default will trigger aging, and blocks
reclaiming from one memcg. This wastes reclaim retry cycles easily. And
in the worst case, if the reclaim is making slower progress and all
following attempts fail due to being blocked by aging, it triggers
unexpected early OOM.
And if a lruvec requires aging, it doesn't mean it's hot. Instead, the
lruvec could be idle for quite a while, and hence it might contain lots of
cold folios to be reclaimed.
While it's helpful to rotate memcg LRU after aging for global reclaim, as
global reclaim fairness is coupled with the rotation in shrink_many, memcg
fairness is instead handled by cgroup iteration in shrink_node_memcgs.
So, for memcg level pressure, this abort is not the key part for keeping
the fairness. And in most cases, there is no need to age, and fairness
must be achieved by upper-level reclaim control.
So instead, just keep the scanning going unless one whole batch of folios
failed to be isolated or enough folios have been scanned, which is
triggered by evict_folios returning 0. And only abort for global reclaim
after one batch, so when there are fewer memcgs, progress is still made,
and the fairness mechanism described above still works fine.
And in most cases, the one more batch attempt for global reclaim might
just be enough to satisfy what the reclaimer needs, hence improving global
reclaim performance by reducing reclaim retry cycles.
Rotation is still there after the reclaim is done, which still follows the
comment in mmzone.h. And fairness still looking good.
Link: https://lore.kernel.org/20260428-mglru-reclaim-v7-8-02fabb92dc43@tencent.com Signed-off-by: Kairui Song <kasong@tencent.com> Reviewed-by: Axel Rasmussen <axelrasmussen@google.com> Reviewed-by: Chen Ridong <chenridong@huaweicloud.com> Reviewed-by: Barry Song <baohua@kernel.org> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Chris Li <chrisl@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: David Stevens <stevensd@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Kalesh Singh <kaleshsingh@google.com> Cc: Leno Hou <lenohou@gmail.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vernon Yang <vernon2gm@gmail.com> Cc: Wei Xu <weixugc@google.com> Cc: Yafang <laoar.shao@gmail.com> Cc: Yuanchu Xie <yuanchu@google.com> Cc: Yu Zhao <yuzhao@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Kairui Song [Mon, 27 Apr 2026 18:06:58 +0000 (02:06 +0800)]
mm/mglru: use a smaller batch for reclaim
With a fixed number to reclaim calculated at the beginning, making each
following step smaller should reduce the lock contention and avoid
over-aggressive reclaim of folios, as it will abort earlier when the
number of folios to be reclaimed is reached.
Link: https://lore.kernel.org/20260428-mglru-reclaim-v7-7-02fabb92dc43@tencent.com Signed-off-by: Kairui Song <kasong@tencent.com> Reviewed-by: Axel Rasmussen <axelrasmussen@google.com> Reviewed-by: Chen Ridong <chenridong@huaweicloud.com> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Reviewed-by: Barry Song <baohua@kernel.org> Cc: Chris Li <chrisl@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: David Stevens <stevensd@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Kalesh Singh <kaleshsingh@google.com> Cc: Leno Hou <lenohou@gmail.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vernon Yang <vernon2gm@gmail.com> Cc: Wei Xu <weixugc@google.com> Cc: Yafang <laoar.shao@gmail.com> Cc: Yuanchu Xie <yuanchu@google.com> Cc: Yu Zhao <yuzhao@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
mm/mglru: avoid reclaim type fall back when isolation makes no progress
While isolation makes no progress in scan_folios(), we quickly fall back
to the other type in isolate_folios(). This is incorrect, as the current
type may still have sufficient folios. Falling back can undermine the
positive_ctrl_err() result from get_type_to_scan(), which is derived from
swappiness.
So just continue scanning this type for another round.
Worth noting if the cold generations are all reclaimed, scan will no
longer make any progress either, which may undermine the swappiness again.
This is not a new issue and hence better be fixed later [1].
Link: https://lore.kernel.org/linux-mm/CAGsJ_4zjdOYEtuO6gNjABm7NDxW0skzBFNRNee-k2D6VwsYEQA@mail.gmail.com/ Link: https://lore.kernel.org/20260428-mglru-reclaim-v7-6-02fabb92dc43@tencent.com Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org> Signed-off-by: Kairui Song <kasong@tencent.com> Reviewed-by: Kairui Song <kasong@tencent.com> Cc: Axel Rasmussen <axelrasmussen@google.com> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Chen Ridong <chenridong@huaweicloud.com> Cc: Chris Li <chrisl@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: David Stevens <stevensd@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Kalesh Singh <kaleshsingh@google.com> Cc: Leno Hou <lenohou@gmail.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vernon Yang <vernon2gm@gmail.com> Cc: Wei Xu <weixugc@google.com> Cc: Yafang <laoar.shao@gmail.com> Cc: Yuanchu Xie <yuanchu@google.com> Cc: Yu Zhao <yuzhao@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Kairui Song [Mon, 27 Apr 2026 18:06:56 +0000 (02:06 +0800)]
mm/mglru: scan and count the exact number of folios
Make the scan helpers return the exact number of folios being scanned or
isolated. Since the reclaim loop now has a natural scan budget that
controls the scan progress, returning the scan number and consuming the
budget makes the scan more accurate and easier to follow.
The number of scanned folios for each iteration is always larger than 0,
unless the reclaim must stop for a forced aging, so there is no more need
for any special handling when there is no progress made:
- `return isolated || !remaining ? scanned : 0` in scan_folios: both
the function and the call now just return the exact scan count, combined
with the scan budget introduced in the previous commit to avoid livelock
or under scan.
- `scanned += try_to_inc_min_seq` in evict_folios: adding a bool as a
scan count was kind of confusing and no longer needed, as scan number
should never be zero as long as there are still evictable gens. We may
encounter a empty old gen that returns 0 scan count, to avoid that, do a
try_to_inc_min_seq before toisolation which have slight to none overhead
in most cases.
- `evictable_min_seq + MIN_NR_GENS > max_seq` guard in evict_folios: the
per-type get_nr_gens == MIN_NR_GENS check in scan_folios naturally
returns 0 when only two gens remain and breaks the loop.
Also change try_to_inc_min_seq to return void, as its return value is no
longer used by any caller. Call it before isolate_folios to flush any
empty gens left by external folio freeing, and again after isolate_folios
when scanning moved or protected folios may have emptied the oldest gen.
The scan still stops if only two gens are left, as the scan number will be
zero. This matches the previous behavior. This forced gen protection may
be removed or softened later to improve reclaim further.
Link: https://lore.kernel.org/20260428-mglru-reclaim-v7-5-02fabb92dc43@tencent.com Signed-off-by: Kairui Song <kasong@tencent.com> Reviewed-by: Axel Rasmussen <axelrasmussen@google.com> Reviewed-by: Chen Ridong <chenridong@huaweicloud.com> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Barry Song <baohua@kernel.org> Cc: Chris Li <chrisl@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: David Stevens <stevensd@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Kalesh Singh <kaleshsingh@google.com> Cc: Leno Hou <lenohou@gmail.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vernon Yang <vernon2gm@gmail.com> Cc: Wei Xu <weixugc@google.com> Cc: Yafang <laoar.shao@gmail.com> Cc: Yuanchu Xie <yuanchu@google.com> Cc: Yu Zhao <yuzhao@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Kairui Song [Mon, 27 Apr 2026 18:06:55 +0000 (02:06 +0800)]
mm/mglru: restructure the reclaim loop
The current loop will calculate the scan number on each iteration. The
number of folios to scan is based on the LRU length, with some unclear
behaviors, e.g, the scan number is only shifted by reclaim priority when
aging is not needed or when at the default priority, and it couples the
number calculation with aging and rotation.
Adjust, simplify it, and decouple aging and rotation. Just calculate the
scan number for once at the beginning of the reclaim, always respect the
reclaim priority, and make the aging and rotation more explicit.
This slightly changes how aging and offline memcg reclaim works:
Previously, aging was skipped at DEF_PRIORITY even when eviction was no
longer possible, so the reclaimer wasted an iteration until the priority
escalated. Now aging runs immediately whenever it is needed to make
progress; the DEF_PRIORITY skip only applies when eviction is still
viable. This may avoid wasted iterations that over-reclaim slab and break
reclaim balance in multi-cgroup setups.
Similar for offline memcg. Previously, offline memcg wouldn't be aged
unless it didn't have any evictable folios. Now, we might age it if it
has only 3 generations, which should be fine. On one hand, offline memcg
might still hold long-term folios, and in fact, a long-existing offline
memcg must be pinned by some long-term folios like shmem. These folios
might be used by other memcg, so aging them as ordinary memcg seems
correct. Besides, aging enables further reclaim of an offlined memcg,
which will certainly happen if we keep shrinking it. And offline memcg
might soon be no longer an issue with reparenting.
Overall, the memcg LRU rotation, as described in mmzone.h, remains the
same.
Note that because the scan budget is now pinned at loop entry, tiny lruvec
might skip this reclaim pass, also skipping aging, which could be
beneficial as aging is not helpful since it will still be un-reclaimable
after aging. Reclaim will go on as usual once priority escalates.
Link: https://lore.kernel.org/20260428-mglru-reclaim-v7-4-02fabb92dc43@tencent.com Signed-off-by: Kairui Song <kasong@tencent.com> Reviewed-by: Axel Rasmussen <axelrasmussen@google.com> Acked-by: Shakeel Butt <shakeel.butt@linux.dev> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Barry Song <baohua@kernel.org> Cc: Chen Ridong <chenridong@huaweicloud.com> Cc: Chris Li <chrisl@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: David Stevens <stevensd@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Kalesh Singh <kaleshsingh@google.com> Cc: Leno Hou <lenohou@gmail.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vernon Yang <vernon2gm@gmail.com> Cc: Wei Xu <weixugc@google.com> Cc: Yafang <laoar.shao@gmail.com> Cc: Yuanchu Xie <yuanchu@google.com> Cc: Yu Zhao <yuzhao@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Kairui Song [Mon, 27 Apr 2026 18:06:54 +0000 (02:06 +0800)]
mm/mglru: relocate the LRU scan batch limit to callers
Same as active / inactive LRU, MGLRU isolates and scans folios in batches.
The batch split is done hidden deep in the helper, which makes the code
harder to follow. The helper's arguments are also confusing since callers
usually request more folios than the batch size, so the helper almost
never processes the full requested amount.
Move the batch splitting into the top loop to make it cleaner, there
should be no behavior change.
Link: https://lore.kernel.org/20260428-mglru-reclaim-v7-3-02fabb92dc43@tencent.com Signed-off-by: Kairui Song <kasong@tencent.com> Reviewed-by: Axel Rasmussen <axelrasmussen@google.com> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Reviewed-by: Barry Song <baohua@kernel.org> Reviewed-by: Chen Ridong <chenridong@huaweicloud.com> Acked-by: Shakeel Butt <shakeel.butt@linux.dev> Cc: Chris Li <chrisl@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: David Stevens <stevensd@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Kalesh Singh <kaleshsingh@google.com> Cc: Leno Hou <lenohou@gmail.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vernon Yang <vernon2gm@gmail.com> Cc: Wei Xu <weixugc@google.com> Cc: Yafang <laoar.shao@gmail.com> Cc: Yuanchu Xie <yuanchu@google.com> Cc: Yu Zhao <yuzhao@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Kairui Song [Mon, 27 Apr 2026 18:06:52 +0000 (02:06 +0800)]
mm/mglru: consolidate common code for retrieving evictable size
Patch series "mm/mglru: improve reclaim loop and dirty folio", v7.
This series cleans up and slightly improves MGLRU's reclaim loop and dirty
writeback handling. As a result, we can see an up to ~30% increase in
some workloads like MongoDB with YCSB and a huge decrease in file refault,
no swap involved. Other common benchmarks have no regression, and LOC is
reduced, with less unexpected OOM, too.
Some of the problems were found in our production environment, and others
were mostly exposed while stress testing during the development of the
LSM/MM/BPF topic on improving MGLRU [1]. This series cleans up the code
base and fixes several performance issues, preparing for further work.
MGLRU's reclaim loop is a bit complex, and hence these problems are
somehow related to each other. The aging, scan number calculation, and
reclaim loop are coupled together, and the dirty folio handling logic is
quite different, making the reclaim loop hard to follow and the dirty
flush ineffective.
This series slightly cleans up and improves these issues using a scan
budget by calculating the number of folios to scan at the beginning of the
loop, and decouples aging from the reclaim calculation helpers. Then,
move the dirty flush logic inside the reclaim loop so it can kick in more
effectively. These issues are somehow related, and this series handles
them and improves MGLRU reclaim in many ways.
Test results: All tests are done on a 48c96t NUMA machine with 2 nodes and
a 128G memory machine using NVME as storage. Classical (non-MGLRU) LRU
numbers are included as "MGLRU disabled" for each benchmark below; see [8]
and [9] for the longer write-up.
MongoDB
=======
Running YCSB workloadb [2] (recordcount:20000000 operationcount:6000000,
threads:32), which does 95% read and 5% update to generate mixed read and
dirty writeback. MongoDB is set up in a 10G cgroup using Docker, and the
WiredTiger cache size is set to 4.5G, using NVME as storage. This is
close to the case we observed regressing in our production environment:
mixed read and writeback pressure, so it is a practical case for
evaluation.
Not using SWAP. The intent is to isolate the file LRU writeback path.
Enabling SWAP would just add noise from anonymous reclaim.
We can see a significant performance improvement after this series. The
test is done on NVME and the performance gap would be even larger for slow
devices, such as HDD or network storage. We observed over 100% gain for
some workloads with slow IO.
Note, classical LRU is still faster for this benchmark, MGLRU may catch up
later with further work [7].
Chrome & Node.js [3]
====================
Using Yu Zhao's test script [3], testing on a x86_64 NUMA machine with 2
nodes and 128G memory, using 256G ZRAM as swap and spawn 32 memcg 64
workers. Many memcgs each applying roughly equal pressure exercises the
LRU's ability to detect/protect each tenant's working set and to balance
reclamation fairly between tenants, which makes this a meaningful test for
the reclaim mechanism.
Fairness is reported via Jain's fairness index (1.0 means all tenants get
exactly equal allocation, lower is worse). Under equal pressure, all
memcgs should make roughly equal forward progress. See [8] for the longer
rationale and per-memcg breakdown.
Reclaim is still fair and effective, total requests number seems slightly
better.
OOM issue with aging and throttling
===================================
For the throttling OOM issue, it can be easily reproduced using dd and
cgroup limit as demonstrated and fixed by a later patch in this series.
The aging OOM is a bit tricky, a specific reproducer can be used to
simulate what we encountered in production environment [4]: Spawns
multiple workers that keep reading the given file using mmap, and pauses
for 120ms after one file read batch. It also spawns another set of
workers that keep allocating and freeing a given size of anonymous memory.
The total memory size exceeds the memory limit (eg. 14G anon + 8G file,
which is 22G vs a 16G memcg limit).
- MGLRU disabled:
Finished 128 iterations.
- MGLRU enabled:
OOM with following info after about ~10-20 iterations:
[ 62.624130] file_anon_mix_p invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
[ 62.624999] memory: usage 16777216kB, limit 16777216kB, failcnt 24460
[ 62.640200] swap: usage 0kB, limit 9007199254740988kB, failcnt 0
[ 62.640823] Memory cgroup stats for /demo:
[ 62.641017] anon 10604879872
[ 62.641941] file 6574858240
OOM occurs despite there being still evictable file folios.
- MGLRU enabled after this series:
Finished 128 iterations.
Worth noting there is another OOM related issue reported in V1 of this
series, which is tested and looking OK now [5].
MySQL:
======
Testing with innodb_buffer_pool_size=26106127360, in a 2G memcg, using
ZRAM as swap and test command:
sysbench /usr/share/sysbench/oltp_read_only.lua --mysql-db=sb \
--tables=48 --table-size=2000000 --threads=48 --time=600 run
A 24G InnoDB buffer pool inside a 2G memcg with ZRAM as swap forces
aggressive eviction of cached database anon pages, which exercises the
LRU's hot page detection and the eviction path under swap pressure. The
workload is practical, and the pressure is higher than what we usually see
in production but it is intended to expose the extreme case.
Random buffered mmap read on a ramdisk strips out storage variance and
stresses purely the LRU's ability to evict and recycle the page cache
under heavy random read pressure.
Also seem only noise level changes and no regression or slightly better.
Build kernel:
=============
Build kernel test using ZRAM as swap, kernel source on tmpfs, in a memcg
with memory.max=3G, using make -j96 and defconfig, measuring system time,
6 test run each. Building the kernel is a classical mixed anon + file
workload (lots of small file reads/writes plus parallel anon allocations
from cc/ld) and is representative of many real compilation jobs.
Also seem only noise level changes, no regression or very slightly better.
Android:
========
Xinyu reported a performance gain on Android, too, with this series. The
test consisted of cold-starting multiple applications sequentially under
moderate system load [6]; this is a real Android user-visible scenario,
dominated by the LRU's ability to keep the right working set resident and
re-fault launch-critical pages quickly.
Before:
Launch Time Summary (all apps, all runs)
Mean 868.0ms
P50 888.0ms
P90 1274.2ms
P95 1399.0ms
After:
Launch Time Summary (all apps, all runs)
Mean 850.5ms (-2.07%)
P50 861.5ms (-3.04%)
P90 1179.0ms (-8.05%)
P95 1228.0ms (-12.2%)
This patch (of 15):
Merge commonly used code for counting evictable folios in a lruvec.
userfaultfd: make functions that are not used outside uffd static
After merging fs/userfaultfd.c into mm/userfaultfd.c, several functions
that were previously shared between the two files are now only used within
mm/userfaultfd.c.
Make them static and remove their declarations from
include/linux/userfaultfd_k.h.
Link: https://lore.kernel.org/20260523173759.3964908-3-rppt@kernel.org Assisted-by: Copilot:claude-opus-4-6 Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Christian Brauner <brauner@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: Jan Kara <jack@suse.cz> Cc: "Kirill A. Shutemov" <kirill@shutemov.name> Cc: Peter Xu <peterx@redhat.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
userfaultfd: merge fs/userfaultfd.c into mm/userfaultfd.c
Patch series "userfaultfd: merge fs/userfaultfd.c into mm/userfaultfd.c",
v3.
These patches merge fs/userfaultfd.c into mm/userfaultfd.c and make
functions used only inside mm/userfaultfd.c static.
This patch (of 2):
Historically userfaultfd implementation has been split between
fs/userfaultfd.c and mm/userfaultfd.c.
The mm/ part implemented memory management operations, while the fs/ part
implemented file descriptor handling and called into the mm/ part for the
actual memory management work.
This separation is quite artificial and fs/userfaultfd.c does not seem to
belong to fs/ because it's only a user if vfs APIs and like for other
users, for example, memfd and secretmem, the file descriptor handling
could live in mm/ as well.
"Append" fs/userfaultfd.c to mm/userfaultfd and update fs/Makefile and
MAINTAINERS accordingly.
No intended functional changes.
Link: https://lore.kernel.org/20260523173759.3964908-1-rppt@kernel.org Link: https://lore.kernel.org/20260523173759.3964908-2-rppt@kernel.org Assisted-by: Copilot:claude-opus-4-6 Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org> Reviewed-by: Christian Brauner (Amutable) <brauner@kernel.org> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: David Hildenbrand <david@kernel.org> Cc: Jan Kara <jack@suse.cz> Cc: "Kirill A. Shutemov" <kirill@shutemov.name> Cc: Peter Xu <peterx@redhat.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
mm/mglru: use folio_mark_accessed to replace folio_set_active
MGLRU gives high priority to folios mapped in page tables. As a result,
folio_set_active() is invoked for all folios read during page faults. In
practice, however, readahead can bring in many folios that are never
accessed via page tables.
A previous attempt by Lei Liu proposed introducing a separate LRU for
readahead[1] to make readahead pages easier to reclaim, but that approach
is likely over-engineered.
Before commit 4d5d14a01e2c ("mm/mglru: rework workingset protection"),
folios with PG_active were always placed in the youngest generation,
leading to over-protection and increased refaults. After that commit,
PG_active folios are placed in the second youngest generation, which is
still too optimistic given the presence of readahead. In contrast, the
classic active/inactive scheme is more conservative.
This patch switches to using folio_mark_accessed() and
begins prefaulted file folios from the second oldest
generation instead of active generations.
We should also adjust the following accordingly:
- WORKINGSET_ACTIVATE: aligned with setting active for refaulted workingset
folios;
- lru_gen_folio_seq(): place (pre)faulted file folios into the second
oldest generation;
- promote second-scanned folios to workingset in
folio_check_references(): we now have to depend on
folio_lru_refs() > 1, since we previously relied on PG_referenced
being set during the first scan, but PG_referenced is now set
earlier.
On x86, running a kernel build inside a memcg with a 1GB memory
limit using 20 threads.
Wang Wensheng [Sun, 24 May 2026 03:10:53 +0000 (11:10 +0800)]
kasan/test: only do kmalloc_double_kzfree for generic mode
kmalloc_double_kzfree() would corrupt kernel memory when the just freed
memory were allocated by another thread before the second call to
kfree_sensitive() and the new allocation tag happened to match the old
one.
This could not happen in GENERIC mode as it uses quarantine.
Link: https://lore.kernel.org/20260524031053.381776-1-wsw9603@163.com Signed-off-by: Wang Wensheng <wsw9603@163.com> Reviewed-by: Andrey Konovalov <andreyknvl@gmail.com> Cc: Alexander Potapenko <glider@google.com> Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Vincenzo Frascino <vincenzo.frascino@arm.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
SeongJae Park [Wed, 20 May 2026 15:03:10 +0000 (08:03 -0700)]
mm/damon/core: trace esz at first setup
DAMON traces effective size quota from the second update, only if a change
has been made by the update. Tracing only changed updates was an
intentional decision to avoid unnecessary same value tracing. Always
skipping the first value is just an unintended mistake.
The mistake makes the tracepoint based investigation incomplete, because
the first effective size quota is never traced. It is not a big issue
when the 'consist' quota tuner is used, because it keeps changing the
quota in the usual setup.
However, when the 'temporal' tuner is used, the quota value is not changed
before the goal achievement status is completely changed. For example, if
the DAMOS scheme is started with an under-achieved goal, the quota is set
to the maximum value, and kept the same value until the goal is achieved.
Because DAMON skips the first value, the user cannot know what effective
quota the current scheme is using. Only after the goal is achieved, the
effective quota is changed to zero, and traced.
Unconditionally trace the initial quota value to fix this problem.
Note that the 'temporal' quota tuner was introduced by commit af738a6a00c1
("mm/damon/core: introduce DAMOS_QUOTA_GOAL_TUNER_TEMPORAL"), which was
added to 7.1-rc1. But even with the 'consist' quota tuner, the tracing is
unintentionally incomplete. Hence this commit marks the introduction of
the trace event as the broken commit.
Link: https://lore.kernel.org/20260520150311.80925-1-sj@kernel.org Fixes: a86d695193bf ("mm/damon: add trace event for effective size quota") Signed-off-by: SeongJae Park <sj@kernel.org> Cc: <stable@vger.kernel.org> # 6.17.x Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Maksym Shcherba [Thu, 21 May 2026 20:20:20 +0000 (23:20 +0300)]
Docs/admin-guide/mm/damon/usage: clarify current_value of quota goals
The sysfs interface for DAMON quota goals includes a `current_value` file.
This file is not updated by the kernel and only serves to receive user
input.
Clarify in the documentation that the kernel does not update
`current_value`, and that reading it only has meaning when `target_metric`
is set to `user_input`.
While at it, fix missing commas in the goal files list.
Maksym Shcherba [Thu, 21 May 2026 20:20:19 +0000 (23:20 +0300)]
mm/damon: fix missing parens in macro arguments
Patch series "mm/damon: fix macro arguments and clarify quota goals doc",
v2.
This patch (of 2):
The DAMON iterator macros do not wrap their pointer arguments with
parentheses. This can cause build failures when the argument is a complex
expression due to operator precedence issues.
Add missing parentheses around the arguments in the following macros
to prevent potential build failures:
- damon_for_each_region()
- damon_for_each_region_from()
- damon_for_each_region_safe()
- damos_for_each_quota_goal()
SeongJae Park [Fri, 22 May 2026 15:40:22 +0000 (08:40 -0700)]
selftests/damon/sysfs.py: stop kdamonds before failing
When an assertion is failed, sysfs.py DAMON selftest immediately exits the
test program leaving the DAMON running behind. Many of the following
tests need to start DAMON on their own. But because DAMON that was
started by sysfs.py is still running, those start attempts fail, and the
tests are failed or skipped. Update sysfs.py to stop DAMON before exiting
the test program due to the assertion failure.
Link: https://lore.kernel.org/20260522154026.80546-12-sj@kernel.org Signed-off-by: SeongJae Park <sj@kernel.org> Cc: Brendan Higgins <brendan.higgins@linux.dev> Cc: Shuah Khan <shuah@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
SeongJae Park [Fri, 22 May 2026 15:40:21 +0000 (08:40 -0700)]
mm/damon/tests/core-kunit: add damon_set_regions() test cases
damon_set_regions() is one of the main DAMON kernel API functions that set
up the monitoring target memory region boundaries. Implement unit tests
for verifying its basic functionalities.
Link: https://lore.kernel.org/20260522154026.80546-11-sj@kernel.org Signed-off-by: SeongJae Park <sj@kernel.org> Cc: Brendan Higgins <brendan.higgins@linux.dev> Cc: Shuah Khan <shuah@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
SeongJae Park [Fri, 22 May 2026 15:40:20 +0000 (08:40 -0700)]
mm/damon/core: remove damon_verify_nr_regions()
When CONFIG_DAMON_DEBUG_SANITY is enabled, damon_verify_nr_regions() is
called for each damon_nr_regions() invocation. damon_veify_nr_regions()
iterates all regions. damon_nr_regions() is called for each region in
kdamond_reset_aggregated() and damos_apply_scheme(). Hence it imposes
O(n**2) overhead where n is the number of regions.
Though the verification is enabled only under DAMON_DEBUG_SANITY, which is
not for production use cases, it could be too high overhead. Meanwhile,
damon_verify_ctx() is doing the damon_nr_regions() test. Because
damon_verify_ctx() is called for each kdamond_call(), the test coverage
from damon_verify_ctx() could be sufficient. Remove damon_nr_regions()
verification.
Link: https://lore.kernel.org/20260522154026.80546-10-sj@kernel.org Signed-off-by: SeongJae Park <sj@kernel.org> Cc: Brendan Higgins <brendan.higgins@linux.dev> Cc: Shuah Khan <shuah@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
kdamond_call() is the place where DAMON API callers are allowed to access
the DAMON context's public internal state including the monitoring
results. Hence it is important to ensure it is called with the expected
DAMON context state. Do the check under DAMON_DEBUG_SANITY.
Link: https://lore.kernel.org/20260522154026.80546-9-sj@kernel.org Signed-off-by: SeongJae Park <sj@kernel.org> Cc: Brendan Higgins <brendan.higgins@linux.dev> Cc: Shuah Khan <shuah@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
SeongJae Park [Fri, 22 May 2026 15:40:18 +0000 (08:40 -0700)]
mm/damon/core: hide damon_destroy_region()
damon_destroy_region() is being used by only DAMON core, but exposed to
DAMON API callers. Exposing something that is not really being used by
others will only increase the maintenance cost. Hide it.
Link: https://lore.kernel.org/20260522154026.80546-8-sj@kernel.org Signed-off-by: SeongJae Park <sj@kernel.org> Cc: Brendan Higgins <brendan.higgins@linux.dev> Cc: Shuah Khan <shuah@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
SeongJae Park [Fri, 22 May 2026 15:40:17 +0000 (08:40 -0700)]
mm/damon/core: hide damon_insert_region()
damon_insert_region() is being used by only DAMON core, but exposed to
DAMON API callers. Exposing something that is not really being used by
others will only increase the maintenance cost. Hide it.
Link: https://lore.kernel.org/20260522154026.80546-7-sj@kernel.org Signed-off-by: SeongJae Park <sj@kernel.org> Cc: Brendan Higgins <brendan.higgins@linux.dev> Cc: Shuah Khan <shuah@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
SeongJae Park [Fri, 22 May 2026 15:40:16 +0000 (08:40 -0700)]
mm/damon/core: hide damon_add_region()
damon_add_region() is being used by only DAMON core, but exposed to DAMON
API callers. Exposing something that is not really being used by others
will only increase the maintenance cost. Hide it.
Link: https://lore.kernel.org/20260522154026.80546-6-sj@kernel.org Signed-off-by: SeongJae Park <sj@kernel.org> Cc: Brendan Higgins <brendan.higgins@linux.dev> Cc: Shuah Khan <shuah@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
projects / thirdparty / kernel / linux.git / log
