Michal Pecio [Wed, 3 Jun 2026 09:11:23 +0000 (12:11 +0300)]
usb: xhci: Improve Soft Retries after short transfers
A short transfer is a successful one, so reset the error count.
Otherwise, endpoints which always complete short are limited to
three retries per endpoint life rather than per URB.
Michal Pecio [Wed, 3 Jun 2026 09:11:21 +0000 (12:11 +0300)]
usb: xhci: Remove skip_isoc_td()
This function is pointless because usb_submit_urb() initializes all
isoc frame descriptors to -EXDEV and 0 length so that HCDs don't need
to do anything with transfers which were never executed.
Other HCDs rely on this (e.g. EHCI itd_complete()), so we can too.
This gets rid of a potentially dangereous function which could corrupt
memory if we weren't super careful to only call it on isoc URBs.
Also, set status to 0 rather than any random status determined by the
later TD which caused skipping. This status will be ignored anyway.
Michal Pecio [Wed, 3 Jun 2026 09:11:20 +0000 (12:11 +0300)]
usb: xhci: Simplify xhci_quiesce()
The function reads USBCMD, clears some bits and writes it back.
Its treatment of the Run bit is weird: the bit is usually written
as 0, as we would expect, but it may also be written as 1 if both
its current value and USBSTS.HCHalted are observed as 1.
Per xHCI 5.4.2, HCHalted is 0 whenever Run is 1, so the above can
only happen due to buggy HW or SW, e.g. concurrent xhci_quiesce()
and xhci_start() execution.
It's unclear why we should treat such cases specially and write
the bit as 1. The logic comes from original PoC implementation
and has never been explained. Just write 0 every time, which
looks like the safer choice when the intent is to stop the xHC.
We could get in trouble if clearing Run causes some very broken
xHC to start running after it was halted, but no such case has
been documented. It seems the logic was just poorly thought out.
Niklas Neronin [Wed, 3 Jun 2026 09:11:19 +0000 (12:11 +0300)]
usb: xhci: remove legacy 'num_trbs_free' tracking
Keeping track of free TRBs in a ring by adding and subtracting each time
a enqueue or dequeue pointer is modified has proven to be buggy and
complicated, especially over long periods of time.
The xhci driver has already moved to calculating free TRBs dynamically
based on ring size and the enqueue/dequeue positions.
The DbC path is the last user of 'num_trbs_free'. Rather than maintaining
two separate accounting mechanisms, remove the field entirely and switch
DbC to use xhci_num_trbs_free(). Since 'num_trbs_free' undercounts by one,
and xhci_num_trbs_free() does not, the check for sufficient free TRBs is
adjusted.
The size limits break some of the CI tests. So drop them for now. Keep
the other overflow tests from commit 5ab62dd3687b ("drm: prevent integer
overflows in dumb buffer creation helpers") in place.
There is still a pre-existing overflow check for 32-bit type limits in
drm_mode_create_dumb() that will catch the really absurd size requests.
Drivers that still do not use drm_mode_size_dumb() should be updated. The
helper calculates dumb-buffer geometry with overflow checks.
Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de> Fixes: 5ab62dd3687b ("drm: prevent integer overflows in dumb buffer creation helpers") Reported-by: Jani Nikula <jani.nikula@linux.intel.com> Closes: https://lore.kernel.org/dri-devel/ddf0233e50044059c85279f928661563ef6a55bf@intel.com/ Cc: Rajat Gupta <rajat.gupta@oss.qualcomm.com> Cc: Thomas Zimmermann <tzimmermann@suse.de> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Cc: Maxime Ripard <mripard@kernel.org> Acked-by: Jani Nikula <jani.nikula@intel.com> Link: https://patch.msgid.link/20260602112842.252279-1-tzimmermann@suse.de
Mukesh Ojha [Wed, 27 May 2026 09:54:26 +0000 (15:24 +0530)]
irqchip/qcom-pdc: Use FIELD_GET() to extract bank index and bit position
The IRQ_ENABLE_BANK register is a bank of 32-bit words where each bit
represents one PDC pin. The bank index and bit position within the bank
are encoded in the flat pin number as bits [31:5] and [4:0] respectively.
Replace the open-coded division and modulo with FIELD_GET() and GENMASK()
to make the bit extraction self-documenting and consistent with the
FIELD_PREP() style already used in the PDC_VERSION() macro.
Mukesh Ojha [Wed, 27 May 2026 09:54:25 +0000 (15:24 +0530)]
irqchip/qcom-pdc: Add PDC_VERSION() macro to describe version register fields
The PDC hardware version register encodes major, minor and step fields
in byte-sized fields at bits [23:16], [15:8] and [7:0] respectively.
The existing PDC_VERSION_3_2 constant was a bare magic number (0x30200)
with no indication of this encoding.
Add GENMASK-based field definitions for each sub-field and a
PDC_VERSION(maj, min, step) constructor macro using FIELD_PREP, making
the encoding self-documenting. Replace the magic constant with
PDC_VERSION(3, 2, 0).
Signed-off-by: Mukesh Ojha <mukesh.ojha@oss.qualcomm.com> Signed-off-by: Thomas Gleixner <tglx@kernel.org> Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com> Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com> Link: https://patch.msgid.link/20260527095426.2324504-4-mukesh.ojha@oss.qualcomm.com
Mukesh Ojha [Wed, 27 May 2026 09:54:24 +0000 (15:24 +0530)]
irqchip/qcom-pdc: Tighten ioremap clamp to single DRV region size
The QCOM_PDC_SIZE constant (0x30000) was introduced to work around old
sm8150 DTs that described a too-small PDC register region, causing the
driver to silently expand the ioremap to cover three DRV regions. Now
that the preceding DT fixes have corrected all platforms to describe only
the APSS DRV region (0x10000), the oversized clamp is no longer needed.
Replace QCOM_PDC_SIZE with PDC_DRV_SIZE (0x10000) in the clamp so the
minimum mapped size matches a single DRV region. The clamp and warning
are intentionally kept to preserve backward compatibility with any old
DTs that may still describe a smaller region.
While at it, rename PDC_DRV_OFFSET to PDC_DRV_SIZE since the constant
represents the size of a DRV region and is used as both the ioremap
minimum size and the offset to the previous DRV region.
Mukesh Ojha [Wed, 27 May 2026 09:54:23 +0000 (15:24 +0530)]
irqchip/qcom-pdc: Split __pdc_enable_intr() into per-version helpers
The __pdc_enable_intr() function contains a version branch that selects
between two distinct enable mechanisms: a bank-based IRQ_ENABLE_BANK
register for HW < 3.2, and a per-pin enable bit in IRQ_i_CFG for
HW >= 3.2. These two paths share no code and serve different hardware.
Split them into two focused static functions: pdc_enable_intr_bank()
for HW < 3.2 and pdc_enable_intr_cfg() for HW >= 3.2. No functional
change.
Signed-off-by: Mukesh Ojha <mukesh.ojha@oss.qualcomm.com> Signed-off-by: Thomas Gleixner <tglx@kernel.org> Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com> Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com> Link: https://patch.msgid.link/20260527095426.2324504-2-mukesh.ojha@oss.qualcomm.com
Marek Szyprowski [Fri, 22 May 2026 06:10:12 +0000 (08:10 +0200)]
irqchip/exynos-combiner: Remove useless spinlock
irq_controller_lock doesn't protect anything, it is a leftover from early
development or copy/paste. Remove it completely.
Fixes: 96031b31a4b3 ("irqchip/exynos-combiner: Switch to raw_spinlock") Suggested-by: Thomas Gleixner <tglx@kernel.org> Suggested-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com> Signed-off-by: Thomas Gleixner <tglx@kernel.org> Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Reviewed-by: Peter Griffin <peter.griffin@linaro.org> Link: https://lore.kernel.org/all/20260521090453.bbUZ00tS@linutronix.de Link: https://patch.msgid.link/20260522061012.2687122-1-m.szyprowski@samsung.com/
The mempool_alloc_bulk was modelled after the alloc_pages_bulk API,
including some misunderstanding of it.
Remove checking for NULL slots in the array, as alloc_pages_bulk and
kmem_cache_alloc_bulk always fill the array from the beginning and thus
we know the offset of the first failing allocation. This removes support
for working well with alloc_pages_bulk used to refill page arrays that
might have an entry removed from in the middle, but that is only used by
sunrpc and hopefully on it's way out.
Also remove the allocated parameter as it is redundant because the caller
can simply specific and offset into the entries array.
The kmem_cache_alloc_bulk return value is weird. It returns the number
of allocated objects, but that must always be 0 or the requested number
based on the implementations and the handling in the callers, but that
assumption is not actually documented anywhere, which confuses automated
review tools.
Fix this by returning a bool if the allocation succeeded and adding a
kerneldoc comment explaining the API.
[rob.clark@oss.qualcomm.com: fixups in
msm_iommu_pagetable_prealloc_allocate() ]
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Alexander Lobakin <aleksander.lobakin@intel.com> # skbuff Link: https://patch.msgid.link/20260528093437.2519248-2-hch@lst.de Signed-off-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Linus Torvalds [Wed, 3 Jun 2026 16:09:24 +0000 (09:09 -0700)]
Merge tag 'mmc-v7.1-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc
Pull MMC fixes from Ulf Hansson:
"MMC core:
- Fix host controller programming for eMMC fixed driver type
MMC host:
- dw_mmc-rockchip: Add missing private data for very old controllers
- litex_mmc: Fix clock management
- renesas_sdhi: Add OF entry for RZ/G2H SoC
- sdhci: Manage signal voltage switch during system resume for some hosts
- sdhci-of-dwcmshc: Fix reset, clk and SDIO support for Eswin EIC7700"
* tag 'mmc-v7.1-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc:
mmc: sdhci: add signal voltage switch in sdhci_resume_host
mmc: dw_mmc-rockchip: Add missing private data for very old controllers
mmc: litex_mmc: Set mandatory idle clocks before CMD0
mmc: litex_mmc: Use DIV_ROUND_UP for more accurate clock calculation
mmc: renesas_sdhi: Add OF entry for RZ/G2H SoC
mmc: sdhci-of-dwcmshc: Fix reset, clk, and SDIO support for Eswin EIC7700
mmc: core: Fix host controller programming for fixed driver type
Chao Gao [Wed, 20 May 2026 22:29:12 +0000 (15:29 -0700)]
x86/virt/tdx: Refresh TDX module version after update
The kernel exposes the TDX module version through sysfs so userspace
can check update compatibility. That information needs to remain
accurate across runtime updates.
A runtime update may change the module's update_version, so refresh
the cached version right after a successful update.
Drop __ro_after_init from tdx_sysinfo because it is now updated at
runtime.
Do not refresh the rest of tdx_sysinfo, even if some values change
across updates. TDX module updates are backward compatible, so
existing tdx_sysinfo consumers, such as KVM, can continue to operate
without seeing the new values.
Dave Hansen [Fri, 22 May 2026 15:56:29 +0000 (08:56 -0700)]
coco/tdx-host: Lock out module updates when reading version
The TDX module version is currently stashed in some global variables
and dumped out to sysfs without locking. This works fine when the
version is static and never changes.
But with runtime module updates, the TDX module version can change.
Some kind of locking is needed. Barring this, userspace could
theoretically see a strange torn module version that is some
Frankenstein version from from two different updates.
Use the new module update lock/unlock to prevent updates while
trying to read the version.
Don't be fussy about it. There's no need to snapshot the version or do
READ_ONCE(), or minimize lock holding times. sysfs_emit() does not
sleep. Also note that the lock/unlock are backed by
preempt_dis/enable() which are really cheap CPU-local operations.
This is not a heavyweight lock.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Dave Hansen [Fri, 22 May 2026 15:43:02 +0000 (08:43 -0700)]
x86/virt/seamldr: Add module update locking
TDX metadata like the version number changes during a module update.
Add functions to lock out module updates.
The current stop_machine() implementation uses worker threads. The
scheduler actually does a full, normal context switch over to that
thread. preempt_disable() obviously inhibits that context switch and
thus, locks out stop_machine() users like the module update.
Thanks to Chao for the idea of using preempt_disable().
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Chao Gao [Wed, 20 May 2026 22:29:10 +0000 (15:29 -0700)]
x86/virt/tdx: Restore TDX module state
After per-CPU initialization, the module is nearly functional. It is
in a similar state to TDX initialization before TDH.SYS.CONFIG.
At this point, the kernel _could_ just repeat the boot-time sequence,
but that would land the new module in a slightly different state than
the old module. This would leave old TDs unrunnable, which is not a
good outcome.
Thankfully, the "handoff" data saved during module shutdown should
contain all the information needed to restore the TDX module state to
exactly what it was before the update.
Restore TDX module state. The TDX module only needs a single copy so
only do this on the lead CPU.
Restoration errors can theoretically be handled in a few ways. For
instance, userspace could try to load a different TDX module version.
Or, the kernel could give up on the handoff process and just
reinitialize the new module from scratch, which would lose all
existing TDs.
Simply propagate errors to userspace. Ignore the idea of a
TD-destroying reinitialization. It would destroy data like a reboot
and if things have gone that wrong a reboot is probably the best
option anyway.
Note: the location and the format of handoff data is defined by the
TDX module. The new module knows where to get handoff data and how to
parse it. The kernel does not touch it at all.
Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Link: https://patch.msgid.link/20260520133909.409394-21-chao.gao@intel.com
Chao Gao [Wed, 20 May 2026 22:29:09 +0000 (15:29 -0700)]
x86/virt/seamldr: Initialize the newly-installed TDX module
Continue fleshing out the update process. At this point the new module
is sitting in memory but has never been called and is not usable. It
is in a similar state to the when the system first boots.
Leave the P-SEAMLDR behind. Stop making calls to it. Transition to
calling the new TDX module itself to set up both global and per-cpu
state.
Share tdx_cpu_enable() with the fresh-boot module initialization code.
Export it and invoke it on all CPUs.
Note: "TDX global initialization" needs to be done once before "TDX
per-CPU initialization". It would be a great fit for the new runtime
update "is_lead_cpu" logic. But tdx_cpu_enable() already has some
logic to do the global initialization properly. Just use it directly
to maximize fresh-boot and runtime update code sharing.
== Background ==
The boot-time and post-update initialization flows share the same first
steps:
- TDX global initialization
- TDX per-CPU initialization
After that, they diverge:
- Fresh boot:
Prepare TDMRs/PAMTs
Configure the TDX module
Configure the global KeyID
Initialize TDMRs
- Runtime update:
Restore TDX module state from handoff data
Future changes will consume the handoff data.
[ dhansen: major changelog munging ]
Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Link: https://patch.msgid.link/20260520133909.409394-20-chao.gao@intel.com
Chao Gao [Wed, 20 May 2026 22:29:08 +0000 (15:29 -0700)]
x86/virt/seamldr: Install a new TDX module
Continue fleshing out the update proces. The old module is shut down
and the system is ready for the new module image. Run the
SEAMLDR.INSTALL SEAMCALL on all CPUs.
Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Link: https://patch.msgid.link/20260520133909.409394-19-chao.gao@intel.com
Chao Gao [Wed, 20 May 2026 22:29:06 +0000 (15:29 -0700)]
x86/virt/tdx: Reset software states during TDX module shutdown
The TDX module requires a one-time global initialization (TDH.SYS.INIT) and
per-CPU initialization (TDH.SYS.LP.INIT) before use. These initializations
are guarded by software flags to prevent repetition.
Reset all software flags guarding the initialization flows to allow the
global and per-CPU initializations to be triggered again after updates.
[ dhansen: trim down changelog ]
Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Link: https://patch.msgid.link/20260520133909.409394-18-chao.gao@intel.com
Linus Torvalds [Wed, 3 Jun 2026 15:59:24 +0000 (08:59 -0700)]
Merge tag 'cgroup-for-7.1-rc6-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
Pull cgroup fixes from Tejun Heo:
"One cpuset fix and a maintenance update, both low-risk:
- Fix cpuset partition CPU accounting under sibling CPU exclusion
that could produce wrong CPU assignments and trigger
scheduling-domain warnings. Includes selftests.
- Update an email address in MAINTAINERS"
* tag 'cgroup-for-7.1-rc6-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup:
cgroup/cpuset: Change Ridong's email
cgroup/cpuset: Add test cases for sibling CPU exclusion on partition update
cgroup/cpuset: Use effective_xcpus in partcmd_update add/del mask calculation
Chao Gao [Wed, 20 May 2026 22:29:05 +0000 (15:29 -0700)]
x86/virt/seamldr: Shut down the current TDX module
The first step of TDX module updates is shutting down the current TDX
module. This step also packs state information that needs to be
preserved across updates, called "handoff data". This handoff data is
consumed by the updated module and stored internally in the SEAM range and
hidden from the kernel.
Since the handoff data layout may change between modules, the handoff
data is versioned. Each module has a native handoff version and
provides backward support for several older versions.
The complete handoff versioning protocol is complex as it supports both
module upgrades and downgrades. See details in "Intel Trust Domain
Extensions (Intel TDX) Module Base Architecture Specification", Chapter
"Handoff Versioning".
Ideally, the kernel needs to retrieve the handoff versions supported by
the current module and the new module and select a version supported by
both. But since this implementation only supports module upgrades, simply
request handoff data from the current module using its highest supported
version. That is sufficient for this upgrade-only implementation.
Retrieve the module's handoff version from TDX global metadata and add an
update step to shut down the module. Module shutdown only needs to run on
one CPU.
Don't cache the handoff information in tdx_sysinfo. It is used only for
module shutdown, and is present only when the TDX module supports updates.
Caching it in get_tdx_sys_info() would require extra update-support guards
and refreshing the cached value across module updates.
[ dhansen: fix up function variables, remove 'cpu'.
Return from tdx_module_shutdown() early if handoff call fails. ]
Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Link: https://patch.msgid.link/20260520133909.409394-17-chao.gao@intel.com
Linus Torvalds [Wed, 3 Jun 2026 15:52:26 +0000 (08:52 -0700)]
Merge tag 'sched_ext-for-7.1-rc6-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext
Pull sched_ext fixes from Tejun Heo:
"Two low-risk fixes:
- Drop a spurious warning that can fire during cgroup migration while
a sched_ext scheduler is loaded
- Fix a drgn-based debug script that broke after scheduler state
moved into a per-scheduler struct"
* tag 'sched_ext-for-7.1-rc6-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext:
sched_ext: Don't warn on NULL cgrp_moving_from in scx_cgroup_move_task()
tools/sched_ext: Fix scx_show_state per-scheduler state reads
Mark Rutland [Wed, 3 Jun 2026 11:06:30 +0000 (12:06 +0100)]
arm64: fpsimd: Remove <asm/fpsimdmacros.h>
We no longer need any of the remaining macros in <asm/fpsimdmacros.h>.
Remove all of it.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:29 +0000 (12:06 +0100)]
arm64: fpsimd: Move SME save/restore inline
Currently the SVE register save/restore sequences are written in
out-of-line assembly routines. While this works, it's somewhat painful:
* For KVM to use the sequences, portions of the logic will need to be
duplicated in KVM hyp code. While the common logic can be shared in
assembly macros, this is very likely to lead to unnecessary divergence
and be a maintenance burden.
* For historical reasons, the assembly macros take some register
arguments as numerical indices (e.g. "sme_save_za 0, x2, 12" uses x0, x1, and
x12), which is simply confusing.
* Address generation and control flow are far clearer in C than in
assembly.
* The assembly sequences can't be instrumented, and so it's harder than
necessary to catch memory safety issues.
To handle the above, move the SME register save/restore sequences
to inline assembly.
Neither GCC nor LLVM instrument memory arguments to inline assembly, so
explicit instrumentation is added in the same manner as other assembly
routines. This instrumentation is implicitly disabled by Kbuild for nVHE
hyp code.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:28 +0000 (12:06 +0100)]
arm64: fpsimd: Move sve_flush_live() inline
Currently sve_flush_live() is written in out-of-line assembly. It would
be nice if we could move it inline such that control flow can be written
more clearly in C, and to permit the removal of otherwise unused
assembly macros.
The 'flush_ffr' argument is redundant as sve_flush_live() is always
called from non-streaming mode, and all callers pass 'true'. Remove the
argument and make it a requirement that the function is called from
non-streaming mode.
The 'vq_minus_1' argument is unnecessary, as sve_flush_live() can read
the live VL directly using the RDVL instruction (wrapped by the
sve_get_vl() helper function).
Move the function to C, with the simplifications above.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:27 +0000 (12:06 +0100)]
arm64: fpsimd: Move SVE save/restore inline
Currently the SVE register save/restore sequences are written in
out-of-line assembly routines. While this works, it's somewhat painful:
* As KVM needs to be able to use the sequences in hyp code, separate
assembly files are used for the regular kernel and KVM code. While the
common logic is shared in assembly macros, this still requires some
duplication, and has lead to some trivial divergence.
* As the SVE LDR/STR instrucitons have limited addressing modes, the
assembly macros use an awkward pattern requiring negative offsets.
This could be written more clearly with addresses being generated in C
code.
* As the FFR does not always exist in streaming mode, some awkward
conditional branching has been written in assembly which could be
clearer in C (and would permit the compiler to optimize out
unnecessary branches in some cases).
* For historical reasons, the assembly macros take some register
arguments as numerical indices (e.g. "sve_save 0, x1" uses x0 and x1),
which is simply confusing.
* For historical reasons, the SVE save/restore code and FPSIMD
save/restore code have a distinct sequences for FPSR and FPCR. Ideally
this logic would be shared.
* The assembly sequences can't be instrumented, and so it's harder than
necessary to catch memory safety issues.
To handle the above, move the SVE register save/restore sequences
to inline assembly.
Neither GCC nor LLVM instrument memory arguments to inline assembly, so
explicit instrumentation is added in the same manner as other assembly
routines. This instrumentation is implicitly disabled by Kbuild for nVHE
hyp code.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:26 +0000 (12:06 +0100)]
arm64: fpsimd: Use opaque type for SME state
As the SME state size can vary at runtime, we don't have a concrete type
for the in-memory SME state, and pass this around using a pointer to
void.
Using pointer to void means that it's very easy to introduce errors that
cannot be caught by the compiler (e.g. as 'void **' can be assigned to
'void *').
Improve this by adding an opaque 'struct arm64_sme_state', and
consistently passing a pointer to this.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:25 +0000 (12:06 +0100)]
arm64: fpsimd: Use opaque type for SVE state
As the SVE state size can vary at runtime, we don't have a concrete type
for the in-memory SVE state, and pass this around using a pointer to
void. The functions which save/restore the SVE state have a very unusual
calling convention, expecting a pointer to the FFR *in the middle of*
the in-memory SVE state, which is also passed as a pointer to void.
Passing a pointer to the FFR also requires that callers find the live VL
and perform some arithmetic, which callers implement differently.
Using pointer to void means that it's very easy to introduce errors that
cannot be caught by the compiler (e.g. as 'void **' can be assigned to
'void *'). In general this is unnecessarily confusing and fragile.
Improve this by adding an opaque 'struct arm64_sve_state', and
consistently passing a pointer to this, performing the necessary
offsetting *within* the save/restore functions.
For the moment, the offsetting is performed in a new '_sve_pffr'
assembly macro, using the ADDVL and ADDPL instructions. These add a
multiple of the live vector length and predicate length respectively.
The ADDVL immediate range cannot encode 32, so this is split into two
increments of 16.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:24 +0000 (12:06 +0100)]
arm64: fpsimd: Move fpsimd save/restore inline
Currently the FPSIMD register save/restore sequences are written in
out-of-line assembly routines. While this works, it's somewhat painful:
* As KVM needs to be able to use the sequences in hyp code, separate
assembly files are used for the regular kernel and KVM code. While the
common logic is shared in assembly macros, this still requires some
duplication, and has lead to some trivial divergence.
* For historical reasons, the assembly macros take some register
arguments as numerical indices (e.g. "fpsimd_save x0, 8" uses x0 and
x8), which is simply confusing.
* For historical reasons, the SVE save/restore code and FPSIMD
save/restore code have distinct sequences for FPSR and FPCR. Ideally
this logic would be shared.
* The assembly sequences can't be instrumented, and so it's harder than
necessary to catch memory safety issues.
To handle the above, move the FPSIMD register save/restore sequences to
inline assembly, and share the FPSR+FPCR save/restore with SVE.
Neither GCC nor LLVM instrument memory arguments to inline assembly, so
explicit instrumentation is added in the same manner as other assembly
routines. This instrumentation is implicitly disabled by Kbuild for nVHE
hyp code.
I've used the SVE sequence for restoring FPCR, which uses an
unconditional write to FPCR, rather than the conditional write used by
the FPSIMD assembly sequence. I believe that in practice, this doesn't
matter to a real workload, and given it's possible for the mis-predicted
branch to cost more than the necessary micro-architectural
synchronization, I strongly suspect any performance impact is within the
noise.
Looking at the history, the FPSIMD assembly sequence was changed to use
a conditional write to FPCR since 2014 in commit:
5959e25729a5 ("arm64: fpsimd: avoid restoring fpcr if the contents haven't change")
... as described in the commit message, this was based on an expectation
of implementation style, and was not based on benchmarking.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:23 +0000 (12:06 +0100)]
arm64: fpsimd: Split FPSR/FPCR from SVE save/restore
Regardless of whether the vector registers are saved in FPSIMD or SVE
format, we store FPSR and FPCR in user_fpsimd_state::{fpsr,fpcr}.
For historical reasons, the functions which save/restore SVE context
take a pointer to user_fpsimd_state::fpsr, and use this to access both
user_fpsimd_state::fpsr and user_fpsimd_state::fpcr. This is
unnecessarily fragile.
Move the save/restore of FPSR and FPCR into separate helper functions
which take a pointer to user_fpsimd_state. I've used read_sysreg_s() and
write_sysreg_s() as contemporary versions of LLVM will refuse to
directly assemble accesses to FPCR or FPSR unless the "fp" arch
extension is enabled.
For the moment, fpsimd_save_state() and fpsimd_load_state() are left
as-is with their own logic to save/restore FPSR and FPCR. This will be
unified in subsequent patches.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:22 +0000 (12:06 +0100)]
arm64: sysreg: Add FPCR and FPSR
Add sysreg definitions for FPCR and FPSR.
Some versions of LLVM will refuse to assemble accesses to FPCR and FPSR
unless the "fp" arch extension is enabled, which we don't currently do
for read_sysreg() and write_sysreg(). In general, handling feature
dependencies would complicate read_sysreg() and write_sysreg(), and it's
simpler to use read_sysreg_s() and write_sysreg_s() instead, requiring
sysreg definitions.
The values used can be found in ARM ARM issue M.b:
* C5.2.8 ("FPCR, Floating-point Control Register")
* C5.2.10 ("FPSR, Floating-point Status Register")
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:21 +0000 (12:06 +0100)]
arm64: fpsimd: Move sve_get_vl() and sme_get_vl() inline
The sve_get_vl() and sme_get_vl() functions are wrappers for the RDVL
and RDSVL instructions respectively. There's no need for those to be
out-of-line.
Replace the out-of-line assembly functions with equivalent inline
functions.
The _sve_rdvl assembly macro is unused, and so it is removed. The
_sme_rdsvl assembly macro is still used elsewhere, and so is kept for
now.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:20 +0000 (12:06 +0100)]
arm64: fpsimd: Use assembler for baseline SME instructions
We currently support assemblers which do not support SME instructions,
and have macros to manually encode SME instructions. This was
necessary historically as SME support was developed before assembler
support was widely available, but things have changed:
* All currently supported versions of LLVM support baseline SME
instructions. Building the kernel requires LLVM 15+, while LLVM 13+
supports SME.
* GNU binutils has supported baseline SME instructions since 2.38, which
was released on 09 February 2022. Toolchains using this or later are
widely available. For example Debian 12 (released on 10 June 2023)
provides binutils 2.40. Toolchains provided kernel.org provide
binutils 2.38+ since the GCC 12.1.0 release (released between 06 May
2022 and 17 August 2022).
* For various reasons, SME support was marked as BROKEN, and re-enabled
in v6.16 (released on 27 July 2025). The earliest support LTS kernel
with SME support is v6.18.y, v6.18 was tagged on 30 November 2025, and
contemporary toolchains (GCC 15.2 and binutils 2.45) supported
baseline SME instructions.
* Any distribution which intends to support SME will presumably have a
toolchain that supports baseline SME instructions such that userspace
can be built.
Considering the above, there's no practical benefit to allowing SME to
be built when the toolchain doesn't support baseline SME instructions.
Make CONFIG_ARM64_SME depend on assembler support for SME, and remove
the manual encoding of SME instructions. The various _sme_<insn> macros
are kept for now, and will be cleaned up in subsequent patches.
A couple of SME2 instructions require a more recent toolchain, and are
left as-is for now. I've looked through releases of binutils and LLVM to
find when support was added, and noted this in a comment.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:19 +0000 (12:06 +0100)]
arm64: fpsimd: Use assembler for SVE instructions
Historically we supported assemblers which could not assemble SVE
instructions. We dropped support for such assemblers in commit:
118c40b7b503 ("kbuild: require gcc-8 and binutils-2.30")
Since that commit, all supported assemblers (binutils and LLVM) are
capable of assembling SVE instructions, and there's no need for us to
manually encode SVE instructions.
Rely on the assembler to encode SVE instructions, and remove the manual
encoding. The various _sve_<insn> macros are kept for now, and will be
cleaned up in subsequent patches.
There should be no functional change as a result of this patch.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:18 +0000 (12:06 +0100)]
arm64: fpsimd: Remove sve_set_vq() and sme_set_vq()
The sve_set_vq() and sme_set_vq() assembly functions (and the
sve_load_vq and sme_load_vq macros they use) are open-coded forms of
sysreg_clear_set*(). There's no need for these to be implemented
out-of-line in assembly, and the 'vq_minus_1' argument is unusual and
confusing.
Use sysreg_clear_set_s() directly, where the necessary 'vq - 1' encoding
is more obviously part of encoding the register value.
For now, sve_flush_live() is left with the unusual vq_minus_1 argument.
This will be addressed in subsequent patches.
There should be no functional change as a result of this patch.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:17 +0000 (12:06 +0100)]
arm64: fpsimd: Fold sve_init_regs() into do_sve_acc()
For historical reasons, do_sve_acc() is structurally different from
do_sme_acc(), and the logic to convert the task from FPSIMD to SVE is
out-of-line in sve_init_regs(). We only use sve_init_regs() within
do_sve_acc(), so it's not necessary for this to be a separate function.
Fold sve_init_regs() into do_sve_acc(), and simplify the associated
comments. This makes do_sve_acc() structurally similar to do_sme_acc(),
making it easier to see similarities and differences.
There should be no functional change as a result of this patch.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:16 +0000 (12:06 +0100)]
KVM: arm64: pkvm: Remove struct cpu_sve_state
There's no need for struct cpu_sve_state. Code would be simpler and more
robust without it, and removing it will simplify further cleanups (e.g.
adding an opaque type for the sve register state).
Protected KVM stores most of the host's system register state in
kvm_host_data::host_ctxt, which is an instance of struct
kvm_cpu_context. As kvm_cpu_context::sys_regs[] has a slot for ZCR_EL1,
we can store the host's ZCR_EL1 there.
While kvm_cpu_context::sys_regs doesn't have slots for FPSR and FPCR,
these are usually expected to be stored in struct user_fpsimd_state.
For historical reasons, __sve_save_state and __sve_restore_state()
expect a pointer to fpsr *within* struct user_fpsimd_state, assuming the
fpcr will immediately follow, as per the order within struct
user_fpsimd_state. We currently match this ordering in struct
cpu_sve_state, but it would be simpler and more robust to use struct
user_fpsimd_state directly.
After moving ZCR_EL1, FPSR, and FPCR out of struct cpu_sve_state, all
that's left is sve_regs, which can be represented as a pointer without
need for a container struct. This is kept as a pointer to u8 (matching
the array type), as this permits the compiler to catch unbalanced
referencing/dereferencing, which is not possible for pointers to void.
Apply the above changes, and remove cpu_sve_state.
I've dropped the comment regarding buffer alignment as AFAICT this was
never necessary. The LDR/STR (vector) instructions only require this
alignment when SCTLR_ELx.A==1, which is not the case for the kernel or
hyp code. Nothing else depends on the alignment.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Mark Brown <broonie@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:15 +0000 (12:06 +0100)]
KVM: arm64: pkvm: Save host FPMR in host cpu context
Protected KVM stores most of the host's system register state in
kvm_host_data::host_ctxt, which is an instance of struct
kvm_cpu_context. As kvm_cpu_context::sys_regs[] has a slot for FPMR, we
can store the host's FPMR there.
Do so, and remove kvm_host_data::fpmr.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Mark Brown <broonie@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
The __sve_save_state() and __sve_restore_state() functions take a
parameter describing whether to save/restore the FFR, but both functions
silently override this with '1'. This has always been benign (and
callers have all passed 'true' since the parameter was introduced), but
clearly this is not intentional.
Historically, the functions always saved/restored the FFR, and there was
no parameter to control this.
In v5.16, the sve_save and sve_load assembly macros used by
__sve_save_state() and __sve_restore_state() were changed to make
saving/restoring FFR optional. The implementations of __sve_save_state()
and __sve_restore_state() were changed to pass '1' to their respective
macros, and the prototypes of __sve_save_state() and
__sve_restore_state() were unchanged. See commit:
9f5848665788 ("arm64/sve: Make access to FFR optional")
In v6.10, the prototypes of __sve_save_state() and __sve_restore_state()
were changed to add 'save_ffr' and 'restore_ffr' parameters
respectively, but the implementations were not changed to stop passing 1
to their respective macros. All callers were changed to pass 'true' to
__sve_save_state() and __sve_restore_state(). See commit:
45f4ea9bcfe9 ("KVM: arm64: Fix prototype for __sve_save_state/__sve_restore_state")
This is all benign, but clearly unintentional, and it gets in the way of
cleaning up the FPSIMD/SVE/SME code. Remove the unnecessary overriding.
The 'save_ffr' and 'restore_ffr' parameters are 32-bit ints, and per the
AAPCS64 parameter passing rules, the upper 32 bits of the register
holding these arguments might contain arbitrary values. Thus it is
necessary to pass 'w2' rather than 'x2' to the sve_load and save_save
macros, such that the upper 32 bits are ignored when deciding whether to
save/restore the FFR.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:13 +0000 (12:06 +0100)]
KVM: arm64: Don't include <asm/fpsimdmacros.h>
There's no need for hyp/entry.S to include <asm/fpsimdmacros.h>.
The fpsimd macros have never been used by code in hyp/entry.S, and were
instead used by code in hyp/fpsimd.S.
Remove the unnecessary include.
There should be no functional change as a result of this patch.
Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:12 +0000 (12:06 +0100)]
arm64: fpsimd: Fix type mismatch in sme_{save,load}_state()
The sme_save_state() and sme_load_state() functions take a 32-bit int
argument that describes whether to save/restore ZT0. Their assembly
implementations consume the entire 64-bit register containing this
32-bit value, and will attempt to save/restore ZT0 if any bit of
that 64-bit register is non-zero.
Per the AAPCS64 parameter passing rules, the callee is responsible for
any necessary widening, and the upper 32-bits are permitted to contain
arbitrary values. If the upper 32 bits are non-zero, this could result
in an unexpected attempt to save/restore ZT0, and consequently could
lead to unexpected traps/undefs/faults.
In practice compilers are very unlikely to generate code where the upper
32-bits would be non-zero, but they are permitted to do so.
Fix this by only consuming the low 32 bits of the register, and update
comments accordingly.
Fixes: 95fcec713259 ("arm64/sme: Implement context switching for ZT0") Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Mark Brown <broonie@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Will Deacon <will@kernel.org> Cc: stable@vger.kernel.org Signed-off-by: Will Deacon <will@kernel.org>
Mark Rutland [Wed, 3 Jun 2026 11:06:11 +0000 (12:06 +0100)]
arm64: fpsimd: Fix type mismatch in sve_{save,load}_state()
The sve_save_state() and sve_load_state() functions take a 32-bit int
argument that describes whether to save/restore the FFR. Their assembly
implementations consume the entire 64-bit register containing this
32-bit value, and will attempt to save/restore the FFR if any bit of
that 64-bit register is non-zero.
Per the AAPCS64 parameter passing rules, the callee is responsible for
any necessary widening, and the upper 32-bits are permitted to contain
arbitrary values. If the upper 32 bits are non-zero, this could result
in an unexpected attempt to save/restore the FFR, and consequently could
lead to unexpected traps/undefs/faults.
In practice compilers are very unlikely to generate code where the upper
32-bits would be non-zero, but they are permitted to do so.
Fix this by only consuming the low 32 bits of the register, and update
comments accordingly.
The hyp code __sve_save_state() and __sve_restore_state() functions
don't have the same latent bug as they override the full 64-bit register
containing the argument.
Fixes: 9f5848665788 ("arm64/sve: Make access to FFR optional") Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Mark Brown <broonie@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Will Deacon <will@kernel.org> Cc: stable@vger.kernel.org Signed-off-by: Will Deacon <will@kernel.org>
Bluetooth: MGMT: Fix backward compatibility with userspace
bluetoothd has a bug with makes it send extra bytes as part of
MGMT_OP_ADD_EXT_ADV_DATA which are now being checked to be the
exact the expected length, relax this so only when the expected
length is greater than the data length to cause an error since
that would result in accessing invalid memory, otherwise just
ignore the extra bytes.
SeungJu Cheon [Mon, 1 Jun 2026 11:19:08 +0000 (20:19 +0900)]
Bluetooth: SCO: Fix data-race on sco_pi fields in sco_connect
sco_sock_connect() copies the destination address into sco_pi(sk)->dst
under lock_sock(), then releases the lock and calls sco_connect(),
which reads dst, src, setting, and codec without holding lock_sock() in
hci_get_route() and hci_connect_sco().
These fields may be modified concurrently by connect(), bind(), or
setsockopt() on the same socket, resulting in data-races reported by
KCSAN.
Fix this by snapshotting dst, src, setting, and codec under lock_sock()
at the start of sco_connect() before passing them to hci_get_route()
and hci_connect_sco().
BUG: KCSAN: data-race in memcmp+0x45/0xb0
race at unknown origin, with read to 0xffff88800e6b0dd0 of 1 bytes
by task 315 on cpu 0:
memcmp+0x45/0xb0
hci_connect_acl+0x1b7/0x6b0
hci_connect_sco+0x4d/0xb30
sco_sock_connect+0x27b/0xd60
__sys_connect_file+0xbd/0xe0
__sys_connect+0xe0/0x110
__x64_sys_connect+0x40/0x50
x64_sys_call+0xcad/0x1c60
do_syscall_64+0x133/0x590
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Fixes: 9a8ec9e8ebb5 ("Bluetooth: SCO: Fix possible circular locking dependency on sco_connect_cfm") Signed-off-by: SeungJu Cheon <suunj1331@gmail.com> Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
SeungJu Cheon [Mon, 1 Jun 2026 11:19:07 +0000 (20:19 +0900)]
Bluetooth: ISO: Fix data-race on iso_pi fields in hci_get_route calls
iso_connect_bis(), iso_connect_cis(), iso_listen_bis(), and
iso_conn_big_sync() call hci_get_route() using iso_pi(sk)->dst,
iso_pi(sk)->src, and iso_pi(sk)->src_type without holding lock_sock().
These fields may be modified concurrently by connect() or setsockopt()
on the same socket, resulting in data-races reported by KCSAN.
Fix this by snapshotting the required fields under lock_sock() before
calling hci_get_route().
BUG: KCSAN: data-race in memcmp+0x45/0xb0
race at unknown origin, with read to 0xffff8880122135cf of 1 bytes
by task 333 on cpu 1:
memcmp+0x45/0xb0
hci_get_route+0x27e/0x490
iso_connect_cis+0x4c/0xa10
iso_sock_connect+0x60e/0xb30
__sys_connect_file+0xbd/0xe0
__sys_connect+0xe0/0x110
__x64_sys_connect+0x40/0x50
x64_sys_call+0xcad/0x1c60
do_syscall_64+0x133/0x590
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Fixes: 241f51931c35 ("Bluetooth: ISO: Avoid circular locking dependency") Signed-off-by: SeungJu Cheon <suunj1331@gmail.com> Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
Bluetooth: ISO: Fix a use-after-free of the hci_conn pointer
In iso_sock_rebind_bc(), the bis pointer is cached, then the socket lock is
dropped:
bis = iso_pi(sk)->conn->hcon;
/* Release the socket before lookups since that requires hci_dev_lock
* which shall not be acquired while holding sock_lock for proper
* ordering.
*/
release_sock(sk);
hci_dev_lock(bis->hdev);
During the unlocked window, could a concurrent close() destroy the connection
and free the bis structure, causing hci_dev_lock(bis->hdev) to access memory
after it is freed, fix this by using the hdev reference which was safely
acquired via iso_conn_get_hdev().
Fixes: d3413703d5f8 ("Bluetooth: ISO: Add support to bind to trigger PAST") Reported-by: Sashiko <sashiko-bot@kernel.org> Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
Bluetooth: ISO: Fix not releasing hdev reference on iso_conn_big_sync
hci_get_route() returns a reference-counted hci_dev pointer via
hci_dev_hold(). The function exits normally or with an error without ever
releasing it.
Fixes: 07a9342b94a9 ("Bluetooth: ISO: Send BIG Create Sync via hci_sync") Reported-by: Sashiko <sashiko-bot@kernel.org> Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
Bharath Reddy [Mon, 1 Jun 2026 03:24:26 +0000 (08:54 +0530)]
Bluetooth: fix memory leak in error path of hci_alloc_dev()
Early failures in Bluetooth HCI UART configuration leak SRCU percpu
memory.
When device initialization fails before hci_register_dev() completes,
the HCI_UNREGISTER flag is never set. As a result, when the device
reference count reaches zero, bt_host_release() evaluates this flag as
false and falls back to a direct kfree(hdev).
Because hci_release_dev() is bypassed, the SRCU struct initialized
early in hci_alloc_dev() is never cleaned up, resulting in a leak of
percpu memory.
Fix the leak by explicitly calling cleanup_srcu_struct() in the
fallback (unregistered) branch of bt_host_release() before freeing
the device.
Reported-by: syzbot+535ecc844591e50588a5@syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=535ecc844591e50588a5 Tested-by: syzbot+535ecc844591e50588a5@syzkaller.appspotmail.com Fixes: 1d6123102e9f ("Bluetooth: hci_core: Fix use-after-free in vhci_flush()") Signed-off-by: Bharath Reddy <kbreddy.rpbc@gmail.com> Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
Zhang Cen [Fri, 29 May 2026 03:22:09 +0000 (11:22 +0800)]
Bluetooth: bnep: reject short frames before parsing
A BNEP peer can send a short BNEP SDU. bnep_rx_frame() reads the
packet type byte immediately and, for control packets, reads the control
opcode and setup UUID-size byte before proving that those bytes are
present. bnep_rx_control() also dereferences the control opcode without
rejecting an empty control payload.
Use skb_pull_data() for the fixed fields in bnep_rx_frame() so a NULL
return gates each dereference. Split the control handler so the frame
path can pass an opcode that has already been pulled, and keep the
byte-buffer wrapper for extension control payloads.
For BNEP_SETUP_CONN_REQ, name the UUID-size byte before pulling the
setup payload. struct bnep_setup_conn_req carries destination and source
service UUIDs after that byte, each uuid_size bytes, so the parser now
documents that tuple explicitly instead of leaving the pull length as an
opaque multiplication.
Validation reproduced this kernel report:
KASAN slab-out-of-bounds in bnep_rx_frame.isra.0+0x130c/0x1790
The buggy address belongs to the object at ffff88800c0f7908 which belongs
to the cache kmalloc-8 of size 8
The buggy address is located 0 bytes to the right of allocated 1-byte
region [ffff88800c0f7908, ffff88800c0f7909)
Read of size 1
Call trace:
dump_stack_lvl+0xb3/0x140 (?:?)
print_address_description+0x57/0x3a0 (?:?)
bnep_rx_frame+0x130c/0x1790 (net/bluetooth/bnep/core.c:306)
print_report+0xb9/0x2b0 (?:?)
__virt_addr_valid+0x1ba/0x3a0 (?:?)
srso_alias_return_thunk+0x5/0xfbef5 (?:?)
kasan_addr_to_slab+0x21/0x60 (?:?)
kasan_report+0xe0/0x110 (?:?)
process_one_work+0xfce/0x17e0 (kernel/workqueue.c:3200)
worker_thread+0x65c/0xe40 (?:?)
__kthread_parkme+0x184/0x230 (?:?)
kthread+0x35e/0x470 (?:?)
_raw_spin_unlock_irq+0x28/0x50 (?:?)
ret_from_fork+0x586/0x870 (?:?)
__switch_to+0x74f/0xdc0 (?:?)
ret_from_fork_asm+0x1a/0x30 (?:?)
Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2") Assisted-by: Codex:gpt-5.5 Signed-off-by: Zhang Cen <rollkingzzc@gmail.com> Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
Existing advertising instances can already hold the maximum extended
advertising payload. When hci_adv_bcast_annoucement() prepends the
Broadcast Announcement service data to that payload, the combined data
may no longer fit in the temporary buffer used to rebuild the
advertising data.
Reject that case before copying the existing payload and report the
failure through the device log. This keeps the existing advertising
data intact and avoids overrunning the temporary buffer.
Fixes: 5725bc608252 ("Bluetooth: hci_sync: Fix broadcast/PA when using an existing instance") Cc: stable@kernel.org Reported-by: Yuan Tan <yuantan098@gmail.com> Reported-by: Zhengchuan Liang <zcliangcn@gmail.com> Reported-by: Xin Liu <bird@lzu.edu.cn> Assisted-by: Codex:GPT-5.4 Signed-off-by: Yuqi Xu <xuyq21@lenovo.com> Signed-off-by: Ren Wei <n05ec@lzu.edu.cn> Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
Bluetooth: L2CAP: reject BR/EDR signaling packets over MTUsig
net/bluetooth/l2cap_core.c:l2cap_sig_channel() accepts BR/EDR
signaling packets up to the channel MTU and dispatches each command
without enforcing the signaling MTU (MTUsig). A Bluetooth BR/EDR peer
within radio range can send a fixed-channel CID 0x0001 packet that is
larger than MTUsig and contains many L2CAP_ECHO_REQ commands before
pairing. In a real-radio stock-kernel run, one 681-byte signaling
packet containing 168 zero-length ECHO_REQ commands made the target
transmit 168 ECHO_RSP frames over about 220 ms.
Impact: a Bluetooth BR/EDR peer within radio range, before pairing, can
force 168 ECHO_RSP frames from one 681-byte fixed-channel signaling
packet containing packed ECHO_REQ commands.
Define Linux's BR/EDR signaling MTU as the spec minimum of 48 bytes and
reject any larger signaling packet with one L2CAP_COMMAND_REJECT_RSP
carrying L2CAP_REJ_MTU_EXCEEDED before any command is dispatched.
The Bluetooth Core spec wording for MTUExceeded says the reject
identifier shall match the first request command in the packet, and
that packets containing only responses shall be silently discarded.
Linux intentionally deviates from that prescription: silently
discarding desynchronizes the peer because the remote stack never
learns its responses were dropped, and locating the first request
command requires walking command headers past MTUsig, i.e. processing
bytes from a packet we have already decided is too large to process.
We therefore always emit one reject and use the identifier from the
first command header, a single fixed-offset byte read.
The unrestricted BR/EDR signaling parser and ECHO_REQ response path both
trace to the initial git import; no later introducing commit is
available for a Fixes tag.
SeungJu Cheon [Mon, 25 May 2026 11:04:43 +0000 (20:04 +0900)]
Bluetooth: RFCOMM: validate skb length in MCC handlers
The RFCOMM MCC handlers cast skb->data to protocol-specific structs
without validating skb->len first. A malicious remote device can send
truncated MCC frames and trigger out-of-bounds reads in these handlers.
Fix this by using skb_pull_data() to validate and access the required
data before dereferencing it.
rfcomm_recv_rpn() requires special handling since ETSI TS 07.10 allows
1-byte RPN requests. Handle this by validating only the DLCI byte first,
and validating the full struct only when len > 1.
Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2") Suggested-by: Muhammad Bilal <meatuni001@gmail.com> Signed-off-by: SeungJu Cheon <suunj1331@gmail.com> Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
Zhang Cen [Thu, 28 May 2026 09:45:06 +0000 (17:45 +0800)]
Bluetooth: MGMT: validate advertising TLV before type checks
tlv_data_is_valid() reads each advertising data field length from
data[i], then inspects data[i + 1] for managed EIR types before
checking that the current field still fits inside the supplied buffer.
A malformed field whose length byte is the last byte of the buffer can
therefore make the parser read one byte past the advertising data.
KASAN reported the following when a malformed MGMT_OP_ADD_ADVERTISING
request reached that path:
BUG: KASAN: vmalloc-out-of-bounds in tlv_data_is_valid()
Read of size 1
Call trace:
tlv_data_is_valid()
add_advertising()
hci_mgmt_cmd()
hci_sock_sendmsg()
Move the existing element-length check before any type-octet inspection
so each non-empty element is proven to contain its type byte before the
parser looks at data[i + 1].
Fixes: 2bb36870e8cb ("Bluetooth: Unify advertising instance flags check") Reviewed-by: Paul Menzel <pmenzel@molgen.mpg.de> Signed-off-by: Zhang Cen <rollkingzzc@gmail.com> Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
Zhang Cen [Thu, 28 May 2026 07:56:41 +0000 (15:56 +0800)]
Bluetooth: RFCOMM: hold listener socket in rfcomm_connect_ind()
rfcomm_get_sock_by_channel() scans rfcomm_sk_list under the list lock,
but returns the selected listener after dropping that lock without
taking a reference. rfcomm_connect_ind() then locks the listener,
queues a child socket on it, and may notify it after unlocking it.
The buggy scenario involves two paths, with each column showing the
order within that path:
rfcomm_connect_ind(): listener close:
1. Find parent in 1. close() enters
rfcomm_get_sock_by_channel() rfcomm_sock_release().
2. Drop rfcomm_sk_list.lock 2. rfcomm_sock_shutdown()
without pinning parent. closes the listener.
3. Call lock_sock(parent) and 3. rfcomm_sock_kill()
bt_accept_enqueue(parent, unlinks and puts parent.
sk, true).
4. Read parent flags and may 4. parent can be freed.
call sk_state_change().
If close wins the race, parent can be freed before
rfcomm_connect_ind() reaches lock_sock(), bt_accept_enqueue(), or the
deferred-setup callback.
Take a reference on the listener before leaving rfcomm_sk_list.lock.
After lock_sock() succeeds, recheck that it is still in BT_LISTEN
before queueing a child, cache the deferred-setup bit while the parent
is locked, and drop the reference after the last parent use.
KASAN reported a slab-use-after-free in lock_sock_nested() from
rfcomm_connect_ind(), with the freeing stack going through
rfcomm_sock_kill() and rfcomm_sock_release().
Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2") Signed-off-by: Zhang Cen <rollkingzzc@gmail.com> Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
Chao Gao [Wed, 20 May 2026 22:29:04 +0000 (15:29 -0700)]
x86/virt/seamldr: Abort updates after a failed step
A TDX module update is a multi-step process, and any step can fail.
The current update flow continues to later steps after an error.
Continuing after a failure can cause the TDX module to enter an
unrecoverable state.
But certain failures during the initial module shutdown step should
simply return an error to userspace, so the update can be retried
cleanly.
To preserve that recoverability, one option would be to abort the
update only for those failures, since they occur before any TDX module
state is changed. But special-casing specific failures in specific
steps would complicate the do-while() update loop for no benefit.
Simply abort update on any failure, at any step.
Track failures for each step, stop the update loop once a failure is
observed, and do not advance the state machine to the next step.
Chao Gao [Wed, 20 May 2026 22:29:02 +0000 (15:29 -0700)]
x86/virt/seamldr: Introduce skeleton for TDX module updates
tl;dr: Use stop_machine() and a state machine based on the
"MULTI_STOP" pattern to implement core TDX module update logic.
Long version:
TDX module updates require careful synchronization with other TDX
operations. The requirements are (#1/#2 reflect current behavior that
must be preserved):
1. SEAMCALLs need to be callable from both process and IRQ contexts.
2. SEAMCALLs need to be able to run concurrently across CPUs
3. During updates, only update-related SEAMCALLs are permitted; all
other SEAMCALLs shouldn't be called.
4. During updates, all online CPUs must participate in the update work.
No single lock primitive satisfies all requirements. For instance,
rwlock_t handles #1/#2 but fails #4: CPUs spinning with IRQs disabled
cannot be directed to perform update work.
Use stop_machine() as it is the only well-understood mechanism that can
meet all requirements.
And TDX module updates consist of several steps (See Intel Trust Domain
Extensions (Intel TDX) Module Base Architecture Specification, Chapter
"TD-Preserving TDX module Update"). Ordering requirements between steps
mandate lockstep synchronization across all CPUs.
multi_cpu_stop() provides a good example of executing a multi-step task
in lockstep across CPUs, but it does not synchronize the individual
steps inside the callback itself.
Implement a similar state machine as the skeleton for TDX module
updates. Each state represents one step in the update flow, and the
state advances only after all CPUs acknowledge completion of the current
step. This acknowledgment mechanism provides the required lockstep
execution.
The update flow is intentionally simpler than multi_cpu_stop() in two ways:
a) use a spinlock to protect the control data instead of atomic_t and
explicit memory barriers.
b) omit touch_nmi_watchdog() and rcu_momentary_eqs(), which exist
there for debugging and are not strictly needed for this update flow
Potential alternative to stop_machine()
=======================================
An alternative approach is to lock all KVM entry points and kick all
vCPUs. Here, KVM entry points refer to KVM VM/vCPU ioctl entry points,
implemented in KVM common code (virt/kvm). Adding a locking mechanism
there would affect all architectures KVM supports. And to lock only TDX
vCPUs, new logic would be needed to identify TDX vCPUs, which the KVM
common code currently lacks. This would add significant complexity and
maintenance overhead to KVM for this TDX-specific use case, so don't take
this approach.
[ dhansen: normal changelog/style munging ]
Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Link: https://patch.msgid.link/20260520133909.409394-15-chao.gao@intel.com
Chao Gao [Wed, 20 May 2026 22:29:01 +0000 (15:29 -0700)]
x86/virt/seamldr: Allocate and populate a module update request
There are two important ABIs here:
'struct tdx_image' - The on-disk and in-memory format for a TDX
module update image.
'struct seamldr_params' - The in-memory ABI passed to the TDX module
loader. Points to a single 'struct tdx_image'
broken up into 4k pages.
Userspace supplies the update image in 'struct tdx_image' format. The
image consists of a header followed by a sigstruct and the module
binary. P-SEAMLDR, however, consumes 'struct seamldr_params' rather
than the image directly.
Parse the 'struct tdx_image' provided by userspace and populate a
matching 'struct seamldr_params'.
The 'tdx_image' ABI is versioned. Two public versions exist today:
0x100 and 0x200. This kernel only accepts 0x200. The older 0x100
format is being deprecated and is intentionally not supported here.
Future versions of the module might be able to use the same ABIs
(user/kernel and kernel/SEAMLDR) but they will not be able to use this
kernel code.
Reject module images without that specific version. This ensures that
the kernel is able to understand the passed-in format.
Validate the 'struct tdx_image' header before using it, because the
header is consumed solely by the kernel to locate the sigstruct and
module within the image. Do not validate the payload itself. The
sigstruct and module pages are passed through to P-SEAMLDR, which
validates them as part of the update.
sigstruct_pages_pa_list currently has only one entry, but it will grow
to four pages in the future. Keep it as an array for symmetry with
module_pages_pa_list and for extensibility.
[ dhansen: normal changelog clarification/munging ]
Chao Gao [Wed, 20 May 2026 22:29:00 +0000 (15:29 -0700)]
coco/tdx-host: Implement firmware upload sysfs ABI for TDX module updates
tl;dr: Select fw_upload for doing TDX module updates. The process of
selecting among available update images is complicated and nuanced. Punt
the selection process out to userspace. One existing userspace
implementation today is the script in the Intel TDX Module Binaries
repository[1].
Long Version:
The kernel supports two primary firmware update mechanisms:
1. request_firmware() - used by microcode, SEV firmware, hundreds of
other drivers
2. 'struct fw_upload' - used by CXL, FPGA updates, dozens of others
The key difference between is that request_firmware() loads a named file
from the filesystem where the filename is kernel-controlled, while
fw_upload accepts firmware data directly from userspace.
TDX module firmware update selection policy is too complex for the kernel.
Leave it to userspace and use fw_upload.
Add a skeleton fw_upload implementation to be fleshed out in subsequent
patches.
Refactor the sysfs visiblity attribute function so it can be used as a
more generic flag for the presence of viable runtime update support.
Why fw_upload instead of request_firmware()?
============================================
Selecting a TDX module update image is not a simple "load the latest"
decision. Userspace needs to choose an image that is compatible with both
the platform and the currently running module.
Some constraints are hard requirements:
a. Module version series are platform-specific. For example, the 1.5.x
series runs on Sapphire Rapids but not Granite Rapids, which needs
2.0.x.
b. Updates are also constrained by version distance. A 1.5.6 module
might permit updates to 1.5.7 but not to 1.5.50.
There may also be userspace policy choices:
c. Decide the update direction: upgrade or downgrade
d. Choose whether to optimize for fewer updates or smaller version
steps, for example, 1.2.3=>1.2.5 versus 1.2.3=>1.2.4=>1.2.5.
Given that complexity, leave module selection to userspace and use
fw_upload.
Chao Gao [Wed, 20 May 2026 22:28:59 +0000 (15:28 -0700)]
coco/tdx-host: Don't expose P-SEAMLDR information on CPUs with erratum
TDX-capable CPUs clobber the current VMCS on P-SEAMLDR calls. Clearing
the current VMCS behind KVM's back breaks KVM.
Future CPUs will fix this by preserving the current VMCS across
P-SEAMLDR calls. A future specification update will describe the
VMCS-clearing behavior as an erratum and to state that it does not
occur when IA32_VMX_BASIC[60] is set.
Add a CPU bug bit and refuse to expose P-SEAMLDR information on
affected CPUs.
Use a CPU bug bit to stay consistent with X86_BUG_TDX_PW_MCE. As a
bonus, the bug bit is visible to userspace, which allows userspace to
determine why these sysfs files are not exposed, and it can also be
checked by other kernel components in the future if needed.
== Alternatives ==
Two workarounds were considered but both were rejected:
1. Save/restore the current VMCS around P-SEAMLDR calls. This produces ugly
assembly code [1] and doesn't play well with #MCE or #NMI if they
need to use the current VMCS.
2. Move KVM's VMCS tracking logic to the TDX core code, which would break
the boundary between KVM and the TDX core code [2].
Chao Gao [Wed, 20 May 2026 22:28:57 +0000 (15:28 -0700)]
coco/tdx-host: Expose P-SEAMLDR information via sysfs
TDX module updates require userspace to select the appropriate module
to load. Expose necessary information to facilitate this decision. Two
values are needed:
- P-SEAMLDR version: for compatibility checks between TDX module and
P-SEAMLDR
- num_remaining_updates: indicates how many updates can be performed
Expose them as tdx-host device attributes visible only when updates
are supported.
Note that the underlying P-SEAMLDR attributes are available regardless
of update support; this only restricts their visibility to userspace.
Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://patch.msgid.link/20260520133909.409394-11-chao.gao@intel.com
Chao Gao [Wed, 20 May 2026 22:28:56 +0000 (15:28 -0700)]
x86/virt/seamldr: Add a helper to retrieve P-SEAMLDR information
P-SEAMLDR reports its state via SEAMLDR.INFO, including its version and
the number of remaining runtime updates.
This information is useful for userspace. For example, userspace can
use the P-SEAMLDR version to determine whether a candidate TDX module
is compatible with the running loader, and can use the remaining
update count to determine whether another runtime update is still
possible.
Add a helper to retrieve P-SEAMLDR information in preparation for
exposing P-SEAMLDR version and other necessary information to userspace.
Export the new kAPI for use by the "tdx_host" device.
Note that there are two distinct P-SEAMLDR APIs with similar names:
"SEAMLDR.INFO" is metadata about the loader. It's metadata for the
update process.
"SEAMLDR.SEAMINFO" is metadata about SEAM mode. It is for the module
init process, not for the update process.
Use SEAMLDR.INFO here.
For details, see "Intel Trust Domain Extensions - SEAM Loader (SEAMLDR)
Interface Specification".
Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://patch.msgid.link/20260520133909.409394-10-chao.gao@intel.com
Chao Gao [Wed, 20 May 2026 22:28:55 +0000 (15:28 -0700)]
x86/virt/seamldr: Introduce a wrapper for P-SEAMLDR SEAMCALLs
The TDX architecture uses the "SEAMCALL" instruction to communicate with
SEAM mode software. Right now, the only SEAM mode software that the kernel
communicates with is the TDX module. But, there is actually another
component that runs in SEAM mode but it is separate from the TDX module:
the persistent SEAM loader or "P-SEAMLDR". Right now, the only component
that communicates with it is the BIOS which loads the TDX module itself at
boot. But, to support updating the TDX module, the kernel now needs to be
able to talk to it.
P-SEAMLDR SEAMCALLs differ from TDX module SEAMCALLs in areas such as
concurrency requirements.
Add a P-SEAMLDR wrapper to handle these differences and prepare for
implementing concrete functions.
Use seamcall_prerr() (not '_ret') because current P-SEAMLDR calls do not
use any output registers other than RAX.
Note: Despite the similar name, the NP-SEAMLDR ("Non-Persistent")
(ACM) invoked exclusively by the BIOS at boot rather than a component
running in SEAM mode. The kernel cannot call it at runtime. It exposes
no SEAMCALL interface.
Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://cdrdv2.intel.com/v1/dl/getContent/733582 Link: https://patch.msgid.link/20260520133909.409394-9-chao.gao@intel.com
Chao Gao [Wed, 20 May 2026 22:28:53 +0000 (15:28 -0700)]
coco/tdx-host: Expose TDX module version
For TDX module updates, userspace needs to select compatible update
versions based on the current module version.
For example, the 1.5.x series runs on Sapphire Rapids but not Granite
Rapids, which needs 2.0.x. Updates are also constrained by version
distance, so a 1.5.6 module might permit updates to 1.5.7 but not to
1.5.20.
Start the process of punting the version selection logic to userspace.
Expose the TDX module version in the new faux device.
Define TDX_VERSION_FMT macro for the TDX version format since it will be
used multiple times. Also convert an existing print statement to use it.
== Background ==
For posterity, here's what other firmware mechanisms do:
1. AMD SEV leverages an existing PCI device for the PSP to expose
metadata. TDX uses a faux device as it doesn't have PCI device
in its architecture.
2. Microcode uses per-CPU virtual devices to report microcode revisions
because CPUs can have different revisions. But, there is only a
single TDX module, so exposing the TDX module version through a global
TDX faux device is appropriate
3. ARM's CCA implementation isn't in-tree yet, but will likely follow a
similar faux device approach, though it's unclear whether they need
to expose firmware version information
[ dhansen: trim changelog ]
Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://lore.kernel.org/all/2025073035-bulginess-rematch-b92e@gregkh/ Link: https://patch.msgid.link/20260520133909.409394-8-chao.gao@intel.com
Chao Gao [Wed, 20 May 2026 22:28:52 +0000 (15:28 -0700)]
coco/tdx-host: Introduce a "tdx_host" device
TDX depends on a platform firmware module that runs on the CPU.
Unlike other CoCo architectures, TDX has no hardware "device"
running the show, just a blob on the CPU.
Create a virtual device to anchor interactions with this platform
firmware. This lets later code:
- expose metadata: TDX module version, seamldr version, to userspace
as device attributes
- implement firmware uploader APIs (which are tied to a device) to
support TDX module runtime updates
Use a faux device because the TDX module is singular within the system
and has no platform resources. Using a faux device eliminates the need
to create a stub bus.
The call to tdx_get_sysinfo() ensures that the TDX module is ready to
provide services.
Note that AMD has a PCI device for the PSP for SEV and ARM CCA will
likely have a faux device [1].
Thanks to Dan and Yilun for all the help on this one.
[ dhansen: trim changelog ]
Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Link: https://lore.kernel.org/all/2025073035-bulginess-rematch-b92e@gregkh/ Link: https://patch.msgid.link/20260520133909.409394-7-chao.gao@intel.com
Kai Huang [Wed, 20 May 2026 22:28:51 +0000 (15:28 -0700)]
x86/virt/tdx: Move low level SEAMCALL helpers out of <asm/tdx.h>
TDX host core code implements three seamcall*() helpers to make SEAMCALLs
to the TDX module. Currently, they are implemented in <asm/tdx.h> and
are exposed to other kernel code which includes <asm/tdx.h>.
However, other than the TDX host core, seamcall*() are not expected to
be used by other kernel code directly. For instance, for all SEAMCALLs
that are used by KVM, the TDX host core exports a wrapper function for
each of them.
Move seamcall*() and related code out of <asm/tdx.h> and make them only
visible to TDX host core.
Since TDX host core tdx.c is already very heavy, don't put low level
seamcall*() code there but to a new dedicated "seamcall_internal.h". Also,
currently tdx.c has seamcall_prerr*() helpers which additionally print
error message when calling seamcall*() fails. Move them to
"seamcall_internal.h" as well. In such way all low level SEAMCALL helpers
are in a dedicated place, which is much more readable.
Copy the copyright notice from the original files and consolidate the
date ranges to:
Copyright (C) 2021-2023 Intel Corporation
Signed-off-by: Kai Huang <kai.huang@intel.com> Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Zhenzhong Duan <zhenzhong.duan@intel.com> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Vishal Annapurve <vannapurve@google.com> Acked-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://patch.msgid.link/20260520133909.409394-6-chao.gao@intel.com
Chao Gao [Wed, 20 May 2026 22:28:49 +0000 (15:28 -0700)]
x86/virt/tdx: Move TDX_FEATURES0 bits to asm/tdx.h
Future changes will add support for new TDX features exposed as
TDX_FEATURES0 bits. The presence of these features will need to be
checked outside of arch/x86/virt. The feature query helpers and
the TDX_FEATURES0 defines they reference will need to live in the
widely accessible asm/tdx.h header. Move the existing TDX_FEATURES0 to
asm/tdx.h so that they can all be kept together.
Opportunistically switch to BIT_ULL() since TDX_FEATURES0 is 64-bit.
Chao Gao [Wed, 20 May 2026 22:28:48 +0000 (15:28 -0700)]
x86/virt/tdx: Consolidate TDX global initialization states
The kernel uses several global flags to guard one-time TDX initialization
flows and prevent them from being repeated.
When the TDX module is updated, all of those states must be reset so that
the module can be initialized again. Today those states are kept as
separate global variables, which makes the reset path awkward and easy to
miss when a new state is added.
Group the states into a single structure so they can be reset together, for
example with memset(), and so a newly added state won't be missed.
Drop the __ro_after_init annotation from tdx_module_initialized because
the other two states do not have it. And with TDX module update support,
all the states need to be writable at runtime.
Chao Gao [Wed, 20 May 2026 22:28:47 +0000 (15:28 -0700)]
x86/virt/tdx: Move TDX global initialization states to file scope
TDX module global initialization is executed only once. The first call
caches both the result and the "done" state, and later callers reuse the
saved result. A lock protects that cached states.
Those states and the lock are currently kept as function-local statics
because they are used only by try_init_module_global().
TDX module updates need to reset the cached states so TDX global
initialization can be run again after an update. That will add another
access site in the same file.
Move the cached states to file scope so it is accessible outside
try_init_module_global(), and move the lock along with the states it
protects.
Chao Gao [Wed, 20 May 2026 22:28:46 +0000 (15:28 -0700)]
x86/virt/tdx: Clarify try_init_module_global() result caching
TDX module global initialization is executed only once. The first call
caches both the return code and the "done" state in static function
variables. Later callers read the variables. A lock protects the
saved state and serializes callers.
These variables will soon be moved to a global structure. Prepare for
that by treating the variables as a unit. Assign them together and
limit accesses to while the lock is held.
Paolo Bonzini [Wed, 3 Jun 2026 15:00:06 +0000 (17:00 +0200)]
Merge branch 'kvm-ghcb-for-7.2' into HEAD
Merge the final part of the GHCB 7.2 fixes at
https://lore.kernel.org/kvm/20260529183549.1104619-1-pbonzini@redhat.com/.
Patches 1-17 have already been included in Linux 7.1; these are minor
cleanups, and fixes for behaviors that are suboptimal or contradicting
the specification.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
KVM: SEV: Remove sometimes-used function-scoped "ret" from #VMGEXIT handler
Now that only two case-statements actually need a local "ret" variable,
refactor sev_handle_vmgexit() to have all flows return directly when
possible, and bury "ret" as "r" in the two paths that need to propagate a
return value from a helper.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-25-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-25-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
KVM: SEV: Turn sev_es_validate_vmgexit() into a dedicated predicate
Now that sev_es_validate_vmgexit() is only responsible for checking that
all required GHCB fields are marked valid, turn it into a predicate whose
name reflects exactly that.
No functional change intended.
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Reviewed-by: Michael Roth <michael.roth@amd.com> Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-24-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-24-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
KVM: SEV: Handle unknown #VMGEXIT reasons in sev_handle_vmgexit()
Handle unknown #VMGEXIT reasons in sev_handle_vmgexit(), not in
sev_es_validate_vmgexit(). This makes it _much_ more obvious that KVM
simply funnels "legacy" exits to the standard SVM interception handlers,
and is the final preparatory change needed to reduce the scope of
sev_es_validate_vmgexit().
No functional change intended.
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-23-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-23-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
KVM: SEV: Return INVALID_INPUT, not MISSING_INPUT, for bad GUEST_REQUEST input(s)
Return INVALID_INPUT, not MISSING_INPUT, if the guest provides an unaligned
address for a GUEST_REQUEST, and/or attempts to use the same page for the
source and destination. The inputs are obviously invalid, not missing.
Opportunistically move the checks out of sev_es_validate_vmgexit(), to
continue the march towards reducing the scope of the helper, and to help
guide future changes into correctly handling bad input.
Fixes: 88caf544c930 ("KVM: SEV: Provide support for SNP_GUEST_REQUEST NAE event") Fixes: 74458e4859d8 ("KVM: SEV: Provide support for SNP_EXTENDED_GUEST_REQUEST NAE event") Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Reviewed-by: Michael Roth <michael.roth@amd.com> Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-22-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-22-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
KVM: SEV: Return INVALID_EVENT for SNP-only #VMGEXIT from non-SNP guest
Signal INVALID_EVENT, not MISSING_INPUT, if a non-SNP guest attempts to
invoke an SNP-only #VMGEXIT. Opportunistically move the checks out of
sev_es_validate_vmgexit() to continue the march towards making said helper
a predicate whose sole purpose is to verify the guest has marked required
GHCB fields as valid.
Fixes: e366f92ea99e ("KVM: SEV: Support SEV-SNP AP Creation NAE event") Fixes: 9b54e248d264 ("KVM: SEV: Add support to handle Page State Change VMGEXIT") Fixes: 88caf544c930 ("KVM: SEV: Provide support for SNP_GUEST_REQUEST NAE event") Fixes: 74458e4859d8 ("KVM: SEV: Provide support for SNP_EXTENDED_GUEST_REQUEST NAE event") Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Reviewed-by: Michael Roth <michael.roth@amd.com> Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-21-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-21-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
KVM: SEV: Move GHCB "usage" check out of sev_es_validate_vmgexit()
Move the check to verify the guest's requested GHCB out of
sev_es_validate_vmgexit() as the first step towards making said helper a
predicate whose sole purpose is to verify the guest has marked required
GHCB fields as valid.
Using a single "validate" helper sounds good on paper, but in practice it's
difficult to verify that KVM is performing the necessary sanity checks (the
usage of state is far removed from the relevant checks), makes it difficult
to understand that "legacy" exits are simply routed to KVM's existing exit
handlers, and most importantly, has directly contributed to a number of
bugs as adding case-statements to the validation subtly removes them from
the default path that rejects unknown exit codes with INVALID_EVENT.
Deliberately extract the usage code check first so as to preserve the order
of KVM's checks, even though future code extraction will technically fix
bugs.
No functional change intended.
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Reviewed-by: Michael Roth <michael.roth@amd.com> Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-20-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-20-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
KVM: SEV: Don't terminate SNP VMs on #VMGEXIT without a registered GHCB
If the guest attempts a non-MSR #VMGEXIT without the registered GHCB,
return a GHCB_HV_RESP_MALFORMED_INPUT+GHCB_ERR_NOT_REGISTERED error to the
guest instead of exiting KVM_RUN with -EINVAL (and in likelihood killing
the VM). KVM has already mapped the requested GHCB, i.e. can cleanly
report an error, and so exiting with -EINVAL is completely unjustified.
Fixes: 0c76b1d08280 ("KVM: SEV: Add support to handle GHCB GPA register VMGEXIT") Cc: stable@vger.kernel.org Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Reviewed-by: Michael Roth <michael.roth@amd.com> Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-19-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-19-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Eliot Courtney [Wed, 3 Jun 2026 07:30:25 +0000 (16:30 +0900)]
gpu: nova-core: add non-sec2 unload path
For non-sec2 it is only required to wait for GSP falcon to halt. This is
because GSP does the main work of unloading on GPUs not using sec2.
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
[ jhubbard: use Result instead of Result<()> in the UnloadBundle impl ] Signed-off-by: John Hubbard <jhubbard@nvidia.com> Link: https://patch.msgid.link/20260603-b4-blackwell-v13-8-d9f3a06939e0@nvidia.com Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
On Hopper and Blackwell, FSP boots GSP with hardware lockdown enabled.
After FSP Chain of Trust completes, the driver must poll for lockdown
release before proceeding with GSP initialization. Add the register
bit and helper functions needed for this polling.
KVM: SEV: Unmap and unpin the GHCB as needed on vCPU free
Unmap and unpin the GHCB as needed when freeing a vCPU. If the VM is
destroyed after mapping+pinning the GHCB on #VMGEXIT, without re-running
the vCPU, KVM will effectively leak the GHCB and any mappings created for
the GHCB.
Fixes: 291bd20d5d88 ("KVM: SVM: Add initial support for a VMGEXIT VMEXIT") Cc: stable@vger.kernel.org Tested-by: Michael Roth <michael.roth@amd.com> Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Reviewed-by: Michael Roth <michael.roth@amd.com> Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-18-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-18-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
KVM: SEV: Move sev_free_vcpu() down below sev_es_unmap_ghcb()
Relocate sev_free_vcpu() down in sev.c so that it's definition comes after
sev_es_unmap_ghcb(). This will allow sharing unmap functionality between
the two functions without needing a forward declaration (or weird placement
of the common code).
No functional change intended.
Cc: stable@vger.kernel.org Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Reviewed-by: Michael Roth <michael.roth@amd.com> Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-16-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-16-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
KVM: Don't WARN if memory is dirtied without a vCPU when the VM is dying
When marking a page dirty, complain about not having a running/loaded vCPU
if and only if the VM is still alive, i.e. its refcount is non-zero. This
will allow fixing a memory leak for x86 SEV-ES guests without hitting what
is effectively a false positive on the WARN.
For some SEV-ES VM-Exits, KVM keeps a writable mapping of a guest page
across an exit to userspace, and typically unmaps the page on the next
KVM_RUN. But if userspace never calls KVM_RUN after such an exit, then KVM
needs to unmap the page when the vCPU is destroyed, which in turn triggers
the WARN about not having a running vCPU.
Alternatively, SEV-ES could temporarily load the vCPU to suppress the WARN,
as is done in nested_vmx_free_vcpu() (but for completely unrelated reasons;
suppressing WARN from nested_put_vmcs12_pages() is pure happenstance). But
loading a vCPU during destruction is gross (ideally nVMX code would be
cleaned up), risks complicating the SEV-ES code (KVM would need to ensure
the temporarily load()+put() only runs when the vCPU isn't already loaded),
and is ultimately pointless.
The motivation for the WARN is to guard against KVM dirtying guest memory
without pushing the corresponding GFN to the active vCPU's dirty ring, e.g.
to ensure userspace doesn't miss a dirty page. But for the VM's refcount
to reach zero, there can't be _any_ userspace mappings to the dirty ring,
as mapping the dirty ring requires doing mmap() on the vCPU FD. I.e. if
userspace had a valid mapping for the dirty ring, then the vCPU file and
thus the owning VM would still be alive. And so since userspace can't
possibly reach the dirty ring, whether or not KVM technically "misses" a
push to the dirty ring is irrelevant.
Reported-by: Michael Roth <michael.roth@amd.com> Cc: stable@vger.kernel.org Reviewed-by: Michael Roth <michael.roth@amd.com> Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-15-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-15-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
KVM: SEV: Read start/end indices of PSC requests exactly once per #VMGEXIT
Rework Page State Change (PSC) handling to read the guest-provided start
and end indices exactly once, at the beginning of the request. Re-reading
the indices is "fine", _if_ the guest is well-behaved. KVM _should_ be
safe against concurrent guest modification of the indices, but there is
zero reason to introduce unnecessary risk.
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Reviewed-by: Michael Roth <michael.roth@amd.com> Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-14-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-14-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
KVM: SEV: Add an anonymous "psc" struct to track current PSC metadata
Add a "psc" struct to vcpu_sev_es_state to avoid having to prefix all of
the fields with "psc_".
Take advantage of the code churn to opportunistically rename local
variables to "guest_psc" to make it more obvious that the buffer is guest
data, and more importantly, guest accessible!
Opportunistically rename inflight => batch_size as well, because there can
really only be one operation in-flight (per-vCPU), i.e. "inflight" _looks_
like a boolean, but in actuality is an integer tracking how many pages are
being handled by the current operation.
No functional change intended.
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-13-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-13-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
KVM: SEV: Make it more obvious when KVM is writing back the current PSC index
Increment the guest-visible "cur_entry" index outside of the for-loop
when processing Page State Change entries, and add a comment to make it
more obvious which code is operating on trusted data, and which code is
touching guest-accessible data.
No functional change intended.
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Reviewed-by: Michael Roth <michael.roth@amd.com> Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260501202250.2115252-12-seanjc@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260529183549.1104619-12-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Guan-Chun Wu [Sun, 31 May 2026 08:00:19 +0000 (16:00 +0800)]
ext4: improve str2hashbuf by processing 4-byte chunks and removing function pointers
The original byte-by-byte implementation with modulo checks is less
efficient. Refactor str2hashbuf_unsigned() and str2hashbuf_signed()
to process input in explicit 4-byte chunks instead of using a
modulus-based loop to emit words byte by byte.
Additionally, the use of function pointers for selecting the appropriate
str2hashbuf implementation has been removed. Instead, the functions are
directly invoked based on the hash type, eliminating the overhead of
dynamic function calls.
Performance test (x86_64, Intel Core i7-10700 @ 2.90GHz, average over 10000
runs, using kernel module for testing):
Guan-Chun Wu [Sun, 31 May 2026 08:00:18 +0000 (16:00 +0800)]
ext4: add Kunit coverage for directory hash computation
Introduce Kunit tests for fs/ext4/hash.c to verify ext4fs_dirhash()
across the legacy, half-MD4, and TEA hash variants.
The tests cover empty, seeded hashing, and non-ASCII name handling.
They also verify error paths, including invalid hash versions and
SipHash without a configured key, and check that the signed and
unsigned hash variants differ on non-ASCII input as expected.
When CONFIG_UNICODE is enabled, the tests further verify casefolded-name
hashing and the fallback behavior for invalid input.
projects / thirdparty / kernel / linux.git / log
