Giovanni Cabiddu [Thu, 17 Jul 2025 10:05:43 +0000 (11:05 +0100)]
crypto: qat - make adf_dev_autoreset() static
The function adf_dev_autoreset() is only used within adf_aer.c and does
not need to be exposed outside the compilation unit. Make it static and
remove it from the header adf_common_drv.h.
This does not introduce any functional change.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Ahsan Atta <ahsan.atta@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: ccp - reduce stack usage in ccp_run_aes_gcm_cmd
A number of functions in this file have large structures on the stack,
ccp_run_aes_gcm_cmd() being the worst, in particular when KASAN
is enabled on gcc:
drivers/crypto/ccp/ccp-ops.c: In function 'ccp_run_sha_cmd':
drivers/crypto/ccp/ccp-ops.c:1833:1: error: the frame size of 1136 bytes is larger than 1024 bytes [-Werror=frame-larger-than=]
drivers/crypto/ccp/ccp-ops.c: In function 'ccp_run_aes_gcm_cmd':
drivers/crypto/ccp/ccp-ops.c:914:1: error: the frame size of 1632 bytes is larger than 1024 bytes [-Werror=frame-larger-than=]
Avoid the issue by using dynamic memory allocation in the worst one
of these.
Signed-off-by: Arnd Bergmann <arnd@arndb.de> Acked-by: Tom Lendacky <thomas.lendacky@amd.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Giovanni Cabiddu [Mon, 14 Jul 2025 07:10:29 +0000 (08:10 +0100)]
crypto: qat - fix seq_file position update in adf_ring_next()
The `adf_ring_next()` function in the QAT debug transport interface
fails to correctly update the position index when reaching the end of
the ring elements. This triggers the following kernel warning when
reading ring files, such as
/sys/kernel/debug/qat_c6xx_<D:B:D:F>/transport/bank_00/ring_00:
[27725.022965] seq_file: buggy .next function adf_ring_next [intel_qat] did not update position index
Ensure that the `*pos` index is incremented before returning NULL when
after the last element in the ring is found, satisfying the seq_file API
requirements and preventing the warning.
Fixes: a672a9dc872e ("crypto: qat - Intel(R) QAT transport code") Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Ahsan Atta <ahsan.atta@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Giovanni Cabiddu [Mon, 14 Jul 2025 07:07:49 +0000 (08:07 +0100)]
crypto: qat - fix DMA direction for compression on GEN2 devices
QAT devices perform an additional integrity check during compression by
decompressing the output. Starting from QAT GEN4, this verification is
done in-line by the hardware. However, on GEN2 devices, the hardware
reads back the compressed output from the destination buffer and performs
a decompression operation using it as the source.
In the current QAT driver, destination buffers are always marked as
write-only. This is incorrect for QAT GEN2 compression, where the buffer
is also read during verification. Since commit 6f5dc7658094
("iommu/vt-d: Restore WO permissions on second-level paging entries"),
merged in v6.16-rc1, write-only permissions are strictly enforced, leading
to DMAR errors when using QAT GEN2 devices for compression, if VT-d is
enabled.
Mark the destination buffers as DMA_BIDIRECTIONAL. This ensures
compatibility with GEN2 devices, even though it is not required for
QAT GEN4 and later.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Fixes: cf5bb835b7c8 ("crypto: qat - fix DMA transfer direction") Reviewed-by: Ahsan Atta <ahsan.atta@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: jitter - replace ARRAY_SIZE definition with header include
The ARRAY_SIZE macro is already defined in linux/array_size.h
This patch replaces the ARRAY_SIZE definition in jitterentropy.c with
an include, to make the code cleaner, and help reduce the number of
duplicate ARRAY_SIZE definitions in the codebase.
Signed-off-by: Ruben Wauters <rubenru09@aol.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The {prepare,unprepare}_crypt_hardware callbacks were added back in 2016
by commit 735d37b5424b ("crypto: engine - Introduce the block request
crypto engine framework"), but they were never implemented by any driver.
Remove them as they are unused.
Since the 'engine->idling' and 'was_busy' flags are no longer needed,
remove them as well.
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Remove request batching support from crypto_engine, as there are no
drivers using this feature and it doesn't really work that well.
Instead of doing batching based on backlog, a more optimal approach
would be for the user to handle the batching (similar to how IPsec
can hook into GSO to get 64K of data each time or how block encryption
can use unit sizes much greater than 4K).
Suggested-by: Herbert Xu <herbert@gondor.apana.org.au> Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com> Reviewed-by: Horia Geantă <horia.geanta@nxp.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Giovanni Cabiddu [Fri, 11 Jul 2025 12:27:43 +0000 (13:27 +0100)]
crypto: qat - flush misc workqueue during device shutdown
Repeated loading and unloading of a device specific QAT driver, for
example qat_4xxx, in a tight loop can lead to a crash due to a
use-after-free scenario. This occurs when a power management (PM)
interrupt triggers just before the device-specific driver (e.g.,
qat_4xxx.ko) is unloaded, while the core driver (intel_qat.ko) remains
loaded.
Since the driver uses a shared workqueue (`qat_misc_wq`) across all
devices and owned by intel_qat.ko, a deferred routine from the
device-specific driver may still be pending in the queue. If this
routine executes after the driver is unloaded, it can dereference freed
memory, resulting in a page fault and kernel crash like the following:
BUG: unable to handle page fault for address: ffa000002e50a01c
#PF: supervisor read access in kernel mode
RIP: 0010:pm_bh_handler+0x1d2/0x250 [intel_qat]
Call Trace:
pm_bh_handler+0x1d2/0x250 [intel_qat]
process_one_work+0x171/0x340
worker_thread+0x277/0x3a0
kthread+0xf0/0x120
ret_from_fork+0x2d/0x50
To prevent this, flush the misc workqueue during device shutdown to
ensure that all pending work items are completed before the driver is
unloaded.
Note: This approach may slightly increase shutdown latency if the
workqueue contains jobs from other devices, but it ensures correctness
and stability.
Fixes: e5745f34113b ("crypto: qat - enable power management for QAT GEN4") Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Cc: stable@vger.kernel.org Reviewed-by: Ahsan Atta <ahsan.atta@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qat - enable rate limiting feature for GEN6 devices
Add support for enabling rate limiting(RL) feature for QAT GEN6 by
initializing the rl_data member in adf_hw_device_data structure.
Implement init_num_svc_aes() for GEN6 which will populate the number of
AEs associated with the RL service type.
Implement adf_gen6_get_svc_slice_cnt() for GEN6 which will return
the slice count that can support the RL service type.
Co-developed-by: George Abraham P <george.abraham.p@intel.com> Signed-off-by: George Abraham P <george.abraham.p@intel.com> Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qat - add compression slice count for rate limiting
In QAT GEN4 devices, the compression slice count was tracked using the
dcpr_cnt field.
Introduce a new cpr_cnt field in the rate limiting (RL) infrastructure to
track the compression (CPR) slice count independently. The cpr_cnt value is
populated via the RL_INIT admin message.
The existing dcpr_cnt field will now be used exclusively to cache the
decompression slice count, ensuring a clear separation between compression
and decompression tracking.
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qat - add get_svc_slice_cnt() in device data structure
Enhance the adf_hw_device_data structure by introducing a new callback
function get_svc_slice_cnt(), which provides a mechanism to query the
total number of accelerator available on the device for a specific
service.
Implement adf_gen4_get_svc_slice_cnt() for QAT GEN4 devices to support this
new interface. This function returns the total accelerator count for a
specific service.
Co-developed-by: George Abraham P <george.abraham.p@intel.com> Signed-off-by: George Abraham P <george.abraham.p@intel.com> Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qat - add adf_rl_get_num_svc_aes() in rate limiting
Enhance the rate limiting (RL) infrastructure by adding
adf_rl_get_num_svc_aes() which can be used to fetch the number of engines
associated with the service type. Expand the structure adf_rl_hw_data
with an array that contains the number of AEs per service.
Implement adf_gen4_init_num_svc_aes() for QAT GEN4 devices to calculate
the total number of acceleration engines dedicated to a specific service.
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Rename (1) is_service_enabled() to adf_is_service_enabled(), and
(2) srv_to_cfg_svc_type() to adf_srv_to_cfg_svc_type(), and move them to
adf_cfg_services.c which is the appropriate place for configuration-related
service logic. This improves code organization and modularity by grouping
related service configuration logic in a single location.
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The enums `adf_base_services` (used in rate limiting) and `adf_services`
define the same values, resulting in code duplication.
To improve consistency across the QAT driver: (1) rename `adf_services`
to `adf_base_services` in adf_cfg_services.c to better reflect its role
in defining core services (those with dedicated accelerators),
(2) introduce a new `adf_extended_services` enum starting from
`SVC_BASE_COUNT`, and move `SVC_DCC` into it, as it represents an
extended service (DC with chaining), and (3) remove the redundant
`adf_base_services` enum from the rate limiting implementation.
This does not introduce any functional change.
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qat - add decompression service for rate limiting
Add a new base service type ADF_SVC_DECOMP to the QAT rate limiting (RL)
infrastructure. This enables RL support for the decompression (DECOMP)
service type, allowing service-level agreements (SLAs) to be enforced
when decompression is configured.
The new service is exposed in the sysfs RL service list for visibility.
Note that this support is applicable only to devices that provide the
decompression service, such as QAT GEN6 devices.
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qat - validate service in rate limiting sysfs api
The sysfs interface 'qat_rl/srv' currently allows all valid services,
even if a service is not configured for the device. This leads to a failure
when attempting to add the SLA using 'qat_rl/sla_op'.
Add a check using is_service_enabled() to ensure the requested service is
enabled. If not, return -EINVAL to prevent invalid configurations.
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Wenkai Lin [Thu, 10 Jul 2025 12:24:57 +0000 (20:24 +0800)]
crypto: hisilicon/sec2 - implement full backlog mode for sec
This patch introduces a hierarchical backlog mechanism to cache
user data in high-throughput encryption/decryption scenarios,
the implementation addresses packet loss issues when hardware
queues overflow during peak loads.
First, we use sec_alloc_req_id to obtain an exclusive resource
from the pre-allocated resource pool of each queue, if no resource
is allocated, perform the DMA map operation on the request memory.
When the task is ready, we will attempt to send it to the hardware,
if the hardware queue is already full, we cache the request into
the backlog list, then return an EBUSY status to the upper layer
and instruct the packet-sending thread to pause transmission.
Simultaneously, when the hardware completes a task, it triggers
the sec callback function, within this function, reattempt to send
the requests from the backlog list and wake up the sending thread
until the hardware queue becomes fully occupied again.
In addition, it handles such exceptions like the hardware is reset
when packets are sent, it will switch to the software computing
and release occupied resources.
Signed-off-by: Wenkai Lin <linwenkai6@hisilicon.com> Signed-off-by: Chenghai Huang <huangchenghai2@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Documentation: qat: update debugfs-driver-qat_telemetry for GEN6 devices
Expands telemetry documentation for supporting QAT GEN6 device. Introduces
new parameters to capture compression, decompression slice utilization and
execution count.
Co-developed-by: George Abraham P <george.abraham.p@intel.com> Signed-off-by: George Abraham P <george.abraham.p@intel.com> Signed-off-by: Vijay Sundar Selvamani <vijay.sundar.selvamani@intel.com> Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Enable telemetry for QAT GEN6 devices by defining the firmware data
structures layouts, implementing the counters parsing logic and setting
the required properties on the adf_tl_hw_data data structure.
As for QAT GEN4, telemetry counters are exposed via debugfs using the
interface described in Documentation/ABI/testing/debugfs-driver-qat_telemetry.
Co-developed-by: George Abraham P <george.abraham.p@intel.com> Signed-off-by: George Abraham P <george.abraham.p@intel.com> Signed-off-by: Vijay Sundar Selvamani <vijay.sundar.selvamani@intel.com> Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qat - add decompression service to telemetry
QAT GEN6 devices offer decompression as an additional service.
Update the telemetry ring pair service interface to support monitoring
decompression operations.
Co-developed-by: George Abraham P <george.abraham.p@intel.com> Signed-off-by: George Abraham P <george.abraham.p@intel.com> Signed-off-by: Vijay Sundar Selvamani <vijay.sundar.selvamani@intel.com> Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Eric Biggers [Wed, 9 Jul 2025 00:59:54 +0000 (17:59 -0700)]
crypto: acomp - Fix CFI failure due to type punning
To avoid a crash when control flow integrity is enabled, make the
workspace ("stream") free function use a consistent type, and call it
through a function pointer that has that same type.
Fixes: 42d9f6c77479 ("crypto: acomp - Move scomp stream allocation code into acomp") Cc: stable@vger.kernel.org Signed-off-by: Eric Biggers <ebiggers@kernel.org> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Eric Biggers [Tue, 8 Jul 2025 19:38:28 +0000 (12:38 -0700)]
crypto: x86/aegis - Fix sleeping when disallowed on PREEMPT_RT
skcipher_walk_done() can call kfree(), which takes a spinlock, which
makes it incorrect to call while preemption is disabled on PREEMPT_RT.
Therefore, end the kernel-mode FPU section before calling
skcipher_walk_done(), and restart it afterwards.
Moreover, pass atomic=false to skcipher_walk_aead_encrypt() instead of
atomic=true. The point of atomic=true was to make skcipher_walk_done()
safe to call while in a kernel-mode FPU section, but that does not
actually work. So just use the usual atomic=false.
Fixes: 1d373d4e8e15 ("crypto: x86 - Add optimized AEGIS implementations") Cc: stable@vger.kernel.org Signed-off-by: Eric Biggers <ebiggers@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qat - enable power management debugfs for GEN6 devices
The QAT driver includes infrastructure to report power management (PM)
information via debugfs. Extend this support to QAT GEN6 devices
by exposing PM debug data through the `pm_status` file.
This implementation reports the current PM state, power management
hardware control and status registers (CSR), and per-domain power
status specific to the QAT GEN6 architecture.
The debug functionality is implemented in adf_gen6_pm_dbgfs.c
and initialized as part of the enable_pm() function.
Co-developed-by: Vijay Sundar Selvamani <vijay.sundar.selvamani@intel.com> Signed-off-by: Vijay Sundar Selvamani <vijay.sundar.selvamani@intel.com> Signed-off-by: George Abraham P <george.abraham.p@intel.com> Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qat - relocate power management debugfs helper APIs
Relocate the power management debugfs helper APIs in a common file
adf_pm_dbgfs_utils.h and adf_pm_dbgfs_utils.c so that it can be shared
between device generations.
When moving logic from adf_gen4_pm_debugfs.c to adf_pm_dbgfs_utils.c, the
include kernel.h has been replaced with the required include.
This does not introduce any functional change.
Signed-off-by: George Abraham P <george.abraham.p@intel.com> Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qat - fix virtual channel configuration for GEN6 devices
The TCVCMAP (Traffic Class to Virtual Channel Mapping) field in the
PVC0CTL and PVC1CTL register controls how traffic classes are mapped to
virtual channels in QAT GEN6 hardware.
The driver previously wrote a default TCVCMAP value to this register, but
this configuration was incorrect.
Modify the TCVCMAP configuration to explicitly enable both VC0 and VC1,
and map Traffic Classes 0 to 7 → VC0 and Traffic Class 8 → VC1.
Replace FIELD_PREP() with FIELD_MODIFY() to ensure that only the intended
TCVCMAP field is updated, preserving other bits in the register. This
prevents unintended overwrites of unrelated configuration fields when
modifying TC to VC mappings.
pm_runtime_put_autosuspend(), pm_runtime_put_sync_autosuspend(),
pm_runtime_autosuspend() and pm_request_autosuspend() now include a call
to pm_runtime_mark_last_busy(). Remove the now-reduntant explicit call to
pm_runtime_mark_last_busy().
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
pm_runtime_put_autosuspend(), pm_runtime_put_sync_autosuspend(),
pm_runtime_autosuspend() and pm_request_autosuspend() now include a call
to pm_runtime_mark_last_busy(). Remove the now-reduntant explicit call to
pm_runtime_mark_last_busy().
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com> Reviewed-by: AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: caam - avoid option aliasing with the CONFIG_CAAM_QI build option
In the Makefile, the new build option CONFIG_CAAM_QI is defined conditioned
on the existence of the CONFIG_CRYPTO_DEV_FSL_CAAM_CRYPTO_API_QI, which is
properly defined in the Kconfig file. So, CONFIG_CAAM_QI is just a local
alias for CONFIG_CRYPTO_DEV_FSL_CAAM_CRYPTO_API_QI.
There is little benefit in the source code of having this slightly shorter
alias for this configuration, but it complicates further maintenance, as
searching for the impact of CONFIG_CRYPTO_DEV_FSL_CAAM_CRYPTO_API_QI
requires to grep once, and then identify the option introduced and continue
searching for that. Further, tools, such as cross referencers, and scripts
to check Kconfig definitions and their use simply do not handle this
situation. Given that this is the only incidence of such a config alias in
the whole kernel tree, just prefer to avoid this pattern of aliasing here.
Use CONFIG_CRYPTO_DEV_FSL_CAAM_CRYPTO_API_QI throughout the Freescale
CAAM-Multicore platform driver backend source code.
crypto: qat - add live migration enablers for GEN6 devices
The current implementation of the QAT live migration enablers is exclusive
to QAT GEN4 devices and resides within QAT GEN4 specific files. However,
the underlying mechanisms, such as the relevant CSRs and offsets,
can be shared between QAT GEN4 and QAT GEN6 devices.
Add the necessary enablers required to implement live migration for QAT
GEN6 devices to the abstraction layer to allow leveraging the existing
QAT GEN4 implementation.
Signed-off-by: Małgorzata Mielnik <malgorzata.mielnik@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qat - relocate and rename bank state structure definition
The `bank_state` structure represents the state of a bank of rings.
As part of recent refactoring, the functions that interact with this
structure have been moved to a new unit, adf_bank_state.c.
To align with this reorganization, rename `struct bank_state` to
`struct adf_bank_state` and move its definition to adf_bank_state.h.
Also relocate the associated `struct ring_config` to the same header
to consolidate related definitions.
Update all references to use the new structure name.
This does not introduce any functional change.
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qat - relocate bank state helper functions
The existing implementation of bank state management functions,
including saving and restoring state, is located within 4xxx device
files. However, these functions do not contain GEN4-specific code and
are applicable to other QAT generations.
Relocate the bank state management functions to a new file,
adf_bank_state.c, and rename them removing the `gen4` prefix. This change
enables the reuse of such functions across different QAT generations.
Add documentation to bank state related functions that were
moved from QAT 4xxx specific files to common files.
This does not introduce any functional change.
Signed-off-by: Małgorzata Mielnik <malgorzata.mielnik@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: cryptd - Use nested-BH locking for cryptd_cpu_queue
cryptd_queue::cryptd_cpu_queue is a per-CPU variable and relies on
disabled BH for its locking. Without per-CPU locking in
local_bh_disable() on PREEMPT_RT this data structure requires explicit
locking.
Add a local_lock_t to the struct cryptd_cpu_queue and use
local_lock_nested_bh() for locking. This change adds only lockdep
coverage and does not alter the functional behaviour for !PREEMPT_RT.
Cc: "David S. Miller" <davem@davemloft.net> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: linux-crypto@vger.kernel.org Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qat - disable ZUC-256 capability for QAT GEN5
The ZUC-256 EEA (encryption) and EIA (integrity) algorithms are not
supported on QAT GEN5 devices, as their current implementation does not
align with the NIST specification. Earlier versions of the ZUC-256
specification used a different initialization scheme, which has since
been revised to comply with the 5G specification.
Due to this misalignment with the updated specification, remove support
for ZUC-256 EEA and EIA for QAT GEN5 by masking out the ZUC-256
capability.
Fixes: fcf60f4bcf549 ("crypto: qat - add support for 420xx devices") Signed-off-by: Bairavi Alagappan <bairavix.alagappan@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Fix the following warnings reported by the static analyzer Smatch:
crypto/zstd.c:273 zstd_decompress()
warn: duplicate check 'scur' (previous on line 235)
Fixes: f5ad93ffb541 ("crypto: zstd - convert to acomp") Reported-by: Dan Carpenter <dan.carpenter@linaro.org> Closes: https://lore.kernel.org/linux-crypto/92929e50-5650-40be-8c0a-de81e77f0acf@sabinyo.mountain/ Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch sets ccp_debugfs_dir to NULL after destroying it in
ccp5_debugfs_destroy, allowing the directory dentry to be
recreated when rebinding the ccp device.
Tested on AMD Ryzen 7 1700X.
Fixes: 3cdbe346ed3f ("crypto: ccp - Add debugfs entries for CCP information") Signed-off-by: Mengbiao Xiong <xisme1998@gmail.com> Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Markus Theil [Sat, 21 Jun 2025 11:36:43 +0000 (13:36 +0200)]
crypto: jitter - fix intermediary handling
The intermediary value was included in the wrong
hash state. While there, adapt to user-space by
setting the timestamp to 0 if stuck and inserting
the values nevertheless.
Acked-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Markus Theil <theil.markus@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Thomas Fourier [Fri, 20 Jun 2025 07:29:26 +0000 (09:29 +0200)]
crypto: inside-secure - Fix `dma_unmap_sg()` nents value
The `dma_unmap_sg()` functions should be called with the same nents as the
`dma_map_sg()`, not the value the map function returned.
Fixes: c957f8b3e2e5 ("crypto: inside-secure - avoid unmapping DMA memory that was not mapped") Signed-off-by: Thomas Fourier <fourier.thomas@gmail.com> Reviewed-by: Antoine Tenart <atenart@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: ccp - Fix locking on alloc failure handling
The __snp_alloc_firmware_pages() helper allocates pages in the firmware
state (alloc + rmpupdate). In case of failed rmpupdate, it tries
reclaiming pages with already changed state. This requires calling
the PSP firmware and since there is sev_cmd_mutex to guard such calls,
the helper takes a "locked" parameter so specify if the lock needs to
be held.
Most calls happen from snp_alloc_firmware_page() which executes without
the lock. However
commit 24512afa4336 ("crypto: ccp: Handle the legacy TMR allocation when SNP is enabled")
switched sev_fw_alloc() from alloc_pages() (which does not call the PSP) to
__snp_alloc_firmware_pages() (which does) but did not account for the fact
that sev_fw_alloc() is called from __sev_platform_init_locked()
(via __sev_platform_init_handle_tmr()) and executes with the lock held.
Add a "locked" parameter to __snp_alloc_firmware_pages().
Make sev_fw_alloc() use the new parameter to prevent potential deadlock in
rmp_mark_pages_firmware() if rmpupdate() failed.
Fixes: 24512afa4336 ("crypto: ccp: Handle the legacy TMR allocation when SNP is enabled") Signed-off-by: Alexey Kardashevskiy <aik@amd.com> Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Reviewed-by: Pratik R. Sampat <prsampat@amd.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
local_lock: Move this_cpu_ptr() notation from internal to main header
local_lock.h is the main header for the local_lock_t type and provides
wrappers around internal functions prefixed with __ in
local_lock_internal.h.
Move the this_cpu_ptr() dereference of the variable from the internal to
the main header. Since it is all macro implemented, this_cpu_ptr() will
still happen within the preempt/ IRQ disabled section.
This frees the internal implementation (__) to be used on local_lock_t
types which are local variables and must not be accessed via
this_cpu_ptr().
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Acked-by: Waiman Long <longman@redhat.com> Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/all/20250630075138.3448715-2-bigeasy@linutronix.de
Add key preparation code in case of selftest running to the phmac
setkey function:
As long as crypto_ahash_tested() returns with false, all setkey()
invocations are assumed to carry sheer hmac clear key values and thus
need some preparation to work with the phmac implementation. Thus it
is possible to use the already available hmac test vectors implemented
in the testmanager to test the phmac code.
When crypto_ahash_tested() returns true (that is after larval state)
the phmac code assumes the key material is a blob digestible by the
pkey kernel module which converts the blob into a working key for the
phmac code.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com> Reviewed-by: Holger Dengler <dengler@linux.ibm.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: ahash - Add crypto_ahash_tested() helper function
Add a little inline helper function
crypto_ahash_tested()
to the internal/hash.h header file to retrieve the tested
status (that is the CRYPTO_ALG_TESTED bit in the cra_flags).
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com> Suggested-by: Herbert Xu <herbert@gondor.apana.org.au> Reviewed-by: Holger Dengler <dengler@linux.ibm.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: s390 - New s390 specific protected key hash phmac
Add support for protected key hmac ("phmac") for s390 arch.
With the latest machine generation there is now support for
protected key (that is a key wrapped by a master key stored
in firmware) hmac for sha2 (sha224, sha256, sha384 and sha512)
for the s390 specific CPACF instruction kmac.
This patch adds support via 4 new ahashes registered as
phmac(sha224), phmac(sha256), phmac(sha384) and phmac(sha512).
Co-developed-by: Holger Dengler <dengler@linux.ibm.com> Signed-off-by: Harald Freudenberger <freude@linux.ibm.com> Reviewed-by: Holger Dengler <dengler@linux.ibm.com> Signed-off-by: Holger Dengler <dengler@linux.ibm.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: ahash - make hash walk functions from ahash.c public
Make the hash walk functions
crypto_hash_walk_done()
crypto_hash_walk_first()
crypto_hash_walk_last()
public again.
These functions had been removed from the header file
include/crypto/internal/hash.h with commit 7fa481734016
("crypto: ahash - make hash walk functions private to ahash.c")
as there was no crypto algorithm code using them.
With the upcoming crypto implementation for s390 phmac
these functions will be exploited and thus need to be
public within the kernel again.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com> Acked-by: Holger Dengler <dengler@linux.ibm.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This occurs in selftests where the input and the output scatterlist point
to the same underlying memory (e.g., when tested with INPLACE_TWO_SGLISTS
mode).
The problem is that the hisi_sec2 driver maps these two different
scatterlists using the DMA_BIDIRECTIONAL flag which leads to overlapped
write mappings which are not supported by the DMA layer.
Fix it by using the fine grained and correct DMA mapping directions. While
at it, switch the DMA directions used by the hisi_zip driver too.
Signed-off-by: Zenghui Yu <yuzenghui@huawei.com> Reviewed-by: Longfang Liu <liulongfang@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In the past %pK was preferable to %p as it would not leak raw pointer
values into the kernel log.
Since commit ad67b74d2469 ("printk: hash addresses printed with %p")
the regular %p has been improved to avoid this issue.
Furthermore, restricted pointers ("%pK") were never meant to be used
through printk(). They can still unintentionally leak raw pointers or
acquire sleeping locks in atomic contexts.
Switch to the regular pointer formatting which is safer and
easier to reason about.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The sha384 and hmac_sha384 DRBGs with and without prediction resistance
were disallowed in an early version of the FIPS 140-3 Implementation
Guidance document. Hence, the fips_allowed flag in struct alg_test_desc
pertaining to the affected DRBGs was unset. The IG has been withdrawn
and they are allowed again.
Furthermore, when the DRBGs are configured, /proc/crypto shows that
drbg_*pr_sha384 and drbg_*pr_hmac_sha384 are fips-approved ("fips: yes")
but because their self-tests are not run (a consequence of unsetting
the fips_allowed flag), the drbgs won't load successfully with the seeming
contradictory "fips: yes" in /proc/crypto.
This series contains a single patch that sets the fips_allowed flag in
the sha384-impacted DRBGs, which restores the ability to load them in
FIPS mode.
crypto: qat - restore ASYM service support for GEN6 devices
Support for asymmetric crypto services was not included in the qat_6xxx
by explicitly setting the asymmetric capabilities to 0 to allow for
additional testing.
Enable asymmetric crypto services on QAT GEN6 devices by setting the
appropriate capability flags.
Ashish Kalra [Mon, 16 Jun 2025 21:50:27 +0000 (21:50 +0000)]
crypto: ccp - Fix SNP panic notifier unregistration
Panic notifiers are invoked with RCU read lock held and when the
SNP panic notifier tries to unregister itself from the panic
notifier callback itself it causes a deadlock as notifier
unregistration does RCU synchronization.
Code flow for SNP panic notifier:
snp_shutdown_on_panic() ->
__sev_firmware_shutdown() ->
__sev_snp_shutdown_locked() ->
atomic_notifier_chain_unregister(.., &snp_panic_notifier)
Fix SNP panic notifier to unregister itself during SNP shutdown
only if panic is not in progress.
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Cc: stable@vger.kernel.org Fixes: 19860c3274fb ("crypto: ccp - Register SNP panic notifier only if SNP is enabled") Signed-off-by: Ashish Kalra <ashish.kalra@amd.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Convert the implementation to a native acomp interface using zstd
streaming APIs, eliminating the need for buffer linearization.
This includes:
- Removal of the scomp interface in favor of acomp
- Refactoring of stream allocation, initialization, and handling for
both compression and decompression using Zstandard streaming APIs
- Replacement of crypto_register_scomp() with crypto_register_acomp()
for module registration
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
virtcrypto_devmgr_get_first() and virtcrypto_dev_in_use() were added in
2016 by
commit dbaf0624ffa5 ("crypto: add virtio-crypto driver")
but have remained unused.
Remove them.
Signed-off-by: Dr. David Alan Gilbert <linux@treblig.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Fri, 13 Jun 2025 08:51:38 +0000 (16:51 +0800)]
crypto: ahash - Stop legacy tfms from using the set_virt fallback path
Ensure that drivers that have not been converted to the ahash API
do not use the ahash_request_set_virt fallback path as they cannot
use the software fallback.
Reported-by: Eric Biggers <ebiggers@kernel.org> Fixes: 9d7a0ab1c753 ("crypto: ahash - Handle partial blocks in API") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
dt-bindings: crypto: Convert ti,omap4-des to DT schema
Convert the TI OMAP DES binding to DT schema format.
Drop "ti,hwmods" as it is not actually used for this binding. Only
OMAP2 platforms are using it.
Signed-off-by: Rob Herring (Arm) <robh@kernel.org> Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
dt-bindings: crypto: Convert ti,omap2-aes to DT schema
Convert the TI OMAP AES binding to DT schema format. It's a straight
forward conversion.
Make "ti,hwmods" not required as it is deprecated and only used on
OMAP2.
Signed-off-by: Rob Herring (Arm) <robh@kernel.org> Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ryan Wanner [Wed, 11 Jun 2025 19:47:27 +0000 (12:47 -0700)]
dt-bindings: crypto: add sama7d65 in Atmel TDES
Add DT bindings for SAMA7D65 SoC Atmel TDES.
The SAMA7D65 SoC has the same capability as the SAM9x75 SoC.
Signed-off-by: Ryan Wanner <Ryan.Wanner@microchip.com> Acked-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ryan Wanner [Wed, 11 Jun 2025 19:47:26 +0000 (12:47 -0700)]
dt-bindings: crypto: add sama7d65 in Atmel SHA
Add DT bindings for SAMA7D65 SoC Atmel SHA.
The SAMA7D65 similar to the SAM9x75 SoC supports SHA1/224/256/384/512
and supports HMAC for the same hashes. They both also support automatic
padding as well as double buffering.
Signed-off-by: Ryan Wanner <Ryan.Wanner@microchip.com> Acked-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ryan Wanner [Wed, 11 Jun 2025 19:47:25 +0000 (12:47 -0700)]
dt-bindings: crypto: add sama7d65 in Atmel AES
Add DT bindings for SAMA7D65 SoC in atmel AES.
The SAMA7D65 similar to the SAM9x75 SoC supports HMAC, dual buffer, and
GCM. And similar all 3 it supports CBC, CFB, CTR, ECB, and XTS.
Signed-off-by: Ryan Wanner <Ryan.Wanner@microchip.com> Acked-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
John Ernberg [Wed, 11 Jun 2025 11:38:09 +0000 (11:38 +0000)]
dt-bindings: crypto: fsl,sec-v4.0: Add power domains for iMX8QM and iMX8QXP
NXP SoCs like the iMX8QM, iMX8QXP or iMX8DXP use power domains for
resource management.
Add compatible strings for these SoCs (QXP and DXP gets to share as their
only difference is a core-count, Q=Quad core and D=Dual core), and allow
power-domains for them only. Keep the old restriction for others.
Signed-off-by: John Ernberg <john.ernberg@actia.se> Reviewed-by: Rob Herring (Arm) <robh@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
John Ernberg [Wed, 11 Jun 2025 11:38:08 +0000 (11:38 +0000)]
crypto: caam - Support iMX8QXP and variants thereof
The iMX8QXP (and variants such as the QX, DX, DXP) all identify as iMX8QXP.
They have the exact same restrictions as the supported iMX8QM introduced
at commit 61bb8db6f682 ("crypto: caam - Add support for i.MX8QM")
Loosen the check a little bit with a wildcard to also match the iMX8QXP
and its variants.
Signed-off-by: John Ernberg <john.ernberg@actia.se> Reviewed-by: Frank Li <Frank.Li@nxp.com> Reviewed-by: Peng Fan <peng.fan@nxp.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
John Ernberg [Wed, 11 Jun 2025 11:38:08 +0000 (11:38 +0000)]
crypto: caam - Prevent crash on suspend with iMX8QM / iMX8ULP
Since the CAAM on these SoCs is managed by another ARM core, called the
SECO (Security Controller) on iMX8QM and Secure Enclave on iMX8ULP, which
also reserves access to register page 0 suspend operations cannot touch
this page.
This is similar to when running OPTEE, where OPTEE will reserve page 0.
Track this situation using a new state variable no_page0, reflecting if
page 0 is reserved elsewhere, either by other management cores in SoC or
by OPTEE.
Replace the optee_en check in suspend/resume with the new check.
optee_en cannot go away as it's needed elsewhere to gate OPTEE specific
situations.
ChengZhenghan [Wed, 11 Jun 2025 02:51:31 +0000 (10:51 +0800)]
crypto: x86 - Fix build warnings about export.h
I got some build warnings with W=1:
arch/x86/coco/sev/core.c:
arch/x86/crypto/aria_aesni_avx2_glue.c:
warning: EXPORT_SYMBOL() is used,
but #include <linux/export.h> is missing
arch/x86/crypto/aria_aesni_avx_glue.c:
warning: EXPORT_SYMBOL() is used,
but #include <linux/export.h> is missing
arch/x86/crypto/camellia_aesni_avx_glue.c:
warning: EXPORT_SYMBOL() is used,
but #include <linux/export.h> is missing
arch/x86/crypto/camellia_glue.c: warning:
EXPORT_SYMBOL() is used,
but #include <linux/export.h> is missing
arch/x86/crypto/curve25519-x86_64.c:
warning: EXPORT_SYMBOL() is used,
but #include <linux/export.h> is missing
arch/x86/crypto/serpent_avx_glue.c:
warning: EXPORT_SYMBOL() is used,
but #include <linux/export.h> is missing
arch/x86/crypto/sm4_aesni_avx_glue.c:
warning: EXPORT_SYMBOL() is used,
but #include <linux/export.h> is missing
arch/x86/crypto/twofish_glue.c:
warning: EXPORT_SYMBOL() is used,
but #include <linux/export.h> is missing
arch/x86/crypto/twofish_glue_3way.c:
warning: EXPORT_SYMBOL() is used,
but #include <linux/export.h> is missing
so I fixed these build warnings for x86_64.
Signed-off-by: ChengZhenghan <chengzhenghan@uniontech.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Arnd Bergmann [Tue, 10 Jun 2025 09:32:52 +0000 (11:32 +0200)]
crypto: arm/aes-neonbs - work around gcc-15 warning
I get a very rare -Wstringop-overread warning with gcc-15 for one function
in aesbs_ctr_encrypt():
arch/arm/crypto/aes-neonbs-glue.c: In function 'ctr_encrypt':
arch/arm/crypto/aes-neonbs-glue.c:212:1446: error: '__builtin_memcpy' offset [17, 2147483647] is out of the bounds [0, 16] of object 'buf' with type 'u8[16]' {aka 'unsigned char[16]'} [-Werror=array-bounds=]
212 | src = dst = memcpy(buf + sizeof(buf) - bytes,
arch/arm/crypto/aes-neonbs-glue.c: In function 'ctr_encrypt':
arch/arm/crypto/aes-neonbs-glue.c:218:17: error: 'aesbs_ctr_encrypt' reading 1 byte from a region of size 0 [-Werror=stringop-overread]
218 | aesbs_ctr_encrypt(dst, src, ctx->rk, ctx->rounds, bytes, walk.iv);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/arm/crypto/aes-neonbs-glue.c:218:17: note: referencing argument 2 of type 'const u8[0]' {aka 'const unsigned char[]'}
arch/arm/crypto/aes-neonbs-glue.c:218:17: note: referencing argument 3 of type 'const u8[0]' {aka 'const unsigned char[]'}
arch/arm/crypto/aes-neonbs-glue.c:218:17: note: referencing argument 6 of type 'u8[0]' {aka 'unsigned char[]'}
arch/arm/crypto/aes-neonbs-glue.c:36:17: note: in a call to function 'aesbs_ctr_encrypt'
36 | asmlinkage void aesbs_ctr_encrypt(u8 out[], u8 const in[], u8 const rk[],
This could happen in theory if walk.nbytes is larger than INT_MAX and gets
converted to a negative local variable.
Keep the type unsigned like the orignal nbytes to be sure there is no
integer overflow.
Fixes: c8bf850e991a ("crypto: arm/aes-neonbs-ctr - deal with non-multiples of AES block size") Signed-off-by: Arnd Bergmann <arnd@arndb.de> Acked-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Mon, 16 Jun 2025 08:38:49 +0000 (16:38 +0800)]
padata: Remove comment for reorder_work
Remove comment for reorder_work which no longer exists.
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au> Fixes: 71203f68c774 ("padata: Fix pd UAF once and for all") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Documentation: qat: update sysfs-driver-qat for GEN6 devices
Reorganize and expand documentation for service configurations. This
reworks the `cfg_services` section by removing explicit service
combinations (e.g., asym;sym, sym;asym) and clarifying that multiple
services can be configured in any order.
Update the documentation to reflect that the attribute previously limited
to qat_4xxx devices is also applicable to qat_6xxx devices.
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qat - add support for decompression service to GEN6 devices
Add support to configure decompression as a separate service for QAT GEN6
devices. A new arbiter configuration has been added to map the hardware
decompression threads to all ring pairs.
The decompression service is enabled via sysfs by writing "decomp" to
"/sys/bus/pci/devices/<BDF>/qat/cfg_services".
The decompression service is not supported on QAT GEN2 and GEN4 devices,
and attempting it results in an invalid write error. The existing
compression service for QAT GEN2 and GEN4 devices remains unchanged and
supports both compression and decompression operations on the same ring
pair.
crypto: qat - fix state restore for banks with exceptions
Change the logic in the restore function to properly handle bank
exceptions.
The check for exceptions in the saved state should be performed before
conducting any other ringstat register checks.
If a bank was saved with an exception, the ringstat will have the
appropriate rp_halt/rp_exception bits set, causing the driver to exit
the restore process with an error. Instead, the restore routine should
first check the ringexpstat register, and if any exception was raised,
it should stop further checks and return without any error. In other
words, if a ring pair is in an exception state at the source, it should
be restored the same way at the destination but without raising an error.
Even though this approach might lead to losing the exception state
during migration, the driver will log the exception from the saved state
during the restore process.
Signed-off-by: Svyatoslav Pankratov <svyatoslav.pankratov@intel.com> Fixes: bbfdde7d195f ("crypto: qat - add bank save and restore flows") Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ahsan Atta [Wed, 4 Jun 2025 08:23:43 +0000 (09:23 +0100)]
crypto: qat - allow enabling VFs in the absence of IOMMU
The commit ca88a2bdd4dd ("crypto: qat - allow disabling SR-IOV VFs")
introduced an unnecessary change that prevented enabling SR-IOV when
IOMMU is disabled. In certain scenarios, it is desirable to enable
SR-IOV even in the absence of IOMMU. Thus, restoring the previous
functionality to allow VFs to be enumerated in the absence of IOMMU.
Fixes: ca88a2bdd4dd ("crypto: qat - allow disabling SR-IOV VFs") Signed-off-by: Ahsan Atta <ahsan.atta@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Michal Witwicki <michal.witwicki@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Amit Singh Tomar [Wed, 28 May 2025 14:59:41 +0000 (20:29 +0530)]
crypto: octeontx2 - get engine group number for asymmetric engine
Cryptographic Accelerator Unit (CPT) support different engine groups, one
for asymmetric algorithms (only AE engines in this group), one for the
most common symmetric algorithms (all SE and all IE engines in this group),
and one for other symmetric algorithms (only SE engines in this group).
For symmetric engine (SE), we obtain the group number using
"MBOX_MSG_GET_ENG_GRP_NUM" mailbox. Let's follow a similar approach to
determine the group number for asymmetric engine (AE).
Signed-off-by: Amit Singh Tomar <amitsinght@marvell.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Amit Singh Tomar [Wed, 28 May 2025 14:59:40 +0000 (20:29 +0530)]
crypto: octeontx2 - Rework how engine group number is obtained
By default, otx2_cpt_get_kcrypto_eng_grp_num() returns the engine group
number of SE engine type. Add an engine type parameter to support
retrieving the engine group number for different engine types.
Since otx2_cpt_get_kcrypto_eng_grp_num() always returns the kernel crypto
engine group number, rename it to otx2_cpt_get_eng_grp_num().
Signed-off-by: Amit Singh Tomar <amitsinght@marvell.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Sat, 24 May 2025 12:32:20 +0000 (20:32 +0800)]
padata: Fix pd UAF once and for all
There is a race condition/UAF in padata_reorder that goes back
to the initial commit. A reference count is taken at the start
of the process in padata_do_parallel, and released at the end in
padata_serial_worker.
This reference count is (and only is) required for padata_replace
to function correctly. If padata_replace is never called then
there is no issue.
In the function padata_reorder which serves as the core of padata,
as soon as padata is added to queue->serial.list, and the associated
spin lock released, that padata may be processed and the reference
count on pd would go away.
Fix this by getting the next padata before the squeue->serial lock
is released.
In order to make this possible, simplify padata_reorder by only
calling it once the next padata arrives.
Fixes: 16295bec6398 ("padata: Generic parallelization/serialization interface") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Fri, 23 May 2025 12:28:56 +0000 (20:28 +0800)]
crypto: s390/sha3 - Use cpu byte-order when exporting
The sha3 partial hash on s390 is in little-endian just like the
final hash. However the generic implementation produces native
or big-endian partial hashes.
Make s390 sha3 conform to that by doing the byte-swap on export
and import.
Reported-by: Ingo Franzki <ifranzki@linux.ibm.com> Fixes: 6f90ba706551 ("crypto: s390/sha3 - Use API partial block handling") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Bharat Bhushan [Thu, 22 May 2025 10:06:27 +0000 (15:36 +0530)]
crypto: octeontx2 - Fix address alignment on CN10KB and CN10KA-B0
octeontx2 crypto driver allocates memory using kmalloc/kzalloc,
and uses this memory for dma (does dma_map_single()). It assumes
that kmalloc/kzalloc will return 128-byte aligned address. But
kmalloc/kzalloc returns 8-byte aligned address after below changes:
"9382bc44b5f5 arm64: allow kmalloc() caches aligned to the
smaller cache_line_size()
Memory allocated are used for following purpose:
- Input data or scatter list address - 8-Byte alignment
- Output data or gather list address - 8-Byte alignment
- Completion address - 32-Byte alignment.
This patch ensures all addresses are aligned as mentioned above.
Bharat Bhushan [Thu, 22 May 2025 10:06:26 +0000 (15:36 +0530)]
crypto: octeontx2 - Fix address alignment on CN10K A0/A1 and OcteonTX2
octeontx2 crypto driver allocates memory using kmalloc/kzalloc,
and uses this memory for dma (does dma_map_single()). It assumes
that kmalloc/kzalloc will return 128-byte aligned address. But
kmalloc/kzalloc returns 8-byte aligned address after below changes:
"9382bc44b5f5 arm64: allow kmalloc() caches aligned to the
smaller cache_line_size()
Memory allocated are used for following purpose:
- Input data or scatter list address - 8-Byte alignment
- Output data or gather list address - 8-Byte alignment
- Completion address - 32-Byte alignment.
This patch ensures all addresses are aligned as mentioned above.
Bharat Bhushan [Thu, 22 May 2025 10:06:25 +0000 (15:36 +0530)]
crypto: octeontx2 - Fix address alignment issue on ucode loading
octeontx2 crypto driver allocates memory using kmalloc/kzalloc,
and uses this memory for dma (does dma_map_single()). It assumes
that kmalloc/kzalloc will return 128-byte aligned address. But
kmalloc/kzalloc returns 8-byte aligned address after below changes:
"9382bc44b5f5 arm64: allow kmalloc() caches aligned to the
smaller cache_line_size()"
Completion address should be 32-Byte alignment when loading
microcode.
projects / thirdparty / kernel / linux.git / log
