Eric Biggers [Tue, 10 Oct 2023 05:59:43 +0000 (22:59 -0700)]
crypto: adiantum - add fast path for single-page messages
When the source scatterlist is a single page, optimize the first hash
step of adiantum to use crypto_shash_digest() instead of
init/update/final, and use the same local kmap for both hashing the bulk
part and loading the narrow part of the source data.
Likewise, when the destination scatterlist is a single page, optimize
the second hash step of adiantum to use crypto_shash_digest() instead of
init/update/final, and use the same local kmap for both hashing the bulk
part and storing the narrow part of the destination data.
In some cases these optimizations improve performance significantly.
Note: ideally, for optimal performance each architecture should
implement the full "adiantum(xchacha12,aes)" algorithm and fully
optimize the contiguous buffer case to use no indirect calls. That's
not something I've gotten around to doing, though. This commit just
makes a relatively small change that provides some benefit with the
existing template-based approach.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
There is no need to free the reset_data structure if the recovery is
unsuccessful and the reset is synchronous. The function
adf_dev_aer_schedule_reset() handles the cleanup properly. Only
asynchronous resets require such structure to be freed inside the reset
worker.
Eric Biggers [Mon, 9 Oct 2023 08:19:00 +0000 (01:19 -0700)]
crypto: x86/sha256 - implement ->digest for sha256
Implement a ->digest function for sha256-ssse3, sha256-avx, sha256-avx2,
and sha256-ni. This improves the performance of crypto_shash_digest()
with these algorithms by reducing the number of indirect calls that are
made.
For now, don't bother with this for sha224, since sha224 is rarely used.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Eric Biggers [Mon, 9 Oct 2023 07:53:27 +0000 (00:53 -0700)]
crypto: arm64/sha2-ce - implement ->digest for sha256
Implement a ->digest function for sha256-ce. This improves the
performance of crypto_shash_digest() with this algorithm by reducing the
number of indirect calls that are made. This only adds ~112 bytes of
code, mostly for the inlined init, as the finup function is tail-called.
For now, don't bother with this for sha224, since sha224 is rarely used.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Eric Biggers [Mon, 9 Oct 2023 07:32:14 +0000 (00:32 -0700)]
crypto: shash - fold shash_digest_unaligned() into crypto_shash_digest()
Fold shash_digest_unaligned() into its only remaining caller. Also,
avoid a redundant check of CRYPTO_TFM_NEED_KEY by replacing the call to
crypto_shash_init() with shash->init(desc). Finally, replace
shash_update_unaligned() + shash_final_unaligned() with
shash_finup_unaligned() which does exactly that.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Eric Biggers [Mon, 9 Oct 2023 07:32:13 +0000 (00:32 -0700)]
crypto: shash - optimize the default digest and finup
For an shash algorithm that doesn't implement ->digest, currently
crypto_shash_digest() with aligned input makes 5 indirect calls: 1 to
shash_digest_unaligned(), 1 to ->init, 2 to ->update ('alignmask + 1'
bytes, then the rest), then 1 to ->final. This is true even if the
algorithm implements ->finup. This is caused by an unnecessary fallback
to code meant to handle unaligned inputs. In fact,
crypto_shash_digest() already does the needed alignment check earlier.
Therefore, optimize the number of indirect calls for aligned inputs to 3
when the algorithm implements ->finup. It remains at 5 when the
algorithm implements neither ->finup nor ->digest.
Similarly, for an shash algorithm that doesn't implement ->finup,
currently crypto_shash_finup() with aligned input makes 4 indirect
calls: 1 to shash_finup_unaligned(), 2 to ->update, and
1 to ->final. Optimize this to 3 calls.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Eric Biggers [Mon, 9 Oct 2023 02:31:16 +0000 (19:31 -0700)]
crypto: xts - use 'spawn' for underlying single-block cipher
Since commit adad556efcdd ("crypto: api - Fix built-in testing
dependency failures"), the following warning appears when booting an
x86_64 kernel that is configured with
CONFIG_CRYPTO_MANAGER_EXTRA_TESTS=y and CONFIG_CRYPTO_AES_NI_INTEL=y,
even when CONFIG_CRYPTO_XTS=y and CONFIG_CRYPTO_AES=y:
alg: skcipher: skipping comparison tests for xts-aes-aesni because xts(ecb(aes-generic)) is unavailable
This is caused by an issue in the xts template where it allocates an
"aes" single-block cipher without declaring a dependency on it via the
crypto_spawn mechanism. This issue was exposed by the above commit
because it reversed the order that the algorithms are tested in.
Specifically, when "xts(ecb(aes-generic))" is instantiated and tested
during the comparison tests for "xts-aes-aesni", the "xts" template
allocates an "aes" crypto_cipher for encrypting tweaks. This resolves
to "aes-aesni". (Getting "aes-aesni" instead of "aes-generic" here is a
bit weird, but it's apparently intended.) Due to the above-mentioned
commit, the testing of "aes-aesni", and the finalization of its
registration, now happens at this point instead of before. At the end
of that, crypto_remove_spawns() unregisters all algorithm instances that
depend on a lower-priority "aes" implementation such as "aes-generic"
but that do not depend on "aes-aesni". However, because "xts" does not
use the crypto_spawn mechanism for its "aes", its dependency on
"aes-aesni" is not recognized by crypto_remove_spawns(). Thus,
crypto_remove_spawns() unexpectedly unregisters "xts(ecb(aes-generic))".
Fix this issue by making the "xts" template use the crypto_spawn
mechanism for its "aes" dependency, like what other templates do.
Note, this fix could be applied as far back as commit f1c131b45410
("crypto: xts - Convert to skcipher"). However, the issue only got
exposed by the much more recent changes to how the crypto API runs the
self-tests, so there should be no need to backport this to very old
kernels. Also, an alternative fix would be to flip the list iteration
order in crypto_start_tests() to restore the original testing order.
I'm thinking we should do that too, since the original order seems more
natural, but it shouldn't be relied on for correctness.
Fixes: adad556efcdd ("crypto: api - Fix built-in testing dependency failures") Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
zhenwei pi [Sat, 7 Oct 2023 06:43:09 +0000 (14:43 +0800)]
crypto: virtio - handle config changed by work queue
MST pointed out: config change callback is also handled incorrectly
in this driver, it takes a mutex from interrupt context.
Handle config changed by work queue instead.
Cc: Gonglei (Arei) <arei.gonglei@huawei.com> Cc: Halil Pasic <pasic@linux.ibm.com> Cc: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: zhenwei pi <pizhenwei@bytedance.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Weili Qian [Sun, 8 Oct 2023 12:36:17 +0000 (20:36 +0800)]
crypto: hisilicon/qm - alloc buffer to set and get xqc
If the temporarily applied memory is used to set or get the xqc
information, the driver releases the memory immediately after the
hardware mailbox operation time exceeds the driver waiting time.
However, the hardware does not cancel the operation, so the hardware
may write data to released memory.
Therefore, when the driver is bound to a device, the driver reserves
memory for the xqc configuration. The subsequent xqc configuration
uses the reserved memory to prevent hardware from accessing the
released memory.
Signed-off-by: Weili Qian <qianweili@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In case a health test error occurs during runtime, the power-up health
tests are rerun to verify that the noise source is still good and
that the reported health test error was an outlier. For performing this
power-up health test, the already existing entropy collector instance
is used instead of allocating a new one. This change has the following
implications:
* The noise that is collected as part of the newly run health tests is
inserted into the entropy collector and thus stirs the existing
data present in there further. Thus, the entropy collected during
the health test is not wasted. This is also allowed by SP800-90B.
* The power-on health test is not affected by the state of the entropy
collector, because it resets the APT / RCT state. The remainder of
the state is unrelated to the health test as it is only applied to
newly obtained time stamps.
This change also fixes a bug report about an allocation while in an
atomic lock (the lock is taken in jent_kcapi_random, jent_read_entropy
is called and this can call jent_entropy_init).
Fixes: 04597c8dd6c4 ("jitter - add RCT/APT support for different OSRs") Reported-by: Dan Carpenter <dan.carpenter@linaro.org> Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Rob Herring [Fri, 6 Oct 2023 21:43:40 +0000 (16:43 -0500)]
hwrng: n2 - Use device_get_match_data()
Use preferred device_get_match_data() instead of of_match_device() to
get the driver match data. With this, adjust the includes to explicitly
include the correct headers.
Signed-off-by: Rob Herring <robh@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Rob Herring [Fri, 6 Oct 2023 21:39:17 +0000 (16:39 -0500)]
crypto: aspeed-hace - Use device_get_match_data()
Use preferred device_get_match_data() instead of of_match_device() to
get the driver match data. With this, adjust the includes to explicitly
include the correct headers.
Signed-off-by: Rob Herring <robh@kernel.org> Reviewed-by: Andrew Jeffery <andrew@codeconstruct.com.au> Reviewed-by: Neal Liu <neal_liu@aspeedtech.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The Compress and Verify (CnV) feature check and ensures data integrity
in the compression operation. The implementation of CnV keeps a record
of the CnV errors that have occurred since the driver was loaded.
Expose CnV error stats by providing the "cnv_errors" file under
debugfs. This includes the number of errors detected up to now and
the type of the last error. The error count is provided on a per
Acceleration Engine basis and it is reset every time the driver is loaded.
Signed-off-by: Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
QAT devices implement a mechanism that allows them to go autonomously
to a low power state depending on the load.
Expose power management info by providing the "pm_status" file under
debugfs. This includes PM state, PM event log, PM event counters, PM HW
CSRs, per-resource type constrain counters and per-domain power gating
status specific to the QAT device.
This information is retrieved from (1) the FW by means of
ICP_QAT_FW_PM_INFO command, (2) CSRs and (3) counters collected by the
device driver.
In addition, add logic to keep track and report power management event
interrupts and acks/nacks sent to FW to allow/prevent state transitions.
Signed-off-by: Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Include kernel.h for GENMASK(), kstrtobool() and types.
Add forward declaration for struct adf_accel_dev. Remove unneeded
include.
This change doesn't introduce any function change.
Signed-off-by: Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qcom-rng - Add hw_random interface support
Add hw_random interface support in qcom-rng driver as new IP block
in Qualcomm SoC has inbuilt NIST SP800 90B compliant entropic source
to generate true random number.
Keeping current rng_alg interface as well for random number generation
using Kernel Crypto API.
Signed-off-by: Om Prakash Singh <quic_omprsing@quicinc.com> Reviewed-by: Bjorn Andersson <quic_bjorande@quicinc.com> Signed-off-by: Neil Armstrong <neil.armstrong@linaro.org> Acked-by: Om Prakash Singh <quic_omprsing@quicinc.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Neil Armstrong [Tue, 3 Oct 2023 07:10:20 +0000 (09:10 +0200)]
dt-bindings: crypto: qcom,prng: document SM8550
Document SM8550 compatible for the True Random Number Generator.
Reviewed-by: Om Prakash Singh <quic_omprsing@quicinc.com> Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> Signed-off-by: Neil Armstrong <neil.armstrong@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Neil Armstrong [Tue, 3 Oct 2023 07:10:19 +0000 (09:10 +0200)]
dt-bindings: crypto: qcom,prng: document that RNG on SM8450 is a TRNG
It has been reported at [1] the RNG HW on SM8450 is in fact a True Random
Number Generator and no more Pseudo, document this by adding
a new qcom,trng and the corresponding SoC specific sm8450 compatible.
Suggested-by: Om Prakash Singh <quic_omprsing@quicinc.com> Suggested-by: Konrad Dybcio <konrad.dybcio@linaro.org> Reviewed-by: Om Prakash Singh <quic_omprsing@quicinc.com> Signed-off-by: Neil Armstrong <neil.armstrong@linaro.org> Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Tue, 3 Oct 2023 03:43:32 +0000 (11:43 +0800)]
crypto: xts - Only access common skcipher fields on spawn
As skcipher spawns may be of the type lskcipher, only the common
fields may be accessed. This was already the case but use the
correct helpers to make this more obvious.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Tue, 3 Oct 2023 03:43:31 +0000 (11:43 +0800)]
crypto: lrw - Only access common skcipher fields on spawn
As skcipher spawns may be of the type lskcipher, only the common
fields may be accessed. This was already the case but use the
correct helpers to make this more obvious.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Tue, 3 Oct 2023 03:43:30 +0000 (11:43 +0800)]
crypto: hctr2 - Only access common skcipher fields on spawn
As skcipher spawns may be of the type lskcipher, only the common
fields may be accessed. This was already the case but use the
correct helpers to make this more obvious.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Tue, 3 Oct 2023 03:43:29 +0000 (11:43 +0800)]
crypto: gcm - Only access common skcipher fields on spawn
As skcipher spawns may be of the type lskcipher, only the common
fields may be accessed. This was already the case but use the
correct helpers to make this more obvious.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Tue, 3 Oct 2023 03:43:28 +0000 (11:43 +0800)]
crypto: cts - Only access common skcipher fields on spawn
As skcipher spawns may be of the type lskcipher, only the common
fields may be accessed. This was already the case but use the
correct helpers to make this more obvious.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Tue, 3 Oct 2023 03:43:27 +0000 (11:43 +0800)]
crypto: ctr - Only access common skcipher fields on spawn
As skcipher spawns may be of the type lskcipher, only the common
fields may be accessed. This was already the case but use the
correct helpers to make this more obvious.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Tue, 3 Oct 2023 03:43:26 +0000 (11:43 +0800)]
crypto: chacha20poly1305 - Only access common skcipher fields on spawn
As skcipher spawns may be of the type lskcipher, only the common
fields may be accessed. This was already the case but use the
correct helpers to make this more obvious.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Tue, 3 Oct 2023 03:43:25 +0000 (11:43 +0800)]
crypto: ccm - Only access common skcipher fields on spawn
As skcipher spawns may be of the type lskcipher, only the common
fields may be accessed. This was already the case but use the
correct helpers to make this more obvious.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Tue, 3 Oct 2023 03:43:24 +0000 (11:43 +0800)]
crypto: authencesn - Only access common skcipher fields on spawn
As skcipher spawns may be of the type lskcipher, only the common
fields may be accessed. This was already the case but use the
correct helpers to make this more obvious.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Tue, 3 Oct 2023 03:43:23 +0000 (11:43 +0800)]
crypto: authenc - Only access common skcipher fields on spawn
As skcipher spawns may be of the type lskcipher, only the common
fields may be accessed. This was already the case but use the
correct helpers to make this more obvious.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Tue, 3 Oct 2023 03:43:22 +0000 (11:43 +0800)]
crypto: adiantum - Only access common skcipher fields on spawn
As skcipher spawns may be of the type lskcipher, only the common
fields may be accessed. This was already the case but use the
correct helpers to make this more obvious.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Tue, 3 Oct 2023 03:43:21 +0000 (11:43 +0800)]
crypto: cryptd - Only access common skcipher fields on spawn
As skcipher spawns may be of the type lskcipher, only the common
fields may be accessed. This was already the case but use the
correct helpers to make this more obvious.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
As skcipher spawns can be of two different types (skcipher vs.
lskcipher), only the common fields can be accessed. Add a helper
to return the common algorithm object.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Tue, 3 Oct 2023 03:31:55 +0000 (11:31 +0800)]
crypto: skcipher - Add dependency on ecb
As lskcipher requires the ecb wrapper for the transition add an
explicit dependency on it so that it is always present. This can
be removed once all simple ciphers have been converted to lskcipher.
Reported-by: Nathan Chancellor <nathan@kernel.org> Fixes: 705b52fef3c7 ("crypto: cbc - Convert from skcipher to lskcipher") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Tested-by: Nathan Chancellor <nathan@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Create CRYPTO_QAT namespace for symbols exported by the qat_common
module and import those in the QAT drivers. It will reduce the global
namespace crowdedness and potential misuse or the API.
This does not introduce any functional change.
Suggested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Remove support for md4 md5 hash and signatures in x.509 certificate
parsers, pkcs7 signature parser, authenticode parser.
All of these are insecure or broken, and everyone has long time ago
migrated to alternative hash implementations.
Also remove md2 & md3 oids which have already didn't have support.
This is also likely the last user of md4 in the kernel, and thus
crypto/md4.c and related tests in tcrypt & testmgr can likely be
removed. Other users such as cifs smbfs ext modpost sumversions have
their own internal implementation as needed.
Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com> Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Randy Dunlap [Sun, 1 Oct 2023 00:00:44 +0000 (17:00 -0700)]
crypto: sig - fix kernel-doc typo
Correct typo of "destination".
Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: "David S. Miller" <davem@davemloft.net> Cc: linux-crypto@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Randy Dunlap [Sun, 1 Oct 2023 00:00:43 +0000 (17:00 -0700)]
crypto: akcipher - fix kernel-doc typos
Correct typos of "destination".
Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: "David S. Miller" <davem@davemloft.net> Cc: linux-crypto@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: hisilicon/qm - check function qp num before alg register
When the Kunpeng accelerator executes tasks such as encryption
and decryption have minimum requirements on the number of device
queues. If the number of queues does not meet the requirement,
the process initialization will fail. Therefore, the driver checks
the number of queues on the device before registering the algorithm.
If the number does not meet the requirements, the driver does not register
the algorithm to crypto subsystem, the device is still added to the
qm_list.
Signed-off-by: Weili Qian <qianweili@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The type of aeq has only 4bits in dw0 17 to 20bits, but 15bits(17 to
31bits) are read in function qm_aeq_thread(). The remaining 11bits(21
to 31bits) are reserved for aeq, but may not be 0. To avoid getting
incorrect value of type, other bits are cleared.
Signed-off-by: Weili Qian <qianweili@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
If the queue isolation feature is enabled, the number of queues
supported by the device changes. When PF is enabled using the
current default number of queues, the default number of queues may
be greater than the number supported by the device. As a result,
the PF fails to be bound to the driver.
After modification, if queue isolation feature is enabled, when
the default queue parameter is greater than the number supported
by the device, the number of enabled queues will be changed to
the number supported by the device, so that the PF and driver
can be properly bound.
Fixes: 8bbecfb402f7 ("crypto: hisilicon/qm - add queue isolation support for Kunpeng930") Signed-off-by: Longfang Liu <liulongfang@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: x86/aesni - Perform address alignment early for XTS mode
Currently, the alignment of each field in struct aesni_xts_ctx occurs
right before every access. However, it's possible to perform this
alignment ahead of time.
Introduce a helper function that converts struct crypto_skcipher *tfm
to struct aesni_xts_ctx *ctx and returns an aligned address. Utilize
this helper function at the beginning of each XTS function and then
eliminate redundant alignment code.
Suggested-by: Eric Biggers <ebiggers@kernel.org> Link: https://lore.kernel.org/all/ZFWQ4sZEVu%2FLHq+Q@gmail.com/ Signed-off-by: Chang S. Bae <chang.seok.bae@intel.com> Cc: linux-crypto@vger.kernel.org Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: x86/aesni - Correct the data type in struct aesni_xts_ctx
Currently, every field in struct aesni_xts_ctx is defined as a byte
array of the same size as struct crypto_aes_ctx. This data type
is obscure and the choice lacks justification.
To rectify this, update the field type in struct aesni_xts_ctx to
match its actual structure.
Suggested-by: Eric Biggers <ebiggers@kernel.org> Link: https://lore.kernel.org/all/ZFWQ4sZEVu%2FLHq+Q@gmail.com/ Signed-off-by: Chang S. Bae <chang.seok.bae@intel.com> Cc: linux-crypto@vger.kernel.org Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: x86/aesni - Refactor the common address alignment code
The address alignment code has been duplicated for each mode. Instead
of duplicating the same code, refactor the alignment code and simplify
the alignment helpers.
Suggested-by: Eric Biggers <ebiggers@kernel.org> Link: https://lore.kernel.org/all/20230526065414.GB875@sol.localdomain/ Signed-off-by: Chang S. Bae <chang.seok.bae@intel.com> Cc: linux-crypto@vger.kernel.org Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
X.509: Add missing IMPLICIT annotations to AKID ASN.1 module
The ASN.1 module in RFC 5280 appendix A.1 uses EXPLICIT TAGS whereas the
one in appendix A.2 uses IMPLICIT TAGS.
The kernel's simplified asn1_compiler.c always uses EXPLICIT TAGS, hence
definitions from appendix A.2 need to be annotated as IMPLICIT for the
compiler to generate RFC-compliant code.
In particular, GeneralName is defined in appendix A.2:
Because appendix A.2 uses IMPLICIT TAGS, the IA5String tag (0x16) of a
dNSName is not rendered. Instead, the string directly succeeds the
[2] tag (0x82).
Likewise, the SEQUENCE tag (0x30) of an OtherName is not rendered.
Instead, only the constituents of the SEQUENCE are rendered: An OID tag
(0x06), a [0] tag (0xa0) and an ANY tag. That's three consecutive tags
instead of a single encompassing tag.
The situation is different for x400Address and directoryName choices:
They reference ORAddress and Name, which are defined in appendix A.1,
therefore use EXPLICIT TAGS.
The AKID ASN.1 module is missing several IMPLICIT annotations, hence
isn't RFC-compliant. In the unlikely event that an AKID contains other
elements beside a directoryName, users may see parse errors.
Add the missing annotations but do not tag this commit for stable as I
am not aware of any issue reports. Fixes are only eligible for stable
if they're "obviously correct" and with ASN.1 there's no such thing.
Signed-off-by: Lukas Wunner <lukas@wunner.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: engine - Make crypto_engine_exit() return void
All callers ignore the return value, so simplify by not providing one.
Note that crypto_engine_exit() is typically called in a device driver's
remove path (or the error path in probe), where errors cannot be handled
anyhow.
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: keembay - Don't pass errors to the caller in .remove()
Returning an error code in the remove function of a platform device has
no effect (compared to returning zero) apart from an error message, that
the error is ignored. Then the device is removed irrespective of the
returned value.
As kmb_ocs_hcu_remove is only called after kmb_ocs_hcu_probe() returned
successfully, platform_get_drvdata() never returns NULL and so the
respective check can just be dropped.
crypto_engine_exit() might return an error code but already emits an
error message in that case, so better return zero in
kmb_ocs_hcu_remove() even in this case to suppress another error
message. All other crypto drivers also ignore the return value of
crypto_engine_exit().
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: qat - Annotate struct adf_fw_counters with __counted_by
Prepare for the coming implementation by GCC and Clang of the __counted_by
attribute. Flexible array members annotated with __counted_by can have
their accesses bounds-checked at run-time checking via CONFIG_UBSAN_BOUNDS
(for array indexing) and CONFIG_FORTIFY_SOURCE (for strcpy/memcpy-family
functions).
As found with Coccinelle[1], add __counted_by for struct adf_fw_counters.
Giovanni Cabiddu [Fri, 22 Sep 2023 09:03:47 +0000 (10:03 +0100)]
crypto: qat - increase size of buffers
Increase the size of the buffers used for composing the names used for
the transport debugfs entries and the vector name to avoid a potential
truncation.
This resolves the following errors when compiling the driver with W=1
and KCFLAGS=-Werror on GCC 12.3.1:
drivers/crypto/intel/qat/qat_common/adf_transport_debug.c: In function ‘adf_ring_debugfs_add’:
drivers/crypto/intel/qat/qat_common/adf_transport_debug.c:100:60: error: ‘snprintf’ output may be truncated before the last format character [-Werror=format-truncation=]
drivers/crypto/intel/qat/qat_common/adf_isr.c: In function ‘adf_isr_resource_alloc’:
drivers/crypto/intel/qat/qat_common/adf_isr.c:197:47: error: ‘%d’ directive output may be truncated writing between 1 and 11 bytes into a region of size between 0 and 5 [-Werror=format-truncation=]
Fixes: a672a9dc872e ("crypto: qat - Intel(R) QAT transport code") Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Damian Muszynski <damian.muszynski@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Stephan Müller [Thu, 21 Sep 2023 11:48:59 +0000 (13:48 +0200)]
crypto: jitter - Allow configuration of oversampling rate
The oversampling rate used by the Jitter RNG allows the configuration of
the heuristically implied entropy in one timing measurement. This
entropy rate is (1 / OSR) bits of entropy per time stamp.
Considering that the Jitter RNG now support APT/RCT health tests for
different OSRs, allow this value to be configured at compile time to
support systems with limited amount of entropy in their timer.
The allowed range of OSR values complies with the APT/RCT cutoff health
test values which range from 1 through 15.
The default value of the OSR selection support is left at 1 which is the
current default. Thus, the addition of the configuration support does
not alter the default Jitter RNG behavior.
Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Stephan Müller [Thu, 21 Sep 2023 11:48:33 +0000 (13:48 +0200)]
crypto: jitter - Allow configuration of memory size
The memory size consumed by the Jitter RNG is one contributing factor in
the amount of entropy that is gathered. As the amount of entropy
directly correlates with the distance of the memory from the CPU, the
caches that are possibly present on a given system have an impact on the
collected entropy.
Thus, the kernel compile time should offer a means to configure the
amount of memory used by the Jitter RNG. Although this option could be
turned into a runtime option (e.g. a kernel command line option), it
should remain a compile time option as otherwise adminsitrators who may
not have performed an entropy assessment may select a value that is
inappropriate.
The default value selected by the configuration is identical to the
current Jitter RNG value. Thus, the patch should not lead to any change
in the Jitter RNG behavior.
To accommodate larger memory buffers, kvzalloc / kvfree is used.
Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Stephan Müller [Thu, 21 Sep 2023 11:48:11 +0000 (13:48 +0200)]
crypto: jitter - add RCT/APT support for different OSRs
The oversampling rate (OSR) value specifies the heuristically implied
entropy in the recorded data - H_submitter = 1/osr. A different entropy
estimate implies a different APT/RCT cutoff value. This change adds
support for OSRs 1 through 15. This OSR can be selected by the caller
of the Jitter RNG.
For this patch, the caller still uses one hard-coded OSR. A subsequent
patch allows this value to be configured.
In addition, the power-up self test is adjusted as follows:
* It allows the caller to provide an oversampling rate that should be
tested with - commonly it should be the same as used for the actual
runtime operation. This makes the power-up testing therefore consistent
with the runtime operation.
* It calls now jent_measure_jitter (i.e. collects the full entropy
that can possibly be harvested by the Jitter RNG) instead of only
jent_condition_data (which only returns the entropy harvested from
the conditioning component). This should now alleviate reports where
the Jitter RNG initialization thinks there is too little entropy.
* The power-up test now solely relies on the (enhanced) APT and RCT
test that is used as a health test at runtime.
The code allowing the different OSRs as well as the power-up test
changes are present in the user space version of the Jitter RNG 3.4.1
and thus was already in production use for some time.
Reported-by "Ospan, Abylay" <aospan@amazon.com> Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Implement stm32_rng_suspend()/stm32_rng_resume() low-power APIs
called when the hardware block context will be lost.
There is no need to save the RNG_CR register in
stm32_rng_runtime_suspend() as the context is not lost. Therefore,
only enable/disable the RNG in the runtime sequences.
Signed-off-by: Gatien Chevallier <gatien.chevallier@foss.st.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
hwrng: stm32 - support RNG configuration locking mechanism
If "st,rng-lock-conf" DT binding property is set for a stm32-rng node,
the RNG configuration will be locked until next hardware block reset
or platform reset.
Signed-off-by: Gatien Chevallier <gatien.chevallier@foss.st.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
For NIST certification the noise source sampling may need to be
restrained.
This change implements an algorithm that gets the rate of the RNG
clock and apply the correct value in CLKDIV field in RNG_CR register
to force the RNG clock rate to be "max_clock_rate" maximum.
As it is platform-specific, implement it as a compat data.
Signed-off-by: Gatien Chevallier <gatien.chevallier@foss.st.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
hwrng: stm32 - rework error handling in stm32_rng_read()
Try to conceal seed errors when possible. If, despite the error
concealing tries, a seed error is still present, then return an error.
A clock error does not compromise the hardware block and data can
still be read from RNG_DR. Just warn that the RNG clock is too slow
and clear RNG_SR.
Signed-off-by: Gatien Chevallier <gatien.chevallier@foss.st.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The RNG present on STM32MP13x platforms introduces a customizable
configuration and the conditional reset.
STM32 RNG configuration should best fit the requirements of the
platform. Therefore, put a platform-specific RNG configuration
field in the platform data. Default RNG configuration for STM32MP13
is the NIST certified configuration [1].
While there, fix and the RNG init sequence to support all RNG
versions.
dt-bindings: rng: introduce new compatible for STM32MP13x
Introduce st,stm32mp13-rng compatible and add st,rng-lock-conf.
If st,rng-lock-conf is set, the RNG configuration in RNG_CR, RNG_HTCR
and RNG_NSCR will be locked. It is supported starting from the RNG
version present in the STM32MP13
Signed-off-by: Gatien Chevallier <gatien.chevallier@foss.st.com> Reviewed-by: Rob Herring <robh@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: x86/sha - load modules based on CPU features
x86 optimized crypto modules are built as modules rather than build-in and
they are not loaded when the crypto API is initialized, resulting in the
generic builtin module (sha1-generic) being used instead.
It was discovered when creating a sha1/sha256 checksum of a 2Gb file by
using kcapi-tools because it would take significantly longer than creating
a sha512 checksum of the same file. trace-cmd showed that for sha1/256 the
generic module was used, whereas for sha512 the optimized module was used
instead.
Add module aliases() for these x86 optimized crypto modules based on CPU
feature bits so udev gets a chance to load them later in the boot
process. This resulted in ~3x decrease in the real-time execution of
kcapi-dsg.
Fix is inspired from commit aa031b8f702e ("crypto: x86/sha512 - load based on CPU features")
where a similar fix was done for sha512.
Cc: stable@vger.kernel.org # 5.15+ Suggested-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com> Suggested-by: Julian Andres Klode <julian.klode@canonical.com> Signed-off-by: Roxana Nicolescu <roxana.nicolescu@canonical.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Adam Guerin [Thu, 14 Sep 2023 14:14:13 +0000 (15:14 +0100)]
crypto: qat - enable dc chaining service
QAT GEN4 devices support chained compression operations. These
allow, with a single request to firmware, to hash then compress
data.
Extend the configuration to enable such mode. The cfg_services
operations in sysfs are extended to allow the string "dcc". When
selected, the driver downloads to the device both the symmetric
crypto and the compression firmware images and sends an admin message
to firmware which enables `chained` operations.
In addition, it sets the device's capabilities as the combination
of compression and symmetric crypto capabilities, while excluding
the ICP_ACCEL_CAPABILITIES_CRYPTO_SYMMETRIC bit to indicate
that in this mode, symmetric crypto instances are not supported.
When "dcc" is enabled, the device will handle compression requests
as if the "dc" configuration is loaded ("dcc" is a variation of "dc")
and the driver will register the acomp algorithms.
As for the other extended configurations, "dcc" is only available for
qat_4xxx devices and the chaining service will be only accessible from
user space.
Signed-off-by: Adam Guerin <adam.guerin@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Giovanni Cabiddu [Thu, 14 Sep 2023 14:14:12 +0000 (15:14 +0100)]
crypto: qat - consolidate services structure
The data structure that associates a service id with its name is
replicated across the driver.
Remove duplication by moving this data structure to a new include file,
adf_cfg_services.h in order to have consistency across the drivers.
Note that the data structure is re-instantiated every time the new
include is added to a compilation unit.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Giovanni Cabiddu [Thu, 14 Sep 2023 09:55:49 +0000 (10:55 +0100)]
crypto: qat - fix unregistration of compression algorithms
The function adf_dev_init(), through the subsystem qat_compression,
populates the list of list of compression instances
accel_dev->compression_list. If the list of instances is not empty,
the function adf_dev_start() will then call qat_compression_registers()
register the compression algorithms into the crypto framework.
If any of the functions in adf_dev_start() fail, the caller of such
function, in the error path calls adf_dev_down() which in turn call
adf_dev_stop() and adf_dev_shutdown(), see for example the function
state_store in adf_sriov.c.
However, if the registration of compression algorithms is not done,
adf_dev_stop() will try to unregister the algorithms regardless.
This might cause the counter active_devs in qat_compression.c to get
to a negative value.
Add a new state, ADF_STATUS_COMPRESSION_ALGS_REGISTERED, which tracks
if the compression algorithms are registered into the crypto framework.
Then use this to unregister the algorithms if such flag is set. This
ensures that the compression algorithms are only unregistered if
previously registered.
Fixes: 1198ae56c9a5 ("crypto: qat - expose deflate through acomp api for QAT GEN2") Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Adam Guerin <adam.guerin@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Giovanni Cabiddu [Thu, 14 Sep 2023 09:55:48 +0000 (10:55 +0100)]
crypto: qat - fix unregistration of crypto algorithms
The function adf_dev_init(), through the subsystem qat_crypto, populates
the list of list of crypto instances accel_dev->crypto_list.
If the list of instances is not empty, the function adf_dev_start() will
then call qat_algs_registers() and qat_asym_algs_register() to register
the crypto algorithms into the crypto framework.
If any of the functions in adf_dev_start() fail, the caller of such
function, in the error path calls adf_dev_down() which in turn call
adf_dev_stop() and adf_dev_shutdown(), see for example the function
state_store in adf_sriov.c.
However, if the registration of crypto algorithms is not done,
adf_dev_stop() will try to unregister the algorithms regardless.
This might cause the counter active_devs in qat_algs.c and
qat_asym_algs.c to get to a negative value.
Add a new state, ADF_STATUS_CRYPTO_ALGS_REGISTERED, which tracks if the
crypto algorithms are registered into the crypto framework. Then use
this to unregister the algorithms if such flag is set. This ensures that
the crypto algorithms are only unregistered if previously registered.
Fixes: d8cba25d2c68 ("crypto: qat - Intel(R) QAT driver framework") Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Adam Guerin <adam.guerin@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Giovanni Cabiddu [Thu, 14 Sep 2023 09:55:47 +0000 (10:55 +0100)]
crypto: qat - ignore subsequent state up commands
If the device is already in the up state, a subsequent write of `up` to
the sysfs attribute /sys/bus/pci/devices/<BDF>/qat/state brings the
device down.
Fix this behaviour by ignoring subsequent `up` commands if the device is
already in the up state.
Fixes: 1bdc85550a2b ("crypto: qat - fix concurrency issue when device state changes") Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Adam Guerin <adam.guerin@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Giovanni Cabiddu [Thu, 14 Sep 2023 09:55:46 +0000 (10:55 +0100)]
crypto: qat - do not shadow error code
Do not shadow the return code from adf_dev_down() in the error path of
the DEV_DOWN command.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Adam Guerin <adam.guerin@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Giovanni Cabiddu [Thu, 14 Sep 2023 09:55:45 +0000 (10:55 +0100)]
crypto: qat - fix state machines cleanup paths
Commit 1bdc85550a2b ("crypto: qat - fix concurrency issue when device
state changes") introduced the function adf_dev_down() which wraps the
functions adf_dev_stop() and adf_dev_shutdown().
In a subsequent change, the sequence adf_dev_stop() followed by
adf_dev_shutdown() was then replaced across the driver with just a call
to the function adf_dev_down().
The functions adf_dev_stop() and adf_dev_shutdown() are called in error
paths to stop the accelerator and free up resources and can be called
even if the counterparts adf_dev_init() and adf_dev_start() did not
complete successfully.
However, the implementation of adf_dev_down() prevents the stop/shutdown
sequence if the device is found already down.
For example, if adf_dev_init() fails, the device status is not set as
started and therefore a call to adf_dev_down() won't be calling
adf_dev_shutdown() to undo what adf_dev_init() did.
Do not check if a device is started in adf_dev_down() but do the
equivalent check in adf_sysfs.c when handling a DEV_DOWN command from
the user.
Fixes: 2b60f79c7b81 ("crypto: qat - replace state machine calls") Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Adam Guerin <adam.guerin@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Yang Shen [Thu, 14 Sep 2023 09:09:08 +0000 (17:09 +0800)]
crypto: hisilicon/zip - remove zlib and gzip
Remove the support of zlib-deflate and gzip.
Signed-off-by: Yang Shen <shenyang39@huawei.com> Reviewed-by: Longfang Liu <liulongfang@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Thu, 14 Sep 2023 08:28:27 +0000 (16:28 +0800)]
crypto: ecb - Convert from skcipher to lskcipher
This patch adds two different implementations of ECB. First of
all an lskcipher wrapper around existing ciphers is introduced as
a temporary transition aid.
Secondly a permanent lskcipher template is also added. It's simply
a wrapper around the underlying lskcipher algorithm.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Thu, 14 Sep 2023 08:28:25 +0000 (16:28 +0800)]
crypto: lskcipher - Add compatibility wrapper around ECB
As an aid to the transition from cipher algorithm implementations
to lskcipher, add a temporary wrapper when creating simple lskcipher
templates by using ecb(X) instead of X if an lskcipher implementation
of X cannot be found.
This can be reverted once all cipher implementations have switched
over to lskcipher.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Thu, 14 Sep 2023 08:28:24 +0000 (16:28 +0800)]
crypto: skcipher - Add lskcipher
Add a new API type lskcipher designed for taking straight kernel
pointers instead of SG lists. Its relationship to skcipher will
be analogous to that between shash and ahash.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
`strncpy` is deprecated for use on NUL-terminated destination strings [1].
We know `hw.partname` is supposed to be NUL-terminated by its later use with seq_printf:
| nitrox_debugfs.c +25
| seq_printf(s, " Part Name: %s\n", ndev->hw.partname);
Let's prefer a more robust and less ambiguous string interface.
A suitable replacement is `strscpy` [2] due to the fact that it guarantees
NUL-termination on the destination buffer.
Martin Kaiser [Tue, 12 Sep 2023 14:31:18 +0000 (16:31 +0200)]
hwrng: imx-rngc - reasonable timeout for initial seed
Set a more reasonable timeout for calculating the initial seed.
The reference manuals says that "The initial seed takes approximately
2,000,000 clock cycles." The rngc peripheral clock runs at >= 33.25MHz,
so seeding takes at most 60ms.
A timeout of 200ms is more appropriate than the current value of 3
seconds.
Signed-off-by: Martin Kaiser <martin@kaiser.cx> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Martin Kaiser [Tue, 12 Sep 2023 14:31:17 +0000 (16:31 +0200)]
hwrng: imx-rngc - reasonable timeout for selftest
Set a more reasonable timeout for the rngc selftest.
According to the reference manual, "The self test takes approximately
29,000 cycles to complete." The lowest possible frequency of the rngc
peripheral clock is 33.25MHz, the selftest would then take about 872us.
2.5ms should be enough for the selftest timeout.
Signed-off-by: Martin Kaiser <martin@kaiser.cx> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
projects / thirdparty / kernel / linux.git / log
