As registration is usually carried out during module init, this
is a context where as little work as possible should be carried
out. Testing may trigger module loads of underlying components,
which could even lead back to the module that is registering at
the moment. This may lead to dead-locks outside of the Crypto API.
Avoid this by not waiting for the tests to complete. They will
be scheduled but completion will be asynchronous. Any users will
still wait for completion.
Reported-by: Russell King <linux@armlinux.org.uk>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In order to allow testing to complete asynchronously after the
registration process, instance larvals need to complete prior
to having a test result. Support this by redoing the lookup for
instance larvals after completion. This should locate the pending
test larval and then repeat the wait on that (if it is still pending).
As the lookup is now repeated there is no longer any need to compute
the fulfilment status and all that code can be removed.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use the generic crypto_authenc_extractkeys helper instead of custom
parsing code that is slightly broken. Also fix a number of memory
leaks by moving memory allocation from setkey to init_tfm (setkey
can be called multiple times over the life of a tfm).
Finally accept all hash key lengths by running the digest over
extra-long keys.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use the generic crypto_authenc_extractkeys helper instead of custom
parsing code that is slightly broken. Also fix a number of memory
leaks by moving memory allocation from setkey to init_tfm (setkey
can be called multiple times over the life of a tfm).
Finally accept all hash key lengths by running the digest over
extra-long keys.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch fixes counter width checks according to the version extension3
register. The counter widths can be 8, 16, 32 and 64 bits as per the
extension3 register.
Signed-off-by: Bhoomika K <bhoomikak@vayavyalabs.com>
Signed-off-by: Pavitrakumar M <pavitrakumarm@vayavyalabs.com>
Acked-by: Ruud Derwig <Ruud.Derwig@synopsys.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The macro `ADF_RP_INT_SRC_SEL_F_RISE_MASK` is currently set to the value
`0100b` which means "Empty Going False". This might cause an incorrect
restore of the bank state during live migration.
Fix the definition of the macro to properly represent the "Full Going
True" state which is encoded as `0011b`.
Fixes: bbfdde7d19 ("crypto: qat - add bank save and restore flows")
Signed-off-by: Svyatoslav Pankratov <svyatoslav.pankratov@intel.com>
Reviewed-by: Xin Zeng <xin.zeng@intel.com>
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Smatch complains that:
drivers/crypto/dwc-spacc/spacc_aead.c:1031 spacc_aead_process()
error: uninitialized symbol 'ptaadsize'.
This could happen if, for example, tctx->mode was CRYPTO_MODE_NULL and
req->cryptlen was less than icvremove.
Fixes: 06af76b46c ("crypto: spacc - Add SPAcc aead support")
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Put the break statement should be on its own line.
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The spacc->config.modes[] array has CRYPTO_MODE_LAST number of elements
so this > comparison should be >= to prevent an out of bounds access.
Fixes: c8981d9230 ("crypto: spacc - Add SPAcc Skcipher support")
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This bounds checking is off by one. The > should be >=. The
spacc->job[] array is allocated in spacc_init() and it has
SPACC_MAX_JOBS elements.
Fixes: 8ebb14deef ("crypto: spacc - Enable SPAcc AUTODETECT")
Fixes: c8981d9230 ("crypto: spacc - Add SPAcc Skcipher support")
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Print additional information, in the form of the old and new versions of
the SEV firmware, so that it can be seen what the base firmware was before
the upgrade.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Ashish Kalra <ashish.kalra@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The macros FOUR_ROUNDS_AND_SCHED and DO_4ROUNDS rely on an
unexpected/undocumented behavior of the GNU assembler, which might
change in the future
(https://sourceware.org/bugzilla/show_bug.cgi?id=32073).
M (1) (2) // 1 arg !? Future: 2 args
M 1 + 2 // 1 arg !? Future: 3 args
M 1 2 // 2 args
Add parentheses around the single arguments to support future GNU
assembler and LLVM integrated assembler (when the IsOperator hack from
the following link is dropped).
Link: 055006475e
Signed-off-by: Fangrui Song <maskray@google.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add MODULE_DEVICE_TABLE(), so modules could be properly autoloaded
based on the alias from of_device_id table.
Signed-off-by: Liao Chen <liaochen4@huawei.com>
Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The user space Jitter RNG library uses the oversampling rate of 3 which
implies that each time stamp is credited with 1/3 bit of entropy. To
obtain 256 bits of entropy, 768 time stamps need to be sampled. The
increase in OSR is applied based on a report where the Jitter RNG is
used on a system exhibiting a challenging environment to collect
entropy.
This OSR default value is now applied to the Linux kernel version of
the Jitter RNG as well.
The increase in the OSR from 1 to 3 also implies that the Jitter RNG is
now slower by default.
Reported-by: Jeff Barnes <jeffbarnes@microsoft.com>
Signed-off-by: Stephan Mueller <smueller@chronox.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The custom swap functions used in octeontx2 driver do not perform any
special operations and can be replaced with the built-in swap function
of sort. This change not only reduces code size but also improves
efficiency, especially in scenarios where CONFIG_RETPOLINE is enabled,
as it makes indirect function calls more expensive.
By using the built-in swap, we avoid these costly indirect function
calls, leading to better performance.
Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The custom swap function used in octeontx driver do not perform any
special operations and can be replaced with the built-in swap function
of sort. This change not only reduces code size but also improves
efficiency, especially in scenarios where CONFIG_RETPOLINE is enabled,
as it makes indirect function calls more expensive.
By using the built-in swap, we avoid these costly indirect function
calls, leading to better performance.
Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Fixes: 6637e11e4a ("crypto: rsa - allow only odd e and restrict value in FIPS mode")
Fixes: f145d411a6 ("crypto: rsa - implement Chinese Remainder Theorem for faster private key operation")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Now that mpi_rshift can return errors, check them.
Fixes: 35d2bf2068 ("crypto: dh - calculate Q from P for the full public key verification")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The remaining functions added by commit
a8ea8bdd9d did not check for memory
allocation errors. Add the checks and change the API to allow errors
to be returned.
Fixes: a8ea8bdd9d ("lib/mpi: Extend the MPI library")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This partially reverts commit a8ea8bdd9d.
Most of it is no longer needed since sm2 has been removed. However,
the following functions have been kept as they have developed other
uses:
mpi_copy
mpi_mod
mpi_test_bit
mpi_set_bit
mpi_rshift
mpi_add
mpi_sub
mpi_addm
mpi_subm
mpi_mul
mpi_mulm
mpi_tdiv_r
mpi_fdiv_r
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In aes-neonbs, instead of going through the crypto API for the parts
that the bit-sliced AES code doesn't handle, namely AES-CBC encryption
and single-block AES, just call the ARM scalar AES cipher directly.
This basically goes back to the original approach that was used before
commit b56f5cbc7e ("crypto: arm/aes-neonbs - resolve fallback cipher
at runtime"). Calling the ARM scalar AES cipher directly is faster,
simpler, and avoids any chance of bugs specific to the use of fallback
ciphers such as module loading deadlocks which have happened twice. The
deadlocks turned out to be fixable in other ways, but there's no need to
rely on anything so fragile in the first place.
The rationale for the above-mentioned commit was to allow people to
choose to use a time-invariant AES implementation for the fallback
cipher. There are a couple problems with that rationale, though:
- In practice the ARM scalar AES cipher (aes-arm) was used anyway, since
it has a higher priority than aes-fixed-time. Users *could* go out of
their way to disable or blacklist aes-arm, or to lower its priority
using NETLINK_CRYPTO, but very few users customize the crypto API to
this extent. Systems with the ARMv8 Crypto Extensions used aes-ce,
but the bit-sliced algorithms are irrelevant on such systems anyway.
- Since commit 913a3aa07d ("crypto: arm/aes - add some hardening
against cache-timing attacks"), the ARM scalar AES cipher is partially
hardened against cache-timing attacks. It actually works like
aes-fixed-time, in that it disables interrupts and prefetches its
lookup table. It does use a larger table than aes-fixed-time, but
even so, it is not clear that aes-fixed-time is meaningfully more
time-invariant than aes-arm. And of course, the real solution for
time-invariant AES is to use a CPU that supports AES instructions.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When submitting more than 2^32 padata objects to padata_do_serial, the
current sorting implementation incorrectly sorts padata objects with
overflowed seq_nr, causing them to be placed before existing objects in
the reorder list. This leads to a deadlock in the serialization process
as padata_find_next cannot match padata->seq_nr and pd->processed
because the padata instance with overflowed seq_nr will be selected
next.
To fix this, we use an unsigned integer wrap around to correctly sort
padata objects in scenarios with integer overflow.
Fixes: bfde23ce20 ("padata: unbind parallel jobs from specific CPUs")
Cc: <stable@vger.kernel.org>
Co-developed-by: Christian Gafert <christian.gafert@rohde-schwarz.com>
Signed-off-by: Christian Gafert <christian.gafert@rohde-schwarz.com>
Co-developed-by: Max Ferger <max.ferger@rohde-schwarz.com>
Signed-off-by: Max Ferger <max.ferger@rohde-schwarz.com>
Signed-off-by: Van Giang Nguyen <vangiang.nguyen@rohde-schwarz.com>
Acked-by: Daniel Jordan <daniel.m.jordan@oracle.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Commit 1e6b251ce1 ("crypto: nx - Avoid -Wflex-array-member-not-at-end
warning") introduced tagged `struct nx842_crypto_header_hdr`. We want
to ensure that when new members need to be added to the flexible
structure, they are always included within this tagged struct.
So, we use `static_assert()` to ensure that the memory layout for
both the flexible structure and the tagged struct is the same after
any changes.
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Commit 140e4c85d5 ("crypto: qat - Avoid -Wflex-array-member-not-at-end
warnings") introduced tagged `struct qat_alg_buf_list_hdr`. We want to
ensure that when new members need to be added to the flexible structure,
they are always included within this tagged struct.
So, we use `static_assert()` to ensure that the memory layout for
both the flexible structure and the tagged struct is the same after
any changes.
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Acked-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When objtool gains support for ARM in the future, it may encounter issues
disassembling the following data in the .text section:
> .Lzeros:
> .long 0,0,0,0,0,0,0,0
> .asciz "Poly1305 for ARMv8, CRYPTOGAMS by \@dot-asm"
> .align 2
Move it to .rodata which is a more appropriate section for read-only data.
There is a limit on how far the label can be from the instruction, hence
use "adrp" and low 12bits offset of the label to avoid the compilation
error.
Signed-off-by: Jia He <justin.he@arm.com>
Tested-by: Daniel Gomez <da.gomez@samsung.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add the __counted_by compiler attribute to the flexible array member
salt to improve access bounds-checking via CONFIG_UBSAN_BOUNDS and
CONFIG_FORTIFY_SOURCE.
Reviewed-by: Kees Cook <kees@kernel.org>
Signed-off-by: Thorsten Blum <thorsten.blum@toblux.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Fixup for "crypto: spacc - Add SPAcc Skcipher support"
interacting with commit
1a251f52cf ("minmax: make generic MIN() and MAX() macros available everywhere")
from Linus' tree.
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
Reintroduced MIN macro with ifndef around it.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
On PREEMPT_RT, kfree() takes sleeping locks and must not be called with
preemption disabled. Therefore, on PREEMPT_RT skcipher_walk_done() must
not be called from within a kernel_fpu_{begin,end}() pair, even when
it's the last call which is guaranteed to not allocate memory.
Therefore, move the last skcipher_walk_done() in gcm_crypt() to the end
of the function so that it goes after the kernel_fpu_end(). To make
this work cleanly, rework the data processing loop to handle only
non-last data segments.
Fixes: b06affb1cb ("crypto: x86/aes-gcm - add VAES and AVX512 / AVX10 optimized AES-GCM")
Reported-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Closes: https://lore.kernel.org/linux-crypto/20240802102333.itejxOsJ@linutronix.de
Signed-off-by: Eric Biggers <ebiggers@google.com>
Tested-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add the missing clk_disable_unprepare() before return in
cctrng_resume().
Fixes: a583ed310b ("hwrng: cctrng - introduce Arm CryptoCell driver")
Cc: <stable@vger.kernel.org>
Signed-off-by: Gaosheng Cui <cuigaosheng1@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Rockchip SoCs used to have a random number generator as part of their
crypto device, and support for it has to be added to the corresponding
driver. However newer Rockchip SoCs like the RK3568 have an independent
True Random Number Generator device. This patch adds a driver for it,
greatly inspired from the downstream driver.
The TRNG device does not seem to have a signal conditionner and the FIPS
140-2 test returns a lot of failures. They can be reduced by increasing
RK_RNG_SAMPLE_CNT, in a tradeoff between quality and speed. This value
has been adjusted to get ~90% of successes and the quality value has
been set accordingly.
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
[daniel@makrotpia.org: code style fixes]
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Acked-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add the True Random Number Generator on the Rockchip RK3568 SoC.
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Switch cpumask_t to cpumask_var_t as the former may be too big
for the stack:
CC [M] drivers/crypto/caam/qi.o
CC [M] drivers/crypto/caam/caamalg_qi2.o
../drivers/crypto/caam/qi.c: In function ‘caam_qi_init’:
../drivers/crypto/caam/qi.c:808:1: warning: the frame size of 1056 bytes is larger than 1024 bytes [-Wframe-larger-than=]
808 | }
| ^
CHECK ../drivers/crypto/caam/qi.c
../drivers/crypto/caam/caamalg_qi2.c: In function ‘dpaa2_dpseci_setup’:
../drivers/crypto/caam/caamalg_qi2.c:5135:1: warning: the frame size of 1032 bytes is larger than 1024 bytes [-Wframe-larger-than=]
5135 | }
| ^
Also fix the error path handling in qi.c.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Before commit addea5858b ("hwrng: Kconfig - Do not enable by default
CN10K driver") the Marvell CN10K Random Number Generator was always
enabled when HW_RANDOM was enabled.
This was changed with that commit to prevent having this driver being
always enabled on arm64. To prevent introducing regression with some old
defconfig enable the driver when ARCH_THUNDER is enabled.
Fixes: addea5858b ("hwrng: Kconfig - Do not enable by default CN10K driver")
Closes: https://lore.kernel.org/all/SN7PR18MB53144B37B82ADEEC5D35AE0CE3AC2@SN7PR18MB5314.namprd18.prod.outlook.com/
Signed-off-by: Francesco Dolcini <francesco.dolcini@toradex.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The free_device_compression_mode(iaa_device, device_mode) function frees
"device_mode" but it iss passed to iaa_compression_modes[i]->free() a few
lines later resulting in a use after free.
The good news is that, so far as I can tell, nothing implements the
->free() function and the use after free happens in dead code. But, with
this fix, when something does implement it, we'll be ready. :)
Fixes: b190447e0f ("crypto: iaa - Add compression mode management along with fixed mode")
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Reviewed-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Since commit 1fffe7a34c ("script: modpost: emit a warning when the
description is missing"), a module without a MODULE_DESCRIPTION() will
result in a warning with make W=1. The following warning is being
observed when building ppc64le with CRYPTO_CURVE25519_PPC64=m:
WARNING: modpost: missing MODULE_DESCRIPTION() in arch/powerpc/crypto/curve25519-ppc64le.o
Add the missing invocation of the MODULE_DESCRIPTION() macro.
Signed-off-by: Jeff Johnson <quic_jjohnson@quicinc.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The QAT driver allows enabling SR-IOV VFs but does not allow them to be
disabled through a write to sysfs.
Disabling SR-IOV VFs can be only achieved by bringing down and up a
device using the attribute /sys/bus/pci/devices/<BDF>/qat/state.
The documentation for the sysfs attribute `sriov_numvfs` specifies
that "a userspace application wanting to disable the VFs would write a
zero to this file".
Add support for disabling SR-IOV VFs by writing '0' to the
'sriov_numvfs' attribute in sysfs.
Enabling or disabling SR-IOV always requires adf_dev_down() to be
called. This action subsequently leads to the deletion of the
ADF_KERNEL_SEC configuration section. The keys ADF_NUM_CY and ADF_NUM_DC
within that section must be set to '0', otherwise, the driver will
register into the Linux Crypto Framework. Because of this, the
configuration in the ADF_KERNEL_SEC section must be added before every
sriov_enable.
Signed-off-by: Michal Witwicki <michal.witwicki@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In the process of sending the ADF_PF2VF_MSGTYPE_RESTARTING message to
Virtual Functions (VFs), the Physical Function (PF) should set the
`vf->restarting` flag to true before dispatching the message.
This change is necessary to prevent a race condition where the handling
of the ADF_VF2PF_MSGTYPE_RESTARTING_COMPLETE message (which sets the
`vf->restarting` flag to false) runs immediately after the message is sent,
but before the flag is set to true.
Set the `vf->restarting` to true before sending the message
ADF_PF2VF_MSGTYPE_RESTARTING, if supported by the version of the
protocol and if the VF is started.
Fixes: ec26f8e6c7 ("crypto: qat - update PFVF protocol for recovery")
Signed-off-by: Michal Witwicki <michal.witwicki@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
