Richard Purdie [Tue, 19 Dec 2023 16:04:24 +0000 (16:04 +0000)]
testimage: Drop target_dumper and most of monitor_dumper
The target_dumper code is basically broken. It has been reading binary files
over the text base serial communication and runs at every command failure which
makes no sense. Each run might overwrite files from the previous run and the
output appears corrupted due to confusion from the binary data.
For now, remove the commands and the target dumper code as the command
and execution point are problematic. Also remove the same pieces of the monitor
code but leave the command list since in theory this can be moved to a more
useful place in the code.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whilst debugging an autobuilder failure, I wondered why it was rebuilding qemu-system-native
instead of reusing from sstate. The reason was it was overwriting DISTRO_FEATURES,
in this case removing opengl which caused much to rebuild.
The test doesn't need that so don't do it.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 18 Dec 2023 15:14:08 +0000 (15:14 +0000)]
qemurunner: Improve handling of serial port output blocking
Similar to stdout in the previous commit, we need to ensure serial output
if written is read and put somewhere, else qemu might block on writes to
the serial port leading to hangs in the kernel. Use our existing logging
thread to log data when run_serial is not in use.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 18 Dec 2023 12:18:45 +0000 (12:18 +0000)]
qemurunner: Improve stdout logging handling
We need to ensure we read from and log the output from qemu stdout
as otherwise the buffers can fill and block, leading qemu to hang.
Use our existing logging thread to do this.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 19 Dec 2023 14:29:32 +0000 (14:29 +0000)]
testimage: Exclude wtmp from target-dumper commands
wtmp is filled with binary data which the run_serial command can't cope with.
Catting this results in confusion of the serial interface and potentially large
backlogs of data in the buffers which can hang qemu.
Exclude the problematic files from the command.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Victor Kamensky [Mon, 18 Dec 2023 06:07:57 +0000 (22:07 -0800)]
systemtap: fix libdebuginfod auto detection logic
Addresses an issue when systemtap is built in situation where debuginfod
is not part of DISTRO_FEATURES and latest SystemTap has problem of
auto detecting libdebuginfod library as it was reported by Martin
Jansa in [1].
After systemtap commit 2e67b053e3796ee7cf29a39f9698729b52078406
"configury: rework debuginfod searches" automatic detection of
libdebuginfod is broken. Let's handle it explicitly with debuginfod
package feature. It is good to have such feature anyway.
Signed-off-by: Victor Kamensky <victor.kamensky7@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Joshua Watt [Fri, 15 Dec 2023 20:34:04 +0000 (13:34 -0700)]
rpcbind: Specify state directory under /run
The state directory must be correctly specified as under /run because
RequiresMountsFor doesn't follow symbolic links which means the unit may
run before /run is mounted if the default of /var/run/rpcbind is kept
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Joshua Watt [Mon, 18 Dec 2023 15:38:02 +0000 (08:38 -0700)]
bitbake.conf: Add runtimedir
Adds the path to the runtime state directory (/run). In particular,
systemd units need to have the correct path to the runtime directory
because RequiresMountsFor doesn't follow symbolic links. This means that
if a unit calls out a directory in /var/run (a symbolic link to /run),
it may actually start before /run is mounted. The fix is to have the
unit specify a directory in /run instead.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tim Orling [Sun, 17 Dec 2023 05:41:33 +0000 (21:41 -0800)]
oeqa: add "maturin develop" SDK test case
'maturin develop' first checks that a virtual environment
has been created, which is a good test for our python3 SDK
environment ;)
Source for guessing-game lifted from https://www.maturin.rs/tutorial
The test case is expected to fetch any necessary crates, build a
development version of the crate and package it as a wheel
Needs at a minimum the following in e.g. local.conf:
TOOLCHAIN_HOST_TASK:append = " nativesdk-python3-maturin"
SDK_INCLUDE_TOOLCHAIN = '1'
SDK_TOOLCHAIN_LANGS += 'rust'
The output of 'maturin develop' should be something like:
...
🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.8
🐍 Not using a specific python interpreter
📡 Using build options features from pyproject.toml
...
Compiling guessing-game v0.1.0 (/path/to/guessing-game)
Finished dev [unoptimized + debuginfo] target(s) in 7.14s
📦 Built wheel for abi3 Python ≥ 3.8 to /path/to/tmpdir/guessing_game-0.1.0-cp38-abi3-linux_x86_64.whl
🛠 Installed guessing-game-0.1.0
Signed-off-by: Tim Orling <tim.orling@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tim Orling [Sun, 17 Dec 2023 05:41:28 +0000 (21:41 -0800)]
python3-maturin: bzip2-sys reproduciblility
The /usr/bin/.debug/maturin binary ends up with the path
to the bzip-sys (bzip2-1.0.8) source in the python3-maturin-dbg package
Fix this by mapping the debug path for CARGO_HOME
to TARGET_DBGSRC_DIR/cargo_home via CFLAGS. This is
required because the bzip-sys crate is compiled using
the cc crate and we override default CFLAGS.
Signed-off-by: Tim Orling <tim.orling@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tim Orling [Sun, 17 Dec 2023 05:41:27 +0000 (21:41 -0800)]
python3-maturin: add v1.4.0
Build and publish crates with pyo3, rust-cpython, cffi and uniffi bindings
as well as rust binaries as python packages.
This project is meant as a zero configuration replacement for setuptools-rust
and milksnake. It supports building wheels for python 3.7+ on windows, linux,
mac and freebsd, can upload them to pypi and has basic pypy and graalpy support.
https://github.com/pyo3/maturin
* Referesh -crates.inc for 1.4.0
Signed-off-by: Tim Orling <tim.orling@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
oeqa/selftest/sstatetests: re-work CDN tests, add local cache tests
With the rework of printdiff, it is not longer useful for checking
absence of sstate objects in a remote http cache, as it would only
report the top level missing signatures, and leave the recursive
investigation to diffsigs (which relies on ability to list cache
files - not available over http).
The CDN check can be performed by simply running 'bitbake -DD -n'
which is very verbose, but neverthless reports the amount
of missing sstate objects and what they are in a way that can
be programmatically extracted and checked (as suggested by RP).
This also adds local sstate tests, as they can be useful to
determine whether the missing cdn objects were never created or
erroneously cleaned up, or if they were created but didn't propagate
to cdn.
[YOCTO #15303]
Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool: modify: Handle recipes with a menuconfig task correctly
This avoids the following error when running `devtool modify` on a
recipe that has a menuconfig task, but does not have
KCONFIG_CONFIG_ENABLE_MENUCONFIG set.
.../temp/run.do_configure.4163366: line 152:
${@ oe.types.boolean('${KCONFIG_CONFIG_ENABLE_MENUCONFIG}') }: bad substitution
WARNING: .../temp/run.do_configure.4163366:152 exit 1 from
'[ ${@ oe.types.boolean('${KCONFIG_CONFIG_ENABLE_MENUCONFIG}') } = True ]'
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Khem Raj [Thu, 14 Dec 2023 19:34:36 +0000 (11:34 -0800)]
util-linux: Delete md-raid tests
These tests are marked as known failures upstream as well [1]
if we delete them then we can drop dependency on mdadm for ptests which
can further reduce the dependency chain for ptest enabled minimal images
Yoann Congal [Wed, 13 Dec 2023 08:56:58 +0000 (09:56 +0100)]
strace: Disable bluetooth support by default
The bluetooth support adds a bluez5 dependency (and,recursively, a lot
of other stuff). Disable it by default to avoid having to build all of
this when it is not needed.
This decrease the number of tasks run for a core-image-minimal build by
~1000 (-21%).
To re-enable bluetooth support in strace, add "bluez" to strace
PACKAGECONFIG. For example, in local.conf:
PACKAGECONFIG:append:pn-strace = " bluez"
Fixes [YOCTO #15323]
[RP: Tweaked to put a message about the change in the recipe] Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Suggested-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sun, 10 Dec 2023 09:33:17 +0000 (09:33 +0000)]
sstate: Fix dir ownership issues in SSTATE_DIR
We currently use mkdir -p to create missing parent directories within SSTATE_DIR.
Reading the man page for mkdir mentions that parent directories are created with
the current umask, *not* the mode passed upon the commandline.
We could fix this by setting and resetting the umask but since we already have
decent python code able to do this, move to using that injecting a python function
into the chain of functions already present.
This should help fix the occasional sstate directory creation with the wrong
permissions.
[YOCTO #14385]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Wed, 13 Dec 2023 20:30:14 +0000 (20:30 +0000)]
runqemu: add qmp socket support
Add support for qmp sockets and defaults to unix:qmp.sock if unspecified
Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add new file for C and C++ build tools. The initial implemmentation
contains a class for CMake and one for Meson. At least these first
tests for the qemu-usermode share most of the code. That's why there
is only one c_ccp.py file and not for example a cmake.py and a
meson.py file.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This simple C++ project supports compilation with CMake and Meson.
(Autotool support could be added later on.)
It's supposed to be used with oe-selftest.
An artificial project has several advantages over compiling a normal
CMake or Meson based project for testing purposes:
- It is much faster because it can be kept minimalistic
- It can cover multiple odd corner cases
- No one will change it in an unpredictable way
- It can support multiple build tools with only one C++ codebase
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make the cmake-qemu.bbclass more usable:
- Drop the CMAKE_EXEWRAPPER_ENABLED variable (which does not make much
sense without the second commit of the original patch series).
- Inherit qemu to make the cmake-qemu a drop in replacement for cmake.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Alex Kiernan [Fri, 8 Dec 2023 14:03:44 +0000 (14:03 +0000)]
rust: Upgrade 1.70.0 -> 1.71.0
Switch libstd-rs to use the dummy `sysroot` crate which represents the
standard library crates. Target getrandom-open64.patch at 0.2.8 (merged
for 0.2.9). Drop bootstrap_fail.patch (backport merged).
Yash Shinde [Fri, 8 Dec 2023 14:03:43 +0000 (14:03 +0000)]
rust: Disable rust oe-selftest
After rust is upgraded to 1.71 and later versions, the rust oe-selftest gives
build errors due to unstable nightly options(see the error mentioned below). Thus, disable the test suite
until the issue is fixed
error: the option `Z` is only accepted on the nightly compiler
thread 'main' panicked at 'failed to gather the target spec for x86_64-poky-linux-gnu', synthetic_targets.rs:66:9
Following issues are created in Yocto bugzilla and rust upstream to track this issue-
Julien Stephan [Mon, 11 Dec 2023 17:01:38 +0000 (18:01 +0100)]
oeqa/selftest/recipetool: fix metadata corruption on meta layer
[YOCTO #15314]
test_recipetool_appendsrcfile_update_recipe_basic is using base-files as
test recipe but modifies it directly which can corrupt metadata for other
tests relying on this recipe.
So use mtd-utils-selftest as test recipe from meta-selftest to avoid
this kind of issues
Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Robert Yang [Mon, 11 Dec 2023 15:13:17 +0000 (07:13 -0800)]
archiver.bbclass: Improve work-shared checking
There are other recipes except the listed ones which use work-shared, improve
the checking to make other recipes such as llvm-project-source work with
do_ar_patched.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Ross Burton [Mon, 11 Dec 2023 13:49:46 +0000 (13:49 +0000)]
go: set vendor in CVE_PRODUCT
It's not uncommon for specific third party modules to use "go" as the
product[1]. However, the canonical CPE for the official Go
language/runtime is always golang:go[2], so use that explicitly.
[1] e.g. https://nvd.nist.gov/vuln/detail/CVE-2023-49292
[2] e.g. https://nvd.nist.gov/vuln/detail/CVE-2023-39320
Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Dhairya Nagodra [Mon, 11 Dec 2023 10:05:00 +0000 (02:05 -0800)]
cve-update-nvd2-native: increase the delay between subsequent request failures
Sometimes NVD servers are unstable and return too many errors.
There is an option to have higher fetch attempts to increase the chances
of successfully fetching the CVE data.
Additionally, it also makes sense to progressively increase the delay
after a failed request to an already unstable or busy server.
The increase in delay is reset after every successful request and
the maximum delay is limited to 30 seconds.
Dhairya Nagodra [Mon, 11 Dec 2023 10:04:59 +0000 (02:04 -0800)]
cve-update-nvd2-native: faster requests with API keys
As per NVD, the public rate limit is 5 requests in 30s (6s delay).
Using an API key increases the limit to 50 requests in 30s (0.6s delay).
However, NVD still recommends sleeping for several seconds so that the
other legitimate requests are serviced without denial or interruption.
Keeping the default sleep at 6 seconds and 2 seconds with an API key.
For failures, the wait time is unchanged (6 seconds).
If a task is adde which has a dependency on the do_populate_sysroot task of
the recipe, it will cause it to be installed into the sysroot (similar to
do_addto_recipe_sysroot). This fails since the postinst script is an overlapping
file:
The copy written out at do_prepare_recipe_sysroot time is just for debug so
rename it, meaning there are no longer overlapping files and the installation
can be successful, removing the error.
[YCOTO #14961]
With the bug fixed, enable the test.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Alex Kiernan [Fri, 8 Dec 2023 12:00:28 +0000 (12:00 +0000)]
cargo: Add CARGO_LOCK_PATH for path to Cargo.lock
When building a workspace enabled project, the Cargo.lock is found at
the root of the project, not alongside the Cargo.toml. Expose
CARGO_LOCK_PATH so it can be explicitly configured.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bluetooth HID Hosts in BlueZ may permit an unauthenticated Peripheral role
HID Device to initiate and establish an encrypted connection, and accept HID
keyboard reports,potentially permitting injection of HID messages when no user
interaction has occurred in the Central role to authorize such access. An example
affected package is bluez 5.64-0ubuntu1 in Ubuntu 22.04LTS. NOTE: in some cases,
a CVE-2020-0556 mitigation would have already addressed this Bluetooth HID Hosts issue.
Ross Burton [Fri, 8 Dec 2023 10:56:55 +0000 (10:56 +0000)]
insane: ensure more paths have the workdir removed
When showing paths to the user we don't want to include the whole build
directory. Passing the package name to package_qa_clean_path strips
this completely.
Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tim Orling [Sat, 9 Dec 2023 01:40:04 +0000 (17:40 -0800)]
recipetool: pypi: do not clobber SRC_URI checksums
The pypi change:
"85a2a6f68af recipetool: create_buildsys_python: add pypi support"
deleted all the SRC_URI variables, including the SRC_URI checksums.
These are not generated by the pypi.bbclass (how could they be trusted?)
Without the checksum(s), we are vulnerable to a man-in-the-middle attack
and zero checks on the validity of the downloaded tarball from pypi.org.
Fix by only setting S and SRC_URI to None.
Signed-off-by: Tim Orling <tim.orling@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
sstatesig/find_siginfo: special-case gcc-source when looking in sstate caches
This is already done for local stamps just above, and will allow enabling
the full selftest that compares gcc-source signatures via printdiff
(that is, both local stamp and sstate variants).
Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If recipe A requires the useradd actions of recipe B we need to
ensure that recipe B is part of the recipe A dependancy chain. In
order to do that, we introduce USERADD_DEPENDS. This makes sure
that the do_populate_sysroot_setscene of recipe B exists for
recipe A in case of a missing TMPDIR. This requires changes made in
runqueue.py by RP.
This commit along with the runqueue fixes effects:
Bug 13419 - recipes that add users to groups cannot rely on other recipes creating those groups (when population from sstate happens)
Bug 13904 - do_prepare_recipe_sysroot: postinst-useradd-* does not run in order of dependency and sometimes fails
Bug 13279 - Make sure users/groups exist for package_write_* tasks
Bug 15084 - For some reason using of same user in two recipes does not work properly
I've included the start of self-testing for useradd by adding tests for
13419 (which ends up testing 13904, 13279, 15084 by virtue of them all
having the same root cause)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids the following error when using --no-extract, introduced in
commit 900129cbdf (devtool: add support for git submodules):
Traceback (most recent call last):
File ".../scripts/devtool", line 349, in <module>
ret = main()
File ".../scripts/devtool", line 336, in main
ret = args.func(args, config, basepath, workspace)
File ".../scripts/lib/devtool/standard.py", line 995, in modify
for commit in commits[name]:
KeyError: '.'
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool: upgrade: Update all existing checksums for the SRC_URI
In addition to updating the sha256sum and removing the md5sum, update
all other existing checksums. If the only existing checksum is md5sum,
then replace it with the default expected checksums (currently only
sha256sum).
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
recipetool: create: Only include the expected SRC_URI checksums
Rather than including all SRC_URI checksums, include the ones that are
expected. These are the same as are output if no checksums are included
when building the recipe.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
recipetool: create: Improve identification of licenses
Rather than having a static list of crunched MD5 checksums for some of
the most common licenses, calculate it for all common licenses. This
should improve the identification of license text variantions.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
oeqa/selftest/devtool: Correct git clone of local repository
If the build environment is setup using `repo`, then poky/.git/object
is a symbolic link rather than a directory. To clone such repositories,
the source path must be prefixed with "file://". This avoids the
following error:
fatal: failed to start iterator over '.../poky/.git/objects': Not a directory
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jose Quaresma [Wed, 6 Dec 2023 15:59:30 +0000 (15:59 +0000)]
go: update 1.20.11 -> 1.20.12
Upgrade to latest 1.20.x release [1]:
$ git log --oneline go1.20.11..go1.20.12 97c8ff8d53 (tag: go1.20.12, origin/release-branch.go1.20) [release-branch.go1.20] go1.20.12 6446af942e [release-branch.go1.20] net/http: limit chunked data overhead 77397ffcb2 [release-branch.go1.20] crypto/rand,runtime: revert "switch RtlGenRandom for ProcessPrng" d77307f855 [release-branch.go1.20] cmd/compile: fix findIndVar so it does not match disjointed loop headers 1bd76576fe [release-branch.go1.20] crypto/rand,runtime: switch RtlGenRandom for ProcessPrng 1b59b017db [release-branch.go1.20] path/filepath: consider \\?\c: as a volume on Windows 46bc33819a [release-branch.go1.20] cmd/go/internal/vcs: error out if the requested repo does not support a secure protocol e1dc209be8 [release-branch.go1.20] cmd/go/internal/modfetch/codehost: set core.longpaths in Git repos on Windows
zstd is dual-licensed under BSD _OR_ GPLv2. License wording in the
README for v1.5.5 is misleading, but license headers in the code clearly
state that there is a choice between the two licenses.
projects / thirdparty / openembedded / openembedded-core-contrib.git / log
