Changelog:
added optional encoding parameter for parsing files
fixed handling of the FilesAnalyzed field in Tag-Value format
fixed the validation of the DownloadLocation field
fixed the error handling while parsing license expressions
fixed output of timezone-sensitive datetimes
added code architecture documentation
Signed-off-by: Marta Rybczynska <marta.rybczynska@syslinbit.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Changelog for 0.16.4 [1]:
In beartype.claw type-check PEP 526-compliant annotated variable
assignments in methods
Fix an inheritance regression introduced in 0.16.3
Changelog for 0.16.3 [2]:
Add hot reloading
Support root superclass validators
Forward reference issubclass() proxying
Readable forward reference exceptions
Class redecoration eliding
Documentation update
William Lyu [Wed, 1 Nov 2023 13:37:41 +0000 (09:37 -0400)]
openssl: improve handshake test error reporting
Fixes [YOCTO #15225]
Yocto Bug #15255 is not reproducible. To obtain more useful information
for debugging, the OpenSSL test code is improved so that more detailed
state information in the handshake loop is printed when an error occurs.
Signed-off-by: William Lyu <William.Lyu@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Randy MacLeod [Wed, 1 Nov 2023 00:54:23 +0000 (17:54 -0700)]
strace: upgrade 6.5 -> 6.6
Update the COPYING checksum; only the copyright date changed.
Noteworthy changes in strace 6.6 (2023-10-31)
=============================================
* Improvements
* Implemented --kill-on-exit option that instructs the tracer to set
PTRACE_O_EXITKILL option to all tracee processes and not to detach them
on cleanup so they will not be left running after the tracer exit.
* Implemented automatic activation of --kill-on-exit option when
--seccomp-bpf is enabled and -p/--attach option is not used.
* Implemented decoding of map_shadow_stack syscall.
* Implemented decoding of FSCONFIG_CMD_CREATE_EXCL fsconfig command.
* Implemented decoding of IFLA_BRPORT_BACKUP_NHID netlink attribute.
* Implemented decoding of SECCOMP_IOCTL_NOTIF_SET_FLAGS ioctl.
* Implemented decoding of UFFDIO_CONTINUE, UFFDIO_POISON, and
UFFDIO_WRITEPROTECT ioctls.
* Updated lists of ARCH_*, BPF_*, DEVCONF_*, IORING_*, KEXEC_*, MAP_*, NT_*,
PTRACE_*, QFMT_*, SEGV_*, UFFD_*, V4L2_*, and XDP_* constants.
* Updated lists of ioctl commands from Linux 6.6.
ptest-runner results on qemux86-64/kvm with qemuparms="-m 1024 -smp 4":
============================================================================
Testsuite summary for strace 6.6
============================================================================
============================================================================
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Niko Mauno [Tue, 31 Oct 2023 14:13:26 +0000 (14:13 +0000)]
ccache.conf: Remove obsolete configuration option
Since ccache version 4.0, according to
https://github.com/ccache/ccache/blob/master/doc/NEWS.adoc#ccache-40
* An appropriate cache directory level structure is now chosen automatically. The cache_dir_levels (CCACHE_NLEVELS) configuration option has therefore been removed.
Therefore remove the option which has not been supported by ccache
recipe version since Yocto Hardknott.
Signed-off-by: Niko Mauno <niko.mauno@vaisala.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Niko Mauno [Mon, 30 Oct 2023 13:12:53 +0000 (13:12 +0000)]
image_types.bbclass: Use xz default compression preset level
Commit ef0654f1453ff0afe98d7e921626b2a96cf2f6f6
("Set XZ_COMPRESSION_LEVEL to -9") changed the xz compression preset
level from previous value of -3 to -9. The commit message explains that
the change was made in order to be consistent with other compressors
that also use their best compression. However looking at xz man page,
under the compression preset level selection chapter there is mentioned
that
The differences between the presets are more significant than with gzip(1) and
bzip2(1). The selected compression settings determine the memory requirements of
the decompressor, thus using a too high preset level might make it painful to
decompress the file on an old system with little RAM. Specifically, it's not a
good idea to blindly use -9 for everything like it often is with gzip(1) and
bzip2(1).
which is then followed by a table, which mentions that the decompressor
memory requirement for preset -9 is 65 MiB, whereas for xz default
preset -6 it is just 9 MiB. Given that the use case where a device
running a Yocto generated Linux OS decompresses an ext4 root filesystem
image to non-volatile memory as part of firmware upgrade process is not
far-fetched, and considering that a range of these devices can run low
on available RAM when there are other applications running at the same
time, the lower decompressor memory requirement of the default preset
level makes sense in order to prevent an OOM situation from occurring.
This change was tested on a 32 CPU core build host with 128 GB RAM by
issuing
$ bitbake -c cleansstate core-image-minimal core-image-sato
$ time bitbake core-image-minimal
$ time bitbake core-image-sato
With MACHINE="qemux86-64" and IMAGE_FSTYPES="ext4 ext4.xz" using
XZ_COMPRESSION_LEVEL values "-6" and "-9". In both cases the resulting
'ext4' image size remained same, 38141952 bytes for core-image-minimal,
and 565043200 bytes for core-image-sato.
The observation was that with this change there is a small increase in
the resulting 'ext4.xz' file size, and a build speed improvement that
was significant for larger rootfs image.
core-image XZ real time time delta ext4.xz size size delta
-----------------------------------------------------------------------
minimal -9 0m44.992s 15932508
minimal -6 0m42.445s -5.66% 16243484 +1.95%
sato -9 2m40.828s 85080416
sato -6 1m38.891s -38.51% 87447456 +2.78%
Regarding decompression speed, issuing following command in qemux86-64
target OS
$ time xz -dkc --memlimit=MEMLIMIT core-image-sato-qemux86-64.rootfs.ext4.xz > /dev/null
using the lowest accepted value for MEMLIMIT for each case (providing a
lower value caused xz to exit with 'Memory usage limit reached' error)
showed that decompression time saw a minuscule improvement with the -6
compression preset level:
XZ MEMLIMIT real time
-------------------------
-9 65M 0m43.83s
-6 9M 0m43.28s
(In the above tables, XZ refers to XZ_COMPRESSION_LEVEL value used when
images were generated with Yocto).
Signed-off-by: Niko Mauno <niko.mauno@vaisala.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Richard Purdie [Sun, 29 Oct 2023 10:21:22 +0000 (10:21 +0000)]
oeqa/selftest: Drop machines support
The machines option to oe-selftest isn't used in our CI and is never likely to
be, we focus and execute testing explictly. The YOCTO #15247 is about how
this code doesn't interact well with build directory cleanup and at this point
I think we should just remove the option/code.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
valgrind: split helper scripts to separate packages, update dependencies
The cachegrind scripts have been rewritten in python3, so the RDEPENDS
on perl is no longer sufficient. This is unfortunately not caught by
QA checks since the scripts use
#! /usr/bin/env python3
as shebang line.
Since the valgrind binary by itself can be quite useful on target,
allow including that in the rootfs without also having to pull in the
whole perl and python interpreters and stdlibs.
Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Alexis Lothoré [Fri, 3 Nov 2023 20:50:18 +0000 (13:50 -0700)]
scripts/resulttool: group all regressions in regression report
Commit c304fcbe0588b1078373558c2ddf36064bcdf214 introduced a grouping when
listing regressions. This grouping has been added only for ptests. It has
been observed that any other kind of tests could benefit from it. For
example, current regression reports can show the following:
1 regression(s) for oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash
oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_help
oescripts.OEPybootchartguyTests.test_pybootchartguy_help: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output: PASSED -> SKIPPED
[...]
This output is not so useful in its current state and should be grouped per
test type too.
Enable grouping for all kind of tests, to make it llok like the following
in reports:
5 regression(s) for oescripts
oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: PASSED -> SKIPPED
oescripts.OEPybootchartguyTests.test_pybootchartguy_help: PASSED -> SKIPPED
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output: PASSED -> SKIPPED
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output: PASSED -> SKIPPED
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output: PASSED -> SKIPPED
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
baremetal-helloworld: Pull in fix for race condition on x86-64
It was previously discovered that there was a race condition during the Makefile
execution between the assemble and compile targets, the previous fix attempted
to serialize the build targets, but the fix was missing for x86-64.
Pull in latest commit from upstream to fix this issue on x86-64.
[YOCTO #15146]
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandro@enedino.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Florian Wickert [Fri, 3 Nov 2023 14:50:37 +0000 (15:50 +0100)]
systemd: fix libnss-mymachines packaging
By removing libnss_mymachines.so.2 from FILES:${PN}-container it
correctly gets packaged into libnss-mymachines, just like the other
libnss-* components, due to PACKAGES_DYNAMIC.
To make up for the missing lib in the container package, I added
libnss-mymachines to RDEPENDS:${PN}-container. As I understand it,
the lib should be installed even without this if anything from the
container package links against it. But since I don't know if there
was a special reason to explicitly add it to the container files list,
like something using dlopen to load the lib, it seemd to be the
safest option.
Signed-off-by: Florian Wickert <fw@ferncast.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
scripts/esdk-tools: use a dedicated, static directory for esdk tools
This allows easier replication of esdk environment (which provides
a curated, limited set of tools that for example does not include bitbake)
in a standard yocto build. Switchover between various sets can be achieved
via PATH manipulation.
Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
meta/lib/oe/copy_buildsystem.py: do not derefence symlinks
This was added (I think) for the purpose of supporting layers that refer
to items outside of the layer via relative symlinks:
https://git.yoctoproject.org/poky-contrib/commit/?id=d31d1ad4e566e42d0bbcf1f41ac25e33181fb517
I do not think copying the link target into the layer that references it is the
correct solution: rather the original target should be included
into the SDK with the same relative path.
This change is done for the sake of preserving symlinks
that are referencing things inside the layer as they are;
particularly the content of scripts/esdk-tools/.
Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
populate_sdk_ext.bbclass: do not symlink unfsd from sdk image sysroot into eSDK tools path
This was done in 2016 to support Eclipse plugin (long dead),
it's currently broken as image sysroot is not in the SDK
until the image is built in that context, and current tools
all rely on runqemu-export-rootfs which does not rely on PATH
and runs unfsd with full path to recipe-specific sysroots.
Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Fri, 3 Nov 2023 13:28:11 +0000 (13:28 +0000)]
cve-check: don't warn if a patch is remote
We don't make do_cve_check depend on do_unpack because that would be a
waste of time 99% of the time. The compromise here is that we can't
scan remote patches for issues, but this isn't a problem so downgrade
the warning to a note.
Also move the check for CVEs in the filename before the local file check
so that even with remote patches, we still check for CVE references in
the name.
Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Trevor Gamblin [Thu, 2 Nov 2023 17:25:49 +0000 (13:25 -0400)]
patchtest-send-results: fix sender parsing
Not all mbox 'from' fields will contain angle brackets, so the
re.findall invocation used for getting a reply_address may fail. Use a
simpler reference to the field to get the sender's email address.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Trevor Gamblin [Wed, 1 Nov 2023 14:20:55 +0000 (10:20 -0400)]
patchtest: remove test for CVE tag in mbox
After patchtest went live it was determined that testing for a CVE tag
in the mbox commit message is unnecessary, since it will already be in
the shortlog and in any carried patches. Remove the test and the
associated selftest files so that its absence isn't flagged in future
test results.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Trevor Gamblin [Tue, 31 Oct 2023 18:16:13 +0000 (14:16 -0400)]
patchtest: make pylint tests compatible with 3.x
pylint 3.x has removed epylint, which is now a separate module. To avoid
adding another recipe or using outdated modules, modify the
test_python_pylint tests so that they use the standard pylint API.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Trevor Gamblin [Wed, 1 Nov 2023 18:08:21 +0000 (14:08 -0400)]
patchtest-send-results: add In-Reply-To
Rework the script for sending results to use send_raw_email and specify
the 'In-Reply-To' field so that patchtest replies to the emails, rather
than sending them standalone to the submitter and mailing list.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
selftest/sstatetests: add a test for CDN sstate cache
Specifically, the test checks that everything needed for building
standard oe-core images for x86_64 and arm64 is available from
the cache (with minor exceptions). Going forward, a complete
world check could be enabled and additional configurations,
but that requires improvements to performance of hash equivalence
server in particular.
RP: I've disabled the tests by default so we can merge them. We will
make them live once we get to the bottom of the failures.
Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Chen Qi [Mon, 30 Oct 2023 06:31:33 +0000 (14:31 +0800)]
kernel.bbclass: add preceding space in appendVar setting
The appendVar setting should have a preceding space, otherwise, when
KERNEL_MODULE_SPLIT is set to "0", we'll sometimes get dependency error
due to lacking of space.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Trevor Gamblin [Mon, 30 Oct 2023 15:44:41 +0000 (11:44 -0400)]
patchtest-send-results: send results to submitter
Modify patchtest-send-results so that it extracts the submitter's email
address and responds to them with the patch testresults. Also make a
minor adjustment to the suggestions provided with each email and include
a link to the Patchtest wiki page for additional clarification on
specific failures.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
git show-ref looks at the _remote_ ref called HEAD, which is fine when it
matches the local HEAD but problematic when you're iterating a series of
commits.
Use rev-parse to resolve the local name to a proper hash.
Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The hooks are pulled from the impure environment and are often broken in
our environments. There is no reason to add extra metadata or verify the
commit message as its arbitrary to turn the tarball into a git repo.
Signed-off-by: William A. Kennington III <wak@google.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Stéphane Veyret [Mon, 28 Aug 2023 17:52:56 +0000 (19:52 +0200)]
volatile-binds: Calculate the name of the /var/lib service
By default, /var/lib is bind mounted on /var/volatile/lib. If this is
the case, the recipe adds conditions on systemd-random-seed in the
service file mounting it. But as the VOLATILE_BINDS may be modified,
/var/lib may be mounted elsewhere, for example in /persistent/var/lib.
In this case, the conditions are not set because the service file name
does not match expected one.
This patch automatically records the name of the service mounting
/var/lib, if any, in order to set the condition in the appropriate file.
Signed-off-by: Stéphane Veyret <sveyret@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Stéphane Veyret [Sun, 27 Aug 2023 16:41:23 +0000 (18:41 +0200)]
volatile-binds: Allow creation of subdirectories
The mount-copybind script will create the parent directory of the bind
mount if it does not exist. But actually, if this is the case, the
service will not even start because of the ConditionPathIsReadWrite.
This patch adds a "or" condition to allow the service to start also if
the parent directory of the bind mount does not exist.
Signed-off-by: Stéphane Veyret <sveyret@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The commit not only removes the dependencies on the cross compiler
but also does not depend on e.g. virtual/${TARGET_PREFIX}compilerlibs
and virtual/libc which in turn makes the file-rdeps qa check fail
if installing binaries linked against e.g. libc or libstdc++.
Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Trevor Gamblin [Sun, 29 Oct 2023 14:54:26 +0000 (10:54 -0400)]
patchtest: shorten test result outputs
Some test result lines in TestMbox and TestPatch are still too long to
avoid being flagged by the mailer script. Clean them up by removing
redundant information, so that they are all under the length limit of
220 characters.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Trevor Gamblin [Fri, 27 Oct 2023 15:43:38 +0000 (11:43 -0400)]
patchtest: reduce checksum test output length
The test_lic_files_chksum_modified_not_mentioned test in TestMetadata is
outputting very long lines that fail the maximum length check when
sending email results, preventing the actual errors from being
displayed. Reduce the length of the failure message by rewording and
removing redundant information.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Fri, 27 Oct 2023 15:29:40 +0000 (16:29 +0100)]
scripts/contrib/patchreview: add commit and recipe count fields to JSON
The autobuilder scripts post-process the generated JSON to inject recipe
and commit counts into the data. We can do this easily in patchreview
instead.
Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Fri, 27 Oct 2023 15:29:39 +0000 (16:29 +0100)]
scripts/patchreview: rework patch detection
A previous patch[1] added the ability to allow the search pattern for
patches to be changed, so that patchreview can be used across the entire
meta-oe repository by changing the patterns.
However, this means the caller needs to write long patterns when calling
patchreview.
Instead, we can see if the specified directory contains a layer by
checking if conf/layer.conf exists. If it does, then search for patches
inside this directory. If it doesn't, assume that the specified
directory is a repository that contains sublayers (such as
meta-openembedded) and look through each of the directories that match
the pattern meta-*.
This means patchreview can both scan either a single layer (eg
.../poky/meta) or a repository of sublayers (eg .../meta-openembedded).
Richard Purdie [Thu, 26 Oct 2023 10:36:10 +0000 (11:36 +0100)]
base: Ensure recipes using mercurial-native have certificates
If you try and fetch using mercurial-native, you see certificate errors since
it is configured to find ones in the sysroot, not the system. Add the missing
dependency so that mercurial recipes using the native tool work.
Found trying to make mirroring for old meta-oe stable branches work.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
lib/oe/sstatesig.py: dump locked.sigs.inc only when explicitly asked via -S lockedsigs
This was writing out locked-sigs.inc into cwd with every
'bitbake -S' invocation. When the intent is only to to get task
stamps (-S none), or print the difference between them (-S printdiff),
the file is unnecessary clutter.
A couple of selftests/scripts were however relying on this, so they're
adjusted to explicitly request the file.
eSDK code calls dump_lockedsigs() separately via
oe.copy_buildsystem.generate_locked_sigs() and so isn't affected.
Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
selftest/sstatetests: add tests for 'bitbake -S printdiff'
'bitbake -S printdiff' is a useful diagnostic facility for finding out
why sstate is not being reused, but until now it had no tests that would
ensure it works. This commit adds three basic scenarios:
1. make a change in a really basic, common recipe that is at the very root
of dependency trees (quilt-native), and ensure that change is correctly discovered when
building an image.
2. make a change in gcc-source recipe, which is somewhat special
(operates in work-shared), and ensure that gcc-runtime builds track
that down as well.
3. make a change in base_do_configure() definition from base.bbclass,
which is not recipe-specific, but affects many basic recipes, and ensure that
is correctly reported as well.
The test itself actually runs twice:
- first against a fully populated build directory, where
the printdiff code is guaranteed to find the correct previous
stamp that can be compared with in a predictable manner.
- then in an empty build directory where the printdiff code
goes to look in the sstate cache, and so the existence of the
previous signature can be tested, but not the difference with it
(what the exact difference would be is unpredictable as the
sstate cache is indeed shared between many builds).
Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Xiangyu Chen [Thu, 26 Oct 2023 06:38:48 +0000 (14:38 +0800)]
linux-yocto: make sure the pahole-native available before do_kernel_configme
When using debug-btf.scc in a clean workspace, the CONFIG_MODULE_ALLOW_BTF_MISMATCH cannot
apply to kernel until clean the kernel code(bitbake linux-yocto -c cleanall) and rebuild.
After tracking the code, some options depend on CONFIG_PAHOLE_VERSION, it was generated by
scripts/pahole-version.sh in kernel, but during do_kernel_configme step, the pahole-native
is not available in sysroot-native, so need to wait pahole-native install to sysroot-native
before do_kernel_configme.
Julien Stephan [Wed, 25 Oct 2023 15:46:59 +0000 (17:46 +0200)]
oeqa/selftest/recipetool: add selftest for PEP-517 recipe creation
Add 3 tests to check the creation of PEP-517 project using the 3
backends supported by bitbake:
- setuptools.build_meta
- poetry.core.masonry.api
- flit_core.buildapi
Theses tests requires the tomllib python module, so skip theses tests
if module is not present. tomllib module is part of python starting from 3.11
Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Julien Stephan [Wed, 25 Oct 2023 15:46:58 +0000 (17:46 +0200)]
recipetool/create_buildsys_python: add PEP517 support
Add basic support for PEP517 [1] for the 3 following backends that are
supported by bitbake:
- setuptools.build_meta
- poetry.core.masonry.api
- flit_core.buildapi
If a pyproject.toml file is found, use it to create the recipe,
otherwise fallback to the old setup.py method.
Some projects can declare a minimal pyproject.toml file, and put all
the metadata in setup.py/setup.cfg/requirements.txt .. theses cases
are not handled. If a pyproject.toml file is found, assumes it has all
necessary metadata.
As for the old setup.py method, version numbers for dependencies are not
handled.
Some features may be missing, such as the extra dependencies.
[YOCTO #14737]
[1]: https://peps.python.org/pep-0517/
Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Julien Stephan [Wed, 25 Oct 2023 15:46:57 +0000 (17:46 +0200)]
recipetool/create_buildsys_python: refactor code for futur PEP517 addition
In order to prepare the support for pyproject.toml (PEP517 [1]) enabled
projects, refactor the code and move setup.py specific code into a
specific class in order to allow sharing the PythonRecipeHandler class
License field of setup is not always standardized, so we usually use the
classifier to determine the correct license format to use in the recipe.
A warning note is added above the LICENSE field of the create recipe
in case a license is provided in setup. But when the plugin is called,
"LICENSE =" is not yet present so we can never display this note.
Replace the "LICENSE =" condition with "##LICENSE_PLACEHOLDER##"
to actually be able to display the note message
Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Julien Stephan [Wed, 25 Oct 2023 16:02:44 +0000 (18:02 +0200)]
oeqa/selftest/devtool: abort if a local workspace already exist
if user run devtool selftests with a local workspacelayer
the tests fail with various error such as:
- devtool.DevtoolAddTests.test_devtool_add just hangs
- devtool.DevtoolModifyTests.* fail with the following error:
ERROR: Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it.
Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it.
Check if a workspacelayer exists, warn the user and abort the tests
Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Khem Raj [Tue, 24 Oct 2023 22:07:35 +0000 (15:07 -0700)]
kernel.bbclass: Use strip utility used for kernel build in do_package
os.environ does not pass this down to runstrip() function and in
strip_execs() its using STRIP bitbake variable to find the strip utility
to use. Since there might be a trailing whitespace in KERNEL_STRIP
remove that otherwise python is not able to launch it.
e.g.
FileNotFoundError: [Errno 2] No such file or directory: 'riscv64-yoe-linux-strip '
This is more evident when STRIP and KERNEL_STRIP are different utilities
e.g. when using clang as default toolchain but using gcc+binutils only for
kernel build.
Signed-off-by: Khem Raj <raj.khem@gmail.com> Cc: Bruce Ashfield <bruce.ashfield@gmail.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Trevor Gamblin [Thu, 26 Oct 2023 14:07:46 +0000 (10:07 -0400)]
patchtest: simplify test directory structure
Consolidate the various mbox tests into a new TestMbox class, metadata
tests into TestMetadata, and patch tests into TestPatch. Also update the
selftest filenames to match the changes. The test contents are not
significantly changed (other than to reference the new class names).
While this doesn't improve overall readability, it does result in more
obvious categorization, and more importantly reduces the number of calls
to setup tinfoil in the tests, resulting in a roughly 25% reduction in
runtime.
Major changes between OpenSSL 3.1.3 and OpenSSL 3.1.4 [24 Oct 2023]
* Mitigate incorrect resize handling for symmetric cipher keys and IVs. (CVE-2023-5363)
Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Niko Mauno [Sat, 21 Oct 2023 15:51:14 +0000 (15:51 +0000)]
package_rpm: Allow compression mode override
Commit 4a4d5f78a6962dda5f63e9891825c80a8a87bf66 ("package_rpm: use zstd
instead of xz") changed the rpm package compressor from 'xz' to 'zstd'
which results in decompression failure with BusyBox-provided 'rpm2cpio'
applet and 'rpm' applet when given the '-i' (Install package) option:
rpm2cpio: no gzip/bzip2/xz magic
Introduce a variable which makes it possible to use a different
compression mode, making it possible to override the default value for
example like
Niko Mauno [Sat, 21 Oct 2023 15:51:11 +0000 (15:51 +0000)]
package_rpm: Fix some pycodestyle issues
Fix following subset of observations reported by version 2.10.0 of
pycodestyle utility:
meta/classes-global/package_rpm.bbclass:65:46: E231 missing whitespace after ','
meta/classes-global/package_rpm.bbclass:66:46: E231 missing whitespace after ','
meta/classes-global/package_rpm.bbclass:107:19: E231 missing whitespace after ','
meta/classes-global/package_rpm.bbclass:109:69: E202 whitespace before ')'
meta/classes-global/package_rpm.bbclass:122:103: W291 trailing whitespace
meta/classes-global/package_rpm.bbclass:194:74: W291 trailing whitespace
meta/classes-global/package_rpm.bbclass:448:16: E713 test for membership should be 'not in'
meta/classes-global/package_rpm.bbclass:450:16: E713 test for membership should be 'not in'
meta/classes-global/package_rpm.bbclass:520:1: W293 blank line contains whitespace
meta/classes-global/package_rpm.bbclass:521:15: E231 missing whitespace after ','
meta/classes-global/package_rpm.bbclass:542:12: E713 test for membership should be 'not in'
meta/classes-global/package_rpm.bbclass:544:12: E713 test for membership should be 'not in'
meta/classes-global/package_rpm.bbclass:647:67: W291 trailing whitespace
Signed-off-by: Niko Mauno <niko.mauno@vaisala.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Mon, 23 Oct 2023 17:46:21 +0000 (18:46 +0100)]
pixman: ignore CVE-2023-37769
This issue relates to a floating point exception in stress-test, which
is an unlikely security exploit at the best of times, but the test is
not installed so isn't relevant.
Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Mon, 23 Oct 2023 17:38:22 +0000 (18:38 +0100)]
cve-check: slightly more verbose warning when adding the same package twice
Occasionally the cve-check tool will warn that it is adding the same
package twice. Knowing what this package is might be the first step
towards understanding where this message comes from.
Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Mon, 23 Oct 2023 17:38:21 +0000 (18:38 +0100)]
cve-check: sort the package list in the JSON report
The JSON report generated by the cve-check class is basically a huge
list of packages. This list of packages is, however, unsorted.
To make things easier for people comparing the JSON, or more
specifically for git when archiving the JSON over time in a git
repository, we can sort the list by package name.
Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Mon, 23 Oct 2023 17:38:19 +0000 (18:38 +0100)]
libxml2: ignore disputed CVE-2023-45322
This CVE is a use-after-free which theoretically can be an exploit
vector, but this UAF only occurs when malloc() fails. As it's
unlikely that the user can orchestrate malloc() failures at just the
place to break on _this_ malloc and not others it is disputed that this
is actually a security issue.
The underlying bug has been fixed, and will be incorporated into the
next release.
Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Trevor Gamblin [Wed, 25 Oct 2023 16:41:33 +0000 (12:41 -0400)]
patchtest/selftest: add XSKIP, update test files
Since we are skipping the merge test, two of the selftests now report
SKIP instead of XPASS/XFAIL as expected. Adjust the two files to have
the right endings for XSKIP, and add the category so that it can be used
for more extensive testing in the future.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Trevor Gamblin [Wed, 25 Oct 2023 14:47:39 +0000 (10:47 -0400)]
patchtest-send-results: check max line length, simplify responses
Check that the maximum line length of the testresult file is less than
220 characters, to help guard against malicious changes being sent in
email responses. If any line exceeds this length, replace the normal
testresults used in the response with a line stating that tests failed,
but the results could not be processed. Also clean up the respone
substrings slightly to go along with the change.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rasmus Villemoes [Mon, 23 Oct 2023 07:03:30 +0000 (09:03 +0200)]
perf: lift TARGET_CC_ARCH modification out of security_flags.inc
Building perf without security_flags.inc being included in one's
distro results in the buildpaths warning
WARNING: perf-1.0-r9 do_package_qa: QA Issue: File /usr/bin/trace in
package perf contains reference to TMPDIR
because the ${DEBUG_PREFIX_MAP} does not get used. Most recipes get
that from CFLAGS, but the perf recipe explicitly unsets that.
Now ${SELECTED_OPTIMIZATION} of course contains more than just
${DEBUG_FLAGS}/${DEBUG_PREFIX_MAP}. For most TUs, perf's build system
adds its own optimization flags (-O6 for odd reasons), so for those
including the -O2 or -Og doesn't change anything. But looking at the
.o.cmd files show that there are some TUs which currently get built
without any -O flag. So for those adding the distro's
SELECTED_OPTIMIZATION seem to be the right thing to do.
Trevor Gamblin [Tue, 24 Oct 2023 18:40:08 +0000 (14:40 -0400)]
patchtest: disable merge test
Disable the merge-on-head test until patchtest properly handles merging
of series subsets and accounts for patches that are rapidly merged (i.e.
before patchtest is run).
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Trevor Gamblin [Tue, 24 Oct 2023 13:14:48 +0000 (09:14 -0400)]
patchtest-send-results: improve subject line
Pull the actual email's subject line from the .mbox file and use that in
patchtest's test results response, so that it's clearer which patch it
is replying to.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Alexis Lothoré [Sun, 22 Oct 2023 17:49:38 +0000 (19:49 +0200)]
scripts/yocto_testresults_query: add option to change display limit
Add a "-l"/"--limit" option to allow changing the display limit in
resulttool.
- If no value is passed, resulttool uses its default value.
- If 0 is passed, the display limit is removed and every regression will be
displayed
- If a custom value is passed, this value overrides the vlaue configured in
resulttool
Alexis Lothoré [Sun, 22 Oct 2023 17:49:35 +0000 (19:49 +0200)]
scripts/resulttool: limit the number of changes displayed per test
Most of the changes list generated in regression reports fall in one
of the two following categories:
- there is only a few (<10) changes listed and the info is
valuable/relevant
- the list is huge (> 100 ? 1000 ?) and basically tells us that the whole
tests category suffers the same status (test missing, test failing, test
skipped, etc)
Prevent those huge, worthless lists by limiting the output for each test
group:
- current default limit is arbitrarily set to 50
- limit can still be overriden with a new "-l"/"--limit" flag, either with
custom value, or with 0 to print the whole lists of changes
- limit is applied per test family: currently it distinguishes only types
of ptests, but it can be adapted to other kind of tests
Glenn Strauss [Fri, 20 Oct 2023 13:30:26 +0000 (09:30 -0400)]
lighttpd: modernize lighttpd.conf
- remove obsolete modules
- replace mod_compress directives with mod_deflate
- do not enable debug.log-request-handling by default
(should not be enabled *by default* on any production system,
especially not an embedded system)
- update TLS syntax for modern recommended use
(separate files for certificate+chain, and private key)
- remove incorrect comment about server.event-handler
lighttpd defaults correctly to use kqueue on *BSD systems
- remove ancient config which disables range requests for PDF
(cargo-culted config from ~15 years ago to address problem
in then-popular PDF client)
- use recommend config file include syntax
(more efficient and more deterministic include file ordering)
Glenn Strauss [Fri, 20 Oct 2023 13:30:25 +0000 (09:30 -0400)]
lighttpd: update init script
- add configtest option
- add configtest before starting, restart, reload, force-reload
- change reload,force-reload to use lighttpd graceful restart
via kill signal USR1
Jörg Sommer [Fri, 20 Oct 2023 07:48:41 +0000 (09:48 +0200)]
package_qa_check_rdepends: Allow /usr/bin/sh if usrmerge
If the distro feature usrmerge is set, all files from /bin are moved to
/usr/bin, i.e. /usr/bin/sh is the same as /bin/sh and should be allowed be
ignored, because it's always present.
Signed-off-by: Jörg Sommer <joerg.sommer@navimatix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Logan Gunthorpe [Thu, 19 Oct 2023 21:05:15 +0000 (15:05 -0600)]
runqemu: Add squashfs filesystem types
When using a squashfs filesystem type, runqemu requires specifying the
full path to the image because it doesn't list squashfs types in its
fstypes variable. Add them to provide the same support as other
filesystem types.
Thomas Wolber [Thu, 19 Oct 2023 14:57:17 +0000 (16:57 +0200)]
kea: drop unused directory
the usage of /var/kea was dropped in the 1.6 release (see
https://gitlab.isc.org/isc-projects/kea/-/issues/538 ).
Creating the directory fails on systems with read-only rootfs.
Eero Aaltonen [Thu, 19 Oct 2023 11:11:52 +0000 (14:11 +0300)]
systemd: add option to use stub-resolv.conf
Add option to use the stub-resolv.conf file, which is the systemd
upstream's recommended default mode
https://www.freedesktop.org/software/systemd/man/systemd-resolved.service.html#/etc/resolv.conf
This enables the resolution of Multicast DNS and Link-Local Multicast
Name Resolution names for programs that do not use Name Service Switch.
Signed-off-by: Eero Aaltonen <eero.aaltonen@vaisala.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Eero Aaltonen [Thu, 19 Oct 2023 11:11:51 +0000 (14:11 +0300)]
base-files, systemd: add nss-resolve plugin
Add nss-resolve plugin to the glibc Name Service Switch (NSS) with
systemd-resolved DISTRO_FEATURE so that systemd-resolved is used in DNS
name resolution.
This enables the resolution of Multicast DNS and Link-Local Multicast
Name Resolution names, depending on the selected options.
Signed-off-by: Eero Aaltonen <eero.aaltonen@vaisala.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Khem Raj [Thu, 19 Oct 2023 03:58:29 +0000 (20:58 -0700)]
llvm: Upgrade to 17.0.3
Brings following fixes
* 888437e1b600 [asan] Ensure __asan_register_elf_globals is called in COMDAT asan.module_ctor (#67745)
* 2e00f4ca4e91 [clang-format][doc] Update the Linux kernel coding style URL
* aeb83c3783a6 [clang-format] Fix a serious bug in git-clang-format (#65723)
* 268faa377aee [LSan] Mark create_thread_leak.cpp as UNSUPPORTED: darwin.
* 491a91e8eea2 [PowerPC] Use zext instead of anyext in custom and combine (#68784)
* 8ce6b65c89ad [PowerPC] Add test for #68783 (NFC)
* 7a23a5d43c67 [clang-format] Fix a bug in RemoveParentheses: ReturnStatement (#67911)
* be4016e52779 [X86] Fix logic for optimizing movmsk(bitcast(shuffle(x))); PR67287
* 496b174053bd [X86] Add tests for incorrectly optimizing out shuffle used in `movmsk`; PR67287
* f50c6382c716 [clang] [MinGW] Explicitly always pass the -fno-use-init-array (#68571)
* d10b731adcc8 [LVI][CVP] Treat undef like a full range (#68190)
* 37b79e779f44 [X86] combineConcatVectorOps - only concatenate single-use subops
* 5a13ce2d6020 Bump version to 17.0.3
* e7b3b94cf500 [clang] Correct behavior of `LLVM_UNREACHABLE_OPTIMIZE=OFF` for `Release` builds (#68284)
* f0a687d821c1 [LLD] [COFF] Fix handling of comdat .drectve sections (#68116)
* 8a8ade49ff49 workflows/release-binaries: Use more cores to avoid the 6 hour timeout (#67874)
* 1090b91a2840 [AArch64] Disable loop alignment for Windows targets (#67894)
* 69c8c96691c7 [Sema] Use underlying type of scoped enum for -Wformat diagnostics (#67378)
* b2417f51dbbd (tag: llvmorg-17.0.2) Fix release/export.sh to export runtimes tarball, too (#67404)
* 23988a1d82d5 [libc++] Fix `std::pair`'s pair-like constructor's incorrect assumption (#66585)
* 33e14ecd6aac [CodeGen] Don't treat thread local globals as large data (#67764)
* 03f797b51df6 [workflow] Fix abi checker in llvm-tests. Same fix as in 99fb0af80d16b0ff886f032441392219e1cac452 (#67957)
* f6cf58eed973 [clang] [MinGW] Tolerate mingw specific linker options during compilation (#67891)
* b338a2830a2c [LLD] [COFF] Restore the current dir as the first entry in the search path (#67857)
* 6a5be8e95b43 [LLD] [COFF] Clarify -print-search-path for the empty string element (#67856)
* 71be0aafe357 [NFC] clang-format lld/COFF/Driver.cpp and lld/Common/Filesystem.cpp
* 0a2d7dae6ef2 [compiler-rt] Reinstate removal of CRT choice flags from CMAKE_*_FLAGS* (#67935)
* 098e653a5bed [MemCpyOpt] Merge alias metadatas when replacing arguments (#67539)
* 78d201ebc3e2 [MemCpyOpt] Add test for #67539 (NFC)
* e718f3240a57 [DependencyScanningFilesystem] Make sure the local/shared cache filename lookups use only absolute paths (#66122)
* 45066b9fbc7b [Sema] Fix fixit cast printing inside macros (#66853)
* 87ec1f460d0e Work around two more instances of __noinline__ conflicts. (#66138)
* 9da5b7a93bca [lldb] Fix building LLDB standlone without framework
* c056d720b534 [lldb][NFCI] Change logic to find clang resource dir in standalone builds
* cb23434f9e63 [XCOFF] Do not generate the special .ref for zero-length sections (#66805)
* 1b55dc9d94c3 Fix buildbot failure caused by D157623
* 28d81a2bfa0a [lld][COFF] Remove incorrect flag from EHcont table
* b7eba056b93c workflows/release-tasks: Setup FileCheck and not for release-lit (#66799)
* 9678f11b057c [StackColoring] Handle fixed object index
* 49e9ee190080 [StackColoring] Handle SEH catch object stack slots conservatively
* 17123a60b87c [X86] Add test for #66984 (NFC)
* 2839aa915066 [SimpleLoopUnswitch] Fix exponential unswitch
* 773f136d6faa [SimpleLoopUnswitch] Fix reversed branch during condition injection
* 4362f3e4cf48 [clang] Include `expected-no-diagnostics` in newly-added test (NFC)
* 5f1fcc43e592 [clang] Bail out when handling union access with virtual inheritance
* 178cf5bc8732 [clang][Diagnostics] Fix wrong line number display (#65238)
* 25a150b830f6 Revert "[InlineCost] Check for conflicting target attributes early"
projects / thirdparty / openembedded / openembedded-core.git / log
