]> git.ipfire.org Git - thirdparty/openembedded/openembedded-core-contrib.git/log
thirdparty/openembedded/openembedded-core-contrib.git
18 months agopackage_rpm: add RPMBUILD_EXTRA_PARAMS variable jansa/rpm-user-defined
Martin Jansa [Thu, 1 Feb 2024 10:45:02 +0000 (11:45 +0100)] 
package_rpm: add RPMBUILD_EXTRA_PARAMS variable

* e.g. for DISTROs which define extra user-defined fields with
  PACKAGE_ADD_METADATA/PACKAGE_ADD_METADATA_RPM
  as undefined fields in packagedata are fatal error for rpmbuild:
  "error: line 9: Unknown tag: Author: Unspecified"
  as shown in:
  http://errors.yoctoproject.org/Errors/Details/751706/
  with "Author" field added with:

  PACKAGE_CLASSES = "package_rpm"
  PACKAGE_ADD_AUTHOR_METADATA = "test-author"
  PACKAGE_ADD_METADATA = "Author: ${PACKAGE_ADD_AUTHOR_METADATA}"

  to fix rpm build you can use:
  RPMBUILD_EXTRA_PARAMS = " --define '_Author Author'"
  keep in mind that this doesn't cause this Author field to be
  added in .rpm, it just avoids the BUILDSPEC failure.

  and for ipk build:
  OPKG_MAKE_INDEX_EXTRA_PARAMS = "-f"

  alternatively you can avoid additional packagedata fields ending
  in the package manager (if you use them only with buildhistory
  or packagedata) with:
  PACKAGE_ADD_METADATA_RPM = ""
  PACKAGE_ADD_METADATA_IPK = ""
  PACKAGE_ADD_METADATA_DEP = ""

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agopackage_manager: ipk: add OPKG_MAKE_INDEX_EXTRA_PARAMS variable
Martin Jansa [Thu, 1 Feb 2024 10:41:22 +0000 (11:41 +0100)] 
package_manager: ipk: add OPKG_MAKE_INDEX_EXTRA_PARAMS variable

* can be used to pass e.g. -f param to preserve user-defined fields
  in the index as added in:
  https://git.yoctoproject.org/opkg-utils/commit/opkg-make-index?id=13f6281d24e17199e0fef6c2984419372ea0f86f

* otherwise it will show a lot of messages like:
  "Lost field Author <value>"
  for every package in the feed

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agoglibc: Upgrade to 2.39
Khem Raj [Sun, 14 Jan 2024 06:08:14 +0000 (22:08 -0800)] 
glibc: Upgrade to 2.39

Upgrade localdef to get glibc 2.39 build fixes
Details of release [1]

[1] https://sourceware.org/glibc/wiki/Release/2.39

Signed-off-by: Khem Raj <raj.khem@gmail.com>
18 months agoio-uring-writev: add simple test for writting file with io_uring
Martin Jansa [Thu, 19 Oct 2023 11:11:11 +0000 (13:11 +0200)] 
io-uring-writev: add simple test for writting file with io_uring

* pseudo doesn't support io_uring yet as shown after nodejs was upgraded
  and nodejs-native >= 20.3.0 with libuv >= 1.45.0 which has:
  https://github.com/libuv/libuv/pull/3952

* files created in do_install with nodejs-native aren't tracked by pseudo
  and will result in host-user-contamination QA issue or
  "KeyError: 'getpwuid(): uid not found" as documented in:
  https://github.com/shr-project/com.webos.app.minimal/commit/bd238047c8ce3cd085041d276613396b863213cf

* this is much simpler test for io_uring without the need to build whole
  nodejs-native, it's based on:
  https://unixism.net/2020/04/io-uring-by-example-part-1-introduction/
  just using writev instead of readv

* if it works fine, the file "test" will be tracked in pseudo database
  since the creation in ${D} like:
  core2-64-oe-linux/io-uring-writev/1.0 $ sqlite3 pseudo/files.db "select * from files"
  1|/OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/io-uring-writev/1.0/image|66305|48357743|0|0|16877|0|0
  2|/OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/io-uring-writev/1.0/image/test|66305|48316709|0|0|33188|0|0

  and it does in this case, because I haven't figured out how to call writev()
  without opening the fd of output file first where the openat() call gets
  intercepted by pseudo

  io-uring-writev/1.0 $ strace -v ./io-uring-writev test2 2>&1 | grep openat
  openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
  openat(AT_FDCWD, "/usr/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
  openat(AT_FDCWD, "test2", O_WRONLY|O_CREAT, 0666) = 4

  while with libuv there was no openat() for the output files in strace

* add test with libuv as well and surprisingly it's still working in current pseudo
  oe-core/tmp-glibc/work/core2-64-oe-linux/io-uring-writev/1.0 $ sqlite3 pseudo/files.db "select * from files"
  1|/OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/io-uring-writev/1.0/image|66305|62072917|0|0|16877|0|0
  2|/OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/io-uring-writev/1.0/image/test|66305|62061857|0|0|33188|0|0
  3|/OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/io-uring-writev/1.0/image/task-copy.h|66305|62061858|0|0|33188|0|0

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agoqemu.*multilib.conf: add qemu with multilib enabled for testing
Martin Jansa [Fri, 1 Sep 2023 06:38:07 +0000 (08:38 +0200)] 
qemu.*multilib.conf: add qemu with multilib enabled for testing

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agotcmode-default.inc: Add LLVM_PREFERRED_VERSION
Martin Jansa [Sat, 1 Apr 2023 16:28:22 +0000 (18:28 +0200)] 
tcmode-default.inc: Add LLVM_PREFERRED_VERSION

* Allow to independently set P_V for llvm* recipes with new LLVM_PREFERRED_VERSION
  and actual llvm version used in other places with existing LLVMVERSION variable

  LLVM_PREFERRED_VERSION is used only to set PREFERRED_VERSION
  of llvm recipes and you might need to change it only if your
  layer provides another set of llvm recipes with different version.

  LLVMVERSION is the actual version of LLVM used during the build
  (and it can be different from llvm recipes, e.g. when LLVM is
  provided by clang recipes based on PREFERRED_PROVIDER setting
  and meta-clang already sets LLVMVERSION in layer.conf).

  PREFERRED_VERSION_llvm doesn't affect version of clang recipes even
  when clang provides llvm, only PREFERRED_VERSION_clang would affect that.

* This is needed for meta-clang which provides different llvm version from clang recipes, see:
  https://github.com/kraj/meta-clang/pull/766
  https://lists.openembedded.org/g/bitbake-devel/message/14521

* as reported in:
  https://lists.openembedded.org/g/bitbake-devel/message/14523
  the other places in kirkstone and langdale which use LLVMVERSION
  (which doesn't have to be the same as LLVM_PREFERRED_VERSION) are:

  meta/classes-recipe/meson.bbclass:llvm-config = 'llvm-config${LLVMVERSION}'
  meta/recipes-graphics/mesa/mesa.inc:MESA_LLVM_RELEASE ?= "${LLVMVERSION}"

  these were removed in mickledore, but there are other places which
  use LLVMVERSION as the actual LLVM version as reported in:
  https://github.com/kraj/meta-clang/pull/766#pullrequestreview-1349170591:

  meta-intel/dynamic-layers/clang-layer/recipes-opencl/igc/intel-graphics-compiler_1.0.12812.24.bb:                  -DIGC_OPTION__LLVM_PREFERRED_VERSION=${LLVMVERSION} \
  meta-intel/dynamic-layers/clang-layer/recipes-opencl/opencl-clang/opencl-clang_14.0.0.bb:                  -DPREFERRED_LLVM_VERSION=${LLVMVERSION} \
  meta-intel/dynamic-layers/clang-layer/recipes-opencl/opencl-clang/opencl-clang_15.0.0.bb:                  -DPREFERRED_LLVM_VERSION=${LLVMVERSION} \
  meta-intel/conf/machine/include/meta-intel.inc:PREFERRED_VERSION_opencl-clang ?= "${@bb.utils.contains('LLVMVERSION', '14.0.3', '14.0.0', '15.0.0', d)}"
  meta-intel/conf/machine/include/meta-intel.inc:PREFERRED_VERSION_opencl-clang-native ?= "${@bb.utils.contains('LLVMVERSION', '14.0.3', '14.0.0', '15.0.0', d)}"
  meta-riscv/recipes-graphics/mesa/mesa-pvr.inc:MESA_LLVM_RELEASE ?= "${LLVMVERSION}"
  meta-browser/meta-chromium/recipes-browser/chromium/chromium-gn.inc:# Check the LLVMVERSION defined in the meta-clang layer. Given Chromium is
  meta-browser/meta-chromium/recipes-browser/chromium/chromium-gn.inc:# developed using new C++ features, the LLVMVERSION has to be >= 12. Otherwise,
  meta-browser/meta-chromium/recipes-browser/chromium/chromium-gn.inc:  llvm_version = d.getVar('LLVMVERSION', False)
  meta-browser/meta-chromium/recipes-browser/chromium/chromium-gn.inc:    bb.fatal("Your LLVMVERSION (%s) is lower than the minimum required "
  meta-browser/meta-chromium/recipes-browser/chromium/chromium-gn.inc:             "LLVMVERSION (%s). If you are using dunfell, make sure you "
  meta-clang/conf/layer.conf:LLVMVERSION = "16.0.0"
  meta-clang/recipes-devtools/spirv-llvm-translator/spirv-llvm-translator_git.bb:        -DBASE_LLVM_VERSION=${LLVMVERSION} \
  meta-clang/dynamic-layers/openembedded-layer/recipes-devtools/bcc/bcc_0.26.0.bb:    -DLLVM_PACKAGE_VERSION=${LLVMVERSION} \
  meta-clang/dynamic-layers/openembedded-layer/recipes-devtools/bpftrace/bpftrace_0.17.0.bb:    pvsplit = d.getVar('LLVMVERSION').split('.')

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
18 months agoreport-error.bbclass: replace angle brackets with &lt; and &gt;
Changqing Li [Mon, 15 Mar 2021 01:03:25 +0000 (18:03 -0700)] 
report-error.bbclass: replace angle brackets with &lt; and &gt;

when we have below content in local.conf or auto.conf:
BUILDHISTORY_COMMIT_AUTHOR ?= "Khem Raj <raj.khem@gmail.com>"
send-error-report will fail with "HTTP Error 500: OK"

error-report-web do rudimentary check on all fields that are
passed to the graphs page to avoid any XSS happening, if contains
'<', the server will return error(Invalid characters in json).
fixed by use escape of <> to replace it.

NOTE: with this change, error-report-web need to add filter 'safe'
for the string wanted to display to avoid further HTML escaping
prior to output. Below is how the content displayed on webpage:
with the filter 'safe':
BUILDHISTORY_COMMIT_AUTHOR ?= "Khem Raj <raj.khem@gmail.com>"
without the filter 'safe':
BUILDHISTORY_COMMIT_AUTHOR ?= "Khem Raj &lt;raj.khem@gmail.com&gt;"

Another patch for error-report-web will send to yocto mail list.

[YOCTO #13252]

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
18 months agoreport-error: Allow to upload reports automatically
Martin Jansa [Fri, 14 Mar 2014 18:12:43 +0000 (19:12 +0100)] 
report-error: Allow to upload reports automatically

* useful when distro wants to collect build statistics from
  all users/developers without any manual interaction from them

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
18 months agoqemux86: Add identical qemux86copy variant for tests
Martin Jansa [Fri, 9 Mar 2012 06:56:57 +0000 (07:56 +0100)] 
qemux86: Add identical qemux86copy variant for tests

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
18 months agoinsane: add patch-status to default ERROR_QA
Richard Purdie [Wed, 18 Jan 2023 13:08:58 +0000 (13:08 +0000)] 
insane: add patch-status to default ERROR_QA

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
18 months agopatchreview: use check_upstream_status() from oe.qa
Martin Jansa [Wed, 15 Feb 2023 15:59:58 +0000 (16:59 +0100)] 
patchreview: use check_upstream_status() from oe.qa

* the idea was to reuse the same function as I've noticed that the
  QA check which was added to insane.bbclass in:
  https://git.openembedded.org/openembedded-core/commit/?id=76a685bfcf927593eac67157762a53259089ea8a
  is in some cases more strcit than scripts/contrib/patchreview.py

  To be honest I wasn't aware of scripts/contrib/patchreview.py
  existence when I've asked about moving check_upstream_status()
  to oe.qa in order to write standalone script just like
  patchreview.py, now I don't feel strongly about sharing this
  functionality (other than adjusting regexes in patchreview.py)

* it finds one "new" issue in oe-core:
  Malformed Upstream-Status 'Malformed Upstream-Status in patch
  meta/recipes-support/libssh2/files/0001-Don-t-let-host-enviroment-to-decide-if-a-test-is-bui.patch
  Please correct according to https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines#Patch_Header_Recommendations:_Upstream-Status :
  Upstream-Status: Inappropriate[oe specific]' (meta/recipes-support/libssh2/files/0001-Don-t-let-host-enviroment-to-decide-if-a-test-is-bui.patch)

* but unlike the QA check patchreview.py will report this:
  -Upstream-Status: Submitted [https://github.com/madler/zlib/pull/599]
  +Upstream-Broken-Status: Submitted [https://github.com/madler/zlib/pull/599]

  as a missing Upstream-Status instead of malformed as reported by QA check:

  ERROR: zlib-native-1.2.13-r0 do_patch: QA Issue: Malformed Upstream-Status in patch
  /OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib/0001-configure-Pass-LDFLAGS-to-link-tests.patch
  Please correct according to https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines#Patch_Header_Recommendations:_Upstream-Status :
  Upstream-Broken-Status: Submitted [https://github.com/madler/zlib/pull/599] [patch-status-core]

* RFC: let me know if you think it's worth re-working this to better
  integrate (e.g. detecting Upstream-Broken-Status and not repeating
  .patch path in patchreview.py output) or if I should just adjust
  regexes in patchreview.py.

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
18 months agopopulate_sdk_ext.bbclass: add SSTATE_MIRRORS for NATIVELSBSTRING when not using uninative
Martin Jansa [Wed, 5 Apr 2023 19:37:46 +0000 (21:37 +0200)] 
populate_sdk_ext.bbclass: add SSTATE_MIRRORS for NATIVELSBSTRING when not using uninative

* this fixes the issue where eSDK preparation fails when uninative isn't enabled:

  Preparing build system...
  WARNING: You are using a local hash equivalence server but have configured an sstate mirror. This will likely mean no sstate will match from the mirror. You may wish to disable the hash equivalence use (BB_HASHSERVE), or use a hash equivalence server alongside the sstate mirror.
  Loading cache: 100% |                                                      | ETA:  --:--:--
  Parsing recipes: 100% |####################################################| Time: 0:00:10
  Initialising tasks: 100% |#################################################| Time: 0:00:02
  Checking sstate mirror object availability: 100% |#########################| Time: 0:00:00
  WARNING: The base-files:do_install sig is computed to be 83b9c9a6ef1145baac5a1e0d08814b9156af239c58fc42df95c25a9cd8a7f201,
    but the sig is locked to 3dc22233059075978e5503691e98e79e7cc60db94259dfcd886bca2291c0add7 in SIGGEN_LOCKEDSIGS_t-qemux86-64
  ERROR: Task glibc-locale.do_deploy_source_date_epoch attempted to execute unexpectedly and should have been setscened
  Task virtual:native:/OE/build/test-sdk-without/layers/poky/meta/recipes-support/libpcre/libpcre2_10.42.bb:do_populate_sysroot,
    unihash 4615cb1da18a0fae9852bbeb9a57bc5858cd3c18489a29dfb7d42bf9e5d26783, taskhash 4615cb1da18a0fae9852bbeb9a57bc5858cd3c18489a29dfb7d42bf9e5d26783
  ...
  Task virtual:native:/OE/build/test-sdk-without/layers/poky/meta/recipes-devtools/rpm/rpm_4.18.0.bb:do_deploy_source_date_epoch,
    unihash 593e6c8bcc68da0b0e1eb562eac65e3805172a92ac5c7fb276a0e8641ebc41d6, taskhash 593e6c8bcc68da0b0e1eb562eac65e3805172a92ac5c7fb276a0e8641ebc41d6
  This is usually due to missing setscene tasks. Those missing in this build were: {
    '/OE/build/test-sdk-without/layers/poky/meta/recipes-bsp/v86d/v86d_0.1.10.bb:do_deploy_source_date_epoch',
    '/OE/build/test-sdk-without/layers/poky/meta/recipes-bsp/v86d/v86d_0.1.10.bb:do_package',
    ...
    'virtual:native:/OE/build/test-sdk-without/layers/poky/meta/recipes-support/re2c/re2c_3.0.bb:do_deploy_source_date_epoch',
    'virtual:native:/OE/build/test-sdk-without/layers/poky/meta/recipes-support/re2c/re2c_3.0.bb:do_populate_sysroot'}
  ERROR: Task (/OE/build/test-sdk-without/layers/poky/meta/recipes-core/glibc/glibc-locale_2.37.bb:do_deploy_source_date_epoch) failed with exit code 'setscene ignore_tasks'
  ERROR: SDK preparation failed: error log written to /OE/build/test-sdk-without/preparing_build_system.log

* the locked sstate-cache is created with:
  fixedlsbstring = "universal%s" % oe.utils.host_gcc_version(d)
  but then inside eSDK, we're using regular NATIVELSBSTRING

  in this SDK I can see:

  test-sdk-without $ find sstate-cache/universal/ -name \*zst | wc -l
  150

  while for my host LSB gentoo-2.13 it's empty (just many empty directories):

  test-sdk-without $ find sstate-cache/gentoo-2.13/ -name \*zst | wc -l
  0
  test-sdk-without $ find sstate-cache/gentoo-2.13/ | wc -l
  263

* smaller test case with much shorter log is e.g. texinfo-dummy-native or m4-native:

  test-sdk-without $ BB_SETSCENE_ENFORCE=1  bitbake texinfo-dummy-native
  WARNING: You are using a local hash equivalence server but have configured an sstate mirror. This will likely mean no sstate will match from the mirror. You may wish to disable the hash equivalence use (BB_HASHSERVE), or use a hash equivalence server alongside the sstate mirror.
  Loading cache...done.
  Loaded 1797 entries from dependency cache.
  NOTE: Resolving any missing task queue dependencies
  Initialising tasks...done.
  Sstate summary: Wanted 4 Local 0 Mirrors 0 Missed 4 Current 2 (0% match, 33% complete)
  NOTE: Executing Tasks
  ERROR: Task quilt-native.do_fetch attempted to execute unexpectedly
  Task /OE/build/test-sdk-without/layers/poky/meta/recipes-extended/texinfo-dummy-native/texinfo-dummy-native.bb:do_deploy_source_date_epoch, unihash b355f17d90c8800c246ec1c597e4726ed57d98e0c67d6f18d1b6d3c821f08e7e, taskhash b355f17d90c8800c246ec1c597e4726ed57d98e0c67d6f18d1b6d3c821f08e7e
  Task /OE/build/test-sdk-without/layers/poky/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_deploy_source_date_epoch, unihash 116f8d4762cd111492fffed036d13d4bcca35e9bafc743e5fce5432ff64b2201, taskhash 116f8d4762cd111492fffed036d13d4bcca35e9bafc743e5fce5432ff64b2201
  Task /OE/build/test-sdk-without/layers/poky/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_populate_sysroot, unihash f25773f9c2ae02d1de6c037b6f4161c79b3ba37ce5add86cf087158ba26c52d9, taskhash f25773f9c2ae02d1de6c037b6f4161c79b3ba37ce5add86cf087158ba26c52d9
  Task /OE/build/test-sdk-without/layers/poky/meta/recipes-extended/texinfo-dummy-native/texinfo-dummy-native.bb:do_populate_sysroot, unihash 1c584073b7e999100258ae34289091b5216d3f921d2210d1f8053fc4c8c516f5, taskhash 1c584073b7e999100258ae34289091b5216d3f921d2210d1f8053fc4c8c516f5
  This is usually due to missing setscene tasks. Those missing in this build were: {'/OE/build/test-sdk-without/layers/poky/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_deploy_source_date_epoch',
   '/OE/build/test-sdk-without/layers/poky/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_populate_sysroot',
   '/OE/build/test-sdk-without/layers/poky/meta/recipes-extended/texinfo-dummy-native/texinfo-dummy-native.bb:do_deploy_source_date_epoch',
   '/OE/build/test-sdk-without/layers/poky/meta/recipes-extended/texinfo-dummy-native/texinfo-dummy-native.bb:do_populate_sysroot'}
  ERROR: Task (/OE/build/test-sdk-without/layers/poky/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_fetch) failed with exit code 'setscene ignore_tasks'
  NOTE: Tasks Summary: Attempted 1 tasks of which 0 didn't need to be rerun and 1 failed.

  Summary: 1 task failed:
    /OE/build/test-sdk-without/layers/poky/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_fetch
  Summary: There was 1 WARNING message.
  Summary: There was 1 ERROR message, returning a non-zero exit code.

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
18 months agoselftest: incompatible_lic.py: run bitbake -c testimage separately for each image
Martin Jansa [Mon, 13 Mar 2023 15:20:06 +0000 (16:20 +0100)] 
selftest: incompatible_lic.py: run bitbake -c testimage separately for each image

* with 1 preconfigured tap device you cannot run testimage on multiple images
  at the same time

  in my case have /tmp/qemu-tap-locks/tap0.skip (tap0 is used by openvpn on builder)
  and /etc/runqemu-nosudo as my build user doesn't have sudo powers

  but then when 2 do_testimage tasks are executed at the same time it fails like this:

    NOTE: recipe core-image-weston-1.0-r0: task do_testimage: Started
    NOTE: recipe core-image-full-cmdline-1.0-r0: task do_testimage: Started
    QMP Available for connection at /OE/build/poky/build-st/tmp/.c2d3k3bh
    QMP Available for connection at /OE/build/poky/build-st/tmp/._4agpi1v
    ERROR: core-image-full-cmdline-1.0-r0 do_testimage: runqemu exited with code 1
    ERROR: core-image-full-cmdline-1.0-r0 do_testimage: Output from runqemu:
    runqemu - INFO - Continuing with the following parameters:
    KERNEL: [/OE/build/poky/build-st/tmp/deploy/images/qemux86-64/bzImage]
    MACHINE: [qemux86-64]
    FSTYPE: [ext4]
    ROOTFS: [/OE/build/poky/build-st/tmp/deploy/images/qemux86-64/core-image-full-cmdline-qemux86-64.ext4]
    CONFFILE: [/OE/build/poky/build-st/tmp/deploy/images/qemux86-64/core-image-full-cmdline-qemux86-64.qemuboot.conf]

    runqemu - INFO - Found /tmp/qemu-tap-locks/tap0.skip, skipping tap0
    runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap1.lock failed: [Errno 11] Resource temporarily unavailable
    runqemu - ERROR - Error: There are no available tap devices to use for networking,
    runqemu - ERROR - and I see /etc/runqemu-nosudo exists, so I am not going to try creating
    runqemu - ERROR - a new one with sudo.
    runqemu - INFO - Cleaning up
    runqemu - INFO - Host uptime: 67474.59

    tput: No value for $TERM and no -T specified

    WARNING: core-image-full-cmdline-1.0-r0 do_testimage: Qemu ended unexpectedly, dump data from host is in /OE/build/poky/build-st/tmp/log/runtime-hostdump/202303131614_qemu
    WARNING: core-image-full-cmdline-1.0-r0 do_testimage: runqemu during is_alive() test
    ERROR: core-image-full-cmdline-1.0-r0 do_testimage: Error executing a python function in exec_func_python() autogenerated:

  which is very reasonable error message, but 600 lines later it also shows:

    RESULTS - xorg.XorgTest.test_xorg_running: SKIPPED (0.00s)
    SUMMARY:
    core-image-weston () - Ran 71 tests in 34.073s
    core-image-weston - OK - All required tests passed (successes=14, skipped=57, failures=0, errors=0)
    NOTE: recipe core-image-weston-1.0-r0: task do_testimage: Succeeded
    NOTE: Tasks Summary: Attempted 1141 tasks of which 1139 didn't need to be rerun and 1 failed.

    Summary: 1 task failed:
      /OE/build/poky/meta/recipes-extended/images/core-image-full-cmdline.bb:do_testimag

  and I didn't notice that there are actually 2 different images core-image-weston
  and core-image-full-cmdline and spent most time trying to figure out how it was
  able to run all those ptest when qemu failed to start at the beginning.

  It's possible to create 2nd tap with runqemu-ifup before running this test.
  Then it works, maybe some check like in DevtoolTests to error out early if
  there isn't enough tap devices? But the implementation in:
  https://git.openembedded.org/openembedded-core/commit/?id=2258345e19efff7717fe19a5026ec55f1b6f90b6
  has its own limitations (e.g. not checking /tmp/qemu-tap-locks/tap0.skip)
  and would probably need to be re-implemented as some generic decorator
  as other tests need this as well.

  It's a bit slower when serialized:
  INFO - RESULTS - incompatible_lic.NoGPL3InImagesTests.test_core_image_full_cmdline_weston: PASSED (145.96s)
  without this change:
  INFO - RESULTS - incompatible_lic.NoGPL3InImagesTests.test_core_image_full_cmdline_weston: PASSED (104.72s)

  I don't mind if this is rejected, sending it mostly to document this behavior.

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
18 months agou-boot.inc: don't replace the binary with symlink
Martin Jansa [Tue, 21 Nov 2023 20:30:27 +0000 (21:30 +0100)] 
u-boot.inc: don't replace the binary with symlink

* when UBOOT_ARTIFACT_NAME and UBOOT_ARTIFACT_LINK_NAME are empty
  the UBOOT_BINARYNAME and UBOOT_IMAGE might be indentical and the
  binary gets overwritten by the symlink to itself (similarly for SPL_*)

$ ls -lah /OE/build/poky/build/tmp/work/qemuarm-poky-linux-gnueabi/u-boot/2023.10/package/boot
total 8.0K
drwxr-xr-x 2 martin martin 4.0K Nov 21 21:23 .
drwxr-xr-x 4 martin martin 4.0K Nov 21 21:23 ..
lrwxrwxrwx 1 martin martin    3 Nov 21 21:23 MLO -> MLO
lrwxrwxrwx 1 martin martin   10 Nov 21 21:23 u-boot.bin -> u-boot.bin

* which causes:
ERROR: u-boot-1_2023.10-r0 do_package: Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:do_package(d)
     0003:
File: '/OE/build/poky/meta/classes-global/package.bbclass', lineno: 536, function: do_package
     0532:    bb.build.exec_func("package_prepare_pkgdata", d)
     0533:    bb.build.exec_func("perform_packagecopy", d)
     0534:    for f in (d.getVar('PACKAGE_PREPROCESS_FUNCS') or '').split():
     0535:        bb.build.exec_func(f, d)
 *** 0536:    oe.package.process_split_and_strip_files(d)
     0537:    oe.package.fixup_perms(d)
     0538:
     0539:    ###########################################################################
     0540:    # Split up PKGD into PKGDEST
File: '/OE/build/poky/meta/lib/oe/package.py', lineno: 1073, function: process_split_and_strip_files
     1069:                    staticlibs.append(file)
     1070:                    continue
     1071:
     1072:                try:
 *** 1073:                    ltarget = cpath.realpath(file, dvar, False)
     1074:                    s = cpath.lstat(ltarget)
     1075:                except OSError as e:
     1076:                    (err, strerror) = e.args
     1077:                    if err != errno.ENOENT:
File: '/OE/build/poky/meta/lib/oe/cachedpath.py', lineno: 231, function: realpath
     0227:            if e.errno == errno.ELOOP:
     0228:                # make ELOOP more readable; without catching it, there will
     0229:                # be printed a backtrace with 100s of OSError exceptions
     0230:                # else
 *** 0231:                raise OSError(errno.ELOOP,
     0232:                              "too much recursions while resolving '%s'; loop in '%s'" %
     0233:                              (file, e.strerror))
     0234:
     0235:            raise
Exception: OSError: [Errno 40] too much recursions while resolving '/OE/build/poky/build/tmp/work/qemuarm-poky-linux-gnueabi/u-boot/2023.10/package/boot/MLO'; loop in '/OE/build/poky/build/tmp/work/qemuarm-poky-linux-gnueabi/u-boot/2023.10/package/boot/MLO'

ERROR: Logfile of failure stored in: /OE/build/poky/build/tmp/work/qemuarm-poky-linux-gnueabi/u-boot/2023.10/temp/log.do_package.3990391

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agopopulate_sdk_ext.bbclass: add *:do_shared_workdir to BB_SETSCENE_ENFORCE_IGNORE_TASKS
Martin Jansa [Mon, 20 Nov 2023 19:10:23 +0000 (20:10 +0100)] 
populate_sdk_ext.bbclass: add *:do_shared_workdir to BB_SETSCENE_ENFORCE_IGNORE_TASKS

* otherwise populate_sdk_ext task will fail as shown e.g. with:
  bitbake core-image-minimal -c populate_sdk_ext
  esdk.oeSDKExtSelfTest.test_image_generation_binary_feeds
  esdk.oeSDKExtSelfTest.test_install_libraries_headers:

ERROR: Task linux-yocto.do_deploy_links attempted to execute unexpectedly
Task tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_qa, unihash 9d177d4c6ca34e68e19b1bc23deec58c3eabe5f9d5808f90402161163a73f22f, taskhash 9d177d4c6ca34e68e19b1bc23deec58c3eabe5f9d5808f90402161163a73f22f
Task tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_complete, unihash 0aff4dcbdb3c5ca68e0ebb39457fbe86beb3482986ddfe0b0b6fc0386807edbf, taskhash 0aff4dcbdb3c5ca68e0ebb39457fbe86beb3482986ddfe0b0b6fc0386807edbf
This is usually due to missing setscene tasks. Those missing in this build were: {'tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_complete',
 'tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_qa'}
ERROR: Task (tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta/recipes-kernel/linux/linux-yocto_6.5.bb:do_deploy_links) failed with exit code 'setscene ignore_tasks'
NOTE: Tasks Summary: Attempted 4975 tasks of which 4971 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta/recipes-kernel/linux/linux-yocto_6.5.bb:do_deploy_links
Summary: There was 1 WARNING message.
Summary: There was 1 ERROR message, returning a non-zero exit code.
ERROR: Logfile of failure stored in: tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0/temp/log.do_populate_sdk_ext.2280835
NOTE: recipe core-image-minimal-1.0-r0: task do_populate_sdk_ext: Failed
ERROR: Task (/OE/build/poky/meta/recipes-core/images/core-image-minimal.bb:do_populate_sdk_ext) failed with exit code '1'
NOTE: Tasks Summary: Attempted 6211 tasks of which 6147 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  /OE/build/poky/meta/recipes-core/images/core-image-minimal.bb:do_populate_sdk_ext
Summary: There was 1 ERROR message, returning a non-zero exit code.

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agoimage.bbclass: remove hardlinks as well
Martin Jansa [Sat, 18 Nov 2023 21:17:21 +0000 (22:17 +0100)] 
image.bbclass: remove hardlinks as well

* it was removing only destination symlinks, but sometimes hardlink might be regenerated
  as well, e.g. in oeqa test wic.Wic.test_permissions which was failing with:

NOTE: recipe core-image-minimal-1.0-r0: task do_image_wic: Started
ERROR: core-image-minimal-1.0-r0 do_image_wic: Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:create_hardlinks(d)
     0003:
File: '/OE/build/poky/meta/classes-recipe/image.bbclass', lineno: 606, function: create_hardlinks
     0602:        if os.path.exists(src):
     0603:            bb.note("Creating hardlink: %s -> %s" % (dst, src))
     0604:            if os.path.islink(dst):
     0605:                os.remove(dst)
 *** 0606:            os.link(src, dst)
     0607:        else:
     0608:            bb.note("Skipping hardlink, source does not exist: %s -> %s" % (dst, src))
     0609:}
     0610:
Exception: FileExistsError: [Errno 17] File exists: 'tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0/deploy-core-image-minimal-image-complete/core-image-minimal-qemux86-64.rootfs.wic' -> 'tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0/deploy-core-image-minimal-image-complete/core-image-minimal-qemux86-64.rootfs--1.0-r0-20110405230000.wic'

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agoselftest: multiconfig-image-packager: use IMAGE_NAME instead of IMAGE_LINK_NAME
Martin Jansa [Sat, 18 Nov 2023 21:05:49 +0000 (22:05 +0100)] 
selftest: multiconfig-image-packager: use IMAGE_NAME instead of IMAGE_LINK_NAME

* the IMAGE_LINK_NAME now contains PKGV, PKGR in the filename, but the
  multiconfig-image-packager and MC_DEPLOY_IMAGE_BASENAME
  (e.g. core-image-minimal) has different PKGV value causing:

  | DEBUG: Executing shell function do_install
  | install: cannot stat 'tmp-mc-musl/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs--0.1-r0-20110405230000.ext4': No such file or directory
  ...
  | install: cannot stat 'tmp-mc-tiny/deploy/images/qemux86/core-image-minimal-qemux86.rootfs--0.1-r0-20110405230000.cpio.gz': No such file or directory

  because the actual filenames are:
  tmp-mc-musl/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs--1.0-r0-20110405230000.ext4
  tmp-mc-tiny/deploy/images/qemux86/core-image-minimal-qemux86.rootfs--1.0-r0-20110405230000.ext4

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agooeqa: fitimage: respect KERNEL_FIT_NAME
Martin Jansa [Sat, 18 Nov 2023 20:24:44 +0000 (21:24 +0100)] 
oeqa: fitimage: respect KERNEL_FIT_NAME

* avoid couple of get_bb_var calls and use get_bb_vars instead

* use KERNEL_FIT_LINK_NAME instead of assuming it's MACHINE as e.g.:
  machine = get_bb_var('MACHINE')
  fitimage_its_path = os.path.join(deploy_dir_image,
     "fitImage-its-%s-%s-%s" % (image_type, machine, machine))

* be aware that KERNEL_FIT_LINK_NAME can still be set to empty
  and then this oeqa check would fail again, because this hardlink:

  ln -vf $deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}${KERNEL_FIT_NAME}.its "$deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}${KERNEL_FIT_LINK_NAME}"

  wouldn't be created and also with KERNEL_FIT_LINK_NAME the
  PKGV in kernel recipe looks differently in the final kernel
  artifact and KERNEL_FIT_LINK_NAME e.g.:

  AssertionError: False is not true:
  tmp/deploy/images/beaglebone-yocto/fitImage-its-core-image-minimal-initramfs-beaglebone-yocto-beaglebone-yocto--6.1.20+git-r0-20230318024804 image tree source doesn't exist

  because it's actually named with SRCPV expanded:
  tmp/deploy/images/beaglebone-yocto/fitImage-its-core-image-minimal-initramfs-beaglebone-yocto-beaglebone-yocto--6.1.20+git0+29ec3dc6f4_423e199669-r0-20230318024804

  Use KERNEL_FIT_NAME instead of KERNEL_FIT_LINK_NAME but then we would
  need to add .its extension to expected filenames as well, but in previous
  commit I've added KERNEL_FIT_ITS_EXT variable and used it for links as well.
  But this doesn't apply for u-boot-its* files which don't use any extension.

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agooeqa: gdbserver: append -dbg suffix at the end of IMAGE_NAME not IMAGE_LINK_NAME
Martin Jansa [Sat, 18 Nov 2023 19:51:18 +0000 (20:51 +0100)] 
oeqa: gdbserver: append -dbg suffix at the end of IMAGE_NAME not IMAGE_LINK_NAME

* the filename is constructed as:
  meta/classes-recipe/image.bbclass:    d.appendVar('IMAGE_NAME','-dbg')
  and IMAGE_LINK_NAME adds ${IMAGE_VERSION_SUFFIX} _after_ this

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agooeqa: imagefeatures: append -dbg suffix at the end of IMAGE_NAME not IMAGE_LINK_NAME
Martin Jansa [Sat, 18 Nov 2023 19:50:51 +0000 (20:50 +0100)] 
oeqa: imagefeatures: append -dbg suffix at the end of IMAGE_NAME not IMAGE_LINK_NAME

* the filename is constructed as:
  meta/classes-recipe/image.bbclass:    d.appendVar('IMAGE_NAME','-dbg')
  and IMAGE_LINK_NAME adds ${IMAGE_VERSION_SUFFIX} _after_ this

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agoimage.bbclass: don't append -dbg suffix twice
Martin Jansa [Sat, 18 Nov 2023 17:39:29 +0000 (18:39 +0100)] 
image.bbclass: don't append -dbg suffix twice

* now with IMAGE_LINK_NAME defined based on IMAGE_NAME we don't want to
  append -dbg to IMAGE_NAME and then again to IMAGE_LINK_NAME

* this resulted in filename like:
  core-image-minimal-qemux86-64.rootfs-dbg--1.0-r0-20110405230000-dbg.tar.bz2

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agolinux-dummy: add do_deploy_links task
Martin Jansa [Thu, 16 Nov 2023 07:42:49 +0000 (08:42 +0100)] 
linux-dummy: add do_deploy_links task

* fixes containerimage.ContainerImageTests.test_expected_files oeqa test failing with:

  Initialising tasks...ERROR: Task do_build in
  /OE/build/poky/build-st/meta-selftest/recipes-test/container-image/container-test-image.bb
  depends upon non-existent task do_deploy_links in
  /OE/build/poky/meta/recipes-kernel/linux/linux-dummy.bb

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agooeqa: wic: use just isfile() instead of islink()
Martin Jansa [Sat, 18 Nov 2023 23:41:25 +0000 (00:41 +0100)] 
oeqa: wic: use just isfile() instead of islink()

* with [YOCTO #12937] changes the manifest is hardlink not symlink

* fixes:
2023-11-18 23:48:55,695 - oe-selftest - INFO -  ... FAIL
2023-11-18 23:48:55,696 - oe-selftest - INFO - Traceback (most recent call last):
  File "/OE/build/poky/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/OE/build/poky/meta/lib/oeqa/selftest/cases/wic.py", line 836, in test_wic_image_type
    self.assertTrue(os.path.islink(path), msg="Link %s wasn't generated as expected" % path)
AssertionError: False is not true : Link tmp/deploy/images/qemux86-64/wic-image-minimal-qemux86-64.rootfs--1.0-r0-20110405230000.wic wasn't generated as expected

18 months agooeqa: bbtests.BitbakeTests.test_image_manifest: use just isfile() instead of islink()
Martin Jansa [Thu, 16 Nov 2023 07:39:54 +0000 (08:39 +0100)] 
oeqa: bbtests.BitbakeTests.test_image_manifest: use just isfile() instead of islink()

* with [YOCTO #12937] changes the manifest is hardlink not symlink

* fixes:
  2023-11-16 00:16:33,967 - oe-selftest - INFO - test_image_manifest (bbtests.BitbakeTests.test_image_manifest)
  2023-11-16 00:19:05,060 - oe-selftest - INFO -  ... FAIL
  2023-11-16 00:19:05,060 - oe-selftest - INFO - Traceback (most recent call last):
    File "/OE/build/poky/meta/lib/oeqa/selftest/cases/bbtests.py", line 139, in test_image_manifest
      self.assertTrue(os.path.islink(manifest), msg="No manifest file created for image. It should have been created in %s" % manifest)
  AssertionError: False is not true : No manifest file created for image. It should have been created in /OE/build/poky/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs--1.0-r0-20110405230000.manifest

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agoimage-artifact-names.bbclass: add IMAGE_VERSION_SUFFIX_DATETIME which uses SOURCE_DAT...
Martin Jansa [Wed, 15 Nov 2023 11:37:40 +0000 (12:37 +0100)] 
image-artifact-names.bbclass: add IMAGE_VERSION_SUFFIX_DATETIME which uses SOURCE_DATE_EPOCH

* since https://git.openembedded.org/openembedded-core/diff/meta/classes/image-artifact-names.bbclass?id=abb0671d2cebfd7e8df94796404bbe9c7f961058
  which removed the
  bb.data.inherits_class('reproducible_build', d)
  condition this was already applied in all the builds which used DATETIME, so we
  can move it to the default value directly and DISTRO configs than can choose
  to use IMAGE_VERSION_SUFFIX_DATETIME as they want

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agoimage.bbclass: rename create_symlinks to create_hardlinks
Martin Jansa [Sat, 18 Nov 2023 09:31:36 +0000 (10:31 +0100)] 
image.bbclass: rename create_symlinks to create_hardlinks

* to make it more clear what this postfunc does now

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agouboot: use ${IMAGE_MACHINE_SUFFIX} instead of -${MACHINE} and use hardlinks
Martin Jansa [Sat, 18 Nov 2023 21:48:35 +0000 (22:48 +0100)] 
uboot: use ${IMAGE_MACHINE_SUFFIX} instead of -${MACHINE} and use hardlinks

* rename variables to match the conventions used in kernel and image recipes
* use versioned hardlinks as kernel and image recipes, but don't split
  the do_deploy_links task (can be split later).

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agokernel-fitimage.bbclass: add .its extension also to links
Martin Jansa [Mon, 20 Nov 2023 10:54:35 +0000 (11:54 +0100)] 
kernel-fitimage.bbclass: add .its extension also to links

* for consistency with the names

[YOCTO #12937]

18 months agokernel-fitimage.bbclass: avoid duplicate .bin extension
Martin Jansa [Mon, 20 Nov 2023 10:48:50 +0000 (11:48 +0100)] 
kernel-fitimage.bbclass: avoid duplicate .bin extension

* the linux.bin was deployed as:
  fitImage-linux.bin${KERNEL_FIT_NAME}${KERNEL_FIT_BIN_EXT}
  where KERNEL_FIT_BIN_EXT is the 2nd ".bin"

* add the${KERNEL_FIT_BIN_EXT} also to corresponding links:
  fitImage-linux${KERNEL_FIT_LINK_NAME}${KERNEL_FIT_BIN_EXT}

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agokernel: move the leading dash into KERNEL_ARTIFACT_NAME
Martin Jansa [Sat, 18 Nov 2023 12:35:26 +0000 (13:35 +0100)] 
kernel: move the leading dash into KERNEL_ARTIFACT_NAME

* this matches how IMAGE_MACHINE_SUFFIX works and we can use
  that for the default value

* allows to set IMAGE_MACHINE_SUFFIX to empty for people
  who prefer to keep MACHINE name only in the directory name
  otherwise there would be a stray dash in:
  lrwxrwxrwx 2 martin martin   12 Nov 18 13:25 bzImage -> bzImage-.bin
  -rw-r--r-- 2 martin martin  12M Nov 18 13:25 bzImage-.bin
  -rw-r--r-- 2 martin martin 182M Nov 18 13:25 modules-.tgz
  if you set
  KERNEL_ARTIFACT_NAME = ""

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agoimage, kernel: allow to disable creating the hardlinks by setting *LINK_NAME variable...
Martin Jansa [Thu, 11 May 2023 07:35:34 +0000 (09:35 +0200)] 
image, kernel: allow to disable creating the hardlinks by setting *LINK_NAME variables to empty

* they can be disabled individually by setting *_LINK_NAME
  to empty or disable them all by setting IMAGE_VERSION_SUFFIX
  to empty (making them equal to *_NAME variables)

There are couple *_LINK_NAME variables:

IMAGE_LINK_NAME = ""
KERNEL_IMAGE_LINK_NAME = ""
KERNEL_DTB_LINK_NAME = ""
KERNEL_FIT_LINK_NAME = ""
MODULE_TARBALL_LINK_NAME = ""
INITRAMFS_LINK_NAME = ""

or

IMAGE_MACHINE_SUFFIX = ""
IMAGE_NAME_SUFFIX = ""
IMAGE_VERSION_SUFFIX = ""

to have really the minimal filenames:

$ ls tmp/deploy/images/qemux86-64/
bzImage                  core-image-minimal.manifest       core-image-minimal.tar.bz2
bzImage-qemux86-64.bin   core-image-minimal.qemuboot.conf  core-image-minimal.testdata.json
core-image-minimal.ext4  core-image-minimal.spdx.tar.zst   modules-qemux86-64.tgz

and to remove MACHINE name from kernel artifacts as well
(if you prefer the MACHINE name in directory only), you can set:
KERNEL_ARTIFACT_NAME = ""

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agocreate-spdx-2.2.bbclass: use hardlink as well
Martin Jansa [Mon, 13 Nov 2023 12:20:43 +0000 (13:20 +0100)] 
create-spdx-2.2.bbclass: use hardlink as well

[YOCTO #12937]

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
18 months agoimage*.bbclass, kernel*.bbclass: create version-less artifacts and versioned hard...
Martin Jansa [Tue, 10 Jul 2018 19:36:10 +0000 (19:36 +0000)] 
image*.bbclass, kernel*.bbclass: create version-less artifacts and versioned hard links

* instead of versioned artifacts and version-less symlinks

* We used to create the actual artifact files with some version
  in the filename and then created symlink without any version
  which was updated to point to the latest one created.
  In some scenarios it's useful to create all artifacts - typically
  rootfs and kernel images with the same version - like release
  build even when the kernel itself wasn't modified since the
  previous release.

  If we include the release version in the regular _NAME variables
  then we'll need to re-run do_deploy and do_image which will cause
  kernel to be rebuilt and image to be re-created even when the
  only change since last build was the version number.

  With this change we can re-use kernel and image from sstate when
  nothing was changed and run only very fast do_deploy_links task
  which just adds another hard link to existing artifact from
  sstate.

* This is already used by various LGE builds as do_webos_deploy_fixup()
  https://github.com/webosose/meta-webosose/blob/master/meta-webos/classes/webos_deploy.bbclass
  but injecting this task in all the right places is difficult
  and sometimes requires whole bbclass to be duplicated. Having
  simpler way of versioning artifacts directly in oe-core might
  be useful for others.

* move IMAGE_VERSION_SUFFIX from _NAME variables to _LINK_NAME
  that way e.g. kernel.do_deploy can be reused from sstate to
  provide "version-less" artifacts and then very fast
  do_deploy_links task just adds links with consistent suffixes
  (by default the version from the recipe but could be easily set
  to e.g. some release name when building some products).
* create hard links instead of symlinks, so that whatever version
  the filename says is really there
* some IMAGE_FSTYPES might need the "version-less" IMAGE_NAME file
  to be removed first or they might either append or update the
  content of the image instead of creating new image file from
  scratch - I have seen this only with one proprietary format we
  generate with our own tool, so hopefully this isn't very common
* this is basically the mechanism are using in webOS with
  WEBOS_IMAGE_NAME_SUFFIX which is for official builds set from
  jenkins job and then all artifacts (images as well as corresponding
  kernel files) have the same version string)

* without this, you can still easily set the variables to contain
  the version from jenkins job (excluded from sstate signature like
  DATETIME currently is to prevent rebuilding it everytime even when
  the content didn't change) but then when kernel is reused from sstate
  you can have version 1.0 used on kernel artifacts and 2.0 on image
  artifacts.

* if you don't exclude the version string with vardepsexclude, then
  you get the right version in the filenames but for cost of
  re-executing do_deploy every single time, which with rm_work will
  cause all kernel tasks to be re-executed (together with everything
  which depends on it like external modules etc).

* the implementation "from outside" is a bit tricky as shown in webOS
  OSE, because first you need to reverse the meaning of IMAGE_NAME
  and IMAGE_LINK_NAME like here, but also replace all symlinks with
  hardlinks and then adjust all recipes/bbclasses to depend on our
  do_deploy_fixup task instead of the original do_deploy
  see the variable modifications:
  https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/conf/distro/include/webos.inc#L65
  and then various bbclasses to hook do_webos_deploy_fixup task creating
  the hardlinks for possible artifacts:
  https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/classes/webos_deploy.bbclass
  https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/classes/kernel.bbclass
  https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/classes/image.bbclass
  so hopefully with all these changes in oe-core other project can
  achieve the same just by setting one variable IMAGE_VERSION_SUFFIX

* drop ${PKGE}-${PKGV}-${PR} from kernel artifacts names (this is the
  latest build) and add it only in hardlinks created in do_deploy_links
  so that we can use PKGR there again (because these links are generally
  used only by human operators and they don't have their own TASKHASH or
  the IMAGE_VERSION_SUFFIX might be set to some release name which they
  do understand

* this allows to drop package_get_auto_pr from kernel do_deploy as well,
  leaving only 2 EXTENDPRAUTO bumps for each kernel build (do_package
  and do_deploy_links, unfortunatelly these will still have different
  value, so if you're looking for the exact kernel image in deploy
  directory based on kernel image package version seen on the device the
  EXTENDPRAUTO part of PR will be different).

[YOCTO #12937]

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
18 months agognutls: Upgrade 3.8.2 -> 3.8.3
Simone Weiß [Sun, 28 Jan 2024 18:47:41 +0000 (18:47 +0000)] 
gnutls: Upgrade 3.8.2 -> 3.8.3

Upgrade version to adress recent CVE findings.

Changelog
=========
** libgnutls: Fix more timing side-channel inside RSA-PSK key exchange
   [GNUTLS-SA-2024-01-14, CVSS: medium] [CVE-2024-0553]

** libgnutls: Fix assertion failure when verifying a certificate chain with a
   cycle of cross signatures
   [GNUTLS-SA-2024-01-09, CVSS: medium] [CVE-2024-0567]

** libgnutls: Fix regression in handling Ed25519 keys stored in PKCS#11 token
   certtool was unable to handle Ed25519 keys generated on PKCS#11
   with pkcs11-tool (OpenSC). This is a regression introduced in 3.8.2.

Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoxwayland: Upgrade 23.2.3 -> 23.2.4
Khem Raj [Sun, 28 Jan 2024 17:05:59 +0000 (09:05 -0800)] 
xwayland: Upgrade 23.2.3 -> 23.2.4

Cumulative Fixes for CVE-2023-6816, CVE-2024-0229, CVE-2024-21885, CVE-2024-21886,
CVE-2024-0408, CVE-2024-0409

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agopython3-yamllint: add missing dependency
Konrad Weihmann [Sun, 28 Jan 2024 08:34:12 +0000 (09:34 +0100)] 
python3-yamllint: add missing dependency

yamllint requires pathspec module to be available

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoqemu: fix target build with ccache enabled
Martin Jansa [Tue, 30 Jan 2024 14:47:48 +0000 (15:47 +0100)] 
qemu: fix target build with ccache enabled

* with ccache inheritted, the BUILD_CC is 'ccache gcc', but because of missing
  quote it ends passing just ccache to host-cc which gets stripped and then
  it calls compiler[0] on empty compiler variable and breaks meson as shown in:
  http://errors.yoctoproject.org/Errors/Details/751436/

python version: Python 3.12.1
Traceback (most recent call last):
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/mesonmain.py", line 194, in run
    return options.run_func(options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/msetup.py", line 358, in run
    app.generate()
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/msetup.py", line 181, in generate
    return self._generate(env, capture, vslite_ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/msetup.py", line 203, in _generate
    intr = interpreter.Interpreter(b, user_defined_options=user_defined_options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/interpreter/interpreter.py", line 331, in __init__
    self.parse_project()
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 132, in parse_project
    self.evaluate_codeblock(self.ast, end=1)
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 198, in evaluate_codeblock
    raise e
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 190, in evaluate_codeblock
    self.evaluate_statement(cur)
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 204, in evaluate_statement
    return self.function_call(cur)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 530, in function_call
    res = func(node, func_args, kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/interpreterbase/decorators.py", line 260, in wrapper
    return f(*nargs, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/interpreterbase/decorators.py", line 579, in wrapper
    return f(*wrapped_args, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/interpreter/interpreter.py", line 1309, in func_project
    self.add_languages(proj_langs, False, MachineChoice.BUILD)
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/interpreter/interpreter.py", line 1476, in add_languages
    success = self.add_languages_for(args, required, for_machine)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/interpreter/interpreter.py", line 1519, in add_languages_for
    comp = compilers.detect_compiler_for(self.environment, lang, for_machine, skip_sanity_check)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/compilers/detect.py", line 115, in detect_compiler_for
    comp = compiler_from_language(env, lang, for_machine)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/compilers/detect.py", line 112, in compiler_from_language
    return lang_map[lang](env, for_machine) if lang in lang_map else None
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/compilers/detect.py", line 614, in detect_c_compiler
    return _detect_c_or_cpp_compiler(env, 'c', for_machine)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot-native/usr/lib/python3.12/site-packages/mesonbuild/compilers/detect.py", line 286, in _detect_c_or_cpp_compiler
    compiler_name = os.path.basename(compiler[0])
                                     ~~~~~~~~^^^
IndexError: list index out of range

...

Project name: qemu
Project version: 8.2.0
C compiler for the host machine: ccache x86_64-oe-linux-clang -target x86_64-oe-linux -m64 -march=core2 -mtune=core2 -msse3 -mfpmath=sse -mlittle-endian --dyld-prefix=/usr -Qunused-arguments --sysroot=TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot -m64 -mcx16 (clang 17.0.6 "clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18)")
C linker for the host machine: x86_64-oe-linux-clang -target x86_64-oe-linux -m64 -march=core2 -mtune=core2 -msse3 -mfpmath=sse -mlittle-endian --dyld-prefix=/usr -Qunused-arguments --sysroot=TOPDIR/tmp-glibc/work/core2-64-oe-linux/qemu/8.2.0/recipe-sysroot -m64 -mcx16 ld.lld 17.0.6

../qemu-8.2.0/meson.build:1:0: ERROR: Unhandled python exception

    This is a Meson bug and should be reported!

ERROR: meson setup failed

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agooeqa/qemurunner: Handle rare shutdown race
Richard Purdie [Fri, 26 Jan 2024 22:52:26 +0000 (22:52 +0000)] 
oeqa/qemurunner: Handle rare shutdown race

The pid file can disappear when qemu is shutting down leading to a
file not found race before it is read.

Tweak the code to handle this and fix a rare but annoying race error
case.

[YOCTO #15036]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoqemu: Replace the basename patch with backport
Khem Raj [Thu, 25 Jan 2024 00:32:50 +0000 (16:32 -0800)] 
qemu: Replace the basename patch with backport

Backport the patch that got applied to fix this issue upstream

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agopiglit: Update to latest revision
Fabio Estevam [Thu, 25 Jan 2024 14:57:48 +0000 (11:57 -0300)] 
piglit: Update to latest revision

Update to the latest piglit revision.

Signed-off-by: Fabio Estevam <festevam@denx.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agopython3: Initialize struct termios before calling tcgetattr()
Khem Raj [Thu, 25 Jan 2024 00:05:24 +0000 (16:05 -0800)] 
python3: Initialize struct termios before calling tcgetattr()

Fixes ptest failures on python 3.12

    Failed ptests:
    {'python3': ['test_tcgetattr',
                 'test_cfmakeraw',
                 'test_setcbreak',
                 'test_setraw',
                 'python3']}

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agolttng-modules: upgrade 2.13.10 -> 2.13.11
Denys Dmytriyenko [Fri, 26 Jan 2024 04:15:59 +0000 (23:15 -0500)] 
lttng-modules: upgrade 2.13.10 -> 2.13.11

Changelog:
https://lwn.net/Articles/957396/

Drop 2 backported patches and 1 implemented differently upstream.

Signed-off-by: Denys Dmytriyenko <denis@denix.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoelfutils: Fix build with gcc trunk
Khem Raj [Fri, 26 Jan 2024 07:21:05 +0000 (23:21 -0800)] 
elfutils: Fix build with gcc trunk

Fixes a build issue seen with latest gcc trunk.

Fixes
| ../../elfutils-0.189/tests/elfstrmerge.c: In function 'main':
| ../../elfutils-0.189/tests/elfstrmerge.c:450:32: error: 'calloc' sizes specified with 'sizeof' in the earlier argument and not in the later argument [-Werror=calloc-transposed-args]
|   450 |   newscnbufs = calloc (sizeof (void *), newshnums);
|       |                                ^~~~
| ../../elfutils-0.189/tests/elfstrmerge.c:450:32: note: earlier argument should specify number of elements, later size of each element
| cc1: all warnings being treated as errors

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoxz: Add ptest support
Chi Xu [Fri, 26 Jan 2024 05:48:20 +0000 (13:48 +0800)] 
xz: Add ptest support

There are two types of cases: executables and POSIX shell scripts.
All test cases PASS.
Add xz-ptest to PTESTS_FAST because test duration less than 30s
on qemux86-64.

root@qemux86-64:~# ptest-runner xz
START: ptest-runner
2024-01-26T03:32
BEGIN: /usr/lib/xz/ptest
=== test_bcj_exact_size.c ===
PASS: test_exact_size
PASS: test_empty_block

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agogtk4: update 4.12.4 -> 4.12.5
Markus Volk [Fri, 26 Jan 2024 00:42:51 +0000 (01:42 +0100)] 
gtk4: update 4.12.4 -> 4.12.5

Overview of Changes in 4.12.5, 17-01-2024
=========================================

* GtkColumnView:
 - Fix a crash on dispose

* GtkEmojiChooser:
 - Update to CLDR v44
 - Add more translations

* GtkFileDialog:
 - Return an error if no file is selected
 - Make closing the portal file chooser work

* GtkDropDown:
 - Fix display of the initial checkmark

* GtkShortcutsWindow:
 - Reduce the minimum width

* GDK:
 - Make the png loader safer against overflow

* Windows:
 - Use new clipboard api

* Wayland:
 - Fix cursor handling with graphics tablets

* macOS:
 - Silence secure-restore message
 - Fix build on macOS < 10.13

* Translation updates
 Basque
 British English
 Catalan
 Chinese (China)
 Czech
 Georgian
 German
 Hebrew
 Indonesian
 Lithuanian
 Persian
 Polish
 Romanian
 Russian
 Slovenian
 Swedish
 Turkish
 Ukrainian
 Vietnamese

Signed-off-by: Markus Volk <f_l_k@t-online.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agorpm: update 4.18.1 -> 4.19.1
Alexander Kanavin [Fri, 26 Jan 2024 13:34:55 +0000 (14:34 +0100)] 
rpm: update 4.18.1 -> 4.19.1

Upstream has replaced autoconf with cmake, which necessitates a rewrite of the
recipe and available options, and a rebase to cmake of
0001-Do-not-hardcode-lib-rpm-as-the-installation-path-for.patch

Correct a mistake in 0001-Do-not-read-config-files-from-HOME.patch :
the patch was removing the NULL marker at the end of function arguments,
and 0002-Add-support-for-prefixing-etc-from-RPM_ETCCONFIGDIR-.patch
was restoring it (in addition to the actual change the patch was making).
Now both patches preserve the NULL terminator.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agooeqa/runtime/rpm: fail tests if test rpm file cannot be found
Alexander Kanavin [Fri, 26 Jan 2024 13:34:54 +0000 (14:34 +0100)] 
oeqa/runtime/rpm: fail tests if test rpm file cannot be found

Discovery of the test file was happening in a class initializer.
That block of code cannot fail (it's not a test), and so it
falls through to completion even if the needed file could not be found.

Then the tests themselves fail later due to class variables not
being set, but all information as to why is already lost at that point.

This converts the discovery to a helper function called from
the tests, so that the function can fail the tests precisely when the
problems occur.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoclasses/package_rpm: set bogus locations for passwd/group files
Alexander Kanavin [Fri, 26 Jan 2024 13:34:53 +0000 (14:34 +0100)] 
classes/package_rpm: set bogus locations for passwd/group files

Since https://github.com/rpm-software-management/rpm/commit/f3eaeeb7341085e1850e914350cf1f33d538320d
rpm does its own parsing of /etc/passwd and /etc/group instead of relying on getpwnam() and friends.

This has an unfortunate effect of leaking build host uid/gid values for users and groups
into the cpio header inside rpm file (where previously those were always zero).

Installation of rpm packages relies on rpm header to set files ownership, and that
is a different structure that is build from .spec information, so we can avoid host
contamination by setting the paths to something bogus.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoclasses/package_rpm: use weak user/group dependencies
Alexander Kanavin [Fri, 26 Jan 2024 13:34:52 +0000 (14:34 +0100)] 
classes/package_rpm: use weak user/group dependencies

rpm 4.19 automatically generates provides and depends for user and
groups:
https://github.com/rpm-software-management/rpm/blob/rpm-4.19.x/docs/manual/users_and_groups.md#dependencies

This mechanism relies on sysusers.d for the 'provides'
part, and thus is systemd-only at best. So we need to disable it for now,
otherwise image generation fails with unresolved dependencies.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoclasses/package_rpm: write file permissions and ownership explicitly into .spec
Alexander Kanavin [Fri, 26 Jan 2024 13:34:51 +0000 (14:34 +0100)] 
classes/package_rpm: write file permissions and ownership explicitly into .spec

Per https://github.com/rpm-software-management/rpm/commit/77d3529c31ca090a40b8d3959a0bcdd721a556d6
rpm 4.19.1+ will not consider actual filesystem permissions and ownership, and will quietly default
to root if not expictly set otherwise in .spec file.

There's also additional diagnostics (printing what is in passwd/group)
when user/group name lookup against the sysroot fails.
That is never supposed to happen, and yet there was one report that it did:
https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/8493/steps/23/logs/stdio

Investigating that issue led to the first three commits in this patchset:

sysroot user management postinsts: run with /bin/sh -e to report errors when they happen
classes/multilib: expand PACKAGE_WRITE_DEPS in addition to DEPENDS
classes/staging: capture output of sysroot postinsts into logs

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoclasses/staging: capture output of sysroot postinsts into logs
Alexander Kanavin [Fri, 26 Jan 2024 13:34:50 +0000 (14:34 +0100)] 
classes/staging: capture output of sysroot postinsts into logs

This particularly helps with user management postinsts as otherwise
there's no trace left of what was run, in which order, and what was the output.

Here's an example from the logs:

NOTE: Running postinst /srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/lib64-recipe-sysroot/usr/bin/postinst-lib64-base-passwd, output:
b'/srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/recipe-sysroot-native/usr/sbin/useradd\nRunning useradd commands...\nNOTE: lib64-ptest-runner: Performing useradd with [--root /srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/lib64-recipe-sysroot --system --no-create-home --home / --user-group ptest]\n'
NOTE: Running postinst /srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/lib64-recipe-sysroot/usr/bin/postinst-useradd-lib64-ptest-runner, output:
b'/srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/recipe-sysroot-native/usr/sbin/useradd\nRunning useradd commands...\nNOTE: lib64-ptest-runner: Performing useradd with [--root /srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/lib64-recipe-sysroot --system --no-create-home --home / --user-group ptest]\nNOTE: lib64-ptest-runner: user ptest already exists, not re-creating it\n'

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoclasses/multilib: expand PACKAGE_WRITE_DEPS in addition to DEPENDS
Alexander Kanavin [Fri, 26 Jan 2024 13:34:49 +0000 (14:34 +0100)] 
classes/multilib: expand PACKAGE_WRITE_DEPS in addition to DEPENDS

Otherwise, PACKAGE_WRITE_DEPS would contain non-multilib variants
of dependencies even when building multilib items, resulting in
sysroots being populated with entirely wrong versions of them.

This hasn't been noticed until now through sheer (bad) luck, I think,
except in the cpio recipe, but the previous commit shows that the issues
did occur, quietly. Every other recipe in oe-core and meta-oe does not
prepend the multilib prefix.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agosysroot user management postinsts: run with /bin/sh -e to report errors when they...
Alexander Kanavin [Fri, 26 Jan 2024 13:34:48 +0000 (14:34 +0100)] 
sysroot user management postinsts: run with /bin/sh -e to report errors when they happen

This exposes the following failure in a multilib setup, when everything
up to do_package_write_rpm is in sstate, but do_package_write_rpm is not
(there's a similar fail for lib64-man-db, and the failures themselves will be fixed separately
in the next commit):

Exception: subprocess.CalledProcessError: Command '/srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/recipe-sysroot/usr/bin/postinst-base-passwd' returned non-zero exit status 1.

Subprocess output:
install: cannot stat '/srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/lib64-recipe-sysroot/usr/share/base-passwd/passwd.master': No such file or directory

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agopython3-numpy: Use Large File Support version of fallocate
Ola x Nilsson [Fri, 26 Jan 2024 13:57:46 +0000 (14:57 +0100)] 
python3-numpy: Use Large File Support version of fallocate

This was supposed to always be the case from upstream but was exposed
by the 64-bit-time QA checker when file function scanning was enabled.

Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agonative.bbclass: base_libdir unique from libdir
William Hauser [Wed, 17 Jan 2024 16:30:03 +0000 (16:30 +0000)] 
native.bbclass: base_libdir unique from libdir

Use STAGING_BASE_LIBDIR_NATIVE for the value of base_libdir instead of
STAGING_LIBDIR_NATIVE. This will avoid conflicts between the two
directories.

Signed-off-by: William Hauser <william.hauser@meraki.net>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agosanity: require bitbake 2.7.2 for the inherit_defer statement
Ross Burton [Fri, 26 Jan 2024 16:19:35 +0000 (16:19 +0000)] 
sanity: require bitbake 2.7.2 for the inherit_defer statement

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agorootfs-postcommands.bbclass: ignore comment mismatch in systemd_user_check
Chen Qi [Wed, 27 Dec 2023 04:20:36 +0000 (12:20 +0800)] 
rootfs-postcommands.bbclass: ignore comment mismatch in systemd_user_check

The check was forcing every field to be identical, this is too
strict. For example, if the comment differs, there's really no
impact. For example, root user's comment is 'root' in passwd,
and it's 'Super User' in sysusers.d/basic.conf. Such difference is
not worth a warning. In fact, previous codes use 'lower()' to
avoid warning between 'nobody' and 'Nobody', and what's more, another
patch puts its own basic.conf.in in systemd's SRC_URI, but it changes
'Super User' to 'root'. Such changes are all unnecessary. We should
just ignore comment mismatch.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agosystemd-boot: upgrade to 255.1
Chen Qi [Wed, 27 Dec 2023 04:20:35 +0000 (12:20 +0800)] 
systemd-boot: upgrade to 255.1

0030-meson-Pass-all-static-pie-args-to-linker.patch is dropped as
it's a backported patch and is now in the new 255.1 version.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agosystemd: upgrade to 255.1
Chen Qi [Wed, 27 Dec 2023 04:20:34 +0000 (12:20 +0800)] 
systemd: upgrade to 255.1

1. Patch changes:

0004-Move-sysusers.d-sysctl.d-binfmt.d-modules-load.d-to-.patch
is removed because it has no real effect now. The /lib is now
/usr/lib because 'usrmerge' is a required distro feature for systemd.

0002-binfmt-Don-t-install-dependency-links-at-install-tim.patch is
refreshed for the new version to avoid patch-fuzz issue.

2. root user's home directory now defaults to "/root":

The sysuers.d/basic.conf is still modified to respect the ROOT_HOME
value, so if users set ROOT_HOME to "/home/root", the behavior is the
same as before. However, this is only for backward compatibility. With
this patch, The ROOT_HOME value is set to "/root" in init-manager-systemd.inc.
This is because systemd's source codes are hardcoding "/root", and other
values are not officially supported. See the list below.
$ grep -rl '"/root"' src/ | grep -v 'src/test'
src/core/namespace.c
src/basic/user-util.c
src/nss-systemd/nss-systemd.c
src/nspawn/nspawn.c
src/firstboot/firstboot.c
src/shared/userdb.c
src/shared/user-record.c
$ grep -rl /root network/ factory/ sysctl.d/ sysusers.d/ rules.d/ tmpfiles.d/ units/ xorg/ tools/
sysusers.d/basic.conf.in
tmpfiles.d/provision.conf
units/emergency.service.in
units/rescue.service.in
tools/list-discoverable-partitions.py
Previously, the recipe was just substituting sysusers.d/basic.conf.in,
which is not enough to be treated as 'fully support'. I deliberately put
a warning message in do_install to warn users about non "/root" ROOT_HOME
value. Don't remove it until all above places are handled.

3. cgroupv2 is now the default.
cgroupv2 is the default for systemd for many years and it's the default
for distros such as ubuntu and fedora. Let's also use it as the default.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agosystemd: refresh musl patches for v255.1
Chen Qi [Tue, 2 Jan 2024 03:31:19 +0000 (11:31 +0800)] 
systemd: refresh musl patches for v255.1

The current patches are refreshed. A new patch is added:
0022-avoid-missing-LOCK_EX-declaration.patch

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agooeqa/selftest: add test case to cover 'devtool modify -n' for a git recipe
Chen Qi [Mon, 22 Jan 2024 05:29:26 +0000 (13:29 +0800)] 
oeqa/selftest: add test case to cover 'devtool modify -n' for a git recipe

Add a test case to ensure the following error does not happen again for
'devtool modify -n'.

Traceback (most recent call last):
  File "/buildarea2/chenqi/poky/scripts/devtool", line 349, in <module>
    ret = main()
  File "/buildarea2/chenqi/poky/scripts/devtool", line 336, in main
    ret = args.func(args, config, basepath, workspace)
  File "/buildarea2/chenqi/poky/scripts/lib/devtool/standard.py", line 924, in modify
    if not initial_revs["."]:
KeyError: '.'

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agorust: Re-write RPATHs in the copies llvm-config
Khem Raj [Mon, 15 Jan 2024 20:55:17 +0000 (12:55 -0800)] 
rust: Re-write RPATHs in the copies llvm-config

Ensure that it can still access the native-sysroot for dependencies,
use ORIGIN to indicate this relative its install location, this also
helps in this not getting into the output of llvm-config which could
otherwise provide incorrect library paths

target rust recipe builds ( cross compile ) calls llvm-config from
target sysroot which works ok as long as C++ runtime it needs is
available on build host e.g. libstdc++ etc. which is commonly the
case, however when using clang and llvm runtime this falters since
it should be using libc++ from native sysroot and if this does not
exist on build machine this fails to find libc++ shared object and
llvm-config fails to run. This ensures that llvm-config version in
use is correctly relocated and can use shared libraries from native
sysroot correctly. Adding ORIGIN to sysroot will look for the .so in
same dir as the binary and there is the libc++.so.1 copied in place

Fixes rust build with clang compiler.

| /mnt/b/yoe/master/build/tmp/work/riscv64-yoe-linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-config: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or director
y
| thread 'main' panicked at llvm.rs:551:19:
| command did not execute successfully: "/mnt/b/yoe/master/build/tmp/work/riscv64-yoe-linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-config" "--version"
| expected success, got: exit status: 127

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agopulseaudio: Update to 17.0
Fabio Estevam [Wed, 24 Jan 2024 13:55:15 +0000 (10:55 -0300)] 
pulseaudio: Update to 17.0

Update to pulseaudio 17.0 version.

Since pulseaudio commit e96d278bfc51 ("daemon/meson.build: Install
dbus policy in /usr, not /etc"), pulseaudio-system.conf needs to be passed
in FILES, otherwise the following QA error is seen:

ERROR: pulseaudio-17.0-r0 do_package: QA Issue: pulseaudio: Files/directories were installed but not shipped in any package:
  /usr/share/dbus-1
  /usr/share/dbus-1/system.d
  /usr/share/dbus-1/system.d/pulseaudio-system.conf

Signed-off-by: Fabio Estevam <festevam@denx.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoperl: Fix perl-module-* being ignored via COMPLEMENTARY_GLOB
William Lyu [Wed, 24 Jan 2024 16:32:34 +0000 (08:32 -0800)] 
perl: Fix perl-module-* being ignored via COMPLEMENTARY_GLOB

When perl-modules is added via COMPLEMENTARY_GLOB, all perl-module-*
packages recommended by perl-modules are ignored due to the defined
behavior of COMPLEMENTARY_GLOB.

This patch changes the relationship between perl-modules and all of its
perl-module-* from RRECOMMENDS to RDEPENDS. This makes sense as
perl-modules should represent the collection of all optional
perl-module-* packages. After this patch, perl-modules itself is being
RRECOMMENDED instead of the individual perl-module-* packages
perl-modules represents.

Signed-off-by: William Lyu <William.Lyu@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoudev-extraconf: fix unmount directories containing octal-escaped chars
Jonathan GUILLOT [Wed, 24 Jan 2024 13:29:19 +0000 (13:29 +0000)] 
udev-extraconf: fix unmount directories containing octal-escaped chars

USB devices are auto-mounted in a directory named like theirs labels.
Special characters like whitespace are octal-escaped in /proc/mounts
output. Using directly this output file as an argument for umount failed
and the mount directory can't be removed as still busy.
Using printf allows these special characters to be unescaped.

Signed-off-by: Jonathan GUILLOT <jonathan@joggee.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agogcc: Update status of CVE-2023-4039
Simone Weiß [Mon, 22 Jan 2024 16:41:32 +0000 (16:41 +0000)] 
gcc: Update status of CVE-2023-4039

This is fixed via a patch added in gcc-13.2.inc already, but still
reported e.g. for libgcc as it is not defining an own source but use the
shared gcc-source.

Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agopython3-yamllint: Add recipe
Ryan Eatmon [Tue, 23 Jan 2024 16:39:57 +0000 (10:39 -0600)] 
python3-yamllint: Add recipe

Add recipe for yamllint.  There is an upcoming change in u-boot where
the binman tool is now configured to call yamllint to verify the configs
during compile time.

There was a previous patch a year ago from Trevor Woerner that never
made it into oe-core.  This patch is a reworking of his patch but
pointing to a newer version.

Signed-off-by: Ryan Eatmon <reatmon@ti.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agop11-kit: fix parallel build failures
Kai Kang [Mon, 22 Jan 2024 04:24:00 +0000 (12:24 +0800)] 
p11-kit: fix parallel build failures

It fails occasionally with missing generated header files:

| ../git/common/asn1.c:42:10: fatal error: openssl.asn.h: No such file or directory
|    42 | #include "openssl.asn.h"
|       |          ^~~~~~~~~~~~~~~
| compilation terminated.

According to meson manual page:

https://mesonbuild.com/Wrap-best-practices-and-tips.html#declare-generated-headers-explicitly

'asn_h_dep' should be a dependency of static_library target 'libp11_asn1'
to make sure that required header files generated before compile
common/asn1.c.

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agonativesdk-cairo: fix build error
Kai Kang [Mon, 22 Jan 2024 09:56:11 +0000 (17:56 +0800)] 
nativesdk-cairo: fix build error

It fails to build nativesdk-cairo:

| ../cairo-1.18.0/meson.build:381:13: ERROR: Can not run test
    applications in this cross environment.

Set meson property ipc_rmid_deferred_release in cross-file to fix the
issue which is as same as for target cairo

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoqemu: Fix segfaults in webkitgtk:do_compile on debian11
Richard Purdie [Thu, 25 Jan 2024 23:31:37 +0000 (23:31 +0000)] 
qemu: Fix segfaults in webkitgtk:do_compile on debian11

Add a work around for segfaults we're seeing with qemu 8.2.0 when
compiling webkitgtk on debian 11 autobuilder workers.

The issue has been reported upstream to work out an official fix.

[YOCTO #15367]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agorust: Enable rust oe-selftest.
Yash Shinde [Thu, 25 Jan 2024 13:16:03 +0000 (05:16 -0800)] 
rust: Enable rust oe-selftest.

* Enable rust oe-selftest.

* Include the dependent patches for rust oe-selftest in
  meta/recipes-devtools/rust/rust-source.inc

* Disable rust oe-selftest for mips32 target (Rust upstream has classified it into tier 3 target,
  for which the Rust project does not build or test automatically) as it is unstable with rust tests.
  https://doc.rust-lang.org/nightly/rustc/platform-support.html#tier-3

* The testing is done on arm32, arm64, mips64, x86 and  x86_64 targets on Ubuntu 22.04.

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agorust: Remove the test cases whose parent dir is also present in the exclude list
Yash Shinde [Thu, 25 Jan 2024 13:16:02 +0000 (05:16 -0800)] 
rust: Remove the test cases whose parent dir is also present in the exclude list

To avoid overlap and redundancy of rust tests, remove the test cases whose
parent dir is already excluded.

Tests which are failing from below dirs are removed as these dirs are
already present in exclude list
            tests/run-make
            tests/rustdoc
            tests/mir-opt
            tests/ui-fulldeps

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agorust: Add new tests in the exclude list for rust oe-selftest
Yash Shinde [Thu, 25 Jan 2024 13:16:01 +0000 (05:16 -0800)] 
rust: Add new tests in the exclude list for rust oe-selftest

Add newly failing tests cases in the exclude list for
rust oe-selftest.

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agorust: Fix assertion failure error on oe-selftest
Yash Shinde [Thu, 25 Jan 2024 13:16:00 +0000 (05:16 -0800)] 
rust: Fix assertion failure error on oe-selftest

Fixes: thread 'main' panicked at 'assertion failed: `(left == right)`
       left: `x86_64-unknown-linux-gnu`,
       right: `x86_64-poky-linux-gnu`: Cannot obtain compiler for non-native build triple at stage 0', compile.rs:1474:13

Add correct target value for cross-compiled targets on stage1 during rust oe-selfest.

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agorust: Enable RUSTC_BOOTSTRAP to use nightly features during rust oe-selftest.
Yash Shinde [Thu, 25 Jan 2024 13:15:59 +0000 (05:15 -0800)] 
rust: Enable RUSTC_BOOTSTRAP to use nightly features during rust oe-selftest.

Fixes: error: the option `Z` is only accepted on the nightly compiler
When rust.channel is set to either beta or stable, we can't use
nightly features on bootstrap without RUSTC_BOOTSTRAP.
Set RUSTC_BOOTSTRAP=1 to use nightly features on stable or beta.

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agorust: detect user-specified custom targets in compiletest
Yash Shinde [Thu, 25 Jan 2024 13:15:58 +0000 (05:15 -0800)] 
rust: detect user-specified custom targets in compiletest

Fixes: thread 'main' panicked at 'failed to gather the target spec
       for '<arch>-unknown-linux-gnu', synthetic_targets.rs:66:9

Detect and fetch custom target configurations when rustc is
bootstrapped in rust oe-selftest.

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agorust: Fetch cargo from rust-snapshot dir.
Yash Shinde [Thu, 25 Jan 2024 13:15:57 +0000 (05:15 -0800)] 
rust: Fetch cargo from rust-snapshot dir.

Fixes: Exception: no cargo executable found at
       `${B}/rustc-1.74.1-src/build/x86_64-unknown-linux-gnu/stage0/bin/cargo`

Fix the cargo binary path error on oe-selftest and path set to rust-snapshot dir.

Patch sent to upstream- https://github.com/rust-lang/rust/pull/120125

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoqemu: Upgrade 8.1.2 -> 8.2.0
Richard Purdie [Tue, 23 Jan 2024 21:27:08 +0000 (21:27 +0000)] 
qemu: Upgrade 8.1.2 -> 8.2.0

We can drop the mips workaround patch since there were fixes in 8.2.0.

The build system changed and we should drop cross.patch and replace it with
explicit settings for cross-prefix, and host-cc. To make that work we need
to indicate we don't use a cross pkg-config.

PIE isn't availale for mingw so tweak PACKAGECONFIG accordingly.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agopython3-subunit: Add missing module dependency
Richard Purdie [Tue, 23 Jan 2024 20:25:28 +0000 (20:25 +0000)] 
python3-subunit: Add missing module dependency

In version 1.4.3:
* Subunit now has a dependency on an external iso8601
  module rather than shipping its own.

so add the missing dependency.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agooeqa/selftest/devtool: add test for recipes with multiple sources in SRC_URI
Julien Stephan [Tue, 23 Jan 2024 14:06:51 +0000 (15:06 +0100)] 
oeqa/selftest/devtool: add test for recipes with multiple sources in SRC_URI

add a non regression test for devtool modify/build on recipe having
several sources in SRC_URI

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agodevtool: modify: add support for multiple source in SRC_URI
Julien Stephan [Tue, 23 Jan 2024 14:06:50 +0000 (15:06 +0100)] 
devtool: modify: add support for multiple source in SRC_URI

[YOCTO #15162]

when doing devtool modify, sources are extracted into a devtool
temporary workdir. The main source is moved inside
build/workspace/sources/${BPN}/ and local files are moved inside
build/workspace/sources/${BPN}/oe-local-files. Secondary sources are
currently not handled and are lost.

Here is the output of devtool modify/build on bzip2 recipe:

NOTE: bzip2: compiling from external source tree <...>/build/workspace/sources/bzip2
ERROR: bzip2-1.0.8-r0 do_install_ptest_base: ExecutionError('<...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368', 1, None, None)
ERROR: Logfile of failure stored in: <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/log.do_install_ptest_base.3368
Log data follows:
| DEBUG: Executing shell function do_install_ptest_base
| NOTE: make -j 16 DESTDIR=<...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest install-ptest
| sed  -n '/^runtest:/,/^install-ptest:/{/^install-ptest:/!p}' \
|            ../../../../../../workspace/sources/bzip2/Makefile.am      > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/Makefile
| cp ../../../../../../workspace/sources/bzip2/sample1.ref      <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| cp ../../../../../../workspace/sources/bzip2/sample2.ref      <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| cp ../../../../../../workspace/sources/bzip2/sample3.ref      <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| cp ../../../../../../workspace/sources/bzip2/sample1.bz2      <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| cp ../../../../../../workspace/sources/bzip2/sample2.bz2      <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| cp ../../../../../../workspace/sources/bzip2/sample3.bz2      <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| ln -s /usr/bin/bzip2          <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/bzip2
| cp: cannot stat '<...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/git/commons-compress': No such file or directory
| WARNING: <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368:189 exit 1 from 'cp -r <...>/build/tmp/work/core2-64-poky-linux/bzip2/
1.0.8/git/commons-compress <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/bzip2-tests/commons-compress'
| WARNING: Backtrace (BB generated script):
|       #1: do_install_ptest, <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368, line 189
|       #2: do_install_ptest_base, <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368, line 158
|       #3: main, <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368, line 226
ERROR: Task (<...>/poky/meta/recipes-extended/bzip2/bzip2_1.0.8.bb:do_install_ptest_base) failed with exit code '1'
NOTE: Tasks Summary: Attempted 776 tasks of which 765 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  <...>/poky/meta/recipes-extended/bzip2/bzip2_1.0.8.bb:do_install_ptest_base

externalsrc class modify SRC_URI to keep only:
* 'file', 'npmsw' and 'crate' sources
* url with type parameter matching 'kmeta' or 'git-dependency'

So by forcing to add type='git-dependency' on secondary sources, we
ensure that when building the recipe, the secondary sources can be
unpacked into WORKDIR.

This allows recipes containing several sources to be built under a
devtool context, but it has some limitations:
* user would not be able to generate patches for the secondary sources
* type="git-dependency" is added for secondary sources even on non git
  sources, so we may want to rename this parameter

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoexternalsrc: fix task dependency for do_populate_lic
Julien Stephan [Tue, 23 Jan 2024 14:06:49 +0000 (15:06 +0100)] 
externalsrc: fix task dependency for do_populate_lic

do_populate_lic dependencies are defined inside license.bbclass such as:

  addtask populate_lic after do_patch before do_build

but externalsrc deletes the do_patch task, so the only dependency left for
do_populate_lic is "before do_build"

On a devtool context, when doing devtool modify, sources are extracted inside
build/workspace/sources/${BPN}/ and local files inside
build/workspace/sources/${BPN}/oe-local-files

When building the recipe after a devtool modify, do_unpack is called again to
unpack (possibly modified) local files from
build/workspace/sources/${BPN}/oe-local-files into ${WORKDIR}.

Since the only left dependency for do_populate_lic is do_build, the
do_populate_lic can be called BEFORE do_unpack. Most of the time this is not a
problem, because license files are generally located inside ${S}, which
corresponds to build/workspace/sources/${BPN} (and is already unpacked),
but this can lead to an issue if recipe sets LIC_FILES_CHKSUM to look for
files in ${WORKDIR} (example from init-ifupdown_1.0.bb):

  LIC_FILES_CHKSUM = "file://${WORKDIR}/copyright;md5=3dd6192d306f582dee7687da3d8748ab"

So devtool modify init-ifupdown && bitbake init-ifupdown gives the following
error:

  WARNING: init-ifupdown-1.0-r0 do_populate_lic: Could not copy license file <...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/copyright to <...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/license-destdir/qemux86_64/init-ifupdown/copyright: [Errno 2] No such file or directory: '<...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/copyright'
  ERROR: init-ifupdown-1.0-r0 do_populate_lic: QA Issue: init-ifupdown: LIC_FILES_CHKSUM points to an invalid file: <...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/copyright [license-checksum]
  ERROR: init-ifupdown-1.0-r0 do_populate_lic: Fatal QA errors were found, failing task.
  ERROR: Logfile of failure stored in: <...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/temp/log.do_populate_lic.838584
  ERROR: Task (<...>/poky/meta/recipes-core/init-ifupdown/init-ifupdown_1.0.bb:do_populate_lic) failed with exit code '1'

Fix this by forcing the do_populate_lic task to run after do_unpack

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoglibc: Do not enable CET on 32bit x86
Khem Raj [Mon, 22 Jan 2024 19:26:25 +0000 (11:26 -0800)] 
glibc: Do not enable CET on 32bit x86

Support has been removed from 2.39+ see [1]

[1] https://sourceware.org/git/?p=glibc.git;a=commit;h=25f1e16ef03a6a8fb1701c4647d46c564480d88c

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agolibssh2: backport fix for CVE-2023-48795
Ross Burton [Mon, 22 Jan 2024 17:20:31 +0000 (17:20 +0000)] 
libssh2: backport fix for CVE-2023-48795

Backport the upstream fix for CVE-2023-48795.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agogrub2: upgrade 2.06 -> 2.12
Anuj Mittal [Tue, 23 Jan 2024 08:47:34 +0000 (16:47 +0800)] 
grub2: upgrade 2.06 -> 2.12

Drop patches that have been upstreamed. Refresh others.

This version dropped extra_deps.lst from the tarball [1] and that leads
to build failures. Restore it in do_configure for now.

[1] https://git.savannah.gnu.org/cgit/grub.git/commit/?id=b835601c7639ed1890f2d3db91900a8506011a8e

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agolibpsl: upgrade 0.21.2 -> 0.21.5
Anuj Mittal [Tue, 23 Jan 2024 08:47:33 +0000 (16:47 +0800)] 
libpsl: upgrade 0.21.2 -> 0.21.5

License-Update: Copyright dates changed

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agocronie: upgrade 1.7.0 -> 1.7.1
Anuj Mittal [Tue, 23 Jan 2024 08:47:32 +0000 (16:47 +0800)] 
cronie: upgrade 1.7.0 -> 1.7.1

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agobluez5: upgrade 5.71 -> 5.72
Anuj Mittal [Tue, 23 Jan 2024 08:47:31 +0000 (16:47 +0800)] 
bluez5: upgrade 5.71 -> 5.72

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agomdadm: Disable ptests
Pavel Zhukov [Tue, 23 Jan 2024 10:27:11 +0000 (11:27 +0100)] 
mdadm: Disable ptests

As agreed upon in the bug triage meeting, disable mdadm ptest for the time being

Related: [Yocto #15181] [Yocto #15159] [Yocto #15308] [Yocto #15309]

Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agolinux-yocto/6.1: drop recipes
Bruce Ashfield [Mon, 22 Jan 2024 14:08:29 +0000 (09:08 -0500)] 
linux-yocto/6.1: drop recipes

6.8-lts + linux-yocto-dev will the active reference kernels in
the upcoming release. We've moved all of the preferred versions
to 6.8, so we can now drop the recipes for 6.1

6.1 will continue to be maintained and updated for released
and supported branches.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agolibxcomposite: Drop obsolete patch
Richard Purdie [Mon, 22 Jan 2024 14:25:49 +0000 (14:25 +0000)] 
libxcomposite: Drop obsolete patch

The patch looks obsolete now we have recipe specific sysroots, drop it.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoxserver-xorg: disable xvfb by default
Ross Burton [Mon, 22 Jan 2024 14:04:06 +0000 (14:04 +0000)] 
xserver-xorg: disable xvfb by default

xvfb has limited use, so to mitigate CVE-2023-5574 out of the box we can
disable the xvfb PACKAGECONFIG.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoxserver-xorg: add PACKAGECONFIG for xvfb
Ross Burton [Mon, 22 Jan 2024 14:04:05 +0000 (14:04 +0000)] 
xserver-xorg: add PACKAGECONFIG for xvfb

Xvfb is pretty niche and has outstanding unsolved security issues, so
let people disable it and add a conditional CVE_STATUS to reflect this.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agocve_check: cleanup logging
Ross Burton [Mon, 22 Jan 2024 14:04:03 +0000 (14:04 +0000)] 
cve_check: cleanup logging

Primarily list the number of patches found, useful when debugging.

Also clean up some bad escaping that caused warnings and use
re.IGNORECASE instead of manually doing case-insenstive rang matches.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agocve_check: handle CVE_STATUS being set to the empty string
Ross Burton [Mon, 22 Jan 2024 14:04:02 +0000 (14:04 +0000)] 
cve_check: handle CVE_STATUS being set to the empty string

Handle CVE_STATUS[...] being set to an empty string just as if it was
not set at all.

This is needed for evaluated CVE_STATUS values to work, i.e. when
setting not-applicable-config if a PACKAGECONFIG is disabled.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agozlib: ignore CVE-2023-6992
Peter Marko [Sat, 13 Jan 2024 18:04:04 +0000 (19:04 +0100)] 
zlib: ignore CVE-2023-6992

This CVE is for iCPE cloudflare:zlib.

Alternative to ignoring would be to limit CVE_PRODUCT, but
historic CVEs already have two - gnu:zlib and zlib:zlib.
So limiting it could miss future CVEs.

Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoxev: Drop diet libx11 related patch
Richard Purdie [Mon, 22 Jan 2024 13:52:15 +0000 (13:52 +0000)] 
xev: Drop diet libx11 related patch

We dropped diet-libx11 back in 2019 in 1a502765db2093e573c9e3ecd6c1bc7621476963.
Drop the patch we don't need from xev to match.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoshaderc: update commit hash to v2023.7
Vincent Davis Jr [Sun, 21 Jan 2024 02:49:29 +0000 (21:49 -0500)] 
shaderc: update commit hash to v2023.7

v2023.7 of google shaderc hash was updated to

https://github.com/google/shaderc/commit/3882b16417077aa8eaa7b5775920e7ba4b8a224d
https://github.com/google/shaderc/releases/tag/v2023.7

Signed-off-by: Vincent Davis Jr <vince@underview.tech>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agoopensbi: bump to 1.4
Thomas Perrot [Sun, 21 Jan 2024 02:10:07 +0000 (18:10 -0800)] 
opensbi: bump to 1.4

This release has:
- Synopsys DesignWare APB GPIO driver
- Zicntr and Zihpm support
- Console print improvements
- Smepmp support
- Simple FDT based syscon regmap driver
- Syscon based reboot and poweroff driver
- Non-contiguous hpm counters
- Smcntrpmf support
- Full sparse hartid support
- IPI improvements
- RFENCE improvements
- Zkr support
- Andes custom PMU support
Overall, this release mainly adds more ISA extensions, drivers and other improvements.

https://github.com/riscv-software-src/opensbi/compare/v1.3.1...v1.4

Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
18 months agolinux-yocto/6.1: update CVE exclusions
Bruce Ashfield [Fri, 19 Jan 2024 21:14:19 +0000 (16:14 -0500)] 
linux-yocto/6.1: update CVE exclusions

Data pulled from: https://github.com/nluedtke/linux_kernel_cves

    1/1 [
        Author: Nicholas Luedtke
        Email: nicholas.luedtke@uwalumni.com
        Subject: Update 15Jan24
        Date: Mon, 15 Jan 2024 12:48:45 -0500

    ]

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>