Bruce Ashfield [Wed, 14 Aug 2019 15:31:33 +0000 (11:31 -0400)]
kconf_check: tweak CONFIG_ regex
As reported in https://bugzilla.yoctoproject.org/show_bug.cgi?id=12563,
the regex that matches valid CONFIG_ options was missing some of the
ones in net/netfilter/ipvs/Kconfig, and hence triggering invalid
option warnings.
By dropping the trailing space on the regex, we'll cover all the cases
for valid option.
Bruce Ashfield [Wed, 14 Aug 2019 15:31:31 +0000 (11:31 -0400)]
linux-yocto: arch/x86/boot: use prefix map to avoid embedded paths
>From the kernel patch:
[
It was observed that the kernel embeds the path in the x86 boot
artifacts.
From https://bugzilla.yoctoproject.org/show_bug.cgi?id=13458:
[
If you turn on the buildpaths QA test, or try a reproducible build, you
discover that the kernel image contains build paths.
$ strings bzImage-5.0.19-yocto-standard |grep tmp/
out of pgt_buf in
/data/poky-tmp/reproducible/tmp/work-shared/qemux86-64/kernel-source/arch/x86/boot/compressed/kaslr_64.c!?
So the __FILE__ shouldn't be using the full path. However
arch/x86/boot/compressed/Makefile has this:
KBUILD_CFLAGS := -m$(BITS) -O2
So that clears KBUILD_FLAGS, removing the -fmacro-prefix-map option.
]
Other architectures do not clear the flags, but instead prune before
adding boot or specific options. There's no obvious reason why x86 isn't
doing the same thing (pruning vs clearing) and no build or boot issues
have been observed.
So we make x86 can do the same thing, and we no longer have embedded paths.
]
This issue has been reported upstream, and a patch submission is
pending, but for now, we'll soak the proposed patch in linux-yocto to
see if any issues are found
Bruce Ashfield [Wed, 14 Aug 2019 15:31:30 +0000 (11:31 -0400)]
kern-tools: Add SPDX license headers to source files
Integrating the following commit:
Add SPDX license headers to source files
Kconfiglib/* were under ISC license before they were imported
here from https://github.com/ulfalizer/Kconfiglib
Adjusting SPDX header to reflect that fact.
tools/* all have some sort of GPLv2 headers; adding SPDX header
to make it obvious.
This address bug #13334 :
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13334
Change-Id: I243f2dd266a398f982798b771e74a67be70ecb52 Signed-off-by: William Bourque <wbourque@gmail.com>
(From OE-Core rev: eb60f1544fcafcfed7baecceec4549c4e86989a3)
Signed-off-by: William Bourque <wbourque@gmail.com> Signen-off-by: Bruce Ashfield <bruce.ashfield@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bruce Ashfield [Wed, 14 Aug 2019 15:31:29 +0000 (11:31 -0400)]
kernel-devsrc: tweak for v5.3+
The 5.3 kernel has two changes that require tweaks to the minimal
kernel-devsrc package.
- 4ce97317f [x86/purgatory: Do not use __builtin_memcpy and __builtin_memset]
This change removes the need for arch/x86/purgatory/string.c and
instead reuses a copy in arch/x86/boot/compressed/, so we can't copy
the file anymore. To support older kernels, we make the copy survive
the non-existence of the file.
- b1663d7e [docs: Kbuild/Makefile: allow check for missing docs at build time]
This change adds the sourceing of Documentation/Kbuild to the top
level Kbuild file. So we now leave the copy of Documention/'s Kbuild
in the devsrc.
Richard Purdie [Thu, 15 Aug 2019 21:54:43 +0000 (22:54 +0100)]
bitbake: hashserv: Ensure we don't accumulate sockets in TIME_WAIT state
This can cause a huge backlog of closing sockets on the server and
in our case we don't really want/need the protection TCP is trying to
give us so work around it.
Robert Yang [Thu, 15 Aug 2019 11:56:51 +0000 (19:56 +0800)]
bitbake: bitbake: server/process: Handle BBHandledException to avoid unexpected exceptions
The parseBaseConfiguration() raises bb.BBHandledException(), but
BitBakeServer() didn't handle it, so we always got unexpected exceptions when
there were errors. For example:
=== Case 1:
* Add "print "hello"' in base.bbclass' def oe_import() function
def oe_import(d):
print "hello"
[snip]
$ bitbake -p
ERROR: Unable to start bitbake server (None)
ERROR: Last 60 lines of server log for this session (/buildarea1/lyang1/test_hy/bitbake-cookerdaemon.log):
File "/buildarea1/lyang1/poky/meta/classes/base.bbclass", line 21
print "hello"
^
SyntaxError: Missing parentheses in call to 'print'
<The first exception>
During handling of the above exception, another exception occurred:
<Tracebacks>
<The second exception>
During handling of the above exception, another exception occurred:
<Tracebacks>
<The third exception>
During handling of the above exception, another exception occurred:
<Tracebacks>
[snip]
Now it looks like:
$ bitbake -p
ERROR: Unable to start bitbake server (None)
ERROR: Server log for this session (/buildarea1/lyang1/test_hy/bitbake-cookerdaemon.log):
ERROR: Error in compiling python function in /buildarea1/lyang1/poky/meta/classes/base.bbclass, line 21:
The code lines resulting in this error were:
0001:def oe_import(d):
*** 0002: print "hello"
0003: import sys
0004:
0005: bbpath = d.getVar("BBPATH").split(":")
0006: sys.path[0:0] = [os.path.join(dir, "lib") for dir in bbpath]
SyntaxError: Missing parentheses in call to 'print' (base.bbclass, line 21)
=== Case 2:
* Add 'HOSTTOOLS += "hello"' to conf/local.conf:
$ bitbake -p
ERROR: Unable to start bitbake server (None)
ERROR: Server log for this session (/buildarea1/lyang1/test_hy/bitbake-cookerdaemon.log):
<Tracebacks>
[snip]
During handling of the above exception, another exception occurred:
[snip]
<Tracebacks>
ERROR: The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed:
hello
The error message is printed by bb.fatal() which raises bb.BBHandledException(),
but BitBakeServer() doesn't handle it, so we got it.
Now it looks like:
ERROR: Unable to start bitbake server (None)
ERROR: Server log for this session (/buildarea1/lyang1/test_hy/bitbake-cookerdaemon.log):
ERROR: The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed:
hello
Richard Purdie [Thu, 15 Aug 2019 07:41:21 +0000 (08:41 +0100)]
bitbake: tests/runqueue: Fix tests
There were paths being accidentally included in some of the hashserv tests. Remove
that and update the hashes so the tests work independently of paths.
SRC_URI was moved from xorg-app-common.inc, since it has hardcoded file
extension (tar.bz2), but upstream stopped publishing tar.bz2 archives
for newer versions of the packages.
Will Page [Tue, 13 Aug 2019 18:37:15 +0000 (11:37 -0700)]
uboot: fixes to uboot-extlinux-config attribute values
The way this class uses overrides to support generation of multiple
sections is subject to two different issues: 1) labels that conflict
with existing override names causing the value for the conflicting label
to be set for all labels, and 2) reusing the override list through each
iteration, prepending each new label to the list of overrides makes
earlier labels' value take precedence over later labels, making later
labels virtually impossible to customize.
The first issue is resolved by removing all label names from overrides
before iterating over labels. The second issue is resolved by
generating a fresh list of overrides with only the current label added.
The current label is also appended to the list of overrides instead of
prepended, which makes it the highest priority override. This is
matches the behavior of devtool-source.bbclass, which similarly
monkey-patches overrides.
insane.bbclass: in file-rdeps do not look into RDEPENDS recursively
Recursive RDEPENDS resolution requires that all of the dependent
recipes' packaging has completed. There is no mechanism to ensure that
and therefore races were observed.
This change effectively requires recipes to list their runtime file
dependencies explicitly rather than have them pulled indirectly.
This may require a bit of fixing in layers, but should result
in a better definition of runtime file dependencies.
Richard Purdie [Tue, 13 Aug 2019 18:48:39 +0000 (19:48 +0100)]
gcc-cross-canadian: Drop obsolete shlibs exclusion
This is a very old change as and be inferred from the name in the comment.
We've since had many changes to pkgdata including separating it
to its own sysroot now so the reasons for this blanket exclusion are
likely long gone.
If the shlib provides were really the problem I'd much rather have
a dedicated variable for that too.
Removing this fixes missing dependencies on nativesdk-libc and other
libs which would then happen automatically.
Khem Raj [Tue, 13 Aug 2019 15:28:35 +0000 (08:28 -0700)]
gcc-runtime: Move content from gcclibdir into libdir
OE does not use the traditional /usr/lib/gcc prefix to store gcc-runtime
it basically is moved into libdir, however some newer files were
installed by newer versions of gcc especially libgomp ( omp.h openacc.h )
into gcclibdir, so we have content in both directories, this confuses
other tools which are trying to guess the gcc installation and its
runtime location, since now we have two directories, the tools either
choose one or other and we get inconsistent behavior, e.g. clang for
aarch64 uses /usr/lib but same clang for riscv64 chose /usr/lib/gcc
This change ensures that OE ends up with single valid location for gcc
runtime files
Khem Raj [Mon, 12 Aug 2019 01:59:45 +0000 (18:59 -0700)]
oeqa/parselogs: grep for exact errors list keywords
Given grep expression looks for error strings regardless of the
semantics and this ends up it flagging things like
***********************
Log: /mnt/a/yoe/build/tmp/work/raspberrypi3_64-yoe-linux-musl/yoe-qt5-wayland-image/1.0-r0/target_logs/weston.log
-----------------------
Central error: EGL_KHR_create_context_no_error EGL_KHR_fence_sync
***********************
here EGL_KHR_create_context_no_error is an API, therefore the check
should be tightened a bit so explicitly look for error words so such
false positives don't show up
This helps in fixing wrong matches e.g. on rpi3 dmesg have strings like
above
Khem Raj [Sat, 10 Aug 2019 18:05:40 +0000 (11:05 -0700)]
oeqa/buildgalculator.py: Add dependency on gtk+3
This helps in skipping this test on images which are not using gtk and
as per the gcalculator website it needs gtk+3 or gtk+2 and libglade as
the build first looks for gtk+3
Changqing Li [Tue, 13 Aug 2019 08:36:22 +0000 (16:36 +0800)]
go-runtime: remove conflict files from -dev packages
fix below error:
file /usr/lib64/go/src/cmd/cgo/zdefaultcc.go conflicts between attempted installs of go-dev-1.12.6-r0.core2_64 and go-runtime-dev-1.12.6-r0.core2_64
file /usr/lib64/go/src/cmd/go/internal/cfg/zdefaultcc.go conflicts between attempted installs of go-dev-1.12.6-r0.core2_64 and go-runtime-dev-1.12.6-r0.core2_64
these 2 files existed in both go-dev and go-runtime-dev
remove it from go-runtime-dev to fix the problem
Richard Purdie [Wed, 14 Aug 2019 14:59:58 +0000 (15:59 +0100)]
bitbake: runqueue: Ensure we clear the stamp cache
When the task hashes change we need to ensure the stampcache is cleared out
else tasks don't rerun when they should as we're basing decisions on stale
cache data.
Richard Purdie [Wed, 14 Aug 2019 14:56:35 +0000 (15:56 +0100)]
bitbake: runqueue: Add further debug information
Further testing shows we should test some extra datastructures to help pinpoint logic
errors more precisely. This adds some further data structure sanity checks.
Richard Purdie [Wed, 14 Aug 2019 13:09:43 +0000 (14:09 +0100)]
bitbake: runqueue: Drop debug statement causing performance issues
This debug statement could result in a long list of tasks which when
repeatedly sent over our IPC, slowed down the builds immensely. Remove
it in favour of other more targeted debugging added recently, bringing
back some lost performance, particularly on builds with large numbers
of tasks.
Richard Purdie [Wed, 14 Aug 2019 10:48:25 +0000 (11:48 +0100)]
bitbake: runqueue: Fix event timing race
The event from the task notifiing of hash equivalency should only be processed
when the task completes. This can otherwise result in a race where a dependent
task may run before the original task completes causing various failures.
To make this work reliably, the code had to be restructured quite a bit.
Richard Purdie [Tue, 13 Aug 2019 23:27:09 +0000 (00:27 +0100)]
bitbake: runqueue: Recompute holdoff tasks from scratch
The changed_setscene variable here is just odd and not needed. Worse,
it could prevent some tasks from being removed from the holdoff tasks
list. The list is being rebuilt and should work as intended just from
the other data, this is a leftover from previous versions of the code
as far as I can tell.
Richard Purdie [Tue, 13 Aug 2019 22:45:55 +0000 (23:45 +0100)]
bitbake: runqueue: Improve scenequeue debugging
Whilst we had good runqueue failure mode debug, it hadn't adapted to the
scenequeue changes. Run the scenequeue sanity tests at the end of
a build and output the results regardless of whether all setscene tasks
completed or not. This *massively* improves the ability to debug runqueue
problems.
Richard Purdie [Tue, 13 Aug 2019 19:14:49 +0000 (20:14 +0100)]
bitbake: runqueue: Fix next_buildable_task performance problem
Looking at the profile information, a lot of time is being spent in
next_buildable_task. This is probably due to the generator expressions
not working well with the empty test.
The easiest way to improve things is to switch to using set manipulations.
We also don't need to update self.buildable the way the original code did
as we don't rely on that anywhere.
Richard Purdie [Mon, 12 Aug 2019 14:56:20 +0000 (15:56 +0100)]
bitbake: cooker: Improve hash server startup code to avoid exit tracebacks
At exit the hashserv code was causing tracebacks as join() wasn't
being called from the thread that started the process. Ensure that
the hashserver is started from the pre_serve hook which is the
final thread the cooker runs in. This avoids the traceback at the
expense of some horrific poking into data stores which will ultimately
need improving through a proper API.
The previous tasks_covered and tasks_notcovered were basically unstable
data structures. We couldn't always tell whether tasks should be covered
or not when trying to repair the sturcture if sstate tasks reran.
In the end its simpler to throw the lists away and rebuild them based upon
current data rather than trying to patch it adhoc. This turns out to be
simpler and much more reliable and I've much more confidence in this code.
Richard Purdie [Sun, 11 Aug 2019 13:59:12 +0000 (14:59 +0100)]
bitbake: runqueue: Fix corruption issue
We need to copy this set, not modify the original else all kinds
of weird and bad things break, mostly from circular references.
We'll not go into how much sleep I lost tracking down the fallout
from this.
Mark Hatle [Thu, 1 Aug 2019 20:36:54 +0000 (16:36 -0400)]
bitbake: layerindexlib: Fix parsing of recursive layer dependencies
[YOCTO #13447]
When running bitbake-layers layerindex-fetch from 'master', there is a
circular dependency between meta-oe and meta-python. This triggered a maximum
recursion depth exception.
To fix the exception, as we walk down a branch (depth first search), we track
the layers we've already seen. If we are about to recurse into a layer we've
already seen we report a warning and then stop recursion.
Wenlin Kang [Fri, 9 Aug 2019 08:39:48 +0000 (16:39 +0800)]
db: add switch for building database verification
Add switch for building database verification, enable
this, it will solve the following issue:
root@qemux86-64:~# db_verify /var/lib/rpm/Packages
db_verify: BDB0571 library build did not include support for database verification BDB5105 Verification of /var/lib/rpm/Packages failed.
mesa: enable glx-tls option in native and nativesdk builds
Before meson conversion this was always enabled; now it needs to
be explicitly switched on.
This un-breaks the virgl headless test in particular, which regressed
with the meson conversion (and the test is not currently run on AB
due to oudated graphics hardware in it).
Bedel, Alban [Wed, 17 Jul 2019 14:20:44 +0000 (14:20 +0000)]
boost: Fix build and enable context and coroutines on aarch64
Like for ARM bjam need some hints about the ABI to properly build on
aarch64. While at it also enable context and coroutine as these are
supported on aarch64.
* this might be related to:
commit d2fcaeb153fdc3f8d7143ea823139f1537055ff1
Author: Douglas Royds <douglas.royds@taitradio.com>
Date: Thu Dec 20 11:59:47 2018 +1300
icecc: Don't generate recipe-sysroot symlinks at recipe-parsing time
* it's still a bit unclear when and why this happends, but I'm seeing
random tasks sometimes failing with:
WARNING: Exception during build_dependencies for set_icecc_env
WARNING: Error during finalise of /build/meta-oe/meta-python/recipes-devtools/python/python-markupsafe_1.0.bb
ERROR: Traceback (most recent call last):
File "/build/bitbake/lib/bb/data_smart.py", line 411, in expandWithRefs
s = __expand_python_regexp__.sub(varparse.python_sub, s)
File "/build/bitbake/lib/bb/data_smart.py", line 136, in python_sub
value = utils.better_eval(codeobj, DataContext(self.d), {'d' : self.d})
File "/build/bitbake/lib/bb/utils.py", line 421, in better_eval
return eval(source, ctx, locals)
File "Var <set_icecc_env>", line 1, in <module>
File "/build/oe-core/meta/classes/icecc.bbclass", line 287, in icecc_get_and_check_tool
link_path = icecc_get_tool_link(t, d)
File "/build/oe-core/meta/classes/icecc.bbclass", line 246, in icecc_get_tool_link
return subprocess.check_output("readlink -f %s" % tool, shell=True).decode("utf-8")[:-1]
File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/usr/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'readlink -f /build/BUILD/work/qemux86-oe-linux/python-markupsafe/1.0-r0/recipe-sysroot-native/usr/bin/x86_64-oe-linux/x86_64-oe-linux-g++' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/build/bitbake/bin/bitbake-worker", line 239, in child
the_data = bb_cache.loadDataFull(fn, appends)
File "/build/bitbake/lib/bb/cache.py", line 327, in loadDataFull
bb_data = self.load_bbfile(virtualfn, appends, virtonly=True)
File "/build/bitbake/lib/bb/cache.py", line 340, in load_bbfile
datastores = parse_recipe(bb_data, bbfile, appends, mc)
File "/build/bitbake/lib/bb/cache.py", line 303, in parse_recipe
bb_data = bb.parse.handle(bbfile, bb_data)
File "/build/bitbake/lib/bb/parse/__init__.py", line 107, in handle
return h['handle'](fn, data, include)
File "/build/bitbake/lib/bb/parse/parse_py/BBHandler.py", line 142, in handle
return ast.multi_finalize(fn, d)
File "/build/bitbake/lib/bb/parse/ast.py", line 386, in multi_finalize
finalize(fn, d)
File "/build/bitbake/lib/bb/parse/ast.py", line 351, in finalize
bb.parse.siggen.finalise(fn, d, variant)
File "/build/bitbake/lib/bb/siggen.py", line 147, in finalise
taskdeps = self._build_data(fn, d)
File "/build/bitbake/lib/bb/siggen.py", line 118, in _build_data
tasklist, gendeps, lookupcache = bb.data.generate_dependencies(d)
File "/build/bitbake/lib/bb/data.py", line 388, in generate_dependencies
deps[dep], values[dep] = build_dependencies(dep, keys, shelldeps, varflagsexcl, d)
File "/build/bitbake/lib/bb/data.py", line 317, in build_dependencies
value, parsedvar = d.getVarFlag(key, "_content", False, retparser=True)
File "/build/bitbake/lib/bb/data_smart.py", line 802, in getVarFlag
parser = self.expandWithRefs(value, cachename)
File "/build/bitbake/lib/bb/data_smart.py", line 424, in expandWithRefs
raise ExpansionError(varname, s, exc).with_traceback(tb) from exc
File "/build/bitbake/lib/bb/data_smart.py", line 411, in expandWithRefs
s = __expand_python_regexp__.sub(varparse.python_sub, s)
File "/build/bitbake/lib/bb/data_smart.py", line 136, in python_sub
value = utils.better_eval(codeobj, DataContext(self.d), {'d' : self.d})
File "/build/bitbake/lib/bb/utils.py", line 421, in better_eval
return eval(source, ctx, locals)
File "Var <set_icecc_env>", line 1, in <module>
File "/build/oe-core/meta/classes/icecc.bbclass", line 287, in icecc_get_and_check_tool
link_path = icecc_get_tool_link(t, d)
File "/build/oe-core/meta/classes/icecc.bbclass", line 246, in icecc_get_tool_link
return subprocess.check_output("readlink -f %s" % tool, shell=True).decode("utf-8")[:-1]
File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/usr/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
bb.data_smart.ExpansionError: Failure expanding variable set_icecc_env, expression was if [ "${@use_icecc(bb, d)}" = "no" ]
then
return
fi
ICECC_VERSION="${@icecc_version(bb, d)}"
if [ "x${ICECC_VERSION}" = "x" ]
then
bbwarn "Cannot use icecc: could not get ICECC_VERSION"
return
fi
ICE_PATH="${@icecc_path(bb, d)}"
if [ "x${ICE_PATH}" = "x" ]
then
bbwarn "Cannot use icecc: could not get ICE_PATH"
return
fi
ICECC_BIN="${@get_icecc(d)}"
if [ -z "${ICECC_BIN}" ]; then
bbwarn "Cannot use icecc: icecc binary not found"
return
fi
if [ -z "$(which patchelf patchelf-uninative)" ]; then
bbwarn "Cannot use icecc: patchelf not found"
return
fi
# Create symlinks to icecc in the recipe-sysroot directory
mkdir -p ${ICE_PATH}
if [ -n "${KERNEL_CC}" ]; then
compilers="${@get_cross_kernel_cc(bb,d)}"
else
compilers="x86_64-oe-linux-gcc x86_64-oe-linux-g++"
fi
for compiler in $compilers; do
ln -sf ${ICECC_BIN} ${ICE_PATH}/$compiler
done
ICECC_CC="${@icecc_get_and_check_tool(bb, d, "gcc")}"
ICECC_CXX="${@icecc_get_and_check_tool(bb, d, "g++")}"
# cannot use icecc_get_and_check_tool here because it assumes as without target_sys prefix
ICECC_WHICH_AS="${@bb.utils.which(os.getenv('PATH'), 'as')}"
if [ ! -x "${ICECC_CC}" -o ! -x "${ICECC_CXX}" ]
then
bbwarn "Cannot use icecc: could not get ICECC_CC or ICECC_CXX"
return
fi
ICE_VERSION=`$ICECC_CC -dumpversion`
ICECC_VERSION=`echo ${ICECC_VERSION} | sed -e "s/@VERSION@/$ICE_VERSION/g"`
if [ ! -x "/build/BUILD/work/qemux86-oe-linux/python-markupsafe/1.0-r0/recipe-sysroot-native/usr/bin/icecc-create-env" ]
then
bbwarn "Cannot use icecc: invalid ICECC_ENV_EXEC"
return
fi
ICECC_AS="`${ICECC_CC} -print-prog-name=as`"
# for target recipes should return something like:
# /OE/tmp-eglibc/sysroots/x86_64-linux/usr/libexec/arm920tt-oe-linux-gnueabi/gcc/arm-oe-linux-gnueabi/4.8.2/as
# and just "as" for native, if it returns "as" in current directory (for whatever reason) use "as" from PATH
if [ "`dirname "${ICECC_AS}"`" = "." ]
then
ICECC_AS="${ICECC_WHICH_AS}"
fi
if [ ! -f "${ICECC_VERSION}.done" ]
then
mkdir -p "`dirname "${ICECC_VERSION}"`"
# the ICECC_VERSION generation step must be locked by a mutex
# in order to prevent race conditions
if flock -n "${ICECC_VERSION}.lock" \
/build/BUILD/work/qemux86-oe-linux/python-markupsafe/1.0-r0/recipe-sysroot-native/usr/bin/icecc-create-env "${ICECC_CC}" "${ICECC_CXX}" "${ICECC_AS}" "${ICECC_VERSION}"
then
touch "${ICECC_VERSION}.done"
elif ! wait_for_file "${ICECC_VERSION}.done" 30
then
# locking failed so wait for ${ICECC_VERSION}.done to appear
bbwarn "Timeout waiting for ${ICECC_VERSION}.done"
return
fi
fi
# Don't let ccache find the icecream compiler links that have been created, otherwise
# it can end up invoking icecream recursively.
export CCACHE_PATH="$PATH"
export CCACHE_DISABLE="1"
Kai Kang [Thu, 8 Aug 2019 15:45:13 +0000 (11:45 -0400)]
e2fsprogs: 1.44.5 -> 1.45.3
Update e2fsprogs from 1.44.5 to 1.45.3:
* rebase ptest.patch
* remove 0001-create_inode-fix-copying-large-files.patch which merged by upstream
* add new sub-package e2fsprogs-e2scrub and disable cron support
Richard Purdie [Fri, 9 Aug 2019 09:41:55 +0000 (10:41 +0100)]
python3-pygobject: Add missing pkgutil RDEPENDS
With the removal of setuputils, there was a missing runtime dependency on pkgutil
which was breaking the selftests for gobject-introspection at runtime. Fix this.
Remove it from RDEPENDS to shrink the runtime dependency chain, as
setuptools includes scripts to download other packages, which might not
be desired on a minimal image.
Jason Wessel [Thu, 8 Aug 2019 20:24:00 +0000 (13:24 -0700)]
dpkg: Provide update-alternative for start-stop-daemon
When using DISTRO_FEATURE usrmerge, busybox and something that
requires dpkg-start-stop there is a conflict which prints the
following error.
===
ERROR: core-image-minimal-1.0-r1 do_rootfs: Postinstall scriptlets of ['busybox'] have failed. If the intention is to defer them to first boot,
then please place them into pkg_postinst_ontarget_${PN} ().
Deferring to first boot via 'exit 1' is no longer supported.
Details of the failure are in /build/tmp-glibc/work/intel_x86_64-linux/core-image-minimal/1.0-r1/temp/log.do_rootfs.
ERROR: core-image-minimal-1.0-r1 do_rootfs: Function failed: do_rootfs
===
Looking at the log file, it shows the problem:
update-alternatives: Error: not linking /build/tmp-glibc/work/intel_x86_64-linux/core-image-minimal/1.0-r1/rootfs/usr/sbin/start-stop-daemon to /usr/bin/busybox.nosuid since /build/tmp-glibc/work/intel_x86_64-linux/core-image-minimal/1.0-r1/rootfs/usr/sbin/start-stop-daemon exists and is not a link
===
The solution is to add an update-alternative to start-stop-daemon for dpkg.
As for the removal of the bb.warn, if the size is not specified, it is
not something to warn the end user about. Some of my default images
make use of the head room + a computed directory size or while generating
images.
Jason Wessel [Thu, 8 Aug 2019 20:23:58 +0000 (13:23 -0700)]
wic: Add partition type for msdos partition tables
In order to create a msdos partition table disk image that can auto
expand after the image is copied to an SD card, wic needs the ability
to have a primary partition as the last entry. The desired use case
is to be able to create an A/B update partition image scheme with a
/var volume that can be auto expanded to the remainder of the SD card
at run time.
The typical .wks file will look similar to the following:
With the patch applied a primary partition can be created at the end
of the image which can be expanded to fill the free space on the media
where the image has been copied, which looks like:
Disk boot.img: 890940s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Jason Wessel [Thu, 8 Aug 2019 20:23:57 +0000 (13:23 -0700)]
image_types_wic.bbclass: Copy the .wks and .env files to deploy image dir
When using a .wks.in file, the only place that the generated .wks file
exists in the tmp/work area. A copy should be left behind in the
deploy directory so that you can easily run the wic tool to
re-generate or modify a new image without running bitbake. Custom
.wks.in files can reference any number of bitbake variables, so it is
important to save the result.
below is an example of using the generated .wks file in the deploy
area. The full name of my generated .wks file was
core-image-minimal-ostree-uboot-ab.wks, but since you usually only
have a single .wks file per image you can use a wild card like:
Ricardo Neri [Mon, 5 Aug 2019 22:18:23 +0000 (18:18 -0400)]
runqemu: Add support to handle EnrollDefaultKeys PK/KEK1 certificate
The EnrollDefaultKeys.efi application (distributed in ovmf-shell-image)
expects the hypervisor to provide a Platform Key and first Key Exchange
Key certificate.
For QEMU, this is done by adding an OEM string in the Type 11 SMBIOS
table. The string contains the EnrollDefaultKeys application GUID followed
by the certificate string. For now, the string is passed in the command
line until QEMU understands OEM strings from regular files (please see
https://bugs.launchpad.net/qemu/+bug/1826200).
If runqemu detects it is given an OVMF binary with support for Secure Boot
(i.e., ovmf.secboot* binaries), extract the certificate string from the
OvmfPkKek1.pem certificate and modify the command-line parameters to
provide the key. Such certificate is created when building OVMF with
support for Secure Boot.
ovmf: Generate test Platform key and first Key Exchange Key
Commit from EDK2 be9470b3c91f ("OvmfPkg/EnrollDefaultKeys: enroll
PK/KEK1 from the Type 11 SMBIOS table") mandates that a Platform Key
and first Key Exchange Key certificate is provided to the
EnrollDefaultKeys application.
Previously, the application was using a hard-coded certificate
from Red Hat embedded in the application.
Create a certificate that can QEMU can subsequently pass to
EnrollDefaultKeys when running qemu-shell-image.
Cc: Ross Burton <ross.burton@intel.com> Cc: Patrick Ohly <patrick.ohly@intel.com> Cc: Alexander Kanavin <alex.kanavin@gmail.com>
(From OE-Core rev: daaf9d7bd8c3586609ab0eccf49af38dbdb0b02e)
Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update OVMF to edk2-stable201905. Since the last update, several things
have changed. Many of the patches we were carrying have now been taken
upstream in EDK2 or become obsolete. See below for details.
Also, as of commit 0c1ffb9504c3("CryptoPkg: Adding OpenSSL as one
submodule of EDKII repo"), openssl is not embedded into EDK2 using a
patching script but a git submodule. Then, use the gitsm bitbabke fetcher
and drop the extra SRC_URI from openssl when building with secureboot
enabled. Also remove all related variables.
OVMF switched to BSD+Patent license as detailed in
https://bugzilla.tianocore.org/show_bug.cgi?id=1373. Hence, update
LIC_FILES_CHKSUM accordingly to reflect this change.
Patches are updated as follows:
Drop 0001-ia32-Dont-use-pie.patch as it is implemented in upstream EDK2
in commits are 11d0cd23dd1b ("BaseTools/tools_def IA32: drop -no-pie linker
option for GCC49" and c25d3905523a ("BaseTools/tools_def IA32: disable PIE
code generation explicitly") for the applicable GCC versions.
Rebase 0003-BaseTools-makefile-adjust-to-build-in-under-bitbake.patch
to edk2-stable201905.
Drop VfrCompile-increase-path-length-limit.patch as it has been taken
upstream in EDK2 in commit ba78032bc8c9f("BaseTools/VfrCompile: Remove the
MAX_PATH limitation").
Rebase no-stack-protector-all-archs.patch to keep behavior on not using
stack protector on all archs.
Drop 0001-BaseTools-header.makefile-add-Wno-stringop-truncatio.patch,
0002-BaseTools-header.makefile-add-Wno-restrict.patch, and
0003-BaseTools-header.makefile-revert-gcc-8-Wno-xxx-optio.patch. These
patches have been taken in upstream EDK2 in commits 9222154ae7b3("BaseTools
/header.makefile: add "-Wno-restrict"), 1d212a83df0e("BaseTools/
header.makefile: add "-Wno-stringop-truncation"), and 777f4aa083e9
("BaseTools/header.makefile: revert gcc-8 '-Wno-xxx' options on OSX"),
respectively.
Also, drop 0004-BaseTools-GenVtf-silence-false-stringop-overflow-war.patch.
GenVtf has been removed from EDK2 in commit 64ab2c82e8f6("BaseTools:
Remove GenVtf"). Also, this patch had been taken in EDK2 upstream in
commit 9de306701312("BaseTools/GenVtf: silence false 'stringop-overflow'
warning with memcpy()").
Drop patch 0007-OvmfPkg-EnrollDefaultKeys-application-for-enrolling-.patch
as it also has been taken by EDK2.
Patches 0002-ovmf-update-path-to-native-BaseTools.patch and
0004-ovmf-enable-long-path-file.patch did not need any update.
do_rootfs failed with error:
file /usr/lib64/grub/arm64-efi/acpi.module conflicts between attempted installs of grub-2.02-r0.aarch64 and grub-efi-2.02-r0.aarch64
file /usr/lib64/grub/arm64-efi/adler32.module conflicts between attempted installs of grub-2.02-r0.aarch64 and grub-efi-2.02-r0.aarch64
...
file /usr/lib64/grub/arm64-efi/zfsinfo.module conflicts between attempted installs of grub-2.02-r0.aarch64 and grub-efi-2.02-r0.aarch64
For arm64, grub and grub-efi both compiled GRUBD2 with platform
arm64-efi, arm64 only support platform efi. So both packages have
almost same folder and content /usr/lib64/grub/arm64-efi/*
if Machine set to qemux86_64, do_rootfs will not have this problem,
since for grub, it compiled with platform i386-pc, for grub-efi,
compiled with platform x86-64-efi.
While installing grub and grub-efi, there are conflict files
in ${sysconfdir} ${datadir} ${bindir} ${sbindir}.
- Since all of the conflicted files are tools which is
common for grub and grub-efi, we split them (except
grub-editenv) to grub-common in grub.
common tools for grub and grub-efi are installed into grub-common,
for aarch64, also shipped libdir into this common packages.
in this way, for qemuarm64, package grub is empty, and package grub-efi
use lib in package grub-common
Tim Blechmann [Thu, 8 Aug 2019 04:58:33 +0000 (12:58 +0800)]
deb: allow custom dpkg command
trying to reduce the turnaround times for local builds, i'm trying to
find a way to inject `-Zgzip -z1` into the dkpg-deb command line.
attached patch introduces a `DPKG_BUILDCMD` variable for this purpose.
(i'm not very familiar with the whole bitbake infrastructure, so i'm
wondering: do i have to add it to the `vardeps` of do_package_deb?)
tia,
tim
>From 62d4930d307d5d07844889001e8a1c3111b72b98 Mon Sep 17 00:00:00 2001
From: Tim Blechmann <tim@klingt.org>
Date: Thu, 8 Aug 2019 06:48:54 +0200
Subject: [PATCH] package_deb: allow dpkg-deb to be customized via
DPKG_BUILDCMD
the command line to invoke `dpkg-deb` is hardcoded. there are certain use
cases where we want to tweak how debian packages are compiled: e.g. the
default uses xz compression, which is rather CPU intensive. for local
builds one might want to pass `-Zgzip -z1` to favor speed over compression
ratio.
we therefore introduce a `DPKG_BUILDCMD` variable which allows downstream
code to customize how dpkg-deb is executed
Robert Yang [Wed, 7 Aug 2019 09:50:49 +0000 (17:50 +0800)]
bitbake: knotty: Fix for the Second Keyboard Interrupt
Fixed:
$ rm -fr tmp-glibc/cache/default-glibc/qemux86/x86_64/bb_cache.dat* ; bitbake -p
Press the first Ctrl-C when the parsing process is at about 50%:
Keyboard Interrupt, closing down...
Then presss the second Ctrl-C:
File "/path/to/bitbake/bitbake/lib/bb/ui/knotty.py", line 619, in main
event = eventHandler.waitEvent(0.25)
File "/path/to/bitbake/lib/bb/server/process.py", line 591, in waitEvent
self.eventQueueNotify.wait(delay)
File "/usr/lib/python3.5/threading.py", line 549, in wait
signaled = self._cond.wait(timeout)
File "/usr/lib/python3.5/threading.py", line 297, in wait
gotit = waiter.acquire(True, timeout)
KeyboardInterrupt
Capture the second KeyboardInterrupt during stateShutdown is running can fix
the problem. There may be still tracebacks for the third KeyboardInterrupt, but
I'm leaning to not fix it since we aimed for supporting 2 KeyboardInterrupts
only.
Robert Yang [Wed, 7 Aug 2019 09:50:48 +0000 (17:50 +0800)]
bitbake: cooker: Cleanup the queue before call process.join()
Fixed:
$ rm -fr tmp-glibc/cache/default-glibc/qemux86/x86_64/bb_cache.dat* ; bitbake -p
Press *one* Ctrl-C when the parsing process is at about 50%, then the processes
are not exited:
Keyboard Interrupt, closing down...
Timeout while waiting for a reply from the bitbake server
local.conf.sample: do not add sdl to nativesdk qemu config
This is unnecessary as the recipe itself already does it:
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-devtools/qemu/qemu_4.0.0.bb?id=1a4e4fb6b0a9d54641bd4193e95311d1f822a9ca#n21
EXCLUDE_PACKAGES_FROM_SHLIBS is too broad: it suppresses both generation
of required and provided shlibs. We need to suppress only the provided shlibs
(to avoid clashes with the main package providing the same shlibs),
and run the required shlib dependencies generator as usual.
When the code is compiled with "-fstack-protector-strong
-D_FORTIFY_SOURCE=2", everytime ftpfd is asked for a non existent file,
it crashes with the following error:
*** buffer overflow detected ***:
Aborted
This seems to be a bug/feature of gcc. A bug has been open on their
bugzilla, and also inetutils have been posted with the proposed patch.
Without this patch, pxelinux fails to boot because it keeps asking the
server for the pxelinux.cfg/00-01-02-03-04 and never jumps to /default.
projects / thirdparty / openembedded / openembedded-core-contrib.git / log
