Richard Purdie [Thu, 18 Jul 2019 11:46:12 +0000 (12:46 +0100)]
staging: Drop clean_recipe_sysroot
With recent changes to runqueue, this fuction is unsafe as setscene tasks can run
at the same time as normal ones and doing things before do_fetch no longer
offers any guarantees.
There is other code which cleans out things from the sysroots as tasks rerun so
we should rely upon that instead.
Scott Rifenbark [Thu, 27 Jun 2019 21:03:59 +0000 (14:03 -0700)]
dev-manual: Provided proper link title
In the "Enabling GObject Introspection Support" section, a link
existed to the "Known Issues" section further down in the
chapter. The target of the link was missing and appeared as the
placeholder "xxx". I put in the proper link.
Scott Rifenbark [Tue, 25 Jun 2019 19:50:14 +0000 (12:50 -0700)]
sdk-manual: Updated devtool to talk about oe-local-files.
Fixes YOCTO #13079
Bug is about making sure the section on the devtool command
talks about oe-local-files. Two devtool commands (modify
and upgrade) needed to mention the role of the oe-local-files
area and how it is used during these commands. I updated the
appropriate sections:
* Use devtool modify to Modify the Source of an Existing Component
* Use devtool upgrade to Create a Version of the Recipe that
Supports a Newer Version of the Software.
Richard Purdie [Wed, 17 Jul 2019 21:46:12 +0000 (22:46 +0100)]
bitbake: runqueue: Fix non setscene tasks targets being lost
If you specify both setscene and non-setscene tasks on the commandline, the
non-setscene tasks could be missed, e.g. "bitbake X:do_patch X:do_populate_sysroot"
and do_patch would fail to run.
Alistair Francis [Wed, 17 Jul 2019 20:51:49 +0000 (13:51 -0700)]
opensbi: Fix installed-vs-shipped warning
Fix the following warning by just deleting the files:
WARNING: opensbi-0.4-r0 do_package: QA Issue: opensbi: Files/directories were installed but not shipped in any package:
/lib
/lib/libsbiutils.a
/lib/libsbi.a
Error info:
getrlimit03.c:104: FAIL: __NR_prlimit64(0) had rlim_cur = ffffffffffffffff but __NR_getrlimit(0) had rlim_cur = 7fffffff
According to kernel code: [arch/mips/include/uapi/asm/resource.h]
RLIM_INFINITY is set to 0x7fffffffUL instead of ULONG_MAX on mips32.
/*
* SuS says limits have to be unsigned.
* Which makes a ton more sense anyway,
* but we keep the old value on MIPS32,
* for compatibility:
*/
#ifndef __mips64
# define RLIM_INFINITY 0x7fffffffUL
#endif
Adding conditional statement about mips to fix this.
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Wed, 17 Jul 2019 10:45:36 +0000 (11:45 +0100)]
cve-check: allow comparison of Vendor as well as Product
Some product names are too vague to be searched without also matching the
vendor, for example Flex could be the parser compiler we ship, or Adobe Flex, or
Apache Flex, or IBM Flex.
If entries in CVE_PRODUCT contain a colon then split it as vendor:product to improve the search.
Also don't use .format() to construct SQL as that can lead to security
issues. Instead, use ? placeholders and lets sqlite3 handle the escaping.
Andrej Valek [Wed, 17 Jul 2019 13:25:44 +0000 (15:25 +0200)]
oe/copy_buildsystem: move layer into layers directory
Layers could be located outside from poky but inside the build directory.
This case should be covered in eSDK.
meta-abc
meta-def/meta-ghi
meta-def/poky
meta-def/meta-oe/meta-oe
...
It should take all enabled layers and put them into 'layers' dir during
build-time with respecting new relative path to poky.
layers/meta-abc
layers/meta-ghi
layers/poky
layers/meta-oe/meta-oe
...
Mikko Rapeli [Wed, 17 Jul 2019 09:08:36 +0000 (12:08 +0300)]
busybox: enable unicode support
While creating and deleting files with unicode or other
encodings works, it's annoying when ls and other core utils
show questionmarks instead of the unicode characters.
In 2019, it's quite common that users of embedded devices
based on yocto need unicode support. Debugging a box with
unicode encoded file names is a bit annoying when core utils
from busybox don't support them.
The unicode config fragment has the same config as Debian in their
deb and udeb builds of version 1:1.30.1-4.
If developers do not want this or other default yocto features in busybox,
or optimize the configuration for size, then they likely run a completely
custom configuration. Thus I think it's safe to enable unicode support
by default.
u-boot.bin with dtb & signature should be placed in ${B} so that
it can be deployed by u-boot as expected. Otherwise, the version
without signature is installed.
Chee Yang Lee [Wed, 17 Jul 2019 02:02:42 +0000 (10:02 +0800)]
wic: add support for kernel with initramfs bundled
when INITRAMFS_IMAGE_BUNDLE and INITRAMFS_IMAGE are set, wic should
look for kernel with initramfs image bundled.
Include required variable MACHINE, INITRAMFS_IMAGE_BUNDLE,
INITRAMFS_IMAGE, INITRAMFS_LINK_NAME and KERNEL_IMAGETYPE in WICVARS.
No longer require default value for variable kernel as KERNEL_IMAGETYPE
is not optional variable and included in WICVARS.
image_types_wic to inherit kernel-artifact-names to obtain default
INITRAMFS_LINK_NAME when INITRAMFS_IMAGE_BUNDLE are set.
update wic.Wic2.test_image_env test case to filter optional
variable INITRAMFS_LINK_NAME, INITRAMFS_IMAGE and INITRAMFS_IMAGE_BUNDLE.
Some distributions for various reasons (like for example mounting a
tmpfs over /var at runtime) can't use /var/lib to store the opkg
metadata, so a different path is required to have a functioning
package manager.
${localstatedir} can't be modified to something other than the
hardcoded value in bitbake.conf because other recipes depending on it
will fail to install.
So the only recourse, which is also the least invasive, is to allow
distros to overwrite the OPKGLIBDIR variable just like they are also
allowed to overwrite OPKGBUILDCMD.
Ross Burton [Tue, 16 Jul 2019 12:47:21 +0000 (13:47 +0100)]
glibc: exclude child recipes from CVE scanning
As glibc will be scanned for CVEs, we don't need to scan glibc-locale,
glibc-mtrace, and glibc-scripts which are all separate recipes for technical
reasons.
Exclude the recipes by setting CVE_PRODUCT in the recipe, instead of using the
global whitelist.
CVE-2014-2524 is a readline CVE that was fixed in 6.3patch3 onwards, but the
tooling wasn't able to detect this version. As we now ship readline 8 we don't
need to manually whitelist it, and if we did then the whitelisting should be in
the readline recipe.
Kai Kang [Mon, 15 Jul 2019 10:20:16 +0000 (06:20 -0400)]
webkitgtk: set incomptible with tune mips
It fails to compile webkit when default tune is 'mips':
| .../tmp-glibc/work/mips-wrs-linux/webkitgtk/2.24.2-r0/webkitgtk-2.24.2
| /Source/JavaScriptCore/assembler/MacroAssemblerMIPS.h:418:23:
| error: static assertion failed: CLZ opcode is not available for this ISA
devtool: provide support for devtool menuconfig command
All packages that support the menuconfig task will be able to run
devtool menuconfig command. This would allow the user to modify the
current configure options and create a config fragment which can be
added to a recipe using devtool finish.
1. The patch checks if devtool menuconfig command is called for a valid
package.
2. It checks for oe-local-files dir within source and creates one if
needed, this directory is needed to store the final generated config
fragment so that devtool finish can update the recipe.
3. Menuconfig command is called for users to make necessary changes.
After saving the changes, diffconfig command is run to generate the
fragment.
Run devtool finish to update the recipe by appending the config fragment
to SRC_URI and place a copy of the fragment within the layer where the
recipe resides.
Ex: devtool finish linux-yocto meta
Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool/standard.py: Create a copy of kernel source within work-shared if not present
If kernel source is not already downloaded i.e staging kernel dir is
empty, place a copy of the source when the user runs devtool modify
linux-yocto. This way the kernel source is available for other packages
that use it.
Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool/standard.py: Update devtool modify to copy source from work-shared if its already downloaded
In the regular devtool modify flow, the kernel source is fetched by
running do_fetch task. This is an overhead in time and space.
This patch updates modify command to check if the kernel source is
already downloaded. If so, then instead of calling do_fetch, copy the
source from work-shared to devtool workspace by creating hard links
else run the usual devtool modify flow and call do_fetch task.
Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 11 Jul 2019 16:05:19 +0000 (17:05 +0100)]
bitbake: runqueue: Optimise multiconfig with overlapping setscene
Currently if a multiconfig build contains different configurations which
have overlapping sstate artefacts, it will build them multiple times.
This is clearly suboptimal and not what users want/expect.
This adds code to detect this and stall all but one of the setscne tasks
so that once its built, it can be found by the other tasks.
We take care to iterate the multiconfigs in order so try and avoid
dependency loops. We also match on PN+taskname+taskhash since this is
what we know sstate in OE-Core would use. There are some tasks even within
a multiconfig which match hashes (mostly do_populate_lic tasks) but those
have a much higher chance of circular dependency so aren't work attempting
to optimise.
If a deadlock does occur the build will be slower but there is code to
unbreak such a deadlock so it hopefully doens't break anything.
Comments are injected into the test tasks so they have different task
hashes and a new test for this optimisation is added.
Briefly:
Brazil no longer observes DST.
'zic -b slim' outputs smaller TZif files; please try it out.
Palestine's 2019 spring-forward transition was on 03-29, not 03-30.
Changes to future timestamps
Brazil has canceled DST and will stay on standard time indefinitely.
(Thanks to Steffen Thorsen, Marcus Diniz, and Daniel Soares de
Oliveira.)
Predictions for Morocco now go through 2087 instead of 2037, to
work around a problem on newlib when using TZif files output by
zic 2019a or earlier. (Problem reported by David Gauchard.)
Changes to past and future timestamps
Palestine's 2019 spring transition was 03-29 at 00:00, not 03-30
at 01:00. (Thanks to Sharef Mustafa and Even Scharning.) Guess
future transitions to be March's last Friday at 00:00.
Changes to past timestamps
Hong Kong's 1941-06-15 spring-forward transition was at 03:00, not
03:30. Its 1945 transition from JST to HKT was on 11-18 at 02:00,
not 09-15 at 00:00. In 1946 its spring-forward transition was on
04-21 at 00:00, not the previous day at 03:30. From 1946 through
1952 its fall-back transitions occurred at 04:30, not at 03:30.
In 1947 its fall-back transition was on 11-30, not 12-30.
(Thanks to P Chan.)
Changes to past time zone abbreviations
Italy's 1866 transition to Rome Mean Time was on December 12, not
September 22. This affects only the time zone abbreviation for
Europe/Rome between those dates. (Thanks to Stephen Trainor and
Luigi Rosa.)
Changes affecting metadata only
Add info about the Crimea situation in zone1970.tab and zone.tab.
(Problem reported by Serhii Demediuk.)
Changes to code
zic's new -b option supports a way to control data bloat and to
test for year-2038 bugs in software that reads TZif files.
'zic -b fat' and 'zic -b slim' generate larger and smaller output;
for example, changing from fat to slim shrinks the Europe/London
file from 3648 to 1599 bytes, saving about 56%. Fat and slim
files represent the same set of timestamps and use the same TZif
format as documented in tzfile(5) and in Internet RFC 8536.
Fat format attempts to work around bugs or incompatibilities in
older software, notably software that mishandles 64-bit TZif data
or uses obsolete TZ strings like "EET-2EEST" that lack DST rules.
Slim format is more efficient and does not work around 64-bit bugs
or obsolete TZ strings. Currently zic defaults to fat format
unless you compile with -DZIC_BLOAT_DEFAULT=\"slim\"; this
out-of-the-box default is intended to change in future releases
as the buggy software often mishandles timestamps anyway.
zic no longer treats a set of rules ending in 2037 specially.
Previously, zic assumed that such a ruleset meant that future
timestamps could not be predicted, and therefore omitted a
POSIX-like TZ string in the TZif output. The old behavior is no
longer needed for current tzdata, and caused problems with newlib
when used with older tzdata (reported by David Gauchard).
zic no longer generates some artifact transitions. For example,
Europe/London no longer has a no-op transition in January 1996.
Changes to build procedure
tzdata.zi now assumes zic 2017c or later. This shrinks tzdata.zi
by a percent or so.
Changes to documentation and commentary
The Makefile now documents the POSIXRULES macro as being obsolete,
and similarly, zic's -p POSIXRULES option is now documented as
being obsolete. Although the POSIXRULES feature still exists and
works as before, in practice it is rarely used for its intended
purpose, and it does not work either in the default reference
implementation (for timestamps after 2037) or in common
implementations such as GNU/Linux (for contemporary timestamps).
Since POSIXRULES was designed primarily as a temporary transition
facility for System V platforms that died off decades ago, it is
being decommissioned rather than institutionalized.
New info on Bonin Islands and Marcus (thanks to Wakaba and Phake
Nick).
The options in ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS} are already passed
via ${CC}/${CXX} and there is no reason to pass them a second time. Thus
we can remove MESON_TOOLCHAIN_ARGS. And when it is removed, the other
MESON_*_ARGS variables revert to the standard CFLAGS, CXXFLAGS and
LDFLAGS, so just use them directly instead.
Apart from the obvious improvement with not passing a lot of options
twice, this also solves a problem where -pie would be passed on the
command line in a way that it would prevent building any dynamic
libraries using meson if using a toolchain that is not built with
--enable-default-pie and if security_flags.inc is used.
Before commit[80d17497b7 mdadm: improve the run-ptest],
the mdadm ptest just run some tests without
"--keep-going" option. After the option added in
commit 80d17497b7, all test cases have chance to
be called.
But the logic in mdadm upstream commit
[e2a8e9d tests: wait for complete rebuild in integrity checks]
will make the test enter infinite loop especially in
qemu env as commit e2a8e9d update the logic from
"check wait" to "check state 'U*'" for testcase
tests/01r5integ and tests/01raid6integ. Considering all
other cases still use "check wait" logic, so revert e2a8e9d
to make tests/01r5integ and tests/01raid6integ also use
"check wait" logic to avoid the infinite loop.
Richard Purdie [Wed, 10 Jul 2019 13:26:37 +0000 (14:26 +0100)]
bitbake: tests: Add initial scenario based test for runqueue
We need some tests for runqueue, its been something which has been hard to test
for a long time. Add some dummy metadata to allow this, mirroring the OE
structure in spirit.
Richard Purdie [Sat, 6 Jul 2019 15:16:54 +0000 (16:16 +0100)]
bitbake: runqueue: Alter setscenewhitelist handling
Since there is now parallel execution of setscene and normal tasks, the way
setscenewhitelist handling worked can't function the way it did. Paul and I
never liked its error output anyway.
This code tries a different approach, checking the task at execution time
but printing the uncovered task list.
This code may need improvement after real world usage but can
work with the new task flows.
Richard Purdie [Wed, 3 Jul 2019 23:14:02 +0000 (00:14 +0100)]
bitbake: runqueue: Merge the queues and execute setscene and normal tasks in parallel
This is the serious functionality change in this runqueue patch series of
changes.
Rather than two phases of execution, the scenequeue setscene phase, followed
by normal task exeuction, this change allows them to execute in parallel
together.
To do this we need to handle marking of tasks as covered/uncovered in a piecemeal
fashion on a task by task basis rather than in a single function.
The code will block normal task exeuction until any setcene task which could
cover that task is executed and its status is known. There is a slight
optimisation which could be possible here at the risk of races but that
doesn't seem worthwhile.
The state engine isn't entirely cleaned up in this commit (see FIXME) and
the setscenewhitelist functionality is broken by it (see following patches)
however its good enough to test with normal workflows.
Richard Purdie [Wed, 3 Jul 2019 22:12:19 +0000 (23:12 +0100)]
bitbake: runqueue: Clarify scenequeue_covered vs. tasks_covered
It wasn't clear whether the variable contained just setscene covered
tasks or all covered tasks. We need both sets of data so lets just have
two clearly named variables.
The existing code to compute the 'unskippable' setscene task list is overcomlicated,
so replace it with something functionally equivalent but simpler and more efficient.
We don't need to process all chains, just the 'top' ones to the first setscene tasks.
Richard Purdie [Wed, 3 Jul 2019 17:34:07 +0000 (18:34 +0100)]
bitbake: event/runqueue: Drop StampUpdate event, its pointless/unused
Whilst this class has existed for years, it doesn't have any
users and has a questionable interface. Drop it to allow for further
simplification and changes.
Richard Purdie [Wed, 3 Jul 2019 16:20:08 +0000 (17:20 +0100)]
bitbake: runqueue: Remove RunQueueExecuteScenequeue and RunQueueExecuteTasks
Replace the remains of the Tasks and Scenequeue Tasks classes with simple
function calls. Also drop the dummy version of the execution class to
simplify further changes as its not needed.
Richard Purdie [Sat, 6 Jul 2019 13:12:55 +0000 (14:12 +0100)]
bitbake: runqueue: Remove now uneeded code
With the removal of the setcene verify code, this additional code block
is also now unneeded since tasks can't be forced at this point in the code
any move. This effectively reverts f21910157d873c030b149c4cdc5b57c5062ab5a6.
Richard Purdie [Thu, 11 Jul 2019 15:58:14 +0000 (16:58 +0100)]
bitbake: siggen: Use unique hashes for tasks
Now that runqueue optimises based on task hash, we need to ensure
tasks have unique hashes even in the simplest siggen mode. Use
the task name to calculate a unique hash.
This fixes runqueue tests when hash optimisations are added.
Robert Yang [Fri, 12 Jul 2019 03:26:15 +0000 (11:26 +0800)]
bitbake: cache: Set packages for skipped recipes
The provides and rprovides had been set for skipped recipes, packages are
similar to them (all of them provide something), so also set it. This makes it
easier to figure out the RDEPENDS issues, for example, lmsensors
(lmsensors_3.5.0.bb) RRECOMMENDS lmsensors-config-fancontrol
(lmsensors-config_1.0.bb), but lmsensors-config is skipped for some reasons,
then if we run:
$ bitbake lmsensors
ERROR: Nothing RPROVIDES 'lmsensors-config-fancontrol' (but /path/to/lmsensors_3.5.0.bb RDEPENDS on or otherwise requires it)
NOTE: Runtime target 'lmsensors-config-fancontrol' is unbuildable, removing...
Missing or unbuildable dependency chain was: ['lmsensors-config-fancontrol']
ERROR: Required build target 'lmsensors' has no buildable providers.
Missing or unbuildable dependency chain was: ['lmsensors', 'lmsensors-config-fancontrol']
We had no way to know who rprovides lmsensors-config-fancontrol, we can figure
it out by bitbake/contrib/dump_cache.py after this patch.
projects / thirdparty / openembedded / openembedded-core-contrib.git / log
