Scott Rifenbark [Wed, 26 Jul 2017 19:04:48 +0000 (12:04 -0700)]
yocto-project-qs, ref-manual: Moved YP intro from QS to ref-manual
Fixes [YOCTO #11630]
The introductory stuff in the QS is really conceptual information and
needs to be in the ref-manual where YP is introduced. Regarding the
QS, all we really need to do is point to places where the reader can
go to find that stuff out.
Part of this move involved getting the diagram of the YP flow for
the environment from the QS to the ref-manual. That figure was
named "YP-flow-diagram.png". It was named "yocto-environment.png"
in the QS but I renamed it when moving it to the ref-manual. This
caused some edits to the "Makefile" to clean up the figure lists
for tarballs.
Tom Rini [Sat, 29 Jul 2017 00:14:36 +0000 (20:14 -0400)]
imagefeatures: Add a test for various hypervisor formats
We add a new test that will create core-image-minimal for wic.vmdk,
wic.vdi and wic.qcow2. We then confirm via qemu-img that the resulting
file is of the expected type.
Tom Rini [Sat, 29 Jul 2017 00:14:35 +0000 (20:14 -0400)]
imagefeatures: Add a test for many CONVERSION_CMDs being chained
Add a new test to create a long (and not otherwise useful) image,
ext4.bmap.gz.bz2.lzo.xz.u-boot and also the sha256sum of it. Check that
the resulting sha256sum is valid.
Tom Rini [Sat, 29 Jul 2017 00:14:34 +0000 (20:14 -0400)]
imagefeatures: Extend test_bmap to validate gzip
We extend the existing test_bmap test to also produce an ext4.bmap.gz
file and then have gzip confirm that it contains valid gzip data. This
tests that we are able to chain at least 2 CONVERSION_CMDs together.
Tom Rini [Sat, 29 Jul 2017 00:14:33 +0000 (20:14 -0400)]
wic: Switch to using --use-uuid by default
The most portable way to specifiy a root device in a disk image that we
create is to use PARTUUID rather than /dev/sda2. As background, both
GPT and MBR tables provide valid UUID values for each partition and the
Linux Kernel contains the logic to parse this value. With this change
we can now boot the default disk images when used as any valid block
device that the included kernel uses. This for example means that
VirtualBox can be used to run vmdk without changes as it uses IDE for
the virtual disk controller.
Tom Rini [Sat, 29 Jul 2017 00:14:32 +0000 (20:14 -0400)]
image: Convert vmdk/vdi/qcow2 to strict CONVERSION_CMD types
The vmdk/vdi/qcow2 IMAGE_FSTYPEs predate wic. As such, they provide
some similar underlying functionality in order to produce a "disk" image
that in turn can be converted into different formats that various
hypervisor types work with. They do not however provide the ability for
other disk image types to be converted into these same output types.
Furthermore, they are less flexible than what wic does provide. This
drops the old style vmdk/vdi/qcow2 types and re-introduces them under
the CONVERSION_CMD framework. The equivalent of vmdk is now wic.vmdk
and so forth for the other types.
cml1.bbclass: wait until menuconfig terminal finishes
There are at least two terminals types (gnome and tmux) that when
launched to show the kernel's menuconfig, we lost track of the corresponding
process ID, thus there is no way to see when they finish, yielding identical
timestamps before and after menuconfig thus compile's task
is never tainted. This commit takes the solution from [1] but now in the menuconfig's
context.
Kai Kang [Fri, 28 Jul 2017 03:21:06 +0000 (11:21 +0800)]
assimp: upgrade to 4.0.0
Upgrade assimp to 4.0.0.
* Because there is also an assimp recipe in meta-oe, so update
DESCRIPTION, HOMEPAGE and SECTION from it.
* Remove redundant license file
* Update EXTRA_OECMAKE to fix library install path for multilib
Patrick Ohly [Fri, 28 Jul 2017 14:49:53 +0000 (16:49 +0200)]
autoconf-archive: simplify and fix recipe
The COPYING file specifies pure GPLv3, not GPLv2 & GPLv3. There is
also the COPYING.EXCEPTION file with the autotools exception, which
gets ignored here in the recipe to keep it simpler.
All of the explicit dependencies seem unnecessary, and RDEPENDS_${PN}
doesn't do anything for native recipes either, so all of that gets
removed.
It also built fine without the m4 and parallel build workarounds.
Patrick Ohly [Fri, 28 Jul 2017 14:49:52 +0000 (16:49 +0200)]
autoconf-archive: move from meta-oe to OE-core
Having common macros in OE-core that are needed by autotools based
projects makes sense. For example, tpm2.0-tools in meta-measured
depended on meta-oe only because of autoconf-archive.
This is a verbatim copy of the autoconf-archive recipe in
meta-openembedded rev 1cbd1bc1, with just one change: the patch which
disabled the installation of ax_code_coverage.m4 and
ax_check_enable_debug.m4 and the dependency on gnome-common were
removed.
So now autoconf-archive in OE-core provides them. gnome-common in
meta-oe will be changed to not install them and instead depend on
autoconf-archive.
Ming Liu [Fri, 28 Jul 2017 14:04:27 +0000 (16:04 +0200)]
rootfsdebugfiles.bbclass: add a optional parameter to choose file mode
This is in case that sometimes the copied target files might be
expected to have a desired mode, for instance, the ssh keys should not
be too open, otherwise the users will get a "Permission denied" error.
Ed Bartosh [Fri, 28 Jul 2017 09:29:28 +0000 (12:29 +0300)]
oe-selftest: wic: fix test_quemu
This test case boots the image in qemu and checks for mounted
partitions. As /boot is mounted automatically the test case fails.
Fixed this by adding /boot to the list of mounted partitions.
Ed Bartosh [Fri, 28 Jul 2017 09:29:27 +0000 (12:29 +0300)]
wic: add /boot mount point to fstab by default
wic avoided adding /boot to fstab for no reason.
This exception was hardcoded in the wic code.
There is no need for this as mountpoint in .wks file is an optional
field. It can be used only if user wants to have partitions
automatically mounted on system boot.
Ed Bartosh [Fri, 28 Jul 2017 09:29:24 +0000 (12:29 +0300)]
wic: use absolute paths in rootfs plugin
Using relative paths can cause copyhardlinktree API to fail as
it changes current directory when working. Converted all paths
to absolute paths using os.path.realpath.
Ed Bartosh [Tue, 25 Jul 2017 11:26:36 +0000 (14:26 +0300)]
wic: copy rootfs directory before changing fstab
wic updates /etc/fstab on root partition if there are
valid mount points in .wks
When wic runs from bitbake this can cause incorrect results
or even breakage of other tasks working with the same rootfs
directory in parallel with do_image_wic.
Implemented copying rootfs directory to a temporary location
using copyhardlinktree before updating fstab to avoid conflicts with
other tasks working with the same rootfs directory.
Ed Bartosh [Fri, 28 Jul 2017 09:29:22 +0000 (12:29 +0300)]
image_types.bbclass: ignore tar exit code 1
tar exists with 1 and produces warning "file changed as we read it"
if content is changed while tar archives it. Even hardlinking content
causes tar to fail this way as it changes file ctime.
Other tasks running in parallel with do_image_tar may need to hardlink
rootfs content in order to change it, e.g. do_image_wic does this to
update etc/fstab.
Ignored tar exit code 1 to be able to hardlink rootfs content while
do_rootfs_tar is tarring it.
Since we are adding the addto_sysroot that that is specific to the
native recipe, we must split this into a seperate -native .bb recipe.
When systemtap-native is built, the intention is that it's usable
from the native sysroot when done, ensure it's there with a forced
addto_recipe_sysroot task, we also don't want to clean the sysroot
when RM_WORK is enabled otherwise we loose the binaries.
oeqa/core/runner: OEStreamLogger don't buffer test execution writes
Since OEQA framework uses Python logging functionality to report test
results there is a class that wraps PyUnit writes into logging commands
(OEStreamLogger), so don't buffer the actual test execution to have
insight of what is currently executing.
This fix will change a little the test output format adding an '\n'
previous the test result, for example:
From:
test_nonmatching_checksum (lic_checksum.LicenseTests) ... ok
To:
test_nonmatching_checksum (lic_checksum.LicenseTests)
... ok
This is because the new line added by the PyUnit StreamLogger because
currently we don't have a manner to identify when a test execution
starts at report level (write msg).
oeqa/{core,selftest}: Add support to validate if a specified test case isn't found
If some test module/case is specified to run and isn't found the OEQA
framework didn't notice it, so complete the implementation using
modules_required and validate for the test case prescense.
Raise an exception when the test module/case required isn't found.
Jackie Huang [Wed, 31 May 2017 08:27:16 +0000 (16:27 +0800)]
ncurses: add SYSROOT_DESTDIR for siteconfig_gencache
After switching to Recipe Specific Sysroots, ncurses
will not be populated in its own sysroots, then
siteconfig_gencache fails to find some headers,
so add ${SYSROOT_DESTDIR}/${includedir} into the
search list to fix the issue.
Jackie Huang [Wed, 31 May 2017 08:27:17 +0000 (16:27 +0800)]
siteinfo: fix siteinfo_get_files to work with RSS
The siteconfig cache files in ACLOCALDIR setup by autotools.bbclass
has been dropped after switching to RSS, so change the siteconfig
search path back to SITECONFIG_SYSROOTCACHE and the parameter
name changed from aclocalcache to sysrootcache.
package/bbclass: sources are packaged separately from debug.
The configuration variable PACKAGE_DEBUG_SPLIT_STYLE includes
the new mode debug-with-srcpkg that instructs the system to
remove the source files from the debug package but include them in
a separate package with a "-src" suffix in the name.
Richard Purdie [Sat, 29 Jul 2017 20:08:43 +0000 (21:08 +0100)]
Revert "core-image-minimal-initramfs: use initramfs-framework by default"
This reverts commit 28fc6ba761ed4a47efa7c43e7f7dff5e2fe72b5e since it causes
runqemu.RunqemuTests.test_boot_machine_iso and runqemu.RunqemuTests.test_boot_deploy_hddimg
to fail in oe-selftest.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bsp/help: include a warning showing deprecation of create-layer plugin
This plugin will be removed starting 2.5 development in favour
of using 'bitbake-layers create-layer' script/plugin, offering a single
script to manage layers.
`git fetch --tags` seems to interact badly with `mirror=fetch`, resulting in
the regular branches not being fetched, so drop the unnecessary `--tags`. This
fixes this unit test failure: `bb.fetch2.FetchError: Fetcher failure: Unable
to resolve 'master' in upstream git repository in git ls-remote output for
/tmp/tmp4ag_mgmn/gitsource`
Richard Purdie [Fri, 28 Jul 2017 15:19:56 +0000 (16:19 +0100)]
bitbake: main: Always return 0 for bitbake --kill-server
If the server isn't running return 0 as to do otherwise complicates
scripts which do cleanup of bitbake servers which would potentially
be memory resident.
Richard Purdie [Fri, 28 Jul 2017 14:46:20 +0000 (15:46 +0100)]
bitbake: process: Clean up server communication timeout errors
This timeout path was commonly hit due to errors starting the server. Now we
have a better way to handle that, the retry logic can be improved and cleaned
up. This patch:
* Makes the timeout 5s rather than intervals of 1s with a message. Paul
noted some commands can take around 1s to run on a server which has just
been started on a loaded system.
* Allows a broke connection to exit immediately rather than retrying something
which will never work.
* Drops the Ctrl+C masking, we shouldn't need that anymore and any issues
would be better handled in other ways.
This should make things clearer and less confusing for users and is much cleaner
code too.
Richard Purdie [Fri, 28 Jul 2017 14:40:02 +0000 (15:40 +0100)]
bitbake: process/cooker: Allow UI process to know if the cooker was started successfully
Currently if the server fails to start, the user sees no error message and
the server will be repeatedly attempted to be started until some longer
timeouts expire. There are error messages in the cookerdeamon log but
nobody thinks to look there.
Add in a pipe which can be used to tell the starting process whether the cooker
did actually start or not. If it fails to start, no further attempts can be
made and if present, the log file can be shown to the user.
Richard Purdie [Fri, 28 Jul 2017 14:33:32 +0000 (15:33 +0100)]
bitbake: process: Allow BBUIEventQueue to exit cleanly
Currently the monitoring thread exits with some error code or runs indefinitely. Allow
closure of the pipe its monitoring to have the thread exit cleanly/silently.
Scott Rifenbark [Tue, 25 Jul 2017 23:57:04 +0000 (16:57 -0700)]
dev-manual: Added Links for specific container use
At the end of the section that describes how to set up the
development machine for CROPS, I added some user links for more
information based on the type of container the use is interested
in.
initramfs-framework: include install-efi module in recipe for installation
Utilized the existing init-install-efi.sh script and renamed it to
install-efi.sh to manage the installation process of images in
initramfs-framework model. This script will be executed when
"install" option is being chosen in the grub menu and install
the image on the target platform. A new install-efi module is
being added in the recipe to handle the installation process
using initramfs-framework.
initramfs-framework: module to support boot live image
setup-live module is a new module being introduced to integrate the
functionality of init-live.sh into new scriptlet named setup-live in
order to support the live boot image. The udev-extraconf rdepends is
being added to perform automounting. It gets to run before the rootfs
and finish module.
The setup-live scriplet include the changes for:
- Create a conditional loop for the bootparam_root variable. If it is
not set, then it will boot from ROOT_IMAGE. Else, it will boot normally
which is not from removable media.
- Gives a standard path to the original boot disk mount which can be
used to. While /media/sda is a good guess, it isn't always right, nor
is it a good assumption that only one boot disk is in the system.
- The current rootfs module has no support for rootfs images, currently
it only support for rootfs partitions for wic image. Therefore, there
is a need to assign the rootfs image for live image.
License checksum change due to copyright year update.
Latest icu will not compile with anything less than C++11, so
drop the enforcement of an earlier C++ version. This should be okay,
as there is now a fix in place for the problem of mixing native gcc 4/5
compiled code:
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=c21cec84886d9c70396e9be0ceb9a8ef300b54be
Andre McCurdy [Wed, 26 Jul 2017 00:18:36 +0000 (17:18 -0700)]
mirrors.bbclass: provide git repo fallbacks using the https protocol
Use MIRRORS to provide git repo fallbacks using the https protocol,
for cases where git native protocol fetches may fail due to local
firewall rules, etc.
These rules should cover all git native repos used by recipes within
oe-core, with the exception of mtd-utils, for which there's currently
no upstream alternative to the git native protocol for anonymous
access ( see http://git.infradead.org/mtd-utils.git ).
Ed Bartosh [Wed, 26 Jul 2017 16:22:29 +0000 (19:22 +0300)]
oe-selftest: wic: change mkfs.btrfs options
test_mkfs_extraopts test case fails on ab with error caused
by using -K --mixed options:
output: extent-tree.c:2696: btrfs_reserve_extent: BUG_ON `ret` triggered, value -28
For this test case it's not important to use particular options,
so changing options to anything less influential is OK.
Changed extra options for mkfs.btrfs to '--quiet' to fix the failure.
RPM file signing is enabled with --with-imaevm during configuration.
If enabled, the RPM signing tool rpmsign will call libimaevm.so provided
by the recipe ima-evm-utils.
Robert Yang [Thu, 13 Jul 2017 08:09:34 +0000 (01:09 -0700)]
runqemu: validate combos
Error out ealier if the combos is invalid, e.g.:
$ runqemu tmp/deploy/images/qemux86/bzImage-qemux86.bin tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic
This will fail at kernel panic, no we check and error out early. We can
add other checkings in the future.
Paul Eggleton [Thu, 27 Jul 2017 12:51:49 +0000 (14:51 +0200)]
bitbake: bitbake-diffsigs: fix regression after recent server changes
We were bridging the gap between the server and UI here by calling a
bb.siggen.find_siginfo, a function defined and set on that module from
the metadata. This worked from the UI side before but since the recent
server changes is no longer accessible. Create a new command so this can
execute on the server side and return the result by way of a new event.
(We're still running compare_sigfiles() on the signature generator but
that isn't quite the same thing and does still work.)
Richard Purdie [Wed, 26 Jul 2017 08:52:43 +0000 (09:52 +0100)]
gnome-themes-standard: Fix packages so dev-pkgs image generation works
A change from Ross removed the dual -dev/-dbg packages but this broke
deb/ipk rootfs constuction due to a dependency on a package which doesn't
exist (for dev-pkgs images). Remove the dependency on that package to
allow images to build.
Andre McCurdy [Tue, 25 Jul 2017 21:30:01 +0000 (14:30 -0700)]
bitbake: fetch2/__init__.py: replace stray logger.warn() with logger.warning()
Update stray usage of deprecated logger.warn(), which was introduced
to fetch2/__init__.py after all other instances had been replaced by
logger.warning():
Tom Rini [Tue, 25 Jul 2017 19:58:09 +0000 (15:58 -0400)]
image: Fix "metadata is not deterministic" when chaining 2+ CONVERSION_CMDs
When we have more than one CONVERSION_CMD being used, for example
ext4.gz.sha256sum we will see errors about "metadata is not
deterministic". This is because we do not have a stable order of
intermediate files that will be removed in the generated shell command.
We fix this by calling sorted() on the set of rm_tmp_images so that we
will have a stable hash again.
oeqa/buildperf: Add 'bitbake -m' on sync function to ensure bitbake is unloaded
Add 'bitbake -m' to the sync method and ensure all process related to
bitbake are correctly unloaded before doing the different measurements.
Also add a call to sync funtion on Test4 before final measurment of
eSDK deploy dir disk usage.
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tom Rini [Fri, 21 Jul 2017 22:06:34 +0000 (18:06 -0400)]
image_types.bbclass: Make u-boot signed images more versatile
With the introduction of chaining compression/conversion support we can
convert the old image_types_uboot.bbclass code that did a hand-chaining
of a set of ${filesystem}.${compression} into generic and arbitrary
support to sign whatever the user wants to sign for their image.
This, for the record, does remove setting a valid compression type in
the record in favour of just saying none. This is not a generally
useful feature in U-Boot and I believe being versatile in terms of being
able to pass in arbitrary compressions is more important.
Signed-off-by: Tom Rini <trini@konsulko.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tom Rini [Fri, 21 Jul 2017 22:06:33 +0000 (18:06 -0400)]
image.bbclass: Correct chaining compression support
When chaining of compression/conversion types was added, we had a new
way to handle doing things like "ext4.bz2.sha256sum" or
"ext2.gz.u-boot". However, because the U-Boot image class isn't
included normally, it wasn't properly converted at the time. After the
support was added the "clean" argument that the .u-boot code uses no
longer functions. The fix for this inadvertently broke chaining
compression/conversion. First, correct the u-boot conversion code.
Fixes: 46bc438374de ("image.bbclass: do exact match for rootfs type") Cc: Zhenhua Luo <zhenhua.luo@nxp.com> Cc: Richard Purdie <richard.purdie@linuxfoundation.org> Cc: Patrick Ohly <patrick.ohly@intel.com>
(From OE-Core rev: 0a7ce0b971a208956cb895ba5a869ec8c5d94703)
Signed-off-by: Tom Rini <trini@konsulko.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Mon, 24 Jul 2017 20:34:49 +0000 (21:34 +0100)]
libpng: use SourceForge mirror
The Gentoo mirror also deletes old versions when they're not used, so revert
back to the canonical SourceForge site, adding /older-releases/ to MIRRORS to
handle new releases moving the version we want.
Original idea by Maxin B. John <maxin.john@intel.com>.
externalsrc: place copy of git index into /tmp and do not use copyfile2
Using shutil.copy2() to copy .git/index to a temporary file tries to
copy SELinux attributes which might fail for confined users in SELinux
environments.
E.g. our builders are running in docker containers and modification of
sources (inclusive updated of .git/index) is done outside. Trying to
copy .git/index fails with
is created. This can not be solved by adapting the SELinux policy because
this is a very deep constraint violation:
| constrain file { create relabelfrom relabelto } ((u1 == u2 -Fail-) or (t1 == can_change_object_identity -Fail-) ); Constraint DENIED
|
| Possible cause is the source user (system_u) and target user (unconfined_u) are different.
I do not see much sense in using 'shutil.copy2()' here; 'shutil.copyfile()'
seems to be a better choice (target file is created in a secure way by
tempfile.NamedTemporaryFile()).
By placing the tempfile into /tmp we avoid potential problems related to
git's 'core.sharedRepository'. As a (positive) side effect, the source
tree will not be modified anymore (at least by this part of code) which
prevented to mount it read-only from somewhere else.
Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 25 Jul 2017 09:12:10 +0000 (11:12 +0200)]
wic-tools: ensure pseudo is available
wic will attempt to use pseudo from the wic-tools sysroot to run, but it
was only sure to be in there if do_install had executed - which is not
the case if it had been restored from sstate, in which case it failed
horribly as seen when running the wic.Wic.test_fs_types and
test_mkfs_extraopts tests on the Yocto Project autobuilder recently. Add
an explicit dependency on pseudo-native to ensure it's always there.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Benjamin Esquivel <code@esquibits.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Mon, 24 Jul 2017 21:42:01 +0000 (22:42 +0100)]
bitbake: fetch/wget: mitigate a wget race condition when listing FTP directories
When wget is fetching a listing for a directory over FTP it writes to a
temporary file called .listing in the current directory. If there are many such
operations happening in parallel - for example during 'bitbake world -c
checkpkg' - then up to BB_NUMBER_THREADS instances of wget will be racing to
write to, read, and delete the same file.
This results in various failures such as the file disappearing before wget has
processed it or the file changing contents, which causes checkpkg to randomly
fail.
Mitigate the race condition by creating a temporary directory to run wget in
when doing directory listings.
projects / thirdparty / openembedded / openembedded-core-contrib.git / log
