Elliot Smith [Tue, 12 Jul 2016 22:54:49 +0000 (15:54 -0700)]
bitbake: buildinfohelper: only record image files for tasks which make images
If a target is built which is classified as an "image" target
(e.g. "core-image-minimal"), Toaster reads the list of files in
the image (from the files-in-image.txt file).
However, Toaster continues to do this for builds which don't
produce images, if the recipe providing the target is an
image recipe. This can result in a list of files in the image
being attached to a target which didn't produce an image (e.g.
rootfs).
When associating files with an image, ensure that only targets
with a task which produces an image have "files in the image"
associated with them.
Elliot Smith [Tue, 12 Jul 2016 22:54:48 +0000 (15:54 -0700)]
bitbake: toaster: improve scan for SDK artifacts
SDK artifacts were previously picked up by toaster.bbclass and
notified to buildinfohelper (via toasterui). The artifacts
were then added to the Build object, so that it wasn't clear
which artifact went with which target; we were also unable
to attach SDK artifacts to a Build if they had already been
attached to a previous build.
Now, toaster.bbclass just notifies the TOOLCHAIN_OUTPUTNAME when
a populate_sdk* target completes. The scan is moved to buildinfohelper,
where we search the SDK deploy directory for files matching
TOOLCHAIN_OUTPUTNAME and attach them to targets (not builds).
If an SDK file is not produced by a target, we now look for a
similar, previously-run target which did produce artifacts.
If there is one, we clone the SDK artifacts from that target
onto the current one.
This all means that we can show SDK artifacts by target, and should
always get artifacts associated with a target, regardless of whether
it really build them.
This requires an additional model, TargetSDKFile, which tracks
the size and path of SDK artifact files with respect to Target
objects.
Elliot Smith [Tue, 12 Jul 2016 22:54:47 +0000 (15:54 -0700)]
bitbake: buildinfohelper: fix retrieval of targets
When buildinfohelper records the targets for a build, it looks
up any existing targets for a build and creates them if they
are not present. This is because in the case of Toaster-triggered
builds, the Target objects have already been created (inside
triggerBuild()) and don't need to be recreated; but in the case
of cli builds, the Target objects have to be created by
buildinfohelper.
The issue is that the code for retrieving an existing target for
a build only looks for Targets with a matching target and build,
e.g. Targets for build X with target "core-image-minimal". But it
is perfectly legitimate to call bitbake with a command like
"bitbake core-image-minimal:do_populate_sdk
core-image-minimal:do_populate_sdk_ext". In such a case, the
code which looks for matching targets finds two objects, as it
doesn't filter by task.
Add the task into the filter for the Target so that only one
Target object is be returned. Note that a command
line like "bitbake recipe:task recipe:task" will still cause an
error as bitbake doesn't de-duplicate the command line arguments
and will run the recipe:task combination twice.
Elliot Smith [Tue, 12 Jul 2016 22:54:46 +0000 (15:54 -0700)]
bitbake: toaster: attach kernel artifacts to targets
The bzImage and modules files were previously attached to a build,
rather than to the target which produced them. This meant it was
not possible to determine which kernel artifact produced by a
build came from which target; which in turn made it difficult to
associate existing kernel artifact with targets when those
targets didn't produce artifacts (e.g. if the same machine + target
combination was built again and didn't produce a bzImage or modules
file because those files already existed).
By associating kernel artifacts with the target (via a new
TargetArtifactFile model), we make it possible to find all
the artifacts for a given machine + target combination. Then, in
cases where a build is completed but its targets don't produce
any artifacts, we can find a previous Target object with the same
machine + target and copy its artifacts to the targets for a
just-completed build.
Note that this doesn't cover SDK artifacts yet, which are still
retrieved in toaster.bbclass and show up as "Other artifacts",
lumped together for the whole build rather than by target.
Refactor retrieval of suffix from image file path, making it a
a method on Target_Image_File. This makes it easier to use this
in the build dashboard for individual images, plus reduces the
complexity of the code required to get all of the image file
suffixes for a build.
Robert Yang [Mon, 18 Jul 2016 03:42:51 +0000 (20:42 -0700)]
bitbake: lib/bb/utils.py: return sorted OrderedDict in explode_dep_versions2
The OrderedDict's item is sorted by insertion order, there might be a
problem when build the same recipe again, for example:
- First build of acl:
Depends: libattr1 (>= 2.4.47), libc6 (>= 2.24)
- Second build of acl:
Depends: libc6 (>= 2.24), libattr1 (>= 2.4.47)
They are exactly the same depends, but tools like "diff" doesn't think
so. Return sorted OrderedDict will fix the problem.
Paul Eggleton [Fri, 15 Jul 2016 08:54:30 +0000 (20:54 +1200)]
bitbake: knotty: avoid errors when fetching outside of a task
In a few places we use the fetcher code to fetch files outside of a
task, for example uninative in OE. In that case the pid of the event is
0 and that was causing an error in BBUIHelper.eventHandler(). Check the
pid and do nothing if it's 0.
Ross Burton [Thu, 14 Jul 2016 18:56:22 +0000 (19:56 +0100)]
bitbake: build: don't use $B as the default cwd for functions
When bitbake executes a shell or Python function it can cd/chdir() into a
directory before executing the task. If no directory is specified then the
default of $B is used. However $B is an OpenEmbedded variable and BitBake
shouldn't be aware of it.
To solve this change the semantics slightly so that if no directory is
specified, the current working directory isn't changed. There's also a sanity
check that emits a warning if a Python task does os.chdir() without restoring
the old path, and the previous working directory is restored.
This does change semantics: whereas before a function in OE would have $B as the
working directory unless specified, now the working directory is the top of the
build tree. Any breakage this causes can be solved by either adding
do_some_task[dirs] = "${B}" or by using absolute paths in the task.
Ross Burton [Fri, 15 Jul 2016 10:25:42 +0000 (11:25 +0100)]
bitbake: lib/bb/build.py: decode the command as UTF-8
The messaging FIFO is UTF-8, so decode the command as UTF-8 as well as the value
as otherwise "bberror" != b("bberror") and none of the messages from shell
functions are ever displayed.
Also add an else to the command parser so unhandled commands are noticed.
In Python3 the itertools module's imap function has been migrated to the
globalname space as map(). Calling itertools.imap() will fail because it
no longer exists.
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildstats-summary: round the floating point percentage
This was rounded in python 2, but python 3 changed the default behavior of /.
We could switch to the same behavior as previous by switching to // rather
than /, but there's value in keeping at least one decimal point, to avoid the
misleading case where it says 0% but the reuse is non-zero.
Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Wed, 6 Jul 2016 23:57:09 +0000 (11:57 +1200)]
lib/oe/patch: handle encoding differences in patch files
With Python 3, the encoding of a file is significant; several recipes in
OE-Core have patches which are not fully utf-8 decodable e.g. man,
lrzsz, and gstreamer1.0-libav, leading to errors when using devtool's
modify, upgrade or extract subcommands on these recipes. To work around
this, try reading the patch file as utf-8 first and if that fails try
latin-1 before giving up.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Wed, 6 Jul 2016 23:57:08 +0000 (11:57 +1200)]
recipetool: create: support specifying a file as the local source
It is currently possible to specify a file (e.g. a tarball) on the local
disk as the source, but you have to know to put file:// in front of it.
There's really no need to force users to jump through that hoop if they
really want to do this so check if the specified source is a file and
prefix it with file:// if that's the case.
Also ensure the same works for "devtool add" at the same time.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Wed, 6 Jul 2016 23:57:07 +0000 (11:57 +1200)]
recipetool: create: fix handling of github URLs
For a while now, Github hasn't been advertising a specific repository
URL since cloning the web URL with git works. Armed with this knowledge
and fully expecting people to just paste the github URL, we need to
handle this situation specially. If it looks like a github URL to the
root of a repository then treat it as a git repository instead of a
normal https URL to be fetched by the wget fetcher.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If devtool returns exit code 4 then record the recipes as "skipped"
rather than "failed" - these are recipes we know cannot work (usually
because they don't provide any source).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Sun, 10 Jul 2016 23:07:56 +0000 (11:07 +1200)]
devtool: return specific exit code for incompatible recipes
Certain recipes cannot be used with devtool extract / modify / upgrade -
usually because they don't provide any source. Return a specific exit
code (4) so that scripts such as scripts/contrib/devtool-stress.py know
the difference between this and a genuine failure.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Wed, 6 Jul 2016 23:57:11 +0000 (11:57 +1200)]
devtool: reset: allow reset to work if the recipe file has been deleted
We were attempting to open the recipe file unconditionally here - we
need to account for the possibility that the recipe file has been
deleted or moved away by the user.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Wed, 6 Jul 2016 23:57:10 +0000 (11:57 +1200)]
devtool: update-recipe: fix --initial-rev option
In OE-Core revision 7baf57ad896112cf2258b3e2c2a1f8b756fb39bc I changed
the default update-recipe behaviour to only update patches for commits
that were changed; unfortunately I failed to handle the --initial-rev
option which was broken after that point. Rework how the initial
revision is passed in so that it now operates correctly.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Egyptian government changed its mind on short notice, and
Africa/Cairo will not introduce DST starting 2016-07-07 after all.
(Thanks to Mina Samuel.)
Asia/Novosibirsk switches from +06 to +07 on 2016-07-24 at 02:00.
(Thanks to Stepan Golosunov.)
Changes to past and future time stamps
Asia/Novokuznetsk and Asia/Novosibirsk now use numeric time zone
abbreviations instead of invented ones.
Changes affecting past time stamps
Europe/Minsk's 1992-03-29 spring-forward transition was at 02:00 not 00:00.
(Thanks to Stepan Golosunov.)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Robert Yang [Thu, 7 Jul 2016 01:37:04 +0000 (18:37 -0700)]
libpcre: 8.38 -> 8.39
* Remove CVE-2016-3191.patch which is already in the source.
* The LIC_FILES_CHKSUM is changed since it has updated the date from
2015 to 2016, the contents are the same.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Jan Remmet <j.remmet@phytec.de> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 8 Jul 2016 17:00:49 +0000 (18:00 +0100)]
linux-yocto-dev: Handle performance regression
After commit 0437a59e3c298d40aaa96af09b80bff8fcbe292d, the linux-yocto-dev
recipe is being parsed every time we run "bitbake -p". This was spotted
on some performance benchmarks and showed up as a performance regression.
We can tweak the recipe to ensure this doesn't happen and that its only
used if selected.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Wed, 6 Jul 2016 23:48:34 +0000 (11:48 +1200)]
gcc: add runtime packages to RRECOMMENDS
In order to use certain features of gcc, you need the corresponding
runtime library. It seems to me that these ought to be installed by
default when installing the compiler since they are required if certain
command line options are used, so add them to RRECOMMENDS. I used
RRECOMMENDS since some of these packages may or may not exist depending
on architecture and build options; additionally it makes it possible to
use BAD_RECOMMENDATIONS if you really want to exclude them.
The impact of this isn't too bad in the context of an image providing
on-target compilation - about a 30MB increase in size for an image
containing gcc and g++.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Mon, 11 Jul 2016 21:03:38 +0000 (22:03 +0100)]
glibc: add more Imagination Meta relocation to elf.h
glibc master added the EM_METAG tag but didn't add the relocation defines.
However the kernel tooling only checks for EM_METAG when defining its own values
so scripts/recordmcount ends up using R_META_* symbols without their definition.
Whilst the kernel can and should be fixed, this breaks all users of recordmcount
so patch elf.h to add the values.
Zhenhua Luo [Tue, 28 Jun 2016 07:25:07 +0000 (15:25 +0800)]
tune-ppce500mc.inc: pass -mcpu=e500mc for ppce500mc kernel compile
Currently the -mcpu parameter is not passed to cross gcc when assembling
kernel .S file, the implicit -mcpu option that defaults to the latest
server cpu might casuse incorrect assembling.
A existent case is that wait instruction of ppce500mc is incorrectly assembled
to power9 version with default -mcpu setting, accordingly kernel boot calltrace
happend when wait instruction is executed on ppce500mc targets.
Signed-off-by: Zhenhua Luo <zhenhua.luo@nxp.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Tue, 5 Jul 2016 16:41:24 +0000 (17:41 +0100)]
classes/libc-package: remove pointless copying when running localedef
localedef handles attempts to read/write the archive in parallel correctly by
creating the file atomically, gracefully handling racing to create, and has
exclusive locks when writing. Therefore I can't see any purpose to copying the
archive to /tmp and back again when manipulating it.
This fragment dates from when this class was used for more than just glibc
locale packaging, and as glibc-locale disables do_configure it can't have been
executed.
Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Tue, 23 Feb 2016 14:05:35 +0000 (14:05 +0000)]
db: use S/B more idiomatically
Instead of setting S to a directory inside the tarball and B to another
directory inside the tarball, use the default value of S and set
AUTOTOOLS_SCRIPT_PATH to the right path to find configure. Update the patches
so they still apply, and clean up the recipe slightly.
Because something is not quite right regarding quilt and patching, add a PR bump
to the recipes to ensure that a clean work directory is used: for some reason
rebuilds will rarely fail to patch correctly.
Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a --disable-debug-info parameter to ./configure, as it otherwise
attempts to execute a small test binary. If this (new) feature is truly needed,
we can set up a qemu wrapper later, or patch autoconf macros.
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 6 Jul 2016 16:21:59 +0000 (17:21 +0100)]
terminal: Drop gnome-terminal --disable-factory workarounds
With the new pid monitoring code we have for recent versions of
gnome-terminal we can just drop the --disable-factory code now since
the other solution handles this case as well.
Richard Purdie [Wed, 6 Jul 2016 16:08:57 +0000 (17:08 +0100)]
terminal: Fix gnome-terminal to work with recent versions
Currently gnome-terminal just returns straight away, opening a terminal in a new
separate process we have no insight into. For patch resolution, this leads to
spawning many different terminal windows, for pydevshell, it just flashes a window
up and then closes.
We need to block until the command completes but gnome-terminal gives us no way
to do this. We therefore write the pid to a file using a "phonehome" wrapper
script, then monitor the pid until it exits.
Richard Purdie [Tue, 5 Jul 2016 12:18:31 +0000 (13:18 +0100)]
pseudo: Upgrade to 1.8.1
* Drop patches where the changes exist upstream
* Fetch from git as no tarball is available for 1.8.1
* Move common code to pseudo.inc
* Update patchset in git recipe
Paul Eggleton [Thu, 7 Jul 2016 21:17:32 +0000 (09:17 +1200)]
oe-selftest: drop test_prepare_unexpected
This test refers to a function that no longer exists after the eSDK
install double execution of bitbake has been removed, and since
test_prepare_unexpected is the only test in this module, drop the
entire module. We can easily resurrect it if we have unit tests to add
in the future.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Wed, 6 Jul 2016 17:22:36 +0000 (18:22 +0100)]
bitbake: toaster: layerdetails api Fix saving of git revision of a layer
Update, clean up and move the api for updating a layerversion from the
views to api. Also update the layerdetails page to include the
layerversion id in the url getter.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Wed, 6 Jul 2016 11:00:35 +0000 (12:00 +0100)]
bitbake: buildinfohelper: ensure task datetimes are timezone-aware
When using toaster-eventreplay to run a bitbake event file
through toasterui/buildinfohelper, errors occur when the
tasks are updated with buildstats info:
RuntimeWarning: DateTimeField Task.started received a naive
datetime (2016-07-06 09:15:22.070000) while time zone support
is active.
This is because a method in buildinfohelper returns a naive
datetime, but Django is expecting timezone-aware datetimes.
Ensure that datetimes used to set the started/ended times on
tasks are converted to timezone-aware datetimes.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Mon, 4 Jul 2016 11:52:53 +0000 (12:52 +0100)]
bitbake: toaster: fix layout for command-line builds in recent builds area
Command-line builds were displayed incorrectly, so that the
HTML elements for other builds were being "consumed" by the
command-line build elements due to incorrect positioning of
element end tags.
Fix by ensuring end tags close elements in the right places.
As the indentation was all over the place in the most recent
builds section template, it was almost impossible to see what the
problem was. So that was fixed, too.
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Worker needs input stream in binary mode as it reads binary content
from it. Current code does it by detaching a buffer from sys.stdin
and assigning it back to sys.stdin.
Detached buffer is io.BufferedReader in binary mode. This operation
is implicit as its purpose is not easily understandable from the code.
Replacing it with fdopen(sys.stdin.fileno(), 'rb') should make the
code more understandable.
Assigning the buffer to sys.stdin is not needed as worker doesn't
use sys.stdin. Moreover, it leads to difficult to debug issues down
the stack. For example, devpyshell doesn't work without reopening
sys.stdin in text mode. This is not needed anymore after this fix as
sys.stdin is not changed in worker code and remains in text mode.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Mon, 4 Jul 2016 15:34:45 +0000 (16:34 +0100)]
bitbake: toaster: layerBtn avoid connecting handler to other build buttons
Some pages contain other build buttons which may have the same class
attached. Make sure that we only select the buttons in the tables where
layerBtn is used.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sujith H [Wed, 22 Jun 2016 09:10:42 +0000 (10:10 +0100)]
bitbake: bitbake: toaster-tests: tests for project config
Add basic tests to validate the value, user types
in the text box for DL_DIR and SSTATE_DIR. Added
test case to validate the first char and inclusion
of space between the characters.
Signed-off-by: Sujith H <sujith.h@gmail.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Sujith H <sujith.h@gmail.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sujith H [Fri, 24 Jun 2016 15:25:54 +0000 (16:25 +0100)]
bitbake: toaster: remove SSTATE_MIRRORS from projectconf
Remove SSTATE_MIRRORS from getting blacklisted. Hence the SSTATE_MIRRORS
should be removed from projectconf.html file so that users can know that
this variable is no more blacklisted.
Signed-off-by: Sujith H <sujith.h@gmail.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Sujith H <sujith.h@gmail.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 7 Jul 2016 13:27:43 +0000 (14:27 +0100)]
bitbake: progress: Ensure missing start event is fired
The init function of the parent class fires a progress event for 0
progress rather than a start event. UI code was assuming that progress
events should always have a start event first. This change ensures that
the start event is correctly generated.
This fixes crashes that were seen in knotty in some configurations.
Richard Purdie [Thu, 7 Jul 2016 13:26:34 +0000 (14:26 +0100)]
bitbake: knotty: Handle process indicators more gracefully
Mistakes can happen with the generation of the progress events, change
knotty to be more tolerant of this rather than crashing, reporting to the
user when something unexpected happens. I haven't debugged why multiple
finish events appear to be triggered.
Paul Eggleton [Thu, 23 Jun 2016 10:59:12 +0000 (22:59 +1200)]
bitbake: runqueue: report progress for "Preparing RunQueue" step
When "Preparing RunQueue" shows up you can expect to wait up to 30
seconds while it works - which is a bit long to leave the user waiting
without any kind of output. Since the work being carried out during this
time is divided into stages such that it's practical to determine
internally how it's progressing, replace the message with a progress
bar.
Actually what happens during this time is two major steps rather than
just one - the runqueue preparation itself, followed by the
initialisation prior to running setscene tasks. I elected to have the
progress bar cover both as one (there doesn't appear to be much point in
doing otherwise from a user perspective). I did however describe it as
"initialising tasks".
Paul Eggleton [Thu, 23 Jun 2016 10:59:11 +0000 (22:59 +1200)]
bitbake: runqueue: add ability to enforce that tasks are setscened
Add the ability to enter a mode where only a specified whitelist of
tasks can be executed outright; everything else must be successfully
provided in the form of a setscene task (or covered by a setscene task).
Any setscene failure outside of the whitelist will cause the build to
fail immediately instead of running the real task, and any real tasks
that would execute outside of the whitelist cause an immediate build
failure when it comes to executing the runqueue as well.
The mode is enabled by setting BB_SETSCENE_ENFORCE="1", and the
whitelist is specified through BB_SETSCENE_ENFORCE_WHITELIST, consisting
of pn:taskname pairs. A single % character can be substituted for the pn
value to match any target explicitly specified on the bitbake command
line. Wildcards * and ? can also be used as per standard unix file name
matching for both pn and taskname.
Paul Eggleton [Thu, 23 Jun 2016 10:59:10 +0000 (22:59 +1200)]
bitbake: knotty: add quiet output mode
Quiet output mode disables printing most messages (below warnings) to
the console; however these messages still go to the console log file.
This is primarily for cases where bitbake is being launched
interactively from some other process, but where full console output is
not needed.
Because of the need to keep logging all normal events to the console
log, this functionality was implemented within the knotty UI rather
than in bb.msg (where verbose mode is implemented). We don't currently
have a means of registering command line options from the UI end, thus
the option actually has to be registered in main.py regardless of the
UI, however I didn't feel like it was worth setting up such a mechanism
just for this option.
Paul Eggleton [Thu, 23 Jun 2016 10:59:09 +0000 (22:59 +1200)]
bitbake: knotty: show task progress bar
In addition to the "currently running n tasks (x of y)" message, show a
progress bar for another view on how much of the build is left. We have
to take care to reset it when moving from the scenequeue to the
runqueue, and explicitly don't include an ETA since not all tasks take
equal time and thus it isn't possible to estimate the time remaining
with the information available.
Paul Eggleton [Thu, 23 Jun 2016 10:59:08 +0000 (22:59 +1200)]
bitbake: knotty: add code to support showing progress for sstate object querying
Add support code on the BitBake side to allow sstate.bbclass in
OpenEmbedded to report progress when it is checking for availability of
artifacts from shared state mirrors.
Add a class to help report progress in a task that consists of multiple
stages, some of which may have internal progress (do_rootfs within
OpenEmbedded is one example). Each stage is weighted to try to give
a reasonable representation of progress over time.
Paul Eggleton [Thu, 23 Jun 2016 10:59:05 +0000 (22:59 +1200)]
bitbake: lib: implement basic task progress support
For long-running tasks where we have some output from the task that
gives us some idea of the progress of the task (such as a percentage
complete), provide the means to scrape the output for that progress
information and show it to the user in the default knotty terminal
output in the form of a progress bar. This is implemented using a new
TaskProgress event as well as some code we can insert to do output
scanning/filtering.
Any task can fire TaskProgress events; however, if you have a shell task
whose output you wish to scan for progress information, you just need to
set the "progress" varflag on the task. This can be set to:
* "percent" to just look for a number followed by a % sign
* "percent:<regex>" to specify your own regex matching a percentage
value (must have a single group which matches the percentage number)
* "outof:<regex>" to look for the specified regex matching x out of y
items completed (must have two groups - first group needs to be x,
second y).
We can potentially extend this in future but this should be a good
start.
Paul Eggleton [Thu, 23 Jun 2016 10:59:04 +0000 (22:59 +1200)]
bitbake: knotty: import latest python-progressbar
Since we're going to make some minor extensions to it, it makes sense to
bring in the latest version of python-progressbar. Its structure has
changed a little but the API hasn't; however we do need to ensure our
overridden _needs_update() function's signature in BBProgress() matches
properly.
projects / thirdparty / openembedded / openembedded-core-contrib.git / log
