Michael Wood [Thu, 24 Nov 2016 11:20:04 +0000 (11:20 +0000)]
toaster: buildinfohelper Simplify layer event to toaster layer function
Simplify the layer event information to layer version object in toaster
function. Previously this attempted many different methods of trying to
obtain the correct layer from toaster by manipulating the data from the
event or the data from the known layers to try and match them together.
We speed up and simplify this process by making better use of django's
orm methods and by working down the most likely matching methods in order
of accuracy.
[YOCTO #10220]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 24 Nov 2016 11:20:03 +0000 (11:20 +0000)]
toaster: bldcontrol Move CustomImageRecipe file creation into own function
Move the custom image file creation (i.e. create the layer file
structure, conf and recipe file) into it's own function and remove the
creation of the BRLayer as this is done at schedule_build just like all
the other layers.
Fix a bug where the toaster-custom-images layer was always being appened
to the layer list if the directory exists.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 24 Nov 2016 11:20:02 +0000 (11:20 +0000)]
toaster: orm models Handle CustomImageRecipe BRLayer here
The schedule_build function on the project object is where the BRLayers
are created for the build. Instead of creating the BRLayer for the
CustomImageRecipe in the localhostbbcontroller create it here so that
all that mechanism is in one place.
Also fix a number of pyflake errors.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes the unidentified layers issue by making the
toaster-custom-images layer a local layer. By doing this we also fix the
git assumptions made for the local layers which stop recipes and other
meta data being associated with them. This also removed some of the
special casing previously needed when we didn't have the concept of a
local (non git) layer.
Also rename created flag var to a have a different var for each returned
value so that the same value isn't used multiple times.
[YOCTO #10220]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sujith H [Thu, 24 Nov 2016 11:19:58 +0000 (11:19 +0000)]
toaster: localhostbecontroller accept custom init script for build
When passed variable CUSTOM_BUILD_INIT_SCRIPT to toaster
setting, it would be nice to use it. Else toaster
can use oe-init script. This gives an oppurtunity to
use customized build init scritps.
Signed-off-by: Sujith H <sujith.h@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reyna, David [Thu, 24 Nov 2016 11:19:56 +0000 (11:19 +0000)]
toaster: orm gen_layerdeps Protect against circular Layer dependencies
Limit the recursion (to say 20 levels) when processing layer dependencies
so that circular dependecies do not cause infinite decent and an
out-of-memory failure. The duplicate found layers are already immediately
filtered in the code.
[YOCTO #10630]
Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 24 Nov 2016 11:19:55 +0000 (11:19 +0000)]
toaster: customrecipejs Consume click event on 'a' link if disabled
Consume the click event on the download recipe link if it's disabled. To
prevent the link from sending user to an error page.
See http://getbootstrap.com/css/#forms-disabled-fieldsets and a link
caveat.
[YOCTO #10151]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 24 Nov 2016 11:19:52 +0000 (11:19 +0000)]
toaster: runbuilds Write the pidfile in python rather than shell script
Write the pid file out in the start up of this management command. This
ensures this has happened instead of relying on the shell command having
been run which may or may not be the case. This also makes it simpler for
testing.
Couple of clean ups of runbuilds as identified by pyflake
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I noiced builds where tasks seemed to be taking a surprisingly long time.
When I looked at the output of top/pstree, these tasks were no longer
running despite being listed in knotty. Some were in D/Z state waiting for
their exit code to be collected, others were simply not present at all.
strace showed communication problems between the worker and cooker, each
was trying to write to the other and nearly deadlocking. Eventually, timeouts
would allow them to echange 64kb of data but this was only happening every
few seconds.
Whilst this particularly affected builds on machines with large numbers
of cores (and hence highly parallal task execution) and in cases where
I had a lot of debug enabled, this situation is clearly bad in general.
This patch introduces a thread to the worker which is used to write data
back to cooker. This means that the deadlock can't occur and data flows
much more freely and effectively.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Kai Kang [Sun, 9 Oct 2016 08:34:32 +0000 (16:34 +0800)]
COW.py: fix sample codes
The call of methods iteritems() and itervalues() in sample codes were
replaced by items() and values() to convert to Python 3 by Bitbake rev d0f904d407f57998419bd9c305ce53e5eaa36b24. But the methods iteritems()
and itervalues() belong to class COWDictMeta not class dict or set. The
modifications should not be made in purpose that it fails to run sample
codes, so revert them.
Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 10 Nov 2016 03:52:34 +0000 (19:52 -0800)]
toaster: layerindex updater Take into account layers being predefined
As we can now provide layer definitions through fixtures we need to be
more clever how we update the metadata in the database to avoid
duplicate metadata being created. To do this we make more effort to
match existing data in the database and update only the fields which
will be better provided by the layer index.
This removes the need for us to special case layers which are provided
as part of poky such as openembedded-core or meta-poky which exist on
the layerindex but with different git urls.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 10 Nov 2016 03:52:33 +0000 (19:52 -0800)]
toaster: orm/fixtures Add the master release and correct morty release
Add the master release option to base your project on and correct the
morty release so that for poky based setups we use the poky provided
version of the layer rather than checking out the layer from its own git
repository.
[YOCTO #10497]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 10 Nov 2016 03:52:32 +0000 (19:52 -0800)]
toaster: settings fixture Set default release to master
Now that morty has been released we now set the DEFAULT_RELEASE back to
master.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Patrick Ohly [Mon, 14 Nov 2016 09:39:14 +0000 (10:39 +0100)]
taskdata.py: improve handling of depends/rdepends
Error handling only caught the cause where a dependency did not have
any colon, but ignored the case where more than one was given. Now
"pn:task:garbage" will raise an error instead of ignoring ":garbage".
The error message had a misplaced line break (?) with the full stop
on the next line. Indenting the explanation with a space might have
been intended and is kept.
split() was called three times instead of just once.
Instead of improving the two instances of the code (one for 'depends',
one for 'rdepends'), the common code is now in a helper function.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Patrick Ohly [Mon, 14 Nov 2016 09:39:00 +0000 (10:39 +0100)]
data_smart.py: don't reorder internal bitbake variables when calculating hash
Commit 260ced745 added __BBTASKS, __BBANONFUNCS, __BBHANDLERS to the
data that gets hashed, but only after reordering these lists. The
intention probably was to make the hash deterministic, but that's
unnecessary (the content of the variables should already be
deterministic) and hides potential reasons that might require
re-parsing.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 16 Nov 2016 11:20:42 +0000 (11:20 +0000)]
data_smart: Default to expansion for getVar/getVarFlags
We've been building to this for a while, default to return expanded
values for getVar/getVarFlags.
We can then go through and remove the "True" option to many of the
calls to this function, all function calls should have a default by now
though since the parameter has been required for a while.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 16 Nov 2016 11:20:06 +0000 (11:20 +0000)]
data: Drop deprecated old style bitbake API
The old style bb.data.getVar/setVar API has long since been deprecated in
favour of d.getVar/setVar and friends.
Now we're about to change the default expansion parameter, drop the old APIs
to simplify the transition and ensure everyone is using the new style functions.
Conversion is trivial if there are remaining stragglers.
I've left bb.data.expand() for now since its more widely used but would make a good
follow up patch series.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 16 Nov 2016 11:19:01 +0000 (11:19 +0000)]
lib/bb: Don't use deprecated bb.data.getVar/setVar API
The old style bb.data.getVar/setVar API is obsolete. Most of bitbake
doesn't use it but there were some pieces that escaped conversion. This
patch fixes the remaining users mostly in the fetchers.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If ud.ignore_checksums is set (which we currently use to suppress the
warnings for missing SRC_URI checksums when fetching files from
scripts), then if we're fetching an npm package we should similarly
suppress the warnings when NPM_LOCKDOWN and NPM_SHRINKWRAP aren't set.
At the same time, make any errors reading either of these files actual
errors since if the file is specified and could not be found, that
should be an error - not the exact same warning.
Fixes [YOCTO #10464].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
brian avery [Fri, 4 Nov 2016 12:27:06 +0000 (12:27 +0000)]
toaster: settings set ALLOWED_HOSTS to * in debug mode
As of Django 1.8.16, Django is rejecting any HTTP_HOST header that is
not on the ALLOWED_HOST list. We often need to reference the
toaster server via a fqdn, if we start it via webport=0.0.0.0:8000 for
instance, and are hitting the server from a laptop. This change does
reduce the protection from a DNS rebinding attack, however, if you are
running the toaster server outside a protected network, you should be
using the production instance.
[YOCTO #10578]
Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Robert Yang [Thu, 27 Oct 2016 08:50:18 +0000 (01:50 -0700)]
lib/bb/cooker.py: fix for BBFILE_PATTERN = ""
There would be error when BBFILE_PATTERN = None:
BBFILE_PATTERN_foo not defined
This is the correct behaviour, but when the layer sets BBFILE_PATTERN = "",
it would match all the remaining recipes, and cause "No bb files matched BBFILE_PATTERN"
warnings for all the layers which behind it.
When a layer sets BBFILE_PATTERN = "" (for example, a layer only
provides git repos and source tarballs), now it means has no recipes.
This is different from BBFILE_PATTERN_IGNORE_EMPTY, the later one means
that it *may* not have any recipes.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Mon, 31 Oct 2016 16:40:40 +0000 (16:40 +0000)]
toaster: add tests/eventreplay/README
Put instructions on how to prepare event log files
and run eventreplay tests.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Mon, 31 Oct 2016 16:40:39 +0000 (16:40 +0000)]
toaster: add eventreplay test case for zlib
Run toaster-eventreplay with zlib.events.
Check if zlib build and package present in Toaster database.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Mon, 31 Oct 2016 16:40:38 +0000 (16:40 +0000)]
toaster: add eventreplay test case for core-image-minimal
Run toaster-eventreplay with core-image-minimal.events and
test if all required packages present in Target_Installed_Package
table.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Mon, 31 Oct 2016 16:40:37 +0000 (16:40 +0000)]
toaster: use current directory if BUILDDIR is not set
If BUILDDIR environment variable is not set signal_runbuilds function
throws TypeError as os.getenv('BUILDDIR') returns None:
ERROR: unsupported operand type(s) for +=: 'NoneType' and 'str'
Traceback (most recent call last):
File "bitbake/lib/bb/ui/toasterui.py", line
391, in main
buildinfohelper.update_build_information(event, errors, warnings,
taskfailures)
File "bitbake/lib/bb/ui/buildinfohelper.py",
line 1184, in update_build_information
self.internal_state['build'], errors, warnings, taskfailures)
File "bitbake/lib/bb/ui/buildinfohelper.py",
line 238, in update_build_stats_and_outcome
signal_runbuilds()
File "bitbake/lib/toaster/orm/models.py", line
1746, in signal_runbuilds
'.runbuilds.pid')) as pidf:
File "/usr/lib64/python3.4/posixpath.py", line 82, in join
path += b
TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'
Used os.getenv('BUILDIR', '.') to make it always return meaningful
directory path. Current directory '.' will be used if BUILDDIR is
not set.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
brian avery [Fri, 28 Oct 2016 16:52:51 +0000 (17:52 +0100)]
toaster: buildinfohelper Handle regex paths
We were presuming that all the layer dependency information was of the
form "^/path/to/layer" to we were just stripping the leading "^" off of
the layer information when we were matching the layer priorities to the
toaster database. This patch splits out the priorities layer match which
gets a regex from the task/recipe match which is gets a path.
Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of searching for the build for each test just use the returned
value of the completed build.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Fri, 28 Oct 2016 15:48:48 +0000 (18:48 +0300)]
toaster: tests builds Update buildtest
Now that we're using fixtures for configuration just load these instead
of trying to search for a toasterconf json file.
Also for convenience add the ability for the tests to source the build
environment script. To use this test make sure that directories are in
the same layout as poky.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Fri, 28 Oct 2016 15:48:47 +0000 (18:48 +0300)]
toaster: orm models Handle run builds process not yet running
During tests we may want to call the runbuilds process manually for
example when doing a "one shot" approach rather than a long running
process during tests.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Fri, 28 Oct 2016 15:48:46 +0000 (18:48 +0300)]
toaster: test browser test_layerdetails_page add wait_until_visible
Add an additional wait_until_visible for the save buttons as firefox
animates this into view so slowly we get a race on them being visible
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Fri, 28 Oct 2016 15:48:45 +0000 (18:48 +0300)]
toaster: Remove contrib tts
Remove the "Toaster test system". We don't need a home brew
test "framework" as the django test runner is more than adequate.
None of these tests here are currently working and have been obsoleted
by the work done on unit and browser tests in ./tests/.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Fri, 28 Oct 2016 15:48:44 +0000 (18:48 +0300)]
toaster: Delete useless bldcontrol/test
It doesn't work nor does it test anything useful
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Fri, 28 Oct 2016 15:48:43 +0000 (18:48 +0300)]
toaster: Move views tests to main testing module
Consolidating all the tests to live in the same place to make them more
discoverable and consistent as well as not cluttering up the django app
directory.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Fri, 28 Oct 2016 15:48:42 +0000 (18:48 +0300)]
toaster: views Tests fix all pyflake identified issues
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Markus Lehtonen [Tue, 1 Nov 2016 15:05:12 +0000 (17:05 +0200)]
bitbake-worker: print full traceback instead of message only
Print full traceback instead of just the exception message in the
child() function inside fork_off_task(). This makes debugging a lot
easier as the function catches a generic "Exception" and the exception
message alone might not give much information.
[YOCTO #10393]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Markus Lehtonen [Tue, 1 Nov 2016 15:05:11 +0000 (17:05 +0200)]
data: fix exception handling in exported_vars()
Fix a bug where a totally wrong value of a variable would be exported if
an exception happened during d.getVar(). Also, print a warning if an
exception happends instead of silently ignoring it. It would probably be
best just to raise the exception, instead, but use the warning for now
in order to avoid breaking existing builds.
[YOCTO #10393]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 2 Nov 2016 15:07:33 +0000 (15:07 +0000)]
siggen: Ensure taskhash mismatches don't override existing data
We recalculate the taskhash to ensure the version we have matches
what we think it should be. When we write out a sigdata file, use
the calculated value so that we don't overwrite any existing file.
This leaves any original taskhash sigdata file intact to allow a
debugging comparison.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 2 Nov 2016 15:06:50 +0000 (15:06 +0000)]
siggen: Pass basehash to worker processes and sanity check reparsing result
Bitbake can parse metadata in the cooker and in the worker during builds. If
the metadata isn't deterministic, it can change between these two parses and
this confuses things a lot. It turns out to be hard to debug these issues
currently.
This patch ensures the basehashes from the original parsing are passed into
the workers and that these are checked when reparsing for consistency. The user
is shown an error message if inconsistencies are found.
There is debug code in siggen.py (see the "Slow but can be useful for debugging
mismatched basehashes" commented code), we don't enable this by default due to
performance issues. If you run into this message, enable this code and you will
find "sigbasedata" files in tmp/stamps which should correspond to the hashes
shown in this error message. bitbake-diffsigs on the files should show which
variables are changing.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
brian avery [Fri, 14 Oct 2016 15:28:01 +0000 (16:28 +0100)]
toaster: Update default release to Morty
Set Morty to be the default release in toaster for the Morty release
when creating new projects.
Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
brian avery [Fri, 14 Oct 2016 15:28:00 +0000 (16:28 +0100)]
toaster: Update poky fixture for Morty release
Update the poky fixture to the Morty release. This removes the
master branch from the release and limits it to the morty branch.
Normally, we would also support at least one past branch but the change
from Python 2.7 -> Python 3 makes that infeasible.
Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
brian avery [Fri, 14 Oct 2016 15:27:59 +0000 (16:27 +0100)]
toaster: Update oe-core fixture for Morty release
Update the oe-core fixture to the Morty release. This removes the
master branch from the release and limits it to the morty branch.
Normally, we would also support at least one past branch but the change
from Python 2.7 -> Python 3 makes that infeasible.
Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Aníbal Limón [Fri, 14 Oct 2016 15:48:39 +0000 (10:48 -0500)]
bb.event: fix infinite loop on print_ui_queue
If bitbake ends before _uiready and bb.event.LogHandler was add
to the bitbake logger it causes an infinite loop when logging
something.
The scenario is print_ui_queue is called at exit and executes
the log handlers [2] one of them is bb.event.LogHandler this handler
appends the same entry to ui_queue causing the inifine loop [3].
In order to fix a new copy of the ui_queue list is created when iterate
ui_queue.
Scott Rifenbark [Thu, 13 Oct 2016 22:00:23 +0000 (15:00 -0700)]
bitbake-user-manual: Changed BB_SETSCENE_VERIFY_FUNCTION name
The BB_SETSCENE_VERIFY_FUNCTION variable has effectively changed
to BB_SETSCENE_VERIFY_FUNCTION2. I changed the three areas in the
book. Basically a name change.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Scott Rifenbark [Mon, 10 Oct 2016 19:49:06 +0000 (12:49 -0700)]
bitbake-user-manual: Updated the [noexec] and [nostamp] flag descriptions
Fixes [YOCTO #10401]
Added some wording to clarify that setting these flags to "1"
causes the desired action. Also, provided a cautionary note
about tasks depending on any [nostamp] task causes the task
to always be executed and could cause unnecessary rebuild time.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Randy Witt [Tue, 11 Oct 2016 22:36:19 +0000 (15:36 -0700)]
runqueue.py: Remove redundant whitelist checks
The whitelist checks for BB_SETSCENE_ENFORCE were running for every call
to execute(). Since the task list doesn't change for each call into
execute, the checks only need to be ran once.
[YOCTO #10369]
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jussi Kukkonen [Mon, 10 Oct 2016 08:30:04 +0000 (11:30 +0300)]
depexp: Close UI with error message on NoProvider event
Without this the UI just sits there doing nothing. Showing an
infobar in-UI would be nicer but not much more useful since currently
user couldn't do anything in-UI to fix the situation. Implementation
is based on the one in knotty.
Fixes [YOCTO #9288]
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Either using the memres script or the bitbake call with --server-only
if the port is a string instead of a number then the process hangs
indefinitely causing a loop that never ends.
Add a check at the beginning for the port being a number otherwise
show an error message and exit cleanly.
[YOCTO #10397]
Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 7 Oct 2016 07:26:48 +0000 (08:26 +0100)]
runqueue: Optimise task id string manipulations
Some task id manipulations were suboptimal:
* taskfn_fromtid and fn_from_tid were effectively the same function
* many calls to split_tid(), then taskfn_fromtid()
* taskfn_fromtid() called split_tid() internally
This patch adds split_tid_mcfn() to replace split_tid() and returns the
"taskfn" variant being used in many places. We update all core calls
to the new function and ignore the return values we don't need since the
function call overhead of the split_tid wrapper is higher than ignoring
a return value.
The one remaining standalone use of taskfn_fromtid is replaced with
fn_from_tid. I couldn't see any external usage so it was dropped.
There is external usage of split_tid so a wrapper remains for it.
Combined together these changes should improve some of the runqueue task
manipulation performance.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
`if w in self.rq.worker` when w *is* self.rq.worker doesn't make a great deal
of sense, and results in this error:
File ".../poky/bitbake/lib/bb/runqueue.py", line 2372, in runQueuePipe.read():
name = None
> if w in self.rq.worker:
name = "Worker"
TypeError: unhashable type: 'dict'
Most likely this was meant to be 'is' rather than 'in', but rather than
checking after the fact, just include the name in the iteration, instead.
While we're here, also clean up and fix the broken error message.
Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Aníbal Limón [Thu, 6 Oct 2016 21:52:07 +0000 (16:52 -0500)]
ui/knotty.py: Fix signal handling of SIGWINCH in BBProgress
Add the ability to pass default signal handler for SIGWINCH in BBProgress
because with multiple instace of BBProgress the original signal handler
set by TerminalFilter (sigwinch_handle) is lost.
This is a fix for stack trace due to multiple async calls of ProgressBar
_handle_resize (ioctl to terminal fd), see:
Current thread 0x00007f70a4793700 (most recent call first):
File
"/home/alimonb/repos/poky/bitbake/lib/progressbar/progressbar.py", line
183 in _handle_resize
File "/home/alimonb/repos/poky/bitbake/lib/bb/ui/knotty.py", line 58
in _handle_resize
File "/home/alimonb/repos/poky/bitbake/lib/bb/ui/knotty.py", line 60
in _handle_resize
...
File "/home/alimonb/repos/poky/bitbake/lib/bb/ui/knotty.py", line 60
in _handle_resize
...
Aborted
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 6 Oct 2016 00:08:54 +0000 (17:08 -0700)]
toaster: Update tests to reflect front end changes
- Browser test we changed the project heading access to use the class name
- Update toastergui unit test for additional gotoUrl property
- On faster browsers we had a race for layer details inputs being
visible
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 6 Oct 2016 00:08:52 +0000 (17:08 -0700)]
toaster: importlayer Fix layer dependencies button state toggle
Fix regression introduced by switching typeahead library. Make sure we
enable and disable the add button based on whether the selection event
has fired or not.
[YOCTO #9936]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 6 Oct 2016 00:08:50 +0000 (17:08 -0700)]
toaster: buildinfohelper: Use correct way to get message from LogMessage
Use the correct method to get a message value from the LogMessage object
rather than constructing it ourselves which is not recommended. This
causes an exception when the msg contains a '%' such as when there are
wildcards in file names (something2.%.bbappends)
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 6 Oct 2016 00:08:49 +0000 (17:08 -0700)]
toaster: api / project Cancel any in progress builds before project delete
Before we finally delete any project make sure we send the cancel command to
any in-progress builds. This ensures that an inaccessible build doesn't block
up the system and that we don't get errors after deletion.
[YOCTO #10289]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Scott Rifenbark [Tue, 4 Oct 2016 16:58:44 +0000 (09:58 -0700)]
bitbake-user-manual: Added new section on BB-style functions
Fixes [YOCTO #10364]
Added a new section titled "Bitbake-Style Python Functions
Versus Python Functions". This section describes differences
for the user between the two types of functions.
Also, cleaned up a consistency problem with the terms
"BitBake style" and "BitBake-style". I used the latter
throughout the manual.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Scott Rifenbark [Wed, 21 Sep 2016 22:30:14 +0000 (15:30 -0700)]
bitbake-user-manual: Added examples for using overrides with functions.
Fixes [YOCTO #10296]
This adds some bits clarifying you can append and prepend to
functions. Added a bit to the introduction paragraph of the
"Appending and Prepending (Override Style Syntax)" section to
note that you can do this. Referenced some new examples.
In the "Shell Functions" section I added an example. In the
"BitBake Style Python Functions" section I also added an example.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bavery [Mon, 3 Oct 2016 23:56:26 +0000 (16:56 -0700)]
toaster: add Font Awesome license
Font Awesome fonts are bundled with the Toaster UI and are released
under the SIL Open Font License 1.1. This patch adds that information
to the LICENSE file.
Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bb.build: in _exec_task, catch errors from TaskStarted
We don't always want a traceback when an exception is raised by the
TaskStarted event handler. Silently return if we get a SystemExit or
HandledException, and print the error and return for FuncFailed.
This is done via a separate try/catch block, to avoid firing TaskFailed if all
the TaskStarted event handlers didn't complete, otherwise the bitbake UIs get
unhappy.
Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Joshua Lock [Tue, 4 Oct 2016 10:03:55 +0000 (11:03 +0100)]
event: prevent unclosed file warning in print_ui_queue
Use logger.addHandler(), rather than assigning an array of Handlers
to the loggers handlers property directly, to avoid a warning from
Python 3 about unclosed files:
$ bitbake
Nothing to do. Use 'bitbake world' to build everything, or run 'bitbake --help' for usage information.
WARNING: /home/joshuagl/Projects/poky/bitbake/lib/bb/event.py:143: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/joshuagl/Projects/poky/build/tmp/log/cooker/qemux86/20161004094928.log' mode='a' encoding='UTF-8'>
logger.handlers = [stdout]
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 30 Sep 2016 16:22:12 +0000 (17:22 +0100)]
data: Fix handling of vardepvalueexclude
The value used for exclusion was always being expanded. This is actually
a bad idea since in most cases you'd want to exclude an unexpanded
value and makes it impossible to use the variable as intended.
This adjusts things so the value is not expanded and we can correctly
remove things from checksums much more easily.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 27 Sep 2016 15:16:56 +0000 (16:16 +0100)]
toaster: make error message more informative
Error message
ERROR: Unprocessed MetadataEvent <bb.event.MetadataEvent object at 0x7f750e671a58>
doesn't give a lot of information about the event. It just prints
event object, which is always bb.event.MetadataEvent.
Including event type into the error message should make it more
informative:
ERROR: Unprocessed MetadataEvent TaskArtifacts
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 27 Sep 2016 15:16:55 +0000 (16:16 +0100)]
toaster: fix 'Unhandled MetadataEvent' error
New MetadataEvent 'TaskArtifacts' causes this error.
Processing of this event will hopefully be implemented in future.
For now it should be enough to just skip it.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 27 Sep 2016 15:16:54 +0000 (16:16 +0100)]
toaster: fix handling of EnvironmentError
Due to the bug in processing EnvironmentError exception,
toasterui ignores it. As EnvironmentError is a base for OSError
and IOError this means that all OSError and IOError exceptions
were silently ignored.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 27 Sep 2016 15:16:53 +0000 (16:16 +0100)]
toaster: check if file exist
Buildinfohelper assumes that all files mentioned in
manifest exist in deploy/ directory, which is not always
the case. Toaster crashes with OSError trying to
call os.stat on non-existing file.
Checking if file exists before processing it should
fix this.
[YOCTO #10185]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 27 Sep 2016 15:16:52 +0000 (16:16 +0100)]
toaster: stop modifying OEROOT in toaster script
Setting OEROOT in toaster script makes oe-init-build-env to
break with error:
bash: ../bitbake/bin/../../scripts/oe-buildenv-internal: No such file or directory
This happens because OEROOT contains path relative to build
directory.
Renamed OEROOT to OE_ROOT and unset it after it's used.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Mon, 26 Sep 2016 10:59:37 +0000 (13:59 +0300)]
toaster: layerdetails Update implementation of delete imported layer
Update the implementation of delete an imported layer so that it is
consistent with the other delete messages and wording. Also use the new
libtoaster way of setting a notification that the delete was successful.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Mon, 26 Sep 2016 10:59:36 +0000 (13:59 +0300)]
toaster: customrecipe Add frontend feature to delete custom image recipe
[YOCTO #8132]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Mon, 26 Sep 2016 10:59:35 +0000 (13:59 +0300)]
toaster: importlayer Convert success import to new notification system
Use the simpler libtoaster method of showing a notification about
successful import of a layer.
Also a number of whitespace clean ups.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Mon, 26 Sep 2016 10:59:34 +0000 (13:59 +0300)]
toaster: Add front end controls for deleting a build
Add front end modal and controls for deleting a build from the build
dashboard.
Also convert the Actions list to links instead of buttons as per the
design.
[YOCTO #6238]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Mon, 26 Sep 2016 10:59:33 +0000 (13:59 +0300)]
toaster: Add backend API for deleting a build
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Mon, 26 Sep 2016 10:59:32 +0000 (13:59 +0300)]
toaster: alerts and modals Avoid modals and alerts overlaying each other
Make sure that when we spawn a modal we clear any notifications and also
make sure that old notifications are cleared before showing a new one.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Mon, 26 Sep 2016 10:59:31 +0000 (13:59 +0300)]
toaster: project page Implement front end feature to delete project
Add confirm modal and api calls to delete a project from the project
dashboard.
[YOCTO #6238]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Mon, 26 Sep 2016 10:59:30 +0000 (13:59 +0300)]
toaster: libtoaster Add a global notification set/show mechanism
We now have a number of places where we show change notifications based
on an event in a previous page (imported a layer, deleted a build,
deleted a project etc) and we show these notifications on various pages
so we add a simple notification utility to libtoaster.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Mon, 26 Sep 2016 10:59:29 +0000 (13:59 +0300)]
toaster: move MostRecentBuildsView to its own widget
This view is specific to the builds dashboard rather than gernic api so
like ToasterTable and ToasterTypeAhead we class it as a widget as it has
a single purpose. Also clean up some flake8 identified issues.
Original author of the code moved in this commit is Elliot Smith.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Mon, 26 Sep 2016 10:59:28 +0000 (13:59 +0300)]
toaster: Clean up and convert to rest api project edit and get calls
Convert the project xhr calls into proper rest api and port the client
side calls to use the new API. Fix all the pyflakes identified issues
and clean up unused fields.
Also remove the api and client side code for changing release on the fly
as this is no longer supported.
[YOCTO #9519]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Fri, 23 Sep 2016 09:14:32 +0000 (21:14 +1200)]
knotty: ensure progress bar output is accounted for in display
When calculating how many lines we'd printed we weren't properly taking
the progress bars into account, with the result that sometimes if the
last line printed on the terminal wrapped to the next line (which is
possible) we backed up less lines than we should have.
Additionally, we should always print a newline after updating the
progress bar - there's no need to check if there wasn't output (there
always will be courtesy of our overridden _need_update()) and we now
allow the line to wrap so we don't need to check the other condition
either.
Hopefully this will fix [YOCTO #10046].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 22 Sep 2016 12:54:43 +0000 (13:54 +0100)]
knotty: Show task elapsed time
Its often useful to know how long a task has been running for. This patch
adds that information to the task display, updating every 5s if there
were no other updates so the user can see how long tasks have been running
for.
[YOCTO #9737]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
projects / thirdparty / openembedded / openembedded-core-contrib.git / log
