]> git.ipfire.org Git - thirdparty/openembedded/openembedded-core-contrib.git/log
thirdparty/openembedded/openembedded-core-contrib.git
8 years agofetch2: clean up remaining cwd saves/changes
Matt Madison [Sun, 21 Aug 2016 15:02:46 +0000 (08:02 -0700)] 
fetch2: clean up remaining cwd saves/changes

Now that the fetchers all preserve the current working
directory, the cwd changes in the try_mirror_url,
download, and checkstatus methods are no longer needed.

Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agosiggen: Fix file variable typo in compare_sigfiles
Jonathan Liu [Sat, 20 Aug 2016 08:58:13 +0000 (18:58 +1000)] 
siggen: Fix file variable typo in compare_sigfiles

Signed-off-by: Jonathan Liu <net147@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agofetch2: preserve current working directory
Matt Madison [Wed, 10 Aug 2016 17:08:16 +0000 (10:08 -0700)] 
fetch2: preserve current working directory

Fix the methods in all fetchers so they don't change
the current working directory of the calling process, which
could lead to "changed cwd" warnings from bitbake.

Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agodump_cache.py: use python3 as interpreter
Robert Yang [Thu, 18 Aug 2016 08:26:13 +0000 (01:26 -0700)] 
dump_cache.py: use python3 as interpreter

Fixed:
  File "bitbake/contrib/dump_cache.py", line 39
    print("Error, need one argument!", file=sys.stderr)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agotoasterui, knotty: don't print taskid followed by taskstring which are now in most...
Martin Jansa [Thu, 18 Aug 2016 15:06:29 +0000 (17:06 +0200)] 
toasterui, knotty: don't print taskid followed by taskstring which are now in most cases identical

* unify the format how the task is described
* don't show taskid followed by taskstring as the taskstring is
  different only for setscene tasks (by _setscene suffix)
* the duplicated output was introduced by:
  2c88afb   taskdata/runqueue: Rewrite without use of ID indirection
  as reported and confirmed as a bug here:
  http://lists.openembedded.org/pipermail/openembedded-core/2016-June/123148.html
* show:
  NOTE: Running task 541 of 548 (/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_package)
  instead of much longer:
  NOTE: Running task 541 of 548 (ID: /OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_package, /OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_package)

  and similarly for failed tasks:
  ERROR: Task (virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_install) failed with exit code '1'
  instead of much longer:
  ERROR: Task virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_install (virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_install) failed with exit code '1'

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agotests: add unit tests for the usehead url parameter
Markus Lehtonen [Thu, 18 Aug 2016 16:12:02 +0000 (19:12 +0300)] 
tests: add unit tests for the usehead url parameter

[YOCTO #9351]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agogit: Allow local repos to use HEAD
Richard Purdie [Thu, 18 Aug 2016 16:12:01 +0000 (19:12 +0300)] 
git: Allow local repos to use HEAD

Introduce a new 'usehead' url parameter for git repositories. Specifying
usehead=1 causes bitbake to use whatever commit the repository HEAD is
pointing to. Usage of usehead=1 is only allowed for local git
repositories, i.e. it must always be accompanied with protocol=file url
parameter.

[YOCTO #9351]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake-selftest: enable bitbake logging to stdout
Markus Lehtonen [Thu, 18 Aug 2016 16:55:54 +0000 (19:55 +0300)] 
bitbake-selftest: enable bitbake logging to stdout

Now you get the bb logger output for failed tests. This helps debugging
problems. Also, all stdout/stderr data for successful tests is silenced
which makes for less cluttered console output.

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake-selftest: introduce BB_TMPDIR_NOCLEAN
Markus Lehtonen [Thu, 18 Aug 2016 16:55:53 +0000 (19:55 +0300)] 
bitbake-selftest: introduce BB_TMPDIR_NOCLEAN

Set this env variable to 'yes' to preserve temporary directories used by
the fetcher tests. Useful for debugging tests.

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake-selftest: add help text for env variable(s)
Markus Lehtonen [Thu, 18 Aug 2016 16:55:52 +0000 (19:55 +0300)] 
bitbake-selftest: add help text for env variable(s)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake-selftest: utilize unittest.main better
Markus Lehtonen [Thu, 18 Aug 2016 16:55:51 +0000 (19:55 +0300)] 
bitbake-selftest: utilize unittest.main better

This simplifies the script, and, gives new features. It is now possible
to run single test functions, for example. This is nice when writing new
test cases.

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake: Update version to 1.31.1
Richard Purdie [Thu, 18 Aug 2016 09:05:26 +0000 (10:05 +0100)] 
bitbake: Update version to 1.31.1

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake: Initial multi-config support
Richard Purdie [Tue, 16 Aug 2016 16:47:06 +0000 (17:47 +0100)] 
bitbake: Initial multi-config support

This patch adds the notion of supporting multiple configurations within
a single build. To enable it, set a line in local.conf like:

BBMULTICONFIG = "configA configB configC"

This would tell bitbake that before it parses the base configuration,
it should load conf/configA.conf and so on for each different
configuration. These would contain lines like:

MACHINE = "A"

or other variables which can be set which can be built in the same
build directory (or change TMPDIR not to conflict).

One downside I've already discovered is that if we want to inherit this
file right at the start of parsing, the only place you can put the
configurations is in "cwd", since BBPATH isn't constructed until the
layers are parsed and therefore using it as a preconf file isn't
possible unless its located there.

Execution of these targets takes the form "bitbake
multiconfig:configA:core-image-minimal core-image-sato" so similar to
our virtclass approach for native/nativesdk/multilib using BBCLASSEXTEND.

Implementation wise, the implication is that instead of tasks being
uniquely referenced with "recipename/fn:task" it now needs to be
"configuration:recipename:task".

We already started using "virtual" filenames for recipes when we
implemented BBCLASSEXTEND and this patch adds a new prefix to
these, "multiconfig:<configname>:" and hence avoid changes to a large
part of the codebase thanks to this. databuilder has an internal array
of data stores and uses the right one depending on the supplied virtual
filename.

That trick allows us to use the existing parsing code including the
multithreading mostly unchanged as well as most of the cache code.

For recipecache, we end up with a dict of these accessed by
multiconfig (mc). taskdata and runqueue can only cope with one recipecache
so for taskdata, we pass in each recipecache and have it compute the result
and end up with an array of taskdatas. We can only have one runqueue so there
extensive changes there.

This initial implementation has some drawbacks:

a) There are no inter-multi-configuration dependencies as yet

b) There are no sstate optimisations. This means if the build uses the
same object twice in say two different TMPDIRs, it will either load from
an existing sstate cache at the start or build it twice. We can then in
due course look at ways in which it would only build it once and then
reuse it. This will likely need significant changes to the way sstate
currently works to make that possible.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agosiggen: properly close files rather than opening them inline
Paul Eggleton [Thu, 11 Aug 2016 03:37:00 +0000 (15:37 +1200)] 
siggen: properly close files rather than opening them inline

If you don't do this, with Python 3 you get a warning on exit under some
circumstances.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agoast/ConfHandler: Add a syntax to clear variable
Jérémy Rosen [Tue, 16 Aug 2016 12:04:47 +0000 (14:04 +0200)] 
ast/ConfHandler: Add a syntax to clear variable

unset VAR
will clear variable VAR
unset VAR[flag]
will clear flag "flag" from var VAR

Signed-off-by: Jérémy Rosen <jeremy.rosen@openwide.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agocache: Build datastores from databuilder object
Richard Purdie [Mon, 15 Aug 2016 17:03:29 +0000 (18:03 +0100)] 
cache: Build datastores from databuilder object

Rather than passing in a datastore to build on top of, use the data builder
object in the cache and base the parsed recipe from this. This turns
things into proper objects building from one another rather than messy
mixes of static and class functions.

This sets things up so we can support parsing and building multiple
configurations.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agocache: Split Cache() into a NoCache() parent object
Richard Purdie [Mon, 15 Aug 2016 17:02:41 +0000 (18:02 +0100)] 
cache: Split Cache() into a NoCache() parent object

There are some cases we want to parse recipes without any cache
setup or involvement. Split out the standalone functions into
a NoCache variant which the Cache is based upon, setting the scene
for further cleanup and restructuring.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agocache/cooker: Pass databuilder into bb.cache.Cache()
Richard Purdie [Mon, 15 Aug 2016 17:01:54 +0000 (18:01 +0100)] 
cache/cooker: Pass databuilder into bb.cache.Cache()

Rather that the current mix of static and class methods, refactor
so that the cache has the databuilder object internally. This becomes
useful for the following patches for multi config support.

It effectively completes some of the object oriented work we've been
working towards in the bitbake core for a while.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agocache: Make virtualfn2realfn/realfn2virtual standalone functions
Richard Purdie [Mon, 15 Aug 2016 17:00:45 +0000 (18:00 +0100)] 
cache: Make virtualfn2realfn/realfn2virtual standalone functions

Needing to access these static methods through a class doesn't
make sense. Move these to become module level standalone functions.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agocache/ast: Move __VARIANTS handling to parse cache function
Richard Purdie [Mon, 15 Aug 2016 16:59:56 +0000 (17:59 +0100)] 
cache/ast: Move __VARIANTS handling to parse cache function

Simple refactoring to allow for multiconfig support.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agorunqueue: Abstract worker functionality to an object/array
Richard Purdie [Mon, 15 Aug 2016 16:58:39 +0000 (17:58 +0100)] 
runqueue: Abstract worker functionality to an object/array

With the introduction of multi-config and the possibility of distributed
builds we need arrays of workers rather than the existing two.

This refactors the code to have a dict() of workers and a dict of
fakeworkers, represented by objects. The code can iterate over these.

This is separated out from the multi-config changes since its separable
and clearer this way.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agocookerdata: Simplify prefiles/postfiles
Richard Purdie [Mon, 15 Aug 2016 16:57:20 +0000 (17:57 +0100)] 
cookerdata: Simplify prefiles/postfiles

The current codepaths are rather confusing. Stop passing these
as parameters and use the ones from when the object is created.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake-user-manual: Re-write "Dependencies Internal to the .bb File"
Scott Rifenbark [Fri, 12 Aug 2016 16:17:02 +0000 (09:17 -0700)] 
bitbake-user-manual: Re-write "Dependencies Internal to the .bb File"

Fixes [YOCTO #10117]

Applied a re-write to better clarify the behavior of dependencies.

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake-user-manual: Added setting variable for a single task
Scott Rifenbark [Mon, 8 Aug 2016 16:38:00 +0000 (09:38 -0700)] 
bitbake-user-manual: Added setting variable for a single task

Fixes [YOCTO #10095]

I added a third case to the "Conditional Metadata" section to
describe setting a variable for a single task.

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake-user-manual: Added more detail to anonymous Python functions.
Scott Rifenbark [Mon, 8 Aug 2016 15:54:41 +0000 (08:54 -0700)] 
bitbake-user-manual: Added more detail to anonymous Python functions.

Fixes [YOCTO #10093]

Provided much more detail on how these functions work.

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake-user-manual: Formatted all "flags" to be consistent
Scott Rifenbark [Thu, 4 Aug 2016 00:44:53 +0000 (17:44 -0700)] 
bitbake-user-manual: Formatted all "flags" to be consistent

Fixes [YOCTO #10071]

The use of any flags throughout the manual was very inconsistent.
I changed all references to any named flag in the text to be
formatted as code and to be enclosed in square brackets.

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake-user-manual: Added detail to [dirs] and [cleardirs] flags
Scott Rifenbark [Thu, 4 Aug 2016 00:22:58 +0000 (17:22 -0700)] 
bitbake-user-manual: Added detail to [dirs] and [cleardirs] flags

Fixes [YOCTO #10071]

Provided more clear descriptions for these two flags.

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobb/utils.py: export_proxies add GIT_PROXY_COMMAND
Francisco Pedraza [Wed, 10 Aug 2016 21:00:04 +0000 (16:00 -0500)] 
bb/utils.py: export_proxies add GIT_PROXY_COMMAND

This was added to enable the usage of git through proxies.

Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agoknotty: don't show number of running tasks in quiet mode
Paul Eggleton [Thu, 11 Aug 2016 03:36:59 +0000 (15:36 +1200)] 
knotty: don't show number of running tasks in quiet mode

There's not a whole lot of point showing how many tasks are running when
we're in quiet mode, it just looks a bit strange particularly when it's
not running any tasks.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agoknotty: fix task progress bar not starting at 0%
Paul Eggleton [Thu, 11 Aug 2016 03:36:58 +0000 (15:36 +1200)] 
knotty: fix task progress bar not starting at 0%

If we have the task number here we need to subtract 1 to get the number
of tasks completed.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agorunqueue: fix two minor issues with the initialising tasks progress
Paul Eggleton [Thu, 11 Aug 2016 03:36:57 +0000 (15:36 +1200)] 
runqueue: fix two minor issues with the initialising tasks progress

A couple of fixes for the "Initialising tasks" progress bar behaviour:
* Properly finish the progress bar when using bitbake -S
* Finish the progress bar before calling BB_HASHCHECK_FUNCTION (so that
  in OE when that shows its own "Checking sstate mirror object
  availability"  progress bar it gets shown on the next line as it
  should).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agolib/bb/tests/fetch: remove URL that doesn't exist anymore
Ross Burton [Mon, 15 Aug 2016 14:20:50 +0000 (15:20 +0100)] 
lib/bb/tests/fetch: remove URL that doesn't exist anymore

The CUPS ipptool URL we were checking now redirects to github where the tarball
isn't present, so remove it from the test suite.

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agotoaster: update web urls for openembedded-core's special case
David Reyna [Mon, 15 Aug 2016 10:41:33 +0000 (11:41 +0100)] 
toaster: update web urls for openembedded-core's special case

The layer index update command has a special case for the
updating 'openembedded-core' layer, and it was missing reading
and updating the git web URL fields.

[YOCTO #8037]

Signed-off-by: David Reyna <david.reyna@windriver.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agotoaster: buildinfohelper Add handling local layers (i.e. non-git) layers
Michael Wood [Tue, 9 Aug 2016 12:50:12 +0000 (13:50 +0100)] 
toaster: buildinfohelper Add handling local layers (i.e. non-git) layers

Adds handling of the non-git layers to create and update the
corresponding layer objects in Toaster.

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
9 years agotoaster: bldcontrol model BRLayer Add corresponding local_source_dir
Michael Wood [Mon, 8 Aug 2016 14:11:27 +0000 (15:11 +0100)] 
toaster: bldcontrol model BRLayer Add corresponding local_source_dir

Sync the BRLayer object with the new field added to the Layer object.
The BRLayer (BuildRequest Layers) are snapshots of the layers in the
project at build time and therefore need to mirror the required fields
of the layer object.

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
9 years agotoaster: adapt xhr_import_layer test locally imported layer
Sujith H [Tue, 2 Aug 2016 15:56:17 +0000 (15:56 +0000)] 
toaster: adapt xhr_import_layer test locally imported layer

Update xhr_import_layer test to use locally imported
layer in local_source_dir.

[YOCO #9911]

Signed-off-by: Sujith H <sujith.h@gmail.com>
9 years agotoaster: update build configuration page
Sujith H [Mon, 1 Aug 2016 07:25:15 +0000 (07:25 +0000)] 
toaster: update build configuration page

Update layer branch and layer commit section
in the build configuration page for locally
imported layers. For locally imported layers
this secion goes as "Not applicable".

[YOCO #9911]

Signed-off-by: Sujith H <sujith.h@gmail.com>
9 years agotoaster: update recipe details page
Sujith H [Fri, 29 Jul 2016 10:42:30 +0000 (10:42 +0000)] 
toaster: update recipe details page

Update Layer branch and Layer commit
in recipe details page. For local layer
imported from toaster these fields are
not applicable.

[YOCO #9911]

Signed-off-by: Sujith H <sujith.h@gmail.com>
9 years agotoaster: update package detail page
Sujith H [Fri, 29 Jul 2016 07:15:24 +0000 (07:15 +0000)] 
toaster: update package detail page

Update package details pages layer branch
and layer commit section for locally imported
layers. We add Not applicable to them along
with a helper text, which helps user understand
why its not applicable to those layers.

[YOCO #9911]

Signed-off-by: Sujith H <sujith.h@gmail.com>
9 years agotoaster: updte build tables for locally imported layers
Sujith H [Fri, 29 Jul 2016 07:11:19 +0000 (07:11 +0000)] 
toaster: updte build tables for locally imported layers

Update the build tables section for locally
imported layers. Here we do not provide informations
such as branch or commit. Because those are locally
imported layer(s).

[YOCO #9911]

Signed-off-by: Sujith H <sujith.h@gmail.com>
9 years agotoaster: update the tables information
Sujith H [Fri, 29 Jul 2016 07:03:39 +0000 (07:03 +0000)] 
toaster: update the tables information

Update table informations for pages:
1) Compatible layers
2) Compatible image recipes
Added Not Applicable to the fields of locally
imported layers because they are not git version.

[YOCO #9911]

Signed-off-by: Sujith H <sujith.h@gmail.com>
9 years agotoaster: collect details for local dir imported from ui
Sujith H [Wed, 20 Jul 2016 08:58:57 +0000 (08:58 +0000)] 
toaster: collect details for local dir imported from ui

Collect the dir path imported from UI and make
sure that its getting added to bblayers.conf.
This patch exactly does the same job. Any layer
which is imported locally need not be cloned
again to _toaster_clones dir.

[YOCO #9911]

Signed-off-by: Sujith H <sujith.h@gmail.com>
9 years agotoaster: add local_source_dir field to model
Sujith H [Wed, 20 Jul 2016 08:57:11 +0000 (08:57 +0000)] 
toaster: add local_source_dir field to model

Add a new field local_source_dir to model.
This will clearly differentiate us from the
vcs_url which is for git path.
Adding migration file 0010_layer_local_source_dir_path.py
along with this patch.

[YOCO #9911]

Signed-off-by: Sujith H <sujith.h@gmail.com>
9 years agotoaster: add local directory to the db
Sujith H [Tue, 26 Jul 2016 09:38:17 +0000 (09:38 +0000)] 
toaster: add local directory to the db

Adding local directory to the database.
The local directory is added to vcs_url,
field of db.

[YOCO #9911]

Signed-off-by: Sujith H <sujith.h@gmail.com>
9 years agotoaster: improve the display when local dir is added
Sujith H [Tue, 19 Jul 2016 07:57:09 +0000 (07:57 +0000)] 
toaster: improve the display when local dir is added

The helper text gets displayed accordingly when mouse
is hovered above the layers. If its a local directory
then no more branch is mentioned. Only directory path
is mentioned.

[YOCTO #9911]

Signed-off-by: Sujith H <sujith.h@gmail.com>
9 years agotoaster: update css file for layer import
Belen Barros Pena [Tue, 26 Jul 2016 08:08:42 +0000 (08:08 +0000)] 
toaster: update css file for layer import

Update the css file for the layer import.
This changes will improve the UI for the
task.

[YOCO #9911]

Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
9 years agotoaster: add local_source_dir to the needed_fields
Sujith H [Wed, 27 Jul 2016 06:16:29 +0000 (06:16 +0000)] 
toaster: add local_source_dir to the needed_fields

This change will help us know if local_source_dir
is null or if there is value associated with this
field. This change will help us display the details
for duplicate layers tryied to import locally.

[YOCOT #9911]

Signed-off-by: Sujith H <sujith.h@gmail.com>
9 years agotoaster: control the selection of git and local repo
Sujith H [Tue, 19 Jul 2016 07:54:49 +0000 (07:54 +0000)] 
toaster: control the selection of git and local repo

Changes made in this patch will help user to select
either repo based on git and local. This patch also
improves the help provided to user so that user can
understand them and take necessary action to proceed.

[YOCOT #9911]

Signed-off-by: Sujith H <sujith.h@gmail.com>
9 years agotoaster: add import of local directory to UI
Sujith H [Tue, 19 Jul 2016 07:51:50 +0000 (07:51 +0000)] 
toaster: add import of local directory to UI

This patch will help to add local directory
to UI. The modification is made in importlayer.html
file. The radio buttons are created to separate
selection of git based repo and local directory.

Co-Author: Belen Barros Pena <belen.barros.pena@linux.intel.com>

[YOCOT #9911]

Signed-off-by: Sujith H <sujith.h@gmail.com>
9 years agotoaster: fix unhandled exception
Ed Bartosh [Wed, 27 Jul 2016 12:15:02 +0000 (15:15 +0300)] 
toaster: fix unhandled exception

manage.py lsupdates throws DataError exception if the recipe can't be
saved to the MySQL database:
    django.db.utils.DataError: (1406, "Data too long for column 'license' at row 1"

Adding DataError exception to the list of exceptions should make
lsupdates to print a warning message and skip the recipe.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
9 years agotoasterui: ensure that the Build object is always available
Elliot Smith [Thu, 4 Aug 2016 15:32:33 +0000 (16:32 +0100)] 
toasterui: ensure that the Build object is always available

Many of the methods in toasterui and buildinfohelper rely
on the internal state of the buildinfohelper; in particular, they
need a Build object to have been created on the buildinfohelper.

If the creation of this Build object is tied to an event which
may or may not occur, there's no guarantee that it will exist.
This then causes assertion errors in those methods.

To prevent this from happening, add an _ensure_build() method
to buildinfohelper. This ensures that a minimal Build object
is always available whenever it is needed, either by retrieving
it from the BuildRequest or creating it; it also ensures that
the Build object is up to date with whatever data is available
on the bitbake server (DISTRO, MACHINE etc.).

This method is then called by any other method which relies on
a Build object being in the internal state, ensuring that the
object is either available, or creating it.

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster-tests: add test for showing self-dependent task
Elliot Smith [Fri, 29 Jul 2016 11:19:18 +0000 (12:19 +0100)] 
toaster-tests: add test for showing self-dependent task

Toaster occasionally records a task which depends on itself.

Add a test which checks that a task which depends on itself
can be displayed in the task page.

[YOCTO #9952]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster: prevent infinite loop when finding task dependencies
Elliot Smith [Fri, 29 Jul 2016 11:25:46 +0000 (12:25 +0100)] 
toaster: prevent infinite loop when finding task dependencies

Toaster occasionally records a task which depends on itself.
This causes a problem when trying to display that task if it
is "covered" by itself, as the code does the following: for
task A, find a task B which covers A; then, recursively
find the task which covers B etc. If B == A, this loop becomes
infinite and never terminates.

To prevent this, add the condition that, when finding a task B
which covers A, don't allow B == A.

[YOCTO #9952]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster-tests: add tests for reverting to default sort
Elliot Smith [Tue, 19 Jul 2016 13:47:12 +0000 (14:47 +0100)] 
toaster-tests: add tests for reverting to default sort

Add tests for ToasterTable UI table sort reverting, which can
only be exercised via the browser.

Check that if a table is sorted by a column, and that column
is hidden, then the sort reverts to the default for the table.

[YOCTO #9836]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster: reset table to default orderby when orderby column is hidden
Elliot Smith [Mon, 18 Jul 2016 15:02:14 +0000 (16:02 +0100)] 
toaster: reset table to default orderby when orderby column is hidden

When a ToasterTable is sorted by a column, and that column is
hidden from view, the sort doesn't revert to the default for the
table.

Modify the JS responsible for reloading the table data so that
it doesn't rely on clicking a table column heading (as this is
inflexible and error-prone). Instead, use a function to apply
the sort to the table; and call that function when column
headings are clicked.

This means that the ordering can be changed programmatically
to a specified default ordering when a column is hidden, without
having to click on a column heading.

Use this function when the current sort column is hidden, to
apply the default sort for the table.

[YOCTO #9836]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster: show loading spinner after creating custom image
Elliot Smith [Fri, 15 Jul 2016 14:20:36 +0000 (15:20 +0100)] 
toaster: show loading spinner after creating custom image

Creating a custom image through the "New custom image" dialog
can sometimes result in a long pause between pressing the button
to create the image, and being transferred to the page showing
details of its content. This can make it appear as though pressing
the button had no effect.

To prevent this from happening, disable the button and text box
in the new custom image dialog after the "Create image" button is
pressed. Also show a loading spinner and "loading..." text on
the button to make it clear that the application is still responding.

[YOCTO #9475]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster-tests: fix erroneous message when test fails
Elliot Smith [Fri, 15 Jul 2016 11:09:57 +0000 (12:09 +0100)] 
toaster-tests: fix erroneous message when test fails

When one of the layer details tests fails (as it occasionally
does, if running on a machine under heavy load, due to sync issues),
the error message shown is misleading, as it is something like:

"Expected 'This was imported' in ['This was imported', ...]"

The string 'This was imported' is in the list shown in the message,
but the message suggests it isn't.

This is because the test compares the string with one list, but
then uses a different list in the fail message if the comparison
fails.

Fix the list shown in the message about the test failing.

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster-tests: add tests for most recent builds state changes
Elliot Smith [Thu, 14 Jul 2016 14:58:00 +0000 (15:58 +0100)] 
toaster-tests: add tests for most recent builds state changes

Add tests for the state transitions in the "most recent builds"
area of the all builds page.

[YOCTO #9631]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster-tests: add tests for build dashboard menu and summary visibility
Elliot Smith [Wed, 13 Jul 2016 14:12:53 +0000 (15:12 +0100)] 
toaster-tests: add tests for build dashboard menu and summary visibility

Test that the build dashboard only shows a menu and a build
summary area if a build has properly "started" (i.e. has at least
one Variable object associated with it).

[YOCTO #8443]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster-tests: add tests for build time links in the all builds page
Elliot Smith [Tue, 26 Apr 2016 15:32:11 +0000 (16:32 +0100)] 
toaster-tests: add tests for build time links in the all builds page

When a build fails, it shouldn't have links on its build time in the
recent builds area or in the all builds table.

[YOCTO #8443]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster: adjust build dashboard for failed builds
Elliot Smith [Wed, 13 Jul 2016 13:42:31 +0000 (14:42 +0100)] 
toaster: adjust build dashboard for failed builds

Remove the side bar and build details modules for failed builds.

[YOCTO #8443]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster: add started property to Build
Elliot Smith [Wed, 13 Jul 2016 13:39:47 +0000 (14:39 +0100)] 
toaster: add started property to Build

Add a property to the Build model which records whether
the BuildStarted event has occurred for the build.

The proxy for this event is the presence of variables recorded
against the Build: as the buildinfohelper only saves variables
when the BuildStarted event occurs (as the variables aren't
available on the bitbake server before that point), we can
tell whether BuildStarted has happened by counting Variable
objects on the Build.

This can then be used to determine whether a Build "properly"
started, enabling a different dashboard display (left-hand menu
hidden) if the build didn't record any useful information (e.g.
if it had a bad target).

[YOCTO #8443]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster: remove links from time field on failed builds
Elliot Smith [Wed, 13 Jul 2016 10:37:01 +0000 (11:37 +0100)] 
toaster: remove links from time field on failed builds

Failed builds don't have any time data recorded for them,
so the time field in the builds table, the time shown
in the recent builds area, and the build time shown in
the build dashboard should not be links for failed builds.

[YOCTO #8443]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster: tweak styling and typos in recent builds area
Elliot Smith [Tue, 12 Jul 2016 15:38:06 +0000 (16:38 +0100)] 
toaster: tweak styling and typos in recent builds area

Help icons need to have the Bootstrap tooltip() method called on
them so that the popups are correctly styled.

Ensure that the colour of the help/error/warning icons is correct,
depending on the build state.

Fix pluralisation of errors and warnings shown.

Add a div around the build state area so it's easy to pick up
where the state is going to display (e.g. in tests).

[YOCTO #9631]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster: show "Tasks starting..." until the first task completes
Elliot Smith [Tue, 12 Jul 2016 15:14:42 +0000 (16:14 +0100)] 
toaster: show "Tasks starting..." until the first task completes

To prevent showing a "0% of tasks complete" message for a long time,
don't show the progress bar until the first task has finished.

While waiting for that first task, show a message about tasks
starting instead.

[YOCTO #9631]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster: show progress of recipe parsing in recent builds area
Elliot Smith [Mon, 11 Jul 2016 13:47:06 +0000 (14:47 +0100)] 
toaster: show progress of recipe parsing in recent builds area

Modify buildinfohelper and toasterui so that they record the
recipe parse progress (from ParseProgress events in bitbake)
on the Build object.

Note that because the Build object is now created at the
point when ParseStarted occurs, it is necessary to set the
build name to the empty string initially (hence the migration).
The build name can be set when the build properly starts,
i.e. at the BuildStarted event.

Then use this additional data to determine whether a Build
is in a "Parsing" state, and report this in the JSON API.
This enables the most recent builds area to show the recipe
parse progress.

Add additional logic to update the progress bar if the progress
for a build object changes.

[YOCTO #9631]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster: move most recent builds templating to client
Elliot Smith [Wed, 29 Jun 2016 14:41:56 +0000 (15:41 +0100)] 
toaster: move most recent builds templating to client

The most recent builds area of the all builds and project builds
table needs to update as a build progresses. It also needs
additional functionality to show other states (e.g. recipe parsing,
queued) which again needs to update on the client side.

Rather than add to the existing mix of server-side templating
with client-side DOM updating, translate all of the server-side
templates to client-side ones (jsrender), and add logic which
updates the most recent builds area as the state of a build changes.

Add a JSON API for mostrecentbuilds, which returns the state of
all "recent" builds. Fetch this via Ajax from the build dashboard
(rather than fetching the ad hoc API as in the previous version).

Then, as new states for builds are fetched via Ajax, determine
whether the build state has changed completely, or whether the progress
has just updated. If the state completely changed, re-render the
template on the client side for that build. If only the progress
changed, just update the progress bar. (NB this fixes the
task progress bar so it works for the project builds and all builds
pages.)

In cases where the builds table needs to update as the result of
a build finishing, reload the whole page.

This work highlighted a variety of other issues, such as
build requests not being able to change state as necessary. This
was one part of the cause of the "cancelling build..." state
being fragile and disappearing entirely when the page refreshed.
The cancelling state now persists between page reloads, as the
logic for determining whether a build is cancelling is now on
the Build object itself.

Note that jsrender is redistributed as part of Toaster, so
a note was added to LICENSE to that effect.

[YOCTO #9631]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agocooker: add BuildInit event
Elliot Smith [Mon, 11 Jul 2016 15:23:05 +0000 (16:23 +0100)] 
cooker: add BuildInit event

In situations where a bitbake run fails before the build
properly starts and BuildStarted is fired, a UI has no way
to get at the targets passed to the build. This makes it
difficult for the UI to report on the targets which failed.

Fire a BuildInit event before running buildTargets() or
buildFile(). This enables a UI to capture targets passed to
buildTargets(), even if the build fails (e.g. the targets
themselves are invalid).

[YOCTO #8440]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
9 years agotoaster: fixtures Add README
Michael Wood [Tue, 2 Aug 2016 13:19:17 +0000 (14:19 +0100)] 
toaster: fixtures Add README

Add README to explain fixtures directory

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
9 years agotoaster: Remove old toaster config loading mechanism
Michael Wood [Mon, 1 Aug 2016 18:49:43 +0000 (19:49 +0100)] 
toaster: Remove old toaster config loading mechanism

This has been replaced using django's inbuilt loaddata.
Django command documented at:
https://docs.djangoproject.com/en/1.8/ref/django-admin/#django-admin-loaddata

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
9 years agotoaster: lsupdates Add spinner for parsing/http fetch
Michael Wood [Mon, 1 Aug 2016 18:38:28 +0000 (19:38 +0100)] 
toaster: lsupdates Add spinner for parsing/http fetch

Adds a spinner so that you know that the parse and http fetch from the
layerindex is in progress.

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
9 years agotoaster: checksettings call django's loaddata instead of custom command
Michael Wood [Mon, 1 Aug 2016 18:32:53 +0000 (19:32 +0100)] 
toaster: checksettings call django's loaddata instead of custom command

Call django's inbuilt loaddata command to load the appropriate fixtures.
We also attempt to load a fixture called "custom" and fail silently if
we don't have one. This is where initial customisations can be done to
load particular settings or data into Toaster (for example layers or
default values for variables)

Make sure the value for TEMPLATECONF is available to checksettings so
that we can have a go a working out which default data to load.

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
9 years agotoaster: Add poky and openembedded configuration fixtures
Michael Wood [Mon, 25 Jul 2016 12:43:08 +0000 (13:43 +0100)] 
toaster: Add poky and openembedded configuration fixtures

These fixtures provide a recommended default configuration of toaster for
either using bitbake and oe-core or as part of poky.
They can be used as the sample configuration for writing custom configurations.

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
9 years agotoaster: tests test_new_project_page Catch button not enabled exception
Michael Wood [Fri, 5 Aug 2016 08:26:49 +0000 (09:26 +0100)] 
toaster: tests test_new_project_page Catch button not enabled exception

When using firefox and selenium we get an exception generated when a
disabled button click is attempted. This should happen in the test but
we need to catch the exception to make sure it doesn't cause the test to
fail.

[YOCTO #10056]

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agotoaster: --help now returns 0 instead of 1
Randy Witt [Tue, 2 Aug 2016 23:43:54 +0000 (16:43 -0700)] 
toaster: --help now returns 0 instead of 1

If the user explicitly passes in "--help" then it should return 0. This
is the convention follow by the typical application. This allows the
user to check for options without triggering an error.

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agosiggen: Fix typo
Ulrich Ölmann [Mon, 1 Aug 2016 08:43:00 +0000 (10:43 +0200)] 
siggen: Fix typo

Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agofetch2/gitannex.py: use 'git annex init' instead of 'git annex sync'
Terry Boese [Wed, 10 Aug 2016 15:14:15 +0000 (09:14 -0600)] 
fetch2/gitannex.py: use 'git annex init' instead of 'git annex sync'

The git annex fetcher needs git annex to be initialized.  Previously
it was using 'git annex sync' to do this, but that has the downside
of moving the checkout to the tip of the default branch.  This means
that tags, SRCREV, etc don't work in the gitannex case.

Signed-off-by: Terry Boese <terry.boese@vecima.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agopersist_data: Fix py3 update stack overflow
Richard Purdie [Wed, 10 Aug 2016 15:00:33 +0000 (16:00 +0100)] 
persist_data: Fix py3 update stack overflow

Revision d0f904d407f57998419bd9c305ce53e5eaa36b24 accidentally broke
items() and values() and made them cause stack overflows. Undo that
breakage.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agofetch: Fix use of tar's --exclude option for tar >= 1.29
Mariano Lopez [Fri, 5 Aug 2016 13:54:44 +0000 (13:54 +0000)] 
fetch: Fix use of tar's --exclude option for tar >= 1.29

Starting from tar 1.29 the --exclude option won't work
anymore if is not used before the path. There are some
fetch modules that copy the ptest using tar and --exclude
option. This fixes these for bitbake.

[YOCTO #9763]

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake-user-manual: Applied typo fix to "Override Style Operation Advantages"
Scott Rifenbark [Mon, 1 Aug 2016 18:52:10 +0000 (11:52 -0700)] 
bitbake-user-manual: Applied typo fix to "Override Style Operation Advantages"

Fixes [YOCTO #9985]

Fixed an operator typo from ":=" to "+=" in the note
at the bottom of the section.

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
9 years agobitbake-user-manual: Changed bitbake command example.
Scott Rifenbark [Fri, 29 Jul 2016 21:18:35 +0000 (14:18 -0700)] 
bitbake-user-manual: Changed bitbake command example.

Fixes [YOCTO #7718]

In the "Executing a List of Task and Recipe Combinations"
section, I changed the improper bitbake command example to
use the correct syntax.  This change was review feedback.

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
9 years agobitbake-user-manual: Updated the PROVIDES variable
Scott Rifenbark [Fri, 29 Jul 2016 21:08:25 +0000 (14:08 -0700)] 
bitbake-user-manual: Updated the PROVIDES variable

Fixes [YOCTO #10011]

Added paragraphs near the end to describe the role of virtual
targets.

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
9 years agobitbake-user-manual: Added non-existant variable expansion
Scott Rifenbark [Fri, 22 Jul 2016 15:36:09 +0000 (08:36 -0700)] 
bitbake-user-manual: Added non-existant variable expansion

Fixes [YOCTO #10003]

I added a small paragraph explaining what happens when expansion
of a variable that does not exist occurs.

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
9 years agobitbake-user-manual: Applied some review comments
Scott Rifenbark [Thu, 21 Jul 2016 18:27:57 +0000 (11:27 -0700)] 
bitbake-user-manual: Applied some review comments

Fixes [YOCTO #9985]

Fixed some problems with the changes.  A small typo for an
example and added a clarifying operator in another sentence.

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
9 years agobitbake-user-manual: Updated the "Inline Python Variable Expansion" section.
Scott Rifenbark [Thu, 21 Jul 2016 18:02:52 +0000 (11:02 -0700)] 
bitbake-user-manual: Updated the "Inline Python Variable Expansion" section.

Fixes [YOCTO #9984]

Added a small note to the bottom to help clarify.

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
9 years agobitbake-user-manual: Fixed override operator syntax.
Scott Rifenbark [Thu, 21 Jul 2016 17:51:11 +0000 (10:51 -0700)] 
bitbake-user-manual: Fixed override operator syntax.

I had inconsistent usage of the "_append" style operator syntax
in the chaper.  I was using a mix of <filename>_append</filename>
and "_append".  I changed to "_append" for consistency.

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
9 years agobitbake-user-manual: Clarified override-style operators.
Scott Rifenbark [Thu, 21 Jul 2016 17:37:09 +0000 (10:37 -0700)] 
bitbake-user-manual: Clarified override-style operators.

Fixes [YOCTO #9985]

Made the following changes:

 * Section Removal (Override Style Syntax):  Added a small
   qualifying sentence at the end to further define behavior

 * Added new section "Override Style Operation Advantages":
   This section provides some rationale behind the "_append"
   style operations.

 * Section "Examples": Changed an example to use the "="
   operator rather than the "+=" operator.

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
9 years agobitbake-user-manual: Updated the variable expansion section.
Scott Rifenbark [Thu, 21 Jul 2016 16:49:08 +0000 (09:49 -0700)] 
bitbake-user-manual: Updated the variable expansion section.

Fixes [YOCTO #9984]

Added more detail to the examples that show the effects of
variable expanison.

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
9 years agoRevert "bitbake-user-manual: Added new section on command-line execution"
Richard Purdie [Tue, 2 Aug 2016 14:10:38 +0000 (15:10 +0100)] 
Revert "bitbake-user-manual: Added new section on command-line execution"

This reverts commit 6f6cd0674fd1595f4e74b7da692e0c348b2660c6 as it was
a duplicated commit.

9 years agolib/toaster: Fix missing new files from previous commits
Richard Purdie [Mon, 1 Aug 2016 08:38:23 +0000 (09:38 +0100)] 
lib/toaster: Fix missing new files from previous commits

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agotoaster-tests: fix URL given for Chromedriver download
Elliot Smith [Wed, 20 Jul 2016 16:37:04 +0000 (17:37 +0100)] 
toaster-tests: fix URL given for Chromedriver download

The link to the Chromedriver downloads page is dead, so
put in the correct URL.

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agotoaster-tests: define capabilities for latest Firefox driver
Elliot Smith [Wed, 20 Jul 2016 16:37:03 +0000 (17:37 +0100)] 
toaster-tests: define capabilities for latest Firefox driver

For the latest Firefox versions, WebDriver requires a download of a
separate binary and an additional capability to be defined on it.

Modify our tests so that when "marionette" is set as the browser,
this capability is defined on the Firefox driver. Also add a note to the
README about the additional installation steps required.

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agolib/bb/checksum: avoid exception on broken symlinks
Paul Eggleton [Tue, 26 Jul 2016 03:36:40 +0000 (15:36 +1200)] 
lib/bb/checksum: avoid exception on broken symlinks

If using OE's externalsrc with a source tree that is not tracked by git
and contains broken symlinks, you can receive "TypeError: unorderable
types: NoneType() < str()" within the file checksum code due to:

 checksums.sort(key=operator.itemgetter(1))

Don't add files with no checksum to the checksums list in order to avoid
this.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agolib/bb/utils: show subprocess output in stack traces
Ross Burton [Mon, 18 Jul 2016 16:22:55 +0000 (17:22 +0100)] 
lib/bb/utils: show subprocess output in stack traces

If better_exec() throws a subprocess.CalledProcessError then show the output to
the user as it likely contains useful information for solving the problem.

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agofetch2: fix pickle issues while switching from master to krogoth
Maxin B. John [Fri, 29 Jul 2016 08:23:35 +0000 (11:23 +0300)] 
fetch2: fix pickle issues while switching from master to krogoth

While switching from master to krogoth build with a common download directory,
got a large number of warnings like the one listed below:

WARNING: freetype-2.6.3-r0 do_fetch: Couldn't load checksums from
donestamp /home/maxin/downloads/freetype-2.6.3.tar.bz2.done: ValueError
(msg: unsupported pickle protocol: 4)

These warnings are caused by the difference in pickle module
implementation in python3(master) and python2(krogoth). Python2 supports
3 different protocols (0, 1, 2) and pickle.HIGHEST_PROTOCOL is 2 where as
Python3 supports 5 different protocols (0, 1, 2, 3, 4) and
pickle.HIGHEST_PROTOCOL is obviously 4.

My suggestion is to use 2 since it is backward compatible with python2
(all the supported distros for krogoth provides python2 which supports
pickle protocol version 2)

Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agolib/bb/build: handle incomplete message fragments in log FIFO
Ross Burton [Mon, 25 Jul 2016 22:32:26 +0000 (23:32 +0100)] 
lib/bb/build: handle incomplete message fragments in log FIFO

It's possible that the logging FIFO doesn't do a complete read (or the sender a
complete write) with the result that an incomplete message is read in bitbake.
This used to result in silently truncated lines but since 42d727 now also
results in a warning as the start of the rest of the message isn't a valid
logging command.

Solve this by storing incoming bytes in a bytearray() across reads, and parsing
complete messages from that.

[ YOCTO #9999 ]

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agotoaster-tests: add tasks and recipes sub-page tests
Elliot Smith [Thu, 21 Jul 2016 15:58:40 +0000 (18:58 +0300)] 
toaster-tests: add tasks and recipes sub-page tests

Add tests for the tasks and recipes sub-pages of the build
dashboard.

[YOCTO #9833]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agotoaster: set non-hideable columns for built recipes table
Elliot Smith [Thu, 21 Jul 2016 15:58:39 +0000 (18:58 +0300)] 
toaster: set non-hideable columns for built recipes table

None of the columns in the built recipes table are marked
as not hideable, so it is possible to remove all the columns
and make the table disappear.

Set the recipe name and version columns as not hideable.

Also rename the "Name" column to "Recipe", for consistency with
the design and with other recipe tables.

[YOCTO #9833]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agotoaster: set non-hideable columns for build tasks table
Elliot Smith [Thu, 21 Jul 2016 15:58:38 +0000 (18:58 +0300)] 
toaster: set non-hideable columns for build tasks table

The task, recipe and order columns in the build tasks table
should not be hideable. If they are, it's possible for the
table to have all of its columns hidden so that it no longer
displays.

Set the hideable property to prevent these columns from being
hidden.

[YOCTO #9833]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agotoaster: fix download URL for task logs
Elliot Smith [Thu, 21 Jul 2016 15:42:35 +0000 (18:42 +0300)] 
toaster: fix download URL for task logs

The task display template formatting had split the Django
url template tag across two lines and broken it. This resulted
in a gibberish URL for task logs.

Fix by placing the tag and its arguments on a single line.

[YOCTO #9837]

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>