Ed Bartosh [Wed, 23 Sep 2015 22:34:54 +0000 (15:34 -0700)]
toaster: use git reset --hard instead of rebase
Replaced 'git checkout <ref> && git rebase' with 'git reset --hard' as
with git checkout repository ends up with detached HEAD. Rebase makes
things even worse as it can cause conflicts. git reset --hard resets
repository to the required state in a most straightforward and
reliable way.
[YOCTO #7505]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Wed, 23 Sep 2015 22:34:53 +0000 (15:34 -0700)]
toaster: don't use --single-branch when cloning
git clone --single-branch works only with ref names. It fails if
commit sha1 is set in layer configuration with this error:
fatal: Remote branch <commit sha1> not found in upstream origin
Cloning repository without using --single-branch should work for
refs and commit sha1.
[YOCTO #7505]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Markus Lehtonen [Thu, 24 Sep 2015 13:02:23 +0000 (16:02 +0300)]
utils: only add layer once in edit_bblayers_conf()
Prevent edit_bblayers_conf() from adding layer(s) multiple times. This
happened when BBLAYERS variable was "listed" multiple times in
bblayer.conf - i.e. the configuration was split into multiple separate
assignments.
[YOCTO #8316]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
David Reyna [Tue, 29 Sep 2015 07:15:19 +0000 (08:15 +0100)]
toaster: display most recent builds for projects
Display the most recent builds in the given project's build page.
[YOCTO #8186]
Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 29 Sep 2015 09:50:10 +0000 (10:50 +0100)]
toaster: orm remove the complicated querying on the ORM
We no longer need to compute each layer_version and all the recipes
which belong to this.
[YOCTO #8147]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 29 Sep 2015 04:45:30 +0000 (21:45 -0700)]
toaster: buildinfohelper Create a copy of the built layer and recipe
Create a copy of the built layer and the recipes associated with it.
This is so that the user can view the historical information about a
build. i.e. a snapshot of the layer version and artifacts produced at
that build.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 29 Sep 2015 04:45:29 +0000 (21:45 -0700)]
toaster: tables show all recipes in the layerdetails even duplicates
For some layers we have multiple recipes for the same software with
differing versions. Change to showing all versions and add a version
column to the table instead.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 29 Sep 2015 04:45:28 +0000 (21:45 -0700)]
toaster: Prioroitise the layer more generic vcs reference over the sha
When we do a build we update the last commit value that the layer was built at
However in future builds we do want to use the named reference rather
than the commit sha, e.g. master/fido
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 29 Sep 2015 04:45:27 +0000 (21:45 -0700)]
toaster: Create a relationship between build information and toaster layers
Previously this layer relationship was done by trying to match path
information that came back to the buildinfohelper with trying to query for
data in toaster's layers table. This rarely matched due to the lose coupling.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 29 Sep 2015 04:45:26 +0000 (21:45 -0700)]
toaster: Special case the openembedded-core layer to avoid duplicates
If the openembedded-core layer is specified in the toasterconf we need
to treat it differently because we may also get this layer either from
the layerindex source or I can also be provided locally.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 29 Sep 2015 04:45:25 +0000 (21:45 -0700)]
toaster: Add test cases for new Image customisation features
- Adds tests for new ToasterTables
- Adds tests for new ReST API
co-author: Elliot Smith <elliot.smith@intel.com>
Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 29 Sep 2015 04:45:24 +0000 (21:45 -0700)]
toaster: Add Image customisation frontend feature
Add the Image customisation front end feature to Toaster.
Caveat - This feature is currently in development and should not be
enabled by default.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 29 Sep 2015 04:45:23 +0000 (21:45 -0700)]
toaster: Add ToasterTables for Image customisation feature
- Split up the recipes tables into Image recipes and Software Recipes
- Add CustomImageRecipe table
- Add SelectPackagesTable table
- Add NewCustomImagesTable table
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 29 Sep 2015 04:45:22 +0000 (21:45 -0700)]
toaster: Add new ReST API for Image Customisation feature
Implemented xhr_customrecipe API. To create a custom recipe from a
base recipe.
Implemented xhr_customrecipe_packages API to add/remove packages
to/from custom recipe.
co-authored see Signed-off-by
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 29 Sep 2015 04:45:21 +0000 (21:45 -0700)]
toaster: Fix indentation of jsunittests view
Fix indentation to 4 spaces
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 29 Sep 2015 04:45:20 +0000 (21:45 -0700)]
toaster: implement decorator for REST responses
Implemented xhr_response decorator to decorate responses
from REST methods into Django HttpResponse objects.
This decorator should reduce amount of repeated code in
REST methods and make them more readable.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 29 Sep 2015 04:45:19 +0000 (21:45 -0700)]
toaster: add toggle for enabling image customisation feeature
This feature is currently under heavy development and should be used
with caution.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 29 Sep 2015 04:45:18 +0000 (21:45 -0700)]
toaster: Add CustomImageRecipe model
This model lists custom image recipes for the project.
It is populated when new custom image is created.
It holds reference to the base recipe and list of packages
included into custom image.
For CustomImageRecipes the packages will be copied in and
therefore not associated with a build so Remove the requirement
for the package to have a Build.
co-author: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 29 Sep 2015 04:45:17 +0000 (21:45 -0700)]
toaster: ToasterTable remove unused class definition
Remove ToasterTemplateView as this isn't used by anything.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 29 Sep 2015 04:45:16 +0000 (21:45 -0700)]
toaster: add nocache option to the ToasterTable widget
Useful for skipping the cache mechanism when debugging.
Simply append nocache=true to the end of the url with the table in
it.
Also adds a debug message if the table is using cached data.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 29 Sep 2015 04:45:15 +0000 (21:45 -0700)]
toaster: widgets ToasterTable Add more info to search field exception
If we get a search fields exception then also print out the model name
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 29 Sep 2015 04:45:14 +0000 (21:45 -0700)]
toaster: widgets ToasterTable add logger to notify when cache hit
Useful for debugging to remind you if you're hitting a cache or not
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 29 Sep 2015 04:45:13 +0000 (21:45 -0700)]
toaster: create custom layer and recipes for Image customisation
When building customised recipes toaster creates custom layer directory
and puts layer.conf and custom recipes to it.
[YOCTO #8075]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 29 Sep 2015 04:45:12 +0000 (21:45 -0700)]
toaster: tables Move the title and name into the widget
For historical reasons this was being set in the urls definition. We
can set this in the actual definition of the table and defaults in the
widget.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 29 Sep 2015 04:45:11 +0000 (21:45 -0700)]
toaster: make a workaround for old style index
For a new style indexes 'inherits' field is used to determine if
recipe is an image recipe.
As old style indexes don't have 'inherits' field this can be
guessed from recipe name. Let's consider recipe an image recipe
if recipe name contains '-image-'.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
prserv/serv.py: Better messaging when starting/stopping the server with port=0
When starting the server using port=0, the server actually starts with a
different port, so print a message with this new value. When stopping the
server with port=0, advise the user which ports the server is listening to,
so next time it tries to close it, user can pick up the correct one.
[YOCTO #8560]
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
prserv/serv: Close the DB connection out of class destructor
When launching the PR server daemon, the PRData __del__ function was being
called (no reason found yet) where the DB connection closed, thus following
PR updates were not getting into the DB. This patch closes the connection
explicitly, not relaying on the __del__ function execution.
Closing the connection in turn causes all WAL file transactions to be moved
into the database (checkpoint), thus effectively updating the database.
[YOCTO #8215]
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
prserv/serv: Start/Stop daemon using ip instead of host
In cases where hostname is given instead of an IP (i.e. localhost
instead of 127.0.0.1) when stopping the server with bitbake-prserv --stop,
the server shows a misleading message indicating that the daemon was not
found, where it is actually stopped. This patch converts host to IP values
before starting/stopping the daemon, so it will always work on IP, not on
hostnames, avoiding problems like the latter.
[YOCTO #8258]
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Markus Lehtonen [Mon, 21 Sep 2015 13:07:57 +0000 (16:07 +0300)]
tinfoil: remove logging handler at shutdown
Otherwise the logger gets multiple handers (and the user get duplicate
logging output) if another tinfoil instance is initialized after one is
shut down().
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the 'latest builds' section of the all builds page,
completed builds show the time they completed at. Builds
in progress should not display such time, since they
haven't completed yet.
This patch removes the time information that was showing
for builds in progress, whatever that time actually was.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Tue, 22 Sep 2015 02:42:35 +0000 (19:42 -0700)]
toaster: Add fake entry to Target_File for filesystem root
The files-in-image.txt file is produced by bitbake after an
image is created, listing all the files in the image.
However, this list doesn't include the root directory ('/').
buildinfohelper.py then tries to construct the filesystem
tree from this file, assuming that every directory apart from
the root directory (which is special-cased) can be assigned
a parent. But because the root directory isn't listed in
files-in-image.txt, an object for the root directory is never
created.
The direct subdirectories of the root ('./bin', './usr' etc.)
then can't be assigned a parent directory, as the object
representing the root directory doesn't exist. This
results in a Target_File lookup error and causes the
directory listing page to fail.
Fix this by creating a fake entry for the root directory
in the Target_File table, so that the direct subdirectories
of / can be assigned a parent. Note that it doesn't matter
that the root is faked, as its properties are never shown
in the directory structure tree.
[YOCTO #8280]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Tue, 22 Sep 2015 02:45:24 +0000 (19:45 -0700)]
toaster: layerdetails Fix back button tab behaviour
This completes the behaviour fix of the back button in the layerdetails
page as we not only have parameters in our history we also have the hash
to indicate which tab is active. As we pop our history we need to show
the corresponding tab.
[YOCTO #8252]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ke Zou [Wed, 23 Sep 2015 14:18:11 +0000 (15:18 +0100)]
toaster: UI test improvements
* Add decorator in logging mechanism
* Add more debug information
Signed-off-by: Ke Zou <ke.zou@windriver.com> Signed-off-by: Stanciu Mihail <stanciux.mihail@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
brian avery [Wed, 23 Sep 2015 14:05:40 +0000 (15:05 +0100)]
toaster: support selenium testing from mac OS X
* added a mac section to the cfg file
* added mac specific screenshot code
Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 22 Sep 2015 09:34:55 +0000 (10:34 +0100)]
toaster: add 2 UI tests
Tested that UI shows task names for the builds in
both all-builds and projectbuilds views.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 22 Sep 2015 09:34:54 +0000 (10:34 +0100)]
toaster: change UI to show tasks
Changed toaster UI to show tasks if they're specified for the
builds and use them when restarting builds.
[YOCTO #7442]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 22 Sep 2015 09:34:53 +0000 (10:34 +0100)]
toaster: don't re-create Target objects
Due to re-creating Target objects from bitbake events task information
stored in original objects is lost.
There is no valid reason to remove existing objects. It's safer to query
them instead of re-creating as original object contain more information
than events coming from bitbake.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 22 Sep 2015 09:34:52 +0000 (10:34 +0100)]
toaster: store task name in Target objects
Information about a task is not stored in Target objects.
This makes it impossible to correctly operate with the builds
where task is specified.
Storing taks name is an enabler for other fixes in UI and backend
related to restarting builds.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently this module is dereferencing errno.ENOENT but the python module "errno"
is not imported, which causes a crash when fetching from a git repository.
Signed-off-by: Romain Perier <romain.perier@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Alexandru DAMIAN [Fri, 24 Jul 2015 15:05:20 +0000 (16:05 +0100)]
toaster: hide irrelevant builds in the project builds view
This patch fixes the project builds view so it doesn't show
"in progress" builds or builds for other projects.
Note that this also modifies the "all builds" view to use
the same queryset filtering as the project builds. This is
to avoid excluding "in progress" builds more than once, which
is what was happening before.
The patch also has a minor change to ensure that when
displaying the project builds page, only builds for that
project are in the results.
The queryset filtering is now split over several lines so
you can see what's going on.
[YOCTO #8236]
[YOCTO #8187]
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 18 Sep 2015 07:55:47 +0000 (07:55 +0000)]
data_smart: Ensure OVERRIDES dependencies account for contains()
The dependencies of OVERRIDES were not including DEFAULTTUNE in OE-Core.
This is pulled in by a bb.utils.contains() reference which the override
dependency tracking code wasn't accounting for.
This patch ensures we do track contains references too.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 16 Sep 2015 20:59:44 +0000 (21:59 +0100)]
data_smart: When considering OVERRIDE dependencies, do so recursively
Sadly its not enough to consider the dependencies of OVERRIDES, we
need to resolve their dependencies and so on recursively. If we don't
do this, some variable can be changed and the resulting data store is
incorrect.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 16 Sep 2015 20:57:55 +0000 (21:57 +0100)]
data_smart: Expand overrides cache recursively
If the values that make up OVERRIDES are themselves overridden,
we end up into some horrible circular logic. Unfortunately some
metadata does depend on this functionality.
e.g:
DEFAULTTUNE_virtclass-multilib-xxx = Y
which changes TUNE_ARCH
which changes TARGET_ARCH
which changes OVERRIDES
As a solution, we iterate override expansion until the values don't
change. If we iterate more than 5 times we abort and tell the user to
report the issue.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bb.fetch2.{git, hg}: remove tarball if it needs updating
We were maintaining state in the form of ud.repochanged to determine whether
we need to write the tarball in the case where the tarball already exists, but
this state is maintained only in memory. If we need to update the git repo,
set ud.repochanged to True, and then are interrupted or killed, the tarball
will then be out of date.
Rather than maintaining this state, simply remove the out of date tarball when
we update the git repo, and it will be re-created with updated content in
build_mirror_data.
[YOCTO #6366]
Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Fri, 4 Sep 2015 09:37:47 +0000 (10:37 +0100)]
toaster: Simplify redirects when build page parameters are missing
A RedirectException is used to redirect the client to the
correct page when their original request is missing the
required parameters (page, orderby etc.). However, the code
is difficult to follow.
Rather than catching RedirectExceptions and rethrowing them with
different view URLs, ensure that the RedirectException has the
correct URL in it when thrown by passing the original page
name to the _build_list_helper() method.
Modified from an original patch by
David Reyna <david.reyna@windriver.com>.
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Fri, 4 Sep 2015 09:37:46 +0000 (10:37 +0100)]
toaster: Fix date range pickers on the project builds page
These were referring to the old HTML elements with ids
"date_from_created" and "date_from_updated", but their
ids have changed to "date_from_started_on" and
"date_from_completed_on". This meant that they weren't
functional.
This fixes the references.
Modified from an original patch by David Reyna
<david.reyna@windriver.com>.
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Fri, 4 Sep 2015 09:37:45 +0000 (10:37 +0100)]
toaster: Show correct builds count on project pages
The counter for completed builds on the project pages
includes builds in progress.
Instead use the completedbuilds queryset to count the
number of completed builds for display on project pages.
Modify how the completedbuilds queryset is constructed so
it only excludes builds with status "in progress".
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
toaster: hide irrelevant builds in the project builds view
This patch fixes the project builds view so it doesn't show
"in progress" builds or builds for other projects.
Note that this also modifies the "all builds" view to use
the same queryset filtering as the project builds. This is
to avoid excluding "in progress" builds more than once, which
is what was happening before.
The patch also has a minor change to ensure that when
displaying the project builds page, only builds for that
project are in the results.
The queryset filtering is now split over several lines so
you can see what's going on.
[YOCTO #8236]
[YOCTO #8187]
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mark Hatle [Thu, 10 Sep 2015 23:15:27 +0000 (18:15 -0500)]
tests/fetch.py: Fix recursion failure in url mapping
Instead of reproducessing the same line over and over and over, we remove the
current line from the mirror list. This permits us to re-evaluate the list
while excluding all matches that have previousily occured.
Without this fix, adding this test results in a failure:
RuntimeError: maximum recursion depth exceeded in cmp
Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Fri, 11 Sep 2015 20:57:30 +0000 (13:57 -0700)]
toaster: Don't def a function for each call to build_artifact()
Cache the mimetype object and only define the function for
getting a mimetype once.
Also ensure that filemagic is listed as a requirement of
toaster. Doing this also means we can remove the code
which tries multiple different "magic" libraries, as we know
we have the right version available.
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Fri, 11 Sep 2015 20:57:29 +0000 (13:57 -0700)]
toaster: Avoid unnecessary local to local copy of cooker log
The cooker log is copied from its original (bitbake) location
to an artifact directory chosen by the user (chosen when
checksettings.py runs as part of the managed mode;
it's one of the annoying questions you're asked at startup).
The copy happens as part of the runbuilds script run, which
is started in a loop from the toaster startup script
in managed mode.
When a user requests the log for a build via toaster, they
are getting the log which has been copied to the artifact
directory, not the original bitbake log.
This works for the managed case, where the runbuilds command is
running in a loop and copying log files for completed builds to
the artifact directory. However, in analysis mode, there are two
problems:
1. checksettings isn't run, so the artifacts directory isn't
set. toaster is then unable to figure out where the log files
should have been copied to.
2. The log files aren't copied to the artifacts directory
anyway, as runbuilds isn't running in analysis mode.
To fix this, just point the user to the local bitbake log file
in its original location. This avoids the copy step, and means
we can remove a whole question from the toaster startup sequence,
as we no longer need an artifact directory.
The only downside to this is that it's not appropriate for
remote bitbake servers. We will need to revisit this and
possibly reinstate the copy step once we have to reconcile
local and remote builds and make their logs available in
the same way.
[YOCTO #8209]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Fri, 11 Sep 2015 20:57:28 +0000 (13:57 -0700)]
toaster: Read correct cooker log path from toasterui
The BB_CONSOLELOG variable changes by the time we read it in
BuildInfoHelper. This means that the log file location we
are using is incorrect, so the links to the cooker logs don't
work.
Instead, read it at the point when the BuildStarted event occurs
in toasterui. The BB_CONSOLELOG variable has the correct value
here, so pass that to BuildInfoHelper.
[YOCTO #8209]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
brian avery [Mon, 14 Sep 2015 15:45:40 +0000 (16:45 +0100)]
toaster: delete multiple builds cleanup
This patch cleans up the multiple delete. It:
1) skips build id's that don't exist rather than giving a traceback.
2) let you pass in the ids as a space separated list
3) fixes the usage to match the space separated list format
[YOCTO #7726]
Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 16 Sep 2015 20:44:20 +0000 (21:44 +0100)]
cache: Handle spaces and colons in directory names for file-checksums
If there is a space in a directory name containing a file in file-checksums
(e.g. from a file:// url), you currently get tracebacks from bitbake. This
improves the code to handle colons and spaces in the file-checksums names
since it possible to figure out the correct names.
[YOCTO #8267]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 9 Sep 2015 21:44:37 +0000 (22:44 +0100)]
bitbake-worker: Ensure pipe closure doesn't crash before killpg()
If the pipe is closed, we want to ensure that we kill any child processes
by triggering the sigterm handler before we exit. This code does that,
hopefully avoiding the remaining process left behind issues on the autobuilder.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Wed, 9 Sep 2015 02:39:44 +0000 (19:39 -0700)]
toaster: increase waiting time
Increased waiting time for toasterUI from 10 to 25 seconds. Bitbake
takes longer time to start, so toaster should wait longer.
[YOCTO #8240]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Changes the breadcrumb to provide access to either
the project builds or the project configuration, as
appropriate
* Changes the left navigation in the project configuration
to reflect the hierarchical relationship between the
basic configuration and all other configuration pages
* Changes the left navigation in the build history to bring
it in line with the changes in the project configuration
This way the breadcrumb explicitly exposes the hierarchy
of the application, which is its correct behaviour, making it
easier to move around within Toaster.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 8 Sep 2015 22:37:34 +0000 (23:37 +0100)]
bitbake-worker: Handle SIGKILL of parent gracefully
If we SIGKILL cooker (the parent process), ensure the worker notices
and shuts down gracefully. To do this:
* trigger the sigterm handler if the parent exits
* ensure broken pipe writes don't trigger backtraces which interfer with
other exit work
* notice if our command pipe is broken due to EOF and sigterm if so
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 8 Sep 2015 22:32:07 +0000 (23:32 +0100)]
prserv: SIGTERM and deamonization fixes
Worryingly, if you SIGKILL the bitbake cooker, an autostarted PR server
will remain behind. It turns out there are a few things we should do:
* The PR service doesn't need to daemonize when started from cooker,
it just complicated the process lifecycle. Add a fork() method
to handle this and use the non-daemon mode for the singleton.
* Reset the sigterm and sigint handlers. Bitbake cooker installs its
own which we inherit meaning PR server was ignoring SIGTERM. Installing
our own handlers which include a sync makes most sense here. Since
we're in the code, make it sync the database on SIGINT.
* Use the new bb.utils.signal_on_parent_exit() call so that we get a
SIGTERM when the parent (usually cooker) exits and we can shutdown
too. Alternatives would be having an open pipe or polling
os.getppid() for changes but this seems more effective.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 8 Sep 2015 22:31:13 +0000 (23:31 +0100)]
utils: Add signal_on_parent_exit() function
Add a new bb.utils.signal_on_parent_exit() function so that a process
can register to recieve a signal when the parent dies. There is no
POSIX standard for this and the implementation is Linux specific.
Alternatives would be having an open pipe or polling os.getppid()
for changes but this seems more effective and less invasive to most
of bitbake's code structure.
We need to be able to determine when parents die to ensure child
processes stop running in a variety of circumstances to avoid
locks being held and ensure clean shutdown.
Roughly based on https://gist.github.com/evansd/2346614
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 7 Sep 2015 15:48:16 +0000 (15:48 +0000)]
server/process: Handle SIGTERM more gracefully
Currently if you send a SIGTERM to the bitbake UI process, the system basically
hangs if tasks are executing. This is because the server process doesn't
actually try any kind of shutdown before exiting.
This patch trys executing a stateForceShutdown command first, which is
enough to stop any active tasks before the system exits.
I also noticed that terminate can execute multiple times, once at SIGTERM
from the handler and once from the real exit. Double execution leads to
stack traces and potential hangs (writes to dead pipes), so ensure
the code only can run once.
With these fixes, bitbake much more correctly deals with SIGTERM to the
UI process.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ETA we show for builds in progress is woefully
inaccurate. In the 1.8 release we replaced it with
the % of tasks completed. Somehow, we regressed to
the ETA, so bringing the task % back.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bb.fetch: don't remove the clone when an update fails
When our clone exists, but is out of date, and the attempt to update it fails,
we don't necessarily want to remove the entire clone, particularly if it's
a large repository.
Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bb.fetch: handle checksums consistently for mirrors
If the main fetch method doesn't support checksums, the user will not be
defining them in the recipe, so we don't want to check them for
premirrors/mirrors either. This ensures that we never error due to missing
checksums on a git mirror tarball.
Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using an absolute file URI, there's no host, and the path starts with
'/', the dir under ${DL_DIR}/git2/ ends up starting with '.', so is hidden.
Remove any leading '.' to fix this.
Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Alex Franco [Wed, 2 Sep 2015 20:11:30 +0000 (15:11 -0500)]
cooker/runqueue: Allow bitbake commands starting with do_
The output of "bitbake, -c listtasks pkg" lists tasks with their real names
(starting with "do_"), but then "bitbake -c do_task" fails, as "do_" always
gets unconditionally prepended to task names. This patch handles this error
by checking whether a task starts with "do_" prior to prepending it with it
when the task runlist is being constructed (and a few other corner cases).
[YOCTO #7818]
Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 3 Sep 2015 14:20:05 +0000 (15:20 +0100)]
event/server: Add _uiready flag to handle missing error messages
If you start and suspend a bitbake execution so the bitbake lock is held,
then try and run "bitbake -w '' X", you will see bitbake return an error exit
code but print no message about what happened at all.
The reason is that the -w option creates a "UI" which swallows the messages. The
code which handles this exit failure mode thinks a UI has printed the messages
and therefore doesn't do so.
This adds in an extra parameter to the UI registration code so that we
can figure out whether its a primary UI or not and base decisions on whether
to display information on that instead. This fixes the error shown above and
some bizarre failures on the Yocto Project Autobuilder.
[YOCTO #8239]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake utils' movefile is now prone to malform the destination
file with duplicated file name strings. Fixing it to force a file
name append iff the dest argument is a dir not a file name
Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Thu, 3 Sep 2015 00:25:12 +0000 (17:25 -0700)]
toaster: Add tests for pages which show the default project
Test that the correct landing page redirect is applied, depending
on the state of the default project.
Test that the default project is only shown on the /projects page
if it has at least one build.
[YOCTO #7932]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Thu, 3 Sep 2015 00:25:11 +0000 (17:25 -0700)]
toaster: Fix test for projects page JSON
The tests expect data to be returned in a particular format,
which doesn't include is_default or the num_builds fields
added to support default project improvements.
Add the extra fields so the tests pass again.
[YOCTO #7932]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Thu, 3 Sep 2015 00:25:10 +0000 (17:25 -0700)]
toaster: Exclude default project unless it has builds
Don't include the default "command line builds" project in
the projects view unless it has builds associated with it.
[YOCTO #7932]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Thu, 3 Sep 2015 00:25:09 +0000 (17:25 -0700)]
toaster: Only redirect to projects page if user has added projects
The landing page currently redirects the user if there are any
projects in the db. Because we now always have at least one
(the default one added by a migration), we always get the redirect.
Change this so that when the user hits the landing page,
we only redirect them to the projects page if there is at least
one user-added project and there are no builds.
[YOCTO #7932]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Thu, 3 Sep 2015 00:25:08 +0000 (17:25 -0700)]
toaster: Improve how default project is identified and fetched
Command line builds are associated with a "default project"
(as we currently require a build to have a project). This
acts as a container for builds initiated outside Toaster.
Currently, this project is marked as the default by its ID
being 0. However, this doesn't work with MySQL, as MySQL
won't allow 0 in a foreign key which references an
autoincrement field.
Instead, use an is_default field to track the default Project
for builds initiated outside Toaster.
Add a method to fetch this default project, rather than fetching
a project with a magic ID.
Add this default project in a migration, rather than as a side
effect of a get_or_create() style method.
Also ensure that builds always have a project explicitly assigned
to avoid any magic with a build's project foreign key defaulting to
0 (as it no longer does).
[YOCTO #7932]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Thu, 3 Sep 2015 00:25:07 +0000 (17:25 -0700)]
toaster: Make 0021 migration compatible with MySQL
Two issues prevent this migration from working correctly
with a MySQL back-end:
1. MySQL won't allow a default value to be set for an
AutoField, which is what the migration tries to do
for project_id ("ValueError: The database backend does not accept
0 as a value for AutoField.")
2. When migrations are applied to a MySQL back-end, Django
(via South) attempts a dry run of the migration first: it
applies the forward migration then rolls it back. However, this
migration raises an exception on roll back, which causes the
whole series of migrations to fail.
This commit fixes both issues.
[YOCTO #7932]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want addtask to be able to bring back a deleted task, but we don't want its
previous dependencies to come back with it, so rather than marking a task as
deleted and then skipping tasks marked as such, actually delete the task and
its dependency information in deltask.
While we're in that part of the code, also fix a couple 'not foo in bar'
instances to 'foo not in bar', which is preferred in python.
Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Mon, 17 Aug 2015 16:47:47 +0000 (17:47 +0100)]
toastergui: Add frontend javascript unit tests
Use Jquery's Qunit tests to create some unit tests for javascript
components used in toaster.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
toaster: fix show / hide columns in all builds table
The classes errors_no and warnings_no were removed from the
td tags in the builds.html template. That broke the show / hide
columns functionality, which would hide only the table heading,
but not the data cells.
This patch brings back those classes so that you can hide
and show the errors and warnings information.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bian Naimeng [Fri, 28 Aug 2015 15:37:38 +0000 (16:37 +0100)]
toaster: Support environments which have proxies set
In an environment with a proxy which requires authentication, e.g. with
$http_proxy = 'http://user:password@ip:port', the following error
occurs when running Toaster:
EE: Using proxy http://user:password@ip:port
EE: could not connect to 'url', skipping update: 'error message'
This prevents Toaster from fetching layer, recipe and machine information
from remote repositories.
This patch allows Toaster to use the proxy settings from the
environment for HTTP/HTTPS requests.
Signed-off-by: Bian Naimeng <biannm@cn.fujitsu.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
David Reyna [Tue, 25 Aug 2015 08:32:15 +0000 (09:32 +0100)]
toaster: update directory tree expansion
The ajax query that fetches the directory data on an expand request
is now returning the response data already in an object form. so the
parseJSON() call is no longer needed (and is currently returning a
parse error).
[YOCTO #7810]
Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Wed, 26 Aug 2015 12:50:47 +0000 (13:50 +0100)]
lib/bb/cooker: add interrupted flag to BuildCompleted event
Allow any listeners for this event (such as buildhistory.bbclass in
OpenEmbedded) to find out if the build was interrupted rather than
completing normally. The value will be 0 if not interrupted, 1 if
interrupted waiting for remaining tasks to complete, or 2 if force
interrupted (stopping any running tasks immediately).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
projects / thirdparty / openembedded / openembedded-core-contrib.git / log
