Michael Wood [Wed, 23 Mar 2016 08:28:38 +0000 (08:28 +0000)]
toaster: localhostbecontroller Allow file:/// uri type for git repo
We don't need to skip file:/// uri type locations for git repositories.
If you're using a file:/// uri you should know that it has to be a local
path.
[YOCTO #9200]
igned-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Wed, 23 Mar 2016 08:28:37 +0000 (08:28 +0000)]
toaster: orm Add a constant for the CustomImageRecipe's layer name
Use a constant to define the name for the toaster custom images layer;
this constant is then used to identify this layer in various places.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Wed, 23 Mar 2016 08:28:36 +0000 (08:28 +0000)]
toaster: localhostbecontroller Don't clear out toaster custom layer dir
We may have a recipe which is based on a custom image recipe that has
already been built so keep the recipe file around so that it can be read
by the generate recipe function.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Brendan Le Foll [Fri, 11 Mar 2016 10:32:58 +0000 (10:32 +0000)]
fetch2/npm: fix ud.registry so that alternative registries can be handled
Fixes [YOCTO #9231]
npm when given an invalid registry URL with --registry actually goes and
fetches from the default registry, but this commit makes sure it goes to the
specified one.
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Wed, 9 Mar 2016 13:01:24 +0000 (13:01 +0000)]
toaster: models List only have the specified project's imported layers
When returning the compatible layers make sure that we are only
listing: All the layers which are for this release && configuration
layers (i.e. aren't part of the build history) and which aren't an
imported layer OR are this project's imported layer(s).
[YOCTO #8944]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Tue, 8 Mar 2016 11:32:12 +0000 (11:32 +0000)]
toaster: rework task buildstats storage and display
The data available from buildstats is now more fine grained than
previously, so take advantage of that to enrich the data we save
against tasks:
* Store the CPU usage for user and system separately, and display
them separately.
* Disk IO is now measured in bytes, not ms. Also store the
read/write bytes separately.
* Store started and ended times, as well as elapsed_time. This
will enable future features such as showing which tasks were
running at a particular point in the build.
There was also a problem with how we were looking up the Task
object, which meant that the buildstats were being added to
new tasks which weren't correctly associated with the build. Fix
how we look up the Task (only looking for tasks which match the
build, and the task and recipe names in the build stats data) so
the build stats are associated with the correct task.
[YOCTO #8842]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sujith H [Tue, 8 Mar 2016 18:09:02 +0000 (18:09 +0000)]
toaster: use force_bytes to display non-ascii project names
When user enters a non-ascii character in the project
name of toaster, the build doesn't get triggered.
Use force_bytes to fix this.
Also deal with non-ascii project names when logging the
build request in runbuilds.
[YOCTO #9071]
Signed-off-by: Sujith H <sujith.h@gmail.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
fetch2: Make SRC_URI[md5sum] and SRC_URI[sha256sum] expand their values
For some reason, the values for SRC_URI[md5sum] and SRC_URI[sha256sum]
were not being expanded. That lead to the following code not working
as expected:
Ed Bartosh [Wed, 9 Mar 2016 08:07:40 +0000 (10:07 +0200)]
xmlrpc: fix bug in setting XMLRPCServer.single_use
XMLRPCServer.single_use attribute was always set to False.
This caused xmlrpc server to keep running after build is done as
BitBakeServerCommands.removeClient only shuts down server if its
single_use attribute is set to True.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Wed, 9 Mar 2016 04:22:13 +0000 (17:22 +1300)]
fetch2/npm: properly handle npm dependencies
The output of "npm view dependencies" isn't entirely JSON if there are
multiple results, but the code here was just discarding the output if
the entire thing didn't parse as JSON. Split the output into lines and
iterate over it, parsing JSON fragments as we find them; this way we end
up with the last package's dependencies since it'll be last in the
output.
Digging further, it seems that the dependencies field reported by "npm
view" also includes optional dependencies. That wouldn't be a problem
except some of these optional dependencies may be OS-specific; for
example the "chokidar" module has "fsevents" in its optional
dependencies, but fsevents only works on MacOS X (and is only needed
there). If we erroneously pull in fsevents, not only is it unnecessary
but it causes "npm shrinkwrap" to throw a tantrum. In the absence of a
better approach, look at the os field and discard the module (along with
any of its dependencies) if it isn't for Linux.
As part of this, we can reduce the calls to npm view to one per package
since we get the entire json output rather than querying twice for two
separate fields. Overall the time taken has probably increased since we
are being more thorough about dependencies, but it's not quite as bad as
it could have been.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Wed, 9 Mar 2016 04:22:12 +0000 (17:22 +1300)]
fetch2/npm: fix errors with some version specifications
"2 || 3" is a valid version specification for a dependency in an npm
package.json file, but of course that looks like something else when
sent to a shell. Quote the version value to avoid this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Thu, 3 Mar 2016 06:47:44 +0000 (22:47 -0800)]
toasterui: fix warning 'Unknown event'
Continue after processing BuildStarted event to fix
WARNING: Unknown event: <bb.event.BuildStarted object at 0x2554150>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Thu, 3 Mar 2016 06:47:43 +0000 (22:47 -0800)]
toasterui: exit on final events
Toasterui exits only if bitbake observer shuts down.
In build mode it should exit when build is done.
Made toasterui exit on bb.command.CommandCompleted,
bb.command.CommandFailed and bb.command.CommandExit events
when it's running in build mode.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Thu, 3 Mar 2016 06:47:42 +0000 (22:47 -0800)]
toasterui: make toasterui to work in build mode
Currently toasterui works only in observer mode. This is
artificial limitation which was made to support current toaster
design. As we decided to stop using bitbake server we'll
need to run toasterui also in build mode.
[YOCTO #7880]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Thu, 3 Mar 2016 06:47:41 +0000 (22:47 -0800)]
toasterui: check if setEventMask succeeded
Currently toasterui ignores return value of setEventMask
command, which created confusing difference between set of
events set by this command and the real set used in the code.
Checked if setEventMask succeeded. Print error message and
exit if it's not.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Thu, 3 Mar 2016 06:47:40 +0000 (22:47 -0800)]
command: make setEventMask readonly
Executing setEventMask command when bitbake server is in readonly
mode causes runCommand to fail with the following error:
'Not able to execute not readonly commands in readonly mode'
Set readonly attribute for setEventMask command to make it working
for Toaster UI. This should not do any harm as this command doesn't
influence cooker state.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Thu, 3 Mar 2016 06:47:39 +0000 (22:47 -0800)]
toasterui: update list of events
Removed events not used in the code from the list.
Added events that are used in the code.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Thu, 3 Mar 2016 06:47:38 +0000 (22:47 -0800)]
toasterui: reformat list of events
Reformatted and reordered list of events to make changes
easily and see them clearly in the diffs.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Thu, 3 Mar 2016 05:26:56 +0000 (21:26 -0800)]
toaster: remove sshbecontroller module
The code of this module is broken for a long time.
The functionality of it can be easily achieved by running
'manage.py runbuilds.py' on remote machine.
[YOCTO #8806]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Thu, 3 Mar 2016 05:26:55 +0000 (21:26 -0800)]
toaster: don't use sshbecontroller
Removed usage of sshbecontroller from bbcontroller, models, tests
and database schema.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Thu, 3 Mar 2016 05:26:54 +0000 (21:26 -0800)]
toaster: raise NotImplementedError
Raised NotImplementedError instead of Exception to be
able to catch it.
This is a preparation for removing sshbecontroller module.
It has to be done as code in bldcontrol/tests.py imports custom
NotImplementedException from sshbecontroller.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I've received some feedback on the 'Revision' label we use in the import
layer page. It is not quite communicating that what's required is a Git
revision. Changing it to 'Git revision' to make it a bit more specific.
The change applies not only to the import layer page, but to all pages
showing revision information in the project configuration section.
Michael Wood [Mon, 7 Mar 2016 12:01:48 +0000 (12:01 +0000)]
toaster: views api Package info return both kinds of RDEPENDS
As we do with the popover snippet which shows dependencies inline in the
table also show dependencies which have both TYPE_TRDEPENDS and
TYPE_RDEPENDS.
Also remove obsolete comment
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Dave Lerner [Fri, 4 Mar 2016 16:41:27 +0000 (16:41 +0000)]
toaster: fixup dependency excludes for customimage
[YOCTO #9156]
For a customized image when adding a dependent package X that depends on
dependency package Y, in addition to adding X to appends_set and Y to
includes_set, make sure that Y is no longer in the excludes_set. Y may
have been added to the excludes_set by a prior package removal.
Signed-off-by: Dave Lerner <dave.lerner@windriver.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Mon, 7 Mar 2016 04:27:37 +0000 (17:27 +1300)]
fetch2/npm: ignore unknown headers in tarballs
Tarballs that are fetched down via npm repositories seem to often have
unknown headers. This doesn't affect our ability to extract the contents
though so we don't really care to see those warnings.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Mon, 7 Mar 2016 04:27:36 +0000 (17:27 +1300)]
fetch2/npm: handle alternative dependency syntax
npm allows you to specify a dependency as a Github username+path, or
omit the version entirely. You can hit these if you don't use a
shrinkwrap file, with the result that the code later fails due to the
output of "npm view" being empty; so handle this lazily by just ignoring
this part of the dependency if it's not really a version.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updated the variable list describing the variables that affect
how environment variables are handled. Also updated the section
on how those variables are passed.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
brian avery [Tue, 1 Mar 2016 11:40:55 +0000 (11:40 +0000)]
toaster: cleanup of bin/toaster startup code
This cleanup fixes a few issues:
1) Removes the superflous code to set toasterconf.json
- this isn't used and referenced meta-yocto
2) Changes exit to return so we don't surprise the user by exiting their shell
- this is necessary because it is being sourced
3) Removes the last references to the old TOASTER_MANAGED variable
- this is historical and no longer used.
4) Adds -t parameter to lsof
- This stops it from dying on odd filesystems and is much
faster since all we are using are processes anyway
5) Handles start and stop as params
- it was easy to confuse the script especially
if we were calling it with parameters.
- if start/stop isn't specified, it will still toggle
Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bavery [Mon, 29 Feb 2016 18:26:02 +0000 (10:26 -0800)]
hob: removal of hob ui and associated ui files
We've been gearing up the Toaster web UI to replace the Hob (GTK+ based) UI
for some time now; Hob has basically been on life support for the past few
releases. As of late last month in master, Toaster has the capability to
select the packages in an image, removing the last thing that Hob could do
that Toaster couldn't.
To recap, the reasons why Hob is being removed include:
- The code is tightly woven into BitBake, making it fragile. This means it
needs significant QA and maintenance on an ongoing basis.
- Some of the implementation is not ideal; we'll be able to remove some cruft
from BitBake and OE-Core at the same time.
- It's GTK+ 2 based, not the current GTK+ 3.
- Toaster is now a much more capable UI and is being actively maintained
The discussion about removing hob can be found at:
http://lists.openembedded.org/pipermail/openembedded-architecture/2016-February/000082.html
Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Brendan Le Foll [Mon, 29 Feb 2016 14:27:43 +0000 (14:27 +0000)]
npm: in cases where shrinkwrap resolved a git URL, ignore it and grab dist.tarball
npm-shrinkwrap will sometimes resolve a git URL which instead of a http url, in
this case go and grab the dist.tarball via npm instead of using the resolved
URL.
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
fetch2: fixes copying of file://dir; subdir=foo, bug 6128 and bug 6129
When in SRC_URI appears file://dir;subdir=foo unpacker copies 'dir' to ${WORKDIR}, not
${WORKDIR}/foo as it should be.
These changes are fixing following bugs as well:
Bug 6128 - Incorrect wildcard unpack behaviour in fetcher
Bug 6129 - Local directories unpack to a different location than local files
Signed-off-by: Alexander Shashkevich <alex@stunpix.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 26 Feb 2016 17:49:33 +0000 (17:49 +0000)]
fetch2/__init__: Fix decodeurl to better handle urls without paths
If we specify urls such as npm://somehost;someparams the fetcher currently
does a poor job of handling mirrors of these urls due to deficiencies in the
way decodeurl works. This is because "somehost" is returned as a path, not
a host.
This tweaks the code so that unless its a file url, the host is returned
correctly.
This patch also adds test cases for these urls to the exist set of test
urls.
We need to tweak the URI() class since this thinks this is a relative url
which is clearly isn't. We also need to handle the fact that encodeurl will
error if passed a url of this form (it would want the path to be '/'.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I have received quite a few complaints about the use of the word
'delete' for layer removal, so change it to 'remove'. That also matches
the language we use for packages in image customisation.
[YOCTO #9165]
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Dave Lerner [Thu, 25 Feb 2016 14:24:49 +0000 (14:24 +0000)]
toaster: fix banner after customimage package add
Correct formatting of the banner message after adding a package to a
custom image.
[YOCTO #9101]
Signed-off-by: Dave Lerner <dave.lerner@windriver.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
toaster: custom breadcrumb for the default project
The default "Command line builds" project does not have a Configuration
page. It therefore needs a custom breadcrumb where the project name goes
to the project builds page, instead of the project configuration page.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Haris Okanovic [Tue, 23 Feb 2016 17:36:33 +0000 (11:36 -0600)]
prserv: Add dump_db()
Returns a script (string) that reconstructs the state of the
entire database at the time this function is called. The script
language is defined by the backing database engine, which is a
function of server configuration.
Returns None if the database engine does not support dumping to
script or if some other error is encountered in processing.
The SQLite3 implementation in db.py calls iterdump() [1] to generate
a script. iterdump() is the library equivalent of the `sqlite3 .dump`
shell command, and the scripts are compatible. Execute the script in
an empty SQLite3 database using the sqlite3 utility to restore a backup
of prserv.
Use case: Backup a live PR server database in a non-racy way, such
that one could snapshot the entire database after a set of bitbake
builds all using a shared server. I.e. All changes made prior to
the start of a dump_db() operation should be committed and captured
in the script. Subsequent changes made during the backup process are
not guaranteed to be captured.
Testing: ~7MB database backs up in ~1s while PR server is under load
from 32 thread bitbake builds on two separate machines.
Signed-off-by: Haris Okanovic <haris.okanovic@ni.com> Reviewed-by: Ken Sharp <ken.sharp@ni.com> Reviewed-by: Bill Pittman <bill.pittman@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Dave Lerner [Tue, 23 Feb 2016 12:34:45 +0000 (12:34 +0000)]
toaster: remove custom images from Image Recipes
Fix the view of 'Image Recipes' under 'Configuration' to only show
image recipes that are not customised since custom images have their
own page.
[YOCTO #9111]
Signed-off-by: Dave Lerner <dave.lerner@windriver.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Tue, 23 Feb 2016 12:17:04 +0000 (12:17 +0000)]
toaster: show suffix for image files and basename for artifact files
The build dashboard doesn't show image and artifact files correctly,
as it shows the full filename for images and the filename plus
path relative to DEPLOY_DIR for artifacts.
Instead, show just the suffix for image files, and the basename
for artifact files.
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the 'New custom image' page, each image recipe name listed should
link to the corresponding image recipe details page, so that users can
look into what packages are installed by a certain image, and decide
based on that if they want to customise it or not.
This patch adds that missing link.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The search field at the top of our tables was using one of the Bootstrap
classes for text field sizing. Those classes are a bit rigid, resulting
in text fields too wide that made other table controls wrap.
Setting a maximum width to the search form using one of the span classes,
combined with a % width css declaration, make for text fields that
adapt a bit better to the horizontal space available in each table.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Our project pages have 4 tabs: builds, configuration, import layer and
new custom image. Even though we treat the 'configuration' as the
default tab, it comes second after the builds tab.
That's a bit strange: the default tab should be the first one listed.
This patch changes the tab order to put 'configuration' first.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The existing breadcrumb does not always provide a link to the project
configuration page. When you are in the build history pages, you must go
back to the builds information first, and from there access the project
configuration. That feels very long.
Change the breadcrumb so that the project name item always provides a
link to the project configuration.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that we use modal dialogs to display dependency information for
packages, we are hitting their maximum height relatively often. It is set
by default to 400px, which makes it a bit tight at a 1280x800 viewport
size.
Reduce the maximum height to 300px to make things a bit more
comfortable.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The 'add layer' button in the project configuration page remains enabled
after you add a layer. If you click it again, the same layer you just
added is added again.
This patch disables the 'add layer' button on click, to avoid this bit
of weirdness.
[YOCTO #8905]
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The form for naming new custom images shows you an error message when
the name already exists or you include an invalid character in it. But
when an error appears, the input field was missing the red highlight.
This patch applies the right class to the form controls whenever an
error message is shown.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The placeholder text in the form where you name your new custom images
didn't display fully. This patch fixes the styles so that the text shows
properly. It also changes the text itself to make it a bit shorter.
[YOCTO #9122]
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Although the support for building more than one release and how we
handle the build directories is the subject of lively discussion, we all
seem to agree on removing the ability to change the release of a
project. The feature is currently not working but exposed to users,
which is not a happy state of affairs.
This patch comments out the controls that give access to the release
changing functionality to hide them from users, but does not touch
anything else. Once all moving pieces start to settle down, we can make
a final decision regarding this feature, and clean up the code
accordingly.
[YOCTO #8917]
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Brendan Le Foll [Thu, 25 Feb 2016 15:07:06 +0000 (15:07 +0000)]
utils.py: Add sha1_file call
This is useful as npm-lockdown uses sha1 because npm releases the sha1 of
packages and whilst this is undocumented it seems no other algorithm is
supported
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Fri, 19 Feb 2016 05:21:52 +0000 (21:21 -0800)]
toaster: toastergui Fix invalid char test and implementation
This test was passing but because the assertion was the wrong way round
and should have expected the first one to pass and second one to fail,
in reality both were failing as the method for checking the invalid char
was incorrect.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Fri, 19 Feb 2016 05:21:51 +0000 (21:21 -0800)]
toaster: PackagesTable show only installed packages
When showing the package list for the image recipe details only show the
packages which are installed in the image rather than all the packages
which are produced.
[YOCTO #9108]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Fri, 19 Feb 2016 05:21:50 +0000 (21:21 -0800)]
toaster: toastergui unit tests convert to use fixtures
Instead of having to manually write up in the setUp function all the
different objects to create in the database, just specifiy them in a
fixture and load it. This has the advantage that it can be used on a
live server by using the ./manage.py loaddata command and can then
be inspected.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure the default orderby for the SoftwareRecipesTable is applied
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Fri, 19 Feb 2016 05:21:48 +0000 (21:21 -0800)]
toaster: orm migrations Sort out migrations mess
We messed up the migrations by squashing some of the image
customisation model definitions into the initial migration which
has meant some irreversible operations on mysql took place.
This deletes, re-orders and fixes the migrations.
If your schema is up to date you may want to use ./manage migrate
with --fake or --fake-initial to avoid re-applying migrations.
[YOCTO #9116]
Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 2 Feb 2016 23:55:37 +0000 (23:55 +0000)]
data_smart: Drop default expand=False to getVarFlag [API change]
At some point in the future, getVarFlag should expand by default. To
get there from the current position, we need a period of time where the
expand parameter is mandatory.
This patch starts that process. Clear errors will result from any code
which doesn't provide this. Layers can be fixed with an expression
like:
sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g' -i `grep -ril getVar *`
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 2 Feb 2016 23:55:40 +0000 (23:55 +0000)]
data_smart: Drop default expand=False to getVar [API change]
At some point in the future, getVar should expand by default. To get
there from the current position, we need a period of time where the
expand parameter is mandatory.
This patch starts that process. Clear errors will result from any code
which doesn't provide this. Layers can be fixed with an expression
like:
sed -e 's:\(\.getVar([^,()]*\)):\1, False):g' -i `grep -ril getVar *`
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Markus Lehtonen [Tue, 26 Jan 2016 13:34:29 +0000 (15:34 +0200)]
bb/runqueue: save task file dependency cache onto disk
Before this patch the usage of cache was quite useless as the file
checksums were not actually cached on disk but re-calculated every time.
This patch utilises the new writeout_file_checksum_cache() method of the
SignatureGenerator class to do the job.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Markus Lehtonen [Tue, 26 Jan 2016 13:34:28 +0000 (15:34 +0200)]
SignatureGenerator: add method for saving the file checksum cache
Extend the API in order to be able to write out the file checksum cache
onto disk. SignatureGeneratorBasic class now implements a method that
update the fetcher local files checksum cache with the task file
dependency checksums.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mirela Rabulea [Tue, 19 Jan 2016 14:31:38 +0000 (16:31 +0200)]
Allow Hob to run images on a custom simulator, other than qemu
The current behavior of Hob is that there is a "Run Image" button which becomes visible only for qemu images.
My suggested change is:
- if an image is selected and it is qemu-compatible, let the "Run image" button be named "Run qemu image"
- if an image is selected and it is not qemu-compatible, let the same button show up with the name "Run custom image", and besides that, an option shows-up to allow the selection of the custom script (by default it points out to runqemu script) to be used for launching this custom image
Note: in case there is more than one toggled image (qemu runnable or deployable), when the user clicks the "Run custom image" button, a dialog will be presented, allowing to choose between any of the existing images.
[YOCTO #8940]
Signed-off-by: Mirela Rabulea <mirela.rabulea@nxp.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sun, 14 Feb 2016 23:09:49 +0000 (23:09 +0000)]
knotty: Set exit failure code on runQueueTaskFailed events
If the worker segfaults, we may never see a TaskFailed event from it, only
a runQueueTaskFailed event. In this case, return_value isn't getting set
leading to an incorrect exit code from bitbake. Fix by setting return_value
in both places.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sun, 14 Feb 2016 10:14:22 +0000 (10:14 +0000)]
taskdata: Fix traceback issue with missing provider
If there is a missing provider and we're using "-k" mode alongside "-w",
we could get a traceback since there was no provider. Add tests to avoid this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sun, 14 Feb 2016 09:11:24 +0000 (09:11 +0000)]
cooker: Improve cache handling
We're seeing issues where the self test, which uses tinfoil doesn't
notice the changed contents of include files. The issue is
cached_statements in the parser being reused when the files have changed.
Whilst looking at this, I realised there were some other issues:
* We need to also invalidate the mtime cache when cooker restarts
* We should pass full filenames to the file invalidation code
* We should process cached_statements as part of inotify invalidation
With these fixes, the caching is more reliable for memory resident
bitbake too. It does raise some questions about cache validation and
lifecycles and indicates bitbake does need more work in the area,
preferably with the removal of the globals. This at least highlights
and works around some of the current issues.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Thu, 11 Feb 2016 10:37:08 +0000 (12:37 +0200)]
cooker: gracefully shutdown parsers
CookerParser.shutdown code doesn't do all required work to shutdown
parser processes. As a result bitbake hangs if interrupted during
parsing. Putting None into the parser_quit queue should fix this issue
as it makes parsers to quit main loop.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Wed, 10 Feb 2016 18:34:10 +0000 (18:34 +0000)]
buildinfohelper: unset brbe variable when build finishes
The brbe variable is set on the bitbake server when the Toaster
UI starts. This enables Toaster to associate events with the
correct build and build environment.
However, the brbe variable is also used when a build starts to
identify whether a new build needs to be created, or an existing
one looked up. This causes a bug for command-line builds which
happen after a Toaster-triggered build: because the brbe variable
is never unset on the server or the buildinfohelper, the new
command-line build events are treated as originating from the
previous build.
Ensure the brbe variable is reset when the buildinfohelper "closes"
a build, so that each build then either sets the brbe variable
(Toaster-triggered builds) or leaves it blank (command-line builds).
Also modify the localhostbecontroller so that the brbe variable
is not set on the server and not looked up from the server. This
ensures that it is only set when the triggerBuild() method is
called, and that it remains as None for command-line builds.
[YOCTO #9021]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 9 Feb 2016 15:08:42 +0000 (15:08 +0000)]
build: Improve python execution tracebacks
If something fails in a exec_func_python() the current stack trace shows
incorrect filenames and linenumbers. For example:
The stack trace of python calls that resulted in this exception/failure was:
File: '/media/build1/poky/meta/recipes-sato/images/core-image-sato.bb', lineno: 200, function: <module>
0196: chksum = bb.utils.sha256_file(fn)
0197: f.write('%s\t%s\n' % (chksum, os.path.relpath(fn, baseoutpath)))
0198:
0199:
*** 0200:copy_buildsystem(d)
0201:
File: '/media/build1/poky/meta/recipes-sato/images/core-image-sato.bb', lineno: 9, function: copy_buildsystem
0005:IMAGE_FEATURES += "splash package-management x11-base x11-sato ssh-server-dropbear hwcodecs"
0006:
0007:LICENSE = "MIT"
0008:
*** 0009:inherit core-image
0010:
0011:IMAGE_INSTALL += "packagegroup-core-x11-sato-games"
File: '/usr/lib/python2.7/subprocess.py', lineno: 535, function: check_call
0531: The arguments are the same as for the Popen constructor. Example:
0532:
0533: check_call(["ls", "-l"])
0534: """
*** 0535: retcode = call(*popenargs, **kwargs)
0536: if retcode:
0537: cmd = kwargs.get("args")
0538: if cmd is None:
0539: cmd = popenargs[0]
The problem is the use of "FILE" to obtain the current filename. Instead,
we therefore inject the function being executed into the methodpool which
allows us to correct its linenumber and filename information. We can then
clearly mark the initial piece as autogenerated and the rest of the linenumber
and filename information should be correct. Afterwards the trace starts:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:copy_buildsystem(d)
0003:
File: '/media/build1/poky/meta/classes/populate_sdk_ext.bbclass', lineno: 66, function: copy_buildsystem
0062: import glob
0063: import oe.copy_buildsystem
0064: import subprocess
0065:
*** 0066: subprocess.check_call("foo")
0067:
0068: oe_init_env_script = d.getVar('OE_INIT_ENV_SCRIPT', True)
0069:
0070: conf_bbpath = ''
File: '/usr/lib/python2.7/subprocess.py', lineno: 535, function: check_call
0531: The arguments are the same as for the Popen constructor. Example:
0532:
0533: check_call(["ls", "-l"])
0534: """
*** 0535: retcode = call(*popenargs, **kwargs)
0536: if retcode:
0537: cmd = kwargs.get("args")
0538: if cmd is None:
0539: cmd = popenargs[0]
We can't inject into methodpool at parsing time, since there may be
_append or other override operations against the function before its
execution.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
then any sane person would expect that Y = "b" at the end of the
function. This is not the case, Y = "a".
This is due to the python function being expanded before execution, the
executed code would read d.setVar("Y", "a"). This understandably
confuses people, it also makes it near impossible to write ${} in a
python function without unintended things happening.
I think there is general agreement we should fix this and standardise
on non-expansion of python functions. We already don't expand anonymous
python (mostly).
I've checked OE-Core with buildhistory before and after this change and
there were a small number of issues this exposed which I've sent
patches for.
I propose we default to not expanding python code and then deal with
any consequences from that if/as/where identified. This will improve
new user understanding and usability of the system, it also allows
several long standing weird expansion issues to be fixed.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 2 Feb 2016 13:59:52 +0000 (13:59 +0000)]
cooker: Don't expand python functions in variable dumps
We don't want to expand python functions since they aren't expanded
at execution time (e.g. anonymous python). They can also have side
effects.
This function is primarily used by toaster for variable dumps for later
display. The lack of expansion of python functions won't matter in this case
and actively helps some variable handling (e.g. SRCPV).
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 2 Feb 2016 13:58:17 +0000 (13:58 +0000)]
data: Don't expand python functions for variable dependencies
Expanding python functions for variable dependencies doesn't really make sense,
not least since this causes execution of any inline python, it also makes it
impossible to write expressions like d.expand("${X}") of d.setVar("X", "${Y}")
which may have the wrong values if expanded now.
This starts to standardise the approach across bitbake for handling python code.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Manually managing the database transactions caused more
problems than it temporarily solved so we return control back to
Django's orm.
[YOCTO #8796]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
toaster: templates make build data breadcrumb consistent
The pages in the build data section of Toaster showed different
breadcrumbs: in some pages the machine was displayed, but not in others.
For builds with more than one target, some pages showed the first
alphabetical target (the correct behaviour), others didn't.
This patch removes the inconsistencies, showing exactly the same
breacrumb across all pages in the section.
The patch also removes the extra space between the '+' and the number of
targets when the builds have more than one target.
Remove an unneeded debug message
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 9 Feb 2016 15:10:56 +0000 (15:10 +0000)]
BBHandler/ast: Merge handMethod and handleMethodFlags
The functionality overlap between these two functions is significant and
its clearer to handle both things together since they are intimately
linked. There should be no behaviour change, just clearer code.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 9 Feb 2016 15:09:45 +0000 (15:09 +0000)]
utils: Drop datastore function inspection during exception
When we use functions from the data store, they now have correct line number
and filename information. This function would attempt to correct line numbers
which doesn't need correcting, leading to misleading messages to the user.
Therefore remove this code as being obsoleted.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extended the dot styling of dependencies created by bitbake -g in dot syntax to differentiate between the various kinds.
depends: solid
rdepends: dashed
rrecommends: dotted
The change observed is that depends get an explicit style which is the same as dot default behavior and the runtime recommends get
dotted while before they were dashed. This helps to distinguish them graphically as well as eases post processing by script.
Signed-off-by: Henning Schroeder <henning.schroeder@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
projects / thirdparty / openembedded / openembedded-core-contrib.git / log
