Sujith H [Tue, 19 Jul 2016 07:54:49 +0000 (07:54 +0000)]
toaster: control the selection of git and local repo
Changes made in this patch will help user to select
either repo based on git and local. This patch also
improves the help provided to user so that user can
understand them and take necessary action to proceed.
Sujith H [Tue, 19 Jul 2016 07:51:50 +0000 (07:51 +0000)]
toaster: add import of local directory to UI
This patch will help to add local directory
to UI. The modification is made in importlayer.html
file. The radio buttons are created to separate
selection of git based repo and local directory.
Ed Bartosh [Wed, 27 Jul 2016 12:15:02 +0000 (15:15 +0300)]
toaster: fix unhandled exception
manage.py lsupdates throws DataError exception if the recipe can't be
saved to the MySQL database:
django.db.utils.DataError: (1406, "Data too long for column 'license' at row 1"
Adding DataError exception to the list of exceptions should make
lsupdates to print a warning message and skip the recipe.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Elliot Smith [Thu, 4 Aug 2016 15:32:33 +0000 (16:32 +0100)]
toasterui: ensure that the Build object is always available
Many of the methods in toasterui and buildinfohelper rely
on the internal state of the buildinfohelper; in particular, they
need a Build object to have been created on the buildinfohelper.
If the creation of this Build object is tied to an event which
may or may not occur, there's no guarantee that it will exist.
This then causes assertion errors in those methods.
To prevent this from happening, add an _ensure_build() method
to buildinfohelper. This ensures that a minimal Build object
is always available whenever it is needed, either by retrieving
it from the BuildRequest or creating it; it also ensures that
the Build object is up to date with whatever data is available
on the bitbake server (DISTRO, MACHINE etc.).
This method is then called by any other method which relies on
a Build object being in the internal state, ensuring that the
object is either available, or creating it.
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Elliot Smith [Fri, 29 Jul 2016 11:25:46 +0000 (12:25 +0100)]
toaster: prevent infinite loop when finding task dependencies
Toaster occasionally records a task which depends on itself.
This causes a problem when trying to display that task if it
is "covered" by itself, as the code does the following: for
task A, find a task B which covers A; then, recursively
find the task which covers B etc. If B == A, this loop becomes
infinite and never terminates.
To prevent this, add the condition that, when finding a task B
which covers A, don't allow B == A.
[YOCTO #9952]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Elliot Smith [Mon, 18 Jul 2016 15:02:14 +0000 (16:02 +0100)]
toaster: reset table to default orderby when orderby column is hidden
When a ToasterTable is sorted by a column, and that column is
hidden from view, the sort doesn't revert to the default for the
table.
Modify the JS responsible for reloading the table data so that
it doesn't rely on clicking a table column heading (as this is
inflexible and error-prone). Instead, use a function to apply
the sort to the table; and call that function when column
headings are clicked.
This means that the ordering can be changed programmatically
to a specified default ordering when a column is hidden, without
having to click on a column heading.
Use this function when the current sort column is hidden, to
apply the default sort for the table.
[YOCTO #9836]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Elliot Smith [Fri, 15 Jul 2016 14:20:36 +0000 (15:20 +0100)]
toaster: show loading spinner after creating custom image
Creating a custom image through the "New custom image" dialog
can sometimes result in a long pause between pressing the button
to create the image, and being transferred to the page showing
details of its content. This can make it appear as though pressing
the button had no effect.
To prevent this from happening, disable the button and text box
in the new custom image dialog after the "Create image" button is
pressed. Also show a loading spinner and "loading..." text on
the button to make it clear that the application is still responding.
[YOCTO #9475]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Elliot Smith [Fri, 15 Jul 2016 11:09:57 +0000 (12:09 +0100)]
toaster-tests: fix erroneous message when test fails
When one of the layer details tests fails (as it occasionally
does, if running on a machine under heavy load, due to sync issues),
the error message shown is misleading, as it is something like:
"Expected 'This was imported' in ['This was imported', ...]"
The string 'This was imported' is in the list shown in the message,
but the message suggests it isn't.
This is because the test compares the string with one list, but
then uses a different list in the fail message if the comparison
fails.
Fix the list shown in the message about the test failing.
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Elliot Smith [Wed, 13 Jul 2016 14:12:53 +0000 (15:12 +0100)]
toaster-tests: add tests for build dashboard menu and summary visibility
Test that the build dashboard only shows a menu and a build
summary area if a build has properly "started" (i.e. has at least
one Variable object associated with it).
[YOCTO #8443]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Elliot Smith [Wed, 13 Jul 2016 13:39:47 +0000 (14:39 +0100)]
toaster: add started property to Build
Add a property to the Build model which records whether
the BuildStarted event has occurred for the build.
The proxy for this event is the presence of variables recorded
against the Build: as the buildinfohelper only saves variables
when the BuildStarted event occurs (as the variables aren't
available on the bitbake server before that point), we can
tell whether BuildStarted has happened by counting Variable
objects on the Build.
This can then be used to determine whether a Build "properly"
started, enabling a different dashboard display (left-hand menu
hidden) if the build didn't record any useful information (e.g.
if it had a bad target).
[YOCTO #8443]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Elliot Smith [Wed, 13 Jul 2016 10:37:01 +0000 (11:37 +0100)]
toaster: remove links from time field on failed builds
Failed builds don't have any time data recorded for them,
so the time field in the builds table, the time shown
in the recent builds area, and the build time shown in
the build dashboard should not be links for failed builds.
[YOCTO #8443]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Elliot Smith [Mon, 11 Jul 2016 13:47:06 +0000 (14:47 +0100)]
toaster: show progress of recipe parsing in recent builds area
Modify buildinfohelper and toasterui so that they record the
recipe parse progress (from ParseProgress events in bitbake)
on the Build object.
Note that because the Build object is now created at the
point when ParseStarted occurs, it is necessary to set the
build name to the empty string initially (hence the migration).
The build name can be set when the build properly starts,
i.e. at the BuildStarted event.
Then use this additional data to determine whether a Build
is in a "Parsing" state, and report this in the JSON API.
This enables the most recent builds area to show the recipe
parse progress.
Add additional logic to update the progress bar if the progress
for a build object changes.
[YOCTO #9631]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Elliot Smith [Wed, 29 Jun 2016 14:41:56 +0000 (15:41 +0100)]
toaster: move most recent builds templating to client
The most recent builds area of the all builds and project builds
table needs to update as a build progresses. It also needs
additional functionality to show other states (e.g. recipe parsing,
queued) which again needs to update on the client side.
Rather than add to the existing mix of server-side templating
with client-side DOM updating, translate all of the server-side
templates to client-side ones (jsrender), and add logic which
updates the most recent builds area as the state of a build changes.
Add a JSON API for mostrecentbuilds, which returns the state of
all "recent" builds. Fetch this via Ajax from the build dashboard
(rather than fetching the ad hoc API as in the previous version).
Then, as new states for builds are fetched via Ajax, determine
whether the build state has changed completely, or whether the progress
has just updated. If the state completely changed, re-render the
template on the client side for that build. If only the progress
changed, just update the progress bar. (NB this fixes the
task progress bar so it works for the project builds and all builds
pages.)
In cases where the builds table needs to update as the result of
a build finishing, reload the whole page.
This work highlighted a variety of other issues, such as
build requests not being able to change state as necessary. This
was one part of the cause of the "cancelling build..." state
being fragile and disappearing entirely when the page refreshed.
The cancelling state now persists between page reloads, as the
logic for determining whether a build is cancelling is now on
the Build object itself.
Note that jsrender is redistributed as part of Toaster, so
a note was added to LICENSE to that effect.
[YOCTO #9631]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Elliot Smith [Mon, 11 Jul 2016 15:23:05 +0000 (16:23 +0100)]
cooker: add BuildInit event
In situations where a bitbake run fails before the build
properly starts and BuildStarted is fired, a UI has no way
to get at the targets passed to the build. This makes it
difficult for the UI to report on the targets which failed.
Fire a BuildInit event before running buildTargets() or
buildFile(). This enables a UI to capture targets passed to
buildTargets(), even if the build fails (e.g. the targets
themselves are invalid).
[YOCTO #8440]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Michael Wood [Mon, 1 Aug 2016 18:49:43 +0000 (19:49 +0100)]
toaster: Remove old toaster config loading mechanism
This has been replaced using django's inbuilt loaddata.
Django command documented at:
https://docs.djangoproject.com/en/1.8/ref/django-admin/#django-admin-loaddata
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Michael Wood [Mon, 1 Aug 2016 18:32:53 +0000 (19:32 +0100)]
toaster: checksettings call django's loaddata instead of custom command
Call django's inbuilt loaddata command to load the appropriate fixtures.
We also attempt to load a fixture called "custom" and fail silently if
we don't have one. This is where initial customisations can be done to
load particular settings or data into Toaster (for example layers or
default values for variables)
Make sure the value for TEMPLATECONF is available to checksettings so
that we can have a go a working out which default data to load.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Michael Wood [Mon, 25 Jul 2016 12:43:08 +0000 (13:43 +0100)]
toaster: Add poky and openembedded configuration fixtures
These fixtures provide a recommended default configuration of toaster for
either using bitbake and oe-core or as part of poky.
They can be used as the sample configuration for writing custom configurations.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Michael Wood [Fri, 5 Aug 2016 08:26:49 +0000 (09:26 +0100)]
toaster: tests test_new_project_page Catch button not enabled exception
When using firefox and selenium we get an exception generated when a
disabled button click is attempted. This should happen in the test but
we need to catch the exception to make sure it doesn't cause the test to
fail.
[YOCTO #10056]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Randy Witt [Tue, 2 Aug 2016 23:43:54 +0000 (16:43 -0700)]
toaster: --help now returns 0 instead of 1
If the user explicitly passes in "--help" then it should return 0. This
is the convention follow by the typical application. This allows the
user to check for options without triggering an error.
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Terry Boese [Wed, 10 Aug 2016 15:14:15 +0000 (09:14 -0600)]
fetch2/gitannex.py: use 'git annex init' instead of 'git annex sync'
The git annex fetcher needs git annex to be initialized. Previously
it was using 'git annex sync' to do this, but that has the downside
of moving the checkout to the tip of the default branch. This means
that tags, SRCREV, etc don't work in the gitannex case.
Signed-off-by: Terry Boese <terry.boese@vecima.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mariano Lopez [Fri, 5 Aug 2016 13:54:44 +0000 (13:54 +0000)]
fetch: Fix use of tar's --exclude option for tar >= 1.29
Starting from tar 1.29 the --exclude option won't work
anymore if is not used before the path. There are some
fetch modules that copy the ptest using tar and --exclude
option. This fixes these for bitbake.
[YOCTO #9763]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the "Executing a List of Task and Recipe Combinations"
section, I changed the improper bitbake command example to
use the correct syntax. This change was review feedback.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
I had inconsistent usage of the "_append" style operator syntax
in the chaper. I was using a mix of <filename>_append</filename>
and "_append". I changed to "_append" for consistency.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Elliot Smith [Wed, 20 Jul 2016 16:37:03 +0000 (17:37 +0100)]
toaster-tests: define capabilities for latest Firefox driver
For the latest Firefox versions, WebDriver requires a download of a
separate binary and an additional capability to be defined on it.
Modify our tests so that when "marionette" is set as the browser,
this capability is defined on the Firefox driver. Also add a note to the
README about the additional installation steps required.
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 26 Jul 2016 03:36:40 +0000 (15:36 +1200)]
lib/bb/checksum: avoid exception on broken symlinks
If using OE's externalsrc with a source tree that is not tracked by git
and contains broken symlinks, you can receive "TypeError: unorderable
types: NoneType() < str()" within the file checksum code due to:
checksums.sort(key=operator.itemgetter(1))
Don't add files with no checksum to the checksums list in order to avoid
this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Mon, 18 Jul 2016 16:22:55 +0000 (17:22 +0100)]
lib/bb/utils: show subprocess output in stack traces
If better_exec() throws a subprocess.CalledProcessError then show the output to
the user as it likely contains useful information for solving the problem.
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These warnings are caused by the difference in pickle module
implementation in python3(master) and python2(krogoth). Python2 supports
3 different protocols (0, 1, 2) and pickle.HIGHEST_PROTOCOL is 2 where as
Python3 supports 5 different protocols (0, 1, 2, 3, 4) and
pickle.HIGHEST_PROTOCOL is obviously 4.
My suggestion is to use 2 since it is backward compatible with python2
(all the supported distros for krogoth provides python2 which supports
pickle protocol version 2)
Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Mon, 25 Jul 2016 22:32:26 +0000 (23:32 +0100)]
lib/bb/build: handle incomplete message fragments in log FIFO
It's possible that the logging FIFO doesn't do a complete read (or the sender a
complete write) with the result that an incomplete message is read in bitbake.
This used to result in silently truncated lines but since 42d727 now also
results in a warning as the start of the rest of the message isn't a valid
logging command.
Solve this by storing incoming bytes in a bytearray() across reads, and parsing
complete messages from that.
[ YOCTO #9999 ]
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Thu, 21 Jul 2016 15:58:40 +0000 (18:58 +0300)]
toaster-tests: add tasks and recipes sub-page tests
Add tests for the tasks and recipes sub-pages of the build
dashboard.
[YOCTO #9833]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Thu, 21 Jul 2016 15:58:39 +0000 (18:58 +0300)]
toaster: set non-hideable columns for built recipes table
None of the columns in the built recipes table are marked
as not hideable, so it is possible to remove all the columns
and make the table disappear.
Set the recipe name and version columns as not hideable.
Also rename the "Name" column to "Recipe", for consistency with
the design and with other recipe tables.
[YOCTO #9833]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Thu, 21 Jul 2016 15:58:38 +0000 (18:58 +0300)]
toaster: set non-hideable columns for build tasks table
The task, recipe and order columns in the build tasks table
should not be hideable. If they are, it's possible for the
table to have all of its columns hidden so that it no longer
displays.
Set the hideable property to prevent these columns from being
hidden.
[YOCTO #9833]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Thu, 21 Jul 2016 15:42:35 +0000 (18:42 +0300)]
toaster: fix download URL for task logs
The task display template formatting had split the Django
url template tag across two lines and broken it. This resulted
in a gibberish URL for task logs.
Fix by placing the tag and its arguments on a single line.
[YOCTO #9837]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 21 Jul 2016 13:43:32 +0000 (14:43 +0100)]
toaster: loadconf Partially add back some of the layerSource parsing
Partially add back a revised version of the layersource handling so that
we can continue to support the old toasterconf.json and it's setup of
the local project.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 21 Jul 2016 13:43:31 +0000 (14:43 +0100)]
toaster: admin Add Layer_Version to the admin-able models
If the migration didn't get the release conversion right for say, a
local or imported layer it would be handy to be able to edit this
in the django admin page.
Also useful for developers to be able to tweak layers on the fly.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 21 Jul 2016 13:43:30 +0000 (14:43 +0100)]
toaster: orm Remove the layerindex specific up_branch fields
We don't need to keep track of layerindex data in our database. And
using branch==release is very confusing in the schema. Instead use the
existing Release definition to keep track of which release a
layer_version is for.
Remove the Branch model and all references to it.
Create a migration path to convert from up_branches to their
corresponding releases.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 21 Jul 2016 13:43:29 +0000 (14:43 +0100)]
toaster: lsupdates Add progress information and clean up logging
Adds basic progress % information and provides better description of
what is happening.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 21 Jul 2016 13:43:28 +0000 (14:43 +0100)]
toaster: tests: Remove references to LayerSource model
Remove and replace layersource model references in the tests and test
data. Remove the orm/test as this only tested LayerSource interactions
which have now been removed.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 21 Jul 2016 13:43:27 +0000 (14:43 +0100)]
toaster: Replace references to LayerSource models
Replace references to the now deprecated layersource models across
Toaster with the new enums for layer source types.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 21 Jul 2016 13:43:26 +0000 (14:43 +0100)]
toaster: lsupdates Add layerindex fetcher
Move and refactor the layerindex layer source update mechanism so that
we don't have to track the layerindex objects in the toaster database.
Move this out of the orm and into the management command.
Paves the way for future improvement to allow you to specify a layer
index server as an argument to the command.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 21 Jul 2016 13:43:25 +0000 (14:43 +0100)]
toaster: models Remove LayerSource models and replace with enum
We had a lot of complexity around different layer sources to allow for
multiple sources and different priorities for these source. This was
implemented using rigged abstract classes which represented the
different layer sources when in fact just an enum/flag on the
layer_version object is sufficient for our needs.
Remove the LayerSourcePriority object as this is not needed. We no longer
have a problem of multiple layers coming from multiple sources so this
is not needed. Two migrations are added to first remove the child models
which represented layersources. Then a second migration is needed to
remove the LayerSource model it's self as Django can't understand the
non-standard base class dependency. Triggering this issue:
https://docs.djangoproject.com/en/1.8/topics/migrations/#dependencies
Clean up a number of flake8 warnings in classes which were modified.
[YOCTO #9853]
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove the LayerSource admin from django admin interface. LayerSources
are not going to be manageable from the admin interface.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood [Thu, 21 Jul 2016 13:43:23 +0000 (14:43 +0100)]
toaster: loadconf remove Loading LayerSources
We don't need to configure layer sources in the initial configuration as
this information is provided by the models.
Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Fri, 22 Jul 2016 12:18:39 +0000 (00:18 +1200)]
lib/bb/progress: avoid possibility of start event being reported twice
In MultiStageProgressReporter, set a guard when we start the progress
so that it can't happen more than once. This fixes "Initialising
tasks.." being shown twice in succession when running bitbake in
non-interactive terminal mode.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Fri, 22 Jul 2016 12:18:38 +0000 (00:18 +1200)]
knotty: don't display ETA for tasks with progress
It turns out that progress information we can extract from a task is
rarely apportioned closely enough to the time taken for the ETA to be
accurate, so showing it is going to be misleading most of the time for
anything but the most basic of examples. Let's just remove it and avoid
misleading (or worse, annoying) the user.
Fixes [YOCTO #9986].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Fri, 22 Jul 2016 12:18:37 +0000 (00:18 +1200)]
knotty: fix some minor bugs in BBProgress
If you specify custom widgets then we don't want to assume where the
"extra" position is - you should have to specify it, and if it isn't
specified it shouldn't just wipe out the last widget or you can start to
see odd behaviour if you're modifying the code.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 22 Jul 2016 10:28:57 +0000 (11:28 +0100)]
cache: Don't interleave pickle cache file writing
For some reason the data written in this way is coming back out the
files out of order. I've not been able to simplify the test case to a
point where this was standalone reproducible. Simplify the code and
write out the cache files sequentially since this seems to avoid the
errors and makes the code more readable.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 22 Jul 2016 10:28:23 +0000 (11:28 +0100)]
cache: Add better cache loading sanity checks
We've seen cache corruption where the pairs come out in a different
order to the way we saved them for unknown reasons. Add better sanity
checking to give a more user friendly error rather than a crash/traceback.
Also allows the system to reparse and recover.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 22 Jul 2016 10:27:51 +0000 (11:27 +0100)]
cache: Drop/simplify pointless type checking
Since we no longer have random data like version fields in these structures
and we can assume any extra cache data subclasses our class, simplify the
code.
This is mostly reindenting after removal of the pointless type checks.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 21 Jul 2016 16:27:35 +0000 (17:27 +0100)]
cache: Correctly handle missing extra caches
If an "extras" cache file is corrupted, the system would not notice
and later fail with errors about missing entries. Add a test for this
which means we can fall back to re-parsing in those cases.
[YOCTO #9902]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 19 Jul 2016 20:20:34 +0000 (23:20 +0300)]
bitbake: implement idle timeout for xmlrpc server
Idle timeout can be specified either by -T/--idle-timeout option or
by sessing BBTIMEOUT environment variable. Bitbake xmlrpc server
will unload itself when timeout exprired, i.e. when server is idle
for more than <idle timeout> seconds.
[YOCTO #5534]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Scott Rifenbark [Thu, 14 Jul 2016 19:52:30 +0000 (12:52 -0700)]
bitbake-user-manual: Added review changes to BBCLASSEXTEND description
Fixes [YOCTO #9909]
Added some technical clarifications to the existing note to be clear
about what is causing the limitations in this case. Applied some
formatting fixed to for the use of the include word.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Scott Rifenbark [Thu, 14 Jul 2016 19:00:09 +0000 (12:00 -0700)]
bitbake-user-manual: Added review changes to recursive section.
Applied formatting to a "after" word that should be
<filename>after</filename>. Also added a new clarifying paragraph in
the "Recursive Dependencies" section.
Fixes [YOCTO #9861]
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Andrew Bradford [Thu, 14 Jul 2016 18:42:56 +0000 (11:42 -0700)]
bitbake-user-manual: Addeds support for the Perforce Fetcher
Added a new Perforce Fetcher section in the same spirit as the existing
sections for other supported fetchers. Changes included the new section,
removal of the bulleted item that mentioned this fetcher as an
"additional" fetcher, and the creation of a new variable in the glossary
named P4DIR.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Scott Rifenbark [Mon, 11 Jul 2016 21:24:22 +0000 (14:24 -0700)]
bitbake-user-manual: Updated dependency section for tasks
Fixes [YOCTO #9861]
I updated the section on dependencies to give a couple of examples
for task dependencies within a single recipe and dependencies for
tasks between two individual recipes.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Richard Purdie [Wed, 20 Jul 2016 15:08:17 +0000 (16:08 +0100)]
cooker: Fix incorrect dot file generation
In the runqueue cleanup/conversion, "dep" was mistakenly used where "tid" should
be leading to incorrect task-depends.dot files and causing general confusion.
Fix this, its clearly incorrect looking at the code.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
toaster-tests: add class SeleniumTestCaseBase for browser tests
In order to reuse the Selenium helper outside a Django
environment (for functional testing), add a new module
containing the base class SeleniumTestCaseBase, which only
inherits unittest.TestCase
Add a class SeleniumTestCase with multiple inheritance of
StaticLiveServerTestCase and SeleniumTestCaseBase to prevent
existing tests from breaking.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Mon, 18 Jul 2016 16:07:18 +0000 (19:07 +0300)]
bitbake: main: implement server autostart feature
If environment variable BBSERVER == 'autostart' bitbake will
automatically load server if it's not running yet.
If host and port are in bitbake.lock then bitbake tries to check
if server is running and responses to commands and starts new
server only if this check fails.
[YOCTO #5534]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Mon, 18 Jul 2016 16:07:16 +0000 (19:07 +0300)]
bitbake: xmlrpc: implement check of connection to server
Implemented check_connection function. The purpose of this function
is to check if bitbake server is accessible and functional.
To check this this function tries to connect to bitbake server and
run getVariable command.
This API is going to be used to implement autoloading of bitbake
server.
[YOCTO #5534]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Wed, 6 Jul 2016 04:26:10 +0000 (16:26 +1200)]
fetch2: implement progress support
Implement progress reporting support specifically for the fetchers. For
fetch tasks we don't necessarily know which fetcher will be used (we
might initially be fetching a git:// URI, but if we instead download a
mirror tarball we may fetch that over http using wget). These programs
also have different abilities as far as reporting progress goes (e.g.
wget gives us percentage complete and rate, git gives this some of the
time depending on what stage it's at). Additionally we filter out the
progress output before it makes it to the logs, in order to prevent the
logs filling up with junk.
At the moment this is only implemented for the wget and git fetchers
since they are the most commonly used (and svn doesn't seem to support
any kind of progress output, at least not without doing a relatively
expensive remote file listing first).
Line changes such as the ones you get in git's output as it progresses
don't make it to the log files, you only get the final state of the line
so the logs aren't filled with progress information that's useless after
the fact.
Part of the implementation for [YOCTO #5383].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Tue, 12 Jul 2016 22:54:56 +0000 (15:54 -0700)]
toaster-tests: add tests for build artifact display on build dashboard
Add tests for display of image, kernel and SDK artifacts on the
build dashboard, checking that the "Images" option in the left-hand
menu and the "Build artifacts" section display correctly for
different types of build.
Also add metadata to elements on the build dashboard so it's clearer
what they represent, and to assist in finding them in the tests.
Add a method to the test helper to make it more convenient to check
whether a single element matching a selector exists.
[YOCTO #8556]
[YOCTO #8563]
[YOCTO #9500]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Tue, 12 Jul 2016 22:54:55 +0000 (15:54 -0700)]
toaster: the customise image button shouldn't rely on targets
The build dashboard customise image button (for creating a new
custom image based on an image recipe used by a build) shouldn't
rely on targets: whether a new custom image can be created or not
depends on whether any of the recipes used by the build are image
recipes.
Modify the method used to determine whether a build has customisable
images to look at the image recipes used during the build, rather
than whether the targets run by the build refer to image recipes.
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Tue, 12 Jul 2016 22:54:54 +0000 (15:54 -0700)]
toaster: add package manifest path to Target objects
Store the path to the *.rootfs.manifest file for targets which
generate images.
A link to the package manifest is displayed in the build dashboard
for targets which produce image files.
Like the license manifest path, if a target would have produced
the package manifest (but didn't, because it already existed), that
path is copied from the target which did produce the package
manifest.
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Elliot Smith [Tue, 12 Jul 2016 22:54:53 +0000 (15:54 -0700)]
toaster: better display of targets which produced no images
SDK targets (populate_sdk) produce SDK artifacts but no image files.
Currently, these targets appear under the "Images" heading in the
build dashboard, even though they aren't strictly image targets.
Change the heading to "Build artifacts". Also remove the section
which states that a build produced no image files: this is not
correct for populate_sdk targets (those targets don't produce
image files under any circumstances); and other changes mean
that all targets which do produce images will now show those
files.
The check for whether to display the "Build artifacts" section also
needs to change, as we show targets here which didn't produce any
images but did produce SDK artifacts.
[YOCTO #8556]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
projects / thirdparty / openembedded / openembedded-core-contrib.git / log
