Richard Purdie [Sat, 31 Aug 2013 22:42:13 +0000 (23:42 +0100)]
serv/db: Don't use BEGIN/COMMIT
Since we don't support using multiple servers on the same database file,
don't use the BEGIN/COMMIT syntax and allow writes to the database
to work ~100 times faster with no transaction locking.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 31 Aug 2013 22:41:35 +0000 (23:41 +0100)]
serv/db: Fix looping upon database locked issues
If the database is locked we will get an immediate error indicating so,
there is no retry timeout. The looping code is therefore useless, the loop
count is near instantly exceeded.
Using a time based retry means we can wait a sensible time, then gracefully
exit.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 31 Aug 2013 22:40:55 +0000 (23:40 +0100)]
server/process, server/xmlrpc, runqueue: Use select.select() on fds, not time.sleep()
The existing backend server implementations were inefficient since they
were sleeping for the full length of the timeouts rather than being woken when
there was data ready for them. It was assumed they would wake and perhaps did
when we forked processes directory but that is no longer the case.
This updates both the process and xmlrpc backends to wait using select(). This
does mean we need to pass the file descriptors to wait on from the internals
who know which these file descriptors are but this is a logical improvement.
Tests of a pathaolgical load on the process server of ~420 rapid tasks
executed on a server with BB_NUMBER_THREAD=48 went from a wall clock
measurement of the overall command execution time of 75s to a much more
reasonable 24s.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 30 Aug 2013 16:41:16 +0000 (17:41 +0100)]
server/xmlrpc: Increase timeout to 60s
This is a better value that the earlier infinite timeout yet still
allows for servers with high loads. It does mean the bitbake process
can hang at exit for the timeout period but that should never happen
and only happened for me in some test cases which wouldn't happen
in normal use.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 29 Aug 2013 13:27:29 +0000 (14:27 +0100)]
cookerdata: Allow bblayers.conf to be found using BBPATH
It should be possible to run a build anywhere on the filesystem and have
bitbake find the correct build directory if its set somehow. The BBPATH
variable makes perfect sense for this usage. Therefore use any available
value of BBPATH to search for conf/bblayers.conf before walking the parent
directory structure.
This restores the option of being able to run bitbake from anywhere if
the user has set things up to operate in that environment.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 29 Aug 2013 13:26:53 +0000 (14:26 +0100)]
cookerdata: Set TOPDIR when using bblayers.conf
By definition, bblayers.conf is at the top of the build tree. We'd like
to support running bitbake anywhere within that build tree but TOPDIR
gets set to wherever cwd is. Change the code to reset TOPDIR
to the top of the build directory.
This shouldn't break anything but does make the system more usable.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 28 Aug 2013 16:10:09 +0000 (16:10 +0000)]
prserv/serv: Multithread the server
This makes the PR server multithreaded and able to handle multiple connections
at once which means its no longer a build bottle neck when serving one connection
at a time. I've experimented and database connection for each thread seems to
cause the least issues, pushing the contention for sqllite to handle itself.
This means moving the db/table connection code into the actual function methods.
It doesn't abstract well as a function since we need the db object around for
the lifetime of the function as well as the table else we lose the connection.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 28 Aug 2013 16:06:10 +0000 (16:06 +0000)]
prserv/db: Threading fixes
Enabling threading for the PRServer causes a number of issues. Firstly is
the obtuse error:
sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type
which is due to the class not being derived from object. See:
http://docs.python.org/2/library/sqlite3.html#registering-an-adapter-callable
Secondly, we want to enable multithreadded access to the database so we do this
when we open it. This opens the way up to multithreading the PR server.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
commit 972bc43e6d5b1207b944b3baa8f9805adb35dda7 (serv.py: Fix hang
when spawned dynamically with bitbake) introduced a regression,
because the wrong patch was submitted. The syntax was incorrect in
the original patch. The logger iterator must be used with a call to
getLogger().
[YOCTO #5059]
Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is more idiomatic, and from the limited performance testing I did, is
faster as well. See https://gist.github.com/kergoth/6360248 for the naive
benchmark.
Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jason Wessel [Tue, 27 Aug 2013 20:12:55 +0000 (15:12 -0500)]
serv.py: Fix hang when spawned dynamically with bitbake
The PRServer has the possibility to hang indefinitely blocking on a
semaphore processing a xmlrpc request to send an event back to the
main bitbake instance. This was observed during a "bitbake -e" on a
heavily loaded machine and the main bitbake instance and cooker exited
before the PRServer emitted its first log.
The stack trace is provided below as to show what happens every time a
logger.info() is executed in the PRServer. Not only does it write to
the stream handler but it also tries to send the event to the main
event processor.
It was never intended for the PRServer to send its logs anywhere but
its own log file. The event processing is an artifact of how the
PRServer was forked and it inherits the event log handlers. The
simple fix is to clean up and purge all the log handlers after the
fork() but before doing any of the typical PRServer work or logging.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 9 Mar 2013 12:47:21 +0000 (12:47 +0000)]
data_smart: Add _remove operator
There are long standing complaints about the fact its very difficult
to remove a portion of a variable. The immediate request is for a -=
and =- operator. The trouble is that += and =+ are "immediate"
operators and are applied straight away. Most people would expect
-= and =- to be deferred to have the effect most people desire and
therefore implementing -= and =- would just make the situation more
confusing.
This deferred operation is much more similar to the override syntax
which happens at data store finalisation. The _remove operator is
therefore in keeping with the _append and _prepend operations.
This code is loosely based on a patch from Peter Seebach although it
has been rewritten to be simpler, more efficient and avoid some
potential bugs.
The code currently only works on space delimited variables, which
are by far the most commom type. If bitbake is ehanced to support
types natively in future, we can adjust this code to adapt to that.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 23 Aug 2013 16:16:50 +0000 (16:16 +0000)]
bitbake: Add ui event handlers filtering
Add functionality to allow UIs to update and change the types of events they
recieve. To do this we need to add a new command and also need to be able
to obtain the current event hander ID. In the case of xmlrpc, this is
straightforward, in the case of the process server we need to save the result
in a multiprocessing.Value() so we can retrive it. An excplit command
was added to the server API to facilitate this.
The same function can also be used to mask or unmask specific log messages,
allowing the UI to optionally differ from the standard set of message
filtering.
Based upon work by Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 24 Aug 2013 12:03:10 +0000 (12:03 +0000)]
server/xmlrpc/prserv: Add sane timeout to default xmlrpc server
The standard python socket connect has long timouts which make sense for remote
connections but not local things like the PR Service. This adds a timeout
parameter to the common xmlrpc server creation function and sets it to a more
reasonable 5 seconds.
Making the PR server instantly exit is a good way to test the effect of this
on bitbake.
We can remove the bodged timeout in the PRServer terminate function which
has the side effect of affecting global scope.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 24 Aug 2013 11:56:31 +0000 (11:56 +0000)]
process: Improve exit handling and hangs
It turns out we have a number of different ways the process server termination can
hang. If we call cancel_join_thread() on the event queue, it means that it can be left
containing partial data. This means the reading of the event queue in the terminate()
function can hang, the timeout and block parameters to Queue.get() don't make any
difference.
Equally, if we don't call cancel_join_thread(), the join_thread in terminate()
will hang giving a different deadlock.
The best solution I could find is to loop over the process is_alive() after requesting
it stops, trying to join the thread and if that fails, try and flush the event
queue again.
It wasn't clear what difference a force option should make in this case, we're
gracefully trying to empty queues and shut down regardless of whether its a SIGTERM
so I've simply removed the force option.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Martin Jansa [Fri, 23 Aug 2013 17:06:26 +0000 (19:06 +0200)]
parse: Return IOError when including file with absolute path
* resolve_file was behaving different when relative and absolute
paths were passed to it
* include relative-path/non-existent-file.inc
works correctly resolve_file throws IOError, BBHandler.py:handle()
doesn't catch it, ConfHandler.py:include() catches IOError and shows:
DEBUG: CONF file 'relative-path/non-existent-file.inc' not found
* include /absolute-path/non-existent-file.inc
was failing, because resolve_file just returns fn,
BBHandler.py:handle() calls bb.parse.mark_dependency(d, abs_fn)
which throws:
OSError: [Errno 2] No such file or directory: '/absolute-path/non-existent-file.inc'
and parsing fails.
Ad isfile() test for absolute fn and throw IOError to make
resolve_file behavior consistent for both paths.
* I know we had some issues with -b relative-path-to-recipe.bb and
absolute path, so consider this patch only as RFC and documentation of
this problem
* Catch OSError too in ConfHandler.py:include() e.g. in case the file exists, but user
cannot read it or something like that.
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 23 Aug 2013 16:41:53 +0000 (16:41 +0000)]
prserv/serv: Fix pid file removal
Mark Hatle spotted there were pid files being left around. This patch
fixes things so the removal function is called correctly, the code
contained a typo.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 23 Aug 2013 14:55:21 +0000 (15:55 +0100)]
event/msg: Add primitive server side UI log record filtering
Currently one of the bigger bottlenecks in bitbake is passing all the
log messages over IPC to the UI. This is worthwhile if the UI is going
to use them, pointless otherwise. The memory resident bitbake suffers
from this performance issue particularly badly.
This patch filters the log events on the server side with the global
log levels and hence reduces the traffic. This speeds up parsing
(18.5s down to 17s) and bitbake general command overhead is reduced
(7.3s for a NOP to 6.2s).
What isn't added here is general event filtering or the ability to
change the log levels once set. Provision is made for adding this
in a follow up patch though.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Nicolas Dechesne [Fri, 16 Aug 2013 09:20:59 +0000 (11:20 +0200)]
build.py: create symlink for run.do_xxx scripts
The 'courtesy' symlink for log.do_xxx are quite useful when debugging, so
with this commit, we now get similar 'courtesy' symlink for run.do_xxx
scripts.
We only create symlink for tasks, not individual functions.
The symlink is create right before the actual runfile is created, indeed
we cannot create the symlink right after running the task since a failure
or execption can happen, in which case the symlink wouldn't be created,
and symlink are particularely useful when the task failed!
Another option would be create the symlink after the runfile is created,
and before the script is executed, but that means we need to duplicate the
code in case of Shell vs Python task.
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Cristiana Voicu [Tue, 20 Aug 2013 13:30:22 +0000 (16:30 +0300)]
hob: changes to image combo box from image configuration page
In some cases when a new machine is selected, the image combo
shows the same option. Fixed this issue, because the image
combo box should be reseted.
Fix the counter for the options in the combobox. This bug
was introduced by the templates functionality. The combo box had some
last changes, and I forgot about this counter.
Andrei Dinu [Wed, 15 May 2013 09:59:26 +0000 (09:59 +0000)]
propertydialog.py : Scrollable windows for long tooltips
In some cases, the length of the description and the brought
in by field was too big. That led to the size of the property
dialog exceeding Hob's size. For long tooltips we use
scrollable windows now.
[HOB #4321]
Signed-off-by: Andrei Dinu <andrei.adrianx.dinu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Peter Seebach [Wed, 10 Oct 2012 22:11:54 +0000 (17:11 -0500)]
build.py: Provide useful diagnostics when exiting.
Running scripts with 'set -e' produces silent failures with no
diagnostic. Add an exit handler which produces diagnostics, including
details of what was running if the shell seems to be bash.
Signed-off-by: Peter Seebach <peter.seebach@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 12 Aug 2013 09:22:40 +0000 (10:22 +0100)]
hob: Add missing file from previous commit
This should have been part of 'hob: implement the "retrieve image dialog" +
changes to image combo box' but got lost in the merge process. This adds
the missing file.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cooker: save packages in IMAGE_INSTALL instead of PACKAGE_INSTALL
Hob retrieves the list of recipes and packages using the IMAGE_INSTALL
variable, so a custom image should be saved using this variable.
Changed how the image is saved in a bb file
[YOCTO #4193] Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: implement the "retrieve image dialog" + changes to image combo box
Tha changes related to the image combo box are related to the
action done in the retrieveImageDialog. When the user wants to select
a customize image, but then he cancels the action, the combo box is set to
--select a base image--.
If the user selects an image using the new dialog, a new item with its name
is added to the combo box list and then it is activated.
[YOCTO #4193] Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The entire file name (with the path) is needed to know
if the image is located in the "build" directory or it comes
from layers. According to this information, the image is placed
differently in the combobox.
[YOCTO #4193] Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob/bitbake: when an image is saved, it should require an image from layers
Saving an image that requires another image will cause issues when
the second on is removed. So, we have agreed to "require" only the images
from layers.
The functionality is implemented in bitbake, in order to be more abstract,
and it is used by Hob when an image recipe is saved.
[YOCTO #4193] Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob & bitbake: append a value to a variable from hob throught bitbake
It was necessary to append ${TOPDIR}/recipes/images to BBFILES.
Implemented the mechanism to append a value to a variable: a command and
the method in cooker.
[YOCTO #4193] Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob/bitbake: save the description of a custom image
When an new image is saved, the dialog for this action has
a field for the description. Changed how an image is saved, by
appending the DESCRIPTION variable at the end of the .bb file.
[YOCTO #4193] Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob/bitbake: create a template (a .bb file) from hob through bitbake
Modified generateNewImage function from cooker, in order to be used to
save a template in Hob.
Created a command to ensure that some dirs are created. The templates
(recipes) will be saved in {TOPDIR}/recipes/images folder.
Called these methods from Hob.
[YOCTO #4193] Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob: create save image dialog used to save a template
Implemented a new dialog used by Hob. This dialog was desinged
in order to permit to save only in a particular directory.
Also, it has a field where the user can type a description
for the image.
Implemented in the handler a method to retrieve the topdir variable,
because the changes will be saved in {topdir}/recipes/images directory.
[YOCTO #4193] Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob/packageslist: change image size label to estimated image size
On the "Edit packages list" page, the image size may be computed before
building. We didn't find another way to give a more accurate size, so
we have agreed to inform the user that this an estimated size.
[YOCTO #4388] Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a file ends with .xz, it currently gets overwritten during unpack:
The decompress command for .xz files is:
'xz -dc %s > %s' % (file, efile)
and as efile == file, we end up overwriting file (the source).
Fix this by adding .xz to the list of suffixes that that need to
be removed from a file name for an extract command, leaving the
bare file name. Now, for a given file foo.xz,
file == foo.xz and efile == foo, similar to how .gz .bz2 and .Z
files are treated.
Until now the split made searching the string "file://", but
this is not ok when SSTATE_VARIABLE has the following form:
SSTATE_MIRRORS ?= "\
file://.* http://someserver.tld/share/sstate/PATH \n \
file://.* file:///some/local/dir/sstate/PATH"
In the documentation I've found that \n is the list separator.
[YOCTO #4857] Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Robert Yang [Fri, 19 Jul 2013 03:07:51 +0000 (03:07 +0000)]
providers.py: print a debug message for dynamic pacakge
We can't know the dynamic pacakge's name exactly, there might be a
problem, for example, when we use:
IMAGE_INSTALL_append += "ncurses-lib12344"
The ncurses-lib12344 matches ncurses' dynamic packages pattern:
PACKAGES_DYNAMIC = "^${PN}-lib.*"
so there is no errors before the rootfs creation though there is no
ncurses-lib12344.
We can warn this, but I think that we'd better not since there are many
dynamic packages, or there would be too many warnings, for example, the
perl and kernel modules, maybe we can print a debug message for it.
[YOCTO #4798]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modal dialogs doesn't run on the main loop so they cannot
catch any signal from the terminal. This patch makes sure
the dialogs are destroyed when a SIGINT is sent to HOB.
[YOCTO #3329]
Signed-off-by: Valentin Popa <valentin.popa@intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 21 Jun 2013 11:31:32 +0000 (11:31 +0000)]
data_smart: Fix bug with overrides and weak default values
(aka pay the cookie monster for weak defaults)
If you have code like:
MYVAR = "a"
MYVAR_override ??= "b"
then MYVAR will get the value "a" even when override is in OVERRIDES. The
reason is that the value of ??= is set as a flag not a value and the cookie
monster isn't paid.
The fix is to ensure appropriate payment is made for a defaultval varflag
matching the usual setVar case.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 20 Jun 2013 20:53:14 +0000 (21:53 +0100)]
data_smart: Ensure variable flags are accounted for in config data hash
Currently if the flags set against a variable in the base data store
change, it doesn't automatically trigger a reparse when it really
should. For example with the blacklist class setting:
will not trigger a reparse if only one entry is changed and a
blacklisted recipe can still be built.
I did consider using BB_SIGNATURE_EXCLUDE_FLAGS in here however it
doesn't make sense, we want to trigger a reparse when any of the
flags change too (which is different to the sstate signatures which
we wouldn't want to change in those cases).
[YOCTO #4627]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Cristiana Voicu [Thu, 20 Jun 2013 14:21:17 +0000 (17:21 +0300)]
hob: save variables in conf/ directory
Until now the variables were saved in bitbake configuration,
now they are saved in configuration files, in order to be read by
bitbake-worker. This helps to assure the consistency for the rest
of the variables.
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Cristiana Voicu [Thu, 20 Jun 2013 14:21:16 +0000 (17:21 +0300)]
hob: replace the init cooker with reset in order to keep the history
When the cooker initialization is called, the variables history is lost.
It need a reset, in order to load the configuration files.
Also, concatenated the value for INHERIT variable.
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Cristiana Voicu [Thu, 20 Jun 2013 14:21:15 +0000 (17:21 +0300)]
bitbake: call the initConfiguration when reparsing
As it is implemented now, when parsing the configuration files is
needed, it also needs an initialization.
Also, implemented a function to create an extra configuration files
which can be used as postfile/prefiles when parsing.
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Cristiana Voicu [Wed, 19 Jun 2013 14:23:28 +0000 (17:23 +0300)]
bitbake: remove the function for reparse because it does nothing
The command for reparse does nothing, so I have managed to remove it,
because it can be misleading. When a reparse is needed, it can be used
the parse function.
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Cristiana Voicu [Thu, 20 Jun 2013 10:19:42 +0000 (13:19 +0300)]
bitbake & hob: modify some commands and call them from hob
This patch adds a new command to create an extra configuration
file, in order to create it from Hob.
Also, it calls the enable/disable data tracking when needed.
Modified the prefiles and postfiles variables because the cooker
expects some lists.
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 19 Jun 2013 13:03:39 +0000 (14:03 +0100)]
runqueue/build: Add recideptask flag
Currently, tasks like fetchall are slightly broken since if a recipe
has specific [depends] which occur after do_fetch and add items not listed
in DEPENDS and RDEPENDS, they are not caught by recrdeptask. We've gone
around in circles on this issue (e.g
http://git.yoctoproject.org/cgit.cgi/poky/commit/bitbake/lib/bb/runqueue.py?id=5fa6036d49ed7befe6ad50ec95c61a50aec48195
) and in many cases the behaviour of recrdepends is correct but tasks like
fetchall need the other behaviour.
To address this we add a recideptask flag which can be used in conjuction
with the recrdeptask flag to specify which task to to the inspection upon.
This means entries like do_rootfs[depends] which have do_fetch tasks are
caught and run.
I'm not 100% happy with needing another flag but I don't see any rational
way to get the correct behaviour in all cases without it.
[YOCTO #4597]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Alexandru DAMIAN [Mon, 17 Jun 2013 11:11:53 +0000 (12:11 +0100)]
bitbake: read BBSERVER environement for remote server
This change allows bitbake to connect to a remote server
by reading the BBSERVER environment variable and effecting
the config. Basically a shortcut to the command line parameters.
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Alexandru DAMIAN [Mon, 17 Jun 2013 11:11:51 +0000 (12:11 +0100)]
knotty, xmlrpc: add observer-only mode
I add an observer only mode for the knotty UI and
the XMLRPC server that will allow the UI to register
a callback with a server in order to receive events.
The observer-UI is able to send read-only commands to the
server, and also is able to register as an event handler.
Read-only commands are the commands that do not change
the state of the server and have been marked as such in
the command module.
The observer can switch to a full client if it calls addClient
at any time, and the server has no other client running.
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the double wrapper script environment, we'd not see issues like
this as it would be masked. The problem theoretically affects code like
useradd too as well as anything using a sstate postinstall.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bogdan Marinescu [Fri, 31 May 2013 14:19:14 +0000 (14:19 +0000)]
bitbake: Add event mask flag support
Add a flag to event handlers which lists the events a given handler wishes to
process. By default event handlers recieve all events but this means
we can stop running code in many cases if we know it doesn't want the event.
This is part of the fix for YOCTO #3812, but implements filtering only
for class event handlers; the other part (events filter for UIs) will be
the subject of a different patch.
Signed-off-by: Bogdan Marinescu <bogdan.a.marinescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 7 Jun 2013 17:13:04 +0000 (18:13 +0100)]
prserv: Adapt autostart to bitbake-worker
With the change to bitbake-worker we need to ensure the workers know
how to contact the PR service, the magic 0 port and singleton is
no longer enough.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 7 Jun 2013 17:11:49 +0000 (18:11 +0100)]
runqueue: Move the bitbake-worker execution to a higher level
The worker was being executed by each execution queue so would get
constructed twice for each build. This is wasteful so move execution
to the main runqueue so we only have to start the worker once.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 7 Jun 2013 17:11:09 +0000 (18:11 +0100)]
runqueue: Split runqueue to use bitbake-worker
This is a pretty fundamental change to the way bitbake operates. It
splits out the task execution part of runqueue into a completely
separately exec'd process called bitbake-worker.
This means that the separate process has to build its own datastore and
that configuration needs to be passed from the cooker over to the
bitbake worker process.
Known issues:
* Hob is broken with this patch since it writes to the configuration
and that configuration isn't preserved in bitbake-worker.
* We create a worker for setscene, then a new worker for the main task
execution. This is wasteful but shouldn't be hard to fix.
* We probably send too much data over to bitbake-worker, need to
see if we can streamline it.
These are issues which will be followed up in subsequent patches.
This patch sets the groundwork for the removal of the double bitbake
execution for psuedo which will be in a follow on patch.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 7 Jun 2013 17:10:36 +0000 (18:10 +0100)]
fetch2: Fix AUTOINC handling
AUTOINC was meant to appear once at the start of the version string.
The list of names may not be sorted meaning it could get inserted in
the middle. This patch simplifies the code and ensures it appears at
the start.
Include cache version bump to ensure the cache picks up these changes.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 4 Jun 2013 08:59:37 +0000 (09:59 +0100)]
lib/bb/cache.py: invalidate cache when file checksum entry no longer exists
Go through the cached list of file checksums and check if any of the
files no longer exist; if any are missing then invalidate the cached
recipe, which will force it to be reparsed and thus force the list of
files to be collected again. This prevents a warning when moving a file
to a different location that is still picked up by the recipe, e.g.
moving a file from a "files" subdirectory to one named with the recipe
name (${BPN}).
Fixes [YOCTO #4474].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
projects / thirdparty / openembedded / openembedded-core-contrib.git / log
