Chris Larson [Tue, 8 Feb 2011 16:53:47 +0000 (09:53 -0700)]
build: improve set -x handling
- set -x always, not just when debugging
- set -x immediately before executing the function, rather than at the top of
the script, so the output is no longer cluttered with all the variable and
function definitions
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Chris Larson [Mon, 7 Feb 2011 17:50:27 +0000 (10:50 -0700)]
Add initial bitbake-layers script
This script has subcommands which operate against your bitbake layers, either
displaying useful information, or acting against them. Currently, it only
provides a show_appends command, which shows you what bbappends are in effect,
and warns you if you have appends which are not being utilized.
Currently, a bug exists when using this due to the DataContext stuff, but I'm
not certain as to the root cause, it appears to be the bb package relying
implicitly on the way the bitbake script does things. A fix for that issue
will be forthcoming, as will further subcommands.
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Richard Purdie [Fri, 28 Jan 2011 10:21:41 +0000 (10:21 +0000)]
bitbake/runqueue.py: Avoid starvation of events to the server
The server UI was reading 1024 bytes, then sleeping for 0.25 seconds. Since
most new LogRecord events are larger than this it leads to a build up of data
which is only processed slowly, leading to a bottleneck and a slow down of
all bitbake processes.
Thanks to Dongxiao Xu <dongxiao.xu@intel.com> for the great work in debugging
this. A large value has been left in for the read() command just to ensure some
fairness amongst process handling if a task tries to log truly huge amounts of
data to the server, or goes crazy and ensures the main loop doesn't stall.
Chris Larson [Mon, 17 Jan 2011 15:50:40 +0000 (08:50 -0700)]
cooker: also respond to SystemExit for execute_runqueue
Without this, bitbake appears to hang when encountering an invalid task. In
reality, the SystemExit flows up uncaught, the server may go away, and the UI
doesn't know about it, so requires us to ^C to exit.
This is a temporary measure. Poky's bitbake handles invalid tasks
differently, and avoids this *particular* issue, but we do need to ensure that
the server responds to SystemExit from the idle function in a sane way.
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Chris Larson [Mon, 10 Jan 2011 16:20:50 +0000 (09:20 -0700)]
Inject taskpid into log records via our log handler
It turns out that while log filters added with addFilter are only associated
with that logger, and not its children, handlers are inherited, and handlers
can be filters. So, let's add filtering to our existing LogHandler class
which dispatches our log records as bitbake events.
Signed-off-by: Chris Larson <chris_larson@mentor.com>
The current parameters are not useful to the stampfile generator function
as they can't uniquely define a task. This updated things so the
parameters can identify unique tasks.
Yu Ke [Wed, 29 Dec 2010 07:28:48 +0000 (15:28 +0800)]
bb.utils: check if lock file is writable, to fix Yocto bug 606
Bug 606 report that if $DL_DIR is read-only, do_fetch will
simply hang without any error message.
The root cause is that: bb.fetch.go()->bb.utils.lockfile()
will try to lock file ${DL_DIR}/xxxxx.lock. Since ${DL_DIR}
is read-only, it will cause IOError exception. Although
lockfile() can catch the exception, currently code simply
ignore all the exception and continue the loop. it make
sense if the exception is caused by locking contention,
but in the read-only $DL_DIR case, it cause endless waiting
unfortunately.
So this patch add read-only check for lockfile to avoid the
silent hang.
Chris Larson [Fri, 7 Jan 2011 15:51:43 +0000 (08:51 -0700)]
Merge branch 'poky-sync'
* poky-sync:
build: kill unneeded environment setup bits
parse: save python functions into the metadata
Include exported variables in task process env
runqueue: start implementing quieterrors
Overhaul environment handling
runqueue: resurrect use of file objects for pipein/pipeout
runqueue: clean up message log levels
runqueue: use correct task ID when checking validity of setscene tasks
Add support for 'noexec' tasks
runqueue: implement cache checking from metadata
runqueue: pass task hash information to tasks
runqueue: avoid unnecessary delays
Implement task signatures
data: Add emit_func and generate_dependencies
codeparser: Implement persistent cache
data_smart: track variable references
codeparser: add module for parsing shell/python
runqueue: implement scenequeue
fetch: add SRC_URI checksum
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Chris Larson [Fri, 7 Jan 2011 15:38:41 +0000 (08:38 -0700)]
cache: don't expand variables for skipped recipes
Errors can result from these expansions, but for skipped recipes, we
shouldn't care about those failures. This fixes the same issue which
Richard Purdie fixed in poky, commit 847b717.
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Chris Larson [Tue, 4 Jan 2011 20:34:08 +0000 (13:34 -0700)]
parse: pass filename, lineno into the ast
We will be needing this information to improve the tracebacks of python code
from the metadata, as well as to give the user information about where
variables were defined, so they know how it ended up the way it is.
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Chris Larson [Fri, 10 Sep 2010 18:14:54 +0000 (11:14 -0700)]
Fix exit code display for task failure
Per the python documentation, os.waitpid returns the exitcode shifted up by 8
bits, and we weren't compensating, resulting in a display of 'failed with 256'
when a worker process exits with a code of 1.
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Only mark fn as clean if it is clean.
This saves us from removing (prematurely added) fn from our clean set
and saves me a few percent of runtime (and misleading debugging output
from remove()).
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Previously we emitted two newlines for export and unexport.
One newline for export and unexport is enough (and makes the scripts
look better and a tad smaller).
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Richard Purdie [Sun, 28 Nov 2010 17:39:09 +0000 (17:39 +0000)]
Overhaul environment handling
Currently, anything whitelisted in the environment makes it into the worker
processes. This is undesireable and the worker environment should be as
clean as possible. This patch adapts bitbake sosme variables are loaded into
bitbake's datastore but not exported by default. Any variable can be exported
by setting its export flag.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
Kevin Tian [Tue, 14 Dec 2010 11:21:24 +0000 (19:21 +0800)]
runqueue: use correct task ID when checking validity of setscene tasks
sstate hash validation is done at initialization of RunQueueExecuteScenequeue.
However the index of 'valid' list returned from the validation doesn't
correspond to setscene task ID. It's just an intermediate namespace between
runqueue and sstate hash func. Use it as setscene task ID fully mess the flow.
Previously this doesn't cause trouble because all setscene tasks are passed. Commit 58396a5d24c62710fd0a9f3780d84ac8a95d8e7c add 'noexec' concept to setscene
tasks which grabs some tasks out of the list and thus trigger this problem
Without this fix there're ~50 recipes (gzip-native, glib, ...) rebuilt weirdly
with a minimal build, even though existing sstate packages could accelerate them.
there's another typo using wrong task ID in a debug message which further hide
this issue
Richard Purdie [Tue, 3 Aug 2010 13:30:20 +0000 (14:30 +0100)]
data: Add emit_func and generate_dependencies
These functions allow generation of dependency data between functions and
variables allowing moves to be made towards generating checksums and allowing
use of the dependency information in other parts of bitbake.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Richard Purdie [Mon, 2 Aug 2010 09:20:20 +0000 (10:20 +0100)]
codeparser: add module for parsing shell/python
This commit is derived from Chris Larson's checksum work, turned into a
standalone piece of code for parsing python and shell functions.
The deindent code has been replaced with code to work around indentation
for speed. The original NodeVisitor in the ast was replaced with a faster
class walk call.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
- control variable BB_STRICT_CHECKSUM
when SRC_URI checksum is missing, this variable decide pass or not
if BB_STRICT_CHECKSUM = "1", missing checksum is fatal
Signed-off-by: Yu Ke <ke.yu@intel.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
Chris Larson [Tue, 28 Dec 2010 17:02:49 +0000 (10:02 -0700)]
bb.debug: handle caller not passing a debug level
This is a compatibility measure, as previous versions did not error out when
this occurred, and will most likely be dropped in the next bitbake minor
version in favor of a fatal error.
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Chris Larson [Wed, 29 Dec 2010 23:23:14 +0000 (16:23 -0700)]
fetch: resurrect branch inclusion in git revision key
This is similar to Richard Purdie's commit, but falls back to checking the
persist_data database for the old key if the new does not exist, so as to
retain compatibility for at least one bitbake minor version.
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Chris Larson [Tue, 28 Dec 2010 17:29:04 +0000 (10:29 -0700)]
build: work around do_clean issue by creating T in exec_func
In some repositories (e.g. poky), do_clean removes WORKDIR, then uses
exec_func, which expects T to exist, so let's make sure it does. I think its
fair to assume that the runfile dir exists in exec_func, so this is a
temporary measure.
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Chris Larson [Fri, 24 Dec 2010 02:22:21 +0000 (19:22 -0700)]
data_smart: don't include functions in name lookups
For raw name lookups in python snippets, don't return the values for functions
in the metadata. This was necessary to fix issues once the methodpool
functions get added to the metadata.
Signed-off-by: Chris Larson <chris_larson@mentor.com>
projects / thirdparty / openembedded / openembedded-core-contrib.git / log
