Chris Larson [Thu, 24 Feb 2011 20:16:43 +0000 (13:16 -0700)]
cooker: use BBHandler.inherit for INHERIT
Ideally we'd avoid direct BBHandler usage, but honestly, to say the whole
bb.parse abstraction is incredibly leaky is an understatement. If we try to
make handle() not reparse classes, things get ugly fairly quickly, as
inherit() calls handle() itself after adding the class to the inherit cache.
This change fixes it so we no longer risk reparsing a class if:
- it's listed in INHERIT multiple times
- it's listed in INHERIT and is 'inherit'ed from a class in INHERIT
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Richard Purdie [Thu, 24 Feb 2011 22:02:19 +0000 (22:02 +0000)]
build.py: Operate logfile handling at the task level
Previously, not all task output could end up in the logfile and
sub functions run using exec_func() calls would end up logging to
/dev/null. This is extrememly confusing for users who want to find
the output from a given task as there are several possible log files,
or it might not be there at all.
This patch changes exec_task() to redirect standard output/error
to the logfile along with any python logging output. This change is
applied for all python/shell tasks so it doesn't matter whether output
comes from os.system() calls or wherever, the logs are always correct.
This also removed the ton of log files previous code could generate
in ${T}, again making this simpler for users.
Based upon patches from Poky.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If expanding a variable triggers an exception the caller currently has no
way to supress the error message or otherwise handle the siutation. An
example of where this is a problem is "bitbake -e" showing tracebacks and
errors for variables like SRCPV in OE/Poky.
Secondly in a chained expansion fails, log mesages are recorded for
every step of the expansion, not just the innermost error which is
where the real failure occured.
To fix this we introduce a new exception ExpansionError which callers
can handle as appropriate.
Richard Purdie [Wed, 23 Feb 2011 11:48:01 +0000 (11:48 +0000)]
bitbake/data_smart: Improve the way lazyassignment works
Currently, if a variable has been set with ??= and the code looks it up
before the data finalisation phase, no value is found. This is causes
serious problems for anonymous python functions which manipulate data, or
for the fetcher revision handling code where revisions can be set with
??=.
There is also a significant performance implication for processing lazy
assignment in finalise.
Moving the check for a default value into getVarFlag addresses both
the timing issue and the performace. This change gives a 7% real time
performance improvement to parsing the Poky metadata. The cost of the
check at this point is minimal since we have all the data flags available.
Richard Purdie [Wed, 23 Feb 2011 11:09:07 +0000 (11:09 +0000)]
bitbake/cooker: Fix parsing failure zombie problem
When parsing if a SystemExit event is triggered, it causes the parsing thread to
exit and the main process hangs waiting for it to finish indefintely. Add code to
catch BaseExceptions and raise these with the main process gracefully instead
of just hanging indefinitely with zombie processes.
Mark Hatle [Fri, 11 Feb 2011 17:43:54 +0000 (11:43 -0600)]
fetch2: unpack revision
Revise the unpack function to have a way to disable the unpack. This is
based on the work from "Andreas Oberritter <obi@opendreambox.org>", see
http://cgit.openembedded.net/cgit.cgi/openembedded/commit/?id=2bdfe8519eda8067845019a699acdf19a21ba380
In addition, the to_boolean function comes from the work of
"Chris Larson <chris_larson@mentor.com>", see
http://cgit.openembedded.net/cgit.cgi/openembedded/commit/?id=900cc29b603691eb3a077cb660545ead3715ed54
Kevin Tian [Thu, 20 Jan 2011 05:54:51 +0000 (13:54 +0800)]
siggen.py: better print for task hash comparison
current bitbake-diffsigs simply print out the whole 'runtaskdeps' when there's mismatch, which
is not very readable. On the other hand, 'runtaskhashes' comparison is broken which assumes
same key existing in two sides. This commit provides better output by figuring out differences
from addition, removal or hash change.
Take a real world testcase where you have two recipes, each of which
contains PACKAGES_DYNAMIC = "gdk-pixbuf-loaders-*" and recipes which
RDEPEND on some gdk-pixbuf-loaders-xxx package. To select between these
you need to set a PREFERRED_PROVIDER.
These are specified in the PN namespace so the locgical conclusion is
that setting PREFERRED_PROVIDER_gdk-pixbuf = "gtk+" should work. It
doesn't and instead checks crazy things.
The code was correctly finding the two possible providers, gtk+ and
gdk-pixbuf. It was however only accepting PREFERRED_PROVIDER_gtk+
= "gdk-pixbuf" to resolve this problem which reads as the exact
opposite to what was wanted.
This patch changes the code to do something that makes sense. I suspect
that before these changes it was pretty much a null operation rubber
stamping the single provider case. For Poky at least it exposes a few
cases where -nativesdk recipes were providing the same things as their
normal counterparts but these are genuine bugs in the metadata.
I've also attempted to make the multiple provider error message human
readable as I counldn't understand it and I doubt anyone else could
either.
Dongxiao Xu [Tue, 18 Jan 2011 08:18:18 +0000 (16:18 +0800)]
Introduce stamp-extra-info task flag into stamp filenames
For certain tasks, we need additional information in build stamp file
other than the task name and file name. stamp-extra-info is introduced as
a task flag which is appended to the stamp file name.
Even when a variable was whitelisted, any dependencies of that variable
could still creep into the task hash due to the way the whitelisting
code worked. This patch changes thing to ensure that when whitelisted,
that whitelisting applies to the variable and any dependencies it has.
Chris Larson [Fri, 11 Feb 2011 16:42:38 +0000 (09:42 -0700)]
persist_data: drop SQLData indirection
The common case (if not the only case) is to only use a single domain. The
only reason SQLData exists is to make it easier to delete a domain. Yet,
there's no need for us to delete a domain if SQLTable knows how to clear
itself out. So, add clear() to the table and pass the domain to persist().
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Mark Hatle [Tue, 8 Feb 2011 00:18:18 +0000 (18:18 -0600)]
fetch2: Add SRPM knowledge
Enable the fetcher to be able to unpack and SRPM. By default the system will
unpack the contents of the SRPM into the WORKDIR.
A new syntax "unpack=file" was developed for the SRC_URI, to allow for a
recipe to extract a specific file within an SRPM. An unpack operation will
then be executed on the extracted file.
In order to apply extracted patches (or unpack files not specified with
unpack), you must specify the path using WORKDIR, i.e.:
Richard Purdie [Mon, 7 Feb 2011 09:13:44 +0000 (09:13 +0000)]
bitbake/fetch2/git: Fix localpath to point at the clone repo since we no longer always generate a mirror tarball and it isn't a good guide to fetcher success
Richard Purdie [Fri, 4 Feb 2011 10:30:54 +0000 (10:30 +0000)]
bitbake/fetch2: Drop old md5 handling code
Drop some old md5 functions since we have improved functionality now which includes
sha256 checksum support. This stops each download being md5 checksumed twice.
Also change ".md5" stamp extentions to ".done" to better describe its use as a
download complete marker file and no longer write the md5 sum to the files.
Richard Purdie [Sun, 6 Feb 2011 10:20:18 +0000 (10:20 +0000)]
bitbake/fetch2/git: Switch to completely use bare clones
We no longer need index/workdir support in the mirror tree and it causes all
kind of reference naming problems.Simplifying the code to remove this and use
just bare clones addresses this problem.
We increase the "version" number on the mirror tarballs to reflect the change
and ensure older mirror tarballs are not used as they would break.
Richard Purdie [Thu, 3 Feb 2011 20:16:05 +0000 (20:16 +0000)]
bitbake/fetch2/git: Ensure deleted branches upstream are removed, all branch tags are forced to the upstream revisions and all revisions and tags are always fetched
Richard Purdie [Thu, 3 Feb 2011 18:24:25 +0000 (18:24 +0000)]
bitbake/fetch2: Remove old local file acceleration code
Since local mirror fetches are always symlinked from the download directory
directly, there is no need for this premirrors hack which doesn't cover
mirrors and also abuses the localpath variable with inconsistent results.
Richard Purdie [Thu, 3 Feb 2011 18:22:06 +0000 (18:22 +0000)]
bitbake/fetch2: Ensure that mirror fetches are symlinked from the download directory
When files are fetched from a mirror source that happens to be local,
ensure links are created for the file since subsequent fetch calls
can then follow the links to find files.
Any other approach such as the existing manipulations of localpath
internally to the fetcher are prone to errors, races and other issues.
Richard Purdie [Thu, 3 Feb 2011 18:19:23 +0000 (18:19 +0000)]
bitbake/fetch2: When encoding a file:// url, drop user and host information
When processing a cvs SRC_URI to a file:// mirror, the user and host information
will break the mirror processing. This patch addresses it by only constructing
valid urls.
Yu Ke [Wed, 26 Jan 2011 12:14:06 +0000 (20:14 +0800)]
fetch2: Allow multiple src rev to be used in one src_uri
* SRC_URI format:
the SRC_URI are extended to allow multiple src rev:
name=<name1>,<name2>,...<name-n>
branch=<branch1>,<branch2>,...,<branch-n>
also SRCREV can be defined with
SRCREV_<name1> = xxxxx
SRCREV_<name2> = xxxxx
* FetchData extention
to support multiple src rev, several FetchData data are added:
- FetchData.names: list of name in SRC_URI, one name per srcrev. name is the index of revision and branch
- FetchData.revisions: dictionary of name->revision.
- FetchData.branches: dictionary of name->branch.
For example, linux-yocto recipes becomes:
SRC_URI = "git://git.pokylinux.org/linux-yocto-2.6.37;protocol=git;branch=${KBRANCH},meta;name=machine,meta"
* generic revision handling extension
the related revision handling code in fetch2.__init__.py are changed accordingly. the major change is add name parameter to indicate which src rev to handling. originally there is one src rev per FetchData, so FetchData parameter is enough. now since one FetchData has multiple src rev, it is necessary to use FetchData + name to specifiy src rev.
* git extension
git fetcher are also revised to take advantage of the multiple src rev in FetchData. especially the download() method are enhanced to fetch multiple src rev.
* other fetcher (svn, hg, ...) does not support multiple src rev. they just sync the API to add name, and then simply ignore the name. no actually functional change
Yu Ke [Tue, 18 Jan 2011 16:58:42 +0000 (00:58 +0800)]
git.py: Remove the source tree tarball archives
Since we're now always providing the git source control files it becomes
pointless to handle the tarballs of specific git revisions so drop this
part of the fetcher.
Yu Ke [Tue, 18 Jan 2011 16:22:13 +0000 (00:22 +0800)]
bb.fetch2: Add git unpack
The git download method clones the git repository to the local machine. The unpack process
can be optimised to be a local to local machine clone or a direct readtree operation to the
destination using git.will clone git repo to local, so git unpack can be simplified
to only checkouting the code to the work dir. For fullclone case, we also
need to manually copy all the ref info, which is needed by the later do_kernel_checkout().
Rather than use hardlinks, we reference the repository using alternatives since the
download directory may be on a different filesystem.
projects / thirdparty / openembedded / openembedded-core-contrib.git / log
