Paul Eggleton [Mon, 2 Sep 2013 10:35:14 +0000 (11:35 +0100)]
runqemu: set memory size to 256M for most qemu machines
Set memory size to 256M for qemuarm, qemux86, qemux86-64, qemumips,
qemumips64, and qemuppc.
This allows the smart automated tests to run on machines with a GUI
environment (such as Sato) running at the same time, for which 128M is
too limiting. Setting this in runqemu allows users manually using
runqemu to avoid the same out-of-memory issues under similar conditions
using smart, on-target compilation or other uses.
Richard Purdie [Sun, 1 Sep 2013 15:19:32 +0000 (16:19 +0100)]
sanity: Don't make assumptions about cwd
When using the recently fixed out of build directory bitbake invocations, I was
puzzled why bitbake seemed to be pausing. The reason was due to running the sanity
tests each and every time. This was due to current working directory assumptions
within the sanity test code. Fix this to use TOPDIR.
Richard Purdie [Sun, 1 Sep 2013 07:52:40 +0000 (08:52 +0100)]
meta: Don't use deprecated bitbake API
These have been deprecated for a long time, convert the remaining
references to the correct modules and prepare for removal of the
compatibility support from bitbake.
Alexandru DAMIAN [Sat, 31 Aug 2013 22:46:06 +0000 (23:46 +0100)]
bitbake: server/xmlrpc: stop server on client exit
If the server only expects a single client to connect (i.e.
no bind parameter set, so there is no way for the
clients to get the server port), stop the server after
the first client exits.
Richard Purdie [Sat, 31 Aug 2013 22:44:42 +0000 (23:44 +0100)]
bitbake: prserv/serv: Settle on two threads for optimal performance
Using the threading mixin class resulted in large amounts of memory
being used by the PR server for no good reason. Using a receiver thread
and a thread to do the actual database operations on a single connection
gives the same performance with a much saner memory overhead so
switch to this.
Richard Purdie [Sat, 31 Aug 2013 22:43:37 +0000 (23:43 +0100)]
bitbake: build: Fix profile file names
Using the basename of the .bb file is not unique, for example xxx-native
and xxx can overwrite each other. If this happens whilst running, you can
get odd backtraces as one file is parsed as another tries to write out
new data.
Avoid issues by using PN for the output filename instead.
Richard Purdie [Sat, 31 Aug 2013 22:42:13 +0000 (23:42 +0100)]
bitbake: serv/db: Don't use BEGIN/COMMIT
Since we don't support using multiple servers on the same database file,
don't use the BEGIN/COMMIT syntax and allow writes to the database
to work ~100 times faster with no transaction locking.
Richard Purdie [Sat, 31 Aug 2013 22:41:35 +0000 (23:41 +0100)]
bitbake: serv/db: Fix looping upon database locked issues
If the database is locked we will get an immediate error indicating so,
there is no retry timeout. The looping code is therefore useless, the loop
count is near instantly exceeded.
Using a time based retry means we can wait a sensible time, then gracefully
exit.
Richard Purdie [Sat, 31 Aug 2013 22:40:55 +0000 (23:40 +0100)]
bitbake: server/process, server/xmlrpc, runqueue: Use select.select() on fds, not time.sleep()
The existing backend server implementations were inefficient since they
were sleeping for the full length of the timeouts rather than being woken when
there was data ready for them. It was assumed they would wake and perhaps did
when we forked processes directory but that is no longer the case.
This updates both the process and xmlrpc backends to wait using select(). This
does mean we need to pass the file descriptors to wait on from the internals
who know which these file descriptors are but this is a logical improvement.
Tests of a pathaolgical load on the process server of ~420 rapid tasks
executed on a server with BB_NUMBER_THREAD=48 went from a wall clock
measurement of the overall command execution time of 75s to a much more
reasonable 24s.
Scott Rifenbark [Mon, 26 Aug 2013 14:06:00 +0000 (17:06 +0300)]
ref-manual, mega-manual: Edits for expanded packaging description
Fixes [YOCTO #2808]
Applied review comments to the package splitting section in the
section that digs deeper into the build process. This involved
updating the figure that resides in both "figures" directories
of the ref-manual and the mega-manual. Also updated supporting
text throughout the section per Paul's comments.
Stefan Stanacar [Fri, 30 Aug 2013 16:48:53 +0000 (19:48 +0300)]
lib/oeqa/runtime: ping: fix ping false fail
We run the ping test as soon as we reach the login prompt.
But sometimes (seen in sato systemd) we end up with link down/link up stuff like:
qemux86-64 login: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
e1000: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
The logic behind ping -w 30 -c 1 was to wait at most 30 seconds
for at least one reply, but there is a catch: reply doesn't seems
to be echo reply but any reply (non-reply means loss not network error)
ping's man page:
-w deadline
Specify a timeout, in seconds, before ping exits regardless of
how many packets have been sent or received. In this case ping
does not stop after count packet are sent, it waits either for
deadline expire or until count probes are answered or for some
error notification from network.
Just when the link up/link down happens ping returns:
From 192.168.7.1 icmp_seq=1 Destination Host Unreachable
--- 192.168.7.2 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
and exits sooner than the 30 seconds timeout.
This patch should do what was originally intended (wait at most
30 seconds for at least one reply).
Roy.Li [Wed, 28 Aug 2013 02:36:27 +0000 (10:36 +0800)]
btrfs-tools: Add dependency on lzo acl e2fsprogs
Add dependency on lzo acl e2fsprogs to fix the below building failure
1. commit b268a417259b9[add lzo compression support to restore] needs lzo
cmds-restore.c:30:25: fatal error: lzo/lzoconf.h: No such file or directory
2. btrfs-tools includes <sys/acl.h> which is provided by acl
btrfs-convert.c:32:21: fatal error: sys/acl.h: No such file or directory
3. btrfs-tools always needs e2fsprogs
btrfs-convert.c:44:28: fatal error: ext2fs/ext2_fs.h: No such file or directory
Signed-off-by: Jonathan Liu <net147@gmail.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fix the segfault issue of dd on e500v2 targets
Since double-float is disabled for e500v2 targets build due to ICE of gcc-4.8.1,
accordingly %Ld format of sprintf is disabled.
Address Bug 4910 - [p1022ds]urandom: segmentation fault
Signed-off-by: Jonathan Liu <net147@gmail.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Peter A. Bigot <pab@pabigot.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Peter A. Bigot <pab@pabigot.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 30 Aug 2013 16:41:16 +0000 (17:41 +0100)]
bitbake: server/xmlrpc: Increase timeout to 60s
This is a better value that the earlier infinite timeout yet still
allows for servers with high loads. It does mean the bitbake process
can hang at exit for the timeout period but that should never happen
and only happened for me in some test cases which wouldn't happen
in normal use.
Richard Purdie [Thu, 29 Aug 2013 13:27:29 +0000 (14:27 +0100)]
bitbake: cookerdata: Allow bblayers.conf to be found using BBPATH
It should be possible to run a build anywhere on the filesystem and have
bitbake find the correct build directory if its set somehow. The BBPATH
variable makes perfect sense for this usage. Therefore use any available
value of BBPATH to search for conf/bblayers.conf before walking the parent
directory structure.
This restores the option of being able to run bitbake from anywhere if
the user has set things up to operate in that environment.
Richard Purdie [Thu, 29 Aug 2013 13:26:53 +0000 (14:26 +0100)]
bitbake: cookerdata: Set TOPDIR when using bblayers.conf
By definition, bblayers.conf is at the top of the build tree. We'd like
to support running bitbake anywhere within that build tree but TOPDIR
gets set to wherever cwd is. Change the code to reset TOPDIR
to the top of the build directory.
This shouldn't break anything but does make the system more usable.
Chen Qi [Wed, 28 Aug 2013 02:52:03 +0000 (10:52 +0800)]
runqemu-ifdown: clean up the remaining iptables rules
The iptables rules for the tap interface are added by runqemu-ifup
everytime we use runqemu to start a qemu target. But it's not cleaned
up when runqemu exits.
This patch cleans up the remaining iptables rules for the tap interface
in runqemu-ifdown.
Chen Qi [Wed, 28 Aug 2013 02:52:05 +0000 (10:52 +0800)]
runqemu-internal: provide more info if a preconfigured tap is used
We should provide the user more information if a preconfigured tap
is used. This is because the user might have manually set up the tap
interface to be used by other qemu binaries.
So at a minimum, we should let the user know how to make runqemu skip
that tap interface.
Chen Qi [Wed, 28 Aug 2013 02:52:04 +0000 (10:52 +0800)]
runqemu-internal: don't bring down preconfigured tap interface
runqemu-ifup and runqemu-ifdown should be pairs. If we're using a
preconfigured tap interface, the runqemu-ifdown should not be invoked
to bring it down.
Run list-packageconfig-flags.py on wrlinux's platform in which
the oe-core layer and bitbake layer in different directories:
----
../layers/oe-core/scripts/contrib/list-packageconfig-flags.py
Traceback (most recent call last):
File "../layers/oe-core/scripts/contrib/list-packageconfig-flags.py", line 28, in <module>
import bb.cache
ImportError: No module named bb.cache
----
The script import bb module from bitbake lib dir, the previous
lib dir was hardcode and only worked on poky but not for others.
In this situation, look for bitbake/bin dir in PATH could fix this issue.
gtk+3: pull forward the gtk+2 hardcoded libtool patch
Without this, it tries to run ./libtool, not the sys-prefixed libtool, which
fails, resulting in dynamic modules being disabled, which in turn results in
compiling the immodules directly into the gtk+3 library.
I tried switching it to using $LIBTOOL rather than hardcoding the path, as
LT_INIT sets LIBTOOL, but it didn't work, I didn't have time to dig further,
and this gets the job done for now.
Jackie Huang [Fri, 30 Aug 2013 01:56:36 +0000 (09:56 +0800)]
remove the unnecessary protocol parameters
It's not necessary to specify the protocol parameter when it's the
default protocol for the fetcher, e.g. the default protocol for
git fetcher it git, "protocol=git" isn't needed.
Stefan Stanacar [Fri, 30 Aug 2013 09:24:32 +0000 (12:24 +0300)]
lib/oeqa: move skeletoninit.py where it belongs
OE-core commit fcc59cbcdb1550489d372edf9f465efa7165245f /
poky commit 748ddc39e56623f4e48987f0467f4722f6e162f2 added a new test, but
in the wrong location.
I took the patch from Alex's branch but renamed it from meta/lib/oeqa/runtime/skeleton.py to meta/lib/oeqa/skeletoninit.py before sending. This was
unintentional, it should have been under meta/lib/oeqa/runtime.
Zhang Xiao [Tue, 27 Aug 2013 02:07:43 +0000 (10:07 +0800)]
libnfsidmap: Add and modify idmapd.conf
add configuration file to make rpc.idmapd to not report below error:
rpc.idmapd: Skipping configuration file "/etc/idmapd.conf": No such file or directory
rpc.idmapd: Could not find group "nobody"
Paul Eggleton [Wed, 28 Aug 2013 15:39:19 +0000 (16:39 +0100)]
busybox: set default log buffer size to 64 at compile time
This provides a more reasonable log buffer size to avoid losing earlier
events, and 64K is not a problem for modern systems. When the buffer is
used on sysvinit-based systems, which it isn't by default, 64K is already
the runtime default size unless /etc/syslog-startup.conf is modified or
deleted, so this only really affects systems using systemd. This
completely removes the need for the busybox bbappend in meta-oe.
Richard Purdie [Wed, 28 Aug 2013 16:10:09 +0000 (16:10 +0000)]
bitbake: prserv/serv: Multithread the server
This makes the PR server multithreaded and able to handle multiple connections
at once which means its no longer a build bottle neck when serving one connection
at a time. I've experimented and database connection for each thread seems to
cause the least issues, pushing the contention for sqllite to handle itself.
This means moving the db/table connection code into the actual function methods.
It doesn't abstract well as a function since we need the db object around for
the lifetime of the function as well as the table else we lose the connection.
Richard Purdie [Wed, 28 Aug 2013 16:06:10 +0000 (16:06 +0000)]
bitbake: prserv/db: Threading fixes
Enabling threading for the PRServer causes a number of issues. Firstly is
the obtuse error:
sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type
which is due to the class not being derived from object. See:
http://docs.python.org/2/library/sqlite3.html#registering-an-adapter-callable
Secondly, we want to enable multithreadded access to the database so we do this
when we open it. This opens the way up to multithreading the PR server.
Stefan Stanacar [Sun, 25 Aug 2013 00:46:55 +0000 (01:46 +0100)]
classes/testimage: add support for finding tests in other layers
A layer can add tests in lib/oeqa/runtime (provided it extends BBPATH as
normal) and enable them with TEST_SUITES_append = " testname". Test
module names shouldn't collide though.
Stefan Stanacar [Wed, 28 Aug 2013 10:52:09 +0000 (13:52 +0300)]
classes/testimage: increase default boot timeout
While I can't reproduce on local builds, sometimes images fail
to boot on AB (which runs many builds at once). Assuming
there isn't something weird going on, let's just give it more time.
Stefan Stanacar [Wed, 28 Aug 2013 09:56:45 +0000 (12:56 +0300)]
lib/oeqa/runtime: remove some unnecessary checks from setUpModule
These checks are unnecessary.
setUpModule is run when a module is loaded and we
shouldn't run commands on the target here, (plus if
ssh doesn't work we error out in setup multiple times, instead
of skipping the real test, which might depend on test_ssh).
Signed-off-by: Tom Zanussi <tom.zanussi@intel.com> Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jason Wessel [Wed, 28 Aug 2013 02:20:55 +0000 (21:20 -0500)]
bitbake: serv.py: Fix regression from 972bc43e6d5b
commit 972bc43e6d5b1207b944b3baa8f9805adb35dda7 (serv.py: Fix hang
when spawned dynamically with bitbake) introduced a regression,
because the wrong patch was submitted. The syntax was incorrect in
the original patch. The logger iterator must be used with a call to
getLogger().
Mihai Prica [Tue, 27 Aug 2013 14:12:05 +0000 (14:12 +0000)]
midori: add version 0.5.5 from meta-gnome
Midori is a lightweight web browser that uses the WebKit
rendering engine and the GTK interface. This allows a more
easy testing of the functionality of webkit-gtk component
that is already in oe-core.
Mihai Prica [Tue, 27 Aug 2013 14:14:13 +0000 (14:14 +0000)]
python-docutils: add version 0.11 from meta-oe
Docutils is a text processing system for processing plaintext
documentation into useful formats(HTML, XML, man-pages). This is
required by the Midori web-browser.
Joe Slater [Mon, 26 Aug 2013 23:38:45 +0000 (23:38 +0000)]
dpkg: fix configuration issue for mips64
CQID: 431771
configure cannot determine the proper cpu, os, or
architecture for mips64, and possibly other arch's,
because of faulty code added to Arch.pm in the latest
release from upstream. We remove that code.
Khem Raj [Thu, 22 Aug 2013 02:16:30 +0000 (02:16 +0000)]
systemd: Upgrade to 206
Add new PACKAGE systemd-rpm-macros, this will hold
the macros which are interesting when rpm is used as
package management backend
Forward port uclibc only patches. Add a new patch
to stub out use of preadv/pwritev in testcases
Delete patches that have been merged upstream in systemd
Remove force export of GPERF variable in environment
this was causing AC_CHECK_TOOL to not populate GPERF
variable as expected
systemd needs kmod to be present on rootfs so add it
to RDEPENDS
some services substitute discovered kmod when the service
file is generated during boot, however the discovered kmod
is from native sysroot and it gets into the service file
with absolute path. So specify the target path of kmod using
KMOD variable so the unit files have correct pointer to kmod
on target
Add a patch to make sure that mknod capability is checked
before the service which excercise mknod, this patch is also
submitted to upstream systemd
bitbake: data_smart: use a split/filter/rejoin for _remove
This is more idiomatic, and from the limited performance testing I did, is
faster as well. See https://gist.github.com/kergoth/6360248 for the naive
benchmark.
Signed-off-by: Darren Hart <dvhart@linux.intel.com> Cc: Ross Burton <ross.burton@intel.com> Cc: Tom Zanussi <tom.zanussi@intel.com> Cc: Bruce Ashfield <bruce.ashfield@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jason Wessel [Tue, 27 Aug 2013 20:12:55 +0000 (15:12 -0500)]
bitbake: serv.py: Fix hang when spawned dynamically with bitbake
The PRServer has the possibility to hang indefinitely blocking on a
semaphore processing a xmlrpc request to send an event back to the
main bitbake instance. This was observed during a "bitbake -e" on a
heavily loaded machine and the main bitbake instance and cooker exited
before the PRServer emitted its first log.
The stack trace is provided below as to show what happens every time a
logger.info() is executed in the PRServer. Not only does it write to
the stream handler but it also tries to send the event to the main
event processor.
It was never intended for the PRServer to send its logs anywhere but
its own log file. The event processing is an artifact of how the
PRServer was forked and it inherits the event log handlers. The
simple fix is to clean up and purge all the log handlers after the
fork() but before doing any of the typical PRServer work or logging.
Signed-off-by: Darren Hart <dvhart@linux.intel.com> Cc: Ross Burton <ross.burton@intel.com> Cc: Scott Rifenbark <scott.m.rifenbark@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
layer.conf: Bumping LAYERVERSION_yoctobsp to account for atom-pc
In order to maintain autobuilder compatibility with prior releases
we'll need to decide on if we should build atom-pc or genericx86
based on LAYERVERSION_yoctobsp.
if LAYERVERSION_yoctobsp < 2 we build out atom-pc else
we build out genericx86
Signed-off-by: Elizabeth Flanagan <elizabeth.flanagan@intel.com> Cc: Ross Burton <ross.burton@intel.com> Signed-off-by: Darren Hart <dvhart@linux.intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Darren Hart [Mon, 26 Aug 2013 21:15:18 +0000 (21:15 +0000)]
meta-yocto-bsp: Purge atom-pc in favor of genericx86
genericx86 is a superset of atom-pc, so remove all references to
atom-pc.
Note that genericx86 only has 3.10 linux-yocto support, that leaves us
with no x86 BSP in meta-yocto-bsp for the 3.4 kernel. As a general rule,
new BSPs are targeted at the current kernel version, so this is normal,
but something worth noting.
Darren Hart [Mon, 26 Aug 2013 20:50:34 +0000 (20:50 +0000)]
meta-yocto-bsp/genericx86: Update kernel, branch, KMACHINE, and SRCREV
For the genericx86 machine:
o Update the PREFERRED_VERSION for linux-yocto to 3.10
o Change the KBRANCH to common-pc/base
o Change the SRCREV to the HEAD
o Change the KMACHINE to a valid linux-yocto BSP name
Bruce Ashfield [Mon, 26 Aug 2013 18:08:35 +0000 (18:08 +0000)]
meta-yocto: change default kernel to 3.10
With 3.10 being the next LTSI kernel, and all qemu reference platforms
tested on that new baseline, the time has come to bump the default
version for qemu* and other boards to 3.10.
Reference platforms continue to explicitly set their own preferred versions
for the kernel as required.
Bruce Ashfield [Mon, 26 Aug 2013 21:47:20 +0000 (21:47 +0000)]
linux-yocto: add bc-native dependency, and move to linux-yocto.inc
As reported by Martin Jansa <martin.jansa@gmail.com>, the following error happens
when building in a minimal environment:
| BC kernel/timeconst.h
| /bin/sh: bc: command not found
| make[3]: *** [kernel/timeconst.h] Error 127
| make[2]: *** [kernel] Error 2
| make[2]: *** Waiting for unfinished jobs....
kernel commit 70730bca [kernel: Replace timeconst.pl with a bc script] added
a kernel dependency on bc. To support the build of linux-yocto recipes in
these configurations, we add bc-native to the common dependencies.
Bruce Ashfield [Mon, 26 Aug 2013 17:28:29 +0000 (17:28 +0000)]
kern-tools: fix patch series to git tree validation
Previous changes to the kern-tools improved functionality to ensure that
as a series is considered, it is checked against the tree to confirm that
all patches are really applied.
There was a bug in the subject based detection, such that the first matching
patch was take, and not the last. This change ensures that we start from
the end of a series, not the start.
and returns a 0 exit code (it thinks it succesfully removed the package,
when in reality it didn't), so we need to catch those specifically.
Also, sometimes output from download command is:
Saving cache...http://192.168.7.1:49456/rpm/x86_64_x32/psplash-default-0.1+git0+afd4e228c6-r15.x86_64_x32.rpm
and that tricks our smart download test, so use a regex there.
projects / thirdparty / openembedded / openembedded-core-contrib.git / log
