selftest: incompatible_lic.py: run bitbake -c testimage separately for each image

* with 1 preconfigured tap device you cannot run testimage on multiple images
  at the same time

  in my case have /tmp/qemu-tap-locks/tap0.skip (tap0 is used by openvpn on builder)
  and /etc/runqemu-nosudo as my build user doesn't have sudo powers

  but then when 2 do_testimage tasks are executed at the same time it fails like this:

    NOTE: recipe core-image-weston-1.0-r0: task do_testimage: Started
    NOTE: recipe core-image-full-cmdline-1.0-r0: task do_testimage: Started
    QMP Available for connection at /OE/build/poky/build-st/tmp/.c2d3k3bh
    QMP Available for connection at /OE/build/poky/build-st/tmp/._4agpi1v
    ERROR: core-image-full-cmdline-1.0-r0 do_testimage: runqemu exited with code 1
    ERROR: core-image-full-cmdline-1.0-r0 do_testimage: Output from runqemu:
    runqemu - INFO - Continuing with the following parameters:
    KERNEL: [/OE/build/poky/build-st/tmp/deploy/images/qemux86-64/bzImage]
    MACHINE: [qemux86-64]
    FSTYPE: [ext4]
    ROOTFS: [/OE/build/poky/build-st/tmp/deploy/images/qemux86-64/core-image-full-cmdline-qemux86-64.ext4]
    CONFFILE: [/OE/build/poky/build-st/tmp/deploy/images/qemux86-64/core-image-full-cmdline-qemux86-64.qemuboot.conf]

    runqemu - INFO - Found /tmp/qemu-tap-locks/tap0.skip, skipping tap0
    runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap1.lock failed: [Errno 11] Resource temporarily unavailable
    runqemu - ERROR - Error: There are no available tap devices to use for networking,
    runqemu - ERROR - and I see /etc/runqemu-nosudo exists, so I am not going to try creating
    runqemu - ERROR - a new one with sudo.
    runqemu - INFO - Cleaning up
    runqemu - INFO - Host uptime: 67474.59

    tput: No value for $TERM and no -T specified

    WARNING: core-image-full-cmdline-1.0-r0 do_testimage: Qemu ended unexpectedly, dump data from host is in /OE/build/poky/build-st/tmp/log/runtime-hostdump/202303131614_qemu
    WARNING: core-image-full-cmdline-1.0-r0 do_testimage: runqemu during is_alive() test
    ERROR: core-image-full-cmdline-1.0-r0 do_testimage: Error executing a python function in exec_func_python() autogenerated:

  which is very reasonable error message, but 600 lines later it also shows:

    RESULTS - xorg.XorgTest.test_xorg_running: SKIPPED (0.00s)
    SUMMARY:
    core-image-weston () - Ran 71 tests in 34.073s
    core-image-weston - OK - All required tests passed (successes=14, skipped=57, failures=0, errors=0)
    NOTE: recipe core-image-weston-1.0-r0: task do_testimage: Succeeded
    NOTE: Tasks Summary: Attempted 1141 tasks of which 1139 didn't need to be rerun and 1 failed.

    Summary: 1 task failed:
      /OE/build/poky/meta/recipes-extended/images/core-image-full-cmdline.bb:do_testimag

  and I didn't notice that there are actually 2 different images core-image-weston
  and core-image-full-cmdline and spent most time trying to figure out how it was
  able to run all those ptest when qemu failed to start at the beginning.

  It's possible to create 2nd tap with runqemu-ifup before running this test.
  Then it works, maybe some check like in DevtoolTests to error out early if
  there isn't enough tap devices? But the implementation in:
  https://git.openembedded.org/openembedded-core/commit/?id=2258345e19efff7717fe19a5026ec55f1b6f90b6
  has its own limitations (e.g. not checking /tmp/qemu-tap-locks/tap0.skip)
  and would probably need to be re-implemented as some generic decorator
  as other tests need this as well.

  It's a bit slower when serialized:
  INFO - RESULTS - incompatible_lic.NoGPL3InImagesTests.test_core_image_full_cmdline_weston: PASSED (145.96s)
  without this change:
  INFO - RESULTS - incompatible_lic.NoGPL3InImagesTests.test_core_image_full_cmdline_weston: PASSED (104.72s)

  I don't mind if this is rejected, sending it mostly to document this behavior.

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>

diff --git a/meta/lib/oeqa/selftest/cases/incompatible_lic.py b/meta/lib/oeqa/selftest/cases/incompatible_lic.py
index 1597d30ab03c1a99d20ce3198ac05762a23b1190..a508534e339f1819f0d8be07bd33acec804f9536 100644 (file)
--- a/meta/lib/oeqa/selftest/cases/incompatible_lic.py
+++ b/meta/lib/oeqa/selftest/cases/incompatible_lic.py
@@ -160,5 +160,6 @@ CORE_IMAGE_EXTRA_INSTALL:pn-core-image-weston += "rpm"
 CORE_IMAGE_BASE_INSTALL:remove:pn-core-image-weston = "matchbox-terminal"
 """)
         bitbake('core-image-full-cmdline core-image-weston')
-        bitbake('-c testimage core-image-full-cmdline core-image-weston')
+        bitbake('-c testimage core-image-full-cmdline')
+        bitbake('-c testimage core-image-weston')