python-gitdb: use pypi.bbclass

* Simplify python3-gitdb recipe using pypi.bblass
* Avoid duplication by using PYTHON_PN in .inc
* Sort RDEPENDS alphabetically

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>

python-mako: use pypi.bbclass

* Simplify python3-mako by using pypi.bbclass
* Avoid duplication by using PYTHON_PN in .inc

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>

python-async: use pypi.bbclass

* Simplify by using pypi.bbclass
* Use PYTHON_PN for RDEPENDS in .inc to avoid duplication

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>

python-six: use pypi.bbclass

* Simplify python3-six by using pypi.bbclass
  - Use PYTHON_PN in .inc to avoid duplication in RDEPENDS

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>

python-nose: use pypi.bbclass

* Simplify python- and python3-nose by using pypi.bbclass
* Consolidate common content in python-nose.inc
  - Avoid duplication by using PYTHON_PN variable
* nosetests RDEPENDS on python-logging

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>

python3-pip: use pypi.bbclass

* Simplify recipe using pypi.bbclass

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>

python*-setuptools: upgrade to 37.0.0; use pypi.bbclass; improvements

* Simplify python- and python3-setuptools with pypi.bbclass
* inherit setuptools rather than distutils
* Consolidate common settings in python-setuptools.inc
  - use PYTHON_PN variable to eliminate duplication
  - python3-setuptools had RDEPENDS on _markupbase
    - prior commit it added to -core rather than -misc catchall
  - installer no longer creates setuptools.pth, drop fixes
  - easy_install script is installed in /usr/bin/, add --script-dir

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>

python*-manifest.json: add dependencies, runpy

* runpy allows running modules/scripts with 'python -m foo'
  - create explicit python2 and 3 packages rather than the
    misc catchall
* python3-setuptools and html.parser RDEPENDS on _markupbase
  - add to python3-core rather than misc catchall
* pip3 RDEPENDS on plistlib, http.client
  - already packaged in python2, add to python3
  - add http/ to -netclient
* "pip3 install" RDEPENDS on encodingds.idna
  - encodings.idna packaged in -core, but missing:
    - stringprep (move from -codecs to -core)
    - unicodedata (move from -codecs to -core)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>

pypi.bbclass: bring in from meta-python

The pypi.bbclass has usefullness in many meta layers, not
just meta-python. Add it to oe-core for the benefit of
everyone.

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>

python-scons: Remove python-importlib dependency

With the new manifest python-importlib became part of python-core
(it was being installed anyway), so now the package cant be found
and it shouldnt be listed as a dependency, this patch fixes
python-scons RDEPENDS to avoid dependency errors.

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>

hwlatdetect: fix RDEPENDS to avoid QA failures

When listing python3 as an RDEPENDS, the image will in fact
contain python3-modules, but the QA check fails to get
RPROVIDERS correctly (since the python3 package does not
actually exist), /usr/bin/python3 is provided by the
python3-core package, so by fixing the RDEPENDS and listing
python3-core instead of python3, the QA check works correctly
and we avoid failures.

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>

python3: fix RDEPENDS on several recipes, due to non-existent python3 packages

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>

python3: Restructure python3 packaging and replace it with autopackaging

See previous commit (python2 version) for more info, since mostly
everything applies here as well.

Old manifest file had several issues:
 - Its unorganized and hard to read and understand it for an average
   human being.
 - When a new package needs to be added, the user actually has to modify
   the script that creates the manifest, then call the script to create
   a new manifest, and then submit a patch for both the script and the
   manifest, so its a little convoluted.
 - Git complains every single time a patch is submitted to the manifest,
   since it violates some of its guidelines.
 - It changes or may change with every release of python, its impossible
   to know if the required files for a certain package have changed
   (it could have more or less dependencies), the only way of doing so
   would be to install and test them all one by one on separate individual
   images, and even then we wouldnt know if they require less dependencies,
   we would just know if an extra dependency is required since it would
   complain, lets face it, this isnt feasible.
 - The same thing happens for new packages, if someone wants to add a new
   package, its dependencies need to be checked manually one by one.

Features/Fixes:
 - A new manifest format is used (JSON), easy to read and understand.
   This file is parsed by the python recipe and python packages
   read from here are passed directly to bitbake during parsing time.
 - It provides an automatic manifest creation task (explained on previous
   commit), which automagically checks for every package dependencies and
   adds them to the new manifest, hence we will have on each package
   exactly what that package needs to be run, providing finer granularity.
  - Dependencies are also checked automagically for new packages
   (explained on previous commit).

This patch has the same features as the python2 version but it differs
in the following ways:
 - Python3 handles precompiled bytecode files  (*.pyc) differently.
   for this reason and since we are cross compiling, wildcards couldnt be
   avoided on python3 (See PEP #3147 [1]).
   Both the manifest and the manifest creation script handle this
   differently, the manifest for python3 has an extra field for cached
   files, which is how it lets the user install the cached files or not
   via : INCLUDE_PYCS = "1" on their local.conf.
 - Shared libraries nomenclature also changed on python3, so again, we
   use wildcards to deal with this issue ( See PEP #3149 [2]):
 - Fixes python3 manifest, python3-core should be base and everything
   should depend on it, hence several packages were deleted:
   python3-enum, re, gdbm, subprocess, signal, readline.
 - When building python3-native it adds as symlink to it called
   nativepython3, which is then isued by the create_manifest task.
 - Fixes [YOCTO #11513] while were at it.

References:
[1] https://www.python.org/dev/peps/pep-3147/
[2] https://www.python.org/dev/peps/pep-3149/

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>

python: Restructure python packaging and replace it with autopackaging

The reason we have a manifest file for python is that our goal is to
keep python-core as small as posible and add other python packages only
when the user needs them, hence why we split upstream python into several
packages.

Although our manifest file has several issues:
 - Its unorganized and hard to read and understand it for an average
   human being.
 - When a new package needs to be added, the user actually has to modify
   the script that creates the manifest, then call the script to create
   a new manifest, and then submit a patch for both the script and the
   manifest, so its a little convoluted.
 - Git complains every single time a patch is submitted to the manifest,
   since it violates some of its guidelines.
 - It changes or may change with every release of python, its impossible
   to know if the required files for a certain package have changed
   (it could have more or less dependencies), the only way of doing so
   would be to install and test them all one by one on separate individual
   images, and even then we wouldnt know if they require less dependencies,
   we would just know if an extra dependency is required since it would
   complain, lets face it, this isnt feasible.
 - The same thing happens for new packages, if someone wants to add a
   new package, its dependencies need to be checked manually one by one.

This patch fixes those issues, while adding some additional features.

Features/Fixes:

 - A new manifest format is used (JSON), easy to read and understand.
   This file is parsed by the python recipe and python packages read
   from here are passed directly to bitbake during parsing time.
 - It provides an automatic manifest creation task (explained below),
   which automagically checks for every package dependencies and adds
   them to the new manifest, hence we will have on each package exactly
   what that package needs to be run, providing finer granularity.
 - Dependencies are also checked automagically for new packages (explained below).
 - Fixes the manifest in the following ways:
   * python-core should be base and all packages should depend on it,
     fixes lang, string, codecs, etc.
   * Fixes packages with repeated files (e.g. bssdb and db, or
     netclient and mime, and many others).
 - Sitecustomize was fixed since encoding was deprecated.
 - The JSON manifest file invalidates bitbake's cache, so if it changes
   the python package will be rebuilt.
 - It creates a solution for users that want precompiled bytecode files
   (*.pyc) INCLUDE_PYCS = "1" can be set by the user on their local.conf to
   include such files, some argument they get faster boot time, even when the
   files would be created on their first run?, but they also sometimes give a
   magic number error and take up space, so we leave it to the user to
   decide if they want them or not.
 - Fixes python-core dependencies, e.g.
   When python is run on an image, it TRIES to import everything it needs,
   but it doesnt necessarily fails when it doesnt find something, so even if
   we didnt know, we had errors like (trimmed on purpose):
   # trying /usr/lib/python2.7/_locale.so
   # trying /usr/lib/python2.7/lib-dynload/_locale.so
   # trying /usr/lib/python2.7/_sysconfigdata.so

   while it didnt complain about _locale it should have imported it,
   after creating a new manifest with the automated script we get:

   # trying /usr/lib/python2.7/lib-dynload/_locale.so
   dlopen("/usr/lib/python2.7/lib-dynload/_locale.so", 2);
   import _locale # dynamically loaded from /usr/lib/python2.7/lib-dynload/_locale.so

How to use (after a new release of python, or maybe before every OE
release):
 - A new task called create_manifest was added to the python package,
   which may be invoked via:

 $ bitbake python -c create_manifest

 This task runs a script on native python on our HOST system, and since
 the python and python-native packages come from the same source, we can
 use it to know the dependencies of each module as if we were doing it
 on an image, this script is called create_manifest.py and in a very
 simplistic way it does the following:

 1. Reads the JSON manifest file and creates a dictionary data structure
    with all of our python packages, their FILES, RDEPENDS and SUMMARY.
 2. Loops through all of them and runs every module listed on them
    asynchronously, determining every dependency that they have.
 3. These module dependencies are then handled, to be able to know which
    packages contain those files and which should RDEPEND on one another.
 4. The data structure that comes out of this, is then used to create a
    new manifest file which is automatically copied onto the user's python
    directory replacing the old one.

 Create_manifest script features:
 - Handles modules which dont exist anymore (new release for example).
 - Handles modules that are builtin.
 - Deals with modules which were not compiled (e.g. bsddb or ossaudiodev)
 - Deals with packages which include folders.
 - Deals with packages which include FILES with a wildcard.
 - The manifest can be constructed on a multilib environment as well.
 - This method works for both python modules and shared libraries used
   by python.

How to add a new package:
 - If a user wants to add a new package all that has to be done is
   modify the python2-manifest.json file, and add the required file(s)
   to the FILES list, the script should handle all the rest.
   Real example:
   We want to add a web browser package, including the file webbrowser.py
   which at the moment is on python-misc.
   "webbrowser": {
       "files": ["${libdir}/python2.7/lib-dynload/webbrowser.py"],
       "rdepends": [],
       "summary": "Python Web Browser support"}

 Run bitbake python -c create_manifest and the resulting manifest
 should  be completed after a few seconds, showing something like:
   "webbrowser": {
      "files": ["${libdir}/python2.7/webbrowser.py"],
      "rdepends": ["core","fcntl","io","pickle","shell","subprocess"],
      "summary": "Python Web Browser support"}

Known errors/issues:
 - Some special packages are handled differently: core, misc,
   modules,dev, staticdev.
   All these should be handled manually, because they either include
   binaries, static libraries, include files, etc. (something that we
   cant import).
   Specifically static libraries are not not supported by this method
   and have to be handled by the user.
 - The change should be transparent to the user, other than the fact
   that now we CANT build python-foo (it was pretty dumb anyway, since
   what building python-foo actually did was building the whole python
   package anyway), but doing IMAGE_INSTALL_append = " python-foo"
   would create an image with the requested package with no issues.

[YOCTO #11510] [YOCTO #11694] [YOCTO #11695]

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>

runqemu: Also specialcase resolution of '.' to the file's location

Similarly to handling "../", handle "." to resovle to the qemuconf
file's current directory.

(From OE-Core rev: 33418ed064fe9cff5b4803f09135a81d9170c189)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

image_types: Add debugging code to ext4 fs creation

We have seen a small number of issues with ROOTFS_SIZE not getting
computed correctly, resulting in a failure in the mke2fs processing
and populating the resulting new filesystem.

This information should help us to reproduce [YOCTO #12304]

(From OE-Core rev: 3a72f6783e142d53d19b37811a854d08d32485ab)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

qemurunner: Simplify binary data handling

I have concerns that bad timing of the flow of data from the logger
might corrupt the output due to the way binary strings are handled
in qemurunner.

This simplifies the code to do the same thing it did before but much
more safely.

(From OE-Core rev: 1e87283e92a2765bb5d54d17138b208bc395953b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

qemuboot: Improve relative path handling

qemuconf files are currently written relative to TOPDIR. What
makes more sense is to write paths relative to the location of the
file. This makes moving them around and decoding the end paths in
runqemu much easier.

The effect of this should allow less use of bitbake to determine
variables and allow us to simplify runqemu.

(From OE-Core rev: 55a0028a961c0ad3c2e5729a9e3919cbbf256fe1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

runqemu: Improve relative path handling in qemuconf files

If a variable starts with "../", its likely its a path and we want to
set it to an absolute path relative to the qemuconf file.

This means we don't have to use bitbake as often to figure out variables.

(From OE-Core rev: dfc7940900d798aa47716288338107e1d46a3972)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

classes/cross: Add addto_recipe_sysroot task to cross recipes

This is particularly useful if you want to use gdb-cross as there
is no other good way to access it now with RSS.

(From OE-Core rev: e8648a022c93175d84baf3852d4d364b19d8d795)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

qemu: Add patch to avoid qemuppc boot hangs

qemuppc boots are occasionally hanging on the autobuilder. This adds a
patch which fixes the issue in local testing. Its being discussed with
upstream qemu.

(From OE-Core rev: 87719e35db08b21cd43ab3ebd72f4567ca0fdc65)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

runqemu: Ensure we process all tap devices

The regexp in the script misses some tap devices, e.g. we see output like:

runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap25.lock failed: [Errno 11] Resource temporarily unavailable
runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap26.lock failed: [Errno 11] Resource temporarily unavailable
runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap27.lock failed: [Errno 11] Resource temporarily unavailable
runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap28.lock failed: [Errno 11] Resource temporarily unavailable
runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap40.lock failed: [Errno 11] Resource temporarily unavailable
runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap41.lock failed: [Errno 11] Resource temporarily unavailable

What happened to tap29 to tap39?

The issue is was we were missing devices with '0' in the number,
like "10:" and so on in the output from "ip link".

(From OE-Core rev: 6447697a48e3b693ee38806bc2ba07c2a65c2bc8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: fetch/wget: use with to ensure the response is closed

(Bitbake rev: 8c487176d311557031cedba76185f14f0e7a14cd)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: fetch/wget: improve proxy object

The connection cache class uses a dummy file object but it doesn't have a closed
attribute, so we can't use it in a context manager.

(Bitbake rev: 7b072ef91d16331eae11bd60f229ce1f0c175995)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: cooker: fix typo in bitbake -g message

Flatened -> flattened.

(Bitbake rev: 80f72ac6c56ba8f3e2f7b5f0cb95bee6c0101323)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: tinfoil: ensure get_recipe_info() returns Null if recipe not found

If a matching recipe is not found then return Null instead of raising
KeyError because we were blindly using None as a key for pkg_fn.

(Bitbake rev: 431e89e322850a2497157c3c0843da9df6bc9a3e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: bitbake-layers: show-recipes: fix help to mention -i supports multiple classes

The -i option supports more than one class, but the help didn't mention
that.

(Bitbake rev: 1060955c4aa2ef66cdb4f0549f9bd8c1c332673c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: bitbake-layers: remove-layer: support removing multiple layers at a time

If you can add multiple layers at once, it stands to reason that you
should also be able to remove more than one at a time.

(Bitbake rev: 2f2033836a5ce4064d9e4f263788a563001bc008)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: bitbake-layers: add-layer: enable adding multiple layers at once

Allow specifying multiple layers with bitbake-layers add-layer so that
you can add more than one in a single command. This is not just useful,
it's actually pretty important if you need to add a layer and its
dependencies at the same time - since we now go through a parse process
when the layer is added, without this you have to add them all in just
the right order and wait for the parse each time which is somewhat
painful.

(Bitbake rev: ad6b14f01aa326a1c6baa31bfac33be238bce805)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: runqueue: Fix typo builable -> buildable

(Bitbake rev: 1e59ae8729513e19a801c723b67911491c2a66fe)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

base: add automatic dependency on xz-native for .txz SRC_URI

.txz is .tar.xz, so add it, as this can actually be found in the
wild.

(From OE-Core rev: 58af8c2e4bd17692274fc5a6ac8f8af84319fec6)

Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

go: Use right dynamic linker on musl

(From OE-Core rev: 0d6e83757fc26d3e88bfe3c2437b5c7c9be09118)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

net-tools: correctly set COPTS and LOPTS

COPTS will be ignored if it is defined in the environment.
It must be passed directly to make.  To be consistent, we
pass LOPTS that way, too.

(From OE-Core rev: dede6d3d37aab72ae897c3709d21108fa75f6673)

Signed-off-by: Joe Slater <jslater@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

strace: 4.18 -> 4.19

Rebase Makefile-ptest.patch

(From OE-Core rev: 481bceb3f23f55a537ad4bde8bf6226f60fdeb46)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

libsdl2: 2.0.5 -> 2.0.7

* Drop linkage.patch, this version already links .lo object.
* Remove backported patch fix-build-failure-on-ppc.patch
* Update LIC_FILES_CHKSUM since the year is changed.

(From OE-Core rev: 2702506f48a1b7e72e1786d24bd0113396dbc85d)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

less: 487 -> 527

(From OE-Core rev: 14620d2882cfb59aac260136bb43568108ab8cf0)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

logrotate: 3.12.3 -> 3.13.0

(From OE-Core rev: bd6bd5f8773459a3671e39925eb5891c0ae74367)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

liberror-perl: 0.17024 -> 0.17025

(From OE-Core rev: 5d148d34642793b7cf0c795321b6779b2f6b0a33)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

git: 2.13.3 -> 2.15.0

(From OE-Core rev: d138c3c40206cc4f5d1ff06663ce9c3ea8846563)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

file: 5.31 -> 5.32

Rebase debian-742262.patch for new source.

(From OE-Core rev: 4231e238d9fe6b2178b900cde49ca17ce5aa1f47)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

e2fsprogs: 1.43.5 -> 1.43.7

Remove reproducible-doc.patch which is already in the source.

(From OE-Core rev: c76d6f22e72399cab4d2652a7431a390907e1ba2)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

autoconf-archive: inherit allarch and fix package splitting

autoconf-archive is a set of common m4 macros, it should be allarch,
just like it is on other distros.

Also fix package splitting. With current package splitting,
autoconf-archive itself contains almost nothing, the m4 macros are
contained in its dev package. However, autoconf-archive should
contain those m4 marcos itself.

(From OE-Core rev: 8cbf5523f002e8ae0d8123755d2cbc3e8b0138ac)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

iproute2: upgrade to 4.13

- 0001-include-stdint.h-explicitly-for-UINT16_MAX.patch is already
  applied to upstream.
- Rebase two patches.

(From OE-Core rev: 9839de8861c5fb2067664542045c0728653bbcb4)

Signed-off-by: Changhyeok Bae <changhyeok.bae@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

mesa: Upgrade to 17.2.5

This upgrades the recipe to 17.2.5 bugfix release.

The changes can be seen at:

https://www.mesa3d.org/relnotes/17.2.5.html

(From OE-Core rev: 638b4ac5279225b89edcd411e39aa2c22617dcd5)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

recipetool: create: fix failure handling included dicts

If a setup dict in a python setup.py file pulled in the contents of
another dict (e.g.  **otherdict), then we got an error when mapping
the keys because the key is None in that case. Skip those keys to avoid
the error (we pick up the values directly in any case).

A quick reproducer for this issue:

recipetool create https://files.pythonhosted.org/packages/source/p/pyqtgraph/pyqtgraph-0.10.0.tar.gz

(From OE-Core rev: ae62a9953e219df5147ed4a5ae3f4163d51cff28)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

linux-firmware: Remove iwlwifi-8000C-19 SRC_URI

Since it's been removed from the upstream repo and not fetchable
remove it here.  The newer firmware supports the device correctly.

(From OE-Core rev: 8b8c40bdbd09ddd1409dc30e04ef847f6a15f109)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

site/microblaze-linux: Add MicroBlaze linux site info

Add linux site info for the MicroBlaze architecture. This is based on
the site info from meta-xilinx and additional options based on config
options that other *-linux architecture site info files define.

(From OE-Core rev: def626a46efdeeaf867a27464aed463d265d777c)

Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

siteinfo.bbclass: Add microblaze(eb/el) linux site info targets

Add mappings for the microblaze(eb/el) linux and linux-musl variants to
use architecture common linux site data.

(From OE-Core rev: 634d1ed2cd89b8ac28b918be3561b135f7418b25)

Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

insane.bbclass: add microblaze-*-elf to machine dictionary

Add the definitions for microblaze-*-elf targets to the machine
dictionary.

(From OE-Core rev: a0104e47217e3a46d319a76929aec5b45c3ea6f9)

Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

icu: CVE-2017-14952

Double free in i18n/zonemeta.cpp in International Components for Unicode
(ICU) for C/C++ through 59.1 allows remote attackers to execute arbitrary
code via a crafted string, aka a "redundant UVector entry clean up
function call" issue.

Reference:
https://nvd.nist.gov/vuln/detail/CVE-2017-14952

Upstream patches:
http://bugs.icu-project.org/trac/changeset/40324/trunk/icu4c/source/i18n/zonemeta.cpp

(From OE-Core rev: 4ff12a8bf2b8d094085afbe8fa1d43f781cfa79d)

Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

qemu.inc: Drop rng-tools

This was included to allow hwrng to feed data to /dev/random. Since the
kernel does this itself (confirmed by the recent kernel Kconfig text change
, code inspection and local testing), we can drop rng-tools from these
images.

(From OE-Core rev: d4b17e841b497f0ee4d31a8c967b5ce1b76157e4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

lttng-modules: Upgrade to 2.9.5 release

This upgrade is critical as it fixes the support for Linux 4.14 LTS kernel.

The changlog is:

2017-10-05 LTTng modules 2.9.5
* Fix: update block instrumentation for 4.14 kernel
* Revert "Fix: update block instrumentation for kernel 4.14"

2017-10-03 (National Techies Day) LTTng modules 2.9.4
* Fix: version check error in btrfs instrumentation
* Fix: update btrfs instrumentation for kernel 4.14
* Fix: update writeback instrumentation for kernel 4.14
* Fix: update block instrumentation for kernel 4.14
* Fix: vmalloc wrapper on kernel < 2.6.38
* Fix: vmalloc wrapper on kernel >= 4.12
* Add kmalloc failover to vmalloc
* Fix: mmap: caches aliased on virtual addresses
* Fix: update ext4 instrumentation for kernel 4.13
* Fix: Sleeping function called from invalid context
* Fix: sched for v4.11.5-rt1
* Fix: handle missing ftrace header on v4.12

This also removes the previously backported patches as they are part
of 2.9.4 release and the missing fix is part of 2.9.5 release.

(From OE-Core rev: 3ca3f8b2dbf0fec6a188ee2c10238704e4642c93)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

gawk: upgrade to 4.2.0

test-arrayind1-Remove-hashbang-line.patch is dropped as it has
been merged in new version.

(From OE-Core rev: c84c8666581bfd64b21498330925146d8fe81075)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

cups: upgrade to 2.2.6

(From OE-Core rev: 3b6d600ac8e91fd489780ca87a21bc56f2aae60f)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

sysstat: upgrade to 11.6.1

(From OE-Core rev: abe2c55b719bdff40d9d423a84bfde08d9e09631)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

sudo: upgrade to 1.8.21p2

(From OE-Core rev: a3eccc5b0f0e95c18b3d40cc82d576fbe45b4547)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

kexec-tools: 2.0.14 -> 2.0.15

1) Upgrade kexec-tools from 2.0.14 to 2.0.15.
2) Remove patches that are included in 2.0.15.
   kexec-tools/0001-arm64-Disable-PIC.patch
   kexec-tools/0001-kexec-exntend-the-semantics-of-kexec_iomem_for_each_.patch
   kexec-tools/0001-vmcore-dmesg-Define-_GNU_SOURCE.patch
   kexec-tools/0001-x86-x86_64-Fix-format-warning-with-die.patch
   kexec-tools/0002-kexec-generalize-and-rename-get_kernel_stext_sym.patch
   kexec-tools/0002-ppc-Fix-format-warning-with-die.patch
   kexec-tools/0003-arm64-identify-PHYS_OFFSET-correctly.patch
   kexec-tools/0004-arm64-kdump-identify-memory-regions.patch
   kexec-tools/0005-arm64-kdump-add-elf-core-header-segment.patch
   kexec-tools/0006-arm64-kdump-set-up-kernel-image-segment.patch
   kexec-tools/0007-arm64-kdump-set-up-other-segments.patch
   kexec-tools/0008-arm64-kdump-add-DT-properties-to-crash-dump-kernel-s.patch
   kexec-tools/0009-arm64-kdump-Add-support-for-binary-image-files.patch

(From OE-Core rev: d2f196351a8e30542d5fb1561cb7960b6673c5ab)

Signed-off-by: Zheng Ruoqin <zhengrq.fnst@cn.fujitsu.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

sbc: fix license

sbc library itself is licensed under LGPLv2.1 or higher as mentioned in
sbc/sbc.h or any other file in sbc directory.

sbc test applications are licensed under GPLv2 or higher as mentioned in
src/sbcenc.c or any other file in src directory

Reported-by: Vladimir Koutny <vladimir.koutny@streamunlimited.com>
(From OE-Core rev: ef34ed6282627005d0077f88235fdbe42dea23a3)

Signed-off-by: Radek Dostál <radek.dostal@streamunlimited.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

sbc: move examples to their own package

Suggested-by: Ross Burton <ross.burton@intel.com>
(From OE-Core rev: ffd9a1b5b32aeb44ff908378fdc6e2cc9595deff)

Signed-off-by: Radek Dostál <radek.dostal@streamunlimited.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

sqlite3: upgrade to 3.21.0

Remove upstreamed patch:
        1. sqlite3-fix-CVE-2017-13685.patch

(From OE-Core rev: 483711e676cd063a873179bdb2daedf56de0aa75)

Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

libproxy: upgrade to 0.4.15

0.4.14 -> 0.4.15

(From OE-Core rev: 4ec39933d2290c484e34ac803420155c7a2feee7)

Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

quota: upgrade to 4.04

4.03 -> 4.04

Removed the following backported patches:
        1. 0001-Do-not-accidentaly-override-commandline-passed-CFLAG.patch
        2. 0001-Fix-build-with-disabled-ldap.patch
        3. 0001-Use-NGROUPS_MAX-instead-of-NGROUPS.patch
        4. 0003-Don-t-build-rpc.rquotad-when-disable-rpc-was-request.patch
        5. 0004-Fix-warnings-due-to-missing-stdlib.h.patch

Refreshed the following patch:
        1. replace_getrpcbynumber_r.patch

(From OE-Core rev: 0fb3d1fa1af5a2ff7bbc4ef1c84e4135f119d701)

Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

harfbuzz: upgrade to 1.6.3

1.4.8 -> 1.6.3

(From OE-Core rev: bbc18d587d2f9283e7f503f82764849d3a3e55cf)

Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

ofono: upgrade to 1.21

1.20 -> 1.21

(From OE-Core rev: 24fe10634c5e6f17bab2fb5235eb96a2700663cd)

Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

hostap-utils: remove obsolete driver

HostAP is the user-space part of the Intersil Prism 2/2.5/3 wifi chipset.  It's
also a decade old and obsolete, so remove it from oe-core.

(From OE-Core rev: 63be0678bbe3c3809cdd75f1aabd8e031341719a)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

packagegroup-base: remove hostap-utils from wifi support

hostap-utils is the userspace part of the Intersil Prism2/2.5/3 wifi driver,
which is a decade old and obsolete.  Stop pulling it into all images via the
base packagegroup, BSPs which still need this driver can do that themselves.

(From OE-Core rev: 5988d9e55bac5d5d703f86b05301812be328602b)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

poky: Switch to post release name/version

(From meta-yocto rev: 5d49cf4cde6945218aade88adcfa049b0e6ac20a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

dev-manual: Updated bitbake-layers overview section.

The section was missing three of the summaries for the command when
you used --help to find out about it.  I added these in to the section.

(From yocto-docs rev: ac9c58322b7f8533335a9d4379aa031bb1de7be6)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

dev-manual: Removed note about yocto-layer being deprecated.

Deprecation of yocto-layer occurred in 2.4.  For 2.5, we do not
need to mention this fact in the note.

(From yocto-docs rev: 619b74422baf3b531e2fc3db50a95019178a1c09)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

ref-manual: Updated supported distro list

Removed Fedora 24.

(From yocto-docs rev: 4383c2ac9f1f9ca925518874dc40b8394a2865a4)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

documentation: Set up for "sumo" (YP 2.5) Release.

poky.ent - updated variables
<manual>.xml - updated manual revision tables
mega-manual.sed - replaced "2.4" with "2.5" for links.

(From yocto-docs rev: 15d1f805ad11fba145155dcec17b7ec8a21ad487)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

ref-manual: 2.4 Migration Revisions

(From yocto-docs rev: 0802de0ca64b7d3581fb2e3ff26d2f72802ade61)

Signed-off-by: Kristi Rifenbark <kristi.rifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

ref-manual: 2.4 Migration Revisions

(From yocto-docs rev: 1778a743ba3a8b82e96d1e9948e86048d77a8e77)

Signed-off-by: Kristi Rifenbark <kristi.rifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

beaglebone-yocto: rename beaglebone machine to beaglebone-yocto

The beaglebone BSP provided in this layer is a reference implementation
usingonly mainline components available in core Yocto Project layers,
whereas Texas Instruments maintain a full-featured BSP in the meta-ti
layer.

Rename the reference machine to prevent namespace collisions with the
version maintained by the SoC vendor.

[YOCTO #12326]

(From meta-yocto rev: e32882938eca7f50548deab84dab78b4aef31b95)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

dhcp: fix build issue with libxml2 support

A missing case breaks the build when libxml2 is
required and found appropriately. The third argument
to the function AC_SEARCH_LIB is action-if-found which
was mistakenly been used for the case where the library
is not found and hence breaks the configure phase
where it shoud actually pass.
We now pass on silently when action-if-found is
executed.

(From OE-Core rev: a17f3ec910366e9e7551fa24fbc07929b9584341)

Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

dhcp: use ${BPN} instead of ${PN} for user

${PN} will include additional prefixes, such as lib32-, which are not
actually a part of the user that is being added. This was creating an unused
user and possibly missing the actually intended user. By using ${BPN} this
will remove all additional extra information and consistently be "dhcp".

(From OE-Core rev: 77375b50e23555c3a2d24f2b41771e4d43c5b026)

Signed-off-by: Dan Dedrick <ddedrick@lexmark.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

package_ipk.bbclass: handle only whitespace in PACKAGE_EXCLUDE

If PACKAGE_EXCLUDE is constructed using _append then it's possible
that the final value will contain only a space. Currently that
results in build failures due to an invalid opkg command line.

(From OE-Core rev: f832f57ba9f6babb946fbde580004acfd85667e1)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

gcc: backport patch to fix miscompilation on mips64

Backport a patch to fix miscompilation on mips64.

We've observed strange behaviour of `systemctl status <xxx> on qemumips64.
The output of the command is like `systemctl show <xxx>', which is incorrect.

The problem is due to the miscompilation of gcc for mips64 platform, thus
backporting patch from upstream to fix this problem.

[YOCTO #12266]

(From OE-Core rev: cfa13e5c756849820644d86d1882602649db6a9c)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

Revert "gcc: fix miscompilation on mips64"

This reverts commit b656fd9267b1f36d46ca20a1c0bcfaedbf7df438.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

initramfs-live-install: Add aarch64 arch to COMPATIBLE_HOST.

So that we can use this on aarch64 with HiKey board.

(From OE-Core rev: d78a3cd2a76b381b0395d99201f670c94fb594c7)

Signed-off-by: Peter Griffin <peter.griffin@linaro.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

linux-firmware: Bump to bf04291 revision

This includes following changes:

bf04291 WHENCE: Add new qed firmware
d8fc990 WHENCE: Add new radeon firmware
7245319 WHENCE: Fix syntax error for iwlwifi-8265-31.ucode entry
18d71a8 Revert "ath10k: QCA988X hw2.0: update firmware to 10.2.4.70.63-2"
4ebfab3 ath10k: QCA6174 hw3.0: update board-2.bin
96a7402 ath10k: QCA6174 hw3.0: update firmware-6.bin to WLAN.RM.4.4.1-00051-QCARMSWP-1
59bf7e2 cxgb4: update firmware to revision 1.16.63.0

The MD5 checksum of WHENCE license file was due the changes above as
the firmware versions are listed there. It had no license term
changes.

(From OE-Core rev: aea6ce797cc1b1ecc199979eb12aa42de8ff73d4)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

cmake: Upgrade to 3.9.5

(From OE-Core rev: 80d3766aab011979ce01ec295696e4939ce43b87)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

libdrm: upgrade to 2.4.88

(From OE-Core rev: 74e4e8277301e85468f1fad68fa52591e5f296fd)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

base.bbclass: increase indent in get_layers_branch_rev() and buildcfg_vars()

Although it may not appeal so much to users to prefer 80x24 consoles,
the general trend is for screens to get bigger and the current output
has started to look a little cramped on a modern HD display.

Increasing from 17 to 20 is obviously arbitrary, but does give enough
space to cleanly display layers such as "meta-nodejs-contrib" and
"meta-virtualization" while still keeping the output fairly compact.

(From OE-Core rev: e556b85a452a4ebf34a0c9581608f135bbad2e2c)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

mesa: Upgrade to 17.2.4 release

Upgrade to a new stable release and drop patches applied on upstream.

For a full release notes, please see:
https://mesa3d.org/relnotes/17.2.0.html
https://mesa3d.org/relnotes/17.2.1.html
https://mesa3d.org/relnotes/17.2.2.html
https://mesa3d.org/relnotes/17.2.3.html
https://mesa3d.org/relnotes/17.2.4.html

(From OE-Core rev: 77e79af4234ac0e96473d4691fd0f1ca759f3447)

Signed-off-by: Fabio Berton <fabio.berton@ossystems.com.br>
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

devtool: implement conditional patch handling

If you have a recipe that uses overrides to conditionally extend
SRC_URI to add additional patches, then you will often need to update
those patches if you're making other changes to the source tree (for
example if you're upgrading the underlying source). Make this possible
with devtool by creating devtool-override-* branches for each override
that conditionally appends/prepends SRC_URI, and have devtool
update-recipe / finish check each branch out in turn and update the
corresponding patches.

A current example of a recipe that does this is the quota recipe - it
applies an additional patch if musl is the selected C library (i.e.
libc-musl is in OVERRIDES).

Note that use of this functionality does require some care - in
particular, updates to patches that appear on the main branch (named
"devtool" by default) should be made there and not only on one of the
specific devtool-override-* branches that are created for each override.
The recommended procedure is to make the changes you want to make to the
main branch first, then check out and rebase each devtool-override-*
branch, testing each one by activating the corresponding configuration,
and then finally run devtool finish.

Fixes [YOCTO #11516].

(From OE-Core rev: aa87603d1ffd695027847f4df75c0406cf4e14d8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

devtool: finish: add dry-run option

If you're not sure what changes devtool finish is going to make, or
you're not sure you're finished with your modifications, it is useful to
be able to see what devtool finish is going to do beforehand, so add
a -N/--dry-run option to make that possible.

(It's also very useful for debugging devtool finish itself.)

(From OE-Core rev: 05f2d5d2ce00c53825ccea5cd9c2262f9d27a638)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

devtool: finish: improve reporting for removed files

If a file is going to be effectively removed from the destination by
devtool finish, we should report that rather than just reporting that
we're removing files from the workspace. This is a little tricky because
the way we actually operate when finishing is to:
 (1) remove all original files (as recorded by devtool upgrade, if that
     was used)
 (2) as part of updating the recipe file, remove the files from next to
     the new recipe (i.e. in the workspace for an upgrade, real recipe
     otherwise) corresponding to commits not in the git tree
 (3) copy over remaining files from the workspace to the destination

To report the files removed with respect to what was originally there,
we need to swap steps 1 and 2 so we can see what no longer exists after
the deletion, and suppress the reporting currently done in step 2 -
however, we still want to report removal in step 2 for the non-upgrade
case, so the latter is conditional.

(From OE-Core rev: db1d663507509cac9d97d7c96ac8590478767ba2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

devtool: show a better error message if meta-files aren't found

If the files that the devtool-source class is supposed to create in the
source tree aren't found in the temporary directory then we know that
the class hasn't worked properly - say that explicitly.

(From OE-Core rev: 4621152509c037532b133e5e6d5b73bda7ddb602)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

devtool: upgrade: reformat --no-patch warning message

* Only log one warning message instead of one per line
* Be a bit more verbose
* "if list" is more pythonic than "if len(list)"

(From OE-Core rev: 2d11e9e6e73648c1cb514c0c10111c7886acae78)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

devtool: upgrade: automatically handle changes to source subdirectory

If the directory where the source code extracts to changes (for
example, when upgrading iucode-tool from 1.5 to 2.1.1, the subdirectory
in the tarball changed from "iucode_tool-${PV}" to "iucode-tool-${PV}")
then handle this automatically. Also handle when it changes to match the
default S value (i.e. "${WORKDIR}/${BP}") in which case we just drop
setting S in the recipe.

Fixes [YOCTO #10939].

(From OE-Core rev: d29881a652bf03627d257a1eac5f02ec17315b8b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

devtool: upgrade: show messages before source extraction steps

Give the user a little more insight into what's being done.

(From OE-Core rev: 9cf2089bd22b9fc4eb0eec8d4924e44519412dad)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

recipetool: create: show a warning for github archive URLs

github archive URLs are not guaranteed to be stable [1] and thus we
should show a warning if a user specifies one to recipetool create (or
devtool add).

[1] http://lists.openembedded.org/pipermail/openembedded-core/2017-September/142519.html

(From OE-Core rev: 7e84a777aa924a237b4e604120ebf8a4b3ba53b2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

devtool: stop always moving workspace to end of BBLAYERS

I noticed that using bitbake-layers add-layer followed by a devtool
command resulted in bitbake re-parsing all of the recipes, which is
annoying. Upon closer inspection I could see that devtool was moving the
workspace layer path to the end of BBLAYERS if it happened to be
somewhere in the middle - there's no need for it to be doing this. This
occurred because we were passing the current workspace path to remove
and the "new" path to add even if the path is not being changed, and I
think earlier versions of bb.utils.edit_bblayers_conf() didn't move the
existing entry under these circumstances as it clearly does now. Fix it
so we only pass the path to be removed if we're actually changing the
path.

(From OE-Core rev: 284426dbad91a3c52eaf7da5c58fe8a2c2dfb826)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

recipetool: create: drop debug print

This looks like some debug printing that was left in by accident.

(From OE-Core rev: b0bfa1b1f4377270af9e7f19949cc1781a4e3b9d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

devtool: upgrade: handle recipes that use named SRC_URI checksums

devtool upgrade did not properly handle setting SRC_URI checksums for
recipes that use named SRC_URI entries and also use those names in the
SRC_URI checksums. A further complication was where the name contained
an expression that changed with the version e.g. ${PV} (probably quite
rare, but the dnsmasq recipe in meta-networking is currently one such
recipe.) All of these are now handled properly.

Additionally, drop the _get_checksums() function that wasn't being
called from anywhere in the code.

Note that this now turns nowrap_vars in recipeutils.py to be a list of
regexes, hence things such as [ and ] need to be appropriately escaped.

(From OE-Core rev: c914a5e1ad6d96e316746222e5d42f2ba9110060)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

devtool: finish: fix "layer not in bblayers.conf" warning when path specified

devtool finish will check if the destination layer is part of
bblayers.conf so that we avoid the user getting confused about the
recipe vanishing from their configuration if it isn't. devtool finish
also accepts a path underneath a layer so that you have a bit
more control over where it ends up. However if you used a path
underneath a layer then it wasn't converting this to the base of the
layer before checking it against BBLAYERS, thus the warning was being
shown erroneously in that case.

(From OE-Core rev: ab1b8d55e551fea3e8656aab7a786d1bfec62d0f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

devtool: finish: ensure repository is clean before proceeding

If the git repository for a recipe in the workspace has uncommitted
changes in it then it's possible that the user has forgotten to commit
something, so check and exit if there are any. Provide a -f/--force
option to continue in the case where the uncommitted changes aren't
needed.

Separately, if the repository is in the middle of a rebase or git am /
apply then error out (without the opportunity to force) since the user
really needs to sort this out before finishing.

(From OE-Core rev: bfebd18982c0c82ef2da63ec8f22175c93b2e308)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

devtool: reset: print source tree base path

If S points to a subdirectory of the source rather than the "base" of
the source tree then print that rather than the subdirectory path when
telling the user they need to remove the source tree, since that is the
directory that they will need to remove.

(From OE-Core rev: 9e8808099046478e98c6cf1903dc6787d69132fc)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

devtool: make find-recipe and edit-recipe always work with any recipe

After some reconsideration I think it's a bit annoying for users to be
forced to use an option to work with recipes where the file isn't in the
workspace, so let's just have these commands check the workspace first
for the recipe, and if it isn't there then load the cache and get it
that way.

(From OE-Core rev: 46683c61069a386658676a79d797062404bf1140)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>