classes/pypi: update the default UPSTREAM_CHECK_URI to use the simple repo API

Update the UPSTREAM_CHECK_URI to leverage the simple repo API.  The
project URLs require javascript which breaks the version checking fetch
and subsequent logic.  The simple repo API provides similar
functionality with a well defined spec which is used by tools such as
pip.  Also update the UPSTREAM_CHECK_REGEX to be compatible with the
information retrieved via the API

Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

diff --git a/meta/classes-recipe/pypi.bbclass b/meta/classes-recipe/pypi.bbclass
index c6bbe8119a683e4b08999fe629b699463b37adac..b0bc167cdfa58ad4892b448e4c43d868ad7401e4 100644 (file)
--- a/meta/classes-recipe/pypi.bbclass
+++ b/meta/classes-recipe/pypi.bbclass
@@ -28,6 +28,14 @@ def pypi_src_uri(d):
     archive_downloadname = d.getVar('PYPI_ARCHIVE_NAME_PREFIX') + archive_name
     return 'https://files.pythonhosted.org/packages/source/%s/%s/%s;downloadfilename=%s' % (package[0], package, archive_name, archive_downloadname)
 
+def pypi_normalize(d):
+    """"
+        Normalize the package names to match PEP625 (https://peps.python.org/pep-0625/).
+        For non-compliant packages, maintainers can set UPSTREAM_CHECK_PYPI_PACKAGE to override the normalization
+    """
+    import re
+    return re.sub(r"[-_.]+", "-", d.getVar('PYPI_PACKAGE')).lower()
+
 PYPI_SRC_URI ?= "${@pypi_src_uri(d)}"
 
 HOMEPAGE ?= "https://pypi.python.org/pypi/${PYPI_PACKAGE}/"
@@ -36,8 +44,14 @@ SRC_URI:prepend = "${PYPI_SRC_URI} "
 S = "${WORKDIR}/${PYPI_PACKAGE}-${PV}"
 
 # Replace any '_' characters in the pypi URI with '-'s to follow the PyPi website naming conventions
-UPSTREAM_CHECK_PYPI_PACKAGE ?= "${@d.getVar('PYPI_PACKAGE').replace('_', '-')}"
-UPSTREAM_CHECK_URI ?= "https://pypi.org/project/${UPSTREAM_CHECK_PYPI_PACKAGE}/"
-UPSTREAM_CHECK_REGEX ?= "/${UPSTREAM_CHECK_PYPI_PACKAGE}/(?P<pver>(\d+[\.\-_]*)+)/"
+UPSTREAM_CHECK_PYPI_PACKAGE ?= "${@pypi_normalize(d)}"
+
+# Use the simple repository API rather than the potentially unstable project URL
+# More information on the pypi API specification is avaialble here:
+# https://packaging.python.org/en/latest/specifications/simple-repository-api/
+#
+# NOTE: All URLs for the simple API MUST request canonical normalized URLs per the spec
+UPSTREAM_CHECK_URI ?= "https://pypi.org/simple/${@pypi_normalize(d)}"
+UPSTREAM_CHECK_REGEX ?= "${UPSTREAM_CHECK_PYPI_PACKAGE}-(?P<pver>(\d+[\.\-_]*)+).(tar\.gz|tgz|zip|tar\.bz2)"
 
 CVE_PRODUCT ?= "python:${PYPI_PACKAGE}"