tox config, README update, listening on Unix socket, added black for code formatting

diff --git a/manager/.gitignore b/manager/.gitignore
index 24690a91ed218f31abcc2f3e73784d7c74e4cb23..ff1cdbf6e481ba178209a9668e64e4da8d043e62 100644 (file)
--- a/manager/.gitignore
+++ b/manager/.gitignore
@@ -5,4 +5,6 @@ __pycache__/
 node_modules/
 yarn.lock
 package-lock.json
-.pytype
\ No newline at end of file
+.pytype
+dist/
+.tox/
\ No newline at end of file

diff --git a/manager/.python-version b/manager/.python-version
index 8b7b0b52e55c33b0447d3fd1c564c8b7b4a42ed5..ba49e60f82a60f69672d4b343d48fa983b6141d1 100644 (file)
--- a/manager/.python-version
+++ b/manager/.python-version
@@ -1 +1,3 @@
 3.6.12
+3.7.9
+3.8.7

diff --git a/manager/README.md b/manager/README.md
index aa39af13795e73be1a80c2ecf4b1888074253ceb..97b8a5e36fa9549504371db3629daa6c46749e04 100644 (file)
--- a/manager/README.md
+++ b/manager/README.md
@@ -9,7 +9,7 @@ Because we want to support multiple versions of Python with one codebase, we dev
 Install these tools:
 * [pyenv](https://github.com/pyenv/pyenv#installation)
 * [Poetry](https://python-poetry.org/docs/#installation)
-* [Yarn](https://yarnpkg.com/) (See FAQ for why do we need JS in Python project)
+* [Yarn](https://yarnpkg.com/) (See FAQ for why do we need JS in Python project) or NPM
 
 The actual development environment can be setup using these commands:
 
@@ -17,7 +17,7 @@ The actual development environment can be setup using these commands:
 pyenv install
 poetry env use $(pyenv which python)
 poetry install
-yarn install
+yarn install # or npm install
 ```
 
 ### Common tasks and interactions with the project
@@ -26,8 +26,10 @@ After setting up the environment, you should be able to interract with the proje
 
 * `poe run` - runs the manager from the source
 * `poe test` - unit tests
+* `poe tox` - unit tests in all supported Python versions
 * `poe check` - static code analysis
 * `poe fixdeps` - update installed dependencies according to the project's configuration
+* `poe clean` - cleanup the repository from unwanted files
 
 All possible commands can be listed by running the `poe` command without arguments. The definition of these commands can be found in the `pyproject.toml` file.
 
@@ -37,7 +39,39 @@ If you don't want to be writing the `./` prefix, you can install [PoeThePoet](ht
 
 Before commiting, please ensure that both `poe check` and `poe test` pass.
 
+### Packaging
+
+Not yet properly implemented. Ideal situation would be a command like `poe package` which would create all possible packages.
+
+Temporary solution to build a wheel/sdist - just call `poetry build`. The result will be in the `dist/` directory.
+
 ## FAQ
+
+### What all those dev dependencies for?
+
+Short answer - mainly for managing other dependencies. By using dependency management systems within the project, anyone can start developing after installing just a few core tools. Everything else will be handled automagically. The main concept behind it is that there should be nothing that can be run only in CI, not on dev machines.
+
+* core dependencies which you have to install manually
+  * pyenv
+    * A tools which allows you to install any version of Python regardless of your system's default. The version used by default in the project is configured in the file `.python-version`.
+    * We should be all developing on the same version, because otherwise we might not be able to reproduce each others bug's.
+    * Written in pure shell, no dependencies on Python. Should therefore work on any Unix-like system.
+  * Poetry
+    * A dependency management system for Python libraries. Normally, all libraries in Python are installed system-wide and dependent on system's Python version. By using virtual environments managed by Poetry, configured to use a the correct Python version through pyenv, we can specify versions of the dependencies in any way we like.
+    * Follows PEP 518 and uses the `pyproject.toml` file for all of it's configuration.
+    * Written in Python, therefore it's problematic if installed system-wide as an ordinary Python package (because it would be unavailable in its own virtual environment).
+  * Yarn or NPM
+    * Dependency management systems from JavaScript development.
+    * Used for installing pyright - the type checker we use.
+* automatically managed dependencies
+  * PoeThePoet - A task management system, or in other words glorified switch statement calling other tools. Used for simplifying interractions with the project.
+  * pytest, pytest-cov - unit testing
+  * pylint, flake8 - linting
+  * pyright - type checking, compatible with VSCode using the Pylance extension
+  * black - autoformatter (might be removed in the future if not used in practice)
+  * tox - testing automation
+  * tox-pyenv - plugin for tox that makes use of pyenv provided Python binaries
+
 ### Why do we need JavaScript in Python project?
 
 We would like to use a type checker. As of writing this, there are 4 possible options:

diff --git a/manager/knot_resolver_manager/__main__.py b/manager/knot_resolver_manager/__main__.py
index 4db2dc782b8863f56c21c606214ab3538fe0eb88..bdf6f199c1a3d0ab3ce3248d8e6681fac2fb0533 100644 (file)
--- a/manager/knot_resolver_manager/__main__.py
+++ b/manager/knot_resolver_manager/__main__.py
@@ -1,7 +1,7 @@
 from aiohttp import web
 
 
-async def hello(_request):
+async def hello(_request: web.Request) -> web.Response:
     return web.Response(text="Hello, world")
 
 
@@ -9,7 +9,7 @@ def main():
     app = web.Application()
     app.add_routes([web.get("/", hello)])
 
-    web.run_app(app)
+    web.run_app(app, path="./manager.sock")
 
 
 if __name__ == "__main__":

diff --git a/manager/poetry.lock b/manager/poetry.lock
index 4c3a51d422ad556c4e075e4e8d5abc40c5d8e378..7cc6477e5a815d7a4e445dc50ae7bbb5ab57cb11 100644 (file)
--- a/manager/poetry.lock
+++ b/manager/poetry.lock
@@ -144,6 +144,22 @@ category = "main"
 optional = false
 python-versions = ">=3.6.0"
 
+[[package]]
+name = "distlib"
+version = "0.3.1"
+description = "Distribution utilities"
+category = "dev"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "filelock"
+version = "3.0.12"
+description = "A platform independent file lock."
+category = "dev"
+optional = false
+python-versions = "*"
+
 [[package]]
 name = "flake8"
 version = "3.8.4"
@@ -193,6 +209,21 @@ zipp = ">=0.5"
 docs = ["sphinx", "jaraco.packaging (>=8.2)", "rst.linker (>=1.9)"]
 testing = ["pytest (>=3.5,!=3.7.3)", "pytest-checkdocs (>=1.2.3)", "pytest-flake8", "pytest-cov", "pytest-enabler", "packaging", "pep517", "pyfakefs", "flufl.flake8", "pytest-black (>=0.3.7)", "pytest-mypy", "importlib-resources (>=1.3)"]
 
+[[package]]
+name = "importlib-resources"
+version = "5.1.0"
+description = "Read resources from Python packages"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+zipp = {version = ">=0.4", markers = "python_version < \"3.8\""}
+
+[package.extras]
+docs = ["sphinx", "jaraco.packaging (>=8.2)", "rst.linker (>=1.9)"]
+testing = ["pytest (>=3.5,!=3.7.3)", "pytest-checkdocs (>=1.2.3)", "pytest-flake8", "pytest-cov", "pytest-enabler", "pytest-black (>=0.3.7)", "pytest-mypy"]
+
 [[package]]
 name = "isort"
 version = "5.7.0"
@@ -416,6 +447,40 @@ category = "dev"
 optional = false
 python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
 
+[[package]]
+name = "tox"
+version = "3.21.4"
+description = "tox is a generic virtualenv management and test command line tool"
+category = "dev"
+optional = false
+python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7"
+
+[package.dependencies]
+colorama = {version = ">=0.4.1", markers = "platform_system == \"Windows\""}
+filelock = ">=3.0.0"
+importlib-metadata = {version = ">=0.12", markers = "python_version < \"3.8\""}
+packaging = ">=14"
+pluggy = ">=0.12.0"
+py = ">=1.4.17"
+six = ">=1.14.0"
+toml = ">=0.9.4"
+virtualenv = ">=16.0.0,<20.0.0 || >20.0.0,<20.0.1 || >20.0.1,<20.0.2 || >20.0.2,<20.0.3 || >20.0.3,<20.0.4 || >20.0.4,<20.0.5 || >20.0.5,<20.0.6 || >20.0.6,<20.0.7 || >20.0.7"
+
+[package.extras]
+docs = ["pygments-github-lexers (>=0.0.5)", "sphinx (>=2.0.0)", "sphinxcontrib-autoprogram (>=0.1.5)", "towncrier (>=18.5.0)"]
+testing = ["flaky (>=3.4.0)", "freezegun (>=0.3.11)", "psutil (>=5.6.1)", "pytest (>=4.0.0)", "pytest-cov (>=2.5.1)", "pytest-mock (>=1.10.0)", "pytest-randomly (>=1.0.0)", "pytest-xdist (>=1.22.2)", "pathlib2 (>=2.3.3)"]
+
+[[package]]
+name = "tox-pyenv"
+version = "1.1.0"
+description = "tox plugin that makes tox use `pyenv which` to find python executables"
+category = "dev"
+optional = false
+python-versions = "*"
+
+[package.dependencies]
+tox = ">=2.0"
+
 [[package]]
 name = "typed-ast"
 version = "1.4.2"
@@ -432,6 +497,26 @@ category = "main"
 optional = false
 python-versions = "*"
 
+[[package]]
+name = "virtualenv"
+version = "20.4.2"
+description = "Virtual Python Environment builder"
+category = "dev"
+optional = false
+python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7"
+
+[package.dependencies]
+appdirs = ">=1.4.3,<2"
+distlib = ">=0.3.1,<1"
+filelock = ">=3.0.0,<4"
+importlib-metadata = {version = ">=0.12", markers = "python_version < \"3.8\""}
+importlib-resources = {version = ">=1.0", markers = "python_version < \"3.7\""}
+six = ">=1.9.0,<2"
+
+[package.extras]
+docs = ["proselint (>=0.10.2)", "sphinx (>=3)", "sphinx-argparse (>=0.2.5)", "sphinx-rtd-theme (>=0.4.3)", "towncrier (>=19.9.0rc1)"]
+testing = ["coverage (>=4)", "coverage-enable-subprocess (>=1)", "flaky (>=3)", "pytest (>=4)", "pytest-env (>=0.6.2)", "pytest-freezegun (>=0.4.1)", "pytest-mock (>=2)", "pytest-randomly (>=1)", "pytest-timeout (>=1)", "packaging (>=20.0)", "xonsh (>=0.9.16)"]
+
 [[package]]
 name = "wcwidth"
 version = "0.2.5"
@@ -476,7 +561,7 @@ testing = ["pytest (>=3.5,!=3.7.3)", "pytest-checkdocs (>=1.2.3)", "pytest-flake
 [metadata]
 lock-version = "1.1"
 python-versions = "^3.6.12"
-content-hash = "2175c570cfdff0c990d9b3e962ece66c377a767c399cdf2113a35f4c57b4d181"
+content-hash = "42429537b63d18fac9e23c847343d899e331f977fb00bd2b686d4d174e930adb"
 
 [metadata.files]
 aiohttp = [
@@ -612,6 +697,14 @@ dbus-next = [
     {file = "dbus_next-0.2.2-py3-none-any.whl", hash = "sha256:7127b484d30606b0daaa5ce481873c57e651bbe6df0949ceb39f7addbca0086b"},
     {file = "dbus_next-0.2.2.tar.gz", hash = "sha256:f656a3d3450b670f228248ffb1c3a703a69c4a8cb10cce63b108f17c8bd6c3de"},
 ]
+distlib = [
+    {file = "distlib-0.3.1-py2.py3-none-any.whl", hash = "sha256:8c09de2c67b3e7deef7184574fc060ab8a793e7adbb183d942c389c8b13c52fb"},
+    {file = "distlib-0.3.1.zip", hash = "sha256:edf6116872c863e1aa9d5bb7cb5e05a022c519a4594dc703843343a9ddd9bff1"},
+]
+filelock = [
+    {file = "filelock-3.0.12-py3-none-any.whl", hash = "sha256:929b7d63ec5b7d6b71b0fa5ac14e030b3f70b75747cef1b10da9b879fef15836"},
+    {file = "filelock-3.0.12.tar.gz", hash = "sha256:18d82244ee114f543149c66a6e0c14e9c4f8a1044b5cdaadd0f82159d6a6ff59"},
+]
 flake8 = [
     {file = "flake8-3.8.4-py2.py3-none-any.whl", hash = "sha256:749dbbd6bfd0cf1318af27bf97a14e28e5ff548ef8e5b1566ccfb25a11e7c839"},
     {file = "flake8-3.8.4.tar.gz", hash = "sha256:aadae8761ec651813c24be05c6f7b4680857ef6afaae4651a4eccaef97ce6c3b"},
@@ -627,6 +720,10 @@ importlib-metadata = [
     {file = "importlib_metadata-3.4.0-py3-none-any.whl", hash = "sha256:ace61d5fc652dc280e7b6b4ff732a9c2d40db2c0f92bc6cb74e07b73d53a1771"},
     {file = "importlib_metadata-3.4.0.tar.gz", hash = "sha256:fa5daa4477a7414ae34e95942e4dd07f62adf589143c875c133c1e53c4eff38d"},
 ]
+importlib-resources = [
+    {file = "importlib_resources-5.1.0-py3-none-any.whl", hash = "sha256:885b8eae589179f661c909d699a546cf10d83692553e34dca1bf5eb06f7f6217"},
+    {file = "importlib_resources-5.1.0.tar.gz", hash = "sha256:bfdad047bce441405a49cf8eb48ddce5e56c696e185f59147a8b79e75e9e6380"},
+]
 isort = [
     {file = "isort-5.7.0-py3-none-any.whl", hash = "sha256:fff4f0c04e1825522ce6949973e83110a6e907750cd92d128b0d14aaaadbffdc"},
     {file = "isort-5.7.0.tar.gz", hash = "sha256:c729845434366216d320e936b8ad6f9d681aab72dc7cbc2d51bedc3582f3ad1e"},
@@ -808,6 +905,14 @@ tomlkit = [
     {file = "tomlkit-0.7.0-py2.py3-none-any.whl", hash = "sha256:6babbd33b17d5c9691896b0e68159215a9387ebfa938aa3ac42f4a4beeb2b831"},
     {file = "tomlkit-0.7.0.tar.gz", hash = "sha256:ac57f29693fab3e309ea789252fcce3061e19110085aa31af5446ca749325618"},
 ]
+tox = [
+    {file = "tox-3.21.4-py2.py3-none-any.whl", hash = "sha256:65d0e90ceb816638a50d64f4b47b11da767b284c0addda2294cb3cd69bd72425"},
+    {file = "tox-3.21.4.tar.gz", hash = "sha256:cf7fef81a3a2434df4d7af2a6d1bf606d2970220addfbe7dea2615bd4bb2c252"},
+]
+tox-pyenv = [
+    {file = "tox-pyenv-1.1.0.tar.gz", hash = "sha256:916c2213577aec0b3b5452c5bfb32fd077f3a3196f50a81ad57d7ef3fc2599e4"},
+    {file = "tox_pyenv-1.1.0-py2.py3-none-any.whl", hash = "sha256:e470c18af115fe52eeff95e7e3cdd0793613eca19709966fc2724b79d55246cb"},
+]
 typed-ast = [
     {file = "typed_ast-1.4.2-cp35-cp35m-manylinux1_i686.whl", hash = "sha256:7703620125e4fb79b64aa52427ec192822e9f45d37d4b6625ab37ef403e1df70"},
     {file = "typed_ast-1.4.2-cp35-cp35m-manylinux1_x86_64.whl", hash = "sha256:c9aadc4924d4b5799112837b226160428524a9a45f830e0d0f184b19e4090487"},
@@ -845,6 +950,10 @@ typing-extensions = [
     {file = "typing_extensions-3.7.4.3-py3-none-any.whl", hash = "sha256:7cb407020f00f7bfc3cb3e7881628838e69d8f3fcab2f64742a5e76b2f841918"},
     {file = "typing_extensions-3.7.4.3.tar.gz", hash = "sha256:99d4073b617d30288f569d3f13d2bd7548c3a7e4c8de87db09a9d29bb3a4a60c"},
 ]
+virtualenv = [
+    {file = "virtualenv-20.4.2-py2.py3-none-any.whl", hash = "sha256:2be72df684b74df0ea47679a7df93fd0e04e72520022c57b479d8f881485dbe3"},
+    {file = "virtualenv-20.4.2.tar.gz", hash = "sha256:147b43894e51dd6bba882cf9c282447f780e2251cd35172403745fc381a0a80d"},
+]
 wcwidth = [
     {file = "wcwidth-0.2.5-py2.py3-none-any.whl", hash = "sha256:beb4802a9cebb9144e99086eff703a642a13d6a0052920003a230f3294bbe784"},
     {file = "wcwidth-0.2.5.tar.gz", hash = "sha256:c4d647b99872929fdb7bdcaa4fbe7f01413ed3d98077df798530e5b04f116c83"},

diff --git a/manager/pyproject.toml b/manager/pyproject.toml
index 9780a33e448caeb2954031648269171ce8597f2d..543ae7637d35e6178b660f1dfdcee76d39f75aa1 100644 (file)
--- a/manager/pyproject.toml
+++ b/manager/pyproject.toml
@@ -14,17 +14,47 @@ dbus-next = "^0.2.2"
 [tool.poetry.dev-dependencies]
 pytest = "^5.2"
 pylint = "^2.6.0"
-poethepoet = "^0.9.0"
 pytest-cov = "^2.11.1"
 flake8 = "^3.8.4"
 black = "^20.8b1"
+tox = "^3.21.4"
+tox-pyenv = "^1.1.0"
+poethepoet = "^0.9.0"
 
 [tool.poe.tasks]
 run = { cmd = "python -m knot_resolver_manager", help = "Run the manager" }
 test = { cmd = "pytest --cov=knot_resolver_manager", help = "Run tests" }
 check = { cmd = "scripts/codecheck", help = "Run static code analysis" }
 fixdeps = { shell = "poetry install; yarn install", help = "Install/update dependencies according to configuration files"}
-format-all = { cmd = "black knot_resolver_manager tests", help = "Reformat all code using black. Dangerous!" }
+clean = """
+  rm -rf .coverage
+         .mypy_cache
+         .pytest_cache
+         ./**/__pycache__
+         dist
+         .tox
+"""
+tox = { cmd = "tox", help = "Run tests in tox" }
+
+
+[tool.tox]
+legacy_tox_ini = """
+[tox]
+isolated_build = True
+envlist = py36, py37, py38
+
+[tox:.package]
+# note tox will use the same python version as under what tox is installed to package
+# so unless this is python 3 you can require a given python version for the packaging
+# environment via the basepython key
+basepython = python3
+
+[testenv]
+deps = poetry
+commands =
+    poetry install -v
+    ./poe test
+"""
 
 
 # pylint configuration copied from apkg (https://gitlab.nic.cz/packaging/apkg/-/blob/master/pylintrc)

diff --git a/manager/scripts/codecheck b/manager/scripts/codecheck
index a8f5e4ff1765cb9619803d5701c42183a34c2d6a..77595b27e5eb805f054809a4a8f815387d88469e 100755 (executable)
--- a/manager/scripts/codecheck
+++ b/manager/scripts/codecheck
@@ -19,6 +19,12 @@ function check_rv {
 # stop failing early, because we wouldn't do anything else than fail
 set +e
 
+# check formatting using black
+echo -e "${yellow}Checking formatting using black...${reset}"
+black knot_resolver_manager tests --check --diff
+check_rv $?
+echo
+
 # check code with pylint
 echo -e "${yellow}Linting using pylint...${reset}"
 pylint knot_resolver_manager
@@ -37,5 +43,12 @@ node_modules/.bin/pyright knot_resolver_manager
 check_rv $?
 echo
 
+# fancy messages at the end :)
+if test "$aggregate_rv" -eq "0"; then
+       echo -e "${green}Everything looks great!${reset} 🥳🎉🥰"
+else
+       echo -e "${red}That's not great. Could you please fix that?${reset} 😲😟"
+fi
+
 # exit with the aggregate return value
 exit $aggregate_rv
\ No newline at end of file