cli: build: No longer use the build logger

We have so many loggers and we need to remove a few...

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

ctx: Move the magic context to here

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Remove build/mkimage/shell from the Pakfire object

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Remove the build service implementation wrapper

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Remove the Python daemon implementation

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

job: Fix typo

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

job: Send log messages to the build service

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Implement adding strings with length to a JSON object

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfer: Make the WebSocket send callback more sensible

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

job: Buffer log lines in a ring buffer

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

logging: Add a simple ring buffer for buffering log lines

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

job: Pipe any log output back to the parent process

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

job: Create an own independant context for the child process

This prevents us to inherit anything from the parent process and allows
us to run a totally different configuration in the child.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

job: Create scaffolding to run a build

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

job: Fix replace error in flags

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

job: Initialize Pakfire in the child process

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Have them monitor themselves and update their state

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Automatically launch jobs once it has connected to the build service

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

job: Show what job we are connecting to

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfer: Should more detail when a transfer fails

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Rename workers to jobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Remove the worker's own HTTP client and event loop

It makes no sense to duplicate so much code, so the control connection
of the worker will have to run in the main process, too.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

worker: Store a reference to the daemon's HTTP client

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

worker: Fetch the daemon's event loop

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Terminate all workers when terminating

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

worker: Initialize the event loop and cURL in the child process

cURL is not multi-threaded and so to avoid any problems, we will
initialize everything in the forked child process.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

worker: Establish a control connection

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buildservice: Use new URL formatting feature

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfer: Make it easier to format URLs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

worker: Create our own event loop and HTTP client

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Connect the worker with the daemon

The daemon will now process when the worker exits and at least clean up.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Move the clone3() wrapper

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Fix format of the stats message

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Enable stat submission only when we have a control connection

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Build out submitting stats

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Add JSON helper function for unsigned integers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Rename JSON function to add integers to int64

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Add new helper function to add double values to JSON objects

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Don't require a reference to pakfire for the JSON functions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: buildservice: Remove submit_stats()

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Keep the stats timer running continuously

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Send dummy stats using the event loop

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfer: Implement sending a WebSocket message

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buildservice: Remove the event loop

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Remove the buildservice

It is becoming too complicated to make the buildservice event-driven
when it does not need to be. It can live as a single request abstraction
model.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buildservice: Remove holding the base URL

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

httpclient: Set the base URL for new xfers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

httpclient: Add option to set a base URL

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Submit stats regularly

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buildservice: Tidy up creating the daemon control connection

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buildservice: Enable authentication when uploading the payload

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Register any workers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Split stuff into a new worker thing

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfer: Ignore CURLE_AGAIN

This is happening when the server is sending a ping over WebSocket which
triggers the socket, but there is no data for us to deal with as it has
already been handled by cURL.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfer: Fail if the connection was unexpectedly closed

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfer: Free the allocated buffer

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Parse the received job

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Automatically try to reconnect when the connection gets lost

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfer: Refactor handling any transfer problems and call the close callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buildservice: Move daemon stuff to the daemon

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

httpclient: Implement basic WebSocket communication

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfer: Send authentication headers straight away when asked to

We don't need to check whether the server does not require
authentication first.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buildservice: Allow passing the event loop to the HTTP client

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

httpclient: Remove queueing our own transfers

We now throw everything into cURL and let it handle all of that...

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

httpclient: Tell cURL how many transfers we want to run in parallel

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

httpclient: Register the progress with the client

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

httpclient: Implement parallel transfers with sd-events

This patch will create a new event loop for each HTTP client which will
be used to connect to cURL.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Use the systemd event loop instead of epoll

This will allow us more flexiblity and already implements a lot of
features.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Makefile: Build the C daemon again

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Fix typo

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mount: Remove the excessive parameter passing

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

deps: Revert all the braindead logging changes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

logging: Add convenience functions to call the logger without having a context

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mount: Loop devices are actually block devices

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

logging: Remove *_ERRNO logging

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pwd: Remove legacy logger

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

compress: Remove legacy logger

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mount: Remove the legacy logger

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Remove the legacy logger

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

dependencies: Drop the legacy logger

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

problem: Store a reference to the context

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

parser: Store a reference to the context

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Replace legacy logger

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Replace (most of) the legacy logger

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

logging: Make the legacy logger configurable

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Move to new logging in pakfire.c

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Make sure we hold the GIL when jumping into the logging callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Re-enable jail tests that pass

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Pakfire.execute(): Fix parsing some arguments

There have been a couple of problems here when no or invalid values have
been passed.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Fix path to test configuration file

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Test the locally built binaries

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Dump log onto the console for Python tests

This is not very elegant, but I don't have a better way to do this now.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Add tests for the Archive class

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Add functions to make opening files in the test env easier

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Add Python tests for context

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: ctx: Cleanup code that dereferences the logging function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: ctx: Setup the default logger

There was some code that started implementing this, but weirdly it was
never called...

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Disable all jail tests

Those mess up the console.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Migrate other Python tests to the new framework

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Build a common basis for Python tests

This new class has some methods that make it easy for us to set up some
simple Pakfire environments.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>