buffer: Always allocate at least one chunk

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buffer: Automatically shrink the buffer

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buffer: Correctly compute the new desired length when growing

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buffer: Add function to configure the maximum length

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buffer: Give the functions slightly better names

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

log stream: Use the new buffer implementation

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buffer: Create a new generic buffer implementation

This is used to hold a dynamically growing buffer.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

log stream: Expect the callback to consume the entire line

This is otherwise getting out of hand and we should not have a place
where this would become an issue.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Use log stream to perform line-buffered output

This helps us to split the complexity between the PTY and some extra
buffering code.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: log stream: Stream another line

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: execute: Use the send buffer callback we already have

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

log stream: Move data as second argument of the callback

That way, we are compatible with the PTY stdout callback.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

log stream: Change first argument of the callback to ctx

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

PTY: Remove the rest of the line buffering code

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

log stream: Add an experimental implementation to stream from PTY

This is the first step to split all the complicated line buffering from
the PTY code.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

log streamer: Move line sanitization from PTY

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Major refactor

This adds loads of comments to hopefully make this thing more robust. So
far we don't really seem to be able to implement all the features we
need without destroying others.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Pass the program exit code as return code of the loop

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: command: Always return zero after the pipe is finished

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Always disable character echoing

This usually happens later in case of the interactive version, but since
we don't need to configure much in the pipe case, we can just set this
as default.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Add a function to set up interactive mode

This way we can split off the initialization code a little bit better
from the main initialization of the PTY object.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Refactor how we capture output

This will now avoid copying all the data around between the kernel and
userland and will also simply use our own callbacks which helps us to
keep the code a little bit less of an if/else hell.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Remove use of removed PTY flags

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Re-organise flags

The CONNECT* flags have been used for two purposes which got mixed up on
the way which is why the standard input was not connected for callbacks.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Add more logging when reading/writing

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Automatically drain when the event loop finishes

There was the problem that some content could have remained stuck in the
buffers on occasion.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Fail with error code if fgetc() returns an error

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Tidy up cleanup code

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Tidy up creating a new jail

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: pakfire: Keep a reference to the Python context

Since we are registering all sorts of callbacks to here, we cannot
deallocate the Ctx object, or we won't be able to use them any more.

This is being fixed by holding a reference to the Python object instead
of the context itself. That will at least help us with lots of these
problems.

There is still, however, the problem that both the Python Ctx and
Pakfire object can be dereferenced when the backend objects will remain
existing.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Pakfire: Remove old callbacks from headers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: python: Don't call depreated unit test function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: ctx: Always forward all messages to the logger

We will let the Python logger filter out what we don't need.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: execute: Implement passing data into stdin

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: execute: Raise any output

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: execute: Allow returning the output of the command

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Remove unused function declaration

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: execute: Implement setting the nice value

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Fix return code of nice function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Bring back a basic version of Pakfire.execute()

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Re-add CommandExecutionError

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Send the progress title in the start callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

progress: Add a status callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

progress: Allow to push/pop the status

That way, we can return to something more useful.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

strings: Add a function to pop the last element of an array

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Fix compilation of the archive test

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Drop old status callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Add a unified progress meter

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

progress: Never show any status for child-progress meters

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

progress: Add support for a status message

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Add support for a parent progress indicator for extraction

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

progress: Rename status to state

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: transaction: Add function to run the transaction

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: ctx: Initialize the callbacks

Just to be safe...

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: ctx: Ensure we are holding the GIL

This is required because we usally return the GIL when we are jumping
into Pakfire functions as these tend to take some time when running.

When we are jumping back into the Python code, we need to acquire the
GIL again.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Add functions to register a progress callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Allow passing a confirm callback from Python

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Add a basic Transaction class

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Remove unused flags

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

db: Fail graciously when we could not open the database

This mainly changes the return code and does some minor code cleanups

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

po: Update translations

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Add yet another kind of GitHub URL

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

arch: Don't use TLS to store the utsname struct

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Ignore cgroups if they are not fully functional

This is an experimental change to see how this will all turn out on
Jenkins.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Fix printing erros when repos config could not be written

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Fix makefile tests

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Fix writing configuration

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

parser: Make it less verbose

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Log less when creating/freeing a jail

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jenkins: Move the asan logs to the root

Because otherwise I am too stupid to find them.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Avoid leaking memory in parser tests

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jenkins: Upload asan logs only when tests have failed

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

parser: Dump the current status less often

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfopen: Include limits.h

We need this for ULONG_MAX.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

strings: Have pakfire_string_format_interval write to a preallocated buffer

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jenkins: Use multiple CPU cores for the static analyzer & LCOV

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jenkins: Run the testsuite with address sanitization

There are still some things that are broken (Python), so for now, we
won't care if this fails.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

configure: Let the compiler figure out what to link for ASAN

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

CLI: Refactor progressbar to use statically allocated buffers

This might take a little bit of pressure off the allocator and we don't
have to deal with asprintf() loosing memory.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transactions: Free the job queue

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

snapshots: Be less verbose when destroying a snapshot

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Fix buffer overrun when reading the scriptlet

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

asan: Use libunwind for better stack traces

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Remove various smaller memory "leaks"

There are not critical because the test suite does not run for very
long.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Free the timeout timer

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Use the reference to the loop that we already have

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Refactor making the filelist

There seems to have been a leak when initializing the string, so we
format it on the stack first and then copy it to the heap later.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

strings: Add function to add formatted strings to a buffer

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

log buffer: Free all held lines when freeing the buffer

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

db: Ensure to free the deps list after each iteration

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

configure: Build with debugging information when using asan

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

configure: Add option to enable the address sanitizer

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jenkins: Remove any traces for Almalinux

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jenkins: Use more modern command line to install dev tools on Fedora

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

parser: Ignore comments regardless what state the scanner is in

Fixes: #13363 - A comment character anywhere in the build or packages section causes the build to fail
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

dist: Fix return codes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

parser: Properly handle yyparse() return codes

This patch converts them into our usual errno codes.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jenkins: Strictly name releases

For Ubuntu, latest points at the latest LTS release, which is not what I
expected. To avoid any confusion, let's name the releases that we want
to support by their number or code name.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jenkins: Add Ubuntu latest for the tests

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Run setuid/gid mapping even when we are running in /

Since we always create a new namespace, we need to always run this, too.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>