jail: Fix return codes of bind function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: execute: Implement bind-mounting mountpoints

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: execute: Capture stdout/stderr separately

That way, we can raise the error output and return the rest

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: execute: Actually verify the set nice level

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: execute: Bring back the open file descriptor test

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: execute: Actually check the output

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Add some simple tests for the buffer

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Only negative return values are considered unsuccessful

Sometimes we want to return the length of consumed input, so this should
all be considered successful.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

po: Update translations

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

log stream: Don't fail if no callback has been registered

This should usually never happen.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Don't drop into a shell when build has been interrupted

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Cleanly terminate on SIGINT

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Move syscall wrapper to where it should be

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: execute: Check that we are not running for forever

There was a bug once where the loop never terminated if the process
actually could never be executed.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Drain the pipes after the process has terminated

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Free the input buffer after we are done

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Free string array after use

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Move the filelist streaming helper from PTY

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Move the send buffer function from the PTY

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Refactor the output callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Remove any traces of the deprecated PTY stdin callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Refactor streaming data into the jail

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: jail: Consume any output of the pipe

This is just because of the log streamer complaining

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buffer: Add a helper function that fills a buffer with a callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

log stream: Return the number of bytes written

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Only make the parent ends of the pipes non-blocking

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buffer: Don't allow to resize the buffer smaller than what it holds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buffer: Don't allow the buffer to grow larger than its defined length

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Don't leak our locking fd into the jail

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Flush any output after we have updated the buffers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Use our own buffer implementation

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: log stream: Replace PTY test with file handle test

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Remove code that maps CRNL to NL

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Remove all sorts of code that is not needed any more

This is all of the non-interactive code.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Bring back pipes for non-interactive sessions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Use the PTY only for interactive sessions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buffer: Always allocate at least one chunk

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buffer: Automatically shrink the buffer

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buffer: Correctly compute the new desired length when growing

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buffer: Add function to configure the maximum length

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buffer: Give the functions slightly better names

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

log stream: Use the new buffer implementation

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buffer: Create a new generic buffer implementation

This is used to hold a dynamically growing buffer.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

log stream: Expect the callback to consume the entire line

This is otherwise getting out of hand and we should not have a place
where this would become an issue.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Use log stream to perform line-buffered output

This helps us to split the complexity between the PTY and some extra
buffering code.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: log stream: Stream another line

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: execute: Use the send buffer callback we already have

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

log stream: Move data as second argument of the callback

That way, we are compatible with the PTY stdout callback.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

log stream: Change first argument of the callback to ctx

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

PTY: Remove the rest of the line buffering code

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

log stream: Add an experimental implementation to stream from PTY

This is the first step to split all the complicated line buffering from
the PTY code.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

log streamer: Move line sanitization from PTY

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Major refactor

This adds loads of comments to hopefully make this thing more robust. So
far we don't really seem to be able to implement all the features we
need without destroying others.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Pass the program exit code as return code of the loop

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: command: Always return zero after the pipe is finished

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Always disable character echoing

This usually happens later in case of the interactive version, but since
we don't need to configure much in the pipe case, we can just set this
as default.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Add a function to set up interactive mode

This way we can split off the initialization code a little bit better
from the main initialization of the PTY object.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Refactor how we capture output

This will now avoid copying all the data around between the kernel and
userland and will also simply use our own callbacks which helps us to
keep the code a little bit less of an if/else hell.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Remove use of removed PTY flags

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Re-organise flags

The CONNECT* flags have been used for two purposes which got mixed up on
the way which is why the standard input was not connected for callbacks.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Add more logging when reading/writing

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pty: Automatically drain when the event loop finishes

There was the problem that some content could have remained stuck in the
buffers on occasion.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Fail with error code if fgetc() returns an error

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Tidy up cleanup code

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Tidy up creating a new jail

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: pakfire: Keep a reference to the Python context

Since we are registering all sorts of callbacks to here, we cannot
deallocate the Ctx object, or we won't be able to use them any more.

This is being fixed by holding a reference to the Python object instead
of the context itself. That will at least help us with lots of these
problems.

There is still, however, the problem that both the Python Ctx and
Pakfire object can be dereferenced when the backend objects will remain
existing.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Pakfire: Remove old callbacks from headers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: python: Don't call depreated unit test function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: ctx: Always forward all messages to the logger

We will let the Python logger filter out what we don't need.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: execute: Implement passing data into stdin

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: execute: Raise any output

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: execute: Allow returning the output of the command

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Remove unused function declaration

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: execute: Implement setting the nice value

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Fix return code of nice function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Bring back a basic version of Pakfire.execute()

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Re-add CommandExecutionError

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Send the progress title in the start callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

progress: Add a status callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

progress: Allow to push/pop the status

That way, we can return to something more useful.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

strings: Add a function to pop the last element of an array

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Fix compilation of the archive test

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Drop old status callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Add a unified progress meter

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

progress: Never show any status for child-progress meters

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

progress: Add support for a status message

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Add support for a parent progress indicator for extraction

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

progress: Rename status to state

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: transaction: Add function to run the transaction

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: ctx: Initialize the callbacks

Just to be safe...

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: ctx: Ensure we are holding the GIL

This is required because we usally return the GIL when we are jumping
into Pakfire functions as these tend to take some time when running.

When we are jumping back into the Python code, we need to acquire the
GIL again.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Add functions to register a progress callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Allow passing a confirm callback from Python

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Add a basic Transaction class

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Remove unused flags

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

db: Fail graciously when we could not open the database

This mainly changes the return code and does some minor code cleanups

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

po: Update translations

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Add yet another kind of GitHub URL

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

arch: Don't use TLS to store the utsname struct

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Ignore cgroups if they are not fully functional

This is an experimental change to see how this will all turn out on
Jenkins.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>