jail: Implement better logging for the child process

The child process used to dump any logging to the standard output/error
which is not useful when we want to collect any actual output of the
process that was called.

This patch adds extra pipes (one for each log level - I know) and passes
those log messages on to the regular logger (past the jail log
callback).

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Automatically determine how many fds we have in the array

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Don't open log descriptors are non-blocking

We want write() to block if the pipe is full and not drop any messages.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Do not remove trailing newline when capturing log

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Add switch to return output on execute

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Actually connect stdout/stderr to logger

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Rename execute.py to jail.py

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: command: Rename pid to print-pid

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Implement setting nice level

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

execute: Drop old environment

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Unify the wait logic for processes and use pidfd

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Rework Pakfire.execute() using jail

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Export in libpakfire

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Move pakfire_execute_shell/ldconfig

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Add option to collect stdout

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

execute: Drop unused pakfire_execute_script function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Replace pakfire_execute_script with new jail functions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

scriptlets: Use new jail functions to run scripts

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Add convenience function to run scripts

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Configure UID/GID mapping correctly for root

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Create fds for logging as non-blocking straight away

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Use struct in execution context

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Move completed_fd into ctx

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Use factory function to create jail

This jail can be configured and customised for the build process.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

execute: Replace old code with new jail

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Drop separate logging callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

execute: Drop unused pakfire_execute_command function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

execute: Use pakfire_jail_run() to call ldconfig

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

execute: Use convenience function to run shell

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Add convenience function to run simple commands

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

execute: Use new jail to run shell

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Add interface to simply execute scripts

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Add function to import environment

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Set default logging callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Implement changing logging callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Import logging stuff from execute.c

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Fix signal handling

We need to stricly send uint64_t.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mount: Add /dev/shm

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mount: Bring back /tmp to jail

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mount: Mount a new instance of /dev/pts

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mount: Make /dev/mqueue available in jail

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mount: Fix mounting any file systems from the host system

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Try bind-mounting device nodes when we cannot use mknod()

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Apply syscall filter

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Execute command

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Drop capabilities

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Set open file limit

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Set personality

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Mount all default filesystems

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Fix setting UID/GID in namespace

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Make the client process wait until the parent has finished initialization

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Perform UID/GID setup for new namespace

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Implement first steps of running a command in jail

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Makefile: Define TEST_STUB_ROOT in testsuite

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Introduce interactive jails

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Add flags

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Add prototype for execution function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Set some default environment variables

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Implement setting environment variables

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Log when jails are created/destroyed

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Add a simple test for jails

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Add basic type

The goal is to split execute.c into a more flexible API.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

execute: Split creation of namespaces into two steps

This is needed to create a new user namespace first in which we can do
some first stage of initialization. Then, we will start a new process in
that new namespace which will finish initialization.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

execute: Add flags to the environment

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

execute: Drop creating a new cgroup in the parent process

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: execute: Check if the process has received PID 1

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: execute: Add test to catch exceptions in logger

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: command: Print PID

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: command: Build a fork bomb

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: execute: Check if logger returned the correct output

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: command: All option to exhaust all memory

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: execute: Generate some random output for tests

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: execute: Replace former dummy commands

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: execute: Add test to return environment variables

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: execute: Add a simple echo command that prints lines

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: execute: Replace exit code tests with the stub command

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Run execute tests in the stub environment

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Build scaffolding for a simple statically linked command

This can then be used to test the execution container.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

snapshot: Show throughput and ETA in progress bar

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Drop any custom timespec functions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

progressbar: Make this a lot smoother

Previously, the progress bar was redrawn very often which did not look
great on the terminal. We are now redrawing the bar about 20 times a
second which gives us a smooth experience.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Adjust flag to enable/disable ccache

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Adjust flag to enable/disable snapshot

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Remove unneeded stuff from snapshot.c

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

snapshot: Include pakfire.h

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Check if snapshot has something in it

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mount: Don't return error in foreach when there are no mountpoints

The loop returned 1 by default when there were no mountpoints to process
which is not what we need here.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

snapshots: Protect against invalid inputs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Add a test that creates and restores a snapshot

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Revert "mount: Adjust mount flags for unprivileged users"

This reverts commit c92f710524a370d8e910b74d7ba062373d02d7a6.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mount: Adjust mount flags for unprivileged users

Bind-mounts require us to set MS_REC and remounting any mountpoint
requires us to now downgrade on noexec/nodev/nosuid.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mount: Use mount(2) to perform any mount operations

libmount did too much voodoo here which prevented us from running
smoothly for unprivileged users.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Don't use path in error message

path seems to have been freed after the extraction has been started and
therefore we cannot use it any more.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mount: Don't mount /tmp in container

If /tmp is a ramdisk, any temporary files written during the build
process will be lost between stages. That is rather unintuitive and we
might use excess memory.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

execute: Copy scripts into /

Formerly they were created in /tmp which could be overlayed by a tmpfs.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Make the static analyzer happy

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Move cache directory into user's home

This is only happening when running as an unprivileged user.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Fix permissions of temporary directories

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Move all temporary files directly into /var/tmp

When running Pakfire as an unprivileged user, we cannot create temporary
files in a subdirectory which has been created earlier by a different
user.

Hence we now put everything directly into /var/tmp where everyone should
have write permissions all of the time.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Store errno when running pakfire_rmtree()

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>