jail: Allow accessing loop devices

This is not great, but the only way we can mount any images inside the
jail as loop devices are not namespaced (yet).

Jails of this style can access any loop devices set up by the system and
for other jails.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Install tools that are required to build a certain image

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Find all packages to be installed and create a new repository with them

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: mkimage: Take a fd for the output

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Add function to copy all data from one fd to another

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Add scaffolding to create images

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: archive: Fix compiling

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Drop PGP test key

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

contrib: Update keys of IPFire 3

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Carry the comment with us and require it

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Import/export keys as strings

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Do not insist on reading the comment line first

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Write database signature to the correct place

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Fix re-reading repository key

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Wipe memory after importing keys

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Export signing/verification routines in Python

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Flush buffers after creating a signature

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Fix handling IDs (again)

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Convert the key ID to integer in Python

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Implement loading keys

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Make the ID an array of bytes again

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: keys: Fix error handling when returning algorithm

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Add some simple tests for keys in Python

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Import everything from _pakfire

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Export the key algorithm constants

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: keys: Treat IDs as integers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Change key id into uint64_t

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Drop listing keys

We no longer keep keys stored.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Drop delete operation

Since we don't have a keystore any more, there is no need to implement
this.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Implement creating a detached signature for databases

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Drop flags argument from compose function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Implement signature verification

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Refactor importing keys

This is now using the base64 decoder and insists on reading the comment
line.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Implement decoding base64 data and add tests

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Drop the old keystore as it is not longer being used

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Replace usage of PGP by signify

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Drop fetching PGP keys from keyservers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Set CCACHE_DIR

This is mostly for completeness and not to cause any problems when there
is a custom ccache configuration inside the jail.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Set CCACHE_TEMPDIR to /tmp

This will cause that ccache creates any temporary files in /tmp instead
of the cache dir. This caused massive bandwidth and slightly slow builds
with a shared NFS cache.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Wrap Pakfire entirely into a thread

This is an attempt to fix a couple of concurrency issues which cause
that Pakfire does not cleanup any files on disk.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Make job_id a property

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

config: Allow longer section & key names

This allows us to use UUIDs as repository names

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Replace /usr/bin/env with the absolute path if possible

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Fix creating the build environment without a snapshot

For some reason, I really messed this one up.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Implement extracting archives into arbitrary locations

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

installcheck: Add a function that checks whether a package can be installed

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: archive: Allow opening packages in any repository

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

request: Implement multiinstall for kernel as pooljobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

request: Fix passing solver flags

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Set up the loopback interface

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

client: Add switch to disable test builds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cgroups: Prevent falling through to default statement all the time

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Do not upload any packages for test jobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cgroups: Don't create groups in system root for unprivileged users

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cgroups: Fix checks for file descriptors

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Log user/group and subids

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Split comment

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pwd: Use libsubid

This is an attempt to read any subids using libsubid from shadow.

However, it seems that libsubid is not entirely thread-safe and randomly
fails. Hence this code is kept disabled for now.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Be more verbose when pakfire_create fails

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Move SUBIDs into user/group structs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Drop function to fetch user home directory

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Fetch more user/group information at startup

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Finish builds with a regular POST request

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Set a default ccache path in the configuration

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Allow setting a different ccache path

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Improve Python exception raising on build

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Drop pakfire_build legacy function

Since we need to extend the interface, it is becoming painful to keep
the compat layer working for only one call.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Avoid having to steps when not using the snapshot

The build environment can be cached in a snapshot which allows much
faster builds. But sometimes, we don't want to use the snapshot.

In those cases, we will install the default set of packages first and
then we will install the source package. In order to find any dependency
problems quicker, this is now being done in just one step.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Correctly read the CPU model

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pwd: Remove static buffer for subid entry

In the build service, it could happen that Pakfire runs concurrently
which might cause that the statically allocated memory might be
overwritten by another thread.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Automatically fetch a TGT when a keytab has been given

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Don't use a default keytab

This might overwrite when a user is logged in.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Attempt to reconnect on 502 Proxy Error

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Refactor message handling

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Build scaffolding to abort builds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Rename log message field

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Update message format for job messages

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Receive jobs over the new control connection

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Use the new control connection for sending stats

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Include timestamp in log messages

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Refactor communication with the hub after merging pakfire-web and -hub

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Move from pakfirehub.ipfire.org to pakfire.ipfire.org

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Check for invalid script interpreters

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Define PY_SSIZE_T_CLEAN

https://docs.python.org/3/c-api/intro.html

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Revert "downloader: Drop using deprecated CURLOPT_PROTOCOLS"

This reverts commit de08af0247aa2bf56251a168eca186c701dce6c2.

This is difficult to mirgate to since Debian Bullseye ships a version of
cURL which does not support CURLOPT_PROTOCOL_STR, yet. So since on
Bookworm there is only a deprecation warning, we will simply revert this
patch for now and deal with the warning.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Implement scaffolding for CF protection check

See #13084

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Don't try to bind-mount @local if it does not exist

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Remove any nested functions from reading files

Nested functions require an executable stack which we want to avoid.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

downloader: Drop using deprecated CURLOPT_PROTOCOLS

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Fix coding style of sysusers macro

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Apply all local sysusers files by default

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Add macro to apply sysusers based users/groups inside the jail

This macro can be called inside a build file and easily allows to apply
any kind of users/groups specified in a sysusers file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Add macro to automatically install any systemd sysusers files

This macros works very similar than the tmpfiles one but handles
sysusers files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

systemd.macro: Avoid declaring the directory for tmpfiles twice

We allready have declared this directory in the arch macro file,
so there is no need in doing this again.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Add macro to automatically install all systemd sysusers files

This macros works very similar than the tmpfiles one but handles
sysusers files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Introduce sysusersdir

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

meson: Avoid calling meson without setup is deprecated

Calling meson without setup as argument when configure
a project is deprecated since a while by the meson developers.

To avoid any problems in future adding this argument.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: sysusers: Fix walking through archive for sysusers files

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Hardening: Declare content of /usr/lib/grub as firmware files

This folder contains the neccessary files, which are written to
the MBR, dealing with EFI, or loading additional required grub
modules unless the whole grub menu can be displayed or a selected
OS will start up.

Some of these files are 32bit ELF files or do not have SSP etc.

So I would suggest to mark them as firmware files and therefore
skip some of the hardening tests.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

parser: Perform a side-lookup for packages in build namespace

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>