keys: Refactor importing keys

This is now using the base64 decoder and insists on reading the comment
line.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Implement decoding base64 data and add tests

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Drop the old keystore as it is not longer being used

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Replace usage of PGP by signify

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Drop fetching PGP keys from keyservers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Set CCACHE_DIR

This is mostly for completeness and not to cause any problems when there
is a custom ccache configuration inside the jail.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Set CCACHE_TEMPDIR to /tmp

This will cause that ccache creates any temporary files in /tmp instead
of the cache dir. This caused massive bandwidth and slightly slow builds
with a shared NFS cache.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Wrap Pakfire entirely into a thread

This is an attempt to fix a couple of concurrency issues which cause
that Pakfire does not cleanup any files on disk.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Make job_id a property

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

config: Allow longer section & key names

This allows us to use UUIDs as repository names

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Replace /usr/bin/env with the absolute path if possible

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Fix creating the build environment without a snapshot

For some reason, I really messed this one up.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Implement extracting archives into arbitrary locations

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

installcheck: Add a function that checks whether a package can be installed

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: archive: Allow opening packages in any repository

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

request: Implement multiinstall for kernel as pooljobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

request: Fix passing solver flags

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Set up the loopback interface

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

client: Add switch to disable test builds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cgroups: Prevent falling through to default statement all the time

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Do not upload any packages for test jobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cgroups: Don't create groups in system root for unprivileged users

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cgroups: Fix checks for file descriptors

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Log user/group and subids

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Split comment

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pwd: Use libsubid

This is an attempt to read any subids using libsubid from shadow.

However, it seems that libsubid is not entirely thread-safe and randomly
fails. Hence this code is kept disabled for now.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Be more verbose when pakfire_create fails

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Move SUBIDs into user/group structs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Drop function to fetch user home directory

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Fetch more user/group information at startup

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Finish builds with a regular POST request

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Set a default ccache path in the configuration

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Allow setting a different ccache path

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Improve Python exception raising on build

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Drop pakfire_build legacy function

Since we need to extend the interface, it is becoming painful to keep
the compat layer working for only one call.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Avoid having to steps when not using the snapshot

The build environment can be cached in a snapshot which allows much
faster builds. But sometimes, we don't want to use the snapshot.

In those cases, we will install the default set of packages first and
then we will install the source package. In order to find any dependency
problems quicker, this is now being done in just one step.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Correctly read the CPU model

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pwd: Remove static buffer for subid entry

In the build service, it could happen that Pakfire runs concurrently
which might cause that the statically allocated memory might be
overwritten by another thread.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Automatically fetch a TGT when a keytab has been given

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Don't use a default keytab

This might overwrite when a user is logged in.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Attempt to reconnect on 502 Proxy Error

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Refactor message handling

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Build scaffolding to abort builds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Rename log message field

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Update message format for job messages

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Receive jobs over the new control connection

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Use the new control connection for sending stats

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Include timestamp in log messages

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Refactor communication with the hub after merging pakfire-web and -hub

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Move from pakfirehub.ipfire.org to pakfire.ipfire.org

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Check for invalid script interpreters

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Define PY_SSIZE_T_CLEAN

https://docs.python.org/3/c-api/intro.html

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Revert "downloader: Drop using deprecated CURLOPT_PROTOCOLS"

This reverts commit de08af0247aa2bf56251a168eca186c701dce6c2.

This is difficult to mirgate to since Debian Bullseye ships a version of
cURL which does not support CURLOPT_PROTOCOL_STR, yet. So since on
Bookworm there is only a deprecation warning, we will simply revert this
patch for now and deal with the warning.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Implement scaffolding for CF protection check

See #13084

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Don't try to bind-mount @local if it does not exist

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Remove any nested functions from reading files

Nested functions require an executable stack which we want to avoid.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

downloader: Drop using deprecated CURLOPT_PROTOCOLS

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Fix coding style of sysusers macro

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Apply all local sysusers files by default

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Add macro to apply sysusers based users/groups inside the jail

This macro can be called inside a build file and easily allows to apply
any kind of users/groups specified in a sysusers file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Add macro to automatically install any systemd sysusers files

This macros works very similar than the tmpfiles one but handles
sysusers files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

systemd.macro: Avoid declaring the directory for tmpfiles twice

We allready have declared this directory in the arch macro file,
so there is no need in doing this again.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Add macro to automatically install all systemd sysusers files

This macros works very similar than the tmpfiles one but handles
sysusers files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Introduce sysusersdir

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

meson: Avoid calling meson without setup is deprecated

Calling meson without setup as argument when configure
a project is deprecated since a while by the meson developers.

To avoid any problems in future adding this argument.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: sysusers: Fix walking through archive for sysusers files

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Hardening: Declare content of /usr/lib/grub as firmware files

This folder contains the neccessary files, which are written to
the MBR, dealing with EFI, or loading additional required grub
modules unless the whole grub menu can be displayed or a selected
OS will start up.

Some of these files are 32bit ELF files or do not have SSP etc.

So I would suggest to mark them as firmware files and therefore
skip some of the hardening tests.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

parser: Perform a side-lookup for packages in build namespace

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Simplify customisation of configure/make/make install

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Move cmake/meson/ninja/perl stuff into the build namespace

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Drop anything related to Python 2

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

FHS: Tolerate runtime linkers in /usr/lib

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: read: Correctly fail if we cannot find a file

The routine did not properly clear up the cookie after it could not find
a file in the archive.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: archive: Free file handle at the end

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Make the _cleanup function configurable to tidy up as well

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Make the testsuite compile again

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Show which step, action and package failed

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Be smarter when removing files

Before, when we tried to remove a file from the filesystem, we tried to
call unlink() and if that failed, we tried rmdir() instead.

This patch changes that we will call rmdir() in the first place for
directories and unlink() for everything else so that we can catch error
codes more granulary.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Use cleanup function to remove all files from a filelist

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Whitelist libgcc_so.* and libmvec.so.* from SSP check

libgcc_s.so cannot be built with SSP, at least it will create some
problems linking start files later on.

libmvec should generally not be on here, but all the assembly magic
seems to confuse something so that it won't be linked okay.

Fixes: #13069
Fixes: #13070
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Make File objects immutable

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Fix potential SEGV when accesing File attributes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Fix repr() output for File objects

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

FHS: Allow /usr/src/kernel

This directory and it's subdirectories will contain the source code
and helper scripts/binaries of the current compiled kernel.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire_format_time(): Fix typo

Fix a small typo when displaying the build time
which is longer than 1 hour.

In such a case the following message got displayed:

Build successfully completed in 01m07m02s

Which should be 01h07m02s

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

FHS: Allow /var/mail owned by root:mail

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Return a file descriptor for any archive files

This is a lot more handy for us later on when we are dealing with any of
the payload which might potentially larger as it can now be read bit by
bit.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

FHS: Drop /usr/bin/su from list of allowed SUID binaries

In the Makefile (util-linx.nm) we specify some capabilities to avoid setting
the suid bit.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Silently ignore if systemd-sysusers could not be executed

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Move flags to individual exec commands

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

strip: Apply hack to preserve capabilities

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

parser: Free regular expressions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

FHS: Allow gpasswd, ksu and pkexec to have the setuid bit set

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

FHS: Fix setuid check

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

FHS: Silence a warning as it gets in the way of the progress bar

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

FHS: Allow installing kernel source in /usr/src

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Check for capabilities being applied to non-executable files

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Automatically create system users

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Check if relative/absolute paths confuse pakfire_path_match

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Log the path of the command we tried to execute

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>