packages: Validate UUIDs when reading from the database

Since we are using the UUID in the filesystem paths, we must make sure
that no malicious content is in the field.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

contrib: Enable the testing repository by default

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packages: Simplify the directory structures once again

I don't think that we will have lots of subdirectories here so that
filesystems won't be able to cope. However, it would be nice to use the
same structure throughout.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

package: Use the correct attribute for directories in the SOLV database

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Fix file structure

Because it is possible to have multiple builds with the same nevra in
the same repository, we need to store packages in a subdirectory to be
able to uniquely identify them. That is also helpful when we want to
avoid downloading a package when expecting a different UUID.

This patch changes that packages will now be put into some subdirectory
structure so multiple archives with the same name can coexist.

The downloader and caching mechanism for repositories had to be tweaked
to honour the subdirectory structure.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Read configuration from a file descriptor

This is easier to handle especially when creating temporary environments
where we just want to shoot the configuration into Pakfire().

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: util: Refactor PyObject_AsFileHandle using fmemcookie

Python is not very good at handling file objects, so we wrap them into
this little tool.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

parser: Remove any global variables

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Fix integer overflow in bit fields

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Consume any exceptions in log callback

We cannot do anything with them here, so we need to de-fuse the
exception so that we won't run into errors when the originally called
Python function returns.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Increase robustness if the server goes away

This patch will retry a couple of requests on certain errors.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: build: Use better return codes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: digest: Use better return codes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: dependencies: Use better return codes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: config: Use better return codes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: arch: Use better return codes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: archive: Return better return codes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: strings: Create better return codes for all functions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

key: Fix wrong passing of string

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: pakfire_create: Return a better return code

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Enhance logging to pass custom errno

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

dependencies: Create and destroy the pool for each version check

The former solution obviously wasn't very thread-safe.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Add version_compare() that does not require Pakfire

This is useful if we do not have a Pakfire instance at hand and will
save us the overhead of creating one every time.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

find-provides: Only handle shared objects

The previous pattern matched other files like "*.socket".

Fixes: #13027 - Pakfire thinks .socket files are .so files
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Remove BUILDROOT check

This searched for any references to BUILDROOT in any generated files.
However, this is hard to implement without using the nested function
which unfortunately requires an executable stack which is not permitted
in IPFire.

Since the check is usually not having a massive impact, this patch
removes it for now with the intention to bring it back at a later time.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Group prctl() calls together

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Create a new time namespace

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Change mount propagation before switching root

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Fix file descriptor check

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Use pivot_root() again instead of chroot()

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Handle signals in epoll() loop

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Initialize all file descriptors with -1

It is not a good idea to use zero as that might be a valid fd.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cgroups: Add BPF program to filter device node access

This is currently permitting everything which we don't want to sustain
in the long-term.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

configure: Link against libbpf

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Don't drop any capabilities

This is not what we finally need, but we will try to give the jail as
many capabilities in its own namespace as possible.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cgroup: Return error when the cgroup could not be created

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Refactor searching for env variable function

This used a variable size array on the stack before which is not needed.
This version should be slightly faster and the compiler should be able
to inline it.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Allow accessing loop devices

This is not great, but the only way we can mount any images inside the
jail as loop devices are not namespaced (yet).

Jails of this style can access any loop devices set up by the system and
for other jails.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Install tools that are required to build a certain image

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Find all packages to be installed and create a new repository with them

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: mkimage: Take a fd for the output

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Add function to copy all data from one fd to another

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Add scaffolding to create images

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: archive: Fix compiling

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Drop PGP test key

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

contrib: Update keys of IPFire 3

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Carry the comment with us and require it

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Import/export keys as strings

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Do not insist on reading the comment line first

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Write database signature to the correct place

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Fix re-reading repository key

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Wipe memory after importing keys

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Export signing/verification routines in Python

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Flush buffers after creating a signature

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Fix handling IDs (again)

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Convert the key ID to integer in Python

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Implement loading keys

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Make the ID an array of bytes again

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: keys: Fix error handling when returning algorithm

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Add some simple tests for keys in Python

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Import everything from _pakfire

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Export the key algorithm constants

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: keys: Treat IDs as integers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Change key id into uint64_t

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Drop listing keys

We no longer keep keys stored.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Drop delete operation

Since we don't have a keystore any more, there is no need to implement
this.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Implement creating a detached signature for databases

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Drop flags argument from compose function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Implement signature verification

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Refactor importing keys

This is now using the base64 decoder and insists on reading the comment
line.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Implement decoding base64 data and add tests

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Drop the old keystore as it is not longer being used

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Replace usage of PGP by signify

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Drop fetching PGP keys from keyservers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Set CCACHE_DIR

This is mostly for completeness and not to cause any problems when there
is a custom ccache configuration inside the jail.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Set CCACHE_TEMPDIR to /tmp

This will cause that ccache creates any temporary files in /tmp instead
of the cache dir. This caused massive bandwidth and slightly slow builds
with a shared NFS cache.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Wrap Pakfire entirely into a thread

This is an attempt to fix a couple of concurrency issues which cause
that Pakfire does not cleanup any files on disk.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Make job_id a property

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

config: Allow longer section & key names

This allows us to use UUIDs as repository names

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Replace /usr/bin/env with the absolute path if possible

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Fix creating the build environment without a snapshot

For some reason, I really messed this one up.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Implement extracting archives into arbitrary locations

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

installcheck: Add a function that checks whether a package can be installed

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: archive: Allow opening packages in any repository

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

request: Implement multiinstall for kernel as pooljobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

request: Fix passing solver flags

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Set up the loopback interface

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

client: Add switch to disable test builds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cgroups: Prevent falling through to default statement all the time

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Do not upload any packages for test jobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cgroups: Don't create groups in system root for unprivileged users

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cgroups: Fix checks for file descriptors

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Log user/group and subids

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Split comment

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pwd: Use libsubid

This is an attempt to read any subids using libsubid from shadow.

However, it seems that libsubid is not entirely thread-safe and randomly
fails. Hence this code is kept disabled for now.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Be more verbose when pakfire_create fails

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Move SUBIDs into user/group structs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Drop function to fetch user home directory

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Fetch more user/group information at startup

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Finish builds with a regular POST request

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>