compress: Swap PAKFIRE_WALK_DONE and *_END for semantic reasons

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Fix reading filelists/extraction on newer formats

Fixes: #12995 - pakfire extracts meta files in archives
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Drop support for legacy package formats

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Fix progress bar on extraction

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Use CET on x86_64

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Enable libstcd++ assertions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Change how we append arguments to the request

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Do not send DEBUG messages to the build service and log file

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repositories: Refactor how we are reading metadata

This is a large rewrite of how we are discovering and reading any
repository metadata.

It first of all makes the code a little bit more straight forward by
breaking steps into their own function.

Those functions will now do "the right thing" depending whether we are
dealing with a local or remote repository and will try to read
repository metdata for local repositories, too.

If that fails, we will of course fall back and scan.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Allow setting a custom cache path through the configuration

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

downloader: Read proxy settings from the general section

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: compose: Ensure that the destination path always exists

realpath() fails if the destination does not exist, so we will try to
create it before.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Store the real path on stack to avoid it being altered later

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Try to hardlink packages when possible

Since we no longer change any packages when composing a repository (no
embedded signatures), we can try to hardlink to save disk space and IO.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Handle relative paths on compose

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Check files for being RELRO

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Check if ELF files have an executable stack

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Refactor hardening checks

Instead of calling many different checks, this will now check once
and store any issues that have been identified.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Compress the SOLV database using Zstandard

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Ensure to close the file descriptor after reading the database

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

CFLAGS: Use -fstack-protector-all over -strong

Although -fstack-protector-strong will omit inserting any canary checks
on functions that cannot possibly overflow their stack, we will need all
functions to be protected in order to run our check.

It is more benefitial to us to have the check than optimise for
performance. We also expect LTO to inline all those functions that
possibly do not have a stack.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

IPFire 3: Disable all repositories for now except Bootstrap

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Make sure the destination directory exists when copying

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Check if binaries have been built with -fPIC

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Set download size for new format packages

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Improve importing filelists from JSON

We no longer build a completely filelist object that is being destroyed
very quickly again. The paths are being sent directly to the package.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Move SSP check into the build process

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Check if files are in ELF format

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Reuse function to walk through the package filelist

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Move checking for broken symlinks into C

Fixes: #13057 - Broken symlink check seems to fail if spaces are in filename
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Use S_IFMT instead of AE_IFMT

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Append symlink target to dump

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Don't install logrotate files

logrotate has been dropped

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Don't run MACRO_FIX_AUTOTOOLS all the time any more

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Fix update config.guess/config.sub macro

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Delete static libraries only when there is a shared object

This brings back the former behaviour.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packagelist: Refactor the whole thing

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

CLI: requires: Search for all reverse dependencies

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

search: Consolidate using Dataiterator and add file search

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire_whatprovides/requires: Write to an existant list

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

ui: Don't ask any questions if the shell isn't interactive

This is a slight hack because we did not finish implementing a proper
solution with callbacks that we would prefer.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Refactor returning all packages in a repository

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packagelist: Return some value on push

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packagelists: Split function to import packages from Queue

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Check if makefile name matches the package name

This is quite important because the build process relies on it.

Fixes: #13041 - pakfire-builder confuses on wrong .nm file name
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Fix sorting when adding more elements

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Add a simple test for filelists

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

db: Don't return filelists in a funny order

Since filelists are now ordered, there is no point in trying to be smart
here. In fact it is more efficient to return it pre-sorted.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Rename *_append() to *_add()

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Drop performance_index function

We no longer use this.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Rename *_size() -> *_length()

Also call *_is_empty() wherever suitable as it is much faster.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Remove any *.a and *.la files internally instead of using a script

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Implement removing files from a filelist

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Implement detecting static libraries and libtool archives

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Drop sort function as filelists are now always sorted

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Refactor the entire module

This carries some changes that will allow us more flexibility with
filelists as they are now based on a slightly easier to manage data
structure.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

find-requires: Only check for interpreters for executable files

This avoids collecting interpreters from firmware likes like QEMU
installs them for instance.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

dependencies: Add support for arch() namespace

This can be used (mainly in source packages) to check on what system a
package is being installed - but not to check whether it is compatible.

Most commonly, this feature is going to be used to pull in dependencies
that should only be installed on certain architectures.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

dependencies: Fix parsing complex dependencies

It is possible that a complex dependency has more than one closing
bracket. In that case, we will take too much of the string.

This patch makes sure we take the first bracket.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mount: Increase size for /tmp in jail

Some build processes require more temporary space than 1GiB, so I up
this to 4GiB hoping that it will suffice for most builds.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Skip calling scripts if no file matches

Fixes: #13038
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cgroups: Make killall function static and fix memory leak

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Try to classify files using libmagic

Just filtering by "*.pm" will result in packages that miss some
dependencies, but the provides script generates too many false-positives
so that we cannot just pass the entire filelist.

This will now filter them using libmagic.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

configure: Depend on libmagic

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Make it easier to build in a different directory

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Define firmwaredir

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

perl: Correctly decode Perl versions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Meson does not support parallel tests

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Implement filter_provides/filter_requires

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Move regex compilation function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Construct LDFLAGS with more information and add -O1

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Implement configuring timeouts for commands

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: cmake: Fix setting paths

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Set container and container_uuid environment variables in jail

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Set SYSTEMD_OFFLINE when in chroot

This will prevent systemctl to talk to systemd which won't work in a
chroot environment.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Set LANG to C.utf-8 by default

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Make sure that CMake always uses the correct CFLAGS

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelist: Drop pattern matching from walk function

This is no longer being used.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Bring back pattern matching which was accidentially dropped

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Do not add dependencies to files provided by the same package

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: communicate: Mark the stdin fd as closed to avoid later writes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Send the scriptlet only once

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: epoll: Only register for the operations we actually need

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Avoid deadlock when sending data to stdin

It could happen that we filled up the buffer the input pipe and the jail
communication got deadlocked.

This is now being solved by using the event loop to find out when we are
ready to send more data. This required some changes into how the
callbacks behave.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Fix searching for descriptions

SEARCH_SUBSTRING and SEARCH_GLOB does not seem to work together.

Therefore I am dropping SEARCH_GLOB because a substring search is the
more intuitive one.

Fixes: #12998
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: jail: Send signals to ourselves

This should not cause the parent process to die.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: jail: Check for correct exit code on SEGV

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: command: Add a command that immediately causes SEGV

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Return 139 when the child was killed

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: jail: Check for correct exit code

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mount: Fix bind-mounting read-only shares

Fixes: #12968
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Remind libtool where to find any system libraries

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Set CC and CXX environment variables to the default compilers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transactions: Fix showing freed space

Fixes: #13013
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Append -pipe to CFLAGS

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

parser: Support export VAR += VALUE

Fixes: #13011
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Move CFLAGS/CXXFLAGS/LDFLAGS into the build section

This is where they belong and it makes it easier to modify them in the
build section of individual packages.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Do not log skipped dependencies

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Add -Wl,--as-needed to LDFLAGS by default

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Fix FTBFS linux/fcntl.h and fcntl.h are incompatible

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>