file: Mark files as executable

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Rename extension check to patterns

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Do not check for SSP for runtime linkers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Release and acquire the GIL when we need it

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Dump the complete filelist

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

compress: Resolve hardlinks when writing archives

Fixes: #13014
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Correctly fail PIE test

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Show build time at the end

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

string: Add function to format elapsed time

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Perform magic check for all files

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Skip hardening checks for firmware files

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

CFLAGS: Move string formatting stuff into an extra variable

That way, we can clear it easily.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Disable all hardening checks for Relocatable Objects

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Skip SSP check for data libraries

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Implement marking configuration files in archives

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Fix digest comment

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Add missing return type

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Store MIME type of files

This is going to be helpful in the build service and generally some
useful metadata.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

files: Fix iterating over extended attributes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Check if ELF files contain debug information

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Show error when the hardening check fails

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Show error when a file has no symbol table

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Show tags for failed execstack/partly RELRO check

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Be more efficient when reading single files

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Fix reading files from archives

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

compress: Swap PAKFIRE_WALK_DONE and *_END for semantic reasons

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Fix reading filelists/extraction on newer formats

Fixes: #12995 - pakfire extracts meta files in archives
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Drop support for legacy package formats

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Fix progress bar on extraction

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Use CET on x86_64

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Enable libstcd++ assertions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Change how we append arguments to the request

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Do not send DEBUG messages to the build service and log file

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repositories: Refactor how we are reading metadata

This is a large rewrite of how we are discovering and reading any
repository metadata.

It first of all makes the code a little bit more straight forward by
breaking steps into their own function.

Those functions will now do "the right thing" depending whether we are
dealing with a local or remote repository and will try to read
repository metdata for local repositories, too.

If that fails, we will of course fall back and scan.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Allow setting a custom cache path through the configuration

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

downloader: Read proxy settings from the general section

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: compose: Ensure that the destination path always exists

realpath() fails if the destination does not exist, so we will try to
create it before.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Store the real path on stack to avoid it being altered later

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Try to hardlink packages when possible

Since we no longer change any packages when composing a repository (no
embedded signatures), we can try to hardlink to save disk space and IO.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Handle relative paths on compose

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Check files for being RELRO

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Check if ELF files have an executable stack

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Refactor hardening checks

Instead of calling many different checks, this will now check once
and store any issues that have been identified.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Compress the SOLV database using Zstandard

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Ensure to close the file descriptor after reading the database

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

CFLAGS: Use -fstack-protector-all over -strong

Although -fstack-protector-strong will omit inserting any canary checks
on functions that cannot possibly overflow their stack, we will need all
functions to be protected in order to run our check.

It is more benefitial to us to have the check than optimise for
performance. We also expect LTO to inline all those functions that
possibly do not have a stack.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

IPFire 3: Disable all repositories for now except Bootstrap

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Make sure the destination directory exists when copying

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Check if binaries have been built with -fPIC

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Set download size for new format packages

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Improve importing filelists from JSON

We no longer build a completely filelist object that is being destroyed
very quickly again. The paths are being sent directly to the package.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Move SSP check into the build process

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Check if files are in ELF format

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Reuse function to walk through the package filelist

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Move checking for broken symlinks into C

Fixes: #13057 - Broken symlink check seems to fail if spaces are in filename
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Use S_IFMT instead of AE_IFMT

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Append symlink target to dump

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Don't install logrotate files

logrotate has been dropped

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Don't run MACRO_FIX_AUTOTOOLS all the time any more

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Fix update config.guess/config.sub macro

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Delete static libraries only when there is a shared object

This brings back the former behaviour.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packagelist: Refactor the whole thing

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

CLI: requires: Search for all reverse dependencies

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

search: Consolidate using Dataiterator and add file search

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire_whatprovides/requires: Write to an existant list

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

ui: Don't ask any questions if the shell isn't interactive

This is a slight hack because we did not finish implementing a proper
solution with callbacks that we would prefer.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Refactor returning all packages in a repository

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packagelist: Return some value on push

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packagelists: Split function to import packages from Queue

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Check if makefile name matches the package name

This is quite important because the build process relies on it.

Fixes: #13041 - pakfire-builder confuses on wrong .nm file name
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Fix sorting when adding more elements

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Add a simple test for filelists

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

db: Don't return filelists in a funny order

Since filelists are now ordered, there is no point in trying to be smart
here. In fact it is more efficient to return it pre-sorted.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Rename *_append() to *_add()

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Drop performance_index function

We no longer use this.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Rename *_size() -> *_length()

Also call *_is_empty() wherever suitable as it is much faster.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Remove any *.a and *.la files internally instead of using a script

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Implement removing files from a filelist

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Implement detecting static libraries and libtool archives

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Drop sort function as filelists are now always sorted

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Refactor the entire module

This carries some changes that will allow us more flexibility with
filelists as they are now based on a slightly easier to manage data
structure.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

find-requires: Only check for interpreters for executable files

This avoids collecting interpreters from firmware likes like QEMU
installs them for instance.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

dependencies: Add support for arch() namespace

This can be used (mainly in source packages) to check on what system a
package is being installed - but not to check whether it is compatible.

Most commonly, this feature is going to be used to pull in dependencies
that should only be installed on certain architectures.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

dependencies: Fix parsing complex dependencies

It is possible that a complex dependency has more than one closing
bracket. In that case, we will take too much of the string.

This patch makes sure we take the first bracket.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mount: Increase size for /tmp in jail

Some build processes require more temporary space than 1GiB, so I up
this to 4GiB hoping that it will suffice for most builds.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Skip calling scripts if no file matches

Fixes: #13038
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cgroups: Make killall function static and fix memory leak

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Try to classify files using libmagic

Just filtering by "*.pm" will result in packages that miss some
dependencies, but the provides script generates too many false-positives
so that we cannot just pass the entire filelist.

This will now filter them using libmagic.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

configure: Depend on libmagic

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Make it easier to build in a different directory

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Define firmwaredir

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

perl: Correctly decode Perl versions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Meson does not support parallel tests

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Implement filter_provides/filter_requires

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Move regex compilation function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Construct LDFLAGS with more information and add -O1

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Implement configuring timeouts for commands

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: cmake: Fix setting paths

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Set container and container_uuid environment variables in jail

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Set SYSTEMD_OFFLINE when in chroot

This will prevent systemctl to talk to systemd which won't work in a
chroot environment.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>