FHS: Do not allow any subdirectories in /usr/bin & /usr/sbin

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

FHS: Implement being able to check for file type

This allows us a more granular filtering

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

FHS: Do not allow any more files in /usr and /usr/src

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

FHS: Implement checking file ownerships

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Drop check-libraries script

This is now covered by the new builtin FHS check.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Drop old FHS script

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

FHS: Implement some simple filesystem checks

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: path_matches: Check if pattern is shorter than string

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Implement a simple path matching function that supports **

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

arch: Drop support for all 32 bit architectures

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

compress: Fix wrong variable in threads code

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packager: Don't initialize an unsigned integer with -1

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

compress: Enable parallel compression for Zstandard if available

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

compress: Create a unified function to create archives

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

snapshots: Call it store/restore

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

snapshots: Do not modify an existing snapshot

Instead, the routines will now write the new snapshot to a temporary
location and replace it more or less atomically.

Fixes: #13045 - Multiple concurrent instances can destroy the snapshot
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Revert "snapshots: Pass path instead of file descriptor"

This reverts commit 4667a2ca811f6f2b20c1cfb3223dd8b90af4952c.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

compress: Do not overwrite configuration on extraction

This is somewhat experimental and I would need to think a little bit
more about this.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Enable all QEMU CPU features by default

When we are emulating a different architecture, QEMU by default emulates
a very basic processor which might not be able to emulate for example
SIMD instructions.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Mark files as executable

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Rename extension check to patterns

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Do not check for SSP for runtime linkers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Release and acquire the GIL when we need it

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Dump the complete filelist

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

compress: Resolve hardlinks when writing archives

Fixes: #13014
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Correctly fail PIE test

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Show build time at the end

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

string: Add function to format elapsed time

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Perform magic check for all files

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Skip hardening checks for firmware files

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

CFLAGS: Move string formatting stuff into an extra variable

That way, we can clear it easily.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Disable all hardening checks for Relocatable Objects

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Skip SSP check for data libraries

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Implement marking configuration files in archives

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Fix digest comment

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Add missing return type

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Store MIME type of files

This is going to be helpful in the build service and generally some
useful metadata.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

files: Fix iterating over extended attributes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Check if ELF files contain debug information

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Show error when the hardening check fails

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Show error when a file has no symbol table

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Show tags for failed execstack/partly RELRO check

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Be more efficient when reading single files

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Fix reading files from archives

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

compress: Swap PAKFIRE_WALK_DONE and *_END for semantic reasons

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Fix reading filelists/extraction on newer formats

Fixes: #12995 - pakfire extracts meta files in archives
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Drop support for legacy package formats

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Fix progress bar on extraction

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Use CET on x86_64

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Enable libstcd++ assertions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Change how we append arguments to the request

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Do not send DEBUG messages to the build service and log file

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repositories: Refactor how we are reading metadata

This is a large rewrite of how we are discovering and reading any
repository metadata.

It first of all makes the code a little bit more straight forward by
breaking steps into their own function.

Those functions will now do "the right thing" depending whether we are
dealing with a local or remote repository and will try to read
repository metdata for local repositories, too.

If that fails, we will of course fall back and scan.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Allow setting a custom cache path through the configuration

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

downloader: Read proxy settings from the general section

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: compose: Ensure that the destination path always exists

realpath() fails if the destination does not exist, so we will try to
create it before.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Store the real path on stack to avoid it being altered later

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Try to hardlink packages when possible

Since we no longer change any packages when composing a repository (no
embedded signatures), we can try to hardlink to save disk space and IO.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Handle relative paths on compose

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Check files for being RELRO

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Check if ELF files have an executable stack

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Refactor hardening checks

Instead of calling many different checks, this will now check once
and store any issues that have been identified.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Compress the SOLV database using Zstandard

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Ensure to close the file descriptor after reading the database

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

CFLAGS: Use -fstack-protector-all over -strong

Although -fstack-protector-strong will omit inserting any canary checks
on functions that cannot possibly overflow their stack, we will need all
functions to be protected in order to run our check.

It is more benefitial to us to have the check than optimise for
performance. We also expect LTO to inline all those functions that
possibly do not have a stack.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

IPFire 3: Disable all repositories for now except Bootstrap

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Make sure the destination directory exists when copying

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Check if binaries have been built with -fPIC

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Set download size for new format packages

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Improve importing filelists from JSON

We no longer build a completely filelist object that is being destroyed
very quickly again. The paths are being sent directly to the package.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Move SSP check into the build process

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Check if files are in ELF format

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Reuse function to walk through the package filelist

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Move checking for broken symlinks into C

Fixes: #13057 - Broken symlink check seems to fail if spaces are in filename
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Use S_IFMT instead of AE_IFMT

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Append symlink target to dump

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Don't install logrotate files

logrotate has been dropped

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Don't run MACRO_FIX_AUTOTOOLS all the time any more

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Fix update config.guess/config.sub macro

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Delete static libraries only when there is a shared object

This brings back the former behaviour.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packagelist: Refactor the whole thing

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

CLI: requires: Search for all reverse dependencies

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

search: Consolidate using Dataiterator and add file search

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire_whatprovides/requires: Write to an existant list

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

ui: Don't ask any questions if the shell isn't interactive

This is a slight hack because we did not finish implementing a proper
solution with callbacks that we would prefer.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Refactor returning all packages in a repository

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packagelist: Return some value on push

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packagelists: Split function to import packages from Queue

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Check if makefile name matches the package name

This is quite important because the build process relies on it.

Fixes: #13041 - pakfire-builder confuses on wrong .nm file name
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Fix sorting when adding more elements

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Add a simple test for filelists

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

db: Don't return filelists in a funny order

Since filelists are now ordered, there is no point in trying to be smart
here. In fact it is more efficient to return it pre-sorted.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Rename *_append() to *_add()

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Drop performance_index function

We no longer use this.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Rename *_size() -> *_length()

Also call *_is_empty() wherever suitable as it is much faster.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Remove any *.a and *.la files internally instead of using a script

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Implement removing files from a filelist

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Implement detecting static libraries and libtool archives

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Drop sort function as filelists are now always sorted

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelists: Refactor the entire module

This carries some changes that will allow us more flexibility with
filelists as they are now based on a slightly easier to manage data
structure.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>