problems: Refactor fetching problems and solutions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Merge requests into the transaction

It makes a lot more sense to have one large object that wraps the
solving process around libsolv instead of having multiple and passing
things around.

This is the first step to move things over and allow more flexibility
when performing depenedency resolution.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

request: Obey policy when best flag is set

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Increase subpriority for commandline repositories

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Changing packages is a re-install

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

request: Fix sync

The dist-upgrade job became a noop when the DROP_ORPHANED flag got set,
because it is not a flag but a separate job instead.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

request: Implement locking the running kernel as a pool job

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Silence debugging messaging saying which step to run

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

request: Automatically attempt to update packages on install

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

contrib: Drop priority from development repositories

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

contrib: Make the local repository preferred over stable

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transactions: Major rewrite of step generation

When we extract from libsolv what steps we need to perform in order to
apply the changes to our system, there was a problem that when updating
packages, the previous version was not properly removed from the
database.

This has now been fixed by adding additional steps which are passive and
will remove the left-over metadata.

This should hopefully make the entire process more robust and easier to
handle in the code as each step has one simple job to do.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

contrib: Remove old signature configuration

We do not support disabling the signature check any more and we no
longer have any host keys to sign packages.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Implement subpriorities to prefer local packages when possible

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Don't wipe dump when we change a package

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Refactor reading userinstalled packages

This could potentially leak memory and was less robust.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

dependencies: Don't log parsed dependencies

This is quite verbose...

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Refactor code to dump a transaction

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

package: Treat sizes as unsigned long long

It looks like there have been some more casting issues which should
hopefully be fixed now.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

compress-man-pages: Support already XZ compressed man pages

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

contrib: Disable the bootstrap repository

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Fix casting issue when writing JSON

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

request: Log dependency issues if not in interactive mode

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Build things with a lower nice level

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Don't abort if /etc/hosts or /etc/resolv.conf cannot be bind-mounted

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Fix compiler warning due to iterating over the wrong type

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

contrib: Update IPFire 3 repositoriy definitions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

downloader: Fix JSON reference counting in mirror list parser

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Correctly handle it no configuration was given

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Fix passing --config option to Pakfire

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

fhs.c: Correct permissions for /var/mail and /tmp

- /var/mail is looking for 0755 but dma is setting 02775
   checking FHS it looks like the settings should be 02775
- /tmp is being checked for 1755 instead of 01755 so the number is
   not being recognised as octal but as decimal
- The /var/mail change confirmed to allow dma to be built in ipfire3.x
   without failing for FHS errors

Tested-by: Adolf Belka <adolf.belka@ipfire.org>
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

configure: Bump version to 0.9.29

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Truncate file before rewriting it

It can happen that the file is shorter after we have fixed the
interpreter, which causes that the remaining data is still in the file.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: util: Fix SEGV when we could not read from file

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Fix potential double-free on error

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Create a ramdisk if no path has been given

This might be more useful when we struggle to cleanup the build
environment due to large IO. The build environment should generally be
small enough to easily fit into memory.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: b64encode: Fix leak of p

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

config: Don't copy section names

This mainly makes the static analyzer happy :)

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Rewrite fixing interpreters

The old version had some weaknesses in finding any arguments that came
after the interpreter which are being fixed here.

This version should also be easier to read.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Fix check if an interpreter has any additional arguments

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Don't fail on non-implemented CF protection check

See: #13084 - Implement CF protection check
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Fix potential NULL pointer dereference in pattern matching

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

package: Make code clearer where dumping deps

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

*list: Fix potential NULL pointer dereference

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Fix pattern matching for files we don't scan

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Never compile the test command using the analyzer

We intentionally have some memory leaks and other bad things in here...

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Initialize buffer when dumping a file

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filter-requires: Remove check if files are on filelist

This does not work in this place because we are streaming the filelist
into the script and the check is being performed in Pakfire.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelist: Fix return code check on pattern matching function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Ensure the target directory exists

This causes that we could not hard-link any repositories since we put
archives into subdirectories resulting in a lot of unnecessary IO.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Fix potential memory leak if entry couldn't be copied

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Don't overwrite noarch

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

parser: Remove debgugging line

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Correctly initialize the dump buffer

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelist: Don't fail scan when the path does not exist

I believe this is what we mostly want.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Ignore if local repositories don't exist when scanning

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

db: Check if schema exists before reading settings

Earlier versions of SQLite did not seem to complain, but newer versions
log an error when we are trying to access the settings table when it
does not exist, yet.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Remove static buffers from basename/dirname

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Drop unused pakfire_path_abspath function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pwd: Don't fail if we cannot read SUBUID/SUBGID from configuration

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pwd: Set maximum range for SUBUID/SUBGIDs by default

This allows us to continue even if we cannot read anything from
/etc/subuid or /etc/subgid.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: packages: Fix group check

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Reformat code that downloads metadata

There are no functional changes here, but this is better to read.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Configure better default refresh times

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Allow storing custom refresh times for metadata

Fixes: #13248 - repo: Build TTL for metadata
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

string: Add function to format/parse intervals

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Increase precision of the logging timestamp

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Implement custom logging that shows build time

Fixes: #13250 - log: Add timestamp for each log line
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Allow customising the log callback

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

db: Fix double free of joined groups string

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jail: Ignore any return codes from the shell

Fixes: #13003 - Python exception when exiting the build jail or requested shell
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: package: Fix cleaning up build arches

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packages: Store groups as array

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: package: Check reading build_id in Python

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: package: Export Build IDs as string

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

downloader: Drop using deprecated CURLOPT_PROTOCOLS

This reverts commit bf0f7d4ad237169f0da6a02c5b19981348616d30.

It looks like we no longer need to support Buster, so this commit is now
safe to apply.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hub: Fix typo in sleep command

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

_pakfire: Fix multiple issues in a utility function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Check if we can build a packager on our architecture

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

arch: Enhance check for support by host and check for interpreter

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

strings: Add function to free string arrays

This should help us to keep code tidier and shorter.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packages: Copy strings when returning arrays

This way, the return value will be owned by the caller which also has
the responsibility to clean it up.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

arch: Teach Pakfire to run a different arch effectively

This is useful to let Pakfire think it is running in "noarch" mode, when
we of course actually have to install some binary packages.

This might also become useful when bootstrapping the distribution for a
new architecture.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

macros: Set arch in pakfire

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Call the correct function to check if the host supports this architecture

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packages: Export build architectures in Python

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Add a simple test to read metadata from Python

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packages: Add support for build arches

This requires that we add a new data type to packages to store a string
array.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Fix expression to find package archives

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packager: Set the sourcepath as path if no path given

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

package: Don't crash if a package has no UUID

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Fix tests and handle invalid inputs better

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Use "struct archive_entry" to store lots of metadata

We used to copy metadata information from the entry format and back
again which does not seem to be too clever. The overhead of the entry
isn't that high that we cannot simply use them here.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

compress: Make sure that paths are relative in archives

When we write archives, we don't want them to contain leading slashes on
the filenames. However, we want to have leading slashes on the
filelists.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

configure: No longer check for FTS functions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Remove using FTS when reading repository configuration

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

arch: Remove using FTS to search interpreter

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

dist: Don't package directories

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

dist: Fix not packaging any temporary files

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

filelist: Use our own pattern matching function throughout

Since we (once again) have another way to match paths in packages, there
is a flag which allows to switch to a different way to match paths.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>