util: Add a simple buffer structure

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

key: Print correct error code when verification fails

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

json: Oops, forgot to add the files

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Move repomd.json into the root

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Don't make the JSON pretty

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Cleanup old metadata before writing new stuff

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Add file type to metadata

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Add filesize to metadata

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Some minor code cleanup when writing metadata

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Only use a few stronger digests for metadata

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Add checksums for files

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Add an array for files

This should allow us to add more items whenever we would need to.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Add the database path to the repository metadata

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

json: Move functions into a separate file

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Use our own JSON functions to build repository metadata

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Sign the repository metadata instead of the database

I don't think that there is any chance that we will be able to get rid
of the JSON object.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

keys: Give signing functions better names

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: key: Remove unused variables

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Refactor importing repositories

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Make the static analyzer happy when importing repositories

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mirrors: Import the correct headers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfers: Use correct error code when a transfer fails

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfer: Return positive error codes on verify

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

httpclient: Prevent that the event loop terminates too early

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

httpclient: Keep WebSocket connections open

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

httpclient: Implicitly handle relaunch

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

httpclient: Launch transfers as soon as they are enqueued when not in standalone mode

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

httpclient: Shorten some function names

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfer: Remove circular dependency to HTTP client

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

httpclient: Refactor the queue management

It does not seem like a very good idea to give cURL all the handles at
the same time and let it sort it all out. Therefore, this is yet another
attempt to implement some half-decent queueing to implement parallel
downloads.

Fixes: #13812 - httpclient: Bring back a queue for transfers
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfer: Don't verify digest when we did not get a positive response

When servers were sending a 404 error page, the transfer failed because
of a digest mismatch which is not what we want.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfer: Ensure we always cleanup on failure

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Remove unused clone function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

snapshot: Clone pakfire with custom flags

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

snapshot: Create a new snapshot if the cache dir does not exist

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

httpclient: Configure some more sensible defaults

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Anchor the key to the appdata

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfer: Tidy up mirror handling

No functional changes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

xfer: For queued xfers, make sure we don't use an older mirror

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mirrorlist: Don't free the JSON object when checking it's type

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mirrors: Disable instead of "broken"

This patch also implements this all with flags.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mirrors: Evaluate the status code

With some errors, there is no point at all to keep trying.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mirrors: Refactor mirror handling

xfers will now immediately get a mirror assigned when they have a
mirrorlist set so that we don't have to play any games later.

When an xfer has failed, we will ask for another mirror and run again.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mirror: Ensure paths always have a trailing slash

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

contrib: Enable mirror list for stable

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Achor the mirrorlist to the appdata

This is now possible because we won't circle-reference Pakfire any more
since we have the ctx object.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Instead of passing around flags, automatically free the repo

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Add reference counting to appdata

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Include flags when cloning an instance

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Clean up after failing to create a temporary directory

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

compress: Free the cookie if level parsing fails

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Include local repo name in path

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Use function to check repo name

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

log buffer: Removed unused r

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Compose path after all other metadata has been set

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Check pointers before trying to free them

This is again for the static analyzer

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Make the static analyzer happy again

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Actually copy files to the right place

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Implement the reverse of pakfire_package_get_path()

This hopefully fixes that we will copy/link packages to themselves.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Don't try to copy the package into the same repository

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Ensure the repo is always internalized

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: repo: Usual EQUALS macro to show the true figures

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packages: Allow passing a repository for faster caching

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Automatically sync local repositories with their index

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packagelist: Add helper function to check if a certain path is on the list

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packages: Fix destroying a package

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Have the function to return all packages as list create the list

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packages: Introduce a function to return the expected path

Since we need the paths in other places, this is the best way.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Don't do any path voodoo for internal repositories

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

transaction: Use the new check to find available packages

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Pass on the correct error code if we could not open the database

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

package: Build a unified way to check if a package exists

This performs a quick stat() as well.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Make path composition public

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Simply scan again on forced refresh of local repos

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Make the write metadata function public

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Remove the refresh function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Don't refresh the local repository sooner than the others

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Drop Pakfire.refresh()

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Refresh repos once all have been created

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Move creating the local repository into Pakfire

Pakfire is kind of the thing that should do the setup. That way, we have
it all in one place instead of two places of setup.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cli: The shell environment is obviously local, too

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

contrib: Remove the local repo from the configuration

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Always create a local build repository in the home directory

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

path: Add function to expand paths

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Create a flag to mark locally run builds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Increase the Zstandard compression level

The data is actually really small and compresses well, so we can spend
some extra CPU cycles to save disk space and bandwidth.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

compress: Implement passing the compression level to XZ & Zstd

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

compress: Fix flushing Zstandard output data

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Move compression detection

It is easier to run this when we read the database instead of in the
lower level SOLV reading function because inheriting the FILE handle
becomes difficult.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

package: Add function to destroy a package

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: archive: Make the test repository virtual

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Make all internal repositories virtual

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Since we now support virtual repositories, the don't have to be local

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Fix return codes when reading SOLV data

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Remove unused function to only add package metadata

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cli: Use import function to import archives

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Optionally return the package object when importing archives

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repo: Introduce the concept of virtual repositories

These repos can import packages, but they won't copy them but rather
leave them where they were before.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Revert changing pakfire_path_relpath

Use pakfire_path_relative instead which writes the path to the stack.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build: Remove fetching unused path

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>