libpakfire: db: Add scriptlets

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: scriptlets live in "scriptlets/"

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: archive: Do not break on successful memory allocation

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: scriptlets: Actually free on free()

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Call chdir("/") after chroot

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: scriptlets: Move them into an own data structure

We deal with scriptlets in several places and carrying them around as
loose pointers to some data is not very comfortable.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Initialise r

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Drop unused build_date column

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Drop dependency on libgcrypt

This library was chosen as a lightweight and more easy to use
alternative to OpenSSL.

Unfortunately it does not seem to be up to standard in terms of
reliability, performance and most importantly security.

Therefore we are using OpenSSL which is the de-facto standard library,
well-audited and has a transparent development process.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

archive: Use OpenSSL instead of libcrypt to compute hashes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Link against OpenSSL

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Add a basic test for filelists

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Drop filelists from packages

Filelists are now being allocated when needed.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Remove dropped functions from symbols table

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: filelist: Implement sorting filelists

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Refactor files

This used to be used to implement lists which was a right mess.

Lists are now filelists and files are now files only.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: filelist: Refactor filelist

The code that was written was not even used and has now been rewritten
in order to make it usable and to remove any linked-list stuff from
PakfireFile.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: filelist: Move object declaration into .c file

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Add files to filelist

This is incomplete because we can currently not access some attributes
from the file object.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: steps: Make them independent from libsolv

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Drop build_id column from INSERT statement

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Add check function to check database integrity

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Add some tests for database

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Fix typo in optimize

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Fix reading settings

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Drop old build_id column

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Add packages to database when installed

This does not add any metadata, yet.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Add foreign keys to tables

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Add scriptlets table

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Add files table

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Add dependencies table

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Create packages table

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Self-populate schema

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Set database to WAL mode

We disable autocheckpointing since we are expected to writer rather
large transactions and we would like to keep the database as compact as
possible, too. That is achieved by truncating the WAL log in the end
where the data is being copied into the main database and the WAL log
being emptied after.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Set some sensible defaults for the connection

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: status might have been used uninitialised

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: execute: Raise OS error when starting the process failed

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Setup some basic database logging

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Use SQLite3

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Add flags to decide whether we need write access

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Export functions like everything else

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Remove LMDB

This does not seem to be the right tool for our task

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Call database routines when packages are added/removed

These functions are empty right now and serve as placeholders

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: transaction: Pass the open database down to steps

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: transaction: Open the database when needed

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: db: Assume that the private directory already exists

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Define and always create private dir

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Skip freeing repositories when pool isn't set

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Add lmdb database environment

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Do not allocate more space than we need

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Depend on lmdb

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Allow passing the logging callback from Python

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Reduce logging callback to one function

This function is now being passed the loglevel and can therefore
distinguish between stdout and stderr.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: archive: Only load support for what we use

This reduces the footprint of libpakfire in memory as well as not
opening us up to any vulnerabilities in filters and formats of
libarchive that we do not use.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Statically allocate the log buffer

This prevents the buffer from growing unboundedly, but limits us to only
process log messages of up to the buffer size.

I have chosen 64k which should be more than enought that we never run
into this situation.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Make logging functions exchangeable

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: execute: Add test with multiple newlines in one read

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Read all output from commands and log it

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Rename LOG_OUTPUT to INTERACTIVE

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Read content from stdout/stderr and sent it to the logger

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Create fds to read output in parent process

This will be used to log any output of the executed command in the
parent process.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Call chroot() and personality() only when necessary

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: python: Check return codes of pakfire_execute()

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Tidy up return codes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Use clone() instead of fork()

This avoids copying the whole process (which could be quite large) and
spawn the new process into a new namespace straight away without needing
to call unshare(2) later.

This should be a little bit faster when executing many commands.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Bring back execute environment

This allows us to only pass one argument to pakfire_execute_fork

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builder: Drop private network setting

The build chroot will never have network enabled, but the shell does.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Drop old unshare code

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Unshare environment when entering chroot

The shell has networking enabled automatically.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Add pakfire_execute_command

This is a convenience function when only a simple command without any
arguments is called and it saves us from allocating argv in the calling
function.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Cast perso to unsigned int for printing

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Re-indent execute.c

This file was accidentially indented using both spaces and tabs.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Cast argv to non-const as late as possible

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builder: Use pakfire_execute to run shell

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Remove environment and accept arguments as they are passed to execve

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Increase i when copying environment

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Increase pool debug level when compiled with SOLVER_DEBUG

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Set pool to RPM format

On Debian and other non-RPM distributions the default format might be
something different which reconfigures evr comparison and the name of
the "all" architecture.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Make "pakfire provides" work

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire-builder: shell: Add --install argument

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: Pass Pakfire to pakfire_create as argument

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Update translations

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Give better names to assertions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Remove all typedefs and use structs instead

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Make all tests static

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Allocate testsuite statically

This allows us to have shorter function calls when setting up a test

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Add macro for shorter listing of tests

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

tests: Add test for pakfire_string_startswith

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: util: Make pakfire_string_startswith return 1 on success

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Check if path is absolute

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

pakfire: Check if path exists

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Throw a clean exception when Pakfire was initialised with an invalid architecture

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

constants: Remove more unused constants

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

constants: Drop unused CHUNK_SIZE

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builder: Move shell environment variables into submodule

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Automatically set personality from arch

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Drop unused define for STACK_SIZE

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

libpakfire: execute: Add option to pass environment

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Add tests for execute()

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Export pakfire_execute() into Python

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>