uploads: Calculate the digest when receiving the file

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Fix sending configuration

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

uploads: Return the received filename in the API response

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

uploads: Add a debug handler to download any uploaded payload

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

uploads: Send an URL to the client where to send the payload

Hopefully this will give us some more flexibility in the future.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

uploads: Manually commit the session

This is required because the session might not be fast enough before the
client comes back with the payload. This ensures that we won't have a
race.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

config: Pass the configuration to Pakfire as string

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

configure: Check for all required Python module

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

constants: Remove unused values

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Rename to PBS

This is easier than a hyphenated Python package name. This is now also
moved out of the pakfire module which has become a pure C module.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Store control connections by their UUID

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Add crash handler

This will be called by the builders when the build process has
unexexpectedly crashed.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Fix retrying jobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Fix aborting jobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

web: Don't fetch index data in a transaction

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Fix fetching running jobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builders: Refactor dispatching jobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

config: Tidy up code

No functional changes.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

config: Make it easier to generate a text configuration

We cannot use the old method any more since we need to call async
methods.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

database: (Re-)raise any exceptions after the task is done

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builders: Fix least busy iterator

This must be an outer join so that we will always get all builds, even
those that don't have a job running.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builders: Fix accidental join

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Use shorter code return queue length

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

database: Fix loads of incorrect joins

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cli: Fix running installchecks

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Send log files with decompressing them if client supports it

If the browser supports gzip, we simply send the raw file as there is no
need to decompress it first and then have the reverse proxy compress it
again.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

events: Rely more on "selectin"

This seems to be slightly faster in case of larger result sets.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

web: Initialize the locale

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

web: Log in the user before checking the XSRF cookie

In case the cookie validation fails, we need to be able to send an error
message which accesses current_user.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

database: Implement custom logger that only logs debugging

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

users: Bring back the welcome push message

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

users: Don't include the TTL in the payload of the push message

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

users: Simplify passing around push messages

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

auth: Correctly set session expiry time in cookies

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sessions: Correctly set expiry time

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

users: Send push messages from the same task

This is necessary because SQLAlchemy cannot delete the push
notification from a different task.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

users: Fix sending push notifications from command line

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

users: Fix deleting push subscriptions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

users: Move JS for push notifications into the main file

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

users: Fix subscribing to push notifications

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builders: Fix showing number of running jobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builders: Store when a builder has connected last and from where

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builders: Hash connections by hostname

Since we now have one builder object per session they don't match their
hashes any more. Therefore we identify them by their name which is also
unique.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builders: Log when builders connect or disconnect

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

uploads: Actually send an empty response

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

watchers: Fetch users as selectin

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Finish relationship to points

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

CSS: Fix broken stuff by importing CSS in the correct order

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

CSS: Drop unused style.css

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builders: Show system information

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builders: Rename a couple of fields and add system

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

database: Add a helper function to trigger a commit

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Limit the amount of concurrent log tail operations

This way we will ensure that we access one file at a time and put the
others in a queue to not starve IO too hard.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Stream the log file faster

When we are downloading the entire file, we only have to decompress it
and we don't have to search for line breaks before sending each chunk.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Ratelimit access to logs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Fix deleting builds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

events: Correctly show the bug ID

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builders: Refactor auto-scaling

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Propagate back the group

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Make repos read-only

SQLAlchemy complained and wanted this flag set.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Add a CTE to fetch all currently running jobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Fix rendering bugs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Fix incorrect attribute access

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Refactor handling bugs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Make the group a relationship again

This is the only way to easily work with the group; to assign it when we
create a build.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Rename "build_group" to just "group"

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Rename "build_repo" to just "repo"

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Fix cloning a build

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

packages: Read files with a larger chunk size

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

database: Directly assign values on delete

I don't know why I implemented it like this.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mirrors: Fix editing and deleting mirrors

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mirrors: Use the correct function to format the check time

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mirrors: Fix check

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

cli: Remove reference to command to generate VAPID keys

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Fix showing logs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Fix fetch builds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Fix order after grouping

It looks like groupby() is sorting the groups before it returns them
which results in our original order being removed. We simply want to
have the most recent build at the top.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

backend: Remove unnecessary imports

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

backend: Run periodic tasks as a separate task

This is the easiest option to ensure that any changes will be committed
to the database in the end. When we run the callback in the same task,
we will never release the database session and therfore have no
automatic commit.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

users: Read the VAPID keys from the configuration file

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

build groups: Fix rendering macro

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Relax relationship between test groups

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Fix the empty state of the job queue

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Fix rendering the job index page

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Show a message if the queue is empty

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Fix rendering the queue

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Fix queries that generate the queue

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Drop message

This was an unused concept.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

database: Use joined loading a little bit more often

Since we have so many objects that have so many very close relationships
between each other, we can have lots of roundtrips to the database when
we fetch them all individually in an extra query.

The joined loads will however increase the size of the result by quite a
siginificant amount of data when applied to 1:N relationships with
larger Ns. It is pretty much free in a 1:1 relationship.

Therefore this commit changes that we fetch lots of stuff in one go so
we don't have as many roundtrips to the database. It will however result
in larger queries and potentially a little bit more bandwidth usage.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Fix running the installcheck

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Fix storing the result of the installcheck

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builders: Add helper function to find the least busy builders

This works out a little bit easier than fetching the job queue for each
individual builder.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Correctly wrap transactions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

mirrors: Add filter to show the name of an ASN

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

web: Add handler that disables authentication

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

repos: Fix rendering the mirror list

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Asynchronously create jobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jobs: Let failed default to False

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Fix wrong argument name for groups

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

builds: Correctly open a transaction

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>