]> git.ipfire.org Git - pbs.git/log
pbs.git
5 months agojobs: Update API master
Michael Tremer [Wed, 1 Nov 2023 10:07:44 +0000 (10:07 +0000)] 
jobs: Update API

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agorepos: Update API authentication
Michael Tremer [Wed, 1 Nov 2023 10:06:12 +0000 (10:06 +0000)] 
repos: Update API authentication

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agobuilders: Fix API authentication
Michael Tremer [Wed, 1 Nov 2023 10:05:51 +0000 (10:05 +0000)] 
builders: Fix API authentication

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agobuilders: Make statistics percentage value range from 0 - 1
Michael Tremer [Tue, 31 Oct 2023 11:51:07 +0000 (11:51 +0000)] 
builders: Make statistics percentage value range from 0 - 1

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agobuilders: Add a new stats handler
Michael Tremer [Tue, 31 Oct 2023 11:46:37 +0000 (11:46 +0000)] 
builders: Add a new stats handler

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agobuilds: Update API error response codes
Michael Tremer [Fri, 20 Oct 2023 09:49:05 +0000 (09:49 +0000)] 
builds: Update API error response codes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agouploads: Show if payload has been received and block sending it again
Michael Tremer [Fri, 20 Oct 2023 09:48:36 +0000 (09:48 +0000)] 
uploads: Show if payload has been received and block sending it again

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agobuilds: Require SPNEGO
Michael Tremer [Fri, 20 Oct 2023 09:34:20 +0000 (09:34 +0000)] 
builds: Require SPNEGO

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agouploads: Use errno codes
Michael Tremer [Fri, 20 Oct 2023 09:33:56 +0000 (09:33 +0000)] 
uploads: Use errno codes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agouploads: Log any incorrect digests
Michael Tremer [Fri, 20 Oct 2023 09:23:40 +0000 (09:23 +0000)] 
uploads: Log any incorrect digests

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agouploads: Align response codes
Michael Tremer [Fri, 20 Oct 2023 09:19:20 +0000 (09:19 +0000)] 
uploads: Align response codes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agouploads: Fail quickly if no payload was received
Michael Tremer [Fri, 20 Oct 2023 09:18:55 +0000 (09:18 +0000)] 
uploads: Fail quickly if no payload was received

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agouploads: Fix deleting uploads if no payload was received
Michael Tremer [Fri, 20 Oct 2023 09:17:53 +0000 (09:17 +0000)] 
uploads: Fix deleting uploads if no payload was received

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agoweb: Do not try to send (multiline) error messages in status
Michael Tremer [Fri, 20 Oct 2023 09:17:13 +0000 (09:17 +0000)] 
web: Do not try to send (multiline) error messages in status

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agoweb: Make APIError inherit from HTTPError
Michael Tremer [Thu, 19 Oct 2023 20:43:15 +0000 (20:43 +0000)] 
web: Make APIError inherit from HTTPError

This way, tornado will catch the exception and log something useful

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agoweb: Send better errors for failed upload requests
Michael Tremer [Thu, 19 Oct 2023 20:40:37 +0000 (20:40 +0000)] 
web: Send better errors for failed upload requests

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agouploads: Rewrite the whole thing
Michael Tremer [Thu, 19 Oct 2023 16:03:55 +0000 (16:03 +0000)] 
uploads: Rewrite the whole thing

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agoweb: Add decorator to require clients to do SPNEGO
Michael Tremer [Thu, 19 Oct 2023 13:25:51 +0000 (13:25 +0000)] 
web: Add decorator to require clients to do SPNEGO

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agousers: Return 0 when no quota has been used
Michael Tremer [Thu, 19 Oct 2023 13:03:29 +0000 (13:03 +0000)] 
users: Return 0 when no quota has been used

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agoconfig: Create a temporary directory
Michael Tremer [Thu, 19 Oct 2023 10:41:50 +0000 (10:41 +0000)] 
config: Create a temporary directory

As a non-privileged user, we cannot mount a ramdisk which is why we
cannot use this and need to pre-create an empty directory.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agorepos: Only set priority if configured
Michael Tremer [Thu, 19 Oct 2023 10:34:38 +0000 (10:34 +0000)] 
repos: Only set priority if configured

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agojobs: Fix quota calculation for release builds
Michael Tremer [Thu, 19 Oct 2023 10:34:08 +0000 (10:34 +0000)] 
jobs: Fix quota calculation for release builds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agouploads: Fix serialising UUIDs
Michael Tremer [Thu, 19 Oct 2023 10:33:41 +0000 (10:33 +0000)] 
uploads: Fix serialising UUIDs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agopackages: Implement sending debuginfo
Michael Tremer [Fri, 6 Oct 2023 15:02:23 +0000 (15:02 +0000)] 
packages: Implement sending debuginfo

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agoevents: Show when builds are moving repository
Michael Tremer [Wed, 20 Sep 2023 16:02:32 +0000 (16:02 +0000)] 
events: Show when builds are moving repository

This keeps the event log shorter and makes it clearer what has happened.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agousers: Show used daily build quota
Michael Tremer [Wed, 20 Sep 2023 15:43:54 +0000 (15:43 +0000)] 
users: Show used daily build quota

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agousers: Obviously we need to account for deleted builds/jobs
Michael Tremer [Wed, 20 Sep 2023 15:35:53 +0000 (15:35 +0000)] 
users: Obviously we need to account for deleted builds/jobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agousers: Introduce daily build quotas
Michael Tremer [Wed, 20 Sep 2023 15:23:54 +0000 (15:23 +0000)] 
users: Introduce daily build quotas

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agodb: Make scratch builds index more efficient
Michael Tremer [Wed, 20 Sep 2023 14:40:32 +0000 (14:40 +0000)] 
db: Make scratch builds index more efficient

This should not include any test builds.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agousers: Fix how we are calculating disk usage
Michael Tremer [Wed, 20 Sep 2023 14:37:30 +0000 (14:37 +0000)] 
users: Fix how we are calculating disk usage

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agousers: Set default storage quota for users
Michael Tremer [Wed, 20 Sep 2023 14:14:13 +0000 (14:14 +0000)] 
users: Set default storage quota for users

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agousers: Remove empty SQL statement
Michael Tremer [Wed, 20 Sep 2023 14:13:44 +0000 (14:13 +0000)] 
users: Remove empty SQL statement

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agobuilds: Fix users delete field update
Michael Tremer [Wed, 20 Sep 2023 14:13:20 +0000 (14:13 +0000)] 
builds: Fix users delete field update

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agousers: Rename quota to storage quota
Michael Tremer [Wed, 20 Sep 2023 14:08:10 +0000 (14:08 +0000)] 
users: Rename quota to storage quota

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agodb: Fix remaining index names
Michael Tremer [Wed, 20 Sep 2023 14:02:09 +0000 (14:02 +0000)] 
db: Fix remaining index names

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agodb: Drop unused repo_builds table
Michael Tremer [Wed, 20 Sep 2023 13:59:15 +0000 (13:59 +0000)] 
db: Drop unused repo_builds table

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agodb: Update users indexes
Michael Tremer [Wed, 20 Sep 2023 13:47:16 +0000 (13:47 +0000)] 
db: Update users indexes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agousers: Rename deleted to deleted_at
Michael Tremer [Wed, 20 Sep 2023 13:43:17 +0000 (13:43 +0000)] 
users: Rename deleted to deleted_at

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agojobs: Show when jobs time out
Michael Tremer [Wed, 20 Sep 2023 13:38:15 +0000 (13:38 +0000)] 
jobs: Show when jobs time out

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agojobs: Automatcially abort any jobs after three hours
Michael Tremer [Wed, 20 Sep 2023 12:16:18 +0000 (12:16 +0000)] 
jobs: Automatcially abort any jobs after three hours

If builds hang or have some other problem, we will automatically abort
them after three hours so that we don't keep the builders up idle.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agodb: Optimise efficiency of the running jobs index
Michael Tremer [Wed, 20 Sep 2023 12:15:28 +0000 (12:15 +0000)] 
db: Optimise efficiency of the running jobs index

This does not need to contain any deleted jobs and therefore can be
slightly smaller.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agopackages: Allow searching for files with wildcards
Michael Tremer [Mon, 18 Sep 2023 10:04:35 +0000 (10:04 +0000)] 
packages: Allow searching for files with wildcards

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agojobs: Fix sorting of the queue
Michael Tremer [Sun, 17 Sep 2023 12:36:39 +0000 (12:36 +0000)] 
jobs: Fix sorting of the queue

Scratch builds are now being moved to the top of the queue and packages
will be built by when their dependency check has succeeded.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agoFix Kerberos authentication
Michael Tremer [Sun, 17 Sep 2023 12:25:10 +0000 (12:25 +0000)] 
Fix Kerberos authentication

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agobuilds: Add controls to approve builds
Michael Tremer [Sun, 17 Sep 2023 09:43:24 +0000 (09:43 +0000)] 
builds: Add controls to approve builds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agobuilds: Move repos above test builds
Michael Tremer [Sun, 17 Sep 2023 08:13:51 +0000 (08:13 +0000)] 
builds: Move repos above test builds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agobuilds: Rewrite all relevant repositories when deleting builds
Michael Tremer [Sun, 17 Sep 2023 08:06:32 +0000 (08:06 +0000)] 
builds: Rewrite all relevant repositories when deleting builds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agobuilds: Remove unnecessary task group when deleting builds
Michael Tremer [Sun, 17 Sep 2023 08:05:44 +0000 (08:05 +0000)] 
builds: Remove unnecessary task group when deleting builds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agodatabase: Allow up to 1024 concurrent database connections
Michael Tremer [Sun, 17 Sep 2023 08:05:13 +0000 (08:05 +0000)] 
database: Allow up to 1024 concurrent database connections

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agologstream: Fix sending the last messages
Michael Tremer [Wed, 13 Sep 2023 17:44:43 +0000 (17:44 +0000)] 
logstream: Fix sending the last messages

Instead, the first messages in the buffer were sent.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agobuilds: Delete builds one step at a time
Michael Tremer [Wed, 13 Sep 2023 17:43:55 +0000 (17:43 +0000)] 
builds: Delete builds one step at a time

Creating a lot of sub-tasks caused some SQL statements to be executed
outside its transaction and the connection pool to starve.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agojobs: Fix UUID comparison that differs in type
Michael Tremer [Wed, 13 Sep 2023 15:52:19 +0000 (15:52 +0000)] 
jobs: Fix UUID comparison that differs in type

We use UUID for PostgreSQL and now receive the UUID in Python's UUID
type which surprisingly does not compare to strings...

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agodatabase: Increase the maximum number of connections again
Michael Tremer [Wed, 13 Sep 2023 15:41:10 +0000 (15:41 +0000)] 
database: Increase the maximum number of connections again

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agojobs: Import packages one by one
Michael Tremer [Wed, 13 Sep 2023 15:40:45 +0000 (15:40 +0000)] 
jobs: Import packages one by one

This could have caused the appliance to run out of available database
connections.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agomisc: Fix format_size from running out of sizes
Michael Tremer [Wed, 13 Sep 2023 15:40:18 +0000 (15:40 +0000)] 
misc: Fix format_size from running out of sizes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agobackend: Move cleanup job into the main thread
Michael Tremer [Sat, 9 Sep 2023 13:50:18 +0000 (13:50 +0000)] 
backend: Move cleanup job into the main thread

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agocron: Remove unneeded cronjobs
Michael Tremer [Sat, 9 Sep 2023 13:49:15 +0000 (13:49 +0000)] 
cron: Remove unneeded cronjobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agodatabase: Close connections after 5 seconds
Michael Tremer [Sat, 9 Sep 2023 13:49:02 +0000 (13:49 +0000)] 
database: Close connections after 5 seconds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agobackend: Sync mirrors from the main thread
Michael Tremer [Sat, 9 Sep 2023 13:47:13 +0000 (13:47 +0000)] 
backend: Sync mirrors from the main thread

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agobackend: Define background tasks here
Michael Tremer [Sat, 9 Sep 2023 13:45:56 +0000 (13:45 +0000)] 
backend: Define background tasks here

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agodatabase: Limit the maximum number of database connections to 64
Michael Tremer [Fri, 8 Sep 2023 16:10:10 +0000 (16:10 +0000)] 
database: Limit the maximum number of database connections to 64

Our servers are configured to only allow up to 100 connections total
which had funny effects here...

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agojobs: Show when jobs have been aborted in queue
Michael Tremer [Fri, 8 Sep 2023 14:53:15 +0000 (14:53 +0000)] 
jobs: Show when jobs have been aborted in queue

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agomonitorings: Perform checks one at a time
Michael Tremer [Fri, 8 Sep 2023 14:49:48 +0000 (14:49 +0000)] 
monitorings: Perform checks one at a time

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agoconfig: Send "noarch" as build architecture to builders
Michael Tremer [Fri, 8 Sep 2023 14:49:01 +0000 (14:49 +0000)] 
config: Send "noarch" as build architecture to builders

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agomonitoring: Catch errors when we cannot create a build
Michael Tremer [Fri, 8 Sep 2023 14:48:35 +0000 (14:48 +0000)] 
monitoring: Catch errors when we cannot create a build

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agosources: Add an extra barrier if we are missing files
Michael Tremer [Sat, 2 Sep 2023 13:09:45 +0000 (13:09 +0000)] 
sources: Add an extra barrier if we are missing files

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agoauth: Revert back to authentication using a web form
Michael Tremer [Sat, 2 Sep 2023 12:32:28 +0000 (12:32 +0000)] 
auth: Revert back to authentication using a web form

Using Kerberos authentication for this part seems to be more complicated
than necessary and does not provide any benefits.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agojobs: Force builders to refresh repository data all the time
Michael Tremer [Sat, 2 Sep 2023 09:30:23 +0000 (09:30 +0000)] 
jobs: Force builders to refresh repository data all the time

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agosync: Write timestamp so that we can track when mirrors were synced last
Michael Tremer [Fri, 1 Sep 2023 16:16:33 +0000 (16:16 +0000)] 
sync: Write timestamp so that we can track when mirrors were synced last

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agomirrors: Fix joining paths
Michael Tremer [Fri, 1 Sep 2023 16:08:54 +0000 (16:08 +0000)] 
mirrors: Fix joining paths

The original path of the mirror was unintentionally dropped.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agopackages: Store groups as string array
Michael Tremer [Thu, 31 Aug 2023 03:28:27 +0000 (03:28 +0000)] 
packages: Store groups as string array

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agopackages: Store build ID as UUID
Michael Tremer [Wed, 30 Aug 2023 18:07:43 +0000 (18:07 +0000)] 
packages: Store build ID as UUID

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agopackages: Store build arches and apply them when creating jobs
Michael Tremer [Wed, 30 Aug 2023 17:49:26 +0000 (17:49 +0000)] 
packages: Store build arches and apply them when creating jobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agosources: Only check out those files that we need
Michael Tremer [Sat, 26 Aug 2023 11:18:22 +0000 (11:18 +0000)] 
sources: Only check out those files that we need

Since the entire build service is very heavy on I/O, we should avoid
writing files that we don't need.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agojobs: Create an index over the superseeded_by field
Michael Tremer [Sat, 26 Aug 2023 11:14:33 +0000 (11:14 +0000)] 
jobs: Create an index over the superseeded_by field

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agosources: Capture & store Pakfire log on dist()
Michael Tremer [Sat, 26 Aug 2023 11:14:03 +0000 (11:14 +0000)] 
sources: Capture & store Pakfire log on dist()

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agoconfig: Add PakfireLogger class
Michael Tremer [Sat, 26 Aug 2023 11:12:10 +0000 (11:12 +0000)] 
config: Add PakfireLogger class

This class can be used to capture the log from Pakfire in order to debug
issues easier.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agobuilders: Fix styling of edit page
Michael Tremer [Sat, 26 Aug 2023 10:11:31 +0000 (10:11 +0000)] 
builders: Fix styling of edit page

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agojobs: Call dispatch after launching any jobs
Michael Tremer [Fri, 18 Aug 2023 16:01:11 +0000 (16:01 +0000)] 
jobs: Call dispatch after launching any jobs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agosources: Fix call to launch builds
Michael Tremer [Fri, 18 Aug 2023 16:00:52 +0000 (16:00 +0000)] 
sources: Fix call to launch builds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agorepo: Add simple API operations
Michael Tremer [Fri, 18 Aug 2023 15:27:31 +0000 (15:27 +0000)] 
repo: Add simple API operations

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agorepos: Don't remove builds that have been removed before
Michael Tremer [Fri, 18 Aug 2023 15:27:16 +0000 (15:27 +0000)] 
repos: Don't remove builds that have been removed before

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agojobs: Reset jobs as successful if following jobs succeed
Michael Tremer [Fri, 18 Aug 2023 14:11:39 +0000 (14:11 +0000)] 
jobs: Reset jobs as successful if following jobs succeed

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agojobs: Show logs of previous attempts
Michael Tremer [Fri, 18 Aug 2023 14:08:57 +0000 (14:08 +0000)] 
jobs: Show logs of previous attempts

Fixes: #13249 - jobs: Add option to download logs from previous attempts
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agobuilds: Immediately launch builds in the transaction
Michael Tremer [Fri, 18 Aug 2023 10:05:02 +0000 (10:05 +0000)] 
builds: Immediately launch builds in the transaction

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agopackages: Read archive metadata in executor thread
Michael Tremer [Fri, 18 Aug 2023 09:53:52 +0000 (09:53 +0000)] 
packages: Read archive metadata in executor thread

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agopackages: Fail more gracefully if a package has been deleted from disk
Michael Tremer [Fri, 18 Aug 2023 09:53:30 +0000 (09:53 +0000)] 
packages: Fail more gracefully if a package has been deleted from disk

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agojobs: Re-add accidentially removed import of the queue module
Michael Tremer [Fri, 18 Aug 2023 09:52:28 +0000 (09:52 +0000)] 
jobs: Re-add accidentially removed import of the queue module

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agodatabase: Don't keep so many connections open and idle
Michael Tremer [Fri, 18 Aug 2023 09:51:10 +0000 (09:51 +0000)] 
database: Don't keep so many connections open and idle

Since we usually peak briefly, we will fork a lot of PostgreSQL
processes that we will never need again. In order to avoid wasting too
much memory, we close those connections quicker, but at least 8 open so
that new tasks will quickly be able to acquire a database connection.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agobuilds+jobs: Use new methods to access database objects
Michael Tremer [Fri, 18 Aug 2023 09:50:51 +0000 (09:50 +0000)] 
builds+jobs: Use new methods to access database objects

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agojobs: Show message if no jobs are queued
Michael Tremer [Fri, 18 Aug 2023 09:50:07 +0000 (09:50 +0000)] 
jobs: Show message if no jobs are queued

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agopackages: Log deleting packages
Michael Tremer [Thu, 17 Aug 2023 11:06:43 +0000 (11:06 +0000)] 
packages: Log deleting packages

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agobuilds: Try to delete packages if creating build fails
Michael Tremer [Thu, 17 Aug 2023 11:05:18 +0000 (11:05 +0000)] 
builds: Try to delete packages if creating build fails

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agopackages: Don't try to overwrite existing packages
Michael Tremer [Thu, 17 Aug 2023 11:05:03 +0000 (11:05 +0000)] 
packages: Don't try to overwrite existing packages

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agojobs: Remove some unnecessary imports
Michael Tremer [Thu, 17 Aug 2023 11:02:09 +0000 (11:02 +0000)] 
jobs: Remove some unnecessary imports

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agopackages: Extract filelists in a separate thread
Michael Tremer [Thu, 17 Aug 2023 10:17:34 +0000 (10:17 +0000)] 
packages: Extract filelists in a separate thread

Since we need to read the entire archive to extract the entire filelist
with all its meta information, we need to move this into a separate
thread.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agojobs: Tail logs more efficiently
Michael Tremer [Thu, 17 Aug 2023 10:16:54 +0000 (10:16 +0000)] 
jobs: Tail logs more efficiently

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agosources: Automatically launch builds after dist
Michael Tremer [Thu, 17 Aug 2023 09:48:50 +0000 (09:48 +0000)] 
sources: Automatically launch builds after dist

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agocache: Tidy up code
Michael Tremer [Fri, 11 Aug 2023 14:49:58 +0000 (14:49 +0000)] 
cache: Tidy up code

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agoweb: Create a simple rate limiter
Michael Tremer [Fri, 11 Aug 2023 14:45:56 +0000 (14:45 +0000)] 
web: Create a simple rate limiter

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>