]>
git.ipfire.org Git - pbs.git/log
Michael Tremer [Sat, 5 Aug 2023 10:11:43 +0000 (10:11 +0000)]
release monitoring: Do not try to update a bug that has never been created
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 5 Aug 2023 10:11:09 +0000 (10:11 +0000)]
cache: Implement an async redis cache driver
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 2 Aug 2023 17:17:36 +0000 (17:17 +0000)]
jobs: Launch installcheck in transactions and wait for it
It can happen that a transaction is being rolled back and an
installcheck has been queued which will then run into errors.
Since the installcheck is now fast enough, we can simply wait for it
with the exception of relaunching any pending jobs in repositories which
will be run as one single task in the background.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 2 Aug 2023 17:02:19 +0000 (17:02 +0000)]
monitoring: Put the rate limiter in the correct place
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 2 Aug 2023 17:01:06 +0000 (17:01 +0000)]
monitoring: Complete configuration before running the first check
Because we are communicating with Bugzilla (potentially) we might undo a
lot of stuff if there is any error.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 2 Aug 2023 15:36:26 +0000 (15:36 +0000)]
config: Pass configuration as file object
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 1 Aug 2023 17:18:19 +0000 (17:18 +0000)]
config: Make Pakfire an async context manager
This should help us to block less when initializing a Pakfire instance.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 1 Aug 2023 17:17:58 +0000 (17:17 +0000)]
users: Show controls to create repos when none exist
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 1 Aug 2023 16:58:59 +0000 (16:58 +0000)]
release monitoring: Use new version compare function
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 1 Aug 2023 16:47:29 +0000 (16:47 +0000)]
backend: Use aiofiles for asyncio handling of temporary files
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 1 Aug 2023 16:31:07 +0000 (16:31 +0000)]
index: Show recently finished jobs instead of pending ones
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 1 Aug 2023 16:23:09 +0000 (16:23 +0000)]
builds: Move job queue
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 1 Aug 2023 16:16:37 +0000 (16:16 +0000)]
jobs: Add option to only show failed builds
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 1 Aug 2023 16:03:13 +0000 (16:03 +0000)]
jobs: queue: Show build time for finished jobs
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 1 Aug 2023 15:52:39 +0000 (15:52 +0000)]
jobs: queue: Show status for failed/finished jobs
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 1 Aug 2023 15:50:06 +0000 (15:50 +0000)]
jobs: Group by date
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 1 Aug 2023 15:47:38 +0000 (15:47 +0000)]
jobs: Add pagination to new page
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 1 Aug 2023 15:45:03 +0000 (15:45 +0000)]
jobs: Create new page that lists all finished jobs
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 31 Jul 2023 17:19:48 +0000 (17:19 +0000)]
web: Create UI module for queued jobs
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 28 Jul 2023 09:24:46 +0000 (09:24 +0000)]
jobs: Fix indefinite loop in dispatch function
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 25 Jul 2023 15:05:46 +0000 (15:05 +0000)]
builds: Fix SQL query to fetch all release/scratch builds
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 25 Jul 2023 14:15:50 +0000 (14:15 +0000)]
logstreams: Increase the maximum number of log messages stored
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 25 Jul 2023 14:15:08 +0000 (14:15 +0000)]
logstream: Lock buffer when it is being modified
This might be cheaper than copying it when it becomes large.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 25 Jul 2023 14:12:18 +0000 (14:12 +0000)]
Revert "logstreams: Don't wait until all messages have been delivered to all consumers"
This reverts commit
1fa487c5455f3e4a63ae0755c466c2a7ee62b4e5 .
This might potentially shuffle the messages :(
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 25 Jul 2023 14:10:02 +0000 (14:10 +0000)]
repos: Run fewer installchecks
Only run this for jobs that have previously failed.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 25 Jul 2023 14:08:57 +0000 (14:08 +0000)]
jobs: Perform installcheck before a job is being dispatched
I would like to perform fewer of the installchecks because they unlikely
to fail and rather expensive to run.
This is a safety net so that we will never dispatch any jobs that cannot
be built.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 25 Jul 2023 14:02:52 +0000 (14:02 +0000)]
logstreams: Copy buffer before sending it off
collections.deque() does not allow us to append new messages while we
are reading from it.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 25 Jul 2023 14:01:28 +0000 (14:01 +0000)]
logstreams: Don't wait until all messages have been delivered to all consumers
This will just queue tasks for all new messages and allows us to get
back to processing the next message quicker.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Jul 2023 18:17:05 +0000 (18:17 +0000)]
builders: Return zero for total build time if there are no builds
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Jul 2023 18:16:53 +0000 (18:16 +0000)]
builders: Include disabled builders in autoscale
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Jul 2023 17:42:12 +0000 (17:42 +0000)]
repositories: Master repositories as async as possible
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Jul 2023 17:21:25 +0000 (17:21 +0000)]
robots.txt: Disallow download files from packages
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Jul 2023 15:17:11 +0000 (15:17 +0000)]
web: make_url: Filter out any values that are None
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Jul 2023 15:14:39 +0000 (15:14 +0000)]
builds: Commit forgotten bug templates
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Jul 2023 15:12:54 +0000 (15:12 +0000)]
packages: Use the more efficient version comparison function
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Jul 2023 15:12:43 +0000 (15:12 +0000)]
database: Close connections sooner
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Jul 2023 15:12:16 +0000 (15:12 +0000)]
repos: Move database operations into the main thread
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Jul 2023 15:11:52 +0000 (15:11 +0000)]
database: Replace more occurences of the UUID type
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Jul 2023 13:31:31 +0000 (13:31 +0000)]
database: Move all sorts of database operations into the main thread
We no longer support any database operations from any other thread than
the main thread (because otherwise we cannot figure out which Task we
are running in).
For that reason, we have to split a couple of tasks off at different
times and send blocking operations to an executor thread later.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Jul 2023 13:30:12 +0000 (13:30 +0000)]
database: psycopg3 is returning UUIDs as UUID type instead of string
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Jul 2023 10:22:44 +0000 (10:22 +0000)]
release monitoring: Fix cleaning up the repository if none was created
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Jul 2023 10:16:51 +0000 (10:16 +0000)]
database: Replace the old database model with a connection pool
This patch introduces a connection pool to the database. When a function
is executing a query, a connection will be handed to the (asyncio) Task
and remains with that task until its end.
That way, multiple concurrent tasks (even when they are running in the
same thread) will have an exclusive connection to the database which
will avoid that we overwrite and commit transactions of other tasks.
Additionally, transactions can now be nested as psycopg's transactions
are more advanced than the previous Transaction class.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Jul 2023 09:54:07 +0000 (09:54 +0000)]
backend: Move temporary tables into CTEs
It was a nice idea, but if we want to pool connections, we will only
create the temporary tables on one connection with the others losing
out.
An alternative could have been to create all tables on all connections
first, but I suppose that is going to create a lot of unnecessary
overhead for short-lived connections.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Jul 2023 09:52:35 +0000 (09:52 +0000)]
web: jobs: Add HTTP status code to exception
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 20 Jul 2023 13:31:32 +0000 (13:31 +0000)]
database: Remove aliases for common exceptions
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 20 Jul 2023 10:44:34 +0000 (10:44 +0000)]
database: Migrate to psycopg3
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 20 Jul 2023 10:16:55 +0000 (10:16 +0000)]
builds: Show better tags
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 20 Jul 2023 10:16:12 +0000 (10:16 +0000)]
CSS: Don't use our custom red as it clashes with the primary colour
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 20 Jul 2023 10:15:34 +0000 (10:15 +0000)]
sources: Fix rendering page when source has never been fetched
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 20 Jul 2023 10:15:05 +0000 (10:15 +0000)]
sources: Create dist jobs for all packages on the initial commit import
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 20 Jul 2023 10:14:33 +0000 (10:14 +0000)]
packages: Store empty strings if summary/description are empty
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 20 Jul 2023 10:14:04 +0000 (10:14 +0000)]
db: Sync schema
I don't quite know why the diff is so large when there have been only a
few fields changed...
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 20 Jul 2023 10:11:38 +0000 (10:11 +0000)]
builders: Catch the correct exception on disconnect
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 20 Jul 2023 10:10:26 +0000 (10:10 +0000)]
backend: Avoid removing the tmp directory
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 14 Jun 2023 14:19:38 +0000 (14:19 +0000)]
distros: Actually filter the latest release
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 14 Jun 2023 14:18:17 +0000 (14:18 +0000)]
releases: Implement controls to publish
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 14 Jun 2023 14:12:42 +0000 (14:12 +0000)]
releases: "Publish" instead of "Release"
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 14 Jun 2023 14:03:52 +0000 (14:03 +0000)]
distros: Link to releases
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 14 Jun 2023 13:53:34 +0000 (13:53 +0000)]
releases: Implement controls to delete a release
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 14 Jun 2023 13:46:04 +0000 (13:46 +0000)]
events: Add releases
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 14 Jun 2023 13:27:38 +0000 (13:27 +0000)]
releases: Add controls to create and edit releases
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 14 Jun 2023 13:27:01 +0000 (13:27 +0000)]
release: Drop reference to repositories
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 14 Jun 2023 10:44:53 +0000 (10:44 +0000)]
releases: Change prerelease to stable
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 14 Jun 2023 10:05:21 +0000 (10:05 +0000)]
distros: Add scaffolding for releases
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 14 Jun 2023 08:53:12 +0000 (08:53 +0000)]
sources: Add detail page for commits
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 14 Jun 2023 08:52:28 +0000 (08:52 +0000)]
sources: Add any watchers from commits to builds
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 14 Jun 2023 08:52:08 +0000 (08:52 +0000)]
sources: Refactor Commit class
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 14 Jun 2023 08:51:06 +0000 (08:51 +0000)]
sources: Fix wrong table name in SQL query
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 14 Jun 2023 08:50:36 +0000 (08:50 +0000)]
web: Text: Implement showing pre-formatted text
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 13 Jun 2023 16:26:55 +0000 (16:26 +0000)]
web: Drop unused source modules
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 13 Jun 2023 15:02:47 +0000 (15:02 +0000)]
sources: Remove old state column
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 13 Jun 2023 14:58:09 +0000 (14:58 +0000)]
sources: Run jobs only once
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 7 Jun 2023 12:32:39 +0000 (12:32 +0000)]
sources: Run jobs after fetching commits
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 7 Jun 2023 10:46:55 +0000 (10:46 +0000)]
builds: Move test builds view into code
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 7 Jun 2023 10:39:33 +0000 (10:39 +0000)]
repos: Fix deleting user repositories
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 7 Jun 2023 10:39:03 +0000 (10:39 +0000)]
repos: Remove all builds before deletion
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 7 Jun 2023 10:27:02 +0000 (10:27 +0000)]
packages: Refactor search
This patch drops the materialized view as it becomes painfully slow to
update. Instead we store the search data as an auto-generated row and
put a GIN index over it.
This should be sufficiently fast even with a large number of packages in
the database.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 7 Jun 2023 10:26:21 +0000 (10:26 +0000)]
packages: Fix search for filenames
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 6 Jun 2023 16:25:46 +0000 (16:25 +0000)]
users: Add binary packages to the disk usage
This patch also creates an index to search through all scratch builds
quicker.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 6 Jun 2023 16:18:35 +0000 (16:18 +0000)]
users: Move disk usage view into the code
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 6 Jun 2023 16:12:28 +0000 (16:12 +0000)]
jobs: Move estimated build time view into code
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 6 Jun 2023 16:04:06 +0000 (16:04 +0000)]
index: Remove log stream
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 6 Jun 2023 15:55:59 +0000 (15:55 +0000)]
release monitoring: Show the follow strategy
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 6 Jun 2023 15:31:51 +0000 (15:31 +0000)]
release monitoring: Implement following the current branch
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 6 Jun 2023 14:29:39 +0000 (14:29 +0000)]
release monitoring: Implement build check
This is so that we won't import any releases that are older than what we
already know.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 6 Jun 2023 14:15:56 +0000 (14:15 +0000)]
release monitoring: Implement following stable releases and the latest
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sun, 4 Jun 2023 11:32:01 +0000 (11:32 +0000)]
packages: Use misc.group() to group packages
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sun, 4 Jun 2023 11:30:23 +0000 (11:30 +0000)]
packages: Fix rendering page when there are no scratch builds
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sun, 4 Jun 2023 11:27:41 +0000 (11:27 +0000)]
builds: Show scratch builds for packages and enhance search
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sun, 4 Jun 2023 11:14:51 +0000 (11:14 +0000)]
web: Build function to easily compose URLs
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 2 Jun 2023 15:59:43 +0000 (15:59 +0000)]
packages: Move name page into its own directory
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 2 Jun 2023 11:39:02 +0000 (11:39 +0000)]
builders: Show status on detail page
Since we are polling the API now, we may as well show the status
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 2 Jun 2023 11:35:49 +0000 (11:35 +0000)]
builders: Add controls to manually start/stop builders
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 2 Jun 2023 11:25:52 +0000 (11:25 +0000)]
builders: Drop sync with AWS
This does nothing apart from logging things.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 1 Jun 2023 21:19:22 +0000 (21:19 +0000)]
keys: Drop download handler
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 1 Jun 2023 21:18:22 +0000 (21:18 +0000)]
keys: Require comments
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 1 Jun 2023 20:45:28 +0000 (20:45 +0000)]
keys: Add comments
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 1 Jun 2023 20:40:04 +0000 (20:40 +0000)]
keys: Refactor with new
Ed25519 keys
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 29 May 2023 14:42:06 +0000 (14:42 +0000)]
builds: Refactor deleting UI
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 29 May 2023 14:32:58 +0000 (14:32 +0000)]
backend: Replace asyncio.gather() with task groups
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>