Robin Jarry [Mon, 8 Jun 2026 11:59:43 +0000 (13:59 +0200)]
parsemail: wrap parse_mail() in a single transaction
Wrap the entire parse_mail() call in transaction.atomic() so that all
database writes from email parsing run inside a single transaction.
The existing transaction.atomic() blocks inside parse_mail() become
savepoints within this outer transaction. The series deduplication
retry logic continues to work since savepoint rollbacks are scoped
to their own savepoint.
This also ensures that any on_commit() callbacks registered by signal
handlers only fire after the full email has been parsed and all
patch/series associations are committed.
Signed-off-by: Robin Jarry <robin@jarry.cc> Signed-off-by: Stephen Finucane <stephen@that.guru> Reviewed-by: Stephen Finucane <stephen@that.guru>
[stephenfin: Slight tweaks to the release notes]
Robin Jarry [Mon, 8 Jun 2026 12:00:55 +0000 (14:00 +0200)]
parsemail: fix SeriesReference race with concurrent delivery
When multiple parsemail processes run in parallel (e.g. postfix
delivering several messages from the same series at once), two
processes can try to create a SeriesReference for the same msgid
simultaneously. The second one fails with an IntegrityError:
This can result in incomplete series that never reach the
"received_all" state because the failed parsemail invocation
prevents one of the patches from being recorded.
The existing get/create pattern has a classic TOCTOU race: the get
succeeds (no reference found), but by the time create runs, another
process has already inserted the row. Replace both the try/get/
except/create block and the bare create call with get_or_create
which handles the race atomically at the database level.
Signed-off-by: Robin Jarry <robin@jarry.cc> Reviwed-by: Stephen Finucane <stephen@that.guru>
Stephen Finucane [Thu, 13 Sep 2018 17:13:20 +0000 (11:13 -0600)]
Add a Makefile
Time to go old school. When using Docker, we can't simply use tox or
manage.py command as commands should run inside the container and not on
the host. To resolve this, we recommend using 'docker-compose run --rm
web tox', which will run tox in the container. Having to remember these
rather esoteric commands is annoying though (particularly when I haven't
run it recently and therefore can't find the command with '<Ctrl>+R'). A
series of make targets make this a heck of a lot easier for us.
Note that this isn't ideal and won't entirely remove the need to run the
above command. This is mostly due to how make is designed: namely, we
need to pass positional arguments but each positional argument in make
should be a target. This does, however, cover the biggest use cases and
is therefore worth doing.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Currently, the web UI allows any logged in user to remove patches from
public bundles. However the correct behaviour is that only the owner of
the bundle should be allowed to update a bundle.
Fix that by adding checks in set_bundle() before adding or removing
patches from bundles.
notifications.py: don't crash if notifications recipient is invalid
On Django 3:3.2.19, if recipient is not filled or is invalid, cron
job fails with:
Traceback (most recent call last):
File "/usr/local/patchwork/./manage.py", line 17, in <module>
execute_from_command_line(sys.argv)
File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
utility.execute()
File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 413, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 354, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 398, in execute
output = self.handle(*args, **options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/patchwork/patchwork/management/commands/cron.py", line 19, in handle
errors = send_notifications()
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/patchwork/patchwork/notifications.py", line 84, in send_notifications
message.send()
File "/usr/lib/python3/dist-packages/django/core/mail/message.py", line 284, in send
return self.get_connection(fail_silently).send_messages([self])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/core/mail/backends/smtp.py", line 109, in send_messages
sent = self._send(message)
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/core/mail/backends/smtp.py", line 122, in _send
recipients = [sanitize_address(addr, encoding) for addr in email_message.recipients()]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/core/mail/backends/smtp.py", line 122, in <listcomp>
recipients = [sanitize_address(addr, encoding) for addr in email_message.recipients()]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/core/mail/message.py", line 99, in sanitize_address
address_parts = nm + localpart + domain
~~~^~~~~~~~~~~
TypeError: can only concatenate str (not "NoneType") to str
As there's no point trying to send a notification to an invalid
e-mail, just drops it.
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org> Co-authored-by: Stephen Finucane <stephen@that.guru>
[stephenfin: Updated to fix linter issues]
The current logic applies checks color only once. That's
weird and makes harder for people to properly idenfify
CI checks when there are multiple CI reports at the same line.
MySQL 9.7 changed the default gtid_mode from OFF to ON (since 9.5). When
the MySQL Docker container initialises, the init scripts run with
GTID=ON, leaving non-empty GTID_EXECUTED in the binary log even though
the final server starts with --gtid-mode=OFF.
This causes two problems:
1. mysqldump always includes SET @@GLOBAL.GTID_PURGED when cloning
test databases for parallel tests. Restoring this requires
SYSTEM_VARIABLES_ADMIN, which was not granted.
2. mysqldump requires PROCESS to dump tablespace information. While
PROCESS was being granted, SYSTEM_VARIABLES_ADMIN was missing
from the grants, causing clone failures (exit code 5) for Django
6.0's stricter parallel test runner.
Fix both issues by:
- Running RESET BINARY LOGS AND GTIDS before granting privileges,
which clears the GTID state left by the init phase so mysqldump
no longer emits GTID_PURGED statements.
- Adding SYSTEM_VARIABLES_ADMIN to the privilege grants so that if
any GTID state is present (e.g. in future MySQL versions), the
restore can set GTID_PURGED.
Signed-off-by: Stephen Finucane <stephen@that.guru> Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
docker: Fix compat with latest patchwork/pyenv image
We switched to the Ubuntu 24.04 base image some time back. This includes
an ubuntu user that conflicts with the patchwork user we were creating.
Simplify this by using the ubuntu user instead.
Signed-off-by: Stephen Finucane <stephen@that.guru>
psycopg (v3) strictly rejects NUL bytes (0x00) in PostgreSQL text
fields, unlike psycopg2 which handled them silently. Malformed emails
(such as the codec-null.mbox fuzz fixture in our test suite) contain NUL
bytes in headers and body content, causing DataError on insert.
Strip NUL bytes from all text fields before they reach the database.
This is safe for all fields:
* Headers are restricted to printable ASCII by RFC 5322 meaning NUL
bytes there always indicate corruption
* Email body / commit message content is plain text so NUL bytes are
equally invalid here
* Unified diffs should also never contain NUL bytes. Git uses their
presence as the heuristic to classify a file as binary, at which point
it either emits 'Binary files ... differ' or, with --binary, a
base85-encoded binary patch, both of which are entirely printable
ASCII. A legitimate patch produced by git format-patch will therefore
never carry NUL bytes in the diff text itself.
Signed-off-by: Stephen Finucane <stephen@that.guru>
We also change our testing matrix so that we only test the upper and
lower bounds of supported Python releases. This should ease pressure on
CI somewhat. We also switch from the deprecated psycopg2 to psycopg
(which actually corresponds to v3 now).
Changes to the source were auto-generated with django-upgrade.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Adam Hassick [Thu, 30 Jan 2025 19:56:42 +0000 (14:56 -0500)]
release-notes: Add release note
Signed-off-by: Adam Hassick <ahassick@iol.unh.edu> Acked-by: Aaron Conole <aconole@redhat.com>
[stephenfin: Rename Project field from parse_dependencies to
show_dependencies] Signed-off-by: Stephen Finucane <stephen@that.guru>
Adam Hassick [Thu, 30 Jan 2025 19:56:41 +0000 (14:56 -0500)]
docs: Add generated API schemas
Signed-off-by: Adam Hassick <ahassick@iol.unh.edu> Acked-by: Aaron Conole <aconole@redhat.com>
[stephenfin: Rename Project field from parse_dependencies to
show_dependencies.] Signed-off-by: Stephen Finucane <stephen@that.guru>
Adam Hassick [Thu, 30 Jan 2025 19:56:40 +0000 (14:56 -0500)]
docs: Add feature info and increment API version
The following patch will add the generated schema files as they are very
large.
* Increment the API version from v1.3 to v1.4.
* Update schema documentation to reflect the version change.
* Add blurb documenting the "Depends-on" tag to the usage overview.
Signed-off-by: Adam Hassick <ahassick@iol.unh.edu> Acked-by: Aaron Conole <aconole@redhat.com>
[stephenfin: Tweak docs] Signed-off-by: Stephen Finucane <stephen@that.guru>
Adam Hassick [Thu, 30 Jan 2025 19:56:39 +0000 (14:56 -0500)]
tests: Add tests for new functionality
* Add tests for the API and parser changes.
* Add new test mbox files.
* Add new patch series tests.
* Add test to check that dependencies are not shown when disabled.
Signed-off-by: Adam Hassick <ahassick@iol.unh.edu> Acked-by: Aaron Conole <aconole@redhat.com>
[stephenfin: Rename Project field from parse_dependencies to
show_dependencies. Add some additional tests] Signed-off-by: Stephen Finucane <stephen@that.guru>
Adam Hassick [Thu, 30 Jan 2025 19:56:38 +0000 (14:56 -0500)]
parser: Parse "Depends-on" tags in emails
Add a new function to parse "Depends-on" tags to the parser. The value
may either be the message ID of a patch or cover letter email already
received by Patchwork, or the web URL of a patch or series. When this
tag is found, the parser will add the series (or the series the patch
belongs to) as a dependency to the series it is creating.
This parser feature is only active when the feature flag is enabled on
the project related to the patch or cover letter.
Signed-off-by: Adam Hassick <ahassick@iol.unh.edu> Acked-by: Aaron Conole <aconole@redhat.com>
[stephenfin: Parse patch dependencies unconditionally] Signed-off-by: Stephen Finucane <stephen@that.guru>
Adam Hassick [Thu, 30 Jan 2025 19:56:37 +0000 (14:56 -0500)]
api: Add fields to series detail view
* Add the "dependencies" and "dependents" fields to the series detail
view.
* Add "parse_dependencies" to the project detail view.
Signed-off-by: Adam Hassick <ahassick@iol.unh.edu> Acked-by: Aaron Conole <aconole@redhat.com>
[stephenfin: Rename Project field from parse_dependencies to
show_dependencies] Signed-off-by: Stephen Finucane <stephen@that.guru>
Adam Hassick [Thu, 30 Jan 2025 19:56:36 +0000 (14:56 -0500)]
models: Add fields for series dependencies
* Add a ManyToMany field to represent a dependency relationship between
patch series and a helper method to add dependencies.
* Add the parse_dependency field to the Project model.
Signed-off-by: Adam Hassick <ahassick@iol.unh.edu> Acked-by: Aaron Conole <aconole@redhat.com>
[stephenfin: Rename Project field from parse_dependencies to
show_dependencies. Also add 'blank=True' to
Series.cover_letter field] Signed-off-by: Stephen Finucane <stephen@that.guru>
Updates the requirements on [django-debug-toolbar](https://github.com/django-commons/django-debug-toolbar) to permit the latest version.
- [Release notes](https://github.com/django-commons/django-debug-toolbar/releases)
- [Changelog](https://github.com/django-commons/django-debug-toolbar/blob/main/docs/changes.rst)
- [Commits](https://github.com/django-commons/django-debug-toolbar/compare/4.4.6...5.0.1)
Raxel Gutierrez [Mon, 23 Aug 2021 18:28:32 +0000 (18:28 +0000)]
views: Style modification forms as an action bar
Add styling to the new patch list html code to make the change property
and bundle action forms more usable. Before [1] and after [2] images for
reference.
Raxel Gutierrez [Mon, 23 Aug 2021 18:28:31 +0000 (18:28 +0000)]
views: Move and refactor patch-forms
Move patch forms in patch-list and detail page to a new template file
patch-forms.html and move them to the top of the patch-list page to
improve their discoverability.
Refactor forms.py, __init__.py, patch.py, and test_bundles.py files so
that the shared bundle form in patch-forms.html works for both the
patch-list and patch-detail pages. In particular, the changes normalize
the behavior of the error and update messages of the patch forms and
updates tests to reflect the changes. Overall, these changes make patch
forms ready for change and more synchronized in their behavior. More
specifically:
- Previously patch forms changes were separated between the patch-detail
and patch-list pages. Thus, desired changes to the patch forms
required changes to patch-list.html, submission.html, and forms.py.
So, the most important benefit to this change is that forms.py and
patch-forms.html become the two places to adjust the forms to handle
form validation and functionality as well as UI changes.
- Previously the patch forms in patch-list.html handled error and
update messages through views in patch.py, whereas the patch forms in
submission.html handled the messages with forms.py. Now, with a single
patch forms component in patch-forms.html, forms.py is set to handle
the messages and handle form validation for both pages.
Raxel Gutierrez [Mon, 23 Aug 2021 18:28:30 +0000 (18:28 +0000)]
views: Move js code to separate file
Move patch-list related JS code to a new patch-list.js file, to
make the JavaScript easy to read and change in one place. This makes
automatic code formatting easier, makes it more straightforward to
measure test coverage and discover opportunities for refactoring, and
simplifies a possible future migration to TypeScript if the project
chooses to go in that direction.
Raxel Gutierrez [Mon, 23 Aug 2021 18:28:29 +0000 (18:28 +0000)]
views: Clean up patch-list page
Add ids to table cells, and rename selectors using hyphen delimited
strings to clean up and improve readability of patch-list.html. Also,
create a partial template errors.html for errors that render with form
submission.These changes make the code healthier, ready for change, and
overall more readable.
No user-visible change should be noticed.
Signed-off-by: Raxel Gutierrez <raxel@google.com> Signed-off-by: Stephen Finucane <stephen@that.guru>
[stephenfin: Addressed merge conflicts and renamed some Python variables
in snake_case also]
views: Don't show chevron if user can't change sort order
We prevent users changing the column that sorting is done on when
allowing users to change the order of patches in a bundle. However, we
still show a chevron and clicking the link will appear to do something.
This is confusing/misleading. Remove the chevron and the link in this
situation.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Robin Jarry [Wed, 5 Oct 2022 14:24:05 +0000 (16:24 +0200)]
css: make diff colors more accessible
The colors used to display patch diffs are confusing. The context color
is very similar to the added line color and the contrast between added
and removed lines is very low.
Originally, the choice of purple/blue (instead of the more common
red/green palette) may have been made with colorblindness accessibility
in mind. However, after inspecting the current colors with
colorblindness "simulators", I found that the low contrast was
consistent no matter what vision deficiency (if any) you might have.
Update the colors to use a more common red/green palette. Add background
colors to increase contrast for colorblind people. Use less confusing
colors for context and diff hunks. Use normal line height to prevent
background colors from overlapping. Use a different color for email
quotes (blue) to avoid confusion with added lines.
I have made a compilation of the current and updated color palette
previews for normal vision and all common color deficiencies. I also
included the same diff as seen from Github interface for reference.
Thomas Monjalon [Fri, 16 Aug 2024 07:20:28 +0000 (09:20 +0200)]
templates/submission: Fix alignment of commit message
Preformatted content must not be indented
because any space is kept in the output,
making the content wrongly indented.
When aligning message headers to the left,
the new HTML code has been indented
including some preformatted content indented with two spaces.
The fix is to remove the indent of the content.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net> Fixes: fe34ab2ffad3 ("patch-detail: left align message headers") Reviewed-by: Robin Jarry <robin@jarry.cc> Reviewed-by: Stephen Finucane <stephen@that.guru>
build(deps): update sqlparse requirement from ~=0.5.0 to ~=0.5.1
Updates the requirements on [sqlparse](https://github.com/andialbrecht/sqlparse) to permit the latest version.
- [Changelog](https://github.com/andialbrecht/sqlparse/blob/master/CHANGELOG)
- [Commits](https://github.com/andialbrecht/sqlparse/compare/0.5.0...0.5.1)
Updates the requirements on [django-debug-toolbar](https://github.com/jazzband/django-debug-toolbar) to permit the latest version.
- [Release notes](https://github.com/jazzband/django-debug-toolbar/releases)
- [Changelog](https://github.com/jazzband/django-debug-toolbar/blob/main/docs/changes.rst)
- [Commits](https://github.com/jazzband/django-debug-toolbar/compare/4.4...4.4.6)
build(deps): update mysqlclient requirement from ~=2.2.0 to ~=2.2.4
Updates the requirements on [mysqlclient](https://github.com/PyMySQL/mysqlclient) to permit the latest version.
- [Release notes](https://github.com/PyMySQL/mysqlclient/releases)
- [Changelog](https://github.com/PyMySQL/mysqlclient/blob/main/HISTORY.rst)
- [Commits](https://github.com/PyMySQL/mysqlclient/compare/v2.2.0...v2.2.4)
build(deps): update openapi-core requirement from ~=0.19.0 to ~=0.19.2
Updates the requirements on [openapi-core](https://github.com/python-openapi/openapi-core) to permit the latest version.
- [Release notes](https://github.com/python-openapi/openapi-core/releases)
- [Commits](https://github.com/python-openapi/openapi-core/compare/0.19.0...0.19.2)
build(deps): update psycopg2-binary requirement from ~=2.9.0 to ~=2.9.9
Updates the requirements on [psycopg2-binary](https://github.com/psycopg/psycopg2) to permit the latest version.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.6...2.9.9)
build(deps): update django requirement from ~=5.0.0 to ~=5.0.7
Updates the requirements on [django](https://github.com/django/django) to permit the latest version.
- [Commits](https://github.com/django/django/compare/5.0...5.0.7)
build(deps): update psycopg2 requirement from ~=2.9.0 to ~=2.9.9
Updates the requirements on [psycopg2](https://github.com/psycopg/psycopg2) to permit the latest version.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.6...2.9.9)
projects / thirdparty / patchwork.git / log
