doc: The REST API can now be used with Django < 1.8
The release notes stated that the REST API was only compatible with
Django 1.8. However, with the merge of commit '646366cc', this is no
longer the case.
We would rather people didn't use these older versions of Django, but
let's not lie.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Use the latest LTS version of Ubuntu. This is mostly a simplification of
the guide, which reverts back to single-node configuration and increases
the emphasis on installing system packages rather than using 'pip'.
There are also a series of corrections, mostly around using the Python 3
variants of packages.
Signed-off-by: Stephen Finucane <stephen@that.guru>
The nginx file was a replacement for '/etc/nginx/nginx.conf' instead of
a "site" file, while the uWSGI file referenced the Python 2 plugin
despite the sample deployment guide, which uses this, being Python
3-based. Correct both issues.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Robin Jarry [Tue, 2 May 2017 14:51:51 +0000 (16:51 +0200)]
pwclient: Fix silent crash on python 2
Replacing sys.stdout and sys.stderr can cause obscure crashes when
trying to write non unicode data. The interpreter is terminated with
SIGINT without any specific error writen on the console.
This happens easily when there is an untrapped exception which should
lead to printing a traceback on stderr.
The only way to prevent UnicodeEncodeErrors is to make sure that one of
the locale-related environment variables (LC_ALL, LANG, LANGUAGE, etc.)
is set. Python will use the correct encoding accordingly.
Add a note about this on `pwclient --help`. Also, display a help message
when an encoding error occurs.
Fixes: 046419a3bf8f ("pwclient: Fix encoding problems") Signed-off-by: Robin Jarry <robin.jarry@6wind.com> Signed-off-by: Stephen Finucane <stephen@that.guru>
Stephen Finucane [Sat, 29 Apr 2017 00:13:06 +0000 (01:13 +0100)]
docs: Emphasise the cron job
Having talked to a few folks deploying Patchwork, it appears not
everyone is aware of/enabling the Patchwork cron job. Emphasise this
feature by moving it to its own section. This section is marked as
optional, given that it's not truly required but is helpful.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Stephen Finucane [Wed, 26 Apr 2017 23:58:03 +0000 (00:58 +0100)]
REST: Fix versioning
One of the few remaining warts in the API is versioning. There is some
basic versioning there, but it doesn't work properly as-is. Fix this by
correcting the index endpoint ('/') to use Django REST Framework's
'reverse' function [1], which handles versioning for us [2] and
switching from 'NamespaceVersioning' to 'URLPathVersioning', the latter
of which does the same thing but in a different way.
In commit febad055, we attempted to start testing older versions of
Django REST Framework for use with older versions of Django. This work
was later reverted in commit 6f2ddab6 due to issues with 'django-filter'
versions.
There are additional reasons for validating older versions of DRF, such
as supporting the versions provide as part of distributions. It turns
out that those issues aren't such a big deal and can be handled with a
shim. All in all, this means we can and should support these older
versions of Django REST Framework, and that is what we'll do.
Note that the minimum version supported in requirements.txt is not
changed as it's good practice to use the latest available version.
However, that doesn't make it any less supported.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Older versions of Django REST Framework's 'RelatedField' did not check
for the presence 'get_queryset' before checking for a 'queryset'
attribute. This was fixed in 3.4.0 [1] and backported to the 3.3 series
in 3.3.3 [2]. However, the fix was not backported to the last release of
3.2 series [3], which is the last release to support Django 1.6. As
such, we must set a 'queryset' attribute, even if it's set to a garbage
value.
Recent versions of Django REST Framework (DRF) have deprecated the
'DjangoFilterBackend' filter found in-tree, in favour of an equivalent
implementation found in django-filter. However, we need to support older
versions of DRF for users who want to use system packages.
Seeing as the two implementations are, for all intents and purposes,
essentially the same thing, provide a shim that will allow us to use
both together.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Partial-bug: #94
Stephen Finucane [Wed, 26 Apr 2017 21:45:33 +0000 (22:45 +0100)]
docs: Split API docs into their own section
Third time lucky. There are two changes:
- Add a new 'clients' section to the usage doc, allowing us to remove a
lot of the API nitty gritty stuff from the users guide. This makes
more sense as users don't really care what API method they're using -
only what application).
- Change the API docs from the developers guide into a quick start
section, allowing us to greatly expand the REST API docs to include
information on pagination, authentication, etc.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Stephen Finucane [Wed, 22 Mar 2017 23:17:08 +0000 (23:17 +0000)]
docs: Add skeleton for Sphinx docs
This is mostly the output of 'sphinx-quickstart' with all non-HTML build
cruft removed and Sphinx minimum version set to 1.5. A tox target is
included and the output of the docs build ignored.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Daniel Axtens [Mon, 27 Mar 2017 05:42:52 +0000 (16:42 +1100)]
Display count of patches
Minor UI tweak - show the number of patches left in the current
view.
Suggested-by: Stewart Smith <stewart@linux.vnet.ibm.com> Signed-off-by: Daniel Axtens <dja@axtens.net> Reviewed-by: Stephen Finucane <stephen@that.guru>
Daniel Axtens [Mon, 27 Mar 2017 05:42:51 +0000 (16:42 +1100)]
'mpe mode': click to copy patch IDs
If 'Show Patch IDs' is turned on in settings, add an extra column
to the patch list, with buttons showing the patch IDs. The buttons
copy the patch IDs to the clipboard.
JavaScript inspired by https://github.com/Triforcey/clip-j and many
many StackOverflow answers.
Suggested-by: Michael Ellerman <mpe@ellerman.id.au> Signed-off-by: Daniel Axtens <dja@axtens.net> Reviewed-by: Stephen Finucane <stephen@that.guru>
In '2a915efd', a check was added to ensure mails prefixed with 'RE:' or
similar would not be parsed as patches. By time this check actually
happens, any patches had already been extracted from the mail thus these
patches were re-added to the mail content before saving the comment.
Unfortunately, this didn't take into account cases where a patch or diff
was not the last part of a mail but rather located somewhere in the
middle of the content:
Introduction content
Diff or patch content ***
Additional content
This would result in mangling of the mail as the patch would _always_ be
appended to the end:
Introduction content
Additional content
Diff or patch content ***
Handle this by only breaking a mail into a comment and a diff if there
is any possibility that we might want to use that diff.
Signed-off-by: Stephen Finucane <stephen@that.guru> Fixes: 2a915efd ("parser: fix wrong parsing of diff comments")
Closes-bug: #95
At present, the 'parsearchive' command only supports parsing of mboxes.
Expand this to support maildirs. This allows us to rewrite the
'parsemail-bulk' script to deliver much improved performance.
Signed-off-by: Stephen Finucane <stephen@that.guru> Suggested-by: Daniel Axtens <dja@axtens.net>
Daniel Axtens [Mon, 27 Mar 2017 05:01:13 +0000 (16:01 +1100)]
parser: cut off patch names after 255 characters
Our model limits the length of patch names to 255 characters.
Enforce this cutoff in the parser, otherwise we throw an
exception and fail to store patches with stupidly long titles.
Reported-by: Russell Currey <ruscur@russell.cc> Signed-off-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Stephen Finucane <stephen@that.guru>
Stephen Finucane [Thu, 23 Mar 2017 13:25:00 +0000 (13:25 +0000)]
views: Use bundle-detail for public URLs
In 'd1c605f', we reworked the 'bundle-list' view to use the new
'Bundle.get_mbox_url' function instead of the 'Bundle.public_url'.
However, these are not the same thing. The latter referred to the
'bundle-detail' view, while the former referred to the 'bundle-mbox'
view.
The easiest fix would be to simply revert that patch. However, it turns
out that 'public_url' isn't actually needed. Commit '5d0140ef' removed a
divide between public and non-public URLs for bundles, meaning we can
actually use an existing function - 'get_absolute_url' - instead.
This also presents the opportunity to clean up the 'bundle-list' page,
favouring a simple public/is-not-public marker and only a single URL.
Signed-off-by: Stephen Finucane <stephen@that.guru> Fixes: d1c605f9 ("views: Stop using Bundle.public_url")
Closes-bug: #92
Stephen Finucane [Thu, 23 Mar 2017 13:24:59 +0000 (13:24 +0000)]
xmlrpc: Remove 'bundle_to_dict'
Turns out this function is not called anywhere nor has it been called
since being introduced in '83964878'. Given that we're now focused on
the REST API, we can simply remove this.
Signed-off-by: Stephen Finucane <stephen@that.guru>
It is possible to download a patch mbox with all dependencies. Now make
it possible to download the entire series. This takes the form of the
following URL when using the default routes:
/series/{seriesID}/mbox/
Like the equivalent patch and bundle links, this will return a 404 if
'{seriesID}' does not match an existing series' ID. However, a 404 will
also be returned in the series is not complete, as indicated by
Series.total > Series.received_total. You can override this behavior by
providing the 'force' parameter:
/series/{seriesID}/mbox/?force=1
Note that there are no current plans to provide a series-specific view,
a.k.a.
/series/{seriesID}/
As a result, this particular URL will continue to return a 404.
Signed-off-by: Stephen Finucane <stephen@that.guru> Reviewed-by: Daniel Axtens <dja@axtens.net>
The 'patch_to_mbox' function returns an object which is coverted to a
string in all places where this call occurs. The string conversion
differs between Python 2 and 3 and while it has been updated in one
place, it was missed in two others. Resolve these issues and ensure they
don't happen again by returning strings from 'patch_to_mbox' instead.
Signed-off-by: Stephen Finucane <stephen@that.guru> Reviewed-by: Daniel Axtens <dja@axtens.net>
Ralf Baechle [Tue, 14 Mar 2017 18:09:12 +0000 (19:09 +0100)]
parsemail.sh: Fix usage via stdin.
Commit e56391f66cb8 ("trivial: Standardize variable naming in scripts")
sneaked in a checked for missing command line arguments in parsemail.sh
which broke typical argument-less usage for mail delivery. Revert that
again.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org> Reviewed-by: Stephen Finucane <stephen@that.guru> Fixes: e56391f6 ("trivial: Standardize variable naming in scripts")
Stephen Finucane [Tue, 21 Feb 2017 16:22:41 +0000 (11:22 -0500)]
REST: Add '/bundle' endpoint
I had initially resisted adding a '/bundle' endpoint to the API as I
wanted to kill this feature in favour of series. However, series are not
a like-for-like replacement for bundles. Among other things, series do
not provide the composability of bundles: bundles can be manually
created, meaning you can use bundles to group not only multiple patches
but also multiple series (or at least the patches in those series).
Seeing as we're not getting rid of this feature, we should expose it via
the API. Bundles are little unusual, in that they can be public or
private, thus, we should only show bundles that are public or belonging
to the currently authenticated user, if any. For now, this is a
read-only endpoint. We may well allow creation of bundles via the API
once we figure out how to do this cleanly.
Signed-off-by: Stephen Finucane <stephen@that.guru> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Sun, 26 Feb 2017 23:57:22 +0000 (23:57 +0000)]
views: Allow use of basic auth for bundle mboxes
API clients are going to talk using basic auth. We also need to do this
for bundles. The alternative is to provide another endpoint for bundles
in the API but that seems unnecessary.
Signed-off-by: Stephen Finucane <stephen@that.guru> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Apparently different environments raise different return codes. Handle
the invalid return codes thrown by subprocess, presumably related to how
Travis checks out things from git.
Instead of exposing differing fields depending on the event category,
expose a consistent set of 'id', 'category', 'date' and 'payload'. This
ensures we can loop through events in a somewhat standardized fashion.
Signed-off-by: Stephen Finucane <stephen@that.guru> Tested-by: Daniel Axtens <dja@axtens.net>
This is a list only endpoint as it's expected that we would kill events
after a certain duration and would have no reason to allow indexing of
past events.
Signed-off-by: Stephen Finucane <stephen@that.guru> Tested-by: Daniel Axtens <dja@axtens.net>
At the moment, Series.received_all is only true is the number of patches
received exactly matches the expected number of patches. However, there
are cases where there may in fact be more patches in the final series
than expected. For example, one could send patches with a subject of
'PATCH 5/4' in-reply-to an original four patch series. The parser
handles this correctly, so the 'received_all' property should too.
We still need to support patterns like '5/n', which are in use and do
not appear to be supported since 'c21b305'. This can be done at a later
stage.
Signed-off-by: Stephen Finucane <stephen@that.guru> Tested-by: Daniel Axtens <dja@axtens.net>
Daniel Axtens [Tue, 21 Feb 2017 05:45:29 +0000 (16:45 +1100)]
REST: allow fetching of subject prefixes
Some mailing lists accept patches for multiple projects, and use
a subject prefix to differentiate the projects.
Therefore, for snowpatch, it's useful to be able to fetch the
subject prefixes.
Export subject prefixes in the REST API.
Signed-off-by: Daniel Axtens <dja@axtens.net> Tested-by: Russell Currey <ruscur@russell.cc> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Reviewed-by: Stephen Finucane <stephen@that.guru>
Stephen Finucane [Sat, 19 Nov 2016 18:31:32 +0000 (18:31 +0000)]
urls: Move 'help/about' to 'about'
This is the standard URL for such pages. This involves removing the
pwclient help page, but this is migrated to the project summary page
and detailed in the documentation.
Permanent redirects are included to prevent dead links.
Signed-off-by: Stephen Finucane <stephen@that.guru> Reviewed-by: Daniel Axtens <dja@axtens.net>
This is reflected in the migration but not the model.
Signed-off-by: Stephen Finucane <stephen@that.guru> Fixes: c1d8a0d ("Fix series.project migration logic") Reviewed-by: Andy Doan <andy.doan@linaro.org>
It would appear that object-based mixins don't work with django-filter's
FilterSet. This should probably be fixed upstream, but for now let's
live with the extra duplication.
Signed-off-by: Stephen Finucane <stephen@that.guru> Fixes: 0fc32337 ("REST: Integrate django-filter support") Reviewed-by: Andy Doan <andy.doan@linaro.org>
Django 1.10.3, 1.9.11 and 1.8.16 changed default behavior for
ALLOWED_HOSTS to prevent DNS rebinding attacks [1]. Unfortunately this
also means we can't access the development Docker or Vagrant installs
by IP address. Sidestep the issue by wildcarding the 'ALLOWED_HOSTS'
setting for development, thus allowing connections from any IP.
This causes two issues. Firstly, on fresh installs you see the following
error message:
"Table 'patchwork.patchwork_state' doesn't exist"
Secondly, any new states created when the process is running will not be
reflected in the API until the server process is restarted.
Resolve this issue by moving the step into a method, thus ensuring it's
continuously refreshed. It doesn't seem possible to add tests to prevent
this regressing but some similarly useful tests are included to at least
validate the behavior of that field.
Signed-off-by: Stephen Finucane <stephen@that.guru> Tested-By: Denis Laxalde <denis@laxalde.org> Fixes: a2993505 ("REST: Make 'Patch.state' editable")
Closes-bug: #67
Closes-bug: #80
Per feedback from FOSDEM, this is still confusing some people. Clarify
things.
You might think we could just strip of the offending prefixes but that
might not always be the thing to do. Other VCSs don't include these
prefixes and both 'a' and 'b' are valid folder names. The risk of false
positives might be small, but it's enough to discourage us from doing
this.
Signed-off-by: Stephen Finucane <stephen@that.guru> Reviewed-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Daniel Axtens [Wed, 8 Feb 2017 04:16:58 +0000 (15:16 +1100)]
Fix double use of mail directory in parsemail-batch.sh
2d142b2c0a27 ("bin: Run scripts through shellcheck") changes an
ls to a find. find output includes the directory name, unlike ls.
This breaks the script, because it prepends the mail directory
later on.
Remove the prepending in the script - leave it exclusively to find.
Signed-off-by: Daniel Axtens <dja@axtens.net> Reviewed-by: stephen@that.guru Fixes: 2d142b2c0a27 ("bin: Run scripts through shellcheck")
projects / thirdparty / patchwork.git / log
