Apparently different environments raise different return codes. Handle
the invalid return codes thrown by subprocess, presumably related to how
Travis checks out things from git.
Instead of exposing differing fields depending on the event category,
expose a consistent set of 'id', 'category', 'date' and 'payload'. This
ensures we can loop through events in a somewhat standardized fashion.
Signed-off-by: Stephen Finucane <stephen@that.guru> Tested-by: Daniel Axtens <dja@axtens.net>
This is a list only endpoint as it's expected that we would kill events
after a certain duration and would have no reason to allow indexing of
past events.
Signed-off-by: Stephen Finucane <stephen@that.guru> Tested-by: Daniel Axtens <dja@axtens.net>
At the moment, Series.received_all is only true is the number of patches
received exactly matches the expected number of patches. However, there
are cases where there may in fact be more patches in the final series
than expected. For example, one could send patches with a subject of
'PATCH 5/4' in-reply-to an original four patch series. The parser
handles this correctly, so the 'received_all' property should too.
We still need to support patterns like '5/n', which are in use and do
not appear to be supported since 'c21b305'. This can be done at a later
stage.
Signed-off-by: Stephen Finucane <stephen@that.guru> Tested-by: Daniel Axtens <dja@axtens.net>
Daniel Axtens [Tue, 21 Feb 2017 05:45:29 +0000 (16:45 +1100)]
REST: allow fetching of subject prefixes
Some mailing lists accept patches for multiple projects, and use
a subject prefix to differentiate the projects.
Therefore, for snowpatch, it's useful to be able to fetch the
subject prefixes.
Export subject prefixes in the REST API.
Signed-off-by: Daniel Axtens <dja@axtens.net> Tested-by: Russell Currey <ruscur@russell.cc> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Reviewed-by: Stephen Finucane <stephen@that.guru>
Stephen Finucane [Sat, 19 Nov 2016 18:31:32 +0000 (18:31 +0000)]
urls: Move 'help/about' to 'about'
This is the standard URL for such pages. This involves removing the
pwclient help page, but this is migrated to the project summary page
and detailed in the documentation.
Permanent redirects are included to prevent dead links.
Signed-off-by: Stephen Finucane <stephen@that.guru> Reviewed-by: Daniel Axtens <dja@axtens.net>
This is reflected in the migration but not the model.
Signed-off-by: Stephen Finucane <stephen@that.guru> Fixes: c1d8a0d ("Fix series.project migration logic") Reviewed-by: Andy Doan <andy.doan@linaro.org>
It would appear that object-based mixins don't work with django-filter's
FilterSet. This should probably be fixed upstream, but for now let's
live with the extra duplication.
Signed-off-by: Stephen Finucane <stephen@that.guru> Fixes: 0fc32337 ("REST: Integrate django-filter support") Reviewed-by: Andy Doan <andy.doan@linaro.org>
Django 1.10.3, 1.9.11 and 1.8.16 changed default behavior for
ALLOWED_HOSTS to prevent DNS rebinding attacks [1]. Unfortunately this
also means we can't access the development Docker or Vagrant installs
by IP address. Sidestep the issue by wildcarding the 'ALLOWED_HOSTS'
setting for development, thus allowing connections from any IP.
This causes two issues. Firstly, on fresh installs you see the following
error message:
"Table 'patchwork.patchwork_state' doesn't exist"
Secondly, any new states created when the process is running will not be
reflected in the API until the server process is restarted.
Resolve this issue by moving the step into a method, thus ensuring it's
continuously refreshed. It doesn't seem possible to add tests to prevent
this regressing but some similarly useful tests are included to at least
validate the behavior of that field.
Signed-off-by: Stephen Finucane <stephen@that.guru> Tested-By: Denis Laxalde <denis@laxalde.org> Fixes: a2993505 ("REST: Make 'Patch.state' editable")
Closes-bug: #67
Closes-bug: #80
Per feedback from FOSDEM, this is still confusing some people. Clarify
things.
You might think we could just strip of the offending prefixes but that
might not always be the thing to do. Other VCSs don't include these
prefixes and both 'a' and 'b' are valid folder names. The risk of false
positives might be small, but it's enough to discourage us from doing
this.
Signed-off-by: Stephen Finucane <stephen@that.guru> Reviewed-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Daniel Axtens [Wed, 8 Feb 2017 04:16:58 +0000 (15:16 +1100)]
Fix double use of mail directory in parsemail-batch.sh
2d142b2c0a27 ("bin: Run scripts through shellcheck") changes an
ls to a find. find output includes the directory name, unlike ls.
This breaks the script, because it prepends the mail directory
later on.
Remove the prepending in the script - leave it exclusively to find.
Signed-off-by: Daniel Axtens <dja@axtens.net> Reviewed-by: stephen@that.guru Fixes: 2d142b2c0a27 ("bin: Run scripts through shellcheck")
Stephen Finucane [Mon, 23 Jan 2017 09:07:29 +0000 (09:07 +0000)]
migrations: Don't use 'noop'
The 'noop' operation was introduced in Django 1.8, meaning this
operation won't work on Django 1.7. Travis caught this, though I don't
know how it was missed locally.
Signed-off-by: Stephen Finucane <stephen@that.guru> Fixes: e3cbe493 ("models: Add 'project' field to Series")
Stephen Finucane [Sun, 22 Jan 2017 22:33:51 +0000 (22:33 +0000)]
Revert "requirements: Test older versions of DRF"
This partially reverts commit febad055fb6609369f1a465a5eec323549c5c065.
While Django REST Framework works with Django 1.6 and 1.7, the versions
of Django Filters that provide DRF integration do not [1]. It doesn't
really make sense to enable only partial REST API support (i.e. no
filtering) for users with older Django versions and this approach will
cause far too much confusion among users. Better to just drop REST API
support for users with these insecure versions and encourage them to
update to supported versions of Django should they wish to use these
features.
This details the various top level elements that Patchwork exposes.
This could be stored as source code documentation, but then users would
need to look at the source.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Stephen Finucane [Fri, 11 Nov 2016 23:19:32 +0000 (23:19 +0000)]
REST: Integrate django-filter support
This mostly works out of the box, thanks to Django REST Framework.
Mostly unique fields, like name or email, are excluded as these will be
handled separately.
Signed-off-by: Stephen Finucane <stephen@that.guru>
This is helpful for filtering. We use RunPython because folks are likely
to have few if any Series objects existing yet. In addition, we update
the unique constraints for SeriesReference as it's now possible to
handle messages with duplicate message IDs.
The update is included in parser to ensure this applies immediately.
Signed-off-by: Stephen Finucane <stephen@that.guru> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Wed, 16 Nov 2016 23:25:00 +0000 (23:25 +0000)]
docs: Update swagger definition
There are a couple of changes needed:
- Add the '/users' endpoint
- Add PATCH support for the '/projects' endpoint
- Add information on authentication
- Clean up some documentation
- Remove some non-implemented parameters
This is still not complete, but it bears a lot closer resemblence to
reality now.
Signed-off-by: Stephen Finucane <stephen@that.guru>
This is necessary due to the N:N mapping of series and patches: it's
possible for a patch to belong to many series, and a series usually
contains many patches. This means it is not possible to rely on the
patch endpoint alone.
It is also necessary to add a cover letter endpoint, such that the
series body can include this.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Stephen Finucane [Thu, 17 Nov 2016 01:00:04 +0000 (01:00 +0000)]
REST: Make 'User.first_name', 'last_name' editable
In an ideal world, everything that can be done with the UI should also
be doable from the command line. The first and last names of the User
are low-value fields to allow customization of, but they are easily
implmented and move us towards the above goal.
Signed-off-by: Stephen Finucane <stephen@that.guru> Reviewed-by: Daniel Axtens <dja@axtens.net> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Wed, 23 Nov 2016 15:53:50 +0000 (15:53 +0000)]
REST: Make 'Patch.state' editable
The 'Patch.state' field is exposed by the API, but is not editable.
Tests exist that suggest the field is editable, but they lie as they
don't actually validate the field is changed (it hasn't). Make this
field editable, using a custom field type to allow said user to submit a
string representation of the state rather than an integer id.
This has the side effect of changing the output representation of the
'state' field for the '/patches' endpoint to a slugified representation,
i.e.:
"state": "under-review",
Instead of:
"state": "Under review",
The same slugified representation will be used in PATCH requests. This
seems more consistent with how APIs generally do this stuff making it a
"good thing" (TM).
Signed-off-by: Stephen Finucane <stephen@that.guru> Reviewed-by: Andy Doan <andy.doan@linaro.org> Reviewed-by: Daniel Axtens <dja@axtens.net>
Stephen Finucane [Thu, 24 Nov 2016 12:55:42 +0000 (12:55 +0000)]
REST: Use generic views instead of ViewSets
ViewSet provide an easy way to define an API, but they don't offer much
flexibility. To get them to work as expected, a lot of hacks were
required. Generic views provide a more verbose, but ultimately easier to
understand, version. Using generic views lets us remove the dependency
of drf-nested-routers, bringing us back down to two main dependencies.
It also lets us remove the AuthenticatedReadOnly permission class, as
the DRF provides a similar permission class that can be used with
generic views.
The main user facing change is that invalid methods, such as POST on an
endpoint that doesn't allow object creation, will now return a HTTP 405
(Method Not Allowed) error code rather than the HTTP 403 (Forbidden)
error code previously returned. This is the semantically correct option
and should have been used all along.
Signed-off-by: Stephen Finucane <stephen@that.guru> Reviewed-by: Daniel Axtens <dja@axtens.net>
Stephen Finucane [Fri, 28 Oct 2016 18:02:41 +0000 (19:02 +0100)]
REST: Resolve performance issues with tags
The list comprehension used to generate a list of tags resulted in a
significant number of duplicated queries. Resolve this by copying the
approach taken to minimize patch queries in the UI.
This changes the output of the response in two ways. First, counts for
all tag patches are now shown, even if the count is 0. Secondly, a
dictionary is returned, with the tag name as the key, rather than a
list. As such, the output for a typical patch is transformed from:
Stephen Finucane [Wed, 23 Nov 2016 19:41:12 +0000 (19:41 +0000)]
REST: Explicitly define fields
Explicitly define included fields (using 'Meta.fields') rather than
those that should be excluded (using 'Meta.exclude'). This ensure fields
that are exposed by the API don't change unless we explicitly change
them
The only side-effect of this change should be a consistent ordering of
the output - the ordering used in the 'Meta.fields' will be consistently
used, thus ordering is now important and some fields are moved around
to reflect this.
Signed-off-by: Stephen Finucane <stephen@that.guru> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Wed, 23 Nov 2016 19:35:40 +0000 (19:35 +0000)]
REST: Use SerializerMethod field
Use 'SerializerMethodField' to override how we generate fields, rather
than hacking the 'to_representation' method. This is the more idiomatic
way to do this.
The Meta class for the Project and Patch serializers are moved to the
bottom of the serializers and some variables renamed, both for
consistency's sake.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Stephen Finucane [Thu, 24 Nov 2016 09:33:53 +0000 (09:33 +0000)]
REST: Remove '_url' suffixes
This was a design decision made when implementing the REST API. The
idea was that these items were URLs to related objects and should be
indicated as such. However, this was a faulty assumption as the
Patchwork API, unlike other some other APIs (GitHub), does not also
include a full representation of said objects, like so:
Since there is no intention to support this design yet, there isn't
really any reason to fight django-rest-framework in appending these
suffixes. Simply remove them. This changes the output for most endpoints
from something like this:
Stephen Finucane [Mon, 31 Oct 2016 18:23:29 +0000 (18:23 +0000)]
REST: Create 'api' directory
Move all REST API-related code into an 'api' directory. This allows us
to break the existing files into endpoint-based files and will allow us
to split the API into a different Django app in the future. This
involves simply shuffling code around for now, so there no functional
changes introduced.
Signed-off-by: Stephen Finucane <stephen@that.guru> Reviewed-by: Daniel Axtens <dja@axtens.net>
Yann E. MORIN [Thu, 22 Dec 2016 21:09:32 +0000 (22:09 +0100)]
bin/pwclient: accept alternate http_proxy forms
The forms for http_proxy (and the likes) is not very well defined nor
documented in any authoritaive place. However, there are two common
forms: http://host:port or http://host:port/
Currently, the code chokes on the latter (e.g. with
http_proxy=http://127.0.0.1:8080/ ):
[...]
File "/usr/lib/python2.7/httplib.py", line 792, in _get_hostport
raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
httplib.InvalidURL: nonnumeric port: '8080/'
Chop off any slash character in the port definition to accept the second
form. If there is no '/' in there, it still works.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Signed-off-by: Stephen Finucane <stephenfinucane@hotmail.com>
Stephen Finucane [Fri, 23 Dec 2016 19:40:59 +0000 (19:40 +0000)]
forms: 'False' != False
Forms cast boolean values to strings, and attempting to coerce using the
'bool' function does not correctly return them to true boolean values.
Correct this by doing a string comparison instead.
Signed-off-by: Stephen Finucane <stephen@that.guru> Fixes: 0abde97aa ("forms: Use TypedChoiceField")
Stephen Finucane [Fri, 23 Dec 2016 20:21:56 +0000 (20:21 +0000)]
tests: Set 'date' when bulk creating patches
Apparently Django 1.8+ changed how patches with identical 'date' fields
were ordered, meaning tests that worked on these versions failed on
previous versions.
Florian Fainelli [Tue, 13 Dec 2016 18:47:37 +0000 (10:47 -0800)]
xmlrpc: Expose patch hash to patch_get
Expose the patch object hash value to xmlrpc::patch_get. This is
particuarly helpful if some patche(s) have been submitted several times
but changed from e.g: RFC to a proper official patch submission. The
hash would typically be identical, but numbers would not.
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com> Reviewed-by: Stephen Finucane <stephen@that.guru>
Doug Anderson [Sat, 19 Nov 2016 19:32:09 +0000 (19:32 +0000)]
views: Don't munge the 'From' field of patches
At the moment patchwork always uses the official submitter name (as
patchwork understands it) as the "From" for patches that you receive.
This isn't quite what users expect and has some unfortunate
consequences.
The biggest problem is that patchwork saves the "official" name for an
email address the first time it sees an email from them. If that name
is wrong (or was missing) patchwork will be confused even if future
emails from this person are fixed. There are similar problems if a
user changes his/her name (get married?).
It seems better to just have each patch report the actual "From" that
was used to send that patch. We'll still return the submitter in
'X-Patchwork-Submitter' just in case someone wants it.
Reported-by: Wolfram Sang <wsa@the-dreams.de> Signed-off-by: Doug Anderson <dianders@chromium.org> Signed-off-by: Stephen Finucane <stephen@that.guru>
Robin Jarry [Thu, 15 Dec 2016 16:56:18 +0000 (17:56 +0100)]
pwclient: Fix encoding problems
All data returned by the xmlrpc object is unicode decoded with 'utf-8' (on
python 3, unicode == str). Add from __future__ import unicode_literals
to make sure that everything is unicode and avoid surprises.
On python 2, printing unicode to stdout causes it to be encoded to str
(byte string) with the 'ascii' codec:
>>> print some_unicode_string
...
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0142'
in position 468: ordinal not in range(128)
Work around ths by avoiding any explicit call to unicode() and by
replacing sys.stdout and sys.stderr by unicode-aware file objects (as
returned by io.open()).
Guess the encoding of stdout and stderr by looking at (in that order):
sys.stdout.encoding, locale.getpreferredencoding(), the PYTHONIOENCODING
environment variable. If no encoding is defined, assume 'utf-8' as
output encoding.
Signed-off-by: Robin Jarry <robin.jarry@6wind.com> Tested-by: Thomas Monjalon <thomas.monjalon@6wind.com> Reviewed-by: Stephen Finucane <stephen@that.guru>
Thomas Monjalon [Tue, 13 Dec 2016 10:37:47 +0000 (11:37 +0100)]
pwclient: Fix Python 3 encoding of received strings
The conversion encode("utf-8") makes a byte stream which is
poorly printed with Python 3.
However this encoding is required for Popen.communicate() but must be
done after str.join() which applies to a real string.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com> Reviewed-by: Stephen Finucane <stephen@that.guru>
Stephen Finucane [Wed, 30 Nov 2016 18:29:37 +0000 (18:29 +0000)]
tools: Update to use 'hasher'
The old 'parser' module used to extract diffs from their surrounding
mbox fluff before hashing this. Seeing as this was only used in the
context of an actual git repo, avoid all of that rigmarole by just using
'git diff', which produces a plain diff, rather than 'git show'.
Signed-off-by: Stephen Finucane <stephen@that.guru> Tested-by: Tom Rini <trini@konsulko.com>
Closes-bug: #63
Stephen Finucane [Fri, 18 Nov 2016 00:54:51 +0000 (00:54 +0000)]
tools: Trivial formatting fixes
These tools are currently broken, but before beginning surgery let's
clean things up. Use standard 4 spaces and the longer, but easier to
read, if-else-fi syntax for comparison.
Missing license headers are added for completeness sake.
Signed-off-by: Stephen Finucane <stephen@that.guru> Tested-by: Tom Rini <trini@konsulko.com>
Andrew Donnellan [Wed, 23 Nov 2016 07:01:31 +0000 (18:01 +1100)]
views: don't duplicate tags in patch message when generating mbox
When generating an mbox for a patch with tags in the original commit
message, e.g.:
Example patch
This patch is awesome!
Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Acked-by: Russell Currey <ruscur@russell.cc>
the tags from the original email are duplicated:
Example patch
This patch is awesome!
Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Acked-by: Russell Currey <ruscur@russell.cc> Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Acked-by: Russell Currey <ruscur@russell.cc>
It appears that during the refactoring in ef56359fb776 ("models: Merge
patch and first comment"), we added a call to patch.patch_responses() to
extract the tags from the initial patch email, which we then append to the
patch email body... which already has the tags in it.
Remove the unnecessary append of patch.patch_responses when generating an
mbox.
Fixes: ef56359fb776 ("models: Merge patch and first comment") Reported-by: Russell Currey <ruscur@russell.cc> Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Reviewed-by: Daniel Axtens <dja@axtens.net> Reviewed-by: Stephen Finucane <stephen@that.guru>
Stephen Finucane [Thu, 24 Nov 2016 09:24:13 +0000 (09:24 +0000)]
trivial: Resolve pep8 issues
flake8 3.2.0 updates the pycodestyle dependency to 2.2.0, which in
turn resolves a bug with E305. Upgrading results in a small number of
additional pep8 issues. Resolve these now.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Stephen Finucane [Sat, 29 Oct 2016 13:18:28 +0000 (14:18 +0100)]
admin: Integrate UserProfile fields into admin
The 'User' model is extended by means of a 'UserProfile' model. These
fields are not correctly displayed in the admin UI, but they should be.
Let's fix this per the recommendations of the Django docs [1].
Daniel Axtens [Thu, 17 Nov 2016 05:11:59 +0000 (16:11 +1100)]
Fix parsing of interesing series reply structures
There are some things you probably shouldn't do on public
mailing lists, but which people do anyway.
The first, and most understandable, is this:
- [PATCH 1/2] test: Add some lorem ipsum
- [PATCH 2/2] test: Convert to Markdown
- [PATCH v2 1/2] test: Add some lorem ipsum
- [PATCH v2 2/2] test: Convert to Markdown
We should correctly parse these by:
- creating a new series if the version number changes
- when deciding whether to create a SeriesReference, search by
message-id alone, not the message-id/series pair. (Otherwise,
we try to create a series ref for v1 2/2 in the series for v2,
which breaks a uniqueness constraint.
The second, and less excusable, is this:
- [PATCH 1/2] test: Add some lorem ipsum
- [PATCH 2/2] test: Convert to Markdown
- [PATCH 1/2] test: Add some lorem ipsum
- [PATCH 2/2] test: Convert to Markdown
With this patch:
- if we get a x/n for a series that already has an x/n, create a
new series for it.
Signed-off-by: Daniel Axtens <dja@axtens.net> Reviewed-by: Stephen Finucane <stephen@that.guru>
projects / thirdparty / patchwork.git / log
