REST: Limit max page size

If this isn't set it's possible to set any page size. Let's restrict
this to the value set in settings.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Cc: Andy Doan <andy.doan@linaro.org>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

Pin psycopg2 to latest version 2.7

REST: Don't iterate through models at class level

This causes two issues. Firstly, on fresh installs you see the following
error message:

  "Table 'patchwork.patchwork_state' doesn't exist"

Secondly, any new states created when the process is running will not be
reflected in the API until the server process is restarted.

Resolve this issue by moving the step into a method, thus ensuring it's
continuously refreshed. It doesn't seem possible to add tests to prevent
this regressing but some similarly useful tests are included to at least
validate the behavior of that field.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Tested-By: Denis Laxalde <denis@laxalde.org>
Fixes: a2993505 ("REST: Make 'Patch.state' editable")
Closes-bug: #67
Closes-bug: #80

Improve documentation of delegation rules

Per feedback from FOSDEM, this is still confusing some people. Clarify
things.

You might think we could just strip of the offending prefixes but that
might not always be the thing to do. Other VCSs don't include these
prefixes and both 'a' and 'b' are valid folder names. The risk of false
positives might be small, but it's enough to discourage us from doing
this.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Thomas Monjalon <thomas.monjalon@6wind.com>

Fix double use of mail directory in parsemail-batch.sh

2d142b2c0a27 ("bin: Run scripts through shellcheck") changes an
ls to a find. find output includes the directory name, unlike ls.
This breaks the script, because it prepends the mail directory
later on.

Remove the prepending in the script - leave it exclusively to find.

Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: stephen@that.guru
Fixes: 2d142b2c0a27 ("bin: Run scripts through shellcheck")

tox: Add 'bashate' target

This is not as powerful as Shellcheck (yet), but it's Python and
therefore installable in a virtualenv.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>

trivial: Standardize variable naming in scripts

Just so we can interact with them as expected.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>

bin: Run scripts through shellcheck

This mostly results in the addition of quotes to prevent globbing.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>

tools: Run scripts through shellcheck

This mostly results in the addition of quotes to prevent globbing.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>

docs: fix links to installation pages

Signed-off-by: Denis Laxalde <denis@laxalde.org>
Reviewed-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Stephen Finucane <stephen@that.guru>

docs: fix links to "contributing" page

Signed-off-by: Denis Laxalde <denis@laxalde.org>
Reviewed-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Stephen Finucane <stephen@that.guru>

docs: link to deployment/development installation page in index

Previous links are broken.

Signed-off-by: Denis Laxalde <denis@laxalde.org>
Reviewed-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Stephen Finucane <stephen@that.guru>

Fix series.project migration logic

The migration logic seems slightly off for 2 reasons:

1) If you already have series objects in your database, then adding a
non-nullable foreign key to projects will fail.

2) This may just be a remnant of my development database, but I had
series entries with no patches.

I think part #1 is *required*. I'm not totally sure about #2, but it
seems safe.

Signed-off-by: Andy Doan <andy.doan@linaro.org>
Reviewed-by: Stephen Finucane <stephen@that.guru>

REST API: add "project" to series filter

It would be nice to allow users to filter on the project they are
interested in.

Signed-off-by: Andy Doan <andy.doan@linaro.org>
Reviewed-by: Stephen Finucane <stephen@that.guru>

migrations: Don't use 'noop'

The 'noop' operation was introduced in Django 1.8, meaning this
operation won't work on Django 1.7. Travis caught this, though I don't
know how it was missed locally.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Fixes: e3cbe493 ("models: Add 'project' field to Series")

requirements: Add django-filter to dev reqs

Forgot to add this to development requirements.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Fixes: 6f2ddab6 ("Revert 'requirements: Test older versions of DRF'")

Revert "requirements: Test older versions of DRF"

This partially reverts commit febad055fb6609369f1a465a5eec323549c5c065.
While Django REST Framework works with Django 1.6 and 1.7, the versions
of Django Filters that provide DRF integration do not [1]. It doesn't
really make sense to enable only partial REST API support (i.e. no
filtering) for users with older Django versions and this approach will
cause far too much confusion among users. Better to just drop REST API
support for users with these insecure versions and encourage them to
update to supported versions of Django should they wish to use these
features.

[1] https://github.com/carltongibson/django-filter/blob/0.15.0/CHANGES.rst

Signed-off-by: Stephen Finucane <stephen@that.guru>
Fixes: 0fc32337 ("REST: Integrate django-filter support")

docs: Provide overview document

This details the various top level elements that Patchwork exposes.
This could be stored as source code documentation, but then users would
need to look at the source.

Signed-off-by: Stephen Finucane <stephen@that.guru>

docs: Bring import of archives up to top-level

This applies whether you're using the Docker-, Vagrant- or manual-
based installation methods.

Signed-off-by: Stephen Finucane <stephen@that.guru>

REST: Integrate ordering

Order by way of the 'order' parameter on many endpoints.

Signed-off-by: Stephen Finucane <stephen@that.guru>

REST: Integrate searching

Search by way of the 'q' parameter on many endpoints.

Signed-off-by: Stephen Finucane <stephen@that.guru>

REST: Integrate django-filter support

This mostly works out of the box, thanks to Django REST Framework.
Mostly unique fields, like name or email, are excluded as these will be
handled separately.

Signed-off-by: Stephen Finucane <stephen@that.guru>

REST: Expose 'Series.project'

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

models: Add 'project' field to Series

This is helpful for filtering. We use RunPython because folks are likely
to have few if any Series objects existing yet. In addition, we update
the unique constraints for SeriesReference as it's now possible to
handle messages with duplicate message IDs.

The update is included in parser to ensure this applies immediately.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

Pin sqlparse to latest version 0.2.2

docs: Update swagger definition

There are a couple of changes needed:

- Add the '/users' endpoint
- Add PATCH support for the '/projects' endpoint
- Add information on authentication
- Clean up some documentation
- Remove some non-implemented parameters

This is still not complete, but it bears a lot closer resemblence to
reality now.

Signed-off-by: Stephen Finucane <stephen@that.guru>

REST: Add '/series' endpoint

Adopt a hybrid approach, by adding an additional series endpoint to the
existing patch endpoint:

    /series/${series_id}/
    /patches/${patch_id}/

This is based on the approach described here:

    http://softwareengineering.stackexchange.com/a/275007/106804

This is necessary due to the N:N mapping of series and patches: it's
possible for a patch to belong to many series, and a series usually
contains many patches. This means it is not possible to rely on the
patch endpoint alone.

It is also necessary to add a cover letter endpoint, such that the
series body can include this.

Signed-off-by: Stephen Finucane <stephen@that.guru>

REST: Add maintainers to '/projects'

This is in the UI. Expose it in the API.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>

REST: Add id field to all endpoints

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>

tests: Standardize validation of REST API JSON

Standardize how the JSON bodies of REST API responses is validated.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Rework REST API tests

- Combine anonymous user tests with authenticated user tests
- Rename and simplify some tests
- Improve documentation

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

REST: Make 'User.first_name', 'last_name' editable

In an ideal world, everything that can be done with the UI should also
be doable from the command line. The first and last names of the User
are low-value fields to allow customization of, but they are easily
implmented and move us towards the above goal.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

REST: Make 'Patch.state' editable

The 'Patch.state' field is exposed by the API, but is not editable.
Tests exist that suggest the field is editable, but they lie as they
don't actually validate the field is changed (it hasn't). Make this
field editable, using a custom field type to allow said user to submit a
string representation of the state rather than an integer id.

This has the side effect of changing the output representation of the
'state' field for the '/patches' endpoint to a slugified representation,
i.e.:

    "state": "under-review",

Instead of:

    "state": "Under review",

The same slugified representation will be used in PATCH requests. This
seems more consistent with how APIs generally do this stuff making it a
"good thing" (TM).

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Andy Doan <andy.doan@linaro.org>
Reviewed-by: Daniel Axtens <dja@axtens.net>

REST: Use generic views instead of ViewSets

ViewSet provide an easy way to define an API, but they don't offer much
flexibility. To get them to work as expected, a lot of hacks were
required. Generic views provide a more verbose, but ultimately easier to
understand, version. Using generic views lets us remove the dependency
of drf-nested-routers, bringing us back down to two main dependencies.
It also lets us remove the AuthenticatedReadOnly permission class, as
the DRF provides a similar permission class that can be used with
generic views.

The main user facing change is that invalid methods, such as POST on an
endpoint that doesn't allow object creation, will now return a HTTP 405
(Method Not Allowed) error code rather than the HTTP 403 (Forbidden)
error code previously returned. This is the semantically correct option
and should have been used all along.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>

REST: Resolve performance issues with tags

The list comprehension used to generate a list of tags resulted in a
significant number of duplicated queries. Resolve this by copying the
approach taken to minimize patch queries in the UI.

This changes the output of the response in two ways. First, counts for
all tag patches are now shown, even if the count is 0. Secondly, a
dictionary is returned, with the tag name as the key, rather than a
list. As such, the output for a typical patch is transformed from:

    [
      {
        'name': 'Reviewed-by',
        'count': 1
      }
    ]

to

    {
      'Reviewed-by': 1
      'Acked-by': 0,
      'Tested-by': 0
    }

How this is achieved is a little hacky, but reworked tags are on the
post-v2.0 which will allow this to be resolved.

Signed-off-by: Stephen Finucane <stephen@that.guru>

REST: Make use of the 'source' property

This is apparently the correct way to rename fields, and will ensure a
future API version that supports writes will work correctly.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

REST: Explicitly define fields

Explicitly define included fields (using 'Meta.fields') rather than
those that should be excluded (using 'Meta.exclude'). This ensure fields
that are exposed by the API don't change unless we explicitly change
them

The only side-effect of this change should be a consistent ordering of
the output - the ordering used in the 'Meta.fields' will be consistently
used, thus ordering is now important and some fields are moved around
to reflect this.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

REST: Use SerializerMethod field

Use 'SerializerMethodField' to override how we generate fields, rather
than hacking the 'to_representation' method. This is the more idiomatic
way to do this.

The Meta class for the Project and Patch serializers are moved to the
bottom of the serializers and some variables renamed, both for
consistency's sake.

Signed-off-by: Stephen Finucane <stephen@that.guru>

REST: Remove '_url' suffixes

This was a design decision made when implementing the REST API. The
idea was that these items were URLs to related objects and should be
indicated as such. However, this was a faulty assumption as the
Patchwork API, unlike other some other APIs (GitHub), does not also
include a full representation of said objects, like so:

    {
      "url": "http://localhost:8000/api/1.0/patches/1/",
      ...
      "delegate_url": "http://localhost:8000/api/1.0/users/1",
      "delegate": {
        "url": "http://localhost:8000/api/1.0/users/1/",
        "username": "admin",
        "first_name": "",
        "last_name": "",
        "email": ""
      }
    }

Since there is no intention to support this design yet, there isn't
really any reason to fight django-rest-framework in appending these
suffixes. Simply remove them. This changes the output for most endpoints
from something like this:

    {
      "url": "http://localhost:8000/api/1.0/patches/1",
      ...
      "delegate_url": "http://localhost:8000/api/1.0/users/1"
    }

to:

    {
      "url": "http://localhost:8000/api/1.0/patches/1",
      ...
      "delegate": "http://localhost:8000/api/1.0/users/1"
    }

This will affect all endpoints with nested resources.

Note that the API version is not bumped as the API is still considered
unreleased.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Andy Doan <andy.doan@linaro.org>
Reviewed-by: Daniel Axtens <dja@axtens.net>

REST: Create 'api' directory

Move all REST API-related code into an 'api' directory. This allows us
to break the existing files into endpoint-based files and will allow us
to split the API into a different Django app in the future. This
involves simply shuffling code around for now, so there no functional
changes introduced.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>

.gitignore: Ignore mbox files

Signed-off-by: Stephen Finucane <stephen@that.guru>

docs: Highlight UID issue

This occurs on Fedora 24, Fedora 25. There was a brief mention of it in
the docker-compose file but no resolution was offered.

Signed-off-by: Stephen Finucane <stephen@that.guru>

pwclient: support 3-way merge

Add support for the 3-way merge in git-am.
This can be done via
  - the commandline
  - the global option section
  - the per-project section

Signed-off-by: Matthias Brugger <mbrugger@suse.com>
Reviewed-by: Stephen Finucane <stephen@that.guru>

bin/pwclient: accept alternate http_proxy forms

The forms for http_proxy (and the likes) is not very well defined nor
documented in any authoritaive place. However, there are two common
forms:  http://host:port  or  http://host:port/

Currently, the code chokes on the latter (e.g. with
http_proxy=http://127.0.0.1:8080/ ):

    [...]
      File "/usr/lib/python2.7/httplib.py", line 792, in _get_hostport
        raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
    httplib.InvalidURL: nonnumeric port: '8080/'

Chop off any slash character in the port definition to accept the second
form. If there is no '/' in there, it still works.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Stephen Finucane <stephenfinucane@hotmail.com>

forms: 'False' != False

Forms cast boolean values to strings, and attempting to coerce using the
'bool' function does not correctly return them to true boolean values.
Correct this by doing a string comparison instead.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Fixes: 0abde97aa ("forms: Use TypedChoiceField")

tests: Set 'date' when bulk creating patches

Apparently Django 1.8+ changed how patches with identical 'date' fields
were ordered, meaning tests that worked on these versions failed on
previous versions.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Fixes: ed5e93242 ("tests: Add rudimentary pagination tests")

tests: Replace 'force_login' with 'login'

The former is only available in Django 1.9+.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Fixes: ed5e93242 ("tests: Add rudimentary pagination tests")

xmlrpc: Expose patch hash to patch_get

Expose the patch object hash value to xmlrpc::patch_get. This is
particuarly helpful if some patche(s) have been submitted several times
but changed from e.g: RFC to a proper official patch submission. The
hash would typically be identical, but numbers would not.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
Reviewed-by: Stephen Finucane <stephen@that.guru>

forms: Trivial reshuffling of file

Group all "optional" fields.

Signed-off-by: Stephen Finucane <stephen@that.guru>

forms: Use TypedChoiceField

This resolves a TODO.

Signed-off-by: Stephen Finucane <stephen@that.guru>

forms: Remove duplication

DelegateField and OptionalDelegateField were basically duplicates of
other fields. Make use of parameters and remove them.

Signed-off-by: Stephen Finucane <stephen@that.guru>

forms: Remove unused args in forms

There were a couple of forms that offered configurabilty, but this
configurability wasn't used anywhere. Simplify things and remove any
related code.

Signed-off-by: Stephen Finucane <stephen@that.guru>

tests: Add rudimentary pagination tests

This should improve coverage and prevent regressions. The 'ppp' header
is removed as this is a non-standard header and not accessible from
a browser.

Signed-off-by: Stephen Finucane <stephen@that.guru>

views: Include 'List-Id' in mbox

Signed-off-by: Stephen Finucane <stephen@that.guru>
Closes-bug: #53

views: Don't munge the 'From' field of patches

At the moment patchwork always uses the official submitter name (as
patchwork understands it) as the "From" for patches that you receive.
This isn't quite what users expect and has some unfortunate
consequences.

The biggest problem is that patchwork saves the "official" name for an
email address the first time it sees an email from them.  If that name
is wrong (or was missing) patchwork will be confused even if future
emails from this person are fixed.  There are similar problems if a
user changes his/her name (get married?).

It seems better to just have each patch report the actual "From" that
was used to send that patch.  We'll still return the submitter in
'X-Patchwork-Submitter' just in case someone wants it.

Reported-by: Wolfram Sang <wsa@the-dreams.de>
Signed-off-by: Doug Anderson <dianders@chromium.org>
Signed-off-by: Stephen Finucane <stephen@that.guru>

trivial: PEP8 fixes

Signed-off-by: Stephen Finucane <stephen@that.guru>

pwclient: Fix encoding problems

All data returned by the xmlrpc object is unicode decoded with 'utf-8' (on
python 3, unicode == str). Add from __future__ import unicode_literals
to make sure that everything is unicode and avoid surprises.

On python 2, printing unicode to stdout causes it to be encoded to str
(byte string) with the 'ascii' codec:

  >>> print some_unicode_string
  ...
  UnicodeEncodeError: 'ascii' codec can't encode character u'\u0142'
  in position 468: ordinal not in range(128)

Work around ths by avoiding any explicit call to unicode() and by
replacing sys.stdout and sys.stderr by unicode-aware file objects (as
returned by io.open()).

Guess the encoding of stdout and stderr by looking at (in that order):
sys.stdout.encoding, locale.getpreferredencoding(), the PYTHONIOENCODING
environment variable. If no encoding is defined, assume 'utf-8' as
output encoding.

Signed-off-by: Robin Jarry <robin.jarry@6wind.com>
Tested-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Reviewed-by: Stephen Finucane <stephen@that.guru>

docs: Document how to backport patches

Signed-off-by: Stephen Finucane <stephen@that.guru>

pwclient: Fix Python 3 encoding of received strings

The conversion encode("utf-8") makes a byte stream which is
poorly printed with Python 3.
However this encoding is required for Popen.communicate() but must be
done after str.join() which applies to a real string.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Reviewed-by: Stephen Finucane <stephen@that.guru>

pwclient: Support proxy configuration

The environment variables http_proxy and https_proxy can be used
to configure the HTTP transport.

The TCP connection is made with the proxy host, whereas the original host
is maintained in the HTTP POST URI via "handler" in "send_request".

The send_request() method of xmlrpclib has a different signature
and behaviour in Python 2 and 3.

Fixes #47

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Reviewed-by: Stephen Finucane <stephen@that.guru>

pwclient: Rework HTTP authentication

Transform the HTTP authentication class into a generic transport class.
The credentials become optional so this transport class is always used.

A side effect is to fix the Python 3 support for the authentication.
Fixes #59

It will help to bring proxy support while combining http/https and
authentication cases.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Reviewed-by: Stephen Finucane <stephen@that.guru>

tools: Update to use 'hasher'

The old 'parser' module used to extract diffs from their surrounding
mbox fluff before hashing this. Seeing as this was only used in the
context of an actual git repo, avoid all of that rigmarole by just using
'git diff', which produces a plain diff, rather than 'git show'.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Tested-by: Tom Rini <trini@konsulko.com>
Closes-bug: #63

tools: Trivial formatting fixes

These tools are currently broken, but before beginning surgery let's
clean things up. Use standard 4 spaces and the longer, but easier to
read, if-else-fi syntax for comparison.

Missing license headers are added for completeness sake.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Tested-by: Tom Rini <trini@konsulko.com>

hasher: Create hasher module

This exposes the hashing functionality of Patchwork without requiring
Django or similar dependencies.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Tested-by: Tom Rini <trini@konsulko.com>

parser: Trivial rename of functions

'auto_delegate' doesn't actually delegate the patches - it merely finds
a suitable delegate based on the filename. Rename the function
accordingly.

'find_delegate' is also renamed to prevent confusion.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Tested-by: Tom Rini <trini@konsulko.com>

parser: Remove unused parameter

The find_content function expected a 'project' parameter but never
actually used it. Remove it and clean up tests accordingly.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Tested-by: Tom Rini <trini@konsulko.com>

views: don't duplicate tags in patch message when generating mbox

When generating an mbox for a patch with tags in the original commit
message, e.g.:

    Example patch

    This patch is awesome!

Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Acked-by: Russell Currey <ruscur@russell.cc>
the tags from the original email are duplicated:

    Example patch

    This patch is awesome!

Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Acked-by: Russell Currey <ruscur@russell.cc>
Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Acked-by: Russell Currey <ruscur@russell.cc>
It appears that during the refactoring in ef56359fb776 ("models: Merge
patch and first comment"), we added a call to patch.patch_responses() to
extract the tags from the initial patch email, which we then append to the
patch email body... which already has the tags in it.

Remove the unnecessary append of patch.patch_responses when generating an
mbox.

Fixes: ef56359fb776 ("models: Merge patch and first comment")
Reported-by: Russell Currey <ruscur@russell.cc>
Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Reviewed-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Stephen Finucane <stephen@that.guru>

trivial: Resolve pep8 issues

flake8 3.2.0 updates the pycodestyle dependency to 2.2.0, which in
turn resolves a bug with E305. Upgrading results in a small number of
additional pep8 issues. Resolve these now.

Signed-off-by: Stephen Finucane <stephen@that.guru>

views: Don't use attr(X, 'patch')

This doesn't play nice with Django 1.6 under Python 3.4, for some odd
reason.

Signed-off-by: Stephen Finucane <stephen@that.guru>

Add support for comment permalink

Signed-off-by: Stephen Finucane <stephen@that.guru>
Closes-bug: #39

forms: Tighten validation on BundleForm

The 'name' field of 'Bundle' cannot be blank, so update the form to
reflect this.

Signed-off-by: Stephen Finucane <stephen@that.guru>

admin: Integrate UserProfile fields into admin

The 'User' model is extended by means of a 'UserProfile' model. These
fields are not correctly displayed in the admin UI, but they should be.
Let's fix this per the recommendations of the Django docs [1].

[1] https://docs.djangoproject.com/en/1.10/topics/auth/customizing/#extending-the-existing-user-model

Signed-off-by: Stephen Finucane <stephen@that.guru>

Fix parsing of interesing series reply structures

There are some things you probably shouldn't do on public
mailing lists, but which people do anyway.

The first, and most understandable, is this:

          - [PATCH 1/2] test: Add some lorem ipsum
            - [PATCH 2/2] test: Convert to Markdown
              - [PATCH v2 1/2] test: Add some lorem ipsum
                - [PATCH v2 2/2] test: Convert to Markdown

We should correctly parse these by:
 - creating a new series if the version number changes
 - when deciding whether to create a SeriesReference, search by
   message-id alone, not the message-id/series pair. (Otherwise,
   we try to create a series ref for v1 2/2 in the series for v2,
   which breaks a uniqueness constraint.

The second, and less excusable, is this:

          - [PATCH 1/2] test: Add some lorem ipsum
            - [PATCH 2/2] test: Convert to Markdown
              - [PATCH 1/2] test: Add some lorem ipsum
                - [PATCH 2/2] test: Convert to Markdown

With this patch:
 - if we get a x/n for a series that already has an x/n, create a
   new series for it.

Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Stephen Finucane <stephen@that.guru>

requirements: Test older versions of DRF

We still care about Django 1.6 and 1.7, at least until 2.0 is released.
Start testing REST functionality on these versions by using older
versions.

Signed-off-by: Stephen Finucane <stephen@that.guru>

requirements: Bump django-rest-framework to 3.5

This requires explicitly declaring the 'field' parameter.

Signed-off-by: Stephen Finucane <stephen@that.guru>

requirements: Bump selenium to 3.0

3.0 introduces no notable changes for our use cases. Use the latest and
greatest.

Signed-off-by: Stephen Finucane <stephen@that.guru>

requirements: Bump django-debug-toolbar to 1.6

1.6 introduces little to no changes [1]. We already use the explicit
setup, so simply switch to the latest and greatest version.

[1] https://django-debug-toolbar.readthedocs.io/en/1.6/changes.html

Signed-off-by: Stephen Finucane <stephen@that.guru>

requirements: Loosen requirements for RDBMS libs

Signed-off-by: Stephen Finucane <stephen@that.guru>

Fix parsing of un-numbered messages in series

Say we are sent the following:

          - [PATCH 0/2] A sample series
            - [PATCH 1/2] test: Add some lorem ipsum
            - Random message with diff

We expect that:
 1) we parse normally without errors
 2) we get a series with a cover letter and a patch
 3) the random message is orphaned

What happens is that we get an integrity error, boiling down to:

(1048, "Column 'number' cannot be null")

This is caused because we believe that the random message belongs
to the series because of the headers, but because there are no
numbers in the Subject, we pass "None" into the number field of
SeriesPatch. That turns into a null, and rightly hits an integrity
error.

Fix this by requring that a message has a series _and_ a number
before we try to add it to the series.

Add a test to verify correctness.

Reported-by: Daniel Wagner <wagi@monom.org>
Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Stephen Finucane <stephen@that.guru>

travis: Disable sudo

We don't need it, and disabling it can make builds faster. Disable it.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>

travis: Integrate with codecov

This mostly involves enabling coverage as a default tox target. We add
pep8 too, while we're at it.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>

travis: Use tox-travis

This simplifies our .travis.yml file

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>

Remove 'htdocs/images'

Remove 'htdocs/images'

Signed-off-by: Bruno Mendes <brunosouza@alunos.utfpr.edu.br>
Reviewed-by: Stephen Finucane <stephen@that.guru>
Closes: #40

Remove 'htdocs/images'

Signed-off-by: Bruno Mendes <brunosouza@alunos.utfpr.edu.br>
Reviewed-by: Stephen Finucane <stephen@that.guru>
Closes: #40

CHANGELOG: Indicate series support

Signed-off-by: Stephen Finucane <stephen@that.guru>

templates: Integrate series view into patches

Patches for related series are listed on the patch page - this
provides a way to quickly grok a given patches location in a
series hierarchy.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Andy Doan <andy.doan@linaro.org>
Reviewed-by: Daniel Axtens <dja@axtens.net>
Tested-by: Russell Currey <ruscur@russell.cc>

templates: Integrate series support

Integrate support for series in the web UI. This is rather
straightforward, the only significant change being the addition of a
filter for series filtering.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Andy Doan <andy.doan@linaro.org>
Tested-by: Russell Currey <ruscur@russell.cc>
Reviewed-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>

filters: Handle invalid ids

Two filters - SubmitterFilter and DelegateFilter - don't attempt to
handle invalid id values and will bubble an exception up as a 5xx
error. Correct this.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>

tests: Add tests for series

Add a number of integration tests for parsing real series mbox files.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Andy Doan <andy.doan@linaro.org>
Tested-by: Russell Currey <ruscur@russell.cc>

tests: Add mbox files for series tests

Add a number of mbox files which we can use to unit test basic series
parsing functionality.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Andy Doan <andy.doan@linaro.org>
Tested-by: Russell Currey <ruscur@russell.cc>

parser: Add series parsing

It is now possible to parse and store series, so do just that.
The parsing at the moment is based on both RFC822 headers and
subject lines.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Andy Doan <andy.doan@linaro.org>
Reviewed-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Tested-by: Russell Currey <ruscur@russell.cc>

models: Add 'Series' model

Add a series model. This model is expected to act like a collection for
patches, similar to bundles but thread-orientated.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Andy Doan <andy.doan@linaro.org>
Reviewed-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Tested-by: Russell Currey <ruscur@russell.cc>

models: Convert functions to properties

A number of models contain functions that are, semantically speaking,
actually properties. Mark them as such.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Andy Doan <andy.doan@linaro.org>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>

Remove 'htdocs/images'

views: Update how patch counts are retrieved

It's no longer possible to access 'Project.patch_set' as
'Project.submission_set' has replaced it. This was missed when the
cover letter feature was merged, so resolve it now.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>
Closes-bug: #54

docs: Add getmail documentation

It seems a lot of people are having success using tools like fetchmail
in combination with IMAP/POP-capable email accounts like Gmail.  While
fetchmail itself is rather decrepit, the Python-based getmail seems
actively supported and pretty easy to configure. Document this process.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>

models: Remove TODO from 'refresh_tag_counts'

There's actually no harm keeping this around as it makes the code.
Remove the TODO, replacing it with a note on why the function exists.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>

models: Remove UserProfile.sync_person

This doesn't seem to do anything, and hasn't for a very long time
(pre-0.9.0, at least).

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>

models: Remove Bundle.n_patches

It doesn't really do much for us. Less LOC = win.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>

models: Make use of aggregates

We're well past Django 1.1 now, so resolve a TODO to use aggregate
support introduced in this version. As part of this change, replace
the use of 'count' to check for presence of matching objects with
'exists'.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Reviewed-by: Daniel Axtens <dja@axtens.net>