docs: Update readthedocs URLs

'readthedocs.io' is preferred to 'readthedocs.org'. Update all relevant
links.

Signed-off-by: Stephen Finucane <stephenfinucane@hotmail.com>

docs: Document SELinux issues

Initial Docker setup caused me some heartache on Fedora. Briefly
document the issues encountered to help other users.

Signed-off-by: Stephen Finucane <stephenfinucane@hotmail.com>
Reviewed-by: Daniel Axtens <dja@axtens.net>

docs: Make Docker the recommended install method

Docker is faster, lighter and "hipper" than Vagrant. Promote this
method over the Vagrant approach.

Signed-off-by: Stephen Finucane <stephenfinucane@hotmail.com>
Reviewed-by: Daniel Axtens <dja@axtens.net>

docker: Move 'docker' directory to 'tools'

...like the Vagrant scripts (which have been renamed also).

Signed-off-by: Stephen Finucane <stephenfinucane@hotmail.com>
Acked-by: Daniel Axtens <dja@axtens.net>

docs: Add a paragraph about the django admin console

Provide a few hints about how to get initial access to the django admin
console as well as a few essential configuration hints regarding
projects and sites setup.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
Reviewed-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>

pwclient: Unhardcode prints to ~/.pwclientrc

Even though the default location is ~/.pwclientrc, we can change it, so
use CONFIG_FILE wherever we can to make the prints match the actual file
location.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>

tox: Disable F405

Recent versions of tox throw a lot of errors when using star-imports.
While these warnings are generally justified, the settings files
require this. Since 'noqa'-ing the offending lines would be rather
arduous, simply disable the warning instead.

Signed-off-by: Stephen Finucane <stephenfinucane@hotmail.com>

trivial: Resolve PEP8 isues

These were introduced with recent commits.

Signed-off-by: Stephen Finucane <stephenfinucane@hotmail.com>

Enable tox tests in Docker

Update the entrypoint script to allow you to do --tox and
--quick-tox in the same way as --test and --quick-test.
Update the readme appropriately.

Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>

Pass more environment variables through to tox

In order to get tox tests working, we want db host, db port, home,
and xauthority.

Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>

Don't import the REST framework if it is disabled

In the disabled case, also stub out the APITestCase class.

Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>

settings: only enable the REST framework for Django 1.7+

This helps with tox tests for Django 1.6

Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>

Convert RunSQL migration commands from list to strings

The ability to use a list was added in Django 1.8 - compare

https://docs.djangoproject.com/en/1.7/ref/migration-operations/#runsql
https://docs.djangoproject.com/en/1.8/ref/migration-operations/#runsql

We don't need a list as we run only 1 SQL command. So convert to
strings. This helps unbreak Django 1.7.

Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>

Add Docker IP to whitelist for django-debug-toolbar

Yay for debugging with Docker.

Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>

Allow use of Docker for development

This makes it possible to use Docker and docker-compose for development
as an alternative to Vagrant.

I quite liked vagrant a couple of years ago, but currently:

 * Trying to install VirtualBox on Ubuntu wants me to disable
   Secure Boot, and I don't want to do that.

 * Trying to use the libvirt plugin for vagrant requires I pick
   from a very small set of possible images, and requires that I
   install the upstream vagrant rather than the vagrant shipped
   with Ubuntu 16.04

 * I find docker containers faster to work with and more transparent.

So I've done the work to make docker work for Patchwork development.
This doesn't break or in any way interfere with using Vagrant, it just
provides an alternative.

It includes support for headless selenium tests using Chromium.

Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>

Put test db host and port into env variables

This is preparation for a Docker-based dev environment.

Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>

Ignore *~ and #*# files.

Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>

Fix test_login in test_user_browser.py

It tried to use the pbkdf2 hash as the password. Use the username instead, as that
is what create_user sets.

Then it compared the test user username to testuser, rather than the username, which
is dynamically generated. Compare to the generated username.

Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>

Fix failure to start with uninitalised database

An OptionalModelChoiceField will attempt to query the database to get choices
in its __init__ method. This fails if the database hasn't been initialised
yet. So, put that in a try/catch block. This lets things work through the
migration and loading of data from fixtures.

Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>

settings: Add PASSWORD_HASHER

This improves run time by a few seconds, and is worth having.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Tested-by: Andy Doan <andy.doan@linaro.org>

settings: Regroup sections per upstream docs

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>

docs: Alter heading of development install guide

Divide the document into a Vagrant-based install section, and a manual
install section (+ footnotes).

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

xmlrpc: Don't load all Patches into memory

The way that reverse indexing of patches was implemented is broken.
At present, it will retrieve all patches in memory and return the
length from that data, then the slicing operation will then happen
without querying the DB and slice the results cached from the len()
evaluation. This is memory intensive, particularly for larger
instances.

Take advantage of Django's lazy loading to avoid this.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Suggested-by: Damien Lespiau <damien.lespiau@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Address nits in 'utils'

Simplify object creation through creation of the 'objects.create'
shortcut. In addition, remove an unncessary 'save' call.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

parsemail: Ignore multiple cover letters

If multiple cover letters are found, parsemail dies with a
MultipleObjectsReturned exception.  This is particularly problematic in
the middle of a parsearchive run, so just ignore multiple cover letters.

Signed-off-by: Russell Currey <ruscur@russell.cc>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

uwsgi: Add python plugin

Given the documentation suggests using Ubuntu, and any Debian derivative
requires manually specifying plugins in uwsgi, add the python27 plugin to
the example uwsgi settings file.

Signed-off-by: Russell Currey <ruscur@russell.cc>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

docs/deployment: Fix systemctl commands

systemctl takes the name of the operation before the name of the service
so that you can do multiple things at once, i.e. "systemctl status nginx
postgresql".  Fix the ordering of the arguments.

Signed-off-by: Russell Currey <ruscur@russell.cc>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

tests: Don't create model objects unless necessary

Creating model objects when not required introduces unnecessary
overhead and delays the tests. Avoid this by only creating new objects
when one isn't provided for us already.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Remove 'default_states' fixture requirement

The 'default_states' fixture is used in many tests. However, the
'default_states' are merely suggestions for various States, and the
names themselves have no meaning from a patchwork perspective. In
addition, these fixtures significantly increase run time of the tests.

Resolve this by creating new State objects on demand, rather than
using versions provided by the fixture.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Remove 'defaults' class

This is no longer needed and can be removed.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_rest_api'

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Move 'create_check' to utils

This is also useful for the REST API tests.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Rework and rename 'test_patchparser'

* Move some functions and variables from 'tests.utils', where they are
  only used in this file
* Rename and regroup some tests to make more sense
* Rename to 'test_parser', as this parses more than patches now

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_patchparser'

* Make use of 'create_' helper functions
* Include every import on its own line
* Use underscore_case, rather than camelCase
* Rename some functions to make more sense

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_xmlrpc'

* Use underscore_case, rather than camelCase

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_user_browser'

* Don't use hardcode routes: use the reverse function instead

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_user'

* Don't use hardcode routes: use the reverse function instead
* Make use of 'create_' helper functions
* Minimize duplication of code
* Remove unneeded 'XXX.objects.delete()' calls (all objects are deleted
  on teardown of each test)
* Include every import on its own line
* Use underscore_case, rather than camelCase

This includes one trivial, albeit necessary, removal of an import from
'test_user_browser'.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_updates'

* Make use of 'create_' helper functions
* Include every import on its own line
* Use underscore_case, rather than camelCase
* Be consistent in how 'Patch.objects.get()' is called

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_tags'

* Make use of 'create_' helper functions
* Remove unneeded 'XXX.objects.delete()' calls (all objects are deleted
  on teardown of each test)
* Use underscore_case, rather than camelCase

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Rename 'test_person'

Reflect the purpose of these tests - validation of the autocomplete
feature - in the filename.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_person'

* Don't use hardcode routes: use the reverse function instead
* Make use of 'create_' helper functions
* Include every import on its own line
* Use underscore_case, rather than camelCase

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_notifications'

* Make use of 'create_' helper functions
* Remove unneeded 'XXX.objects.delete()' calls (all objects are deleted
  on teardown of each test)
* Include every import on its own line
* Use underscore_case, rather than camelCase

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_mboxviews'

* Don't use hardcode routes: use the reverse function instead
* Make use of 'create_' helper functions
* Include every import on its own line
* Use underscore_case, rather than camelCase
* Rename and regroup some tests to make more sense

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_mail_settings'

* Make use of 'create_' helper functions
* Include every import on its own line
* Use underscore_case, rather than camelCase
* Don't use class level variables when not required

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_list'

* Make use of 'create_' helper functions
* Include every import on its own line
* Use underscore_case, rather than camelCase
* Don't use list comprehension without generating a list: use a for
  loop instead

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_filters'

* Don't use hardcode routes: use the reverse function instead
* Make use of 'create_' helper functions
* Use underscore_case, rather than camelCase

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_expiry'

* Make use of 'create_' helper functions
* Include every import on its own line
* Use underscore_case, rather than camelCase

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_encodings'

* Don't use hardcode routes: use the reverse function instead
* Make use of 'create_' helper functions
* Remove unneeded 'XXX.objects.delete()' calls (all objects are deleted
  on teardown of each test)
* Include every import on its own line
* Use underscore_case, rather than camelCase

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_detail'

* Use underscore_case, rather than camelCase

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_confirm'

* Make use of 'create_' helper functions
* Remove unneeded 'XXX.objects.delete()' calls (all objects are deleted
  on teardown of each test)
* Include every import on its own line
* Use underscore_case, rather than camelCase

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_checks'

* Make use of 'create_' helper
* Remove unneeded 'XXX.objects.delete()' calls (all objects are deleted
  on teardown of each test)
* Include every import on its own line

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Clean up 'test_bundles'

* Don't use hardcode routes: use the reverse function instead
* Make use of 'create_' helper functions
* Minimize duplication of code
* Use underscore_case, rather than camelCase

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Add additional 'create_' helpers

Add some additional object creation helpers, as a way to help
eventually remove the 'default' class.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Remove use of global value

This isn't needed, so remove it.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

tests: Remove 'find_in_context'

The aformentioned function handled both dicts and lists in a request's
context field. Seeing as only dicts are ever returned, this is not
necessary.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

templates: Display '[owner]/[context]' for checks

This is useful for highlighting ownership of a check.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

models: Don't group checks with different owners

This prevents CIs from overriding the results of another CI by using
the same context.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

models: Return QuerySet from 'Patch.checks'

Previously this returned a list. However, a QuerySet is more flexible
and is required to provide a generic check endpoint at some point in
the future.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

models: Return string from 'combined_check_status'

Previously the 'combined_check_status' function returned the checks
enum integer for the status field. However, the string representation
is more meaningful and is the only representation seen by users at
moment. Change this function so it returns, for example, 'success'
instead of '1'.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

models: Use non-null slugs for 'Check.name'

The schema for 'Check' defines 'Check.name' as a 'CharField'. This is
less than ideal as names with spaces and special characters can't be
represented cleanly in URLs etc. We should use 'SlugField' instead.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>
Closes: #33

tests: Don't save multiple projects

There were issues introduced in the rebase of 'ee15585' that resulted
in two projects being saved with the same ID for one test, leading to
a test failure. Resolve this.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>

REST: Allow projects to be retrieved by linkname

Building a user-friendly CLI becomes difficult when project-ids are
required. It also makes it almost impossible to work with the current
format of the .pwclientrc file.

Signed-off-by: Andy Doan <andy.doan@linaro.org>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

REST API: expose patch.checks

Signed-off-by: Andy Doan <andy.doan@linaro.org>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

REST: Add Patch.mbox_url

Provide a URL to the raw patch.

Signed-off-by: Andy Doan <andy.doan@linaro.org>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

REST: Add Patch Checks to the API

This exports patch checks via the REST API.

The drf-nested-routers package is used to handle the fact Checks are
nested under a Patch.

Security Constraints:
 * Anyone (logged in or not) can read all objects.
 * No one can update/delete objects.
 * Project maintainers and patch owners may create objects.

Signed-off-by: Andy Doan <andy.doan@linaro.org>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

REST: Add Patches to the API

This exposes patches via the REST API.

Security Constraints:
 * Anyone (logged in or not) can read all objects.
 * No one can create/delete objects.
 * Project maintainers are allowed to update (ie "patch"
   attributes)

NOTE: Patch.save was overridden incorrectly and had to be
fixed to work with DRF.

Signed-off-by: Andy Doan <andy.doan@linaro.org>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>

REST: Add Persons to the API

This exports person objects via the REST API.

Security Constraints:
 * The API is read-only to authenticated users

Signed-off-by: Andy Doan <andy.doan@linaro.org>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

REST: Add Users to the API

This exports user objects via the REST API.

Security Constraints:
 * The API is read-only to authenticated users

Signed-off-by: Andy Doan <andy.doan@linaro.org>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>

REST: Add Projects to the API

This exports projects via the REST API.

Security Constraints:
 * Anyone (logged in or not) can read all objects.
 * No one can create/delete objects.
 * Project maintainers are allowed to update (ie "patch"
   attributes)

Signed-off-by: Andy Doan <andy.doan@linaro.org>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Inspired-by: Damien Lespiau <damien.lespiau@intel.com>

REST: Add base configuration hooks for a REST API

This adds the ability to expose a REST API based on the Django REST
framework project. Since this project isn't packaged in most current
distributions, we ensure that its both installed and enabled before
trying to use it.

Signed-off-by: Andy Doan <andy.doan@linaro.org>
Inspired-by: Damien Lespiau <damien.lespiau@intel.com>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

docs: Add prototype API specification

Add specification for a REST API. It documents a number of endpoints
for the following objects/models:

* Projects
* People
* Patches
* Checks

The specification is provided in OpenAPI format [1]. This specification
can be used to automatically generate REST client libraries and to
validate the implemented API.

The API is currently read-only. It is expected that this will expanded
upon as the API grows.

[1] https://openapis.org/specification

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Andy Doan <andy.doan@linaro.org>

post-receive: Use correct default installation path

The documentation recommends to use /opt/patchwork instead of
/srv/patchwork since version 1.1. This change should be reflected in the
PWDIR variable of the post-receive hook.

Signed-off-by: Sven Eckelmann <sven@narfation.org>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

patchwork/templates: Fix path to delete button image

Signed-off-by: Sven Eckelmann <sven@narfation.org>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

post-receive.hook: rename detection

patches which descripes renamed files show different hashes with git
show and pwclient. Use git show -C (Detect copies as well as renames.)

Signed-off-by: Jan Remmet <j.remmet@phytec.de>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

parsemail: Fix typo

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

pwclient: Fix typos

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

tests: Fix spelling mistakes and typos

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

templates: Fix spelling mistakes and typos

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

sql: Fix spelling mistakes and typos

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

docs: Fix spelling mistakes and typos

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>

parsearchive: Show breakdown of parsed mails

Since the parse_mail function now returns a single object, it is
easy to keep stats for the number and type of different mails parsed.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

parsemail: Always update Person.email

Patches applied using pwclient will always use the first name that has
been seen for a particular email address. While this is usually fine,
ot actually causes issues for people changing name without changing
email addresses or people changing the capitalization of their name.
Resolve this by always updating the Person object when we receive a
new submission or comment.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>
Closes: #24

parsemail: Handle cover letters sent in reply

Some people send cover letters (and therefore series) as replies to
existing series. Apply a second set of heuristics to cover this case:

* The message contains a '[0/n]' marker tag in the subject
* The message is not the root message, but the subject is unique, i.e.
  it is not a reply to a cover letter already stored in Patchwork

It is theoretically possible that the message could be a reply to a
cover letter that's been missed, but such occurences should be rare
and only get rarer the longer an instance is running.

Note that there is a third case which is not yet covered: a cover
letter with the same name but a different number of patches. This
will have to wait until series support is fully implemented.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

parsemail: Add cover letter parsing

Add support for both cover letters and comments on cover letters. This
works using the following heuristics:

* The message contains a '[0/n]' marker tag in the subject
* The message is the root message

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

parsemail: Be consistent in variable naming

Don't refer to a variable using different names in different
locations unless there's a good reason to do so.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

parsemail: Use consistent 'find_' format

Most of the functions in the parsemail file work by taking a Mail
object, extracting the data they require, and formatting this data to
the expected format. This is not applied across the board, however.
Fix this oversight.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

parsemail: Restructure 'parse_content'

The 'parse_content' function is designed, as the title suggests, to
parse the content of an email. It should not be used to build objects
or similar. Move this functionality out of the function, updating the
unit tests as necessary.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

parsemail: Flatten 'try_decode' method

This function is small, is only called once and isn't unit tested. Save
a few lines and some cognitive effort by folding it in where it's used.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

parsemail: Extract building the list of mail references

We'll need to figure out whether the mail we are parsing is the root of
the thread to automatically build series, and we'll need the list of
references for that.

Signed-off-by: Damien Lespiau <damien.lespiau@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>

parsemail: Parse series markers e.g. "1/12"

This can be used to identify cover letters, patches part of series,
length of series, etc.

Signed-off-by: Damien Lespiau <damien.lespiau@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>

parsemail: Extract prefixes from subject

The patch is a preparation step towards understanding series. It will be
handy to parse those prefixes, looking for 'x/n' to retrieve the order
of a patch in a series.

Signed-off-by: Damien Lespiau <damien.lespiau@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>

views: Integrate cover letter support

There isn't really any need to list cover letters right now, seeing as
they're really only valuable in the context of series. However, if
someone requests a cover letter by ID then this should be displayed.
To this effect, add a new "covers" endpoint that can display the basic
elements of the cover letter. This includes redirects from/to the
"patches" endpoint.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Reviewed-by: Andy Doan <andy.doan@linaro.org>

pep8: Resolve issues in settings file

Introduced in '2822854'.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>

tests: Fix issue with user profile page test

The title of the user profile page was changed in 'b3d1917', but the
test wasn't updated to reflect this. Fix this.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>

settings: Remove hard coded login url value

Use the url name for the login page rather than a hard coded value
this means that if you have patchwork in a prefix it will redirect
to the correct location without any additional configuration.

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>

models: Add Cover Letter support

Cover letters contain useful information that should not be discarded.
Store them.

This is a prerequisite of series support.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>

models: Split Patch into two models

There are a lot of similarities between cover letters and patches: so
many, in fact, that it would be helpful to occasionally treat them as
the same thing. Achieve this by extracting out the fields that would be
shared between a Patch and a hypothetical cover letter into a "sub
model". This allows us to do cool stuff like assigning comments to both
patches and cover letters or listing both patches and cover letters on
the main screen in a natural way.

The migrations for this are really the only complicated part. There are
three, broken up into schema and data migrations per Django customs,
and they works as follows:

* Rename the 'Patch' model to 'Submission', then create a subclass
  called 'Patch' that includes duplicates of the patch-specific fields
  of Submission (with changed names to prevent conflicts). Rename
  non-patch specific references to the renamed 'Submission' model
  as necessary.
* Duplicate the contents of the patch-specific fields from 'Submission'
  to 'Patch'
* Remove the patch-specific fields from 'Submission', renaming the
  'Patch' model to take their place. Update the patch-specific
  references to point the new 'Patch' model, rather than 'Submission'.

This comes at the cost of an additional JOIN per item on the main
screen, but this seems a small price to pay for the additional
functionality gained. To minimise this, however, caching will be added.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Andy Doan <andy.doan@linaro.org>

utils: Rework how notifications are selected

The current method of filtering valid patches from invalid ones is
effective, but incompatible with forthcoming rework that splits
the 'Patch' model into two models. Rework this, removing the
soon-to-be non-existant 'User.patch' reference.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>

parsearchive: Fix default value of verbosity argument

With the current parsearchive.py, the default value for the verbosity
argument does not exist in the VERBOSITY_LEVELS dict, so we get:

  Traceback (most recent call last):
    File "./patchwork/bin/parsemail.py", line 569, in <module>
      sys.exit(main(sys.argv))
    File "./patchwork/bin/parsemail.py", line 555, in main
      logging.basicConfig(level=VERBOSITY_LEVELS[args['verbosity']])
  KeyError: 20

This change uses an actual key instead.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>

ui: Add user bundle link into the dropdown menu

Since the link to TODOs (reviews pending) can already be seen there,
it makes sense to link to bundles also.

A divider is added to logically group these elements.

Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>