Aspects of both Patchwork and Django, such as the 'makemigrations'
management command, function differently under Python 3. It should be
possible to run these commands under Python 2. Install dependencies in
the Python 2 environment to make this a possibility.
Signed-off-by: Stephen Finucane <stephenfinucane@hotmail.com> Acked-by: Daniel Axtens <dja@axtens.net>
Daniel Axtens [Tue, 9 Aug 2016 04:27:41 +0000 (14:27 +1000)]
Add .travis.yml file
This automatically runs the tests (except the browser ones) via tox.
Tested on my github account. This will require some setup on the GitHub
end: Travis will need to be turned on for the project, and ideally a
badge added to the readme so we can see the build status.
Signed-off-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Stephen Finucane <stephenfinucane@hotmail.com>
Thomas Monjalon [Tue, 6 Sep 2016 20:37:03 +0000 (22:37 +0200)]
lib: Refresh nginx/uwsgi configuration
As stated in http://uwsgi-docs.readthedocs.io/en/latest/Nginx.html:
"
Unfortunately nginx is not able to rewrite PATH_INFO accordingly to
SCRIPT_NAME. For such reason you need to instruct uWSGI to map specific
apps in the so called “mountpoint” and rewrite SCRIPT_NAME and PATH_INFO
automatically
[...]
ancient uWSGI versions used to support the so called “uwsgi_modifier1 30”
approach. Do not do it. it is a really ugly hack
"
Replacing the uwsgi_modifier1 hack by a mount point seems to work.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
In '8e58c29', code was added to delete '.pyc' files. However, this was
later reverted in '9fd161' due to the increase in run time that this
incurred. However, the need to avoid stale '.pyc' files still exists.
To resolve this, add the 'PYTHONDONTWRITEBYTECODE' environment option
to disable generation of these files. This includes a slight
performance increase.
Signed-off-by: Stephen Finucane <stephenfinucane@hotmail.com>
Daniel Axtens [Sat, 3 Sep 2016 07:07:20 +0000 (17:07 +1000)]
xmlrpc: massively increase test coverage
XMLRPC defines a bunch of very similar methods for getting and
listing a bunch of different models. So create a mixin that allows
us to test an arbitrary model, then use that to create test classes
to test the various models.
Add other tests as well.
Signed-off-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Stephen Finucane <stephenfinucane@hotmail.com>
Daniel Axtens [Sat, 3 Sep 2016 07:07:17 +0000 (17:07 +1000)]
xmlrpc: catch possible exceptions in patch filtering
Currently, filtering by project, submitter, delegate or state uses a
filter(id=filt[key])[0]. This will throw an exception when something
isn't found, as filter will return [], and getting the first element of
that will fail.
Convert them to explicit get()s, so it's clearer that they can throw an
exception, then catch the 3 possible types of DoesNotExists exceptions.
Signed-off-by: Daniel Axtens <dja@axtens.net> Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>
Daniel Axtens [Sat, 3 Sep 2016 07:07:14 +0000 (17:07 +1000)]
xmlrpc: Change filter(...)[0] to get(...)
filter() returns a list or [], and get() returns an object or raises an
exception.
Currently, we use filter()[0] inside a try/except model.DoesNotExist
block. This doesn't work: if filter returns [] we'll get an exception
about list indices, not a DoesNotExist exception.
Switch these over to get(), allowing them to use the existing exception
flow.
Signed-off-by: Daniel Axtens <dja@axtens.net> Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>
Stephen Finucane [Mon, 18 Apr 2016 10:21:09 +0000 (11:21 +0100)]
parser: Correct empty email value check
The check for empty emails in 'find_author' checked for None, but it
was not possible to ever return None unless the 'From:' header was
missing altogether. Seeing as, per RFC822 and its revisions, this is
not possible, check for the empty string instead.
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Thu, 14 Apr 2016 16:38:57 +0000 (17:38 +0100)]
parser: Put more specific 'From' regex first
Multiple different 'From:' header styles are supported by the parser,
but some of these are less specific than others. Attempt to parse using
a more specific style before going with the more generic one.
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Sun, 10 Apr 2016 12:45:13 +0000 (13:45 +0100)]
parsemail: Move parsing to 'parser'
Separate the parsing of mails from the CLI aspects of parsemail. Since
there is already a 'parser' module, it makes sense to place the parsing
functionality here. This will allow for additional uses of this parsing
functionality in the future.
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Sun, 10 Apr 2016 12:45:13 +0000 (13:45 +0100)]
trivial: Cleanup of 'parser'
Do some cleanup of the file by removing excess whitespace, adding some
documentation, removing shadowing of keywords and renaming some
functions to more accurately reflect their purpose.
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Sun, 21 Aug 2016 17:06:55 +0000 (18:06 +0100)]
requirements: Update django debug toolbar to 1.5
This is the latest version. This breaks compatibility with Django 1.7,
but this version is unsupported and should not be used for development.
We do, however, continue to test support for Django 1.7 and 1.6.
This also prevents the need to manually track sqlparse dependency [1].
Daniel Axtens [Tue, 9 Aug 2016 04:55:23 +0000 (14:55 +1000)]
tox: Add Python 3.5
Python 3.5 is shipped with Ubuntu 16.04, and it's the new stable
release. Add support for this runtime in tox. Note that Django 1.6 and
1.7 don't work with Python 3.5, so these are excluded.
Signed-off-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Stephen Finucane <stephenfinucane@hotmail.com>
Daniel Axtens [Fri, 26 Aug 2016 06:55:59 +0000 (16:55 +1000)]
docker: Install Python 3.4
Currently, we tox test against Python 3.4. Python 3.4 is included with
Ubuntu 14.04, which is supported for several years to come, so we want
to keep supporting it. However, Python 3.4 isn't included with Ubuntu
16.04, which is what the Docker image is based on.
We could downgrade the container to 14.04, but then we'd struggle to
get Python 3.5 into the container. Python 3.5 is the most recent in the
Python 3 series, so we should also be supporting it.
Add the apt sources for Trusty and pull in Python 3.4 from there.
Signed-off-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Stephen Finucane <stephenfinucane@hotmail.com>
Florian Fainelli [Tue, 30 Aug 2016 01:12:15 +0000 (18:12 -0700)]
docs: Add a paragraph about the django admin console
Provide a few hints about how to get initial access to the django admin
console as well as a few essential configuration hints regarding
projects and sites setup.
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com> Reviewed-by: Daniel Axtens <dja@axtens.net> Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>
Florian Fainelli [Tue, 30 Aug 2016 01:12:14 +0000 (18:12 -0700)]
pwclient: Unhardcode prints to ~/.pwclientrc
Even though the default location is ~/.pwclientrc, we can change it, so
use CONFIG_FILE wherever we can to make the prints match the actual file
location.
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com> Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>
Stephen Finucane [Sat, 13 Aug 2016 22:43:04 +0000 (23:43 +0100)]
tox: Disable F405
Recent versions of tox throw a lot of errors when using star-imports.
While these warnings are generally justified, the settings files
require this. Since 'noqa'-ing the offending lines would be rather
arduous, simply disable the warning instead.
Signed-off-by: Stephen Finucane <stephenfinucane@hotmail.com>
Daniel Axtens [Tue, 9 Aug 2016 04:38:45 +0000 (14:38 +1000)]
Add Docker IP to whitelist for django-debug-toolbar
Yay for debugging with Docker.
Signed-off-by: Daniel Axtens <dja@axtens.net> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>
Daniel Axtens [Tue, 9 Aug 2016 04:27:40 +0000 (14:27 +1000)]
Allow use of Docker for development
This makes it possible to use Docker and docker-compose for development
as an alternative to Vagrant.
I quite liked vagrant a couple of years ago, but currently:
* Trying to install VirtualBox on Ubuntu wants me to disable
Secure Boot, and I don't want to do that.
* Trying to use the libvirt plugin for vagrant requires I pick
from a very small set of possible images, and requires that I
install the upstream vagrant rather than the vagrant shipped
with Ubuntu 16.04
* I find docker containers faster to work with and more transparent.
So I've done the work to make docker work for Patchwork development.
This doesn't break or in any way interfere with using Vagrant, it just
provides an alternative.
It includes support for headless selenium tests using Chromium.
Signed-off-by: Daniel Axtens <dja@axtens.net> Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>
Daniel Axtens [Tue, 9 Aug 2016 04:27:39 +0000 (14:27 +1000)]
Put test db host and port into env variables
This is preparation for a Docker-based dev environment.
Signed-off-by: Daniel Axtens <dja@axtens.net> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>
Daniel Axtens [Tue, 9 Aug 2016 04:27:38 +0000 (14:27 +1000)]
Ignore *~ and #*# files.
Signed-off-by: Daniel Axtens <dja@axtens.net> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>
Daniel Axtens [Tue, 9 Aug 2016 04:27:37 +0000 (14:27 +1000)]
Fix test_login in test_user_browser.py
It tried to use the pbkdf2 hash as the password. Use the username instead, as that
is what create_user sets.
Then it compared the test user username to testuser, rather than the username, which
is dynamically generated. Compare to the generated username.
Signed-off-by: Daniel Axtens <dja@axtens.net> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>
Daniel Axtens [Tue, 9 Aug 2016 04:27:36 +0000 (14:27 +1000)]
Fix failure to start with uninitalised database
An OptionalModelChoiceField will attempt to query the database to get choices
in its __init__ method. This fails if the database hasn't been initialised
yet. So, put that in a try/catch block. This lets things work through the
migration and loading of data from fixtures.
Signed-off-by: Daniel Axtens <dja@axtens.net> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> Reviewed-by: Stephen Finucane <stephenfinucane@hotmail.com>
Stephen Finucane [Wed, 13 Jul 2016 09:57:29 +0000 (10:57 +0100)]
xmlrpc: Don't load all Patches into memory
The way that reverse indexing of patches was implemented is broken.
At present, it will retrieve all patches in memory and return the
length from that data, then the slicing operation will then happen
without querying the DB and slice the results cached from the len()
evaluation. This is memory intensive, particularly for larger
instances.
Take advantage of Django's lazy loading to avoid this.
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Suggested-by: Damien Lespiau <damien.lespiau@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Russell Currey [Fri, 15 Jul 2016 06:29:10 +0000 (16:29 +1000)]
parsemail: Ignore multiple cover letters
If multiple cover letters are found, parsemail dies with a
MultipleObjectsReturned exception. This is particularly problematic in
the middle of a parsearchive run, so just ignore multiple cover letters.
Signed-off-by: Russell Currey <ruscur@russell.cc> Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>
Russell Currey [Fri, 15 Jul 2016 05:42:09 +0000 (15:42 +1000)]
uwsgi: Add python plugin
Given the documentation suggests using Ubuntu, and any Debian derivative
requires manually specifying plugins in uwsgi, add the python27 plugin to
the example uwsgi settings file.
Signed-off-by: Russell Currey <ruscur@russell.cc> Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>
Russell Currey [Fri, 15 Jul 2016 04:46:28 +0000 (14:46 +1000)]
docs/deployment: Fix systemctl commands
systemctl takes the name of the operation before the name of the service
so that you can do multiple things at once, i.e. "systemctl status nginx
postgresql". Fix the ordering of the arguments.
Signed-off-by: Russell Currey <ruscur@russell.cc> Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>
Stephen Finucane [Thu, 30 Jun 2016 17:30:27 +0000 (18:30 +0100)]
tests: Don't create model objects unless necessary
Creating model objects when not required introduces unnecessary
overhead and delays the tests. Avoid this by only creating new objects
when one isn't provided for us already.
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
The 'default_states' fixture is used in many tests. However, the
'default_states' are merely suggestions for various States, and the
names themselves have no meaning from a patchwork perspective. In
addition, these fixtures significantly increase run time of the tests.
Resolve this by creating new State objects on demand, rather than
using versions provided by the fixture.
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Tue, 21 Jun 2016 21:20:10 +0000 (22:20 +0100)]
tests: Rework and rename 'test_patchparser'
* Move some functions and variables from 'tests.utils', where they are
only used in this file
* Rename and regroup some tests to make more sense
* Rename to 'test_parser', as this parses more than patches now
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Tue, 21 Jun 2016 20:45:06 +0000 (21:45 +0100)]
tests: Clean up 'test_patchparser'
* Make use of 'create_' helper functions
* Include every import on its own line
* Use underscore_case, rather than camelCase
* Rename some functions to make more sense
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Mon, 20 Jun 2016 10:43:32 +0000 (11:43 +0100)]
tests: Clean up 'test_user'
* Don't use hardcode routes: use the reverse function instead
* Make use of 'create_' helper functions
* Minimize duplication of code
* Remove unneeded 'XXX.objects.delete()' calls (all objects are deleted
on teardown of each test)
* Include every import on its own line
* Use underscore_case, rather than camelCase
This includes one trivial, albeit necessary, removal of an import from
'test_user_browser'.
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Tue, 21 Jun 2016 17:04:56 +0000 (18:04 +0100)]
tests: Clean up 'test_updates'
* Make use of 'create_' helper functions
* Include every import on its own line
* Use underscore_case, rather than camelCase
* Be consistent in how 'Patch.objects.get()' is called
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Tue, 21 Jun 2016 14:23:25 +0000 (15:23 +0100)]
tests: Clean up 'test_tags'
* Make use of 'create_' helper functions
* Remove unneeded 'XXX.objects.delete()' calls (all objects are deleted
on teardown of each test)
* Use underscore_case, rather than camelCase
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Tue, 21 Jun 2016 10:13:41 +0000 (11:13 +0100)]
tests: Clean up 'test_person'
* Don't use hardcode routes: use the reverse function instead
* Make use of 'create_' helper functions
* Include every import on its own line
* Use underscore_case, rather than camelCase
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Tue, 21 Jun 2016 08:20:42 +0000 (09:20 +0100)]
tests: Clean up 'test_notifications'
* Make use of 'create_' helper functions
* Remove unneeded 'XXX.objects.delete()' calls (all objects are deleted
on teardown of each test)
* Include every import on its own line
* Use underscore_case, rather than camelCase
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Mon, 20 Jun 2016 16:02:49 +0000 (17:02 +0100)]
tests: Clean up 'test_mboxviews'
* Don't use hardcode routes: use the reverse function instead
* Make use of 'create_' helper functions
* Include every import on its own line
* Use underscore_case, rather than camelCase
* Rename and regroup some tests to make more sense
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Mon, 20 Jun 2016 15:11:44 +0000 (16:11 +0100)]
tests: Clean up 'test_mail_settings'
* Make use of 'create_' helper functions
* Include every import on its own line
* Use underscore_case, rather than camelCase
* Don't use class level variables when not required
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Mon, 20 Jun 2016 14:39:14 +0000 (15:39 +0100)]
tests: Clean up 'test_list'
* Make use of 'create_' helper functions
* Include every import on its own line
* Use underscore_case, rather than camelCase
* Don't use list comprehension without generating a list: use a for
loop instead
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Mon, 20 Jun 2016 14:07:50 +0000 (15:07 +0100)]
tests: Clean up 'test_encodings'
* Don't use hardcode routes: use the reverse function instead
* Make use of 'create_' helper functions
* Remove unneeded 'XXX.objects.delete()' calls (all objects are deleted
on teardown of each test)
* Include every import on its own line
* Use underscore_case, rather than camelCase
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Mon, 20 Jun 2016 13:47:24 +0000 (14:47 +0100)]
tests: Clean up 'test_confirm'
* Make use of 'create_' helper functions
* Remove unneeded 'XXX.objects.delete()' calls (all objects are deleted
on teardown of each test)
* Include every import on its own line
* Use underscore_case, rather than camelCase
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Mon, 20 Jun 2016 13:05:16 +0000 (14:05 +0100)]
tests: Clean up 'test_checks'
* Make use of 'create_' helper
* Remove unneeded 'XXX.objects.delete()' calls (all objects are deleted
on teardown of each test)
* Include every import on its own line
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Sun, 19 Jun 2016 19:42:57 +0000 (20:42 +0100)]
tests: Clean up 'test_bundles'
* Don't use hardcode routes: use the reverse function instead
* Make use of 'create_' helper functions
* Minimize duplication of code
* Use underscore_case, rather than camelCase
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Fri, 24 Jun 2016 16:28:13 +0000 (17:28 +0100)]
models: Return string from 'combined_check_status'
Previously the 'combined_check_status' function returned the checks
enum integer for the status field. However, the string representation
is more meaningful and is the only representation seen by users at
moment. Change this function so it returns, for example, 'success'
instead of '1'.
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org>
Stephen Finucane [Fri, 24 Jun 2016 16:28:12 +0000 (17:28 +0100)]
models: Use non-null slugs for 'Check.name'
The schema for 'Check' defines 'Check.name' as a 'CharField'. This is
less than ideal as names with spaces and special characters can't be
represented cleanly in URLs etc. We should use 'SlugField' instead.
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com> Reviewed-by: Andy Doan <andy.doan@linaro.org> Closes: #33
Stephen Finucane [Tue, 28 Jun 2016 09:14:44 +0000 (10:14 +0100)]
tests: Don't save multiple projects
There were issues introduced in the rebase of 'ee15585' that resulted
in two projects being saved with the same ID for one test, leading to
a test failure. Resolve this.
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Andy Doan [Thu, 16 Jun 2016 21:13:25 +0000 (16:13 -0500)]
REST: Allow projects to be retrieved by linkname
Building a user-friendly CLI becomes difficult when project-ids are
required. It also makes it almost impossible to work with the current
format of the .pwclientrc file.
Signed-off-by: Andy Doan <andy.doan@linaro.org> Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>
Andy Doan [Thu, 16 Jun 2016 21:13:22 +0000 (16:13 -0500)]
REST: Add Patch Checks to the API
This exports patch checks via the REST API.
The drf-nested-routers package is used to handle the fact Checks are
nested under a Patch.
Security Constraints:
* Anyone (logged in or not) can read all objects.
* No one can update/delete objects.
* Project maintainers and patch owners may create objects.
Signed-off-by: Andy Doan <andy.doan@linaro.org> Reviewed-by: Stephen Finucane <stephen.finucane@intel.com>
Andy Doan [Thu, 16 Jun 2016 21:13:21 +0000 (16:13 -0500)]
REST: Add Patches to the API
This exposes patches via the REST API.
Security Constraints:
* Anyone (logged in or not) can read all objects.
* No one can create/delete objects.
* Project maintainers are allowed to update (ie "patch"
attributes)
NOTE: Patch.save was overridden incorrectly and had to be
fixed to work with DRF.
Signed-off-by: Andy Doan <andy.doan@linaro.org> Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>