docs: Structure docs folder like online

The XML-RPC doc is renamed accordingly.

Signed-off-by: Stephen Finucane <stephen@that.guru>

4 files changed, 484 insertions, 0 deletions

diff --git a/docs/development/contributing.md b/docs/development/contributing.md
new file mode 100644
index 0000000..cc5f1d6
--- /dev/null
+++ b/docs/development/contributing.md
@@ -0,0 +1,59 @@
+# Tips and Tricks
+
+## Coding Standards
+
+**Follow PEP8**. All code is currently PEP8 compliant and it should stay this
+way.
+
+Changes that fix semantic issues will be generally be happily received, but
+please keep such changes separate from functional changes.
+
+`pep8` targets are provided via tox. Refer to the [testing section](#testing)
+below for more information on usage of this tool.
+
+## Testing
+
+Patchwork includes a [tox][ref-tox] script to automate testing. This requires
+a functional database and some Python requirements like `tox`. Please refer
+to the [development guide][doc-development] for information on how to configure
+these.
+
+Assuming these requirements are met, actually testing Patchwork is quite easy
+to do. To start, you can show the default targets like so:
+
+    $ tox --list
+
+You'll see that this includes a number of targets to run unit tests against
+the different versions of Django supported, along with some other targets
+related to code coverage and code quality. To run one of these, use the `-e`
+parameter:
+
+    $ tox -e py27-django18
+
+In the case of the unit tests targets, you can also run specific tests by
+passing the fully qualified test name as an additional argument to this
+command:
+
+    $ tox -e py27-django18 patchwork.tests.SubjectCleanUpTest
+
+Because Patchwork support multiple versions of Django, it's very important
+that you test against all supported versions. When run without argument, tox
+will do this:
+
+    $ tox
+
+## Submitting Changes
+
+All patches should be sent to the [mailing list][pw-ml]. When doing so, please
+abide by the [QEMU guidelines][ref-qemu-contrib] on contributing or submitting
+patches. This covers both the initial submission and any follow up to the
+patches. In particular, please ensure:
+
+* [All tests pass](#testing)
+* Documentation has been updated with new requirements, new script names etc.
+* The `CHANGES` file has been updated with any added or removed features
+
+[doc-development]: development.md
+[pw-ml]: https://ozlabs.org/mailman/listinfo/patchwork
+[ref-qemu-contrib]: http://wiki.qemu.org/Contribute/SubmitAPatch
+[ref-tox]: https://tox.readthedocs.io/en/latest/
diff --git a/docs/development/installation.md b/docs/development/installation.md
new file mode 100644
index 0000000..36d2fdf
--- /dev/null
+++ b/docs/development/installation.md
@@ -0,0 +1,342 @@
+# Development
+
+This document describes the necessary steps to configure Patchwork in a
+development environment. If you are interested in deploying Patchwork in a
+production environment, please refer to [the deployment guide][doc-deployment]
+instead.
+
+To begin, you should clone Patchwork:
+
+    $ git clone git://github.com/getpatchwork/patchwork.git
+
+## Docker-Based Installation
+
+Patchwork provides a Docker-based environment for quick configuration of a
+development environment. This is the preferred installation method. To
+configure Patchwork using Docker:
+
+1. Install [**Docker**][ref-docker] and [**docker-compose**][ref-compose].
+2. Build the images. This will download over 200MB from the internet:
+
+        $ docker-compose build
+
+3. Run `docker-compose up`:
+
+        $ docker-compose up
+
+    This will be visible at http://localhost:8000/.
+
+To run a shell within this environment, run:
+
+    $ docker-compose run --rm web --shell
+
+To run unit tests, excluding Selenium UI interaction tests, using only the
+package versions installed during container initialization, run:
+
+    $ docker-compose run --rm web --quick-test
+
+To run the same against all supported versions of Django (via tox), run:
+
+    $ docker-compose run --rm web --quick-tox
+
+To run all tests, including Selenium UI interaction tests, using only the
+package versions installed container initialization, run:
+
+    $ docker-compose run --rm web --test
+
+To run the same against all supported versions of Django (via tox), run:
+
+    $ docker-compose run --rm web --tox
+
+To run all tests, including Selenium UI interaction tests in non-headless mode,
+run:
+
+    $ docker run -it --rm -v (pwd):/home/patchwork/patchwork/ \
+        --link patchwork_db_1:db -p 8000:8000 \
+        -v /tmp/.X11-unix:/tmp/.X11-unix \
+        -e PW_TEST_DB_HOST=db -e DISPLAY patchwork_web bash
+
+To reset the database before any of these commands, add `--reset` to the
+command line after `web` and before any other arguments.
+
+Any local edits to the project files made locally are immediately visible to
+the Docker container, and so should be picked up by the Django auto-reloader.
+
+For more information on Docker itself, please refer to the [Docker][ref-docker]
+and [docker-compose][ref-compose] documentation.
+
+**NOTE:** If using SELinux, you will need to create a custom SELinux rule to
+allow the Docker process to access your working directory. Run:
+
+    $ chcon -RT svirt_sandbox_file_t $PATCHWORK_DIR
+
+where `$PATCHWORK_DIR` is the absolute patch to the `patchwork` folder created
+when you cloned the repo. For more information, see `man docker run`.
+
+**NOTE:** If you see an error like the below:
+
+    ERROR: Couldn't connect to the Docker daemon at
+    http+docker://localunixsocket - is it running?
+
+ensure you have correctly installed Docker, added your user to the `docker`
+group, and started the daemon, per the [Docker documentation][ref-docker].
+
+## Vagrant-Based Installation
+
+Patchwork provides a Vagrant-based environment as an alternative to Docker.
+Like Docker, Vagrant can be used to quickly configure Patchwork in a
+development environment. To configure Patchwork using Vagrant:
+
+1. Install [**Vagrant**][ref-vagrant]
+2. Run `vagrant up` from the project directory:
+
+        $ cd patchwork
+        $ vagrant up
+
+Once stacked, follow the on-screen instructions. For more information on
+Vagrant itself, please refer to the [Vagrant documentation][ref-vagrant].
+
+## Manual Installation
+
+Manual installation can be used where use of Docker or Vagrant is not possible or
+desired.
+
+### Install Required Packages
+
+There are a number of different requirements for developing Patchwork:
+
+* Python and libraries
+* A supported database (RDBMS)
+
+These are detailed below.
+
+#### Python Requirements
+
+To develop Python-based software you first need Python. Patchwork supports
+both Python 2.7 and Python 3.3+. One of these will be installed by default
+on many installations, though they can also be installed manually using the
+`python` or `python3` packages.
+
+It's a good idea to use [virtual environments][ref-venv] to develop Python
+software. Virtual environments are "instances" of your system Python without
+any of the additional Python packages installed. They are useful to develop and
+possibly deploy Patchwork against a "well known" set of dependencies, but they
+can also be used to test Patchwork against several versions of Django.
+
+If you do not have `virtualenv` installed then you should install it now. This
+can be installed using the `python-virtualenv` or `python3-virtualenv`
+packages. Alternatively you can install these using `pip`.
+
+It is also helpful to install [`tox`][ref-tox] which is used for running tests
+in Patchwork. This can be installed using the `python-tox` or `python3-tox`
+packages, or via `pip`.
+
+#### Database Requirements
+
+If not already installed, you may need to install an RDBMS. You can use either
+MariaDB/MySQL or PostgreSQL for this purpose. You should also install the
+development headers, known as `libmysqlclient-dev` or `libpq-dev` respectively
+on Debian-based Debian-based distros like Ubuntu and `mysql-devel` or
+`postgresql-devel` on RHEL-based distros.
+
+**NOTE:** While Django provides support for
+[multiple database backends][ref-django-db], Patchwork itself is only tested
+against MySQL/MariaDB and PostgreSQL. Should you wish to use a different
+backend, ensure you validate this first (and perhaps
+[upstream][doc-contributing] any changes you may find necessary).
+
+**NOTE:** You may be tempted to use SQLite to develop Patchwork. We'd advise
+against doing this. SQLite supports a subset of the functionality of "full"
+RDBMS like MySQL: for example, case-sensitive matching of Unicode
+[is not supported][ref-sqlite-utf8]. You will find some tests provided by
+Patchwork fail and some patches you develop may fail in production due to these
+differences.
+
+#### Example Installation
+
+An example for installing all these packages and the MySQL RDBMS on Ubuntu
+15.04 is given below:
+
+    $ sudo apt-get install python python-pip python-dev python-virtualenv \
+        python-tox mysql-server libmysqlclient-dev
+
+If you have an existing MariaDB/MySQL installation and have installed `pip`
+already/are using [Python 3.4+][ref-py34-pip] then you can install all
+packages using `pip`:
+
+    $ sudo pip install virtualenv tox
+
+If you wish to use Python 3 then simply replace 'python' with 'python3' in
+the above command.
+
+### Configure Virtual Environment
+
+**NOTE:** If you are interested in simply [testing Patchwork][doc-testing],
+many of the below steps are not required. tox will automatically install
+dependencies and use virtual environments when testing.
+
+Once these requirements are installed, you should create and activate a new
+virtual environment. This can be done like so:
+
+    $ virtualenv .venv
+    $ source .venv/bin/activate
+    (.venv)$
+
+**NOTE:** It you installed a Python 3.x-based virtual environment package,
+adjust the executable indicated above as necessary, e.g. `virtualenv-3.4`.
+
+Now install the packages. Patchwork provides three requirements files.
+
+* `requirements-dev.txt`: Packages required to configure a development
+  environment
+* `requirements-prod.txt`: Packages required for deploying Patchwork in
+  production
+* `requirements-test.txt`: Packages required to run tests
+
+We're going to install the first of these, which can be done like so:
+
+    (.venv)$ cd patchwork
+    (.venv)$ pip install -r requirements-dev.txt
+
+**NOTE:** Once configured this does not need to be done again *unless* the
+requirements change, e.g. Patchwork requires an updated version of Django.
+
+### Initialize the Database
+
+One installed, the database must be configured. We will assume you have root
+access to the database for these steps.
+
+To begin, export your database credentials as follows:
+
+    (.venv)$ db_user=root
+    (.venv)$ db_pass=password
+
+Now, create the database. If this is your first time configuring the database,
+you must create a `patchwork` user (or similar) along with the database
+instance itself. The commands below will do this, dropping existing databases
+if necessary:
+
+    (.venv)$ mysql -u$db_user -p$db_pass << EOF
+    DROP DATABASE IF EXISTS patchwork;
+    CREATE DATABASE patchwork CHARACTER SET utf8;
+    GRANT ALL PRIVILEGES ON patchwork.* TO 'patchwork'@'localhost'
+        IDENTIFIED BY 'password';
+    EOF
+
+**NOTE:** The `patchwork` username and `password` password are the defaults
+expected by the provided `dev` settings files. If using something different,
+please export the `PW_TEST_DB_USER` and `PW_TEST_DB_PASS` variables described
+in the [Environment Variables](#environment-variables) section below.
+Alternatively, you can create your own settings file with these variables
+hardcoded and change the value of `DJANGO_SETTINGS_MODULE` as described below.
+
+### Load Initial Data
+
+Before continuing, we need to tell Django where it can find our configuration.
+Patchwork provides a default development `settings.py` file for this purpose.
+To use this, export the `DJANGO_SETTINGS_MODULE` environment variable as
+described below:
+
+    (.venv)$ export DJANGO_SETTINGS_MODULE=patchwork.settings.dev
+
+Alternatively you can provide your own `settings.py` file and provide the path
+to that instead.
+
+Once done, we need to create the tables in the database. This can be done using
+the `migrate` command of the `manage.py` executable:
+
+    (.venv)$ ./manage.py migrate
+
+Next, you should load the initial fixtures into Patchwork. These initial
+fixtures provide.
+
+* `default_tags.xml`: The tags that Patchwork will extract from mails.
+  Examples: `Acked-By`, `Reviewed-By`
+* `default_states.xml`: The states that a patch can be in. Examples:
+  `Accepted`, `Rejected`
+* `default_projects.xml`: A default project that you can then upload patches
+  for
+
+These can be loaded using the `loaddata` command:
+
+    (.venv)$ ./manage.py loaddata default_tags default_states default_projects
+
+You should also take the opportunity to create a "superuser". You can do this
+using the aptly-named `createsuperuser` command:
+
+    (.venv)$ ./manage.py createsuperuser
+
+Once this is done, it's beneficial to load some real emails into the system.
+This can be done manually, however it's generally much easier to download
+an archive from a Mailman instance and load these using the `parsearchive`
+command. You can do this like so:
+
+    (.venv)$ mm_user=myusername
+    (.venv)$ mm_pass=mypassword
+    (.venv)$ mm_host=https://lists.ozlabs.org
+    (.venv)$ mm_url=$mm_host/private/patchwork.mbox/patchwork.mbox
+    (.venv)$ curl -F username=$mm_user -F password=$mm_pass -k -O $mm_url
+
+Where `mm_user` and `mm_pass` are the username and password you have registered
+with on the Mailman instance found at `mm_host`.
+
+**NOTE:** We provide instructions for downloading archives from the Patchwork
+mailing list, but almost any instance of Mailman will allow downloading of
+archives as seen above; simply change the `pw_url` variable defined. You can
+find more informations about this [here][ref-mman-bulk].
+
+Load these archives into Patchwork. Depending on the size of the downloaded
+archives this may take some time:
+
+    (.venv)$ ./manage.py parsearchive --list-id=patchwork.ozlabs.org \
+      patchwork.mbox
+
+Finally, run the server and browse to the IP address of your board using your
+browser of choice:
+
+    (.venv)$ ./manage.py runserver 0.0.0.0:8000
+
+Once finished, you can kill the server (`Ctrl` + `C`) and exit the virtual
+environment:
+
+    (.venv)$ deactivate
+    $
+
+Should you wish to re-enter this environment, simply source the `activate`
+script again.
+
+## Django Debug Toolbar
+
+Patchwork installs and enables the 'Django Debug Toolbar' by default. However,
+by default this is only displayed if you are developing on localhost. If
+developing on a different machine, you should configure an SSH tunnel such
+that, for example, `localhost:8000` points to `[DEV_MACHINE_IP]:8000`.
+
+## Environment Variables
+
+The following environment variables are available to configure settings when
+using the provided `dev` settings file.
+
+<dl>
+  <dt>PW_TEST_DB_NAME = 'patchwork'</dt>
+  <dd>Name of the database</dd>
+  <dt>PW_TEST_DB_USER = 'patchwork'</dt>
+  <dd>Username to access the database with</dd>
+  <dt>PW_TEST_DB_PASS = 'password'</dt>
+  <dd>Password to access the database with</dd>
+  <dt>PW_TEST_DB_TYPE = 'mysql'</dt>
+  <dd>Type of database to use. Options: 'mysql', 'postgres'</dd>
+</dl>
+
+[doc-contributing]: ../CONTRIBUTING.md
+[doc-deployment]: development.md
+[doc-testing]: testing.md
+[ref-django-db]: https://docs.djangoproject.com/en/1.8/ref/databases/
+[ref-mman-bulk]: http://blog.behnel.de/posts/indexp118.html
+[ref-py34-pip]: http://legacy.python.org/dev/peps/pep-0453/
+[ref-sqlite-utf8]: https://www.sqlite.org/faq.html#q18
+[ref-tox]: https://tox.readthedocs.io/en/latest/
+[ref-compose]: https://docs.docker.com/compose/install/
+[ref-docker]: https://docs.docker.com/engine/installation/linux/
+[ref-vagrant]: https://www.vagrantup.com/docs/getting-started/
+[ref-venv]: https://virtualenv.readthedocs.io/en/latest/
diff --git a/docs/development/releasing.md b/docs/development/releasing.md
new file mode 100644
index 0000000..5974da2
--- /dev/null
+++ b/docs/development/releasing.md
@@ -0,0 +1,32 @@
+# Release Process
+
+## Versioning
+
+Since version 1.0, Patchwork has implemented a version of
+[Semantic Versioning][ref-semver]. To summarise, releases take the format
+**MAJOR.MINOR.PATCH** (or just **MAJOR.MINOR**). We increment:
+
+1. **MAJOR** version when we make major UI changes or functionality updates
+2. **MINOR** version when we make minor UI changes or functionality updates
+3. **PATCH** version when we make make bug fixes, dependency updates etc.
+
+In Git, each release will have a tag indicating the version number. In
+addition, each release series has it's own branch called `stable/MAJOR.MINOR`
+to allow backporting of bugfixes or security updates to older versions.
+
+## Release Cycle
+
+There is no cadence for releases: they are made available as necessary.
+
+## Supported Versions
+
+Typically all development should occur on `master`. While we will backport
+bugfixes and security updates, we will not backport any new features. This
+is to ensure stability for users of these versions of Patchwork.
+
+## Release Checklist
+
+* Documentation has been updated with latest release version
+* Documentation references latest supported version of Django
+
+[ref-semver]: http://semver.org/
diff --git a/docs/development/xmlrpc.md b/docs/development/xmlrpc.md
new file mode 100644
index 0000000..cfa5acf
--- /dev/null
+++ b/docs/development/xmlrpc.md
@@ -0,0 +1,51 @@
+# The XML-RPC API
+
+Patchwork provides an XML-RPC API. This API can be used to be used to retrieve
+and modify information about patches, projects and more.
+
+**NOTE:** The XML-RPC API can be enabled/disabled by the administrator: it may
+not be available in every instance.
+
+## Patchwork API Documentation
+
+Patchwork provides automatically generated documentation for the XML-RPC API.
+You can find this at the following URL:
+
+    http://patchwork.example.com/xmlrpc/
+
+Where `patchwork.example.com` refers to the URL of your Patchwork instance.
+
+**NOTE:** Automatic documentation generation for the Patchwork API was
+introduced in Patchwork v1.1. Prior versions of Patchwork do not offer this
+functionality.
+
+## Developing Your Own Client
+
+You need to connect to the server. Some methods require authentication (via
+HTTP Basic Auth) while others do not. Authentication uses your Patchwork
+account and the on-server documentation will indicate where it is necessary.
+We will only cover the unauthenticated method here for brevity - please
+consult the [`xmlrpclib`] documentation for more detailed examples:
+
+    from __future__ import print_function
+    import sys
+    import xmlrpclib
+
+    url = 'http://patchwork.example.org/xmlrpc/'
+
+    try:
+        rpc = xmlrpclib.ServerProxy(url)
+    except:
+        print('Unable to connect to %s\n' % url, file=sys.stderr)
+        sys.exit(1)
+
+After connecting, the `rpc` object will be populated with a list of available
+functions (or procedures, in RPC terminology). For example, if we continue
+with the above example:
+
+    print(rpc.pw_rpc_version())
+
+It should be possible to use all the methods listed in the
+[server's documentation](#patchwork-api-documentation).
+
+[`xmlrpclib`]: https://docs.python.org/2/library/xmlrpclib.html