docs: Covert usage guide

Signed-off-by: Stephen Finucane <stephen@that.guru>

8 files changed, 612 insertions, 62 deletions

diff --git a/docs/conf.py b/docs/conf.py
index a46bea0..06b1dea 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -11,7 +11,7 @@ needs_sphinx = '1.5'
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = []
+extensions = ['sphinx.ext.todo']
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = []
diff --git a/docs/index.rst b/docs/index.rst
index 7a9ce09..b7fb8d9 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -1,10 +1,6 @@
 Patchwork
 =========
 
-.. toctree::
-   :maxdepth: 2
-   :caption: Contents:
-
 Patchwork is a patch tracking system for community-based projects. It is
 intended to make the patch management process easier for both the project's
 contributors and maintainers, leaving time for the more important (and more
@@ -21,61 +17,15 @@ subsystems of the Linux kernel. Although Patchwork has been developed with the
 kernel workflow in mind, the aim is to be flexible enough to suit the majority
 of community projects.
 
-Download
---------
-
-The latest version of Patchwork is available with git. To download:
-
-.. code-block:: shell
-
-   $ git clone git://github.com/getpatchwork/patchwork
-
-Patchwork is distributed under the `GNU General Public License (v2)`__.
-
-__ http://www.gnu.org/licenses/gpl-2.0.html
-
-Design
-------
-
-Patchwork should supplement mailing lists, not replace them
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Patchwork isn't intended to replace a community mailing list; that's why you
-can't comment on a patch in Patchwork. If this were the case, then there would
-be two forums of discussion on patches, which fragments the patch review
-process. Developers who don't use Patchwork would get left out of the
-discussion.
-
-However, a future development item for Patchwork is to facilitate on-list
-commenting, by providing a "send a reply to the list" feature for logged-in
-users.
+.. _user-docs:
 
-Don't pollute the project's changelogs with Patchwork poop
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-A project's changelogs are valuable - we don't want to add Patchwork-specific
-metadata.
-
-Patchwork users shouldn't require a specific version control system
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Not everyone uses git for kernel development, and not everyone uses git for
-Patchwork-tracked projects.
-
-It's still possible to hook other programs into Patchwork, using the pwclient
-command-line client for Patchwork, or directly to the XML RPC interface.
-
-Getting Started
----------------
-
-You should check out the `deployment/installation` and
-`development/installation` guides for information on how to configure
-Patchwork for production and development environments, respectively.
-
-Support
--------
-
-All questions and contributions should be sent to the
-`Patchwork mailing list`__
-
-__ https://ozlabs.org/mailman/listinfo/patchwork
+.. toctree::
+   :maxdepth: 2
+   :caption: User Documentation
+
+   usage/overview
+   usage/design
+   usage/delegation
+   usage/headers
+   usage/rest
+   usage/xmlrpc
diff --git a/docs/usage/delegation.rst b/docs/usage/delegation.rst
new file mode 100644
index 0000000..eaabc20
--- /dev/null
+++ b/docs/usage/delegation.rst
@@ -0,0 +1,52 @@
+Autodelegation
+==============
+
+Autodelegation allows patches to be automatically delegated to a user based on
+the files modified by the patch. To do this, a number of rules can be
+configured in the project administration page. This can usually be found at:
+
+    /admin/patchwork/project/<project_id>/change
+
+.. note::
+
+   Autodelegation can only be configured by Patchwork administrators, i.e.
+   those that can access the 'admin' panel. If you require configuration of
+   autodelegation rules on a local instance, contact your Patchwork
+   administrator.
+
+In this section there are the following fields:
+
+User
+
+  The patchwork user that should be autodelegated to the patch
+
+Priority
+
+  The priority of the rule relative to other patches. Higher values indicate
+  higher priority. If two rules have the same priority, ordering will be based
+  on the path.
+
+Path
+
+  A path in `fnmatch`__ format. The fnmatch library allows for limited, Unix
+  shell-style wildcarding. Filenames are extracted from patch lines beginning
+  with ``---`` or ``+++``. Note that for projects using Git or Mercurial, the
+  tools these VCS provide for producing patches are prefixed with `a` or `b`.
+  You should account for this in your path. For example, to match the path
+  `patchwork/views` (relative to the top of a Git repo) your pattern should
+  be::
+
+      ?/patchwork/views/*
+
+  It is also possible to use relative paths, such as::
+
+      */manage.py
+
+  For projects using other VCSs like Subversion can simply use a bare path::
+
+      patchwork/views/*
+
+Rules are configured by setting the above fields and saving the rules. These
+rules will be applied at patch parse time.
+
+__ https://docs.python.org/2/library/fnmatch.html
diff --git a/docs/usage/design.rst b/docs/usage/design.rst
new file mode 100644
index 0000000..2d8fea7
--- /dev/null
+++ b/docs/usage/design.rst
@@ -0,0 +1,30 @@
+Design
+======
+
+Patchwork should supplement mailing lists, not replace them
+-----------------------------------------------------------
+
+Patchwork isn't intended to replace a community mailing list; that's why you
+can't comment on a patch in Patchwork. If this were the case, then there would
+be two forums of discussion on patches, which fragments the patch review
+process. Developers who don't use Patchwork would get left out of the
+discussion.
+
+However, a future development item for Patchwork is to facilitate on-list
+commenting, by providing a "send a reply to the list" feature for logged-in
+users.
+
+Don't pollute the project's changelogs with Patchwork poop
+----------------------------------------------------------
+
+A project's changelogs are valuable - we don't want to add Patchwork-specific
+metadata.
+
+Patchwork users shouldn't require a specific version control system
+-------------------------------------------------------------------
+
+Not everyone uses git for kernel development, and not everyone uses git for
+Patchwork-tracked projects.
+
+It's still possible to hook other programs into Patchwork, using the pwclient
+command-line client for Patchwork, or directly to the XML RPC interface.
diff --git a/docs/usage/headers.rst b/docs/usage/headers.rst
new file mode 100644
index 0000000..8741d9c
--- /dev/null
+++ b/docs/usage/headers.rst
@@ -0,0 +1,41 @@
+Hint Headers
+============
+
+Patchwork provides a number of special email headers to control how a patch is
+handled when it is received. The examples provided below use `git-send-email`,
+but custom headers can also be set when using tools like `mutt`.
+
+`X-Patchwork-Ignore`
+
+  Valid values: *
+
+  When set, the mere presence of this header will ensure the provided email is
+  not parsed by Patchwork. For example:
+
+  .. code-block:: shell
+
+     $ git send-email --add-header="X-Patchwork-Ignore: test" master
+
+`X-Patchwork-Delegate`
+
+  Valid values: An email address associated with a Patchwork user
+
+  If set and valid, the user corresponding to the provided email address will
+  be assigned as the delegate of any patch parsed. If invalid, it will be
+  ignored.  For example:
+
+  .. code-block:: shell
+
+     $ git send-email --add-header="X-Patchwork-Delegate: a@example.com" master
+
+`X-Patchwork-State`
+
+  Valid values: Varies between deployments. This can usually be one of
+  "Accepted", "Rejected", "RFC" or "Awaiting Upstream", among others.
+
+  If set and valid, the state provided will be assigned as the state of any
+  patch parsed. If invalid, it will be ignored. For example:
+
+  .. code-block:: shell
+
+     $ git send-email --add-header="X-Patchwork-State: RFC" master
diff --git a/docs/usage/overview.rst b/docs/usage/overview.rst
new file mode 100644
index 0000000..40f49e1
--- /dev/null
+++ b/docs/usage/overview.rst
@@ -0,0 +1,368 @@
+Overview
+========
+
+The key concepts or models of Patchwork are outlined below.
+
+Projects
+--------
+
+Projects typically represent a software project or sub-project. A Patchwork
+server can host multiple projects. Each project can have multiple maintainers.
+Projects usually have a 1:1 mapping with a mailing list, though it's also
+possible to have multiple projects in the same list using the subject as
+filter. Patches, cover letters, and series are all associated with a single
+project.
+
+People
+------
+
+People are anyone who has submitted a patch, cover letter, or comment to a
+Patchwork instance.
+
+Users
+-----
+
+Users are anyone who has created an account on a given Patchwork instance.
+
+Standard Users
+~~~~~~~~~~~~~~
+
+A standard user can associate multiple email addresses with their user account,
+create bundles and store TODO lists.
+
+Maintainers
+~~~~~~~~~~~
+
+Maintainers are a special type of user that with permissions to do certain
+operations that regular Patchwork users can't. Patchwork maintainers usually
+have a 1:1 mapping with a project's code maintainers though this is not
+necessary.
+
+The operations that a maintainer can invoke include:
+
+- Change the state of a patch
+- Archive a patch
+- Delegate a patch, or be delegated a patch
+
+Submissions
+-----------
+
+Patchwork captures three types of mail to mailing lists: patches, cover
+letters, and replies to either patches or cover letters, a.k.a. comments. Any
+mail that does not fit one of these categories is ignored.
+
+Patches
+~~~~~~~
+
+Patches are the central object in Patchwork structure. A patch contains both a
+diff and some metadata, such as the name, the description, the author, the
+version of the patch etc. Patchwork stores not only the patch itself but also
+various metadata associated with the email that the patch was parsed from, such
+as the message headers or the date the message itself was received.
+
+Cover Letters
+~~~~~~~~~~~~~
+
+Cover letters provide a way to offer a "big picture" overview of a series of
+patches. When using Git, these mails can be recognised by way of their `0/N`
+subject prefix, e.g. `[00/11] A sample series`. Like patches, Patchwork stores
+not only the various aspects of the cover letter itself, such as the name and
+body of the cover letter, but also various metadata associated with the email
+that the cover letter was parsed from.
+
+Comments
+~~~~~~~~
+
+Comments are replies to a submission - either a patch or a cover letter. Unlike
+a Mail User Agent (MUA) like Gmail, Patchwork does not thread comments.
+Instead, every comment is associated with either a patch or a cover letter, and
+organized by date.
+
+Patch Metadata
+--------------
+
+Patchwork allows users to store various metadata against patches. This metadata
+is only configurable by a maintainer.
+
+States
+~~~~~~
+
+States track the state of patch in its lifecycle. States vary from project to
+project, but generally a minimum subset of "new", "rejected" and "accepted"
+will exist.
+
+Delegates
+~~~~~~~~~
+
+Delegates are Patchwork users who are responsible for both reviewing a patch
+and setting its eventual state in Patchwork. This makes them akin to reviewers
+in other tools. Delegation works particularly well for larger projects where
+various subsystems, each with their own maintainer(s), can be identified. Only
+one delegate can be assigned to a patch.
+
+.. note::
+
+   Patchwork supports automatic delegation of patches. Refer to
+   :doc:`delegation` for more information.
+
+Tags
+~~~~
+
+Tags are specially formatted metadata appended to the foot the body of a patch
+or a comment on a patch. Patchwork extracts these tags at parse time and
+associates them with the patch. You add extra tags to an email by replying to
+the email. The following tags are available on a standard Patchwork install:
+
+Acked-by:
+
+  For example::
+
+      Acked-by: Stephen Finucane <stephen@that.guru>
+
+Tested-by:
+
+  For example::
+
+      Tested-by: Stephen Finucane <stephen@that.guru>
+
+Reviewed-by:
+
+  For example::
+
+      Tested-by: Stephen Finucane <stephen@that.guru>
+
+The available tags, along with the significance of said tags, varies from
+project to project and Patchwork instance to Patchwork instance. The `kernel
+project documentation`__ provides an overview of the supported tags for the
+Linux kernel project.
+
+__ https://www.kernel.org/doc/Documentation/SubmittingPatches
+
+Checks
+~~~~~~
+
+Checks store the results of any tests executed (or executing) for a given
+patch. This is useful, for example, when using a continuous integration (CI)
+system to test patches. Checks have a number of fields associated with them:
+
+Context
+
+  A label to discern check from the checks of other testing systems
+
+Description
+
+  A brief, optional description of the check
+
+Target URL
+
+  A target URL where a user can find information related to this check, such as
+  test logs.
+
+State
+
+  The state of the check. One of: pending, success, warning, fail
+
+User
+
+  The user creating the check
+
+.. note::
+
+   Checks can only be created through the Patchwork APIs. Refer to `../api`
+   for more information.
+
+.. todo::
+
+   Provide information on building a CI system that reports check results back
+   to Patchwork.
+
+Collections
+-----------
+
+Patchwork provides a number of ways to store groups of patches. Some of these
+are automatically generated, while others are user-defined.
+
+Series
+~~~~~~
+
+Series are groups of patches, along with an optional cover letter. Series are
+mostly dumb containers, though they also contain some metadata themselves such
+as a version (which is inherited by the patches and cover letter) and a count
+of the number of patches found in the series.
+
+Bundles
+~~~~~~~
+
+Bundles are custom, user-defined groups of patches. Bundles can be used to keep
+patch lists, preserving order, for future inclusion in a tree. There's no
+restriction of number of patches and they don't even need to be in the same
+project. A single patch also can be part of multiple bundles at the same time.
+An example of Bundle usage would be keeping track of the Patches that are ready
+for merge to the tree.
+
+To-do Lists
+~~~~~~~~~~~
+
+Patchwork users can store a to-do list of patches.
+
+Events
+------
+
+Events are raised whenever patches are created or modified.
+
+All events have a number of common properties, along with some event-specific
+properties:
+
+`category`
+
+  The type of event
+
+`project`
+
+  The project this event belongs to
+
+`date`
+
+  When this event was created
+
+.. note::
+
+   Checks can only be created and read through the Patchwork APIs. Refer to
+   `../api/index` for more information.
+
+Cover Letter Created
+~~~~~~~~~~~~~~~~~~~~
+
+Sent when a cover letter is created.
+
+`category`
+
+  ``cover-created``
+
+`cover`
+
+  Created cover letter
+
+Patch Created
+~~~~~~~~~~~~~
+
+Sent when a patch is created.
+
+`category`
+
+  ``patch-created``
+
+`patch`
+
+  Created patch
+
+Patch Completed
+~~~~~~~~~~~~~~~
+
+Sent when a patch in a series has its dependencies met, or when a patch that is
+not in a series is created (since that patch has no dependencies).
+
+`category`
+
+  ``patch-completed``
+
+`patch`
+
+  Completed patch
+
+`series`
+
+  Series from which patch dependencies were extracted, if any
+
+Patch Delegated
+~~~~~~~~~~~~~~~
+
+Sent when a patch's delegate is changed.
+
+`category`
+
+  ``patch-delegated``
+
+`patch`
+
+  Updated patch
+
+`previous`
+
+  Previous delegate, if any
+
+`current`
+
+  Current delegate, if any
+
+Patch State Changed
+~~~~~~~~~~~~~~~~~~~
+
+Sent when a patch's state is changed.
+
+`category`
+
+  ``patch-state-changed``
+
+`patch`
+
+  Updated patch
+
+`previous`
+
+  Previous state
+
+`current`
+
+  Current state
+
+Check Created
+~~~~~~~~~~~~~
+
+Sent when a patch check is created.
+
+`category`
+
+  ``check-created``
+
+`check`
+
+  Created check
+
+Series Created
+~~~~~~~~~~~~~~
+
+Sent when a series is created.
+
+`category`
+
+  ``series-created``
+
+`series`
+
+  Created series
+
+Series Completed
+~~~~~~~~~~~~~~~~
+
+Sent when a series is completed.
+
+`category`
+
+  ``series-completed``
+
+`series`
+
+  Completed series
+
+What's Not Exposed
+~~~~~~~~~~~~~~~~~~
+
+* Bundles
+
+  We don't expose an "added to bundle" event as it's unlikely that this will
+  be useful to either users or CI setters.
+
+* Comments
+
+  Like Bundles, there likely isn't much value in exposing these via the API.
diff --git a/docs/usage/rest.rst b/docs/usage/rest.rst
new file mode 100644
index 0000000..8d6e036
--- /dev/null
+++ b/docs/usage/rest.rst
@@ -0,0 +1,63 @@
+The REST API
+============
+
+Patchwork provides a REST API. This API can be used to retrieve and modify
+information about patches, projects and more.
+
+.. note::
+
+   The REST API was introduced in Patchwork v2.0. Users of earlier Patchwork
+   versions should instead refer to :doc:`xmlrpc`.
+
+Documentation
+-------------
+
+Patchwork provides automatically generated documentation for the RESET API.
+You can find this at the following URL:
+
+    http://patchwork.example.com/api/
+
+where `patchwork.example.com` refers to the URL of your Patchwork instance.
+
+Interacting with the API
+------------------------
+
+REST APIs run over plain HTTP(S), thus, the API can be interfaced using
+applications or libraries that support this widespread protocol. One such
+application is `curl`__, which can be used to both retrieve and send
+information to the REST API. For example, to get the version of the REST API
+for a Patchwork instance hosted at `patchwork.example.com`, run:
+
+.. code-block:: shell
+
+    $ curl -s http://localhost:8000/api/1.0/ | python -m json.tool
+    {
+        "patches": "http://localhost:8000/api/1.0/patches/",
+        "people": "http://localhost:8000/api/1.0/people/",
+        "projects": "http://localhost:8000/api/1.0/projects/",
+        "users": "http://localhost:8000/api/1.0/users/"
+    }
+
+In addition, a huge variety of libraries are avaiable for interacting with and
+parsing the output of REST APIs. The `requests`__ library is wide-spread and
+well-supported. To repeat the above example using `requests`:
+
+.. code-block:: pycon
+
+    $ python
+    >>> import json
+    >>> import requests
+    >>> r = requests.get('http://patchwork.example.com/api/1.0/')
+    >>> print(json.dumps(r.json(), indent=2))
+    {
+          "users": "http://localhost:8000/api/1.0/users/",
+          "patches": "http://localhost:8000/api/1.0/patches/",
+          "projects": "http://localhost:8000/api/1.0/projects/",
+          "people": "http://localhost:8000/api/1.0/people/"
+    }
+
+Tools like `curl` and libraries like `requests` can be used to build anything
+from small utilities to full-fledged clients targeting the REST API.
+
+__ https://curl.haxx.se/
+__ http://docs.python-requests.org/en/master/
diff --git a/docs/usage/xmlrpc.rst b/docs/usage/xmlrpc.rst
new file mode 100644
index 0000000..644e8e2
--- /dev/null
+++ b/docs/usage/xmlrpc.rst
@@ -0,0 +1,46 @@
+The XML-RPC API
+===============
+
+.. note::
+
+   This guide covers usage information for the Patchwork XML-RPC API.  For
+   information on using the REST API, refer to :doc:`rest`. For information on
+   developing custom applications or clients for this API, refer to the
+   `../development/xmlrpc`.
+
+Patchwork provides an XML-RPC API. This API can be used to be used to retrieve
+and modify information about patches, projects and more.
+
+.. note::
+
+   The XML-RPC API can be enabled/disabled by the administrator: it may not be
+   available in every instance.
+
+pwclient
+--------
+
+The `pwclient` application, provided with Patchwork, can be used to interact
+with Patchwork from the command line. Functionality provided by `pwclient`
+includes:
+
+* Listing patches, projects, and checks
+* Downloading and applying patches to a local code base
+* Modifying the status of patches
+* Creating new checks
+
+pwclient can be downloaded from the `Ozlabs Patchwork instance`__, or at the
+following path for other Patchwork instances:
+
+    http://patchwork.example.com/pwclient/
+
+where `patchwork.example.com` corresponds to the URL a Patchwork instance is
+hosted at.
+
+Once downloaded, to view information about all the operations supported by
+`pwclient`, run:
+
+.. code-block:: shell
+
+    $ pwclient --help
+
+__ https://patchwork.ozlabs.org/pwclient/