diff options
author | Stephen Finucane <stephen@that.guru> | 2017-03-22 23:17:25 +0000 |
---|---|---|
committer | Stephen Finucane <stephen@that.guru> | 2017-04-18 22:08:32 +0100 |
commit | 1cb38956875fecd42e1eaaf4c0c4064c577c7aab (patch) | |
tree | 564b0bb6903ba41dce8db8b74b19b12b7ff15025 | |
parent | 5d608aeaa86f2819770098214e92f1dbc1c72d00 (diff) | |
download | patchwork-1cb38956875fecd42e1eaaf4c0c4064c577c7aab.tar patchwork-1cb38956875fecd42e1eaaf4c0c4064c577c7aab.tar.gz |
docs: Remove old Markdown files
Everything is now ported to rST/Sphinx meaning we can remove the
Markdown versions.
Signed-off-by: Stephen Finucane <stephen@that.guru>
-rw-r--r-- | docs/deployment/installation.md | 489 | ||||
-rw-r--r-- | docs/deployment/upgrading.md | 138 | ||||
-rw-r--r-- | docs/development/contributing.md | 59 | ||||
-rw-r--r-- | docs/development/installation.md | 379 | ||||
-rw-r--r-- | docs/development/releasing.md | 49 | ||||
-rw-r--r-- | docs/development/xmlrpc.md | 54 | ||||
-rw-r--r-- | docs/index.md | 68 | ||||
-rw-r--r-- | docs/usage/delegation.md | 45 | ||||
-rw-r--r-- | docs/usage/headers.md | 34 | ||||
-rw-r--r-- | docs/usage/overview.md | 313 | ||||
-rw-r--r-- | docs/usage/rest.md | 56 | ||||
-rw-r--r-- | docs/usage/xmlrpc.md | 39 | ||||
-rw-r--r-- | mkdocs.yml | 22 |
13 files changed, 0 insertions, 1745 deletions
diff --git a/docs/deployment/installation.md b/docs/deployment/installation.md deleted file mode 100644 index 48bf979..0000000 --- a/docs/deployment/installation.md +++ /dev/null @@ -1,489 +0,0 @@ -# Deployment - -This document describes the necessary steps to configure Patchwork in a -production environment. This requires a significantly "harder" deployment than -the one used for development. If you are interested in developing Patchwork, -please refer to [the development guide][doc-development] instead. - -This document describes a two-node installation of Patchwork, consisting of -a database sever and an application server. It should be possible to combine -these machines for smaller Patchwork instances. It should also be possible to -configure high availability deployment through use of additional database and -application machines, though this is out of the scope of this document. - -## Deployment Guides, Provisioning Tools and Platform-as-a-Service - -Before continuing, it's worth noting that Patchwork is a Django application. -With the exception of the handling of incoming mail (described below), it -can be deployed like any other Django application. This means there are tens, -if not hundreds, of existing articles and blogs detailing how to deploy an -application like this. As such, if any of the below information is unclear -then we'd suggest you go search for "Django deployment guide" or similar, -deploy your application, and submit [a patch for this guide][doc-contributing] -to clear up that confusion for others. - -You'll also find that the same search reveals a significant number of existing -deployment tools aimed at Django. These tools, be they written in Ansible, -Puppet, Chef or something else entirely, can be used to avoid much of the -manual configuration described below. If possible, embrace these tools to make -your life easier. - -Finally, many Platform-as-a-Service (PaaS) providers and tools support -deployment of Django applications with minimal effort. Should you wish to -avoid much of the manual configuration, we suggest you investigate the many -options available to find one that best suits your requirements. The only issue -here will likely be the handling of incoming mail - something which many of -these providers don't support. We address this in the appropriate section -below. - -## Requirements - -For the purpose of this guide, we will assume the following machines: - -| server role | IP address | -|-------------|------------| -| database | 10.1.1.1 | -| application | 10.1.1.2 | - -We will use the database server to, ostensibly enough, host the database for -the Patchwork instance. The application server, on the other hand, will host -the Patchwork instance along with the required reverse proxy and WSGI HTTP -servers. - -We expect a Ubuntu 15.04 installation on each of these hosts: commands, -package names and/or package versions will likely change if using a -different distro or release. In addition, usage of different package versions -to the ones suggested may require slightly different configuration. - -Before beginning, you should update these systems: - - $ sudo apt-get update - $ sudo apt-get upgrade - -We also need to configure some environment variables to ease deployment. These -should be exported on all systems: - -<dl> - <dt>PW_HOST_DB=10.1.1.1</dt> - <dd>IP of the database host</dd> - <dt>PW_HOST_APP=10.1.1.2</dt> - <dd>IP of the application host</dd> - <dt>PW_DB_NAME=patchwork</dt> - <dd>Name of the database</dd> - <dt>PW_DB_USER=www-data</dt> - <dd>Username that the Patchwork app will access the database with</dd> -</dl> - -## Database - -These steps should be run on the database server. - -**NOTE:** If you already have a database server on site, you can skip much of -this section. - -### Install Requirements - -We're going to rely on PostgreSQL. You can adjust the below steps if using a -different RDBMS. Install the required packages. - - $ sudo apt-get install -y postgresql postgresql-contrib - -### Configure Database - -PostgreSQL created a user account called `postgres`; you will need to run -commands as this user. Use this account to create the database that Patchwork -will use, using the credentials we configured earlier. - - $ sudo -u postgres createdb $PW_DB_NAME - $ sudo -u postgres createuser $PW_DB_USER - -We will also need to apply permissions to the tables in this database but -seeing as the tables haven't actually been created yet this will have to be -done later. - -**TODO** `pg_hba.conf` configuration - -## Patchwork - -These steps should be run on the application server. - -### Install Packages - -The first requirement is Patchwork itself. It can be downloaded like so: - - $ wget https://github.com/getpatchwork/patchwork/archive/v1.1.0.tar.gz - -We will install this under `/opt`, though this is only a suggestion: - - $ tar -xvzf v1.1.0.tar.gz - $ sudo mv v1.1.0 /opt/patchwork - -**NOTE:** Per the [Django documentation][ref-django-files], source code should -not be placed in your web server's document root as this risks the possibility -that people may be able to view your code over the Web. This is a security -risk. - -Next we require Python. If not already installed, then you should do so now. -Patchwork supports both Python 2.7 and Python 3.3+, though we would suggest -using the latter to ease future upgrades: - - $ sudo apt-get install python3 # or 'python' if using Python 2.7 - -We require a number of Python packages. These can be installed using `pip`: - - $ sudo pip install -r /opt/patchwork/requirements-prod.txt - -If you're not using `pip`, you will need to identify and install the -corresponding distro package for each of these requirements. For example: - - $ sudo apt-get install python3-django - -**NOTE:** The [pkgs.org][ref-pkgs] website provides a great reference for -identifying the name of these dependencies. - -### Configure Patchwork - -You will also need to configure a [settings][ref-django-settings] file for -Django. A sample settings file is provided that defines default settings for -Patchwork. You'll need to configure settings for your own setup and save this -as `production.py`. - - $ cp patchwork/settings/production.example.py \ - patchwork/settings/production.py - -Alternatively, you can override the `DJANGO_SETTINGS_MODULE` environment -variable and provide a completely custom settings file. - -**NOTE:** You should not include shell variables in settings but rather -hardcoded values. These settings files are evaluated in Python - not a shell. - -### Databases - -You can configure the `DATABASES` setting using the variables we set earlier. - - DATABASES = { - 'default': { - 'ENGINE': 'django.db.backends.postgresql_psycopg2', - 'HOST': '$PW_HOST_DB', # don't use sh variables but actual values - 'PORT': '', - 'NAME': '$PW_DB_NAME', - 'USER': '$PW_DB_USER', - 'PASSWORD': '$PW_DB_PASS', - 'TEST': { - 'CHARSET': 'utf8', - }, - }, - } - -**NOTE:** `TEST/CHARSET` is used when creating tables for the test suite. -Without it, tests checking for the correct handling of non-ASCII characters -fail. It is not necessary if you don't plan to run tests, however. - -#### Static Files - -While we have not yet configured our proxy server, we do need to configure -the location that these files will be stored in. We will install these under -`/var/www/patchwork`, though this is only a suggestion and can be changed. - - $ mkdir /var/www/patchwork - -You can configure this by setting the `STATIC_ROOT` variable. - - STATIC_ROOT = '/var/www/patchwork' - -#### Other Options - -Finally, the following settings need to be configured. The purpose of these -variables is described in the [Django documentation][ref-django-settings]: - -* `SECRET_KEY` -* `ADMINS` -* `TIME_ZONE` -* `LANGUAGE_CODE` -* `DEFAULT_FROM_EMAIL` -* `NOTIFICATION_FROM_EMAIL` - -You can generate the `SECRET_KEY` with the following python code: - - import string, random - chars = string.letters + string.digits + string.punctuation - print repr("".join([random.choice(chars) for i in range(0,50)])) - -If you wish to enable the XML-RPC interface, you should add the following to -the file: - - ENABLE_XMLRPC = True - -### Final Steps - -Once done, we should be able to check that all requirements are met using -the `check` command of the `manage.py` executable: - - $ /opt/patchwork/manage.py check - -We should also take this opportunity to both configure the database and static -files: - - $ /opt/patchwork/manage.py migrate - $ /opt/patchwork/manage.py loaddata \ - /opt/patchwork/patchwork/fixtures/default_tags.xml - $ /opt/patchwork/manage.py loaddata \ - /opt/patchwork/patchwork/fixtures/default_states.xml - $ /opt/patchwork/manage.py collectstatic - -**NOTE:** The above `default_tags` and `default_states` are just that: -defaults. You can modify these to fit your own requirements. - -Finally, it may be helpful to start the development server quickly to ensure -you can see *something*: - - $ /opt/patchwork/manage.py runserver 0.0.0.0:8080 - -Browse this instance at `http://[your_server_ip]:8000`. If everything is -working, kill the development server using `Ctrl`+`C`. - -## Reverse Proxy and WSGI HTTP Servers - -These steps should be run on the application server. - -### Install Packages - -We will use nginx and uWSGI to deploy Patchwork, acting as reverse proxy server -and WSGI HTTP server respectively. Other options are available, such as -Apache+mod_wsgi or nginx+Gunicorn. While we don't document these, sample -configuration files for the former case are provided in `lib/apache2/`. - - $ sudo apt-get install nginx-full uwsgi uwsgi-plugin-python - -### Configure nginx and uWSGI - -Configuration files for nginx and uWSGI are provided in the `lib` subdirectory -of the Patchwork source code. These can be modified as necessary, but for now -we will simply copy them. - -First, let's load the provided configuration for nginx: - - $ sudo cp /opt/patchwork/lib/nginx/patchwork.conf \ - /etc/nginx/sites-available/ - -If you wish to modify this configuration, now is the time to do so. Once done, -validate and enable your configuration: - - $ sudo nginx -t - $ sudo ln -s /etc/nginx/sites-available/patchwork.conf \ - /etc/nginx/sites-enabled/patchwork.conf - -Now use the provided configuration for uWSGI: - - $ sudo mkdir -p /etc/uwsgi/sites - $ sudo cp /opt/patchwork/lib/uwsgi/patchwork.ini \ - /etc/uwsgi/sites/patchwork.ini - -**NOTE** We created the `/etc/uwsgi` directory above because we're going to run -uWSGI in ["emperor mode][ref-uwsgi-emperor]". This has benefits for multi-app -deployments. - -### Create systemd Unit File - -As things stand, uWSGI will need to be started manually every time the system -boots, in addition to any time it may fail. We can automate this process using -systemd. To this end a [systemd unit file][ref-uwsgi-systemd] should be created -to start uWSGI at boot: - - $ sudo cat << EOF > /etc/systemd/system/uwsgi.service - [Unit] - Description=uWSGI Emperor service - - [Service] - ExecStartPre=/usr/bin/bash -c 'mkdir -p /run/uwsgi; chown user:nginx /run/uwsgi' - ExecStart=/usr/bin/uwsgi --emperor /etc/uwsgi/sites - Restart=always - KillSignal=SIGQUIT - Type=notify - NotifyAccess=all - - [Install] - WantedBy=multi-user.target - EOF - -**NOTE:** On older version of Ubuntu you may need to tweak these steps to use -[upstart][ref-uwsgi-upstart] instead. - -### Final Steps - -Start the uWSGI service we created above: - - $ sudo systemctl start uwsgi - $ sudo systemctl status uwsgi - -Next up, restart the nginx service: - - $ sudo systemctl restart nginx - $ sudo systemctl status nginx - -Patchwork uses a cron script to clean up expired registrations and send -notifications of patch changes (for projects with this enabled). Something like -this in your crontab should work. - - # m h dom mon dow command - */10 * * * * cd patchwork; ./manage.py cron - -**NOTE**: The frequency should be the same as the `NOTIFICATION_DELAY_MINUTES` -setting, which defaults to 10 minutes. - -Finally, browse to the instance using your browser of choice. - -You may wish to take this opportunity to setup your projects and configure your -website address (in the Sites section of the admin console, found at `/admin`). - -## Django administrative console - -In order to access the administrative console at `/admin`, you need at least -one user account to be registered and configured as a super user or staff -account to access the Django administrative console. This can be achieved by -doing the following: - - $ /opt/patchwork/manage.py createsuperuser - -Once the administrative console is accessible, you would want to configure your -different sites and their corresponding domain names, which is required for the -different emails sent by patchwork (registration, password recovery) as well as -the sample `pwclientrc` files provided by your project's page. - -## Incoming Email - -Patchwork is designed to parse incoming mails which means you need an address -to receive email at. This is a problem that has been solved for many webapps, -thus there are many ways to go about this. Some of these ways are discussed -below. - -### IMAP/POP3 - -The easiest option for getting mail into Patchwork is to use an existing email -address in combination with a mail retriever like [getmail][ref-getmail], which -will download mails from your inbox and pass them to Patchwork for processing. -getmail is easy to set up and configure: to begin, you need to install it: - - $ sudo apt-get install getmail4 - -Once installed, you should configure it, sustituting your own configuration -details where required below: - - $ sudo cat << EOF > /etc/getmail/user@example.com/getmailrc - [retriever] - type = SimpleIMAPSSLRetriever - server = imap.example.com - port = 993 - username = XXX - password = XXX - mailboxes = ALL - - [destination] - # we configure Patchwork as a "mail delivery agent", in that it will - # handle our mails - type = MDA_external - path = /opt/patchwork/patchwork/bin/parsemail.sh - - [options] - # retrieve only new emails - read_all = false - # do not add a Delivered-To: header field - delivered_to = false - # do not add a Received: header field - received = false - EOF - -Validate that this works as expected by starting `getmail`: - - $ getmail --getmaildir=/etc/getmail/user@example.com --idle INBOX - -If everything works as expected, you can create a systemd script to ensure this -starts on boot: - - $ sudo cat << EOF > /etc/systemd/system/getmail.service - [Unit] - Description=Getmail for user@example.com - - [Service] - User=pathwork - ExecStart=/usr/bin/getmail --getmaildir=/etc/getmail/user@example.com --idle INBOX - Restart=always - - [Install] - WantedBy=multi-user.target - EOF - -And start the service: - - $ sudo systemctl start getmail - $ sudo systemctl status getmail - -### Mail Transfer Agent (MTA) - -The most flexible option is to configure our own mail transfer agent (MTA) or -"email server". There are many options, of which [Postfix][ref-postfix] is one. -While we don't cover setting up Postfix here (it's complicated and there are -many guides already available), Patchwork does include a script to take -received mails and create the relevant entries in Patchwork for you. To use -this, you should configure your system to forward all emails to a given -localpart (the bit before the `@`) to this script. Using the `patchwork` -localpart (e.g. `patchwork@example.com`) you can do this like so: - - $ sudo cat << EOF > /etc/aliases - patchwork: "|/opt/patchwork/patchwork/bin/parsemail.sh" - EOF - -You should ensure the appropriate user is created in PostgreSQL and that -it has (minimal) access to the database. Patchwork provides scripts for the -latter and they can be loaded as seen below: - - $ sudo -u postgres createuser nobody - $ sudo -u postgre psql -f \ - /opt/patchwork/lib/sql/grant-all.postgres.sql patchwork - -**NOTE:** This assumes your Postfix process is running as the `nobody` user. -If this is not correct (use of `postfix` user is also common), you should -change both the username in the `createuser` command above and substitute the -username in the `grant-all-postgres.sql` script with the appropriate -alternative. - -### Use a Email-as-a-Service Provider - -Setting up an email server can be a difficult task and, in the case of -deployment on PaaS provider, may not even be an option. In this case, there -are a variety of web services available that offer "Email-as-as-Service". -These services typically convert received emails into HTTP POST requests to -your endpoint of choice, allowing you to sidestep configuration issues. We -don't cover this here, but a simple wrapper script coupled with one of these -services can be more than to get email into Patchwork. - -You can also create such as service yourself using a PaaS provider that -supports incoming mail and writing a little web app. - -## (Optional) Configure your VCS to Automatically Update Patches - -The `tools` directory of the Patchwork distribution contains a file named -`post-receive.hook` which is a sample git hook that can be used to -automatically update patches to the `Accepted` state when corresponding -commits are pushed via git. - -To install this hook, simply copy it to the `.git/hooks` directory on your -server, name it `post-receive`, and make it executable. - -This sample hook has support to update patches to different states depending -on which branch is being pushed to. See the `STATE_MAP` setting in that file. - -If you are using a system other than git, you can likely write a similar hook -using `pwclient` to update patch state. If you do write one, please contribute -it. - -[doc-contributing]: ../development/contributing.md -[doc-development]: ../development/installation.md -[ref-django-files]: https://docs.djangoproject.com/en/dev/intro/tutorial01/#creating-a-project -[ref-django-settings]: https://docs.djangoproject.com/en/1.8/ref/settings/ -[ref-getmail]: http://pyropus.ca/software/getmail/ -[ref-pkgs]: http://pkgs.org/ -[ref-postfix]: http://www.postfix.org/ -[ref-uwsgi-emperor]: https://uwsgi-docs.readthedocs.io/en/latest/Emperor.html -[ref-uwsgi-systemd]: https://uwsgi-docs.readthedocs.io/en/latest/Systemd.html -[ref-uwsgi-upstart]: https://uwsgi-docs.readthedocs.io/en/latest/Upstart.html diff --git a/docs/deployment/upgrading.md b/docs/deployment/upgrading.md deleted file mode 100644 index 014ec51..0000000 --- a/docs/deployment/upgrading.md +++ /dev/null @@ -1,138 +0,0 @@ -# Upgrading - -**NOTE:** This document provides some general tips and tricks that one can use -when upgrading an existing, production installation of Patchwork. If you are -interested in the specific changes between each release, please refer to the -[`UPGRADING` document][gh-upgrading] instead. If this is your first time -installing Patchwork, please refer to the -[installation guide][doc-installation] instead. - -## Before You Start - -Before doing anything, always **backup your data**. This generally means -backing up your database, but it might also be a good idea to backup your -environment in case you encounter issues during the upgrade process. - -While Patchwork won't explicitly prevent it, it's generally wise to avoid -upgrades spanning multiple releases in one go. An iterative upgrade approach -will provide an easier, if slower, upgrade process. - -## Identify Changed Scripts, Requirements, etc. - -The `UPGRADING` document provides a comprehensive listing of all -backwards-incompatible changes that occur between releases of Patchwork. -Examples of such changes include: - -* Moved/removed scripts and files -* Changes to the requirements, e.g. supported Django versions -* Changes to API that may affect, for example, third-party tools - -It is important that you understand these changes and ensure any scripts you -may have, such as systemd/upstart scripts, are modified accordingly. - -## Understand What Requirements Have Changed - -New versions of Patchwork can often require additional or updated version of -dependencies, e.g. newer versions of Django. It is important that you -understand these requirements and can fulfil them. This is particularly true -for users relying on distro-provided packages, who may have to deal with older -versions of a package or may be missing a package altogether (though we try to -avoid this). Such changes are usually listed in the `UPGRADING` document, but -you can also diff the `requirements.txt` files in each release for comparison. - -## Collect Static Files - -New versions of Patchwork generally contain changes to the additional files -like images, CSS and JavaScript. To do this, run the `collectstatic` -management commands: - - $ ./manage.py collectstatic - -## Upgrade Your Database - -Migrations of the database can be tricky. Prior to [`v1.0.0`][gh-v1], database -migrations were provided by way of manual, SQL migration scripts. After this -release, Patchwork moved to support [Django migrations][ref-django-migrate]. -If you are upgrading from `v1.0.0` or later, it is likely that you can rely -entirely on the later to bring your database up-to-date. This can be done like -so: - - $ ./manage.py migrate - -However, there are a number of scenarios in which you may need to fall back to -the provided SQL migrations or provide your own: - -* You are using Django < 1.6 - - Patchwork supports Django 1.6. However, Django Migrations was added in 1.7 - and is [not available for previous versions][ref-south2]. As such, you must - continue to use manual migrations or upgrade your version of Django. For - many of the migrations, this can be done automatically: - - $ ./manage.py sqlmigrate patchwork 0003_add_check_model - - However, this only works for schema migrations. For data migrations, - however, this will fail. In this cases, these migrations will need to be - handwritten. - -* You are using Django > 1.6, but upgrading from Patchwork < 1.0.0 - - Patchwork only started providing migrations in `v1.0.0`. SQL migrations are - provided for versions prior to this and must be applied to get the database - to the "initial" state that Django migrations expects. - -* You have diverged from upstream Patchwork - - If you have applied custom patches that change the database models, the - database in an "inconsistent state" and the provided migrations will likely - fail to apply. - -Steps to handle the latter two of these are described below. - -### Upgrading a pre-v1.0.0 Patchwork instance - -The process for this type of upgrade is quite simple: upgrade using manual -SQL upgrades until better options become available. As such, you should apply -all unapplied SQL migrations that are not duplicated by Django migrations. -Once such duplication occurs, rely on the Django migrations only and continue -to do so going forward. - -### Upgrading a "diverged" Patchwork instance - -This type of upgrade is a little trickier. There are two options you can take: - -1. Bring your Patchwork instance back in sync with upstream -2. Provide your own migrations - -The former option is particularly suitable if you decide to upstream your -change or decide it's not valuable enough to retain. This will require either -reworking any migrations that exist prior to your feature being upstreamed, or -deleting any added database fields and tables, respectively. In both cases, -manually, hand-written SQL migrations will be required to get the databse into -a consistent state (remember: **backup**!). Once this is done, you can resume -using the upstream-provided migrations, ensuring any Django migrations that you -may have skipped are not applied again: - - $ ./manage.py migrate 000x-abc --fake # when 000x-abc is last "skippable" - -It's worth adding that with the databases now back in sync it should be -possible to return to using upstream code rather than maintaining a fork. - -The latter option is best chosen if you wish to retain the aforementioned fork. -How you do this depends on the extensiveness of your changes, but getting the -latest version of Patchwork, deleting the provided migrations, applying any -patches you may have and regenerating the migrations seems like the best -option. - -**NOTE**: To prevent the latter case above from occurring, we'd ask that you -submit any patches you may have to the upstream Patchwork so that the wider -community can benefit from this new functionality. Please see -[the contributing document][doc-contributing] for more information on this -process. - -[doc-installation]: installation.md -[doc-contributing]: ../development/contributing.md -[gh-upgrading]: https://github.com/getpatchwork/patchwork/blob/master/UPGRADING.md -[gh-v1]: https://github.com/getpatchwork/patchwork/releases/tag/v1.0.0 -[ref-django-migrate]: https://docs.djangoproject.com/en/1.8/topics/migrations/ -[ref-south2]: http://blog.allenap.me/2015/05/south-south-2-and-django-migrations.html diff --git a/docs/development/contributing.md b/docs/development/contributing.md deleted file mode 100644 index aefd8a6..0000000 --- a/docs/development/contributing.md +++ /dev/null @@ -1,59 +0,0 @@ -# Tips and Tricks - -## Coding Standards - -**Follow PEP8**. All code is currently PEP8 compliant and it should stay this -way. - -Changes that fix semantic issues will be generally be happily received, but -please keep such changes separate from functional changes. - -`pep8` targets are provided via tox. Refer to the [testing section](#testing) -below for more information on usage of this tool. - -## Testing - -Patchwork includes a [tox][ref-tox] script to automate testing. This requires -a functional database and some Python requirements like `tox`. Please refer -to the [development guide][doc-development] for information on how to configure -these. - -Assuming these requirements are met, actually testing Patchwork is quite easy -to do. To start, you can show the default targets like so: - - $ tox --list - -You'll see that this includes a number of targets to run unit tests against -the different versions of Django supported, along with some other targets -related to code coverage and code quality. To run one of these, use the `-e` -parameter: - - $ tox -e py27-django18 - -In the case of the unit tests targets, you can also run specific tests by -passing the fully qualified test name as an additional argument to this -command: - - $ tox -e py27-django18 patchwork.tests.SubjectCleanUpTest - -Because Patchwork support multiple versions of Django, it's very important -that you test against all supported versions. When run without argument, tox -will do this: - - $ tox - -## Submitting Changes - -All patches should be sent to the [mailing list][pw-ml]. When doing so, please -abide by the [QEMU guidelines][ref-qemu-contrib] on contributing or submitting -patches. This covers both the initial submission and any follow up to the -patches. In particular, please ensure: - -* [All tests pass](#testing) -* Documentation has been updated with new requirements, new script names etc. -* The `CHANGES` file has been updated with any added or removed features - -[doc-development]: installation.md -[pw-ml]: https://ozlabs.org/mailman/listinfo/patchwork -[ref-qemu-contrib]: http://wiki.qemu.org/Contribute/SubmitAPatch -[ref-tox]: https://tox.readthedocs.io/en/latest/ diff --git a/docs/development/installation.md b/docs/development/installation.md deleted file mode 100644 index 90e8a11..0000000 --- a/docs/development/installation.md +++ /dev/null @@ -1,379 +0,0 @@ -# Development - -This document describes the necessary steps to configure Patchwork in a -development environment. If you are interested in deploying Patchwork in a -production environment, please refer to [the deployment guide][doc-deployment] -instead. - -To begin, you should clone Patchwork: - - $ git clone git://github.com/getpatchwork/patchwork.git - -## Docker-Based Installation - -Patchwork provides a Docker-based environment for quick configuration of a -development environment. This is the preferred installation method. To -configure Patchwork using Docker: - -1. Install [**Docker**][ref-docker] and [**docker-compose**][ref-compose]. - -2. Build the images. This will download over 200MB from the internet: - - $ docker-compose build - -3. Run `docker-compose up`: - - $ docker-compose up - - This will be visible at http://localhost:8000/. - -To run a shell within this environment, run: - - $ docker-compose run --rm web --shell - -To run django-manage commands, such as `createsuperuser` or `migrate`, run: - - $ docker-compose run --rm web python manage.py createsuperuser - -To run unit tests, excluding Selenium UI interaction tests, using only the -package versions installed during container initialization, run: - - $ docker-compose run --rm web --quick-test - -To run the same against all supported versions of Django (via tox), run: - - $ docker-compose run --rm web --quick-tox - -To run specific tox targets or tests, pass arguments to the above: - - $ docker-compose run --rm web --quick-tox -e py27-django17 \ - patchwork.tests.test_bundles - -To run all tests, including Selenium UI interaction tests, using only the -package versions installed container initialization, run: - - $ docker-compose run --rm web --test - -To run the same against all supported versions of Django (via tox), run: - - $ docker-compose run --rm web --tox - -To run all tests, including Selenium UI interaction tests in non-headless mode, -run: - - $ docker run -it --rm -v (pwd):/home/patchwork/patchwork/ \ - --link patchwork_db_1:db -p 8000:8000 \ - -v /tmp/.X11-unix:/tmp/.X11-unix \ - -e PW_TEST_DB_HOST=db -e DISPLAY patchwork_web bash - -To reset the database before any of these commands, add `--reset` to the -command line after `web` and before any other arguments. - -Any local edits to the project files made locally are immediately visible to -the Docker container, and so should be picked up by the Django auto-reloader. - -For more information on Docker itself, please refer to the [Docker][ref-docker] -and [docker-compose][ref-compose] documentation. - -**NOTE:** If using SELinux, you will need to create a custom SELinux rule to -allow the Docker process to access your working directory. Run: - - $ chcon -RT svirt_sandbox_file_t $PATCHWORK_DIR - -where `$PATCHWORK_DIR` is the absolute patch to the `patchwork` folder created -when you cloned the repo. For more information, see `man docker run`. - -**NOTE:** If you see an error like the below: - - ERROR: Couldn't connect to the Docker daemon at http+docker://localunixsocket - is it running? - -ensure you have correctly installed Docker, added your user to the `docker` -group, and started the daemon, per the [Docker documentation][ref-docker]. - -**NOTE:** If you see an error like the below: - - py.error.EACCES: [Permission denied]: open('/home/patchwork/patchwork/.tox/py27-django18/.tox-config1', 'w') - -your host user account is likely using a different UID to the one hardcoded in -the Dockerfile. You can confirm this like so: - - $ echo $UID - 1234 - -If this is something other than `1000`, you must must modify the `Dockerfile` -found in `tools/docker` to use your UID and then rebuild: - - $ sed -i "/ARG UID=/c\ARG UID=$(echo $UID)" tools/docker/Dockerfile - $ docker-compose build web - -This change must be retained in the event that you rebuild the container. You -can "hide" the change from Git like so: - - $ git update-index --assume-unchanged tools/docker/Dockerfile - $ git update-index --skip-worktree tools/docker/Dockerfile - -This should be resolved in a future release when we support docker-compose 2.1 -syntax in `docker-compose.yml`. - -## Vagrant-Based Installation - -Patchwork provides a Vagrant-based environment as an alternative to Docker. -Like Docker, Vagrant can be used to quickly configure Patchwork in a -development environment. To configure Patchwork using Vagrant: - -1. Install [**Vagrant**][ref-vagrant] - -2. Run `vagrant up` from the project directory: - - $ cd patchwork - $ vagrant up - -Once stacked, follow the on-screen instructions. For more information on -Vagrant itself, please refer to the [Vagrant documentation][ref-vagrant]. - -## Manual Installation - -Manual installation can be used where use of Docker or Vagrant is not possible or -desired. - -### Install Required Packages - -There are a number of different requirements for developing Patchwork: - -* Python and libraries -* A supported database (RDBMS) - -These are detailed below. - -#### Python Requirements - -To develop Python-based software you first need Python. Patchwork supports -both Python 2.7 and Python 3.3+. One of these will be installed by default -on many installations, though they can also be installed manually using the -`python` or `python3` packages. - -It's a good idea to use [virtual environments][ref-venv] to develop Python -software. Virtual environments are "instances" of your system Python without -any of the additional Python packages installed. They are useful to develop and -possibly deploy Patchwork against a "well known" set of dependencies, but they -can also be used to test Patchwork against several versions of Django. - -If you do not have `virtualenv` installed then you should install it now. This -can be installed using the `python-virtualenv` or `python3-virtualenv` -packages. Alternatively you can install these using `pip`. - -It is also helpful to install [`tox`][ref-tox] which is used for running tests -in Patchwork. This can be installed using the `python-tox` or `python3-tox` -packages, or via `pip`. - -#### Database Requirements - -If not already installed, you may need to install an RDBMS. You can use either -MariaDB/MySQL or PostgreSQL for this purpose. You should also install the -development headers, known as `libmysqlclient-dev` or `libpq-dev` respectively -on Debian-based Debian-based distros like Ubuntu and `mysql-devel` or -`postgresql-devel` on RHEL-based distros. - -**NOTE:** While Django provides support for -[multiple database backends][ref-django-db], Patchwork itself is only tested -against MySQL/MariaDB and PostgreSQL. Should you wish to use a different -backend, ensure you validate this first (and perhaps -[upstream][doc-contributing] any changes you may find necessary). - -**NOTE:** You may be tempted to use SQLite to develop Patchwork. We'd advise -against doing this. SQLite supports a subset of the functionality of "full" -RDBMS like MySQL: for example, case-sensitive matching of Unicode -[is not supported][ref-sqlite-utf8]. You will find some tests provided by -Patchwork fail and some patches you develop may fail in production due to these -differences. - -#### Example Installation - -An example for installing all these packages and the MySQL RDBMS on Ubuntu -15.04 is given below: - - $ sudo apt-get install python python-pip python-dev python-virtualenv \ - python-tox mysql-server libmysqlclient-dev - -If you have an existing MariaDB/MySQL installation and have installed `pip` -already/are using [Python 3.4+][ref-py34-pip] then you can install all -packages using `pip`: - - $ sudo pip install virtualenv tox - -If you wish to use Python 3 then simply replace 'python' with 'python3' in -the above command. - -### Configure Virtual Environment - -**NOTE:** If you are interested in simply [testing Patchwork][doc-testing], -many of the below steps are not required. tox will automatically install -dependencies and use virtual environments when testing. - -Once these requirements are installed, you should create and activate a new -virtual environment. This can be done like so: - - $ virtualenv .venv - $ source .venv/bin/activate - (.venv)$ - -**NOTE:** It you installed a Python 3.x-based virtual environment package, -adjust the executable indicated above as necessary, e.g. `virtualenv-3.4`. - -Now install the packages. Patchwork provides three requirements files. - -* `requirements-dev.txt`: Packages required to configure a development - environment -* `requirements-prod.txt`: Packages required for deploying Patchwork in - production -* `requirements-test.txt`: Packages required to run tests - -We're going to install the first of these, which can be done like so: - - (.venv)$ cd patchwork - (.venv)$ pip install -r requirements-dev.txt - -**NOTE:** Once configured this does not need to be done again *unless* the -requirements change, e.g. Patchwork requires an updated version of Django. - -### Initialize the Database - -One installed, the database must be configured. We will assume you have root -access to the database for these steps. - -To begin, export your database credentials as follows: - - (.venv)$ db_user=root - (.venv)$ db_pass=password - -Now, create the database. If this is your first time configuring the database, -you must create a `patchwork` user (or similar) along with the database -instance itself. The commands below will do this, dropping existing databases -if necessary: - - (.venv)$ mysql -u$db_user -p$db_pass << EOF - DROP DATABASE IF EXISTS patchwork; - CREATE DATABASE patchwork CHARACTER SET utf8; - GRANT ALL PRIVILEGES ON patchwork.* TO 'patchwork'@'localhost' - IDENTIFIED BY 'password'; - EOF - -**NOTE:** The `patchwork` username and `password` password are the defaults -expected by the provided `dev` settings files. If using something different, -please export the `PW_TEST_DB_USER` and `PW_TEST_DB_PASS` variables described -in the [Environment Variables](#environment-variables) section below. -Alternatively, you can create your own settings file with these variables -hardcoded and change the value of `DJANGO_SETTINGS_MODULE` as described below. - -### Load Initial Data - -Before continuing, we need to tell Django where it can find our configuration. -Patchwork provides a default development `settings.py` file for this purpose. -To use this, export the `DJANGO_SETTINGS_MODULE` environment variable as -described below: - - (.venv)$ export DJANGO_SETTINGS_MODULE=patchwork.settings.dev - -Alternatively you can provide your own `settings.py` file and provide the path -to that instead. - -Once done, we need to create the tables in the database. This can be done using -the `migrate` command of the `manage.py` executable: - - (.venv)$ ./manage.py migrate - -Next, you should load the initial fixtures into Patchwork. These initial -fixtures provide. - -* `default_tags.xml`: The tags that Patchwork will extract from mails. - Examples: `Acked-By`, `Reviewed-By` -* `default_states.xml`: The states that a patch can be in. Examples: - `Accepted`, `Rejected` -* `default_projects.xml`: A default project that you can then upload patches - for - -These can be loaded using the `loaddata` command: - - (.venv)$ ./manage.py loaddata default_tags default_states default_projects - -You should also take the opportunity to create a "superuser". You can do this -using the aptly-named `createsuperuser` command: - - (.venv)$ ./manage.py createsuperuser - -## Import Mailing List Archives - -Once this is done, it's beneficial to load some real emails into the system. -This can be done manually, however it's generally much easier to download -an archive from a Mailman instance and load these using the `parsearchive` -command. You can do this like so: - - (.venv)$ mm_user=<myusername> - (.venv)$ mm_pass=<mypassword> - (.venv)$ mm_host=https://lists.ozlabs.org - (.venv)$ mm_url=$mm_host/private/patchwork.mbox/patchwork.mbox - (.venv)$ curl -F username=$mm_user -F password=$mm_pass -k -O $mm_url - -where `mm_user` and `mm_pass` are the username and password you have registered -with on the Mailman instance found at `mm_host`. - -**NOTE:** We provide instructions for downloading archives from the Patchwork -mailing list, but almost any instance of Mailman will allow downloading of -archives as seen above; simply change the `pw_url` variable defined. You can -find more informations about this [here][ref-mman-bulk]. - -Load these archives into Patchwork. Depending on the size of the downloaded -archives this may take some time: - - (.venv)$ ./manage.py parsearchive --list-id=patchwork.ozlabs.org \ - patchwork.mbox - -Finally, run the server and browse to the IP address of your board using your -browser of choice: - - (.venv)$ ./manage.py runserver 0.0.0.0:8000 - -Once finished, you can kill the server (`Ctrl` + `C`) and exit the virtual -environment: - - (.venv)$ deactivate - $ - -Should you wish to re-enter this environment, simply source the `activate` -script again. - -## Django Debug Toolbar - -Patchwork installs and enables the 'Django Debug Toolbar' by default. However, -by default this is only displayed if you are developing on localhost. If -developing on a different machine, you should configure an SSH tunnel such -that, for example, `localhost:8000` points to `[DEV_MACHINE_IP]:8000`. - -## Environment Variables - -The following environment variables are available to configure settings when -using the provided `dev` settings file. - -<dl> - <dt>PW_TEST_DB_NAME = 'patchwork'</dt> - <dd>Name of the database</dd> - <dt>PW_TEST_DB_USER = 'patchwork'</dt> - <dd>Username to access the database with</dd> - <dt>PW_TEST_DB_PASS = 'password'</dt> - <dd>Password to access the database with</dd> - <dt>PW_TEST_DB_TYPE = 'mysql'</dt> - <dd>Type of database to use. Options: 'mysql', 'postgres'</dd> -</dl> - -[doc-contributing]: ../development/contributing.md -[doc-deployment]: ../deployment/installation.md -[doc-testing]: testing.md -[ref-django-db]: https://docs.djangoproject.com/en/1.8/ref/databases/ -[ref-mman-bulk]: http://blog.behnel.de/posts/indexp118.html -[ref-py34-pip]: http://legacy.python.org/dev/peps/pep-0453/ -[ref-sqlite-utf8]: https://www.sqlite.org/faq.html#q18 -[ref-tox]: https://tox.readthedocs.io/en/latest/ -[ref-compose]: https://docs.docker.com/compose/install/ -[ref-docker]: https://docs.docker.com/engine/installation/linux/ -[ref-vagrant]: https://www.vagrantup.com/docs/getting-started/ -[ref-venv]: https://virtualenv.readthedocs.io/en/latest/ diff --git a/docs/development/releasing.md b/docs/development/releasing.md deleted file mode 100644 index e09b524..0000000 --- a/docs/development/releasing.md +++ /dev/null @@ -1,49 +0,0 @@ -# Release Process - -## Versioning - -Since version 1.0, Patchwork has implemented a version of -[Semantic Versioning][ref-semver]. To summarise, releases take the format -**MAJOR.MINOR.PATCH** (or just **MAJOR.MINOR**). We increment: - -1. **MAJOR** version when we make major UI changes or functionality updates -2. **MINOR** version when we make minor UI changes or functionality updates -3. **PATCH** version when we make make bug fixes, dependency updates etc. - -In Git, each release will have a tag indicating the version number. In -addition, each release series has it's own branch called `stable/MAJOR.MINOR` -to allow backporting of bugfixes or security updates to older versions. - -## Release Cycle - -There is no cadence for releases: they are made available as necessary. - -## Supported Versions - -Typically all development should occur on `master`. While we will backport -bugfixes and security updates, we will not backport any new features. This -is to ensure stability for users of these versions of Patchwork. - -## Release Checklist - -* Documentation has been updated with latest release version -* Documentation references latest supported version of Django - -## Backporting - -We will occasionally backport bugfixes and security updates. When backporting -a patch, said patch should first be merged into `master`. Once merged, you can -backport by cherry-picking commits, using the `-x` flag for posterity: - - $ git cherry-pick <master_commit> -x - -There may be some conflicts; resolve these, uncommenting the `Conflicts` line -when commiting: - - Conflicts - patchwork/bin/pwclient - -When enough patches have been backported, you should release a new `PATCH` -release. - -[ref-semver]: http://semver.org/ diff --git a/docs/development/xmlrpc.md b/docs/development/xmlrpc.md deleted file mode 100644 index 1ebc540..0000000 --- a/docs/development/xmlrpc.md +++ /dev/null @@ -1,54 +0,0 @@ -# The XML-RPC API - -**NOTE:** This guide covers development information for the Patchwork XML-RPC -API. For information on using the REST API, refer to the [REST API -documentation][doc-rest]. For information on general usage of the XML-RPC API, -refer to the [user documentation][doc-usage]. - -Patchwork provides an XML-RPC API. This API can be used to be used to retrieve -and modify information about patches, projects and more. - -**NOTE:** The XML-RPC API can be enabled/disabled by the administrator: it may -not be available in every instance. - -## Patchwork API Documentation - -Patchwork provides automatically generated documentation for the XML-RPC API. -You can find this at the following URL: - - http://patchwork.example.com/xmlrpc/ - -Where `patchwork.example.com` refers to the URL of your Patchwork instance. - -**NOTE:** Automatic documentation generation for the Patchwork API was -introduced in Patchwork v1.1. Prior versions of Patchwork do not offer this -functionality. - -## Interacting with the API - -**NOTE:** The Patchwork XML-RPC API provides a number of "methods". Some -methods require authentication (via HTTP Basic Auth) while others do not. -Authentication uses your Patchwork account and the on-server documentation will -indicate where it is necessary. We will only cover the unauthenticated method -here for brevity - please consult the [`xmlrpclib`][ref-xmlrpclib] -documentation for more detailed examples: - -To interact with the Patchwork XML-RPC API, a XML-RPC library should be used. -Python provides such a library - [`xmlrpclib`][ref-xmlrpclib] - in its standard -library. For example, to get the version of the XML-RPC API for a Patchwork -instance hosted at `patchwork.example.com`, run: - - $ python - >>> import xmlrpclib # or 'xmlrpc.client' for Python 3 - >>> rpc = xmlrpclib.ServerProxy('http://patchwork.example.com/xmlrpc/') - >>> rpc.pw_rpc_version() - 1.1 - -Once connected, the `rpc` object will be populated with a list of available -functions (or procedures, in RPC terminology). In the above example, we used -the `pw_rpc_version` method, however, it should be possible to use all the -methods listed in the [server documentation](#patchwork-api-documentation). - -[doc-rest]: ../usage/rest.md -[doc-usage]: ../usage/xmlrpc.md -[ref-xmlrpclib]: https://docs.python.org/2/library/xmlrpclib.html diff --git a/docs/index.md b/docs/index.md deleted file mode 100644 index 526ede0..0000000 --- a/docs/index.md +++ /dev/null @@ -1,68 +0,0 @@ -# Patchwork - -Patchwork is a patch tracking system for community-based projects. It is -intended to make the patch management process easier for both the project's -contributors and maintainers, leaving time for the more important (and more -interesting) stuff. - -Patches that have been sent to a mailing list are 'caught' by the system, and -appear on a web page. Any comments posted that reference the patch are appended -to the patch page too. The project's maintainer can then scan through the list -of patches, marking each with a certain state, such as Accepted, Rejected or -Under Review. Old patches can be sent to the archive or deleted. - -Currently, Patchwork is being used for a number of open-source projects, mostly -subsystems of the Linux kernel. Although Patchwork has been developed with the -kernel workflow in mind, the aim is to be flexible enough to suit the majority -of community projects. - -# Download - -The latest version of Patchwork is available with git. To download: - - $ git clone git://github.com/getpatchwork/patchwork - -Patchwork is distributed under the [GNU General Public License][ref-gpl]. - -# Design - -## Patchwork should supplement mailing lists, not replace them - -Patchwork isn't intended to replace a community mailing list; that's why you -can't comment on a patch in Patchwork. If this were the case, then there would -be two forums of discussion on patches, which fragments the patch review -process. Developers who don't use Patchwork would get left out of the -discussion. - -However, a future development item for Patchwork is to facilitate on-list -commenting, by providing a "send a reply to the list" feature for logged-in -users. - -## Don't pollute the project's changelogs with Patchwork poop - -A project's changelogs are valuable - we don't want to add Patchwork-specific -metadata. - -## Patchwork users shouldn't require a specific version control system - -Not everyone uses git for kernel development, and not everyone uses git for -Patchwork-tracked projects. - -It's still possible to hook other programs into Patchwork, using the pwclient -command-line client for Patchwork, or directly to the XML RPC interface. - -# Getting Started - -You should check out the [deployment][doc-deployment] and -[development][doc-development] guides for information on how to configure -Patchwork for production and development environments, respectively. - -# Support - -All questions and contributions should be sent to the -[Patchwork mailing list][ref-pw-ml]. - -[ref-gpl]: http://www.gnu.org/licenses/gpl-2.0.html -[ref-pw-ml]: https://ozlabs.org/mailman/listinfo/patchwork -[doc-deployment]: deployment/installation.md -[doc-development]: development/installation.md diff --git a/docs/usage/delegation.md b/docs/usage/delegation.md deleted file mode 100644 index 7e86cfb..0000000 --- a/docs/usage/delegation.md +++ /dev/null @@ -1,45 +0,0 @@ -# Autodelegation - -Autodelegation allows patches to be automatically delegated to a user based on -the files modified by the patch. To do this, a number of rules can be -configured in the project administration page. This can usually be found at -`/admin/patchwork/project/<project_id>/change`. - -**NOTE:** Autodelegation can only be configured by Patchwork administrators, -i.e. those that can access the 'admin' panel. If you require configuration of -autodelegation rules on a local instance, contact your Patchwork administrator. - -In this section there are the following fields: - -- User - - The patchwork user that should be autodelegated to the patch - -- Priority - - The priority of the rule relative to other patches. Higher values indicate - higher priority. If two rules have the same priority, ordering will be - based on the path. - -- Path - - A path in [fnmatch](https://docs.python.org/2/library/fnmatch.html) format. - The fnmatch library allows for limited, Unix shell-style wildcarding. - Filenames are extracted from patch lines beginning with `--- ` or `+++ `. - Note that for projects using Git or Mercurial, the tools these VCS provide - for producing patches are prefixed with `a` or `b`. You should account for - this in your path. For example, to match the path `patchwork/views` - (relative to the top of a Git repo) your pattern should be: - - ?/patchwork/views/* - - It is also possible to use relative paths, such as: - - */manage.py - - For projects using other VCSs like Subversion can simply use a bare path: - - patchwork/views/* - -Rules are configured by setting the above fields and saving the rules. These -rules will be applied at patch parse time. diff --git a/docs/usage/headers.md b/docs/usage/headers.md deleted file mode 100644 index dc87397..0000000 --- a/docs/usage/headers.md +++ /dev/null @@ -1,34 +0,0 @@ -# Hint Headers - -Patchwork provides a number of special email headers to control how a patch is -handled when it is received. The examples provided below use `git-send-email`, -but custom headers can also be set when using tools like `mutt`. - -## `X-Patchwork-Ignore` - -Valid values: * - -When set, the mere presence of this header will ensure the provided email is -not parsed by Patchwork. For example: - - $ git send-email --add-header="X-Patchwork-Ignore: test" master - -## `X-Patchwork-Delegate` - -Valid values: An email address associated with a Patchwork user - -If set and valid, the user corresponding to the provided email address will be -assigned as the delegate of any patch parsed. If invalid, it will be ignored. -For example: - - $ git send-email --add-header="X-Patchwork-Delegate: a@example.com" master - -## `X-Patchwork-State` - -Valid values: Varies between deployments. This can usually be one of -"Accepted", "Rejected", "RFC" or "Awaiting Upstream", among others. - -If set and valid, the state provided will be assigned as the state of any patch -parsed. If invalid, it will be ignored. For example: - - $ git send-email --add-header="X-Patchwork-State: RFC" master diff --git a/docs/usage/overview.md b/docs/usage/overview.md deleted file mode 100644 index de808c8..0000000 --- a/docs/usage/overview.md +++ /dev/null @@ -1,313 +0,0 @@ -# Overview - -The key concepts or models of Patchwork are outlined below. - -## Projects - -Projects typically represent a software project or sub-project. A Patchwork -server can host multiple projects. Each project can have multiple maintainers. -Projects usually have a 1:1 mapping with a mailing list, though it's also -possible to have multiple projects in the same list using the subject as -filter. Patches, cover letters, and series are all associated with a single -project. - -## People - -People are anyone who has submitted a patch, cover letter, or comment to a -Patchwork instance. - -## Users - -Users are anyone who has created an account on a given Patchwork instance. - -### Standard Users - -A standard user can associate multiple email addresses with their user account, -create bundles and store TODO lists. - -### Maintainers - -Maintainers are a special type of user that with permissions to do certain -operations that regular Patchwork users can't. Patchwork maintainers usually -have a 1:1 mapping with a project's code maintainers though this is not -necessary. - -The operations that a maintainer can invoke include: - -- Change the state of a patch -- Archive a patch -- Delegate a patch, or be delegated a patch - -## Submissions - -Patchwork captures three types of mail to mailing lists: patches, cover -letters, and replies to either patches or cover letters, a.k.a. comments. Any -mail that does not fit one of these categories is ignored. - -### Patches - -Patches are the central object in Patchwork structure. A patch contains both a -diff and some metadata, such as the name, the description, the author, the -version of the patch etc. Patchwork stores not only the patch itself but also -various metadata associated with the email that the patch was parsed from, such -as the message headers or the date the message itself was received. - -### Cover Letters - -Cover letters provide a way to offer a "big picture" overview of a series of -patches. When using Git, these mails can be recognised by way of their `0/N` -subject prefix, e.g. `[00/11] A sample series`. Like patches, Patchwork stores -not only the various aspects of the cover letter itself, such as the name and -body of the cover letter, but also various metadata associated with the email -that the cover letter was parsed from. - -### Comments - -Comments are replies to a submission - either a patch or a cover letter. Unlike -a Mail User Agent (MUA) like Gmail, Patchwork does not thread comments. -Instead, every comment is associated with either a patch or a cover letter, and -organized by date. - -## Patch Metadata - -Patchwork allows users to store various metadata against patches. This metadata -is only configurable by a maintainer. - -### States - -States track the state of patch in its lifecycle. States vary from project to -project, but generally a minimum subset of "new", "rejected" and "accepted" -will exist. - -### Delegates - -Delegates are Patchwork users who are responsible for both reviewing a patch -and setting its eventual state in Patchwork. This makes them akin to reviewers -in other tools. Delegation works particularly well for larger projects where -various subsystems, each with their own maintainer(s), can be identified. Only -one delegate can be assigned to a patch. - -**NOTE:** Patchwork supports automatic delegation of patches. Refer to the -[Autodelegation Guide][doc-autodelegation] for more information. - -### Tags - -Tags are specially formatted metadata appended to the foot the body of a patch -or a comment on a patch. Patchwork extracts these tags at parse time and -associates them with the patch. You add extra tags to an email by replying to -the email. The following tags are available on a standard Patchwork install: - -- Acked-by: - - For example: - - Acked-by: Stephen Finucane <stephen@that.guru> - -- Tested-by: - - For example: - - Tested-by: Stephen Finucane <stephen@that.guru> - -- Reviewed-by: - - For example: - - Tested-by: Stephen Finucane <stephen@that.guru> - -The available tags, along with the significance of said tags, varies from -project to project and Patchwork instance to Patchwork instance. The [kernel -project documentation][ref-kernel-submission] provides an overview of the -supported tags for the Linux kernel project. - -### Checks - -Checks store the results of any tests executed (or executing) for a given -patch. This is useful, for example, when using a continuous integration (CI) -system to test patches. Checks have a number of fields associated with them: - -- Context - - A label to discern check from the checks of other testing systems - -- Description - - A brief, optional description of the check - -- Target URL - - A target URL where a user can find information related to this check, such - as test logs. - -- State - - The state of the check. One of: pending, success, warning, fail - -- User - - The user creating the check - -**NOTE:** Checks can only be created through the Patchwork APIs. Refer to the -[API documentation][doc-api] for more information. - -**TODO:** Provide information on building a CI system that reports check -results back to Patchwork. - -## Collections - -Patchwork provides a number of ways to store groups of patches. Some of these -are automatically generated, while others are user-defined. - -### Series - -Series are groups of patches, along with an optional cover letter. Series are -mostly dumb containers, though they also contain some metadata themselves such -as a version (which is inherited by the patches and cover letter) and a count -of the number of patches found in the series. - -### Bundles - -Bundles are custom, user-defined groups of patches. Bundles can be used to keep -patch lists, preserving order, for future inclusion in a tree. There's no -restriction of number of patches and they don't even need to be in the same -project. A single patch also can be part of multiple bundles at the same time. -An example of Bundle usage would be keeping track of the Patches that are ready -for merge to the tree. - -### To-do Lists - -Patchwork users can store a to-do list of patches. - -## Events - -Events are raised whenever patches are created or modified. - -All events have a number of common properties, along with some event-specific -properties: - -<dl> - <dt>category</dt> - <dd>The type of event</dd> - <dt>project<dt> - <dd>The project this event belongs to</dd> - <dt>date</dt> - <dd>When this event was created</dd> -</dl> - -**NOTE:** Checks can only be created and read through the Patchwork APIs. Refer -to the [API documentation][doc-api] for more information. - -### Cover Letter Created - -Sent when a cover letter is created. - -<dl> - <dt>category</dt> - <dd><code>cover-created</code></dd> - <dt>cover<dt> - <dd>Created cover letter</dd> -</dl> - -### Patch Created - -Sent when a patch is created. - -<dl> - <dt>category</dt> - <dd><code>patch-created</code></dd> - <dt>patch<dt> - <dd>Created patch</dd> -</dl> - -### Patch Completed - -Sent when a patch in a series has its dependencies met, or when a patch that is -not in a series is created (since that patch has no dependencies). - -<dl> - <dt>category</dt> - <dd><code>patch-completed</code></dd> - <dt>patch<dt> - <dd>Completed patch</dd> - <dt>series<dt> - <dd>Series from which patch dependencies were extracted, if any</dd> -</dl> - -### Patch Delegated - -Sent when a patch's delegate is changed. - -<dl> - <dt>category</dt> - <dd><code>patch-delegated</code></dd> - <dt>patch<dt> - <dd>Updated patch</dd> - <dt>previous</dt> - <dd>Previous delegate, if any</dd> - <dt>current</dt> - <dd>Current delegate, if any</dd> -</dl> - -### Patch State Changed - -Sent when a patch's state is changed. - -<dl> - <dt>category</dt> - <dd><code>patch-state-changed</code></dd> - <dt>patch<dt> - <dd>Updated patch</dd> - <dt>previous</dt> - <dd>Previous state</dd> - <dt>current</dt> - <dd>Current state</dd> -</dl> - -### Check Created - -Sent when a patch check is created. - -<dl> - <dt>category</dt> - <dd><code>check-created</code></dd> - <dt>check<dt> - <dd>Created check</dd> -</dl> - - -### Series Created - -Sent when a series is created. - -<dl> - <dt>category</dt> - <dd><code>series-created</code></dd> - <dt>series<dt> - <dd>Created series</dd> -</dl> - -### Series Completed - -Sent when a series is completed. - -<dl> - <dt>category</dt> - <dd><code>series-completed</code></dd> - <dt>series<dt> - <dd>Completed series</dd> -</dl> - -### What's Not Exposed - -* Bundles - - We don't expose an "added to bundle" event as it's unlikely that this will - be useful to either users or CI setters. - -* Comments - - Like Bundles, there likely isn't much value in exposing these via the API. - -[doc-api]: rest.md -[doc-autodelegation]: delegation.md -[ref-kernel-submission]: https://www.kernel.org/doc/Documentation/SubmittingPatches diff --git a/docs/usage/rest.md b/docs/usage/rest.md deleted file mode 100644 index e0fbf24..0000000 --- a/docs/usage/rest.md +++ /dev/null @@ -1,56 +0,0 @@ -# The REST API - -Patchwork provides a REST API. This API can be used to retrieve and modify -information about patches, projects and more. - -**NOTE:** The REST API was introduced in Patchwork v2.0. Users of earlier -Patchwork versions should instead refer to the [XML-RPC API -documentation][doc-xmlrpc]. - -## Patchwork REST API documentation - -Patchwork provides automatically generated documentation for the RESET API. -You can find this at the following URL: - - http://patchwork.example.com/api/ - -where `patchwork.example.com` refers to the URL of your Patchwork instance. - -## Interacting with the API - -REST APIs run over plain HTTP(S), thus, the API can be interfaced using -applications or libraries that support this widespread protocol. One such -application is [`curl`][ref-curl], which can be used to both retrieve and send -information to the REST API. For example, to get the version of the REST API -for a Patchwork instance hosted at `patchwork.example.com`, run: - - $ curl -s http://localhost:8000/api/1.0/ | python -m json.tool - { - "patches": "http://localhost:8000/api/1.0/patches/", - "people": "http://localhost:8000/api/1.0/people/", - "projects": "http://localhost:8000/api/1.0/projects/", - "users": "http://localhost:8000/api/1.0/users/" - } - -In addition, a huge variety of libraries are avaiable for interacting with and -parsing the output of REST APIs. The [`requests`][ref-requests] library is -wide-spread and well-supported. To repeat the above example using `requests`: - - $ python - >>> import json - >>> import requests - >>> r = requests.get('http://patchwork.example.com/api/1.0/') - >>> print(json.dumps(r.json(), indent=2)) - { - "users": "http://localhost:8000/api/1.0/users/", - "patches": "http://localhost:8000/api/1.0/patches/", - "projects": "http://localhost:8000/api/1.0/projects/", - "people": "http://localhost:8000/api/1.0/people/" - } - -Tools like `curl` and libraries like `requests` can be used to build anything -from small utilities to full-fledged clients targeting the REST API. - -[doc-xmlrpc]: xmlrpc.md -[ref-curl]: https://curl.haxx.se/ -[ref-requests]: http://docs.python-requests.org/en/master/ diff --git a/docs/usage/xmlrpc.md b/docs/usage/xmlrpc.md deleted file mode 100644 index 782edae..0000000 --- a/docs/usage/xmlrpc.md +++ /dev/null @@ -1,39 +0,0 @@ -# The XML-RPC API - -**NOTE:** This guide covers usage information for the Patchwork XML-RPC API. -For information on using the REST API, refer to the [REST API -documentation][doc-rest]. For information on developing custom applications or -clients for this API, refer to the [developers documentation][doc-development]. - -Patchwork provides an XML-RPC API. This API can be used to be used to retrieve -and modify information about patches, projects and more. - -**NOTE:** The XML-RPC API can be enabled/disabled by the administrator: it may -not be available in every instance. - -## pwclient - -The `pwclient` application, provided with Patchwork, can be used to interact -with Patchwork from the command line. Functionality provided by `pwclient` -includes: - -* Listing patches, projects, and checks -* Downloading and applying patches to a local code base -* Modifying the status of patches -* Creating new checks - -pwclient can be downloaded from the [Ozlabs Patchwork instance][ref-pw-oz], or -at the following path for other Patchwork instances: - - http://patchwork.example.com/pwclient/ - -where `patchwork.example.com` corresponds to the URL a Patchwork instance is -hosted at. - -Once downloaded, to view information about all the operations supported by -`pwclient`, run: - - $ pwclient --help - -[doc-development]: ../development/xmlrpc.md -[doc-rest]: rest.md diff --git a/mkdocs.yml b/mkdocs.yml deleted file mode 100644 index cdce06f..0000000 --- a/mkdocs.yml +++ /dev/null @@ -1,22 +0,0 @@ -site_name: patchwork -site_url: http://jk.ozlabs.org/projects/patchwork/ -site_description: patchwork - the web-based patch tracking system - -repo_url: git://github.com/getpatchwork/patchwork - -pages: - - Home: 'index.md' - - Development Guide: - - Installation: 'development/installation.md' - - Contributing: 'development/contributing.md' - - Release Process: 'development/releasing.md' - - XML-RPC API: 'development/xmlrpc.md' - - Deployment Guide: - - Installation: 'deployment/installation.md' - - Upgrading: 'deployment/upgrading.md' - - Usage Guide: - - Overview: 'usage/overview.md' - - REST API: 'usage/rest.md' - - XML-RPC API: 'usage/xmlrpc.md' - - Hint Headers: 'usage/headers.md' - - Delegation: 'usage/delegation.md' |