aboutsummaryrefslogtreecommitdiff
path: root/docs/deployment
diff options
context:
space:
mode:
authorStephen Finucane <stephen@that.guru>2017-04-18 21:57:46 +0100
committerStephen Finucane <stephen@that.guru>2017-04-18 22:08:32 +0100
commit7ebaba698c6b4f7e6a048f291a087fcfc81d58dd (patch)
tree90677f04f7052cf73234dbe205ae69217a01488f /docs/deployment
parent06378101d9cfc0d4aec78980ae4c7eaf66a187ab (diff)
downloadpatchwork-7ebaba698c6b4f7e6a048f291a087fcfc81d58dd.tar
patchwork-7ebaba698c6b4f7e6a048f291a087fcfc81d58dd.tar.gz
docs: Convert deployment guide
Signed-off-by: Stephen Finucane <stephen@that.guru>
Diffstat (limited to 'docs/deployment')
-rw-r--r--docs/deployment/installation.rst615
-rw-r--r--docs/deployment/upgrading.rst152
2 files changed, 767 insertions, 0 deletions
diff --git a/docs/deployment/installation.rst b/docs/deployment/installation.rst
new file mode 100644
index 0000000..3b3636d
--- /dev/null
+++ b/docs/deployment/installation.rst
@@ -0,0 +1,615 @@
+Installation
+============
+
+This document describes the necessary steps to configure Patchwork in a
+production environment. This requires a significantly "harder" deployment than
+the one used for development. If you are interested in developing Patchwork,
+refer to the **development guide** instead.
+
+This document describes a two-node installation of Patchwork, consisting of a
+database sever and an application server. It should be possible to combine
+these machines for smaller Patchwork instances. It should also be possible to
+configure high availability deployment through use of additional database and
+application machines, though this is out of the scope of this document.
+
+.. todo::
+
+ Update this guide to the latest Ubuntu LTS (16.04)
+
+Deployment Guides, Provisioning Tools and Platform-as-a-Service
+---------------------------------------------------------------
+
+Before continuing, it's worth noting that Patchwork is a Django application.
+With the exception of the handling of incoming mail (described below), it
+can be deployed like any other Django application. This means there are tens,
+if not hundreds, of existing articles and blogs detailing how to deploy an
+application like this. As such, if any of the below information is unclear
+then we'd suggest you go search for "Django deployment guide" or similar,
+deploy your application, and submit a patch for this guide to clear up that
+confusion for others.
+
+You'll also find that the same search reveals a significant number of existing
+deployment tools aimed at Django. These tools, be they written in Ansible,
+Puppet, Chef or something else entirely, can be used to avoid much of the
+manual configuration described below. If possible, embrace these tools to make
+your life easier.
+
+Finally, many Platform-as-a-Service (PaaS) providers and tools support
+deployment of Django applications with minimal effort. Should you wish to avoid
+much of the manual configuration, we suggest you investigate the many options
+available to find one that best suits your requirements. The only issue here
+will likely be the handling of incoming mail - something which many of these
+providers don't support. We address this in the appropriate section below.
+
+Requirements
+------------
+
+For the purpose of this guide, we will assume the following machines:
+
++-------------+------------+
+| server role | IP address |
++=============+============+
+| database | 10.1.1.1 |
++-------------+------------+
+| application | 10.1.1.2 |
++-------------+------------+
+
+We will use the database server to, ostensibly enough, host the database for
+the Patchwork instance. The application server, on the other hand, will host
+the Patchwork instance along with the required reverse proxy and WSGI HTTP
+servers.
+
+We expect a Ubuntu 15.04 installation on each of these hosts: commands, package
+names and/or package versions will likely change if using a different distro or
+release. In addition, usage of different package versions to the ones suggested
+may require slightly different configuration.
+
+Before beginning, you should update these systems:
+
+.. code-block:: shell
+
+ $ sudo apt-get update
+ $ sudo apt-get upgrade
+
+We also need to configure some environment variables to ease deployment. These
+should be exported on all systems:
+
+`PW_HOST_DB=10.1.1.1`
+
+ IP of the database host
+
+`PW_HOST_APP=10.1.1.2`
+
+ IP of the application host
+
+`PW_DB_NAME=patchwork`
+
+ Name of the database
+
+`PW_DB_USER=www-data`
+
+ Username that the Patchwork app will access the database with
+
+Database
+--------
+
+These steps should be run on the database server.
+
+.. note::
+
+ If you already have a database server on site, you can skip much of this
+ section.
+
+Install Requirements
+~~~~~~~~~~~~~~~~~~~~
+
+We're going to rely on PostgreSQL. You can adjust the below steps if using a
+different RDBMS. Install the required packages.
+
+.. code-block:: shell
+
+ $ sudo apt-get install -y postgresql postgresql-contrib
+
+Configure Database
+~~~~~~~~~~~~~~~~~~
+
+PostgreSQL created a user account called `postgres`; you will need to run
+commands as this user. Use this account to create the database that Patchwork
+will use, using the credentials we configured earlier.
+
+.. code-block:: shell
+
+ $ sudo -u postgres createdb $PW_DB_NAME
+ $ sudo -u postgres createuser $PW_DB_USER
+
+We will also need to apply permissions to the tables in this database but
+seeing as the tables haven't actually been created yet this will have to be
+done later.
+
+.. todo::
+
+ Add `pg_hba.conf` configuration
+
+Patchwork
+---------
+
+These steps should be run on the application server.
+
+Install Packages
+~~~~~~~~~~~~~~~~
+
+The first requirement is Patchwork itself. It can be downloaded like so:
+
+.. code-block:: shell
+
+ $ wget https://github.com/getpatchwork/patchwork/archive/v1.1.0.tar.gz
+
+We will install this under `/opt`, though this is only a suggestion:
+
+.. code-block:: shell
+
+ $ tar -xvzf v1.1.0.tar.gz
+ $ sudo mv v1.1.0 /opt/patchwork
+
+.. important::
+
+ Per the `Django documentation`__, source code should not be placed in your
+ web server's document root as this risks the possibility that people may be
+ able to view your code over the Web. This is a security risk.
+
+__ https://docs.djangoproject.com/en/dev/intro/tutorial01/#creating-a-project
+
+Next we require Python. If not already installed, then you should do so now.
+Patchwork supports both Python 2.7 and Python 3.3+, though we would suggest
+using the latter to ease future upgrades:
+
+.. code-block:: shell
+
+ $ sudo apt-get install python3 # or 'python' if using Python 2.7
+
+We require a number of Python packages. These can be installed using `pip`:
+
+.. code-block:: shell
+
+ $ sudo pip install -r /opt/patchwork/requirements-prod.txt
+
+If you're not using `pip`, you will need to identify and install the
+corresponding distro package for each of these requirements. For example:
+
+.. code-block:: shell
+
+ $ sudo apt-get install python3-django
+
+.. tip::
+
+ The `pkgs.org`__ website provides a great reference for identifying the name
+ of these dependencies.
+
+__ https://pkgs.org/
+
+Configure Patchwork
+~~~~~~~~~~~~~~~~~~~
+
+You will also need to configure a `settings file`__ for Django. A sample
+settings file is provided that defines default settings for Patchwork. You'll
+need to configure settings for your own setup and save this as `production.py`.
+
+.. code-block:: shell
+
+ $ cp patchwork/settings/production.example.py \
+ patchwork/settings/production.py
+
+Alternatively, you can override the `DJANGO_SETTINGS_MODULE` environment
+variable and provide a completely custom settings file.
+
+.. important::
+
+ You should not include shell variables in settings but rather hardcoded
+ values. These settings files are evaluated in Python - not a shell.
+
+__ https://docs.djangoproject.com/en/1.8/ref/settings/
+
+Databases
+^^^^^^^^^
+
+You can configure the `DATABASES` setting using the variables we set earlier.
+
+.. code-block:: python
+
+ DATABASES = {
+ 'default': {
+ 'ENGINE': 'django.db.backends.postgresql_psycopg2',
+ 'HOST': '$PW_HOST_DB', # don't use sh variables but actual values
+ 'PORT': '',
+ 'NAME': '$PW_DB_NAME',
+ 'USER': '$PW_DB_USER',
+ 'PASSWORD': '$PW_DB_PASS',
+ 'TEST': {
+ 'CHARSET': 'utf8',
+ },
+ },
+ }
+
+.. note::
+
+ `TEST/CHARSET` is used when creating tables for the test suite. Without it,
+ tests checking for the correct handling of non-ASCII characters fail. It is
+ not necessary if you don't plan to run tests, however.
+
+Static Files
+^^^^^^^^^^^^
+
+While we have not yet configured our proxy server, we do need to configure the
+location that these files will be stored in. We will install these under
+`/var/www/patchwork`, though this is only a suggestion and can be changed.
+
+.. code-block:: shell
+
+ $ mkdir /var/www/patchwork
+
+You can configure this by setting the `STATIC_ROOT` variable.
+
+.. code-block:: shell
+
+ STATIC_ROOT = '/var/www/patchwork'
+
+Other Options
+^^^^^^^^^^^^^
+
+Finally, the following settings need to be configured. The purpose of these
+variables is described in the `Django documentation`__:
+
+* `SECRET_KEY`
+* `ADMINS`
+* `TIME_ZONE`
+* `LANGUAGE_CODE`
+* `DEFAULT_FROM_EMAIL`
+* `NOTIFICATION_FROM_EMAIL`
+
+You can generate the `SECRET_KEY` with the following python code:
+
+.. code-block:: python
+
+ import string, random
+ chars = string.letters + string.digits + string.punctuation
+ print repr("".join([random.choice(chars) for i in range(0,50)]))
+
+If you wish to enable the XML-RPC interface, you should add the following to
+the file:
+
+.. code-block:: python
+
+ ENABLE_XMLRPC = True
+
+__ https://docs.djangoproject.com/en/1.8/ref/settings/
+
+Final Steps
+~~~~~~~~~~~
+
+Once done, we should be able to check that all requirements are met using the
+`check` command of the `manage.py` executable:
+
+.. code-block:: shell
+
+ $ /opt/patchwork/manage.py check
+
+We should also take this opportunity to both configure the database and static
+files:
+
+.. code-block:: shell
+
+ $ /opt/patchwork/manage.py migrate
+ $ /opt/patchwork/manage.py loaddata \
+ /opt/patchwork/patchwork/fixtures/default_tags.xml
+ $ /opt/patchwork/manage.py loaddata \
+ /opt/patchwork/patchwork/fixtures/default_states.xml
+ $ /opt/patchwork/manage.py collectstatic
+
+.. note::
+
+ The above `default_tags` and `default_states` are just that: defaults. You
+ can modify these to fit your own requirements.
+
+Finally, it may be helpful to start the development server quickly to ensure
+you can see *something*:
+
+.. code-block:: shell
+
+ $ /opt/patchwork/manage.py runserver 0.0.0.0:8080
+
+Browse this instance at `http://[your_server_ip]:8000`. If everything is
+working, kill the development server using `Ctrl`+`C`.
+
+Reverse Proxy and WSGI HTTP Servers
+-----------------------------------
+
+These steps should be run on the application server.
+
+Install Packages
+~~~~~~~~~~~~~~~~
+
+We will use nginx and uWSGI to deploy Patchwork, acting as reverse proxy server
+and WSGI HTTP server respectively. Other options are available, such as
+Apache+mod_wsgi or nginx+Gunicorn. While we don't document these, sample
+configuration files for the former case are provided in `lib/apache2/`.
+
+.. code-block:: shell
+
+ $ sudo apt-get install nginx-full uwsgi uwsgi-plugin-python
+
+Configure nginx and uWSGI
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Configuration files for nginx and uWSGI are provided in the `lib` subdirectory
+of the Patchwork source code. These can be modified as necessary, but for now
+we will simply copy them.
+
+First, let's load the provided configuration for nginx:
+
+.. code-block:: shell
+
+ $ sudo cp /opt/patchwork/lib/nginx/patchwork.conf \
+ /etc/nginx/sites-available/
+
+If you wish to modify this configuration, now is the time to do so. Once done,
+validate and enable your configuration:
+
+.. code-block:: shell
+
+ $ sudo nginx -t
+ $ sudo ln -s /etc/nginx/sites-available/patchwork.conf \
+ /etc/nginx/sites-enabled/patchwork.conf
+
+Now use the provided configuration for uWSGI:
+
+.. code-block:: shell
+
+ $ sudo mkdir -p /etc/uwsgi/sites
+ $ sudo cp /opt/patchwork/lib/uwsgi/patchwork.ini \
+ /etc/uwsgi/sites/patchwork.ini
+
+.. note::
+
+ We created the `/etc/uwsgi` directory above because we're going to run uWSGI
+ in `emperor mode`__. This has benefits for multi-app deployments.
+
+__ https://uwsgi-docs.readthedocs.io/en/latest/Emperor.html
+
+Create systemd Unit File
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+As things stand, uWSGI will need to be started manually every time the system
+boots, in addition to any time it may fail. We can automate this process using
+systemd. To this end a `systemd unit file`__ should be created to start uWSGI
+at boot:
+
+.. code-block:: shell
+
+ $ sudo cat << EOF > /etc/systemd/system/uwsgi.service
+ [Unit]
+ Description=uWSGI Emperor service
+
+ [Service]
+ ExecStartPre=/usr/bin/bash -c 'mkdir -p /run/uwsgi; chown user:nginx /run/uwsgi'
+ ExecStart=/usr/bin/uwsgi --emperor /etc/uwsgi/sites
+ Restart=always
+ KillSignal=SIGQUIT
+ Type=notify
+ NotifyAccess=all
+
+ [Install]
+ WantedBy=multi-user.target
+ EOF
+
+.. note::
+
+ On older version of Ubuntu you may need to tweak these steps to use
+ `upstart`__ instead.
+
+__ https://uwsgi-docs.readthedocs.io/en/latest/Systemd.html
+__ https://uwsgi-docs.readthedocs.io/en/latest/Upstart.html
+
+Final Steps
+~~~~~~~~~~~
+
+Start the uWSGI service we created above:
+
+.. code-block:: shell
+
+ $ sudo systemctl start uwsgi
+ $ sudo systemctl status uwsgi
+
+Next up, restart the nginx service:
+
+.. code-block:: shell
+
+ $ sudo systemctl restart nginx
+ $ sudo systemctl status nginx
+
+Patchwork uses a cron script to clean up expired registrations and send
+notifications of patch changes (for projects with this enabled). Something like
+this in your crontab should work.
+
+::
+
+ # m h dom mon dow command
+ */10 * * * * cd patchwork; ./manage.py cron
+
+.. note::
+
+ The frequency should be the same as the `NOTIFICATION_DELAY_MINUTES`
+ setting, which defaults to 10 minutes.
+
+Finally, browse to the instance using your browser of choice.
+
+You may wish to take this opportunity to setup your projects and configure your
+website address (in the Sites section of the admin console, found at `/admin`).
+
+Django administrative console
+-----------------------------
+
+In order to access the administrative console at `/admin`, you need at least
+one user account to be registered and configured as a super user or staff
+account to access the Django administrative console. This can be achieved by
+doing the following:
+
+.. code-block:: shell
+
+ $ /opt/patchwork/manage.py createsuperuser
+
+Once the administrative console is accessible, you would want to configure your
+different sites and their corresponding domain names, which is required for the
+different emails sent by patchwork (registration, password recovery) as well as
+the sample `pwclientrc` files provided by your project's page.
+
+Incoming Email
+--------------
+
+Patchwork is designed to parse incoming mails which means you need an address
+to receive email at. This is a problem that has been solved for many webapps,
+thus there are many ways to go about this. Some of these ways are discussed
+below.
+
+IMAP/POP3
+~~~~~~~~~
+
+The easiest option for getting mail into Patchwork is to use an existing email
+address in combination with a mail retriever like `getmail`__, which will
+download mails from your inbox and pass them to Patchwork for processing.
+getmail is easy to set up and configure: to begin, you need to install it:
+
+.. code-block:: shell
+
+ $ sudo apt-get install getmail4
+
+Once installed, you should configure it, substituting your own configuration
+details where required below:
+
+.. code-block:: shell
+
+ $ sudo cat << EOF > /etc/getmail/user@example.com/getmailrc
+ [retriever]
+ type = SimpleIMAPSSLRetriever
+ server = imap.example.com
+ port = 993
+ username = XXX
+ password = XXX
+ mailboxes = ALL
+
+ [destination]
+ # we configure Patchwork as a "mail delivery agent", in that it will
+ # handle our mails
+ type = MDA_external
+ path = /opt/patchwork/patchwork/bin/parsemail.sh
+
+ [options]
+ # retrieve only new emails
+ read_all = false
+ # do not add a Delivered-To: header field
+ delivered_to = false
+ # do not add a Received: header field
+ received = false
+ EOF
+
+Validate that this works as expected by starting `getmail`:
+
+.. code-block:: shell
+
+ $ getmail --getmaildir=/etc/getmail/user@example.com --idle INBOX
+
+If everything works as expected, you can create a systemd script to ensure this
+starts on boot:
+
+.. code-block:: shell
+
+ $ sudo cat << EOF > /etc/systemd/system/getmail.service
+ [Unit]
+ Description=Getmail for user@example.com
+
+ [Service]
+ User=pathwork
+ ExecStart=/usr/bin/getmail --getmaildir=/etc/getmail/user@example.com --idle INBOX
+ Restart=always
+
+ [Install]
+ WantedBy=multi-user.target
+ EOF
+
+And start the service:
+
+.. code-block:: shell
+
+ $ sudo systemctl start getmail
+ $ sudo systemctl status getmail
+
+__ http://pyropus.ca/software/getmail/
+
+Mail Transfer Agent (MTA)
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The most flexible option is to configure our own mail transfer agent (MTA) or
+"email server". There are many options, of which `Postfix`__ is one. While we
+don't cover setting up Postfix here (it's complicated and there are many guides
+already available), Patchwork does include a script to take received mails and
+create the relevant entries in Patchwork for you. To use this, you should
+configure your system to forward all emails to a given localpart (the bit
+before the `@`) to this script. Using the `patchwork` localpart (e.g.
+`patchwork@example.com`) you can do this like so:
+
+.. code-block:: shell
+
+ $ sudo cat << EOF > /etc/aliases
+ patchwork: "|/opt/patchwork/patchwork/bin/parsemail.sh"
+ EOF
+
+You should ensure the appropriate user is created in PostgreSQL and that it has
+(minimal) access to the database. Patchwork provides scripts for the latter and
+they can be loaded as seen below:
+
+.. code-block:: shell
+
+ $ sudo -u postgres createuser nobody
+ $ sudo -u postgre psql -f \
+ /opt/patchwork/lib/sql/grant-all.postgres.sql patchwork
+
+.. note::
+
+ This assumes your Postfix process is running as the `nobody` user. If this
+ is not correct (use of `postfix` user is also common), you should change
+ both the username in the `createuser` command above and substitute the
+ username in the `grant-all-postgres.sql` script with the appropriate
+ alternative.
+
+__ http://www.postfix.org/
+
+Use a Email-as-a-Service Provider
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Setting up an email server can be a difficult task and, in the case of
+deployment on PaaS provider, may not even be an option. In this case, there
+are a variety of web services available that offer "Email-as-as-Service".
+These services typically convert received emails into HTTP POST requests to
+your endpoint of choice, allowing you to sidestep configuration issues. We
+don't cover this here, but a simple wrapper script coupled with one of these
+services can be more than to get email into Patchwork.
+
+You can also create such as service yourself using a PaaS provider that
+supports incoming mail and writing a little web app.
+
+(Optional) Configure your VCS to Automatically Update Patches
+-------------------------------------------------------------
+
+The `tools` directory of the Patchwork distribution contains a file named
+`post-receive.hook` which is a sample git hook that can be used to
+automatically update patches to the `Accepted` state when corresponding
+commits are pushed via git.
+
+To install this hook, simply copy it to the `.git/hooks` directory on your
+server, name it `post-receive`, and make it executable.
+
+This sample hook has support to update patches to different states depending
+on which branch is being pushed to. See the `STATE_MAP` setting in that file.
+
+If you are using a system other than git, you can likely write a similar hook
+using `pwclient` to update patch state. If you do write one, please contribute
+it.
diff --git a/docs/deployment/upgrading.rst b/docs/deployment/upgrading.rst
new file mode 100644
index 0000000..298db11
--- /dev/null
+++ b/docs/deployment/upgrading.rst
@@ -0,0 +1,152 @@
+Upgrading
+=========
+
+This document provides some general tips and tricks that one can use when
+upgrading an existing, production installation of Patchwork. If you are
+interested in the specific changes between each release, refer to the
+`UPGRADING` document instead. If this is your first time installing Patchwork,
+refer to the :doc:`installation` instead.
+
+Before You Start
+----------------
+
+Before doing anything, always **backup your data**. This generally means
+backing up your database, but it might also be a good idea to backup your
+environment in case you encounter issues during the upgrade process.
+
+While Patchwork won't explicitly prevent it, it's generally wise to avoid
+upgrades spanning multiple releases in one go. An iterative upgrade approach
+will provide an easier, if slower, upgrade process.
+
+Identify Changed Scripts, Requirements, etc.
+--------------------------------------------
+
+The `CHANGELOG` document provides a comprehensive listing of all
+backwards-incompatible changes that occur between releases of Patchwork.
+Examples of such changes include:
+
+* Moved/removed scripts and files
+
+* Changes to the requirements, e.g. supported Django versions
+
+* Changes to API that may affect, for example, third-party tools
+
+It is important that you understand these changes and ensure any scripts you
+may have, such as systemd scripts, are modified accordingly.
+
+Understand What Requirements Have Changed
+-----------------------------------------
+
+New versions of Patchwork can often require additional or updated version of
+dependencies, e.g. newer versions of Django. It is important that you
+understand these requirements and can fulfil them. This is particularly true
+for users relying on distro-provided packages, who may have to deal with older
+versions of a package or may be missing a package altogether (though we try to
+avoid this). Such changes are usually listed in the `UPGRADING` document, but
+you can also diff the `requirements.txt` files in each release for comparison.
+
+Collect Static Files
+--------------------
+
+New versions of Patchwork generally contain changes to the additional files
+like images, CSS and JavaScript. To do this, run the `collectstatic`
+management commands:
+
+.. code-block:: shell
+
+ $ ./manage.py collectstatic
+
+Upgrade Your Database
+---------------------
+
+Migrations of the database can be tricky. Prior to `v1.0.0`__, database
+migrations were provided by way of manual, SQL migration scripts. After this
+release, Patchwork moved to support `Django migrations`__. If you are
+upgrading from `v1.0.0` or later, it is likely that you can rely entirely on
+the later to bring your database up-to-date. This can be done like so:
+
+.. code-block:: shell
+
+ $ ./manage.py migrate
+
+However, there are a number of scenarios in which you may need to fall back to
+the provided SQL migrations or provide your own:
+
+* You are using Django < 1.6
+
+ Patchwork supports Django 1.6. However, Django Migrations was added in 1.7
+ and is `not available for previous versions`__. As such, you must continue to
+ use manual migrations or upgrade your version of Django. For many of the
+ migrations, this can be done automatically:
+
+ .. code-block:: shell
+
+ $ ./manage.py sqlmigrate patchwork 0003_add_check_model
+
+ However, this only works for schema migrations. For data migrations,
+ however, this will fail. In this cases, these migrations will need to be
+ handwritten.
+
+* You are using Django > 1.6, but upgrading from Patchwork < 1.0.0
+
+ Patchwork only started providing migrations in `v1.0.0`. SQL migrations are
+ provided for versions prior to this and must be applied to get the database
+ to the "initial" state that Django migrations expects.
+
+* You have diverged from upstream Patchwork
+
+ If you have applied custom patches that change the database models, the
+ database in an "inconsistent state" and the provided migrations will likely
+ fail to apply.
+
+Steps to handle the latter two of these are described below.
+
+__ https://github.com/getpatchwork/patchwork/releases/tag/v1.0.0
+__ https://docs.djangoproject.com/en/1.8/topics/migrations/
+__ http://blog.allenap.me/2015/05/south-south-2-and-django-migrations.html
+
+Upgrading a pre-v1.0.0 Patchwork instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The process for this type of upgrade is quite simple: upgrade using manual SQL
+upgrades until better options become available. As such, you should apply all
+unapplied SQL migrations that are not duplicated by Django migrations. Once
+such duplication occurs, rely on the Django migrations only and continue to do
+so going forward.
+
+Upgrading a "diverged" Patchwork instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This type of upgrade is a little trickier. There are two options you can take:
+
+1. Bring your Patchwork instance back in sync with upstream
+
+2. Provide your own migrations
+
+The former option is particularly suitable if you decide to upstream your
+change or decide it's not valuable enough to retain. This will require either
+reworking any migrations that exist prior to your feature being upstreamed, or
+deleting any added database fields and tables, respectively. In both cases,
+manually, hand-written SQL migrations will be required to get the databse into
+a consistent state (remember: **backup**!). Once this is done, you can resume
+using the upstream-provided migrations, ensuring any Django migrations that you
+may have skipped are not applied again:
+
+.. code-block:: shell
+
+ $ ./manage.py migrate 000x-abc --fake # when 000x-abc is last "skippable"
+
+It's worth adding that with the databases now back in sync it should be
+possible to return to using upstream code rather than maintaining a fork.
+
+The latter option is best chosen if you wish to retain the aforementioned fork.
+How you do this depends on the extensiveness of your changes, but getting the
+latest version of Patchwork, deleting the provided migrations, applying any
+patches you may have and regenerating the migrations seems like the best
+option.
+
+.. note::
+
+ To prevent the latter case above from occurring, we'd ask that you submit
+ any patches you may have to the upstream Patchwork so that the wider
+ community can benefit from this new functionality.