From 1cb38956875fecd42e1eaaf4c0c4064c577c7aab Mon Sep 17 00:00:00 2001 From: Stephen Finucane Date: Wed, 22 Mar 2017 23:17:25 +0000 Subject: docs: Remove old Markdown files Everything is now ported to rST/Sphinx meaning we can remove the Markdown versions. Signed-off-by: Stephen Finucane --- docs/development/contributing.md | 59 ------ docs/development/installation.md | 379 --------------------------------------- docs/development/releasing.md | 49 ----- docs/development/xmlrpc.md | 54 ------ 4 files changed, 541 deletions(-) delete mode 100644 docs/development/contributing.md delete mode 100644 docs/development/installation.md delete mode 100644 docs/development/releasing.md delete mode 100644 docs/development/xmlrpc.md (limited to 'docs/development') diff --git a/docs/development/contributing.md b/docs/development/contributing.md deleted file mode 100644 index aefd8a6..0000000 --- a/docs/development/contributing.md +++ /dev/null @@ -1,59 +0,0 @@ -# Tips and Tricks - -## Coding Standards - -**Follow PEP8**. All code is currently PEP8 compliant and it should stay this -way. - -Changes that fix semantic issues will be generally be happily received, but -please keep such changes separate from functional changes. - -`pep8` targets are provided via tox. Refer to the [testing section](#testing) -below for more information on usage of this tool. - -## Testing - -Patchwork includes a [tox][ref-tox] script to automate testing. This requires -a functional database and some Python requirements like `tox`. Please refer -to the [development guide][doc-development] for information on how to configure -these. - -Assuming these requirements are met, actually testing Patchwork is quite easy -to do. To start, you can show the default targets like so: - - $ tox --list - -You'll see that this includes a number of targets to run unit tests against -the different versions of Django supported, along with some other targets -related to code coverage and code quality. To run one of these, use the `-e` -parameter: - - $ tox -e py27-django18 - -In the case of the unit tests targets, you can also run specific tests by -passing the fully qualified test name as an additional argument to this -command: - - $ tox -e py27-django18 patchwork.tests.SubjectCleanUpTest - -Because Patchwork support multiple versions of Django, it's very important -that you test against all supported versions. When run without argument, tox -will do this: - - $ tox - -## Submitting Changes - -All patches should be sent to the [mailing list][pw-ml]. When doing so, please -abide by the [QEMU guidelines][ref-qemu-contrib] on contributing or submitting -patches. This covers both the initial submission and any follow up to the -patches. In particular, please ensure: - -* [All tests pass](#testing) -* Documentation has been updated with new requirements, new script names etc. -* The `CHANGES` file has been updated with any added or removed features - -[doc-development]: installation.md -[pw-ml]: https://ozlabs.org/mailman/listinfo/patchwork -[ref-qemu-contrib]: http://wiki.qemu.org/Contribute/SubmitAPatch -[ref-tox]: https://tox.readthedocs.io/en/latest/ diff --git a/docs/development/installation.md b/docs/development/installation.md deleted file mode 100644 index 90e8a11..0000000 --- a/docs/development/installation.md +++ /dev/null @@ -1,379 +0,0 @@ -# Development - -This document describes the necessary steps to configure Patchwork in a -development environment. If you are interested in deploying Patchwork in a -production environment, please refer to [the deployment guide][doc-deployment] -instead. - -To begin, you should clone Patchwork: - - $ git clone git://github.com/getpatchwork/patchwork.git - -## Docker-Based Installation - -Patchwork provides a Docker-based environment for quick configuration of a -development environment. This is the preferred installation method. To -configure Patchwork using Docker: - -1. Install [**Docker**][ref-docker] and [**docker-compose**][ref-compose]. - -2. Build the images. This will download over 200MB from the internet: - - $ docker-compose build - -3. Run `docker-compose up`: - - $ docker-compose up - - This will be visible at http://localhost:8000/. - -To run a shell within this environment, run: - - $ docker-compose run --rm web --shell - -To run django-manage commands, such as `createsuperuser` or `migrate`, run: - - $ docker-compose run --rm web python manage.py createsuperuser - -To run unit tests, excluding Selenium UI interaction tests, using only the -package versions installed during container initialization, run: - - $ docker-compose run --rm web --quick-test - -To run the same against all supported versions of Django (via tox), run: - - $ docker-compose run --rm web --quick-tox - -To run specific tox targets or tests, pass arguments to the above: - - $ docker-compose run --rm web --quick-tox -e py27-django17 \ - patchwork.tests.test_bundles - -To run all tests, including Selenium UI interaction tests, using only the -package versions installed container initialization, run: - - $ docker-compose run --rm web --test - -To run the same against all supported versions of Django (via tox), run: - - $ docker-compose run --rm web --tox - -To run all tests, including Selenium UI interaction tests in non-headless mode, -run: - - $ docker run -it --rm -v (pwd):/home/patchwork/patchwork/ \ - --link patchwork_db_1:db -p 8000:8000 \ - -v /tmp/.X11-unix:/tmp/.X11-unix \ - -e PW_TEST_DB_HOST=db -e DISPLAY patchwork_web bash - -To reset the database before any of these commands, add `--reset` to the -command line after `web` and before any other arguments. - -Any local edits to the project files made locally are immediately visible to -the Docker container, and so should be picked up by the Django auto-reloader. - -For more information on Docker itself, please refer to the [Docker][ref-docker] -and [docker-compose][ref-compose] documentation. - -**NOTE:** If using SELinux, you will need to create a custom SELinux rule to -allow the Docker process to access your working directory. Run: - - $ chcon -RT svirt_sandbox_file_t $PATCHWORK_DIR - -where `$PATCHWORK_DIR` is the absolute patch to the `patchwork` folder created -when you cloned the repo. For more information, see `man docker run`. - -**NOTE:** If you see an error like the below: - - ERROR: Couldn't connect to the Docker daemon at http+docker://localunixsocket - is it running? - -ensure you have correctly installed Docker, added your user to the `docker` -group, and started the daemon, per the [Docker documentation][ref-docker]. - -**NOTE:** If you see an error like the below: - - py.error.EACCES: [Permission denied]: open('/home/patchwork/patchwork/.tox/py27-django18/.tox-config1', 'w') - -your host user account is likely using a different UID to the one hardcoded in -the Dockerfile. You can confirm this like so: - - $ echo $UID - 1234 - -If this is something other than `1000`, you must must modify the `Dockerfile` -found in `tools/docker` to use your UID and then rebuild: - - $ sed -i "/ARG UID=/c\ARG UID=$(echo $UID)" tools/docker/Dockerfile - $ docker-compose build web - -This change must be retained in the event that you rebuild the container. You -can "hide" the change from Git like so: - - $ git update-index --assume-unchanged tools/docker/Dockerfile - $ git update-index --skip-worktree tools/docker/Dockerfile - -This should be resolved in a future release when we support docker-compose 2.1 -syntax in `docker-compose.yml`. - -## Vagrant-Based Installation - -Patchwork provides a Vagrant-based environment as an alternative to Docker. -Like Docker, Vagrant can be used to quickly configure Patchwork in a -development environment. To configure Patchwork using Vagrant: - -1. Install [**Vagrant**][ref-vagrant] - -2. Run `vagrant up` from the project directory: - - $ cd patchwork - $ vagrant up - -Once stacked, follow the on-screen instructions. For more information on -Vagrant itself, please refer to the [Vagrant documentation][ref-vagrant]. - -## Manual Installation - -Manual installation can be used where use of Docker or Vagrant is not possible or -desired. - -### Install Required Packages - -There are a number of different requirements for developing Patchwork: - -* Python and libraries -* A supported database (RDBMS) - -These are detailed below. - -#### Python Requirements - -To develop Python-based software you first need Python. Patchwork supports -both Python 2.7 and Python 3.3+. One of these will be installed by default -on many installations, though they can also be installed manually using the -`python` or `python3` packages. - -It's a good idea to use [virtual environments][ref-venv] to develop Python -software. Virtual environments are "instances" of your system Python without -any of the additional Python packages installed. They are useful to develop and -possibly deploy Patchwork against a "well known" set of dependencies, but they -can also be used to test Patchwork against several versions of Django. - -If you do not have `virtualenv` installed then you should install it now. This -can be installed using the `python-virtualenv` or `python3-virtualenv` -packages. Alternatively you can install these using `pip`. - -It is also helpful to install [`tox`][ref-tox] which is used for running tests -in Patchwork. This can be installed using the `python-tox` or `python3-tox` -packages, or via `pip`. - -#### Database Requirements - -If not already installed, you may need to install an RDBMS. You can use either -MariaDB/MySQL or PostgreSQL for this purpose. You should also install the -development headers, known as `libmysqlclient-dev` or `libpq-dev` respectively -on Debian-based Debian-based distros like Ubuntu and `mysql-devel` or -`postgresql-devel` on RHEL-based distros. - -**NOTE:** While Django provides support for -[multiple database backends][ref-django-db], Patchwork itself is only tested -against MySQL/MariaDB and PostgreSQL. Should you wish to use a different -backend, ensure you validate this first (and perhaps -[upstream][doc-contributing] any changes you may find necessary). - -**NOTE:** You may be tempted to use SQLite to develop Patchwork. We'd advise -against doing this. SQLite supports a subset of the functionality of "full" -RDBMS like MySQL: for example, case-sensitive matching of Unicode -[is not supported][ref-sqlite-utf8]. You will find some tests provided by -Patchwork fail and some patches you develop may fail in production due to these -differences. - -#### Example Installation - -An example for installing all these packages and the MySQL RDBMS on Ubuntu -15.04 is given below: - - $ sudo apt-get install python python-pip python-dev python-virtualenv \ - python-tox mysql-server libmysqlclient-dev - -If you have an existing MariaDB/MySQL installation and have installed `pip` -already/are using [Python 3.4+][ref-py34-pip] then you can install all -packages using `pip`: - - $ sudo pip install virtualenv tox - -If you wish to use Python 3 then simply replace 'python' with 'python3' in -the above command. - -### Configure Virtual Environment - -**NOTE:** If you are interested in simply [testing Patchwork][doc-testing], -many of the below steps are not required. tox will automatically install -dependencies and use virtual environments when testing. - -Once these requirements are installed, you should create and activate a new -virtual environment. This can be done like so: - - $ virtualenv .venv - $ source .venv/bin/activate - (.venv)$ - -**NOTE:** It you installed a Python 3.x-based virtual environment package, -adjust the executable indicated above as necessary, e.g. `virtualenv-3.4`. - -Now install the packages. Patchwork provides three requirements files. - -* `requirements-dev.txt`: Packages required to configure a development - environment -* `requirements-prod.txt`: Packages required for deploying Patchwork in - production -* `requirements-test.txt`: Packages required to run tests - -We're going to install the first of these, which can be done like so: - - (.venv)$ cd patchwork - (.venv)$ pip install -r requirements-dev.txt - -**NOTE:** Once configured this does not need to be done again *unless* the -requirements change, e.g. Patchwork requires an updated version of Django. - -### Initialize the Database - -One installed, the database must be configured. We will assume you have root -access to the database for these steps. - -To begin, export your database credentials as follows: - - (.venv)$ db_user=root - (.venv)$ db_pass=password - -Now, create the database. If this is your first time configuring the database, -you must create a `patchwork` user (or similar) along with the database -instance itself. The commands below will do this, dropping existing databases -if necessary: - - (.venv)$ mysql -u$db_user -p$db_pass << EOF - DROP DATABASE IF EXISTS patchwork; - CREATE DATABASE patchwork CHARACTER SET utf8; - GRANT ALL PRIVILEGES ON patchwork.* TO 'patchwork'@'localhost' - IDENTIFIED BY 'password'; - EOF - -**NOTE:** The `patchwork` username and `password` password are the defaults -expected by the provided `dev` settings files. If using something different, -please export the `PW_TEST_DB_USER` and `PW_TEST_DB_PASS` variables described -in the [Environment Variables](#environment-variables) section below. -Alternatively, you can create your own settings file with these variables -hardcoded and change the value of `DJANGO_SETTINGS_MODULE` as described below. - -### Load Initial Data - -Before continuing, we need to tell Django where it can find our configuration. -Patchwork provides a default development `settings.py` file for this purpose. -To use this, export the `DJANGO_SETTINGS_MODULE` environment variable as -described below: - - (.venv)$ export DJANGO_SETTINGS_MODULE=patchwork.settings.dev - -Alternatively you can provide your own `settings.py` file and provide the path -to that instead. - -Once done, we need to create the tables in the database. This can be done using -the `migrate` command of the `manage.py` executable: - - (.venv)$ ./manage.py migrate - -Next, you should load the initial fixtures into Patchwork. These initial -fixtures provide. - -* `default_tags.xml`: The tags that Patchwork will extract from mails. - Examples: `Acked-By`, `Reviewed-By` -* `default_states.xml`: The states that a patch can be in. Examples: - `Accepted`, `Rejected` -* `default_projects.xml`: A default project that you can then upload patches - for - -These can be loaded using the `loaddata` command: - - (.venv)$ ./manage.py loaddata default_tags default_states default_projects - -You should also take the opportunity to create a "superuser". You can do this -using the aptly-named `createsuperuser` command: - - (.venv)$ ./manage.py createsuperuser - -## Import Mailing List Archives - -Once this is done, it's beneficial to load some real emails into the system. -This can be done manually, however it's generally much easier to download -an archive from a Mailman instance and load these using the `parsearchive` -command. You can do this like so: - - (.venv)$ mm_user= - (.venv)$ mm_pass= - (.venv)$ mm_host=https://lists.ozlabs.org - (.venv)$ mm_url=$mm_host/private/patchwork.mbox/patchwork.mbox - (.venv)$ curl -F username=$mm_user -F password=$mm_pass -k -O $mm_url - -where `mm_user` and `mm_pass` are the username and password you have registered -with on the Mailman instance found at `mm_host`. - -**NOTE:** We provide instructions for downloading archives from the Patchwork -mailing list, but almost any instance of Mailman will allow downloading of -archives as seen above; simply change the `pw_url` variable defined. You can -find more informations about this [here][ref-mman-bulk]. - -Load these archives into Patchwork. Depending on the size of the downloaded -archives this may take some time: - - (.venv)$ ./manage.py parsearchive --list-id=patchwork.ozlabs.org \ - patchwork.mbox - -Finally, run the server and browse to the IP address of your board using your -browser of choice: - - (.venv)$ ./manage.py runserver 0.0.0.0:8000 - -Once finished, you can kill the server (`Ctrl` + `C`) and exit the virtual -environment: - - (.venv)$ deactivate - $ - -Should you wish to re-enter this environment, simply source the `activate` -script again. - -## Django Debug Toolbar - -Patchwork installs and enables the 'Django Debug Toolbar' by default. However, -by default this is only displayed if you are developing on localhost. If -developing on a different machine, you should configure an SSH tunnel such -that, for example, `localhost:8000` points to `[DEV_MACHINE_IP]:8000`. - -## Environment Variables - -The following environment variables are available to configure settings when -using the provided `dev` settings file. - -
-
PW_TEST_DB_NAME = 'patchwork'
-
Name of the database
-
PW_TEST_DB_USER = 'patchwork'
-
Username to access the database with
-
PW_TEST_DB_PASS = 'password'
-
Password to access the database with
-
PW_TEST_DB_TYPE = 'mysql'
-
Type of database to use. Options: 'mysql', 'postgres'
-
- -[doc-contributing]: ../development/contributing.md -[doc-deployment]: ../deployment/installation.md -[doc-testing]: testing.md -[ref-django-db]: https://docs.djangoproject.com/en/1.8/ref/databases/ -[ref-mman-bulk]: http://blog.behnel.de/posts/indexp118.html -[ref-py34-pip]: http://legacy.python.org/dev/peps/pep-0453/ -[ref-sqlite-utf8]: https://www.sqlite.org/faq.html#q18 -[ref-tox]: https://tox.readthedocs.io/en/latest/ -[ref-compose]: https://docs.docker.com/compose/install/ -[ref-docker]: https://docs.docker.com/engine/installation/linux/ -[ref-vagrant]: https://www.vagrantup.com/docs/getting-started/ -[ref-venv]: https://virtualenv.readthedocs.io/en/latest/ diff --git a/docs/development/releasing.md b/docs/development/releasing.md deleted file mode 100644 index e09b524..0000000 --- a/docs/development/releasing.md +++ /dev/null @@ -1,49 +0,0 @@ -# Release Process - -## Versioning - -Since version 1.0, Patchwork has implemented a version of -[Semantic Versioning][ref-semver]. To summarise, releases take the format -**MAJOR.MINOR.PATCH** (or just **MAJOR.MINOR**). We increment: - -1. **MAJOR** version when we make major UI changes or functionality updates -2. **MINOR** version when we make minor UI changes or functionality updates -3. **PATCH** version when we make make bug fixes, dependency updates etc. - -In Git, each release will have a tag indicating the version number. In -addition, each release series has it's own branch called `stable/MAJOR.MINOR` -to allow backporting of bugfixes or security updates to older versions. - -## Release Cycle - -There is no cadence for releases: they are made available as necessary. - -## Supported Versions - -Typically all development should occur on `master`. While we will backport -bugfixes and security updates, we will not backport any new features. This -is to ensure stability for users of these versions of Patchwork. - -## Release Checklist - -* Documentation has been updated with latest release version -* Documentation references latest supported version of Django - -## Backporting - -We will occasionally backport bugfixes and security updates. When backporting -a patch, said patch should first be merged into `master`. Once merged, you can -backport by cherry-picking commits, using the `-x` flag for posterity: - - $ git cherry-pick -x - -There may be some conflicts; resolve these, uncommenting the `Conflicts` line -when commiting: - - Conflicts - patchwork/bin/pwclient - -When enough patches have been backported, you should release a new `PATCH` -release. - -[ref-semver]: http://semver.org/ diff --git a/docs/development/xmlrpc.md b/docs/development/xmlrpc.md deleted file mode 100644 index 1ebc540..0000000 --- a/docs/development/xmlrpc.md +++ /dev/null @@ -1,54 +0,0 @@ -# The XML-RPC API - -**NOTE:** This guide covers development information for the Patchwork XML-RPC -API. For information on using the REST API, refer to the [REST API -documentation][doc-rest]. For information on general usage of the XML-RPC API, -refer to the [user documentation][doc-usage]. - -Patchwork provides an XML-RPC API. This API can be used to be used to retrieve -and modify information about patches, projects and more. - -**NOTE:** The XML-RPC API can be enabled/disabled by the administrator: it may -not be available in every instance. - -## Patchwork API Documentation - -Patchwork provides automatically generated documentation for the XML-RPC API. -You can find this at the following URL: - - http://patchwork.example.com/xmlrpc/ - -Where `patchwork.example.com` refers to the URL of your Patchwork instance. - -**NOTE:** Automatic documentation generation for the Patchwork API was -introduced in Patchwork v1.1. Prior versions of Patchwork do not offer this -functionality. - -## Interacting with the API - -**NOTE:** The Patchwork XML-RPC API provides a number of "methods". Some -methods require authentication (via HTTP Basic Auth) while others do not. -Authentication uses your Patchwork account and the on-server documentation will -indicate where it is necessary. We will only cover the unauthenticated method -here for brevity - please consult the [`xmlrpclib`][ref-xmlrpclib] -documentation for more detailed examples: - -To interact with the Patchwork XML-RPC API, a XML-RPC library should be used. -Python provides such a library - [`xmlrpclib`][ref-xmlrpclib] - in its standard -library. For example, to get the version of the XML-RPC API for a Patchwork -instance hosted at `patchwork.example.com`, run: - - $ python - >>> import xmlrpclib # or 'xmlrpc.client' for Python 3 - >>> rpc = xmlrpclib.ServerProxy('http://patchwork.example.com/xmlrpc/') - >>> rpc.pw_rpc_version() - 1.1 - -Once connected, the `rpc` object will be populated with a list of available -functions (or procedures, in RPC terminology). In the above example, we used -the `pw_rpc_version` method, however, it should be possible to use all the -methods listed in the [server documentation](#patchwork-api-documentation). - -[doc-rest]: ../usage/rest.md -[doc-usage]: ../usage/xmlrpc.md -[ref-xmlrpclib]: https://docs.python.org/2/library/xmlrpclib.html -- cgit v1.2.3