Metadata-Version: 1.1
Name: urllib3
Version: 1.5
Summary: HTTP library with thread-safe connection pooling, file post, and more.
Home-page: http://urllib3.readthedocs.org/
Author: Andrey Petrov
Author-email: andrey.petrov@shazow.net
License: MIT
Description: Highlights
==========
- Re-use the same socket connection for multiple requests
(``HTTPConnectionPool`` and ``HTTPSConnectionPool``)
(with optional client-side certificate verification).
- File posting (``encode_multipart_formdata``).
- Built-in redirection and retries (optional).
- Supports gzip and deflate decoding.
- Thread-safe and sanity-safe.
- Works with AppEngine, gevent, and eventlib.
- Tested on Python 2.6+ and Python 3.2+, 100% unit test coverage.
- Small and easy to understand codebase perfect for extending and building upon.
For a more comprehensive solution, have a look at
`Requests `_ which is also powered by urllib3.
What's wrong with urllib and urllib2?
=====================================
There are two critical features missing from the Python standard library:
Connection re-using/pooling and file posting. It's not terribly hard to
implement these yourself, but it's much easier to use a module that already
did the work for you.
The Python standard libraries ``urllib`` and ``urllib2`` have little to do
with each other. They were designed to be independent and standalone, each
solving a different scope of problems, and ``urllib3`` follows in a similar
vein.
Why do I want to reuse connections?
===================================
Performance. When you normally do a urllib call, a separate socket
connection is created with each request. By reusing existing sockets
(supported since HTTP 1.1), the requests will take up less resources on the
server's end, and also provide a faster response time at the client's end.
With some simple benchmarks (see `test/benchmark.py
`_
), downloading 15 URLs from google.com is about twice as fast when using
HTTPConnectionPool (which uses 1 connection) than using plain urllib (which
uses 15 connections).
This library is perfect for:
- Talking to an API
- Crawling a website
- Any situation where being able to post files, handle redirection, and
retrying is useful. It's relatively lightweight, so it can be used for
anything!
Examples
========
Go to `urllib3.readthedocs.org `_
for more nice syntax-highlighted examples.
But, long story short::
import urllib3
http = urllib3.PoolManager()
r = http.request('GET', 'http://google.com/')
print r.status, r.data
The ``PoolManager`` will take care of reusing connections for you whenever
you request the same host. For more fine-grained control of your connection
pools, you should look at
`ConnectionPool `_.
Run the tests
=============
We use some external dependencies to run the urllib3 test suite. Easiest way to
run the tests is thusly from the urllib3 source root: ::
$ pip install -r test-requirements.txt
$ nosetests
.....................................................
Success! You could also ``pip install coverage`` to get code coverage reporting.
Contributing
============
#. `Check for open issues `_ or open
a fresh issue to start a discussion around a feature idea or a bug. There is
a *Contributor Friendly* tag for issues that should be ideal for people who
are not very familiar with the codebase yet.
#. Fork the `urllib3 repository on Github `_
to start making your changes.
#. Write a test which shows that the bug was fixed or that the feature works
as expected.
#. Send a pull request and bug the maintainer until it gets merged and published.
:) Make sure to add yourself to ``CONTRIBUTORS.txt``.
Changes
=======
1.5 (2012-08-02)
++++++++++++++++
* Added ``urllib3.add_stderr_logger()`` for quickly enabling STDERR debug
logging in urllib3.
* Native full URL parsing (including auth, path, query, fragment) available in
``urllib3.util.parse_url(url)``.
* Built-in redirect will switch method to 'GET' if status code is 303.
(Issue #11)
* ``urllib3.PoolManager`` strips the scheme and host before sending the request
uri. (Issue #8)
* New ``urllib3.exceptions.DecodeError`` exception for when automatic decoding,
based on the Content-Type header, fails.
* Fixed bug with pool depletion and leaking connections (Issue #76). Added
explicit connection closing on pool eviction. Added
``urllib3.PoolManager.clear()``.
* 99% -> 100% unit test coverage.
1.4 (2012-06-16)
++++++++++++++++
* Minor AppEngine-related fixes.
* Switched from ``mimetools.choose_boundary`` to ``uuid.uuid4()``.
* Improved url parsing. (Issue #73)
* IPv6 url support. (Issue #72)
1.3 (2012-03-25)
++++++++++++++++
* Removed pre-1.0 deprecated API.
* Refactored helpers into a ``urllib3.util`` submodule.
* Fixed multipart encoding to support list-of-tuples for keys with multiple
values. (Issue #48)
* Fixed multiple Set-Cookie headers in response not getting merged properly in
Python 3. (Issue #53)
* AppEngine support with Py27. (Issue #61)
* Minor ``encode_multipart_formdata`` fixes related to Python 3 strings vs
bytes.
1.2.2 (2012-02-06)
++++++++++++++++++
* Fixed packaging bug of not shipping ``test-requirements.txt``. (Issue #47)
1.2.1 (2012-02-05)
++++++++++++++++++
* Fixed another bug related to when ``ssl`` module is not available. (Issue #41)
* Location parsing errors now raise ``urllib3.exceptions.LocationParseError``
which inherits from ``ValueError``.
1.2 (2012-01-29)
++++++++++++++++
* Added Python 3 support (tested on 3.2.2)
* Dropped Python 2.5 support (tested on 2.6.7, 2.7.2)
* Use ``select.poll`` instead of ``select.select`` for platforms that support
it.
* Use ``Queue.LifoQueue`` instead of ``Queue.Queue`` for more aggressive
connection reusing. Configurable by overriding ``ConnectionPool.QueueCls``.
* Fixed ``ImportError`` during install when ``ssl`` module is not available.
(Issue #41)
* Fixed ``PoolManager`` redirects between schemes (such as HTTP -> HTTPS) not
completing properly. (Issue #28, uncovered by Issue #10 in v1.1)
* Ported ``dummyserver`` to use ``tornado`` instead of ``webob`` +
``eventlet``. Removed extraneous unsupported dummyserver testing backends.
Added socket-level tests.
* More tests. Achievement Unlocked: 99% Coverage.
1.1 (2012-01-07)
++++++++++++++++
* Refactored ``dummyserver`` to its own root namespace module (used for
testing).
* Added hostname verification for ``VerifiedHTTPSConnection`` by vendoring in
Py32's ``ssl_match_hostname``. (Issue #25)
* Fixed cross-host HTTP redirects when using ``PoolManager``. (Issue #10)
* Fixed ``decode_content`` being ignored when set through ``urlopen``. (Issue
#27)
* Fixed timeout-related bugs. (Issues #17, #23)
1.0.2 (2011-11-04)
++++++++++++++++++
* Fixed typo in ``VerifiedHTTPSConnection`` which would only present as a bug if
you're using the object manually. (Thanks pyos)
* Made RecentlyUsedContainer (and consequently PoolManager) more thread-safe by
wrapping the access log in a mutex. (Thanks @christer)
* Made RecentlyUsedContainer more dict-like (corrected ``__delitem__`` and
``__getitem__`` behaviour), with tests. Shouldn't affect core urllib3 code.
1.0.1 (2011-10-10)
++++++++++++++++++
* Fixed a bug where the same connection would get returned into the pool twice,
causing extraneous "HttpConnectionPool is full" log warnings.
1.0 (2011-10-08)
++++++++++++++++
* Added ``PoolManager`` with LRU expiration of connections (tested and
documented).
* Added ``ProxyManager`` (needs tests, docs, and confirmation that it works
with HTTPS proxies).
* Added optional partial-read support for responses when
``preload_content=False``. You can now make requests and just read the headers
without loading the content.
* Made response decoding optional (default on, same as before).
* Added optional explicit boundary string for ``encode_multipart_formdata``.
* Convenience request methods are now inherited from ``RequestMethods``. Old
helpers like ``get_url`` and ``post_url`` should be abandoned in favour of
the new ``request(method, url, ...)``.
* Refactored code to be even more decoupled, reusable, and extendable.
* License header added to ``.py`` files.
* Embiggened the documentation: Lots of Sphinx-friendly docstrings in the code
and docs in ``docs/`` and on urllib3.readthedocs.org.
* Embettered all the things!
* Started writing this file.
0.4.1 (2011-07-17)
++++++++++++++++++
* Minor bug fixes, code cleanup.
0.4 (2011-03-01)
++++++++++++++++
* Better unicode support.
* Added ``VerifiedHTTPSConnection``.
* Added ``NTLMConnectionPool`` in contrib.
* Minor improvements.
0.3.1 (2010-07-13)
++++++++++++++++++
* Added ``assert_host_name`` optional parameter. Now compatible with proxies.
0.3 (2009-12-10)
++++++++++++++++
* Added HTTPS support.
* Minor bug fixes.
* Refactored, broken backwards compatibility with 0.2.
* API to be treated as stable from this version forward.
0.2 (2008-11-17)
++++++++++++++++
* Added unit tests.
* Bug fixes.
0.1 (2008-11-16)
++++++++++++++++
* First release.
Keywords: urllib httplib threadsafe filepost http https ssl pooling
Platform: UNKNOWN
Classifier: Environment :: Web Environment
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Software Development :: Libraries