Compare commits
No commits in common. "master" and "drop-title-version-api" have entirely different histories.
master
...
drop-title
236
.github/CONTRIBUTING.md
vendored
236
.github/CONTRIBUTING.md
vendored
@ -1,236 +0,0 @@
|
||||
# Contributing
|
||||
|
||||
Thank you for being interested in contributing to HTTPX.
|
||||
There are many ways you can contribute to the project:
|
||||
|
||||
- Try HTTPX and [report bugs/issues you find](https://github.com/encode/httpx/issues/new)
|
||||
- [Implement new features](https://github.com/encode/httpx/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)
|
||||
- [Review Pull Requests of others](https://github.com/encode/httpx/pulls)
|
||||
- Write documentation
|
||||
- Participate in discussions
|
||||
|
||||
## Reporting Bugs or Other Issues
|
||||
|
||||
Found something that HTTPX should support?
|
||||
Stumbled upon some unexpected behaviour?
|
||||
|
||||
Contributions should generally start out with [a discussion](https://github.com/encode/httpx/discussions).
|
||||
Possible bugs may be raised as a "Potential Issue" discussion, feature requests may
|
||||
be raised as an "Ideas" discussion. We can then determine if the discussion needs
|
||||
to be escalated into an "Issue" or not, or if we'd consider a pull request.
|
||||
|
||||
Try to be more descriptive as you can and in case of a bug report,
|
||||
provide as much information as possible like:
|
||||
|
||||
- OS platform
|
||||
- Python version
|
||||
- Installed dependencies and versions (`python -m pip freeze`)
|
||||
- Code snippet
|
||||
- Error traceback
|
||||
|
||||
You should always try to reduce any examples to the *simplest possible case*
|
||||
that demonstrates the issue.
|
||||
|
||||
Some possibly useful tips for narrowing down potential issues...
|
||||
|
||||
- Does the issue exist on HTTP/1.1, or HTTP/2, or both?
|
||||
- Does the issue exist with `Client`, `AsyncClient`, or both?
|
||||
- When using `AsyncClient` does the issue exist when using `asyncio` or `trio`, or both?
|
||||
|
||||
## Development
|
||||
|
||||
To start developing HTTPX create a **fork** of the
|
||||
[HTTPX repository](https://github.com/encode/httpx) on GitHub.
|
||||
|
||||
Then clone your fork with the following command replacing `YOUR-USERNAME` with
|
||||
your GitHub username:
|
||||
|
||||
```shell
|
||||
$ git clone https://github.com/YOUR-USERNAME/httpx
|
||||
```
|
||||
|
||||
You can now install the project and its dependencies using:
|
||||
|
||||
```shell
|
||||
$ cd httpx
|
||||
$ scripts/install
|
||||
```
|
||||
|
||||
## Testing and Linting
|
||||
|
||||
We use custom shell scripts to automate testing, linting,
|
||||
and documentation building workflow.
|
||||
|
||||
To run the tests, use:
|
||||
|
||||
```shell
|
||||
$ scripts/test
|
||||
```
|
||||
|
||||
!!! warning
|
||||
The test suite spawns testing servers on ports **8000** and **8001**.
|
||||
Make sure these are not in use, so the tests can run properly.
|
||||
|
||||
You can run a single test script like this:
|
||||
|
||||
```shell
|
||||
$ scripts/test -- tests/test_multipart.py
|
||||
```
|
||||
|
||||
To run the code auto-formatting:
|
||||
|
||||
```shell
|
||||
$ scripts/lint
|
||||
```
|
||||
|
||||
Lastly, to run code checks separately (they are also run as part of `scripts/test`), run:
|
||||
|
||||
```shell
|
||||
$ scripts/check
|
||||
```
|
||||
|
||||
## Documenting
|
||||
|
||||
Documentation pages are located under the `docs/` folder.
|
||||
|
||||
To run the documentation site locally (useful for previewing changes), use:
|
||||
|
||||
```shell
|
||||
$ scripts/docs
|
||||
```
|
||||
|
||||
## Resolving Build / CI Failures
|
||||
|
||||
Once you've submitted your pull request, the test suite will automatically run, and the results will show up in GitHub.
|
||||
If the test suite fails, you'll want to click through to the "Details" link, and try to identify why the test suite failed.
|
||||
|
||||
<p align="center" style="margin: 0 0 10px">
|
||||
<img src="https://raw.githubusercontent.com/encode/httpx/master/docs/img/gh-actions-fail.png" alt='Failing PR commit status'>
|
||||
</p>
|
||||
|
||||
Here are some common ways the test suite can fail:
|
||||
|
||||
### Check Job Failed
|
||||
|
||||
<p align="center" style="margin: 0 0 10px">
|
||||
<img src="https://raw.githubusercontent.com/encode/httpx/master/docs/img/gh-actions-fail-check.png" alt='Failing GitHub action lint job'>
|
||||
</p>
|
||||
|
||||
This job failing means there is either a code formatting issue or type-annotation issue.
|
||||
You can look at the job output to figure out why it's failed or within a shell run:
|
||||
|
||||
```shell
|
||||
$ scripts/check
|
||||
```
|
||||
|
||||
It may be worth it to run `$ scripts/lint` to attempt auto-formatting the code
|
||||
and if that job succeeds commit the changes.
|
||||
|
||||
### Docs Job Failed
|
||||
|
||||
This job failing means the documentation failed to build. This can happen for
|
||||
a variety of reasons like invalid markdown or missing configuration within `mkdocs.yml`.
|
||||
|
||||
### Python 3.X Job Failed
|
||||
|
||||
<p align="center" style="margin: 0 0 10px">
|
||||
<img src="https://raw.githubusercontent.com/encode/httpx/master/docs/img/gh-actions-fail-test.png" alt='Failing GitHub action test job'>
|
||||
</p>
|
||||
|
||||
This job failing means the unit tests failed or not all code paths are covered by unit tests.
|
||||
|
||||
If tests are failing you will see this message under the coverage report:
|
||||
|
||||
`=== 1 failed, 435 passed, 1 skipped, 1 xfailed in 11.09s ===`
|
||||
|
||||
If tests succeed but coverage doesn't reach our current threshold, you will see this
|
||||
message under the coverage report:
|
||||
|
||||
`FAIL Required test coverage of 100% not reached. Total coverage: 99.00%`
|
||||
|
||||
## Releasing
|
||||
|
||||
*This section is targeted at HTTPX maintainers.*
|
||||
|
||||
Before releasing a new version, create a pull request that includes:
|
||||
|
||||
- **An update to the changelog**:
|
||||
- We follow the format from [keepachangelog](https://keepachangelog.com/en/1.0.0/).
|
||||
- [Compare](https://github.com/encode/httpx/compare/) `master` with the tag of the latest release, and list all entries that are of interest to our users:
|
||||
- Things that **must** go in the changelog: added, changed, deprecated or removed features, and bug fixes.
|
||||
- Things that **should not** go in the changelog: changes to documentation, tests or tooling.
|
||||
- Try sorting entries in descending order of impact / importance.
|
||||
- Keep it concise and to-the-point. 🎯
|
||||
- **A version bump**: see `__version__.py`.
|
||||
|
||||
For an example, see [#1006](https://github.com/encode/httpx/pull/1006).
|
||||
|
||||
Once the release PR is merged, create a
|
||||
[new release](https://github.com/encode/httpx/releases/new) including:
|
||||
|
||||
- Tag version like `0.13.3`.
|
||||
- Release title `Version 0.13.3`
|
||||
- Description copied from the changelog.
|
||||
|
||||
Once created this release will be automatically uploaded to PyPI.
|
||||
|
||||
If something goes wrong with the PyPI job the release can be published using the
|
||||
`scripts/publish` script.
|
||||
|
||||
## Development proxy setup
|
||||
|
||||
To test and debug requests via a proxy it's best to run a proxy server locally.
|
||||
Any server should do but HTTPCore's test suite uses
|
||||
[`mitmproxy`](https://mitmproxy.org/) which is written in Python, it's fully
|
||||
featured and has excellent UI and tools for introspection of requests.
|
||||
|
||||
You can install `mitmproxy` using `pip install mitmproxy` or [several
|
||||
other ways](https://docs.mitmproxy.org/stable/overview-installation/).
|
||||
|
||||
`mitmproxy` does require setting up local TLS certificates for HTTPS requests,
|
||||
as its main purpose is to allow developers to inspect requests that pass through
|
||||
it. We can set them up follows:
|
||||
|
||||
1. [`pip install trustme-cli`](https://github.com/sethmlarson/trustme-cli/).
|
||||
2. `trustme-cli -i example.org www.example.org`, assuming you want to test
|
||||
connecting to that domain, this will create three files: `server.pem`,
|
||||
`server.key` and `client.pem`.
|
||||
3. `mitmproxy` requires a PEM file that includes the private key and the
|
||||
certificate so we need to concatenate them:
|
||||
`cat server.key server.pem > server.withkey.pem`.
|
||||
4. Start the proxy server `mitmproxy --certs server.withkey.pem`, or use the
|
||||
[other mitmproxy commands](https://docs.mitmproxy.org/stable/) with different
|
||||
UI options.
|
||||
|
||||
At this point the server is ready to start serving requests, you'll need to
|
||||
configure HTTPX as described in the
|
||||
[proxy section](https://www.python-httpx.org/advanced/#http-proxying) and
|
||||
the [SSL certificates section](https://www.python-httpx.org/advanced/#ssl-certificates),
|
||||
this is where our previously generated `client.pem` comes in:
|
||||
|
||||
```
|
||||
import httpx
|
||||
|
||||
ssl_context = httpx.SSLContext()
|
||||
ssl_context.load_verify_locations("/path/to/client.pem")
|
||||
|
||||
with httpx.Client(proxy="http://127.0.0.1:8080/", ssl_context=ssl_context) as client:
|
||||
response = client.get("https://example.org")
|
||||
print(response.status_code) # should print 200
|
||||
```
|
||||
|
||||
Note, however, that HTTPS requests will only succeed to the host specified
|
||||
in the SSL/TLS certificate we generated, HTTPS requests to other hosts will
|
||||
raise an error like:
|
||||
|
||||
```
|
||||
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate
|
||||
verify failed: Hostname mismatch, certificate is not valid for
|
||||
'duckduckgo.com'. (_ssl.c:1108)
|
||||
```
|
||||
|
||||
If you want to make requests to more hosts you'll need to regenerate the
|
||||
certificates and include all the hosts you intend to connect to in the
|
||||
seconds step, i.e.
|
||||
|
||||
`trustme-cli -i example.org www.example.org duckduckgo.com www.duckduckgo.com`
|
||||
1
.github/FUNDING.yml
vendored
1
.github/FUNDING.yml
vendored
@ -1 +0,0 @@
|
||||
github: encode
|
||||
16
.github/ISSUE_TEMPLATE/1-issue.md
vendored
16
.github/ISSUE_TEMPLATE/1-issue.md
vendored
@ -1,16 +0,0 @@
|
||||
---
|
||||
name: Issue
|
||||
about: Please only raise an issue if you've been advised to do so after discussion. Thanks! 🙏
|
||||
---
|
||||
|
||||
The starting point for issues should usually be a discussion...
|
||||
|
||||
https://github.com/encode/httpx/discussions
|
||||
|
||||
Possible bugs may be raised as a "Potential Issue" discussion, feature requests may be raised as an "Ideas" discussion. We can then determine if the discussion needs to be escalated into an "Issue" or not.
|
||||
|
||||
This will help us ensure that the "Issues" list properly reflects ongoing or needed work on the project.
|
||||
|
||||
---
|
||||
|
||||
- [ ] Initially raised as discussion #...
|
||||
10
.github/ISSUE_TEMPLATE/config.yml
vendored
10
.github/ISSUE_TEMPLATE/config.yml
vendored
@ -1,11 +1 @@
|
||||
# Ref: https://help.github.com/en/github/building-a-strong-community/configuring-issue-templates-for-your-repository#configuring-the-template-chooser
|
||||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: Discussions
|
||||
url: https://github.com/encode/httpx/discussions
|
||||
about: >
|
||||
The "Discussions" forum is where you want to start. 💖
|
||||
- name: Chat
|
||||
url: https://gitter.im/encode/community
|
||||
about: >
|
||||
Our community chat forum.
|
||||
|
||||
10
.github/ISSUE_TEMPLATE/read-only-issues.md
vendored
Normal file
10
.github/ISSUE_TEMPLATE/read-only-issues.md
vendored
Normal file
@ -0,0 +1,10 @@
|
||||
---
|
||||
name: Read-only issues
|
||||
about: Restricted Zone ⛔️
|
||||
title: ''
|
||||
labels: ''
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
Issues on this repository are considered read-only, and currently reserved for the maintenance team.
|
||||
12
.github/PULL_REQUEST_TEMPLATE.md
vendored
12
.github/PULL_REQUEST_TEMPLATE.md
vendored
@ -1,12 +0,0 @@
|
||||
<!-- Thanks for contributing to HTTPX! 💚
|
||||
Given this is a project maintained by volunteers, please read this template to not waste your time, or ours! 😁 -->
|
||||
|
||||
# Summary
|
||||
|
||||
<!-- Write a small summary about what is happening here. -->
|
||||
|
||||
# Checklist
|
||||
|
||||
- [ ] I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
|
||||
- [ ] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
|
||||
- [ ] I've updated the documentation accordingly.
|
||||
14
.github/dependabot.yml
vendored
14
.github/dependabot.yml
vendored
@ -1,14 +0,0 @@
|
||||
version: 2
|
||||
updates:
|
||||
- package-ecosystem: "pip"
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: "monthly"
|
||||
groups:
|
||||
python-packages:
|
||||
patterns:
|
||||
- "*"
|
||||
- package-ecosystem: "github-actions"
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: monthly
|
||||
29
.github/workflows/publish.yml
vendored
29
.github/workflows/publish.yml
vendored
@ -1,29 +0,0 @@
|
||||
name: Publish
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- '*'
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
name: "Publish release"
|
||||
runs-on: "ubuntu-latest"
|
||||
|
||||
environment:
|
||||
name: deploy
|
||||
|
||||
steps:
|
||||
- uses: "actions/checkout@v4"
|
||||
- uses: "actions/setup-python@v6"
|
||||
with:
|
||||
python-version: 3.9
|
||||
- name: "Install dependencies"
|
||||
run: "scripts/install"
|
||||
- name: "Build package & docs"
|
||||
run: "scripts/build"
|
||||
- name: "Publish to PyPI & deploy docs"
|
||||
run: "scripts/publish"
|
||||
env:
|
||||
TWINE_USERNAME: __token__
|
||||
TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
|
||||
14
.github/workflows/test-suite.yml
vendored
14
.github/workflows/test-suite.yml
vendored
@ -3,9 +3,9 @@ name: Test Suite
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: ["master"]
|
||||
branches: ["v1"]
|
||||
pull_request:
|
||||
branches: ["master", "version-*"]
|
||||
branches: ["v1"]
|
||||
|
||||
jobs:
|
||||
tests:
|
||||
@ -14,21 +14,15 @@ jobs:
|
||||
|
||||
strategy:
|
||||
matrix:
|
||||
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
|
||||
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
|
||||
|
||||
steps:
|
||||
- uses: "actions/checkout@v4"
|
||||
- uses: "actions/setup-python@v6"
|
||||
- uses: "actions/setup-python@v5"
|
||||
with:
|
||||
python-version: "${{ matrix.python-version }}"
|
||||
allow-prereleases: true
|
||||
- name: "Install dependencies"
|
||||
run: "scripts/install"
|
||||
- name: "Run linting checks"
|
||||
run: "scripts/check"
|
||||
- name: "Build package & docs"
|
||||
run: "scripts/build"
|
||||
- name: "Run tests"
|
||||
run: "scripts/test"
|
||||
- name: "Enforce coverage"
|
||||
run: "scripts/coverage"
|
||||
|
||||
10
.gitignore
vendored
10
.gitignore
vendored
@ -1,12 +1,8 @@
|
||||
*.pyc
|
||||
.coverage
|
||||
.pytest_cache/
|
||||
.mypy_cache/
|
||||
.pytest_cache/
|
||||
__pycache__/
|
||||
htmlcov/
|
||||
site/
|
||||
*.egg-info/
|
||||
venv*/
|
||||
.python-version
|
||||
build/
|
||||
dist/
|
||||
venv/
|
||||
build/
|
||||
|
||||
1142
CHANGELOG.md
1142
CHANGELOG.md
File diff suppressed because it is too large
Load Diff
12
LICENSE.md
12
LICENSE.md
@ -1,12 +0,0 @@
|
||||
Copyright © 2019, [Encode OSS Ltd](https://www.encode.io/).
|
||||
All rights reserved.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
|
||||
|
||||
* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
|
||||
|
||||
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
|
||||
|
||||
* Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
165
README.md
165
README.md
@ -1,32 +1,22 @@
|
||||
<p align="center">
|
||||
<a href="https://www.python-httpx.org/"><img width="350" height="208" src="https://raw.githubusercontent.com/encode/httpx/master/docs/img/butterfly.png" alt='HTTPX'></a>
|
||||
<img width="350" height="208" src="https://raw.githubusercontent.com/encode/httpx/master/docs/img/butterfly.png" alt='HTTPX'>
|
||||
</p>
|
||||
|
||||
<p align="center"><strong>HTTPX</strong> <em>- A next-generation HTTP client for Python.</em></p>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/encode/httpx/actions">
|
||||
<img src="https://github.com/encode/httpx/workflows/Test%20Suite/badge.svg" alt="Test Suite">
|
||||
</a>
|
||||
<a href="https://pypi.org/project/httpx/">
|
||||
<img src="https://badge.fury.io/py/httpx.svg" alt="Package version">
|
||||
</a>
|
||||
</p>
|
||||
|
||||
HTTPX is a fully featured HTTP client library for Python 3. It includes **an integrated command line client**, has support for both **HTTP/1.1 and HTTP/2**, and provides both **sync and async APIs**.
|
||||
<p align="center"><em>HTTPX 1.0 — Design proposal.</em></p>
|
||||
|
||||
---
|
||||
|
||||
Install HTTPX using pip:
|
||||
A complete HTTP framework for Python.
|
||||
|
||||
*Installation...*
|
||||
|
||||
```shell
|
||||
$ pip install httpx
|
||||
$ pip install --pre httpx
|
||||
```
|
||||
|
||||
Now, let's get started:
|
||||
*Making requests as a client...*
|
||||
|
||||
```pycon
|
||||
>>> import httpx
|
||||
```python
|
||||
>>> r = httpx.get('https://www.example.org/')
|
||||
>>> r
|
||||
<Response [200 OK]>
|
||||
@ -38,110 +28,45 @@ Now, let's get started:
|
||||
'<!doctype html>\n<html>\n<head>\n<title>Example Domain</title>...'
|
||||
```
|
||||
|
||||
Or, using the command-line client.
|
||||
*Serving responses as the server...*
|
||||
|
||||
```shell
|
||||
$ pip install 'httpx[cli]' # The command line client is an optional dependency.
|
||||
```python
|
||||
>>> def app(request):
|
||||
... content = httpx.HTML('<html><body>hello, world.</body></html>')
|
||||
... return httpx.Response(200, content=content)
|
||||
|
||||
>>> httpx.run(app)
|
||||
Serving on http://127.0.0.1:8080/ (Press CTRL+C to quit)
|
||||
```
|
||||
|
||||
Which now allows us to use HTTPX directly from the command-line...
|
||||
|
||||
<p align="center">
|
||||
<img width="700" src="docs/img/httpx-help.png" alt='httpx --help'>
|
||||
</p>
|
||||
|
||||
Sending a request...
|
||||
|
||||
<p align="center">
|
||||
<img width="700" src="docs/img/httpx-request.png" alt='httpx http://httpbin.org/json'>
|
||||
</p>
|
||||
|
||||
## Features
|
||||
|
||||
HTTPX builds on the well-established usability of `requests`, and gives you:
|
||||
|
||||
* A broadly [requests-compatible API](https://www.python-httpx.org/compatibility/).
|
||||
* An integrated command-line client.
|
||||
* HTTP/1.1 [and HTTP/2 support](https://www.python-httpx.org/http2/).
|
||||
* Standard synchronous interface, but with [async support if you need it](https://www.python-httpx.org/async/).
|
||||
* Ability to make requests directly to [WSGI applications](https://www.python-httpx.org/advanced/transports/#wsgi-transport) or [ASGI applications](https://www.python-httpx.org/advanced/transports/#asgi-transport).
|
||||
* Strict timeouts everywhere.
|
||||
* Fully type annotated.
|
||||
* 100% test coverage.
|
||||
|
||||
Plus all the standard features of `requests`...
|
||||
|
||||
* International Domains and URLs
|
||||
* Keep-Alive & Connection Pooling
|
||||
* Sessions with Cookie Persistence
|
||||
* Browser-style SSL Verification
|
||||
* Basic/Digest Authentication
|
||||
* Elegant Key/Value Cookies
|
||||
* Automatic Decompression
|
||||
* Automatic Content Decoding
|
||||
* Unicode Response Bodies
|
||||
* Multipart File Uploads
|
||||
* HTTP(S) Proxy Support
|
||||
* Connection Timeouts
|
||||
* Streaming Downloads
|
||||
* .netrc Support
|
||||
* Chunked Requests
|
||||
|
||||
## Installation
|
||||
|
||||
Install with pip:
|
||||
|
||||
```shell
|
||||
$ pip install httpx
|
||||
```
|
||||
|
||||
Or, to include the optional HTTP/2 support, use:
|
||||
|
||||
```shell
|
||||
$ pip install httpx[http2]
|
||||
```
|
||||
|
||||
HTTPX requires Python 3.9+.
|
||||
|
||||
## Documentation
|
||||
|
||||
Project documentation is available at [https://www.python-httpx.org/](https://www.python-httpx.org/).
|
||||
|
||||
For a run-through of all the basics, head over to the [QuickStart](https://www.python-httpx.org/quickstart/).
|
||||
|
||||
For more advanced topics, see the [Advanced Usage](https://www.python-httpx.org/advanced/) section, the [async support](https://www.python-httpx.org/async/) section, or the [HTTP/2](https://www.python-httpx.org/http2/) section.
|
||||
|
||||
The [Developer Interface](https://www.python-httpx.org/api/) provides a comprehensive API reference.
|
||||
|
||||
To find out about tools that integrate with HTTPX, see [Third Party Packages](https://www.python-httpx.org/third_party_packages/).
|
||||
|
||||
## Contribute
|
||||
|
||||
If you want to contribute with HTTPX check out the [Contributing Guide](https://www.python-httpx.org/contributing/) to learn how to start.
|
||||
|
||||
## Dependencies
|
||||
|
||||
The HTTPX project relies on these excellent libraries:
|
||||
|
||||
* `httpcore` - The underlying transport implementation for `httpx`.
|
||||
* `h11` - HTTP/1.1 support.
|
||||
* `certifi` - SSL certificates.
|
||||
* `idna` - Internationalized domain name support.
|
||||
* `sniffio` - Async library autodetection.
|
||||
|
||||
As well as these optional installs:
|
||||
|
||||
* `h2` - HTTP/2 support. *(Optional, with `httpx[http2]`)*
|
||||
* `socksio` - SOCKS proxy support. *(Optional, with `httpx[socks]`)*
|
||||
* `rich` - Rich terminal support. *(Optional, with `httpx[cli]`)*
|
||||
* `click` - Command line client support. *(Optional, with `httpx[cli]`)*
|
||||
* `brotli` or `brotlicffi` - Decoding for "brotli" compressed responses. *(Optional, with `httpx[brotli]`)*
|
||||
* `zstandard` - Decoding for "zstd" compressed responses. *(Optional, with `httpx[zstd]`)*
|
||||
|
||||
A huge amount of credit is due to `requests` for the API layout that
|
||||
much of this work follows, as well as to `urllib3` for plenty of design
|
||||
inspiration around the lower-level networking details.
|
||||
|
||||
---
|
||||
|
||||
<p align="center"><i>HTTPX is <a href="https://github.com/encode/httpx/blob/master/LICENSE.md">BSD licensed</a> code.<br/>Designed & crafted with care.</i><br/>— 🦋 —</p>
|
||||
# Documentation
|
||||
|
||||
The [HTTPX 1.0 design proposal](https://www.encode.io/httpnext/) is now available.
|
||||
|
||||
* [Quickstart](https://www.encode.io/httpnext/quickstart)
|
||||
* [Clients](https://www.encode.io/httpnext/clients)
|
||||
* [Servers](https://www.encode.io/httpnext/servers)
|
||||
* [Requests](https://www.encode.io/httpnext/requests)
|
||||
* [Responses](https://www.encode.io/httpnext/responses)
|
||||
* [URLs](https://www.encode.io/httpnext/urls)
|
||||
* [Headers](https://www.encode.io/httpnext/headers)
|
||||
* [Content Types](https://www.encode.io/httpnext/content-types)
|
||||
* [Connections](https://www.encode.io/httpnext/connections)
|
||||
* [Parsers](https://www.encode.io/httpnext/parsers)
|
||||
* [Network Backends](https://www.encode.io/httpnext/networking)
|
||||
|
||||
---
|
||||
|
||||
# Collaboration
|
||||
|
||||
We are not currently accepting unsolicted pull requests against the 1.0 pre-release branch.
|
||||
|
||||
We’re looking at creating paid opportunities for working on open source software *which are properly compensated, flexible & well balanced.*
|
||||
|
||||
If you're interested in a working on this project, please <a href="mailto:kim@encode.io">send an intro</a>.
|
||||
|
||||
---
|
||||
|
||||
<p align="center"><i>This provisional design work is <a href="https://github.com/encode/httpnext/blob/master/LICENSE.md">not currently licensed</a> for reuse.<br/>Designed & crafted with care.</i><br/>— 🦋 —</p>
|
||||
|
||||
@ -1 +0,0 @@
|
||||
www.python-httpx.org
|
||||
19
docs/about.md
Normal file
19
docs/about.md
Normal file
@ -0,0 +1,19 @@
|
||||
# About
|
||||
|
||||
This work is a design proposal for an `httpx` 1.0 release.
|
||||
|
||||
---
|
||||
|
||||
## Sponsorship
|
||||
|
||||
We are currently seeking forward-looking investment that recognises the value of the infrastructure development on it's own merit. Sponsorships may be [made through GitHub](https://github.com/encode).
|
||||
|
||||
We do not offer equity, placements, or endorsments.
|
||||
|
||||
## License
|
||||
|
||||
The rights of the author have been asserted.
|
||||
|
||||
---
|
||||
|
||||
<p align="center"><i><a href="https://www.encode.io/httpnext">home</a><i></p>
|
||||
@ -1,232 +0,0 @@
|
||||
Authentication can either be included on a per-request basis...
|
||||
|
||||
```pycon
|
||||
>>> auth = httpx.BasicAuth(username="username", password="secret")
|
||||
>>> client = httpx.Client()
|
||||
>>> response = client.get("https://www.example.com/", auth=auth)
|
||||
```
|
||||
|
||||
Or configured on the client instance, ensuring that all outgoing requests will include authentication credentials...
|
||||
|
||||
```pycon
|
||||
>>> auth = httpx.BasicAuth(username="username", password="secret")
|
||||
>>> client = httpx.Client(auth=auth)
|
||||
>>> response = client.get("https://www.example.com/")
|
||||
```
|
||||
|
||||
## Basic authentication
|
||||
|
||||
HTTP basic authentication is an unencrypted authentication scheme that uses a simple encoding of the username and password in the request `Authorization` header. Since it is unencrypted it should typically only be used over `https`, although this is not strictly enforced.
|
||||
|
||||
```pycon
|
||||
>>> auth = httpx.BasicAuth(username="finley", password="secret")
|
||||
>>> client = httpx.Client(auth=auth)
|
||||
>>> response = client.get("https://httpbin.org/basic-auth/finley/secret")
|
||||
>>> response
|
||||
<Response [200 OK]>
|
||||
```
|
||||
|
||||
## Digest authentication
|
||||
|
||||
HTTP digest authentication is a challenge-response authentication scheme. Unlike basic authentication it provides encryption, and can be used over unencrypted `http` connections. It requires an additional round-trip in order to negotiate the authentication.
|
||||
|
||||
```pycon
|
||||
>>> auth = httpx.DigestAuth(username="olivia", password="secret")
|
||||
>>> client = httpx.Client(auth=auth)
|
||||
>>> response = client.get("https://httpbin.org/digest-auth/auth/olivia/secret")
|
||||
>>> response
|
||||
<Response [200 OK]>
|
||||
>>> response.history
|
||||
[<Response [401 UNAUTHORIZED]>]
|
||||
```
|
||||
|
||||
## NetRC authentication
|
||||
|
||||
HTTPX can be configured to use [a `.netrc` config file](https://everything.curl.dev/usingcurl/netrc) for authentication.
|
||||
|
||||
The `.netrc` config file allows authentication credentials to be associated with specified hosts. When a request is made to a host that is found in the netrc file, the username and password will be included using HTTP basic authentication.
|
||||
|
||||
Example `.netrc` file:
|
||||
|
||||
```
|
||||
machine example.org
|
||||
login example-username
|
||||
password example-password
|
||||
|
||||
machine python-httpx.org
|
||||
login other-username
|
||||
password other-password
|
||||
```
|
||||
|
||||
Some examples of configuring `.netrc` authentication with `httpx`.
|
||||
|
||||
Use the default `.netrc` file in the users home directory:
|
||||
|
||||
```pycon
|
||||
>>> auth = httpx.NetRCAuth()
|
||||
>>> client = httpx.Client(auth=auth)
|
||||
```
|
||||
|
||||
Use an explicit path to a `.netrc` file:
|
||||
|
||||
```pycon
|
||||
>>> auth = httpx.NetRCAuth(file="/path/to/.netrc")
|
||||
>>> client = httpx.Client(auth=auth)
|
||||
```
|
||||
|
||||
Use the `NETRC` environment variable to configure a path to the `.netrc` file,
|
||||
or fallback to the default.
|
||||
|
||||
```pycon
|
||||
>>> auth = httpx.NetRCAuth(file=os.environ.get("NETRC"))
|
||||
>>> client = httpx.Client(auth=auth)
|
||||
```
|
||||
|
||||
The `NetRCAuth()` class uses [the `netrc.netrc()` function from the Python standard library](https://docs.python.org/3/library/netrc.html). See the documentation there for more details on exceptions that may be raised if the `.netrc` file is not found, or cannot be parsed.
|
||||
|
||||
## Custom authentication schemes
|
||||
|
||||
When issuing requests or instantiating a client, the `auth` argument can be used to pass an authentication scheme to use. The `auth` argument may be one of the following...
|
||||
|
||||
* A two-tuple of `username`/`password`, to be used with basic authentication.
|
||||
* An instance of `httpx.BasicAuth()`, `httpx.DigestAuth()`, or `httpx.NetRCAuth()`.
|
||||
* A callable, accepting a request and returning an authenticated request instance.
|
||||
* An instance of subclasses of `httpx.Auth`.
|
||||
|
||||
The most involved of these is the last, which allows you to create authentication flows involving one or more requests. A subclass of `httpx.Auth` should implement `def auth_flow(request)`, and yield any requests that need to be made...
|
||||
|
||||
```python
|
||||
class MyCustomAuth(httpx.Auth):
|
||||
def __init__(self, token):
|
||||
self.token = token
|
||||
|
||||
def auth_flow(self, request):
|
||||
# Send the request, with a custom `X-Authentication` header.
|
||||
request.headers['X-Authentication'] = self.token
|
||||
yield request
|
||||
```
|
||||
|
||||
If the auth flow requires more than one request, you can issue multiple yields, and obtain the response in each case...
|
||||
|
||||
```python
|
||||
class MyCustomAuth(httpx.Auth):
|
||||
def __init__(self, token):
|
||||
self.token = token
|
||||
|
||||
def auth_flow(self, request):
|
||||
response = yield request
|
||||
if response.status_code == 401:
|
||||
# If the server issues a 401 response then resend the request,
|
||||
# with a custom `X-Authentication` header.
|
||||
request.headers['X-Authentication'] = self.token
|
||||
yield request
|
||||
```
|
||||
|
||||
Custom authentication classes are designed to not perform any I/O, so that they may be used with both sync and async client instances. If you are implementing an authentication scheme that requires the request body, then you need to indicate this on the class using a `requires_request_body` property.
|
||||
|
||||
You will then be able to access `request.content` inside the `.auth_flow()` method.
|
||||
|
||||
```python
|
||||
class MyCustomAuth(httpx.Auth):
|
||||
requires_request_body = True
|
||||
|
||||
def __init__(self, token):
|
||||
self.token = token
|
||||
|
||||
def auth_flow(self, request):
|
||||
response = yield request
|
||||
if response.status_code == 401:
|
||||
# If the server issues a 401 response then resend the request,
|
||||
# with a custom `X-Authentication` header.
|
||||
request.headers['X-Authentication'] = self.sign_request(...)
|
||||
yield request
|
||||
|
||||
def sign_request(self, request):
|
||||
# Create a request signature, based on `request.method`, `request.url`,
|
||||
# `request.headers`, and `request.content`.
|
||||
...
|
||||
```
|
||||
|
||||
Similarly, if you are implementing a scheme that requires access to the response body, then use the `requires_response_body` property. You will then be able to access response body properties and methods such as `response.content`, `response.text`, `response.json()`, etc.
|
||||
|
||||
```python
|
||||
class MyCustomAuth(httpx.Auth):
|
||||
requires_response_body = True
|
||||
|
||||
def __init__(self, access_token, refresh_token, refresh_url):
|
||||
self.access_token = access_token
|
||||
self.refresh_token = refresh_token
|
||||
self.refresh_url = refresh_url
|
||||
|
||||
def auth_flow(self, request):
|
||||
request.headers["X-Authentication"] = self.access_token
|
||||
response = yield request
|
||||
|
||||
if response.status_code == 401:
|
||||
# If the server issues a 401 response, then issue a request to
|
||||
# refresh tokens, and resend the request.
|
||||
refresh_response = yield self.build_refresh_request()
|
||||
self.update_tokens(refresh_response)
|
||||
|
||||
request.headers["X-Authentication"] = self.access_token
|
||||
yield request
|
||||
|
||||
def build_refresh_request(self):
|
||||
# Return an `httpx.Request` for refreshing tokens.
|
||||
...
|
||||
|
||||
def update_tokens(self, response):
|
||||
# Update the `.access_token` and `.refresh_token` tokens
|
||||
# based on a refresh response.
|
||||
data = response.json()
|
||||
...
|
||||
```
|
||||
|
||||
If you _do_ need to perform I/O other than HTTP requests, such as accessing a disk-based cache, or you need to use concurrency primitives, such as locks, then you should override `.sync_auth_flow()` and `.async_auth_flow()` (instead of `.auth_flow()`). The former will be used by `httpx.Client`, while the latter will be used by `httpx.AsyncClient`.
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
import threading
|
||||
import httpx
|
||||
|
||||
|
||||
class MyCustomAuth(httpx.Auth):
|
||||
def __init__(self):
|
||||
self._sync_lock = threading.RLock()
|
||||
self._async_lock = asyncio.Lock()
|
||||
|
||||
def sync_get_token(self):
|
||||
with self._sync_lock:
|
||||
...
|
||||
|
||||
def sync_auth_flow(self, request):
|
||||
token = self.sync_get_token()
|
||||
request.headers["Authorization"] = f"Token {token}"
|
||||
yield request
|
||||
|
||||
async def async_get_token(self):
|
||||
async with self._async_lock:
|
||||
...
|
||||
|
||||
async def async_auth_flow(self, request):
|
||||
token = await self.async_get_token()
|
||||
request.headers["Authorization"] = f"Token {token}"
|
||||
yield request
|
||||
```
|
||||
|
||||
If you only want to support one of the two methods, then you should still override it, but raise an explicit `RuntimeError`.
|
||||
|
||||
```python
|
||||
import httpx
|
||||
import sync_only_library
|
||||
|
||||
|
||||
class MyCustomAuth(httpx.Auth):
|
||||
def sync_auth_flow(self, request):
|
||||
token = sync_only_library.get_token(...)
|
||||
request.headers["Authorization"] = f"Token {token}"
|
||||
yield request
|
||||
|
||||
async def async_auth_flow(self, request):
|
||||
raise RuntimeError("Cannot use a sync authentication class with httpx.AsyncClient")
|
||||
```
|
||||
@ -1,328 +0,0 @@
|
||||
!!! hint
|
||||
If you are coming from Requests, `httpx.Client()` is what you can use instead of `requests.Session()`.
|
||||
|
||||
## Why use a Client?
|
||||
|
||||
!!! note "TL;DR"
|
||||
If you do anything more than experimentation, one-off scripts, or prototypes, then you should use a `Client` instance.
|
||||
|
||||
**More efficient usage of network resources**
|
||||
|
||||
When you make requests using the top-level API as documented in the [Quickstart](../quickstart.md) guide, HTTPX has to establish a new connection _for every single request_ (connections are not reused). As the number of requests to a host increases, this quickly becomes inefficient.
|
||||
|
||||
On the other hand, a `Client` instance uses [HTTP connection pooling](https://en.wikipedia.org/wiki/HTTP_persistent_connection). This means that when you make several requests to the same host, the `Client` will reuse the underlying TCP connection, instead of recreating one for every single request.
|
||||
|
||||
This can bring **significant performance improvements** compared to using the top-level API, including:
|
||||
|
||||
- Reduced latency across requests (no handshaking).
|
||||
- Reduced CPU usage and round-trips.
|
||||
- Reduced network congestion.
|
||||
|
||||
**Extra features**
|
||||
|
||||
`Client` instances also support features that aren't available at the top-level API, such as:
|
||||
|
||||
- Cookie persistence across requests.
|
||||
- Applying configuration across all outgoing requests.
|
||||
- Sending requests through HTTP proxies.
|
||||
- Using [HTTP/2](../http2.md).
|
||||
|
||||
The other sections on this page go into further detail about what you can do with a `Client` instance.
|
||||
|
||||
## Usage
|
||||
|
||||
The recommended way to use a `Client` is as a context manager. This will ensure that connections are properly cleaned up when leaving the `with` block:
|
||||
|
||||
```python
|
||||
with httpx.Client() as client:
|
||||
...
|
||||
```
|
||||
|
||||
Alternatively, you can explicitly close the connection pool without block-usage using `.close()`:
|
||||
|
||||
```python
|
||||
client = httpx.Client()
|
||||
try:
|
||||
...
|
||||
finally:
|
||||
client.close()
|
||||
```
|
||||
|
||||
## Making requests
|
||||
|
||||
Once you have a `Client`, you can send requests using `.get()`, `.post()`, etc. For example:
|
||||
|
||||
```pycon
|
||||
>>> with httpx.Client() as client:
|
||||
... r = client.get('https://example.com')
|
||||
...
|
||||
>>> r
|
||||
<Response [200 OK]>
|
||||
```
|
||||
|
||||
These methods accept the same arguments as `httpx.get()`, `httpx.post()`, etc. This means that all features documented in the [Quickstart](../quickstart.md) guide are also available at the client level.
|
||||
|
||||
For example, to send a request with custom headers:
|
||||
|
||||
```pycon
|
||||
>>> with httpx.Client() as client:
|
||||
... headers = {'X-Custom': 'value'}
|
||||
... r = client.get('https://example.com', headers=headers)
|
||||
...
|
||||
>>> r.request.headers['X-Custom']
|
||||
'value'
|
||||
```
|
||||
|
||||
## Sharing configuration across requests
|
||||
|
||||
Clients allow you to apply configuration to all outgoing requests by passing parameters to the `Client` constructor.
|
||||
|
||||
For example, to apply a set of custom headers _on every request_:
|
||||
|
||||
```pycon
|
||||
>>> url = 'http://httpbin.org/headers'
|
||||
>>> headers = {'user-agent': 'my-app/0.0.1'}
|
||||
>>> with httpx.Client(headers=headers) as client:
|
||||
... r = client.get(url)
|
||||
...
|
||||
>>> r.json()['headers']['User-Agent']
|
||||
'my-app/0.0.1'
|
||||
```
|
||||
|
||||
## Merging of configuration
|
||||
|
||||
When a configuration option is provided at both the client-level and request-level, one of two things can happen:
|
||||
|
||||
- For headers, query parameters and cookies, the values are combined together. For example:
|
||||
|
||||
```pycon
|
||||
>>> headers = {'X-Auth': 'from-client'}
|
||||
>>> params = {'client_id': 'client1'}
|
||||
>>> with httpx.Client(headers=headers, params=params) as client:
|
||||
... headers = {'X-Custom': 'from-request'}
|
||||
... params = {'request_id': 'request1'}
|
||||
... r = client.get('https://example.com', headers=headers, params=params)
|
||||
...
|
||||
>>> r.request.url
|
||||
URL('https://example.com?client_id=client1&request_id=request1')
|
||||
>>> r.request.headers['X-Auth']
|
||||
'from-client'
|
||||
>>> r.request.headers['X-Custom']
|
||||
'from-request'
|
||||
```
|
||||
|
||||
- For all other parameters, the request-level value takes priority. For example:
|
||||
|
||||
```pycon
|
||||
>>> with httpx.Client(auth=('tom', 'mot123')) as client:
|
||||
... r = client.get('https://example.com', auth=('alice', 'ecila123'))
|
||||
...
|
||||
>>> _, _, auth = r.request.headers['Authorization'].partition(' ')
|
||||
>>> import base64
|
||||
>>> base64.b64decode(auth)
|
||||
b'alice:ecila123'
|
||||
```
|
||||
|
||||
If you need finer-grained control on the merging of client-level and request-level parameters, see [Request instances](#request-instances).
|
||||
|
||||
## Other Client-only configuration options
|
||||
|
||||
Additionally, `Client` accepts some configuration options that aren't available at the request level.
|
||||
|
||||
For example, `base_url` allows you to prepend an URL to all outgoing requests:
|
||||
|
||||
```pycon
|
||||
>>> with httpx.Client(base_url='http://httpbin.org') as client:
|
||||
... r = client.get('/headers')
|
||||
...
|
||||
>>> r.request.url
|
||||
URL('http://httpbin.org/headers')
|
||||
```
|
||||
|
||||
For a list of all available client parameters, see the [`Client`](../api.md#client) API reference.
|
||||
|
||||
---
|
||||
|
||||
## Request instances
|
||||
|
||||
For maximum control on what gets sent over the wire, HTTPX supports building explicit [`Request`](../api.md#request) instances:
|
||||
|
||||
```python
|
||||
request = httpx.Request("GET", "https://example.com")
|
||||
```
|
||||
|
||||
To dispatch a `Request` instance across to the network, create a [`Client` instance](#client-instances) and use `.send()`:
|
||||
|
||||
```python
|
||||
with httpx.Client() as client:
|
||||
response = client.send(request)
|
||||
...
|
||||
```
|
||||
|
||||
If you need to mix client-level and request-level options in a way that is not supported by the default [Merging of parameters](#merging-of-parameters), you can use `.build_request()` and then make arbitrary modifications to the `Request` instance. For example:
|
||||
|
||||
```python
|
||||
headers = {"X-Api-Key": "...", "X-Client-ID": "ABC123"}
|
||||
|
||||
with httpx.Client(headers=headers) as client:
|
||||
request = client.build_request("GET", "https://api.example.com")
|
||||
|
||||
print(request.headers["X-Client-ID"]) # "ABC123"
|
||||
|
||||
# Don't send the API key for this particular request.
|
||||
del request.headers["X-Api-Key"]
|
||||
|
||||
response = client.send(request)
|
||||
...
|
||||
```
|
||||
|
||||
## Monitoring download progress
|
||||
|
||||
If you need to monitor download progress of large responses, you can use response streaming and inspect the `response.num_bytes_downloaded` property.
|
||||
|
||||
This interface is required for properly determining download progress, because the total number of bytes returned by `response.content` or `response.iter_content()` will not always correspond with the raw content length of the response if HTTP response compression is being used.
|
||||
|
||||
For example, showing a progress bar using the [`tqdm`](https://github.com/tqdm/tqdm) library while a response is being downloaded could be done like this…
|
||||
|
||||
```python
|
||||
import tempfile
|
||||
|
||||
import httpx
|
||||
from tqdm import tqdm
|
||||
|
||||
with tempfile.NamedTemporaryFile() as download_file:
|
||||
url = "https://speed.hetzner.de/100MB.bin"
|
||||
with httpx.stream("GET", url) as response:
|
||||
total = int(response.headers["Content-Length"])
|
||||
|
||||
with tqdm(total=total, unit_scale=True, unit_divisor=1024, unit="B") as progress:
|
||||
num_bytes_downloaded = response.num_bytes_downloaded
|
||||
for chunk in response.iter_bytes():
|
||||
download_file.write(chunk)
|
||||
progress.update(response.num_bytes_downloaded - num_bytes_downloaded)
|
||||
num_bytes_downloaded = response.num_bytes_downloaded
|
||||
```
|
||||
|
||||
|
||||
|
||||
Or an alternate example, this time using the [`rich`](https://github.com/willmcgugan/rich) library…
|
||||
|
||||
```python
|
||||
import tempfile
|
||||
import httpx
|
||||
import rich.progress
|
||||
|
||||
with tempfile.NamedTemporaryFile() as download_file:
|
||||
url = "https://speed.hetzner.de/100MB.bin"
|
||||
with httpx.stream("GET", url) as response:
|
||||
total = int(response.headers["Content-Length"])
|
||||
|
||||
with rich.progress.Progress(
|
||||
"[progress.percentage]{task.percentage:>3.0f}%",
|
||||
rich.progress.BarColumn(bar_width=None),
|
||||
rich.progress.DownloadColumn(),
|
||||
rich.progress.TransferSpeedColumn(),
|
||||
) as progress:
|
||||
download_task = progress.add_task("Download", total=total)
|
||||
for chunk in response.iter_bytes():
|
||||
download_file.write(chunk)
|
||||
progress.update(download_task, completed=response.num_bytes_downloaded)
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Monitoring upload progress
|
||||
|
||||
If you need to monitor upload progress of large responses, you can use request content generator streaming.
|
||||
|
||||
For example, showing a progress bar using the [`tqdm`](https://github.com/tqdm/tqdm) library.
|
||||
|
||||
```python
|
||||
import io
|
||||
import random
|
||||
|
||||
import httpx
|
||||
from tqdm import tqdm
|
||||
|
||||
|
||||
def gen():
|
||||
"""
|
||||
this is a complete example with generated random bytes.
|
||||
you can replace `io.BytesIO` with real file object.
|
||||
"""
|
||||
total = 32 * 1024 * 1024 # 32m
|
||||
with tqdm(ascii=True, unit_scale=True, unit='B', unit_divisor=1024, total=total) as bar:
|
||||
with io.BytesIO(random.randbytes(total)) as f:
|
||||
while data := f.read(1024):
|
||||
yield data
|
||||
bar.update(len(data))
|
||||
|
||||
|
||||
httpx.post("https://httpbin.org/post", content=gen())
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Multipart file encoding
|
||||
|
||||
As mentioned in the [quickstart](../quickstart.md#sending-multipart-file-uploads)
|
||||
multipart file encoding is available by passing a dictionary with the
|
||||
name of the payloads as keys and either tuple of elements or a file-like object or a string as values.
|
||||
|
||||
```pycon
|
||||
>>> with open('report.xls', 'rb') as report_file:
|
||||
... files = {'upload-file': ('report.xls', report_file, 'application/vnd.ms-excel')}
|
||||
... r = httpx.post("https://httpbin.org/post", files=files)
|
||||
>>> print(r.text)
|
||||
{
|
||||
...
|
||||
"files": {
|
||||
"upload-file": "<... binary content ...>"
|
||||
},
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
More specifically, if a tuple is used as a value, it must have between 2 and 3 elements:
|
||||
|
||||
- The first element is an optional file name which can be set to `None`.
|
||||
- The second element may be a file-like object or a string which will be automatically
|
||||
encoded in UTF-8.
|
||||
- An optional third element can be used to specify the
|
||||
[MIME type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_Types)
|
||||
of the file being uploaded. If not specified HTTPX will attempt to guess the MIME type based
|
||||
on the file name, with unknown file extensions defaulting to "application/octet-stream".
|
||||
If the file name is explicitly set to `None` then HTTPX will not include a content-type
|
||||
MIME header field.
|
||||
|
||||
```pycon
|
||||
>>> files = {'upload-file': (None, 'text content', 'text/plain')}
|
||||
>>> r = httpx.post("https://httpbin.org/post", files=files)
|
||||
>>> print(r.text)
|
||||
{
|
||||
...
|
||||
"files": {},
|
||||
"form": {
|
||||
"upload-file": "text-content"
|
||||
},
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
!!! tip
|
||||
It is safe to upload large files this way. File uploads are streaming by default, meaning that only one chunk will be loaded into memory at a time.
|
||||
|
||||
Non-file data fields can be included in the multipart form using by passing them to `data=...`.
|
||||
|
||||
You can also send multiple files in one go with a multiple file field form.
|
||||
To do that, pass a list of `(field, <file>)` items instead of a dictionary, allowing you to pass multiple items with the same `field`.
|
||||
For instance this request sends 2 files, `foo.png` and `bar.png` in one request on the `images` form field:
|
||||
|
||||
```pycon
|
||||
>>> with open('foo.png', 'rb') as foo_file, open('bar.png', 'rb') as bar_file:
|
||||
... files = [
|
||||
... ('images', ('foo.png', foo_file, 'image/png')),
|
||||
... ('images', ('bar.png', bar_file, 'image/png')),
|
||||
... ]
|
||||
... r = httpx.post("https://httpbin.org/post", files=files)
|
||||
```
|
||||
@ -1,65 +0,0 @@
|
||||
HTTPX allows you to register "event hooks" with the client, that are called
|
||||
every time a particular type of event takes place.
|
||||
|
||||
There are currently two event hooks:
|
||||
|
||||
* `request` - Called after a request is fully prepared, but before it is sent to the network. Passed the `request` instance.
|
||||
* `response` - Called after the response has been fetched from the network, but before it is returned to the caller. Passed the `response` instance.
|
||||
|
||||
These allow you to install client-wide functionality such as logging, monitoring or tracing.
|
||||
|
||||
```python
|
||||
def log_request(request):
|
||||
print(f"Request event hook: {request.method} {request.url} - Waiting for response")
|
||||
|
||||
def log_response(response):
|
||||
request = response.request
|
||||
print(f"Response event hook: {request.method} {request.url} - Status {response.status_code}")
|
||||
|
||||
client = httpx.Client(event_hooks={'request': [log_request], 'response': [log_response]})
|
||||
```
|
||||
|
||||
You can also use these hooks to install response processing code, such as this
|
||||
example, which creates a client instance that always raises `httpx.HTTPStatusError`
|
||||
on 4xx and 5xx responses.
|
||||
|
||||
```python
|
||||
def raise_on_4xx_5xx(response):
|
||||
response.raise_for_status()
|
||||
|
||||
client = httpx.Client(event_hooks={'response': [raise_on_4xx_5xx]})
|
||||
```
|
||||
|
||||
!!! note
|
||||
Response event hooks are called before determining if the response body
|
||||
should be read or not.
|
||||
|
||||
If you need access to the response body inside an event hook, you'll
|
||||
need to call `response.read()`, or for AsyncClients, `response.aread()`.
|
||||
|
||||
The hooks are also allowed to modify `request` and `response` objects.
|
||||
|
||||
```python
|
||||
def add_timestamp(request):
|
||||
request.headers['x-request-timestamp'] = datetime.now(tz=datetime.utc).isoformat()
|
||||
|
||||
client = httpx.Client(event_hooks={'request': [add_timestamp]})
|
||||
```
|
||||
|
||||
Event hooks must always be set as a **list of callables**, and you may register
|
||||
multiple event hooks for each type of event.
|
||||
|
||||
As well as being able to set event hooks on instantiating the client, there
|
||||
is also an `.event_hooks` property, that allows you to inspect and modify
|
||||
the installed hooks.
|
||||
|
||||
```python
|
||||
client = httpx.Client()
|
||||
client.event_hooks['request'] = [log_request]
|
||||
client.event_hooks['response'] = [log_response, raise_on_4xx_5xx]
|
||||
```
|
||||
|
||||
!!! note
|
||||
If you are using HTTPX's async support, then you need to be aware that
|
||||
hooks registered with `httpx.AsyncClient` MUST be async functions,
|
||||
rather than plain functions.
|
||||
@ -1,242 +0,0 @@
|
||||
# Extensions
|
||||
|
||||
Request and response extensions provide a untyped space where additional information may be added.
|
||||
|
||||
Extensions should be used for features that may not be available on all transports, and that do not fit neatly into [the simplified request/response model](https://www.encode.io/httpcore/extensions/) that the underlying `httpcore` package uses as its API.
|
||||
|
||||
Several extensions are supported on the request:
|
||||
|
||||
```python
|
||||
# Request timeouts actually implemented as an extension on
|
||||
# the request, ensuring that they are passed throughout the
|
||||
# entire call stack.
|
||||
client = httpx.Client()
|
||||
response = client.get(
|
||||
"https://www.example.com",
|
||||
extensions={"timeout": {"connect": 5.0}}
|
||||
)
|
||||
response.request.extensions["timeout"]
|
||||
{"connect": 5.0}
|
||||
```
|
||||
|
||||
And on the response:
|
||||
|
||||
```python
|
||||
client = httpx.Client()
|
||||
response = client.get("https://www.example.com")
|
||||
print(response.extensions["http_version"]) # b"HTTP/1.1"
|
||||
# Other server responses could have been
|
||||
# b"HTTP/0.9", b"HTTP/1.0", or b"HTTP/1.1"
|
||||
```
|
||||
|
||||
## Request Extensions
|
||||
|
||||
### `"trace"`
|
||||
|
||||
The trace extension allows a callback handler to be installed to monitor the internal
|
||||
flow of events within the underlying `httpcore` transport.
|
||||
|
||||
The simplest way to explain this is with an example:
|
||||
|
||||
```python
|
||||
import httpx
|
||||
|
||||
def log(event_name, info):
|
||||
print(event_name, info)
|
||||
|
||||
client = httpx.Client()
|
||||
response = client.get("https://www.example.com/", extensions={"trace": log})
|
||||
# connection.connect_tcp.started {'host': 'www.example.com', 'port': 443, 'local_address': None, 'timeout': None}
|
||||
# connection.connect_tcp.complete {'return_value': <httpcore.backends.sync.SyncStream object at 0x1093f94d0>}
|
||||
# connection.start_tls.started {'ssl_context': <ssl.SSLContext object at 0x1093ee750>, 'server_hostname': b'www.example.com', 'timeout': None}
|
||||
# connection.start_tls.complete {'return_value': <httpcore.backends.sync.SyncStream object at 0x1093f9450>}
|
||||
# http11.send_request_headers.started {'request': <Request [b'GET']>}
|
||||
# http11.send_request_headers.complete {'return_value': None}
|
||||
# http11.send_request_body.started {'request': <Request [b'GET']>}
|
||||
# http11.send_request_body.complete {'return_value': None}
|
||||
# http11.receive_response_headers.started {'request': <Request [b'GET']>}
|
||||
# http11.receive_response_headers.complete {'return_value': (b'HTTP/1.1', 200, b'OK', [(b'Age', b'553715'), (b'Cache-Control', b'max-age=604800'), (b'Content-Type', b'text/html; charset=UTF-8'), (b'Date', b'Thu, 21 Oct 2021 17:08:42 GMT'), (b'Etag', b'"3147526947+ident"'), (b'Expires', b'Thu, 28 Oct 2021 17:08:42 GMT'), (b'Last-Modified', b'Thu, 17 Oct 2019 07:18:26 GMT'), (b'Server', b'ECS (nyb/1DCD)'), (b'Vary', b'Accept-Encoding'), (b'X-Cache', b'HIT'), (b'Content-Length', b'1256')])}
|
||||
# http11.receive_response_body.started {'request': <Request [b'GET']>}
|
||||
# http11.receive_response_body.complete {'return_value': None}
|
||||
# http11.response_closed.started {}
|
||||
# http11.response_closed.complete {'return_value': None}
|
||||
```
|
||||
|
||||
The `event_name` and `info` arguments here will be one of the following:
|
||||
|
||||
* `{event_type}.{event_name}.started`, `<dictionary of keyword arguments>`
|
||||
* `{event_type}.{event_name}.complete`, `{"return_value": <...>}`
|
||||
* `{event_type}.{event_name}.failed`, `{"exception": <...>}`
|
||||
|
||||
Note that when using async code the handler function passed to `"trace"` must be an `async def ...` function.
|
||||
|
||||
The following event types are currently exposed...
|
||||
|
||||
**Establishing the connection**
|
||||
|
||||
* `"connection.connect_tcp"`
|
||||
* `"connection.connect_unix_socket"`
|
||||
* `"connection.start_tls"`
|
||||
|
||||
**HTTP/1.1 events**
|
||||
|
||||
* `"http11.send_request_headers"`
|
||||
* `"http11.send_request_body"`
|
||||
* `"http11.receive_response"`
|
||||
* `"http11.receive_response_body"`
|
||||
* `"http11.response_closed"`
|
||||
|
||||
**HTTP/2 events**
|
||||
|
||||
* `"http2.send_connection_init"`
|
||||
* `"http2.send_request_headers"`
|
||||
* `"http2.send_request_body"`
|
||||
* `"http2.receive_response_headers"`
|
||||
* `"http2.receive_response_body"`
|
||||
* `"http2.response_closed"`
|
||||
|
||||
The exact set of trace events may be subject to change across different versions of `httpcore`. If you need to rely on a particular set of events it is recommended that you pin installation of the package to a fixed version.
|
||||
|
||||
### `"sni_hostname"`
|
||||
|
||||
The server's hostname, which is used to confirm the hostname supplied by the SSL certificate.
|
||||
|
||||
If you want to connect to an explicit IP address rather than using the standard DNS hostname lookup, then you'll need to use this request extension.
|
||||
|
||||
For example:
|
||||
|
||||
``` python
|
||||
# Connect to '185.199.108.153' but use 'www.encode.io' in the Host header,
|
||||
# and use 'www.encode.io' when SSL verifying the server hostname.
|
||||
client = httpx.Client()
|
||||
headers = {"Host": "www.encode.io"}
|
||||
extensions = {"sni_hostname": "www.encode.io"}
|
||||
response = client.get(
|
||||
"https://185.199.108.153/path",
|
||||
headers=headers,
|
||||
extensions=extensions
|
||||
)
|
||||
```
|
||||
|
||||
### `"timeout"`
|
||||
|
||||
A dictionary of `str: Optional[float]` timeout values.
|
||||
|
||||
May include values for `'connect'`, `'read'`, `'write'`, or `'pool'`.
|
||||
|
||||
For example:
|
||||
|
||||
```python
|
||||
# Timeout if a connection takes more than 5 seconds to established, or if
|
||||
# we are blocked waiting on the connection pool for more than 10 seconds.
|
||||
client = httpx.Client()
|
||||
response = client.get(
|
||||
"https://www.example.com",
|
||||
extensions={"timeout": {"connect": 5.0, "pool": 10.0}}
|
||||
)
|
||||
```
|
||||
|
||||
This extension is how the `httpx` timeouts are implemented, ensuring that the timeout values are associated with the request instance and passed throughout the stack. You shouldn't typically be working with this extension directly, but use the higher level `timeout` API instead.
|
||||
|
||||
### `"target"`
|
||||
|
||||
The target that is used as [the HTTP target instead of the URL path](https://datatracker.ietf.org/doc/html/rfc2616#section-5.1.2).
|
||||
|
||||
This enables support constructing requests that would otherwise be unsupported.
|
||||
|
||||
* URL paths with non-standard escaping applied.
|
||||
* Forward proxy requests using an absolute URI.
|
||||
* Tunneling proxy requests using `CONNECT` with hostname as the target.
|
||||
* Server-wide `OPTIONS *` requests.
|
||||
|
||||
Some examples:
|
||||
|
||||
Using the 'target' extension to send requests without the standard path escaping rules...
|
||||
|
||||
```python
|
||||
# Typically a request to "https://www.example.com/test^path" would
|
||||
# connect to "www.example.com" and send an HTTP/1.1 request like...
|
||||
#
|
||||
# GET /test%5Epath HTTP/1.1
|
||||
#
|
||||
# Using the target extension we can include the literal '^'...
|
||||
#
|
||||
# GET /test^path HTTP/1.1
|
||||
#
|
||||
# Note that requests must still be valid HTTP requests.
|
||||
# For example including whitespace in the target will raise a `LocalProtocolError`.
|
||||
extensions = {"target": b"/test^path"}
|
||||
response = httpx.get("https://www.example.com", extensions=extensions)
|
||||
```
|
||||
|
||||
The `target` extension also allows server-wide `OPTIONS *` requests to be constructed...
|
||||
|
||||
```python
|
||||
# This will send the following request...
|
||||
#
|
||||
# CONNECT * HTTP/1.1
|
||||
extensions = {"target": b"*"}
|
||||
response = httpx.request("CONNECT", "https://www.example.com", extensions=extensions)
|
||||
```
|
||||
|
||||
## Response Extensions
|
||||
|
||||
### `"http_version"`
|
||||
|
||||
The HTTP version, as bytes. Eg. `b"HTTP/1.1"`.
|
||||
|
||||
When using HTTP/1.1 the response line includes an explicit version, and the value of this key could feasibly be one of `b"HTTP/0.9"`, `b"HTTP/1.0"`, or `b"HTTP/1.1"`.
|
||||
|
||||
When using HTTP/2 there is no further response versioning included in the protocol, and the value of this key will always be `b"HTTP/2"`.
|
||||
|
||||
### `"reason_phrase"`
|
||||
|
||||
The reason-phrase of the HTTP response, as bytes. For example `b"OK"`. Some servers may include a custom reason phrase, although this is not recommended.
|
||||
|
||||
HTTP/2 onwards does not include a reason phrase on the wire.
|
||||
|
||||
When no key is included, a default based on the status code may be used.
|
||||
|
||||
### `"stream_id"`
|
||||
|
||||
When HTTP/2 is being used the `"stream_id"` response extension can be accessed to determine the ID of the data stream that the response was sent on.
|
||||
|
||||
### `"network_stream"`
|
||||
|
||||
The `"network_stream"` extension allows developers to handle HTTP `CONNECT` and `Upgrade` requests, by providing an API that steps outside the standard request/response model, and can directly read or write to the network.
|
||||
|
||||
The interface provided by the network stream:
|
||||
|
||||
* `read(max_bytes, timeout = None) -> bytes`
|
||||
* `write(buffer, timeout = None)`
|
||||
* `close()`
|
||||
* `start_tls(ssl_context, server_hostname = None, timeout = None) -> NetworkStream`
|
||||
* `get_extra_info(info) -> Any`
|
||||
|
||||
This API can be used as the foundation for working with HTTP proxies, WebSocket upgrades, and other advanced use-cases.
|
||||
|
||||
See the [network backends documentation](https://www.encode.io/httpcore/network-backends/) for more information on working directly with network streams.
|
||||
|
||||
**Extra network information**
|
||||
|
||||
The network stream abstraction also allows access to various low-level information that may be exposed by the underlying socket:
|
||||
|
||||
```python
|
||||
response = httpx.get("https://www.example.com")
|
||||
network_stream = response.extensions["network_stream"]
|
||||
|
||||
client_addr = network_stream.get_extra_info("client_addr")
|
||||
server_addr = network_stream.get_extra_info("server_addr")
|
||||
print("Client address", client_addr)
|
||||
print("Server address", server_addr)
|
||||
```
|
||||
|
||||
The socket SSL information is also available through this interface, although you need to ensure that the underlying connection is still open, in order to access it...
|
||||
|
||||
```python
|
||||
with httpx.stream("GET", "https://www.example.com") as response:
|
||||
network_stream = response.extensions["network_stream"]
|
||||
|
||||
ssl_object = network_stream.get_extra_info("ssl_object")
|
||||
print("TLS version", ssl_object.version())
|
||||
```
|
||||
@ -1,83 +0,0 @@
|
||||
HTTPX supports setting up [HTTP proxies](https://en.wikipedia.org/wiki/Proxy_server#Web_proxy_servers) via the `proxy` parameter to be passed on client initialization or top-level API functions like `httpx.get(..., proxy=...)`.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://upload.wikimedia.org/wikipedia/commons/thumb/2/27/Open_proxy_h2g2bob.svg/480px-Open_proxy_h2g2bob.svg.png"/>
|
||||
<figcaption><em>Diagram of how a proxy works (source: Wikipedia). The left hand side "Internet" blob may be your HTTPX client requesting <code>example.com</code> through a proxy.</em></figcaption>
|
||||
</div>
|
||||
|
||||
## HTTP Proxies
|
||||
|
||||
To route all traffic (HTTP and HTTPS) to a proxy located at `http://localhost:8030`, pass the proxy URL to the client...
|
||||
|
||||
```python
|
||||
with httpx.Client(proxy="http://localhost:8030") as client:
|
||||
...
|
||||
```
|
||||
|
||||
For more advanced use cases, pass a mounts `dict`. For example, to route HTTP and HTTPS requests to 2 different proxies, respectively located at `http://localhost:8030`, and `http://localhost:8031`, pass a `dict` of proxy URLs:
|
||||
|
||||
```python
|
||||
proxy_mounts = {
|
||||
"http://": httpx.HTTPTransport(proxy="http://localhost:8030"),
|
||||
"https://": httpx.HTTPTransport(proxy="http://localhost:8031"),
|
||||
}
|
||||
|
||||
with httpx.Client(mounts=proxy_mounts) as client:
|
||||
...
|
||||
```
|
||||
|
||||
For detailed information about proxy routing, see the [Routing](#routing) section.
|
||||
|
||||
!!! tip "Gotcha"
|
||||
In most cases, the proxy URL for the `https://` key _should_ use the `http://` scheme (that's not a typo!).
|
||||
|
||||
This is because HTTP proxying requires initiating a connection with the proxy server. While it's possible that your proxy supports doing it via HTTPS, most proxies only support doing it via HTTP.
|
||||
|
||||
For more information, see [FORWARD vs TUNNEL](#forward-vs-tunnel).
|
||||
|
||||
## Authentication
|
||||
|
||||
Proxy credentials can be passed as the `userinfo` section of the proxy URL. For example:
|
||||
|
||||
```python
|
||||
with httpx.Client(proxy="http://username:password@localhost:8030") as client:
|
||||
...
|
||||
```
|
||||
|
||||
## Proxy mechanisms
|
||||
|
||||
!!! note
|
||||
This section describes **advanced** proxy concepts and functionality.
|
||||
|
||||
### FORWARD vs TUNNEL
|
||||
|
||||
In general, the flow for making an HTTP request through a proxy is as follows:
|
||||
|
||||
1. The client connects to the proxy (initial connection request).
|
||||
2. The proxy transfers data to the server on your behalf.
|
||||
|
||||
How exactly step 2/ is performed depends on which of two proxying mechanisms is used:
|
||||
|
||||
* **Forwarding**: the proxy makes the request for you, and sends back the response it obtained from the server.
|
||||
* **Tunnelling**: the proxy establishes a TCP connection to the server on your behalf, and the client reuses this connection to send the request and receive the response. This is known as an [HTTP Tunnel](https://en.wikipedia.org/wiki/HTTP_tunnel). This mechanism is how you can access websites that use HTTPS from an HTTP proxy (the client "upgrades" the connection to HTTPS by performing the TLS handshake with the server over the TCP connection provided by the proxy).
|
||||
|
||||
### Troubleshooting proxies
|
||||
|
||||
If you encounter issues when setting up proxies, please refer to our [Troubleshooting guide](../troubleshooting.md#proxies).
|
||||
|
||||
## SOCKS
|
||||
|
||||
In addition to HTTP proxies, `httpcore` also supports proxies using the SOCKS protocol.
|
||||
This is an optional feature that requires an additional third-party library be installed before use.
|
||||
|
||||
You can install SOCKS support using `pip`:
|
||||
|
||||
```shell
|
||||
$ pip install httpx[socks]
|
||||
```
|
||||
|
||||
You can now configure a client to make requests via a proxy using the SOCKS protocol:
|
||||
|
||||
```python
|
||||
httpx.Client(proxy='socks5://user:pass@host:port')
|
||||
```
|
||||
@ -1,13 +0,0 @@
|
||||
You can control the connection pool size using the `limits` keyword
|
||||
argument on the client. It takes instances of `httpx.Limits` which define:
|
||||
|
||||
- `max_keepalive_connections`, number of allowable keep-alive connections, or `None` to always
|
||||
allow. (Defaults 20)
|
||||
- `max_connections`, maximum number of allowable connections, or `None` for no limits.
|
||||
(Default 100)
|
||||
- `keepalive_expiry`, time limit on idle keep-alive connections in seconds, or `None` for no limits. (Default 5)
|
||||
|
||||
```python
|
||||
limits = httpx.Limits(max_keepalive_connections=5, max_connections=10)
|
||||
client = httpx.Client(limits=limits)
|
||||
```
|
||||
@ -1,89 +0,0 @@
|
||||
When making a request over HTTPS, HTTPX needs to verify the identity of the requested host. To do this, it uses a bundle of SSL certificates (a.k.a. CA bundle) delivered by a trusted certificate authority (CA).
|
||||
|
||||
### Enabling and disabling verification
|
||||
|
||||
By default httpx will verify HTTPS connections, and raise an error for invalid SSL cases...
|
||||
|
||||
```pycon
|
||||
>>> httpx.get("https://expired.badssl.com/")
|
||||
httpx.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)
|
||||
```
|
||||
|
||||
You can disable SSL verification completely and allow insecure requests...
|
||||
|
||||
```pycon
|
||||
>>> httpx.get("https://expired.badssl.com/", verify=False)
|
||||
<Response [200 OK]>
|
||||
```
|
||||
|
||||
### Configuring client instances
|
||||
|
||||
If you're using a `Client()` instance you should pass any `verify=<...>` configuration when instantiating the client.
|
||||
|
||||
By default the [certifi CA bundle](https://certifiio.readthedocs.io/en/latest/) is used for SSL verification.
|
||||
|
||||
For more complex configurations you can pass an [SSL Context](https://docs.python.org/3/library/ssl.html) instance...
|
||||
|
||||
```python
|
||||
import certifi
|
||||
import httpx
|
||||
import ssl
|
||||
|
||||
# This SSL context is equivalent to the default `verify=True`.
|
||||
ctx = ssl.create_default_context(cafile=certifi.where())
|
||||
client = httpx.Client(verify=ctx)
|
||||
```
|
||||
|
||||
Using [the `truststore` package](https://truststore.readthedocs.io/) to support system certificate stores...
|
||||
|
||||
```python
|
||||
import ssl
|
||||
import truststore
|
||||
import httpx
|
||||
|
||||
# Use system certificate stores.
|
||||
ctx = truststore.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
|
||||
client = httpx.Client(verify=ctx)
|
||||
```
|
||||
|
||||
Loding an alternative certificate verification store using [the standard SSL context API](https://docs.python.org/3/library/ssl.html)...
|
||||
|
||||
```python
|
||||
import httpx
|
||||
import ssl
|
||||
|
||||
# Use an explicitly configured certificate store.
|
||||
ctx = ssl.create_default_context(cafile="path/to/certs.pem") # Either cafile or capath.
|
||||
client = httpx.Client(verify=ctx)
|
||||
```
|
||||
|
||||
### Client side certificates
|
||||
|
||||
Client side certificates allow a remote server to verify the client. They tend to be used within private organizations to authenticate requests to remote servers.
|
||||
|
||||
You can specify client-side certificates, using the [`.load_cert_chain()`](https://docs.python.org/3/library/ssl.html#ssl.SSLContext.load_cert_chain) API...
|
||||
|
||||
```python
|
||||
ctx = ssl.create_default_context()
|
||||
ctx.load_cert_chain(certfile="path/to/client.pem") # Optionally also keyfile or password.
|
||||
client = httpx.Client(verify=ctx)
|
||||
```
|
||||
|
||||
### Working with `SSL_CERT_FILE` and `SSL_CERT_DIR`
|
||||
|
||||
`httpx` does respect the `SSL_CERT_FILE` and `SSL_CERT_DIR` environment variables by default. For details, refer to [the section on the environment variables page](../environment_variables.md#ssl_cert_file).
|
||||
|
||||
### Making HTTPS requests to a local server
|
||||
|
||||
When making requests to local servers, such as a development server running on `localhost`, you will typically be using unencrypted HTTP connections.
|
||||
|
||||
If you do need to make HTTPS connections to a local server, for example to test an HTTPS-only service, you will need to create and use your own certificates. Here's one way to do it...
|
||||
|
||||
1. Use [trustme](https://github.com/python-trio/trustme) to generate a pair of server key/cert files, and a client cert file.
|
||||
2. Pass the server key/cert files when starting your local server. (This depends on the particular web server you're using. For example, [Uvicorn](https://www.uvicorn.org) provides the `--ssl-keyfile` and `--ssl-certfile` options.)
|
||||
3. Configure `httpx` to use the certificates stored in `client.pem`.
|
||||
|
||||
```python
|
||||
ctx = ssl.create_default_context(cafile="client.pem")
|
||||
client = httpx.Client(verify=ctx)
|
||||
```
|
||||
@ -1,75 +0,0 @@
|
||||
When accessing `response.text`, we need to decode the response bytes into a unicode text representation.
|
||||
|
||||
By default `httpx` will use `"charset"` information included in the response `Content-Type` header to determine how the response bytes should be decoded into text.
|
||||
|
||||
In cases where no charset information is included on the response, the default behaviour is to assume "utf-8" encoding, which is by far the most widely used text encoding on the internet.
|
||||
|
||||
## Using the default encoding
|
||||
|
||||
To understand this better let's start by looking at the default behaviour for text decoding...
|
||||
|
||||
```python
|
||||
import httpx
|
||||
# Instantiate a client with the default configuration.
|
||||
client = httpx.Client()
|
||||
# Using the client...
|
||||
response = client.get(...)
|
||||
print(response.encoding) # This will either print the charset given in
|
||||
# the Content-Type charset, or else "utf-8".
|
||||
print(response.text) # The text will either be decoded with the Content-Type
|
||||
# charset, or using "utf-8".
|
||||
```
|
||||
|
||||
This is normally absolutely fine. Most servers will respond with a properly formatted Content-Type header, including a charset encoding. And in most cases where no charset encoding is included, UTF-8 is very likely to be used, since it is so widely adopted.
|
||||
|
||||
## Using an explicit encoding
|
||||
|
||||
In some cases we might be making requests to a site where no character set information is being set explicitly by the server, but we know what the encoding is. In this case it's best to set the default encoding explicitly on the client.
|
||||
|
||||
```python
|
||||
import httpx
|
||||
# Instantiate a client with a Japanese character set as the default encoding.
|
||||
client = httpx.Client(default_encoding="shift-jis")
|
||||
# Using the client...
|
||||
response = client.get(...)
|
||||
print(response.encoding) # This will either print the charset given in
|
||||
# the Content-Type charset, or else "shift-jis".
|
||||
print(response.text) # The text will either be decoded with the Content-Type
|
||||
# charset, or using "shift-jis".
|
||||
```
|
||||
|
||||
## Using auto-detection
|
||||
|
||||
In cases where the server is not reliably including character set information, and where we don't know what encoding is being used, we can enable auto-detection to make a best-guess attempt when decoding from bytes to text.
|
||||
|
||||
To use auto-detection you need to set the `default_encoding` argument to a callable instead of a string. This callable should be a function which takes the input bytes as an argument and returns the character set to use for decoding those bytes to text.
|
||||
|
||||
There are two widely used Python packages which both handle this functionality:
|
||||
|
||||
* [`chardet`](https://chardet.readthedocs.io/) - This is a well established package, and is a port of [the auto-detection code in Mozilla](https://www-archive.mozilla.org/projects/intl/chardet.html).
|
||||
* [`charset-normalizer`](https://charset-normalizer.readthedocs.io/) - A newer package, motivated by `chardet`, with a different approach.
|
||||
|
||||
Let's take a look at installing autodetection using one of these packages...
|
||||
|
||||
```shell
|
||||
$ pip install httpx
|
||||
$ pip install chardet
|
||||
```
|
||||
|
||||
Once `chardet` is installed, we can configure a client to use character-set autodetection.
|
||||
|
||||
```python
|
||||
import httpx
|
||||
import chardet
|
||||
|
||||
def autodetect(content):
|
||||
return chardet.detect(content).get("encoding")
|
||||
|
||||
# Using a client with character-set autodetection enabled.
|
||||
client = httpx.Client(default_encoding=autodetect)
|
||||
response = client.get(...)
|
||||
print(response.encoding) # This will either print the charset given in
|
||||
# the Content-Type charset, or else the auto-detected
|
||||
# character set.
|
||||
print(response.text)
|
||||
```
|
||||
@ -1,71 +0,0 @@
|
||||
HTTPX is careful to enforce timeouts everywhere by default.
|
||||
|
||||
The default behavior is to raise a `TimeoutException` after 5 seconds of
|
||||
network inactivity.
|
||||
|
||||
## Setting and disabling timeouts
|
||||
|
||||
You can set timeouts for an individual request:
|
||||
|
||||
```python
|
||||
# Using the top-level API:
|
||||
httpx.get('http://example.com/api/v1/example', timeout=10.0)
|
||||
|
||||
# Using a client instance:
|
||||
with httpx.Client() as client:
|
||||
client.get("http://example.com/api/v1/example", timeout=10.0)
|
||||
```
|
||||
|
||||
Or disable timeouts for an individual request:
|
||||
|
||||
```python
|
||||
# Using the top-level API:
|
||||
httpx.get('http://example.com/api/v1/example', timeout=None)
|
||||
|
||||
# Using a client instance:
|
||||
with httpx.Client() as client:
|
||||
client.get("http://example.com/api/v1/example", timeout=None)
|
||||
```
|
||||
|
||||
## Setting a default timeout on a client
|
||||
|
||||
You can set a timeout on a client instance, which results in the given
|
||||
`timeout` being used as the default for requests made with this client:
|
||||
|
||||
```python
|
||||
client = httpx.Client() # Use a default 5s timeout everywhere.
|
||||
client = httpx.Client(timeout=10.0) # Use a default 10s timeout everywhere.
|
||||
client = httpx.Client(timeout=None) # Disable all timeouts by default.
|
||||
```
|
||||
|
||||
## Fine tuning the configuration
|
||||
|
||||
HTTPX also allows you to specify the timeout behavior in more fine grained detail.
|
||||
|
||||
There are four different types of timeouts that may occur. These are **connect**,
|
||||
**read**, **write**, and **pool** timeouts.
|
||||
|
||||
* The **connect** timeout specifies the maximum amount of time to wait until
|
||||
a socket connection to the requested host is established. If HTTPX is unable to connect
|
||||
within this time frame, a `ConnectTimeout` exception is raised.
|
||||
* The **read** timeout specifies the maximum duration to wait for a chunk of
|
||||
data to be received (for example, a chunk of the response body). If HTTPX is
|
||||
unable to receive data within this time frame, a `ReadTimeout` exception is raised.
|
||||
* The **write** timeout specifies the maximum duration to wait for a chunk of
|
||||
data to be sent (for example, a chunk of the request body). If HTTPX is unable
|
||||
to send data within this time frame, a `WriteTimeout` exception is raised.
|
||||
* The **pool** timeout specifies the maximum duration to wait for acquiring
|
||||
a connection from the connection pool. If HTTPX is unable to acquire a connection
|
||||
within this time frame, a `PoolTimeout` exception is raised. A related
|
||||
configuration here is the maximum number of allowable connections in the
|
||||
connection pool, which is configured by the `limits` argument.
|
||||
|
||||
You can configure the timeout behavior for any of these values...
|
||||
|
||||
```python
|
||||
# A client with a 60s timeout for connecting, and a 10s timeout elsewhere.
|
||||
timeout = httpx.Timeout(10.0, connect=60.0)
|
||||
client = httpx.Client(timeout=timeout)
|
||||
|
||||
response = client.get('http://example.com/')
|
||||
```
|
||||
@ -1,454 +0,0 @@
|
||||
HTTPX's `Client` also accepts a `transport` argument. This argument allows you
|
||||
to provide a custom Transport object that will be used to perform the actual
|
||||
sending of the requests.
|
||||
|
||||
## HTTP Transport
|
||||
|
||||
For some advanced configuration you might need to instantiate a transport
|
||||
class directly, and pass it to the client instance. One example is the
|
||||
`local_address` configuration which is only available via this low-level API.
|
||||
|
||||
```pycon
|
||||
>>> import httpx
|
||||
>>> transport = httpx.HTTPTransport(local_address="0.0.0.0")
|
||||
>>> client = httpx.Client(transport=transport)
|
||||
```
|
||||
|
||||
Connection retries are also available via this interface. Requests will be retried the given number of times in case an `httpx.ConnectError` or an `httpx.ConnectTimeout` occurs, allowing smoother operation under flaky networks. If you need other forms of retry behaviors, such as handling read/write errors or reacting to `503 Service Unavailable`, consider general-purpose tools such as [tenacity](https://github.com/jd/tenacity).
|
||||
|
||||
```pycon
|
||||
>>> import httpx
|
||||
>>> transport = httpx.HTTPTransport(retries=1)
|
||||
>>> client = httpx.Client(transport=transport)
|
||||
```
|
||||
|
||||
Similarly, instantiating a transport directly provides a `uds` option for
|
||||
connecting via a Unix Domain Socket that is only available via this low-level API:
|
||||
|
||||
```pycon
|
||||
>>> import httpx
|
||||
>>> # Connect to the Docker API via a Unix Socket.
|
||||
>>> transport = httpx.HTTPTransport(uds="/var/run/docker.sock")
|
||||
>>> client = httpx.Client(transport=transport)
|
||||
>>> response = client.get("http://docker/info")
|
||||
>>> response.json()
|
||||
{"ID": "...", "Containers": 4, "Images": 74, ...}
|
||||
```
|
||||
|
||||
## WSGI Transport
|
||||
|
||||
You can configure an `httpx` client to call directly into a Python web application using the WSGI protocol.
|
||||
|
||||
This is particularly useful for two main use-cases:
|
||||
|
||||
* Using `httpx` as a client inside test cases.
|
||||
* Mocking out external services during tests or in dev or staging environments.
|
||||
|
||||
### Example
|
||||
|
||||
Here's an example of integrating against a Flask application:
|
||||
|
||||
```python
|
||||
from flask import Flask
|
||||
import httpx
|
||||
|
||||
|
||||
app = Flask(__name__)
|
||||
|
||||
@app.route("/")
|
||||
def hello():
|
||||
return "Hello World!"
|
||||
|
||||
transport = httpx.WSGITransport(app=app)
|
||||
with httpx.Client(transport=transport, base_url="http://testserver") as client:
|
||||
r = client.get("/")
|
||||
assert r.status_code == 200
|
||||
assert r.text == "Hello World!"
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
For some more complex cases you might need to customize the WSGI transport. This allows you to:
|
||||
|
||||
* Inspect 500 error responses rather than raise exceptions by setting `raise_app_exceptions=False`.
|
||||
* Mount the WSGI application at a subpath by setting `script_name` (WSGI).
|
||||
* Use a given client address for requests by setting `remote_addr` (WSGI).
|
||||
|
||||
For example:
|
||||
|
||||
```python
|
||||
# Instantiate a client that makes WSGI requests with a client IP of "1.2.3.4".
|
||||
transport = httpx.WSGITransport(app=app, remote_addr="1.2.3.4")
|
||||
with httpx.Client(transport=transport, base_url="http://testserver") as client:
|
||||
...
|
||||
```
|
||||
|
||||
## ASGI Transport
|
||||
|
||||
You can configure an `httpx` client to call directly into an async Python web application using the ASGI protocol.
|
||||
|
||||
This is particularly useful for two main use-cases:
|
||||
|
||||
* Using `httpx` as a client inside test cases.
|
||||
* Mocking out external services during tests or in dev or staging environments.
|
||||
|
||||
### Example
|
||||
|
||||
Let's take this Starlette application as an example:
|
||||
|
||||
```python
|
||||
from starlette.applications import Starlette
|
||||
from starlette.responses import HTMLResponse
|
||||
from starlette.routing import Route
|
||||
|
||||
|
||||
async def hello(request):
|
||||
return HTMLResponse("Hello World!")
|
||||
|
||||
|
||||
app = Starlette(routes=[Route("/", hello)])
|
||||
```
|
||||
|
||||
We can make requests directly against the application, like so:
|
||||
|
||||
```python
|
||||
transport = httpx.ASGITransport(app=app)
|
||||
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as client:
|
||||
r = await client.get("/")
|
||||
assert r.status_code == 200
|
||||
assert r.text == "Hello World!"
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
For some more complex cases you might need to customise the ASGI transport. This allows you to:
|
||||
|
||||
* Inspect 500 error responses rather than raise exceptions by setting `raise_app_exceptions=False`.
|
||||
* Mount the ASGI application at a subpath by setting `root_path`.
|
||||
* Use a given client address for requests by setting `client`.
|
||||
|
||||
For example:
|
||||
|
||||
```python
|
||||
# Instantiate a client that makes ASGI requests with a client IP of "1.2.3.4",
|
||||
# on port 123.
|
||||
transport = httpx.ASGITransport(app=app, client=("1.2.3.4", 123))
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as client:
|
||||
...
|
||||
```
|
||||
|
||||
See [the ASGI documentation](https://asgi.readthedocs.io/en/latest/specs/www.html#connection-scope) for more details on the `client` and `root_path` keys.
|
||||
|
||||
### ASGI startup and shutdown
|
||||
|
||||
It is not in the scope of HTTPX to trigger ASGI lifespan events of your app.
|
||||
|
||||
However it is suggested to use `LifespanManager` from [asgi-lifespan](https://github.com/florimondmanca/asgi-lifespan#usage) in pair with `AsyncClient`.
|
||||
|
||||
## Custom transports
|
||||
|
||||
A transport instance must implement the low-level Transport API which deals
|
||||
with sending a single request, and returning a response. You should either
|
||||
subclass `httpx.BaseTransport` to implement a transport to use with `Client`,
|
||||
or subclass `httpx.AsyncBaseTransport` to implement a transport to
|
||||
use with `AsyncClient`.
|
||||
|
||||
At the layer of the transport API we're using the familiar `Request` and
|
||||
`Response` models.
|
||||
|
||||
See the `handle_request` and `handle_async_request` docstrings for more details
|
||||
on the specifics of the Transport API.
|
||||
|
||||
A complete example of a custom transport implementation would be:
|
||||
|
||||
```python
|
||||
import json
|
||||
import httpx
|
||||
|
||||
class HelloWorldTransport(httpx.BaseTransport):
|
||||
"""
|
||||
A mock transport that always returns a JSON "Hello, world!" response.
|
||||
"""
|
||||
|
||||
def handle_request(self, request):
|
||||
return httpx.Response(200, json={"text": "Hello, world!"})
|
||||
```
|
||||
|
||||
Or this example, which uses a custom transport and `httpx.Mounts` to always redirect `http://` requests.
|
||||
|
||||
```python
|
||||
class HTTPSRedirect(httpx.BaseTransport):
|
||||
"""
|
||||
A transport that always redirects to HTTPS.
|
||||
"""
|
||||
def handle_request(self, request):
|
||||
url = request.url.copy_with(scheme="https")
|
||||
return httpx.Response(303, headers={"Location": str(url)})
|
||||
|
||||
# A client where any `http` requests are always redirected to `https`
|
||||
transport = httpx.Mounts({
|
||||
'http://': HTTPSRedirect()
|
||||
'https://': httpx.HTTPTransport()
|
||||
})
|
||||
client = httpx.Client(transport=transport)
|
||||
```
|
||||
|
||||
A useful pattern here is custom transport classes that wrap the default HTTP implementation. For example...
|
||||
|
||||
```python
|
||||
class DebuggingTransport(httpx.BaseTransport):
|
||||
def __init__(self, **kwargs):
|
||||
self._wrapper = httpx.HTTPTransport(**kwargs)
|
||||
|
||||
def handle_request(self, request):
|
||||
print(f">>> {request}")
|
||||
response = self._wrapper.handle_request(request)
|
||||
print(f"<<< {response}")
|
||||
return response
|
||||
|
||||
def close(self):
|
||||
self._wrapper.close()
|
||||
|
||||
transport = DebuggingTransport()
|
||||
client = httpx.Client(transport=transport)
|
||||
```
|
||||
|
||||
Here's another case, where we're using a round-robin across a number of different proxies...
|
||||
|
||||
```python
|
||||
class ProxyRoundRobin(httpx.BaseTransport):
|
||||
def __init__(self, proxies, **kwargs):
|
||||
self._transports = [
|
||||
httpx.HTTPTransport(proxy=proxy, **kwargs)
|
||||
for proxy in proxies
|
||||
]
|
||||
self._idx = 0
|
||||
|
||||
def handle_request(self, request):
|
||||
transport = self._transports[self._idx]
|
||||
self._idx = (self._idx + 1) % len(self._transports)
|
||||
return transport.handle_request(request)
|
||||
|
||||
def close(self):
|
||||
for transport in self._transports:
|
||||
transport.close()
|
||||
|
||||
proxies = [
|
||||
httpx.Proxy("http://127.0.0.1:8081"),
|
||||
httpx.Proxy("http://127.0.0.1:8082"),
|
||||
httpx.Proxy("http://127.0.0.1:8083"),
|
||||
]
|
||||
transport = ProxyRoundRobin(proxies=proxies)
|
||||
client = httpx.Client(transport=transport)
|
||||
```
|
||||
|
||||
## Mock transports
|
||||
|
||||
During testing it can often be useful to be able to mock out a transport,
|
||||
and return pre-determined responses, rather than making actual network requests.
|
||||
|
||||
The `httpx.MockTransport` class accepts a handler function, which can be used
|
||||
to map requests onto pre-determined responses:
|
||||
|
||||
```python
|
||||
def handler(request):
|
||||
return httpx.Response(200, json={"text": "Hello, world!"})
|
||||
|
||||
|
||||
# Switch to a mock transport, if the TESTING environment variable is set.
|
||||
if os.environ.get('TESTING', '').upper() == "TRUE":
|
||||
transport = httpx.MockTransport(handler)
|
||||
else:
|
||||
transport = httpx.HTTPTransport()
|
||||
|
||||
client = httpx.Client(transport=transport)
|
||||
```
|
||||
|
||||
For more advanced use-cases you might want to take a look at either [the third-party
|
||||
mocking library, RESPX](https://lundberg.github.io/respx/), or the [pytest-httpx library](https://github.com/Colin-b/pytest_httpx).
|
||||
|
||||
## Mounting transports
|
||||
|
||||
You can also mount transports against given schemes or domains, to control
|
||||
which transport an outgoing request should be routed via, with [the same style
|
||||
used for specifying proxy routing](#routing).
|
||||
|
||||
```python
|
||||
import httpx
|
||||
|
||||
class HTTPSRedirectTransport(httpx.BaseTransport):
|
||||
"""
|
||||
A transport that always redirects to HTTPS.
|
||||
"""
|
||||
|
||||
def handle_request(self, method, url, headers, stream, extensions):
|
||||
scheme, host, port, path = url
|
||||
if port is None:
|
||||
location = b"https://%s%s" % (host, path)
|
||||
else:
|
||||
location = b"https://%s:%d%s" % (host, port, path)
|
||||
stream = httpx.ByteStream(b"")
|
||||
headers = [(b"location", location)]
|
||||
extensions = {}
|
||||
return 303, headers, stream, extensions
|
||||
|
||||
|
||||
# A client where any `http` requests are always redirected to `https`
|
||||
mounts = {'http://': HTTPSRedirectTransport()}
|
||||
client = httpx.Client(mounts=mounts)
|
||||
```
|
||||
|
||||
A couple of other sketches of how you might take advantage of mounted transports...
|
||||
|
||||
Disabling HTTP/2 on a single given domain...
|
||||
|
||||
```python
|
||||
mounts = {
|
||||
"all://": httpx.HTTPTransport(http2=True),
|
||||
"all://*example.org": httpx.HTTPTransport()
|
||||
}
|
||||
client = httpx.Client(mounts=mounts)
|
||||
```
|
||||
|
||||
Mocking requests to a given domain:
|
||||
|
||||
```python
|
||||
# All requests to "example.org" should be mocked out.
|
||||
# Other requests occur as usual.
|
||||
def handler(request):
|
||||
return httpx.Response(200, json={"text": "Hello, World!"})
|
||||
|
||||
mounts = {"all://example.org": httpx.MockTransport(handler)}
|
||||
client = httpx.Client(mounts=mounts)
|
||||
```
|
||||
|
||||
Adding support for custom schemes:
|
||||
|
||||
```python
|
||||
# Support URLs like "file:///Users/sylvia_green/websites/new_client/index.html"
|
||||
mounts = {"file://": FileSystemTransport()}
|
||||
client = httpx.Client(mounts=mounts)
|
||||
```
|
||||
|
||||
### Routing
|
||||
|
||||
HTTPX provides a powerful mechanism for routing requests, allowing you to write complex rules that specify which transport should be used for each request.
|
||||
|
||||
The `mounts` dictionary maps URL patterns to HTTP transports. HTTPX matches requested URLs against URL patterns to decide which transport should be used, if any. Matching is done from most specific URL patterns (e.g. `https://<domain>:<port>`) to least specific ones (e.g. `https://`).
|
||||
|
||||
HTTPX supports routing requests based on **scheme**, **domain**, **port**, or a combination of these.
|
||||
|
||||
### Wildcard routing
|
||||
|
||||
Route everything through a transport...
|
||||
|
||||
```python
|
||||
mounts = {
|
||||
"all://": httpx.HTTPTransport(proxy="http://localhost:8030"),
|
||||
}
|
||||
```
|
||||
|
||||
### Scheme routing
|
||||
|
||||
Route HTTP requests through one transport, and HTTPS requests through another...
|
||||
|
||||
```python
|
||||
mounts = {
|
||||
"http://": httpx.HTTPTransport(proxy="http://localhost:8030"),
|
||||
"https://": httpx.HTTPTransport(proxy="http://localhost:8031"),
|
||||
}
|
||||
```
|
||||
|
||||
### Domain routing
|
||||
|
||||
Proxy all requests on domain "example.com", let other requests pass through...
|
||||
|
||||
```python
|
||||
mounts = {
|
||||
"all://example.com": httpx.HTTPTransport(proxy="http://localhost:8030"),
|
||||
}
|
||||
```
|
||||
|
||||
Proxy HTTP requests on domain "example.com", let HTTPS and other requests pass through...
|
||||
|
||||
```python
|
||||
mounts = {
|
||||
"http://example.com": httpx.HTTPTransport(proxy="http://localhost:8030"),
|
||||
}
|
||||
```
|
||||
|
||||
Proxy all requests to "example.com" and its subdomains, let other requests pass through...
|
||||
|
||||
```python
|
||||
mounts = {
|
||||
"all://*example.com": httpx.HTTPTransport(proxy="http://localhost:8030"),
|
||||
}
|
||||
```
|
||||
|
||||
Proxy all requests to strict subdomains of "example.com", let "example.com" and other requests pass through...
|
||||
|
||||
```python
|
||||
mounts = {
|
||||
"all://*.example.com": httpx.HTTPTransport(proxy="http://localhost:8030"),
|
||||
}
|
||||
```
|
||||
|
||||
### Port routing
|
||||
|
||||
Proxy HTTPS requests on port 1234 to "example.com"...
|
||||
|
||||
```python
|
||||
mounts = {
|
||||
"https://example.com:1234": httpx.HTTPTransport(proxy="http://localhost:8030"),
|
||||
}
|
||||
```
|
||||
|
||||
Proxy all requests on port 1234...
|
||||
|
||||
```python
|
||||
mounts = {
|
||||
"all://*:1234": httpx.HTTPTransport(proxy="http://localhost:8030"),
|
||||
}
|
||||
```
|
||||
|
||||
### No-proxy support
|
||||
|
||||
It is also possible to define requests that _shouldn't_ be routed through the transport.
|
||||
|
||||
To do so, pass `None` as the proxy URL. For example...
|
||||
|
||||
```python
|
||||
mounts = {
|
||||
# Route requests through a proxy by default...
|
||||
"all://": httpx.HTTPTransport(proxy="http://localhost:8031"),
|
||||
# Except those for "example.com".
|
||||
"all://example.com": None,
|
||||
}
|
||||
```
|
||||
|
||||
### Complex configuration example
|
||||
|
||||
You can combine the routing features outlined above to build complex proxy routing configurations. For example...
|
||||
|
||||
```python
|
||||
mounts = {
|
||||
# Route all traffic through a proxy by default...
|
||||
"all://": httpx.HTTPTransport(proxy="http://localhost:8030"),
|
||||
# But don't use proxies for HTTPS requests to "domain.io"...
|
||||
"https://domain.io": None,
|
||||
# And use another proxy for requests to "example.com" and its subdomains...
|
||||
"all://*example.com": httpx.HTTPTransport(proxy="http://localhost:8031"),
|
||||
# And yet another proxy if HTTP is used,
|
||||
# and the "internal" subdomain on port 5550 is requested...
|
||||
"http://internal.example.com:5550": httpx.HTTPTransport(proxy="http://localhost:8032"),
|
||||
}
|
||||
```
|
||||
|
||||
### Environment variables
|
||||
|
||||
There are also environment variables that can be used to control the dictionary of the client mounts.
|
||||
They can be used to configure HTTP proxying for clients.
|
||||
|
||||
See documentation on [`HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`](../environment_variables.md#http_proxy-https_proxy-all_proxy)
|
||||
and [`NO_PROXY`](../environment_variables.md#no_proxy) for more information.
|
||||
176
docs/api.md
176
docs/api.md
@ -1,176 +0,0 @@
|
||||
# Developer Interface
|
||||
|
||||
## Helper Functions
|
||||
|
||||
!!! note
|
||||
Only use these functions if you're testing HTTPX in a console
|
||||
or making a small number of requests. Using a `Client` will
|
||||
enable HTTP/2 and connection pooling for more efficient and
|
||||
long-lived connections.
|
||||
|
||||
::: httpx.request
|
||||
:docstring:
|
||||
|
||||
::: httpx.get
|
||||
:docstring:
|
||||
|
||||
::: httpx.options
|
||||
:docstring:
|
||||
|
||||
::: httpx.head
|
||||
:docstring:
|
||||
|
||||
::: httpx.post
|
||||
:docstring:
|
||||
|
||||
::: httpx.put
|
||||
:docstring:
|
||||
|
||||
::: httpx.patch
|
||||
:docstring:
|
||||
|
||||
::: httpx.delete
|
||||
:docstring:
|
||||
|
||||
::: httpx.stream
|
||||
:docstring:
|
||||
|
||||
## `Client`
|
||||
|
||||
::: httpx.Client
|
||||
:docstring:
|
||||
:members: headers cookies params auth request get head options post put patch delete stream build_request send close
|
||||
|
||||
## `AsyncClient`
|
||||
|
||||
::: httpx.AsyncClient
|
||||
:docstring:
|
||||
:members: headers cookies params auth request get head options post put patch delete stream build_request send aclose
|
||||
|
||||
|
||||
## `Response`
|
||||
|
||||
*An HTTP response.*
|
||||
|
||||
* `def __init__(...)`
|
||||
* `.status_code` - **int**
|
||||
* `.reason_phrase` - **str**
|
||||
* `.http_version` - `"HTTP/2"` or `"HTTP/1.1"`
|
||||
* `.url` - **URL**
|
||||
* `.headers` - **Headers**
|
||||
* `.content` - **bytes**
|
||||
* `.text` - **str**
|
||||
* `.encoding` - **str**
|
||||
* `.is_redirect` - **bool**
|
||||
* `.request` - **Request**
|
||||
* `.next_request` - **Optional[Request]**
|
||||
* `.cookies` - **Cookies**
|
||||
* `.history` - **List[Response]**
|
||||
* `.elapsed` - **[timedelta](https://docs.python.org/3/library/datetime.html)**
|
||||
* The amount of time elapsed between sending the request and calling `close()` on the corresponding response received for that request.
|
||||
[total_seconds()](https://docs.python.org/3/library/datetime.html#datetime.timedelta.total_seconds) to correctly get
|
||||
the total elapsed seconds.
|
||||
* `def .raise_for_status()` - **Response**
|
||||
* `def .json()` - **Any**
|
||||
* `def .read()` - **bytes**
|
||||
* `def .iter_raw([chunk_size])` - **bytes iterator**
|
||||
* `def .iter_bytes([chunk_size])` - **bytes iterator**
|
||||
* `def .iter_text([chunk_size])` - **text iterator**
|
||||
* `def .iter_lines()` - **text iterator**
|
||||
* `def .close()` - **None**
|
||||
* `def .next()` - **Response**
|
||||
* `def .aread()` - **bytes**
|
||||
* `def .aiter_raw([chunk_size])` - **async bytes iterator**
|
||||
* `def .aiter_bytes([chunk_size])` - **async bytes iterator**
|
||||
* `def .aiter_text([chunk_size])` - **async text iterator**
|
||||
* `def .aiter_lines()` - **async text iterator**
|
||||
* `def .aclose()` - **None**
|
||||
* `def .anext()` - **Response**
|
||||
|
||||
## `Request`
|
||||
|
||||
*An HTTP request. Can be constructed explicitly for more control over exactly
|
||||
what gets sent over the wire.*
|
||||
|
||||
```pycon
|
||||
>>> request = httpx.Request("GET", "https://example.org", headers={'host': 'example.org'})
|
||||
>>> response = client.send(request)
|
||||
```
|
||||
|
||||
* `def __init__(method, url, [params], [headers], [cookies], [content], [data], [files], [json], [stream])`
|
||||
* `.method` - **str**
|
||||
* `.url` - **URL**
|
||||
* `.content` - **byte**, **byte iterator**, or **byte async iterator**
|
||||
* `.headers` - **Headers**
|
||||
* `.cookies` - **Cookies**
|
||||
|
||||
## `URL`
|
||||
|
||||
*A normalized, IDNA supporting URL.*
|
||||
|
||||
```pycon
|
||||
>>> url = URL("https://example.org/")
|
||||
>>> url.host
|
||||
'example.org'
|
||||
```
|
||||
|
||||
* `def __init__(url, **kwargs)`
|
||||
* `.scheme` - **str**
|
||||
* `.authority` - **str**
|
||||
* `.host` - **str**
|
||||
* `.port` - **int**
|
||||
* `.path` - **str**
|
||||
* `.query` - **str**
|
||||
* `.raw_path` - **str**
|
||||
* `.fragment` - **str**
|
||||
* `.is_ssl` - **bool**
|
||||
* `.is_absolute_url` - **bool**
|
||||
* `.is_relative_url` - **bool**
|
||||
* `def .copy_with([scheme], [authority], [path], [query], [fragment])` - **URL**
|
||||
|
||||
## `Headers`
|
||||
|
||||
*A case-insensitive multi-dict.*
|
||||
|
||||
```pycon
|
||||
>>> headers = Headers({'Content-Type': 'application/json'})
|
||||
>>> headers['content-type']
|
||||
'application/json'
|
||||
```
|
||||
|
||||
* `def __init__(self, headers, encoding=None)`
|
||||
* `def copy()` - **Headers**
|
||||
|
||||
## `Cookies`
|
||||
|
||||
*A dict-like cookie store.*
|
||||
|
||||
```pycon
|
||||
>>> cookies = Cookies()
|
||||
>>> cookies.set("name", "value", domain="example.org")
|
||||
```
|
||||
|
||||
* `def __init__(cookies: [dict, Cookies, CookieJar])`
|
||||
* `.jar` - **CookieJar**
|
||||
* `def extract_cookies(response)`
|
||||
* `def set_cookie_header(request)`
|
||||
* `def set(name, value, [domain], [path])`
|
||||
* `def get(name, [domain], [path])`
|
||||
* `def delete(name, [domain], [path])`
|
||||
* `def clear([domain], [path])`
|
||||
* *Standard mutable mapping interface*
|
||||
|
||||
## `Proxy`
|
||||
|
||||
*A configuration of the proxy server.*
|
||||
|
||||
```pycon
|
||||
>>> proxy = Proxy("http://proxy.example.com:8030")
|
||||
>>> client = Client(proxy=proxy)
|
||||
```
|
||||
|
||||
* `def __init__(url, [ssl_context], [auth], [headers])`
|
||||
* `.url` - **URL**
|
||||
* `.auth` - **tuple[str, str]**
|
||||
* `.headers` - **Headers**
|
||||
* `.ssl_context` - **SSLContext**
|
||||
194
docs/async.md
194
docs/async.md
@ -1,194 +0,0 @@
|
||||
# Async Support
|
||||
|
||||
HTTPX offers a standard synchronous API by default, but also gives you
|
||||
the option of an async client if you need it.
|
||||
|
||||
Async is a concurrency model that is far more efficient than multi-threading,
|
||||
and can provide significant performance benefits and enable the use of
|
||||
long-lived network connections such as WebSockets.
|
||||
|
||||
If you're working with an async web framework then you'll also want to use an
|
||||
async client for sending outgoing HTTP requests.
|
||||
|
||||
## Making Async requests
|
||||
|
||||
To make asynchronous requests, you'll need an `AsyncClient`.
|
||||
|
||||
```pycon
|
||||
>>> async with httpx.AsyncClient() as client:
|
||||
... r = await client.get('https://www.example.com/')
|
||||
...
|
||||
>>> r
|
||||
<Response [200 OK]>
|
||||
```
|
||||
|
||||
!!! tip
|
||||
Use [IPython](https://ipython.readthedocs.io/en/stable/) or Python 3.9+ with `python -m asyncio` to try this code interactively, as they support executing `async`/`await` expressions in the console.
|
||||
|
||||
## API Differences
|
||||
|
||||
If you're using an async client then there are a few bits of API that
|
||||
use async methods.
|
||||
|
||||
### Making requests
|
||||
|
||||
The request methods are all async, so you should use `response = await client.get(...)` style for all of the following:
|
||||
|
||||
* `AsyncClient.get(url, ...)`
|
||||
* `AsyncClient.options(url, ...)`
|
||||
* `AsyncClient.head(url, ...)`
|
||||
* `AsyncClient.post(url, ...)`
|
||||
* `AsyncClient.put(url, ...)`
|
||||
* `AsyncClient.patch(url, ...)`
|
||||
* `AsyncClient.delete(url, ...)`
|
||||
* `AsyncClient.request(method, url, ...)`
|
||||
* `AsyncClient.send(request, ...)`
|
||||
|
||||
### Opening and closing clients
|
||||
|
||||
Use `async with httpx.AsyncClient()` if you want a context-managed client...
|
||||
|
||||
```python
|
||||
async with httpx.AsyncClient() as client:
|
||||
...
|
||||
```
|
||||
|
||||
!!! warning
|
||||
In order to get the most benefit from connection pooling, make sure you're not instantiating multiple client instances - for example by using `async with` inside a "hot loop". This can be achieved either by having a single scoped client that's passed throughout wherever it's needed, or by having a single global client instance.
|
||||
|
||||
Alternatively, use `await client.aclose()` if you want to close a client explicitly:
|
||||
|
||||
```python
|
||||
client = httpx.AsyncClient()
|
||||
...
|
||||
await client.aclose()
|
||||
```
|
||||
|
||||
### Streaming responses
|
||||
|
||||
The `AsyncClient.stream(method, url, ...)` method is an async context block.
|
||||
|
||||
```pycon
|
||||
>>> client = httpx.AsyncClient()
|
||||
>>> async with client.stream('GET', 'https://www.example.com/') as response:
|
||||
... async for chunk in response.aiter_bytes():
|
||||
... ...
|
||||
```
|
||||
|
||||
The async response streaming methods are:
|
||||
|
||||
* `Response.aread()` - For conditionally reading a response inside a stream block.
|
||||
* `Response.aiter_bytes()` - For streaming the response content as bytes.
|
||||
* `Response.aiter_text()` - For streaming the response content as text.
|
||||
* `Response.aiter_lines()` - For streaming the response content as lines of text.
|
||||
* `Response.aiter_raw()` - For streaming the raw response bytes, without applying content decoding.
|
||||
* `Response.aclose()` - For closing the response. You don't usually need this, since `.stream` block closes the response automatically on exit.
|
||||
|
||||
For situations when context block usage is not practical, it is possible to enter "manual mode" by sending a [`Request` instance](advanced/clients.md#request-instances) using `client.send(..., stream=True)`.
|
||||
|
||||
Example in the context of forwarding the response to a streaming web endpoint with [Starlette](https://www.starlette.io):
|
||||
|
||||
```python
|
||||
import httpx
|
||||
from starlette.background import BackgroundTask
|
||||
from starlette.responses import StreamingResponse
|
||||
|
||||
client = httpx.AsyncClient()
|
||||
|
||||
async def home(request):
|
||||
req = client.build_request("GET", "https://www.example.com/")
|
||||
r = await client.send(req, stream=True)
|
||||
return StreamingResponse(r.aiter_text(), background=BackgroundTask(r.aclose))
|
||||
```
|
||||
|
||||
!!! warning
|
||||
When using this "manual streaming mode", it is your duty as a developer to make sure that `Response.aclose()` is called eventually. Failing to do so would leave connections open, most likely resulting in resource leaks down the line.
|
||||
|
||||
### Streaming requests
|
||||
|
||||
When sending a streaming request body with an `AsyncClient` instance, you should use an async bytes generator instead of a bytes generator:
|
||||
|
||||
```python
|
||||
async def upload_bytes():
|
||||
... # yield byte content
|
||||
|
||||
await client.post(url, content=upload_bytes())
|
||||
```
|
||||
|
||||
### Explicit transport instances
|
||||
|
||||
When instantiating a transport instance directly, you need to use `httpx.AsyncHTTPTransport`.
|
||||
|
||||
For instance:
|
||||
|
||||
```pycon
|
||||
>>> import httpx
|
||||
>>> transport = httpx.AsyncHTTPTransport(retries=1)
|
||||
>>> async with httpx.AsyncClient(transport=transport) as client:
|
||||
>>> ...
|
||||
```
|
||||
|
||||
## Supported async environments
|
||||
|
||||
HTTPX supports either `asyncio` or `trio` as an async environment.
|
||||
|
||||
It will auto-detect which of those two to use as the backend
|
||||
for socket operations and concurrency primitives.
|
||||
|
||||
### [AsyncIO](https://docs.python.org/3/library/asyncio.html)
|
||||
|
||||
AsyncIO is Python's [built-in library](https://docs.python.org/3/library/asyncio.html)
|
||||
for writing concurrent code with the async/await syntax.
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
import httpx
|
||||
|
||||
async def main():
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get('https://www.example.com/')
|
||||
print(response)
|
||||
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
### [Trio](https://github.com/python-trio/trio)
|
||||
|
||||
Trio is [an alternative async library](https://trio.readthedocs.io/en/stable/),
|
||||
designed around the [the principles of structured concurrency](https://en.wikipedia.org/wiki/Structured_concurrency).
|
||||
|
||||
```python
|
||||
import httpx
|
||||
import trio
|
||||
|
||||
async def main():
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get('https://www.example.com/')
|
||||
print(response)
|
||||
|
||||
trio.run(main)
|
||||
```
|
||||
|
||||
!!! important
|
||||
The `trio` package must be installed to use the Trio backend.
|
||||
|
||||
|
||||
### [AnyIO](https://github.com/agronholm/anyio)
|
||||
|
||||
AnyIO is an [asynchronous networking and concurrency library](https://anyio.readthedocs.io/) that works on top of either `asyncio` or `trio`. It blends in with native libraries of your chosen backend (defaults to `asyncio`).
|
||||
|
||||
```python
|
||||
import httpx
|
||||
import anyio
|
||||
|
||||
async def main():
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get('https://www.example.com/')
|
||||
print(response)
|
||||
|
||||
anyio.run(main, backend='trio')
|
||||
```
|
||||
|
||||
## Calling into Python Web Apps
|
||||
|
||||
For details on calling directly into ASGI applications, see [the `ASGITransport` docs](../advanced/transports#asgitransport).
|
||||
311
docs/clients.md
Normal file
311
docs/clients.md
Normal file
@ -0,0 +1,311 @@
|
||||
# Clients
|
||||
|
||||
HTTP requests are sent by using a `Client` instance. Client instances are thread safe interfaces that maintain a pool of HTTP connections.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> cli = httpx.Client()
|
||||
>>> cli
|
||||
<Client [0 active]>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> cli = ahttpx.Client()
|
||||
>>> cli
|
||||
<Client [0 active]>
|
||||
```
|
||||
|
||||
The client representation provides an indication of how many connections are currently in the pool.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> r = cli.get("https://www.example.com")
|
||||
>>> r = cli.get("https://www.wikipedia.com")
|
||||
>>> r = cli.get("https://www.theguardian.com/uk")
|
||||
>>> cli
|
||||
<Client [0 active, 3 idle]>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> r = await cli.get("https://www.example.com")
|
||||
>>> r = await cli.get("https://www.wikipedia.com")
|
||||
>>> r = await cli.get("https://www.theguardian.com/uk")
|
||||
>>> cli
|
||||
<Client [0 active, 3 idle]>
|
||||
```
|
||||
|
||||
The connections in the pool can be explicitly closed, using the `close()` method...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> cli.close()
|
||||
>>> cli
|
||||
<Client [0 active]>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> await cli.close()
|
||||
>>> cli
|
||||
<Client [0 active]>
|
||||
```
|
||||
|
||||
Client instances support being used in a context managed scope. You can use this style to enforce properly scoped resources, ensuring that the connection pool is cleanly closed when no longer required.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> with httpx.Client() as cli:
|
||||
... r = cli.get("https://www.example.com")
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> async with ahttpx.Client() as cli:
|
||||
... r = await cli.get("https://www.example.com")
|
||||
```
|
||||
|
||||
It is important to scope the use of client instances as widely as possible.
|
||||
|
||||
Typically you should have a single client instance that is used throughout the lifespan of your application. This ensures that connection pooling is maximised, and minmises unneccessary reloading of SSL certificate stores.
|
||||
|
||||
The recommened usage is to *either* a have single global instance created at import time, *or* a single context scoped instance that is passed around wherever it is required.
|
||||
|
||||
## Setting a base URL
|
||||
|
||||
Client instances can be configured with a base URL that is used when constructing requests...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> with httpx.Client(url="https://www.httpbin.org") as cli:
|
||||
>>> r = cli.get("/json")
|
||||
>>> print(r)
|
||||
<Response [200 OK]>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> async with ahttpx.Client(url="https://www.httpbin.org") as cli:
|
||||
>>> r = cli.get("/json")
|
||||
>>> print(r)
|
||||
<Response [200 OK]>
|
||||
```
|
||||
|
||||
## Setting client headers
|
||||
|
||||
Client instances include a set of headers that are used on every outgoing request.
|
||||
|
||||
The default headers are:
|
||||
|
||||
* `Accept: */*` - Indicates to servers that any media type may be returned.
|
||||
* `Accept-Encoding: gzip` - Indicates to servers that gzip compression may be used on responses.
|
||||
* `Connection: keep-alive` - Indicates that HTTP/1.1 connections should be reused over multiple requests.
|
||||
* `User-Agent: python-httpx/1.0` - Identify the client as `httpx`.
|
||||
|
||||
You can override this behavior by explicitly specifying the default headers...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> headers = {"User-Agent": "dev", "Accept-Encoding": "gzip"}
|
||||
>>> with httpx.Client(headers=headers) as cli:
|
||||
>>> r = cli.get("https://www.example.com/")
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> headers = {"User-Agent": "dev", "Accept-Encoding": "gzip"}
|
||||
>>> async with ahttpx.Client(headers=headers) as cli:
|
||||
>>> r = await cli.get("https://www.example.com/")
|
||||
```
|
||||
|
||||
## Configuring the connection pool
|
||||
|
||||
The connection pool used by the client can be configured in order to customise the SSL context, the maximum number of concurrent connections, or the network backend.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> # Setup an SSL context to allow connecting to improperly configured SSL.
|
||||
>>> no_verify = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
|
||||
>>> no_verify.check_hostname = False
|
||||
>>> no_verify.verify_mode = ssl.CERT_NONE
|
||||
>>> # Instantiate a client with our custom SSL context.
|
||||
>>> pool = httpx.ConnectionPool(ssl_context=no_verify)
|
||||
>>> with httpx.Client(transport=pool) as cli:
|
||||
>>> ...
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> # Setup an SSL context to allow connecting to improperly configured SSL.
|
||||
>>> no_verify = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
|
||||
>>> no_verify.check_hostname = False
|
||||
>>> no_verify.verify_mode = ssl.CERT_NONE
|
||||
>>> # Instantiate a client with our custom SSL context.
|
||||
>>> pool = ahttpx.ConnectionPool(ssl_context=no_verify)
|
||||
>>> async with ahttpx.Client(transport=pool) as cli:
|
||||
>>> ...
|
||||
```
|
||||
|
||||
## Sending requests
|
||||
|
||||
* `.request()` - Send an HTTP request, reading the response to completion.
|
||||
* `.stream()` - Send an HTTP request, streaming the response.
|
||||
|
||||
Shortcut methods...
|
||||
|
||||
* `.get()` - Send an HTTP `GET` request.
|
||||
* `.post()` - Send an HTTP `POST` request.
|
||||
* `.put()` - Send an HTTP `PUT` request.
|
||||
* `.delete()` - Send an HTTP `DELETE` request.
|
||||
|
||||
---
|
||||
|
||||
## Transports
|
||||
|
||||
By default requests are sent using the `ConnectionPool` class. Alternative implementations for sending requests can be created by subclassing the `Transport` interface.
|
||||
|
||||
For example, a mock transport class that doesn't make any network requests and instead always returns a fixed response.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
class MockTransport(httpx.Transport):
|
||||
def __init__(self, response):
|
||||
self._response = response
|
||||
|
||||
@contextlib.contextmanager
|
||||
def send(self, request):
|
||||
yield response
|
||||
|
||||
def close(self):
|
||||
pass
|
||||
|
||||
response = httpx.Response(200, content=httpx.Text('Hello, world'))
|
||||
transport = MockTransport(response=response)
|
||||
with httpx.Client(transport=transport) as cli:
|
||||
r = cli.get('https://www.example.com')
|
||||
print(r)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
class MockTransport(ahttpx.Transport):
|
||||
def __init__(self, response):
|
||||
self._response = response
|
||||
|
||||
@contextlib.contextmanager
|
||||
def send(self, request):
|
||||
yield response
|
||||
|
||||
def close(self):
|
||||
pass
|
||||
|
||||
response = ahttpx.Response(200, content=httpx.Text('Hello, world'))
|
||||
transport = MockTransport(response=response)
|
||||
async with ahttpx.Client(transport=transport) as cli:
|
||||
r = await cli.get('https://www.example.com')
|
||||
print(r)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Middleware
|
||||
|
||||
In addition to maintaining an HTTP connection pool, client instances are responsible for two other pieces of functionality...
|
||||
|
||||
* Dealing with HTTP redirects.
|
||||
* Maintaining an HTTP cookie store.
|
||||
|
||||
### `RedirectMiddleware`
|
||||
|
||||
Wraps a transport class, adding support for HTTP redirect handling.
|
||||
|
||||
### `CookieMiddleware`
|
||||
|
||||
Wraps a transport class, adding support for HTTP cookie persistence.
|
||||
|
||||
---
|
||||
|
||||
## Custom client implementations
|
||||
|
||||
The `Client` implementation in `httpx` is intentionally lightweight.
|
||||
|
||||
If you're working with a large codebase you might want to create a custom client implementation in order to constrain the types of request that are sent.
|
||||
|
||||
The following example demonstrates a custom API client that only exposes `GET` and `POST` requests, and always uses JSON payloads.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
class APIClient:
|
||||
def __init__(self):
|
||||
self.url = httpx.URL('https://www.example.com')
|
||||
self.headers = httpx.Headers({
|
||||
'Accept-Encoding': 'gzip',
|
||||
'Connection': 'keep-alive',
|
||||
'User-Agent': 'dev'
|
||||
})
|
||||
self.via = httpx.RedirectMiddleware(httpx.ConnectionPool())
|
||||
|
||||
def get(self, path: str) -> Response:
|
||||
request = httpx.Request(
|
||||
method="GET",
|
||||
url=self.url.join(path),
|
||||
headers=self.headers,
|
||||
)
|
||||
with self.via.send(request) as response:
|
||||
response.read()
|
||||
return response
|
||||
|
||||
def post(self, path: str, payload: Any) -> httpx.Response:
|
||||
request = httpx.Request(
|
||||
method="POST",
|
||||
url=self.url.join(path),
|
||||
headers=self.headers,
|
||||
content=httpx.JSON(payload),
|
||||
)
|
||||
with self.via.send(request) as response:
|
||||
response.read()
|
||||
return response
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
class APIClient:
|
||||
def __init__(self):
|
||||
self.url = ahttpx.URL('https://www.example.com')
|
||||
self.headers = ahttpx.Headers({
|
||||
'Accept-Encoding': 'gzip',
|
||||
'Connection': 'keep-alive',
|
||||
'User-Agent': 'dev'
|
||||
})
|
||||
self.via = ahttpx.RedirectMiddleware(ahttpx.ConnectionPool())
|
||||
|
||||
async def get(self, path: str) -> Response:
|
||||
request = ahttpx.Request(
|
||||
method="GET",
|
||||
url=self.url.join(path),
|
||||
headers=self.headers,
|
||||
)
|
||||
async with self.via.send(request) as response:
|
||||
await response.read()
|
||||
return response
|
||||
|
||||
async def post(self, path: str, payload: Any) -> ahttpx.Response:
|
||||
request = ahttpx.Request(
|
||||
method="POST",
|
||||
url=self.url.join(path),
|
||||
headers=self.headers,
|
||||
content=httpx.JSON(payload),
|
||||
)
|
||||
async with self.via.send(request) as response:
|
||||
await response.read()
|
||||
return response
|
||||
```
|
||||
|
||||
You can expand on this pattern to provide behavior such as request or response schema validation, consistent timeouts, or standardised logging and exception handling.
|
||||
|
||||
---
|
||||
|
||||
<span class="link-prev">← [Quickstart](quickstart.md)</span>
|
||||
<span class="link-next">[Servers](servers.md) →</span>
|
||||
<span> </span>
|
||||
@ -1,56 +0,0 @@
|
||||
# Code of Conduct
|
||||
|
||||
We expect contributors to our projects and online spaces to follow [the Python Software Foundation’s Code of Conduct](https://www.python.org/psf/conduct/).
|
||||
|
||||
The Python community is made up of members from around the globe with a diverse set of skills, personalities, and experiences. It is through these differences that our community experiences great successes and continued growth. When you're working with members of the community, this Code of Conduct will help steer your interactions and keep Python a positive, successful, and growing community.
|
||||
|
||||
## Our Community
|
||||
|
||||
Members of the Python community are **open, considerate, and respectful**. Behaviours that reinforce these values contribute to a positive environment, and include:
|
||||
|
||||
* **Being open.** Members of the community are open to collaboration, whether it's on PEPs, patches, problems, or otherwise.
|
||||
* **Focusing on what is best for the community.** We're respectful of the processes set forth in the community, and we work within them.
|
||||
* **Acknowledging time and effort.** We're respectful of the volunteer efforts that permeate the Python community. We're thoughtful when addressing the efforts of others, keeping in mind that often times the labor was completed simply for the good of the community.
|
||||
* **Being respectful of differing viewpoints and experiences.** We're receptive to constructive comments and criticism, as the experiences and skill sets of other members contribute to the whole of our efforts.
|
||||
* **Showing empathy towards other community members.** We're attentive in our communications, whether in person or online, and we're tactful when approaching differing views.
|
||||
* **Being considerate.** Members of the community are considerate of their peers -- other Python users.
|
||||
* **Being respectful.** We're respectful of others, their positions, their skills, their commitments, and their efforts.
|
||||
* **Gracefully accepting constructive criticism.** When we disagree, we are courteous in raising our issues.
|
||||
* **Using welcoming and inclusive language.** We're accepting of all who wish to take part in our activities, fostering an environment where anyone can participate and everyone can make a difference.
|
||||
|
||||
## Our Standards
|
||||
|
||||
Every member of our community has the right to have their identity respected. The Python community is dedicated to providing a positive experience for everyone, regardless of age, gender identity and expression, sexual orientation, disability, physical appearance, body size, ethnicity, nationality, race, or religion (or lack thereof), education, or socio-economic status.
|
||||
|
||||
## Inappropriate Behavior
|
||||
|
||||
Examples of unacceptable behavior by participants include:
|
||||
|
||||
* Harassment of any participants in any form
|
||||
* Deliberate intimidation, stalking, or following
|
||||
* Logging or taking screenshots of online activity for harassment purposes
|
||||
* Publishing others' private information, such as a physical or electronic address, without explicit permission
|
||||
* Violent threats or language directed against another person
|
||||
* Incitement of violence or harassment towards any individual, including encouraging a person to commit suicide or to engage in self-harm
|
||||
* Creating additional online accounts in order to harass another person or circumvent a ban
|
||||
* Sexual language and imagery in online communities or in any conference venue, including talks
|
||||
* Insults, put downs, or jokes that are based upon stereotypes, that are exclusionary, or that hold others up for ridicule
|
||||
* Excessive swearing
|
||||
* Unwelcome sexual attention or advances
|
||||
* Unwelcome physical contact, including simulated physical contact (eg, textual descriptions like "hug" or "backrub") without consent or after a request to stop
|
||||
* Pattern of inappropriate social contact, such as requesting/assuming inappropriate levels of intimacy with others
|
||||
* Sustained disruption of online community discussions, in-person presentations, or other in-person events
|
||||
* Continued one-on-one communication after requests to cease
|
||||
* Other conduct that is inappropriate for a professional audience including people of many different backgrounds
|
||||
|
||||
Community members asked to stop any inappropriate behavior are expected to comply immediately.
|
||||
|
||||
## Enforcement
|
||||
|
||||
We take Code of Conduct violations seriously, and will act to ensure our spaces are welcoming, inclusive, and professional environments to communicate in.
|
||||
|
||||
If you need to raise a Code of Conduct report, you may do so privately by email to tom@tomchristie.com.
|
||||
|
||||
Reports will be treated confidentially.
|
||||
|
||||
Alternately you may [make a report to the Python Software Foundation](https://www.python.org/psf/conduct/reporting/).
|
||||
@ -1,232 +0,0 @@
|
||||
# Requests Compatibility Guide
|
||||
|
||||
HTTPX aims to be broadly compatible with the `requests` API, although there are a
|
||||
few design differences in places.
|
||||
|
||||
This documentation outlines places where the API differs...
|
||||
|
||||
## Redirects
|
||||
|
||||
Unlike `requests`, HTTPX does **not follow redirects by default**.
|
||||
|
||||
We differ in behaviour here [because auto-redirects can easily mask unnecessary network
|
||||
calls being made](https://github.com/encode/httpx/discussions/1785).
|
||||
|
||||
You can still enable behaviour to automatically follow redirects, but you need to
|
||||
do so explicitly...
|
||||
|
||||
```python
|
||||
response = client.get(url, follow_redirects=True)
|
||||
```
|
||||
|
||||
Or else instantiate a client, with redirect following enabled by default...
|
||||
|
||||
```python
|
||||
client = httpx.Client(follow_redirects=True)
|
||||
```
|
||||
|
||||
## Client instances
|
||||
|
||||
The HTTPX equivalent of `requests.Session` is `httpx.Client`.
|
||||
|
||||
```python
|
||||
session = requests.Session(**kwargs)
|
||||
```
|
||||
|
||||
is generally equivalent to
|
||||
|
||||
```python
|
||||
client = httpx.Client(**kwargs)
|
||||
```
|
||||
|
||||
## Request URLs
|
||||
|
||||
Accessing `response.url` will return a `URL` instance, rather than a string.
|
||||
|
||||
Use `str(response.url)` if you need a string instance.
|
||||
|
||||
## Determining the next redirect request
|
||||
|
||||
The `requests` library exposes an attribute `response.next`, which can be used to obtain the next redirect request.
|
||||
|
||||
```python
|
||||
session = requests.Session()
|
||||
request = requests.Request("GET", ...).prepare()
|
||||
while request is not None:
|
||||
response = session.send(request, allow_redirects=False)
|
||||
request = response.next
|
||||
```
|
||||
|
||||
In HTTPX, this attribute is instead named `response.next_request`. For example:
|
||||
|
||||
```python
|
||||
client = httpx.Client()
|
||||
request = client.build_request("GET", ...)
|
||||
while request is not None:
|
||||
response = client.send(request)
|
||||
request = response.next_request
|
||||
```
|
||||
|
||||
## Request Content
|
||||
|
||||
For uploading raw text or binary content we prefer to use a `content` parameter,
|
||||
in order to better separate this usage from the case of uploading form data.
|
||||
|
||||
For example, using `content=...` to upload raw content:
|
||||
|
||||
```python
|
||||
# Uploading text, bytes, or a bytes iterator.
|
||||
httpx.post(..., content=b"Hello, world")
|
||||
```
|
||||
|
||||
And using `data=...` to send form data:
|
||||
|
||||
```python
|
||||
# Uploading form data.
|
||||
httpx.post(..., data={"message": "Hello, world"})
|
||||
```
|
||||
|
||||
Using the `data=<text/byte content>` will raise a deprecation warning,
|
||||
and is expected to be fully removed with the HTTPX 1.0 release.
|
||||
|
||||
## Upload files
|
||||
|
||||
HTTPX strictly enforces that upload files must be opened in binary mode, in order
|
||||
to avoid character encoding issues that can result from attempting to upload files
|
||||
opened in text mode.
|
||||
|
||||
## Content encoding
|
||||
|
||||
HTTPX uses `utf-8` for encoding `str` request bodies. For example, when using `content=<str>` the request body will be encoded to `utf-8` before being sent over the wire. This differs from Requests which uses `latin1`. If you need an explicit encoding, pass encoded bytes explicitly, e.g. `content=<str>.encode("latin1")`.
|
||||
For response bodies, assuming the server didn't send an explicit encoding then HTTPX will do its best to figure out an appropriate encoding. HTTPX makes a guess at the encoding to use for decoding the response using `charset_normalizer`. Fallback to that or any content with less than 32 octets will be decoded using `utf-8` with the `error="replace"` decoder strategy.
|
||||
|
||||
## Cookies
|
||||
|
||||
If using a client instance, then cookies should always be set on the client rather than on a per-request basis.
|
||||
|
||||
This usage is supported:
|
||||
|
||||
```python
|
||||
client = httpx.Client(cookies=...)
|
||||
client.post(...)
|
||||
```
|
||||
|
||||
This usage is **not** supported:
|
||||
|
||||
```python
|
||||
client = httpx.Client()
|
||||
client.post(..., cookies=...)
|
||||
```
|
||||
|
||||
We prefer enforcing a stricter API here because it provides clearer expectations around cookie persistence, particularly when redirects occur.
|
||||
|
||||
## Status Codes
|
||||
|
||||
In our documentation we prefer the uppercased versions, such as `codes.NOT_FOUND`, but also provide lower-cased versions for API compatibility with `requests`.
|
||||
|
||||
Requests includes various synonyms for status codes that HTTPX does not support.
|
||||
|
||||
## Streaming responses
|
||||
|
||||
HTTPX provides a `.stream()` interface rather than using `stream=True`. This ensures that streaming responses are always properly closed outside of the stream block, and makes it visually clearer at which points streaming I/O APIs may be used with a response.
|
||||
|
||||
For example:
|
||||
|
||||
```python
|
||||
with httpx.stream("GET", "https://www.example.com") as response:
|
||||
...
|
||||
```
|
||||
|
||||
Within a `stream()` block request data is made available with:
|
||||
|
||||
* `.iter_bytes()` - Instead of `response.iter_content()`
|
||||
* `.iter_text()` - Instead of `response.iter_content(decode_unicode=True)`
|
||||
* `.iter_lines()` - Corresponding to `response.iter_lines()`
|
||||
* `.iter_raw()` - Use this instead of `response.raw`
|
||||
* `.read()` - Read the entire response body, making `response.text` and `response.content` available.
|
||||
|
||||
## Timeouts
|
||||
|
||||
HTTPX defaults to including reasonable [timeouts](quickstart.md#timeouts) for all network operations, while Requests has no timeouts by default.
|
||||
|
||||
To get the same behavior as Requests, set the `timeout` parameter to `None`:
|
||||
|
||||
```python
|
||||
httpx.get('https://www.example.com', timeout=None)
|
||||
```
|
||||
|
||||
## Proxy keys
|
||||
|
||||
HTTPX uses the mounts argument for HTTP proxying and transport routing.
|
||||
It can do much more than proxies and allows you to configure more than just the proxy route.
|
||||
For more detailed documentation, see [Mounting Transports](advanced/transports.md#mounting-transports).
|
||||
|
||||
When using `httpx.Client(mounts={...})` to map to a selection of different transports, we use full URL schemes, such as `mounts={"http://": ..., "https://": ...}`.
|
||||
|
||||
This is different to the `requests` usage of `proxies={"http": ..., "https": ...}`.
|
||||
|
||||
This change is for better consistency with more complex mappings, that might also include domain names, such as `mounts={"all://": ..., httpx.HTTPTransport(proxy="all://www.example.com": None})` which maps all requests onto a proxy, except for requests to "www.example.com" which have an explicit exclusion.
|
||||
|
||||
Also note that `requests.Session.request(...)` allows a `proxies=...` parameter, whereas `httpx.Client.request(...)` does not allow `mounts=...`.
|
||||
|
||||
## SSL configuration
|
||||
|
||||
When using a `Client` instance, the ssl configurations should always be passed on client instantiation, rather than passed to the request method.
|
||||
|
||||
If you need more than one different SSL configuration, you should use different client instances for each SSL configuration.
|
||||
|
||||
## Request body on HTTP methods
|
||||
|
||||
The HTTP `GET`, `DELETE`, `HEAD`, and `OPTIONS` methods are specified as not supporting a request body. To stay in line with this, the `.get`, `.delete`, `.head` and `.options` functions do not support `content`, `files`, `data`, or `json` arguments.
|
||||
|
||||
If you really do need to send request data using these http methods you should use the generic `.request` function instead.
|
||||
|
||||
```python
|
||||
httpx.request(
|
||||
method="DELETE",
|
||||
url="https://www.example.com/",
|
||||
content=b'A request body on a DELETE request.'
|
||||
)
|
||||
```
|
||||
|
||||
## Checking for success and failure responses
|
||||
|
||||
We don't support `response.is_ok` since the naming is ambiguous there, and might incorrectly imply an equivalence to `response.status_code == codes.OK`. Instead we provide the `response.is_success` property, which can be used to check for a 2xx response.
|
||||
|
||||
## Request instantiation
|
||||
|
||||
There is no notion of [prepared requests](https://requests.readthedocs.io/en/stable/user/advanced/#prepared-requests) in HTTPX. If you need to customize request instantiation, see [Request instances](advanced/clients.md#request-instances).
|
||||
|
||||
Besides, `httpx.Request()` does not support the `auth`, `timeout`, `follow_redirects`, `mounts`, `verify` and `cert` parameters. However these are available in `httpx.request`, `httpx.get`, `httpx.post` etc., as well as on [`Client` instances](advanced/clients.md#client-instances).
|
||||
|
||||
## Mocking
|
||||
|
||||
If you need to mock HTTPX the same way that test utilities like `responses` and `requests-mock` does for `requests`, see [RESPX](https://github.com/lundberg/respx).
|
||||
|
||||
## Caching
|
||||
|
||||
If you use `cachecontrol` or `requests-cache` to add HTTP Caching support to the `requests` library, you can use [Hishel](https://hishel.com) for HTTPX.
|
||||
|
||||
## Networking layer
|
||||
|
||||
`requests` defers most of its HTTP networking code to the excellent [`urllib3` library](https://urllib3.readthedocs.io/en/latest/).
|
||||
|
||||
On the other hand, HTTPX uses [HTTPCore](https://github.com/encode/httpcore) as its core HTTP networking layer, which is a different project than `urllib3`.
|
||||
|
||||
## Query Parameters
|
||||
|
||||
`requests` omits `params` whose values are `None` (e.g. `requests.get(..., params={"foo": None})`). This is not supported by HTTPX.
|
||||
|
||||
For both query params (`params=`) and form data (`data=`), `requests` supports sending a list of tuples (e.g. `requests.get(..., params=[('key1', 'value1'), ('key1', 'value2')])`). This is not supported by HTTPX. Instead, use a dictionary with lists as values. E.g.: `httpx.get(..., params={'key1': ['value1', 'value2']})` or with form data: `httpx.post(..., data={'key1': ['value1', 'value2']})`.
|
||||
|
||||
## Event Hooks
|
||||
|
||||
`requests` allows event hooks to mutate `Request` and `Response` objects. See [examples](https://requests.readthedocs.io/en/master/user/advanced/#event-hooks) given in the documentation for `requests`.
|
||||
|
||||
In HTTPX, event hooks may access properties of requests and responses, but event hook callbacks cannot mutate the original request/response.
|
||||
|
||||
If you are looking for more control, consider checking out [Custom Transports](advanced/transports.md#custom-transports).
|
||||
|
||||
## Exceptions and Errors
|
||||
|
||||
`requests` exception hierarchy is slightly different to the `httpx` exception hierarchy. `requests` exposes a top level `RequestException`, where as `httpx` exposes a top level `HTTPError`. see the exceptions exposes in requests [here](https://requests.readthedocs.io/en/latest/_modules/requests/exceptions/). See the `httpx` error hierarchy [here](https://www.python-httpx.org/exceptions/).
|
||||
245
docs/connections.md
Normal file
245
docs/connections.md
Normal file
@ -0,0 +1,245 @@
|
||||
# Connections
|
||||
|
||||
The mechanics of sending HTTP requests is dealt with by the `ConnectionPool` and `Connection` classes.
|
||||
|
||||
We can introspect a `Client` instance to get some visibility onto the state of the connection pool.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> with httpx.Client() as cli
|
||||
>>> urls = [
|
||||
... "https://www.wikipedia.org/",
|
||||
... "https://www.theguardian.com/",
|
||||
... "https://news.ycombinator.com/",
|
||||
... ]
|
||||
... for url in urls:
|
||||
... cli.get(url)
|
||||
... print(cli.transport)
|
||||
... # <ConnectionPool [3 idle]>
|
||||
... print(cli.transport.connections)
|
||||
... # [
|
||||
... # <Connection "https://www.wikipedia.org/" IDLE>,
|
||||
... # <Connection "https://www.theguardian.com/" IDLE>,
|
||||
... # <Connection "https://news.ycombinator.com/" IDLE>,
|
||||
... # ]
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> async with ahttpx.Client() as cli
|
||||
>>> urls = [
|
||||
... "https://www.wikipedia.org/",
|
||||
... "https://www.theguardian.com/",
|
||||
... "https://news.ycombinator.com/",
|
||||
... ]
|
||||
... for url in urls:
|
||||
... await cli.get(url)
|
||||
... print(cli.transport)
|
||||
... # <ConnectionPool [3 idle]>
|
||||
... print(cli.transport.connections)
|
||||
... # [
|
||||
... # <Connection "https://www.wikipedia.org/" IDLE>,
|
||||
... # <Connection "https://www.theguardian.com/" IDLE>,
|
||||
... # <Connection "https://news.ycombinator.com/" IDLE>,
|
||||
... # ]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Understanding the stack
|
||||
|
||||
The `Client` class is responsible for handling redirects and cookies.
|
||||
|
||||
It also ensures that outgoing requests include a default set of headers such as `User-Agent` and `Accept-Encoding`.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> with httpx.Client() as cli:
|
||||
>>> r = cli.request("GET", "https://www.example.com/")
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> async with ahttpx.Client() as cli:
|
||||
>>> r = await cli.request("GET", "https://www.example.com/")
|
||||
```
|
||||
|
||||
The `Client` class sends requests using a `ConnectionPool`, which is responsible for managing a pool of HTTP connections. This ensures quicker and more efficient use of resources than opening and closing a TCP connection with each request. The connection pool also handles HTTP proxying if required.
|
||||
|
||||
A single connection pool is able to handle multiple concurrent requests, with locking in place to ensure that the pool does not become over-saturated.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> with httpx.ConnectionPool() as pool:
|
||||
>>> r = pool.request("GET", "https://www.example.com/")
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> async with ahttpx.ConnectionPool() as pool:
|
||||
>>> r = await pool.request("GET", "https://www.example.com/")
|
||||
```
|
||||
|
||||
Individual HTTP connections can be managed directly with the `Connection` class. A single connection can only handle requests sequentially. Locking is provided to ensure that requests are strictly queued sequentially.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> with httpx.open_connection("https://www.example.com/") as conn:
|
||||
>>> r = conn.request("GET", "/")
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> async with ahttpx.open_connection("https://www.example.com/") as conn:
|
||||
>>> r = await conn.request("GET", "/")
|
||||
```
|
||||
|
||||
The `NetworkBackend` is responsible for managing the TCP stream, providing a raw byte-wise interface onto the underlying socket.
|
||||
|
||||
---
|
||||
|
||||
## ConnectionPool
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> pool = httpx.ConnectionPool()
|
||||
>>> pool
|
||||
<ConnectionPool [0 active]>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> pool = ahttpx.ConnectionPool()
|
||||
>>> pool
|
||||
<ConnectionPool [0 active]>
|
||||
```
|
||||
|
||||
### `.request(method, url, headers=None, content=None)`
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> with httpx.ConnectionPool() as pool:
|
||||
>>> res = pool.request("GET", "https://www.example.com")
|
||||
>>> res, pool
|
||||
<Response [200 OK]>, <ConnectionPool [1 idle]>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> async with ahttpx.ConnectionPool() as pool:
|
||||
>>> res = await pool.request("GET", "https://www.example.com")
|
||||
>>> res, pool
|
||||
<Response [200 OK]>, <ConnectionPool [1 idle]>
|
||||
```
|
||||
|
||||
### `.stream(method, url, headers=None, content=None)`
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> with httpx.ConnectionPool() as pool:
|
||||
>>> with pool.stream("GET", "https://www.example.com") as res:
|
||||
>>> res, pool
|
||||
<Response [200 OK]>, <ConnectionPool [1 active]>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> async with ahttpx.ConnectionPool() as pool:
|
||||
>>> async with await pool.stream("GET", "https://www.example.com") as res:
|
||||
>>> res, pool
|
||||
<Response [200 OK]>, <ConnectionPool [1 active]>
|
||||
```
|
||||
|
||||
### `.send(request)`
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> with httpx.ConnectionPool() as pool:
|
||||
>>> req = httpx.Request("GET", "https://www.example.com")
|
||||
>>> with pool.send(req) as res:
|
||||
>>> res.read()
|
||||
>>> res, pool
|
||||
<Response [200 OK]>, <ConnectionPool [1 idle]>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> async with ahttpx.ConnectionPool() as pool:
|
||||
>>> req = ahttpx.Request("GET", "https://www.example.com")
|
||||
>>> async with await pool.send(req) as res:
|
||||
>>> await res.read()
|
||||
>>> res, pool
|
||||
<Response [200 OK]>, <ConnectionPool [1 idle]>
|
||||
```
|
||||
|
||||
### `.close()`
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> with httpx.ConnectionPool() as pool:
|
||||
>>> pool.close()
|
||||
<ConnectionPool [0 active]>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> async with ahttpx.ConnectionPool() as pool:
|
||||
>>> await pool.close()
|
||||
<ConnectionPool [0 active]>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Connection
|
||||
|
||||
*TODO*
|
||||
|
||||
---
|
||||
|
||||
## Protocol upgrades
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
with httpx.open_connection("https://www.example.com/") as conn:
|
||||
with conn.upgrade("GET", "/feed", {"Upgrade": "WebSocket"}) as stream:
|
||||
...
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
async with await ahttpx.open_connection("https://www.example.com/") as conn:
|
||||
async with await conn.upgrade("GET", "/feed", {"Upgrade": "WebSocket"}) as stream:
|
||||
...
|
||||
```
|
||||
|
||||
`<Connection “https://www.example.com/feed” WEBSOCKET>`
|
||||
|
||||
## Proxy `CONNECT` requests
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
with httpx.open_connection("http://127.0.0.1:8080") as conn:
|
||||
with conn.upgrade("CONNECT", "www.encode.io:443") as stream:
|
||||
stream.start_tls(ctx, hostname="www.encode.io")
|
||||
...
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
async with await ahttpx.open_connection("http://127.0.0.1:8080") as conn:
|
||||
async with await conn.upgrade("CONNECT", "www.encode.io:443") as stream:
|
||||
await stream.start_tls(ctx, hostname="www.encode.io")
|
||||
...
|
||||
```
|
||||
|
||||
`<Connection "https://www.encode.io" VIA “http://127.0.0.1:8080” CONNECT>`
|
||||
|
||||
---
|
||||
|
||||
*Describe the `Transport` interface.*
|
||||
|
||||
---
|
||||
|
||||
<span class="link-prev">← [Streams](streams.md)</span>
|
||||
<span class="link-next">[Parsers](parsers.md) →</span>
|
||||
<span> </span>
|
||||
174
docs/content-types.md
Normal file
174
docs/content-types.md
Normal file
@ -0,0 +1,174 @@
|
||||
# Content Types
|
||||
|
||||
Some HTTP requests including `POST`, `PUT` and `PATCH` can include content in the body of the request.
|
||||
|
||||
The most common content types for upload data are...
|
||||
|
||||
* HTML form submissions use the `application/x-www-form-urlencoded` content type.
|
||||
* HTML form submissions including file uploads use the `multipart/form-data` content type.
|
||||
* JSON data uses the `application/json` content type.
|
||||
|
||||
Content can be included directly in a request by using bytes or a byte iterator and setting the appropriate `Content-Type` header.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> headers = {'Content-Type': 'application/json'}
|
||||
>>> content = json.dumps({"number": 123.5, "bool": [True, False], "text": "hello"})
|
||||
>>> response = cli.put(url, headers=headers, content=content)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> headers = {'Content-Type': 'application/json'}
|
||||
>>> content = json.dumps({"number": 123.5, "bool": [True, False], "text": "hello"})
|
||||
>>> response = await cli.put(url, headers=headers, content=content)
|
||||
```
|
||||
|
||||
There are also several classes provided for setting the request content. These implement either the `Content` or `StreamingContent` API, and handle constructing the content and setting the relevant headers.
|
||||
|
||||
* `<Form {“email”: “heya@noodles.com”}>`
|
||||
* `<Files {“upload”: File("README.md”)}>`
|
||||
* `<File “README.md” [123MB]>`
|
||||
* `<MultiPart {} {“upload”: File("README.md”)}>`
|
||||
* `<JSON {"number": 123.5, "bool": [True, False], ...}>`
|
||||
|
||||
For example, sending a JSON request...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> data = httpx.JSON({"number": 123.5, "bool": [True, False], "text": "hello"})
|
||||
>>> cli.post(url, content=data)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> data = httpx.JSON({"number": 123.5, "bool": [True, False], "text": "hello"})
|
||||
>>> await cli.post(url, content=data)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Form
|
||||
|
||||
The `Form` class provides an immutable multi-dict for accessing HTML form data. This class implements the `Content` interface, allowing for HTML form uploads.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> form = httpx.Form({'name': '...'})
|
||||
>>> form
|
||||
...
|
||||
>>> form['name']
|
||||
...
|
||||
>>> res = cli.post(url, content=form)
|
||||
...
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> form = httpx.Form({'name': '...'})
|
||||
>>> form
|
||||
...
|
||||
>>> form['name']
|
||||
...
|
||||
>>> res = await cli.post(url, content=form)
|
||||
...
|
||||
```
|
||||
|
||||
## Files
|
||||
|
||||
The `Files` class provides an immutable multi-dict for accessing HTML form file uploads. This class implements the `StreamingContent` interface, allowing for HTML form file uploads.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> files = httpx.Files({'upload': httpx.File('data.json')})
|
||||
>>> files
|
||||
...
|
||||
>>> files['upload']
|
||||
...
|
||||
>>> res = cli.post(url, content=files)
|
||||
...
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> files = httpx.Files({'upload': httpx.File('data.json')})
|
||||
>>> files
|
||||
...
|
||||
>>> files['upload']
|
||||
...
|
||||
>>> res = await cli.post(url, content=files)
|
||||
...
|
||||
```
|
||||
|
||||
## MultiPart
|
||||
|
||||
The `MultiPart` class provides a wrapper for HTML form and files uploads. This class implements the `StreamingContent` interface, allowing for allowing for HTML form uploads including both data and file uploads.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> multipart = httpx.MultiPart(form={'name': '...'}, files={'avatar': httpx.File('image.png')})
|
||||
>>> multipart.form['name']
|
||||
...
|
||||
>>> multipart.files['avatar']
|
||||
...
|
||||
>>> res = cli.post(url, content=multipart)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> multipart = httpx.MultiPart(form={'name': '...'}, files={'avatar': httpx.File('image.png')})
|
||||
>>> multipart.form['name']
|
||||
...
|
||||
>>> multipart.files['avatar']
|
||||
...
|
||||
>>> res = await cli.post(url, content=multipart)
|
||||
```
|
||||
|
||||
## File
|
||||
|
||||
The `File` class provides a wrapper for file uploads, and is used for uploads instead of passing a file object directly.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> file = httpx.File('upload.json')
|
||||
>>> cli.post(url, content=file)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> file = httpx.File('upload.json')
|
||||
>>> await cli.post(url, content=file)
|
||||
```
|
||||
|
||||
## JSON
|
||||
|
||||
The `JSON` class provides a wrapper for JSON uploads. This class implements the `Content` interface, allowing for HTTP JSON uploads.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> data = httpx.JSON({...})
|
||||
>>> cli.put(url, content=data)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> data = httpx.JSON({...})
|
||||
>>> await cli.put(url, content=data)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Content
|
||||
|
||||
An interface for constructing HTTP content, along with relevant headers.
|
||||
|
||||
The following method must be implemented...
|
||||
|
||||
* `.encode()` - Returns an `httx.Stream` representing the encoded data.
|
||||
* `.content_type()` - Returns a `str` indicating the content type.
|
||||
|
||||
---
|
||||
|
||||
<span class="link-prev">← [Headers](headers.md)</span>
|
||||
<span class="link-next">[Streams](streams.md) →</span>
|
||||
<span> </span>
|
||||
@ -1,232 +0,0 @@
|
||||
# Contributing
|
||||
|
||||
Thank you for being interested in contributing to HTTPX.
|
||||
There are many ways you can contribute to the project:
|
||||
|
||||
- Try HTTPX and [report bugs/issues you find](https://github.com/encode/httpx/issues/new)
|
||||
- [Implement new features](https://github.com/encode/httpx/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)
|
||||
- [Review Pull Requests of others](https://github.com/encode/httpx/pulls)
|
||||
- Write documentation
|
||||
- Participate in discussions
|
||||
|
||||
## Reporting Bugs or Other Issues
|
||||
|
||||
Found something that HTTPX should support?
|
||||
Stumbled upon some unexpected behaviour?
|
||||
|
||||
Contributions should generally start out with [a discussion](https://github.com/encode/httpx/discussions).
|
||||
Possible bugs may be raised as a "Potential Issue" discussion, feature requests may
|
||||
be raised as an "Ideas" discussion. We can then determine if the discussion needs
|
||||
to be escalated into an "Issue" or not, or if we'd consider a pull request.
|
||||
|
||||
Try to be more descriptive as you can and in case of a bug report,
|
||||
provide as much information as possible like:
|
||||
|
||||
- OS platform
|
||||
- Python version
|
||||
- Installed dependencies and versions (`python -m pip freeze`)
|
||||
- Code snippet
|
||||
- Error traceback
|
||||
|
||||
You should always try to reduce any examples to the *simplest possible case*
|
||||
that demonstrates the issue.
|
||||
|
||||
Some possibly useful tips for narrowing down potential issues...
|
||||
|
||||
- Does the issue exist on HTTP/1.1, or HTTP/2, or both?
|
||||
- Does the issue exist with `Client`, `AsyncClient`, or both?
|
||||
- When using `AsyncClient` does the issue exist when using `asyncio` or `trio`, or both?
|
||||
|
||||
## Development
|
||||
|
||||
To start developing HTTPX create a **fork** of the
|
||||
[HTTPX repository](https://github.com/encode/httpx) on GitHub.
|
||||
|
||||
Then clone your fork with the following command replacing `YOUR-USERNAME` with
|
||||
your GitHub username:
|
||||
|
||||
```shell
|
||||
$ git clone https://github.com/YOUR-USERNAME/httpx
|
||||
```
|
||||
|
||||
You can now install the project and its dependencies using:
|
||||
|
||||
```shell
|
||||
$ cd httpx
|
||||
$ scripts/install
|
||||
```
|
||||
|
||||
## Testing and Linting
|
||||
|
||||
We use custom shell scripts to automate testing, linting,
|
||||
and documentation building workflow.
|
||||
|
||||
To run the tests, use:
|
||||
|
||||
```shell
|
||||
$ scripts/test
|
||||
```
|
||||
|
||||
!!! warning
|
||||
The test suite spawns testing servers on ports **8000** and **8001**.
|
||||
Make sure these are not in use, so the tests can run properly.
|
||||
|
||||
Any additional arguments will be passed to `pytest`. See the [pytest documentation](https://docs.pytest.org/en/latest/how-to/usage.html) for more information.
|
||||
|
||||
For example, to run a single test script:
|
||||
|
||||
```shell
|
||||
$ scripts/test tests/test_multipart.py
|
||||
```
|
||||
|
||||
To run the code auto-formatting:
|
||||
|
||||
```shell
|
||||
$ scripts/lint
|
||||
```
|
||||
|
||||
Lastly, to run code checks separately (they are also run as part of `scripts/test`), run:
|
||||
|
||||
```shell
|
||||
$ scripts/check
|
||||
```
|
||||
|
||||
## Documenting
|
||||
|
||||
Documentation pages are located under the `docs/` folder.
|
||||
|
||||
To run the documentation site locally (useful for previewing changes), use:
|
||||
|
||||
```shell
|
||||
$ scripts/docs
|
||||
```
|
||||
|
||||
## Resolving Build / CI Failures
|
||||
|
||||
Once you've submitted your pull request, the test suite will automatically run, and the results will show up in GitHub.
|
||||
If the test suite fails, you'll want to click through to the "Details" link, and try to identify why the test suite failed.
|
||||
|
||||
<p align="center" style="margin: 0 0 10px">
|
||||
<img src="https://raw.githubusercontent.com/encode/httpx/master/docs/img/gh-actions-fail.png" alt='Failing PR commit status'>
|
||||
</p>
|
||||
|
||||
Here are some common ways the test suite can fail:
|
||||
|
||||
### Check Job Failed
|
||||
|
||||
<p align="center" style="margin: 0 0 10px">
|
||||
<img src="https://raw.githubusercontent.com/encode/httpx/master/docs/img/gh-actions-fail-check.png" alt='Failing GitHub action lint job'>
|
||||
</p>
|
||||
|
||||
This job failing means there is either a code formatting issue or type-annotation issue.
|
||||
You can look at the job output to figure out why it's failed or within a shell run:
|
||||
|
||||
```shell
|
||||
$ scripts/check
|
||||
```
|
||||
|
||||
It may be worth it to run `$ scripts/lint` to attempt auto-formatting the code
|
||||
and if that job succeeds commit the changes.
|
||||
|
||||
### Docs Job Failed
|
||||
|
||||
This job failing means the documentation failed to build. This can happen for
|
||||
a variety of reasons like invalid markdown or missing configuration within `mkdocs.yml`.
|
||||
|
||||
### Python 3.X Job Failed
|
||||
|
||||
<p align="center" style="margin: 0 0 10px">
|
||||
<img src="https://raw.githubusercontent.com/encode/httpx/master/docs/img/gh-actions-fail-test.png" alt='Failing GitHub action test job'>
|
||||
</p>
|
||||
|
||||
This job failing means the unit tests failed or not all code paths are covered by unit tests.
|
||||
|
||||
If tests are failing you will see this message under the coverage report:
|
||||
|
||||
`=== 1 failed, 435 passed, 1 skipped, 1 xfailed in 11.09s ===`
|
||||
|
||||
If tests succeed but coverage doesn't reach our current threshold, you will see this
|
||||
message under the coverage report:
|
||||
|
||||
`FAIL Required test coverage of 100% not reached. Total coverage: 99.00%`
|
||||
|
||||
## Releasing
|
||||
|
||||
*This section is targeted at HTTPX maintainers.*
|
||||
|
||||
Before releasing a new version, create a pull request that includes:
|
||||
|
||||
- **An update to the changelog**:
|
||||
- We follow the format from [keepachangelog](https://keepachangelog.com/en/1.0.0/).
|
||||
- [Compare](https://github.com/encode/httpx/compare/) `master` with the tag of the latest release, and list all entries that are of interest to our users:
|
||||
- Things that **must** go in the changelog: added, changed, deprecated or removed features, and bug fixes.
|
||||
- Things that **should not** go in the changelog: changes to documentation, tests or tooling.
|
||||
- Try sorting entries in descending order of impact / importance.
|
||||
- Keep it concise and to-the-point. 🎯
|
||||
- **A version bump**: see `__version__.py`.
|
||||
|
||||
For an example, see [#1006](https://github.com/encode/httpx/pull/1006).
|
||||
|
||||
Once the release PR is merged, create a
|
||||
[new release](https://github.com/encode/httpx/releases/new) including:
|
||||
|
||||
- Tag version like `0.13.3`.
|
||||
- Release title `Version 0.13.3`
|
||||
- Description copied from the changelog.
|
||||
|
||||
Once created this release will be automatically uploaded to PyPI.
|
||||
|
||||
If something goes wrong with the PyPI job the release can be published using the
|
||||
`scripts/publish` script.
|
||||
|
||||
## Development proxy setup
|
||||
|
||||
To test and debug requests via a proxy it's best to run a proxy server locally.
|
||||
Any server should do but HTTPCore's test suite uses
|
||||
[`mitmproxy`](https://mitmproxy.org/) which is written in Python, it's fully
|
||||
featured and has excellent UI and tools for introspection of requests.
|
||||
|
||||
You can install `mitmproxy` using `pip install mitmproxy` or [several
|
||||
other ways](https://docs.mitmproxy.org/stable/overview-installation/).
|
||||
|
||||
`mitmproxy` does require setting up local TLS certificates for HTTPS requests,
|
||||
as its main purpose is to allow developers to inspect requests that pass through
|
||||
it. We can set them up follows:
|
||||
|
||||
1. [`pip install trustme-cli`](https://github.com/sethmlarson/trustme-cli/).
|
||||
2. `trustme-cli -i example.org www.example.org`, assuming you want to test
|
||||
connecting to that domain, this will create three files: `server.pem`,
|
||||
`server.key` and `client.pem`.
|
||||
3. `mitmproxy` requires a PEM file that includes the private key and the
|
||||
certificate so we need to concatenate them:
|
||||
`cat server.key server.pem > server.withkey.pem`.
|
||||
4. Start the proxy server `mitmproxy --certs server.withkey.pem`, or use the
|
||||
[other mitmproxy commands](https://docs.mitmproxy.org/stable/) with different
|
||||
UI options.
|
||||
|
||||
At this point the server is ready to start serving requests, you'll need to
|
||||
configure HTTPX as described in the
|
||||
[proxy section](https://www.python-httpx.org/advanced/proxies/#http-proxies) and
|
||||
the [SSL certificates section](https://www.python-httpx.org/advanced/ssl/),
|
||||
this is where our previously generated `client.pem` comes in:
|
||||
|
||||
```python
|
||||
ctx = ssl.create_default_context(cafile="/path/to/client.pem")
|
||||
client = httpx.Client(proxy="http://127.0.0.1:8080/", verify=ctx)
|
||||
```
|
||||
|
||||
Note, however, that HTTPS requests will only succeed to the host specified
|
||||
in the SSL/TLS certificate we generated, HTTPS requests to other hosts will
|
||||
raise an error like:
|
||||
|
||||
```
|
||||
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate
|
||||
verify failed: Hostname mismatch, certificate is not valid for
|
||||
'duckduckgo.com'. (_ssl.c:1108)
|
||||
```
|
||||
|
||||
If you want to make requests to more hosts you'll need to regenerate the
|
||||
certificates and include all the hosts you intend to connect to in the
|
||||
seconds step, i.e.
|
||||
|
||||
`trustme-cli -i example.org www.example.org duckduckgo.com www.duckduckgo.com`
|
||||
@ -1,10 +0,0 @@
|
||||
div.autodoc-docstring {
|
||||
padding-left: 20px;
|
||||
margin-bottom: 30px;
|
||||
border-left: 5px solid rgba(230, 230, 230);
|
||||
}
|
||||
|
||||
div.autodoc-members {
|
||||
padding-left: 20px;
|
||||
margin-bottom: 15px;
|
||||
}
|
||||
@ -1,79 +0,0 @@
|
||||
# Environment Variables
|
||||
|
||||
The HTTPX library can be configured via environment variables.
|
||||
Environment variables are used by default. To ignore environment variables, `trust_env` has to be set `False`. There are two ways to set `trust_env` to disable environment variables:
|
||||
|
||||
* On the client via `httpx.Client(trust_env=False)`.
|
||||
* Using the top-level API, such as `httpx.get("<url>", trust_env=False)`.
|
||||
|
||||
Here is a list of environment variables that HTTPX recognizes and what function they serve:
|
||||
|
||||
## Proxies
|
||||
|
||||
The environment variables documented below are used as a convention by various HTTP tooling, including:
|
||||
|
||||
* [cURL](https://github.com/curl/curl/blob/master/docs/MANUAL.md#environment-variables)
|
||||
* [requests](https://github.com/psf/requests/blob/master/docs/user/advanced.rst#proxies)
|
||||
|
||||
For more information on using proxies in HTTPX, see [HTTP Proxying](advanced/proxies.md#http-proxying).
|
||||
|
||||
### `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`
|
||||
|
||||
Valid values: A URL to a proxy
|
||||
|
||||
`HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY` set the proxy to be used for `http`, `https`, or all requests respectively.
|
||||
|
||||
```bash
|
||||
export HTTP_PROXY=http://my-external-proxy.com:1234
|
||||
|
||||
# This request will be sent through the proxy
|
||||
python -c "import httpx; httpx.get('http://example.com')"
|
||||
|
||||
# This request will be sent directly, as we set `trust_env=False`
|
||||
python -c "import httpx; httpx.get('http://example.com', trust_env=False)"
|
||||
|
||||
```
|
||||
|
||||
### `NO_PROXY`
|
||||
|
||||
Valid values: a comma-separated list of hostnames/urls
|
||||
|
||||
`NO_PROXY` disables the proxy for specific urls
|
||||
|
||||
```bash
|
||||
export HTTP_PROXY=http://my-external-proxy.com:1234
|
||||
export NO_PROXY=http://127.0.0.1,python-httpx.org
|
||||
|
||||
# As in the previous example, this request will be sent through the proxy
|
||||
python -c "import httpx; httpx.get('http://example.com')"
|
||||
|
||||
# These requests will be sent directly, bypassing the proxy
|
||||
python -c "import httpx; httpx.get('http://127.0.0.1:5000/my-api')"
|
||||
python -c "import httpx; httpx.get('https://www.python-httpx.org')"
|
||||
```
|
||||
|
||||
## `SSL_CERT_FILE`
|
||||
|
||||
Valid values: a filename
|
||||
|
||||
If this environment variable is set then HTTPX will load
|
||||
CA certificate from the specified file instead of the default
|
||||
location.
|
||||
|
||||
Example:
|
||||
|
||||
```console
|
||||
SSL_CERT_FILE=/path/to/ca-certs/ca-bundle.crt python -c "import httpx; httpx.get('https://example.com')"
|
||||
```
|
||||
|
||||
## `SSL_CERT_DIR`
|
||||
|
||||
Valid values: a directory following an [OpenSSL specific layout](https://www.openssl.org/docs/manmaster/man3/SSL_CTX_load_verify_locations.html).
|
||||
|
||||
If this environment variable is set and the directory follows an [OpenSSL specific layout](https://www.openssl.org/docs/manmaster/man3/SSL_CTX_load_verify_locations.html) (ie. you ran `c_rehash`) then HTTPX will load CA certificates from this directory instead of the default location.
|
||||
|
||||
Example:
|
||||
|
||||
```console
|
||||
SSL_CERT_DIR=/path/to/ca-certs/ python -c "import httpx; httpx.get('https://example.com')"
|
||||
```
|
||||
@ -1,124 +0,0 @@
|
||||
# Exceptions
|
||||
|
||||
This page lists exceptions that may be raised when using HTTPX.
|
||||
|
||||
For an overview of how to work with HTTPX exceptions, see [Exceptions (Quickstart)](quickstart.md#exceptions).
|
||||
|
||||
## The exception hierarchy
|
||||
|
||||
* HTTPError
|
||||
* RequestError
|
||||
* TransportError
|
||||
* TimeoutException
|
||||
* ConnectTimeout
|
||||
* ReadTimeout
|
||||
* WriteTimeout
|
||||
* PoolTimeout
|
||||
* NetworkError
|
||||
* ConnectError
|
||||
* ReadError
|
||||
* WriteError
|
||||
* CloseError
|
||||
* ProtocolError
|
||||
* LocalProtocolError
|
||||
* RemoteProtocolError
|
||||
* ProxyError
|
||||
* UnsupportedProtocol
|
||||
* DecodingError
|
||||
* TooManyRedirects
|
||||
* HTTPStatusError
|
||||
* InvalidURL
|
||||
* CookieConflict
|
||||
* StreamError
|
||||
* StreamConsumed
|
||||
* ResponseNotRead
|
||||
* RequestNotRead
|
||||
* StreamClosed
|
||||
|
||||
---
|
||||
|
||||
## Exception classes
|
||||
|
||||
::: httpx.HTTPError
|
||||
:docstring:
|
||||
|
||||
::: httpx.RequestError
|
||||
:docstring:
|
||||
|
||||
::: httpx.TransportError
|
||||
:docstring:
|
||||
|
||||
::: httpx.TimeoutException
|
||||
:docstring:
|
||||
|
||||
::: httpx.ConnectTimeout
|
||||
:docstring:
|
||||
|
||||
::: httpx.ReadTimeout
|
||||
:docstring:
|
||||
|
||||
::: httpx.WriteTimeout
|
||||
:docstring:
|
||||
|
||||
::: httpx.PoolTimeout
|
||||
:docstring:
|
||||
|
||||
::: httpx.NetworkError
|
||||
:docstring:
|
||||
|
||||
::: httpx.ConnectError
|
||||
:docstring:
|
||||
|
||||
::: httpx.ReadError
|
||||
:docstring:
|
||||
|
||||
::: httpx.WriteError
|
||||
:docstring:
|
||||
|
||||
::: httpx.CloseError
|
||||
:docstring:
|
||||
|
||||
::: httpx.ProtocolError
|
||||
:docstring:
|
||||
|
||||
::: httpx.LocalProtocolError
|
||||
:docstring:
|
||||
|
||||
::: httpx.RemoteProtocolError
|
||||
:docstring:
|
||||
|
||||
::: httpx.ProxyError
|
||||
:docstring:
|
||||
|
||||
::: httpx.UnsupportedProtocol
|
||||
:docstring:
|
||||
|
||||
::: httpx.DecodingError
|
||||
:docstring:
|
||||
|
||||
::: httpx.TooManyRedirects
|
||||
:docstring:
|
||||
|
||||
::: httpx.HTTPStatusError
|
||||
:docstring:
|
||||
|
||||
::: httpx.InvalidURL
|
||||
:docstring:
|
||||
|
||||
::: httpx.CookieConflict
|
||||
:docstring:
|
||||
|
||||
::: httpx.StreamError
|
||||
:docstring:
|
||||
|
||||
::: httpx.StreamConsumed
|
||||
:docstring:
|
||||
|
||||
::: httpx.StreamClosed
|
||||
:docstring:
|
||||
|
||||
::: httpx.ResponseNotRead
|
||||
:docstring:
|
||||
|
||||
::: httpx.RequestNotRead
|
||||
:docstring:
|
||||
54
docs/headers.md
Normal file
54
docs/headers.md
Normal file
@ -0,0 +1,54 @@
|
||||
# Headers
|
||||
|
||||
The `Headers` class provides an immutable case-insensitive multidict interface for accessing HTTP headers.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> headers = httpx.Headers({"Accept": "*/*"})
|
||||
>>> headers
|
||||
<Headers {"Accept": "*/*"}>
|
||||
>>> headers['accept']
|
||||
'*/*'
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> headers = ahttpx.Headers({"Accept": "*/*"})
|
||||
>>> headers
|
||||
<Headers {"Accept": "*/*"}>
|
||||
>>> headers['accept']
|
||||
'*/*'
|
||||
```
|
||||
|
||||
Header values should always be printable ASCII strings. Attempting to set invalid header name or value strings will raise a `ValueError`.
|
||||
|
||||
### Accessing headers
|
||||
|
||||
Headers are accessed using a standard dictionary style interface...
|
||||
|
||||
* `.get(key, default=None)` - *Return the value for a given key, or a default value. If multiple values for the key are present, only the first will be returned.*
|
||||
* `.keys()` - *Return the unique keys of the headers. Each key will be a `str`.*
|
||||
* `.values()` - *Return the values of the headers. Each value will be a `str`. If multiple values for a key are present, only the first will be returned.*
|
||||
* `.items()` - *Return the key value pairs of the headers. Each item will be a two-tuple `(str, str)`. If multiple values for a key are present, only the first will be returned.*
|
||||
|
||||
The following methods are also available for accessing headers as a multidict...
|
||||
|
||||
* `.get_all(key, comma_delimited=False)` - *Return all the values for a given key. Returned as a list of zero or more `str` instances. If `comma_delimited` is set to `True` then any comma separated header values are split into a list of strings.*
|
||||
* `.multi_items()` - *Return the key value pairs of the headers. Each item will be a two-tuple `(str, str)`. Repeated keys may occur.*
|
||||
* `.multi_dict()` - *Return the headers as a dictionary, with each value being a list of one or more `str` instances.*
|
||||
|
||||
### Modifying headers
|
||||
|
||||
The following methods can be used to create modified header instances...
|
||||
|
||||
* `.copy_set(key, value)` - *Return a new `Headers` instances, setting a header. Eg. `headers = headers.copy_set("Connection": "close")`*.
|
||||
* `.copy_setdefault(key, value)` - *Return a new `Headers` instances, setting a header if it does not yet exist. Eg. `headers = headers.copy_setdefault("Content-Type": "text/html")`*.
|
||||
* `.copy_append(key, value, comma_delimited=False)` - *Return a new `Headers` instances, setting or appending a header. If `comma_delimited` is set to `True`, then the append will be handled using comma delimiting instead of creating a new header. Eg. `headers = headers.copy_append("Accept-Encoding", "gzip", comma_delimited=True)`*.
|
||||
* `.copy_remove(key)` - *Return a new `Headers` instances, removing a header. Eg. `headers = headers.copy_remove("User-Agent")`*.
|
||||
* `.copy_update(headers)` - *Return a new `Headers` instances, updating multiple headers. Eg. `headers = headers.copy_update({"Authorization": "top secret"})`*.
|
||||
|
||||
---
|
||||
|
||||
<span class="link-prev">← [URLs](urls.md)</span>
|
||||
<span class="link-next">[Content Types](content-types.md) →</span>
|
||||
<span> </span>
|
||||
@ -1,68 +0,0 @@
|
||||
# HTTP/2
|
||||
|
||||
HTTP/2 is a major new iteration of the HTTP protocol, that provides a far more
|
||||
efficient transport, with potential performance benefits. HTTP/2 does not change
|
||||
the core semantics of the request or response, but alters the way that data is
|
||||
sent to and from the server.
|
||||
|
||||
Rather than the text format that HTTP/1.1 uses, HTTP/2 is a binary format.
|
||||
The binary format provides full request and response multiplexing, and efficient
|
||||
compression of HTTP headers. The stream multiplexing means that where HTTP/1.1
|
||||
requires one TCP stream for each concurrent request, HTTP/2 allows a single TCP
|
||||
stream to handle multiple concurrent requests.
|
||||
|
||||
HTTP/2 also provides support for functionality such as response prioritization,
|
||||
and server push.
|
||||
|
||||
For a comprehensive guide to HTTP/2 you may want to check out "[http2 explained](https://http2-explained.haxx.se/)".
|
||||
|
||||
## Enabling HTTP/2
|
||||
|
||||
When using the `httpx` client, HTTP/2 support is not enabled by default, because
|
||||
HTTP/1.1 is a mature, battle-hardened transport layer, and our HTTP/1.1
|
||||
implementation may be considered the more robust option at this point in time.
|
||||
It is possible that a future version of `httpx` may enable HTTP/2 support by default.
|
||||
|
||||
If you're issuing highly concurrent requests you might want to consider
|
||||
trying out our HTTP/2 support. You can do so by first making sure to install
|
||||
the optional HTTP/2 dependencies...
|
||||
|
||||
```shell
|
||||
$ pip install httpx[http2]
|
||||
```
|
||||
|
||||
And then instantiating a client with HTTP/2 support enabled:
|
||||
|
||||
```python
|
||||
client = httpx.AsyncClient(http2=True)
|
||||
...
|
||||
```
|
||||
|
||||
You can also instantiate a client as a context manager, to ensure that all
|
||||
HTTP connections are nicely scoped, and will be closed once the context block
|
||||
is exited.
|
||||
|
||||
```python
|
||||
async with httpx.AsyncClient(http2=True) as client:
|
||||
...
|
||||
```
|
||||
|
||||
HTTP/2 support is available on both `Client` and `AsyncClient`, although it's
|
||||
typically more useful in async contexts if you're issuing lots of concurrent
|
||||
requests.
|
||||
|
||||
## Inspecting the HTTP version
|
||||
|
||||
Enabling HTTP/2 support on the client does not *necessarily* mean that your
|
||||
requests and responses will be transported over HTTP/2, since both the client
|
||||
*and* the server need to support HTTP/2. If you connect to a server that only
|
||||
supports HTTP/1.1 the client will use a standard HTTP/1.1 connection instead.
|
||||
|
||||
You can determine which version of the HTTP protocol was used by examining
|
||||
the `.http_version` property on the response.
|
||||
|
||||
```python
|
||||
client = httpx.AsyncClient(http2=True)
|
||||
response = await client.get(...)
|
||||
print(response.http_version) # "HTTP/1.0", "HTTP/1.1", or "HTTP/2".
|
||||
```
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 242 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 416 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 108 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 356 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 201 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 15 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 72 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 113 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 84 KiB |
182
docs/index.md
182
docs/index.md
@ -1,40 +1,34 @@
|
||||
<p align="center" style="margin: 0 0 10px">
|
||||
<p align="center">
|
||||
<img width="350" height="208" src="https://raw.githubusercontent.com/encode/httpx/master/docs/img/butterfly.png" alt='HTTPX'>
|
||||
</p>
|
||||
|
||||
<h1 align="center" style="font-size: 3rem; margin: -15px 0">
|
||||
HTTPX
|
||||
</h1>
|
||||
<p align="center"><em>HTTPX 1.0 — Prelease.</em></p>
|
||||
|
||||
---
|
||||
|
||||
<div align="center">
|
||||
<p>
|
||||
<a href="https://github.com/encode/httpx/actions">
|
||||
<img src="https://github.com/encode/httpx/workflows/Test%20Suite/badge.svg" alt="Test Suite">
|
||||
</a>
|
||||
<a href="https://pypi.org/project/httpx/">
|
||||
<img src="https://badge.fury.io/py/httpx.svg" alt="Package version">
|
||||
</a>
|
||||
</p>
|
||||
|
||||
<em>A next-generation HTTP client for Python.</em>
|
||||
</div>
|
||||
|
||||
HTTPX is a fully featured HTTP client for Python 3, which provides sync and async APIs, and support for both HTTP/1.1 and HTTP/2.
|
||||
A complete HTTP toolkit for Python. Supporting both client & server, and available in either sync or async flavors.
|
||||
|
||||
---
|
||||
|
||||
Install HTTPX using pip:
|
||||
*Installation...*
|
||||
|
||||
```shell
|
||||
$ pip install httpx
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .shell .httpx }
|
||||
$ pip install --pre httpx
|
||||
```
|
||||
|
||||
Now, let's get started:
|
||||
```{ .shell .ahttpx .hidden }
|
||||
$ pip install --pre ahttpx
|
||||
```
|
||||
|
||||
```pycon
|
||||
*Making requests as a client...*
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> import httpx
|
||||
|
||||
>>> r = httpx.get('https://www.example.org/')
|
||||
>>> r
|
||||
<Response [200 OK]>
|
||||
@ -46,105 +40,73 @@ Now, let's get started:
|
||||
'<!doctype html>\n<html>\n<head>\n<title>Example Domain</title>...'
|
||||
```
|
||||
|
||||
Or, using the command-line client.
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> import ahttpx
|
||||
|
||||
```shell
|
||||
# The command line client is an optional dependency.
|
||||
$ pip install 'httpx[cli]'
|
||||
>>> r = await ahttpx.get('https://www.example.org/')
|
||||
>>> r
|
||||
<Response [200 OK]>
|
||||
>>> r.status_code
|
||||
200
|
||||
>>> r.headers['content-type']
|
||||
'text/html; charset=UTF-8'
|
||||
>>> r.text
|
||||
'<!doctype html>\n<html>\n<head>\n<title>Example Domain</title>...'
|
||||
```
|
||||
|
||||
Which now allows us to use HTTPX directly from the command-line...
|
||||
*Serving responses as the server...*
|
||||
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
Sending a request...
|
||||
```{ .python .httpx }
|
||||
>>> import httpx
|
||||
|
||||
|
||||
>>> def app(request):
|
||||
... content = httpx.HTML('<html><body>hello, world.</body></html>')
|
||||
... return httpx.Response(200, content=content)
|
||||
|
||||
## Features
|
||||
|
||||
HTTPX builds on the well-established usability of `requests`, and gives you:
|
||||
|
||||
* A broadly [requests-compatible API](compatibility.md).
|
||||
* Standard synchronous interface, but with [async support if you need it](async.md).
|
||||
* HTTP/1.1 [and HTTP/2 support](http2.md).
|
||||
* Ability to make requests directly to [WSGI applications](advanced/transports.md#wsgi-transport) or [ASGI applications](advanced/transports.md#asgi-transport).
|
||||
* Strict timeouts everywhere.
|
||||
* Fully type annotated.
|
||||
* 100% test coverage.
|
||||
|
||||
Plus all the standard features of `requests`...
|
||||
|
||||
* International Domains and URLs
|
||||
* Keep-Alive & Connection Pooling
|
||||
* Sessions with Cookie Persistence
|
||||
* Browser-style SSL Verification
|
||||
* Basic/Digest Authentication
|
||||
* Elegant Key/Value Cookies
|
||||
* Automatic Decompression
|
||||
* Automatic Content Decoding
|
||||
* Unicode Response Bodies
|
||||
* Multipart File Uploads
|
||||
* HTTP(S) Proxy Support
|
||||
* Connection Timeouts
|
||||
* Streaming Downloads
|
||||
* .netrc Support
|
||||
* Chunked Requests
|
||||
|
||||
## Documentation
|
||||
|
||||
For a run-through of all the basics, head over to the [QuickStart](quickstart.md).
|
||||
|
||||
For more advanced topics, see the **Advanced** section,
|
||||
the [async support](async.md) section, or the [HTTP/2](http2.md) section.
|
||||
|
||||
The [Developer Interface](api.md) provides a comprehensive API reference.
|
||||
|
||||
To find out about tools that integrate with HTTPX, see [Third Party Packages](third_party_packages.md).
|
||||
|
||||
## Dependencies
|
||||
|
||||
The HTTPX project relies on these excellent libraries:
|
||||
|
||||
* `httpcore` - The underlying transport implementation for `httpx`.
|
||||
* `h11` - HTTP/1.1 support.
|
||||
* `certifi` - SSL certificates.
|
||||
* `idna` - Internationalized domain name support.
|
||||
* `sniffio` - Async library autodetection.
|
||||
|
||||
As well as these optional installs:
|
||||
|
||||
* `h2` - HTTP/2 support. *(Optional, with `httpx[http2]`)*
|
||||
* `socksio` - SOCKS proxy support. *(Optional, with `httpx[socks]`)*
|
||||
* `rich` - Rich terminal support. *(Optional, with `httpx[cli]`)*
|
||||
* `click` - Command line client support. *(Optional, with `httpx[cli]`)*
|
||||
* `brotli` or `brotlicffi` - Decoding for "brotli" compressed responses. *(Optional, with `httpx[brotli]`)*
|
||||
* `zstandard` - Decoding for "zstd" compressed responses. *(Optional, with `httpx[zstd]`)*
|
||||
|
||||
A huge amount of credit is due to `requests` for the API layout that
|
||||
much of this work follows, as well as to `urllib3` for plenty of design
|
||||
inspiration around the lower-level networking details.
|
||||
|
||||
## Installation
|
||||
|
||||
Install with pip:
|
||||
|
||||
```shell
|
||||
$ pip install httpx
|
||||
>>> httpx.run(app)
|
||||
Serving on http://127.0.0.1:8080/ (Press CTRL+C to quit)
|
||||
```
|
||||
|
||||
Or, to include the optional HTTP/2 support, use:
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> import ahttpx
|
||||
|
||||
```shell
|
||||
$ pip install httpx[http2]
|
||||
>>> async def app(request):
|
||||
... content = httpx.HTML('<html><body>hello, world.</body></html>')
|
||||
... return httpx.Response(200, content=content)
|
||||
|
||||
>>> await httpx.run(app)
|
||||
Serving on http://127.0.0.1:8080/ (Press CTRL+C to quit)
|
||||
```
|
||||
|
||||
To include the optional brotli and zstandard decoders support, use:
|
||||
---
|
||||
|
||||
```shell
|
||||
$ pip install httpx[brotli,zstd]
|
||||
```
|
||||
# Documentation
|
||||
|
||||
HTTPX requires Python 3.9+
|
||||
* [Quickstart](quickstart.md)
|
||||
* [Clients](clients.md)
|
||||
* [Servers](servers.md)
|
||||
* [Requests](requests.md)
|
||||
* [Responses](responses.md)
|
||||
* [URLs](urls.md)
|
||||
* [Headers](headers.md)
|
||||
* [Content Types](content-types.md)
|
||||
* [Streams](streams.md)
|
||||
* [Connections](connections.md)
|
||||
* [Parsers](parsers.md)
|
||||
* [Network Backends](networking.md)
|
||||
|
||||
[sync-support]: https://github.com/encode/httpx/issues/572
|
||||
---
|
||||
|
||||
# Collaboration
|
||||
|
||||
The repository for this project is currently private.
|
||||
|
||||
We’re looking at creating paid opportunities for working on open source software *which are properly compensated, flexible & well balanced.*
|
||||
|
||||
If you're interested in a position working on this project, please send an intro: *kim@encode.io*
|
||||
|
||||
---
|
||||
|
||||
<p align="center"><i>This design work is <a href="https://www.encode.io/httpnext/about">not yet licensed</a> for reuse.</i><br/>— 🦋 —</p>
|
||||
|
||||
@ -1,81 +0,0 @@
|
||||
# Logging
|
||||
|
||||
If you need to inspect the internal behaviour of `httpx`, you can use Python's standard logging to output information about the underlying network behaviour.
|
||||
|
||||
For example, the following configuration...
|
||||
|
||||
```python
|
||||
import logging
|
||||
import httpx
|
||||
|
||||
logging.basicConfig(
|
||||
format="%(levelname)s [%(asctime)s] %(name)s - %(message)s",
|
||||
datefmt="%Y-%m-%d %H:%M:%S",
|
||||
level=logging.DEBUG
|
||||
)
|
||||
|
||||
httpx.get("https://www.example.com")
|
||||
```
|
||||
|
||||
Will send debug level output to the console, or wherever `stdout` is directed too...
|
||||
|
||||
```
|
||||
DEBUG [2024-09-28 17:27:40] httpcore.connection - connect_tcp.started host='www.example.com' port=443 local_address=None timeout=5.0 socket_options=None
|
||||
DEBUG [2024-09-28 17:27:41] httpcore.connection - connect_tcp.complete return_value=<httpcore._backends.sync.SyncStream object at 0x101f1e8e0>
|
||||
DEBUG [2024-09-28 17:27:41] httpcore.connection - start_tls.started ssl_context=SSLContext(verify=True) server_hostname='www.example.com' timeout=5.0
|
||||
DEBUG [2024-09-28 17:27:41] httpcore.connection - start_tls.complete return_value=<httpcore._backends.sync.SyncStream object at 0x1020f49a0>
|
||||
DEBUG [2024-09-28 17:27:41] httpcore.http11 - send_request_headers.started request=<Request [b'GET']>
|
||||
DEBUG [2024-09-28 17:27:41] httpcore.http11 - send_request_headers.complete
|
||||
DEBUG [2024-09-28 17:27:41] httpcore.http11 - send_request_body.started request=<Request [b'GET']>
|
||||
DEBUG [2024-09-28 17:27:41] httpcore.http11 - send_request_body.complete
|
||||
DEBUG [2024-09-28 17:27:41] httpcore.http11 - receive_response_headers.started request=<Request [b'GET']>
|
||||
DEBUG [2024-09-28 17:27:41] httpcore.http11 - receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK', [(b'Content-Encoding', b'gzip'), (b'Accept-Ranges', b'bytes'), (b'Age', b'407727'), (b'Cache-Control', b'max-age=604800'), (b'Content-Type', b'text/html; charset=UTF-8'), (b'Date', b'Sat, 28 Sep 2024 13:27:42 GMT'), (b'Etag', b'"3147526947+gzip"'), (b'Expires', b'Sat, 05 Oct 2024 13:27:42 GMT'), (b'Last-Modified', b'Thu, 17 Oct 2019 07:18:26 GMT'), (b'Server', b'ECAcc (dcd/7D43)'), (b'Vary', b'Accept-Encoding'), (b'X-Cache', b'HIT'), (b'Content-Length', b'648')])
|
||||
INFO [2024-09-28 17:27:41] httpx - HTTP Request: GET https://www.example.com "HTTP/1.1 200 OK"
|
||||
DEBUG [2024-09-28 17:27:41] httpcore.http11 - receive_response_body.started request=<Request [b'GET']>
|
||||
DEBUG [2024-09-28 17:27:41] httpcore.http11 - receive_response_body.complete
|
||||
DEBUG [2024-09-28 17:27:41] httpcore.http11 - response_closed.started
|
||||
DEBUG [2024-09-28 17:27:41] httpcore.http11 - response_closed.complete
|
||||
DEBUG [2024-09-28 17:27:41] httpcore.connection - close.started
|
||||
DEBUG [2024-09-28 17:27:41] httpcore.connection - close.complete
|
||||
```
|
||||
|
||||
Logging output includes information from both the high-level `httpx` logger, and the network-level `httpcore` logger, which can be configured separately.
|
||||
|
||||
For handling more complex logging configurations you might want to use the dictionary configuration style...
|
||||
|
||||
```python
|
||||
import logging.config
|
||||
import httpx
|
||||
|
||||
LOGGING_CONFIG = {
|
||||
"version": 1,
|
||||
"handlers": {
|
||||
"default": {
|
||||
"class": "logging.StreamHandler",
|
||||
"formatter": "http",
|
||||
"stream": "ext://sys.stderr"
|
||||
}
|
||||
},
|
||||
"formatters": {
|
||||
"http": {
|
||||
"format": "%(levelname)s [%(asctime)s] %(name)s - %(message)s",
|
||||
"datefmt": "%Y-%m-%d %H:%M:%S",
|
||||
}
|
||||
},
|
||||
'loggers': {
|
||||
'httpx': {
|
||||
'handlers': ['default'],
|
||||
'level': 'DEBUG',
|
||||
},
|
||||
'httpcore': {
|
||||
'handlers': ['default'],
|
||||
'level': 'DEBUG',
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
logging.config.dictConfig(LOGGING_CONFIG)
|
||||
httpx.get('https://www.example.com')
|
||||
```
|
||||
|
||||
The exact formatting of the debug logging may be subject to change across different versions of `httpx` and `httpcore`. If you need to rely on a particular format it is recommended that you pin installation of these packages to fixed versions.
|
||||
381
docs/networking.md
Normal file
381
docs/networking.md
Normal file
@ -0,0 +1,381 @@
|
||||
# Network Backends
|
||||
|
||||
The lowest level network abstractions in `httpx` are the `NetworkBackend` and `NetworkStream` classes. These provide a consistent interface onto the operations for working with a network stream, typically over a TCP connection. Different runtimes (threaded, trio & asyncio) are supported via alternative implementations of the core interface.
|
||||
|
||||
---
|
||||
|
||||
## `NetworkBackend()`
|
||||
|
||||
The default backend is instantiated via the `NetworkBackend` class...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> net = httpx.NetworkBackend()
|
||||
>>> net
|
||||
<NetworkBackend [threaded]>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> net = ahttpx.NetworkBackend()
|
||||
>>> net
|
||||
<NetworkBackend [asyncio]>
|
||||
```
|
||||
|
||||
### `.connect(host, port)`
|
||||
|
||||
A TCP stream is created using the `connect` method...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> net = httpx.NetworkBackend()
|
||||
>>> stream = net.connect("www.encode.io", 80)
|
||||
>>> stream
|
||||
<NetworkStream ["168.0.0.1:80"]>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> net = ahttpx.NetworkBackend()
|
||||
>>> stream = await net.connect("www.encode.io", 80)
|
||||
>>> stream
|
||||
<NetworkStream ["168.0.0.1:80"]>
|
||||
```
|
||||
|
||||
Streams support being used in a context managed style. The cleanest approach to resource management is to use `.connect(...)` in the context of a `with` block.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> net = httpx.NetworkBackend()
|
||||
>>> with net.connect("dev.encode.io", 80) as stream:
|
||||
>>> ...
|
||||
>>> stream
|
||||
<NetworkStream ["168.0.0.1:80" CLOSED]>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> net = ahttpx.NetworkBackend()
|
||||
>>> async with await net.connect("dev.encode.io", 80) as stream:
|
||||
>>> ...
|
||||
>>> stream
|
||||
<NetworkStream ["168.0.0.1:80" CLOSED]>
|
||||
```
|
||||
|
||||
## `NetworkStream(sock)`
|
||||
|
||||
The `NetworkStream` class provides TCP stream abstraction, by providing a thin wrapper around a socket instance.
|
||||
|
||||
Network streams do not provide any built-in thread or task locking.
|
||||
Within `httpx` thread and task saftey is handled at the `Connection` layer.
|
||||
|
||||
### `.read(max_bytes=None)`
|
||||
|
||||
Read up to `max_bytes` bytes of data from the network stream.
|
||||
If no limit is provided a default value of 64KB will be used.
|
||||
|
||||
### `.write(data)`
|
||||
|
||||
Write the given bytes of `data` to the network stream.
|
||||
|
||||
### `.start_tls(ctx, hostname)`
|
||||
|
||||
Upgrade a stream to TLS (SSL) connection for sending secure `https://` requests.
|
||||
|
||||
`<NetworkStream [“168.0.0.1:443” TLS]>`
|
||||
|
||||
### `.get_extra_info(key)`
|
||||
|
||||
Return information about the underlying resource. May include...
|
||||
|
||||
* `"client_addr"` - Return the client IP and port.
|
||||
* `"server_addr"` - Return the server IP and port.
|
||||
* `"ssl_object"` - Return an `ssl.SSLObject` instance.
|
||||
* `"socket"` - Access the raw socket instance.
|
||||
|
||||
### `.close()`
|
||||
|
||||
Close the network stream. For TLS streams this will attempt to send a closing handshake before terminating the conmection.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> net = httpx.NetworkBackend()
|
||||
>>> stream = net.connect("dev.encode.io", 80)
|
||||
>>> try:
|
||||
>>> ...
|
||||
>>> finally:
|
||||
>>> stream.close()
|
||||
>>> stream
|
||||
<NetworkStream ["168.0.0.1:80" CLOSED]>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> net = ahttpx.NetworkBackend()
|
||||
>>> stream = await net.connect("dev.encode.io", 80)
|
||||
>>> try:
|
||||
>>> ...
|
||||
>>> finally:
|
||||
>>> await stream.close()
|
||||
>>> stream
|
||||
<NetworkStream ["168.0.0.1:80" CLOSED]>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Timeouts
|
||||
|
||||
Network timeouts are handled using a context block API.
|
||||
|
||||
This [design approach](https://vorpus.org/blog/timeouts-and-cancellation-for-humans) avoids timeouts needing to passed around throughout the stack, and provides an obvious and natural API to dealing with timeout contexts.
|
||||
|
||||
### timeout(duration)
|
||||
|
||||
The timeout context manager can be used to wrap socket operations anywhere in the stack.
|
||||
|
||||
Here's an example of enforcing an overall 3 second timeout on a request.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> with httpx.Client() as cli:
|
||||
>>> with httpx.timeout(3.0):
|
||||
>>> res = cli.get('https://www.example.com')
|
||||
>>> print(res)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> async with ahttpx.Client() as cli:
|
||||
>>> async with ahttpx.timeout(3.0):
|
||||
>>> res = await cli.get('https://www.example.com')
|
||||
>>> print(res)
|
||||
```
|
||||
|
||||
Timeout contexts provide an API allowing for deadlines to be cancelled.
|
||||
|
||||
### .cancel()
|
||||
|
||||
In this example we enforce a 3 second timeout on *receiving the start of* a streaming HTTP response...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> with httpx.Client() as cli:
|
||||
>>> with httpx.timeout(3.0) as t:
|
||||
>>> with cli.stream('https://www.example.com') as r:
|
||||
>>> t.cancel()
|
||||
>>> print(">>>", res)
|
||||
>>> for chunk in r.stream:
|
||||
>>> print("...", chunk)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> async with ahttpx.Client() as cli:
|
||||
>>> async with ahttpx.timeout(3.0) as t:
|
||||
>>> async with await cli.stream('https://www.example.com') as r:
|
||||
>>> t.cancel()
|
||||
>>> print(">>>", res)
|
||||
>>> async for chunk in r.stream:
|
||||
>>> print("...", chunk)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Sending HTTP requests
|
||||
|
||||
Let's take a look at how we can work directly with a network backend to send an HTTP request, and recieve an HTTP response.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
import httpx
|
||||
import ssl
|
||||
import truststore
|
||||
|
||||
net = httpx.NetworkBackend()
|
||||
ctx = truststore.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
|
||||
req = b'\r\n'.join([
|
||||
b'GET / HTTP/1.1',
|
||||
b'Host: www.example.com',
|
||||
b'User-Agent: python/dev',
|
||||
b'Connection: close',
|
||||
b'',
|
||||
b'',
|
||||
])
|
||||
|
||||
# Use a 10 second overall timeout for the entire request/response.
|
||||
with httpx.timeout(10.0):
|
||||
# Use a 3 second timeout for the initial connection.
|
||||
with httpx.timeout(3.0) as t:
|
||||
# Open the connection & establish SSL.
|
||||
with net.connect("www.example.com", 443) as stream:
|
||||
stream.start_tls(ctx, hostname="www.example.com")
|
||||
t.cancel()
|
||||
# Send the request & read the response.
|
||||
stream.write(req)
|
||||
buffer = []
|
||||
while part := stream.read():
|
||||
buffer.append(part)
|
||||
resp = b''.join(buffer)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
import ahttpx
|
||||
import ssl
|
||||
import truststore
|
||||
|
||||
net = ahttpx.NetworkBackend()
|
||||
ctx = truststore.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
|
||||
req = b'\r\n'.join([
|
||||
b'GET / HTTP/1.1',
|
||||
b'Host: www.example.com',
|
||||
b'User-Agent: python/dev',
|
||||
b'Connection: close',
|
||||
b'',
|
||||
b'',
|
||||
])
|
||||
|
||||
# Use a 10 second overall timeout for the entire request/response.
|
||||
async with ahttpx.timeout(10.0):
|
||||
# Use a 3 second timeout for the initial connection.
|
||||
async with ahttpx.timeout(3.0) as t:
|
||||
# Open the connection & establish SSL.
|
||||
async with await net.connect("www.example.com", 443) as stream:
|
||||
await stream.start_tls(ctx, hostname="www.example.com")
|
||||
t.cancel()
|
||||
# Send the request & read the response.
|
||||
await stream.write(req)
|
||||
buffer = []
|
||||
while part := await stream.read():
|
||||
buffer.append(part)
|
||||
resp = b''.join(buffer)
|
||||
```
|
||||
|
||||
The example above is somewhat contrived, there's no HTTP parsing implemented so we can't actually determine when the response is complete. We're using a `Connection: close` header to request that the server close the connection once the response is complete.
|
||||
|
||||
A more complete example would require proper HTTP parsing. The `Connection` class implements an HTTP request/response interface, layered over a `NetworkStream`.
|
||||
|
||||
---
|
||||
|
||||
## Custom network backends
|
||||
|
||||
The interface for implementing custom network backends is provided by two classes...
|
||||
|
||||
### `NetworkBackendInterface`
|
||||
|
||||
The abstract interface implemented by `NetworkBackend`. See above for details.
|
||||
|
||||
### `NetworkStreamInterface`
|
||||
|
||||
The abstract interface implemented by `NetworkStream`. See above for details.
|
||||
|
||||
### An example backend
|
||||
|
||||
We can use these interfaces to implement custom functionality. For example, here we're providing a network backend that logs all the ingoing and outgoing bytes.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
class RecordingBackend(httpx.NetworkBackendInterface):
|
||||
def __init__(self):
|
||||
self._backend = NetworkBackend()
|
||||
|
||||
def connect(self, host, port):
|
||||
# Delegate creating connections to the default
|
||||
# network backend, and return a wrapped stream.
|
||||
stream = self._backend.connect(host, port)
|
||||
return RecordingStream(stream)
|
||||
|
||||
|
||||
class RecordingStream(httpx.NetworkStreamInterface):
|
||||
def __init__(self, stream):
|
||||
self._stream = stream
|
||||
|
||||
def read(self, max_bytes: int = None):
|
||||
# Print all incoming data to the terminal.
|
||||
data = self._stream.read(max_bytes)
|
||||
lines = data.decode('ascii', errors='replace').splitlines()
|
||||
for line in lines:
|
||||
print("<<< ", line)
|
||||
return data
|
||||
|
||||
def write(self, data):
|
||||
# Print all outgoing data to the terminal.
|
||||
lines = data.decode('ascii', errors='replace').splitlines()
|
||||
for line in lines:
|
||||
print(">>> ", line)
|
||||
self._stream.write(data)
|
||||
|
||||
def start_tls(ctx, hostname):
|
||||
self._stream.start_tls(ctx, hostname)
|
||||
|
||||
def get_extra_info(key):
|
||||
return self._stream.get_extra_info(key)
|
||||
|
||||
def close():
|
||||
self._stream.close()
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
class RecordingBackend(ahhtpx.NetworkBackendInterface):
|
||||
def __init__(self):
|
||||
self._backend = NetworkBackend()
|
||||
|
||||
async def connect(self, host, port):
|
||||
# Delegate creating connections to the default
|
||||
# network backend, and return a wrapped stream.
|
||||
stream = await self._backend.connect(host, port)
|
||||
return RecordingStream(stream)
|
||||
|
||||
|
||||
class RecordingStream(ahttpx.NetworkStreamInterface):
|
||||
def __init__(self, stream):
|
||||
self._stream = stream
|
||||
|
||||
async def read(self, max_bytes: int = None):
|
||||
# Print all incoming data to the terminal.
|
||||
data = await self._stream.read(max_bytes)
|
||||
lines = data.decode('ascii', errors='replace').splitlines()
|
||||
for line in lines:
|
||||
print("<<< ", line)
|
||||
return data
|
||||
|
||||
async def write(self, data):
|
||||
# Print all outgoing data to the terminal.
|
||||
lines = data.decode('ascii', errors='replace').splitlines()
|
||||
for line in lines:
|
||||
print(">>> ", line)
|
||||
await self._stream.write(data)
|
||||
|
||||
async def start_tls(ctx, hostname):
|
||||
await self._stream.start_tls(ctx, hostname)
|
||||
|
||||
def get_extra_info(key):
|
||||
return self._stream.get_extra_info(key)
|
||||
|
||||
async def close():
|
||||
await self._stream.close()
|
||||
```
|
||||
|
||||
We can now instantiate a client using this network backend.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> transport = httpx.ConnectionPool(backend=RecordingBackend())
|
||||
>>> cli = httpx.Client(transport=transport)
|
||||
>>> cli.get('https://www.example.com')
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> transport = ahttpx.ConnectionPool(backend=RecordingBackend())
|
||||
>>> cli = ahttpx.Client(transport=transport)
|
||||
>>> await cli.get('https://www.example.com')
|
||||
```
|
||||
|
||||
Custom network backends can also be used to provide functionality such as handling DNS caching for name lookups, or connecting via a UNIX domain socket instead of a TCP connection.
|
||||
|
||||
---
|
||||
|
||||
<span class="link-prev">← [Parsers](parsers.md)</span>
|
||||
<span> </span>
|
||||
@ -1,54 +0,0 @@
|
||||
{% import "partials/nav-item.html" as item with context %}
|
||||
|
||||
<!-- Determine class according to configuration -->
|
||||
{% set class = "md-nav md-nav--primary" %}
|
||||
{% if "navigation.tabs" in features %}
|
||||
{% set class = class ~ " md-nav--lifted" %}
|
||||
{% endif %}
|
||||
{% if "toc.integrate" in features %}
|
||||
{% set class = class ~ " md-nav--integrated" %}
|
||||
{% endif %}
|
||||
|
||||
<!-- Main navigation -->
|
||||
<nav
|
||||
class="{{ class }}"
|
||||
aria-label="{{ lang.t('nav.title') }}"
|
||||
data-md-level="0"
|
||||
>
|
||||
|
||||
<!-- Site title -->
|
||||
<label class="md-nav__title" for="__drawer">
|
||||
<a
|
||||
href="{{ config.extra.homepage | d(nav.homepage.url, true) | url }}"
|
||||
title="{{ config.site_name | e }}"
|
||||
class="md-nav__button md-logo"
|
||||
aria-label="{{ config.site_name }}"
|
||||
data-md-component="logo"
|
||||
>
|
||||
{% include "partials/logo.html" %}
|
||||
</a>
|
||||
{{ config.site_name }}
|
||||
</label>
|
||||
|
||||
<!-- Repository information -->
|
||||
{% if config.repo_url %}
|
||||
<div class="md-nav__source">
|
||||
{% include "partials/source.html" %}
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
<!-- Navigation list -->
|
||||
<ul class="md-nav__list" data-md-scrollfix>
|
||||
{% for nav_item in nav %}
|
||||
{% set path = "__nav_" ~ loop.index %}
|
||||
{{ item.render(nav_item, path, 1) }}
|
||||
{% endfor %}
|
||||
</ul>
|
||||
|
||||
<ul class="md-nav__list" data-md-scrollfix style="padding-top: 15px; padding-left: 10px">
|
||||
<div>
|
||||
<a href="https://speakeasy.com"><img src="/img/speakeasy.png" width=150px style=></img></a>
|
||||
</div>
|
||||
</ul>
|
||||
</nav>
|
||||
|
||||
110
docs/parsers.md
Normal file
110
docs/parsers.md
Normal file
@ -0,0 +1,110 @@
|
||||
# Parsers
|
||||
|
||||
### Client
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
stream = httpx.DuplexStream(
|
||||
b'HTTP/1.1 200 OK\r\n'
|
||||
b'Content-Length: 23\r\n'
|
||||
b'Content-Type: application/json\r\n'
|
||||
b'\r\n'
|
||||
b'{"msg": "hello, world"}'
|
||||
)
|
||||
p = ahttpx.HTTPParser(stream, mode='CLIENT')
|
||||
|
||||
# Send the request...
|
||||
p.send_method_line(b'GET', b'/', b'HTTP/1.1')
|
||||
p.send_headers([(b'Host', b'www.example.com')])
|
||||
p.send_body(b'')
|
||||
|
||||
# Receive the response...
|
||||
protocol, code, reason_phase = p.recv_status_line()
|
||||
headers = p.recv_headers()
|
||||
body = b''
|
||||
while buffer := p.recv_body():
|
||||
body += buffer
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
stream = ahttpx.DuplexStream(
|
||||
b'HTTP/1.1 200 OK\r\n'
|
||||
b'Content-Length: 23\r\n'
|
||||
b'Content-Type: application/json\r\n'
|
||||
b'\r\n'
|
||||
b'{"msg": "hello, world"}'
|
||||
)
|
||||
p = ahttpx.HTTPParser(stream, mode='CLIENT')
|
||||
|
||||
# Send the request...
|
||||
await p.send_method_line(b'GET', b'/', b'HTTP/1.1')
|
||||
await p.send_headers([(b'Host', b'www.example.com')])
|
||||
await p.send_body(b'')
|
||||
|
||||
# Receive the response...
|
||||
protocol, code, reason_phase = await p.recv_status_line()
|
||||
headers = await p.recv_headers()
|
||||
body = b''
|
||||
while buffer := await p.recv_body():
|
||||
body += buffer
|
||||
```
|
||||
|
||||
### Server
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
stream = httpx.DuplexStream(
|
||||
b'GET / HTTP/1.1\r\n'
|
||||
b'Host: www.example.com\r\n'
|
||||
b'\r\n'
|
||||
)
|
||||
p = httpx.HTTPParser(stream, mode='SERVER')
|
||||
|
||||
# Receive the request...
|
||||
method, target, protocol = p.recv_method_line()
|
||||
headers = p.recv_headers()
|
||||
body = b''
|
||||
while buffer := p.recv_body():
|
||||
body += buffer
|
||||
|
||||
# Send the response...
|
||||
p.send_status_line(b'HTTP/1.1', 200, b'OK')
|
||||
p.send_headers([
|
||||
(b'Content-Length', b'23'),
|
||||
(b'Content-Type', b'application/json')
|
||||
])
|
||||
p.send_body(b'{"msg": "hello, world"}')
|
||||
p.send_body(b'')
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
stream = ahttpx.DuplexStream(
|
||||
b'GET / HTTP/1.1\r\n'
|
||||
b'Host: www.example.com\r\n'
|
||||
b'\r\n'
|
||||
)
|
||||
p = ahttpx.HTTPParser(stream, mode='SERVER')
|
||||
|
||||
# Receive the request...
|
||||
method, target, protocol = await p.recv_method_line()
|
||||
headers = await p.recv_headers()
|
||||
body = b''
|
||||
while buffer := await p.recv_body():
|
||||
body += buffer
|
||||
|
||||
# Send the response...
|
||||
await p.send_status_line(b'HTTP/1.1', 200, b'OK')
|
||||
await p.send_headers([
|
||||
(b'Content-Length', b'23'),
|
||||
(b'Content-Type', b'application/json')
|
||||
])
|
||||
await p.send_body(b'{"msg": "hello, world"}')
|
||||
await p.send_body(b'')
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
<span class="link-prev">← [Connections](connections.md)</span>
|
||||
<span class="link-next">[Low Level Networking](networking.md) →</span>
|
||||
@ -1,146 +1,207 @@
|
||||
# QuickStart
|
||||
|
||||
First, start by importing HTTPX:
|
||||
Install using ...
|
||||
|
||||
```pycon
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .shell .httpx }
|
||||
$ pip install --pre httpx
|
||||
```
|
||||
|
||||
```{ .shell .ahttpx .hidden }
|
||||
$ pip install --pre ahttpx
|
||||
```
|
||||
|
||||
First, start by importing `httpx`...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> import httpx
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> import ahttpx
|
||||
```
|
||||
|
||||
Now, let’s try to get a webpage.
|
||||
|
||||
```pycon
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> r = httpx.get('https://httpbin.org/get')
|
||||
>>> r
|
||||
<Response [200 OK]>
|
||||
```
|
||||
|
||||
Similarly, to make an HTTP POST request:
|
||||
|
||||
```pycon
|
||||
>>> r = httpx.post('https://httpbin.org/post', data={'key': 'value'})
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> r = await ahttpx.get('https://httpbin.org/get')
|
||||
>>> r
|
||||
<Response [200 OK]>
|
||||
```
|
||||
|
||||
The PUT, DELETE, HEAD, and OPTIONS requests all follow the same style:
|
||||
To make an HTTP `POST` request, including some content...
|
||||
|
||||
```pycon
|
||||
>>> r = httpx.put('https://httpbin.org/put', data={'key': 'value'})
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> form = httpx.Form({'key': 'value'})
|
||||
>>> r = httpx.post('https://httpbin.org/post', content=form)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> form = httpx.Form({'key': 'value'})
|
||||
>>> r = await ahttpx.post('https://httpbin.org/post', content=form)
|
||||
```
|
||||
|
||||
Shortcut methods for `PUT`, `PATCH`, and `DELETE` requests follow the same style...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> r = httpx.put('https://httpbin.org/put', content=form)
|
||||
>>> r = httpx.patch('https://httpbin.org/patch', content=form)
|
||||
>>> r = httpx.delete('https://httpbin.org/delete')
|
||||
>>> r = httpx.head('https://httpbin.org/get')
|
||||
>>> r = httpx.options('https://httpbin.org/get')
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> r = await ahttpx.put('https://httpbin.org/put', content=form)
|
||||
>>> r = await ahttpx.patch('https://httpbin.org/patch', content=form)
|
||||
>>> r = await ahttpx.delete('https://httpbin.org/delete')
|
||||
```
|
||||
|
||||
## Passing Parameters in URLs
|
||||
|
||||
To include URL query parameters in the request, use the `params` keyword:
|
||||
To include URL query parameters in the request, construct a URL using the `params` keyword...
|
||||
|
||||
```pycon
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> params = {'key1': 'value1', 'key2': 'value2'}
|
||||
>>> r = httpx.get('https://httpbin.org/get', params=params)
|
||||
>>> url = httpx.URL('https://httpbin.org/get', params=params)
|
||||
>>> r = httpx.get(url)
|
||||
```
|
||||
|
||||
To see how the values get encoding into the URL string, we can inspect the
|
||||
resulting URL that was used to make the request:
|
||||
|
||||
```pycon
|
||||
>>> r.url
|
||||
URL('https://httpbin.org/get?key2=value2&key1=value1')
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> params = {'key1': 'value1', 'key2': 'value2'}
|
||||
>>> url = ahttpx.URL('https://httpbin.org/get', params=params)
|
||||
>>> r = await ahttpx.get(url)
|
||||
```
|
||||
|
||||
You can also pass a list of items as a value:
|
||||
You can also pass a list of items as a value...
|
||||
|
||||
```pycon
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> params = {'key1': 'value1', 'key2': ['value2', 'value3']}
|
||||
>>> r = httpx.get('https://httpbin.org/get', params=params)
|
||||
>>> r.url
|
||||
URL('https://httpbin.org/get?key1=value1&key2=value2&key2=value3')
|
||||
>>> url = httpx.URL('https://httpbin.org/get', params=params)
|
||||
>>> r = httpx.get(url)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> params = {'key1': 'value1', 'key2': ['value2', 'value3']}
|
||||
>>> url = ahttpx.URL('https://httpbin.org/get', params=params)
|
||||
>>> r = await ahttpx.get(url)
|
||||
```
|
||||
|
||||
## Custom Headers
|
||||
|
||||
To include additional headers in the outgoing request, use the `headers` keyword argument...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> url = 'https://httpbin.org/headers'
|
||||
>>> headers = {'User-Agent': 'my-app/0.0.1'}
|
||||
>>> r = httpx.get(url, headers=headers)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> url = 'https://httpbin.org/headers'
|
||||
>>> headers = {'User-Agent': 'my-app/0.0.1'}
|
||||
>>> r = await ahttpx.get(url, headers=headers)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Response Content
|
||||
|
||||
HTTPX will automatically handle decoding the response content into Unicode text.
|
||||
HTTPX will automatically handle decoding the response content into unicode text.
|
||||
|
||||
```pycon
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> r = httpx.get('https://www.example.org/')
|
||||
>>> r.text
|
||||
'<!doctype html>\n<html>\n<head>\n<title>Example Domain</title>...'
|
||||
```
|
||||
|
||||
You can inspect what encoding will be used to decode the response.
|
||||
|
||||
```pycon
|
||||
>>> r.encoding
|
||||
'UTF-8'
|
||||
```
|
||||
|
||||
In some cases the response may not contain an explicit encoding, in which case HTTPX
|
||||
will attempt to automatically determine an encoding to use.
|
||||
|
||||
```pycon
|
||||
>>> r.encoding
|
||||
None
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> r = await ahttpx.get('https://www.example.org/')
|
||||
>>> r.text
|
||||
'<!doctype html>\n<html>\n<head>\n<title>Example Domain</title>...'
|
||||
```
|
||||
|
||||
If you need to override the standard behaviour and explicitly set the encoding to
|
||||
use, then you can do that too.
|
||||
|
||||
```pycon
|
||||
>>> r.encoding = 'ISO-8859-1'
|
||||
```
|
||||
|
||||
## Binary Response Content
|
||||
|
||||
The response content can also be accessed as bytes, for non-text responses:
|
||||
The response content can also be accessed as bytes, for non-text responses.
|
||||
|
||||
```pycon
|
||||
>>> r.content
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> r.body
|
||||
b'<!doctype html>\n<html>\n<head>\n<title>Example Domain</title>...'
|
||||
```
|
||||
|
||||
Any `gzip` and `deflate` HTTP response encodings will automatically
|
||||
be decoded for you. If `brotlipy` is installed, then the `brotli` response
|
||||
encoding will be supported. If `zstandard` is installed, then `zstd`
|
||||
response encodings will also be supported.
|
||||
|
||||
For example, to create an image from binary data returned by a request, you can use the following code:
|
||||
|
||||
```pycon
|
||||
>>> from PIL import Image
|
||||
>>> from io import BytesIO
|
||||
>>> i = Image.open(BytesIO(r.content))
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> r.body
|
||||
b'<!doctype html>\n<html>\n<head>\n<title>Example Domain</title>...'
|
||||
```
|
||||
|
||||
## JSON Response Content
|
||||
|
||||
Often Web API responses will be encoded as JSON.
|
||||
|
||||
```pycon
|
||||
>>> r = httpx.get('https://api.github.com/events')
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> r = httpx.get('https://httpbin.org/get')
|
||||
>>> r.json()
|
||||
[{u'repository': {u'open_issues': 0, u'url': 'https://github.com/...' ... }}]
|
||||
{'args': {}, 'headers': {'Host': 'httpbin.org', 'User-Agent': 'dev', 'X-Amzn-Trace-Id': 'Root=1-679814d5-0f3d46b26686f5013e117085'}, 'origin': '21.35.60.128', 'url': 'https://httpbin.org/get'}
|
||||
```
|
||||
|
||||
## Custom Headers
|
||||
|
||||
To include additional headers in the outgoing request, use the `headers` keyword argument:
|
||||
|
||||
```pycon
|
||||
>>> url = 'https://httpbin.org/headers'
|
||||
>>> headers = {'user-agent': 'my-app/0.0.1'}
|
||||
>>> r = httpx.get(url, headers=headers)
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> r = await ahttpx.get('https://httpbin.org/get')
|
||||
>>> await r.json()
|
||||
{'args': {}, 'headers': {'Host': 'httpbin.org', 'User-Agent': 'dev', 'X-Amzn-Trace-Id': 'Root=1-679814d5-0f3d46b26686f5013e117085'}, 'origin': '21.35.60.128', 'url': 'https://httpbin.org/get'}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Sending Form Encoded Data
|
||||
|
||||
Some types of HTTP requests, such as `POST` and `PUT` requests, can include data
|
||||
in the request body. One common way of including that is as form-encoded data,
|
||||
which is used for HTML forms.
|
||||
Some types of HTTP requests, such as `POST` and `PUT` requests, can include data in the request body. One common way of including that is as form-encoded data, which is used for HTML forms.
|
||||
|
||||
```pycon
|
||||
>>> data = {'key1': 'value1', 'key2': 'value2'}
|
||||
>>> r = httpx.post("https://httpbin.org/post", data=data)
|
||||
>>> print(r.text)
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> form = httpx.Form({'key1': 'value1', 'key2': 'value2'})
|
||||
>>> r = httpx.post("https://httpbin.org/post", content=form)
|
||||
>>> r.json()
|
||||
{
|
||||
...
|
||||
"form": {
|
||||
"key2": "value2",
|
||||
"key1": "value1"
|
||||
},
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> form = ahttpx.Form({'key1': 'value1', 'key2': 'value2'})
|
||||
>>> r = await ahttpx.post("https://httpbin.org/post", content=form)
|
||||
>>> await r.json()
|
||||
{
|
||||
...
|
||||
"form": {
|
||||
@ -153,10 +214,28 @@ which is used for HTML forms.
|
||||
|
||||
Form encoded data can also include multiple values from a given key.
|
||||
|
||||
```pycon
|
||||
>>> data = {'key1': ['value1', 'value2']}
|
||||
>>> r = httpx.post("https://httpbin.org/post", data=data)
|
||||
>>> print(r.text)
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> form = httpx.Form({'key1': ['value1', 'value2']})
|
||||
>>> r = httpx.post("https://httpbin.org/post", content=form)
|
||||
>>> r.json()
|
||||
{
|
||||
...
|
||||
"form": {
|
||||
"key1": [
|
||||
"value1",
|
||||
"value2"
|
||||
]
|
||||
},
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> form = ahttpx.Form({'key1': ['value1', 'value2']})
|
||||
>>> r = await ahttpx.post("https://httpbin.org/post", content=form)
|
||||
>>> await r.json()
|
||||
{
|
||||
...
|
||||
"form": {
|
||||
@ -171,34 +250,31 @@ Form encoded data can also include multiple values from a given key.
|
||||
|
||||
## Sending Multipart File Uploads
|
||||
|
||||
You can also upload files, using HTTP multipart encoding:
|
||||
You can also upload files, using HTTP multipart encoding.
|
||||
|
||||
```pycon
|
||||
>>> with open('report.xls', 'rb') as report_file:
|
||||
... files = {'upload-file': report_file}
|
||||
... r = httpx.post("https://httpbin.org/post", files=files)
|
||||
>>> print(r.text)
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> files = httpx.Files({'upload': httpx.File('uploads/report.xls')})
|
||||
>>> r = httpx.post("https://httpbin.org/post", content=files)
|
||||
>>> r.json()
|
||||
{
|
||||
...
|
||||
"files": {
|
||||
"upload-file": "<... binary content ...>"
|
||||
"upload": "<... binary content ...>"
|
||||
},
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
You can also explicitly set the filename and content type, by using a tuple
|
||||
of items for the file value:
|
||||
|
||||
```pycon
|
||||
>>> with open('report.xls', 'rb') as report_file:
|
||||
... files = {'upload-file': ('report.xls', report_file, 'application/vnd.ms-excel')}
|
||||
... r = httpx.post("https://httpbin.org/post", files=files)
|
||||
>>> print(r.text)
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> files = ahttpx.Files({'upload': httpx.File('uploads/report.xls')})
|
||||
>>> r = await ahttpx.post("https://httpbin.org/post", content=files)
|
||||
>>> await r.json()
|
||||
{
|
||||
...
|
||||
"files": {
|
||||
"upload-file": "<... binary content ...>"
|
||||
"upload": "<... binary content ...>"
|
||||
},
|
||||
...
|
||||
}
|
||||
@ -206,16 +282,36 @@ of items for the file value:
|
||||
|
||||
If you need to include non-file data fields in the multipart form, use the `data=...` parameter:
|
||||
|
||||
```pycon
|
||||
>>> data = {'message': 'Hello, world!'}
|
||||
>>> with open('report.xls', 'rb') as report_file:
|
||||
... files = {'file': report_file}
|
||||
... r = httpx.post("https://httpbin.org/post", data=data, files=files)
|
||||
>>> print(r.text)
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> form = {'message': 'Hello, world!'}
|
||||
>>> files = {'upload': httpx.File('uploads/report.xls')}
|
||||
>>> data = httpx.MultiPart(form=form, files=files)
|
||||
>>> r = httpx.post("https://httpbin.org/post", content=data)
|
||||
>>> r.json()
|
||||
{
|
||||
...
|
||||
"files": {
|
||||
"file": "<... binary content ...>"
|
||||
"upload": "<... binary content ...>"
|
||||
},
|
||||
"form": {
|
||||
"message": "Hello, world!",
|
||||
},
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> form = {'message': 'Hello, world!'}
|
||||
>>> files = {'upload': httpx.File('uploads/report.xls')}
|
||||
>>> data = ahttpx.MultiPart(form=form, files=files)
|
||||
>>> r = await ahttpx.post("https://httpbin.org/post", content=data)
|
||||
>>> await r.json()
|
||||
{
|
||||
...
|
||||
"files": {
|
||||
"upload": "<... binary content ...>"
|
||||
},
|
||||
"form": {
|
||||
"message": "Hello, world!",
|
||||
@ -229,10 +325,31 @@ If you need to include non-file data fields in the multipart form, use the `data
|
||||
Form encoded data is okay if all you need is a simple key-value data structure.
|
||||
For more complicated data structures you'll often want to use JSON encoding instead.
|
||||
|
||||
```pycon
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> data = {'integer': 123, 'boolean': True, 'list': ['a', 'b', 'c']}
|
||||
>>> r = httpx.post("https://httpbin.org/post", json=data)
|
||||
>>> print(r.text)
|
||||
>>> r = httpx.post("https://httpbin.org/post", content=httpx.JSON(data))
|
||||
>>> r.json()
|
||||
{
|
||||
...
|
||||
"json": {
|
||||
"boolean": true,
|
||||
"integer": 123,
|
||||
"list": [
|
||||
"a",
|
||||
"b",
|
||||
"c"
|
||||
]
|
||||
},
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> data = {'integer': 123, 'boolean': True, 'list': ['a', 'b', 'c']}
|
||||
>>> r = await ahttpx.post("https://httpbin.org/post", content=httpx.JSON(data))
|
||||
>>> await r.json()
|
||||
{
|
||||
...
|
||||
"json": {
|
||||
@ -253,88 +370,92 @@ For more complicated data structures you'll often want to use JSON encoding inst
|
||||
For other encodings, you should use the `content=...` parameter, passing
|
||||
either a `bytes` type or a generator that yields `bytes`.
|
||||
|
||||
```pycon
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> content = b'Hello, world'
|
||||
>>> r = httpx.post("https://httpbin.org/post", content=content)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> content = b'Hello, world'
|
||||
>>> r = await ahttpx.post("https://httpbin.org/post", content=content)
|
||||
```
|
||||
|
||||
You may also want to set a custom `Content-Type` header when uploading
|
||||
binary data.
|
||||
|
||||
---
|
||||
|
||||
## Response Status Codes
|
||||
|
||||
We can inspect the HTTP status code of the response:
|
||||
|
||||
```pycon
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> r = httpx.get('https://httpbin.org/get')
|
||||
>>> r.status_code
|
||||
200
|
||||
```
|
||||
|
||||
HTTPX also includes an easy shortcut for accessing status codes by their text phrase.
|
||||
|
||||
```pycon
|
||||
>>> r.status_code == httpx.codes.OK
|
||||
True
|
||||
```
|
||||
|
||||
We can raise an exception for any responses which are not a 2xx success code:
|
||||
|
||||
```pycon
|
||||
>>> not_found = httpx.get('https://httpbin.org/status/404')
|
||||
>>> not_found.status_code
|
||||
404
|
||||
>>> not_found.raise_for_status()
|
||||
Traceback (most recent call last):
|
||||
File "/Users/tomchristie/GitHub/encode/httpcore/httpx/models.py", line 837, in raise_for_status
|
||||
raise HTTPStatusError(message, response=self)
|
||||
httpx._exceptions.HTTPStatusError: 404 Client Error: Not Found for url: https://httpbin.org/status/404
|
||||
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404
|
||||
```
|
||||
|
||||
Any successful response codes will return the `Response` instance rather than raising an exception.
|
||||
|
||||
```pycon
|
||||
>>> r.raise_for_status()
|
||||
```
|
||||
|
||||
The method returns the response instance, allowing you to use it inline. For example:
|
||||
|
||||
```pycon
|
||||
>>> r = httpx.get('...').raise_for_status()
|
||||
>>> data = httpx.get('...').raise_for_status().json()
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> r = await ahttpx.get('https://httpbin.org/get')
|
||||
>>> r.status_code
|
||||
200
|
||||
```
|
||||
|
||||
## Response Headers
|
||||
|
||||
The response headers are available as a dictionary-like interface.
|
||||
|
||||
```pycon
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> r.headers
|
||||
Headers({
|
||||
'content-encoding': 'gzip',
|
||||
'transfer-encoding': 'chunked',
|
||||
'connection': 'close',
|
||||
'server': 'nginx/1.0.4',
|
||||
'x-runtime': '148ms',
|
||||
'etag': '"e1ca502697e5c9317743dc078f67693f"',
|
||||
'content-type': 'application/json'
|
||||
})
|
||||
<Headers {
|
||||
'Content-Encoding': 'gzip',
|
||||
'Connection': 'close',
|
||||
'Server': 'nginx/1.0.4',
|
||||
'ETag': 'e1ca502697e5c9317743dc078f67693f',
|
||||
'Content-Type': 'application/json',
|
||||
'Content-Length': 2126,
|
||||
}>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> r.headers
|
||||
<Headers {
|
||||
'Content-Encoding': 'gzip',
|
||||
'Connection': 'close',
|
||||
'Server': 'nginx/1.0.4',
|
||||
'ETag': 'e1ca502697e5c9317743dc078f67693f',
|
||||
'Content-Type': 'application/json',
|
||||
'Content-Length': 2126,
|
||||
}>
|
||||
```
|
||||
|
||||
The `Headers` data type is case-insensitive, so you can use any capitalization.
|
||||
|
||||
```pycon
|
||||
>>> r.headers['Content-Type']
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> r.headers.get('Content-Type')
|
||||
'application/json'
|
||||
|
||||
>>> r.headers.get('content-type')
|
||||
'application/json'
|
||||
```
|
||||
|
||||
Multiple values for a single response header are represented as a single comma-separated value, as per [RFC 7230](https://tools.ietf.org/html/rfc7230#section-3.2):
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> r.headers.get('Content-Type')
|
||||
'application/json'
|
||||
|
||||
> A recipient MAY combine multiple header fields with the same field name into one “field-name: field-value” pair, without changing the semantics of the message, by appending each subsequent field-value to the combined field value in order, separated by a comma.
|
||||
>>> r.headers.get('content-type')
|
||||
'application/json'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Streaming Responses
|
||||
|
||||
@ -342,206 +463,22 @@ For large downloads you may want to use streaming responses that do not load the
|
||||
|
||||
You can stream the binary content of the response...
|
||||
|
||||
```pycon
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> with httpx.stream("GET", "https://www.example.com") as r:
|
||||
... for data in r.iter_bytes():
|
||||
... for data in r.stream:
|
||||
... print(data)
|
||||
```
|
||||
|
||||
Or the text of the response...
|
||||
|
||||
```pycon
|
||||
>>> with httpx.stream("GET", "https://www.example.com") as r:
|
||||
... for text in r.iter_text():
|
||||
... print(text)
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> async with ahttpx.stream("GET", "https://www.example.com") as r:
|
||||
... async for data in r.stream:
|
||||
... print(data)
|
||||
```
|
||||
|
||||
Or stream the text, on a line-by-line basis...
|
||||
---
|
||||
|
||||
```pycon
|
||||
>>> with httpx.stream("GET", "https://www.example.com") as r:
|
||||
... for line in r.iter_lines():
|
||||
... print(line)
|
||||
```
|
||||
|
||||
HTTPX will use universal line endings, normalising all cases to `\n`.
|
||||
|
||||
In some cases you might want to access the raw bytes on the response without applying any HTTP content decoding. In this case any content encoding that the web server has applied such as `gzip`, `deflate`, `brotli`, or `zstd` will
|
||||
not be automatically decoded.
|
||||
|
||||
```pycon
|
||||
>>> with httpx.stream("GET", "https://www.example.com") as r:
|
||||
... for chunk in r.iter_raw():
|
||||
... print(chunk)
|
||||
```
|
||||
|
||||
If you're using streaming responses in any of these ways then the `response.content` and `response.text` attributes will not be available, and will raise errors if accessed. However you can also use the response streaming functionality to conditionally load the response body:
|
||||
|
||||
```pycon
|
||||
>>> with httpx.stream("GET", "https://www.example.com") as r:
|
||||
... if int(r.headers['Content-Length']) < TOO_LONG:
|
||||
... r.read()
|
||||
... print(r.text)
|
||||
```
|
||||
|
||||
## Cookies
|
||||
|
||||
Any cookies that are set on the response can be easily accessed:
|
||||
|
||||
```pycon
|
||||
>>> r = httpx.get('https://httpbin.org/cookies/set?chocolate=chip')
|
||||
>>> r.cookies['chocolate']
|
||||
'chip'
|
||||
```
|
||||
|
||||
To include cookies in an outgoing request, use the `cookies` parameter:
|
||||
|
||||
```pycon
|
||||
>>> cookies = {"peanut": "butter"}
|
||||
>>> r = httpx.get('https://httpbin.org/cookies', cookies=cookies)
|
||||
>>> r.json()
|
||||
{'cookies': {'peanut': 'butter'}}
|
||||
```
|
||||
|
||||
Cookies are returned in a `Cookies` instance, which is a dict-like data structure
|
||||
with additional API for accessing cookies by their domain or path.
|
||||
|
||||
```pycon
|
||||
>>> cookies = httpx.Cookies()
|
||||
>>> cookies.set('cookie_on_domain', 'hello, there!', domain='httpbin.org')
|
||||
>>> cookies.set('cookie_off_domain', 'nope.', domain='example.org')
|
||||
>>> r = httpx.get('http://httpbin.org/cookies', cookies=cookies)
|
||||
>>> r.json()
|
||||
{'cookies': {'cookie_on_domain': 'hello, there!'}}
|
||||
```
|
||||
|
||||
## Redirection and History
|
||||
|
||||
By default, HTTPX will **not** follow redirects for all HTTP methods, although
|
||||
this can be explicitly enabled.
|
||||
|
||||
For example, GitHub redirects all HTTP requests to HTTPS.
|
||||
|
||||
```pycon
|
||||
>>> r = httpx.get('http://github.com/')
|
||||
>>> r.status_code
|
||||
301
|
||||
>>> r.history
|
||||
[]
|
||||
>>> r.next_request
|
||||
<Request('GET', 'https://github.com/')>
|
||||
```
|
||||
|
||||
You can modify the default redirection handling with the `follow_redirects` parameter:
|
||||
|
||||
```pycon
|
||||
>>> r = httpx.get('http://github.com/', follow_redirects=True)
|
||||
>>> r.url
|
||||
URL('https://github.com/')
|
||||
>>> r.status_code
|
||||
200
|
||||
>>> r.history
|
||||
[<Response [301 Moved Permanently]>]
|
||||
```
|
||||
|
||||
The `history` property of the response can be used to inspect any followed redirects.
|
||||
It contains a list of any redirect responses that were followed, in the order
|
||||
in which they were made.
|
||||
|
||||
## Timeouts
|
||||
|
||||
HTTPX defaults to including reasonable timeouts for all network operations,
|
||||
meaning that if a connection is not properly established then it should always
|
||||
raise an error rather than hanging indefinitely.
|
||||
|
||||
The default timeout for network inactivity is five seconds. You can modify the
|
||||
value to be more or less strict:
|
||||
|
||||
```pycon
|
||||
>>> httpx.get('https://github.com/', timeout=0.001)
|
||||
```
|
||||
|
||||
You can also disable the timeout behavior completely...
|
||||
|
||||
```pycon
|
||||
>>> httpx.get('https://github.com/', timeout=None)
|
||||
```
|
||||
|
||||
For advanced timeout management, see [Timeout fine-tuning](advanced/timeouts.md#fine-tuning-the-configuration).
|
||||
|
||||
## Authentication
|
||||
|
||||
HTTPX supports Basic and Digest HTTP authentication.
|
||||
|
||||
To provide Basic authentication credentials, pass a 2-tuple of
|
||||
plaintext `str` or `bytes` objects as the `auth` argument to the request
|
||||
functions:
|
||||
|
||||
```pycon
|
||||
>>> httpx.get("https://example.com", auth=("my_user", "password123"))
|
||||
```
|
||||
|
||||
To provide credentials for Digest authentication you'll need to instantiate
|
||||
a `DigestAuth` object with the plaintext username and password as arguments.
|
||||
This object can be then passed as the `auth` argument to the request methods
|
||||
as above:
|
||||
|
||||
```pycon
|
||||
>>> auth = httpx.DigestAuth("my_user", "password123")
|
||||
>>> httpx.get("https://example.com", auth=auth)
|
||||
<Response [200 OK]>
|
||||
```
|
||||
|
||||
## Exceptions
|
||||
|
||||
HTTPX will raise exceptions if an error occurs.
|
||||
|
||||
The most important exception classes in HTTPX are `RequestError` and `HTTPStatusError`.
|
||||
|
||||
The `RequestError` class is a superclass that encompasses any exception that occurs
|
||||
while issuing an HTTP request. These exceptions include a `.request` attribute.
|
||||
|
||||
```python
|
||||
try:
|
||||
response = httpx.get("https://www.example.com/")
|
||||
except httpx.RequestError as exc:
|
||||
print(f"An error occurred while requesting {exc.request.url!r}.")
|
||||
```
|
||||
|
||||
The `HTTPStatusError` class is raised by `response.raise_for_status()` on responses which are not a 2xx success code.
|
||||
These exceptions include both a `.request` and a `.response` attribute.
|
||||
|
||||
```python
|
||||
response = httpx.get("https://www.example.com/")
|
||||
try:
|
||||
response.raise_for_status()
|
||||
except httpx.HTTPStatusError as exc:
|
||||
print(f"Error response {exc.response.status_code} while requesting {exc.request.url!r}.")
|
||||
```
|
||||
|
||||
There is also a base class `HTTPError` that includes both of these categories, and can be used
|
||||
to catch either failed requests, or 4xx and 5xx responses.
|
||||
|
||||
You can either use this base class to catch both categories...
|
||||
|
||||
```python
|
||||
try:
|
||||
response = httpx.get("https://www.example.com/")
|
||||
response.raise_for_status()
|
||||
except httpx.HTTPError as exc:
|
||||
print(f"Error while requesting {exc.request.url!r}.")
|
||||
```
|
||||
|
||||
Or handle each case explicitly...
|
||||
|
||||
```python
|
||||
try:
|
||||
response = httpx.get("https://www.example.com/")
|
||||
response.raise_for_status()
|
||||
except httpx.RequestError as exc:
|
||||
print(f"An error occurred while requesting {exc.request.url!r}.")
|
||||
except httpx.HTTPStatusError as exc:
|
||||
print(f"Error response {exc.response.status_code} while requesting {exc.request.url!r}.")
|
||||
```
|
||||
|
||||
For a full list of available exceptions, see [Exceptions (API Reference)](exceptions.md).
|
||||
<span class="link-prev">← [Home](index.md)</span>
|
||||
<span class="link-next">[Clients](clients.md) →</span>
|
||||
<span> </span>
|
||||
178
docs/requests.md
Normal file
178
docs/requests.md
Normal file
@ -0,0 +1,178 @@
|
||||
# Requests
|
||||
|
||||
The core elements of an HTTP request are the `method`, `url`, `headers` and `body`.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> req = httpx.Request('GET', 'https://www.example.com/')
|
||||
>>> req
|
||||
<Request [GET 'https://www.example.com/']>
|
||||
>>> req.method
|
||||
'GET'
|
||||
>>> req.url
|
||||
<URL 'https://www.example.com/'>
|
||||
>>> req.headers
|
||||
<Headers {'Host': 'www.example.com'}>
|
||||
>>> req.body
|
||||
b''
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> req = ahttpx.Request('GET', 'https://www.example.com/')
|
||||
>>> req
|
||||
<Request [GET 'https://www.example.com/']>
|
||||
>>> req.method
|
||||
'GET'
|
||||
>>> req.url
|
||||
<URL 'https://www.example.com/'>
|
||||
>>> req.headers
|
||||
<Headers {'Host': 'www.example.com'}>
|
||||
>>> req.body
|
||||
b''
|
||||
```
|
||||
|
||||
## Working with the request headers
|
||||
|
||||
The following headers have automatic behavior with `Requests` instances...
|
||||
|
||||
* `Host` - A `Host` header must always be included on a request. This header is automatically populated from the `url`, using the `url.netloc` property.
|
||||
* `Content-Length` - Requests including a request body must always include either a `Content-Length` header or a `Transfer-Encoding: chunked` header. This header is automatically populated if `content` is not `None` and the content is a known size.
|
||||
* `Transfer-Encoding` - Requests automatically include a `Transfer-Encoding: chunked` header if `content` is not `None` and the content is an unkwown size.
|
||||
* `Content-Type` - Requests automatically include a `Content-Type` header if `content` is set using the [Content Type] API.
|
||||
|
||||
## Working with the request body
|
||||
|
||||
Including binary data directly...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> headers = {'Content-Type': 'application/json'}
|
||||
>>> content = json.dumps(...)
|
||||
>>> httpx.Request('POST', 'https://echo.encode.io/', content=content)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> headers = {'Content-Type': 'application/json'}
|
||||
>>> content = json.dumps(...)
|
||||
>>> ahttpx.Request('POST', 'https://echo.encode.io/', content=content)
|
||||
```
|
||||
|
||||
## Working with content types
|
||||
|
||||
Including JSON request content...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> data = httpx.JSON(...)
|
||||
>>> httpx.Request('POST', 'https://echo.encode.io/', content=data)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> data = ahttpx.JSON(...)
|
||||
>>> ahttpx.Request('POST', 'https://echo.encode.io/', content=data)
|
||||
```
|
||||
|
||||
Including form encoded request content...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> data = httpx.Form(...)
|
||||
>>> httpx.Request('PUT', 'https://echo.encode.io/', content=data)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> data = ahttpx.Form(...)
|
||||
>>> ahttpx.Request('PUT', 'https://echo.encode.io/', content=data)
|
||||
```
|
||||
|
||||
Including multipart file uploads...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> form = httpx.MultiPart(form={...}, files={...})
|
||||
>>> with httpx.Request('POST', 'https://echo.encode.io/', content=form) as req:
|
||||
>>> req.headers
|
||||
{...}
|
||||
>>> req.stream
|
||||
<MultiPartStream [0% of ...MB]>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> form = ahttpx.MultiPart(form={...}, files={...})
|
||||
>>> async with ahttpx.Request('POST', 'https://echo.encode.io/', content=form) as req:
|
||||
>>> req.headers
|
||||
{...}
|
||||
>>> req.stream
|
||||
<MultiPartStream [0% of ...MB]>
|
||||
```
|
||||
|
||||
Including direct file uploads...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> file = httpx.File('upload.json')
|
||||
>>> with httpx.Request('POST', 'https://echo.encode.io/', content=file) as req:
|
||||
>>> req.headers
|
||||
{...}
|
||||
>>> req.stream
|
||||
<FileStream [0% of ...MB]>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> file = ahttpx.File('upload.json')
|
||||
>>> async with ahttpx.Request('POST', 'https://echo.encode.io/', content=file) as req:
|
||||
>>> req.headers
|
||||
{...}
|
||||
>>> req.stream
|
||||
<FileStream [0% of ...MB]>
|
||||
```
|
||||
|
||||
## Accessing request content
|
||||
|
||||
*In progress...*
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> data = request.json()
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> data = await request.json()
|
||||
```
|
||||
|
||||
...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> form = request.form()
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> form = await request.form()
|
||||
```
|
||||
|
||||
...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> files = request.files()
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> files = await request.files()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
<span class="link-prev">← [Servers](servers.md)</span>
|
||||
<span class="link-next">[Responses](responses.md) →</span>
|
||||
<span> </span>
|
||||
131
docs/responses.md
Normal file
131
docs/responses.md
Normal file
@ -0,0 +1,131 @@
|
||||
# Responses
|
||||
|
||||
The core elements of an HTTP response are the `status_code`, `headers` and `body`.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> resp = httpx.Response(200, headers={'Content-Type': 'text/plain'}, content=b'hello, world')
|
||||
>>> resp
|
||||
<Response [200 OK]>
|
||||
>>> resp.status_code
|
||||
200
|
||||
>>> resp.headers
|
||||
<Headers {'Content-Type': 'text/html'}>
|
||||
>>> resp.body
|
||||
b'hello, world'
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> resp = ahttpx.Response(200, headers={'Content-Type': 'text/plain'}, content=b'hello, world')
|
||||
>>> resp
|
||||
<Response [200 OK]>
|
||||
>>> resp.status_code
|
||||
200
|
||||
>>> resp.headers
|
||||
<Headers {'Content-Type': 'text/html'}>
|
||||
>>> resp.body
|
||||
b'hello, world'
|
||||
```
|
||||
|
||||
## Working with the response headers
|
||||
|
||||
The following headers have automatic behavior with `Response` instances...
|
||||
|
||||
* `Content-Length` - Responses including a response body must always include either a `Content-Length` header or a `Transfer-Encoding: chunked` header. This header is automatically populated if `content` is not `None` and the content is a known size.
|
||||
* `Transfer-Encoding` - Responses automatically include a `Transfer-Encoding: chunked` header if `content` is not `None` and the content is an unkwown size.
|
||||
* `Content-Type` - Responses automatically include a `Content-Type` header if `content` is set using the [Content Type] API.
|
||||
|
||||
## Working with content types
|
||||
|
||||
Including HTML content...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> content = httpx.HTML('<html><head>...</head><body>...</body></html>')
|
||||
>>> response = httpx.Response(200, content=content)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> content = ahttpx.HTML('<html><head>...</head><body>...</body></html>')
|
||||
>>> response = ahttpx.Response(200, content=content)
|
||||
```
|
||||
|
||||
Including plain text content...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> content = httpx.Text('hello, world')
|
||||
>>> response = httpx.Response(200, content=content)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> content = ahttpx.Text('hello, world')
|
||||
>>> response = ahttpx.Response(200, content=content)
|
||||
```
|
||||
|
||||
Including JSON data...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> content = httpx.JSON({'message': 'hello, world'})
|
||||
>>> response = httpx.Response(200, content=content)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> content = ahttpx.JSON({'message': 'hello, world'})
|
||||
>>> response = ahttpx.Response(200, content=content)
|
||||
```
|
||||
|
||||
Including content from a file...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> content = httpx.File('index.html')
|
||||
>>> with httpx.Response(200, content=content) as response:
|
||||
... pass
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> content = ahttpx.File('index.html')
|
||||
>>> async with ahttpx.Response(200, content=content) as response:
|
||||
... pass
|
||||
```
|
||||
|
||||
## Accessing response content
|
||||
|
||||
...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> response.body
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> response.body
|
||||
```
|
||||
|
||||
...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> response.text
|
||||
...
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> response.text
|
||||
...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
<span class="link-prev">← [Requests](requests.md)</span>
|
||||
<span class="link-next">[URLs](urls.md) →</span>
|
||||
<span> </span>
|
||||
85
docs/servers.md
Normal file
85
docs/servers.md
Normal file
@ -0,0 +1,85 @@
|
||||
# Servers
|
||||
|
||||
The HTTP server provides a simple request/response API.
|
||||
This gives you a lightweight way to build web applications or APIs.
|
||||
|
||||
### `serve_http(endpoint)`
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> website = """
|
||||
... <html>
|
||||
... <head>
|
||||
... <style>
|
||||
... body {
|
||||
... font-family: courier;
|
||||
... text-align: center;
|
||||
... padding: 3rem;
|
||||
... background: #111;
|
||||
... color: #ddd;
|
||||
... font-size: 3rem;
|
||||
... }
|
||||
... </style>
|
||||
... </head>
|
||||
... <body>
|
||||
... <div>hello, world</div>
|
||||
... </body>
|
||||
... </html>
|
||||
... """
|
||||
|
||||
>>> def hello_world(request):
|
||||
... content = httpx.HTML(website)
|
||||
... return httpx.Response(200, content=content)
|
||||
|
||||
>>> with httpx.serve_http(hello_world) as server:
|
||||
... print(f"Serving on {server.url} (Press CTRL+C to quit)")
|
||||
... server.wait()
|
||||
Serving on http://127.0.0.1:8080/ (Press CTRL+C to quit)
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> import httpx
|
||||
|
||||
>>> website = """
|
||||
... <html>
|
||||
... <head>
|
||||
... <style>
|
||||
... body {
|
||||
... font-family: courier;
|
||||
... text-align: center;
|
||||
... padding: 3rem;
|
||||
... background: #111;
|
||||
... color: #ddd;
|
||||
... font-size: 3rem;
|
||||
... }
|
||||
... </style>
|
||||
... </head>
|
||||
... <body>
|
||||
... <div>hello, world</div>
|
||||
... </body>
|
||||
... </html>
|
||||
... """
|
||||
|
||||
>>> async def hello_world(request):
|
||||
... if request.path != '/':
|
||||
... content = httpx.Text("Not found")
|
||||
... return httpx.Response(404, content=content)
|
||||
... content = httpx.HTML(website)
|
||||
... return httpx.Response(200, content=content)
|
||||
|
||||
>>> async with httpx.serve_http(hello_world) as server:
|
||||
... print(f"Serving on {server.url} (Press CTRL+C to quit)")
|
||||
... await server.wait()
|
||||
Serving on http://127.0.0.1:8080/ (Press CTRL+C to quit)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Docs in progress...*
|
||||
|
||||
---
|
||||
|
||||
<span class="link-prev">← [Clients](clients.md)</span>
|
||||
<span class="link-next">[Requests](requests.md) →</span>
|
||||
<span> </span>
|
||||
88
docs/streams.md
Normal file
88
docs/streams.md
Normal file
@ -0,0 +1,88 @@
|
||||
# Streams
|
||||
|
||||
Streams provide a minimal file-like interface for reading bytes from a data source. They are used as the abstraction for reading the body of a request or response.
|
||||
|
||||
The interfaces here are simplified versions of Python's standard I/O operations.
|
||||
|
||||
## Stream
|
||||
|
||||
The base `Stream` class. The core of the interface is a subset of Python's `io.IOBase`...
|
||||
|
||||
* `.read(size=-1)` - *(bytes)* Return the bytes from the data stream. If the `size` argument is omitted or negative then the entire stream will be read. If `size` is an positive integer then the call returns at most `size` bytes. A return value of `b''` indicates the end of the stream has been reached.
|
||||
* `.write(self, data: bytes)` - *None* Write the given bytes to the data stream. May raise `NotImplmentedError` if this is not a writeable stream.
|
||||
* `.close()` - Close the stream. Any further operations will raise a `ValueError`.
|
||||
|
||||
Additionally, the following property is also defined...
|
||||
|
||||
* `.size` - *(int or None)* Return an integer indicating the size of the stream, or `None` if the size is unknown. When working with HTTP this is used to either set a `Content-Length: <size>` header, or a `Content-Encoding: chunked` header.
|
||||
|
||||
The `Stream` interface and `ContentType` interface are related, with streams being used as the abstraction for the bytewise representation, and content types being used to encapsulate the parsed data structure.
|
||||
|
||||
For example, encoding some `JSON` data...
|
||||
|
||||
```python
|
||||
>>> data = httpx.JSON({'name': 'zelda', 'score': '478'})
|
||||
>>> stream = data.encode()
|
||||
>>> stream.read()
|
||||
b'{"name":"zelda","score":"478"}'
|
||||
>>> stream.content_type
|
||||
'application/json'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ByteStream
|
||||
|
||||
A byte stream returning fixed byte content. Similar to Python's `io.BytesIO` class.
|
||||
|
||||
```python
|
||||
>>> s = httpx.ByteStream(b'{"msg": "Hello, world!"}')
|
||||
>>> s.read()
|
||||
b'{"msg": "Hello, world!"}'
|
||||
```
|
||||
|
||||
## FileStream
|
||||
|
||||
A byte stream returning content from a file.
|
||||
|
||||
The standard pattern for instantiating a `FileStream` is to use `File` as a context manager:
|
||||
|
||||
```python
|
||||
>>> with httpx.File('upload.json') as s:
|
||||
... s.read()
|
||||
b'{"msg": "Hello, world!"}'
|
||||
```
|
||||
|
||||
## MultiPartStream
|
||||
|
||||
A byte stream returning multipart upload data.
|
||||
|
||||
The standard pattern for instantiating a `MultiPartStream` is to use `MultiPart` as a context manager:
|
||||
|
||||
```python
|
||||
>>> files = {'avatar-upload': 'image.png'}
|
||||
>>> with httpx.MultiPart(files=files) as s:
|
||||
... s.read()
|
||||
# ...
|
||||
```
|
||||
|
||||
## HTTPStream
|
||||
|
||||
A byte stream returning unparsed content from an HTTP request or response.
|
||||
|
||||
```python
|
||||
>>> with httpx.Client() as cli:
|
||||
... r = cli.get('https://www.example.com/')
|
||||
... r.stream.read()
|
||||
# ...
|
||||
```
|
||||
|
||||
## GZipStream
|
||||
|
||||
...
|
||||
|
||||
---
|
||||
|
||||
<span class="link-prev">← [Content Types](content-types.md)</span>
|
||||
<span class="link-next">[Connections](connections.md) →</span>
|
||||
<span> </span>
|
||||
186
docs/templates/base.html
vendored
Normal file
186
docs/templates/base.html
vendored
Normal file
@ -0,0 +1,186 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
|
||||
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'><text y='.9em' font-size='90'>%F0%9F%8C%B1</text></svg>">
|
||||
<title>httpx</title>
|
||||
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com/">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com/" crossorigin="">
|
||||
|
||||
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/atom-one-dark.min.css">
|
||||
<link rel="stylesheet" href="https://unpkg.com/highlightjs-copy/dist/highlightjs-copy.min.css">
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"></script>
|
||||
<script src="https://unpkg.com/highlightjs-copy/dist/highlightjs-copy.min.js"></script>
|
||||
|
||||
<script>
|
||||
function hook(text, el) {
|
||||
const lines = text.split('\n');
|
||||
|
||||
if (lines[0].startsWith('>>> ')) {
|
||||
return lines
|
||||
.filter(line => line.startsWith('>>> ') || line.startsWith('... '))
|
||||
.map(line => line.replace(/^>>> |^\.{3} /, ''))
|
||||
.join('\n');
|
||||
}
|
||||
|
||||
if (lines[0].startsWith('$ ')) {
|
||||
return lines
|
||||
.map(line => line.startsWith('$ ') ? line.slice(2) : line)
|
||||
.join('\n');
|
||||
}
|
||||
|
||||
return text;
|
||||
}
|
||||
|
||||
function httpx() {
|
||||
const httpx = document.querySelectorAll('.httpx');
|
||||
httpx.forEach(function(el) {
|
||||
el.classList.remove('hidden');
|
||||
});
|
||||
const ahttpx = document.querySelectorAll('.ahttpx');
|
||||
ahttpx.forEach(function(el) {
|
||||
el.classList.add('hidden');
|
||||
});
|
||||
document.cookie = "selection=httpx; path=/;"
|
||||
}
|
||||
function ahttpx() {
|
||||
const httpx = document.querySelectorAll('.httpx');
|
||||
httpx.forEach(function(el) {
|
||||
el.classList.add('hidden');
|
||||
});
|
||||
const ahttpx = document.querySelectorAll('.ahttpx');
|
||||
ahttpx.forEach(function(el) {
|
||||
el.classList.remove('hidden');
|
||||
});
|
||||
document.cookie = "selection=ahttpx; path=/;"
|
||||
}
|
||||
hljs.addPlugin(new CopyButtonPlugin({autohide: false, hook: hook}));
|
||||
hljs.highlightAll();
|
||||
</script>
|
||||
<style>
|
||||
* {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
body {
|
||||
font-family: "Poppins", Sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #ffffffcc;
|
||||
background-color: black;
|
||||
}
|
||||
|
||||
main {
|
||||
max-width: 680px;
|
||||
margin: 0 auto;
|
||||
padding: 3rem 1rem 1rem 1rem;
|
||||
}
|
||||
|
||||
h1, h2, h3, h4 {
|
||||
font-family: "Poppins", Sans-serif;
|
||||
margin-top: 2rem;
|
||||
margin-bottom: 1rem;
|
||||
line-height: 1.3;
|
||||
color: #474747;
|
||||
}
|
||||
|
||||
h1 { text-align: center; font-weight: 300; font-size: 4rem; }
|
||||
h2 { font-size: 2rem; }
|
||||
h3 { font-size: 1.5rem; }
|
||||
h4 { font-size: 1.25rem; }
|
||||
|
||||
h1 img {
|
||||
width: 30%;
|
||||
}
|
||||
|
||||
p {
|
||||
margin-top: 1rem;
|
||||
margin-bottom: 1rem;
|
||||
font-size: 1rem;
|
||||
}
|
||||
|
||||
strong {
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
ul, ol {
|
||||
margin-bottom: 1.5rem;
|
||||
padding-left: 2rem;
|
||||
}
|
||||
|
||||
li {
|
||||
margin-bottom: 0.5rem;
|
||||
}
|
||||
|
||||
a {
|
||||
color: #FF9300;
|
||||
}
|
||||
|
||||
hr {
|
||||
margin-top: 1rem;
|
||||
margin-bottom: 1rem;
|
||||
border: none;
|
||||
border-top: 1px solid #FF9300;
|
||||
}
|
||||
|
||||
div.tabs {
|
||||
font-size: small;
|
||||
font-family: courier;
|
||||
text-align: right;
|
||||
border-bottom: 1px solid grey;
|
||||
}
|
||||
|
||||
div.tabs a {
|
||||
color: white;
|
||||
text-decoration: none;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
div.tabs a.hidden {
|
||||
color: grey;
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
pre.hidden {
|
||||
display: none;
|
||||
}
|
||||
|
||||
span.link-prev {
|
||||
float: left;
|
||||
}
|
||||
|
||||
span.link-next {
|
||||
float: right;
|
||||
clear: right;
|
||||
}
|
||||
|
||||
@media (max-width: 600px) {
|
||||
main {
|
||||
padding: 1.5rem 1rem;
|
||||
}
|
||||
|
||||
h1 { font-size: 2rem; }
|
||||
h2 { font-size: 1.75rem; }
|
||||
h3 { font-size: 1.375rem; }
|
||||
h4 { font-size: 1.125rem; }
|
||||
|
||||
p {
|
||||
font-size: 0.95rem;
|
||||
}
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<main>
|
||||
{{ content }}
|
||||
</main>
|
||||
<script>
|
||||
document.addEventListener('DOMContentLoaded', function() {
|
||||
if (document.cookie.match("selection=ahttpx")) {
|
||||
ahttpx();
|
||||
}
|
||||
})
|
||||
</script>
|
||||
</body></html>
|
||||
@ -1,107 +0,0 @@
|
||||
# Third Party Packages
|
||||
|
||||
As HTTPX usage grows, there is an expanding community of developers building tools and libraries that integrate with HTTPX, or depend on HTTPX. Here are some of them.
|
||||
|
||||
<!-- NOTE: Entries are alphabetised. -->
|
||||
|
||||
## Plugins
|
||||
|
||||
### Hishel
|
||||
|
||||
[GitHub](https://github.com/karpetrosyan/hishel) - [Documentation](https://hishel.com/)
|
||||
|
||||
An elegant HTTP Cache implementation for HTTPX and HTTP Core.
|
||||
|
||||
### HTTPX-Auth
|
||||
|
||||
[GitHub](https://github.com/Colin-b/httpx_auth) - [Documentation](https://colin-b.github.io/httpx_auth/)
|
||||
|
||||
Provides authentication classes to be used with HTTPX's [authentication parameter](advanced/authentication.md#customizing-authentication).
|
||||
|
||||
### httpx-caching
|
||||
|
||||
[Github](https://github.com/johtso/httpx-caching)
|
||||
|
||||
This package adds caching functionality to HTTPX
|
||||
|
||||
### httpx-secure
|
||||
|
||||
[GitHub](https://github.com/Zaczero/httpx-secure)
|
||||
|
||||
Drop-in SSRF protection for httpx with DNS caching and custom validation support.
|
||||
|
||||
### httpx-socks
|
||||
|
||||
[GitHub](https://github.com/romis2012/httpx-socks)
|
||||
|
||||
Proxy (HTTP, SOCKS) transports for httpx.
|
||||
|
||||
### httpx-sse
|
||||
|
||||
[GitHub](https://github.com/florimondmanca/httpx-sse)
|
||||
|
||||
Allows consuming Server-Sent Events (SSE) with HTTPX.
|
||||
|
||||
### httpx-retries
|
||||
|
||||
[GitHub](https://github.com/will-ockmore/httpx-retries) - [Documentation](https://will-ockmore.github.io/httpx-retries/)
|
||||
|
||||
A retry layer for HTTPX.
|
||||
|
||||
### httpx-ws
|
||||
|
||||
[GitHub](https://github.com/frankie567/httpx-ws) - [Documentation](https://frankie567.github.io/httpx-ws/)
|
||||
|
||||
WebSocket support for HTTPX.
|
||||
|
||||
### pytest-HTTPX
|
||||
|
||||
[GitHub](https://github.com/Colin-b/pytest_httpx) - [Documentation](https://colin-b.github.io/pytest_httpx/)
|
||||
|
||||
Provides a [pytest](https://docs.pytest.org/en/latest/) fixture to mock HTTPX within test cases.
|
||||
|
||||
### RESPX
|
||||
|
||||
[GitHub](https://github.com/lundberg/respx) - [Documentation](https://lundberg.github.io/respx/)
|
||||
|
||||
A utility for mocking out HTTPX.
|
||||
|
||||
### rpc.py
|
||||
|
||||
[Github](https://github.com/abersheeran/rpc.py) - [Documentation](https://github.com/abersheeran/rpc.py#rpcpy)
|
||||
|
||||
A fast and powerful RPC framework based on ASGI/WSGI. Use HTTPX as the client of the RPC service.
|
||||
|
||||
## Libraries with HTTPX support
|
||||
|
||||
### Authlib
|
||||
|
||||
[GitHub](https://github.com/lepture/authlib) - [Documentation](https://docs.authlib.org/en/latest/)
|
||||
|
||||
A python library for building OAuth and OpenID Connect clients and servers. Includes an [OAuth HTTPX client](https://docs.authlib.org/en/latest/client/httpx.html).
|
||||
|
||||
### Gidgethub
|
||||
|
||||
[GitHub](https://github.com/brettcannon/gidgethub) - [Documentation](https://gidgethub.readthedocs.io/en/latest/index.html)
|
||||
|
||||
An asynchronous GitHub API library. Includes [HTTPX support](https://gidgethub.readthedocs.io/en/latest/httpx.html).
|
||||
|
||||
### httpdbg
|
||||
|
||||
[GitHub](https://github.com/cle-b/httpdbg) - [Documentation](https://httpdbg.readthedocs.io/)
|
||||
|
||||
A tool for python developers to easily debug the HTTP(S) client requests in a python program.
|
||||
|
||||
### VCR.py
|
||||
|
||||
[GitHub](https://github.com/kevin1024/vcrpy) - [Documentation](https://vcrpy.readthedocs.io/)
|
||||
|
||||
Record and repeat requests.
|
||||
|
||||
## Gists
|
||||
|
||||
### urllib3-transport
|
||||
|
||||
[GitHub](https://gist.github.com/florimondmanca/d56764d78d748eb9f73165da388e546e)
|
||||
|
||||
This public gist provides an example implementation for a [custom transport](advanced/transports.md#custom-transports) implementation on top of the battle-tested [`urllib3`](https://urllib3.readthedocs.io) library.
|
||||
@ -1,63 +0,0 @@
|
||||
# Troubleshooting
|
||||
|
||||
This page lists some common problems or issues you could encounter while developing with HTTPX, as well as possible solutions.
|
||||
|
||||
## Proxies
|
||||
|
||||
---
|
||||
|
||||
### "`The handshake operation timed out`" on HTTPS requests when using a proxy
|
||||
|
||||
**Description**: When using a proxy and making an HTTPS request, you see an exception looking like this:
|
||||
|
||||
```console
|
||||
httpx.ProxyError: _ssl.c:1091: The handshake operation timed out
|
||||
```
|
||||
|
||||
**Similar issues**: [encode/httpx#1412](https://github.com/encode/httpx/issues/1412), [encode/httpx#1433](https://github.com/encode/httpx/issues/1433)
|
||||
|
||||
**Resolution**: it is likely that you've set up your proxies like this...
|
||||
|
||||
```python
|
||||
mounts = {
|
||||
"http://": httpx.HTTPTransport(proxy="http://myproxy.org"),
|
||||
"https://": httpx.HTTPTransport(proxy="https://myproxy.org"),
|
||||
}
|
||||
```
|
||||
|
||||
Using this setup, you're telling HTTPX to connect to the proxy using HTTP for HTTP requests, and using HTTPS for HTTPS requests.
|
||||
|
||||
But if you get the error above, it is likely that your proxy doesn't support connecting via HTTPS. Don't worry: that's a [common gotcha](advanced/proxies.md#http-proxies).
|
||||
|
||||
Change the scheme of your HTTPS proxy to `http://...` instead of `https://...`:
|
||||
|
||||
```python
|
||||
mounts = {
|
||||
"http://": httpx.HTTPTransport(proxy="http://myproxy.org"),
|
||||
"https://": httpx.HTTPTransport(proxy="http://myproxy.org"),
|
||||
}
|
||||
```
|
||||
|
||||
This can be simplified to:
|
||||
|
||||
```python
|
||||
proxy = "http://myproxy.org"
|
||||
with httpx.Client(proxy=proxy) as client:
|
||||
...
|
||||
```
|
||||
|
||||
For more information, see [Proxies: FORWARD vs TUNNEL](advanced/proxies.md#forward-vs-tunnel).
|
||||
|
||||
---
|
||||
|
||||
### Error when making requests to an HTTPS proxy
|
||||
|
||||
**Description**: your proxy _does_ support connecting via HTTPS, but you are seeing errors along the lines of...
|
||||
|
||||
```console
|
||||
httpx.ProxyError: [SSL: PRE_MAC_LENGTH_TOO_LONG] invalid alert (_ssl.c:1091)
|
||||
```
|
||||
|
||||
**Similar issues**: [encode/httpx#1424](https://github.com/encode/httpx/issues/1424).
|
||||
|
||||
**Resolution**: HTTPX does not properly support HTTPS proxies at this time. If that's something you're interested in having, please see [encode/httpx#1434](https://github.com/encode/httpx/issues/1434) and consider lending a hand there.
|
||||
240
docs/urls.md
Normal file
240
docs/urls.md
Normal file
@ -0,0 +1,240 @@
|
||||
# URLs
|
||||
|
||||
The `URL` class handles URL validation and parsing.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> url = httpx.URL('https://www.example.com/')
|
||||
>>> url
|
||||
<URL 'https://www.example.com/'>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> url = ahttpx.URL('https://www.example.com/')
|
||||
>>> url
|
||||
<URL 'https://www.example.com/'>
|
||||
```
|
||||
|
||||
URL components are normalised, following the same rules as internet browsers.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> url = httpx.URL('https://www.EXAMPLE.com:443/path/../main')
|
||||
>>> url
|
||||
<URL 'https://www.example.com/main'>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> url = ahttpx.URL('https://www.EXAMPLE.com:443/path/../main')
|
||||
>>> url
|
||||
<URL 'https://www.example.com/main'>
|
||||
```
|
||||
|
||||
Both absolute and relative URLs are valid.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> url = httpx.URL('/README.md')
|
||||
>>> url
|
||||
<URL '/README.md'>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> url = ahttpx.URL('/README.md')
|
||||
>>> url
|
||||
<URL '/README.md'>
|
||||
```
|
||||
|
||||
Coercing a URL to a `str` will always result in a printable ASCII string.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> url = httpx.URL('https://example.com/path to here?search=🦋')
|
||||
>>> str(url)
|
||||
'https://example.com/path%20to%20here?search=%F0%9F%A6%8B'
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> url = ahttpx.URL('https://example.com/path to here?search=🦋')
|
||||
>>> str(url)
|
||||
'https://example.com/path%20to%20here?search=%F0%9F%A6%8B'
|
||||
```
|
||||
|
||||
### URL components
|
||||
|
||||
The following properties are available for accessing the component parts of a URL.
|
||||
|
||||
* `.scheme` - *str. ASCII. Normalised to lowercase.*
|
||||
* `.userinfo` - *str. ASCII. URL encoded.*
|
||||
* `.username` - *str. Unicode.*
|
||||
* `.password` - *str. Unicode.*
|
||||
* `.host` - *str. ASCII. IDNA encoded.*
|
||||
* `.port` - *int or None. Scheme default ports are normalised to None.*
|
||||
* `.authority` - *str. ASCII. IDNA encoded. Eg. "example.com", "example.com:1337", "xn--p1ai".*
|
||||
* `.path` - *str. Unicode.*
|
||||
* `.query` - *str. ASCII. URL encoded.*
|
||||
* `.target` - *str. ASCII. URL encoded.*
|
||||
* `.fragment` - *str. ASCII. URL encoded.*
|
||||
|
||||
A parsed representation of the query parameters is accessible with the `.params` property.
|
||||
|
||||
* `.params` - [`QueryParams`](#query-parameters)
|
||||
|
||||
URLs can be instantiated from their components...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> httpx.URL(scheme="https", host="example.com", path="/")
|
||||
<URL 'https://example.com/'>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> ahttpx.URL(scheme="https", host="example.com", path="/")
|
||||
<URL 'https://example.com/'>
|
||||
```
|
||||
|
||||
Or using both the string form and query parameters...
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> httpx.URL("https://example.com/", params={"search": "some text"})
|
||||
<URL 'https://example.com/?search=some+text'>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> ahttpx.URL("https://example.com/", params={"search": "some text"})
|
||||
<URL 'https://example.com/?search=some+text'>
|
||||
```
|
||||
|
||||
### Modifying URLs
|
||||
|
||||
Instances of `URL` are immutable, meaning their value cannot be changed. Instead new modified instances may be created.
|
||||
|
||||
* `.copy_with(**components)` - *Return a new URL, updating one or more components. Eg. `url = url.copy_with(scheme="https")`*.
|
||||
* `.copy_set_param(key, value)` - *Return a new URL, setting a query parameter. Eg. `url = url.copy_set_param("sort_by", "price")`*.
|
||||
* `.copy_append_param(key, value)` - *Return a new URL, setting or appending a query parameter. Eg. `url = url.copy_append_param("tag", "sale")`*.
|
||||
* `.copy_remove_param(key)` - *Return a new URL, removing a query parameter. Eg. `url = url.copy_remove_param("max_price")`*.
|
||||
* `.copy_update_params(params)` - *Return a new URL, updating the query parameters. Eg. `url = url.copy_update_params({"color_scheme": "dark"})`*.
|
||||
* `.join(url)` - *Return a new URL, given this URL as the base and another URL as the target. Eg. `url = url.join("../navigation")`*.
|
||||
|
||||
---
|
||||
|
||||
## Query Parameters
|
||||
|
||||
The `QueryParams` class provides an immutable multi-dict for accessing URL query parameters.
|
||||
|
||||
They can be instantiated from a dictionary.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> params = httpx.QueryParams({"color": "black", "size": "medium"})
|
||||
>>> params
|
||||
<QueryParams 'color=black&size=medium'>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> params = ahttpx.QueryParams({"color": "black", "size": "medium"})
|
||||
>>> params
|
||||
<QueryParams 'color=black&size=medium'>
|
||||
```
|
||||
|
||||
Multiple values for a single key are valid.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> params = httpx.QueryParams({"filter": ["60GHz", "75GHz", "100GHz"]})
|
||||
>>> params
|
||||
<QueryParams 'filter=60GHz&filter=75GHz&filter=100GHz'>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> params = ahttpx.QueryParams({"filter": ["60GHz", "75GHz", "100GHz"]})
|
||||
>>> params
|
||||
<QueryParams 'filter=60GHz&filter=75GHz&filter=100GHz'>
|
||||
```
|
||||
|
||||
They can also be instantiated directly from a query string.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> params = httpx.QueryParams("color=black&size=medium")
|
||||
>>> params
|
||||
<QueryParams 'color=black&size=medium'>
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> params = ahttpx.QueryParams("color=black&size=medium")
|
||||
>>> params
|
||||
<QueryParams 'color=black&size=medium'>
|
||||
```
|
||||
|
||||
Keys and values are always represented as strings.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> params = httpx.QueryParams("sort_by=published&author=natalie")
|
||||
>>> params["sort_by"]
|
||||
'published'
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> params = ahttpx.QueryParams("sort_by=published&author=natalie")
|
||||
>>> params["sort_by"]
|
||||
'published'
|
||||
```
|
||||
|
||||
When coercing query parameters to strings you'll see the same escaping behavior as HTML form submissions. The result will always be a printable ASCII string.
|
||||
|
||||
<div class="tabs"><a onclick="httpx()" class="httpx">httpx</a> <a onclick="ahttpx()" class="ahttpx hidden">ahttpx</a></div>
|
||||
|
||||
```{ .python .httpx }
|
||||
>>> params = httpx.QueryParams({"email": "user@example.com", "search": "How HTTP works!"})
|
||||
>>> str(params)
|
||||
'email=user%40example.com&search=How+HTTP+works%21'
|
||||
```
|
||||
|
||||
```{ .python .ahttpx .hidden }
|
||||
>>> params = ahttpx.QueryParams({"email": "user@example.com", "search": "How HTTP works!"})
|
||||
>>> str(params)
|
||||
'email=user%40example.com&search=How+HTTP+works%21'
|
||||
```
|
||||
|
||||
### Accessing query parameters
|
||||
|
||||
Query parameters are accessed using a standard dictionary style interface...
|
||||
|
||||
* `.get(key, default=None)` - *Return the value for a given key, or a default value. If multiple values for the key are present, only the first will be returned.*
|
||||
* `.keys()` - *Return the unique keys of the query parameters. Each key will be a `str` instance.*
|
||||
* `.values()` - *Return the values of the query parameters. Each value will be a list of one or more `str` instances.*
|
||||
* `.items()` - *Return the key value pairs of the query parameters. Each item will be a two-tuple including a `str` instance as the key, and a list of one or more `str` instances as the value.*
|
||||
|
||||
The following methods are also available for accessing query parameters as a multidict...
|
||||
|
||||
* `.get_all(key)` - *Return all the values for a given key. Returned as a list of zero or more `str` instances.*
|
||||
* `.multi_items()` - *Return the key value pairs of the query parameters. Each item will be a two-tuple `(str, str)`. Repeated keys may occur.*
|
||||
* `.multi_dict()` - *Return the query parameters as a dictionary, with each value being a list of one or more `str` instances.*
|
||||
|
||||
### Modifying query parameters
|
||||
|
||||
The following methods can be used to create modified query parameter instances...
|
||||
|
||||
* `.copy_set(key, value)`
|
||||
* `.copy_append(key, value)`
|
||||
* `.copy_remove(key)`
|
||||
* `.copy_update(params)`
|
||||
|
||||
---
|
||||
|
||||
<span class="link-prev">← [Responses](responses.md)</span>
|
||||
<span class="link-next">[Headers](headers.md) →</span>
|
||||
<span> </span>
|
||||
@ -1,106 +0,0 @@
|
||||
from .__version__ import __description__, __title__, __version__
|
||||
from ._api import *
|
||||
from ._auth import *
|
||||
from ._client import *
|
||||
from ._config import *
|
||||
from ._content import *
|
||||
from ._exceptions import *
|
||||
from ._models import *
|
||||
from ._status_codes import *
|
||||
from ._transports import *
|
||||
from ._types import *
|
||||
from ._urls import *
|
||||
|
||||
try:
|
||||
from ._main import main
|
||||
except ImportError: # pragma: no cover
|
||||
|
||||
def main() -> None: # type: ignore
|
||||
import sys
|
||||
|
||||
print(
|
||||
"The httpx command line client could not run because the required "
|
||||
"dependencies were not installed.\nMake sure you've installed "
|
||||
"everything with: pip install 'httpx[cli]'"
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"__description__",
|
||||
"__title__",
|
||||
"__version__",
|
||||
"ASGITransport",
|
||||
"AsyncBaseTransport",
|
||||
"AsyncByteStream",
|
||||
"AsyncClient",
|
||||
"AsyncHTTPTransport",
|
||||
"Auth",
|
||||
"BaseTransport",
|
||||
"BasicAuth",
|
||||
"ByteStream",
|
||||
"Client",
|
||||
"CloseError",
|
||||
"codes",
|
||||
"ConnectError",
|
||||
"ConnectTimeout",
|
||||
"CookieConflict",
|
||||
"Cookies",
|
||||
"create_ssl_context",
|
||||
"DecodingError",
|
||||
"delete",
|
||||
"DigestAuth",
|
||||
"FunctionAuth",
|
||||
"get",
|
||||
"head",
|
||||
"Headers",
|
||||
"HTTPError",
|
||||
"HTTPStatusError",
|
||||
"HTTPTransport",
|
||||
"InvalidURL",
|
||||
"Limits",
|
||||
"LocalProtocolError",
|
||||
"main",
|
||||
"MockTransport",
|
||||
"NetRCAuth",
|
||||
"NetworkError",
|
||||
"options",
|
||||
"patch",
|
||||
"PoolTimeout",
|
||||
"post",
|
||||
"ProtocolError",
|
||||
"Proxy",
|
||||
"ProxyError",
|
||||
"put",
|
||||
"QueryParams",
|
||||
"ReadError",
|
||||
"ReadTimeout",
|
||||
"RemoteProtocolError",
|
||||
"request",
|
||||
"Request",
|
||||
"RequestError",
|
||||
"RequestNotRead",
|
||||
"Response",
|
||||
"ResponseNotRead",
|
||||
"stream",
|
||||
"StreamClosed",
|
||||
"StreamConsumed",
|
||||
"StreamError",
|
||||
"SyncByteStream",
|
||||
"Timeout",
|
||||
"TimeoutException",
|
||||
"TooManyRedirects",
|
||||
"TransportError",
|
||||
"UnsupportedProtocol",
|
||||
"URL",
|
||||
"USE_CLIENT_DEFAULT",
|
||||
"WriteError",
|
||||
"WriteTimeout",
|
||||
"WSGITransport",
|
||||
]
|
||||
|
||||
|
||||
__locals = locals()
|
||||
for __name in __all__:
|
||||
if not __name.startswith("__"):
|
||||
setattr(__locals[__name], "__module__", "httpx") # noqa
|
||||
@ -1,3 +0,0 @@
|
||||
__title__ = "httpx"
|
||||
__description__ = "A next generation HTTP client, for Python 3."
|
||||
__version__ = "0.28.1"
|
||||
438
httpx/_api.py
438
httpx/_api.py
@ -1,438 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import typing
|
||||
from contextlib import contextmanager
|
||||
|
||||
from ._client import Client
|
||||
from ._config import DEFAULT_TIMEOUT_CONFIG
|
||||
from ._models import Response
|
||||
from ._types import (
|
||||
AuthTypes,
|
||||
CookieTypes,
|
||||
HeaderTypes,
|
||||
ProxyTypes,
|
||||
QueryParamTypes,
|
||||
RequestContent,
|
||||
RequestData,
|
||||
RequestFiles,
|
||||
TimeoutTypes,
|
||||
)
|
||||
from ._urls import URL
|
||||
|
||||
if typing.TYPE_CHECKING:
|
||||
import ssl # pragma: no cover
|
||||
|
||||
|
||||
__all__ = [
|
||||
"delete",
|
||||
"get",
|
||||
"head",
|
||||
"options",
|
||||
"patch",
|
||||
"post",
|
||||
"put",
|
||||
"request",
|
||||
"stream",
|
||||
]
|
||||
|
||||
|
||||
def request(
|
||||
method: str,
|
||||
url: URL | str,
|
||||
*,
|
||||
params: QueryParamTypes | None = None,
|
||||
content: RequestContent | None = None,
|
||||
data: RequestData | None = None,
|
||||
files: RequestFiles | None = None,
|
||||
json: typing.Any | None = None,
|
||||
headers: HeaderTypes | None = None,
|
||||
cookies: CookieTypes | None = None,
|
||||
auth: AuthTypes | None = None,
|
||||
proxy: ProxyTypes | None = None,
|
||||
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
|
||||
follow_redirects: bool = False,
|
||||
verify: ssl.SSLContext | str | bool = True,
|
||||
trust_env: bool = True,
|
||||
) -> Response:
|
||||
"""
|
||||
Sends an HTTP request.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
* **method** - HTTP method for the new `Request` object: `GET`, `OPTIONS`,
|
||||
`HEAD`, `POST`, `PUT`, `PATCH`, or `DELETE`.
|
||||
* **url** - URL for the new `Request` object.
|
||||
* **params** - *(optional)* Query parameters to include in the URL, as a
|
||||
string, dictionary, or sequence of two-tuples.
|
||||
* **content** - *(optional)* Binary content to include in the body of the
|
||||
request, as bytes or a byte iterator.
|
||||
* **data** - *(optional)* Form data to include in the body of the request,
|
||||
as a dictionary.
|
||||
* **files** - *(optional)* A dictionary of upload files to include in the
|
||||
body of the request.
|
||||
* **json** - *(optional)* A JSON serializable object to include in the body
|
||||
of the request.
|
||||
* **headers** - *(optional)* Dictionary of HTTP headers to include in the
|
||||
request.
|
||||
* **cookies** - *(optional)* Dictionary of Cookie items to include in the
|
||||
request.
|
||||
* **auth** - *(optional)* An authentication class to use when sending the
|
||||
request.
|
||||
* **proxy** - *(optional)* A proxy URL where all the traffic should be routed.
|
||||
* **timeout** - *(optional)* The timeout configuration to use when sending
|
||||
the request.
|
||||
* **follow_redirects** - *(optional)* Enables or disables HTTP redirects.
|
||||
* **verify** - *(optional)* Either `True` to use an SSL context with the
|
||||
default CA bundle, `False` to disable verification, or an instance of
|
||||
`ssl.SSLContext` to use a custom context.
|
||||
* **trust_env** - *(optional)* Enables or disables usage of environment
|
||||
variables for configuration.
|
||||
|
||||
**Returns:** `Response`
|
||||
|
||||
Usage:
|
||||
|
||||
```
|
||||
>>> import httpx
|
||||
>>> response = httpx.request('GET', 'https://httpbin.org/get')
|
||||
>>> response
|
||||
<Response [200 OK]>
|
||||
```
|
||||
"""
|
||||
with Client(
|
||||
cookies=cookies,
|
||||
proxy=proxy,
|
||||
verify=verify,
|
||||
timeout=timeout,
|
||||
trust_env=trust_env,
|
||||
) as client:
|
||||
return client.request(
|
||||
method=method,
|
||||
url=url,
|
||||
content=content,
|
||||
data=data,
|
||||
files=files,
|
||||
json=json,
|
||||
params=params,
|
||||
headers=headers,
|
||||
auth=auth,
|
||||
follow_redirects=follow_redirects,
|
||||
)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def stream(
|
||||
method: str,
|
||||
url: URL | str,
|
||||
*,
|
||||
params: QueryParamTypes | None = None,
|
||||
content: RequestContent | None = None,
|
||||
data: RequestData | None = None,
|
||||
files: RequestFiles | None = None,
|
||||
json: typing.Any | None = None,
|
||||
headers: HeaderTypes | None = None,
|
||||
cookies: CookieTypes | None = None,
|
||||
auth: AuthTypes | None = None,
|
||||
proxy: ProxyTypes | None = None,
|
||||
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
|
||||
follow_redirects: bool = False,
|
||||
verify: ssl.SSLContext | str | bool = True,
|
||||
trust_env: bool = True,
|
||||
) -> typing.Iterator[Response]:
|
||||
"""
|
||||
Alternative to `httpx.request()` that streams the response body
|
||||
instead of loading it into memory at once.
|
||||
|
||||
**Parameters**: See `httpx.request`.
|
||||
|
||||
See also: [Streaming Responses][0]
|
||||
|
||||
[0]: /quickstart#streaming-responses
|
||||
"""
|
||||
with Client(
|
||||
cookies=cookies,
|
||||
proxy=proxy,
|
||||
verify=verify,
|
||||
timeout=timeout,
|
||||
trust_env=trust_env,
|
||||
) as client:
|
||||
with client.stream(
|
||||
method=method,
|
||||
url=url,
|
||||
content=content,
|
||||
data=data,
|
||||
files=files,
|
||||
json=json,
|
||||
params=params,
|
||||
headers=headers,
|
||||
auth=auth,
|
||||
follow_redirects=follow_redirects,
|
||||
) as response:
|
||||
yield response
|
||||
|
||||
|
||||
def get(
|
||||
url: URL | str,
|
||||
*,
|
||||
params: QueryParamTypes | None = None,
|
||||
headers: HeaderTypes | None = None,
|
||||
cookies: CookieTypes | None = None,
|
||||
auth: AuthTypes | None = None,
|
||||
proxy: ProxyTypes | None = None,
|
||||
follow_redirects: bool = False,
|
||||
verify: ssl.SSLContext | str | bool = True,
|
||||
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
|
||||
trust_env: bool = True,
|
||||
) -> Response:
|
||||
"""
|
||||
Sends a `GET` request.
|
||||
|
||||
**Parameters**: See `httpx.request`.
|
||||
|
||||
Note that the `data`, `files`, `json` and `content` parameters are not available
|
||||
on this function, as `GET` requests should not include a request body.
|
||||
"""
|
||||
return request(
|
||||
"GET",
|
||||
url,
|
||||
params=params,
|
||||
headers=headers,
|
||||
cookies=cookies,
|
||||
auth=auth,
|
||||
proxy=proxy,
|
||||
follow_redirects=follow_redirects,
|
||||
verify=verify,
|
||||
timeout=timeout,
|
||||
trust_env=trust_env,
|
||||
)
|
||||
|
||||
|
||||
def options(
|
||||
url: URL | str,
|
||||
*,
|
||||
params: QueryParamTypes | None = None,
|
||||
headers: HeaderTypes | None = None,
|
||||
cookies: CookieTypes | None = None,
|
||||
auth: AuthTypes | None = None,
|
||||
proxy: ProxyTypes | None = None,
|
||||
follow_redirects: bool = False,
|
||||
verify: ssl.SSLContext | str | bool = True,
|
||||
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
|
||||
trust_env: bool = True,
|
||||
) -> Response:
|
||||
"""
|
||||
Sends an `OPTIONS` request.
|
||||
|
||||
**Parameters**: See `httpx.request`.
|
||||
|
||||
Note that the `data`, `files`, `json` and `content` parameters are not available
|
||||
on this function, as `OPTIONS` requests should not include a request body.
|
||||
"""
|
||||
return request(
|
||||
"OPTIONS",
|
||||
url,
|
||||
params=params,
|
||||
headers=headers,
|
||||
cookies=cookies,
|
||||
auth=auth,
|
||||
proxy=proxy,
|
||||
follow_redirects=follow_redirects,
|
||||
verify=verify,
|
||||
timeout=timeout,
|
||||
trust_env=trust_env,
|
||||
)
|
||||
|
||||
|
||||
def head(
|
||||
url: URL | str,
|
||||
*,
|
||||
params: QueryParamTypes | None = None,
|
||||
headers: HeaderTypes | None = None,
|
||||
cookies: CookieTypes | None = None,
|
||||
auth: AuthTypes | None = None,
|
||||
proxy: ProxyTypes | None = None,
|
||||
follow_redirects: bool = False,
|
||||
verify: ssl.SSLContext | str | bool = True,
|
||||
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
|
||||
trust_env: bool = True,
|
||||
) -> Response:
|
||||
"""
|
||||
Sends a `HEAD` request.
|
||||
|
||||
**Parameters**: See `httpx.request`.
|
||||
|
||||
Note that the `data`, `files`, `json` and `content` parameters are not available
|
||||
on this function, as `HEAD` requests should not include a request body.
|
||||
"""
|
||||
return request(
|
||||
"HEAD",
|
||||
url,
|
||||
params=params,
|
||||
headers=headers,
|
||||
cookies=cookies,
|
||||
auth=auth,
|
||||
proxy=proxy,
|
||||
follow_redirects=follow_redirects,
|
||||
verify=verify,
|
||||
timeout=timeout,
|
||||
trust_env=trust_env,
|
||||
)
|
||||
|
||||
|
||||
def post(
|
||||
url: URL | str,
|
||||
*,
|
||||
content: RequestContent | None = None,
|
||||
data: RequestData | None = None,
|
||||
files: RequestFiles | None = None,
|
||||
json: typing.Any | None = None,
|
||||
params: QueryParamTypes | None = None,
|
||||
headers: HeaderTypes | None = None,
|
||||
cookies: CookieTypes | None = None,
|
||||
auth: AuthTypes | None = None,
|
||||
proxy: ProxyTypes | None = None,
|
||||
follow_redirects: bool = False,
|
||||
verify: ssl.SSLContext | str | bool = True,
|
||||
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
|
||||
trust_env: bool = True,
|
||||
) -> Response:
|
||||
"""
|
||||
Sends a `POST` request.
|
||||
|
||||
**Parameters**: See `httpx.request`.
|
||||
"""
|
||||
return request(
|
||||
"POST",
|
||||
url,
|
||||
content=content,
|
||||
data=data,
|
||||
files=files,
|
||||
json=json,
|
||||
params=params,
|
||||
headers=headers,
|
||||
cookies=cookies,
|
||||
auth=auth,
|
||||
proxy=proxy,
|
||||
follow_redirects=follow_redirects,
|
||||
verify=verify,
|
||||
timeout=timeout,
|
||||
trust_env=trust_env,
|
||||
)
|
||||
|
||||
|
||||
def put(
|
||||
url: URL | str,
|
||||
*,
|
||||
content: RequestContent | None = None,
|
||||
data: RequestData | None = None,
|
||||
files: RequestFiles | None = None,
|
||||
json: typing.Any | None = None,
|
||||
params: QueryParamTypes | None = None,
|
||||
headers: HeaderTypes | None = None,
|
||||
cookies: CookieTypes | None = None,
|
||||
auth: AuthTypes | None = None,
|
||||
proxy: ProxyTypes | None = None,
|
||||
follow_redirects: bool = False,
|
||||
verify: ssl.SSLContext | str | bool = True,
|
||||
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
|
||||
trust_env: bool = True,
|
||||
) -> Response:
|
||||
"""
|
||||
Sends a `PUT` request.
|
||||
|
||||
**Parameters**: See `httpx.request`.
|
||||
"""
|
||||
return request(
|
||||
"PUT",
|
||||
url,
|
||||
content=content,
|
||||
data=data,
|
||||
files=files,
|
||||
json=json,
|
||||
params=params,
|
||||
headers=headers,
|
||||
cookies=cookies,
|
||||
auth=auth,
|
||||
proxy=proxy,
|
||||
follow_redirects=follow_redirects,
|
||||
verify=verify,
|
||||
timeout=timeout,
|
||||
trust_env=trust_env,
|
||||
)
|
||||
|
||||
|
||||
def patch(
|
||||
url: URL | str,
|
||||
*,
|
||||
content: RequestContent | None = None,
|
||||
data: RequestData | None = None,
|
||||
files: RequestFiles | None = None,
|
||||
json: typing.Any | None = None,
|
||||
params: QueryParamTypes | None = None,
|
||||
headers: HeaderTypes | None = None,
|
||||
cookies: CookieTypes | None = None,
|
||||
auth: AuthTypes | None = None,
|
||||
proxy: ProxyTypes | None = None,
|
||||
follow_redirects: bool = False,
|
||||
verify: ssl.SSLContext | str | bool = True,
|
||||
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
|
||||
trust_env: bool = True,
|
||||
) -> Response:
|
||||
"""
|
||||
Sends a `PATCH` request.
|
||||
|
||||
**Parameters**: See `httpx.request`.
|
||||
"""
|
||||
return request(
|
||||
"PATCH",
|
||||
url,
|
||||
content=content,
|
||||
data=data,
|
||||
files=files,
|
||||
json=json,
|
||||
params=params,
|
||||
headers=headers,
|
||||
cookies=cookies,
|
||||
auth=auth,
|
||||
proxy=proxy,
|
||||
follow_redirects=follow_redirects,
|
||||
verify=verify,
|
||||
timeout=timeout,
|
||||
trust_env=trust_env,
|
||||
)
|
||||
|
||||
|
||||
def delete(
|
||||
url: URL | str,
|
||||
*,
|
||||
params: QueryParamTypes | None = None,
|
||||
headers: HeaderTypes | None = None,
|
||||
cookies: CookieTypes | None = None,
|
||||
auth: AuthTypes | None = None,
|
||||
proxy: ProxyTypes | None = None,
|
||||
follow_redirects: bool = False,
|
||||
timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
|
||||
verify: ssl.SSLContext | str | bool = True,
|
||||
trust_env: bool = True,
|
||||
) -> Response:
|
||||
"""
|
||||
Sends a `DELETE` request.
|
||||
|
||||
**Parameters**: See `httpx.request`.
|
||||
|
||||
Note that the `data`, `files`, `json` and `content` parameters are not available
|
||||
on this function, as `DELETE` requests should not include a request body.
|
||||
"""
|
||||
return request(
|
||||
"DELETE",
|
||||
url,
|
||||
params=params,
|
||||
headers=headers,
|
||||
cookies=cookies,
|
||||
auth=auth,
|
||||
proxy=proxy,
|
||||
follow_redirects=follow_redirects,
|
||||
verify=verify,
|
||||
timeout=timeout,
|
||||
trust_env=trust_env,
|
||||
)
|
||||
348
httpx/_auth.py
348
httpx/_auth.py
@ -1,348 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import os
|
||||
import re
|
||||
import time
|
||||
import typing
|
||||
from base64 import b64encode
|
||||
from urllib.request import parse_http_list
|
||||
|
||||
from ._exceptions import ProtocolError
|
||||
from ._models import Cookies, Request, Response
|
||||
from ._utils import to_bytes, to_str, unquote
|
||||
|
||||
if typing.TYPE_CHECKING: # pragma: no cover
|
||||
from hashlib import _Hash
|
||||
|
||||
|
||||
__all__ = ["Auth", "BasicAuth", "DigestAuth", "FunctionAuth", "NetRCAuth"]
|
||||
|
||||
|
||||
class Auth:
|
||||
"""
|
||||
Base class for all authentication schemes.
|
||||
|
||||
To implement a custom authentication scheme, subclass `Auth` and override
|
||||
the `.auth_flow()` method.
|
||||
|
||||
If the authentication scheme does I/O such as disk access or network calls, or uses
|
||||
synchronization primitives such as locks, you should override `.sync_auth_flow()`
|
||||
and/or `.async_auth_flow()` instead of `.auth_flow()` to provide specialized
|
||||
implementations that will be used by `Client` and `AsyncClient` respectively.
|
||||
"""
|
||||
|
||||
requires_request_body = False
|
||||
requires_response_body = False
|
||||
|
||||
def auth_flow(self, request: Request) -> typing.Generator[Request, Response, None]:
|
||||
"""
|
||||
Execute the authentication flow.
|
||||
|
||||
To dispatch a request, `yield` it:
|
||||
|
||||
```
|
||||
yield request
|
||||
```
|
||||
|
||||
The client will `.send()` the response back into the flow generator. You can
|
||||
access it like so:
|
||||
|
||||
```
|
||||
response = yield request
|
||||
```
|
||||
|
||||
A `return` (or reaching the end of the generator) will result in the
|
||||
client returning the last response obtained from the server.
|
||||
|
||||
You can dispatch as many requests as is necessary.
|
||||
"""
|
||||
yield request
|
||||
|
||||
def sync_auth_flow(
|
||||
self, request: Request
|
||||
) -> typing.Generator[Request, Response, None]:
|
||||
"""
|
||||
Execute the authentication flow synchronously.
|
||||
|
||||
By default, this defers to `.auth_flow()`. You should override this method
|
||||
when the authentication scheme does I/O and/or uses concurrency primitives.
|
||||
"""
|
||||
if self.requires_request_body:
|
||||
request.read()
|
||||
|
||||
flow = self.auth_flow(request)
|
||||
request = next(flow)
|
||||
|
||||
while True:
|
||||
response = yield request
|
||||
if self.requires_response_body:
|
||||
response.read()
|
||||
|
||||
try:
|
||||
request = flow.send(response)
|
||||
except StopIteration:
|
||||
break
|
||||
|
||||
async def async_auth_flow(
|
||||
self, request: Request
|
||||
) -> typing.AsyncGenerator[Request, Response]:
|
||||
"""
|
||||
Execute the authentication flow asynchronously.
|
||||
|
||||
By default, this defers to `.auth_flow()`. You should override this method
|
||||
when the authentication scheme does I/O and/or uses concurrency primitives.
|
||||
"""
|
||||
if self.requires_request_body:
|
||||
await request.aread()
|
||||
|
||||
flow = self.auth_flow(request)
|
||||
request = next(flow)
|
||||
|
||||
while True:
|
||||
response = yield request
|
||||
if self.requires_response_body:
|
||||
await response.aread()
|
||||
|
||||
try:
|
||||
request = flow.send(response)
|
||||
except StopIteration:
|
||||
break
|
||||
|
||||
|
||||
class FunctionAuth(Auth):
|
||||
"""
|
||||
Allows the 'auth' argument to be passed as a simple callable function,
|
||||
that takes the request, and returns a new, modified request.
|
||||
"""
|
||||
|
||||
def __init__(self, func: typing.Callable[[Request], Request]) -> None:
|
||||
self._func = func
|
||||
|
||||
def auth_flow(self, request: Request) -> typing.Generator[Request, Response, None]:
|
||||
yield self._func(request)
|
||||
|
||||
|
||||
class BasicAuth(Auth):
|
||||
"""
|
||||
Allows the 'auth' argument to be passed as a (username, password) pair,
|
||||
and uses HTTP Basic authentication.
|
||||
"""
|
||||
|
||||
def __init__(self, username: str | bytes, password: str | bytes) -> None:
|
||||
self._auth_header = self._build_auth_header(username, password)
|
||||
|
||||
def auth_flow(self, request: Request) -> typing.Generator[Request, Response, None]:
|
||||
request.headers["Authorization"] = self._auth_header
|
||||
yield request
|
||||
|
||||
def _build_auth_header(self, username: str | bytes, password: str | bytes) -> str:
|
||||
userpass = b":".join((to_bytes(username), to_bytes(password)))
|
||||
token = b64encode(userpass).decode()
|
||||
return f"Basic {token}"
|
||||
|
||||
|
||||
class NetRCAuth(Auth):
|
||||
"""
|
||||
Use a 'netrc' file to lookup basic auth credentials based on the url host.
|
||||
"""
|
||||
|
||||
def __init__(self, file: str | None = None) -> None:
|
||||
# Lazily import 'netrc'.
|
||||
# There's no need for us to load this module unless 'NetRCAuth' is being used.
|
||||
import netrc
|
||||
|
||||
self._netrc_info = netrc.netrc(file)
|
||||
|
||||
def auth_flow(self, request: Request) -> typing.Generator[Request, Response, None]:
|
||||
auth_info = self._netrc_info.authenticators(request.url.host)
|
||||
if auth_info is None or not auth_info[2]:
|
||||
# The netrc file did not have authentication credentials for this host.
|
||||
yield request
|
||||
else:
|
||||
# Build a basic auth header with credentials from the netrc file.
|
||||
request.headers["Authorization"] = self._build_auth_header(
|
||||
username=auth_info[0], password=auth_info[2]
|
||||
)
|
||||
yield request
|
||||
|
||||
def _build_auth_header(self, username: str | bytes, password: str | bytes) -> str:
|
||||
userpass = b":".join((to_bytes(username), to_bytes(password)))
|
||||
token = b64encode(userpass).decode()
|
||||
return f"Basic {token}"
|
||||
|
||||
|
||||
class DigestAuth(Auth):
|
||||
_ALGORITHM_TO_HASH_FUNCTION: dict[str, typing.Callable[[bytes], _Hash]] = {
|
||||
"MD5": hashlib.md5,
|
||||
"MD5-SESS": hashlib.md5,
|
||||
"SHA": hashlib.sha1,
|
||||
"SHA-SESS": hashlib.sha1,
|
||||
"SHA-256": hashlib.sha256,
|
||||
"SHA-256-SESS": hashlib.sha256,
|
||||
"SHA-512": hashlib.sha512,
|
||||
"SHA-512-SESS": hashlib.sha512,
|
||||
}
|
||||
|
||||
def __init__(self, username: str | bytes, password: str | bytes) -> None:
|
||||
self._username = to_bytes(username)
|
||||
self._password = to_bytes(password)
|
||||
self._last_challenge: _DigestAuthChallenge | None = None
|
||||
self._nonce_count = 1
|
||||
|
||||
def auth_flow(self, request: Request) -> typing.Generator[Request, Response, None]:
|
||||
if self._last_challenge:
|
||||
request.headers["Authorization"] = self._build_auth_header(
|
||||
request, self._last_challenge
|
||||
)
|
||||
|
||||
response = yield request
|
||||
|
||||
if response.status_code != 401 or "www-authenticate" not in response.headers:
|
||||
# If the response is not a 401 then we don't
|
||||
# need to build an authenticated request.
|
||||
return
|
||||
|
||||
for auth_header in response.headers.get_list("www-authenticate"):
|
||||
if auth_header.lower().startswith("digest "):
|
||||
break
|
||||
else:
|
||||
# If the response does not include a 'WWW-Authenticate: Digest ...'
|
||||
# header, then we don't need to build an authenticated request.
|
||||
return
|
||||
|
||||
self._last_challenge = self._parse_challenge(request, response, auth_header)
|
||||
self._nonce_count = 1
|
||||
|
||||
request.headers["Authorization"] = self._build_auth_header(
|
||||
request, self._last_challenge
|
||||
)
|
||||
if response.cookies:
|
||||
Cookies(response.cookies).set_cookie_header(request=request)
|
||||
yield request
|
||||
|
||||
def _parse_challenge(
|
||||
self, request: Request, response: Response, auth_header: str
|
||||
) -> _DigestAuthChallenge:
|
||||
"""
|
||||
Returns a challenge from a Digest WWW-Authenticate header.
|
||||
These take the form of:
|
||||
`Digest realm="realm@host.com",qop="auth,auth-int",nonce="abc",opaque="xyz"`
|
||||
"""
|
||||
scheme, _, fields = auth_header.partition(" ")
|
||||
|
||||
# This method should only ever have been called with a Digest auth header.
|
||||
assert scheme.lower() == "digest"
|
||||
|
||||
header_dict: dict[str, str] = {}
|
||||
for field in parse_http_list(fields):
|
||||
key, value = field.strip().split("=", 1)
|
||||
header_dict[key] = unquote(value)
|
||||
|
||||
try:
|
||||
realm = header_dict["realm"].encode()
|
||||
nonce = header_dict["nonce"].encode()
|
||||
algorithm = header_dict.get("algorithm", "MD5")
|
||||
opaque = header_dict["opaque"].encode() if "opaque" in header_dict else None
|
||||
qop = header_dict["qop"].encode() if "qop" in header_dict else None
|
||||
return _DigestAuthChallenge(
|
||||
realm=realm, nonce=nonce, algorithm=algorithm, opaque=opaque, qop=qop
|
||||
)
|
||||
except KeyError as exc:
|
||||
message = "Malformed Digest WWW-Authenticate header"
|
||||
raise ProtocolError(message, request=request) from exc
|
||||
|
||||
def _build_auth_header(
|
||||
self, request: Request, challenge: _DigestAuthChallenge
|
||||
) -> str:
|
||||
hash_func = self._ALGORITHM_TO_HASH_FUNCTION[challenge.algorithm.upper()]
|
||||
|
||||
def digest(data: bytes) -> bytes:
|
||||
return hash_func(data).hexdigest().encode()
|
||||
|
||||
A1 = b":".join((self._username, challenge.realm, self._password))
|
||||
|
||||
path = request.url.raw_path
|
||||
A2 = b":".join((request.method.encode(), path))
|
||||
# TODO: implement auth-int
|
||||
HA2 = digest(A2)
|
||||
|
||||
nc_value = b"%08x" % self._nonce_count
|
||||
cnonce = self._get_client_nonce(self._nonce_count, challenge.nonce)
|
||||
self._nonce_count += 1
|
||||
|
||||
HA1 = digest(A1)
|
||||
if challenge.algorithm.lower().endswith("-sess"):
|
||||
HA1 = digest(b":".join((HA1, challenge.nonce, cnonce)))
|
||||
|
||||
qop = self._resolve_qop(challenge.qop, request=request)
|
||||
if qop is None:
|
||||
# Following RFC 2069
|
||||
digest_data = [HA1, challenge.nonce, HA2]
|
||||
else:
|
||||
# Following RFC 2617/7616
|
||||
digest_data = [HA1, challenge.nonce, nc_value, cnonce, qop, HA2]
|
||||
|
||||
format_args = {
|
||||
"username": self._username,
|
||||
"realm": challenge.realm,
|
||||
"nonce": challenge.nonce,
|
||||
"uri": path,
|
||||
"response": digest(b":".join(digest_data)),
|
||||
"algorithm": challenge.algorithm.encode(),
|
||||
}
|
||||
if challenge.opaque:
|
||||
format_args["opaque"] = challenge.opaque
|
||||
if qop:
|
||||
format_args["qop"] = b"auth"
|
||||
format_args["nc"] = nc_value
|
||||
format_args["cnonce"] = cnonce
|
||||
|
||||
return "Digest " + self._get_header_value(format_args)
|
||||
|
||||
def _get_client_nonce(self, nonce_count: int, nonce: bytes) -> bytes:
|
||||
s = str(nonce_count).encode()
|
||||
s += nonce
|
||||
s += time.ctime().encode()
|
||||
s += os.urandom(8)
|
||||
|
||||
return hashlib.sha1(s).hexdigest()[:16].encode()
|
||||
|
||||
def _get_header_value(self, header_fields: dict[str, bytes]) -> str:
|
||||
NON_QUOTED_FIELDS = ("algorithm", "qop", "nc")
|
||||
QUOTED_TEMPLATE = '{}="{}"'
|
||||
NON_QUOTED_TEMPLATE = "{}={}"
|
||||
|
||||
header_value = ""
|
||||
for i, (field, value) in enumerate(header_fields.items()):
|
||||
if i > 0:
|
||||
header_value += ", "
|
||||
template = (
|
||||
QUOTED_TEMPLATE
|
||||
if field not in NON_QUOTED_FIELDS
|
||||
else NON_QUOTED_TEMPLATE
|
||||
)
|
||||
header_value += template.format(field, to_str(value))
|
||||
|
||||
return header_value
|
||||
|
||||
def _resolve_qop(self, qop: bytes | None, request: Request) -> bytes | None:
|
||||
if qop is None:
|
||||
return None
|
||||
qops = re.split(b", ?", qop)
|
||||
if b"auth" in qops:
|
||||
return b"auth"
|
||||
|
||||
if qops == [b"auth-int"]:
|
||||
raise NotImplementedError("Digest auth-int support is not yet implemented")
|
||||
|
||||
message = f'Unexpected qop value "{qop!r}" in digest auth'
|
||||
raise ProtocolError(message, request=request)
|
||||
|
||||
|
||||
class _DigestAuthChallenge(typing.NamedTuple):
|
||||
realm: bytes
|
||||
nonce: bytes
|
||||
algorithm: str
|
||||
opaque: bytes | None
|
||||
qop: bytes | None
|
||||
2019
httpx/_client.py
2019
httpx/_client.py
File diff suppressed because it is too large
Load Diff
248
httpx/_config.py
248
httpx/_config.py
@ -1,248 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import typing
|
||||
|
||||
from ._models import Headers
|
||||
from ._types import CertTypes, HeaderTypes, TimeoutTypes
|
||||
from ._urls import URL
|
||||
|
||||
if typing.TYPE_CHECKING:
|
||||
import ssl # pragma: no cover
|
||||
|
||||
__all__ = ["Limits", "Proxy", "Timeout", "create_ssl_context"]
|
||||
|
||||
|
||||
class UnsetType:
|
||||
pass # pragma: no cover
|
||||
|
||||
|
||||
UNSET = UnsetType()
|
||||
|
||||
|
||||
def create_ssl_context(
|
||||
verify: ssl.SSLContext | str | bool = True,
|
||||
cert: CertTypes | None = None,
|
||||
trust_env: bool = True,
|
||||
) -> ssl.SSLContext:
|
||||
import ssl
|
||||
import warnings
|
||||
|
||||
import certifi
|
||||
|
||||
if verify is True:
|
||||
if trust_env and os.environ.get("SSL_CERT_FILE"): # pragma: nocover
|
||||
ctx = ssl.create_default_context(cafile=os.environ["SSL_CERT_FILE"])
|
||||
elif trust_env and os.environ.get("SSL_CERT_DIR"): # pragma: nocover
|
||||
ctx = ssl.create_default_context(capath=os.environ["SSL_CERT_DIR"])
|
||||
else:
|
||||
# Default case...
|
||||
ctx = ssl.create_default_context(cafile=certifi.where())
|
||||
elif verify is False:
|
||||
ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
|
||||
ctx.check_hostname = False
|
||||
ctx.verify_mode = ssl.CERT_NONE
|
||||
elif isinstance(verify, str): # pragma: nocover
|
||||
message = (
|
||||
"`verify=<str>` is deprecated. "
|
||||
"Use `verify=ssl.create_default_context(cafile=...)` "
|
||||
"or `verify=ssl.create_default_context(capath=...)` instead."
|
||||
)
|
||||
warnings.warn(message, DeprecationWarning)
|
||||
if os.path.isdir(verify):
|
||||
return ssl.create_default_context(capath=verify)
|
||||
return ssl.create_default_context(cafile=verify)
|
||||
else:
|
||||
ctx = verify
|
||||
|
||||
if cert: # pragma: nocover
|
||||
message = (
|
||||
"`cert=...` is deprecated. Use `verify=<ssl_context>` instead,"
|
||||
"with `.load_cert_chain()` to configure the certificate chain."
|
||||
)
|
||||
warnings.warn(message, DeprecationWarning)
|
||||
if isinstance(cert, str):
|
||||
ctx.load_cert_chain(cert)
|
||||
else:
|
||||
ctx.load_cert_chain(*cert)
|
||||
|
||||
return ctx
|
||||
|
||||
|
||||
class Timeout:
|
||||
"""
|
||||
Timeout configuration.
|
||||
|
||||
**Usage**:
|
||||
|
||||
Timeout(None) # No timeouts.
|
||||
Timeout(5.0) # 5s timeout on all operations.
|
||||
Timeout(None, connect=5.0) # 5s timeout on connect, no other timeouts.
|
||||
Timeout(5.0, connect=10.0) # 10s timeout on connect. 5s timeout elsewhere.
|
||||
Timeout(5.0, pool=None) # No timeout on acquiring connection from pool.
|
||||
# 5s timeout elsewhere.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
timeout: TimeoutTypes | UnsetType = UNSET,
|
||||
*,
|
||||
connect: None | float | UnsetType = UNSET,
|
||||
read: None | float | UnsetType = UNSET,
|
||||
write: None | float | UnsetType = UNSET,
|
||||
pool: None | float | UnsetType = UNSET,
|
||||
) -> None:
|
||||
if isinstance(timeout, Timeout):
|
||||
# Passed as a single explicit Timeout.
|
||||
assert connect is UNSET
|
||||
assert read is UNSET
|
||||
assert write is UNSET
|
||||
assert pool is UNSET
|
||||
self.connect = timeout.connect # type: typing.Optional[float]
|
||||
self.read = timeout.read # type: typing.Optional[float]
|
||||
self.write = timeout.write # type: typing.Optional[float]
|
||||
self.pool = timeout.pool # type: typing.Optional[float]
|
||||
elif isinstance(timeout, tuple):
|
||||
# Passed as a tuple.
|
||||
self.connect = timeout[0]
|
||||
self.read = timeout[1]
|
||||
self.write = None if len(timeout) < 3 else timeout[2]
|
||||
self.pool = None if len(timeout) < 4 else timeout[3]
|
||||
elif not (
|
||||
isinstance(connect, UnsetType)
|
||||
or isinstance(read, UnsetType)
|
||||
or isinstance(write, UnsetType)
|
||||
or isinstance(pool, UnsetType)
|
||||
):
|
||||
self.connect = connect
|
||||
self.read = read
|
||||
self.write = write
|
||||
self.pool = pool
|
||||
else:
|
||||
if isinstance(timeout, UnsetType):
|
||||
raise ValueError(
|
||||
"httpx.Timeout must either include a default, or set all "
|
||||
"four parameters explicitly."
|
||||
)
|
||||
self.connect = timeout if isinstance(connect, UnsetType) else connect
|
||||
self.read = timeout if isinstance(read, UnsetType) else read
|
||||
self.write = timeout if isinstance(write, UnsetType) else write
|
||||
self.pool = timeout if isinstance(pool, UnsetType) else pool
|
||||
|
||||
def as_dict(self) -> dict[str, float | None]:
|
||||
return {
|
||||
"connect": self.connect,
|
||||
"read": self.read,
|
||||
"write": self.write,
|
||||
"pool": self.pool,
|
||||
}
|
||||
|
||||
def __eq__(self, other: typing.Any) -> bool:
|
||||
return (
|
||||
isinstance(other, self.__class__)
|
||||
and self.connect == other.connect
|
||||
and self.read == other.read
|
||||
and self.write == other.write
|
||||
and self.pool == other.pool
|
||||
)
|
||||
|
||||
def __repr__(self) -> str:
|
||||
class_name = self.__class__.__name__
|
||||
if len({self.connect, self.read, self.write, self.pool}) == 1:
|
||||
return f"{class_name}(timeout={self.connect})"
|
||||
return (
|
||||
f"{class_name}(connect={self.connect}, "
|
||||
f"read={self.read}, write={self.write}, pool={self.pool})"
|
||||
)
|
||||
|
||||
|
||||
class Limits:
|
||||
"""
|
||||
Configuration for limits to various client behaviors.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
* **max_connections** - The maximum number of concurrent connections that may be
|
||||
established.
|
||||
* **max_keepalive_connections** - Allow the connection pool to maintain
|
||||
keep-alive connections below this point. Should be less than or equal
|
||||
to `max_connections`.
|
||||
* **keepalive_expiry** - Time limit on idle keep-alive connections in seconds.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
max_connections: int | None = None,
|
||||
max_keepalive_connections: int | None = None,
|
||||
keepalive_expiry: float | None = 5.0,
|
||||
) -> None:
|
||||
self.max_connections = max_connections
|
||||
self.max_keepalive_connections = max_keepalive_connections
|
||||
self.keepalive_expiry = keepalive_expiry
|
||||
|
||||
def __eq__(self, other: typing.Any) -> bool:
|
||||
return (
|
||||
isinstance(other, self.__class__)
|
||||
and self.max_connections == other.max_connections
|
||||
and self.max_keepalive_connections == other.max_keepalive_connections
|
||||
and self.keepalive_expiry == other.keepalive_expiry
|
||||
)
|
||||
|
||||
def __repr__(self) -> str:
|
||||
class_name = self.__class__.__name__
|
||||
return (
|
||||
f"{class_name}(max_connections={self.max_connections}, "
|
||||
f"max_keepalive_connections={self.max_keepalive_connections}, "
|
||||
f"keepalive_expiry={self.keepalive_expiry})"
|
||||
)
|
||||
|
||||
|
||||
class Proxy:
|
||||
def __init__(
|
||||
self,
|
||||
url: URL | str,
|
||||
*,
|
||||
ssl_context: ssl.SSLContext | None = None,
|
||||
auth: tuple[str, str] | None = None,
|
||||
headers: HeaderTypes | None = None,
|
||||
) -> None:
|
||||
url = URL(url)
|
||||
headers = Headers(headers)
|
||||
|
||||
if url.scheme not in ("http", "https", "socks5", "socks5h"):
|
||||
raise ValueError(f"Unknown scheme for proxy URL {url!r}")
|
||||
|
||||
if url.username or url.password:
|
||||
# Remove any auth credentials from the URL.
|
||||
auth = (url.username, url.password)
|
||||
url = url.copy_with(username=None, password=None)
|
||||
|
||||
self.url = url
|
||||
self.auth = auth
|
||||
self.headers = headers
|
||||
self.ssl_context = ssl_context
|
||||
|
||||
@property
|
||||
def raw_auth(self) -> tuple[bytes, bytes] | None:
|
||||
# The proxy authentication as raw bytes.
|
||||
return (
|
||||
None
|
||||
if self.auth is None
|
||||
else (self.auth[0].encode("utf-8"), self.auth[1].encode("utf-8"))
|
||||
)
|
||||
|
||||
def __repr__(self) -> str:
|
||||
# The authentication is represented with the password component masked.
|
||||
auth = (self.auth[0], "********") if self.auth else None
|
||||
|
||||
# Build a nice concise representation.
|
||||
url_str = f"{str(self.url)!r}"
|
||||
auth_str = f", auth={auth!r}" if auth else ""
|
||||
headers_str = f", headers={dict(self.headers)!r}" if self.headers else ""
|
||||
return f"Proxy({url_str}{auth_str}{headers_str})"
|
||||
|
||||
|
||||
DEFAULT_TIMEOUT_CONFIG = Timeout(timeout=5.0)
|
||||
DEFAULT_LIMITS = Limits(max_connections=100, max_keepalive_connections=20)
|
||||
DEFAULT_MAX_REDIRECTS = 20
|
||||
@ -1,240 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import inspect
|
||||
import warnings
|
||||
from json import dumps as json_dumps
|
||||
from typing import (
|
||||
Any,
|
||||
AsyncIterable,
|
||||
AsyncIterator,
|
||||
Iterable,
|
||||
Iterator,
|
||||
Mapping,
|
||||
)
|
||||
from urllib.parse import urlencode
|
||||
|
||||
from ._exceptions import StreamClosed, StreamConsumed
|
||||
from ._multipart import MultipartStream
|
||||
from ._types import (
|
||||
AsyncByteStream,
|
||||
RequestContent,
|
||||
RequestData,
|
||||
RequestFiles,
|
||||
ResponseContent,
|
||||
SyncByteStream,
|
||||
)
|
||||
from ._utils import peek_filelike_length, primitive_value_to_str
|
||||
|
||||
__all__ = ["ByteStream"]
|
||||
|
||||
|
||||
class ByteStream(AsyncByteStream, SyncByteStream):
|
||||
def __init__(self, stream: bytes) -> None:
|
||||
self._stream = stream
|
||||
|
||||
def __iter__(self) -> Iterator[bytes]:
|
||||
yield self._stream
|
||||
|
||||
async def __aiter__(self) -> AsyncIterator[bytes]:
|
||||
yield self._stream
|
||||
|
||||
|
||||
class IteratorByteStream(SyncByteStream):
|
||||
CHUNK_SIZE = 65_536
|
||||
|
||||
def __init__(self, stream: Iterable[bytes]) -> None:
|
||||
self._stream = stream
|
||||
self._is_stream_consumed = False
|
||||
self._is_generator = inspect.isgenerator(stream)
|
||||
|
||||
def __iter__(self) -> Iterator[bytes]:
|
||||
if self._is_stream_consumed and self._is_generator:
|
||||
raise StreamConsumed()
|
||||
|
||||
self._is_stream_consumed = True
|
||||
if hasattr(self._stream, "read"):
|
||||
# File-like interfaces should use 'read' directly.
|
||||
chunk = self._stream.read(self.CHUNK_SIZE)
|
||||
while chunk:
|
||||
yield chunk
|
||||
chunk = self._stream.read(self.CHUNK_SIZE)
|
||||
else:
|
||||
# Otherwise iterate.
|
||||
for part in self._stream:
|
||||
yield part
|
||||
|
||||
|
||||
class AsyncIteratorByteStream(AsyncByteStream):
|
||||
CHUNK_SIZE = 65_536
|
||||
|
||||
def __init__(self, stream: AsyncIterable[bytes]) -> None:
|
||||
self._stream = stream
|
||||
self._is_stream_consumed = False
|
||||
self._is_generator = inspect.isasyncgen(stream)
|
||||
|
||||
async def __aiter__(self) -> AsyncIterator[bytes]:
|
||||
if self._is_stream_consumed and self._is_generator:
|
||||
raise StreamConsumed()
|
||||
|
||||
self._is_stream_consumed = True
|
||||
if hasattr(self._stream, "aread"):
|
||||
# File-like interfaces should use 'aread' directly.
|
||||
chunk = await self._stream.aread(self.CHUNK_SIZE)
|
||||
while chunk:
|
||||
yield chunk
|
||||
chunk = await self._stream.aread(self.CHUNK_SIZE)
|
||||
else:
|
||||
# Otherwise iterate.
|
||||
async for part in self._stream:
|
||||
yield part
|
||||
|
||||
|
||||
class UnattachedStream(AsyncByteStream, SyncByteStream):
|
||||
"""
|
||||
If a request or response is serialized using pickle, then it is no longer
|
||||
attached to a stream for I/O purposes. Any stream operations should result
|
||||
in `httpx.StreamClosed`.
|
||||
"""
|
||||
|
||||
def __iter__(self) -> Iterator[bytes]:
|
||||
raise StreamClosed()
|
||||
|
||||
async def __aiter__(self) -> AsyncIterator[bytes]:
|
||||
raise StreamClosed()
|
||||
yield b"" # pragma: no cover
|
||||
|
||||
|
||||
def encode_content(
|
||||
content: str | bytes | Iterable[bytes] | AsyncIterable[bytes],
|
||||
) -> tuple[dict[str, str], SyncByteStream | AsyncByteStream]:
|
||||
if isinstance(content, (bytes, str)):
|
||||
body = content.encode("utf-8") if isinstance(content, str) else content
|
||||
content_length = len(body)
|
||||
headers = {"Content-Length": str(content_length)} if body else {}
|
||||
return headers, ByteStream(body)
|
||||
|
||||
elif isinstance(content, Iterable) and not isinstance(content, dict):
|
||||
# `not isinstance(content, dict)` is a bit oddly specific, but it
|
||||
# catches a case that's easy for users to make in error, and would
|
||||
# otherwise pass through here, like any other bytes-iterable,
|
||||
# because `dict` happens to be iterable. See issue #2491.
|
||||
content_length_or_none = peek_filelike_length(content)
|
||||
|
||||
if content_length_or_none is None:
|
||||
headers = {"Transfer-Encoding": "chunked"}
|
||||
else:
|
||||
headers = {"Content-Length": str(content_length_or_none)}
|
||||
return headers, IteratorByteStream(content) # type: ignore
|
||||
|
||||
elif isinstance(content, AsyncIterable):
|
||||
headers = {"Transfer-Encoding": "chunked"}
|
||||
return headers, AsyncIteratorByteStream(content)
|
||||
|
||||
raise TypeError(f"Unexpected type for 'content', {type(content)!r}")
|
||||
|
||||
|
||||
def encode_urlencoded_data(
|
||||
data: RequestData,
|
||||
) -> tuple[dict[str, str], ByteStream]:
|
||||
plain_data = []
|
||||
for key, value in data.items():
|
||||
if isinstance(value, (list, tuple)):
|
||||
plain_data.extend([(key, primitive_value_to_str(item)) for item in value])
|
||||
else:
|
||||
plain_data.append((key, primitive_value_to_str(value)))
|
||||
body = urlencode(plain_data, doseq=True).encode("utf-8")
|
||||
content_length = str(len(body))
|
||||
content_type = "application/x-www-form-urlencoded"
|
||||
headers = {"Content-Length": content_length, "Content-Type": content_type}
|
||||
return headers, ByteStream(body)
|
||||
|
||||
|
||||
def encode_multipart_data(
|
||||
data: RequestData, files: RequestFiles, boundary: bytes | None
|
||||
) -> tuple[dict[str, str], MultipartStream]:
|
||||
multipart = MultipartStream(data=data, files=files, boundary=boundary)
|
||||
headers = multipart.get_headers()
|
||||
return headers, multipart
|
||||
|
||||
|
||||
def encode_text(text: str) -> tuple[dict[str, str], ByteStream]:
|
||||
body = text.encode("utf-8")
|
||||
content_length = str(len(body))
|
||||
content_type = "text/plain; charset=utf-8"
|
||||
headers = {"Content-Length": content_length, "Content-Type": content_type}
|
||||
return headers, ByteStream(body)
|
||||
|
||||
|
||||
def encode_html(html: str) -> tuple[dict[str, str], ByteStream]:
|
||||
body = html.encode("utf-8")
|
||||
content_length = str(len(body))
|
||||
content_type = "text/html; charset=utf-8"
|
||||
headers = {"Content-Length": content_length, "Content-Type": content_type}
|
||||
return headers, ByteStream(body)
|
||||
|
||||
|
||||
def encode_json(json: Any) -> tuple[dict[str, str], ByteStream]:
|
||||
body = json_dumps(
|
||||
json, ensure_ascii=False, separators=(",", ":"), allow_nan=False
|
||||
).encode("utf-8")
|
||||
content_length = str(len(body))
|
||||
content_type = "application/json"
|
||||
headers = {"Content-Length": content_length, "Content-Type": content_type}
|
||||
return headers, ByteStream(body)
|
||||
|
||||
|
||||
def encode_request(
|
||||
content: RequestContent | None = None,
|
||||
data: RequestData | None = None,
|
||||
files: RequestFiles | None = None,
|
||||
json: Any | None = None,
|
||||
boundary: bytes | None = None,
|
||||
) -> tuple[dict[str, str], SyncByteStream | AsyncByteStream]:
|
||||
"""
|
||||
Handles encoding the given `content`, `data`, `files`, and `json`,
|
||||
returning a two-tuple of (<headers>, <stream>).
|
||||
"""
|
||||
if data is not None and not isinstance(data, Mapping):
|
||||
# We prefer to separate `content=<bytes|str|byte iterator|bytes aiterator>`
|
||||
# for raw request content, and `data=<form data>` for url encoded or
|
||||
# multipart form content.
|
||||
#
|
||||
# However for compat with requests, we *do* still support
|
||||
# `data=<bytes...>` usages. We deal with that case here, treating it
|
||||
# as if `content=<...>` had been supplied instead.
|
||||
message = "Use 'content=<...>' to upload raw bytes/text content."
|
||||
warnings.warn(message, DeprecationWarning, stacklevel=2)
|
||||
return encode_content(data)
|
||||
|
||||
if content is not None:
|
||||
return encode_content(content)
|
||||
elif files:
|
||||
return encode_multipart_data(data or {}, files, boundary)
|
||||
elif data:
|
||||
return encode_urlencoded_data(data)
|
||||
elif json is not None:
|
||||
return encode_json(json)
|
||||
|
||||
return {}, ByteStream(b"")
|
||||
|
||||
|
||||
def encode_response(
|
||||
content: ResponseContent | None = None,
|
||||
text: str | None = None,
|
||||
html: str | None = None,
|
||||
json: Any | None = None,
|
||||
) -> tuple[dict[str, str], SyncByteStream | AsyncByteStream]:
|
||||
"""
|
||||
Handles encoding the given `content`, returning a two-tuple of
|
||||
(<headers>, <stream>).
|
||||
"""
|
||||
if content is not None:
|
||||
return encode_content(content)
|
||||
elif text is not None:
|
||||
return encode_text(text)
|
||||
elif html is not None:
|
||||
return encode_html(html)
|
||||
elif json is not None:
|
||||
return encode_json(json)
|
||||
|
||||
return {}, ByteStream(b"")
|
||||
@ -1,393 +0,0 @@
|
||||
"""
|
||||
Handlers for Content-Encoding.
|
||||
|
||||
See: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import codecs
|
||||
import io
|
||||
import typing
|
||||
import zlib
|
||||
|
||||
from ._exceptions import DecodingError
|
||||
|
||||
# Brotli support is optional
|
||||
try:
|
||||
# The C bindings in `brotli` are recommended for CPython.
|
||||
import brotli
|
||||
except ImportError: # pragma: no cover
|
||||
try:
|
||||
# The CFFI bindings in `brotlicffi` are recommended for PyPy
|
||||
# and other environments.
|
||||
import brotlicffi as brotli
|
||||
except ImportError:
|
||||
brotli = None
|
||||
|
||||
|
||||
# Zstandard support is optional
|
||||
try:
|
||||
import zstandard
|
||||
except ImportError: # pragma: no cover
|
||||
zstandard = None # type: ignore
|
||||
|
||||
|
||||
class ContentDecoder:
|
||||
def decode(self, data: bytes) -> bytes:
|
||||
raise NotImplementedError() # pragma: no cover
|
||||
|
||||
def flush(self) -> bytes:
|
||||
raise NotImplementedError() # pragma: no cover
|
||||
|
||||
|
||||
class IdentityDecoder(ContentDecoder):
|
||||
"""
|
||||
Handle unencoded data.
|
||||
"""
|
||||
|
||||
def decode(self, data: bytes) -> bytes:
|
||||
return data
|
||||
|
||||
def flush(self) -> bytes:
|
||||
return b""
|
||||
|
||||
|
||||
class DeflateDecoder(ContentDecoder):
|
||||
"""
|
||||
Handle 'deflate' decoding.
|
||||
|
||||
See: https://stackoverflow.com/questions/1838699
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.first_attempt = True
|
||||
self.decompressor = zlib.decompressobj()
|
||||
|
||||
def decode(self, data: bytes) -> bytes:
|
||||
was_first_attempt = self.first_attempt
|
||||
self.first_attempt = False
|
||||
try:
|
||||
return self.decompressor.decompress(data)
|
||||
except zlib.error as exc:
|
||||
if was_first_attempt:
|
||||
self.decompressor = zlib.decompressobj(-zlib.MAX_WBITS)
|
||||
return self.decode(data)
|
||||
raise DecodingError(str(exc)) from exc
|
||||
|
||||
def flush(self) -> bytes:
|
||||
try:
|
||||
return self.decompressor.flush()
|
||||
except zlib.error as exc: # pragma: no cover
|
||||
raise DecodingError(str(exc)) from exc
|
||||
|
||||
|
||||
class GZipDecoder(ContentDecoder):
|
||||
"""
|
||||
Handle 'gzip' decoding.
|
||||
|
||||
See: https://stackoverflow.com/questions/1838699
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.decompressor = zlib.decompressobj(zlib.MAX_WBITS | 16)
|
||||
|
||||
def decode(self, data: bytes) -> bytes:
|
||||
try:
|
||||
return self.decompressor.decompress(data)
|
||||
except zlib.error as exc:
|
||||
raise DecodingError(str(exc)) from exc
|
||||
|
||||
def flush(self) -> bytes:
|
||||
try:
|
||||
return self.decompressor.flush()
|
||||
except zlib.error as exc: # pragma: no cover
|
||||
raise DecodingError(str(exc)) from exc
|
||||
|
||||
|
||||
class BrotliDecoder(ContentDecoder):
|
||||
"""
|
||||
Handle 'brotli' decoding.
|
||||
|
||||
Requires `pip install brotlipy`. See: https://brotlipy.readthedocs.io/
|
||||
or `pip install brotli`. See https://github.com/google/brotli
|
||||
Supports both 'brotlipy' and 'Brotli' packages since they share an import
|
||||
name. The top branches are for 'brotlipy' and bottom branches for 'Brotli'
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
if brotli is None: # pragma: no cover
|
||||
raise ImportError(
|
||||
"Using 'BrotliDecoder', but neither of the 'brotlicffi' or 'brotli' "
|
||||
"packages have been installed. "
|
||||
"Make sure to install httpx using `pip install httpx[brotli]`."
|
||||
) from None
|
||||
|
||||
self.decompressor = brotli.Decompressor()
|
||||
self.seen_data = False
|
||||
self._decompress: typing.Callable[[bytes], bytes]
|
||||
if hasattr(self.decompressor, "decompress"):
|
||||
# The 'brotlicffi' package.
|
||||
self._decompress = self.decompressor.decompress # pragma: no cover
|
||||
else:
|
||||
# The 'brotli' package.
|
||||
self._decompress = self.decompressor.process # pragma: no cover
|
||||
|
||||
def decode(self, data: bytes) -> bytes:
|
||||
if not data:
|
||||
return b""
|
||||
self.seen_data = True
|
||||
try:
|
||||
return self._decompress(data)
|
||||
except brotli.error as exc:
|
||||
raise DecodingError(str(exc)) from exc
|
||||
|
||||
def flush(self) -> bytes:
|
||||
if not self.seen_data:
|
||||
return b""
|
||||
try:
|
||||
if hasattr(self.decompressor, "finish"):
|
||||
# Only available in the 'brotlicffi' package.
|
||||
|
||||
# As the decompressor decompresses eagerly, this
|
||||
# will never actually emit any data. However, it will potentially throw
|
||||
# errors if a truncated or damaged data stream has been used.
|
||||
self.decompressor.finish() # pragma: no cover
|
||||
return b""
|
||||
except brotli.error as exc: # pragma: no cover
|
||||
raise DecodingError(str(exc)) from exc
|
||||
|
||||
|
||||
class ZStandardDecoder(ContentDecoder):
|
||||
"""
|
||||
Handle 'zstd' RFC 8878 decoding.
|
||||
|
||||
Requires `pip install zstandard`.
|
||||
Can be installed as a dependency of httpx using `pip install httpx[zstd]`.
|
||||
"""
|
||||
|
||||
# inspired by the ZstdDecoder implementation in urllib3
|
||||
def __init__(self) -> None:
|
||||
if zstandard is None: # pragma: no cover
|
||||
raise ImportError(
|
||||
"Using 'ZStandardDecoder', ..."
|
||||
"Make sure to install httpx using `pip install httpx[zstd]`."
|
||||
) from None
|
||||
|
||||
self.decompressor = zstandard.ZstdDecompressor().decompressobj()
|
||||
self.seen_data = False
|
||||
|
||||
def decode(self, data: bytes) -> bytes:
|
||||
assert zstandard is not None
|
||||
self.seen_data = True
|
||||
output = io.BytesIO()
|
||||
try:
|
||||
output.write(self.decompressor.decompress(data))
|
||||
while self.decompressor.eof and self.decompressor.unused_data:
|
||||
unused_data = self.decompressor.unused_data
|
||||
self.decompressor = zstandard.ZstdDecompressor().decompressobj()
|
||||
output.write(self.decompressor.decompress(unused_data))
|
||||
except zstandard.ZstdError as exc:
|
||||
raise DecodingError(str(exc)) from exc
|
||||
return output.getvalue()
|
||||
|
||||
def flush(self) -> bytes:
|
||||
if not self.seen_data:
|
||||
return b""
|
||||
ret = self.decompressor.flush() # note: this is a no-op
|
||||
if not self.decompressor.eof:
|
||||
raise DecodingError("Zstandard data is incomplete") # pragma: no cover
|
||||
return bytes(ret)
|
||||
|
||||
|
||||
class MultiDecoder(ContentDecoder):
|
||||
"""
|
||||
Handle the case where multiple encodings have been applied.
|
||||
"""
|
||||
|
||||
def __init__(self, children: typing.Sequence[ContentDecoder]) -> None:
|
||||
"""
|
||||
'children' should be a sequence of decoders in the order in which
|
||||
each was applied.
|
||||
"""
|
||||
# Note that we reverse the order for decoding.
|
||||
self.children = list(reversed(children))
|
||||
|
||||
def decode(self, data: bytes) -> bytes:
|
||||
for child in self.children:
|
||||
data = child.decode(data)
|
||||
return data
|
||||
|
||||
def flush(self) -> bytes:
|
||||
data = b""
|
||||
for child in self.children:
|
||||
data = child.decode(data) + child.flush()
|
||||
return data
|
||||
|
||||
|
||||
class ByteChunker:
|
||||
"""
|
||||
Handles returning byte content in fixed-size chunks.
|
||||
"""
|
||||
|
||||
def __init__(self, chunk_size: int | None = None) -> None:
|
||||
self._buffer = io.BytesIO()
|
||||
self._chunk_size = chunk_size
|
||||
|
||||
def decode(self, content: bytes) -> list[bytes]:
|
||||
if self._chunk_size is None:
|
||||
return [content] if content else []
|
||||
|
||||
self._buffer.write(content)
|
||||
if self._buffer.tell() >= self._chunk_size:
|
||||
value = self._buffer.getvalue()
|
||||
chunks = [
|
||||
value[i : i + self._chunk_size]
|
||||
for i in range(0, len(value), self._chunk_size)
|
||||
]
|
||||
if len(chunks[-1]) == self._chunk_size:
|
||||
self._buffer.seek(0)
|
||||
self._buffer.truncate()
|
||||
return chunks
|
||||
else:
|
||||
self._buffer.seek(0)
|
||||
self._buffer.write(chunks[-1])
|
||||
self._buffer.truncate()
|
||||
return chunks[:-1]
|
||||
else:
|
||||
return []
|
||||
|
||||
def flush(self) -> list[bytes]:
|
||||
value = self._buffer.getvalue()
|
||||
self._buffer.seek(0)
|
||||
self._buffer.truncate()
|
||||
return [value] if value else []
|
||||
|
||||
|
||||
class TextChunker:
|
||||
"""
|
||||
Handles returning text content in fixed-size chunks.
|
||||
"""
|
||||
|
||||
def __init__(self, chunk_size: int | None = None) -> None:
|
||||
self._buffer = io.StringIO()
|
||||
self._chunk_size = chunk_size
|
||||
|
||||
def decode(self, content: str) -> list[str]:
|
||||
if self._chunk_size is None:
|
||||
return [content] if content else []
|
||||
|
||||
self._buffer.write(content)
|
||||
if self._buffer.tell() >= self._chunk_size:
|
||||
value = self._buffer.getvalue()
|
||||
chunks = [
|
||||
value[i : i + self._chunk_size]
|
||||
for i in range(0, len(value), self._chunk_size)
|
||||
]
|
||||
if len(chunks[-1]) == self._chunk_size:
|
||||
self._buffer.seek(0)
|
||||
self._buffer.truncate()
|
||||
return chunks
|
||||
else:
|
||||
self._buffer.seek(0)
|
||||
self._buffer.write(chunks[-1])
|
||||
self._buffer.truncate()
|
||||
return chunks[:-1]
|
||||
else:
|
||||
return []
|
||||
|
||||
def flush(self) -> list[str]:
|
||||
value = self._buffer.getvalue()
|
||||
self._buffer.seek(0)
|
||||
self._buffer.truncate()
|
||||
return [value] if value else []
|
||||
|
||||
|
||||
class TextDecoder:
|
||||
"""
|
||||
Handles incrementally decoding bytes into text
|
||||
"""
|
||||
|
||||
def __init__(self, encoding: str = "utf-8") -> None:
|
||||
self.decoder = codecs.getincrementaldecoder(encoding)(errors="replace")
|
||||
|
||||
def decode(self, data: bytes) -> str:
|
||||
return self.decoder.decode(data)
|
||||
|
||||
def flush(self) -> str:
|
||||
return self.decoder.decode(b"", True)
|
||||
|
||||
|
||||
class LineDecoder:
|
||||
"""
|
||||
Handles incrementally reading lines from text.
|
||||
|
||||
Has the same behaviour as the stdllib splitlines,
|
||||
but handling the input iteratively.
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.buffer: list[str] = []
|
||||
self.trailing_cr: bool = False
|
||||
|
||||
def decode(self, text: str) -> list[str]:
|
||||
# See https://docs.python.org/3/library/stdtypes.html#str.splitlines
|
||||
NEWLINE_CHARS = "\n\r\x0b\x0c\x1c\x1d\x1e\x85\u2028\u2029"
|
||||
|
||||
# We always push a trailing `\r` into the next decode iteration.
|
||||
if self.trailing_cr:
|
||||
text = "\r" + text
|
||||
self.trailing_cr = False
|
||||
if text.endswith("\r"):
|
||||
self.trailing_cr = True
|
||||
text = text[:-1]
|
||||
|
||||
if not text:
|
||||
# NOTE: the edge case input of empty text doesn't occur in practice,
|
||||
# because other httpx internals filter out this value
|
||||
return [] # pragma: no cover
|
||||
|
||||
trailing_newline = text[-1] in NEWLINE_CHARS
|
||||
lines = text.splitlines()
|
||||
|
||||
if len(lines) == 1 and not trailing_newline:
|
||||
# No new lines, buffer the input and continue.
|
||||
self.buffer.append(lines[0])
|
||||
return []
|
||||
|
||||
if self.buffer:
|
||||
# Include any existing buffer in the first portion of the
|
||||
# splitlines result.
|
||||
lines = ["".join(self.buffer) + lines[0]] + lines[1:]
|
||||
self.buffer = []
|
||||
|
||||
if not trailing_newline:
|
||||
# If the last segment of splitlines is not newline terminated,
|
||||
# then drop it from our output and start a new buffer.
|
||||
self.buffer = [lines.pop()]
|
||||
|
||||
return lines
|
||||
|
||||
def flush(self) -> list[str]:
|
||||
if not self.buffer and not self.trailing_cr:
|
||||
return []
|
||||
|
||||
lines = ["".join(self.buffer)]
|
||||
self.buffer = []
|
||||
self.trailing_cr = False
|
||||
return lines
|
||||
|
||||
|
||||
SUPPORTED_DECODERS = {
|
||||
"identity": IdentityDecoder,
|
||||
"gzip": GZipDecoder,
|
||||
"deflate": DeflateDecoder,
|
||||
"br": BrotliDecoder,
|
||||
"zstd": ZStandardDecoder,
|
||||
}
|
||||
|
||||
|
||||
if brotli is None:
|
||||
SUPPORTED_DECODERS.pop("br") # pragma: no cover
|
||||
if zstandard is None:
|
||||
SUPPORTED_DECODERS.pop("zstd") # pragma: no cover
|
||||
@ -1,377 +0,0 @@
|
||||
"""
|
||||
Our exception hierarchy:
|
||||
|
||||
* HTTPError
|
||||
x RequestError
|
||||
+ TransportError
|
||||
- TimeoutException
|
||||
· ConnectTimeout
|
||||
· ReadTimeout
|
||||
· WriteTimeout
|
||||
· PoolTimeout
|
||||
- NetworkError
|
||||
· ConnectError
|
||||
· ReadError
|
||||
· WriteError
|
||||
· CloseError
|
||||
- ProtocolError
|
||||
· LocalProtocolError
|
||||
· RemoteProtocolError
|
||||
- ProxyError
|
||||
- UnsupportedProtocol
|
||||
+ DecodingError
|
||||
+ TooManyRedirects
|
||||
x HTTPStatusError
|
||||
* InvalidURL
|
||||
* CookieConflict
|
||||
* StreamError
|
||||
x StreamConsumed
|
||||
x StreamClosed
|
||||
x ResponseNotRead
|
||||
x RequestNotRead
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import typing
|
||||
|
||||
if typing.TYPE_CHECKING:
|
||||
from ._models import Request, Response # pragma: no cover
|
||||
|
||||
__all__ = [
|
||||
"CloseError",
|
||||
"ConnectError",
|
||||
"ConnectTimeout",
|
||||
"CookieConflict",
|
||||
"DecodingError",
|
||||
"HTTPError",
|
||||
"HTTPStatusError",
|
||||
"InvalidURL",
|
||||
"LocalProtocolError",
|
||||
"NetworkError",
|
||||
"PoolTimeout",
|
||||
"ProtocolError",
|
||||
"ProxyError",
|
||||
"ReadError",
|
||||
"ReadTimeout",
|
||||
"RemoteProtocolError",
|
||||
"RequestError",
|
||||
"RequestNotRead",
|
||||
"ResponseNotRead",
|
||||
"StreamClosed",
|
||||
"StreamConsumed",
|
||||
"StreamError",
|
||||
"TimeoutException",
|
||||
"TooManyRedirects",
|
||||
"TransportError",
|
||||
"UnsupportedProtocol",
|
||||
"WriteError",
|
||||
"WriteTimeout",
|
||||
]
|
||||
|
||||
|
||||
class HTTPError(Exception):
|
||||
"""
|
||||
Base class for `RequestError` and `HTTPStatusError`.
|
||||
|
||||
Useful for `try...except` blocks when issuing a request,
|
||||
and then calling `.raise_for_status()`.
|
||||
|
||||
For example:
|
||||
|
||||
```
|
||||
try:
|
||||
response = httpx.get("https://www.example.com")
|
||||
response.raise_for_status()
|
||||
except httpx.HTTPError as exc:
|
||||
print(f"HTTP Exception for {exc.request.url} - {exc}")
|
||||
```
|
||||
"""
|
||||
|
||||
def __init__(self, message: str) -> None:
|
||||
super().__init__(message)
|
||||
self._request: Request | None = None
|
||||
|
||||
@property
|
||||
def request(self) -> Request:
|
||||
if self._request is None:
|
||||
raise RuntimeError("The .request property has not been set.")
|
||||
return self._request
|
||||
|
||||
@request.setter
|
||||
def request(self, request: Request) -> None:
|
||||
self._request = request
|
||||
|
||||
|
||||
class RequestError(HTTPError):
|
||||
"""
|
||||
Base class for all exceptions that may occur when issuing a `.request()`.
|
||||
"""
|
||||
|
||||
def __init__(self, message: str, *, request: Request | None = None) -> None:
|
||||
super().__init__(message)
|
||||
# At the point an exception is raised we won't typically have a request
|
||||
# instance to associate it with.
|
||||
#
|
||||
# The 'request_context' context manager is used within the Client and
|
||||
# Response methods in order to ensure that any raised exceptions
|
||||
# have a `.request` property set on them.
|
||||
self._request = request
|
||||
|
||||
|
||||
class TransportError(RequestError):
|
||||
"""
|
||||
Base class for all exceptions that occur at the level of the Transport API.
|
||||
"""
|
||||
|
||||
|
||||
# Timeout exceptions...
|
||||
|
||||
|
||||
class TimeoutException(TransportError):
|
||||
"""
|
||||
The base class for timeout errors.
|
||||
|
||||
An operation has timed out.
|
||||
"""
|
||||
|
||||
|
||||
class ConnectTimeout(TimeoutException):
|
||||
"""
|
||||
Timed out while connecting to the host.
|
||||
"""
|
||||
|
||||
|
||||
class ReadTimeout(TimeoutException):
|
||||
"""
|
||||
Timed out while receiving data from the host.
|
||||
"""
|
||||
|
||||
|
||||
class WriteTimeout(TimeoutException):
|
||||
"""
|
||||
Timed out while sending data to the host.
|
||||
"""
|
||||
|
||||
|
||||
class PoolTimeout(TimeoutException):
|
||||
"""
|
||||
Timed out waiting to acquire a connection from the pool.
|
||||
"""
|
||||
|
||||
|
||||
# Core networking exceptions...
|
||||
|
||||
|
||||
class NetworkError(TransportError):
|
||||
"""
|
||||
The base class for network-related errors.
|
||||
|
||||
An error occurred while interacting with the network.
|
||||
"""
|
||||
|
||||
|
||||
class ReadError(NetworkError):
|
||||
"""
|
||||
Failed to receive data from the network.
|
||||
"""
|
||||
|
||||
|
||||
class WriteError(NetworkError):
|
||||
"""
|
||||
Failed to send data through the network.
|
||||
"""
|
||||
|
||||
|
||||
class ConnectError(NetworkError):
|
||||
"""
|
||||
Failed to establish a connection.
|
||||
"""
|
||||
|
||||
|
||||
class CloseError(NetworkError):
|
||||
"""
|
||||
Failed to close a connection.
|
||||
"""
|
||||
|
||||
|
||||
# Other transport exceptions...
|
||||
|
||||
|
||||
class ProxyError(TransportError):
|
||||
"""
|
||||
An error occurred while establishing a proxy connection.
|
||||
"""
|
||||
|
||||
|
||||
class UnsupportedProtocol(TransportError):
|
||||
"""
|
||||
Attempted to make a request to an unsupported protocol.
|
||||
|
||||
For example issuing a request to `ftp://www.example.com`.
|
||||
"""
|
||||
|
||||
|
||||
class ProtocolError(TransportError):
|
||||
"""
|
||||
The protocol was violated.
|
||||
"""
|
||||
|
||||
|
||||
class LocalProtocolError(ProtocolError):
|
||||
"""
|
||||
A protocol was violated by the client.
|
||||
|
||||
For example if the user instantiated a `Request` instance explicitly,
|
||||
failed to include the mandatory `Host:` header, and then issued it directly
|
||||
using `client.send()`.
|
||||
"""
|
||||
|
||||
|
||||
class RemoteProtocolError(ProtocolError):
|
||||
"""
|
||||
The protocol was violated by the server.
|
||||
|
||||
For example, returning malformed HTTP.
|
||||
"""
|
||||
|
||||
|
||||
# Other request exceptions...
|
||||
|
||||
|
||||
class DecodingError(RequestError):
|
||||
"""
|
||||
Decoding of the response failed, due to a malformed encoding.
|
||||
"""
|
||||
|
||||
|
||||
class TooManyRedirects(RequestError):
|
||||
"""
|
||||
Too many redirects.
|
||||
"""
|
||||
|
||||
|
||||
# Client errors
|
||||
|
||||
|
||||
class HTTPStatusError(HTTPError):
|
||||
"""
|
||||
The response had an error HTTP status of 4xx or 5xx.
|
||||
|
||||
May be raised when calling `response.raise_for_status()`
|
||||
"""
|
||||
|
||||
def __init__(self, message: str, *, request: Request, response: Response) -> None:
|
||||
super().__init__(message)
|
||||
self.request = request
|
||||
self.response = response
|
||||
|
||||
|
||||
class InvalidURL(Exception):
|
||||
"""
|
||||
URL is improperly formed or cannot be parsed.
|
||||
"""
|
||||
|
||||
def __init__(self, message: str) -> None:
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
class CookieConflict(Exception):
|
||||
"""
|
||||
Attempted to lookup a cookie by name, but multiple cookies existed.
|
||||
|
||||
Can occur when calling `response.cookies.get(...)`.
|
||||
"""
|
||||
|
||||
def __init__(self, message: str) -> None:
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
# Stream exceptions...
|
||||
|
||||
# These may occur as the result of a programming error, by accessing
|
||||
# the request/response stream in an invalid manner.
|
||||
|
||||
|
||||
class StreamError(RuntimeError):
|
||||
"""
|
||||
The base class for stream exceptions.
|
||||
|
||||
The developer made an error in accessing the request stream in
|
||||
an invalid way.
|
||||
"""
|
||||
|
||||
def __init__(self, message: str) -> None:
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
class StreamConsumed(StreamError):
|
||||
"""
|
||||
Attempted to read or stream content, but the content has already
|
||||
been streamed.
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
message = (
|
||||
"Attempted to read or stream some content, but the content has "
|
||||
"already been streamed. For requests, this could be due to passing "
|
||||
"a generator as request content, and then receiving a redirect "
|
||||
"response or a secondary request as part of an authentication flow."
|
||||
"For responses, this could be due to attempting to stream the response "
|
||||
"content more than once."
|
||||
)
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
class StreamClosed(StreamError):
|
||||
"""
|
||||
Attempted to read or stream response content, but the request has been
|
||||
closed.
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
message = "Attempted to read or stream content, but the stream has been closed."
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
class ResponseNotRead(StreamError):
|
||||
"""
|
||||
Attempted to access streaming response content, without having called `read()`.
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
message = (
|
||||
"Attempted to access streaming response content,"
|
||||
" without having called `read()`."
|
||||
)
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
class RequestNotRead(StreamError):
|
||||
"""
|
||||
Attempted to access streaming request content, without having called `read()`.
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
message = (
|
||||
"Attempted to access streaming request content,"
|
||||
" without having called `read()`."
|
||||
)
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def request_context(
|
||||
request: Request | None = None,
|
||||
) -> typing.Iterator[None]:
|
||||
"""
|
||||
A context manager that can be used to attach the given request context
|
||||
to any `RequestError` exceptions that are raised within the block.
|
||||
"""
|
||||
try:
|
||||
yield
|
||||
except RequestError as exc:
|
||||
if request is not None:
|
||||
exc.request = request
|
||||
raise exc
|
||||
506
httpx/_main.py
506
httpx/_main.py
@ -1,506 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import functools
|
||||
import json
|
||||
import sys
|
||||
import typing
|
||||
|
||||
import click
|
||||
import pygments.lexers
|
||||
import pygments.util
|
||||
import rich.console
|
||||
import rich.markup
|
||||
import rich.progress
|
||||
import rich.syntax
|
||||
import rich.table
|
||||
|
||||
from ._client import Client
|
||||
from ._exceptions import RequestError
|
||||
from ._models import Response
|
||||
from ._status_codes import codes
|
||||
|
||||
if typing.TYPE_CHECKING:
|
||||
import httpcore # pragma: no cover
|
||||
|
||||
|
||||
def print_help() -> None:
|
||||
console = rich.console.Console()
|
||||
|
||||
console.print("[bold]HTTPX :butterfly:", justify="center")
|
||||
console.print()
|
||||
console.print("A next generation HTTP client.", justify="center")
|
||||
console.print()
|
||||
console.print(
|
||||
"Usage: [bold]httpx[/bold] [cyan]<URL> [OPTIONS][/cyan] ", justify="left"
|
||||
)
|
||||
console.print()
|
||||
|
||||
table = rich.table.Table.grid(padding=1, pad_edge=True)
|
||||
table.add_column("Parameter", no_wrap=True, justify="left", style="bold")
|
||||
table.add_column("Description")
|
||||
table.add_row(
|
||||
"-m, --method [cyan]METHOD",
|
||||
"Request method, such as GET, POST, PUT, PATCH, DELETE, OPTIONS, HEAD.\n"
|
||||
"[Default: GET, or POST if a request body is included]",
|
||||
)
|
||||
table.add_row(
|
||||
"-p, --params [cyan]<NAME VALUE> ...",
|
||||
"Query parameters to include in the request URL.",
|
||||
)
|
||||
table.add_row(
|
||||
"-c, --content [cyan]TEXT", "Byte content to include in the request body."
|
||||
)
|
||||
table.add_row(
|
||||
"-d, --data [cyan]<NAME VALUE> ...", "Form data to include in the request body."
|
||||
)
|
||||
table.add_row(
|
||||
"-f, --files [cyan]<NAME FILENAME> ...",
|
||||
"Form files to include in the request body.",
|
||||
)
|
||||
table.add_row("-j, --json [cyan]TEXT", "JSON data to include in the request body.")
|
||||
table.add_row(
|
||||
"-h, --headers [cyan]<NAME VALUE> ...",
|
||||
"Include additional HTTP headers in the request.",
|
||||
)
|
||||
table.add_row(
|
||||
"--cookies [cyan]<NAME VALUE> ...", "Cookies to include in the request."
|
||||
)
|
||||
table.add_row(
|
||||
"--auth [cyan]<USER PASS>",
|
||||
"Username and password to include in the request. Specify '-' for the password"
|
||||
" to use a password prompt. Note that using --verbose/-v will expose"
|
||||
" the Authorization header, including the password encoding"
|
||||
" in a trivially reversible format.",
|
||||
)
|
||||
|
||||
table.add_row(
|
||||
"--proxy [cyan]URL",
|
||||
"Send the request via a proxy. Should be the URL giving the proxy address.",
|
||||
)
|
||||
|
||||
table.add_row(
|
||||
"--timeout [cyan]FLOAT",
|
||||
"Timeout value to use for network operations, such as establishing the"
|
||||
" connection, reading some data, etc... [Default: 5.0]",
|
||||
)
|
||||
|
||||
table.add_row("--follow-redirects", "Automatically follow redirects.")
|
||||
table.add_row("--no-verify", "Disable SSL verification.")
|
||||
table.add_row(
|
||||
"--http2", "Send the request using HTTP/2, if the remote server supports it."
|
||||
)
|
||||
|
||||
table.add_row(
|
||||
"--download [cyan]FILE",
|
||||
"Save the response content as a file, rather than displaying it.",
|
||||
)
|
||||
|
||||
table.add_row("-v, --verbose", "Verbose output. Show request as well as response.")
|
||||
table.add_row("--help", "Show this message and exit.")
|
||||
console.print(table)
|
||||
|
||||
|
||||
def get_lexer_for_response(response: Response) -> str:
|
||||
content_type = response.headers.get("Content-Type")
|
||||
if content_type is not None:
|
||||
mime_type, _, _ = content_type.partition(";")
|
||||
try:
|
||||
return typing.cast(
|
||||
str, pygments.lexers.get_lexer_for_mimetype(mime_type.strip()).name
|
||||
)
|
||||
except pygments.util.ClassNotFound: # pragma: no cover
|
||||
pass
|
||||
return "" # pragma: no cover
|
||||
|
||||
|
||||
def format_request_headers(request: httpcore.Request, http2: bool = False) -> str:
|
||||
version = "HTTP/2" if http2 else "HTTP/1.1"
|
||||
headers = [
|
||||
(name.lower() if http2 else name, value) for name, value in request.headers
|
||||
]
|
||||
method = request.method.decode("ascii")
|
||||
target = request.url.target.decode("ascii")
|
||||
lines = [f"{method} {target} {version}"] + [
|
||||
f"{name.decode('ascii')}: {value.decode('ascii')}" for name, value in headers
|
||||
]
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def format_response_headers(
|
||||
http_version: bytes,
|
||||
status: int,
|
||||
reason_phrase: bytes | None,
|
||||
headers: list[tuple[bytes, bytes]],
|
||||
) -> str:
|
||||
version = http_version.decode("ascii")
|
||||
reason = (
|
||||
codes.get_reason_phrase(status)
|
||||
if reason_phrase is None
|
||||
else reason_phrase.decode("ascii")
|
||||
)
|
||||
lines = [f"{version} {status} {reason}"] + [
|
||||
f"{name.decode('ascii')}: {value.decode('ascii')}" for name, value in headers
|
||||
]
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def print_request_headers(request: httpcore.Request, http2: bool = False) -> None:
|
||||
console = rich.console.Console()
|
||||
http_text = format_request_headers(request, http2=http2)
|
||||
syntax = rich.syntax.Syntax(http_text, "http", theme="ansi_dark", word_wrap=True)
|
||||
console.print(syntax)
|
||||
syntax = rich.syntax.Syntax("", "http", theme="ansi_dark", word_wrap=True)
|
||||
console.print(syntax)
|
||||
|
||||
|
||||
def print_response_headers(
|
||||
http_version: bytes,
|
||||
status: int,
|
||||
reason_phrase: bytes | None,
|
||||
headers: list[tuple[bytes, bytes]],
|
||||
) -> None:
|
||||
console = rich.console.Console()
|
||||
http_text = format_response_headers(http_version, status, reason_phrase, headers)
|
||||
syntax = rich.syntax.Syntax(http_text, "http", theme="ansi_dark", word_wrap=True)
|
||||
console.print(syntax)
|
||||
syntax = rich.syntax.Syntax("", "http", theme="ansi_dark", word_wrap=True)
|
||||
console.print(syntax)
|
||||
|
||||
|
||||
def print_response(response: Response) -> None:
|
||||
console = rich.console.Console()
|
||||
lexer_name = get_lexer_for_response(response)
|
||||
if lexer_name:
|
||||
if lexer_name.lower() == "json":
|
||||
try:
|
||||
data = response.json()
|
||||
text = json.dumps(data, indent=4)
|
||||
except ValueError: # pragma: no cover
|
||||
text = response.text
|
||||
else:
|
||||
text = response.text
|
||||
|
||||
syntax = rich.syntax.Syntax(text, lexer_name, theme="ansi_dark", word_wrap=True)
|
||||
console.print(syntax)
|
||||
else:
|
||||
console.print(f"<{len(response.content)} bytes of binary data>")
|
||||
|
||||
|
||||
_PCTRTT = typing.Tuple[typing.Tuple[str, str], ...]
|
||||
_PCTRTTT = typing.Tuple[_PCTRTT, ...]
|
||||
_PeerCertRetDictType = typing.Dict[str, typing.Union[str, _PCTRTTT, _PCTRTT]]
|
||||
|
||||
|
||||
def format_certificate(cert: _PeerCertRetDictType) -> str: # pragma: no cover
|
||||
lines = []
|
||||
for key, value in cert.items():
|
||||
if isinstance(value, (list, tuple)):
|
||||
lines.append(f"* {key}:")
|
||||
for item in value:
|
||||
if key in ("subject", "issuer"):
|
||||
for sub_item in item:
|
||||
lines.append(f"* {sub_item[0]}: {sub_item[1]!r}")
|
||||
elif isinstance(item, tuple) and len(item) == 2:
|
||||
lines.append(f"* {item[0]}: {item[1]!r}")
|
||||
else:
|
||||
lines.append(f"* {item!r}")
|
||||
else:
|
||||
lines.append(f"* {key}: {value!r}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def trace(
|
||||
name: str, info: typing.Mapping[str, typing.Any], verbose: bool = False
|
||||
) -> None:
|
||||
console = rich.console.Console()
|
||||
if name == "connection.connect_tcp.started" and verbose:
|
||||
host = info["host"]
|
||||
console.print(f"* Connecting to {host!r}")
|
||||
elif name == "connection.connect_tcp.complete" and verbose:
|
||||
stream = info["return_value"]
|
||||
server_addr = stream.get_extra_info("server_addr")
|
||||
console.print(f"* Connected to {server_addr[0]!r} on port {server_addr[1]}")
|
||||
elif name == "connection.start_tls.complete" and verbose: # pragma: no cover
|
||||
stream = info["return_value"]
|
||||
ssl_object = stream.get_extra_info("ssl_object")
|
||||
version = ssl_object.version()
|
||||
cipher = ssl_object.cipher()
|
||||
server_cert = ssl_object.getpeercert()
|
||||
alpn = ssl_object.selected_alpn_protocol()
|
||||
console.print(f"* SSL established using {version!r} / {cipher[0]!r}")
|
||||
console.print(f"* Selected ALPN protocol: {alpn!r}")
|
||||
if server_cert:
|
||||
console.print("* Server certificate:")
|
||||
console.print(format_certificate(server_cert))
|
||||
elif name == "http11.send_request_headers.started" and verbose:
|
||||
request = info["request"]
|
||||
print_request_headers(request, http2=False)
|
||||
elif name == "http2.send_request_headers.started" and verbose: # pragma: no cover
|
||||
request = info["request"]
|
||||
print_request_headers(request, http2=True)
|
||||
elif name == "http11.receive_response_headers.complete":
|
||||
http_version, status, reason_phrase, headers = info["return_value"]
|
||||
print_response_headers(http_version, status, reason_phrase, headers)
|
||||
elif name == "http2.receive_response_headers.complete": # pragma: no cover
|
||||
status, headers = info["return_value"]
|
||||
http_version = b"HTTP/2"
|
||||
reason_phrase = None
|
||||
print_response_headers(http_version, status, reason_phrase, headers)
|
||||
|
||||
|
||||
def download_response(response: Response, download: typing.BinaryIO) -> None:
|
||||
console = rich.console.Console()
|
||||
console.print()
|
||||
content_length = response.headers.get("Content-Length")
|
||||
with rich.progress.Progress(
|
||||
"[progress.description]{task.description}",
|
||||
"[progress.percentage]{task.percentage:>3.0f}%",
|
||||
rich.progress.BarColumn(bar_width=None),
|
||||
rich.progress.DownloadColumn(),
|
||||
rich.progress.TransferSpeedColumn(),
|
||||
) as progress:
|
||||
description = f"Downloading [bold]{rich.markup.escape(download.name)}"
|
||||
download_task = progress.add_task(
|
||||
description,
|
||||
total=int(content_length or 0),
|
||||
start=content_length is not None,
|
||||
)
|
||||
for chunk in response.iter_bytes():
|
||||
download.write(chunk)
|
||||
progress.update(download_task, completed=response.num_bytes_downloaded)
|
||||
|
||||
|
||||
def validate_json(
|
||||
ctx: click.Context,
|
||||
param: click.Option | click.Parameter,
|
||||
value: typing.Any,
|
||||
) -> typing.Any:
|
||||
if value is None:
|
||||
return None
|
||||
|
||||
try:
|
||||
return json.loads(value)
|
||||
except json.JSONDecodeError: # pragma: no cover
|
||||
raise click.BadParameter("Not valid JSON")
|
||||
|
||||
|
||||
def validate_auth(
|
||||
ctx: click.Context,
|
||||
param: click.Option | click.Parameter,
|
||||
value: typing.Any,
|
||||
) -> typing.Any:
|
||||
if value == (None, None):
|
||||
return None
|
||||
|
||||
username, password = value
|
||||
if password == "-": # pragma: no cover
|
||||
password = click.prompt("Password", hide_input=True)
|
||||
return (username, password)
|
||||
|
||||
|
||||
def handle_help(
|
||||
ctx: click.Context,
|
||||
param: click.Option | click.Parameter,
|
||||
value: typing.Any,
|
||||
) -> None:
|
||||
if not value or ctx.resilient_parsing:
|
||||
return
|
||||
|
||||
print_help()
|
||||
ctx.exit()
|
||||
|
||||
|
||||
@click.command(add_help_option=False)
|
||||
@click.argument("url", type=str)
|
||||
@click.option(
|
||||
"--method",
|
||||
"-m",
|
||||
"method",
|
||||
type=str,
|
||||
help=(
|
||||
"Request method, such as GET, POST, PUT, PATCH, DELETE, OPTIONS, HEAD. "
|
||||
"[Default: GET, or POST if a request body is included]"
|
||||
),
|
||||
)
|
||||
@click.option(
|
||||
"--params",
|
||||
"-p",
|
||||
"params",
|
||||
type=(str, str),
|
||||
multiple=True,
|
||||
help="Query parameters to include in the request URL.",
|
||||
)
|
||||
@click.option(
|
||||
"--content",
|
||||
"-c",
|
||||
"content",
|
||||
type=str,
|
||||
help="Byte content to include in the request body.",
|
||||
)
|
||||
@click.option(
|
||||
"--data",
|
||||
"-d",
|
||||
"data",
|
||||
type=(str, str),
|
||||
multiple=True,
|
||||
help="Form data to include in the request body.",
|
||||
)
|
||||
@click.option(
|
||||
"--files",
|
||||
"-f",
|
||||
"files",
|
||||
type=(str, click.File(mode="rb")),
|
||||
multiple=True,
|
||||
help="Form files to include in the request body.",
|
||||
)
|
||||
@click.option(
|
||||
"--json",
|
||||
"-j",
|
||||
"json",
|
||||
type=str,
|
||||
callback=validate_json,
|
||||
help="JSON data to include in the request body.",
|
||||
)
|
||||
@click.option(
|
||||
"--headers",
|
||||
"-h",
|
||||
"headers",
|
||||
type=(str, str),
|
||||
multiple=True,
|
||||
help="Include additional HTTP headers in the request.",
|
||||
)
|
||||
@click.option(
|
||||
"--cookies",
|
||||
"cookies",
|
||||
type=(str, str),
|
||||
multiple=True,
|
||||
help="Cookies to include in the request.",
|
||||
)
|
||||
@click.option(
|
||||
"--auth",
|
||||
"auth",
|
||||
type=(str, str),
|
||||
default=(None, None),
|
||||
callback=validate_auth,
|
||||
help=(
|
||||
"Username and password to include in the request. "
|
||||
"Specify '-' for the password to use a password prompt. "
|
||||
"Note that using --verbose/-v will expose the Authorization header, "
|
||||
"including the password encoding in a trivially reversible format."
|
||||
),
|
||||
)
|
||||
@click.option(
|
||||
"--proxy",
|
||||
"proxy",
|
||||
type=str,
|
||||
default=None,
|
||||
help="Send the request via a proxy. Should be the URL giving the proxy address.",
|
||||
)
|
||||
@click.option(
|
||||
"--timeout",
|
||||
"timeout",
|
||||
type=float,
|
||||
default=5.0,
|
||||
help=(
|
||||
"Timeout value to use for network operations, such as establishing the "
|
||||
"connection, reading some data, etc... [Default: 5.0]"
|
||||
),
|
||||
)
|
||||
@click.option(
|
||||
"--follow-redirects",
|
||||
"follow_redirects",
|
||||
is_flag=True,
|
||||
default=False,
|
||||
help="Automatically follow redirects.",
|
||||
)
|
||||
@click.option(
|
||||
"--no-verify",
|
||||
"verify",
|
||||
is_flag=True,
|
||||
default=True,
|
||||
help="Disable SSL verification.",
|
||||
)
|
||||
@click.option(
|
||||
"--http2",
|
||||
"http2",
|
||||
type=bool,
|
||||
is_flag=True,
|
||||
default=False,
|
||||
help="Send the request using HTTP/2, if the remote server supports it.",
|
||||
)
|
||||
@click.option(
|
||||
"--download",
|
||||
type=click.File("wb"),
|
||||
help="Save the response content as a file, rather than displaying it.",
|
||||
)
|
||||
@click.option(
|
||||
"--verbose",
|
||||
"-v",
|
||||
type=bool,
|
||||
is_flag=True,
|
||||
default=False,
|
||||
help="Verbose. Show request as well as response.",
|
||||
)
|
||||
@click.option(
|
||||
"--help",
|
||||
is_flag=True,
|
||||
is_eager=True,
|
||||
expose_value=False,
|
||||
callback=handle_help,
|
||||
help="Show this message and exit.",
|
||||
)
|
||||
def main(
|
||||
url: str,
|
||||
method: str,
|
||||
params: list[tuple[str, str]],
|
||||
content: str,
|
||||
data: list[tuple[str, str]],
|
||||
files: list[tuple[str, click.File]],
|
||||
json: str,
|
||||
headers: list[tuple[str, str]],
|
||||
cookies: list[tuple[str, str]],
|
||||
auth: tuple[str, str] | None,
|
||||
proxy: str,
|
||||
timeout: float,
|
||||
follow_redirects: bool,
|
||||
verify: bool,
|
||||
http2: bool,
|
||||
download: typing.BinaryIO | None,
|
||||
verbose: bool,
|
||||
) -> None:
|
||||
"""
|
||||
An HTTP command line client.
|
||||
Sends a request and displays the response.
|
||||
"""
|
||||
if not method:
|
||||
method = "POST" if content or data or files or json else "GET"
|
||||
|
||||
try:
|
||||
with Client(proxy=proxy, timeout=timeout, http2=http2, verify=verify) as client:
|
||||
with client.stream(
|
||||
method,
|
||||
url,
|
||||
params=list(params),
|
||||
content=content,
|
||||
data=dict(data),
|
||||
files=files, # type: ignore
|
||||
json=json,
|
||||
headers=headers,
|
||||
cookies=dict(cookies),
|
||||
auth=auth,
|
||||
follow_redirects=follow_redirects,
|
||||
extensions={"trace": functools.partial(trace, verbose=verbose)},
|
||||
) as response:
|
||||
if download is not None:
|
||||
download_response(response, download)
|
||||
else:
|
||||
response.read()
|
||||
if response.content:
|
||||
print_response(response)
|
||||
|
||||
except RequestError as exc:
|
||||
console = rich.console.Console()
|
||||
console.print(f"[red]{type(exc).__name__}[/red]: {exc}")
|
||||
sys.exit(1)
|
||||
|
||||
sys.exit(0 if response.is_success else 1)
|
||||
1277
httpx/_models.py
1277
httpx/_models.py
File diff suppressed because it is too large
Load Diff
@ -1,300 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import mimetypes
|
||||
import os
|
||||
import re
|
||||
import typing
|
||||
from pathlib import Path
|
||||
|
||||
from ._types import (
|
||||
AsyncByteStream,
|
||||
FileContent,
|
||||
FileTypes,
|
||||
RequestData,
|
||||
RequestFiles,
|
||||
SyncByteStream,
|
||||
)
|
||||
from ._utils import (
|
||||
peek_filelike_length,
|
||||
primitive_value_to_str,
|
||||
to_bytes,
|
||||
)
|
||||
|
||||
_HTML5_FORM_ENCODING_REPLACEMENTS = {'"': "%22", "\\": "\\\\"}
|
||||
_HTML5_FORM_ENCODING_REPLACEMENTS.update(
|
||||
{chr(c): "%{:02X}".format(c) for c in range(0x1F + 1) if c != 0x1B}
|
||||
)
|
||||
_HTML5_FORM_ENCODING_RE = re.compile(
|
||||
r"|".join([re.escape(c) for c in _HTML5_FORM_ENCODING_REPLACEMENTS.keys()])
|
||||
)
|
||||
|
||||
|
||||
def _format_form_param(name: str, value: str) -> bytes:
|
||||
"""
|
||||
Encode a name/value pair within a multipart form.
|
||||
"""
|
||||
|
||||
def replacer(match: typing.Match[str]) -> str:
|
||||
return _HTML5_FORM_ENCODING_REPLACEMENTS[match.group(0)]
|
||||
|
||||
value = _HTML5_FORM_ENCODING_RE.sub(replacer, value)
|
||||
return f'{name}="{value}"'.encode()
|
||||
|
||||
|
||||
def _guess_content_type(filename: str | None) -> str | None:
|
||||
"""
|
||||
Guesses the mimetype based on a filename. Defaults to `application/octet-stream`.
|
||||
|
||||
Returns `None` if `filename` is `None` or empty.
|
||||
"""
|
||||
if filename:
|
||||
return mimetypes.guess_type(filename)[0] or "application/octet-stream"
|
||||
return None
|
||||
|
||||
|
||||
def get_multipart_boundary_from_content_type(
|
||||
content_type: bytes | None,
|
||||
) -> bytes | None:
|
||||
if not content_type or not content_type.startswith(b"multipart/form-data"):
|
||||
return None
|
||||
# parse boundary according to
|
||||
# https://www.rfc-editor.org/rfc/rfc2046#section-5.1.1
|
||||
if b";" in content_type:
|
||||
for section in content_type.split(b";"):
|
||||
if section.strip().lower().startswith(b"boundary="):
|
||||
return section.strip()[len(b"boundary=") :].strip(b'"')
|
||||
return None
|
||||
|
||||
|
||||
class DataField:
|
||||
"""
|
||||
A single form field item, within a multipart form field.
|
||||
"""
|
||||
|
||||
def __init__(self, name: str, value: str | bytes | int | float | None) -> None:
|
||||
if not isinstance(name, str):
|
||||
raise TypeError(
|
||||
f"Invalid type for name. Expected str, got {type(name)}: {name!r}"
|
||||
)
|
||||
if value is not None and not isinstance(value, (str, bytes, int, float)):
|
||||
raise TypeError(
|
||||
"Invalid type for value. Expected primitive type,"
|
||||
f" got {type(value)}: {value!r}"
|
||||
)
|
||||
self.name = name
|
||||
self.value: str | bytes = (
|
||||
value if isinstance(value, bytes) else primitive_value_to_str(value)
|
||||
)
|
||||
|
||||
def render_headers(self) -> bytes:
|
||||
if not hasattr(self, "_headers"):
|
||||
name = _format_form_param("name", self.name)
|
||||
self._headers = b"".join(
|
||||
[b"Content-Disposition: form-data; ", name, b"\r\n\r\n"]
|
||||
)
|
||||
|
||||
return self._headers
|
||||
|
||||
def render_data(self) -> bytes:
|
||||
if not hasattr(self, "_data"):
|
||||
self._data = to_bytes(self.value)
|
||||
|
||||
return self._data
|
||||
|
||||
def get_length(self) -> int:
|
||||
headers = self.render_headers()
|
||||
data = self.render_data()
|
||||
return len(headers) + len(data)
|
||||
|
||||
def render(self) -> typing.Iterator[bytes]:
|
||||
yield self.render_headers()
|
||||
yield self.render_data()
|
||||
|
||||
|
||||
class FileField:
|
||||
"""
|
||||
A single file field item, within a multipart form field.
|
||||
"""
|
||||
|
||||
CHUNK_SIZE = 64 * 1024
|
||||
|
||||
def __init__(self, name: str, value: FileTypes) -> None:
|
||||
self.name = name
|
||||
|
||||
fileobj: FileContent
|
||||
|
||||
headers: dict[str, str] = {}
|
||||
content_type: str | None = None
|
||||
|
||||
# This large tuple based API largely mirror's requests' API
|
||||
# It would be good to think of better APIs for this that we could
|
||||
# include in httpx 2.0 since variable length tuples(especially of 4 elements)
|
||||
# are quite unwieldly
|
||||
if isinstance(value, tuple):
|
||||
if len(value) == 2:
|
||||
# neither the 3rd parameter (content_type) nor the 4th (headers)
|
||||
# was included
|
||||
filename, fileobj = value
|
||||
elif len(value) == 3:
|
||||
filename, fileobj, content_type = value
|
||||
else:
|
||||
# all 4 parameters included
|
||||
filename, fileobj, content_type, headers = value # type: ignore
|
||||
else:
|
||||
filename = Path(str(getattr(value, "name", "upload"))).name
|
||||
fileobj = value
|
||||
|
||||
if content_type is None:
|
||||
content_type = _guess_content_type(filename)
|
||||
|
||||
has_content_type_header = any("content-type" in key.lower() for key in headers)
|
||||
if content_type is not None and not has_content_type_header:
|
||||
# note that unlike requests, we ignore the content_type provided in the 3rd
|
||||
# tuple element if it is also included in the headers requests does
|
||||
# the opposite (it overwrites the headerwith the 3rd tuple element)
|
||||
headers["Content-Type"] = content_type
|
||||
|
||||
if isinstance(fileobj, io.StringIO):
|
||||
raise TypeError(
|
||||
"Multipart file uploads require 'io.BytesIO', not 'io.StringIO'."
|
||||
)
|
||||
if isinstance(fileobj, io.TextIOBase):
|
||||
raise TypeError(
|
||||
"Multipart file uploads must be opened in binary mode, not text mode."
|
||||
)
|
||||
|
||||
self.filename = filename
|
||||
self.file = fileobj
|
||||
self.headers = headers
|
||||
|
||||
def get_length(self) -> int | None:
|
||||
headers = self.render_headers()
|
||||
|
||||
if isinstance(self.file, (str, bytes)):
|
||||
return len(headers) + len(to_bytes(self.file))
|
||||
|
||||
file_length = peek_filelike_length(self.file)
|
||||
|
||||
# If we can't determine the filesize without reading it into memory,
|
||||
# then return `None` here, to indicate an unknown file length.
|
||||
if file_length is None:
|
||||
return None
|
||||
|
||||
return len(headers) + file_length
|
||||
|
||||
def render_headers(self) -> bytes:
|
||||
if not hasattr(self, "_headers"):
|
||||
parts = [
|
||||
b"Content-Disposition: form-data; ",
|
||||
_format_form_param("name", self.name),
|
||||
]
|
||||
if self.filename:
|
||||
filename = _format_form_param("filename", self.filename)
|
||||
parts.extend([b"; ", filename])
|
||||
for header_name, header_value in self.headers.items():
|
||||
key, val = f"\r\n{header_name}: ".encode(), header_value.encode()
|
||||
parts.extend([key, val])
|
||||
parts.append(b"\r\n\r\n")
|
||||
self._headers = b"".join(parts)
|
||||
|
||||
return self._headers
|
||||
|
||||
def render_data(self) -> typing.Iterator[bytes]:
|
||||
if isinstance(self.file, (str, bytes)):
|
||||
yield to_bytes(self.file)
|
||||
return
|
||||
|
||||
if hasattr(self.file, "seek"):
|
||||
try:
|
||||
self.file.seek(0)
|
||||
except io.UnsupportedOperation:
|
||||
pass
|
||||
|
||||
chunk = self.file.read(self.CHUNK_SIZE)
|
||||
while chunk:
|
||||
yield to_bytes(chunk)
|
||||
chunk = self.file.read(self.CHUNK_SIZE)
|
||||
|
||||
def render(self) -> typing.Iterator[bytes]:
|
||||
yield self.render_headers()
|
||||
yield from self.render_data()
|
||||
|
||||
|
||||
class MultipartStream(SyncByteStream, AsyncByteStream):
|
||||
"""
|
||||
Request content as streaming multipart encoded form data.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
data: RequestData,
|
||||
files: RequestFiles,
|
||||
boundary: bytes | None = None,
|
||||
) -> None:
|
||||
if boundary is None:
|
||||
boundary = os.urandom(16).hex().encode("ascii")
|
||||
|
||||
self.boundary = boundary
|
||||
self.content_type = "multipart/form-data; boundary=%s" % boundary.decode(
|
||||
"ascii"
|
||||
)
|
||||
self.fields = list(self._iter_fields(data, files))
|
||||
|
||||
def _iter_fields(
|
||||
self, data: RequestData, files: RequestFiles
|
||||
) -> typing.Iterator[FileField | DataField]:
|
||||
for name, value in data.items():
|
||||
if isinstance(value, (tuple, list)):
|
||||
for item in value:
|
||||
yield DataField(name=name, value=item)
|
||||
else:
|
||||
yield DataField(name=name, value=value)
|
||||
|
||||
file_items = files.items() if isinstance(files, typing.Mapping) else files
|
||||
for name, value in file_items:
|
||||
yield FileField(name=name, value=value)
|
||||
|
||||
def iter_chunks(self) -> typing.Iterator[bytes]:
|
||||
for field in self.fields:
|
||||
yield b"--%s\r\n" % self.boundary
|
||||
yield from field.render()
|
||||
yield b"\r\n"
|
||||
yield b"--%s--\r\n" % self.boundary
|
||||
|
||||
def get_content_length(self) -> int | None:
|
||||
"""
|
||||
Return the length of the multipart encoded content, or `None` if
|
||||
any of the files have a length that cannot be determined upfront.
|
||||
"""
|
||||
boundary_length = len(self.boundary)
|
||||
length = 0
|
||||
|
||||
for field in self.fields:
|
||||
field_length = field.get_length()
|
||||
if field_length is None:
|
||||
return None
|
||||
|
||||
length += 2 + boundary_length + 2 # b"--{boundary}\r\n"
|
||||
length += field_length
|
||||
length += 2 # b"\r\n"
|
||||
|
||||
length += 2 + boundary_length + 4 # b"--{boundary}--\r\n"
|
||||
return length
|
||||
|
||||
# Content stream interface.
|
||||
|
||||
def get_headers(self) -> dict[str, str]:
|
||||
content_length = self.get_content_length()
|
||||
content_type = self.content_type
|
||||
if content_length is None:
|
||||
return {"Transfer-Encoding": "chunked", "Content-Type": content_type}
|
||||
return {"Content-Length": str(content_length), "Content-Type": content_type}
|
||||
|
||||
def __iter__(self) -> typing.Iterator[bytes]:
|
||||
for chunk in self.iter_chunks():
|
||||
yield chunk
|
||||
|
||||
async def __aiter__(self) -> typing.AsyncIterator[bytes]:
|
||||
for chunk in self.iter_chunks():
|
||||
yield chunk
|
||||
@ -1,162 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from enum import IntEnum
|
||||
|
||||
__all__ = ["codes"]
|
||||
|
||||
|
||||
class codes(IntEnum):
|
||||
"""HTTP status codes and reason phrases
|
||||
|
||||
Status codes from the following RFCs are all observed:
|
||||
|
||||
* RFC 7231: Hypertext Transfer Protocol (HTTP/1.1), obsoletes 2616
|
||||
* RFC 6585: Additional HTTP Status Codes
|
||||
* RFC 3229: Delta encoding in HTTP
|
||||
* RFC 4918: HTTP Extensions for WebDAV, obsoletes 2518
|
||||
* RFC 5842: Binding Extensions to WebDAV
|
||||
* RFC 7238: Permanent Redirect
|
||||
* RFC 2295: Transparent Content Negotiation in HTTP
|
||||
* RFC 2774: An HTTP Extension Framework
|
||||
* RFC 7540: Hypertext Transfer Protocol Version 2 (HTTP/2)
|
||||
* RFC 2324: Hyper Text Coffee Pot Control Protocol (HTCPCP/1.0)
|
||||
* RFC 7725: An HTTP Status Code to Report Legal Obstacles
|
||||
* RFC 8297: An HTTP Status Code for Indicating Hints
|
||||
* RFC 8470: Using Early Data in HTTP
|
||||
"""
|
||||
|
||||
def __new__(cls, value: int, phrase: str = "") -> codes:
|
||||
obj = int.__new__(cls, value)
|
||||
obj._value_ = value
|
||||
|
||||
obj.phrase = phrase # type: ignore[attr-defined]
|
||||
return obj
|
||||
|
||||
def __str__(self) -> str:
|
||||
return str(self.value)
|
||||
|
||||
@classmethod
|
||||
def get_reason_phrase(cls, value: int) -> str:
|
||||
try:
|
||||
return codes(value).phrase # type: ignore
|
||||
except ValueError:
|
||||
return ""
|
||||
|
||||
@classmethod
|
||||
def is_informational(cls, value: int) -> bool:
|
||||
"""
|
||||
Returns `True` for 1xx status codes, `False` otherwise.
|
||||
"""
|
||||
return 100 <= value <= 199
|
||||
|
||||
@classmethod
|
||||
def is_success(cls, value: int) -> bool:
|
||||
"""
|
||||
Returns `True` for 2xx status codes, `False` otherwise.
|
||||
"""
|
||||
return 200 <= value <= 299
|
||||
|
||||
@classmethod
|
||||
def is_redirect(cls, value: int) -> bool:
|
||||
"""
|
||||
Returns `True` for 3xx status codes, `False` otherwise.
|
||||
"""
|
||||
return 300 <= value <= 399
|
||||
|
||||
@classmethod
|
||||
def is_client_error(cls, value: int) -> bool:
|
||||
"""
|
||||
Returns `True` for 4xx status codes, `False` otherwise.
|
||||
"""
|
||||
return 400 <= value <= 499
|
||||
|
||||
@classmethod
|
||||
def is_server_error(cls, value: int) -> bool:
|
||||
"""
|
||||
Returns `True` for 5xx status codes, `False` otherwise.
|
||||
"""
|
||||
return 500 <= value <= 599
|
||||
|
||||
@classmethod
|
||||
def is_error(cls, value: int) -> bool:
|
||||
"""
|
||||
Returns `True` for 4xx or 5xx status codes, `False` otherwise.
|
||||
"""
|
||||
return 400 <= value <= 599
|
||||
|
||||
# informational
|
||||
CONTINUE = 100, "Continue"
|
||||
SWITCHING_PROTOCOLS = 101, "Switching Protocols"
|
||||
PROCESSING = 102, "Processing"
|
||||
EARLY_HINTS = 103, "Early Hints"
|
||||
|
||||
# success
|
||||
OK = 200, "OK"
|
||||
CREATED = 201, "Created"
|
||||
ACCEPTED = 202, "Accepted"
|
||||
NON_AUTHORITATIVE_INFORMATION = 203, "Non-Authoritative Information"
|
||||
NO_CONTENT = 204, "No Content"
|
||||
RESET_CONTENT = 205, "Reset Content"
|
||||
PARTIAL_CONTENT = 206, "Partial Content"
|
||||
MULTI_STATUS = 207, "Multi-Status"
|
||||
ALREADY_REPORTED = 208, "Already Reported"
|
||||
IM_USED = 226, "IM Used"
|
||||
|
||||
# redirection
|
||||
MULTIPLE_CHOICES = 300, "Multiple Choices"
|
||||
MOVED_PERMANENTLY = 301, "Moved Permanently"
|
||||
FOUND = 302, "Found"
|
||||
SEE_OTHER = 303, "See Other"
|
||||
NOT_MODIFIED = 304, "Not Modified"
|
||||
USE_PROXY = 305, "Use Proxy"
|
||||
TEMPORARY_REDIRECT = 307, "Temporary Redirect"
|
||||
PERMANENT_REDIRECT = 308, "Permanent Redirect"
|
||||
|
||||
# client error
|
||||
BAD_REQUEST = 400, "Bad Request"
|
||||
UNAUTHORIZED = 401, "Unauthorized"
|
||||
PAYMENT_REQUIRED = 402, "Payment Required"
|
||||
FORBIDDEN = 403, "Forbidden"
|
||||
NOT_FOUND = 404, "Not Found"
|
||||
METHOD_NOT_ALLOWED = 405, "Method Not Allowed"
|
||||
NOT_ACCEPTABLE = 406, "Not Acceptable"
|
||||
PROXY_AUTHENTICATION_REQUIRED = 407, "Proxy Authentication Required"
|
||||
REQUEST_TIMEOUT = 408, "Request Timeout"
|
||||
CONFLICT = 409, "Conflict"
|
||||
GONE = 410, "Gone"
|
||||
LENGTH_REQUIRED = 411, "Length Required"
|
||||
PRECONDITION_FAILED = 412, "Precondition Failed"
|
||||
REQUEST_ENTITY_TOO_LARGE = 413, "Request Entity Too Large"
|
||||
REQUEST_URI_TOO_LONG = 414, "Request-URI Too Long"
|
||||
UNSUPPORTED_MEDIA_TYPE = 415, "Unsupported Media Type"
|
||||
REQUESTED_RANGE_NOT_SATISFIABLE = 416, "Requested Range Not Satisfiable"
|
||||
EXPECTATION_FAILED = 417, "Expectation Failed"
|
||||
IM_A_TEAPOT = 418, "I'm a teapot"
|
||||
MISDIRECTED_REQUEST = 421, "Misdirected Request"
|
||||
UNPROCESSABLE_ENTITY = 422, "Unprocessable Entity"
|
||||
LOCKED = 423, "Locked"
|
||||
FAILED_DEPENDENCY = 424, "Failed Dependency"
|
||||
TOO_EARLY = 425, "Too Early"
|
||||
UPGRADE_REQUIRED = 426, "Upgrade Required"
|
||||
PRECONDITION_REQUIRED = 428, "Precondition Required"
|
||||
TOO_MANY_REQUESTS = 429, "Too Many Requests"
|
||||
REQUEST_HEADER_FIELDS_TOO_LARGE = 431, "Request Header Fields Too Large"
|
||||
UNAVAILABLE_FOR_LEGAL_REASONS = 451, "Unavailable For Legal Reasons"
|
||||
|
||||
# server errors
|
||||
INTERNAL_SERVER_ERROR = 500, "Internal Server Error"
|
||||
NOT_IMPLEMENTED = 501, "Not Implemented"
|
||||
BAD_GATEWAY = 502, "Bad Gateway"
|
||||
SERVICE_UNAVAILABLE = 503, "Service Unavailable"
|
||||
GATEWAY_TIMEOUT = 504, "Gateway Timeout"
|
||||
HTTP_VERSION_NOT_SUPPORTED = 505, "HTTP Version Not Supported"
|
||||
VARIANT_ALSO_NEGOTIATES = 506, "Variant Also Negotiates"
|
||||
INSUFFICIENT_STORAGE = 507, "Insufficient Storage"
|
||||
LOOP_DETECTED = 508, "Loop Detected"
|
||||
NOT_EXTENDED = 510, "Not Extended"
|
||||
NETWORK_AUTHENTICATION_REQUIRED = 511, "Network Authentication Required"
|
||||
|
||||
|
||||
# Include lower-case styles for `requests` compatibility.
|
||||
for code in codes:
|
||||
setattr(codes, code._name_.lower(), int(code))
|
||||
@ -1,15 +0,0 @@
|
||||
from .asgi import *
|
||||
from .base import *
|
||||
from .default import *
|
||||
from .mock import *
|
||||
from .wsgi import *
|
||||
|
||||
__all__ = [
|
||||
"ASGITransport",
|
||||
"AsyncBaseTransport",
|
||||
"BaseTransport",
|
||||
"AsyncHTTPTransport",
|
||||
"HTTPTransport",
|
||||
"MockTransport",
|
||||
"WSGITransport",
|
||||
]
|
||||
@ -1,187 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import typing
|
||||
|
||||
from .._models import Request, Response
|
||||
from .._types import AsyncByteStream
|
||||
from .base import AsyncBaseTransport
|
||||
|
||||
if typing.TYPE_CHECKING: # pragma: no cover
|
||||
import asyncio
|
||||
|
||||
import trio
|
||||
|
||||
Event = typing.Union[asyncio.Event, trio.Event]
|
||||
|
||||
|
||||
_Message = typing.MutableMapping[str, typing.Any]
|
||||
_Receive = typing.Callable[[], typing.Awaitable[_Message]]
|
||||
_Send = typing.Callable[
|
||||
[typing.MutableMapping[str, typing.Any]], typing.Awaitable[None]
|
||||
]
|
||||
_ASGIApp = typing.Callable[
|
||||
[typing.MutableMapping[str, typing.Any], _Receive, _Send], typing.Awaitable[None]
|
||||
]
|
||||
|
||||
__all__ = ["ASGITransport"]
|
||||
|
||||
|
||||
def is_running_trio() -> bool:
|
||||
try:
|
||||
# sniffio is a dependency of trio.
|
||||
|
||||
# See https://github.com/python-trio/trio/issues/2802
|
||||
import sniffio
|
||||
|
||||
if sniffio.current_async_library() == "trio":
|
||||
return True
|
||||
except ImportError: # pragma: nocover
|
||||
pass
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def create_event() -> Event:
|
||||
if is_running_trio():
|
||||
import trio
|
||||
|
||||
return trio.Event()
|
||||
|
||||
import asyncio
|
||||
|
||||
return asyncio.Event()
|
||||
|
||||
|
||||
class ASGIResponseStream(AsyncByteStream):
|
||||
def __init__(self, body: list[bytes]) -> None:
|
||||
self._body = body
|
||||
|
||||
async def __aiter__(self) -> typing.AsyncIterator[bytes]:
|
||||
yield b"".join(self._body)
|
||||
|
||||
|
||||
class ASGITransport(AsyncBaseTransport):
|
||||
"""
|
||||
A custom AsyncTransport that handles sending requests directly to an ASGI app.
|
||||
|
||||
```python
|
||||
transport = httpx.ASGITransport(
|
||||
app=app,
|
||||
root_path="/submount",
|
||||
client=("1.2.3.4", 123)
|
||||
)
|
||||
client = httpx.AsyncClient(transport=transport)
|
||||
```
|
||||
|
||||
Arguments:
|
||||
|
||||
* `app` - The ASGI application.
|
||||
* `raise_app_exceptions` - Boolean indicating if exceptions in the application
|
||||
should be raised. Default to `True`. Can be set to `False` for use cases
|
||||
such as testing the content of a client 500 response.
|
||||
* `root_path` - The root path on which the ASGI application should be mounted.
|
||||
* `client` - A two-tuple indicating the client IP and port of incoming requests.
|
||||
```
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
app: _ASGIApp,
|
||||
raise_app_exceptions: bool = True,
|
||||
root_path: str = "",
|
||||
client: tuple[str, int] = ("127.0.0.1", 123),
|
||||
) -> None:
|
||||
self.app = app
|
||||
self.raise_app_exceptions = raise_app_exceptions
|
||||
self.root_path = root_path
|
||||
self.client = client
|
||||
|
||||
async def handle_async_request(
|
||||
self,
|
||||
request: Request,
|
||||
) -> Response:
|
||||
assert isinstance(request.stream, AsyncByteStream)
|
||||
|
||||
# ASGI scope.
|
||||
scope = {
|
||||
"type": "http",
|
||||
"asgi": {"version": "3.0"},
|
||||
"http_version": "1.1",
|
||||
"method": request.method,
|
||||
"headers": [(k.lower(), v) for (k, v) in request.headers.raw],
|
||||
"scheme": request.url.scheme,
|
||||
"path": request.url.path,
|
||||
"raw_path": request.url.raw_path.split(b"?")[0],
|
||||
"query_string": request.url.query,
|
||||
"server": (request.url.host, request.url.port),
|
||||
"client": self.client,
|
||||
"root_path": self.root_path,
|
||||
}
|
||||
|
||||
# Request.
|
||||
request_body_chunks = request.stream.__aiter__()
|
||||
request_complete = False
|
||||
|
||||
# Response.
|
||||
status_code = None
|
||||
response_headers = None
|
||||
body_parts = []
|
||||
response_started = False
|
||||
response_complete = create_event()
|
||||
|
||||
# ASGI callables.
|
||||
|
||||
async def receive() -> dict[str, typing.Any]:
|
||||
nonlocal request_complete
|
||||
|
||||
if request_complete:
|
||||
await response_complete.wait()
|
||||
return {"type": "http.disconnect"}
|
||||
|
||||
try:
|
||||
body = await request_body_chunks.__anext__()
|
||||
except StopAsyncIteration:
|
||||
request_complete = True
|
||||
return {"type": "http.request", "body": b"", "more_body": False}
|
||||
return {"type": "http.request", "body": body, "more_body": True}
|
||||
|
||||
async def send(message: typing.MutableMapping[str, typing.Any]) -> None:
|
||||
nonlocal status_code, response_headers, response_started
|
||||
|
||||
if message["type"] == "http.response.start":
|
||||
assert not response_started
|
||||
|
||||
status_code = message["status"]
|
||||
response_headers = message.get("headers", [])
|
||||
response_started = True
|
||||
|
||||
elif message["type"] == "http.response.body":
|
||||
assert not response_complete.is_set()
|
||||
body = message.get("body", b"")
|
||||
more_body = message.get("more_body", False)
|
||||
|
||||
if body and request.method != "HEAD":
|
||||
body_parts.append(body)
|
||||
|
||||
if not more_body:
|
||||
response_complete.set()
|
||||
|
||||
try:
|
||||
await self.app(scope, receive, send)
|
||||
except Exception: # noqa: PIE-786
|
||||
if self.raise_app_exceptions:
|
||||
raise
|
||||
|
||||
response_complete.set()
|
||||
if status_code is None:
|
||||
status_code = 500
|
||||
if response_headers is None:
|
||||
response_headers = {}
|
||||
|
||||
assert response_complete.is_set()
|
||||
assert status_code is not None
|
||||
assert response_headers is not None
|
||||
|
||||
stream = ASGIResponseStream(body_parts)
|
||||
|
||||
return Response(status_code, headers=response_headers, stream=stream)
|
||||
@ -1,86 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import typing
|
||||
from types import TracebackType
|
||||
|
||||
from .._models import Request, Response
|
||||
|
||||
T = typing.TypeVar("T", bound="BaseTransport")
|
||||
A = typing.TypeVar("A", bound="AsyncBaseTransport")
|
||||
|
||||
__all__ = ["AsyncBaseTransport", "BaseTransport"]
|
||||
|
||||
|
||||
class BaseTransport:
|
||||
def __enter__(self: T) -> T:
|
||||
return self
|
||||
|
||||
def __exit__(
|
||||
self,
|
||||
exc_type: type[BaseException] | None = None,
|
||||
exc_value: BaseException | None = None,
|
||||
traceback: TracebackType | None = None,
|
||||
) -> None:
|
||||
self.close()
|
||||
|
||||
def handle_request(self, request: Request) -> Response:
|
||||
"""
|
||||
Send a single HTTP request and return a response.
|
||||
|
||||
Developers shouldn't typically ever need to call into this API directly,
|
||||
since the Client class provides all the higher level user-facing API
|
||||
niceties.
|
||||
|
||||
In order to properly release any network resources, the response
|
||||
stream should *either* be consumed immediately, with a call to
|
||||
`response.stream.read()`, or else the `handle_request` call should
|
||||
be followed with a try/finally block to ensuring the stream is
|
||||
always closed.
|
||||
|
||||
Example usage:
|
||||
|
||||
with httpx.HTTPTransport() as transport:
|
||||
req = httpx.Request(
|
||||
method=b"GET",
|
||||
url=(b"https", b"www.example.com", 443, b"/"),
|
||||
headers=[(b"Host", b"www.example.com")],
|
||||
)
|
||||
resp = transport.handle_request(req)
|
||||
body = resp.stream.read()
|
||||
print(resp.status_code, resp.headers, body)
|
||||
|
||||
|
||||
Takes a `Request` instance as the only argument.
|
||||
|
||||
Returns a `Response` instance.
|
||||
"""
|
||||
raise NotImplementedError(
|
||||
"The 'handle_request' method must be implemented."
|
||||
) # pragma: no cover
|
||||
|
||||
def close(self) -> None:
|
||||
pass
|
||||
|
||||
|
||||
class AsyncBaseTransport:
|
||||
async def __aenter__(self: A) -> A:
|
||||
return self
|
||||
|
||||
async def __aexit__(
|
||||
self,
|
||||
exc_type: type[BaseException] | None = None,
|
||||
exc_value: BaseException | None = None,
|
||||
traceback: TracebackType | None = None,
|
||||
) -> None:
|
||||
await self.aclose()
|
||||
|
||||
async def handle_async_request(
|
||||
self,
|
||||
request: Request,
|
||||
) -> Response:
|
||||
raise NotImplementedError(
|
||||
"The 'handle_async_request' method must be implemented."
|
||||
) # pragma: no cover
|
||||
|
||||
async def aclose(self) -> None:
|
||||
pass
|
||||
@ -1,406 +0,0 @@
|
||||
"""
|
||||
Custom transports, with nicely configured defaults.
|
||||
|
||||
The following additional keyword arguments are currently supported by httpcore...
|
||||
|
||||
* uds: str
|
||||
* local_address: str
|
||||
* retries: int
|
||||
|
||||
Example usages...
|
||||
|
||||
# Disable HTTP/2 on a single specific domain.
|
||||
mounts = {
|
||||
"all://": httpx.HTTPTransport(http2=True),
|
||||
"all://*example.org": httpx.HTTPTransport()
|
||||
}
|
||||
|
||||
# Using advanced httpcore configuration, with connection retries.
|
||||
transport = httpx.HTTPTransport(retries=1)
|
||||
client = httpx.Client(transport=transport)
|
||||
|
||||
# Using advanced httpcore configuration, with unix domain sockets.
|
||||
transport = httpx.HTTPTransport(uds="socket.uds")
|
||||
client = httpx.Client(transport=transport)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import typing
|
||||
from types import TracebackType
|
||||
|
||||
if typing.TYPE_CHECKING:
|
||||
import ssl # pragma: no cover
|
||||
|
||||
import httpx # pragma: no cover
|
||||
|
||||
from .._config import DEFAULT_LIMITS, Limits, Proxy, create_ssl_context
|
||||
from .._exceptions import (
|
||||
ConnectError,
|
||||
ConnectTimeout,
|
||||
LocalProtocolError,
|
||||
NetworkError,
|
||||
PoolTimeout,
|
||||
ProtocolError,
|
||||
ProxyError,
|
||||
ReadError,
|
||||
ReadTimeout,
|
||||
RemoteProtocolError,
|
||||
TimeoutException,
|
||||
UnsupportedProtocol,
|
||||
WriteError,
|
||||
WriteTimeout,
|
||||
)
|
||||
from .._models import Request, Response
|
||||
from .._types import AsyncByteStream, CertTypes, ProxyTypes, SyncByteStream
|
||||
from .._urls import URL
|
||||
from .base import AsyncBaseTransport, BaseTransport
|
||||
|
||||
T = typing.TypeVar("T", bound="HTTPTransport")
|
||||
A = typing.TypeVar("A", bound="AsyncHTTPTransport")
|
||||
|
||||
SOCKET_OPTION = typing.Union[
|
||||
typing.Tuple[int, int, int],
|
||||
typing.Tuple[int, int, typing.Union[bytes, bytearray]],
|
||||
typing.Tuple[int, int, None, int],
|
||||
]
|
||||
|
||||
__all__ = ["AsyncHTTPTransport", "HTTPTransport"]
|
||||
|
||||
HTTPCORE_EXC_MAP: dict[type[Exception], type[httpx.HTTPError]] = {}
|
||||
|
||||
|
||||
def _load_httpcore_exceptions() -> dict[type[Exception], type[httpx.HTTPError]]:
|
||||
import httpcore
|
||||
|
||||
return {
|
||||
httpcore.TimeoutException: TimeoutException,
|
||||
httpcore.ConnectTimeout: ConnectTimeout,
|
||||
httpcore.ReadTimeout: ReadTimeout,
|
||||
httpcore.WriteTimeout: WriteTimeout,
|
||||
httpcore.PoolTimeout: PoolTimeout,
|
||||
httpcore.NetworkError: NetworkError,
|
||||
httpcore.ConnectError: ConnectError,
|
||||
httpcore.ReadError: ReadError,
|
||||
httpcore.WriteError: WriteError,
|
||||
httpcore.ProxyError: ProxyError,
|
||||
httpcore.UnsupportedProtocol: UnsupportedProtocol,
|
||||
httpcore.ProtocolError: ProtocolError,
|
||||
httpcore.LocalProtocolError: LocalProtocolError,
|
||||
httpcore.RemoteProtocolError: RemoteProtocolError,
|
||||
}
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def map_httpcore_exceptions() -> typing.Iterator[None]:
|
||||
global HTTPCORE_EXC_MAP
|
||||
if len(HTTPCORE_EXC_MAP) == 0:
|
||||
HTTPCORE_EXC_MAP = _load_httpcore_exceptions()
|
||||
try:
|
||||
yield
|
||||
except Exception as exc:
|
||||
mapped_exc = None
|
||||
|
||||
for from_exc, to_exc in HTTPCORE_EXC_MAP.items():
|
||||
if not isinstance(exc, from_exc):
|
||||
continue
|
||||
# We want to map to the most specific exception we can find.
|
||||
# Eg if `exc` is an `httpcore.ReadTimeout`, we want to map to
|
||||
# `httpx.ReadTimeout`, not just `httpx.TimeoutException`.
|
||||
if mapped_exc is None or issubclass(to_exc, mapped_exc):
|
||||
mapped_exc = to_exc
|
||||
|
||||
if mapped_exc is None: # pragma: no cover
|
||||
raise
|
||||
|
||||
message = str(exc)
|
||||
raise mapped_exc(message) from exc
|
||||
|
||||
|
||||
class ResponseStream(SyncByteStream):
|
||||
def __init__(self, httpcore_stream: typing.Iterable[bytes]) -> None:
|
||||
self._httpcore_stream = httpcore_stream
|
||||
|
||||
def __iter__(self) -> typing.Iterator[bytes]:
|
||||
with map_httpcore_exceptions():
|
||||
for part in self._httpcore_stream:
|
||||
yield part
|
||||
|
||||
def close(self) -> None:
|
||||
if hasattr(self._httpcore_stream, "close"):
|
||||
self._httpcore_stream.close()
|
||||
|
||||
|
||||
class HTTPTransport(BaseTransport):
|
||||
def __init__(
|
||||
self,
|
||||
verify: ssl.SSLContext | str | bool = True,
|
||||
cert: CertTypes | None = None,
|
||||
trust_env: bool = True,
|
||||
http1: bool = True,
|
||||
http2: bool = False,
|
||||
limits: Limits = DEFAULT_LIMITS,
|
||||
proxy: ProxyTypes | None = None,
|
||||
uds: str | None = None,
|
||||
local_address: str | None = None,
|
||||
retries: int = 0,
|
||||
socket_options: typing.Iterable[SOCKET_OPTION] | None = None,
|
||||
) -> None:
|
||||
import httpcore
|
||||
|
||||
proxy = Proxy(url=proxy) if isinstance(proxy, (str, URL)) else proxy
|
||||
ssl_context = create_ssl_context(verify=verify, cert=cert, trust_env=trust_env)
|
||||
|
||||
if proxy is None:
|
||||
self._pool = httpcore.ConnectionPool(
|
||||
ssl_context=ssl_context,
|
||||
max_connections=limits.max_connections,
|
||||
max_keepalive_connections=limits.max_keepalive_connections,
|
||||
keepalive_expiry=limits.keepalive_expiry,
|
||||
http1=http1,
|
||||
http2=http2,
|
||||
uds=uds,
|
||||
local_address=local_address,
|
||||
retries=retries,
|
||||
socket_options=socket_options,
|
||||
)
|
||||
elif proxy.url.scheme in ("http", "https"):
|
||||
self._pool = httpcore.HTTPProxy(
|
||||
proxy_url=httpcore.URL(
|
||||
scheme=proxy.url.raw_scheme,
|
||||
host=proxy.url.raw_host,
|
||||
port=proxy.url.port,
|
||||
target=proxy.url.raw_path,
|
||||
),
|
||||
proxy_auth=proxy.raw_auth,
|
||||
proxy_headers=proxy.headers.raw,
|
||||
ssl_context=ssl_context,
|
||||
proxy_ssl_context=proxy.ssl_context,
|
||||
max_connections=limits.max_connections,
|
||||
max_keepalive_connections=limits.max_keepalive_connections,
|
||||
keepalive_expiry=limits.keepalive_expiry,
|
||||
http1=http1,
|
||||
http2=http2,
|
||||
socket_options=socket_options,
|
||||
)
|
||||
elif proxy.url.scheme in ("socks5", "socks5h"):
|
||||
try:
|
||||
import socksio # noqa
|
||||
except ImportError: # pragma: no cover
|
||||
raise ImportError(
|
||||
"Using SOCKS proxy, but the 'socksio' package is not installed. "
|
||||
"Make sure to install httpx using `pip install httpx[socks]`."
|
||||
) from None
|
||||
|
||||
self._pool = httpcore.SOCKSProxy(
|
||||
proxy_url=httpcore.URL(
|
||||
scheme=proxy.url.raw_scheme,
|
||||
host=proxy.url.raw_host,
|
||||
port=proxy.url.port,
|
||||
target=proxy.url.raw_path,
|
||||
),
|
||||
proxy_auth=proxy.raw_auth,
|
||||
ssl_context=ssl_context,
|
||||
max_connections=limits.max_connections,
|
||||
max_keepalive_connections=limits.max_keepalive_connections,
|
||||
keepalive_expiry=limits.keepalive_expiry,
|
||||
http1=http1,
|
||||
http2=http2,
|
||||
)
|
||||
else: # pragma: no cover
|
||||
raise ValueError(
|
||||
"Proxy protocol must be either 'http', 'https', 'socks5', or 'socks5h',"
|
||||
f" but got {proxy.url.scheme!r}."
|
||||
)
|
||||
|
||||
def __enter__(self: T) -> T: # Use generics for subclass support.
|
||||
self._pool.__enter__()
|
||||
return self
|
||||
|
||||
def __exit__(
|
||||
self,
|
||||
exc_type: type[BaseException] | None = None,
|
||||
exc_value: BaseException | None = None,
|
||||
traceback: TracebackType | None = None,
|
||||
) -> None:
|
||||
with map_httpcore_exceptions():
|
||||
self._pool.__exit__(exc_type, exc_value, traceback)
|
||||
|
||||
def handle_request(
|
||||
self,
|
||||
request: Request,
|
||||
) -> Response:
|
||||
assert isinstance(request.stream, SyncByteStream)
|
||||
import httpcore
|
||||
|
||||
req = httpcore.Request(
|
||||
method=request.method,
|
||||
url=httpcore.URL(
|
||||
scheme=request.url.raw_scheme,
|
||||
host=request.url.raw_host,
|
||||
port=request.url.port,
|
||||
target=request.url.raw_path,
|
||||
),
|
||||
headers=request.headers.raw,
|
||||
content=request.stream,
|
||||
extensions=request.extensions,
|
||||
)
|
||||
with map_httpcore_exceptions():
|
||||
resp = self._pool.handle_request(req)
|
||||
|
||||
assert isinstance(resp.stream, typing.Iterable)
|
||||
|
||||
return Response(
|
||||
status_code=resp.status,
|
||||
headers=resp.headers,
|
||||
stream=ResponseStream(resp.stream),
|
||||
extensions=resp.extensions,
|
||||
)
|
||||
|
||||
def close(self) -> None:
|
||||
self._pool.close()
|
||||
|
||||
|
||||
class AsyncResponseStream(AsyncByteStream):
|
||||
def __init__(self, httpcore_stream: typing.AsyncIterable[bytes]) -> None:
|
||||
self._httpcore_stream = httpcore_stream
|
||||
|
||||
async def __aiter__(self) -> typing.AsyncIterator[bytes]:
|
||||
with map_httpcore_exceptions():
|
||||
async for part in self._httpcore_stream:
|
||||
yield part
|
||||
|
||||
async def aclose(self) -> None:
|
||||
if hasattr(self._httpcore_stream, "aclose"):
|
||||
await self._httpcore_stream.aclose()
|
||||
|
||||
|
||||
class AsyncHTTPTransport(AsyncBaseTransport):
|
||||
def __init__(
|
||||
self,
|
||||
verify: ssl.SSLContext | str | bool = True,
|
||||
cert: CertTypes | None = None,
|
||||
trust_env: bool = True,
|
||||
http1: bool = True,
|
||||
http2: bool = False,
|
||||
limits: Limits = DEFAULT_LIMITS,
|
||||
proxy: ProxyTypes | None = None,
|
||||
uds: str | None = None,
|
||||
local_address: str | None = None,
|
||||
retries: int = 0,
|
||||
socket_options: typing.Iterable[SOCKET_OPTION] | None = None,
|
||||
) -> None:
|
||||
import httpcore
|
||||
|
||||
proxy = Proxy(url=proxy) if isinstance(proxy, (str, URL)) else proxy
|
||||
ssl_context = create_ssl_context(verify=verify, cert=cert, trust_env=trust_env)
|
||||
|
||||
if proxy is None:
|
||||
self._pool = httpcore.AsyncConnectionPool(
|
||||
ssl_context=ssl_context,
|
||||
max_connections=limits.max_connections,
|
||||
max_keepalive_connections=limits.max_keepalive_connections,
|
||||
keepalive_expiry=limits.keepalive_expiry,
|
||||
http1=http1,
|
||||
http2=http2,
|
||||
uds=uds,
|
||||
local_address=local_address,
|
||||
retries=retries,
|
||||
socket_options=socket_options,
|
||||
)
|
||||
elif proxy.url.scheme in ("http", "https"):
|
||||
self._pool = httpcore.AsyncHTTPProxy(
|
||||
proxy_url=httpcore.URL(
|
||||
scheme=proxy.url.raw_scheme,
|
||||
host=proxy.url.raw_host,
|
||||
port=proxy.url.port,
|
||||
target=proxy.url.raw_path,
|
||||
),
|
||||
proxy_auth=proxy.raw_auth,
|
||||
proxy_headers=proxy.headers.raw,
|
||||
proxy_ssl_context=proxy.ssl_context,
|
||||
ssl_context=ssl_context,
|
||||
max_connections=limits.max_connections,
|
||||
max_keepalive_connections=limits.max_keepalive_connections,
|
||||
keepalive_expiry=limits.keepalive_expiry,
|
||||
http1=http1,
|
||||
http2=http2,
|
||||
socket_options=socket_options,
|
||||
)
|
||||
elif proxy.url.scheme in ("socks5", "socks5h"):
|
||||
try:
|
||||
import socksio # noqa
|
||||
except ImportError: # pragma: no cover
|
||||
raise ImportError(
|
||||
"Using SOCKS proxy, but the 'socksio' package is not installed. "
|
||||
"Make sure to install httpx using `pip install httpx[socks]`."
|
||||
) from None
|
||||
|
||||
self._pool = httpcore.AsyncSOCKSProxy(
|
||||
proxy_url=httpcore.URL(
|
||||
scheme=proxy.url.raw_scheme,
|
||||
host=proxy.url.raw_host,
|
||||
port=proxy.url.port,
|
||||
target=proxy.url.raw_path,
|
||||
),
|
||||
proxy_auth=proxy.raw_auth,
|
||||
ssl_context=ssl_context,
|
||||
max_connections=limits.max_connections,
|
||||
max_keepalive_connections=limits.max_keepalive_connections,
|
||||
keepalive_expiry=limits.keepalive_expiry,
|
||||
http1=http1,
|
||||
http2=http2,
|
||||
)
|
||||
else: # pragma: no cover
|
||||
raise ValueError(
|
||||
"Proxy protocol must be either 'http', 'https', 'socks5', or 'socks5h',"
|
||||
f" but got {proxy.url.scheme!r}."
|
||||
)
|
||||
|
||||
async def __aenter__(self: A) -> A: # Use generics for subclass support.
|
||||
await self._pool.__aenter__()
|
||||
return self
|
||||
|
||||
async def __aexit__(
|
||||
self,
|
||||
exc_type: type[BaseException] | None = None,
|
||||
exc_value: BaseException | None = None,
|
||||
traceback: TracebackType | None = None,
|
||||
) -> None:
|
||||
with map_httpcore_exceptions():
|
||||
await self._pool.__aexit__(exc_type, exc_value, traceback)
|
||||
|
||||
async def handle_async_request(
|
||||
self,
|
||||
request: Request,
|
||||
) -> Response:
|
||||
assert isinstance(request.stream, AsyncByteStream)
|
||||
import httpcore
|
||||
|
||||
req = httpcore.Request(
|
||||
method=request.method,
|
||||
url=httpcore.URL(
|
||||
scheme=request.url.raw_scheme,
|
||||
host=request.url.raw_host,
|
||||
port=request.url.port,
|
||||
target=request.url.raw_path,
|
||||
),
|
||||
headers=request.headers.raw,
|
||||
content=request.stream,
|
||||
extensions=request.extensions,
|
||||
)
|
||||
with map_httpcore_exceptions():
|
||||
resp = await self._pool.handle_async_request(req)
|
||||
|
||||
assert isinstance(resp.stream, typing.AsyncIterable)
|
||||
|
||||
return Response(
|
||||
status_code=resp.status,
|
||||
headers=resp.headers,
|
||||
stream=AsyncResponseStream(resp.stream),
|
||||
extensions=resp.extensions,
|
||||
)
|
||||
|
||||
async def aclose(self) -> None:
|
||||
await self._pool.aclose()
|
||||
@ -1,43 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import typing
|
||||
|
||||
from .._models import Request, Response
|
||||
from .base import AsyncBaseTransport, BaseTransport
|
||||
|
||||
SyncHandler = typing.Callable[[Request], Response]
|
||||
AsyncHandler = typing.Callable[[Request], typing.Coroutine[None, None, Response]]
|
||||
|
||||
|
||||
__all__ = ["MockTransport"]
|
||||
|
||||
|
||||
class MockTransport(AsyncBaseTransport, BaseTransport):
|
||||
def __init__(self, handler: SyncHandler | AsyncHandler) -> None:
|
||||
self.handler = handler
|
||||
|
||||
def handle_request(
|
||||
self,
|
||||
request: Request,
|
||||
) -> Response:
|
||||
request.read()
|
||||
response = self.handler(request)
|
||||
if not isinstance(response, Response): # pragma: no cover
|
||||
raise TypeError("Cannot use an async handler in a sync Client")
|
||||
return response
|
||||
|
||||
async def handle_async_request(
|
||||
self,
|
||||
request: Request,
|
||||
) -> Response:
|
||||
await request.aread()
|
||||
response = self.handler(request)
|
||||
|
||||
# Allow handler to *optionally* be an `async` function.
|
||||
# If it is, then the `response` variable need to be awaited to actually
|
||||
# return the result.
|
||||
|
||||
if not isinstance(response, Response):
|
||||
response = await response
|
||||
|
||||
return response
|
||||
@ -1,149 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import itertools
|
||||
import sys
|
||||
import typing
|
||||
|
||||
from .._models import Request, Response
|
||||
from .._types import SyncByteStream
|
||||
from .base import BaseTransport
|
||||
|
||||
if typing.TYPE_CHECKING:
|
||||
from _typeshed import OptExcInfo # pragma: no cover
|
||||
from _typeshed.wsgi import WSGIApplication # pragma: no cover
|
||||
|
||||
_T = typing.TypeVar("_T")
|
||||
|
||||
|
||||
__all__ = ["WSGITransport"]
|
||||
|
||||
|
||||
def _skip_leading_empty_chunks(body: typing.Iterable[_T]) -> typing.Iterable[_T]:
|
||||
body = iter(body)
|
||||
for chunk in body:
|
||||
if chunk:
|
||||
return itertools.chain([chunk], body)
|
||||
return []
|
||||
|
||||
|
||||
class WSGIByteStream(SyncByteStream):
|
||||
def __init__(self, result: typing.Iterable[bytes]) -> None:
|
||||
self._close = getattr(result, "close", None)
|
||||
self._result = _skip_leading_empty_chunks(result)
|
||||
|
||||
def __iter__(self) -> typing.Iterator[bytes]:
|
||||
for part in self._result:
|
||||
yield part
|
||||
|
||||
def close(self) -> None:
|
||||
if self._close is not None:
|
||||
self._close()
|
||||
|
||||
|
||||
class WSGITransport(BaseTransport):
|
||||
"""
|
||||
A custom transport that handles sending requests directly to an WSGI app.
|
||||
The simplest way to use this functionality is to use the `app` argument.
|
||||
|
||||
```
|
||||
client = httpx.Client(app=app)
|
||||
```
|
||||
|
||||
Alternatively, you can setup the transport instance explicitly.
|
||||
This allows you to include any additional configuration arguments specific
|
||||
to the WSGITransport class:
|
||||
|
||||
```
|
||||
transport = httpx.WSGITransport(
|
||||
app=app,
|
||||
script_name="/submount",
|
||||
remote_addr="1.2.3.4"
|
||||
)
|
||||
client = httpx.Client(transport=transport)
|
||||
```
|
||||
|
||||
Arguments:
|
||||
|
||||
* `app` - The WSGI application.
|
||||
* `raise_app_exceptions` - Boolean indicating if exceptions in the application
|
||||
should be raised. Default to `True`. Can be set to `False` for use cases
|
||||
such as testing the content of a client 500 response.
|
||||
* `script_name` - The root path on which the WSGI application should be mounted.
|
||||
* `remote_addr` - A string indicating the client IP of incoming requests.
|
||||
```
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
app: WSGIApplication,
|
||||
raise_app_exceptions: bool = True,
|
||||
script_name: str = "",
|
||||
remote_addr: str = "127.0.0.1",
|
||||
wsgi_errors: typing.TextIO | None = None,
|
||||
) -> None:
|
||||
self.app = app
|
||||
self.raise_app_exceptions = raise_app_exceptions
|
||||
self.script_name = script_name
|
||||
self.remote_addr = remote_addr
|
||||
self.wsgi_errors = wsgi_errors
|
||||
|
||||
def handle_request(self, request: Request) -> Response:
|
||||
request.read()
|
||||
wsgi_input = io.BytesIO(request.content)
|
||||
|
||||
port = request.url.port or {"http": 80, "https": 443}[request.url.scheme]
|
||||
environ = {
|
||||
"wsgi.version": (1, 0),
|
||||
"wsgi.url_scheme": request.url.scheme,
|
||||
"wsgi.input": wsgi_input,
|
||||
"wsgi.errors": self.wsgi_errors or sys.stderr,
|
||||
"wsgi.multithread": True,
|
||||
"wsgi.multiprocess": False,
|
||||
"wsgi.run_once": False,
|
||||
"REQUEST_METHOD": request.method,
|
||||
"SCRIPT_NAME": self.script_name,
|
||||
"PATH_INFO": request.url.path,
|
||||
"QUERY_STRING": request.url.query.decode("ascii"),
|
||||
"SERVER_NAME": request.url.host,
|
||||
"SERVER_PORT": str(port),
|
||||
"SERVER_PROTOCOL": "HTTP/1.1",
|
||||
"REMOTE_ADDR": self.remote_addr,
|
||||
}
|
||||
for header_key, header_value in request.headers.raw:
|
||||
key = header_key.decode("ascii").upper().replace("-", "_")
|
||||
if key not in ("CONTENT_TYPE", "CONTENT_LENGTH"):
|
||||
key = "HTTP_" + key
|
||||
environ[key] = header_value.decode("ascii")
|
||||
|
||||
seen_status = None
|
||||
seen_response_headers = None
|
||||
seen_exc_info = None
|
||||
|
||||
def start_response(
|
||||
status: str,
|
||||
response_headers: list[tuple[str, str]],
|
||||
exc_info: OptExcInfo | None = None,
|
||||
) -> typing.Callable[[bytes], typing.Any]:
|
||||
nonlocal seen_status, seen_response_headers, seen_exc_info
|
||||
seen_status = status
|
||||
seen_response_headers = response_headers
|
||||
seen_exc_info = exc_info
|
||||
return lambda _: None
|
||||
|
||||
result = self.app(environ, start_response)
|
||||
|
||||
stream = WSGIByteStream(result)
|
||||
|
||||
assert seen_status is not None
|
||||
assert seen_response_headers is not None
|
||||
if seen_exc_info and seen_exc_info[0] and self.raise_app_exceptions:
|
||||
raise seen_exc_info[1]
|
||||
|
||||
status_code = int(seen_status.split()[0])
|
||||
headers = [
|
||||
(key.encode("ascii"), value.encode("ascii"))
|
||||
for key, value in seen_response_headers
|
||||
]
|
||||
|
||||
return Response(status_code, headers=headers, stream=stream)
|
||||
114
httpx/_types.py
114
httpx/_types.py
@ -1,114 +0,0 @@
|
||||
"""
|
||||
Type definitions for type checking purposes.
|
||||
"""
|
||||
|
||||
from http.cookiejar import CookieJar
|
||||
from typing import (
|
||||
IO,
|
||||
TYPE_CHECKING,
|
||||
Any,
|
||||
AsyncIterable,
|
||||
AsyncIterator,
|
||||
Callable,
|
||||
Dict,
|
||||
Iterable,
|
||||
Iterator,
|
||||
List,
|
||||
Mapping,
|
||||
Optional,
|
||||
Sequence,
|
||||
Tuple,
|
||||
Union,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover
|
||||
from ._auth import Auth # noqa: F401
|
||||
from ._config import Proxy, Timeout # noqa: F401
|
||||
from ._models import Cookies, Headers, Request # noqa: F401
|
||||
from ._urls import URL, QueryParams # noqa: F401
|
||||
|
||||
|
||||
PrimitiveData = Optional[Union[str, int, float, bool]]
|
||||
|
||||
URLTypes = Union["URL", str]
|
||||
|
||||
QueryParamTypes = Union[
|
||||
"QueryParams",
|
||||
Mapping[str, Union[PrimitiveData, Sequence[PrimitiveData]]],
|
||||
List[Tuple[str, PrimitiveData]],
|
||||
Tuple[Tuple[str, PrimitiveData], ...],
|
||||
str,
|
||||
bytes,
|
||||
]
|
||||
|
||||
HeaderTypes = Union[
|
||||
"Headers",
|
||||
Mapping[str, str],
|
||||
Mapping[bytes, bytes],
|
||||
Sequence[Tuple[str, str]],
|
||||
Sequence[Tuple[bytes, bytes]],
|
||||
]
|
||||
|
||||
CookieTypes = Union["Cookies", CookieJar, Dict[str, str], List[Tuple[str, str]]]
|
||||
|
||||
TimeoutTypes = Union[
|
||||
Optional[float],
|
||||
Tuple[Optional[float], Optional[float], Optional[float], Optional[float]],
|
||||
"Timeout",
|
||||
]
|
||||
ProxyTypes = Union["URL", str, "Proxy"]
|
||||
CertTypes = Union[str, Tuple[str, str], Tuple[str, str, str]]
|
||||
|
||||
AuthTypes = Union[
|
||||
Tuple[Union[str, bytes], Union[str, bytes]],
|
||||
Callable[["Request"], "Request"],
|
||||
"Auth",
|
||||
]
|
||||
|
||||
RequestContent = Union[str, bytes, Iterable[bytes], AsyncIterable[bytes]]
|
||||
ResponseContent = Union[str, bytes, Iterable[bytes], AsyncIterable[bytes]]
|
||||
ResponseExtensions = Mapping[str, Any]
|
||||
|
||||
RequestData = Mapping[str, Any]
|
||||
|
||||
FileContent = Union[IO[bytes], bytes, str]
|
||||
FileTypes = Union[
|
||||
# file (or bytes)
|
||||
FileContent,
|
||||
# (filename, file (or bytes))
|
||||
Tuple[Optional[str], FileContent],
|
||||
# (filename, file (or bytes), content_type)
|
||||
Tuple[Optional[str], FileContent, Optional[str]],
|
||||
# (filename, file (or bytes), content_type, headers)
|
||||
Tuple[Optional[str], FileContent, Optional[str], Mapping[str, str]],
|
||||
]
|
||||
RequestFiles = Union[Mapping[str, FileTypes], Sequence[Tuple[str, FileTypes]]]
|
||||
|
||||
RequestExtensions = Mapping[str, Any]
|
||||
|
||||
__all__ = ["AsyncByteStream", "SyncByteStream"]
|
||||
|
||||
|
||||
class SyncByteStream:
|
||||
def __iter__(self) -> Iterator[bytes]:
|
||||
raise NotImplementedError(
|
||||
"The '__iter__' method must be implemented."
|
||||
) # pragma: no cover
|
||||
yield b"" # pragma: no cover
|
||||
|
||||
def close(self) -> None:
|
||||
"""
|
||||
Subclasses can override this method to release any network resources
|
||||
after a request/response cycle is complete.
|
||||
"""
|
||||
|
||||
|
||||
class AsyncByteStream:
|
||||
async def __aiter__(self) -> AsyncIterator[bytes]:
|
||||
raise NotImplementedError(
|
||||
"The '__aiter__' method must be implemented."
|
||||
) # pragma: no cover
|
||||
yield b"" # pragma: no cover
|
||||
|
||||
async def aclose(self) -> None:
|
||||
pass
|
||||
242
httpx/_utils.py
242
httpx/_utils.py
@ -1,242 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import os
|
||||
import re
|
||||
import typing
|
||||
from urllib.request import getproxies
|
||||
|
||||
from ._types import PrimitiveData
|
||||
|
||||
if typing.TYPE_CHECKING: # pragma: no cover
|
||||
from ._urls import URL
|
||||
|
||||
|
||||
def primitive_value_to_str(value: PrimitiveData) -> str:
|
||||
"""
|
||||
Coerce a primitive data type into a string value.
|
||||
|
||||
Note that we prefer JSON-style 'true'/'false' for boolean values here.
|
||||
"""
|
||||
if value is True:
|
||||
return "true"
|
||||
elif value is False:
|
||||
return "false"
|
||||
elif value is None:
|
||||
return ""
|
||||
return str(value)
|
||||
|
||||
|
||||
def get_environment_proxies() -> dict[str, str | None]:
|
||||
"""Gets proxy information from the environment"""
|
||||
|
||||
# urllib.request.getproxies() falls back on System
|
||||
# Registry and Config for proxies on Windows and macOS.
|
||||
# We don't want to propagate non-HTTP proxies into
|
||||
# our configuration such as 'TRAVIS_APT_PROXY'.
|
||||
proxy_info = getproxies()
|
||||
mounts: dict[str, str | None] = {}
|
||||
|
||||
for scheme in ("http", "https", "all"):
|
||||
if proxy_info.get(scheme):
|
||||
hostname = proxy_info[scheme]
|
||||
mounts[f"{scheme}://"] = (
|
||||
hostname if "://" in hostname else f"http://{hostname}"
|
||||
)
|
||||
|
||||
no_proxy_hosts = [host.strip() for host in proxy_info.get("no", "").split(",")]
|
||||
for hostname in no_proxy_hosts:
|
||||
# See https://curl.haxx.se/libcurl/c/CURLOPT_NOPROXY.html for details
|
||||
# on how names in `NO_PROXY` are handled.
|
||||
if hostname == "*":
|
||||
# If NO_PROXY=* is used or if "*" occurs as any one of the comma
|
||||
# separated hostnames, then we should just bypass any information
|
||||
# from HTTP_PROXY, HTTPS_PROXY, ALL_PROXY, and always ignore
|
||||
# proxies.
|
||||
return {}
|
||||
elif hostname:
|
||||
# NO_PROXY=.google.com is marked as "all://*.google.com,
|
||||
# which disables "www.google.com" but not "google.com"
|
||||
# NO_PROXY=google.com is marked as "all://*google.com,
|
||||
# which disables "www.google.com" and "google.com".
|
||||
# (But not "wwwgoogle.com")
|
||||
# NO_PROXY can include domains, IPv6, IPv4 addresses and "localhost"
|
||||
# NO_PROXY=example.com,::1,localhost,192.168.0.0/16
|
||||
if "://" in hostname:
|
||||
mounts[hostname] = None
|
||||
elif is_ipv4_hostname(hostname):
|
||||
mounts[f"all://{hostname}"] = None
|
||||
elif is_ipv6_hostname(hostname):
|
||||
mounts[f"all://[{hostname}]"] = None
|
||||
elif hostname.lower() == "localhost":
|
||||
mounts[f"all://{hostname}"] = None
|
||||
else:
|
||||
mounts[f"all://*{hostname}"] = None
|
||||
|
||||
return mounts
|
||||
|
||||
|
||||
def to_bytes(value: str | bytes, encoding: str = "utf-8") -> bytes:
|
||||
return value.encode(encoding) if isinstance(value, str) else value
|
||||
|
||||
|
||||
def to_str(value: str | bytes, encoding: str = "utf-8") -> str:
|
||||
return value if isinstance(value, str) else value.decode(encoding)
|
||||
|
||||
|
||||
def to_bytes_or_str(value: str, match_type_of: typing.AnyStr) -> typing.AnyStr:
|
||||
return value if isinstance(match_type_of, str) else value.encode()
|
||||
|
||||
|
||||
def unquote(value: str) -> str:
|
||||
return value[1:-1] if value[0] == value[-1] == '"' else value
|
||||
|
||||
|
||||
def peek_filelike_length(stream: typing.Any) -> int | None:
|
||||
"""
|
||||
Given a file-like stream object, return its length in number of bytes
|
||||
without reading it into memory.
|
||||
"""
|
||||
try:
|
||||
# Is it an actual file?
|
||||
fd = stream.fileno()
|
||||
# Yup, seems to be an actual file.
|
||||
length = os.fstat(fd).st_size
|
||||
except (AttributeError, OSError):
|
||||
# No... Maybe it's something that supports random access, like `io.BytesIO`?
|
||||
try:
|
||||
# Assuming so, go to end of stream to figure out its length,
|
||||
# then put it back in place.
|
||||
offset = stream.tell()
|
||||
length = stream.seek(0, os.SEEK_END)
|
||||
stream.seek(offset)
|
||||
except (AttributeError, OSError):
|
||||
# Not even that? Sorry, we're doomed...
|
||||
return None
|
||||
|
||||
return length
|
||||
|
||||
|
||||
class URLPattern:
|
||||
"""
|
||||
A utility class currently used for making lookups against proxy keys...
|
||||
|
||||
# Wildcard matching...
|
||||
>>> pattern = URLPattern("all://")
|
||||
>>> pattern.matches(httpx.URL("http://example.com"))
|
||||
True
|
||||
|
||||
# Witch scheme matching...
|
||||
>>> pattern = URLPattern("https://")
|
||||
>>> pattern.matches(httpx.URL("https://example.com"))
|
||||
True
|
||||
>>> pattern.matches(httpx.URL("http://example.com"))
|
||||
False
|
||||
|
||||
# With domain matching...
|
||||
>>> pattern = URLPattern("https://example.com")
|
||||
>>> pattern.matches(httpx.URL("https://example.com"))
|
||||
True
|
||||
>>> pattern.matches(httpx.URL("http://example.com"))
|
||||
False
|
||||
>>> pattern.matches(httpx.URL("https://other.com"))
|
||||
False
|
||||
|
||||
# Wildcard scheme, with domain matching...
|
||||
>>> pattern = URLPattern("all://example.com")
|
||||
>>> pattern.matches(httpx.URL("https://example.com"))
|
||||
True
|
||||
>>> pattern.matches(httpx.URL("http://example.com"))
|
||||
True
|
||||
>>> pattern.matches(httpx.URL("https://other.com"))
|
||||
False
|
||||
|
||||
# With port matching...
|
||||
>>> pattern = URLPattern("https://example.com:1234")
|
||||
>>> pattern.matches(httpx.URL("https://example.com:1234"))
|
||||
True
|
||||
>>> pattern.matches(httpx.URL("https://example.com"))
|
||||
False
|
||||
"""
|
||||
|
||||
def __init__(self, pattern: str) -> None:
|
||||
from ._urls import URL
|
||||
|
||||
if pattern and ":" not in pattern:
|
||||
raise ValueError(
|
||||
f"Proxy keys should use proper URL forms rather "
|
||||
f"than plain scheme strings. "
|
||||
f'Instead of "{pattern}", use "{pattern}://"'
|
||||
)
|
||||
|
||||
url = URL(pattern)
|
||||
self.pattern = pattern
|
||||
self.scheme = "" if url.scheme == "all" else url.scheme
|
||||
self.host = "" if url.host == "*" else url.host
|
||||
self.port = url.port
|
||||
if not url.host or url.host == "*":
|
||||
self.host_regex: typing.Pattern[str] | None = None
|
||||
elif url.host.startswith("*."):
|
||||
# *.example.com should match "www.example.com", but not "example.com"
|
||||
domain = re.escape(url.host[2:])
|
||||
self.host_regex = re.compile(f"^.+\\.{domain}$")
|
||||
elif url.host.startswith("*"):
|
||||
# *example.com should match "www.example.com" and "example.com"
|
||||
domain = re.escape(url.host[1:])
|
||||
self.host_regex = re.compile(f"^(.+\\.)?{domain}$")
|
||||
else:
|
||||
# example.com should match "example.com" but not "www.example.com"
|
||||
domain = re.escape(url.host)
|
||||
self.host_regex = re.compile(f"^{domain}$")
|
||||
|
||||
def matches(self, other: URL) -> bool:
|
||||
if self.scheme and self.scheme != other.scheme:
|
||||
return False
|
||||
if (
|
||||
self.host
|
||||
and self.host_regex is not None
|
||||
and not self.host_regex.match(other.host)
|
||||
):
|
||||
return False
|
||||
if self.port is not None and self.port != other.port:
|
||||
return False
|
||||
return True
|
||||
|
||||
@property
|
||||
def priority(self) -> tuple[int, int, int]:
|
||||
"""
|
||||
The priority allows URLPattern instances to be sortable, so that
|
||||
we can match from most specific to least specific.
|
||||
"""
|
||||
# URLs with a port should take priority over URLs without a port.
|
||||
port_priority = 0 if self.port is not None else 1
|
||||
# Longer hostnames should match first.
|
||||
host_priority = -len(self.host)
|
||||
# Longer schemes should match first.
|
||||
scheme_priority = -len(self.scheme)
|
||||
return (port_priority, host_priority, scheme_priority)
|
||||
|
||||
def __hash__(self) -> int:
|
||||
return hash(self.pattern)
|
||||
|
||||
def __lt__(self, other: URLPattern) -> bool:
|
||||
return self.priority < other.priority
|
||||
|
||||
def __eq__(self, other: typing.Any) -> bool:
|
||||
return isinstance(other, URLPattern) and self.pattern == other.pattern
|
||||
|
||||
|
||||
def is_ipv4_hostname(hostname: str) -> bool:
|
||||
try:
|
||||
ipaddress.IPv4Address(hostname.split("/")[0])
|
||||
except Exception:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def is_ipv6_hostname(hostname: str) -> bool:
|
||||
try:
|
||||
ipaddress.IPv6Address(hostname.split("/")[0])
|
||||
except Exception:
|
||||
return False
|
||||
return True
|
||||
61
mkdocs.yml
61
mkdocs.yml
@ -1,61 +0,0 @@
|
||||
site_name: HTTPX
|
||||
site_description: A next-generation HTTP client for Python.
|
||||
site_url: https://www.python-httpx.org/
|
||||
|
||||
theme:
|
||||
name: 'material'
|
||||
custom_dir: 'docs/overrides'
|
||||
palette:
|
||||
- scheme: 'default'
|
||||
media: '(prefers-color-scheme: light)'
|
||||
toggle:
|
||||
icon: 'material/lightbulb'
|
||||
name: "Switch to dark mode"
|
||||
- scheme: 'slate'
|
||||
media: '(prefers-color-scheme: dark)'
|
||||
primary: 'blue'
|
||||
toggle:
|
||||
icon: 'material/lightbulb-outline'
|
||||
name: 'Switch to light mode'
|
||||
|
||||
repo_name: encode/httpx
|
||||
repo_url: https://github.com/encode/httpx/
|
||||
edit_uri: ""
|
||||
|
||||
nav:
|
||||
- Introduction: 'index.md'
|
||||
- QuickStart: 'quickstart.md'
|
||||
- Advanced:
|
||||
- Clients: 'advanced/clients.md'
|
||||
- Authentication: 'advanced/authentication.md'
|
||||
- SSL: 'advanced/ssl.md'
|
||||
- Proxies: 'advanced/proxies.md'
|
||||
- Timeouts: 'advanced/timeouts.md'
|
||||
- Resource Limits: 'advanced/resource-limits.md'
|
||||
- Event Hooks: 'advanced/event-hooks.md'
|
||||
- Transports: 'advanced/transports.md'
|
||||
- Text Encodings: 'advanced/text-encodings.md'
|
||||
- Extensions: 'advanced/extensions.md'
|
||||
- Guides:
|
||||
- Async Support: 'async.md'
|
||||
- HTTP/2 Support: 'http2.md'
|
||||
- Logging: 'logging.md'
|
||||
- Requests Compatibility: 'compatibility.md'
|
||||
- Troubleshooting: 'troubleshooting.md'
|
||||
- API Reference:
|
||||
- Developer Interface: 'api.md'
|
||||
- Exceptions: 'exceptions.md'
|
||||
- Environment Variables: 'environment_variables.md'
|
||||
- Community:
|
||||
- Third Party Packages: 'third_party_packages.md'
|
||||
- Contributing: 'contributing.md'
|
||||
- Code of Conduct: 'code_of_conduct.md'
|
||||
|
||||
markdown_extensions:
|
||||
- admonition
|
||||
- codehilite:
|
||||
css_class: highlight
|
||||
- mkautodoc
|
||||
|
||||
extra_css:
|
||||
- css/custom.css
|
||||
113
pyproject.toml
113
pyproject.toml
@ -1,26 +1,21 @@
|
||||
[build-system]
|
||||
requires = ["hatchling", "hatch-fancy-pypi-readme"]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "httpx"
|
||||
description = "The next generation HTTP client."
|
||||
license = "BSD-3-Clause"
|
||||
requires-python = ">=3.9"
|
||||
description = "HTTP, for Python."
|
||||
version = "1.0.dev3"
|
||||
requires-python = ">=3.11"
|
||||
authors = [
|
||||
{ name = "Tom Christie", email = "tom@tomchristie.com" },
|
||||
]
|
||||
classifiers = [
|
||||
"Development Status :: 4 - Beta",
|
||||
"Environment :: Web Environment",
|
||||
"Framework :: AsyncIO",
|
||||
"Framework :: Trio",
|
||||
"Intended Audience :: Developers",
|
||||
"License :: OSI Approved :: BSD License",
|
||||
"Operating System :: OS Independent",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3 :: Only",
|
||||
"Programming Language :: Python :: 3.9",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
@ -29,104 +24,4 @@ classifiers = [
|
||||
]
|
||||
dependencies = [
|
||||
"certifi",
|
||||
"httpcore==1.*",
|
||||
"anyio",
|
||||
"idna",
|
||||
]
|
||||
dynamic = ["readme", "version"]
|
||||
|
||||
[project.optional-dependencies]
|
||||
brotli = [
|
||||
"brotli; platform_python_implementation == 'CPython'",
|
||||
"brotlicffi; platform_python_implementation != 'CPython'",
|
||||
]
|
||||
cli = [
|
||||
"click==8.*",
|
||||
"pygments==2.*",
|
||||
"rich>=10,<15",
|
||||
]
|
||||
http2 = [
|
||||
"h2>=3,<5",
|
||||
]
|
||||
socks = [
|
||||
"socksio==1.*",
|
||||
]
|
||||
zstd = [
|
||||
"zstandard>=0.18.0",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
httpx = "httpx:main"
|
||||
|
||||
[project.urls]
|
||||
Changelog = "https://github.com/encode/httpx/blob/master/CHANGELOG.md"
|
||||
Documentation = "https://www.python-httpx.org"
|
||||
Homepage = "https://github.com/encode/httpx"
|
||||
Source = "https://github.com/encode/httpx"
|
||||
|
||||
[tool.hatch.version]
|
||||
path = "httpx/__version__.py"
|
||||
|
||||
[tool.hatch.build.targets.sdist]
|
||||
include = [
|
||||
"/httpx",
|
||||
"/CHANGELOG.md",
|
||||
"/README.md",
|
||||
"/tests",
|
||||
]
|
||||
|
||||
[tool.hatch.metadata.hooks.fancy-pypi-readme]
|
||||
content-type = "text/markdown"
|
||||
|
||||
[[tool.hatch.metadata.hooks.fancy-pypi-readme.fragments]]
|
||||
path = "README.md"
|
||||
|
||||
[[tool.hatch.metadata.hooks.fancy-pypi-readme.fragments]]
|
||||
text = "\n## Release Information\n\n"
|
||||
|
||||
[[tool.hatch.metadata.hooks.fancy-pypi-readme.fragments]]
|
||||
path = "CHANGELOG.md"
|
||||
pattern = "\n(###.+?\n)## "
|
||||
|
||||
[[tool.hatch.metadata.hooks.fancy-pypi-readme.fragments]]
|
||||
text = "\n---\n\n[Full changelog](https://github.com/encode/httpx/blob/master/CHANGELOG.md)\n"
|
||||
|
||||
[[tool.hatch.metadata.hooks.fancy-pypi-readme.substitutions]]
|
||||
pattern = 'src="(docs/img/.*?)"'
|
||||
replacement = 'src="https://raw.githubusercontent.com/encode/httpx/master/\1"'
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "I", "B", "PIE"]
|
||||
ignore = ["B904", "B028"]
|
||||
|
||||
[tool.ruff.lint.isort]
|
||||
combine-as-imports = true
|
||||
|
||||
[tool.ruff.lint.per-file-ignores]
|
||||
"__init__.py" = ["F403", "F405"]
|
||||
|
||||
[tool.mypy]
|
||||
ignore_missing_imports = true
|
||||
strict = true
|
||||
|
||||
[[tool.mypy.overrides]]
|
||||
module = "tests.*"
|
||||
disallow_untyped_defs = false
|
||||
check_untyped_defs = true
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
addopts = "-rxXs"
|
||||
filterwarnings = [
|
||||
"error",
|
||||
"ignore: You seem to already have a custom sys.excepthook handler installed. I'll skip installing Trio's custom handler, but this means MultiErrors will not show full tracebacks.:RuntimeWarning",
|
||||
# See: https://github.com/agronholm/anyio/issues/508
|
||||
"ignore: trio.MultiError is deprecated since Trio 0.22.0:trio.TrioDeprecationWarning"
|
||||
]
|
||||
markers = [
|
||||
"copied_from(source, changes=None): mark test as copied from somewhere else, along with a description of changes made to accodomate e.g. our test setup",
|
||||
"network: marks tests which require network connection. Used in 3rd-party build environments that have network disabled."
|
||||
]
|
||||
|
||||
[tool.coverage.run]
|
||||
omit = ["venv/*"]
|
||||
include = ["httpx/*", "tests/*"]
|
||||
|
||||
@ -1,29 +1,17 @@
|
||||
# We're pinning our tooling, because it's an environment we can strictly control.
|
||||
# On the other hand, we're not pinning package dependencies, because our tests
|
||||
# needs to pass with the latest version of the packages.
|
||||
# Reference: https://github.com/encode/httpx/pull/1721#discussion_r661241588
|
||||
-e .[brotli,cli,http2,socks,zstd]
|
||||
-e .
|
||||
|
||||
# Optional charset auto-detection
|
||||
# Used in our test cases
|
||||
chardet==5.2.0
|
||||
# Build...
|
||||
build==1.2.2
|
||||
|
||||
# Documentation
|
||||
mkdocs==1.6.1
|
||||
mkautodoc==0.2.0
|
||||
mkdocs-material==9.6.18
|
||||
# Test...
|
||||
mypy==1.15.0
|
||||
pytest==8.3.5
|
||||
pytest-cov==6.1.1
|
||||
|
||||
# Packaging
|
||||
build==1.3.0
|
||||
twine==6.1.0
|
||||
# Sync & Async mirroring...
|
||||
unasync==0.6.0
|
||||
|
||||
# Tests & Linting
|
||||
coverage[toml]==7.10.6
|
||||
cryptography==45.0.7
|
||||
mypy==1.17.1
|
||||
pytest==8.4.1
|
||||
ruff==0.12.11
|
||||
trio==0.31.0
|
||||
trio-typing==0.10.0
|
||||
trustme==1.2.1
|
||||
uvicorn==0.35.0
|
||||
# Documentation...
|
||||
click==8.2.1
|
||||
jinja2==3.1.6
|
||||
markdown==3.8
|
||||
|
||||
@ -1,13 +1,32 @@
|
||||
#!/bin/sh -e
|
||||
#!/bin/sh
|
||||
|
||||
if [ -d 'venv' ] ; then
|
||||
PREFIX="venv/bin/"
|
||||
else
|
||||
PREFIX=""
|
||||
PKG=$1
|
||||
|
||||
if [ "$PKG" != "httpx" ] && [ "$PKG" != "ahttpx" ] ; then
|
||||
echo "build [httpx|ahttpx]"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
set -x
|
||||
export PREFIX=""
|
||||
if [ -d 'venv' ] ; then
|
||||
export PREFIX="venv/bin/"
|
||||
fi
|
||||
|
||||
# Create pyproject-httpx.toml and pyproject-ahttpx.toml
|
||||
cp pyproject.toml pyproject-httpx.toml
|
||||
cat pyproject-httpx.toml | sed 's/name = "httpx"/name = "ahttpx"/' > pyproject-ahttpx.toml
|
||||
|
||||
# Build the releases
|
||||
if [ "$PKG" == "httpx" ]; then
|
||||
${PREFIX}python -m build
|
||||
${PREFIX}twine check dist/*
|
||||
${PREFIX}mkdocs build
|
||||
fi
|
||||
if [ "$PKG" == "ahttpx" ]; then
|
||||
cp pyproject-ahttpx.toml pyproject.toml
|
||||
${PREFIX}python -m build
|
||||
cp pyproject-httpx.toml pyproject.toml
|
||||
fi
|
||||
|
||||
# Clean up
|
||||
rm pyproject-httpx.toml pyproject-ahttpx.toml
|
||||
|
||||
echo $PKG
|
||||
@ -1,14 +0,0 @@
|
||||
#!/bin/sh -e
|
||||
|
||||
export PREFIX=""
|
||||
if [ -d 'venv' ] ; then
|
||||
export PREFIX="venv/bin/"
|
||||
fi
|
||||
export SOURCE_FILES="httpx tests"
|
||||
|
||||
set -x
|
||||
|
||||
./scripts/sync-version
|
||||
${PREFIX}ruff format $SOURCE_FILES --diff
|
||||
${PREFIX}mypy $SOURCE_FILES
|
||||
${PREFIX}ruff check $SOURCE_FILES
|
||||
@ -1,14 +0,0 @@
|
||||
#!/bin/sh -e
|
||||
|
||||
if [ -d 'dist' ] ; then
|
||||
rm -r dist
|
||||
fi
|
||||
if [ -d 'site' ] ; then
|
||||
rm -r site
|
||||
fi
|
||||
if [ -d 'htmlcov' ] ; then
|
||||
rm -r htmlcov
|
||||
fi
|
||||
if [ -d 'httpx.egg-info' ] ; then
|
||||
rm -r httpx.egg-info
|
||||
fi
|
||||
@ -1,11 +0,0 @@
|
||||
#!/bin/sh -e
|
||||
|
||||
export PREFIX=""
|
||||
if [ -d 'venv' ] ; then
|
||||
export PREFIX="venv/bin/"
|
||||
fi
|
||||
export SOURCE_FILES="httpx tests"
|
||||
|
||||
set -x
|
||||
|
||||
${PREFIX}coverage report --show-missing --skip-covered --fail-under=100
|
||||
157
scripts/docs
157
scripts/docs
@ -1,10 +1,153 @@
|
||||
#!/bin/sh -e
|
||||
#!venv/bin/python
|
||||
import pathlib
|
||||
import posixpath
|
||||
|
||||
export PREFIX=""
|
||||
if [ -d 'venv' ] ; then
|
||||
export PREFIX="venv/bin/"
|
||||
fi
|
||||
import click
|
||||
import ghp_import
|
||||
import logging
|
||||
import httpx
|
||||
import jinja2
|
||||
import markdown
|
||||
|
||||
set -x
|
||||
import xml.etree.ElementTree as etree
|
||||
|
||||
${PREFIX}mkdocs serve
|
||||
|
||||
pages = {
|
||||
'/': 'docs/index.md',
|
||||
'/quickstart': 'docs/quickstart.md',
|
||||
'/clients': 'docs/clients.md',
|
||||
'/servers': 'docs/servers.md',
|
||||
'/requests': 'docs/requests.md',
|
||||
'/responses': 'docs/responses.md',
|
||||
'/urls': 'docs/urls.md',
|
||||
'/headers': 'docs/headers.md',
|
||||
'/content-types': 'docs/content-types.md',
|
||||
'/streams': 'docs/streams.md',
|
||||
'/connections': 'docs/connections.md',
|
||||
'/parsers': 'docs/parsers.md',
|
||||
'/networking': 'docs/networking.md',
|
||||
'/about': 'docs/about.md',
|
||||
}
|
||||
|
||||
def path_to_url(path):
|
||||
if path == "index.md":
|
||||
return "/"
|
||||
return f"/{path[:-3]}"
|
||||
|
||||
|
||||
class URLsProcessor(markdown.treeprocessors.Treeprocessor):
|
||||
def __init__(self, state):
|
||||
self.state = state
|
||||
|
||||
def run(self, root: etree.Element) -> etree.Element:
|
||||
for element in root.iter():
|
||||
if element.tag == 'a':
|
||||
key = 'href'
|
||||
elif element.tag == 'img':
|
||||
key = 'src'
|
||||
else:
|
||||
continue
|
||||
|
||||
url_or_path = element.get(key)
|
||||
if url_or_path is not None:
|
||||
output_url = self.rewrite_url(url_or_path)
|
||||
element.set(key, output_url)
|
||||
|
||||
return root
|
||||
|
||||
def rewrite_url(self, href: str) -> str:
|
||||
if not href.endswith('.md'):
|
||||
return href
|
||||
|
||||
current_url = path_to_url(self.state.file)
|
||||
linked_url = path_to_url(href)
|
||||
return posixpath.relpath(linked_url, start=current_url)
|
||||
|
||||
|
||||
class BuildState:
|
||||
def __init__(self):
|
||||
self.file = ''
|
||||
|
||||
|
||||
state = BuildState()
|
||||
env = jinja2.Environment(
|
||||
loader=jinja2.FileSystemLoader('docs/templates'),
|
||||
autoescape=False
|
||||
)
|
||||
template = env.get_template('base.html')
|
||||
md = markdown.Markdown(extensions=['fenced_code'])
|
||||
md.treeprocessors.register(
|
||||
item=URLsProcessor(state),
|
||||
name='urls',
|
||||
priority=10,
|
||||
)
|
||||
|
||||
|
||||
def not_found():
|
||||
text = httpx.Text('Not Found')
|
||||
return httpx.Response(404, content=text)
|
||||
|
||||
|
||||
def web_server(request):
|
||||
if request.url.path not in pages:
|
||||
return not_found()
|
||||
|
||||
file = pages[request.url.path]
|
||||
text = pathlib.Path(file).read_text()
|
||||
|
||||
state.file = file
|
||||
content = md.convert(text)
|
||||
html = template.render(content=content).encode('utf-8')
|
||||
content = httpx.HTML(html)
|
||||
return httpx.Response(200, content=html)
|
||||
|
||||
|
||||
@click.group()
|
||||
def main():
|
||||
pass
|
||||
|
||||
|
||||
@main.command()
|
||||
def build():
|
||||
pathlib.Path("build").mkdir(exist_ok=True)
|
||||
|
||||
for url, path in pages.items():
|
||||
basename = url.lstrip("/")
|
||||
output = f"build/{basename}.html" if basename else "build/index.html"
|
||||
text = pathlib.Path(path).read_text()
|
||||
content = md.convert(text)
|
||||
html = template.render(content=content)
|
||||
pathlib.Path(output).write_text(html)
|
||||
print(f"Built {output}")
|
||||
|
||||
|
||||
@main.command()
|
||||
def serve():
|
||||
logging.basicConfig(
|
||||
format="%(levelname)s [%(asctime)s] %(name)s - %(message)s",
|
||||
datefmt="%Y-%m-%d %H:%M:%S",
|
||||
level=logging.INFO
|
||||
)
|
||||
|
||||
with httpx.serve_http(web_server) as server:
|
||||
server.wait()
|
||||
|
||||
|
||||
@main.command()
|
||||
def deploy():
|
||||
ghp_import.ghp_import(
|
||||
"build",
|
||||
mesg="Documentation deploy",
|
||||
remote="origin",
|
||||
branch="gh-pages",
|
||||
push=True,
|
||||
force=False,
|
||||
use_shell=False,
|
||||
no_history=False,
|
||||
nojekyll=True,
|
||||
)
|
||||
print(f"Deployed to GitHub")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
@ -1,19 +1,13 @@
|
||||
#!/bin/sh -e
|
||||
|
||||
# Use the Python executable provided from the `-p` option, or a default.
|
||||
[ "$1" = "-p" ] && PYTHON=$2 || PYTHON="python3"
|
||||
|
||||
REQUIREMENTS="requirements.txt"
|
||||
VENV="venv"
|
||||
#!/bin/sh
|
||||
|
||||
set -x
|
||||
|
||||
if [ -z "$GITHUB_ACTIONS" ]; then
|
||||
"$PYTHON" -m venv "$VENV"
|
||||
PIP="$VENV/bin/pip"
|
||||
python3 -m venv venv
|
||||
PIP="venv/bin/pip"
|
||||
else
|
||||
PIP="pip"
|
||||
fi
|
||||
|
||||
"$PIP" install -U pip
|
||||
"$PIP" install -r "$REQUIREMENTS"
|
||||
"$PIP" install -r requirements.txt
|
||||
|
||||
12
scripts/lint
12
scripts/lint
@ -1,12 +0,0 @@
|
||||
#!/bin/sh -e
|
||||
|
||||
export PREFIX=""
|
||||
if [ -d 'venv' ]; then
|
||||
export PREFIX="venv/bin/"
|
||||
fi
|
||||
export SOURCE_FILES="httpx tests"
|
||||
|
||||
set -x
|
||||
|
||||
${PREFIX}ruff check --fix $SOURCE_FILES
|
||||
${PREFIX}ruff format $SOURCE_FILES
|
||||
@ -1,26 +1,15 @@
|
||||
#!/bin/sh -e
|
||||
#!/bin/sh
|
||||
|
||||
VERSION_FILE="httpx/__version__.py"
|
||||
PKG=$1
|
||||
|
||||
if [ -d 'venv' ] ; then
|
||||
PREFIX="venv/bin/"
|
||||
else
|
||||
PREFIX=""
|
||||
fi
|
||||
|
||||
if [ ! -z "$GITHUB_ACTIONS" ]; then
|
||||
git config --local user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
git config --local user.name "GitHub Action"
|
||||
|
||||
VERSION=`grep __version__ ${VERSION_FILE} | grep -o '[0-9][^"]*'`
|
||||
|
||||
if [ "refs/tags/${VERSION}" != "${GITHUB_REF}" ] ; then
|
||||
echo "GitHub Ref '${GITHUB_REF}' did not match package version '${VERSION}'"
|
||||
if [ "$PKG" != "httpx" ] && [ "$PKG" != "ahttpx" ] ; then
|
||||
echo "publish [httpx|ahttpx]"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
export PREFIX=""
|
||||
if [ -d 'venv' ] ; then
|
||||
export PREFIX="venv/bin/"
|
||||
fi
|
||||
|
||||
set -x
|
||||
|
||||
${PREFIX}twine upload dist/*
|
||||
${PREFIX}mkdocs gh-deploy --force
|
||||
${PREFIX}pip install -q twine
|
||||
${PREFIX}twine upload dist/$PKG-*
|
||||
|
||||
@ -1,11 +0,0 @@
|
||||
#!/bin/sh -e
|
||||
|
||||
SEMVER_REGEX="([0-9]+)\.([0-9]+)\.([0-9]+)(?:-([0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*))?(?:\+[0-9A-Za-z-]+)?"
|
||||
CHANGELOG_VERSION=$(grep -o -E $SEMVER_REGEX CHANGELOG.md | sed -n 2p)
|
||||
VERSION=$(grep -o -E $SEMVER_REGEX httpx/__version__.py | head -1)
|
||||
echo "CHANGELOG_VERSION: $CHANGELOG_VERSION"
|
||||
echo "VERSION: $VERSION"
|
||||
if [ "$CHANGELOG_VERSION" != "$VERSION" ]; then
|
||||
echo "Version in changelog does not match version in httpx/__version__.py!"
|
||||
exit 1
|
||||
fi
|
||||
14
scripts/test
14
scripts/test
@ -5,14 +5,6 @@ if [ -d 'venv' ] ; then
|
||||
export PREFIX="venv/bin/"
|
||||
fi
|
||||
|
||||
set -ex
|
||||
|
||||
if [ -z $GITHUB_ACTIONS ]; then
|
||||
scripts/check
|
||||
fi
|
||||
|
||||
${PREFIX}coverage run -m pytest "$@"
|
||||
|
||||
if [ -z $GITHUB_ACTIONS ]; then
|
||||
scripts/coverage
|
||||
fi
|
||||
${PREFIX}mypy src/httpx
|
||||
${PREFIX}mypy src/ahttpx
|
||||
${PREFIX}pytest --cov src/httpx tests
|
||||
|
||||
29
scripts/unasync
Executable file
29
scripts/unasync
Executable file
@ -0,0 +1,29 @@
|
||||
#!venv/bin/python
|
||||
import unasync
|
||||
|
||||
unasync.unasync_files(
|
||||
fpath_list = [
|
||||
"src/ahttpx/__init__.py",
|
||||
"src/ahttpx/__version__.py",
|
||||
"src/ahttpx/_client.py",
|
||||
"src/ahttpx/_content.py",
|
||||
"src/ahttpx/_headers.py",
|
||||
"src/ahttpx/_parsers.py",
|
||||
"src/ahttpx/_pool.py",
|
||||
"src/ahttpx/_quickstart.py",
|
||||
"src/ahttpx/_response.py",
|
||||
"src/ahttpx/_request.py",
|
||||
"src/ahttpx/_server.py",
|
||||
"src/ahttpx/_streams.py",
|
||||
"src/ahttpx/_urlencode.py",
|
||||
"src/ahttpx/_urlparse.py",
|
||||
"src/ahttpx/_urls.py",
|
||||
],
|
||||
rules = [
|
||||
unasync.Rule(
|
||||
"src/ahttpx/",
|
||||
"src/httpx/",
|
||||
additional_replacements={"ahttpx": "httpx"}
|
||||
),
|
||||
]
|
||||
)
|
||||
62
src/ahttpx/__init__.py
Normal file
62
src/ahttpx/__init__.py
Normal file
@ -0,0 +1,62 @@
|
||||
from ._client import * # Client
|
||||
from ._content import * # Content, File, Files, Form, HTML, JSON, MultiPart, Text
|
||||
from ._headers import * # Headers
|
||||
from ._network import * # NetworkBackend, NetworkStream, timeout
|
||||
from ._parsers import * # HTTPParser, ProtocolError
|
||||
from ._pool import * # Connection, ConnectionPool, Transport
|
||||
from ._quickstart import * # get, post, put, patch, delete
|
||||
from ._response import * # Response
|
||||
from ._request import * # Request
|
||||
from ._streams import * # ByteStream, DuplexStream, FileStream, HTTPStream, Stream
|
||||
from ._server import * # serve_http, run
|
||||
from ._urlencode import * # quote, unquote, urldecode, urlencode
|
||||
from ._urls import * # QueryParams, URL
|
||||
|
||||
|
||||
__all__ = [
|
||||
"ByteStream",
|
||||
"Client",
|
||||
"Connection",
|
||||
"ConnectionPool",
|
||||
"Content",
|
||||
"delete",
|
||||
"DuplexStream",
|
||||
"File",
|
||||
"FileStream",
|
||||
"Files",
|
||||
"Form",
|
||||
"get",
|
||||
"Headers",
|
||||
"HTML",
|
||||
"HTTPParser",
|
||||
"HTTPStream",
|
||||
"JSON",
|
||||
"MultiPart",
|
||||
"NetworkBackend",
|
||||
"NetworkStream",
|
||||
"open_connection",
|
||||
"post",
|
||||
"ProtocolError",
|
||||
"put",
|
||||
"patch",
|
||||
"Response",
|
||||
"Request",
|
||||
"run",
|
||||
"serve_http",
|
||||
"Stream",
|
||||
"Text",
|
||||
"timeout",
|
||||
"Transport",
|
||||
"QueryParams",
|
||||
"quote",
|
||||
"unquote",
|
||||
"URL",
|
||||
"urldecode",
|
||||
"urlencode",
|
||||
]
|
||||
|
||||
|
||||
__locals = locals()
|
||||
for __name in __all__:
|
||||
if not __name.startswith('__'):
|
||||
setattr(__locals[__name], "__module__", "httpx")
|
||||
156
src/ahttpx/_client.py
Normal file
156
src/ahttpx/_client.py
Normal file
@ -0,0 +1,156 @@
|
||||
import types
|
||||
import typing
|
||||
|
||||
from ._content import Content
|
||||
from ._headers import Headers
|
||||
from ._pool import ConnectionPool, Transport
|
||||
from ._request import Request
|
||||
from ._response import Response
|
||||
from ._streams import Stream
|
||||
from ._urls import URL
|
||||
|
||||
__all__ = ["Client"]
|
||||
|
||||
|
||||
class Client:
|
||||
def __init__(
|
||||
self,
|
||||
url: URL | str | None = None,
|
||||
headers: Headers | typing.Mapping[str, str] | None = None,
|
||||
transport: Transport | None = None,
|
||||
):
|
||||
if url is None:
|
||||
url = ""
|
||||
if headers is None:
|
||||
headers = {"User-Agent": "dev"}
|
||||
if transport is None:
|
||||
transport = ConnectionPool()
|
||||
|
||||
self.url = URL(url)
|
||||
self.headers = Headers(headers)
|
||||
self.transport = transport
|
||||
self.via = RedirectMiddleware(self.transport)
|
||||
|
||||
def build_request(
|
||||
self,
|
||||
method: str,
|
||||
url: URL | str,
|
||||
headers: Headers | typing.Mapping[str, str] | None = None,
|
||||
content: Content | Stream | bytes | None = None,
|
||||
) -> Request:
|
||||
return Request(
|
||||
method=method,
|
||||
url=self.url.join(url),
|
||||
headers=self.headers.copy_update(headers),
|
||||
content=content,
|
||||
)
|
||||
|
||||
async def request(
|
||||
self,
|
||||
method: str,
|
||||
url: URL | str,
|
||||
headers: Headers | typing.Mapping[str, str] | None = None,
|
||||
content: Content | Stream | bytes | None = None,
|
||||
) -> Response:
|
||||
request = self.build_request(method, url, headers=headers, content=content)
|
||||
async with await self.via.send(request) as response:
|
||||
await response.read()
|
||||
return response
|
||||
|
||||
async def stream(
|
||||
self,
|
||||
method: str,
|
||||
url: URL | str,
|
||||
headers: Headers | typing.Mapping[str, str] | None = None,
|
||||
content: Content | Stream | bytes | None = None,
|
||||
) -> Response:
|
||||
request = self.build_request(method, url, headers=headers, content=content)
|
||||
return await self.via.send(request)
|
||||
|
||||
async def get(
|
||||
self,
|
||||
url: URL | str,
|
||||
headers: Headers | typing.Mapping[str, str] | None = None,
|
||||
):
|
||||
return await self.request("GET", url, headers=headers)
|
||||
|
||||
async def post(
|
||||
self,
|
||||
url: URL | str,
|
||||
headers: Headers | typing.Mapping[str, str] | None = None,
|
||||
content: Content | Stream | bytes | None = None,
|
||||
):
|
||||
return await self.request("POST", url, headers=headers, content=content)
|
||||
|
||||
async def put(
|
||||
self,
|
||||
url: URL | str,
|
||||
headers: Headers | typing.Mapping[str, str] | None = None,
|
||||
content: Content | Stream | bytes | None = None,
|
||||
):
|
||||
return await self.request("PUT", url, headers=headers, content=content)
|
||||
|
||||
async def patch(
|
||||
self,
|
||||
url: URL | str,
|
||||
headers: Headers | typing.Mapping[str, str] | None = None,
|
||||
content: Content | Stream | bytes | None = None,
|
||||
):
|
||||
return await self.request("PATCH", url, headers=headers, content=content)
|
||||
|
||||
async def delete(
|
||||
self,
|
||||
url: URL | str,
|
||||
headers: Headers | typing.Mapping[str, str] | None = None,
|
||||
):
|
||||
return await self.request("DELETE", url, headers=headers)
|
||||
|
||||
async def close(self):
|
||||
await self.transport.close()
|
||||
|
||||
async def __aenter__(self):
|
||||
return self
|
||||
|
||||
async def __aexit__(
|
||||
self,
|
||||
exc_type: type[BaseException] | None = None,
|
||||
exc_value: BaseException | None = None,
|
||||
traceback: types.TracebackType | None = None
|
||||
):
|
||||
await self.close()
|
||||
|
||||
def __repr__(self):
|
||||
return f"<Client [{self.transport.description()}]>"
|
||||
|
||||
|
||||
class RedirectMiddleware(Transport):
|
||||
def __init__(self, transport: Transport) -> None:
|
||||
self._transport = transport
|
||||
|
||||
def is_redirect(self, response: Response) -> bool:
|
||||
return (
|
||||
response.status_code in (301, 302, 303, 307, 308)
|
||||
and "Location" in response.headers
|
||||
)
|
||||
|
||||
def build_redirect_request(self, request: Request, response: Response) -> Request:
|
||||
raise NotImplementedError()
|
||||
|
||||
async def send(self, request: Request) -> Response:
|
||||
while True:
|
||||
response = await self._transport.send(request)
|
||||
|
||||
if not self.is_redirect(response):
|
||||
return response
|
||||
|
||||
# If we have a redirect, then we read the body of the response.
|
||||
# Ensures that the HTTP connection is available for a new
|
||||
# request/response cycle.
|
||||
await response.read()
|
||||
await response.close()
|
||||
|
||||
# We've made a request-response and now need to issue a redirect request.
|
||||
request = self.build_redirect_request(request, response)
|
||||
|
||||
async def close(self):
|
||||
pass
|
||||
379
src/ahttpx/_content.py
Normal file
379
src/ahttpx/_content.py
Normal file
@ -0,0 +1,379 @@
|
||||
import json
|
||||
import os
|
||||
import typing
|
||||
|
||||
from ._streams import Stream, ByteStream, FileStream, MultiPartStream
|
||||
from ._urlencode import urldecode, urlencode
|
||||
|
||||
__all__ = [
|
||||
"Content",
|
||||
"Form",
|
||||
"File",
|
||||
"Files",
|
||||
"JSON",
|
||||
"MultiPart",
|
||||
"Text",
|
||||
"HTML",
|
||||
]
|
||||
|
||||
# https://github.com/nginx/nginx/blob/master/conf/mime.types
|
||||
_content_types = {
|
||||
".json": "application/json",
|
||||
".js": "application/javascript",
|
||||
".html": "text/html",
|
||||
".css": "text/css",
|
||||
".png": "image/png",
|
||||
".jpeg": "image/jpeg",
|
||||
".jpg": "image/jpeg",
|
||||
".gif": "image/gif",
|
||||
}
|
||||
|
||||
|
||||
class Content:
|
||||
def encode(self) -> Stream:
|
||||
raise NotImplementedError()
|
||||
|
||||
def content_type(self) -> str:
|
||||
raise NotImplementedError()
|
||||
|
||||
|
||||
class Form(typing.Mapping[str, str], Content):
|
||||
"""
|
||||
HTML form data, as an immutable multi-dict.
|
||||
Form parameters, as a multi-dict.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
form: (
|
||||
typing.Mapping[str, str | typing.Sequence[str]]
|
||||
| typing.Sequence[tuple[str, str]]
|
||||
| str
|
||||
| None
|
||||
) = None,
|
||||
) -> None:
|
||||
d: dict[str, list[str]] = {}
|
||||
|
||||
if form is None:
|
||||
d = {}
|
||||
elif isinstance(form, str):
|
||||
d = urldecode(form)
|
||||
elif isinstance(form, typing.Mapping):
|
||||
# Convert dict inputs like:
|
||||
# {"a": "123", "b": ["456", "789"]}
|
||||
# To dict inputs where values are always lists, like:
|
||||
# {"a": ["123"], "b": ["456", "789"]}
|
||||
d = {k: [v] if isinstance(v, str) else list(v) for k, v in form.items()}
|
||||
else:
|
||||
# Convert list inputs like:
|
||||
# [("a", "123"), ("a", "456"), ("b", "789")]
|
||||
# To a dict representation, like:
|
||||
# {"a": ["123", "456"], "b": ["789"]}
|
||||
for k, v in form:
|
||||
d.setdefault(k, []).append(v)
|
||||
|
||||
self._dict = d
|
||||
|
||||
# Content API
|
||||
|
||||
def encode(self) -> Stream:
|
||||
content = str(self).encode("ascii")
|
||||
return ByteStream(content)
|
||||
|
||||
def content_type(self) -> str:
|
||||
return "application/x-www-form-urlencoded"
|
||||
|
||||
# Dict operations
|
||||
|
||||
def keys(self) -> typing.KeysView[str]:
|
||||
return self._dict.keys()
|
||||
|
||||
def values(self) -> typing.ValuesView[str]:
|
||||
return {k: v[0] for k, v in self._dict.items()}.values()
|
||||
|
||||
def items(self) -> typing.ItemsView[str, str]:
|
||||
return {k: v[0] for k, v in self._dict.items()}.items()
|
||||
|
||||
def get(self, key: str, default: typing.Any = None) -> typing.Any:
|
||||
if key in self._dict:
|
||||
return self._dict[key][0]
|
||||
return default
|
||||
|
||||
# Multi-dict operations
|
||||
|
||||
def multi_items(self) -> list[tuple[str, str]]:
|
||||
multi_items: list[tuple[str, str]] = []
|
||||
for k, v in self._dict.items():
|
||||
multi_items.extend([(k, i) for i in v])
|
||||
return multi_items
|
||||
|
||||
def multi_dict(self) -> dict[str, list[str]]:
|
||||
return {k: list(v) for k, v in self._dict.items()}
|
||||
|
||||
def get_list(self, key: str) -> list[str]:
|
||||
return list(self._dict.get(key, []))
|
||||
|
||||
# Update operations
|
||||
|
||||
def copy_set(self, key: str, value: str) -> "Form":
|
||||
d = self.multi_dict()
|
||||
d[key] = [value]
|
||||
return Form(d)
|
||||
|
||||
def copy_append(self, key: str, value: str) -> "Form":
|
||||
d = self.multi_dict()
|
||||
d[key] = d.get(key, []) + [value]
|
||||
return Form(d)
|
||||
|
||||
def copy_remove(self, key: str) -> "Form":
|
||||
d = self.multi_dict()
|
||||
d.pop(key, None)
|
||||
return Form(d)
|
||||
|
||||
# Accessors & built-ins
|
||||
|
||||
def __getitem__(self, key: str) -> str:
|
||||
return self._dict[key][0]
|
||||
|
||||
def __contains__(self, key: typing.Any) -> bool:
|
||||
return key in self._dict
|
||||
|
||||
def __iter__(self) -> typing.Iterator[str]:
|
||||
return iter(self.keys())
|
||||
|
||||
def __len__(self) -> int:
|
||||
return len(self._dict)
|
||||
|
||||
def __bool__(self) -> bool:
|
||||
return bool(self._dict)
|
||||
|
||||
def __hash__(self) -> int:
|
||||
return hash(str(self))
|
||||
|
||||
def __eq__(self, other: typing.Any) -> bool:
|
||||
return (
|
||||
isinstance(other, Form) and
|
||||
sorted(self.multi_items()) == sorted(other.multi_items())
|
||||
)
|
||||
|
||||
def __str__(self) -> str:
|
||||
return urlencode(self.multi_dict())
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return f"<Form {self.multi_items()!r}>"
|
||||
|
||||
|
||||
class File(Content):
|
||||
"""
|
||||
Wrapper class used for files in uploads and multipart requests.
|
||||
"""
|
||||
|
||||
def __init__(self, path: str):
|
||||
self._path = path
|
||||
|
||||
def name(self) -> str:
|
||||
return os.path.basename(self._path)
|
||||
|
||||
def size(self) -> int:
|
||||
return os.path.getsize(self._path)
|
||||
|
||||
def encode(self) -> Stream:
|
||||
fin = open(self._path, 'rb')
|
||||
return FileStream(self._path, fin)
|
||||
|
||||
def content_type(self) -> str:
|
||||
_, ext = os.path.splitext(self._path)
|
||||
ct = _content_types.get(ext, "application/octet-stream")
|
||||
if ct.startswith('text/'):
|
||||
ct += "; charset='utf-8'"
|
||||
return ct
|
||||
|
||||
def __lt__(self, other: typing.Any) -> bool:
|
||||
return isinstance(other, File) and other._path < self._path
|
||||
|
||||
def __eq__(self, other: typing.Any) -> bool:
|
||||
return isinstance(other, File) and other._path == self._path
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return f"<File {self._path!r}>"
|
||||
|
||||
|
||||
class Files(typing.Mapping[str, File], Content):
|
||||
"""
|
||||
File parameters, as a multi-dict.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
files: (
|
||||
typing.Mapping[str, File | typing.Sequence[File]]
|
||||
| typing.Sequence[tuple[str, File]]
|
||||
| None
|
||||
) = None,
|
||||
boundary: str = ''
|
||||
) -> None:
|
||||
d: dict[str, list[File]] = {}
|
||||
|
||||
if files is None:
|
||||
d = {}
|
||||
elif isinstance(files, typing.Mapping):
|
||||
d = {k: [v] if isinstance(v, File) else list(v) for k, v in files.items()}
|
||||
else:
|
||||
d = {}
|
||||
for k, v in files:
|
||||
d.setdefault(k, []).append(v)
|
||||
|
||||
self._dict = d
|
||||
self._boundary = boundary or os.urandom(16).hex()
|
||||
|
||||
# Standard dict interface
|
||||
def keys(self) -> typing.KeysView[str]:
|
||||
return self._dict.keys()
|
||||
|
||||
def values(self) -> typing.ValuesView[File]:
|
||||
return {k: v[0] for k, v in self._dict.items()}.values()
|
||||
|
||||
def items(self) -> typing.ItemsView[str, File]:
|
||||
return {k: v[0] for k, v in self._dict.items()}.items()
|
||||
|
||||
def get(self, key: str, default: typing.Any = None) -> typing.Any:
|
||||
if key in self._dict:
|
||||
return self._dict[key][0]
|
||||
return None
|
||||
|
||||
# Multi dict interface
|
||||
def multi_items(self) -> list[tuple[str, File]]:
|
||||
multi_items: list[tuple[str, File]] = []
|
||||
for k, v in self._dict.items():
|
||||
multi_items.extend([(k, i) for i in v])
|
||||
return multi_items
|
||||
|
||||
def multi_dict(self) -> dict[str, list[File]]:
|
||||
return {k: list(v) for k, v in self._dict.items()}
|
||||
|
||||
def get_list(self, key: str) -> list[File]:
|
||||
return list(self._dict.get(key, []))
|
||||
|
||||
# Content interface
|
||||
def encode(self) -> Stream:
|
||||
return MultiPart(files=self).encode()
|
||||
|
||||
def content_type(self) -> str:
|
||||
return f"multipart/form-data; boundary={self._boundary}"
|
||||
|
||||
# Builtins
|
||||
def __getitem__(self, key: str) -> File:
|
||||
return self._dict[key][0]
|
||||
|
||||
def __contains__(self, key: typing.Any) -> bool:
|
||||
return key in self._dict
|
||||
|
||||
def __iter__(self) -> typing.Iterator[str]:
|
||||
return iter(self.keys())
|
||||
|
||||
def __len__(self) -> int:
|
||||
return len(self._dict)
|
||||
|
||||
def __bool__(self) -> bool:
|
||||
return bool(self._dict)
|
||||
|
||||
def __eq__(self, other: typing.Any) -> bool:
|
||||
return (
|
||||
isinstance(other, Files) and
|
||||
sorted(self.multi_items()) == sorted(other.multi_items())
|
||||
)
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return f"<Files {self.multi_items()!r}>"
|
||||
|
||||
|
||||
class JSON(Content):
|
||||
def __init__(self, data: typing.Any) -> None:
|
||||
self._data = data
|
||||
|
||||
def encode(self) -> Stream:
|
||||
content = json.dumps(
|
||||
self._data,
|
||||
ensure_ascii=False,
|
||||
separators=(",", ":"),
|
||||
allow_nan=False
|
||||
).encode("utf-8")
|
||||
return ByteStream(content)
|
||||
|
||||
def content_type(self) -> str:
|
||||
return "application/json"
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return f"<JSON {self._data!r}>"
|
||||
|
||||
|
||||
class Text(Content):
|
||||
def __init__(self, text: str) -> None:
|
||||
self._text = text
|
||||
|
||||
def encode(self) -> Stream:
|
||||
content = self._text.encode("utf-8")
|
||||
return ByteStream(content)
|
||||
|
||||
def content_type(self) -> str:
|
||||
return "text/plain; charset='utf-8'"
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return f"<Text {self._text!r}>"
|
||||
|
||||
|
||||
class HTML(Content):
|
||||
def __init__(self, text: str) -> None:
|
||||
self._text = text
|
||||
|
||||
def encode(self) -> Stream:
|
||||
content = self._text.encode("utf-8")
|
||||
return ByteStream(content)
|
||||
|
||||
def content_type(self) -> str:
|
||||
return "text/html; charset='utf-8'"
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return f"<HTML {self._text!r}>"
|
||||
|
||||
|
||||
class MultiPart(Content):
|
||||
def __init__(
|
||||
self,
|
||||
form: (
|
||||
Form
|
||||
| typing.Mapping[str, str | typing.Sequence[str]]
|
||||
| typing.Sequence[tuple[str, str]]
|
||||
| str
|
||||
| None
|
||||
) = None,
|
||||
files: (
|
||||
Files
|
||||
| typing.Mapping[str, File | typing.Sequence[File]]
|
||||
| typing.Sequence[tuple[str, File]]
|
||||
| None
|
||||
) = None,
|
||||
boundary: str | None = None
|
||||
):
|
||||
self._form = form if isinstance(form , Form) else Form(form)
|
||||
self._files = files if isinstance(files, Files) else Files(files)
|
||||
self._boundary = os.urandom(16).hex() if boundary is None else boundary
|
||||
|
||||
@property
|
||||
def form(self) -> Form:
|
||||
return self._form
|
||||
|
||||
@property
|
||||
def files(self) -> Files:
|
||||
return self._files
|
||||
|
||||
def encode(self) -> Stream:
|
||||
form = [(key, value) for key, value in self._form.items()]
|
||||
files = [(key, file._path) for key, file in self._files.items()]
|
||||
return MultiPartStream(form, files, boundary=self._boundary)
|
||||
|
||||
def content_type(self) -> str:
|
||||
return f"multipart/form-data; boundary={self._boundary}"
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return f"<MultiPart form={self._form.multi_items()!r}, files={self._files.multi_items()!r}>"
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user