Merge branch 'master' into cookie-persistence-option
This commit is contained in:
Mark Wine
GitHub
commit
2dfafb79f9
20
CHANGELOG.md
20
CHANGELOG.md
@ -6,6 +6,16 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
||||
|
||||
## Unreleased
|
||||
|
||||
## Added
|
||||
|
||||
* Support for `zstd` content decoding using the python `zstandard` package is added. Installable using `httpx[zstd]`. (#3139)
|
||||
|
||||
### Fixed
|
||||
|
||||
* Fix `app` type signature in `ASGITransport`. (#3109)
|
||||
|
||||
## 0.27.0 (21st February, 2024)
|
||||
|
||||
### Deprecated
|
||||
|
||||
* The `app=...` shortcut has been deprecated. Use the explicit style of `transport=httpx.WSGITransport()` or `transport=httpx.ASGITransport()` instead.
|
||||
@ -92,7 +102,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
||||
* The logging behaviour has been changed to be more in-line with other standard Python logging usages. We no longer have a custom `TRACE` log level, and we no longer use the `HTTPX_LOG_LEVEL` environment variable to auto-configure logging. We now have a significant amount of `DEBUG` logging available at the network level. Full documentation is available at https://www.python-httpx.org/logging/ (#2547, encode/httpcore#648)
|
||||
* The `Response.iter_lines()` method now matches the stdlib behaviour and does not include the newline characters. It also resolves a performance issue. (#2423)
|
||||
* Query parameter encoding switches from using + for spaces and %2F for forward slash, to instead using %20 for spaces and treating forward slash as a safe, unescaped character. This differs from `requests`, but is in line with browser behavior in Chrome, Safari, and Firefox. Both options are RFC valid. (#2543)
|
||||
* NetRC authentication is no longer automatically handled, but is instead supported by an explicit `httpx.NetRCAuth()` authentication class. See the documentation at https://www.python-httpx.org/advanced/#netrc-support (#2525)
|
||||
* NetRC authentication is no longer automatically handled, but is instead supported by an explicit `httpx.NetRCAuth()` authentication class. See the documentation at https://www.python-httpx.org/advanced/authentication/#netrc-authentication (#2525)
|
||||
|
||||
### Removed
|
||||
|
||||
@ -145,7 +155,7 @@ See the "Removed" section of these release notes for details.
|
||||
### Changed
|
||||
|
||||
* Drop support for Python 3.6. (#2097)
|
||||
* Use `utf-8` as the default character set, instead of falling back to `charset-normalizer` for auto-detection. To enable automatic character set detection, see [the documentation](https://www.python-httpx.org/advanced/#character-set-encodings-and-auto-detection). (#2165)
|
||||
* Use `utf-8` as the default character set, instead of falling back to `charset-normalizer` for auto-detection. To enable automatic character set detection, see [the documentation](https://www.python-httpx.org/advanced/text-encodings/#using-auto-detection). (#2165)
|
||||
|
||||
### Fixed
|
||||
|
||||
@ -164,7 +174,7 @@ See the "Removed" section of these release notes for details.
|
||||
|
||||
### Added
|
||||
|
||||
* Support for [the SOCKS5 proxy protocol](https://www.python-httpx.org/advanced/#socks) via [the `socksio` package](https://github.com/sethmlarson/socksio). (#2034)
|
||||
* Support for [the SOCKS5 proxy protocol](https://www.python-httpx.org/advanced/proxies/#socks) via [the `socksio` package](https://github.com/sethmlarson/socksio). (#2034)
|
||||
* Support for custom headers in multipart/form-data requests (#1936)
|
||||
|
||||
### Fixed
|
||||
@ -319,7 +329,7 @@ finally:
|
||||
|
||||
The 0.18.x release series formalises our low-level Transport API, introducing the base classes `httpx.BaseTransport` and `httpx.AsyncBaseTransport`.
|
||||
|
||||
See the "[Writing custom transports](https://www.python-httpx.org/advanced/#writing-custom-transports)" documentation and the [`httpx.BaseTransport.handle_request()`](https://github.com/encode/httpx/blob/397aad98fdc8b7580a5fc3e88f1578b4302c6382/httpx/_transports/base.py#L77-L147) docstring for more complete details on implementing custom transports.
|
||||
See the "[Custom transports](https://www.python-httpx.org/advanced/transports/#custom-transports)" documentation and the [`httpx.BaseTransport.handle_request()`](https://github.com/encode/httpx/blob/397aad98fdc8b7580a5fc3e88f1578b4302c6382/httpx/_transports/base.py#L77-L147) docstring for more complete details on implementing custom transports.
|
||||
|
||||
Pull request #1522 includes a checklist of differences from the previous `httpcore` transport API, for developers implementing custom transports.
|
||||
|
||||
@ -636,7 +646,7 @@ This release switches to `httpcore` for all the internal networking, which means
|
||||
|
||||
It also means we've had to remove our UDS support, since maintaining that would have meant having to push back our work towards a 1.0 release, which isn't a trade-off we wanted to make.
|
||||
|
||||
We also now have [a public "Transport API"](https://www.python-httpx.org/advanced/#custom-transports), which you can use to implement custom transport implementations against. This formalises and replaces our previously private "Dispatch API".
|
||||
We also now have [a public "Transport API"](https://www.python-httpx.org/advanced/transports/#custom-transports), which you can use to implement custom transport implementations against. This formalises and replaces our previously private "Dispatch API".
|
||||
|
||||
### Changed
|
||||
|
||||
|
||||
@ -138,6 +138,7 @@ As well as these optional installs:
|
||||
* `rich` - Rich terminal support. *(Optional, with `httpx[cli]`)*
|
||||
* `click` - Command line client support. *(Optional, with `httpx[cli]`)*
|
||||
* `brotli` or `brotlicffi` - Decoding for "brotli" compressed responses. *(Optional, with `httpx[brotli]`)*
|
||||
* `zstandard` - Decoding for "zstd" compressed responses. *(Optional, with `httpx[zstd]`)*
|
||||
|
||||
A huge amount of credit is due to `requests` for the API layout that
|
||||
much of this work follows, as well as to `urllib3` for plenty of design
|
||||
|
||||
@ -1,144 +0,0 @@
|
||||
<p align="center">
|
||||
<a href="https://www.python-httpx.org/"><img width="350" height="208" src="https://raw.githubusercontent.com/encode/httpx/master/docs/img/butterfly.png" alt='HTTPX'></a>
|
||||
</p>
|
||||
|
||||
<p align="center"><strong>HTTPX</strong> <em>- 适用于 Python 的下一代 HTTP 客户端</em></p>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/encode/httpx/actions">
|
||||
<img src="https://github.com/encode/httpx/workflows/Test%20Suite/badge.svg" alt="Test Suite">
|
||||
</a>
|
||||
<a href="https://pypi.org/project/httpx/">
|
||||
<img src="https://badge.fury.io/py/httpx.svg" alt="Package version">
|
||||
</a>
|
||||
</p>
|
||||
|
||||
HTTPX 是适用于 Python3 的功能齐全的 HTTP 客户端。 它集成了 **一个命令行客户端**，同时支持 **HTTP/1.1 和 HTTP/2**，并提供了 **同步和异步 API**。
|
||||
|
||||
---
|
||||
|
||||
通过 pip 安装 HTTPX：
|
||||
|
||||
```shell
|
||||
$ pip install httpx
|
||||
```
|
||||
|
||||
使用 httpx：
|
||||
|
||||
```pycon
|
||||
>>> import httpx
|
||||
>>> r = httpx.get('https://www.example.org/')
|
||||
>>> r
|
||||
<Response [200 OK]>
|
||||
>>> r.status_code
|
||||
200
|
||||
>>> r.headers['content-type']
|
||||
'text/html; charset=UTF-8'
|
||||
>>> r.text
|
||||
'<!doctype html>\n<html>\n<head>\n<title>Example Domain</title>...'
|
||||
```
|
||||
|
||||
或者使用命令行客户端。
|
||||
|
||||
```shell
|
||||
$ pip install 'httpx[cli]' # 命令行功能是可选的。
|
||||
```
|
||||
|
||||
它允许我们直接通过命令行来使用 HTTPX...
|
||||
|
||||
<p align="center">
|
||||
<img width="700" src="docs/img/httpx-help.png" alt='httpx --help'>
|
||||
</p>
|
||||
|
||||
发送一个请求...
|
||||
|
||||
<p align="center">
|
||||
<img width="700" src="docs/img/httpx-request.png" alt='httpx http://httpbin.org/json'>
|
||||
</p>
|
||||
|
||||
## 特性
|
||||
|
||||
HTTPX 建立在成熟的 requests 可用性基础上，为您提供以下功能：
|
||||
|
||||
* 广泛的 [requests 兼容 API](https://www.python-httpx.org/compatibility/)。
|
||||
* 内置的命令行客户端功能。
|
||||
* HTTP/1.1 [和 HTTP/2 支持](https://www.python-httpx.org/http2/)。
|
||||
* 标准同步接口，也支持 [异步](https://www.python-httpx.org/async/)。
|
||||
* 能够直接向 [WSGI 应用发送请求](https://www.python-httpx.org/advanced/#calling-into-python-web-apps) 或向 [ASGI 应用发送请求](https://www.python-httpx.org/async/#calling-into-python-web-apps)。
|
||||
* 每一处严格的超时控制。
|
||||
* 完整的类型注解。
|
||||
* 100% 测试。
|
||||
|
||||
加上这些应该具备的标准功能...
|
||||
|
||||
* 国际化域名与 URL
|
||||
* Keep-Alive & 连接池
|
||||
* Cookie 持久性会话
|
||||
* 浏览器风格的 SSL 验证
|
||||
* 基础或摘要身份验证
|
||||
* 优雅的键值 Cookies
|
||||
* 自动解压缩
|
||||
* 内容自动解码
|
||||
* Unicode 响应正文
|
||||
* 分段文件上传
|
||||
* HTTP(S)代理支持
|
||||
* 可配置的连接超时
|
||||
* 流式下载
|
||||
* .netrc 支持
|
||||
* 分块请求
|
||||
|
||||
## 安装
|
||||
|
||||
使用 pip 安装：
|
||||
|
||||
```shell
|
||||
$ pip install httpx
|
||||
```
|
||||
|
||||
或者，安装可选的 HTTP/2 支持：
|
||||
|
||||
```shell
|
||||
$ pip install httpx[http2]
|
||||
```
|
||||
|
||||
HTTPX 要求 Python 3.8+ 版本。
|
||||
|
||||
## 文档
|
||||
|
||||
项目文档现已就绪，请访问 [https://www.python-httpx.org/](https://www.python-httpx.org/) 来阅读。
|
||||
|
||||
要浏览所有基础知识，请访问 [快速开始](https://www.python-httpx.org/quickstart/)。
|
||||
|
||||
更高级的主题，可参阅 [高级用法](https://www.python-httpx.org/advanced/) 章节, [异步支持](https://www.python-httpx.org/async/) 或者 [HTTP/2](https://www.python-httpx.org/http2/) 章节。
|
||||
|
||||
[Developer Interface](https://www.python-httpx.org/api/) 提供了全面的 API 参考。
|
||||
|
||||
要了解与 HTTPX 集成的工具, 请访问 [第三方包](https://www.python-httpx.org/third_party_packages/)。
|
||||
|
||||
## 贡献
|
||||
|
||||
如果您想对本项目做出贡献，请访问 [贡献者指南](https://www.python-httpx.org/contributing/) 来了解如何开始。
|
||||
|
||||
## 依赖
|
||||
|
||||
HTTPX 项目依赖于这些优秀的库：
|
||||
|
||||
* `httpcore` - `httpx` 基础传输接口实现。
|
||||
* `h11` - HTTP/1.1 支持。
|
||||
* `certifi` - SSL 证书。
|
||||
* `idna` - 国际化域名支持。
|
||||
* `sniffio` - 异步库自动检测。
|
||||
|
||||
以及这些可选的安装：
|
||||
|
||||
* `h2` - HTTP/2 支持。 *(可选的，通过 `httpx[http2]`)*
|
||||
* `socksio` - SOCKS 代理支持。 *(可选的， 通过 `httpx[socks]`)*
|
||||
* `rich` - 丰富的终端支持。 *(可选的，通过 `httpx[cli]`)*
|
||||
* `click` - 命令行客户端支持。 *(可选的，通过 `httpx[cli]`)*
|
||||
* `brotli` 或者 `brotlicffi` - 对 “brotli” 压缩响应的解码。*(可选的，通过 `httpx[brotli]`)*
|
||||
|
||||
这项工作的大量功劳都归功于参考了 `requests` 所遵循的 API 结构，以及 `urllib3` 中众多围绕底层网络细节的设计灵感。
|
||||
|
||||
---
|
||||
|
||||
<p align="center"><i>HTTPX 使用 <a href="https://github.com/encode/httpx/blob/master/LICENSE.md">BSD 开源协议</a> code。<br/>精心设计和制作。</i><br/>— 🦋 —</p>
|
||||
@ -1,7 +1,7 @@
|
||||
Authentication can either be included on a per-request basis...
|
||||
|
||||
```pycon
|
||||
>>> auth = httpx.BasicAuthentication(username="username", password="secret")
|
||||
>>> auth = httpx.BasicAuth(username="username", password="secret")
|
||||
>>> client = httpx.Client()
|
||||
>>> response = client.get("https://www.example.com/", auth=auth)
|
||||
```
|
||||
@ -9,7 +9,7 @@ Authentication can either be included on a per-request basis...
|
||||
Or configured on the client instance, ensuring that all outgoing requests will include authentication credentials...
|
||||
|
||||
```pycon
|
||||
>>> auth = httpx.BasicAuthentication(username="username", password="secret")
|
||||
>>> auth = httpx.BasicAuth(username="username", password="secret")
|
||||
>>> client = httpx.Client(auth=auth)
|
||||
>>> response = client.get("https://www.example.com/")
|
||||
```
|
||||
@ -19,7 +19,7 @@ Or configured on the client instance, ensuring that all outgoing requests will i
|
||||
HTTP basic authentication is an unencrypted authentication scheme that uses a simple encoding of the username and password in the request `Authorization` header. Since it is unencrypted it should typically only be used over `https`, although this is not strictly enforced.
|
||||
|
||||
```pycon
|
||||
>>> auth = httpx.BasicAuthentication(username="finley", password="secret")
|
||||
>>> auth = httpx.BasicAuth(username="finley", password="secret")
|
||||
>>> client = httpx.Client(auth=auth)
|
||||
>>> response = client.get("https://httpbin.org/basic-auth/finley/secret")
|
||||
>>> response
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
|
||||
Request and response extensions provide a untyped space where additional information may be added.
|
||||
|
||||
Extensions should be used for features that may not be available on all transports, and that do not fit neatly into [the simplified request/response model](https://www.encode.io/httpcore/extensions/) that the underlying `httpcore` pacakge uses as it's API.
|
||||
Extensions should be used for features that may not be available on all transports, and that do not fit neatly into [the simplified request/response model](https://www.encode.io/httpcore/extensions/) that the underlying `httpcore` package uses as its API.
|
||||
|
||||
Several extensions are supported on the request:
|
||||
|
||||
@ -138,6 +138,47 @@ response = client.get(
|
||||
|
||||
This extension is how the `httpx` timeouts are implemented, ensuring that the timeout values are associated with the request instance and passed throughout the stack. You shouldn't typically be working with this extension directly, but use the higher level `timeout` API instead.
|
||||
|
||||
### `"target"`
|
||||
|
||||
The target that is used as [the HTTP target instead of the URL path](https://datatracker.ietf.org/doc/html/rfc2616#section-5.1.2).
|
||||
|
||||
This enables support constructing requests that would otherwise be unsupported.
|
||||
|
||||
* URL paths with non-standard escaping applied.
|
||||
* Forward proxy requests using an absolute URI.
|
||||
* Tunneling proxy requests using `CONNECT` with hostname as the target.
|
||||
* Server-wide `OPTIONS *` requests.
|
||||
|
||||
Some examples:
|
||||
|
||||
Using the 'target' extension to send requests without the standard path escaping rules...
|
||||
|
||||
```python
|
||||
# Typically a request to "https://www.example.com/test^path" would
|
||||
# connect to "www.example.com" and send an HTTP/1.1 request like...
|
||||
#
|
||||
# GET /test%5Epath HTTP/1.1
|
||||
#
|
||||
# Using the target extension we can include the literal '^'...
|
||||
#
|
||||
# GET /test^path HTTP/1.1
|
||||
#
|
||||
# Note that requests must still be valid HTTP requests.
|
||||
# For example including whitespace in the target will raise a `LocalProtocolError`.
|
||||
extensions = {"target": b"/test^path"}
|
||||
response = httpx.get("https://www.example.com", extensions=extensions)
|
||||
```
|
||||
|
||||
The `target` extension also allows server-wide `OPTIONS *` requests to be constructed...
|
||||
|
||||
```python
|
||||
# This will send the following request...
|
||||
#
|
||||
# CONNECT * HTTP/1.1
|
||||
extensions = {"target": b"*"}
|
||||
response = httpx.request("CONNECT", "https://www.example.com", extensions=extensions)
|
||||
```
|
||||
|
||||
## Response Extensions
|
||||
|
||||
### `"http_version"`
|
||||
@ -198,4 +239,4 @@ with httpx.stream("GET", "https://www.example.com") as response:
|
||||
|
||||
ssl_object = network_stream.get_extra_info("ssl_object")
|
||||
print("TLS version", ssl_object.version())
|
||||
```
|
||||
```
|
||||
|
||||
@ -2,7 +2,7 @@ HTTPX's `Client` also accepts a `transport` argument. This argument allows you
|
||||
to provide a custom Transport object that will be used to perform the actual
|
||||
sending of the requests.
|
||||
|
||||
## HTTPTransport
|
||||
## HTTP Transport
|
||||
|
||||
For some advanced configuration you might need to instantiate a transport
|
||||
class directly, and pass it to the client instance. One example is the
|
||||
@ -83,7 +83,7 @@ with httpx.Client(transport=transport, base_url="http://testserver") as client:
|
||||
...
|
||||
```
|
||||
|
||||
## ASGITransport
|
||||
## ASGI Transport
|
||||
|
||||
You can configure an `httpx` client to call directly into an async Python web application using the ASGI protocol.
|
||||
|
||||
@ -148,7 +148,7 @@ However it is suggested to use `LifespanManager` from [asgi-lifespan](https://gi
|
||||
|
||||
## Custom transports
|
||||
|
||||
A transport instance must implement the low-level Transport API, which deals
|
||||
A transport instance must implement the low-level Transport API which deals
|
||||
with sending a single request, and returning a response. You should either
|
||||
subclass `httpx.BaseTransport` to implement a transport to use with `Client`,
|
||||
or subclass `httpx.AsyncBaseTransport` to implement a transport to
|
||||
@ -166,28 +166,81 @@ A complete example of a custom transport implementation would be:
|
||||
import json
|
||||
import httpx
|
||||
|
||||
|
||||
class HelloWorldTransport(httpx.BaseTransport):
|
||||
"""
|
||||
A mock transport that always returns a JSON "Hello, world!" response.
|
||||
"""
|
||||
|
||||
def handle_request(self, request):
|
||||
message = {"text": "Hello, world!"}
|
||||
content = json.dumps(message).encode("utf-8")
|
||||
stream = httpx.ByteStream(content)
|
||||
headers = [(b"content-type", b"application/json")]
|
||||
return httpx.Response(200, headers=headers, stream=stream)
|
||||
return httpx.Response(200, json={"text": "Hello, world!"})
|
||||
```
|
||||
|
||||
Which we can use in the same way:
|
||||
Or this example, which uses a custom transport and `httpx.Mounts` to always redirect `http://` requests.
|
||||
|
||||
```pycon
|
||||
>>> import httpx
|
||||
>>> client = httpx.Client(transport=HelloWorldTransport())
|
||||
>>> response = client.get("https://example.org/")
|
||||
>>> response.json()
|
||||
{"text": "Hello, world!"}
|
||||
```python
|
||||
class HTTPSRedirect(httpx.BaseTransport):
|
||||
"""
|
||||
A transport that always redirects to HTTPS.
|
||||
"""
|
||||
def handle_request(self, request):
|
||||
url = request.url.copy_with(scheme="https")
|
||||
return httpx.Response(303, headers={"Location": str(url)})
|
||||
|
||||
# A client where any `http` requests are always redirected to `https`
|
||||
transport = httpx.Mounts({
|
||||
'http://': HTTPSRedirect()
|
||||
'https://': httpx.HTTPTransport()
|
||||
})
|
||||
client = httpx.Client(transport=transport)
|
||||
```
|
||||
|
||||
A useful pattern here is custom transport classes that wrap the default HTTP implementation. For example...
|
||||
|
||||
```python
|
||||
class DebuggingTransport(httpx.BaseTransport):
|
||||
def __init__(self, **kwargs):
|
||||
self._wrapper = httpx.HTTPTransport(**kwargs)
|
||||
|
||||
def handle_request(self, request):
|
||||
print(f">>> {request}")
|
||||
response = self._wrapper.handle_request(request)
|
||||
print(f"<<< {response}")
|
||||
return response
|
||||
|
||||
def close(self):
|
||||
self._wrapper.close()
|
||||
|
||||
transport = DebuggingTransport()
|
||||
client = httpx.Client(transport=transport)
|
||||
```
|
||||
|
||||
Here's another case, where we're using a round-robin across a number of different proxies...
|
||||
|
||||
```python
|
||||
class ProxyRoundRobin(httpx.BaseTransport):
|
||||
def __init__(self, proxies, **kwargs):
|
||||
self._transports = [
|
||||
httpx.HTTPTransport(proxy=proxy, **kwargs)
|
||||
for proxy in proxies
|
||||
]
|
||||
self._idx = 0
|
||||
|
||||
def handle_request(self, request):
|
||||
transport = self._transports[self._idx]
|
||||
self._idx = (self._idx + 1) % len(self._transports)
|
||||
return transport.handle_request(request)
|
||||
|
||||
def close(self):
|
||||
for transport in self._transports:
|
||||
transport.close()
|
||||
|
||||
proxies = [
|
||||
httpx.Proxy("http://127.0.0.1:8081"),
|
||||
httpx.Proxy("http://127.0.0.1:8082"),
|
||||
httpx.Proxy("http://127.0.0.1:8083"),
|
||||
]
|
||||
transport = ProxyRoundRobin(proxies=proxies)
|
||||
client = httpx.Client(transport=transport)
|
||||
```
|
||||
|
||||
## Mock transports
|
||||
|
||||
@ -114,7 +114,7 @@ what gets sent over the wire.*
|
||||
'example.org'
|
||||
```
|
||||
|
||||
* `def __init__(url, allow_relative=False, params=None)`
|
||||
* `def __init__(url, **kwargs)`
|
||||
* `.scheme` - **str**
|
||||
* `.authority` - **str**
|
||||
* `.host` - **str**
|
||||
|
||||
@ -206,8 +206,8 @@ UI options.
|
||||
|
||||
At this point the server is ready to start serving requests, you'll need to
|
||||
configure HTTPX as described in the
|
||||
[proxy section](https://www.python-httpx.org/advanced/#http-proxying) and
|
||||
the [SSL certificates section](https://www.python-httpx.org/advanced/#ssl-certificates),
|
||||
[proxy section](https://www.python-httpx.org/advanced/proxies/#http-proxies) and
|
||||
the [SSL certificates section](https://www.python-httpx.org/advanced/ssl/),
|
||||
this is where our previously generated `client.pem` comes in:
|
||||
|
||||
```
|
||||
|
||||
@ -119,6 +119,7 @@ As well as these optional installs:
|
||||
* `rich` - Rich terminal support. *(Optional, with `httpx[cli]`)*
|
||||
* `click` - Command line client support. *(Optional, with `httpx[cli]`)*
|
||||
* `brotli` or `brotlicffi` - Decoding for "brotli" compressed responses. *(Optional, with `httpx[brotli]`)*
|
||||
* `zstandard` - Decoding for "zstd" compressed responses. *(Optional, with `httpx[zstd]`)*
|
||||
|
||||
A huge amount of credit is due to `requests` for the API layout that
|
||||
much of this work follows, as well as to `urllib3` for plenty of design
|
||||
@ -138,10 +139,10 @@ Or, to include the optional HTTP/2 support, use:
|
||||
$ pip install httpx[http2]
|
||||
```
|
||||
|
||||
To include the optional brotli decoder support, use:
|
||||
To include the optional brotli and zstandard decoders support, use:
|
||||
|
||||
```shell
|
||||
$ pip install httpx[brotli]
|
||||
$ pip install httpx[brotli,zstd]
|
||||
```
|
||||
|
||||
HTTPX requires Python 3.8+
|
||||
|
||||
@ -100,7 +100,8 @@ b'<!doctype html>\n<html>\n<head>\n<title>Example Domain</title>...'
|
||||
|
||||
Any `gzip` and `deflate` HTTP response encodings will automatically
|
||||
be decoded for you. If `brotlipy` is installed, then the `brotli` response
|
||||
encoding will also be supported.
|
||||
encoding will be supported. If `zstandard` is installed, then `zstd`
|
||||
response encodings will also be supported.
|
||||
|
||||
For example, to create an image from binary data returned by a request, you can use the following code:
|
||||
|
||||
@ -362,7 +363,8 @@ Or stream the text, on a line-by-line basis...
|
||||
|
||||
HTTPX will use universal line endings, normalising all cases to `\n`.
|
||||
|
||||
In some cases you might want to access the raw bytes on the response without applying any HTTP content decoding. In this case any content encoding that the web server has applied such as `gzip`, `deflate`, or `brotli` will not be automatically decoded.
|
||||
In some cases you might want to access the raw bytes on the response without applying any HTTP content decoding. In this case any content encoding that the web server has applied such as `gzip`, `deflate`, `brotli`, or `zstd` will
|
||||
not be automatically decoded.
|
||||
|
||||
```pycon
|
||||
>>> with httpx.stream("GET", "https://www.example.com") as r:
|
||||
|
||||
@ -1,48 +1,15 @@
|
||||
from .__version__ import __description__, __title__, __version__
|
||||
from ._api import delete, get, head, options, patch, post, put, request, stream
|
||||
from ._auth import Auth, BasicAuth, DigestAuth, NetRCAuth
|
||||
from ._client import USE_CLIENT_DEFAULT, AsyncClient, Client
|
||||
from ._config import Limits, Proxy, Timeout, create_ssl_context
|
||||
from ._content import ByteStream
|
||||
from ._exceptions import (
|
||||
CloseError,
|
||||
ConnectError,
|
||||
ConnectTimeout,
|
||||
CookieConflict,
|
||||
DecodingError,
|
||||
HTTPError,
|
||||
HTTPStatusError,
|
||||
InvalidURL,
|
||||
LocalProtocolError,
|
||||
NetworkError,
|
||||
PoolTimeout,
|
||||
ProtocolError,
|
||||
ProxyError,
|
||||
ReadError,
|
||||
ReadTimeout,
|
||||
RemoteProtocolError,
|
||||
RequestError,
|
||||
RequestNotRead,
|
||||
ResponseNotRead,
|
||||
StreamClosed,
|
||||
StreamConsumed,
|
||||
StreamError,
|
||||
TimeoutException,
|
||||
TooManyRedirects,
|
||||
TransportError,
|
||||
UnsupportedProtocol,
|
||||
WriteError,
|
||||
WriteTimeout,
|
||||
)
|
||||
from ._models import Cookies, Headers, Request, Response
|
||||
from ._status_codes import codes
|
||||
from ._transports.asgi import ASGITransport
|
||||
from ._transports.base import AsyncBaseTransport, BaseTransport
|
||||
from ._transports.default import AsyncHTTPTransport, HTTPTransport
|
||||
from ._transports.mock import MockTransport
|
||||
from ._transports.wsgi import WSGITransport
|
||||
from ._types import AsyncByteStream, SyncByteStream
|
||||
from ._urls import URL, QueryParams
|
||||
from ._api import *
|
||||
from ._auth import *
|
||||
from ._client import *
|
||||
from ._config import *
|
||||
from ._content import *
|
||||
from ._exceptions import *
|
||||
from ._models import *
|
||||
from ._status_codes import *
|
||||
from ._transports import *
|
||||
from ._types import *
|
||||
from ._urls import *
|
||||
|
||||
try:
|
||||
from ._main import main
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
__title__ = "httpx"
|
||||
__description__ = "A next generation HTTP client, for Python 3."
|
||||
__version__ = "0.26.0"
|
||||
__version__ = "0.27.0"
|
||||
|
||||
@ -22,6 +22,18 @@ from ._types import (
|
||||
VerifyTypes,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"delete",
|
||||
"get",
|
||||
"head",
|
||||
"options",
|
||||
"patch",
|
||||
"post",
|
||||
"put",
|
||||
"request",
|
||||
"stream",
|
||||
]
|
||||
|
||||
|
||||
def request(
|
||||
method: str,
|
||||
|
||||
@ -16,6 +16,9 @@ if typing.TYPE_CHECKING: # pragma: no cover
|
||||
from hashlib import _Hash
|
||||
|
||||
|
||||
__all__ = ["Auth", "BasicAuth", "DigestAuth", "NetRCAuth"]
|
||||
|
||||
|
||||
class Auth:
|
||||
"""
|
||||
Base class for all authentication schemes.
|
||||
|
||||
@ -58,6 +58,8 @@ from ._utils import (
|
||||
same_origin,
|
||||
)
|
||||
|
||||
__all__ = ["USE_CLIENT_DEFAULT", "AsyncClient", "Client"]
|
||||
|
||||
# The type annotation for @classmethod and context managers here follows PEP 484
|
||||
# https://www.python.org/dev/peps/pep-0484/#annotating-instance-and-class-methods
|
||||
T = typing.TypeVar("T", bound="Client")
|
||||
@ -342,7 +344,7 @@ class BaseClient:
|
||||
|
||||
See also: [Request instances][0]
|
||||
|
||||
[0]: /advanced/#request-instances
|
||||
[0]: /advanced/clients/#request-instances
|
||||
"""
|
||||
url = self._merge_url(url)
|
||||
headers = self._merge_headers(headers)
|
||||
@ -562,6 +564,15 @@ class BaseClient:
|
||||
|
||||
return request.stream
|
||||
|
||||
def _set_timeout(self, request: Request) -> None:
|
||||
if "timeout" not in request.extensions:
|
||||
timeout = (
|
||||
self.timeout
|
||||
if isinstance(self.timeout, UseClientDefault)
|
||||
else Timeout(self.timeout)
|
||||
)
|
||||
request.extensions = dict(**request.extensions, timeout=timeout.as_dict())
|
||||
|
||||
|
||||
class Client(BaseClient):
|
||||
"""
|
||||
@ -808,7 +819,7 @@ class Client(BaseClient):
|
||||
[Merging of configuration][0] for how the various parameters
|
||||
are merged with client-level configuration.
|
||||
|
||||
[0]: /advanced/#merging-of-configuration
|
||||
[0]: /advanced/clients/#merging-of-configuration
|
||||
"""
|
||||
if cookies is not None:
|
||||
message = (
|
||||
@ -912,7 +923,7 @@ class Client(BaseClient):
|
||||
|
||||
See also: [Request instances][0]
|
||||
|
||||
[0]: /advanced/#request-instances
|
||||
[0]: /advanced/clients/#request-instances
|
||||
"""
|
||||
if self._state == ClientState.CLOSED:
|
||||
raise RuntimeError("Cannot send a request, as the client has been closed.")
|
||||
@ -929,6 +940,8 @@ class Client(BaseClient):
|
||||
else merge_response_cookies
|
||||
)
|
||||
|
||||
self._set_timeout(request)
|
||||
|
||||
auth = self._build_request_auth(request, auth)
|
||||
|
||||
response = self._send_handling_auth(
|
||||
@ -1068,7 +1081,7 @@ class Client(BaseClient):
|
||||
params: QueryParamTypes | None = None,
|
||||
headers: HeaderTypes | None = None,
|
||||
cookies: CookieTypes | None = None,
|
||||
auth: AuthTypes | UseClientDefault = USE_CLIENT_DEFAULT,
|
||||
auth: AuthTypes | UseClientDefault | None = USE_CLIENT_DEFAULT,
|
||||
follow_redirects: bool | UseClientDefault = USE_CLIENT_DEFAULT,
|
||||
merge_response_cookies: bool | UseClientDefault = USE_CLIENT_DEFAULT,
|
||||
timeout: TimeoutTypes | UseClientDefault = USE_CLIENT_DEFAULT,
|
||||
@ -1422,8 +1435,7 @@ class AsyncClient(BaseClient):
|
||||
merge_response_cookies: bool = True,
|
||||
limits: Limits = DEFAULT_LIMITS,
|
||||
max_redirects: int = DEFAULT_MAX_REDIRECTS,
|
||||
event_hooks: None
|
||||
| (typing.Mapping[str, list[typing.Callable[..., typing.Any]]]) = None,
|
||||
event_hooks: None | (typing.Mapping[str, list[EventHook]]) = None,
|
||||
base_url: URLTypes = "",
|
||||
transport: AsyncBaseTransport | None = None,
|
||||
app: typing.Callable[..., typing.Any] | None = None,
|
||||
@ -1470,7 +1482,7 @@ class AsyncClient(BaseClient):
|
||||
)
|
||||
warnings.warn(message, DeprecationWarning)
|
||||
|
||||
allow_env_proxies = trust_env and transport is None
|
||||
allow_env_proxies = trust_env and app is None and transport is None
|
||||
proxy_map = self._get_proxy_map(proxies or proxy, allow_env_proxies)
|
||||
|
||||
self._transport = self._init_transport(
|
||||
@ -1593,7 +1605,7 @@ class AsyncClient(BaseClient):
|
||||
and [Merging of configuration][0] for how the various parameters
|
||||
are merged with client-level configuration.
|
||||
|
||||
[0]: /advanced/#merging-of-configuration
|
||||
[0]: /advanced/clients/#merging-of-configuration
|
||||
"""
|
||||
|
||||
if cookies is not None: # pragma: no cover
|
||||
@ -1637,7 +1649,7 @@ class AsyncClient(BaseClient):
|
||||
params: QueryParamTypes | None = None,
|
||||
headers: HeaderTypes | None = None,
|
||||
cookies: CookieTypes | None = None,
|
||||
auth: AuthTypes | UseClientDefault = USE_CLIENT_DEFAULT,
|
||||
auth: AuthTypes | UseClientDefault | None = USE_CLIENT_DEFAULT,
|
||||
follow_redirects: bool | UseClientDefault = USE_CLIENT_DEFAULT,
|
||||
merge_response_cookies: bool | UseClientDefault = USE_CLIENT_DEFAULT,
|
||||
timeout: TimeoutTypes | UseClientDefault = USE_CLIENT_DEFAULT,
|
||||
@ -1698,7 +1710,7 @@ class AsyncClient(BaseClient):
|
||||
|
||||
See also: [Request instances][0]
|
||||
|
||||
[0]: /advanced/#request-instances
|
||||
[0]: /advanced/clients/#request-instances
|
||||
"""
|
||||
if self._state == ClientState.CLOSED:
|
||||
raise RuntimeError("Cannot send a request, as the client has been closed.")
|
||||
@ -1715,6 +1727,8 @@ class AsyncClient(BaseClient):
|
||||
else merge_response_cookies
|
||||
)
|
||||
|
||||
self._set_timeout(request)
|
||||
|
||||
auth = self._build_request_auth(request, auth)
|
||||
|
||||
response = await self._send_handling_auth(
|
||||
|
||||
@ -2,8 +2,12 @@
|
||||
The _compat module is used for code which requires branching between different
|
||||
Python environments. It is excluded from the code coverage checks.
|
||||
"""
|
||||
|
||||
import re
|
||||
import ssl
|
||||
import sys
|
||||
from types import ModuleType
|
||||
from typing import Optional
|
||||
|
||||
# Brotli support is optional
|
||||
# The C bindings in `brotli` are recommended for CPython.
|
||||
@ -16,6 +20,24 @@ except ImportError: # pragma: no cover
|
||||
except ImportError:
|
||||
brotli = None
|
||||
|
||||
# Zstandard support is optional
|
||||
zstd: Optional[ModuleType] = None
|
||||
try:
|
||||
import zstandard as zstd
|
||||
except (AttributeError, ImportError, ValueError): # Defensive:
|
||||
zstd = None
|
||||
else:
|
||||
# The package 'zstandard' added the 'eof' property starting
|
||||
# in v0.18.0 which we require to ensure a complete and
|
||||
# valid zstd stream was fed into the ZstdDecoder.
|
||||
# See: https://github.com/urllib3/urllib3/pull/2624
|
||||
_zstd_version = tuple(
|
||||
map(int, re.search(r"^([0-9]+)\.([0-9]+)", zstd.__version__).groups()) # type: ignore[union-attr]
|
||||
)
|
||||
if _zstd_version < (0, 18): # Defensive:
|
||||
zstd = None
|
||||
|
||||
|
||||
if sys.version_info >= (3, 10) or ssl.OPENSSL_VERSION_INFO >= (1, 1, 0, 7):
|
||||
|
||||
def set_minimum_tls_version_1_2(context: ssl.SSLContext) -> None:
|
||||
|
||||
@ -14,6 +14,8 @@ from ._types import CertTypes, HeaderTypes, TimeoutTypes, URLTypes, VerifyTypes
|
||||
from ._urls import URL
|
||||
from ._utils import get_ca_bundle_from_env
|
||||
|
||||
__all__ = ["Limits", "Proxy", "Timeout", "create_ssl_context"]
|
||||
|
||||
DEFAULT_CIPHERS = ":".join(
|
||||
[
|
||||
"ECDHE+AESGCM",
|
||||
|
||||
@ -25,6 +25,8 @@ from ._types import (
|
||||
)
|
||||
from ._utils import peek_filelike_length, primitive_value_to_str
|
||||
|
||||
__all__ = ["ByteStream"]
|
||||
|
||||
|
||||
class ByteStream(AsyncByteStream, SyncByteStream):
|
||||
def __init__(self, stream: bytes) -> None:
|
||||
|
||||
@ -3,6 +3,7 @@ Handlers for Content-Encoding.
|
||||
|
||||
See: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import codecs
|
||||
@ -10,7 +11,7 @@ import io
|
||||
import typing
|
||||
import zlib
|
||||
|
||||
from ._compat import brotli
|
||||
from ._compat import brotli, zstd
|
||||
from ._exceptions import DecodingError
|
||||
|
||||
|
||||
@ -139,6 +140,44 @@ class BrotliDecoder(ContentDecoder):
|
||||
raise DecodingError(str(exc)) from exc
|
||||
|
||||
|
||||
class ZStandardDecoder(ContentDecoder):
|
||||
"""
|
||||
Handle 'zstd' RFC 8878 decoding.
|
||||
|
||||
Requires `pip install zstandard`.
|
||||
Can be installed as a dependency of httpx using `pip install httpx[zstd]`.
|
||||
"""
|
||||
|
||||
# inspired by the ZstdDecoder implementation in urllib3
|
||||
def __init__(self) -> None:
|
||||
if zstd is None: # pragma: no cover
|
||||
raise ImportError(
|
||||
"Using 'ZStandardDecoder', ..."
|
||||
"Make sure to install httpx using `pip install httpx[zstd]`."
|
||||
) from None
|
||||
|
||||
self.decompressor = zstd.ZstdDecompressor().decompressobj()
|
||||
|
||||
def decode(self, data: bytes) -> bytes:
|
||||
assert zstd is not None
|
||||
output = io.BytesIO()
|
||||
try:
|
||||
output.write(self.decompressor.decompress(data))
|
||||
while self.decompressor.eof and self.decompressor.unused_data:
|
||||
unused_data = self.decompressor.unused_data
|
||||
self.decompressor = zstd.ZstdDecompressor().decompressobj()
|
||||
output.write(self.decompressor.decompress(unused_data))
|
||||
except zstd.ZstdError as exc:
|
||||
raise DecodingError(str(exc)) from exc
|
||||
return output.getvalue()
|
||||
|
||||
def flush(self) -> bytes:
|
||||
ret = self.decompressor.flush() # note: this is a no-op
|
||||
if not self.decompressor.eof:
|
||||
raise DecodingError("Zstandard data is incomplete") # pragma: no cover
|
||||
return bytes(ret)
|
||||
|
||||
|
||||
class MultiDecoder(ContentDecoder):
|
||||
"""
|
||||
Handle the case where multiple encodings have been applied.
|
||||
@ -322,8 +361,11 @@ SUPPORTED_DECODERS = {
|
||||
"gzip": GZipDecoder,
|
||||
"deflate": DeflateDecoder,
|
||||
"br": BrotliDecoder,
|
||||
"zstd": ZStandardDecoder,
|
||||
}
|
||||
|
||||
|
||||
if brotli is None:
|
||||
SUPPORTED_DECODERS.pop("br") # pragma: no cover
|
||||
if zstd is None:
|
||||
SUPPORTED_DECODERS.pop("zstd") # pragma: no cover
|
||||
|
||||
@ -30,6 +30,7 @@ Our exception hierarchy:
|
||||
x ResponseNotRead
|
||||
x RequestNotRead
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
@ -38,6 +39,37 @@ import typing
|
||||
if typing.TYPE_CHECKING:
|
||||
from ._models import Request, Response # pragma: no cover
|
||||
|
||||
__all__ = [
|
||||
"CloseError",
|
||||
"ConnectError",
|
||||
"ConnectTimeout",
|
||||
"CookieConflict",
|
||||
"DecodingError",
|
||||
"HTTPError",
|
||||
"HTTPStatusError",
|
||||
"InvalidURL",
|
||||
"LocalProtocolError",
|
||||
"NetworkError",
|
||||
"PoolTimeout",
|
||||
"ProtocolError",
|
||||
"ProxyError",
|
||||
"ReadError",
|
||||
"ReadTimeout",
|
||||
"RemoteProtocolError",
|
||||
"RequestError",
|
||||
"RequestNotRead",
|
||||
"ResponseNotRead",
|
||||
"StreamClosed",
|
||||
"StreamConsumed",
|
||||
"StreamError",
|
||||
"TimeoutException",
|
||||
"TooManyRedirects",
|
||||
"TransportError",
|
||||
"UnsupportedProtocol",
|
||||
"WriteError",
|
||||
"WriteTimeout",
|
||||
]
|
||||
|
||||
|
||||
class HTTPError(Exception):
|
||||
"""
|
||||
|
||||
@ -53,6 +53,8 @@ from ._utils import (
|
||||
parse_header_links,
|
||||
)
|
||||
|
||||
__all__ = ["Cookies", "Headers", "Request", "Response"]
|
||||
|
||||
|
||||
class Headers(typing.MutableMapping[str, str]):
|
||||
"""
|
||||
@ -816,7 +818,7 @@ class Response:
|
||||
def iter_bytes(self, chunk_size: int | None = None) -> typing.Iterator[bytes]:
|
||||
"""
|
||||
A byte-iterator over the decoded response content.
|
||||
This allows us to handle gzip, deflate, and brotli encoded responses.
|
||||
This allows us to handle gzip, deflate, brotli, and zstd encoded responses.
|
||||
"""
|
||||
if hasattr(self, "_content"):
|
||||
chunk_size = len(self._content) if chunk_size is None else chunk_size
|
||||
@ -916,7 +918,7 @@ class Response:
|
||||
) -> typing.AsyncIterator[bytes]:
|
||||
"""
|
||||
A byte-iterator over the decoded response content.
|
||||
This allows us to handle gzip, deflate, and brotli encoded responses.
|
||||
This allows us to handle gzip, deflate, brotli, and zstd encoded responses.
|
||||
"""
|
||||
if hasattr(self, "_content"):
|
||||
chunk_size = len(self._content) if chunk_size is None else chunk_size
|
||||
|
||||
@ -2,6 +2,8 @@ from __future__ import annotations
|
||||
|
||||
from enum import IntEnum
|
||||
|
||||
__all__ = ["codes"]
|
||||
|
||||
|
||||
class codes(IntEnum):
|
||||
"""HTTP status codes and reason phrases
|
||||
|
||||
@ -0,0 +1,15 @@
|
||||
from .asgi import *
|
||||
from .base import *
|
||||
from .default import *
|
||||
from .mock import *
|
||||
from .wsgi import *
|
||||
|
||||
__all__ = [
|
||||
"ASGITransport",
|
||||
"AsyncBaseTransport",
|
||||
"BaseTransport",
|
||||
"AsyncHTTPTransport",
|
||||
"HTTPTransport",
|
||||
"MockTransport",
|
||||
"WSGITransport",
|
||||
]
|
||||
@ -16,15 +16,17 @@ if typing.TYPE_CHECKING: # pragma: no cover
|
||||
Event = typing.Union[asyncio.Event, trio.Event]
|
||||
|
||||
|
||||
_Message = typing.Dict[str, typing.Any]
|
||||
_Message = typing.MutableMapping[str, typing.Any]
|
||||
_Receive = typing.Callable[[], typing.Awaitable[_Message]]
|
||||
_Send = typing.Callable[
|
||||
[typing.Dict[str, typing.Any]], typing.Coroutine[None, None, None]
|
||||
[typing.MutableMapping[str, typing.Any]], typing.Awaitable[None]
|
||||
]
|
||||
_ASGIApp = typing.Callable[
|
||||
[typing.Dict[str, typing.Any], _Receive, _Send], typing.Coroutine[None, None, None]
|
||||
[typing.MutableMapping[str, typing.Any], _Receive, _Send], typing.Awaitable[None]
|
||||
]
|
||||
|
||||
__all__ = ["ASGITransport"]
|
||||
|
||||
|
||||
def create_event() -> Event:
|
||||
if sniffio.current_async_library() == "trio":
|
||||
@ -139,7 +141,7 @@ class ASGITransport(AsyncBaseTransport):
|
||||
return {"type": "http.request", "body": b"", "more_body": False}
|
||||
return {"type": "http.request", "body": body, "more_body": True}
|
||||
|
||||
async def send(message: dict[str, typing.Any]) -> None:
|
||||
async def send(message: typing.MutableMapping[str, typing.Any]) -> None:
|
||||
nonlocal status_code, response_headers, response_started
|
||||
|
||||
if message["type"] == "http.response.start":
|
||||
|
||||
@ -8,6 +8,8 @@ from .._models import Request, Response
|
||||
T = typing.TypeVar("T", bound="BaseTransport")
|
||||
A = typing.TypeVar("A", bound="AsyncBaseTransport")
|
||||
|
||||
__all__ = ["AsyncBaseTransport", "BaseTransport"]
|
||||
|
||||
|
||||
class BaseTransport:
|
||||
def __enter__(self: T) -> T:
|
||||
|
||||
@ -23,6 +23,7 @@ client = httpx.Client(transport=transport)
|
||||
transport = httpx.HTTPTransport(uds="socket.uds")
|
||||
client = httpx.Client(transport=transport)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
@ -62,6 +63,8 @@ SOCKET_OPTION = typing.Union[
|
||||
typing.Tuple[int, int, None, int],
|
||||
]
|
||||
|
||||
__all__ = ["AsyncHTTPTransport", "HTTPTransport"]
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def map_httpcore_exceptions() -> typing.Iterator[None]:
|
||||
|
||||
@ -9,6 +9,9 @@ SyncHandler = typing.Callable[[Request], Response]
|
||||
AsyncHandler = typing.Callable[[Request], typing.Coroutine[None, None, Response]]
|
||||
|
||||
|
||||
__all__ = ["MockTransport"]
|
||||
|
||||
|
||||
class MockTransport(AsyncBaseTransport, BaseTransport):
|
||||
def __init__(self, handler: SyncHandler | AsyncHandler) -> None:
|
||||
self.handler = handler
|
||||
|
||||
@ -16,6 +16,9 @@ if typing.TYPE_CHECKING:
|
||||
_T = typing.TypeVar("_T")
|
||||
|
||||
|
||||
__all__ = ["WSGITransport"]
|
||||
|
||||
|
||||
def _skip_leading_empty_chunks(body: typing.Iterable[_T]) -> typing.Iterable[_T]:
|
||||
body = iter(body)
|
||||
for chunk in body:
|
||||
|
||||
@ -108,6 +108,8 @@ RequestFiles = Union[Mapping[str, FileTypes], Sequence[Tuple[str, FileTypes]]]
|
||||
|
||||
RequestExtensions = MutableMapping[str, Any]
|
||||
|
||||
__all__ = ["AsyncByteStream", "SyncByteStream"]
|
||||
|
||||
|
||||
class SyncByteStream:
|
||||
def __iter__(self) -> Iterator[bytes]:
|
||||
|
||||
@ -15,6 +15,7 @@ Previously we relied on the excellent `rfc3986` package to handle URL parsing an
|
||||
validation, but this module provides a simpler alternative, with less indirection
|
||||
required.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
|
||||
@ -9,6 +9,8 @@ from ._types import QueryParamTypes, RawURL, URLTypes
|
||||
from ._urlparse import urlencode, urlparse
|
||||
from ._utils import primitive_value_to_str
|
||||
|
||||
__all__ = ["URL", "QueryParams"]
|
||||
|
||||
|
||||
class URL:
|
||||
"""
|
||||
|
||||
@ -52,6 +52,9 @@ http2 = [
|
||||
socks = [
|
||||
"socksio==1.*",
|
||||
]
|
||||
zstd = [
|
||||
"zstandard>=0.18.0",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
httpx = "httpx:main"
|
||||
@ -93,14 +96,16 @@ text = "\n---\n\n[Full changelog](https://github.com/encode/httpx/blob/master/CH
|
||||
pattern = 'src="(docs/img/.*?)"'
|
||||
replacement = 'src="https://raw.githubusercontent.com/encode/httpx/master/\1"'
|
||||
|
||||
# https://beta.ruff.rs/docs/configuration/#using-rufftoml
|
||||
[tool.ruff]
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "I", "B", "PIE"]
|
||||
ignore = ["B904", "B028"]
|
||||
|
||||
[tool.ruff.isort]
|
||||
[tool.ruff.lint.isort]
|
||||
combine-as-imports = true
|
||||
|
||||
[tool.ruff.lint.per-file-ignores]
|
||||
"__init__.py" = ["F403", "F405"]
|
||||
|
||||
[tool.mypy]
|
||||
ignore_missing_imports = true
|
||||
strict = true
|
||||
|
||||
@ -2,28 +2,28 @@
|
||||
# On the other hand, we're not pinning package dependencies, because our tests
|
||||
# needs to pass with the latest version of the packages.
|
||||
# Reference: https://github.com/encode/httpx/pull/1721#discussion_r661241588
|
||||
-e .[brotli,cli,http2,socks]
|
||||
-e .[brotli,cli,http2,socks,zstd]
|
||||
|
||||
# Optional charset auto-detection
|
||||
# Used in our test cases
|
||||
chardet==5.2.0
|
||||
|
||||
# Documentation
|
||||
mkdocs==1.5.3
|
||||
mkdocs==1.6.0
|
||||
mkautodoc==0.2.0
|
||||
mkdocs-material==9.5.6
|
||||
mkdocs-material==9.5.20
|
||||
|
||||
# Packaging
|
||||
build==1.0.3
|
||||
twine==4.0.2
|
||||
build==1.2.1
|
||||
twine==5.0.0
|
||||
|
||||
# Tests & Linting
|
||||
coverage[toml]==7.4.1
|
||||
cryptography==42.0.2
|
||||
mypy==1.8.0
|
||||
pytest==8.0.0
|
||||
ruff==0.1.15
|
||||
trio==0.24.0
|
||||
coverage[toml]==7.5.0
|
||||
cryptography==42.0.5
|
||||
mypy==1.10.0
|
||||
pytest==8.2.0
|
||||
ruff==0.4.2
|
||||
trio==0.25.0
|
||||
trio-typing==0.10.0
|
||||
trustme==1.1.0
|
||||
uvicorn==0.27.0.post1
|
||||
uvicorn==0.29.0
|
||||
|
||||
@ -3,6 +3,7 @@ Integration tests for authentication.
|
||||
|
||||
Unit tests for auth classes also exist in tests/test_auth.py
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import netrc
|
||||
import os
|
||||
|
||||
@ -357,7 +357,7 @@ def test_raw_client_header():
|
||||
assert response.json() == [
|
||||
["Host", "example.org"],
|
||||
["Accept", "*/*"],
|
||||
["Accept-Encoding", "gzip, deflate, br"],
|
||||
["Accept-Encoding", "gzip, deflate, br, zstd"],
|
||||
["Connection", "keep-alive"],
|
||||
["User-Agent", f"python-httpx/{httpx.__version__}"],
|
||||
["Example-Header", "example-value"],
|
||||
|
||||
@ -36,7 +36,7 @@ def test_event_hooks():
|
||||
"host": "127.0.0.1:8000",
|
||||
"user-agent": f"python-httpx/{httpx.__version__}",
|
||||
"accept": "*/*",
|
||||
"accept-encoding": "gzip, deflate, br",
|
||||
"accept-encoding": "gzip, deflate, br, zstd",
|
||||
"connection": "keep-alive",
|
||||
"authorization": "Basic dXNlcm5hbWU6cGFzc3dvcmQ=",
|
||||
},
|
||||
@ -87,7 +87,7 @@ async def test_async_event_hooks():
|
||||
"host": "127.0.0.1:8000",
|
||||
"user-agent": f"python-httpx/{httpx.__version__}",
|
||||
"accept": "*/*",
|
||||
"accept-encoding": "gzip, deflate, br",
|
||||
"accept-encoding": "gzip, deflate, br, zstd",
|
||||
"connection": "keep-alive",
|
||||
"authorization": "Basic dXNlcm5hbWU6cGFzc3dvcmQ=",
|
||||
},
|
||||
@ -144,7 +144,7 @@ def test_event_hooks_with_redirect():
|
||||
"host": "127.0.0.1:8000",
|
||||
"user-agent": f"python-httpx/{httpx.__version__}",
|
||||
"accept": "*/*",
|
||||
"accept-encoding": "gzip, deflate, br",
|
||||
"accept-encoding": "gzip, deflate, br, zstd",
|
||||
"connection": "keep-alive",
|
||||
"authorization": "Basic dXNlcm5hbWU6cGFzc3dvcmQ=",
|
||||
},
|
||||
@ -159,7 +159,7 @@ def test_event_hooks_with_redirect():
|
||||
"host": "127.0.0.1:8000",
|
||||
"user-agent": f"python-httpx/{httpx.__version__}",
|
||||
"accept": "*/*",
|
||||
"accept-encoding": "gzip, deflate, br",
|
||||
"accept-encoding": "gzip, deflate, br, zstd",
|
||||
"connection": "keep-alive",
|
||||
"authorization": "Basic dXNlcm5hbWU6cGFzc3dvcmQ=",
|
||||
},
|
||||
@ -201,7 +201,7 @@ async def test_async_event_hooks_with_redirect():
|
||||
"host": "127.0.0.1:8000",
|
||||
"user-agent": f"python-httpx/{httpx.__version__}",
|
||||
"accept": "*/*",
|
||||
"accept-encoding": "gzip, deflate, br",
|
||||
"accept-encoding": "gzip, deflate, br, zstd",
|
||||
"connection": "keep-alive",
|
||||
"authorization": "Basic dXNlcm5hbWU6cGFzc3dvcmQ=",
|
||||
},
|
||||
@ -216,7 +216,7 @@ async def test_async_event_hooks_with_redirect():
|
||||
"host": "127.0.0.1:8000",
|
||||
"user-agent": f"python-httpx/{httpx.__version__}",
|
||||
"accept": "*/*",
|
||||
"accept-encoding": "gzip, deflate, br",
|
||||
"accept-encoding": "gzip, deflate, br, zstd",
|
||||
"connection": "keep-alive",
|
||||
"authorization": "Basic dXNlcm5hbWU6cGFzc3dvcmQ=",
|
||||
},
|
||||
|
||||
@ -34,7 +34,7 @@ def test_client_header():
|
||||
assert response.json() == {
|
||||
"headers": {
|
||||
"accept": "*/*",
|
||||
"accept-encoding": "gzip, deflate, br",
|
||||
"accept-encoding": "gzip, deflate, br, zstd",
|
||||
"connection": "keep-alive",
|
||||
"example-header": "example-value",
|
||||
"host": "example.org",
|
||||
@ -56,7 +56,7 @@ def test_header_merge():
|
||||
assert response.json() == {
|
||||
"headers": {
|
||||
"accept": "*/*",
|
||||
"accept-encoding": "gzip, deflate, br",
|
||||
"accept-encoding": "gzip, deflate, br, zstd",
|
||||
"connection": "keep-alive",
|
||||
"host": "example.org",
|
||||
"user-agent": "python-myclient/0.2.1",
|
||||
@ -78,7 +78,7 @@ def test_header_merge_conflicting_headers():
|
||||
assert response.json() == {
|
||||
"headers": {
|
||||
"accept": "*/*",
|
||||
"accept-encoding": "gzip, deflate, br",
|
||||
"accept-encoding": "gzip, deflate, br, zstd",
|
||||
"connection": "keep-alive",
|
||||
"host": "example.org",
|
||||
"user-agent": f"python-httpx/{httpx.__version__}",
|
||||
@ -100,7 +100,7 @@ def test_header_update():
|
||||
assert first_response.json() == {
|
||||
"headers": {
|
||||
"accept": "*/*",
|
||||
"accept-encoding": "gzip, deflate, br",
|
||||
"accept-encoding": "gzip, deflate, br, zstd",
|
||||
"connection": "keep-alive",
|
||||
"host": "example.org",
|
||||
"user-agent": f"python-httpx/{httpx.__version__}",
|
||||
@ -111,7 +111,7 @@ def test_header_update():
|
||||
assert second_response.json() == {
|
||||
"headers": {
|
||||
"accept": "*/*",
|
||||
"accept-encoding": "gzip, deflate, br",
|
||||
"accept-encoding": "gzip, deflate, br, zstd",
|
||||
"another-header": "AThing",
|
||||
"connection": "keep-alive",
|
||||
"host": "example.org",
|
||||
@ -164,7 +164,7 @@ def test_remove_default_header():
|
||||
assert response.json() == {
|
||||
"headers": {
|
||||
"accept": "*/*",
|
||||
"accept-encoding": "gzip, deflate, br",
|
||||
"accept-encoding": "gzip, deflate, br, zstd",
|
||||
"connection": "keep-alive",
|
||||
"host": "example.org",
|
||||
}
|
||||
@ -192,7 +192,7 @@ def test_host_with_auth_and_port_in_url():
|
||||
assert response.json() == {
|
||||
"headers": {
|
||||
"accept": "*/*",
|
||||
"accept-encoding": "gzip, deflate, br",
|
||||
"accept-encoding": "gzip, deflate, br, zstd",
|
||||
"connection": "keep-alive",
|
||||
"host": "example.org",
|
||||
"user-agent": f"python-httpx/{httpx.__version__}",
|
||||
@ -215,7 +215,7 @@ def test_host_with_non_default_port_in_url():
|
||||
assert response.json() == {
|
||||
"headers": {
|
||||
"accept": "*/*",
|
||||
"accept-encoding": "gzip, deflate, br",
|
||||
"accept-encoding": "gzip, deflate, br, zstd",
|
||||
"connection": "keep-alive",
|
||||
"host": "example.org:123",
|
||||
"user-agent": f"python-httpx/{httpx.__version__}",
|
||||
|
||||
@ -236,7 +236,7 @@ class TestServer(Server):
|
||||
def install_signal_handlers(self) -> None:
|
||||
# Disable the default installation of handlers for signals such as SIGTERM,
|
||||
# because it can only be done in the main thread.
|
||||
pass
|
||||
pass # pragma: nocover
|
||||
|
||||
async def serve(self, sockets=None):
|
||||
self.restart_requested = asyncio.Event()
|
||||
|
||||
@ -157,7 +157,7 @@ async def test_asgi_headers():
|
||||
"headers": [
|
||||
["host", "www.example.org"],
|
||||
["accept", "*/*"],
|
||||
["accept-encoding", "gzip, deflate, br"],
|
||||
["accept-encoding", "gzip, deflate, br, zstd"],
|
||||
["connection", "keep-alive"],
|
||||
["user-agent", f"python-httpx/{httpx.__version__}"],
|
||||
]
|
||||
|
||||
@ -3,6 +3,7 @@ Unit tests for auth classes.
|
||||
|
||||
Integration tests also exist in tests/client/test_auth.py
|
||||
"""
|
||||
|
||||
from urllib.request import parse_keqv_list
|
||||
|
||||
import pytest
|
||||
|
||||
@ -1,10 +1,12 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import typing
|
||||
import zlib
|
||||
|
||||
import chardet
|
||||
import pytest
|
||||
import zstandard as zstd
|
||||
|
||||
import httpx
|
||||
|
||||
@ -73,6 +75,53 @@ def test_brotli():
|
||||
assert response.content == body
|
||||
|
||||
|
||||
def test_zstd():
|
||||
body = b"test 123"
|
||||
compressed_body = zstd.compress(body)
|
||||
|
||||
headers = [(b"Content-Encoding", b"zstd")]
|
||||
response = httpx.Response(
|
||||
200,
|
||||
headers=headers,
|
||||
content=compressed_body,
|
||||
)
|
||||
assert response.content == body
|
||||
|
||||
|
||||
def test_zstd_decoding_error():
|
||||
compressed_body = "this_is_not_zstd_compressed_data"
|
||||
|
||||
headers = [(b"Content-Encoding", b"zstd")]
|
||||
with pytest.raises(httpx.DecodingError):
|
||||
httpx.Response(
|
||||
200,
|
||||
headers=headers,
|
||||
content=compressed_body,
|
||||
)
|
||||
|
||||
|
||||
def test_zstd_multiframe():
|
||||
# test inspired by urllib3 test suite
|
||||
data = (
|
||||
# Zstandard frame
|
||||
zstd.compress(b"foo")
|
||||
# skippable frame (must be ignored)
|
||||
+ bytes.fromhex(
|
||||
"50 2A 4D 18" # Magic_Number (little-endian)
|
||||
"07 00 00 00" # Frame_Size (little-endian)
|
||||
"00 00 00 00 00 00 00" # User_Data
|
||||
)
|
||||
# Zstandard frame
|
||||
+ zstd.compress(b"bar")
|
||||
)
|
||||
compressed_body = io.BytesIO(data)
|
||||
|
||||
headers = [(b"Content-Encoding", b"zstd")]
|
||||
response = httpx.Response(200, headers=headers, content=compressed_body)
|
||||
response.read()
|
||||
assert response.content == b"foobar"
|
||||
|
||||
|
||||
def test_multi():
|
||||
body = b"test 123"
|
||||
|
||||
|
||||
@ -129,7 +129,7 @@ def test_verbose(server):
|
||||
"GET / HTTP/1.1",
|
||||
f"Host: {server.url.netloc.decode('ascii')}",
|
||||
"Accept: */*",
|
||||
"Accept-Encoding: gzip, deflate, br",
|
||||
"Accept-Encoding: gzip, deflate, br, zstd",
|
||||
"Connection: keep-alive",
|
||||
f"User-Agent: python-httpx/{httpx.__version__}",
|
||||
"",
|
||||
@ -154,7 +154,7 @@ def test_auth(server):
|
||||
"GET / HTTP/1.1",
|
||||
f"Host: {server.url.netloc.decode('ascii')}",
|
||||
"Accept: */*",
|
||||
"Accept-Encoding: gzip, deflate, br",
|
||||
"Accept-Encoding: gzip, deflate, br, zstd",
|
||||
"Connection: keep-alive",
|
||||
f"User-Agent: python-httpx/{httpx.__version__}",
|
||||
"Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=",
|
||||
|
||||
@ -462,8 +462,8 @@ class TestHeaderParamHTML5Formatting:
|
||||
assert expected in request.read()
|
||||
|
||||
def test_unicode_with_control_character(self):
|
||||
filename = "hello\x1A\x1B\x1C"
|
||||
expected = b'filename="hello%1A\x1B%1C"'
|
||||
filename = "hello\x1a\x1b\x1c"
|
||||
expected = b'filename="hello%1A\x1b%1C"'
|
||||
files = {"upload": (filename, b"<file content>")}
|
||||
request = httpx.Request("GET", "https://www.example.com", files=files)
|
||||
assert expected in request.read()
|
||||
|
||||
@ -42,3 +42,14 @@ async def test_pool_timeout(server):
|
||||
with pytest.raises(httpx.PoolTimeout):
|
||||
async with client.stream("GET", server.url):
|
||||
await client.get(server.url)
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_async_client_new_request_send_timeout(server):
|
||||
timeout = httpx.Timeout(1e-6)
|
||||
|
||||
async with httpx.AsyncClient(timeout=timeout) as client:
|
||||
with pytest.raises(httpx.TimeoutException):
|
||||
await client.send(
|
||||
httpx.Request("GET", server.url.copy_with(path="/slow_response"))
|
||||
)
|
||||
|
||||
@ -11,8 +11,6 @@ from httpx._utils import (
|
||||
URLPattern,
|
||||
get_ca_bundle_from_env,
|
||||
get_environment_proxies,
|
||||
is_https_redirect,
|
||||
same_origin,
|
||||
)
|
||||
|
||||
from .common import TESTS_DIR
|
||||
@ -225,33 +223,59 @@ def test_obfuscate_sensitive_headers(headers, output):
|
||||
|
||||
|
||||
def test_same_origin():
|
||||
origin1 = httpx.URL("https://example.com")
|
||||
origin2 = httpx.URL("HTTPS://EXAMPLE.COM:443")
|
||||
assert same_origin(origin1, origin2)
|
||||
origin = httpx.URL("https://example.com")
|
||||
request = httpx.Request("GET", "HTTPS://EXAMPLE.COM:443")
|
||||
|
||||
client = httpx.Client()
|
||||
headers = client._redirect_headers(request, origin, "GET")
|
||||
|
||||
assert headers["Host"] == request.url.netloc.decode("ascii")
|
||||
|
||||
|
||||
def test_not_same_origin():
|
||||
origin1 = httpx.URL("https://example.com")
|
||||
origin2 = httpx.URL("HTTP://EXAMPLE.COM")
|
||||
assert not same_origin(origin1, origin2)
|
||||
origin = httpx.URL("https://example.com")
|
||||
request = httpx.Request("GET", "HTTP://EXAMPLE.COM:80")
|
||||
|
||||
client = httpx.Client()
|
||||
headers = client._redirect_headers(request, origin, "GET")
|
||||
|
||||
assert headers["Host"] == origin.netloc.decode("ascii")
|
||||
|
||||
|
||||
def test_is_https_redirect():
|
||||
url = httpx.URL("http://example.com")
|
||||
location = httpx.URL("https://example.com")
|
||||
assert is_https_redirect(url, location)
|
||||
url = httpx.URL("https://example.com")
|
||||
request = httpx.Request(
|
||||
"GET", "http://example.com", headers={"Authorization": "empty"}
|
||||
)
|
||||
|
||||
client = httpx.Client()
|
||||
headers = client._redirect_headers(request, url, "GET")
|
||||
|
||||
assert "Authorization" in headers
|
||||
|
||||
|
||||
def test_is_not_https_redirect():
|
||||
url = httpx.URL("http://example.com")
|
||||
location = httpx.URL("https://www.example.com")
|
||||
assert not is_https_redirect(url, location)
|
||||
url = httpx.URL("https://www.example.com")
|
||||
request = httpx.Request(
|
||||
"GET", "http://example.com", headers={"Authorization": "empty"}
|
||||
)
|
||||
|
||||
client = httpx.Client()
|
||||
headers = client._redirect_headers(request, url, "GET")
|
||||
|
||||
assert "Authorization" not in headers
|
||||
|
||||
|
||||
def test_is_not_https_redirect_if_not_default_ports():
|
||||
url = httpx.URL("http://example.com:9999")
|
||||
location = httpx.URL("https://example.com:1337")
|
||||
assert not is_https_redirect(url, location)
|
||||
url = httpx.URL("https://example.com:1337")
|
||||
request = httpx.Request(
|
||||
"GET", "http://example.com:9999", headers={"Authorization": "empty"}
|
||||
)
|
||||
|
||||
client = httpx.Client()
|
||||
headers = client._redirect_headers(request, url, "GET")
|
||||
|
||||
assert "Authorization" not in headers
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
|
||||
Loading…
Reference in New Issue
Block a user