fix: httpx.Client is slow when sending large data

This commit addresses an issue where sending large data with httpx.Client was
extremely slow. The problem was that the entire data payload was being loaded
into memory and sent in a single operation.

This fix refactors the sending process for iterable bytes types to use a
chunked approach with `yield`, which significantly improves performance and
reduces memory usage for large payloads.

Signed-off-by: xiexianbin <me@xiexianbin.cn>
This commit is contained in:
xiexianbin 2025-09-07 22:22:53 +08:00
parent 3fee27838e
commit 78377fa6d7
No known key found for this signature in database
GPG Key ID: D6B71988603A67D2

View File

@ -61,7 +61,11 @@ class IteratorByteStream(SyncByteStream):
else:
# Otherwise iterate.
for part in self._stream:
yield part
# For bytes types, should use yield to send data in
# chunks. Sending a very large part at once will
# result in extremely slow transmission.
for i in range(0, len(part), self.CHUNK_SIZE):
yield part[i : i + self.CHUNK_SIZE]
class AsyncIteratorByteStream(AsyncByteStream):