Skip to content

feat(google-api-core): add support for resumable uploads - #18352

Merged
parthea merged 81 commits into
mainfrom
feat/resumable-transfer-api-core
Sep 23, 2026
Merged

parthea merged 81 commits into
mainfrom
feat/resumable-transfer-api-core

Conversation

@parthea

@parthea parthea commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Towards b/457416314, b/556259599

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new resumable transfer library for Google APIs, implementing both synchronous (using requests) and asynchronous (using aiohttp) resumable upload sessions, supported by a sans-I/O protocol state machine and comprehensive tests. The feedback highlights several key areas for improvement: ensuring backward compatibility with Python 3.7/3.8 by replacing asyncio.to_thread with loop.run_in_executor, handling byte-type header keys in the state machine, rejecting unsupported str and dict stream types early, retrying timeouts globally in the synchronous session, and removing redundant deadline checks.

Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_state.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Outdated
@parthea parthea changed the title [DRAFT] feat: add support for resumable uploads [DRAFT] feat(google-api-core): add support for resumable uploads Sep 11, 2026
parthea and others added 9 commits September 11, 2026 21:58
…load_state.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…load_async.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…load.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@parthea parthea changed the title [DRAFT] feat(google-api-core): add support for resumable uploads feat(google-api-core): add support for resumable uploads Sep 14, 2026
@parthea
parthea marked this pull request as ready for review September 14, 2026 19:39
@parthea
parthea requested a review from a team as a code owner September 14, 2026 19:39

@daniel-sanche daniel-sanche left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still digesting this, but giving a quick first round of comments

Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Outdated

@daniel-sanche daniel-sanche left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its looking really close, but a couple more comments

Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload.py Outdated

@daniel-sanche daniel-sanche left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall LGTM, but a few final comments to consider

Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_state.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Outdated
Comment thread packages/google-api-core/google/api_core/resumable_transfer/upload_async.py Outdated
@parthea

parthea commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

All review comments have been addressed. I'm going to hold off on merging this until #18354 is green

@parthea

parthea commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

Merging. #18354 is close to green

@parthea
parthea merged commit 17d6c53 into main Sep 23, 2026
111 checks passed
@parthea
parthea deleted the feat/resumable-transfer-api-core branch September 23, 2026 20:19
@release-please release-please Bot mentioned this pull request Sep 23, 2026
sofisl pushed a commit that referenced this pull request Sep 24, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>gapic-generator: 1.40.0</summary>

##
[1.40.0](gapic-generator-v1.39.0...gapic-generator-v1.40.0)
(2026-09-24)


### Features

* add 3.15 support to gapic library classifiers
([#18450](#18450))
([5fe1ef3](5fe1ef3))


### Bug Fixes

* **bazel:** allow CustomProtoInfo in gapic_compat_proto_library
([#18417](#18417))
([11bdf92](11bdf92))
</details>

<details><summary>google-ads-marketingplatform-admin: 0.6.1</summary>

##
[0.6.1](google-ads-marketingplatform-admin-v0.6.0...google-ads-marketingplatform-admin-v0.6.1)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-analytics-data: 0.23.2</summary>

##
[0.23.2](google-analytics-data-v0.23.1...google-analytics-data-v0.23.2)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-api-core: 2.39.0</summary>

##
[2.39.0](google-api-core-v2.38.0...google-api-core-v2.39.0)
(2026-09-24)


### Features

* **google-api-core:** add support for resumable uploads
([#18352](#18352))
([17d6c53](17d6c53))
</details>

<details><summary>google-apps-chat: 0.10.6</summary>

##
[0.10.6](google-apps-chat-v0.10.5...google-apps-chat-v0.10.6)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-cloud-agentregistry: 0.1.1</summary>

##
[0.1.1](google-cloud-agentregistry-v0.1.0...google-cloud-agentregistry-v0.1.1)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-cloud-bigtable: 2.47.0</summary>

##
[2.47.0](google-cloud-bigtable-v2.46.0...google-cloud-bigtable-v2.47.0)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-cloud-firestore: 2.32.0</summary>

##
[2.32.0](google-cloud-firestore-v2.31.0...google-cloud-firestore-v2.32.0)
(2026-09-24)


### Features

* **firestore:** add BSONBinary support
([#18389](#18389))
([85556ad](85556ad))
* **firestore:** add BSONDecimal128 support
([#18395](#18395))
([a04c831](a04c831))
* **firestore:** add BSONRegex support
([#18394](#18394))
([322c9bc](322c9bc))
* **firestore:** add BSONTimestamp support
([#18390](#18390))
([a1b2d63](a1b2d63))
</details>

<details><summary>google-cloud-geminidataanalytics: 0.13.4</summary>

##
[0.13.4](google-cloud-geminidataanalytics-v0.13.3...google-cloud-geminidataanalytics-v0.13.4)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-cloud-lustre: 0.4.2</summary>

##
[0.4.2](google-cloud-lustre-v0.4.1...google-cloud-lustre-v0.4.2)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-cloud-monitoring-dashboards: 3.1.0</summary>

##
[3.1.0](google-cloud-monitoring-dashboards-v3.0.0...google-cloud-monitoring-dashboards-v3.1.0)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-cloud-ndb: 2.6.1</summary>

##
[2.6.1](google-cloud-ndb-v2.6.0...google-cloud-ndb-v2.6.1)
(2026-09-24)


### Bug Fixes

* **google-cloud-ndb:** allow redis 8.x
([#18438](#18438))
([9630fe4](9630fe4))
</details>

<details><summary>google-cloud-parametermanager: 0.4.2</summary>

##
[0.4.2](google-cloud-parametermanager-v0.4.1...google-cloud-parametermanager-v0.4.2)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-cloud-storage: 3.15.0</summary>

##
[3.15.0](google-cloud-storage-v3.14.1...google-cloud-storage-v3.15.0)
(2026-09-24)


### Features

* **storage:** support storage_class in AsyncAppendableObjectWriter
([#18289](#18289))
([dc30df2](dc30df2))
* **storage:** support storage_class via Blob in
AsyncAppendableObjectWriter
([#18318](#18318))
([b0872f2](b0872f2))
</details>

<details><summary>google-cloud-tasks: 2.25.0</summary>

##
[2.25.0](google-cloud-tasks-v2.24.0...google-cloud-tasks-v2.25.0)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>google-cloud-vectorsearch: 0.11.4</summary>

##
[0.11.4](google-cloud-vectorsearch-v0.11.3...google-cloud-vectorsearch-v0.11.4)
(2026-09-24)


### Features

* update API sources and regenerate
([#18446](#18446))
([54f1019](54f1019))
</details>

<details><summary>googleapis-common-protos: 1.75.4</summary>

##
[1.75.4](googleapis-common-protos-v1.75.3...googleapis-common-protos-v1.75.4)
(2026-09-24)


### Bug Fixes

* regenerate gencode with protobuf 6.33.5
([#18461](#18461))
([9cb21d0](9cb21d0))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants