Skip to content

fix(upload): send the zip upload's requiredHeaders on the signed PUT - #561

Open
imbgar-roboflow wants to merge 2 commits into
mainfrom
feature/zip-upload-required-headers
Open

imbgar-roboflow wants to merge 2 commits into
mainfrom
feature/zip-upload-required-headers

Conversation

@imbgar-roboflow

Copy link
Copy Markdown
Contributor

init_zip_upload returns requiredHeaders beside signedUrl, but upload_zip_to_signed_url PUT only Content-Type: application/zip.

On a Roboflow appliance backed by Azure Blob, the signed URL is a SAS that validates only with x-ms-blob-type: BlockBlob, plus If-None-Match: * when the SAS is create-only (srh=if-none-match). The API already returns both in requiredHeaders. Without them, every SDK zip upload fails with 400 MissingRequiredHeader x-ms-blob-type. This was observed on the azure-scus lab estate on 2026-10-19.

Change:

  • upload_zip_to_signed_url(signed_url, zip_path, required_headers=None) merges the headers into the PUT, the same way put_video_upload already does.
  • Workspace.upload_dataset passes init.get("requiredHeaders") or {}.
  • Hosted GCS responses carry no requiredHeaders, so the request is unchanged there.

Tests: new TestUploadZipToSignedUrl (headers sent, and Content-Type only when there are none), and TestZipUpload asserts the headers are forwarded. python -m unittest tests.adapters.test_rfapi_phase2 tests.test_project passes 127 tests, and ruff is clean.

init_zip_upload returns requiredHeaders beside signedUrl, and
upload_zip_to_signed_url dropped them. On a Roboflow appliance backed by
Azure Blob the signed URL is a SAS that only validates with
x-ms-blob-type: BlockBlob (and If-None-Match: * when it is create-only),
so every SDK zip upload failed 400 MissingRequiredHeader. Pass them
through, the same way put_video_upload already does. GCS URLs return no
requiredHeaders and send exactly what they sent before.
@imbgar-roboflow

Copy link
Copy Markdown
Contributor Author

/mina deepbulk

@imbgar-roboflow

Copy link
Copy Markdown
Contributor Author

/jarbas review

@imbgar-roboflow

Copy link
Copy Markdown
Contributor Author

/mina deepbulk

@imbgar-roboflow

Copy link
Copy Markdown
Contributor Author

/jarbas review

mina-roboflow[bot]

This comment was marked as outdated.

@imbgar-roboflow

Copy link
Copy Markdown
Contributor Author

Follow-ups from the review are addressed in 5242983:

  • TestUploadZipToSignedUrl now sits above the module's unittest.main() guard.
  • The Windows legs failed with PermissionError: Windows can't reopen a NamedTemporaryFile that's still open. The helper now writes a closed file inside a TemporaryDirectory.

On the question: yes, a server-supplied Content-Type in requiredHeaders overrides the application/zip default on purpose. A signed URL can be signed over the content type, and the server issuing it knows the value it signed. The default only fills in when the server doesn't name one.

@imbgar-roboflow

Copy link
Copy Markdown
Contributor Author

/mina deepbulk

@mina-roboflow mina-roboflow Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TL;DR

Ready with nits — 2 follow-ups.

Follow-ups

  • [deepseek-v4.1-flash] No test pins the Content-Type override precedence — tests/adapters/test_rfapi_phase2.py:939
  • [ember-1] No test pins the intentional server-Content-Type override — tests/adapters/test_rfapi_phase2.py:934

Since the last review (e0f6b3b)

  • 5 new · 0 raised again · 2 no longer raised after the code under them changed
    • no longer raised: tests/adapters/test_rfapi_phase2.py [deepseek-v4.1-flash] New zip-upload test class is defined after the module's main guard
    • no longer raised: tests/adapters/test_rfapi_phase2.py [ember-1] New TestUploadZipToSignedUrl class is defined after the module's unittest.main() guard
  • Still open from the earlier review — the code under them is unchanged:
    • [ember-1] A server-supplied Content-Type in requiredHeaders overrides the application/zip default — intended? — roboflow/adapters/rfapi.py:1056 (not raised again)
    • [deepseek-v4.1-flash] Docstring states Azure Blob SAS header requirements without a citable source — roboflow/adapters/rfapi.py:1049 (not raised again)
How this verdict was reached
  • Jury: superduper-trio (reviewer) · superduper-adversarial (cross_examiner) · superduperduper-glm (adjudicator)
  • Reviewer seats: astra · deepseek-v4.1-flash · ember-1
  • Work: 5 raised · 5 accepted · 0 refuted · 0 unverifiable · 0 blocking
  • Vendor docs: nothing in this diff needed them
  • Verifier: not required (no blocking candidates)
  • Cross-examination: astra · deepseek-v4.1-flash · qwen3.8-max (qwen3.8-max examines in ember-1's place; -kimicross keeps ember-1)
  • Bias lenses: 22 ran · 0 recovered by corrective turn · 0 failed
  • Lens flags on accepted findings (advisory; two or more lenses agreeing, never a verdict):
    • lens tests/adapters/test_rfapi_phase2.py No test pins the intentional server-Content-Type override: anchoring, availability, confirmation, duplication-blindness, false-precision, missing-context, test-tunnel-vision
    • lens roboflow/adapters/rfapi.py Fix reuses the established put_video_upload header-merge pattern and stays backward compatible: anchoring, authority-deference, blast-radius-underestimate, confirmation, false-precision, happy-path
    • lens roboflow/adapters/rfapi.py required_headers is merged without a mapping/type check: availability, blast-radius-underestimate, false-precision, framework-dogma, nit-amplification, pattern-match
    • lens tests/adapters/test_rfapi_phase2.py No test pins the Content-Type override precedence: availability, confirmation, duplication-blindness, missing-context, test-tunnel-vision
    • lens roboflow/adapters/rfapi.py Header merge keeps responses without requiredHeaders byte-identical: blast-radius-underestimate, confirmation, false-precision, happy-path
  • Candidate gaps (omission lens, verifier-checked; not findings): 2 demonstrated · 2 not demonstrated (withheld)
    • gap: No finding anchors tests/test_project.py even though the diff modifies it in two hunks, so the project-level test changes are entirely unreviewed by any engine.
    • gap: No finding flags that arbitrary server-supplied header names (e.g. Content-Length, Host, Authorization) are merged into the outgoing request without an allowlist; P4 only covers non-mapping/non-string shapes.
  • Withheld: 1 take-or-leave note (below the public evidence bar)
  • Coverage: superduper-trio reviewed 4/4 files and 5/5 hunks · superduper-adversarial independently reviewed 4/4 files and 5/5 hunks

Reviewed commit 524298341df5c2fec466d45c026d9291fc0ca5d4.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant