Skip to content

[io] Place transparent images on a white background in read_img_as_numpy - #2183

Open
MohammadHijjawi97 wants to merge 3 commits into
mindee:mainfrom
MohammadHijjawi97:fix/read-img-transparent-background
Open

MohammadHijjawi97 wants to merge 3 commits into
mindee:mainfrom
MohammadHijjawi97:fix/read-img-transparent-background

Conversation

@MohammadHijjawi97

Copy link
Copy Markdown
Contributor

read_img_as_numpy (used by DocumentFile.from_images) decodes images with cv2.IMREAD_COLOR, which drops the alpha channel and keeps the color stored under the transparent pixels, usually black. A PNG with dark text on a transparent background (logo, screenshot, exported scan) is read as an all-black page and nothing gets recognized.

Reproduction:

import numpy as np
from io import BytesIO
from PIL import Image, ImageDraw
from doctr.io import DocumentFile

img = Image.new("RGBA", (120, 40), (0, 0, 0, 0))
ImageDraw.Draw(img).text((10, 10), "Hello", fill=(0, 0, 0, 255))
buf = BytesIO()
img.save(buf, format="PNG")
page = DocumentFile.from_images(buf.getvalue())[0]
print(np.unique(page))

Before: [0] (the whole page is black)
After: the text pixels plus a white (255) background

Fix: images with transparency (checked from the PIL header, has_transparency_data) are composited over a white background, with the EXIF orientation applied as cv2.imdecode does. Every other image keeps the existing OpenCV path, and files PIL cannot open still fall back to OpenCV (same ValueError for invalid files).

Tests: extended test_read_img_as_numpy (transparent PNG -> white background, colors and BGR order kept) and test_read_img_as_numpy_exif_orientation (EXIF orientation is still applied to a transparent PNG). The first fails on main and passes with this change. ruff check, ruff format --check and mypy doctr/ pass.

Any feedback is welcome 🤗

Decoding with cv2.IMREAD_COLOR drops the alpha channel and keeps the
color stored under the transparent pixels, which is usually black. A
PNG with dark text on a transparent background was therefore read as
an all-black page. Images with transparency are now composited over
white (with the EXIF orientation still applied).
@codecov

codecov Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.74%. Comparing base (1a4a35b) to head (113390c).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2183   +/-   ##
=======================================
  Coverage   97.74%   97.74%           
=======================================
  Files         170      170           
  Lines       10932    10958   +26     
=======================================
+ Hits        10685    10711   +26     
  Misses        247      247           
Flag Coverage Δ
unittests 97.74% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@felixdittrich92 felixdittrich92 self-assigned this Oct 27, 2026
@felixdittrich92 felixdittrich92 added this to the 1.2.0 milestone Oct 27, 2026
@felixdittrich92 felixdittrich92 added type: enhancement Improvement module: io Related to doctr.io ext: tests Related to tests folder labels Oct 27, 2026

@felixdittrich92 felixdittrich92 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @MohammadHijjawi97 👍

Two small things:

  1. Skip compositing for fully opaque images. has_transparency_data is true for any RGBA image, so opaque RGBA files (screenshots, many exports) now take the slower Pillow path. On a 2500×3000 opaque RGBA PNG this went from ~50 ms to ~130–150 ms. Please return early when alpha.min() == 255. Doing the compositing in numpy instead of alpha_composite would also cut the extra image copies. With an early exit I measured ~68 ms.

  2. Keep the tensor readers consistent. read_img_as_tensor and decode_img_as_tensor in doctr/io/image/pytorch.py still use .convert("RGB"), so transparent pixels come out black there and white in read_img_as_numpy. Please share the helper between both readers.

Minor:

  • Avoid the RGB→BGR→RGB round trip when rgb_output=True.
  • In the EXIF test with a transparent image, also assert that a corner pixel is white, not just the shape.

…readers

- Return early when the alpha channel is fully opaque, and composite in
  place on the decoded array instead of building Pillow images.
- read_img_as_tensor and decode_img_as_tensor now use the same helper,
  so transparent pixels are white there too (they were black).
- Keep Pillow's RGB output as is when rgb_output=True.
@MohammadHijjawi97

Copy link
Copy Markdown
Contributor Author

Thanks for the review, all four points are addressed in 113390c (and main is merged in).

  1. Fully opaque images now return early (alpha.min() == 255), and the compositing runs in place on the decoded array instead of alpha_composite. I used OpenCV ops for the blend since a pure numpy uint16 version was about 5x slower on this size; the output is identical to the previous result. On a 2500×3000 opaque RGBA PNG I now get ~43–49 ms vs ~96–100 ms before (main: ~42 ms).
  2. read_img_as_tensor and decode_img_as_tensor share the same helper, so transparent pixels are white there too, with a new test for both readers.
  3. The RGB→BGR→RGB round trip is gone.
  4. The EXIF test now also checks that a corner pixel is white.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ext: tests Related to tests folder module: io Related to doctr.io type: enhancement Improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants