Repository navigation
pdf_io: write the formatted buffer in pdf_dev_begin_actualtext - #1401
Open
peisbacher wants to merge 1 commit into
Open
peisbacher wants to merge 1 commit into
peisbacher wants to merge 1 commit into
Conversation
The function formats each character of the /ActualText string into the local `buf`, but hands `work_buffer` to dev_out(). Nothing in this function writes work_buffer, so the string that reaches the PDF is whatever that shared scratch buffer happens to hold, in practice a run of NUL bytes. Upstream texk/dvipdfm-x/pdfdev.c passes `buf` here. The divergence came in with 3f16760, which rewrote several pdf_doc_add_page_content(work_buffer, len) calls into dev_out(p, buf, len) and changed only the sprintf target in this one. Because /ActualText takes precedence over the font's ToUnicode CMap in poppler, a document built with \XeTeXgenerateactualtext=1 ends up with no usable text layer at all: pdftotext returns NUL bytes for every glyph in the affected spans. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
pdf_dev_begin_actualtextformats each character into the localbuf, but handswork_buffertodev_out. That buffer is never written in this function, so the/ActualTextstring in the output consists of whateverwork_bufferhappens tohold, in practice a run of NUL bytes.
Upstream
texk/dvipdfm-x/pdfdev.cin TeX Live passesbufhere. The divergencecame in with 3f16760 ("crates/pdf_io: update for TeXLive 2021 xdvipdfmx"), which
rewrote several
pdf_doc_add_page_content(work_buffer, len)calls intodev_out(p, buf, len)and changed only the sprintf target in this one.Reproducing
The seven characters produce seven NUL bytes:
Why it is worth fixing
/ActualTexttakes precedence over the font's ToUnicode CMap in poppler, so adocument built with
\XeTeXgenerateactualtext=1has no usable text layer at all:pdftotextreturns NUL bytes for every glyph in the affected spans. Copy-paste,search, screen readers and CV/resume parsers all see nothing. Extractors that
ignore
/ActualText, such as pypdf, are unaffected, which makes this easy to miss.The flag exists precisely to correct a wrong ToUnicode mapping, so the documents
that set it are the ones that can least afford a broken text layer. In my case the
font maps U+002D, U+00AD, U+2010 and U+2011 to one glyph and xdvipdfmx picks the
highest codepoint for the reverse mapping, which put U+2011 where hyphens belonged.
Verification
Built this branch on macOS 27 (aarch64) and ran the snippet above:
All three encoding paths of the function check out:
/ActualTextbytespdftotextPhilippPhilippPhilippGröße Öl StraßeGr\xf6\xdfe,\xd6l,Stra\xdfe(PDFDocEncoding)Größe Öl StraßeC‑Level(U+2011)\xfe\xff\000C \021\000L...(UTF-16BE)C‑LevelHappy to add a regression test if you would like one, though I did not see an
existing pattern for asserting on content-stream output.