Skip to content

GH-68 Fix self-referential entity expansion - #69

Merged
oalders merged 3 commits into
libwww-perl:masterfrom
pjcj:GH-68-self-entity
Sep 19, 2026
Merged

oalders merged 3 commits into
libwww-perl:masterfrom
pjcj:GH-68-self-entity

Conversation

@pjcj

@pjcj pjcj commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

An entity whose value is the string being decoded produced a result that mixed decoded output, stale bytes and undecoded input, as reported in the ticket. The result now follows the rule the routine has always followed when the entity table holds a separate copy of that string. The entity expands once, to the value the table held when the call began.

Changes

  • Copy the string before the first write and expand a self-referential entity from that copy.
    • Record the encoding flag the string had at entry, so an earlier entity that upgrades the string to UTF-8 cannot mislabel the copy.
  • Drop the buffer added for CVE-2026-8829, which is not needed once no pointer into the string survives a realloc.
  • Stop an entity name at the end of the input matching a name; key on the byte beyond the end, which holds stale content once the string has grown. This is its own commit, since the bug predates the ticket.
  • Die if fetching an entity value runs Perl code that modifies the string, since the routine holds pointers into it. Hold a reference to the string for the whole call and free the copy from the savestack, so a lookup that dies leaks nothing.
  • Document the rule for _decode_entities, which nothing stated before.

Implications

  • The two examples in the ticket give different output from the values the ticket expects. The rule suggested there builds the replacement from the text decoded so far joined to the rest of the input, so whatever has already been decoded appears a second time.
    • This branch takes the value from the table instead, which is what _decode_entities already gives when the table holds a copy of the string rather than the string itself. Those two calls look identical in Perl, so they ought to agree, and now they do.
  • The use-after-free stays fixed. The full suite runs clean under the address sanitiser with both perl and the extension instrumented, and t/entities.t still covers the payload which triggered the fault.
  • Decoding a string that expands anything costs one copy of it. On a 2 MB string holding a single entity that measures around 3%.
  • A tied or overloaded table whose code modifies the input now gets an exception instead of a crash.
  • Dying on a modified string is a guard, not a cure. Decoding into a private buffer and assigning the result at the end would remove the hazard altogether, along with grow_gap, but that is a rewrite of a hot loop and deserves its own ticket.

Ticket

Closes #68

Ticket libwww-perlGH-68

- After the string grows, the byte at its end holds whatever the buffer
  held before, so an entity name that ran to the end could match a
  "name;" key on that stale byte.
Ticket libwww-perlGH-68

- Copy the string before the first write and expand from that copy, so
  `_decode_entities` gives the same result whether the entity table
  holds the string itself or a separate copy of it.
- Drop the buffer added for CVE-2026-8829, which is not needed now that
  no pointer into the string survives a realloc.
- Die if looking up a value runs Perl code that changes the string,
  since we hold pointers into it. Hold a reference to the string for
  the duration of the call and free the copy from the savestack, so
  neither is lost when that code dies.
A tied entity table whose FETCH frees and reallocates the string being
decoded must be caught by the mutation guard rather than left to read
through a dangling pointer.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@oalders
oalders merged commit 203758d into libwww-perl:master Sep 19, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Clarification on _decode_entities

2 participants