Fix XML character references not being decoded in cell values and attributes (regression in 9.3.0)

First I wanted to thank you for your work !

What's broken

Since 9.3.0 (migration from the DOM parser to the SAX parser), XML character references and predefined entities come through literally in parsed data:

// A cell containing "Café", in a file written by openpyxl
// (openpyxl escapes all non-ASCII text as numeric character references):
readXlsxFile(...) // → "Café" instead of "Café"

Affected: any .xlsx whose XML escapes characters as references. openpyxl-generated files are the most visible case (all non-ASCII text is escaped), but files written by Excel itself are affected too wherever &, < or " appear in values — including sheet names (<sheet name="P&amp;L"/> is returned as P&amp;L).

Reading a real-world openpyxl-generated file here (54 data rows): 101 cells contain literal &#...; references on master, 0 with this fix.

Root cause (two layers)

  1. source/xml/parseXmlStream.saxen.jssaxen doesn't decode character references itself; by design it passes a decodeEntities() function to its text / openTag handlers for the consumer to call. The ontext handler ignored that argument, and attribute values were passed through raw as well.

  2. source/saxen/parser.js — the copy-pasted saxen wrapper had return Parser(options) at the top of the function body. Function declarations are hoisted, so the parser worked, but none of the top-level var initializers below the return ever ran: ENTITY_PATTERN, ENTITY_MAPPING, fromCharCode, … all stayed undefined. decodeEntities() was therefore a silent no-op even when called — s.replace(undefined, …) searches for the literal string "undefined" and matches nothing. (The other, currently unused copy parseXmlStream.saxen.code.js already has the return at the bottom; this MR mirrors it.)

Changes

  • source/saxen/parser.js — moved return Parser(options) to the bottom of the wrapper so the var initializers actually run. This also fixes NAME_CACHE, NON_WHITESPACE_OUTSIDE_ROOT_NODE and the nsUri = decodeEntities(value) path, which were all silently undefined for the same reason.
  • source/xml/parseXmlStream.saxen.js — call decodeEntities() on text nodes and on attribute values.
  • source/xml/parseXmlStream.test.js — new tests: decimal/hex numeric references, the five predefined XML entities, pass-through of entity-free text, attribute values.

Performance

decodeEntities() short-circuits (indexOf('&') === -1), so entity-free values — the overwhelming majority in real files — never touch the regex. On a 25 MB synthetic, attribute-heavy sheet (1M attributes): ~180 ms vs ~170 ms parse time (≈5% on that worst-case profile). Typical files show no measurable difference.

Tests

npm test: no new failures. (The 2 failures in parseSheetData.test.js on my machine are pre-existing on master and timezone-related — Europe/Paris DST; unrelated to this change.)

Edited by Etienne Prothon

Merge request reports

Loading