mirror of
https://github.com/h44z/wg-portal.git
synced 2026-09-03 21:36:44 +00:00
Jacopo ClarkandGitHub
a0cb7d9d39
NFC normalisation ran before control and format characters were stripped.
Removing a character can leave a base letter next to a combining mark the
earlier pass never saw as a pair, so a second call composes it:
input U+0041 U+0009 U+0300 ("A", tab, combining grave)
once -> U+0041 U+0300
twice -> U+00C0
Found by TestPropertySanitizeStringIdempotent. Category Cf characters
behave the same way.
Strip first, then normalise. Normalisation still precedes truncation
because composing changes the rune count. One side effect: invalid UTF-8
is now dropped by the strip loop instead of surviving as U+FFFD, so an
identifier containing such bytes sanitises differently than before.
Signed-off-by: clark-ja <37738506+clark-ja@users.noreply.github.com>