Tcl Source Code

% string is word \u203f
1
% regexp \\w \u203f
0

It appears that the character UNDERTIE (\u203f) is considered a word character by "string is word", but not in the regexp engine. In total, 9 characters are missing, all characters in the Unicode category Pc (Puntuation Connector)

> Something in this collection of facts isn't right.

Indeed, in this case the documentation is not correct. In the pre-unicode world, "word" characters were [a-zA-Z0-9_]. This is indeed the alphanumeric characters and the underscore.

In the Unicode world, "word" characters are the alphanumeric characters and the "CONNECTOR_PUNCTUATION" characters (Pc). That's the underscore, but also 9 more:

UNDERTIE ‿ CHARACTER TIE ⁀ INVERTED UNDERTIE ⁔ PRESENTATION FORM FOR VERTICAL LOW LINE ︳ PRESENTATION FORM FOR VERTICAL WAVY LOW LINE ︴ DASHED LOW LINE ﹍ CENTRELINE LOW LINE ﹎ WAVY LOW LINE ﹏ FULLWIDTH LOW LINE ＿

So, the documentation should be adapted. The implementation is already doing this correctly. Done now in core-8-6-branch and up.

Tcl docs (re_syntax.n) say that the "CLASS-SHORTHAND ESCAPES"
are defined:

 \w        [[:alnum:]_] (note underscore)

By this description, \u203f ought to match
regexp {\w} only if it matches {[[:alnum:]]}.

% info patch
8.6.6
% regexp {\w} \u203f
1
% regexp {[[:alnum:]]} \u203f
0

Something in this collection of facts isn't right.

Please notify Tom Lane (user 'tgl') about this change.

Fixed in all active branches