Tcl Source Code

Non-greedy quantifiers revert to greedy when used in conjunction with branches (pipe character).

The following branch-free expression matches each letter in the test string and therefore correctly returns 14
regexp -all .+? {this is a test}

The following branched expression matches the entire string only once because the quantifier reverts to greedy and therefore incorrectly returns 1
regexp -all .+?|x {this is a test}

This breaks a lot of complex regular expressions that require non-greedy quantifiers and branches.

Tcl and postgres make use of (forks of)
the same regexp engine.

Still true for 8.6.6

But this behaviour is stated clearly in re_syntax(n) as:

  An RE consisting of two or more branches connected by the | operator prefers longest match.

BTW, PostgreSQL has same behaviour. (confirmed in PG 9.6.4)

Surely you realise that this "fix" cannot possibly apply to complex alternations. The non-greedy quantifiers within the alternation group are all broken and this functionality can never be replaced by a single quantifier on the group as a whole.

IIRC, the problem is that the "first" quantifier in the syntax tree is the alternation, which can't be made non-greedy. The fix is to use {1,1}? to force the whole RE to be non-greedy (a non-capturing paren is also needed, but that's pretty trivial).

% regexp -all (?:.+?|x){1,1}? {this is a test}
14

Note that this is different from Perl/PCRE; they use a different model for greedy/non-greedy handling that does this in a way that is more as people intuitively expect (in return for being worse in other cases).