Tcl Source Code

This script, when evaluated (note the missing 
closing square bracket):

catch {
    if {![catch [list foo]} {
    }
} err
puts $err
puts $errorInfo

gives me:



invalid command name "fo"
    while executing
"fo"
    invoked from within
"catch [list foo"

i.e. 'err' is empty, and errorInfo has somehow 
dropped the last 'o' from foo.

Both of these seem incorrect.

Logged In: YES 
user_id=80530

the bug is fixed.  Other changes
will be dealt with in other reports.

Logged In: YES 
user_id=148712

Reviewed, teste, committed.
Assigning to dgp for the [subst] fixes - or else, for
branching the subst bugs to another ticket and closing thisone

File Added - 42334: 681841-final.patch

Logged In: YES 
user_id=80530

Here's the final version of the patch.
Ambitious changes to [subst] have
been removed.  Just the bug fix, ma'am.

Please review, test, and commit.

Logged In: YES 
user_id=80530

yes, for the sake of good
patch-release compatibility,
I am separating the bug fix
patch from the "improve subst"
patch.

update soon.

Logged In: YES 
user_id=79902

Ultimately, the behaviour of [subst] is strange in so many
ways in these edge cases.  Given that, it's hard to believe
that many people are encountering them, so fixing them any
way you like (especially if it is more consistent with the
documentation of the command) will be no problem.  Perhaps
the alteration of that aspect should wait for 8.4.2 to be
branched off though, for maximum backward compatability.

Logged In: YES 
user_id=80530

I'm having second thoughts about
tests subst-10.4 and subst-10.5.

In those tests a [break] is being
used to halt substitution.  In that
case, it seems strange for a parse
error beyond the [break] to cause
an error.  Thoughts?

Logged In: YES 
user_id=79902

OK, I prefer the behaviour of [subst] forced by these
changes.  It's even less surprising given the documentation
of the command as being like substitution inside a "quoted"
string; we're actually closer to the documentation with this
alteration!  Go ahead and modify subst.test so that the
following tests do not fail:
  subst-8.6, subst-8.7, subst-8.8,
  subst-10.4, subst-10.5, subst-11.4, subst-11.5

Logged In: YES 
user_id=79902

And people wonder why Tcl_SubstObj has undefined
behaviour...

File Deleted - 42217: 



File Added - 42249: 681841.patch

Logged In: YES 
user_id=80530

another updated patch.
Now includes changes to
Tcl_EvalEx in tclBasic.c
No more code changes to make.

With this patch, 8 tests in subst.test
fail, but arguably they are testing
bugward compatibility, and the
new behavior is better.  Assigning
to the [subst] maintainer for his
opinion on that.

File Deleted - 41816: 



File Added - 42217: 681841.patch

Logged In: YES 
user_id=80530

Here's an updated patch that
takes care of updating 
tclCompile.c.

More bugs fixed, and ::errorInfo
trace should now be improved
when parsing errors occur.

Please review.

Still more work to do on Tcl_EvalEx.

Logged In: YES 
user_id=80530

Here's another demo of the original bug:

% if {1 && "" == [puts \]} {}
\

Note that a backslash is printed, not a ] !

File Deleted - 41799: 



File Added - 41816: 681841.patch

Logged In: YES 
user_id=80530

Here is an updated patch that
also includes a fix for Tcl_EvalEx().

I suspect there are still errors 
involving the setting of
iPtr->termOffset.  Not ready
to commit yet.

Logged In: YES 
user_id=80530

There is a similar error in Tcl_EvalEx()
that can be demo'd by

    subst {a[set b [format cd]}

no syntax error reported.

Logged In: YES 
user_id=148712

Patch looks good! 

Q: does it also solve the old bug you mentioned? (I never
got around to understanding what it is ...)

File Added - 41799: 681841.patch

Logged In: YES 
user_id=80530

please review this patch.

Logged In: YES 
user_id=80530

patch coming soon.

Logged In: YES 
user_id=79902

I don't know which function has responsibility for stripping the 
close bracket.  The open bracket is obviously consumed by 
ParseTokens(), but who gets the close?

Logged In: YES 
user_id=79902

Further investigation seems to indicate the problem as being 
that when parsing bracketed things, the trailing bracket is 
consumed by Tcl_ParseCommand, stopping us from telling 
whether it should have done so.  Perhaps.  Everything's so 
recursive it's hard to tell for sure...

Logged In: YES 
user_id=79902

Obviously not (or we barf on init.tcl!)  :^/

Logged In: YES 
user_id=79902

Yes, shouldn't the tests there come after the test to see if 
the end of the string has been reached?

Logged In: YES 
user_id=79902

I suspect a problem near tclParse.c:865

Logged In: YES 
user_id=148712

Debug notes RE the new error: something subtle going on, witness

% set x {set a [format bc]}
set a [format bc]
% set y [string range $x 0 15]
set a [format bc
% testparser $x 16
- {set a [format bc} 3 simple set 1 text set 0 simple a 1
text a 0 word {[format bc} 1 command {[format bc} 0 \]
% testparser $y 16
missing close-bracket

Logged In: YES 
user_id=80530

Curious.  There is a parsing error there,
but there always has been.

Before, Tcl would parse the expression
wrong, allowing the error to go unnoticed,
but then the compiler would raise an error
when trying to compile the command
substitutions in the expr.

The *new* error is that Tcl_ParseCommand
itself is buggy in handling strings that have
a ] following the string to be parsed.  See
this with

    testparser {set a [format bc]} 16

in both 8.3 and 8.4.  (length 16 does
not include the closing ].  Thus a
parse error in 8.3, but a mistaken
acceptance in 8.4, apparently due
to an effective read beyond numbytes
when parsing.)

Both errors should be fixed.

Logged In: YES 
user_id=80530

Error is in parsing nested command
evaluations in an expression.

Simpler test case:

expr {[set a [format bc]}

Miguel, error should be found in
lines 1261-1313 of tclParseExpr.c

Logged In: YES 
user_id=80530

Ouch!  Note that the return from
[catch] here is TCL_OK, not
TCL_ERROR !

Logged In: YES 
user_id=80530

Therefore my parser rewrite is likely to blame.

Logged In: YES 
user_id=148712

AIEEEEE! 

Mem corruption in the parser? Introduced between 8.4b1 and
8.4b2 ...