Tcl Source Code

as advised on comp.lang.tcl I issue this bug report,  happens with Expect 5.45.3  under macosx 10.10.3: 

I want to use `expect' as an interactive debugger for a very modest tcl script. running the script either via `/usr/bin/env expect -D 1' or by adding somewhere a `debug 1' to the script and loading the Expect package from within `tclsh' and sourcing the script  from within tclsh initially does the expected (...), i.e. I get the `dbg' prompt of `Expect'.  simple things (looking at variables, single-stepping, etc. work). but as soon as I issue `c' there is a short, notable delay (somewhat larger than usual runtime of the script I'd say) followed by a memory fault. 

the problem is most easily reproduced by running the script (called "breakdown" for totally unrelated reasons ;-) like
 
    expect -D 1 breakdown
    c  #(issued at the dbg prompt)

this triggers the memfault each and every time on my machine.

the script does nothing fancy. it uses 

 namespace path {::tcl::mathop} 
 package require cmdline 

otherwise it's "vanilla tcl" (making some use of `clock')

I have attached the affected script plus some small required data files read by the script as a tarfile.

Only convenient Expect sources I found were these:

https://sourceforge.net/projects/expect/files/Expect/5.45/

They segfault just as described, because the routine

print_argv(interp, argc, argv)

is not written to be able to handle the case argc==0.

Bug in expect.  No flaw in Tcl demonstrated.  Take
complaints to expect maintainers if you can find any.

FYI: in the meantime I've cloned the tcl repo and compiled version 52e60be8fa (actually the latter was easier than anticipated ;-)). I can confirm that the reported memfault does no longer show up with this tclsh incarnation.

but playing around with the debugger some more, I've found that the `w' command (intended to show current call stack) still reliably causes a memfault/crash of the running tclsh instance (with the initially attached script demonstrating the initial memfault encounter).

irrespective of the question, whether one should or should not use Expect as a debugger (I really am starting to like it for that purpose since it has some nice features like continuing to completion of current proc, e.g., setting breakpoints via glob or regex patterns, etc. -- and in case someone is not aware of this, there even is something like a manual: expect.sourceforge.net/doc/tcl-debug.ps), it would be good if the cause of the memory fault could be identified. if someone finds the time, that is....

I'd appreciate if someone having the dev trunk sources at hand could test this by executing in tclsh e.g. this:

    package require Expect;
    debug 1;
    clock scan 2015-07-12 -format %Y-%m-%d

and hit `c' at the "dbg" prompt. would save me from trying to download/compile the tcl sources ...

From the symptoms, this might be a duplicate of 

http://core.tcl.tk/tcl/tktview?name=a0ece9d6d4

would be worth testing the development trunk sources
before beating the bushes for Expect help.

Expect is a venerable, but aging, part of Tcl history. By looking at  http://expect.cvs.sourceforge.net/viewvc/expect/expect/?sortby=date , you can see the recent activity and people involved. 

Now Expect's essential added value over Tcl is the ability to wrap pseudoterminals around a keyboard-interactive tool  so that they never notice they are being automated. You're highlighting a secondary item which I wasn't aware of: a cli debugger. But if it's all you need from Expect, it might be a good idea to look for alternatives.

By "instrumented [proc]" I mean the recent thread "Run tcl script in verbose mode" on c.l.t : https://groups.google.com/d/msg/comp.lang.tcl/w_qauHmQW5E/oNObR4ei4Z0J

Note that thread also mentions [trace add execution] which is the more natural answer (though more heavyweight).

As an aside, "don't want/need any GUI" sounds like me... but if you continue along that (good) path, you may soon realize that stepping through is not really needed. Tcl's seamless stringification of any datastructure makes it trivial to dump any part of the state by a simple [puts $something]. No real need to step through when an a posteriori observation of logs says it all. Moreover, that'll work also for time-sensitive programs.

regarding Expect: do hope there still _are_ maintainers (are there?)...

regarding debugging: I'd be happy essentially with setting breakpoints plus step-through and continue in the CLI. (don't want/need any GUI)  that's what `Expect' allows so it would suit me fine. I only _very_ recently have seriously looked at Tcl so I'm not sure what can be done in vanilla Tcl instead. have to look at `trace'. don't know what "instrumented [proc]" means ...

actually, I believe a decent default ("batteries included") debugger having that functionality (set breakpoints, inspect/modify vars, step-through, continue)  would be a great plus for Tcl.  I have come to appreciate this sort of thing in R, for example (matlab has something similar). I found out that `Expect' can do this just by chance. at least it is not mentioned at a prominent place in the documenation of Tcl. so, if `Expect'  could be fixed to work with current Tcl and the docs would point out that `Expect' can act as a quite capable debugger this would seem to be a good idea.

OK, this minimization will certainly help Expect maintainers (assuming there are).
In the meantime, can you elaborate on exactly which debugging features you need ?
If you only want to set breakpoints at specific places, inspect vars, and resume, that can easily be done in vanilla Tcl (just call a proc with a read-eval-print loop from/to the terminal). If you want step-by-step and like GUIs, you might be interested by ActiveState's one. If you just want verbose execution, [trace] or even instrumented [proc] will do the job.

bingo, you're right: forget about my lengthy script and it's data files. a minimal example script need only contain the single line (e.g.):

 clock scan 2015-07-12 -format %Y-%m-%d

running this script with `expect --D 1 scriptname' and issuing the `c' command produces the memfault just fine. same if you source a script containing the three lines

package require Expect;
debug 1;
clock scan 2015-07-12 -format %Y-%m-%d

from within tclsh and then issue the `c' there. hope there is a chance that it can be fixed. it's a pity that Expect chokes on this. apart from it's "real" purpose Expect is the only (free) usable interactive tcl debugger I'm aware of. (if there are other good ones a hint would be appreciated...)

Interesting. Since "getLimits" is the only user of [clock], it may be worth a second look. Indeed, [clock] is a pretty tricky beast that dynamically creates procs on first invocation. That might interfere with some debuggers, like Expect's facility (which I don't know). To factor out this effect, just call [clock format] and [clock scan] once at the beginning of your program, passing the same format strings as you do for the real job.

I have tried to find out where approximately the problem occurs. so far I can only state:

it happens in procedure `getLimits'. if this proc is bypassed (by providing a valid value for its return argument so that the script can proceed) the memfault does not occur. it also does not occur if only a _single_ data file (not necessarily the first one) is processed. so editing the `set databases {d1 d2 d3 d4}'
line at the end of the script accordingly (e.g. `set databases d2') the memfault after issuing `c' from within the expect debugger does also not occur.

so only if  a second call to `getLimits' occurs the memfault is triggered (it also seems that in this case the memfault occurs immediately at time of the second call and nothing more is executed).

I would suspect that it has to do with the use of the `clock' command but this is only a wild guess. not sure whether this is helpful ...