Tcl Source Code

This report holds the patch implementing TIP #59.
See http://www.purl.org/tcl/tip/59.html for the
contents of the TIP.

Beyond holding the patch itself I will use this place 
to record how the implementation deviates from the
specification in the TIP (born from experience during 
the creation of the implementation).

Logged In: YES 
user_id=75003

This has been merged into the HEAD now.

Logged In: YES 
user_id=7549

Does this need to be assigned to me?

Logged In: YES 
user_id=75003

Not really. During compilation we don't have tclsh and 
[file normalize]. And when the information is burnt into 
the binary library we don't know anymore that the 
information is a path we should normalize.

Logged In: YES 
user_id=80530

TIP 17 introduced [file normalize].  Does that help?

Logged In: YES 
user_id=75003

Changes committed.

David, please check that the changes to makefile.vc work.
They did not for me, see below, and I don't know how to
find this problem.

makefile.vc(717) : fatal error U1020: end-of-file found 
before next directive
Stop.

Logged In: YES 
user_id=75003

I have decided to remove the prefix/exec_prefix stuff and 
use the individual paths instead, i.e. libdir, bindir, 
includedir, scriptdir, docdir, for both runtime and install.

A problematic thing is that sometimes a path may look 
like "/home/andreask/sf-dev/tip59/b/../i/lib/tcl8.4".

To avoid this we either have to specify absolute paths to 
configure, or perform a processing step before transfering 
the information into the .c file. Note that this processing 
can be done only lexically, i.e. with sed. As the paths are 
created only during installation (or may not even exist 
from the view of the installer) it is not possible to 
perform "cd path; x=`pwd`; cd back".

Example output
[andreask@pliers b]$ ./tclsh ../query
Querying embedded configuration
_______________________________
debug   = 1
threaded        = 0
profiled        = 0
64bit   = 0
optimized       = 0
mem_debug       = 0
compile_debug   = 0
compile_stats   = 0
libdir,runtime  = /home/andreask/sf-dev/tip59/b/../i/lib
bindir,runtime  = /home/andreask/sf-dev/tip59/b/../i/bin
scriptdir,runtime       = /home/andreask/sf-
dev/tip59/b/../i/lib/tcl8.4
includedir,runtime      = /home/andreask/sf-
dev/tip59/b/../i/include
docdir,runtime  = /home/andreask/sf-dev/tip59/b/../i/man
libdir,install  = /home/andreask/sf-dev/tip59/b/../i/lib
bindir,install  = /home/andreask/sf-dev/tip59/b/../i/bin
scriptdir,install       = /home/andreask/sf-
dev/tip59/b/../i/lib/tcl8.4
includedir,install      = /home/andreask/sf-
dev/tip59/b/../i/include
docdir,install  = /home/andreask/sf-dev/tip59/b/../i/man
_______________________________

Logged In: YES 
user_id=75003

Thanks for the fix David. I appreciate it. Regarding what 
we can store in there go to the Tcl-Core archives or the 
TIP and look at the older versions. I trimmed the number of 
items stored down a lot to avoid a compiler specifics etc. 
in this first cut. Yes, I plan to extend this in the future.

Logged In: YES 
user_id=79902

I have no idea about whether anything in makefile.vc works at all; I've got 98 on this machine, but no compiler (it isn't my 
dev machine, which is a Solaris/SPARC box. This machine is for other people to do things like games, scanning, a bit of 
word-processing and spreadsheet stuff, etc. That it can let me connect to SF or code Tcl is great, but not part of its core 
activities.)  IOW, I can run batch files for people or do simple command-line stuff, but only with what comes "out of the 
box".  The only advanced tools it has (apart from Tcl) are CVS, Emacs and PuTTY. :^)

Logged In: YES 
user_id=7549

I committed a small change to tclConfig.c to handle 
existing namespaces and not cause a panic.  I don't mean to 
be disrespectful by committing changes, but branches are 
play spaces and are easy for sharing.

I played with adding this to an extension, and noticed the 
panic for this sequence:

 Tcl_CreateObjCommand(...);
 Tcl_CreateObjCommand(interp, "winutils::launch", 
LaunchObjCmd, 0L, 0L);
 Tcl_CreateObjCommand(interp, "winutils::registerservice", 
RegServiceObjCmd, 0L, 0L);
 Tcl_PkgProvide(interp, "winutils", "0.5");

#if HAVE_TIP59
 Tcl_RegisterConfig(interp, "winutils", cfg, "cp1252");
#endif

a screen full of success:
% load winutils05
% winutils::pkgconfig list
debug threaded profiled 64bit optimized mem_debug 
compile_debug compile_stats
% winutils::pkgconfig get threaded
1
% winutils::pkgconfig get debug
1
% winutils::pkgconfig get 64bit
0
%

Some ideas come to mind like the name of the stubs library 
and the filename of the public include file should the 
extension be providing an API.  maybe the man page or 
helpfile name, too.  URLs, too?  CVS repository?  What else 
could be stored there?

fun stuff.  I feel I'm on the cutting edge.

Donal,  I was trusting someone's conclusion of find.exe.  
Does makefile.vc on the HEAD have problems specifying stuff 
in the OPTS macro?  Someone told me it was hosed for 
find.exe being broke.

Logged In: YES 
user_id=79902

I'm not quite sure what is meant by "find.exe not working correctly on 9x"; included below is a sample.

  C:\WINDOWS>ver
  
  Windows 98 [Version 4.10.2222]
  
  C:\WINDOWS>echo foo | find "foo"
  foo
  
  C:\WINDOWS>if errorlevel 1 echo hi
  
  C:\WINDOWS>echo foo | find "bar"
  
  C:\WINDOWS>if errorlevel 1 echo hi
  hi

Still, problems wouldn't surprise me (I only develop Tcl scripts on this platform, not build systems... :^)

Logged In: YES 
user_id=80530

As we discussed in the chat, Mo has been working to get
Tcl to honor the --libdir= and --includedir= options
to configure.  When that change is made, knowing the
two paths, prefix and exec_prefix, will not be enough
to determine where important parts of Tcl are installed.
You should consider adding these other paths to the
information stored by Tcl_RegisterConfig.

It seems to me one needs --includedir to find header
files for compiling, --linkdir to find libraries for
linking and maybe tclConfig.sh, and finally tcl_library
to know where to install the non-binary parts of
packages.  With those things, you may not need exec_prefix
and prefix at all.

There's probably a --mandir to contend with in here
somewhere too.

Logged In: YES 
user_id=7549

lucky you.  I'm trying my best to limit makefile.vc to only 
the tools that came on the VC++ CD-ROM.  very painful :)

find.exe exists on Win9x, but just doesn't work.  I'm 
tempted to have rules.vc build a text search tool to make 
up for the limited environment.

There's a speed problem with nmake, too.  On a fast 
computer, file writes aren't flushed immediatly, so 
timestamps can get skewed such as creating a directory, 
then relying on it being there sometimes causes an error.  
If I breakdown and write text search tool, I'll add stuff 
for the skew problem to it, too.

PS. I like GNU make very much.

Logged In: YES 
user_id=75003

Ok, I understand your edit/continue now :)
Regarding the check, 'find not working correctly' means: 
Does not exist ? Crashes ? ... Actually not relevant: 
win/configure assumes a cygwin environment, which means I 
can use 'grep' instead. Thanks.

Logged In: YES 
user_id=7549

The optimizing test is simple:

$(cc32) -nologo -Ox -c -Zs -TC -Fdtemp nul 2>&1 | 
find "D4002"

If "D4002" is found in stderr, then the VC++ compiler 
doesn't support optimization which can be the case for the 
low cost VC++ packages like the standard and student ones.  
I hear find.exe doesn't work correctly on Win9x, but can't 
verify that here.

Logged In: YES 
user_id=7549

Being an over user of step-debugging <G>, I really like.  
Basically, it's compiling with -Zi instead of -Z7.  It's 
rather tied into the IDE, though, and just building with -
Zi doesn't mean the IDE can do an in-place 
rebuild/reload/remap without using a .dsp for the build 
instructions.  I'll need to look into it and see.

Logged In: YES 
user_id=75003

The optimization changes are fine with me. For 
unix/configure we have either optimization on (-O2) or 
symbols. The win/configure code does it similarly. I 
believe that we should have a look and add the check you 
did for VC there too.

edit/continue support ?

Logged In: YES 
user_id=7549

committed some minor changes to makefile.vc and rules.vc

Just because symbols are turned off, doesn't mean the 
compiler is optimizing.  I added a compiler test to make 
sure it supports optimizing and sets the flag 
appropriately.  I feel it's a more accurate way to go.  I 
also added the pentium errata switches which supposedly 
fixes the FDIV hardware bug.  More compiler testing could 
be added, now that I figured a good way to do it.  I'm 
thinking of adding 'edit and continue' support...

Logged In: YES 
user_id=75003

Added manpage for Tcl_RegisterConfig. This should address 
the missing documentation and also spots which had to be 
clarified.

Logged In: YES 
user_id=75003

Comitted changes which address the following topics
from the comments below:

@ Reformatted to 72 columns per line (mostly).
  Some character strings (Panic messages) are
  still longer.

@ Better panic messages.

@ Allocation of wrapper uses sizeof(Tcl_Obj*)
  instead of sizeof (char*).

@ Terminate array with either "" or NULL.

@ Changed to use ckalloc/ckfree instead of
  Tcl_Alloc/Tcl_Free.

Logged In: YES 
user_id=75003

I see your point regarding Tcl_Alloc/ckalloc now. I will 
change my code.

Logged In: YES 
user_id=80530

BTW, see Bug 497459 for a patch that improves
the documentation about ckalloc/ckfree.

Logged In: YES 
user_id=80530

Regarding ckalloc vs. Tcl_Alloc, you are correct that
memory debuggin will still be done.  The problem is that
you must use ckalloc() to get the right values of __LINE__
and __FILE__ into the debugging reports.  If you use
Tcl_Alloc, they will be "unknown" and 0.

It's not very helpful to learn that you have a memory
error at line 0 of an "unknown" file.

Logged In: YES 
user_id=75003

Any ideas on how to test this ?
Docu, yes.

INSTALL_PREFIX, ... Yes these are the one critical part I 
wanted reviewed. I'll try libdir and includedir.

Empty key / NULL. Ok, somehwere in my brain these two are 
equivalent. I will clarify.

Regarding same encoding for all values, see the third 
comment I made (counted from below).

Optimized as integer value: Is there agreement between the 
various compilers (especially cross-platform) about the 
meaning of an optimization level ? I don't believe so.

72 characters/line: Will reformat code.

Doc non-copying of array, making it static: Will do.

Alloc of (char*): I missed this one. I had char*'s first 
and then switched to Tcl_Obj*'s.

Tcl_Alloc/ckalloc: Note line 701ff, file "tclCkAlloc" where 
Tcl_Alloc etc are defined in terms of the debugging 
functions if MEM_DEBUG is set. And line 989ff in the same 
file if MEM_DEBUG is not set. IOW, both ckalloc and 
Tcl_Alloc do the same regarding memory debugging.

Better panic: Ok

Logged In: YES 
user_id=80530

Missing documentation and tests, of course.

The Makefile.in definitions of INSTALL_PREFIX,
INSTALL_EXEC_PREFIX, RUNTIME_EXEC_PREFIX do
not appear to be bulletproof.  Consider the effects
of configure options --libdir= or --includedir= or
a value of TCL_PACKAGE_PATH that consists of more
than one directory.

The TIP says the array of Tcl_Config values is
terminated with an empty key, but it appears that
a NULL character pointer is used instead.  Clarify
that when docs are written.

I note that all values are required to have the
same encoding.  I think that's OK, but it is a
limitation.

"optimized" is stored as a boolean flag, but an
integer describing various levels of optimization
may match the possibilities better.

If line wrapping at 72 columns is not part of the
Tcl Engineering Manual, it should be.

The Tcl_Config array is not copied by Tcl_RegisterConfig;
instead a pointer to it is kept, so it needs to be
documented that the caller of Tcl_RegisterConfig cannot
allow that value to go away.  The Tcl_Config array ought
to be static.

When creating the "wrap" to be used as ClientData for
the [pkgconfig] command, it makes more sense to me to
allocate space for n (Tcl_Obj *)'s rather than n (char *)'s
even though it technically doesn't make any difference.

Tcl_Alloc and Tcl_Free should not be used.  Use ckalloc
and ckfree instead so that they are included in any
memory debugging.

Probably should have a better panic message than
"This can't happen".  At least include the name of
the procedure.

Otherwise the implementation (at least the Unix parts)
appear to do what's specified in TIP 59.  As recognized
in that TIP, though, that's just one step toward what
we need.

Logged In: YES 
user_id=75003

Committed the current state of the work into the branch
"tip-59-implementation".

Logged In: YES 
user_id=75003

Can do.

Logged In: NO 

Could this be put in CVS on a branch?  I would like it if 
it could be.

Logged In: YES 
user_id=75003

Fixed bug in generic support part (Reference counting).
Did gathering of configuration information for the three
windows build systems:

- configure/Makefile    ok, tested
- makefile.vc/rules.vc  ok, untested
- makefile.bc           ok, untested

I do not have a Macintosh, so no way for me to implement 
the changes for that platform.

Changes from specification: The default encoding on windows 
was changed from "iso8859-1" to "cp1252".

This entry will now be routed to a number of maintainers 
for a first round of reviews based upon platform. I ask all 
maintainers to pay particular attention and thought to my 
comment here about the encoding for values (use 
indicriminately for all, or only for user-specified values 
like paths ?)

(1) Don Porter      - Unix
(2) David Graveraux - Windows
(3) Daniel Steffen  - Mac
(4) Jim Ingham      - MacOSX

As there is no Mac stuff I hope that Daniel and Jim have 
the time to implement this.

Assigning to Don now ...

File Deleted - 16573:

File Added - 16701: tclConfig.c

File Added - 16699: tclPkgConfig.c

File Added - 16698: 59-3.diff

Logged In: YES 
user_id=75003

Updating the patch and the tcl configuration itself.

File Added - 16585: tclPkgConfig.c

File Added - 16584: 59-2.diff

File Added - 16575: tclPkgConfig.c

Logged In: YES 
user_id=75003

Looking at the tcl specific information I notice that not 
all keys need the system encoding from --with-encoding. 
Only the paths (...prefix,...) do. Using a global encoding 
token means that I have to write the boolean values ("0", 
and "1") in all possible encodings. This is not good.

A possible solution creates another deviation from the
specification: Remove "valEncoding" from the interface 
of "Tcl_RegisterConfig" and move it into the structure 
("Tcl_Config"). Then the writer of an extension has to 
determine which values are encoded and which are
ASCII/UTF-8.

File Added - 16574: tclConfig.c

Logged In: YES 
user_id=75003

Deviations from the specification:

* Tcl_RegisterConfig creates a namespace and a
  command in it. Both operations are require an
  interpreter to modify. Thus added an interp
  argument to the function.

* The struct holding key and value is public.
  Hence it has to have the "Tcl_" prefix. Added.

File Added - 16573: 59.diff

Logged In: YES 
user_id=75003

The following files contain a partial implementation of TIP 
#59. The generic parts are mostly done, also the gathering 
of information for Unix. Windows and Mac are not adressed 
yet.