|Title:||TIP 59 Implementation|
|Submitter:||andreas_kupries||Created on:||2002-01-22 17:57:44|
|Subsystem:||53. Configure and Build Tools||Assigned To:||andreas_kupries|
|Status:||Closed||Last Modified:||2003-08-09 01:49:57|
|Closed on:||2003-08-08 18:49:57|
This report holds the patch implementing TIP #59. See http://www.purl.org/tcl/tip/59.html for the contents of the TIP. Beyond holding the patch itself I will use this place to record how the implementation deviates from the specification in the TIP (born from experience during the creation of the implementation).
andreas_kupries added on 2003-08-09 01:49:57:
Logged In: YES user_id=75003 This has been merged into the HEAD now.
davygrvy added on 2002-05-28 07:55:12:
Logged In: YES user_id=7549 Does this need to be assigned to me?
andreas_kupries added on 2002-02-20 05:15:58:
Logged In: YES user_id=75003 Not really. During compilation we don't have tclsh and [file normalize]. And when the information is burnt into the binary library we don't know anymore that the information is a path we should normalize.
dgp added on 2002-02-20 05:11:23:
Logged In: YES user_id=80530 TIP 17 introduced [file normalize]. Does that help?
andreas_kupries added on 2002-02-20 05:02:46:
Logged In: YES user_id=75003 Changes committed. David, please check that the changes to makefile.vc work. They did not for me, see below, and I don't know how to find this problem. makefile.vc(717) : fatal error U1020: end-of-file found before next directive Stop.
andreas_kupries added on 2002-02-20 04:44:05:
Logged In: YES user_id=75003 I have decided to remove the prefix/exec_prefix stuff and use the individual paths instead, i.e. libdir, bindir, includedir, scriptdir, docdir, for both runtime and install. A problematic thing is that sometimes a path may look like "/home/andreask/sf-dev/tip59/b/../i/lib/tcl8.4". To avoid this we either have to specify absolute paths to configure, or perform a processing step before transfering the information into the .c file. Note that this processing can be done only lexically, i.e. with sed. As the paths are created only during installation (or may not even exist from the view of the installer) it is not possible to perform "cd path; x=`pwd`; cd back". Example output [andreask@pliers b]$ ./tclsh ../query Querying embedded configuration _______________________________ debug = 1 threaded = 0 profiled = 0 64bit = 0 optimized = 0 mem_debug = 0 compile_debug = 0 compile_stats = 0 libdir,runtime = /home/andreask/sf-dev/tip59/b/../i/lib bindir,runtime = /home/andreask/sf-dev/tip59/b/../i/bin scriptdir,runtime = /home/andreask/sf- dev/tip59/b/../i/lib/tcl8.4 includedir,runtime = /home/andreask/sf- dev/tip59/b/../i/include docdir,runtime = /home/andreask/sf-dev/tip59/b/../i/man libdir,install = /home/andreask/sf-dev/tip59/b/../i/lib bindir,install = /home/andreask/sf-dev/tip59/b/../i/bin scriptdir,install = /home/andreask/sf- dev/tip59/b/../i/lib/tcl8.4 includedir,install = /home/andreask/sf- dev/tip59/b/../i/include docdir,install = /home/andreask/sf-dev/tip59/b/../i/man _______________________________
andreas_kupries added on 2002-02-06 23:57:06:
Logged In: YES user_id=75003 Thanks for the fix David. I appreciate it. Regarding what we can store in there go to the Tcl-Core archives or the TIP and look at the older versions. I trimmed the number of items stored down a lot to avoid a compiler specifics etc. in this first cut. Yes, I plan to extend this in the future.
dkf added on 2002-02-06 23:51:28:
Logged In: YES user_id=79902 I have no idea about whether anything in makefile.vc works at all; I've got 98 on this machine, but no compiler (it isn't my dev machine, which is a Solaris/SPARC box. This machine is for other people to do things like games, scanning, a bit of word-processing and spreadsheet stuff, etc. That it can let me connect to SF or code Tcl is great, but not part of its core activities.) IOW, I can run batch files for people or do simple command-line stuff, but only with what comes "out of the box". The only advanced tools it has (apart from Tcl) are CVS, Emacs and PuTTY. :^)
davygrvy added on 2002-02-06 19:37:05:
Logged In: YES user_id=7549 I committed a small change to tclConfig.c to handle existing namespaces and not cause a panic. I don't mean to be disrespectful by committing changes, but branches are play spaces and are easy for sharing. I played with adding this to an extension, and noticed the panic for this sequence: Tcl_CreateObjCommand(...); Tcl_CreateObjCommand(interp, "winutils::launch", LaunchObjCmd, 0L, 0L); Tcl_CreateObjCommand(interp, "winutils::registerservice", RegServiceObjCmd, 0L, 0L); Tcl_PkgProvide(interp, "winutils", "0.5"); #if HAVE_TIP59 Tcl_RegisterConfig(interp, "winutils", cfg, "cp1252"); #endif a screen full of success: % load winutils05 % winutils::pkgconfig list debug threaded profiled 64bit optimized mem_debug compile_debug compile_stats % winutils::pkgconfig get threaded 1 % winutils::pkgconfig get debug 1 % winutils::pkgconfig get 64bit 0 % Some ideas come to mind like the name of the stubs library and the filename of the public include file should the extension be providing an API. maybe the man page or helpfile name, too. URLs, too? CVS repository? What else could be stored there? fun stuff. I feel I'm on the cutting edge. Donal, I was trusting someone's conclusion of find.exe. Does makefile.vc on the HEAD have problems specifying stuff in the OPTS macro? Someone told me it was hosed for find.exe being broke.
dkf added on 2002-02-06 16:45:15:
Logged In: YES user_id=79902 I'm not quite sure what is meant by "find.exe not working correctly on 9x"; included below is a sample. C:\WINDOWS>ver Windows 98 [Version 4.10.2222] C:\WINDOWS>echo foo | find "foo" foo C:\WINDOWS>if errorlevel 1 echo hi C:\WINDOWS>echo foo | find "bar" C:\WINDOWS>if errorlevel 1 echo hi hi Still, problems wouldn't surprise me (I only develop Tcl scripts on this platform, not build systems... :^)
dgp added on 2002-02-06 13:12:30:
Logged In: YES user_id=80530 As we discussed in the chat, Mo has been working to get Tcl to honor the --libdir= and --includedir= options to configure. When that change is made, knowing the two paths, prefix and exec_prefix, will not be enough to determine where important parts of Tcl are installed. You should consider adding these other paths to the information stored by Tcl_RegisterConfig. It seems to me one needs --includedir to find header files for compiling, --linkdir to find libraries for linking and maybe tclConfig.sh, and finally tcl_library to know where to install the non-binary parts of packages. With those things, you may not need exec_prefix and prefix at all. There's probably a --mandir to contend with in here somewhere too.
davygrvy added on 2002-02-06 07:17:43:
Logged In: YES user_id=7549 lucky you. I'm trying my best to limit makefile.vc to only the tools that came on the VC++ CD-ROM. very painful :) find.exe exists on Win9x, but just doesn't work. I'm tempted to have rules.vc build a text search tool to make up for the limited environment. There's a speed problem with nmake, too. On a fast computer, file writes aren't flushed immediatly, so timestamps can get skewed such as creating a directory, then relying on it being there sometimes causes an error. If I breakdown and write text search tool, I'll add stuff for the skew problem to it, too. PS. I like GNU make very much.
andreas_kupries added on 2002-02-06 06:55:47:
Logged In: YES user_id=75003 Ok, I understand your edit/continue now :) Regarding the check, 'find not working correctly' means: Does not exist ? Crashes ? ... Actually not relevant: win/configure assumes a cygwin environment, which means I can use 'grep' instead. Thanks.
davygrvy added on 2002-02-06 06:50:44:
Logged In: YES user_id=7549 The optimizing test is simple: $(cc32) -nologo -Ox -c -Zs -TC -Fdtemp nul 2>&1 | find "D4002" If "D4002" is found in stderr, then the VC++ compiler doesn't support optimization which can be the case for the low cost VC++ packages like the standard and student ones. I hear find.exe doesn't work correctly on Win9x, but can't verify that here.
davygrvy added on 2002-02-06 06:39:47:
Logged In: YES user_id=7549 Being an over user of step-debugging <G>, I really like. Basically, it's compiling with -Zi instead of -Z7. It's rather tied into the IDE, though, and just building with - Zi doesn't mean the IDE can do an in-place rebuild/reload/remap without using a .dsp for the build instructions. I'll need to look into it and see.
andreas_kupries added on 2002-02-06 06:26:23:
Logged In: YES user_id=75003 The optimization changes are fine with me. For unix/configure we have either optimization on (-O2) or symbols. The win/configure code does it similarly. I believe that we should have a look and add the check you did for VC there too. edit/continue support ?
davygrvy added on 2002-02-06 06:19:02:
Logged In: YES user_id=7549 committed some minor changes to makefile.vc and rules.vc Just because symbols are turned off, doesn't mean the compiler is optimizing. I added a compiler test to make sure it supports optimizing and sets the flag appropriately. I feel it's a more accurate way to go. I also added the pentium errata switches which supposedly fixes the FDIV hardware bug. More compiler testing could be added, now that I figured a good way to do it. I'm thinking of adding 'edit and continue' support...
andreas_kupries added on 2002-02-06 05:16:19:
Logged In: YES user_id=75003 Added manpage for Tcl_RegisterConfig. This should address the missing documentation and also spots which had to be clarified.
andreas_kupries added on 2002-02-06 03:49:49:
Logged In: YES user_id=75003 Comitted changes which address the following topics from the comments below: @ Reformatted to 72 columns per line (mostly). Some character strings (Panic messages) are still longer. @ Better panic messages. @ Allocation of wrapper uses sizeof(Tcl_Obj*) instead of sizeof (char*). @ Terminate array with either "" or NULL. @ Changed to use ckalloc/ckfree instead of Tcl_Alloc/Tcl_Free.
andreas_kupries added on 2002-02-06 02:46:59:
Logged In: YES user_id=75003 I see your point regarding Tcl_Alloc/ckalloc now. I will change my code.
dgp added on 2002-02-06 02:42:22:
Logged In: YES user_id=80530 BTW, see Bug 497459 for a patch that improves the documentation about ckalloc/ckfree.
dgp added on 2002-02-06 02:41:10:
Logged In: YES user_id=80530 Regarding ckalloc vs. Tcl_Alloc, you are correct that memory debuggin will still be done. The problem is that you must use ckalloc() to get the right values of __LINE__ and __FILE__ into the debugging reports. If you use Tcl_Alloc, they will be "unknown" and 0. It's not very helpful to learn that you have a memory error at line 0 of an "unknown" file.
andreas_kupries added on 2002-02-06 01:13:32:
Logged In: YES user_id=75003 Any ideas on how to test this ? Docu, yes. INSTALL_PREFIX, ... Yes these are the one critical part I wanted reviewed. I'll try libdir and includedir. Empty key / NULL. Ok, somehwere in my brain these two are equivalent. I will clarify. Regarding same encoding for all values, see the third comment I made (counted from below). Optimized as integer value: Is there agreement between the various compilers (especially cross-platform) about the meaning of an optimization level ? I don't believe so. 72 characters/line: Will reformat code. Doc non-copying of array, making it static: Will do. Alloc of (char*): I missed this one. I had char*'s first and then switched to Tcl_Obj*'s. Tcl_Alloc/ckalloc: Note line 701ff, file "tclCkAlloc" where Tcl_Alloc etc are defined in terms of the debugging functions if MEM_DEBUG is set. And line 989ff in the same file if MEM_DEBUG is not set. IOW, both ckalloc and Tcl_Alloc do the same regarding memory debugging. Better panic: Ok
dgp added on 2002-02-05 07:57:22:
Logged In: YES user_id=80530 Missing documentation and tests, of course. The Makefile.in definitions of INSTALL_PREFIX, INSTALL_EXEC_PREFIX, RUNTIME_EXEC_PREFIX do not appear to be bulletproof. Consider the effects of configure options --libdir= or --includedir= or a value of TCL_PACKAGE_PATH that consists of more than one directory. The TIP says the array of Tcl_Config values is terminated with an empty key, but it appears that a NULL character pointer is used instead. Clarify that when docs are written. I note that all values are required to have the same encoding. I think that's OK, but it is a limitation. "optimized" is stored as a boolean flag, but an integer describing various levels of optimization may match the possibilities better. If line wrapping at 72 columns is not part of the Tcl Engineering Manual, it should be. The Tcl_Config array is not copied by Tcl_RegisterConfig; instead a pointer to it is kept, so it needs to be documented that the caller of Tcl_RegisterConfig cannot allow that value to go away. The Tcl_Config array ought to be static. When creating the "wrap" to be used as ClientData for the [pkgconfig] command, it makes more sense to me to allocate space for n (Tcl_Obj *)'s rather than n (char *)'s even though it technically doesn't make any difference. Tcl_Alloc and Tcl_Free should not be used. Use ckalloc and ckfree instead so that they are included in any memory debugging. Probably should have a better panic message than "This can't happen". At least include the name of the procedure. Otherwise the implementation (at least the Unix parts) appear to do what's specified in TIP 59. As recognized in that TIP, though, that's just one step toward what we need.
andreas_kupries added on 2002-01-25 08:48:51:
Logged In: YES user_id=75003 Committed the current state of the work into the branch "tip-59-implementation".
andreas_kupries added on 2002-01-25 06:00:31:
Logged In: YES user_id=75003 Can do.
nobody added on 2002-01-25 05:50:06:
Logged In: NO Could this be put in CVS on a branch? I would like it if it could be.
andreas_kupries added on 2002-01-25 00:44:16:
Logged In: YES user_id=75003 Fixed bug in generic support part (Reference counting). Did gathering of configuration information for the three windows build systems: - configure/Makefile ok, tested - makefile.vc/rules.vc ok, untested - makefile.bc ok, untested I do not have a Macintosh, so no way for me to implement the changes for that platform. Changes from specification: The default encoding on windows was changed from "iso8859-1" to "cp1252". This entry will now be routed to a number of maintainers for a first round of reviews based upon platform. I ask all maintainers to pay particular attention and thought to my comment here about the encoding for values (use indicriminately for all, or only for user-specified values like paths ?) (1) Don Porter - Unix (2) David Graveraux - Windows (3) Daniel Steffen - Mac (4) Jim Ingham - MacOSX As there is no Mac stuff I hope that Daniel and Jim have the time to implement this. Assigning to Don now ...
andreas_kupries added on 2002-01-25 00:33:53:
File Deleted - 16573:
andreas_kupries added on 2002-01-25 00:33:32:
File Added - 16701: tclConfig.c
andreas_kupries added on 2002-01-25 00:32:17:
File Added - 16699: tclPkgConfig.c
andreas_kupries added on 2002-01-25 00:31:42:
File Added - 16698: 59-3.diff
andreas_kupries added on 2002-01-25 00:31:41:
Logged In: YES user_id=75003 Updating the patch and the tcl configuration itself.
andreas_kupries added on 2002-01-23 03:27:27:
File Added - 16585: tclPkgConfig.c
andreas_kupries added on 2002-01-23 03:26:25:
File Added - 16584: 59-2.diff
andreas_kupries added on 2002-01-23 01:08:36:
File Added - 16575: tclPkgConfig.c
andreas_kupries added on 2002-01-23 01:08:35:
Logged In: YES user_id=75003 Looking at the tcl specific information I notice that not all keys need the system encoding from --with-encoding. Only the paths (...prefix,...) do. Using a global encoding token means that I have to write the boolean values ("0", and "1") in all possible encodings. This is not good. A possible solution creates another deviation from the specification: Remove "valEncoding" from the interface of "Tcl_RegisterConfig" and move it into the structure ("Tcl_Config"). Then the writer of an extension has to determine which values are encoded and which are ASCII/UTF-8.
andreas_kupries added on 2002-01-23 01:04:00:
File Added - 16574: tclConfig.c
andreas_kupries added on 2002-01-23 01:03:58:
Logged In: YES user_id=75003 Deviations from the specification: * Tcl_RegisterConfig creates a namespace and a command in it. Both operations are require an interpreter to modify. Thus added an interp argument to the function. * The struct holding key and value is public. Hence it has to have the "Tcl_" prefix. Added.
andreas_kupries added on 2002-01-23 01:00:50:
File Added - 16573: 59.diff
andreas_kupries added on 2002-01-23 01:00:49:
Logged In: YES user_id=75003 The following files contain a partial implementation of TIP #59. The generic parts are mostly done, also the gathering of information for Unix. Windows and Mac are not adressed yet.