TIP 59: Embed Build Information in Tcl Binary Library

Login
Bounty program for improvements to Tcl and certain Tcl packages.
Tcl 2017 Conference, Houston/TX, US, Oct 16-20
Send your abstracts to tclconference@googlegroups.com
by Aug 21.
Author:         Andreas Kupries <andreas_kupries@users.sourceforge.net>
State:          Final
Type:           Project
Vote:           Done
Created:        04-Sep-2001
Post-History:   
Tcl-Version:    8.5

Abstract

This TIP provides an interface through which Tcl may be queried for information on its own configuration, in order to extract the information directly instead of reading it from a Bourne shell file. An important reason to do this is to have the information not only available but also tightly bound to the binary configured by it, so that the information doesn't get lost.

Foreword

This TIP proposes a rather small change to Tcl and tries very hard to follow the KISS principle. Given that the casual observer might find it rather long, be assured, the actual specification in here is not very long, nor complicated. Most of the following explanations were added to preserve the KISS principle and head off attempts to extend the TIP beyond its small goal and scope.

Note: All instances of "Tcl library" in the following text refer to the generated installable library and not the script library coming with the core.

Background and Rationale

The main reason for writing this TIP are the disadvantages inherent in the current way of storing the configuration of Tcl, namely in the file tclConfig.sh.

Thus, this TIP proposes:

The file tclConfig.sh is not replaced by this system, both sets of information exist in parallel.

Neither is the variable tcl_platform replaced. This means that some information, like threaded, is held redundantly. Other information in tcl_platform, like user is runtime and not configuration information. The operating system information is important to a build system, but this is out of the scope of this TIP.

Interface Specification

Any embedded information is made accessible at the Tcl level through a new command. The name of the command used by Tcl itself is ::tcl::pkgconfig. Extensions have to use their own commands. These commands will be named pkgconfig too and have to be placed within in the namespaces owned by the extensions initializing them.

At the C-level the public API of the Tcl core is extended with a single function to register the embedded configuration information. This function is added to the public stub table of the Tcl core so that it can be used by Tcl and extensions to register their own configuration information in the system during initialization.

The function takes three (3) arguments; first, the name of the package registering its configuration information, second, a pointer to an array of structures, and third a string declaring the encoding used by the configuration values. Each element of the array refers to two strings containing the key and the value associated with that key. The end of the array is signaled by an empty key.

Formalized, name and signature of this new function are

 Tcl_RegisterConfig (CONST char* pkgName, Config* configuration, CONST char* valEncoding)

 typedef struct Config {
    char* key;
    char* value;
 }

The string valEncoding contains the name of an encoding known to Tcl. All these names are use only characters in the ASCII subset of UTF-8 and are thus implicity in the UTF-8 encoding. It is expected that keys are legible english text and therefore using the ASCII subset of UTF-8. In other words, they are expected to be in UTF-8 too. The values associated with the keys can be any string however. For these the contents of valEncoding define which encoding was used to represent the characters of the strings.

During compile time the value of valEncoding is specified as a makefile variable for non-configure based build systems and through the new option --with-encoding=FOO of configure otherwise. The default value is iso8859-1.

This approach gives us all what we desire with not too many drawbacks.

The function will

The command pkgconfig will provide two subcommands, list and get. The first subcommand, list takes no arguments and returns a list containing the names of the defined keys. The second subcommand takes one argument, the name of a key and returns the string associated with that key.

How to Gather the Embedded Information

The information to be embedded is gathered in a platform-specific way and written into the file generic/tclPkgConfig.c. The different platforms may employ platform specific intermediate files to hold the information, but in the compilation phase only tclPkgConfig.c will be used.

Specification of Tcl Configuration Information

The configuration information registered by Tcl itself is specified here. A discussion of the choices made here follows in the next section. Please read this discussion before commenting on the specification.

The values associated with the keys below are all of one of the following types:

The registered keys follow below. They will be always present with some value. Non-boolean keys not applicable to a particular platform will contain the empty string as their value.

Discussion

The placement of this information into a separate package was proposed but rejected because of the trouble of finding the right information for the right library in the case of multiple configurations installed into the same directory space. Embedding into the library does not cost much space and binds the information tightly to the right spot.

Another reason to do it this way is that this enables us to embed information coming from the Makefile itself (like MEM_DEBUG) or from other compile time means. This would not be possible for a file generated solely by the Tcl configure. It would also restrict the embedding to the platforms which allow the use of configure script.

The usage of a separate package to just access the information placed into the Tcl library was also proposed. This was rejected too, due to the overhead for the management of the package in comparison to the small size of the code actually involved.

Another proposal rejected in the early discussions was to have this TIP define an entire build system based upon Tcl. This TIP is certainly a step in this direction and facilitates the building of such a build system (sic!). Still, specifying such here was seen as too large a step right now, with too many issues to be solved and thus delaying the implementation of this TIP.

Only the configuration of the particular variant of the Tcl library or extension which was generated is recorded in the library. No attempt is made to record the information required to allow the compilation of any possible variant of an extension. Doing so would reach again into the bigger topic of specifying a full build system. We've already established that as being out of the intended scope of this TIP.

Note further that the scheme as specified above does not prevent us from adding the full information in a later stage. In other words, it does not restrict the development of a more powerful system in the future.

This should be enough reasoning to allow the acceptance of even this admittedly simple system.

The configuration information registered by Tcl is currently a very small subset of the information in tclConfig.sh. A future TIP is planned to provide the missing information in a regular and generalized manner.

If an extension requires more information than provided by the Tcl configuration it will have to obtain this information itself. For instance, TclBlend requires a CLASSPATH, the name of a Java compiler, etc. whereas the TclPython and TclPerl extensions require paths to those environments, etc. It is not reasonable that the configure script for Tcl itself have to accommodate all requirements of all extensions of Tcl. Instead, the configure scripts or whatever other means is used to obtain the configuration information for the extensions should reflect their needs, and register the requirements gathered into their own configuration command. Note that an extension is only expected to create variables for information unique to it. Everything else can be had from the configuration command of Tcl and the extensions it depends on.

This TIP is not in opposition to [34] but rather fleshes out one of the many details in the specification which were left open by that TIP.

This TIP also does not propose to change the process for building Tcl itself. The goal is rather to make the building of extensions easier in the future.

A naming convention for keys returned by the config command would have been possible but would also require quite a lot more text, both in careful definition of the general categories and in explanations of the choices made.

Implementation

Work on implementing this feature is tracked at Tcl Patch 507083 at the Tcl project at SourceForge. Implementation effort also takes place on the tip-59-implementation branch in the Tcl CVS repository (see [31]).

Copyright

This document is in the public domain.

History