Check-in [9f0c27f8d1]

TIP:            0
Title:          Tcl Core Team Basic Rules
Version:        $Revision: 2.6 $
Author:         John Ousterhout <[email protected]>
State:          Final
Type:           Process
Vote:           Done
Created:        11-Dec-2000
Post-History:


~ Abstract

This TIP describes the mission, structure, and operating procedures
of the Tcl Core Team (TCT).  When in doubt about how the TCT works,
consult this document as the final authority.

~ Introduction

The Tcl Core Team is a self-organizing group of Tcl experts who are
responsible for the evolution and management of the Tcl core.  The
Tcl Core Team decides what goes into releases; it implements, tests,
and documents new features and bug fixes; it manages the release
process; and it also manages the Tcl Developer Exchange Web site.

~ Scope: the Tcl core

The phrase "Tcl core" refers to the Tcl interpreter and the Tk
toolkit (the packages previously released by Sun, Scriptics, and
Ajuba).  We also include the Tcl Developer Exchange Web site and
the Tcl bug database in the Tcl core.  The Tcl Core Team may also
choose to take on additional responsibilities such as the creation
of more comprehensive "batteries included" releases.  We expect
other Tcl development teams to form independently from the Tcl
Core Team to manage additional projects, such as popular extensions.
The Tcl Core Team may eventually spin off some of its activities
into separate teams.

~ Team membership

The Tcl Core Team is a small group of people who are making major
contributions to the development of the Tcl core and who are highly
respected and trusted by the Tcl community.  Team members are expected
to invest significant amounts of their time to improve the Tcl core.

The original group of Team members was elected by the Tcl community,
but the TCT now handles its own membership according to rules described
here.  To become a member of the Team you must be nominated by an
existing member and voted on by the existing Team; you must receive
2/3 of the votes cast.  If you would like to join the Tcl Core Team,
you should first demonstrate your development skills and leadership by
participating in development projects under the auspices of an
existing team member.

Inactive or disruptive members of the team can be removed by a vote
of other Team members: a 2/3 majority of those voting is required to
remove a Team member.

~ Communication

The primary mechanism for communicating with the Tcl Core Team is the
mail alias [email protected].  This is a public mailing
list;  anyone interested in following the discussions of the TCT is
welcome to join the mailing list.  Email sent to this alias is
archived, so you can review previous discussions at SourceForge.

~ Basic organizational structure

The team structure is simple and flat.  All members have equal
standing: there is no Chairman.  The Tcl Core Team makes its own
rules and chooses its own members as described in this document.
Anyone on the Tcl Core Team can propose a change in the rules;
after discussion, the change is voted on by the Team and must
receive 2/3 of the votes cast.  The person proposing a rules change
is responsible for making sure that the change is properly
implemented after it has been approved (e.g. by modifying this
TIP, creating additional tools, etc.).

~ 2/3 vote

Wherever a 2/3 vote is called for in this document, it means that a
proposal must receive ''at least two-thirds of the votes cast'', not
votes from at least two-thirds of all TCT members.

~ Projects and maintainers

Tcl improvements are organized around two key ideas: ''projects''
and ''maintainers''.  Most of the activities of the Tcl Core
Team consist of projects.  A project can consist of a bug
fix, a new feature in the Tcl core, a new facility in the Tcl
Developer Exchange, or anything else except a change to this
TIP.  We divide projects into two general categories: bug
fixes and feature changes.  In general, if a project requires
manual entries to be updated then it is a feature change;  when
in doubt, a project is a feature change .  Bug fixes use a
more streamlined process for implementation, whereas feature
changes require discussion and approval in advance.

A maintainer is someone who has taken primary responsibility for
a portion of the Tcl sources.  Many maintainers will be members of
the Tcl Core Team, but the Team may also select maintainers from
outside the Tcl Core Team.  We hope to find enough maintainers to
cover all of the Tcl sources, but we will appoint a ''default
maintainer'' to handle the parts of Tcl for which no other maintainer
has volunteered.  We'll also try to have backup maintainers who
can step in when the primary maintainers are on vacation or
otherwise unavailable.

A maintainer accepts several responsibilities, including the
following:

     * Monitoring the bug database for bugs in his/her area.

     * Arranging for bugs to be fixed, either by doing it
       himself/herself or finding someone else to do it.

     * Coordinating and reviewing all modifications to his/her area.

     * Providing assistance to other people working in his/her area.

~ Project life-cycle: approval, implementation, integration; TYANNOTT

The project for a feature change goes through three stages: approval,
implementation, and integration.

A project starts when a member of the Tcl Core Team proposes it to
the Team.  Proposals are submitted by emailing TIPs (Tcl Improvement
Proposals) to the Tcl Core Team.  The format of TIPs is described
in a separate TIP.  Whoever proposes a project is responsible for
making sure it is properly implemented.  A proposal without a
committed implementor cannot be approved.

Project approval is done through a process called ''TYANNOTT'':
Two Yesses And No No's Or Two Thirds.  In order for a project to be
approved it must have support from at least one other member of the
Tcl Core Team besides the proposer.  Once a project has been proposed
and discussed, if there are no objections and there is a vote of
confidence from a second team member ("Two Yesses And No No's"),
then the project is approved.  If objections remain after the
discussion, then the proposer must summarize the objections and
call for a vote of the TCT; a 2/3 vote is required for approval.
The idea here is that most projects will be no-brainers and we want
a simple decision process that doesn't get in the way of progress.
On the other hand, the Tcl Core Team can only work effectively if
it is highly collegial; if the Team can't reach pretty clear
agreement on a project (i.e more than 1/3 of the TCT objects to it)
then the project needs to be rethought.

The second phase of a project is implementation.  The proposer is
responsible for the implementation, either doing it himself/herself
or arranging for someone else to do it.  The implementation is done
in a private work area and may not be integrated with the official
sources until the third phase, below.

The third phase of a project is integrating the results back into
the official Tcl repository.  This is where maintainers come in.
First, before any change can be applied to the official Tcl sources,
the implementor must post it as a patch to the SourceForge patch
manager.  This rule applies regardless of the type of change (anything
from a 1-line bug fix to a major new feature) and regardless of who
is proposing the change.  We use the SourceForge patch manager to
record all changes and also to facilitate discussion about the
changes before they are applied.

When a patch arrives in the SourceForge patch manager, the
appropriate maintainer reviews it and works with the proposer
to revise it as necessary.  Other people can also review the
patch, since it is public.  If changes are needed, a revised
patch is logged in the patch manager (the final version of
the patch must always appear in the SourceForge patch manager).
Once the maintainer is satisfied with the patch, it can be applied
to the Tcl sources.  If the patch implementor has write access
to the sources that he or she can apply the patch once the
maintainer has approved it.  If the patch implementor doesn't
have write access to the sources than the maintainer applies
the patch.

Maintainers are responsible for watching the SourceForge patch
manager to make sure that incoming patches in their area are dealt
with quickly.

If the implementor of a patch is the maintainer, then he/she can
apply the patch to the Tcl sources immediately after logging it in
the SourceForge patch manager, without waiting for additional
approval.  However, if someone objects to the patch then the
maintainer must be prepared to revise it after the fact.

~ Fast path for bug fixes

For a bug fix, no initial proposal or approval is required.  The
only approval needed is for the maintainer to review the patch
before it is applied to the sources.  For example, we invite everyone
in the Tcl community to fix bugs and submit patches to the SourceForge
patch manager.

~ Implementors outside the Tcl Core Team

We encourage people outside the Tcl Core Team to get involved with
Tcl development.  For example, anyone can submit patches for bug
fixes.  It's also fine for someone outside the Tcl core team to
propose a feature change and then implement it, but there must be
a sponsor on the Tcl Core Team who will take personal responsibility
for it.  Typically the sponsor will be the maintainer for the area
of the change.  It is the sponsor's responsibility to provide whatever
level of supervision is appropriate to ensure that the project is
executed well.  If the implementor for a project is not a TCT member
then they cannot vote for approval: TYANNOTT requires the sponsor
plus one other Team member.

~ Raising concerns

If you have concerns about a project, the best time to raise them is
during the initial discussion.  Once a project has been approved, the
best approach is to raise the issue directly with the implementor;
most issues should get resolved quickly this way.  If you can't find
the implementor or can't reach agreement, and if the implementor is
not a member of the Tcl Core Team, the next person to talk to is the
Tcl Core Team member in charge of the project.  If you still can't get
satisfaction, then raise the issue with the entire Tcl Core Team by
leading a discussion.  Once all the issues are out, you can either
withdraw your objection or summarize the issues (on both sides!) and
call for a vote.  If you aren't a member of the Tcl Core Team
you will need to convince a Team member to manage the discussion
and vote.

Even if a project has received initial approval, a Team member can
object to the project later (e.g. if they believe it hasn't been
implemented properly).  If the objection isn't resolved there will
be an additional vote of the Team, and the project cannot be applied
to the official sources unless it receives a 2/3 majority of
the votes cast.  At the same time, Team members are expected to
raise their objections as early as possible; it would be somewhat
anti-social to raise a basic design objection late in the
implementation of a project when it could have been raised during
the initial approval.

~ Disagreements over patches

Normally, patches are not reviewed by the entire TCT; once the
relevant maintainer has reviewed and approved them then they can
be integrated.  However, everyone is invited to review as many
patches as they wish.  If someone on the TCT objects to a patch
and can't resolve the objection with the implementor and/or
maintainer, then it gets discussed by the entire Tcl Core Team with
the usual rules: if anyone on the Tcl Core Team has an objection
that isn't resolved by the discussion, then a 2/3 vote is required
to retain the patch.  Thus if an implementor reaches a disagreement
with a maintainer he/she can appeal to the entire Tcl Core Team.
Or, if someone on the Tcl Core Team objects to a patch applied by
a maintainer, they too can start a discussion in the whole team.
The goal of the maintainer mechanism is to simplify and speed up
improvements in the common case where everyone is in agreement,
while still allowing the entire Tcl Core Team to offer input and
resolve disagreements.

~ Changes that span areas

If a change involves several different areas of the Tcl sources,
with different maintainers, then one of the maintainers acts as
coordinator (presumably the one whose area has the most changes).
It is their responsibility to consult with other maintainers whose
areas are affected, so that all relevant maintainers are happy
before the patch is applied to the sources.

~ Write access to the Tcl sources and the Web site

Everyone in the Tcl Core Team has write access to all the sources
and the Web site, but they may only make changes consistent with
approved projects.  The Tcl Core Team can also give access to other
people who are working on projects.  For example, as part of a project
proposal a Tcl Core Team member can propose that the work will be
done by someone outside the team, and that that person should have
write access for putting back changes.  Giving out write access is
part of a project decision, with the associated rules for approval.
However, if someone outside the Tcl Core Team has write access, it
must be under the auspices of a Tcl Core Team member; the Tcl
Core Team member is personally responsible for making sure the
project is completed satisfactorily and/or cleaning up any messes.

~ Deadlock resolution

If something should go wrong with the TCT organization and the
Tcl Core Team deadlocks to a point where it can't make meaningful
progress, then John Ousterhout will step in as benevolent dictator
and make enough unilateral decisions to break the deadlock.

~ Copyright

This document has been placed in the public domain.

TIP:            1
Title:          TIP Index
Version:        $Revision: 1.6 $
Author:         TIP Editor <[email protected]>
State:          Active
Type:           Informational
Vote:           No voting
Created:        14-Sep-2000
Post-History:


~ Abstract

This TIP contains the index of all TIPs published over the lifetime of
the TCT. It will be continually and automatically updated.

~ Index

#index:

White backgrounds indicate that the TIP is still a draft, yellow
backgrounds highlight TIPs being voted on, and where a TIP has been
rejected, withdrawn or obsoleted its index entry has a dark grey
background.  Blue backgrounds indicate a TIP has been accepted,
but still needs an implementation approved by maintainers.  Green
backgrounds indicate that the TIP is deferred, waiting for someone
to work on it.

~ Explanations and How To Submit New TIPs

See [2] for a description of the editorial process a TIP has to go
through and [3] for a description of their structure and the commands
used to write them. You submit a TIP to this archive by emailing it
(preferably in source form) to the TIP editor <[email protected]>
who will check it for following of the guidelines, style and general
relevance to Tcl/Tk before checking it into the CVS archive and notifying
the author, the rest of the Tcl Core Team, and the relevant newsgroups.

~ Copyright

This document has been placed in the public domain.

TIP:            10
Title:          Tcl I/O Enhancement: Thread-Aware Channels
Version:        $Revision: 1.6 $
Author:         Andreas Kupries <[email protected]>
State:          Final
Type:           Project
Vote:           Done
Created:        08-Nov-2000
Post-History:   
Tcl-Version:    8.4


~ Abstract

This TIP describes how to change the generic I/O layer in the Tcl core
to make channels aware of the thread they are managed by.

~ Rationale

To explain the motives behind this TIP first a short look at the
history of channels and threading.

In ancient times the Tcl core was not thread safe and did not employ
threads.  All channels belonged to a single interpreter. Later on
interpreter hierarchies were introduced and the ability to move or
share a channel between the interpreters in a hierarchy. When the Tcl
core was made thread safe a short time after the ability to move
channels between threads was added (Helper APIs in the core, main
functionality in the Thread extension).  The goal behind these
modifications was to enable the creation of stream-like communication
paths between threads to complement the message based facilities
(thread send). The modifications were only a partial success because
an in-depth analysis of the relevant data structures showed that the
sharing of a channel between threads is not possible with the current
design, only moving. This was implemented to allow at least the
dispatcher- / worker-thread pattern for structuring a threaded
application.

In further pursuit of the original goal the currently chosen approach
is to define a channeltype where two channels are connected internally
through in-memory fifo buffers where access to these shared structures
is protected by mutexes.

During the implementation of fileevents for this channeltype it was
discovered that an efficient implementation of this part is ''not''
possible because of the inability to post file events to the
eventqueue of the thread the other channel of the pair resides in. An
API to post such events is available (''Tcl_ThreadQueueEvent''), but
not the information which thread actually manages the other
channel. Because of this the current implementation of the channeltype
uses polling based upon timer events posted by each side/thread to
itself to manage file events in a rather inefficient way.

~ Reference implementation

This TIP now proposes to change the internals of the generic I/O
layers in the core so that

   1. Channels know the thread they are managed by, and

   1. are able to deliver this information to an extension querying
      the core.

This then allows the two sides of the channeltype mentioned above to
post events to each other, facilitating an efficient implementation of
fileevents.

The changes necessary to accomplish this are:

   1. Extend the structure ''ChannelState'' in tclIO.h with a new
      field of type ''Tcl_ThreadId'' to hold the id of the thread
      currently managing all channels with this state. Note: This
      structure is shared by all channels in a stack of
      transformations.

   1. Modify the procedure ''Tcl_CreateChannel'' to store the Id of
      the current thread in the ''ChannelState'' of the new channel.
      This information can be obtained with ''Tcl_GetCurrentThread''.
      It is ''not'' necessary to modify ''Tcl_StackChannel'' as the
      thread information is already part of the state when it is
      called, and won't be changed by the call.

   1. If some sort of NIL/NULL value meaning "No thread" is available
      for ''Tcl_ThreadId'', then we should modify ''Tcl_CutChannel''
      to insert this value into the state of the channel it is called
      with, as this channel will not be managed by any thread
      afterward (the procedure removes the channel from the list of
      all channels managed by the current thread).

   1. Modify ''Tcl_SpliceChannel'' in the same manner as
      ''Tcl_CreateChannel'' as the channel will be managed by the
      current thread afterward (The procedure adds the channel to the
      list of all channels managed by the current thread).

   1. Declare a new API function to retrieve the Id of the managing
      thread from a channel. Add this declaration to generic/tcl.decls
      and implement the function in the file generic/tclIO.c.  I
      propose ''Tcl_GetChannelThread'' as the name of this new API
      function.

A patch implementing all of the changes described above and
additionally extending the documentation and the test-suite
is available here:
http://www.cs.man.ac.uk/fellowsd-bin/TIP/10.patch

~ Copyright

This document has been placed in the public domain.

TIP:            100
Title:          Add Support for Unloading Dynamic Libraries Loaded with [load]
Version:        $Revision: 1.10 $
Author:         George Petasis <[email protected]>
State:          Final
Type:           Project
Vote:           Done
Created:        11-Jun-2002
Post-History:   
Discussions-To: news:comp.lang.tcl
Keywords:       load,unload,dynamic library
Tcl-Version:    8.5


~ Abstract

Tcl already provides facilities for loading dynamic libraries, through
the ''load'' command.  However no facilities are currently offered in
order to ''unload'' dynamic libraries already loaded with the ''load''
command.  This TIP tries to add support for unloading libraries, by
introducing a new Tcl command (''unload'') and the guidelines that
dynamic libraries must follow, in order to be unloadable.  Note that
the ''unload'' command will operate only on libraries that are
designed to be unloadable by their developers.  This way backward
compatibility with older extensions is maintained, as unload will
never try to unload libraries unaware of this new functionality.

~ Rationale

Tcl is an ideal language for component-based applications.  Usually
these applications offer a framework in which components developed by
the users of the application can be embedded, in order to extend the
functionality of the framework.  Usually, these extensions are
implemented as C/C++ dynamic libraries that are loaded through the
''load'' Tcl command.

However the development of such components can be a time-consuming
process, as developers have to restart the framework application in
order to be able to reload the library into it and test its altered
functionality.  And this can be quite annoying (depending on the
application of course), as usually processing within the application
is required in order to bring it into a proper state before testing
the library.

The development cycle can be significantly shortened if Tcl provides a
mechanism for unloading a dynamic library.  A new version of the
library can be created, as its object file can now be written, and the
updated library can be re-loaded.

However, this is not the only application of unload.
Services running for long periods of time and want to unload
no longer needed functionality, replacing parts of an applications
(i.e. from an automatic update procedure) or functionality
temporarily needed (i.e. a web browser that loads a plugin
to display a file of a particular file type) are some
additional fields of application.

~ Introduction

Unload functionality has been left out of the Tcl core, mainly because
library unloading was poorly implemented in many operating systems.
But almost all operating systems have been improved in the meantime,
and as a result most modern operating systems now support library
unloading.

The main idea of this TIP is to enable dynamic library unloading at
the Tcl level, in the same sense ''load'' provides dynamic library
loading.  However, library unloading will be provided only when the
underlying operating system support this feature (as is also the case
for ''load'') and only when the library to be unloaded provides a set
of functions that can "undo" the changes the library has made to the
interpreter.  In all other cases, unloading a library will result in
an error.

This TIP proposes the insertion of a new Tcl command named ''unload''
and two functions ''pkg_Unload'' and ''pkg_SafeUnload'', modelled
after ''pkg_Init'' and ''pkg_SafeInit'', that libraries which can be
unloaded should implement.

~ Unloadable Libraries

A main concern is related to when a shared library can be "unloadable".
An unloadable library is a library characterised as such by its
developer. The developer of the library must export a function
from the library, similar to the library's initialisation
function. The unload command will never try to unload a library
that does not provide such a function. This makes old libraries
(before the introduction of the unload functionality) safe.

There is a category of libraries that
can never be unloaded (i.e. libraries that register new
tcl object types). However, the choice is upon the developer:
the developer knows if the library can be unloadable. The simpler
case, libraries that only register new commands are the most
probable libraries to be unloadable.
Libraries that export functions through a stub mechanism cannot be
unloaded, as the were meant for having dependencies with
other libraries that use the exported API. There is no way for
the provider library to know wether it is used or not.

~ Specification

Actually, all the facilities for unloading dynamic libraries already
exist in the Tcl core and simply they are not yet exposed at the Tcl
level.  As a result, the implementation of the unload command should
be fairly easy.

''load'' as it is currently implemented loads a dynamic library only
the first time this library is loaded.  It keeps an internal cache of
all loaded libraries and if an already loaded library is requested
again, only its initialisation function is called.  This cache should
be extended to keep some additional information:

   1.  Two reference counts, counting how many times a specific
       library has been loaded.  This reference count should be
       increased by each ''load'' and decreased for each ''unload''.
       When it reaches 0, the library can be unloaded.  Safe
       interpreters and normal interpreters should have different
       reference counts.  Both should be 0 in order for a library to
       be unloaded.

   2.  The addresses of the ''pkg_Unload'' and ''pkg_SafeUnload''
       functions, if these are implemented by the library.  If both of
       these functions are missing, the library will never be
       unloaded.  If only ''pkg_Unload'' is implemented, the library
       can be unloaded if it never has been loaded in a safe
       interpreter.  Finally, if ''pkg_SafeUnload'' is implemented,
       the library can be unloaded if it has never been loaded in a
       normal interpreter.

The ''unload'' command will always return an error, if the operating
system does not support library unloading.  In case the operating
system supports library unloading:

   1.  ''unload'' will examine the cache of ''load'' to locate the
       entry for the library to be unloaded.  It is an error to unload
       a library that has not been loaded with ''load''.

   2.  If the entry in the cache is found, ''unload'' checks whether
       the corresponding for the interpreter type unload function
       pointer is NULL or not.  If it is NULL, the library cannot be
       unloaded under this interpreter and again an error is returned.

   3.  If the unload function pointer is not NULL, it is executed.  If
       an error is returned by this function, ''unload'' also returns
       an error.

   4.  If the unload function finishes without errors, the reference
       count corresponding to the interpreter type is decreased.  If
       both reference counts reach 0, the library is unloaded.

~ Responsibilities of the Unload Functions

Its up to the developer of the library to decide if its library can be
unloaded or not.  A library can be unloaded if the function
''pkg_Unload'' is implemented and exported as a symbol from the
library, and the library will never be loaded in a safe interpreter.
A library that can be also loaded in safe interpreters is unloadable
if the function ''pkg_SafeUnload'' is also available.  These two
functions will accept two arguments, the interpreter under which the
library is unloaded and an integer, holding various flags.  The flags argument
can be either ''TCL_UNLOAD_DETACH_FROM_INTERPRETER'' or
''TCL_UNLOAD_DETACH_FROM_PROCESS''. In case the library will remain attached to
the process after the unload procedure returns (i.e. because the library is
used by other interpreters), TCL_UNLOAD_DETACH_FROM_INTERPRETER will be defined.
However, if the library is used only by the target interpreter and the library
will be detached from the application as soon as the unload procedure returns,
the flags argument will be set to TCL_UNLOAD_DETACH_FROM_PROCESS. 

The main responsibility of these functions is to remove from the
interpreter they are unloaded under any reference to a function
residing inside the library.  For example, such a function must:

   1.  Unregister any commands that have been registered by the
       ''Init()'' function to the ''interpreter'' given by the first
       argument.  In order to do this, the library should keep
       internally the tokens returned by each ''Tcl_Create*Command'',
       as command may have been renamed.

   2.  Unregister any other commands that may have been registered to
       the interpreter during the use of the library (usually used to
       represent special special data structures).

If the flag ''TCL_UNLOAD_DETACH_FROM_PROCESS'' is defined, the
developer must do additional task, that are not normally required when
the library gets unloaded from an interpreter:

   3.  Free any memory occupied by the internal structures of
       the library.

   4.  In general, try to remove any references Tcl may have
       to functions provided by the library.

If the developer cannot remove all reference to functions to the
library, its better to not provide at all these two functions, so as
unload to never attempt to unload the library.

~ Dependencies Among Libraries

It is possible that a library A has been loaded that exports some
symbols.  Then a library B is loaded, that has dependencies (i.e. uses
some exported symbols) on A.  What if A gets unloaded?

Actually, most modern operating systems seem to provide a solution to
this problem, as reference counts are hold internally by the operating
system for each library.  Newer Windows, Solaris and Linux seem to
provide similar solutions and in reality they don't unload the library
if such symbols remain, even if the unload system call has been made
for the library.  Both libraries A and B have to be unloaded in order
for A to be really removed from the address space.

~ Reference Implementation

A reference implementation can be found at:
http://sf.net/tracker/?func=detail&aid=823486&group_id=10894&atid=310894

~ Copyright

This document has been placed in the public domain.

~ Appendix: The unload man page.

     NAME
     unload - Unload machine code.
     SYNOPSIS
     unload ?switches? fileName
     unload ?switches? fileName packageName
     unload ?switches? fileName packageName interp

This command tries to unload shared libraries previously
loaded with load from the application's address space.
fileName is the name of the file containing the library
file to be unload; it must be the same as the filename
provided to load for loading the library. packageName is
the name of the package, and is used to compute the name
of the unload procedure. interp is the path name of the
interpreter from which to unload the package (see the
interp manual entry for details); if interp is omitted,
it defaults to the interpreter in which the unload
command was invoked.

If the initial arguments to unload start with - then they
are treated as switches. The following switches are
currently supported:

     Marks the end of switches. The argument following
     this one will be treated as a fileName even if it
     starts with a -.

When a file containing a shared library is loaded through
the load command, Tcl associates two reference counts to
the library file. The first counter shows how many times
the library has been loaded into normal (trusted)
interpreters while the second describes how many times
the library has been loaded into safe interpreters. As a
file containing a shared library can be loaded only once
by Tcl (with the first load call on the file), these
counters track how many interpreters use the library.
Each subsequent call to load after the first, simply
increaments the proper reference count.

unload works in the opposite direction. As a first step,
unload will check whether the library is unloadable: an
unloadable library exports a special unload procedure.
The name of the unload procedure is determined by
packageName and whether or not the target interpreter is
a safe one. For normal interpreters the name of the
initialization procedure will have the form pkg_Unload,
where pkg is the same as packageName except that the
first letter is converted to upper case and all other
letters are converted to lower case. For example, if
packageName is foo or FOo, the initialization procedure's
name will be Foo_Unload. If the target interpreter is a
safe interpreter, then the name of the initialization
procedure will be pkg_SafeUnload instead of pkg_Unload.

If unload determines that a library is not unloadable (or
unload functionality has been disabled during
compilation), an error will be returned. If the library
is unloadable, then unload will call the unload
procedure. If the unload procedure returns TCL_OK, unload
will proceed and decrease the proper reference count
(depending on the target interpreter type). When both
reference counts have reached 0, the library will be
detached from the process.

The unload procedure must match the following prototype:
typedef int Tcl_PackageUnloadProc(Tcl_Interp *interp, int
flags);

The interp argument identifies the interpreter from which
the library is to be unloaded. The unload procedure must
return TCL_OK or TCL_ERROR to indicate whether or not it
completed successfully; in the event of an error it
should set the interpreter's result to point to an error
message. In this case, the result of the unload command
will be the result returned by the unload procedure. The
flags argument can be either
TCL_UNLOAD_DETACH_FROM_INTERPRETER or
TCL_UNLOAD_DETACH_FROM_PROCESS. In case the library will
remain attached to the process after the unload procedure
returns (i.e. because the library is used by other
interpreters), TCL_UNLOAD_DETACH_FROM_INTERPRETER will be
defined. However, if the library is used only by the
target interpreter and the library will be detached from
the application as soon as the unload procedure returns,
the flags argument will be set to
TCL_UNLOAD_DETACH_FROM_PROCESS.

The unload command cannot unload libraries that are
statically linked with the application. If fileName is an
empty string, then packageName must be specified.
If packageName is omitted or specified as an empty
string, Tcl tries to guess the name of the package. This
may be done differently on different platforms. The
default guess, which is used on most UNIX platforms, is
to take the last element of fileName, strip off the first
three characters if they are lib, and use any following
alphabetic and underline characters as the module name.
For example, the command unload libxyz4.2.so uses the
module name xyz and the command unload bin/last.so {}
uses the module name last.

PORTABILITY ISSUES

Unix
     Not all unix operating systems support library
     unloading. Under such an operating system unload
     returns an error (unless -nocomplain has been
     specified).

Macintosh
     <Somebody to comment on this?>

BUGS

If the same file is loaded by different fileNames, it
will be loaded into the process's address space multiple
times. The behavior of this varies from system to system
(some systems may detect the redundant loads, others may
not). In case a library has been silently detached by the
operating system (and as a result Tcl thinks the library
is still loaded), it may be dangerous to use unload on
such a library (as the library will be completely
detached from the application while some interpreters
will continue to use it).

SEE ALSO

info sharedlibextension, load, safe

KEYWORDS

binary code, unloading, safe interpreter, shared library

TIP:		101
Title:		Export Tcltest Configuration
Version:	$Revision: 1.4 $
Author:		Don Porter <[email protected]>
State:		Final
Type:		Project
Vote:		Done
Created:	11-Jun-2002
Post-History:	
Tcl-Version:	8.4


~ Abstract

Proposes public command ''tcltest::configure'' to give programmatic
control to processing configuration options of the tcltest package.

~ Rationale

During ''package require tcltest'', the internal command
''ProcessCmdLineArgs'' is evaluated.  This command uses the contents
of ''$::argv'' as option-value pairs to configure several aspects of
the tcltest package.

This approach leaves two aspect of package configuration hardwired,
and outside of the control of users of ''tcltest''.  First, the timing
of configuration is fixed to package load time.  Second, the source of
configuration data is fixed to be the global variable ''argv''.

It would improve flexible use of tcltest to export a public command,
''tcltest::configure'', that will allow configuration of ''tcltest''
by its users at any time and from any source.

~ Proposal

Add and export the command ''tcltest::configure'', with the syntax:

|	tcltest::configure ?option? ?value option value ...?

With no options, ''configure'' returns a list of the available
configurable options.  With a single ''option'' argument,
''configure'' returns the corresponding value of that option, or an
error if no such option exists.  In the most general form,
''configure'' accepts an even number of arguments that are alternating
options and values, and sets each option to each value.

The list of options and acceptable values are to be the same as those
currently recognized by ''tcltest'' as its command line options.  The
difference is that this configuration can now be performed
programmatically, not only on the command line.

With complete programmatic access to ''tcltest'' configuration made
available, the special customization hooks ''processCmdLineArgsHook''
and ''processCmdLineArgsAddFlagsHook'' will be deprecated and removed
from the documentation.  Compatibility support for their existing use
will be provided as described below.

~ Compatibility

Many existing test suites have been written depending on the
auto-configuration from ''$::argv'' at package load time.  Some of
them may also be using the special customization hooks that allow the
addition of more command line options.  For compatibility, if the
presence of any of these hooks is detected, ''tcltest'' will fall back
to performing its load-time configuration.  Also, if any command that
can be influenced by configured values is called prior to any call to
''configure'', then automatic configuration from ''::argv'' will be
performed.

~ Copyright

This document has been placed in the public domain.

TIP:		102
Title:		Change [trace list] to [trace info]
Author:		Reinhard Max <[email protected]>
Type:		Project
Tcl-Version:	8.4
State:		Final
Created:	12-Jun-2002
Keywords:	trace, info, introspection
Version:	$Revision: 1.4 $
Vote:		Done
Post-History:	


~ Abstract

This TIP proposes to change the name of the introspection subcommand
of the ''trace'' command from ''list'' to ''info''.

~ Rationale

Although the functionality of the ''trace'' command (as extended by
[62]) is good, the name of the introspection subcommand, ''list'' is
not such a good choice:

 * The name ''info'' is already well known for introspection purposes
   in Tcl (e.g. [[info]] and [[after info]].)

 * The name [[trace list]] could be misunderstood as having to do with
   tracing lists.

 * The introspection subcommand to trace could be extended to allow
   more specific queries along the lines of [[info commands]], [[info
   procs]], and [[info vars]].  ''(This is outside the scope of this
   TIP.)''

Hence, this TIP calls for the ''list'' subcommand to be renamed to
''info''.  (Note that this also makes the subcommand for introspecting
on variable traces more similar to its old form ''vinfo''.)

~ Copyright

This document is placed in the public domain.

TIP:            103
Title:          Argument Expansion Command
Version:        $Revision: 1.13 $
Author:         Peter Spjuth <[email protected]>
Author:         Donal K. Fellows <[email protected]>
Author:         Andreas Leitgeb <[email protected]>
State:          Rejected
Type:           Project
Vote:           Done
Created:        15-Jun-2002
Post-History:   
Tcl-Version:    8.5


~ Abstract

This TIP proposes to add a command that can perform argument expansion
in a safe and efficient manner.

~ Introduction

Many commands take a variable number of arguments and often you find
yourself with those arguments in a list.  This list must then be
expanded into individual arguments to the command.  This is currently
done with eval:

|eval destroy [winfo children .]

This is a bit obscure and also very error prone when the command
becomes more complex.  It is also inefficient and not object safe, why
a command specialised in doing this would be better.

~ Rationale

There have been suggestions of introducing some new syntax to Tcl to
handle argument expansion.  That is a big and controversial step, and
not anything this TIP wants to meddle in.  A command can improve every
point where eval has shortcomings and thus give a good result with
less means.  It can also serve as a bridge to a future global syntax.

Such a command can be done in several ways and below the choice in
this TIP's specification is defended.

As examples three statements are used which will be repeated for
different alternatives.  This is the eval version:

|eval destroy [winfo children .]
|eval button .b $stdargs -text \$mytext -bd $border
|eval exec \$prog $opts1 [getMoreopts] \$file1 \$file2

The eval version would be even more complex if the lists that are to
be expanded are not known to be pure. To be really safe the last
would be:

|eval exec \$prog [lrange $opts1 0 end] [lrange [getMoreopts] 0 end] \$file1 \$file2

With the proposed command they become:

|expand { destroy `[winfo children .] }
|expand { button .b `$stdargs -text $mytext -bd $border }
|expand { exec $prog `$opts1 `[getMoreopts] $file1 $file2 }

An alternative to having a local syntax is to point at the arguments
that should be expanded, either by index:

|expand {end} destroy [winfo children .]
|expand {2} button .b $stdargs -text $mytext -bd $border
|expand {2 3} exec $prog $opts1 [getMoreopts] $file1 $file2

Or by some flag mechanism:

|expand destroy + [winfo children .]
|expand button .b + $stdargs -text - $mytext -bd $border
|expand exec - $prog + $opts1 + [getMoreopts] - $file1 - $file2

Those lack in writability/readability/maintainability in a disturbing
manner.

For the choice of local syntax one goal is that it should not
violate Tcl's rules, which simplifies implementation since Tcl's
parser can do the job.

Any char that fulfils that could be used but the choice fell on ` for
forward compatibility reasons.  See below.

An alternative syntax could be using enclosing `` or some
other enclosing construct like:

|expand { destroy <[winfo children .]> }
|expand { button .b <$stdargs> -text $mytext -bd $border }
|expand { exec $prog <$opts1> <[getMoreopts]> $file1 $file2 }

Paired characters are good for delimiting things.  Here is the
beginning; here is the end.  But this is not about a new way to
delimit things.  It is about indicating a boolean choice:  expand
or do not expand a word into multiple words.  Whatever character
is chosen to be that indicator, it should be a single, leading
one.  No pairs.

In the specification a restrictive rule was chosen that makes
it an error to use ` in a way that do not fit.  This is to make
it easier to change things in the future should ideas come up
for new features.  E.g., should this become a global syntax
in Tcl 9.0 it can be chosen a bit differently and be backward
compatible with the expand command.

~ Specification

A new command "expand" is added.  It takes one argument, which
contains a Tcl script consisting of one command.  The script may
contain comments but only one command is permitted.

The command is processed in the following manner:

 1. Parse into words according to Tcl's standard rules.

 2. Any word starting with ` must be followed by a single variable
    or command substitution.  The word is remembered and the ` is
    removed.

 3. Perform Tcl's normal execution steps on the new line up to the
    point where the command should have been called.

 4. Expand the arguments that should be expanded.

 5. Execute the command.

The return value of expand is the return value of the command.

''Note 1:'' A word should really start with ` to trigger expansion
which means that words like these are not expanded:

|cmd "`$temp" \`[something]

''Note 2:'' Expansion is only performed with words like:

|cmd `$var `[somecmd $arg] `$arr([cmd $arg])

Words like these are a syntax error:

|cmd `word` `$x,$y `[foo]xy[apa]

~ Forward compatibility

One aspect of choosing a syntax here is to think about the
future.  Should there later be a wish for a global syntax for argument
expansion it would be nice if it were the same as the one chosen in
the expand command.  If an agreement
can be made for what may be acceptable in the future, this should
affect the specification in this TIP.

If a single character like ` is chosen for a global expand syntax
it means a backwards compatibility break.  So, what chars
are likely to be used by people and thus causing problems or confusion
when backwards compatibility is broken?

Some food for thought about different chars:

|_     # Word char
|:     # Gets ugly with namespace qualifiers:  :$::var

|!   if !$var {...}
|*   string match *$suffix $line
|^   regexp ^$prefix $line
|~   cd ~$user
||   open |$prog
|.   button .$w ; glob -nocomplain .$str
|=   wm geometry .e =$geo
|@   .x conf -bitmap @$bmp -cursor @$cur
|<   bind . <$left>  ; set html <$tag>

|(   expr ($a + $b) * $c ;# Confuses paren-matching
|)     # Odd enough as opening, but would confuse any paren-matching

|+   expr $a +$b  ;# Same for any operator
|-   
|%
|&   
|?




|/   open /$path   

|'     # Makes more sense as enclosing?
|`     # Makes more sense as enclosing?
|>   exec foobar >/some/file
|,   append recipients ,[join $header($ccL) ,]

|{}  completely forward-compatible, as {} currently cannot be
|    trailed by anything but whitespace. (This would limit the
|    originally proposed global syntax change to the argument
|    of the expand command)

Example usage of those that seem reasonable:

|expand { exec $prog '$opts1 '[getMoreopts] $file1 $file2 }
|expand { exec $prog `$opts1 `[getMoreopts] $file1 $file2 }
|expand { exec $prog ,$opts1 ,[getMoreopts] $file1 $file2 }

For comparison, the syntax that has been proposed earlier that
would not break backwards compatibility:

|expand { exec $prog {}$opts1 {}[getMoreopts] $file1 $file2 }
|expand { exec $prog {expand}$opts1 {expand}[getMoreopts] $file1 $file2 }

~ Discussion

When first issued the TIP caused some discussion on c.l.t.  Until a
summary is made, here is the thread:

http://groups.google.com/groups?th=9e77d5836b06ab1b

Another thread about it:

http://groups.google.com/groups?th=2ba287be87c2678c

~ Reference Implementation

Patch #570201

http://sourceforge.net/tracker/index.php?func=detail&aid=570201&group_id=10894&atid=310894

~ Copyright

This document has been placed in the public domain.

TIP:            104
Title:          Generalization of the Tk Undo Subsystem
Version:        $Revision: 1.6 $
Author:         Ludwig Callewaert <[email protected]>
Author:         Larry W. Virden. <[email protected]>
State:          Final
Type:           Project
Vote:           Done
Created:        19-Jun-2002
Post-History:   
Discussions-To: news:comp.lang.tcl
Tcl-Version:    8.4


~ Abstract

This TIP proposes a reimplementation of the Tk Text widget undo
feature.  The text widget interface is not affected.  No functional
changes are made at the Tcl level.  The purpose of the
reimplementation is to move the undo feature from a text only
implementation to a general implementation also usable by other
widgets.  This opens the door to undoing also tag, mark and other
operations, and allows for an exposure of the undo stack at the Tcl
level.  These new features are however not part of this TIP.

~ Rationale

As stated in the abstract, the current implementation of the text
widget undo feature only allows for text changes to be undone.  The
usefulness of the undo feature would increase tremendously if
other operations could be undone (tags, marks, embedded windows, ...).
This was already part of the [26] discussions.  This TIP deals with
the generalization of the undo stack to cope with these requirements.

~ Specification

As the undo functionality is no longer text widget specific, it has
been put in a separate file ''generic/tkUndo.c'' along with its header
file ''generic/tkUndo.h''.  The ''TkUndoRedoStack'' is a structure
containing the undo and redo stacks.  These undo and redo stacks are
linked lists of ''TkUndoAtom'' structures.  There are two types of
these atoms: the separator (similar to the previous implementation)
and the command.  When the type is command, both an ''apply'' and a
''revert'' action need to be provided.  The apply action is for the
redo.  The revert action is for the undo.  Both are pointers to a
''Tcl_Obj'', so they can (and should) contain a Tcl script.

The following functions all operating on a ''TkUndoRedoStack''
stack are provided to implement the undo/redo functionality.

   1. ''TkUndoInitStack(interp)'': returns a pointer to an initialized
      TkUndoRedoStack and stores ''interp'' in that stack for script
      evaluation.

   2. ''TkUndoClearStacks(stack)'': clears both the undo and the redo
      stacks.

   3. ''TkUndoFreeStack(stack)'': clears both undo and redo stacks and
      frees any memory allocated to ''stack''.

   4. ''TkUndoInsertUndoSeparator(stack)'': inserts a
      separator on the undo stack.  Note that there is currently no
      need for a ''TkUndoInsertRedoSeparator'' function, as the redo
      stack is managed by the internals of ''TkUndo''.

   5. ''TkUndoPushAction(stack, actionScript, revertScript)'': pushes 
      an action of the undo stack (an atom of type command).
      ''actionScript'' and ''revertScript'' are ''Tcl_DString''
      pointers that provide the script to redo and undo the action
      respectively.  The redo stack is cleared.

   6. ''TkUndoRevert(stack)'': undo a compound action.
      Compound means all revert script of action between two
      separators on the undo stack are evaluated in the stack's 
      interpreter and the actions are moved to the redo stack.
      Returns TCL_ERROR when unsuccessful (stack empty for instance),
      and TCL_OK otherwise.

   7. ''TkUndoApply(stack)'': redo a compound action.  The
      apply script of all actions between two separators on the redo
      stack is evaluated in the stack's interpreter.  The actions are
      moved to the undo stack.  Returns TCL_ERROR when unsuccessful
      (stack empty for instance), and TCL_OK otherwise.

   8. ''TkUndoSetDepth(stack, maxDepth)'': sets the maximum number
      of compound actions stored on the stack to ''maxDepth''.  By
      default, stacks are unlimited, and a value of ''maxDepth'' <= 0
      resets the stack to be unlimited.

   9. The option ''-maxundo'' is added to the text widget to access the
      stack limit feature of the text widget's undo stack from the
      script level.

These functions are sufficient to implement the current undo
functionality of the text widget, and they have been used for this
purpose.

~ Reference Implementation

See patch #554763 on SourceForge: ''
https://sourceforge.net/tracker/?func=detail&atid=312997&aid=554763&group_id=12997
''


~ Copyright

This document has been placed in the public domain.

TIP:		105
Title:		Add Prefix Matching for Switch
State:		Withdrawn
Type:		Project
Tcl-Version:	8.5
Vote:		Pending
Post-History:	
Version:	$Revision: 1.4 $
Author:		Donal K. Fellows <[email protected]>
Created:	03-Jul-2002
Obsoleted-By:	195


~ Abstract

This TIP adds a new option to the [[switch]] command to support
matching of strings to unique prefixes of patterns, similar to Tcl's
existing subcommand-name matching or Tk's option-name matching.

~ Rationale

When code (particularly in script libraries) wants to support shortest
unique prefix matching in the manner of the Tcl core (as provided by
''Tcl_GetIndexFromObj'') currently either the prefixes have to be
precomputed (by hand or by script) or the matching has to be done
backwards.  In the first case, this is either error-prone or requires
an extra piece of code that has to be developed by the programmer.  In
the second case, the code has to be converted into a pattern which is
matched against the list of supported options in some way, which is
either inefficient or has hazards if the string being matched contains
characters that are meaningful to the matching engine being used.
Instead, it would be far nicer if we could make the core support this
directly, so that script authors could just say what they mean.

~ Proposed Change

To support this, I propose modifying the ''switch'' command to take an
extra option ''-prefix'' (which should be mutually exclusive with
''-exact'', ''-glob'' and ''-regexp'' of course) to enable prefix
matching.  When prefix matching is enabled, the arm chosen for
execution will be the one such that the switch value is identical to
or an unambiguous prefix of its pattern (i.e. it will not be a prefix
of any other pattern listed, unless the pattern of the arm chosen is
exactly equal to the switch value.)  If there is no arm whose pattern
is an unambiguous prefix of the switch value, the default arm will be
selected for execution, or if there is no default arm, the switch
command will terminate without an error and with an empty result (this
is in contrast to the behaviour of ''Tcl_GetIndexFromObj''.)

~ Examples

The command:

|switch -prefix f {
|   foo {
|      puts "matched foo"
|   }

|   bar {
|      puts "matched bar"
|   }
|}



prints "matched foo".  The command:

|switch -prefix b {
|   bar {
|      puts "matched bar"
|   }

|   boo {
|      puts "matched boo"
|   }

|   default {
|      puts "the default action"
|   }
|}



prints "the default action" ("b" is a prefix of two patterns.)  The
command:

|switch -prefix tcl {
|   tcl {
|      puts "The Tool Command Language"
|   }

|   tk {
|      puts "The Tk Toolkit"
|   }

|   tcl/tk {
|      puts "A cool combination"
|   }
|}



prints "The Tool Command Language" (although "tcl" is a prefix of two
patterns, it matches one of them exactly.)

~ Copyright

This document has been placed in the public domain.

TIP:            106
Title:          Add Encoding Abilities to the [dde] Command
Version:        $Revision: 1.14 $
Author:         Harald Oehlmann <[email protected]>
State:          Final
Type:           Project
Vote:           Done
Created:        13-Aug-2002
Post-History:   
Tcl-Version:    8.6


~ Abstract

When using Windows DDE communication with non-Tcl programs, the encoding of
the exchanged strings is mostly needed to be the system encoding.  Selection
of this behaviour should be possible with in the '''dde''' command should be
done by a parameter.

~ Specification

Extend the '''dde''' commands taking a data argument by the switch
'''-binary''':

 > '''dde execute''' ?'''-async'''? ?'''-binary'''? ''service topic data''

 > '''dde poke''' ?'''-binary'''? ''service topic item data''

The argument ''data'' is taken as a binary string if the '''-binary''' switch
is given.  Otherwise, it is interpreted as utf-8.

Examples:

| dde execute -binary CS CS [encoding convertto [encoding system] �pfel]\0
| dde poke -binary CS CS I [encoding convertto [encoding system] �pfel]\0

~ Rationale

The communication with DDE with external programs uses the format clipboard
''CF_TEXT'' and the sent text should be coded in the system encoding
(''cp1252'' in my locale).

Most people who use DDE to communicate with, for example, Excel use the fact
that what Excel expects (''cp1252'') and what Tcl actually sends (''utf-8'')
is identical for 7-bit values and they don't use 8-bit values.  Unfortunately,
characters used in languages like German, French, etc., are located over this
limit and thus are not transferable.

Peter Hardie addressed this point on 2000-10-26 in the Tcl Feature Request at
SourceForge (#219185: "dde only handles UTF-8 data (-binary patch available)"
[http://sf.net/tracker/?func=detail&aid=219185&group_id=10894&atid=360894]).
His proposal was to add a '''-binary''' option.

This is a reasonable solution, because any encoding including the system
encoding may be used as shown in the upper example.

~ Reference Implementation

See the ''tip-106-impl'' branch in Tcl's fossil repository
[https://core.tcl.tk/tcl/timeline?r=tip-106-impl].

~ Rejected Alternatives

I proposed to use a switch ?'''-encoding''' ''encoding''? which would
avoid the preparation of an encoded string by '''encoding convertto'''.
DDE is so little used at those days so a minimal support is sufficient.

The '''dde request''' subcommand already has a parameter ''-binary''.
It is more logical to extend this to the other commands.

~ Copyright

This document has been placed in the public domain

TIP:		107
Title:		Fix the 2-second "raise delay" in Tk
Version:	$Revision: 1.4 $
Author:		Joe English <[email protected]>
State:		Final
Type:		Project
Created:	28-Aug-2002
Tcl-Version:	8.4
Vote:		Done
Post-History:	


~ Abstract

This TIP explains the genesis of the long delays often associated with
the [[raise]] and [[lower]] commands under Unix/X with some window
managers, as well as describing the solution.

~ Rationale

Currently, Tk's [[raise]] and [[lower]] commands do not return to the
caller until the operation has completed.  Under Unix, the window
manager is responsible for changing the stacking order of toplevel
windows, so [[raise]] and [[lower]] must wait for a notification from
the WM before returning.  Not all window managers are ICCCM-compliant
in this regard, however, so the operation may time out instead.

This two-second "raise delay" has been a longstanding, persistent
problem in Tk.  It has supposedly been fixed several times, but the
problem keeps reoccurring under new window managers and new
environments.  At present, the problem is most noticeable under KDE 2
and KDE 3.

~ Proposal

Change Tk so that [[raise]] and [[lower]] return immediately, without
waiting for a notification that may not be forthcoming.

This should not be be a controversial change.  This behaviour is not
documented anywhere, and is not observable by Tk programs except via
[[wm stackorder]] (see [74]).

Moreover, the guarantee is largely meaningless.  After [[raise]]
returns, the window ''contents'' may still not be visible (there may
be pending <Expose> events, for example), and the actual position in
the stacking order is still subject to window manager intervention.

~ Compatibility Issues

The only Tk programs that would break with this change are ones which
expect the return value of [[wm stackorder]] to reflect the results of
any immediately-preceding [[raise]] and [[lower]] commands.  (The Tk
test suite is one such program, and would need to be modified).

Unfortunately there is no reliable way to fix such programs -
[[update]] will not work, and the ICCCM does not, to the author's
knowledge, provide a way to synchronize with the window manager to
make sure it has processed all outstanding client requests.  Even if
it did, this wouldn't help - the raise delay problem only occurs under
non-compliant window managers to begin with!

Since the stacking order is not observable except through [[wm
stackorder]] - that was the whole point of [74] - no other programs
will be affected.  (Note that [[wm stackorder]] will still work: the
only difference is that it may return soon-to-be out-of-date
information.  Since this is the case already - the user may restack or
iconify windows at any time - this change should be low-impact.)

~ Reference Implementation

See Sourceforge Tk Patch #601518.
http://sourceforge.net/tracker/index.php?func=detail&aid=601518&group_id=12997&atid=312997

~ Author's Note

Could we fast-track this?  It's a longstanding problem with a simple
fix and ought to make it in before 8.4 goes final.

~ Detailed Analysis

First, some terminology:

    *	''toplevel'': a Tk [[toplevel]] window.

    *	''wrapper'': an auxiliary window created by Tk to hold the
	toplevel and its (optional) menubar.  Initially created as a
	child of the root window.

    *	''client window'': From the window manager's perspective, any
	window created as a child of the root window by an X client.
	Tk wrapper windows are client windows.  Most window managers
	reparent client windows under a new frame window which holds
	window manager decorations.

    *	''reparent'': Used as a noun, the immediate parent of a
	wrapper which has been reparented by a window manager.

    *	''frame'': The immediate child of the root window (or virtual
	root window) created by the window manager to hold a client
	window and its decorations.  May or may not be the same as the
	reparent window.

Next, some methodology:

The correct way to change the stacking order of a client window is to
make a ''ConfigureRequest'' on the client window with ''stack_mode''
set appropriately.  If the client has not been reparented, then the X
server performs the operation directly, and will send a
''ConfigureNotify'' back to the client if, and only if, the actual
stacking order has changed.  (Raising a window which is already at the
top of the stacking order will not result in a ''ConfigureNotify'',
for example).

If the client window ''has'' been reparented (which is usually the
case), then the window manager intercepts the request and, at its
discretion, restacks the frame window instead.  It then sends a
synthetic ''ConfigureNotify'' back to the client, regardless of
whether or not it honored the request.

If the stacking order is to be changed relative to some other window -
that is, if the ''sibling'' field is also set - and the client has
been reparented, then the ''ConfigureRequest'' will fail with a
''BadMatch'' error before it gets to the WM.  Clients must be prepared
to handle this case by catching the error and re-sending a synthetic
''ConfigureRequest'' to the root window, which the WM receives and
handles as above.

See ICCCM section 4.1.5 "Configuring the Window" for the full
specification.  The Xlib function ''XReconfigureWMWindow()'' takes
care of all these details.

Now, some archaeology:

Tk 4.0 did not do this: instead, it called ''XConfigureWindow()'' on
the ''reparent'' window, then waited for a ''ConfigureNotify'' on that
window.

This was wrong on at least two counts.  First, the reparent window
might not be the same as the frame window, in which case this would
have no effect at all.  (In 4DWm and Sawfish, for example, the
reparent window is a child of an outer frame window).  Second, it's
not ICCCM-compliant (Tk doesn't own the reparent window and shouldn't
be mucking with it).

Tk 4.0 also included several heuristics that attempted to determine
when the operation was unnecessary, to avoid waiting for a
''ConfigureNotify'' on the reparent that was not forthcoming.

In Tk 4.1, the (incorrect) call to ''XConfigureWindow()'' on the
reparent was changed to a (correct) call to ''XReconfigureWMWindow()''
on the wrapper, but the old heuristic code was left mostly intact.

Browsing the CVS logs and the older Tk Changelog, we see that this
code has been updated several times to account for new conditions, but
ultimately without success: the problem persists.

These heuristics are not needed at all under WMs which send a
synthetic ''ConfigureNotify'' in response to client window stacking
order changes.  On some non-compliant WMs, however, they may help
lessen the problem - more by accident than by design - if the reparent
is the same as the frame window then the Tk 4.0 heuristics sometimes
still work.  But even then the heuristics are not reliable.  For
instance under KDE 2.2 and KDE 3, calling [[raise]] twice in
succession always results in a 2-second delay.

It is the author's opinion that the only way forward is to let
[[raise]] and [[lower]] run asynchronously, and fix the two-second
raise delay once and for all.

~ Copyright

This document is hereby placed in the public domain.

TIP:            108
Title:          Summary of Changes to Generic Tcl/Tk Code to Enable Mac OS X Port
Version:        $Revision: 1.2 $
Author:         Jim Ingham <[email protected]>
State:          Final
Type:           Informative
Vote:           No voting
Created:        29-Aug-2002
Post-History:   


~ Abstract

The submission of the changes to generic Tcl/Tk necessary for the Mac
OS X port was discussed on the Tcl Core mailing list.  In light of the
very minor changes to shared code involved, and to facilitate
including the port in the 8.4 release, it was decided that we would
not hold a formal vote.  This informational TIP is offered to
summarize the work done, however, and to maintain a more complete
record.

~ What Changes are Required for the Mac OS X Port?

These fall into two parts: macosx only changes, and changes that
effect generic code.

The Mac OS X-only changes again fall into two parts.  On the one hand,
we introduced new macosx directories to the Tcl and Tk trees, at the
same level as the win, unix and mac directories.  At present, the
''tcl/macosx'' directory only contains one ''.c'' file and a project
file.  The ''tk/macosx'' directory is much more substantial.  This set
of changes manifestly only effects this port, and since something is
better than nothing, should be uncontroversial.

The other Mac OS X-only part is the addition of Mac OS X specific
elements to the ''.decls'' files.  This should also be
uncontroversial, though I had to add some slightly non-trivial code to
the ''genStubs.tcl'' file to handle the fact that Tk loosely uses
"unix" to mean "X11", which for Mac OS X is not the case.

The Tcl side of Mac OS X is clearly unix, but the same Tcl can in fact
be used with X11 (there is a native X Server in quite common use on
Mac OS X) and with the Aqua based Tk port.

In the end, however, the stubs generated for the generic, mac, win &
X11 parts of Tcl/Tk are the same, and there is just some extra logic
for the aqua part, so the result effects only Mac OS X code.

The generic code changes are quite small - a testament to the design
of the Tcl/Tk porting layers.

 1. We changed the ''configure.in'', ''Makefile.in'' and ''tcl.m4'' to
    handle building Tcl in a the Framework form that is common on Mac
    OS X.

 2. I added a bit of code (conditionalized to Mac OS X) to
    ''tclUnixInit.c'' (in the ''TclpSetVariables'' function) to
    support looking for script files embedded in the Mac OS X
    Framework bundle.  This fits the Mac OS X model better than
    putting files in ''/usr/local'' or such-like.

 3. I added a few more elements to the ''notifierProcPtr''.  For the
    aqua Tk, we need to swap the Unix notifier with our own, and so we
    needed more control over the notifier than was allowed.  This
    change has no effect if you don't use it, however.

 4. I added a function, ''TkGetFirstTextLayout'', which gets the run
    of text up the the first wrap.  I have to get this because the Mac
    OS X button control doesn't like a newline in the button text.  It
    is a private function, however, so it doesn't cause any
    incompatibilities.

 5. We had to change various places in the Tk script code and the
    demos where the implicit assumption was made that [[string equal
    $tcl_platform(platform) "unix"]] meant you were using X11.  To
    this end, we will add a ''windowingsystem'' subcommand to the
    ''tk'' command, and then using it to replace the cases where
    ''tcl_platform'' was being erroneously checked.  This command will
    return "x11" for an X11 server, "aqua" for the native Mac OS X
    window manager, "win32" for Windows, and "classic" for Classic
    MacOS.

~ Reference Implementation

The reference implementation is on the ''macosx-8-4-branch'' in the
SourceForge CVS repository.

~ Copyright

This document has been placed in the public domain.

TIP:		109
Title:		New Look for Checkbutton and Radiobutton on Unix
State:		Final
Type:		Project
Tcl-Version:	8.5
Vote:		Done
Post-History:	
Version:	$Revision: 1.4 $
Author:		Brian Griffin <[email protected]>
Created:	01-Oct-2002


~ Abstract

This TIP proposes changing the look of the Tk checkbutton and
radiobutton widgets on Unix to more closely match the Windows
counterparts.

~ Rationale

The current visual aspect of the Unix version checkbutton and
radiobutton has proved to be confusing to users.  The distinction
between selected (On) and unselected (Off) states are not visually
different enough to clearly identify one from another.  Indeed, in the
rare case where only one checkbutton is present, one cannot tell for
certain if the state is On or Off.  With a check or dot mark icon
(dependent on the type of widget) displayed in the Windows version,
there is no question about the state of the widget.

~ Proposed Changes

The checkbutton shall (when the indicator is turned on) display an
check-mark or other distinguishing icon/symbol that clearly indicates
an On state.  The Off state will be displayed with an empty box.  The
state values will ''not'' be indicated by changing relief or
background color.

The radiobutton shall (when the indicator is turned on) display an dot
mark or other distinguishing icon/symbol that clearly indicates a
selected or On state.  The unselected or Off state shall be displayed
with an empty diamond.  The state values will ''not'' be indicated by
changing relief or background color.

~ Copyright

This document has been placed in the public domain.

TIP:            11
Title:          Tk Menubutton Enhancement: -compound option for menubutton
Version:        $Revision: 1.5 $
Author:         Todd Helfter <[email protected]>
State:          Final
Type:           Project
Tcl-Version:    8.4
Vote:           Done
Created:        16-Nov-2000
Post-History:


~ Abstract

This TIP describes how to change the menubutton in the Tk core to add
a -compound option to display both text and images.  This behavior
already exists in the button widget.

~ Rationale

In order to have a menubutton with both text and images, this change
is needed.  This change facilitates the use of an image for the
menubutton face with text on top.  Like the button widget, the
-compound option will accept these values: none, center, left, right,
top, bottom.

~ Reference Implementation

This TIP proposes to change the internals of the menubutton.

The changes necessary to accomplish this are:

   1. Extend the structure ''TkMenuButton'' in
      ''generic/tkMenubutton.h'' with a new field of type ''int'' to
      hold the value of the compound setting.

   2. Add an enumeration of valid -compound options in
      ''generic/tkMenubutton.h''.

   3. Modify ''generic/tkMenuButton.c'' and ''unix/tkUnixMenubu.c'' in
      such a way to process this new option.  Note: The windows port
      of Tk uses the ''unix/tkUnixMenubu.c'' file.  So this change is
      portable to both Unix and windows.

   4. Change ''tests/menubut.test'' so that the test for configure
      options checks for 33 instead of the current 32.

   5. Change ''doc/menubutton.n'' to show the new option under widget
      specific options.

~ Copyright

This document has been placed in the public domain.

~ Patch

|Index: doc/menubutton.n
|===================================================================
|RCS file: /cvsroot/tk/doc/menubutton.n,v
|retrieving revision 1.3
|diff -c -r1.3 menubutton.n
|*** menubutton.n	2000/08/25 06:58:32	1.3
|--- menubutton.n	2000/11/16 14:37:15
|***************
|*** 26,31 ****
|--- 26,39 ----
|  \-disabledforeground	\-padx
|  .SE
|  .SH "WIDGET-SPECIFIC OPTIONS"
|+ .OP \-compound compound Compound
|+ Specifies whether the menubutton should display both an image and text,
|+ and if so, where the image should be placed relative to the text.
|+ Valid values for this option are \fBbottom\fR, \fBcenter\fR,
|+ \fBleft\fR, \fBnone\fR, \fBright\fR and \fBtop\fR.  The default value
|+ is \fBnone\fR, meaning that the menubutton will display either an image or
|+ text, depending on the values of the \fB\-image\fR and \fB\-bitmap\fR
|+ options.
|  .VS
|  .OP \-direction direction Height
|  Specifies where the menu is going to be popup up. \fBabove\fR tries to
|Index: generic/tkMenubutton.c
|===================================================================
|RCS file: /cvsroot/tk/generic/tkMenubutton.c,v
|retrieving revision 1.4
|diff -c -r1.4 tkMenubutton.c
|*** tkMenubutton.c	1999/04/24 01:50:49	1.4
|--- tkMenubutton.c	2000/11/16 14:37:16
|***************
|*** 37,42 ****
|--- 37,51 ----
|  };
|  
|  /*
|+  * The following table defines the legal values for the -compound option.
|+  * It is used with the "enum compound" declaration in tkButton.h
|+  */
|+ 

|+ static char *compoundStrings[] = {
|+     "bottom", "center", "left", "none", "right", "top", (char *) NULL
|+ };
|+ 

|+ /*
|   * Information used for parsing configuration specs:
|   */
|  
|***************
|*** 113,118 ****
|--- 122,130 ----
|      {TK_OPTION_RELIEF, "-relief", "relief", "Relief",
|	 DEF_MENUBUTTON_RELIEF, -1, Tk_Offset(TkMenuButton, relief), 
|	   0, 0, 0},
|+     {TK_OPTION_STRING_TABLE, "-compound", "compound", "Compound",
|+          DEF_BUTTON_COMPOUND, -1, Tk_Offset(TkMenuButton, compound), 0,
|+          (ClientData) compoundStrings, 0},
|      {TK_OPTION_STRING_TABLE, "-state", "state", "State",
|	 DEF_MENUBUTTON_STATE, -1, Tk_Offset(TkMenuButton, state),
|	 0, (ClientData) stateStrings, 0},
|Index: generic/tkMenubutton.h
|===================================================================
|RCS file: /cvsroot/tk/generic/tkMenubutton.h,v
|retrieving revision 1.5
|diff -c -r1.5 tkMenubutton.h
|*** tkMenubutton.h	1999/04/16 01:51:19	1.5
|--- tkMenubutton.h	2000/11/16 14:37:16
|***************
|*** 25,30 ****
|--- 25,39 ----
|  #endif
|  
|  /*
|+  * Legal values for the "compound" field of TkButton records.
|+  */
|+ 

|+ enum compound {
|+     COMPOUND_BOTTOM, COMPOUND_CENTER, COMPOUND_LEFT, COMPOUND_NONE,
|+         COMPOUND_RIGHT, COMPOUND_TOP
|+ };

|+ 






|+ /*
|   * Legal values for the "orient" field of TkMenubutton records.
|   */
|  
|***************
|*** 161,166 ****
|--- 170,179 ----
|      /*
|	* Miscellaneous information:
|	*/
|+ 

|+     int compound;               /* Value of -compound option; specifies whether
|+                                  * the button should show both an image and
|+                                  * text, and, if so, how. */
|  
|      enum direction direction;	/* Direction for where to pop the menu.
|				  * Valid directions are "above", "below",
|Index: tests/menubut.test
|===================================================================
|RCS file: /cvsroot/tk/tests/menubut.test,v
|retrieving revision 1.5
|diff -c -r1.5 menubut.test
|*** menubut.test	1999/04/21 21:53:29	1.5
|--- menubut.test	2000/11/16 14:37:18
|***************
|*** 138,144 ****
|  } {3}
|  test menubutton-3.7 {ButtonWidgetCmd procedure, "configure" option} {
|      llength [.mb configure]
|! } {32}
|  test menubutton-3.8 {ButtonWidgetCmd procedure, "configure" option} {
|      list [catch {.mb configure -gorp} msg] $msg
|  } {1 {unknown option "-gorp"}}
|--- 138,144 ----
|  } {3}
|  test menubutton-3.7 {ButtonWidgetCmd procedure, "configure" option} {
|      llength [.mb configure]
|! } {33}
|  test menubutton-3.8 {ButtonWidgetCmd procedure, "configure" option} {
|      list [catch {.mb configure -gorp} msg] $msg
|  } {1 {unknown option "-gorp"}}
|Index: unix/tkUnixMenubu.c
|===================================================================
|RCS file: /cvsroot/tk/unix/tkUnixMenubu.c,v
|retrieving revision 1.4
|diff -c -r1.4 tkUnixMenubu.c
|*** tkUnixMenubu.c	1999/09/21 06:43:01	1.4
|--- tkUnixMenubu.c	2000/11/16 14:37:18
|***************
|*** 75,83 ****
|      Pixmap pixmap;
|      int x = 0;			/* Initialization needed only to stop
|				  * compiler warning. */
|!     int y;
|      register Tk_Window tkwin = mbPtr->tkwin;
|!     int width, height;
|  
|      mbPtr->flags &= ~REDRAW_PENDING;
|      if ((mbPtr->tkwin == NULL) || !Tk_IsMapped(tkwin)) {
|--- 75,85 ----
|      Pixmap pixmap;
|      int x = 0;			/* Initialization needed only to stop
|				  * compiler warning. */
|!     int y = 0;
|      register Tk_Window tkwin = mbPtr->tkwin;
|!     int width, height, fullWidth, fullHeight;
|!     int imageXOffset, imageYOffset, textXOffset, textYOffset;
|!     int haveImage = 0, haveText = 0;
|  
|      mbPtr->flags &= ~REDRAW_PENDING;
|      if ((mbPtr->tkwin == NULL) || !Tk_IsMapped(tkwin)) {
|***************
|*** 96,101 ****
|--- 98,112 ----
|	 border = mbPtr->normalBorder;
|      }

|  
|+     if (mbPtr->image != None) {
|+ 	Tk_SizeOfImage(mbPtr->image, &width, &height);
|+ 	haveImage = 1;
|+     } else if (mbPtr->bitmap != None) {
|+ 	Tk_SizeOfBitmap(mbPtr->display, mbPtr->bitmap, &width, &height);
|+ 	haveImage = 1;
|+     }
|+     haveText = (mbPtr->textWidth != 0 && mbPtr->textHeight != 0);
|+ 

|      /*
|	* In order to avoid screen flashes, this procedure redraws
|	* the menu button in a pixmap, then copies the pixmap to the
|***************
|*** 107,141 ****
|	     Tk_Width(tkwin), Tk_Height(tkwin), Tk_Depth(tkwin));
|      Tk_Fill3DRectangle(tkwin, pixmap, border, 0, 0, Tk_Width(tkwin),
|	     Tk_Height(tkwin), 0, TK_RELIEF_FLAT);
|- 

|-     /*
|-      * Display image or bitmap or text for button.
|-      */
|  
|!     if (mbPtr->image != None) {
|! 	Tk_SizeOfImage(mbPtr->image, &width, &height);
|! 

|! 	imageOrBitmap:
|! 	TkComputeAnchor(mbPtr->anchor, tkwin, 0, 0, 
|! 		width + mbPtr->indicatorWidth, height, &x, &y);
|! 	if (mbPtr->image != NULL) {
|! 	    Tk_RedrawImage(mbPtr->image, 0, 0, width, height, pixmap,
|! 		    x, y);
|! 	} else {
|! 	    XCopyPlane(mbPtr->display, mbPtr->bitmap, pixmap,
|! 		    gc, 0, 0, (unsigned) width, (unsigned) height, x, y, 1);
|! 	}
|!     } else if (mbPtr->bitmap != None) {
|! 	Tk_SizeOfBitmap(mbPtr->display, mbPtr->bitmap, &width, &height);
|! 	goto imageOrBitmap;
|      } else {
|! 	TkComputeAnchor(mbPtr->anchor, tkwin, mbPtr->padX, mbPtr->padY,
|! 		mbPtr->textWidth + mbPtr->indicatorWidth,
|! 		mbPtr->textHeight, &x, &y);
|! 	Tk_DrawTextLayout(mbPtr->display, pixmap, gc, mbPtr->textLayout, x, y,
|! 		0, -1);
|! 	Tk_UnderlineTextLayout(mbPtr->display, pixmap, gc, mbPtr->textLayout,
|! 		x, y, mbPtr->underline);
|      }

|  
|      /*
|--- 118,223 ----
|	     Tk_Width(tkwin), Tk_Height(tkwin), Tk_Depth(tkwin));
|      Tk_Fill3DRectangle(tkwin, pixmap, border, 0, 0, Tk_Width(tkwin),
|	     Tk_Height(tkwin), 0, TK_RELIEF_FLAT);
|  
|!     imageXOffset = 0;
|!     imageYOffset = 0;
|!     textXOffset = 0;
|!     textYOffset = 0;
|!     fullWidth = 0;
|!     fullHeight = 0;
|! 

|!     if (mbPtr->compound != COMPOUND_NONE && haveImage && haveText) {
|! 

|!         switch ((enum compound) mbPtr->compound) {
|!             case COMPOUND_TOP:
|!             case COMPOUND_BOTTOM: {
|!                 /* Image is above or below text */
|!                 if (mbPtr->compound == COMPOUND_TOP) {
|!                     textYOffset = height + mbPtr->padY;
|!                 } else {
|!                     imageYOffset = mbPtr->textHeight + mbPtr->padY;
|!                 }
|!                 fullHeight = height + mbPtr->textHeight + mbPtr->padY;
|!                 fullWidth = (width > mbPtr->textWidth ? width :
|!                         mbPtr->textWidth);
|!                 textXOffset = (fullWidth - mbPtr->textWidth)/2;
|!                 imageXOffset = (fullWidth - width)/2;
|!                 break;
|!             }
|!             case COMPOUND_LEFT:
|!             case COMPOUND_RIGHT: {
|!                 /* Image is left or right of text */
|!                 if (mbPtr->compound == COMPOUND_LEFT) {
|!                     textXOffset = width + mbPtr->padX;
|!                 } else {
|!                     imageXOffset = mbPtr->textWidth + mbPtr->padX;
|!                 }
|!                 fullWidth = mbPtr->textWidth + mbPtr->padX + width;
|!                 fullHeight = (height > mbPtr->textHeight ? height :
|!                         mbPtr->textHeight);
|!                 textYOffset = (fullHeight - mbPtr->textHeight)/2;
|!                 imageYOffset = (fullHeight - height)/2;
|!                 break;
|!             }
|!             case COMPOUND_CENTER: {
|!                 /* Image and text are superimposed */
|!                 fullWidth = (width > mbPtr->textWidth ? width :
|!                         mbPtr->textWidth);
|!                 fullHeight = (height > mbPtr->textHeight ? height :
|!                         mbPtr->textHeight);
|!                 textXOffset = (fullWidth - mbPtr->textWidth)/2;
|!                 imageXOffset = (fullWidth - width)/2;
|!                 textYOffset = (fullHeight - mbPtr->textHeight)/2;
|!                 imageYOffset = (fullHeight - height)/2;
|!                 break;
|!             }
|!             case COMPOUND_NONE: {break;}
|!         }
|! 

|!         TkComputeAnchor(mbPtr->anchor, tkwin, 0, 0,
|!                 mbPtr->indicatorWidth + fullWidth, fullHeight,
|! 		&x, &y);
|! 

|!         if (mbPtr->image != NULL) {
|!             Tk_RedrawImage(mbPtr->image, 0, 0, width, height, pixmap,
|!                     x + imageXOffset, y + imageYOffset);
|!         }
|!         if (mbPtr->bitmap != None) {
|!             XCopyPlane(mbPtr->display, mbPtr->bitmap, pixmap,
|!                     gc, 0, 0, (unsigned) width, (unsigned) height, 
|! 		    x + imageXOffset, y + imageYOffset, 1);
|!         }
|!         if (haveText) {
|!             Tk_DrawTextLayout(mbPtr->display, pixmap, gc, mbPtr->textLayout, 
|! 		    x  + textXOffset, y + textYOffset ,
|!                     0, -1);
|!             Tk_UnderlineTextLayout(mbPtr->display, pixmap, gc, 
|! 		    mbPtr->textLayout, x + textXOffset, y + textYOffset ,
|! 		    mbPtr->underline);
|!         }
|      } else {
|!        if (mbPtr->image != NULL) {
|!            TkComputeAnchor(mbPtr->anchor, tkwin, 0, 0,
|!                    width + mbPtr->indicatorWidth, height, &x, &y);
|!            Tk_RedrawImage(mbPtr->image, 0, 0, width, height, pixmap,
|!                    x + imageXOffset, y + imageYOffset);
|!        } else if (mbPtr->bitmap != None) {
|!            TkComputeAnchor(mbPtr->anchor, tkwin, 0, 0,
|!                    width + mbPtr->indicatorWidth, height, &x, &y);
|!            XCopyPlane(mbPtr->display, mbPtr->bitmap, pixmap,
|!                    gc, 0, 0, (unsigned) width, (unsigned) height, 
|! 		   x + imageXOffset, y + imageYOffset, 1);
|!        } else {
|!            TkComputeAnchor(mbPtr->anchor, tkwin, mbPtr->padX, mbPtr->padY,
|!                    mbPtr->textWidth + mbPtr->indicatorWidth,
|!                    mbPtr->textHeight, &x, &y);
|!            Tk_DrawTextLayout(mbPtr->display, pixmap, gc, mbPtr->textLayout, 
|! 		   x  + textXOffset, y + textYOffset ,
|!                    0, -1);
|!            Tk_UnderlineTextLayout(mbPtr->display, pixmap, gc, 
|! 		   mbPtr->textLayout, x + textXOffset, y + textYOffset ,
|! 		   mbPtr->underline);
|!         }
|      }

|  
|      /*
|***************
|*** 252,305 ****
|      TkMenuButton *mbPtr;	/* Widget record for menu button. */
|  {

|      int width, height, mm, pixels;
|  
|      mbPtr->inset = mbPtr->highlightWidth + mbPtr->borderWidth;
|      if (mbPtr->image != None) {
|	 Tk_SizeOfImage(mbPtr->image, &width, &height);
|! 	if (mbPtr->width > 0) {
|! 	    width = mbPtr->width;
|! 	}
|! 	if (mbPtr->height > 0) {
|! 	    height = mbPtr->height;
|! 	}
|      } else if (mbPtr->bitmap != None) {
|	 Tk_SizeOfBitmap(mbPtr->display, mbPtr->bitmap, &width, &height);
|! 	if (mbPtr->width > 0) {
|! 	    width = mbPtr->width;
|! 	}
|! 	if (mbPtr->height > 0) {
|! 	    height = mbPtr->height;
|! 	}
|!     } else {
|	 Tk_FreeTextLayout(mbPtr->textLayout);
|	 mbPtr->textLayout = Tk_ComputeTextLayout(mbPtr->tkfont, mbPtr->text,
|		 -1, mbPtr->wrapLength, mbPtr->justify, 0, &mbPtr->textWidth,
|		 &mbPtr->textHeight);
|! 	width = mbPtr->textWidth;
|! 	height = mbPtr->textHeight;
|! 	if (mbPtr->width > 0) {
|! 	    width = mbPtr->width * Tk_TextWidth(mbPtr->tkfont, "0", 1);
|! 	}
|! 	if (mbPtr->height > 0) {
|! 	    Tk_FontMetrics fm;
|  
|! 	    Tk_GetFontMetrics(mbPtr->tkfont, &fm);
|! 	    height = mbPtr->height * fm.linespace;
|	 }

|! 	width += 2*mbPtr->padX;
|! 	height += 2*mbPtr->padY;
|      }

|  
|      if (mbPtr->indicatorOn) {
|! 	mm = WidthMMOfScreen(Tk_Screen(mbPtr->tkwin));
|! 	pixels = WidthOfScreen(Tk_Screen(mbPtr->tkwin));
|! 	mbPtr->indicatorHeight= (INDICATOR_HEIGHT * pixels)/(10*mm);
|! 	mbPtr->indicatorWidth = (INDICATOR_WIDTH * pixels)/(10*mm)
|! 		+ 2*mbPtr->indicatorHeight;
|! 	width += mbPtr->indicatorWidth;
|      } else {
|! 	mbPtr->indicatorHeight = 0;
|! 	mbPtr->indicatorWidth = 0;
|      }

|  
|      Tk_GeometryRequest(mbPtr->tkwin, (int) (width + 2*mbPtr->inset),
|--- 334,446 ----
|      TkMenuButton *mbPtr;	/* Widget record for menu button. */
|  {

|      int width, height, mm, pixels;
|+     int  avgWidth, txtWidth, txtHeight;
|+     int haveImage = 0, haveText = 0;
|+     Tk_FontMetrics fm;
|  
|      mbPtr->inset = mbPtr->highlightWidth + mbPtr->borderWidth;
|+ 

|+     width = 0;
|+     height = 0;
|+     txtWidth = 0;
|+     txtHeight = 0;
|+     avgWidth = 0;
|+ 

|      if (mbPtr->image != None) {
|	 Tk_SizeOfImage(mbPtr->image, &width, &height);
|! 	haveImage = 1;
|      } else if (mbPtr->bitmap != None) {
|	 Tk_SizeOfBitmap(mbPtr->display, mbPtr->bitmap, &width, &height);
|! 	haveImage = 1;

|!     }
|! 

|!     if (haveImage == 0 || mbPtr->compound != COMPOUND_NONE) {
|	 Tk_FreeTextLayout(mbPtr->textLayout);
|+ 

|	 mbPtr->textLayout = Tk_ComputeTextLayout(mbPtr->tkfont, mbPtr->text,
|		 -1, mbPtr->wrapLength, mbPtr->justify, 0, &mbPtr->textWidth,
|		 &mbPtr->textHeight);
|! 	txtWidth = mbPtr->textWidth;
|! 	txtHeight = mbPtr->textHeight;
|!         avgWidth = Tk_TextWidth(mbPtr->tkfont, "0", 1);
|!         Tk_GetFontMetrics(mbPtr->tkfont, &fm);
|!         haveText = (txtWidth != 0 && txtHeight != 0);
|!     }
|! 

|!     /*
|!      * If the menubutton is compound (ie, it shows both an image and text),
|!      * the new geometry is a combination of the image and text geometry.
|!      * We only honor the compound bit if the menubutton has both text and
|!      * an image, because otherwise it is not really a compound menubutton.
|!      */
|  
|!     if (mbPtr->compound != COMPOUND_NONE && haveImage && haveText) {
|!         switch ((enum compound) mbPtr->compound) {
|!             case COMPOUND_TOP:
|!             case COMPOUND_BOTTOM: {
|!                 /* Image is above or below text */
|!                 height += txtHeight + mbPtr->padY;
|!                 width = (width > txtWidth ? width : txtWidth);
|!                 break;
|!             }
|!             case COMPOUND_LEFT:
|!             case COMPOUND_RIGHT: {
|!                 /* Image is left or right of text */
|!                 width += txtWidth + mbPtr->padX;
|!                 height = (height > txtHeight ? height : txtHeight);
|!                 break;
|!             }
|!             case COMPOUND_CENTER: {
|!                 /* Image and text are superimposed */
|!                 width = (width > txtWidth ? width : txtWidth);
|!                 height = (height > txtHeight ? height : txtHeight);
|!                 break;
|!             }
|!             case COMPOUND_NONE: {break;}
|!         }
|!         if (mbPtr->width > 0) {
|!             width = mbPtr->width;
|!         }
|!         if (mbPtr->height > 0) {
|!             height = mbPtr->height;
|!         }
|!         width += 2*mbPtr->padX;
|!         height += 2*mbPtr->padY;
|!     } else {
|! 	if (haveImage) {
|!             if (mbPtr->width > 0) {
|!                 width = mbPtr->width;
|!             }
|!             if (mbPtr->height > 0) {
|!                 height = mbPtr->height;
|!             }
|! 	} else {
|! 	    width = txtWidth;
|! 	    height = txtHeight;
|!             if (mbPtr->width > 0) {
|!                 width = mbPtr->width * avgWidth;
|!             }
|!             if (mbPtr->height > 0) {
|!                 height = mbPtr->height * fm.linespace;
|!             }
|	 }

|!     }
|! 

|!     if (! haveImage) {
|!         width += 2*mbPtr->padX;
|!         height += 2*mbPtr->padY;
|      }

|  
|      if (mbPtr->indicatorOn) {
|!         mm = WidthMMOfScreen(Tk_Screen(mbPtr->tkwin));
|!         pixels = WidthOfScreen(Tk_Screen(mbPtr->tkwin));
|!         mbPtr->indicatorHeight= (INDICATOR_HEIGHT * pixels)/(10*mm);
|!         mbPtr->indicatorWidth = (INDICATOR_WIDTH * pixels)/(10*mm)
|!     	    + 2*mbPtr->indicatorHeight;
|!         width += mbPtr->indicatorWidth;
|      } else {
|!         mbPtr->indicatorHeight = 0;
|!         mbPtr->indicatorWidth = 0;
|      }

|  
|      Tk_GeometryRequest(mbPtr->tkwin, (int) (width + 2*mbPtr->inset),

TIP:            110
Title:          Add a Tristate Mode to the Checkbutton and Radiobutton
Version:        $Revision: 1.12 $
Author:         Brian Griffin <[email protected]>
State:          Final
Type:           Project
Vote:           Done
Created:        01-Oct-2002
Post-History:   
Tcl-Version:    8.5


~ Abstract

This TIP proposes adding a third value (tristate) to the checkbutton
and radiobutton widgets and corresponding display.

~ Rationale

In order to meet more demanding requirements of todays complex
graphical user interfaces, it would help add to the functionality of
the basic checkbutton and radiobutton widgets. To support using
checkbuttons and radiobuttons along with multiple selection, it is
necessary for these buttons to be able to display a third state,
i.e., both On and Off.  This indicates the situation where a property
has a particular value for some members of a selection set, but
not others.

~ Proposed Change

The change would add a third "tristate" value to current On Off values
of the checkbutton and radiobutton widgets.  The widget would then
display the check/dot mark (as appropriate) along with a "grayed"
background, for example.  When the checkbutton or radiobutton is 
invoked (i.e. clicked on) it would behave exactly as it does currently, 
setting the variable to the On value.

There is a concern that the Unix version of these widgets do not have
a sufficiently different appearance when in the tristate state
compared with the On and Off states.  This issue is addressed in
[109].

~ Propsal Specifics

   1. Identify a third, platform specific, presentation (e.g. check
      with grayed background) to represent the tristate value,

   1. Add the option "-tristatevalue" to specify the match value
      only. The default value of this option will be "{}",

   1. Change the behavior to display the indeterminate or tristate
      presentation when the associated variable's value matches the
      -tristatevalue, and

   1. Add the option "-tristateimage" to specify an image to display
      (in place of the image option) when the checkbutton or
      radiobutton has the tristate value (as defined above.)

~ Copyright

This document has been placed in the public domain.

TIP:            111
Title:          Dictionary Values and Manipulators
Version:        $Revision: 1.12 $
Author:         Donal K. Fellows <[email protected]>
Author:         David S. Cargo <[email protected]>
State:          Final
Type:           Project
Vote:           Done
Created:        05-Oct-2002
Post-History:   
Tcl-Version:    8.5


~ Abstract

This TIP proposes adding a standard value format (and supporting
commands) to Tcl that implements a value-to-value mapping, just as
Tcl's list values can be regarded as implementing a number-to-value
mapping.

~ Rationale

What is a dictionary?  It is a translation from arbitrary values to
arbitrary values, often also known as an associative map.  Many
computer languages, especially higher-level ones, have them as part of
the language or the standard library.  It would be nice to have them
in Tcl too.

Now, I realise that Tcl already contains arrays which provide
dictionary functionality, but they are not quite the same thing.
Tcl's arrays are collections of variables indexable by name, and not
collections of values.  This has some far-reaching implications; it is
possible to set traces on individual elements of the array, but it is
not possible to pass the array by value.  However, one of the main
concerns is the sheer cost of arrays in terms of memory space; aside
from the hash table used as the core of the implementation (and the
representations of the keys and values, of course) there is a
substantial overhead for each array to support traces on the array as
a whole, plus a similar overhead ''per element'' that stems from the
fact that elements are variables in their own right.  By contrast, a
dictionary value should be a lot more frugal.

~ Value Syntax and Semantics

Naturally, it is desirable for dictionary values to have
human-readable forms that are similar to those that currently exist.
I propose using ''key value key value ...'' form with list-style
quoting for keys and values that contain characters that are
significant to Tcl, which should be immediately familiar to users of
the '''array get''' and '''array set''' commands.  No special
interpretation will be placed on the amount of whitespace separating
keys and values, just as with lists (indeed, any list with an even
number of elements can be regarded as a dictionary.)  For example, the
following value represents a mapping from selected languages to a
possible program to invoke to compile them:

|	C gcc C++ g++ FORTRAN f77 Java javac

Empty dictionaries are those that contain no mappings from keys to
values.  Any representation of an empty list will also be a
representation of an empty dictionary.  There is no upper bound on the
number of items that a dictionary may hold.

It should be specially noted that dictionary values have copy-on-write
semantics just like lists.  This means that if I hand a dictionary
value into a procedure as an argument, and that procedure updates the
variable containing that value, the value as seen by the caller will
not have changed.  This is in complete contrast with arrays which
cannot (currently) be passed by value other than through using '''array
get''' to convert the array to a list form and '''array set''' to convert
back again.

This specification does not state what order the keys and values are
listed in.  That depends on the implementation.

~ Command Syntax and Semantics

I propose that all operations that work with dictionary values (where
not done through adaptations of existing commands) will go through the
'''dict''' command.  The alternatives are "array" which is already in
use, "dictionary" which is rather long for what I believe will be a
fairly commonly used command, "alist" (association list) which is
probably too easy to confuse with existing commands, and "map" which
is probably better reserved for future use as something for applying
an operation to a list (or other collection of values), "hash" (which
is perhaps too common), and "table" (which is used for this type of
data structure in the Icon programming language).

Most subcommands operate on either a dictionary value (''exists'',
''for'', ''get'', ''info'', ''keys'', ''remove'', ''replace'',
''size'', and ''values''), or on a variable containing a dictionary
value (''append'', ''incr'', ''lappend'', ''set'', and ''unset'').

Proposed subcommands:

 dict create:	Make a dictionary.

 > '''dict create''' ?''key1 value1 key2 value2 ...''?

 > This will create a new dictionary from the given keys and values
   and return it as the result.  The command will take an even number
   of arbitrary strings (or other objects, naturally) and will use the
   first, third, fifth, etc. as keys and the second, fourth, sixth,
   etc. as values.  From the point of view of string representations,
   this command will behave the same as the '''list''' command with an
   even number of arguments.  There is no restriction on the possible
   representations of keys or values.  It is legal to call this
   command with no arguments at all, which creates an empty
   dictionary.

 dict get:	Get value for given key.

 > '''dict get''' ''dictionaryValue'' ?''key ...''?

 > Given a dictionary value (first argument) and a key (second
   argument), this will retrieve the value for that key.  Where
   several keys are supplied, the behaviour of the command shall be as
   if the result of '''dict get''' ''dictVal key'' was passed as the first
   argument to '''dict get'' with the remaining arguments as second
   (and possibly subsequent) arguments.  This facilitates lookups in
   nested dictionaries.  For example, the following two commands are
   equivalent:

|	dict get $dict foo bar spong
|	dict get [dict get [dict get $dict foo] bar] spong

 > If no keys are provided, dict would return a list containing pairs
   of elements in a manner similar to '''array get'''.  That is, the
   first element of each pair would be the key and the second element
   would be the value for that key.

 > It is an error to attempt to retrieve a value for a key that is not
   present in the dictionary.

 dict replace:	Create a new dictionary that is a copy of an old one
		except with some values different or some extra
		key/value pairs added.

 > '''dict replace''' ''dictionaryValue'' ?''key value ...''?

 > This is very much the analogue of '''lreplace''', taking a dictionary
   value as its first argument and then a list of key/value pairs.
   The result of the command is a new dictionary value that is a copy
   of the supplied dictionary other than that whenever a key is one of
   those supplied to this command, the returned dictionary will map
   that key to the associated value.  It is legal for this command to
   be called with no key/value pairs, but illegal for this command to
   be called with a key but no value.

 dict remove:	Create a new dictionary that is a copy of an old one
		except without the key/value mappings whose keys are
		listed.

 > '''dict remove''' ''dictionaryValue'' ?''key key ...''?

 > This operation does what '''dict replace''' can't do; removes keys
   and values.  The result of the command is a new dictionary value
   that does not contain mappings for any of the keys listed; it is
   not an error if either there are no keys listed, or if any of the
   listed keys does not exist in the supplied dictionary.

 dict set:	Set value for given key in a dictionary in a variable.

 > '''dict set''' ''dictionaryVar key'' ?''key ...''? ''value''

 > This operation takes the name of a variable containing a dictionary
   value and places an updated dictionary value in that variable
   containing a mapping from the given key to the given value.  In a
   manner analogous to '''lset''', where multiple keys are present, they
   do indexing into nested dictionaries.

 dict unset:	Remove association for given key in a dictionary in a
		variable.

 > '''dict unset''' ''dictionaryVar key'' ?''key ...''?

 > This operation takes the name of a variable containing a dictionary
   value and places an updated dictionary value in that variable that
   does not contain a mapping for the given key.  Where multiple keys
   are present, this describes a path through nested dictionaries to
   the mapping to remove.  At least one key must be specified.

 dict keys:	List all keys (with optional criteria matching) in
		dictionary.

 > '''dict keys''' ''dictionaryValue'' ?''globPattern''?

 > Return a list of all keys in the given dictionary value.  If a
   pattern is supplied, only those keys that match it (according to
   the rules of '''string match''') will be returned.  The returned keys
   will be in an arbitrary implementation-specific order.

 dict values:	List all values (with optional criteria matching) in
		the dictionary.

 > '''dict values''' ''dictionaryValue'' ?''globPattern''?

 > Return a list of all values in the given dictionary value.  If a
   pattern is supplied, only those values that match it (according to
   the rules of '''string match''') will be returned.  The returned keys
   will be in an arbitrary implementation-specific order, though where
   no pattern is supplied the ''i''th key returned by '''dict keys'''
   will be the key for the ''i''th value returned by '''dict values'''
   applied to the same dictionary value.

 dict for:	Iterate across all key/value mappings in the
		dictionary.

 > '''dict for''' {''keyVar valueVar''} ''dictionaryValue body''

 > This takes three arguments, the first a pair of variable names (for
   the key and value respectively of each mapping in the dictionary),
   the second the dictionary value to iterate across, and the third a
   script to be evaluated for each mapping with the key and value
   variables set appropriately (in the manner of '''foreach'''.)  The
   result of the command is an empty string.  If any evaluation of the
   body generates a ''TCL_BREAK'' result, no further pairs from the
   dictionary will be iterated over and the '''dict for''' command will
   terminate successfully immediately.  If any evaluation of the body
   generates a ''TCL_CONTINUE'' result, this shall be treated exactly
   like a normal ''TCL_OK'' result.

 dict filter:	Create a new dictionary from an old one containing just
		a selection of key/value pairs.

 > '''dict filter''' ''dictionaryValue'' '''key''' ''globPattern''

 > '''dict filter''' ''dictionaryValue'' '''value''' ''globPattern''

 > '''dict filter''' ''dictionaryValue'' '''script''' {''keyVar valueVar''} ''script''

 > This takes a dictionary value and returns a new dictionary that
   contains just those key/value pairs that match the specified rule.
   Three rules are outlined above.  The '''key''' rule only matches those
   key/value pairs whose keys match the given glob-style pattern.  The
   '''value''' rule only matches those key/value pairs whose values match
   the given glob-style pattern.  The '''script''' rule tests for matching
   by assigning the key to the ''keyVar'' and the value to the
   ''valueVar'', and then evaluating the given script which should
   return a boolean value (with the key/value pair only being included
   in the result of the '''dict filter''' when a true value is returned.)

 dict append:	Append a string to the value for a particular key in
		the dictionary.

 > '''dict append''' ''dictionaryVar key'' ?''string ...''?

 > This appends the given string (or strings) to the value that the
   given key maps to in the dictionary value contained in the given
   variable, writing the resulting dictionary value back to that
   variable.  Non-existent keys are treated as if they map to an empty
   string.

 dict incr:	Increment the value for a particular key in the
		dictionary.

 > '''dict incr''' ''dictionaryVar key'' ?''increment''?

 > This adds the given increment value (an integer that defaults to 1
   if not specified) to the value that the given key maps to in the
   dictionary value contained in the given variable, writing the
   resulting dictionary value back to that variable.  Non-existent
   keys are treated as if they map to 0.  It is an error to increment
   a value for an existing key if that value is not an integer.

 dict lappend:	Append an item to the list-value for a particular key
		in the dictionary.

 > '''dict lappend''' ''dictionaryVar key'' ?''item ...''?

 > This appends the given items to the list value that the given key
   maps to in the dictionary value contained in the given variable,
   writing the resulting dictionary value back to that variable.
   Non-existent keys are treated as if they map to an empty list, and
   it is legal for there to be no items to append to the list.  It is
   an error for the value that the key maps to to not be representable
   as a list.

 dict exists:	Test whether a mapping exists for a key.

 > '''dict exists''' ''dictionaryValue key'' ?''key ...''?

 > This returns a boolean value indicating whether the given key (or
   path of keys through a set of nested dictionaries) exists in the
   given dictionary value.  This returns a true value exactly when
   '''dict get''' on that path will succeed.

 dict size:	Get the number of key/value mappings in a dictionary.

 > '''dict size''' ''dictionaryValue''

 > This returns the size of the dictionary, which will be exactly half
   the value that '''llength''' ''dictionaryValue'' would return.  It is an
   error to apply this command to a non-dictionary value.

 dict info:	Get implementation-specific information about the
		dictionary value.

 > '''dict info''' ''dictionaryValue''

 > This returns information (intended for display to people) about the
   given dictionary though the format of this data is dependent on the
   implementation of the dictionary.  For dictionaries that are
   implemented by hash tables, it is expected that this will return
   the string produced by ''Tcl_HashStats()''.

~ Other Related Changes

There are a few other commands that change:

 * '''array set''' will take a dictionary instead of (or as well as) a
   list as its final argument.

 * '''array get''' will return a dictionary.

 * '''string map''' will take a dictionary instead of (or as well as) a
   list as its map argument.

Naturally, dictionary handling will form its own maintenance area.  [16] and [24] will be updated as necessary.

~ C API

There will be a new public structure and a few new public functions to
allow C-level access to dictionary values:

The new structure (called ''Tcl_DictSearch'') will be there to allow
for searches (i.e. traversals) of a dictionary.  This TIP does not
specify the fields of the structure; the declaration is just to allow
for allocation of these structures on the C stack.

Many public API functions are capable of generating error messages;
these generally indicate some type-conversion failure.  Sharing
constraint violations (where applicable) cause panics as they indicate
basic programming errors which should not be causable by scripts.  The
public API functions are:

 > Tcl_Obj *'''Tcl_NewDictObj'''(void);

 > Tcl_Obj *'''Tcl_DbNewDictObj'''(CONST char *''file'', int ''line'');

These functions (in non-debug and debug versions) create a new
dictionary object and return it.

 > int '''Tcl_DictObjPut'''(Tcl_Interp *''interp'', Tcl_Obj *''dictPtr'',
	Tcl_Obj *''keyPtr'', Tcl_Obj *''valuePtr'');

This function inserts a new key/value pair into a dictionary, or
updates a key/value pair already in the dictionary.  The dictionary
object must be unshared.  Note that both the key and value objects
will have their reference count increased.  The return value is a
normal TCL_OK/TCL_ERROR result, with the ''interp'' for error
reporting.

 > int '''Tcl_DictObjGet'''(Tcl_Interp *''interp'', Tcl_Obj *''dictPtr'',
	Tcl_Obj *''keyPtr'', Tcl_Obj **''valuePtrPtr'');

This function looks up the value for a key in a dictionary.  The
variable pointed to by the ''valuePtrPtr'' argument is updated to
contain a reference to the value, or a NULL if there is no mapping for
the key in the dictionary.  No reference counts are manipulated by
this function.  The return value is a normal TCL_OK/TCL_ERROR result,
with the ''interp'' for error reporting.

 > int '''Tcl_DictObjRemove'''(Tcl_Interp *''interp'', Tcl_Obj *''dictPtr'',
	Tcl_Obj *''keyPtr'');

This function removes the key/value pair with the given key from the
dictionary.  It is not an error if the key is not present in the
dictionary.  The dictionary must be unshared.  The return value is a
normal TCL_OK/TCL_ERROR result, with the ''interp'' for error
reporting.

 > int '''Tcl_DictObjSize'''(Tcl_Interp *''interp'', Tcl_Obj *''dictPtr'',
	int *''sizePtr'');

This function updates the integer variable pointed to by ''sizePtr''
with the number of key/value pairs in the dictionary.  The return
value is a normal TCL_OK/TCL_ERROR result, with the ''interp'' for
error reporting.

 > int '''Tcl_DictObjFirst'''(Tcl_Interp *''interp'', Tcl_Obj *''dictPtr'',
	Tcl_DictSearch *''searchPtr'',
	Tcl_Obj **''keyPtrPtr'', Tcl_Obj **''valuePtrPtr'', int *''donePtr'');

This function starts a search of (i.e. iteration over) the given
dictionary, using the structure pointed to by ''searchPtr'' as
context.  The return value is a normal TCL_OK/TCL_ERROR result, with
the ''interp'' for error reporting.  Three variables are updated to
indicate what was found; ''keyPtrPtr'' is used for reporting the key
of a key/value pair and ''valuePtrPtr'' is used for reporting the
corresponding value.  Finally, ''donePtr'' is used for indicating
whether the search has found all the values; if the variable it points
to is set to 1 there are no key/value pairs in the dictionary
(i.e. the variables pointed to by ''keyPtrPtr'' and ''valuePtrPtr''
were not updated), but if the variable is set to 0, a key/value pair
was found and ''Tcl_DictObjNext()'' should be called to discover
whether that was the last value or if there are further ones in the
dictionary.  Note that if this function indicates that the search is
not done but the calling code wishes to extract no further values from
the dictionary, ''Tcl_DictObjDone()'' ''must'' be called to release
the internal locks on the representation of the value.

 > void '''Tcl_DictObjNext'''(Tcl_DictSearch *''searchPtr'',
	Tcl_Obj **''keyPtrPtr'', Tcl_Obj **''valuePtrPtr'', int *''donePtr'');

This function gets the next key/value pair from the search of a
dictionary, using the search referenced by ''searchPtr''.  The meaning
of the ''keyPtrPtr'', ''valuePtrPtr'' and ''donePtr'' variables is
much the same as in ''Tcl_DictObjFirst()'', along with the restriction
that if the function indicates that the search is not done but the
calling code wishes to extract no further values from the dictionary,
''Tcl_DictObjDone()'' ''must'' be called to release the internal locks
on the representation of the value.

 > void '''Tcl_DictObjDone'''(Tcl_DictSearch *''searchPtr'');

This function terminates a search of a dictionary before all the
values in the dictionary have been iterated over, releasing the
internal locks on the dictionary representation.

 > int '''Tcl_DictObjPutKeyList'''(Tcl_Interp *''interp'', Tcl_Obj *''dictPtr'',
	int ''keyc'', Tcl_Obj *CONST *''keyv'', Tcl_Obj *''valuePtr'');

This function is a variant on ''Tcl_DictObjPut()'' that takes a list
of keys so as to work with nested dictionaries.

 > int '''Tcl_DictObjRemoveKeyList'''(Tcl_Interp *''interp'', Tcl_Obj *''dictPtr'',
	int ''keyc'', Tcl_Obj *CONST *''keyv'');

This function is a variant on ''Tcl_DictObjRemove()'' that takes a
list of keys so as to work with nested dictionaries.

~ Examples

Counting the number of unique words in a file and the number of times
each word occurs:

|set f [open someFile.txt]
|set contents [read $f]
|close $f
|foreach word [regexp -all -inline {\w+} $contents] {
|   dict incr count $word
|}

|puts "There are [dict size $count] unique words."
|foreach word [lsort -dictionary [dict keys $count]] {
|   puts "${word}: [dict get $count $word] occurrences"
|}

A localisable '''string toupper''' implementation:


|set capital [dict create C [dict create]]
|foreach c {abcdefghijklmnopqrstuvwxyz} {
|   dict set capital C $c [string toupper $c]
|}

|dict set capital en [dict get $capital C]
|# ... and so on for other supported languages ...
|set upperCase [string map [dict get $capital $env(LANG)] $string

~ Copyright

This document has been placed in the public domain.

----

~ Appendices

''These appendices are not formally part of the proposal and exist
merely to help understanding.''

~ ~ Implementation Notes

Implement using hash tables (of course.)  Need efficient ways to
convert to/from lists, perhaps making lists know what's going on
underneath the covers?

~ ~ Future Directions

Alternate implementations of mappings, like trees or disk-backed
databases?

TIP:            112
Title:          Ensembles are Namespaces are Commands
Version:        $Revision: 2.28 $
Author:         Donal K. Fellows <[email protected]>
State:          Final
Type:           Project
Vote:           Done
Created:        10-Oct-2002
Post-History:   
Tcl-Version:    8.5


~ Abstract

This TIP proposes unifying the concept of ensembles (from [[Incr
Tcl]]) with namespaces and commands.  It also adds control of command
rewriting to allow for more efficient support for object systems like
Snit.

~ Rationale

Tcl's subcommand-style command collections (e.g. ''array'', ''info'',
''string'', ''interp'', etc.) are a very intuitive and popular way of
structuring collections of related commands.  However, it is quite
awkward to write Tcl code that behaves that way.  Users of [[Incr
Tcl]] have access to ensembles which provide that, but it would be a
very useful feature for many other uses too.

At the same time, it is becoming clear that many applications want to
commonly refer to commands inside other namespaces directly (instead
of through the [[namespace import]] mechanism) but the syntax for
doing this is verbose and not as elegant as it might be.

I believe that the same solution can address these two problems in one
go, and make the language stronger and more usable for it.

Furthermore, by giving the programmer control over the mapping from
the ensemble subcommands to their implementing commands, we can build
a simple class system on the cheap since we can, in effect, import
commands from elsewhere into the ensemble.  By extending the mapping
so that it allows the specification of not just the implementing
command, but also some leading arguments to that command (similar to
what you can do with the [[interp alias]] mechanism) this becomes a
very powerful mechanism indeed.

Finally, a sophisticated extra capability is the addition of an unknown
subcommand callback to allow the creator of the ensemble to specify a
customized strategy for handling subcommands that are not recognized by
the ensemble machinery.  This allows an ensemble to be dynamically
updated to include those subcommands that the user asks for, which is
a potential route for doing things like using an ensemble as a wrapper
round a Tk widget.

~ Proposed Change

I propose to add a new subcommand to the [[namespace]] command,
''ensemble'', that creates and manipulates the ensemble command for a
namespace.  Each namespace may have any number of ensemble commands
associated with it, with the default name of an ensemble command
being the fully-qualified name of the namespace itself, though it will
be legal to rename ensemble commands (anyone wanting to track such
events should use the [[trace]] command.)  Tcl will not create an
ensemble command for any namespace by default.

The [[namespace ensemble]] command will have the following subcommands:

 create: For creating a new ensemble for the ''current'' namespace. The
   command takes a list of option value pairs (as defined below) to
   set the ensemble up with.  As stated above, the default command name
   is exactly the name of the namespace, but any other name may be
   supplied (via the -command option); if it does not start with a
   namespace separator, the name will be taken as being relative to
   the current namespace.  If another command with the same name exists,
   it is deleted as part of the creation of the ensemble.

 configure: For reading and writing the configuration of a particular
   ensemble.  The command takes an argument specifying the name of the
   ensemble to configure, and then works with either no extra arguments
   (when it retrieves the entire ensemble configuration), one extra
   argument (the name of the option to retrieve), or a list of option
   value pairs to configure the ensemble with.  This command returns an error if applied to anything that is not an ensemble.

 exists: This subcommand (which takes a single argument) tests whether
   a command (with the given name) exists and is an ensemble.  This command only returns an error if the wrong number of arguments are supplied.

The options available for creation and configuring are:

 -subcommands: This option (if non-empty) specifies a list of ensemble
   subcommands that the ensemble supports.  It does not need to be
   sorted.  Each command is mapped according to the dictionary in the
   ''-map'' option (if a non-empty map is present and the command has
   a mapping) or to the correspondingly named command (which is not
   required to exist at the time this is specified) in the context
   namespace.

 -map: This option (if non-empty) specifies a dictionary that maps
   from ensemble subcommand names to lists of arguments to substitute
   into the ensemble invokation in place of the ensemble command name
   and the subcommand name.  See the ''-subcommands'' option for the meaning
   of this option when that option is also non-empty.  If this option is
   empty and the ''-subcommands'' option is empty too, the namespace will use
   the exported commands of the namespace as its command set, dynamically
   determining them (subject to cacheing) every time the ensemble is
   ''invoked''.  Note that the words in the dictionary values that give
   the commands to map to (as opposed to any arguments to them) are
   always resolved (if not absolute) relative to the namespace which is
   running [[namespace ensemble create]] or [[namespace ensemble
   configure]].

 -prefixes: This boolean option (which is on by default) controls whether
   unambiguous prefixes of ensemble commands are recognized as if they
   were the fully specified ensemble command names.

 -unknown: This provides (when non-empty) a partial command to handle
   the case where an ensemble subcommand is not recognized and would
   otherwise generate an error.  When empty (the default) an error (in
   the style of ''Tcl_GetIndexFromObj'') is generated whenever the
   ensemble is unable to determine how to implement a particular
   subcommand.

 > See ''Unknown Handler Behaviour'' below for details of how the
   ensemble interacts with its unknown handler.

Ensemble creation takes an extra option value pair:

 -command: This option allows you to override what the name of the
   ensemble to create is.

Ensemble configuring allows the querying of an extra read-only option:

 -namespace: This ''read-only'' option allows the retrieval of the name
   of the namespace that the ensemble was created in.

Given an ensemble command created by the above mechanism, calling the
command will first of all match the subcommand to its implementing
command (or command/argument list, as derived from the dictionary) in
a manner that will be recognizably similar to that enforced by
''Tcl_GetIndexFromObj()'' (unless the ''-unknown'' option override this
behaviour.)  For details of how the ensemble determines what its subcommands are, please see the ''-subcommands'' and ''-map'' options above.
Then the ensemble command will rewrite the command and arguments so
that the ensemble command and subcommand are replaced by the
implementing command and any specified arguments, with the resulting
word list being fed to ''Tcl_EvalObjv()'' for execution.  Note that
this does not increase the stack depth in terms of [[uplevel]], and
that the implementing command may itself be an ensemble command.

Note that if the namespace for an ensemble is deleted, the ensemble will
also be deleted though there is no lifetime constraint in the other
direction (i.e. using [[rename $theEnsemble {}]] will not cause the
namespace to vanish.)  Obviously, any ensemble which has the global
namespace backing it up will have the same natural lifespan as the
hosting Tcl interpreter.

~ Unknown Handler Behaviour

If an unknown handler is specified for an ensemble, that handler is called when the ensemble command would otherwise return an error due to it being unable to decide which subcommand to invoke.  The exact conditions under which that occurs are controlled by the ''-subcommands'', ''-map'' and ''-prefixes'' options as described above.

To execute the unknown handler, the ensemble mechanism takes the specified -unknown option and appends each argument of the attempted ensemble command invocation (including the ensemble command itself, expressed as a fully qualified name). It invokes the resulting command in the scope of the attempted call. If the execution of the unknown handler terminates normally, the ensemble engine reparses the subcommand (as described below) and tries to dispatch it again, which is ideal for when the ensemble's configuration has been updated by the unknown subcommand handler.  Any other kind of termination of the unknown handler is treated as an error.

The result of the unknown handler is expected to be a list (it is an error if it is not).  If the list is an empty list, the ensemble command attempts to look up the original subcommand again and, if it is not found this time, an error will be generated just as if the -unknown handler was not there (i.e. for any particular invokation of an ensemble, its unknown handler will be called at most once.)  This makes it easy for the unknown handler to update the ensemble or its backing namespace so as to provide an implementation of the desired subcommand and reparse.

When the result is a non-empty list, the words of that list are used to replace the ensemble command and subcommand, just as if they had been looked up in the -map.  It is up to the unknown handler to supply all namespace qualifiers if the implementing subcommand is not in the namespace of the caller of the ensemble command.  Also note that when ensemble commands are chained (e.g. if you make one of the commands that implement an ensemble subcommand into an ensemble, in a manner similar to the text widget's ''tag'' and ''mark'' subcommands) then the rewrite happens in the context of the caller of the outermost ensemble.  That is to say that ensembles do not in themselves place any namespace contexts on the Tcl call stack.

Where an empty -unknown handler is given (the default), the ensemble command will generate an error message based on the list of commands that the ensemble has defined (formatted similarly to the error message from ''Tcl_GetIndexFromObj()'').  This is the error that will be thrown when the subcommand is still not recognized during reparsing. It is also an error for an -unknown handler to delete its namespace.

~ Examples

|namespace eval carrot {
|   namespace export foo bar potato
|   namespace ensemble create     ;# Creates command ::carrot
|
|   proc foo {} {puts 1}          ;# Exported
|   proc bar {} {puts 2}          ;# Exported
|   proc boo {} {puts 3}          ;# Not exported
|
|   namespace eval turnip {       ;# Not exported
|      namespace export alpha
|      proc alpha {} {puts 4}     ;# Exported
|      proc beta {} {puts 5}      ;# Not exported
|      namespace ensemble create
|   }

|
|   namespace eval potato {       ;# Exported
|      proc north {x} {puts 6,$x} ;# Not exported
|      proc south {x} {puts 7,$x} ;# Not exported
|      # Notice we resolve names locally, and Tcl metachars are not special
|      namespace ensemble create -map {
|         north {north [$meta[$chars}
|      }
|   }
|}



|
|
|carrot foo                       ;# Prints 1
|carrot bar                       ;# Prints 2
|carrot b                         ;# Also prints 2 ("boo" not exported)
|carrot ?                         ;# ERROR: Alternatives "bar", "foo" and "potato"
|carrot potato                    ;# ERROR: Missing argument
|carrot potato ?                  ;# ERROR: Try "north" instead
|carrot potato north              ;# Prints 6,[$meta[$chars
|carrot turnip alpha              ;# ERROR: "turnip" not known
|carrot::turnip alpha             ;# Prints 4
|carrot::turnip::beta             ;# Prints 5
|
|rename ::carrot::potato ::spud
|spud north                       ;# Prints 6,[$meta[$chars
|spud south                       ;# ERROR: "south" not known
|carrot potato north              ;# ERROR: No ::carrot::potato command
|# Reconfigure spud; notice we get different name resolution now!
|namespace ensemble configure spud -map {
|   north {puts NORTH} south {puts SOUTH}
|}

|spud north                       ;# Prints NORTH
|spud south                       ;# Prints SOUTH
|namespace delete carrot
|spud north                       ;# ERROR: spud command already deleted
|
|
|namespace eval A {
|   proc a args {puts A::a=>$args}
|}

|namespace eval B {
|   proc b args {puts B::b=>$args}
|}

|# Create an ensemble in the global namespace
|namespace ensemble create -command C -map {
|   eg1 {::A::a foo bar}
|   eg2 {::B::b 1 2 3}
|   eg3 ::string
|}

|C eg1 spong                      ;# Prints A::a=>foo bar spong
|C eg2 evil code {[exit]}         ;# Prints B::b=>1 2 3 evil code [exit]
|C eg3 length qwertyuiop          ;# Returns 10
|
|
|# An example demonstrating the use of -unknown to do delegation
|# This uses an ensemble to add a subcommand to a frame
|package require Tk
|rename [frame .f] _impl.f
|namespace ensemble create -command .f -unknown {delegate _impl.f} -map {
|   flash {flashFrame _impl.f}
|}

|# General delegation framework handler
|proc delegate {target ensemble subcmd args} {
|   # Read the current map
|   set map [namespace ensemble configure $ensemble -map]
|   # Update it to include the new subcommand
|   dict set map $subcmd [list $target $subcmd]
|   # Install back into the ensemble
|   namespace ensemble configure $ensemble -map $map
|   # Result is empty string, so we reparse
|   return {}
|}

|# Our new subcommand implementation
|proc flashFrame w {
|   set bg [$w cget -background]
|   foreach colour {black white black white black white} {
|      $w configure -background $colour
|      update idletasks
|      after 150
|   }

|   $w configure -background $bg
|}


~ Consequences

Many commands in both Tcl and Tk would benefit from leveraging this,
and it would enable straight-forward implementations of things like
[65] in pure Tcl code.  It would also make doing things like partial
exposure of ensemble-like commands in safe interpreters much easier.

~ Sample Implementation

http://sf.net/tracker/?func=detail&aid=786509&group_id=10894&atid=310894

~ Copyright

This document has been placed in the public domain.

~ Acknowledgements

Thanks very much to Joe English, Don Porter and Kevin Kenny for their
suggestions in the development of this TIP.  Without them, it would have
been a far worse suggestion.  And thanks to Will Duquette for writing a
piece of software (Snit) that would benefit immensely from pushing the
ensemble stuff as hard as possible and then a bit.  :^)

TIP:            113
Title:          Multi-Line Searches in the Text Widget
Version:        $Revision: 1.13 $
Author:         Vince Darley <[email protected]>
State:          Final
Type:           Project
Vote:           Done
Created:        11-Oct-2002
Post-History:   
Tcl-Version:    8.5


~ Abstract

This TIP proposes enhancing the implementation of the ''$textwidget
search'' subcommand to allow matching of both exact strings and regexp
patterns which span multiple lines, and to allow reporting on all matches.

~ Proposal

If the string/pattern given to the ''search'' subcommand contains
sub-strings/patterns which match newlines, then it will be possible
for that command to return a match which spans multiple lines.  Where
a match could occur both within a single line and across multiple
lines, the first such match will be found, and the length of the match
will follow the usual regexp rules, as documented in the regexp man
page.  Since the text widget is inherently a line-based container of text, regexp searches will implicitly use the regexp ''-line'' functionality so that ''^'', ''$'' matching beginning and end of any line, and ''.'', ''[^'' will not match a newline (but see the ''-nolinestop'' flag to turn off the latter behaviour).

This can be implemented very efficiently, given
the ''TCL_REG_CANMATCH'' flag supported by the regexp library, with no
impact at all on the speed of matching single lines.

In addition, two new options to the ''search'' subcommand are available:

If the new ''-all'' option is given to the ''search'' subcommand, then all matches within the given range will be reported. This means the return result of the command will be a list of indices, and, if a ''-count var'' option was given, ''var'' will be set to a list of match-index-lengths.

If the new ''-nolinestop'' option is given then regexp searches will allow ''.'' and ''[^'' sequences to match newline characters (which is normally not the case).  This is equivalent to ''not'' providing the ''-linestop'' flag to Tcl's ''regexp'' command.

The text widget man page will be updated to reflect the new ''-all'' and ''-nolinestop'' options, and to remove the "single line" caveat.

~ Reference implementation

This is available from:

http://sourceforge.net/tracker/?func=detail&aid=621901&group_id=12997&atid=312997

The patch includes objectification of the entire Text widget, so the multi-line search changes are not obvious to isolate.  In fact the changes required are < 100 lines of code (given that the rest has been objectified, that is).  Of course one nice side-effect of objectification is that regexp objects used in searches are actually cached, which they previously couldn't be.

Note: this patch has to workaround a crashing bug in Tcl's unicode string manipulation.  It would be best if that bug was fixed before applying this patch.

~ Issues

On the implementation side, it might be interesting to abstract the search interface away from the text widget, so that it could in principle be applied to any line-based textual source.

As in the single-line matching implementation in Tcl 8.x, the lack of support for backwards matching in Tcl's regexp library means that backwards matching can only be implemented as repeated forward matches, with a commensurate performance penalty (the solution to which is outside the scope of this tip).

Tk has a curious misfeature that ''$text search -regexp "\n" $pos'' will always fail to match anything. This behaviour will change as a result of this TIP (it will match the first newline after ''$pos''), and any code which somehow depended on that peculiarity will therefore break.

~ Copyright

This document has been placed in the public domain.

TIP:            114
Title:          Eliminate Octal Parsing of Leading Zero Integer Strings
Author:         Don Porter <[email protected]>
Created:        16-Oct-2007
Type:           Project
State:          Draft
Vote:           Done
Version:        $Revision: 2.3 $
Tcl-Version:    9.0
Post-History:   
Keywords:       octal


~Abstract

This TIP proposes elimination of Tcl's practice of using octal
notation to interpret a string with a leading zero when an
integer value is expected.

~History and Rationale

There are several places in the syntax of several Tcl commands
where an integer value may be accepted.  Routines such as
'''Tcl_GetInt()''' perform the task of parsing an integer value
from the string value in these places.  Ultimately, these routines
have been built on C standard library functions such as
'''strtol()'''.  Due to this implementation choice, Tcl integer
parsing has inherited features from '''strtol()''' including
the feature that a leading zero in a string has been taken
as a signal that the string is an integer value in octal format.

Several programmers and programs have hit this feature by
surprise, resulting in nasty bugs such as:

| % proc m date {
|      lassign [split $date -] y m d
|      return [string index _JFMAMJJASOND $m]
|   }

| % m 2007-02-14
| F

| % m 2007-12-25
| D

| % m 2007-09-26
| bad index "09": must be integer?[+-]integer? or end?[+-]integer? (looks like invalid octal number)

There are very few places in Tcl scripts where this feature
is actually useful.  Octal format for integers simply isn't
encountered all that often in most programming tasks tackled by
Tcl scripts.  The main counterexample is the use of octal format
integers to describe filesystem permissions on unix systems.
The Tcl commands that operate on filesystem permission values
are '''open''' and '''file attributes''', and it is a simple matter
to directly code them to recognize octal format, rather that have
then rely on octal parsing as a general integer value recognition
feature.  On the HEAD, these commands have already been so revised.
With those few cases accounted for, it's been observed that
removing this feature of Tcl integer parsing "will likely fix
more scripts than it breaks."

The opportunity to make this change in Tcl 8.5 arises because
we've already replaced our old parsing routines based on
'''strtol()'''  with our own number parser [249].

~ Proposal

Revise all integer parsing in Tcl by making modifications to
the '''TclParseNumber()''' routine.  With reference to the
state machine graph in [249], we change the exit edges of
state ''integer[[1]]''.  Characters '''0''' - '''7''' and
characters '''8''' - '''9''' should now lead to state ''integer[[4]]'',
so that they continue decimal parsing, and not octal parsing.
The states ''integer[[2]]'' and ''error[[5]]'' will now be
accessible only if the character '''o''' or '''O''' is seen
while in state ''integer[[1]]'' and there will no longer be
any exit from those states when the characters '''.''' or '''e'''
or '''E''' are observed.

This change to '''TclParseNumber()''' is achieved with a
'''#define KILL_OCTAL''' in the file ''tclStrToD.c''.

~ Compatibility

This change is an incompatibility.  It's long
been believed that such a change should not happen until Tcl 9
because of this, but over time the consensus belief has developed
that far fewer programs and programmers will be harmed by the
incompatibility than will be helped by removing the misfeature.

That said, the incompatibility is serious.  The same string in
the same place in a script can now have a completely different
meaning.  Before the change:

| % lindex {a b c d e f g h i j k} 010
| i


After the change:

| % lindex {a b c d e f g h i j k} 010
| k


This is not the usual situation where new feature causes scripts
that were an error to become non-errors -- a compatible change.

This is also not a situation where a change causes legal scripts
to become errors.  Such a change would break scripts, but would
at least leave behind scripts that raise noisy errors alerting
about the breakage.

This is the most serious kind of incompatibility, where we replace
a working script with another working script that does something
completely different.  An illustration of the problem from Tcl's
own test suite highlights the danger.  Some of Tcl's tests in
'''io.test''' depend on the umask value, so that value is captured:

| set umaskValue [exec /bin/sh -c umask]

Note that the shell command '''umask''' returns a mask value as
an integer in octal format.  The test suite has relied on Tcl's
built-in ability to recognize this format, and the expected
result of test '''io-40.3''' has been computed:

| format %04o [expr {0666 & ~$umaskValue}]

After the proposed change, ''$umaskValue'' is treated as a decimal
number, and the wrong expected result is computed.  (This test
has already been updated on the HEAD to avoid such problems.)

It is not difficult to imagine more serious problems in scripts
that make use of the result returned by the shell command '''umask'''
where a file might be created or modified with completely unintended
permissions as a result of the proposed change.  Such scripts
might easily raise security concerns.

Even in the light of the judgment that such (hopefully rare) compatibility
issues are acceptable in exchange for the benefits of purging the
misfeature, we really ought to consider seriously how we can alert
those migrating to Tcl 8.5 to this possibility and to the need to
examine their scripts for this issue.

Besides the impact on Tcl commands, this change may also cause
incompatibilities in extensions, to the extent their commands
rely on Tcl's integer parsing to support octal notation.

~ Rejected Alternatives

Motivated largely by the serious incompatibilities lurking here,
a few people have suggested that some means be provided to toggle
Tcl's integer parsing behavior between two modes, one which
recognizes octal and one which does not.  This idea appears inspired
in part by the '''::tcl_precision''' variable, which has long
exercised control over Tcl's floating point number formatting.

While the motivation may be well-intended, this proposal is basically
unworkable, and can't really help anybody.  The point of the proposed
change is to make simple code work as simple coders expect it to.
Our original example proc can already be corrected like so:

| proc m date {
|    scan $date %d-%d-%d y m d
|    return [string index _JFMAMJJASOND $m]
| }


The point of this proposal is to make the original code just work.
It doesn't help to offer this complexity as a solution:

| % proc m date {
|      set mode [tcl::unsupported::octal]
|      tcl::unsupported::octal off
|      lassign [split $date -] y m d
|      set result [string index _JFMAMJJASOND $m]
|      tcl::unsupported::octal $mode
|      return $result
|   }


No one would choose that over just fixing the code
to use '''scan''', in which case this proposal won't be needed.
Also, this kind of management of a shared mode setting cannot
(easily and cheaply) be avoided, because at the point we need
to control the Tcl number parser, the most specific context
we have is the thread, so the mode has to be set thread-wide.

Likewise, the (hopefully rare) set of scripts that would actually
want to turn octal parsing back on are not going to announce themselves.
In order to know that 

| tcl::unsupported::octal on

needs to be added to a script to make it function correctly,
some kind of audit has to reach that conclusion, and once that
conclusion is reached and the issues are understood, it's just
as easy to insert '''scan %o''' in the proper places as it would
be to insert the '''tcl::unsupported::octal''' stopgap.

In short, any coder finding themselves in a position to consider
using a '''tcl::unsupported::octal''' tool, would quickly decide
not to use it in favor of just fixing their code.  Thus users
of this feature are mythical, and it will not be implemented.

~ Note

This TIP has been ''explicitly'' rejected as a feature for Tcl 8.5.
Consensus was that the type of breakage it inherently induces is not
acceptable in a minor version change.

~Copyright

This document is placed in the public domain.

TIP:            115
Title:          Making Tcl Truly 64-Bit Ready
Version:        $Revision: 1.3 $
Author:         Donal K. Fellows <[email protected]>
State:          Draft
Type:           Project
Vote:           Pending
Created:        23-Oct-2002
Post-History:   
Tcl-Version:    9.0


~ Abstract

This TIP proposes changes to Tcl to make it operate more effectively
on 64-bit systems.

~ Rationale

It is a fact of life that 64-bit platforms are becoming more common.
While once the assumption that virtually everything was a 32-bit
machine (where not smaller) was valid, this is no longer the case.
Particularly on modern supercomputers (though increasingly in
workstations and high-end desktop systems too), the amount of memory
that the machine contains is exceeding 2GB, and the need to address
very large amounts of memory is certainly there in scientific and
engineering applications.  And where they lead, consumer systems will
probably follow too.

At the moment, Tcl is ill-prepared for this.  In particular, the type
used for expressing sizes of entities in Tcl (whether strings, lists
or undifferentiated blocks of memory) is ''int'' (and cannot be made
into an ''unsigned int'' in most of those places where it is not
already an unsigned value) but on the majority of 64-bit platforms
this is still a 32-bit type, which is a major restriction.  However,
on the vast majority of those platforms ''long'' is a 64-bit type, and
so a suitable replacement.  (The exceptions to this are the Alpha - but
that is unusual in that both ''int'' and ''long'' are 64-bit types
there, meaning that the platform will be unaffected by such an
alteration - and Win64, which has a 32-bit ''long'' but 64-bit
pointers.)

Luckily, standards like POSIX have already been dealing with this
problem before us, and the types ''size_t'' (which is unsigned) and
''ssize_t'' (which is signed) exist for the sorts of uses we're
interested in (i.e. they are both the same size as each other, and
''size_t'' is large enough to describe the size of any allocatable
memory chunk.)

~ Details of Changes

The key changes will be to change the lengths of the following types
from ''int'' to ''ssize_t'' in all appropriate places, and ''unsigned
int'' to ''size_t'' likewise (mainly in memory allocation routines.)

 * ''Tcl_Obj'' - the ''length'' member.  (Potentially the ''refCount''
   member needs updating as well, but that's less critical.)

 * ''Tcl_SavedResult'' - the ''appendAvl'' and ''appendUsed'' members.

 * ''Tcl_DString'' - the ''length'' and ''spaceAvl'' members.

 * ''Tcl_Token'' - the ''size'' and ''numComponents'' members.

 * ''Tcl_Parse'' - the ''commentSize'', ''commandSize'', numWords'',
   ''numTokens'' and ''tokensAvailable'' members.

 * ''CompiledLocal'' - the ''nameLength'' member.

 * ''Interp'' - the ''appendAvl'', ''appendUsed'' and ''termOffset''
   members.

 * ''List'' - the ''maxElemCount'' and ''elemCount'' members.

 * ''ByteArray'' - the ''used'' and ''allocated'' members.

 * ''SortElement'' - the ''count'' member.

 * ''SortInfo'' - the ''index'' member.

 * ''CopyState'' - the ''toRead'' and ''total'' members.

 * ''GetsState'' - the ''rawRead'', ''bytesWrote'', ''charsWrote'' and
   ''totalChars'' members.

 * ''ParseInfo'' - the ''size'' member.

 * ''String'' - the ''numChars'' member (see also the ''TestString''
   structure.)

Changes to the bytecode-related structures might be worthwhile doing
too, though there are more backward-compatibility issues there.

These changes will force many of the types used in the public API to
change as well.  Notable highlights:

 * ''Tcl_Alloc'' will now take an ''size_t''.

 * ''Tcl_GetByteArrayFromObj'' will now take a pointer to a ''ssize_t''.

 * ''Tcl_GetStringFromObj'' will now take a pointer to a ''ssize_t''.

 * ''Tcl_ListObjLength'' will now take a pointer to a ''ssize_t''.

 * ''Tcl_GetUnicodeFromObj'' will now take a pointer to a ''ssize_t''.

In the internal API, the following notable change will happen:

 * ''TclGetIntForIndex'' will now take a pointer to a ''ssize_t''.

There are probably other similar API changes required.

~ What This TIP Does Not Do

This TIP does not rearrange structure orderings.  Although this would
be very useful for some common structures (notably ''Tcl_Obj'') if the
common arithmetic types were smaller than the word size, it turns out
that the changes in types required to deal with larger entities will
make these rearrangements largely unnecessary and/or pointless.
(Inefficiency in statically-allocated structures won't matter as the
number of instances will remain comparatively small, even in very
large programs.)  Once the changes are applied, there is typically at
most a single ''int'' field per structure, usually holding either a
reference count, a set of flags, or a Tcl result code.

It should also be noted that all structures are always going to be
correctly aligned internally as we never use C's bitfield support, so
structure alignment is purely an issue of efficiency, and not of correct
access to the fields.

~ Copyright

This document has been placed in the public domain.

TIP:            116
Title:          More Safety for Large Images
Version:        $Revision: 1.6 $
Author:         Donal K. Fellows <[email protected]>
State:          Final
Type:           Project
Vote:           Done
Created:        28-Oct-2002
Post-History:   
Tcl-Version:    8.5


~ Abstract

This TIP alters the C API for Tk's images so that failures to allocate
sufficient memory for a large image can be handled more gracefully
than a straight ''panic()''.

~ Rationale

Tk's image mechanism is nice and flexible, but it can run into
problems with a large image.  If we consider a square image that is
2000 pixels on each side (such sizes of images are becoming more
common with the increasing popularity and sophistication of digital
photography), we find it requires about 16MB of memory to load (4
million pixels, four bytes per pixel) but obviously just because an
application fails to load that image (or something even larger), it
doesn't mean that the best course of action is a ''panic()''-induced
crash.  Instead, a graceful failure back to the Tcl interpreter would
allow for scripts to find a way to report this low-memory situation in
a way that users can understand.

The problem with this is that for many of the routes through the Tk
image API, there is no way to report a memory allocation failure;
currently, the only failure mode is total.  This TIP changes this.

~ Proposed Changes

I propose making the following functions that currently return
''void'' return ''int'' instead and that they should additionally take
a standard ''Tcl_Interp *interp'' argument to allow an error message
describing the failure (currently only due to insufficient memory) to
be added to the current interpreter.  It will not be an error for the
''interp'' argument to be ''NULL'', though it will be up to the caller
to guess why the failure happened.

 * ''Tk_PhotoExpand'' will become:

|int
|Tk_PhotoExpand(Tcl_Interp *interp, Tk_PhotoHandle handle,
|               int width, int height)

 * ''Tk_PhotoPutBlock'' will become:

|int
|Tk_PhotoPutBlock(Tcl_Interp *interp, Tk_PhotoHandle handle,
|                 Tk_PhotoImageBlock *blockPtr,
|                 int x, int y, int width, int height, int compRule)

 * ''Tk_PhotoPutZoomedBlock'' will become:

|int
|Tk_PhotoPutZoomedBlock(Tcl_Interp *interp, Tk_PhotoHandle handle,
|                       Tk_PhotoImageBlock *blockPtr,
|                       int x, int y, int width, int height,
|                       int zoomX, int zoomY, int subsampleX, int subsampleY,
|                       int compRule)

 * ''Tk_PhotoSetSize'' will become:

|int
|Tk_PhotoSetSize(Tcl_Interp *interp, Tk_PhotoHandle handle,
|                int width, int height)

Also note that as a consequence of this, some image-related Tk
commands will also gain additional error return situations.  Since
these all trigger abnormal process termination (and potentially a
core-dump too) at the moment, this change in behaviour is believed to
be wholly beneficial.

~ Backward Compatibility

This TIP also proposes a backward compatibility interface, so that
extensions need not be heavily modified to work with new versions of
Tk.  This is done by leaving backwardly-compatible functions in the
old locations in Tk's stub table and adding a ''#define'' to allow
selection of the old API with the standard names.  I propose doing this
when the symbol ''USE_PANIC_ON_PHOTO_ALLOC_FAILURE'' is defined; like
this, extension authors can switch from compiling with Tk 8.4 to using
later versions by adding just one flag to their makefile (or other build
script/environment.)

Note that this interacts with the backward-compatability interface
defined in [98]; if that is enabled, the back-compat interface defined
here is enabled as well.

~ Sample Implementation

http://sf.net/tracker/?func=detail&aid=646382&group_id=12997&atid=312997

~ Copyright

This document has been placed in the public domain.

TIP:            117
Title:          Object Type Introspection
Version:        $Revision: 1.9 $
Author:         Peter Spjuth <[email protected]>
State:          Withdrawn
Type:           Project
Vote:           Pending
Created:        01-Nov-2002
Post-History:   
Tcl-Version:    8.5
Obsoleted-By:	214


~ Abstract

This TIP proposes to add a command to give information
on the current internal representation of an object.

~ Rationale

When trying to understand the internals of objects and when trying to
track down shimmering problems it is very helpful to be able to see
what type an object currently has.

~ Specification

A new package called "Debug" is created as part of the core.

A new command, ''objtype'', is added to the Debug package.
The command has the syntax:

 > '''Debug::objtype''' ?''value''?

If no value is submitted, a list of all currently registered object
types are returned.

If a value is submitted, it returns a list where the first element
is the name of value's object type, or "none" if there is no
internal representation.  The second element is a boolean stating
if the object has a valid string representation.

Examples:

|% Debug::objtype
|boolean index double end-offset wideInt list cmdName bytecode
|nsName procbody bytearray int {array search} string
|% set apa hejsan
|hejsan
|% Debug::objtype $apa
|none 1
|% string length $apa
|6

|% Debug::objtype $apa
|string 1
|% regexp $apa hoppsan
|0

|% Debug::objtype $apa
|regexp 1
|% Debug::objtype [expr 1+1]
|int 0

~ Discussion

The first proposal was to add this as a subcommand to info.  This
was considered bad since this should be a pure debug command
and should never be used in live code.  Thus the proposition
came up to create a Debug package and put it in there to emphasise
its debug nature.  The Debug package can also be extended in the
future with other utilities.

~ Reference Implementation

|static int
|ObjTypeCmd(dummy, interp, objc, objv)
|    ClientData dummy;		/* Not used. */
|    Tcl_Interp *interp;	/* Current interpreter. */
|    int objc;			/* Number of arguments. */
|    Tcl_Obj *CONST objv[];	/* Argument objects. */
|{

|    Tcl_Obj *listPtr;
|
|    if ((objc != 2) && (objc != 3)) {
|        Tcl_WrongNumArgs(interp, 2, objv, "?value?");
|        return TCL_ERROR;
|    }

|
|    listPtr = Tcl_NewListObj(0, (Tcl_Obj **) NULL);
|
|    if (objc == 2) {
|        Tcl_AppendAllObjTypes(interp, listPtr);
|    } else {
|        Tcl_ListObjAppendElement(interp, listPtr, Tcl_NewStringObj(
|                (objv[2]->typePtr != NULL) ? objv[2]->typePtr->name : "none", -1));
|        Tcl_ListObjAppendElement(interp, listPtr,
|                Tcl_NewBooleanObj(objv[2]->bytes != NULL));
|    }

|    Tcl_SetObjResult(interp, listPtr);
|    return TCL_OK;
|}


~ Future possibilities

For further object introspection the command can easily be 
extended by options.

~ Copyright

This document has been placed in the public domain.