TIP 10: Tcl I/O Enhancement: Thread-Aware Channels

Bounty program for improvements to Tcl and certain Tcl packages.
Tcl 2017 Conference, Houston/TX, US, Oct 16-20
Send your abstracts to tclconference@googlegroups.com
by Aug 21.
Author:         Andreas Kupries <a.kupries@westend.com>
State:          Final
Type:           Project
Vote:           Done
Created:        08-Nov-2000
Tcl-Version:    8.4


This TIP describes how to change the generic I/O layer in the Tcl core to make channels aware of the thread they are managed by.


To explain the motives behind this TIP first a short look at the history of channels and threading.

In ancient times the Tcl core was not thread safe and did not employ threads. All channels belonged to a single interpreter. Later on interpreter hierarchies were introduced and the ability to move or share a channel between the interpreters in a hierarchy. When the Tcl core was made thread safe a short time after the ability to move channels between threads was added (Helper APIs in the core, main functionality in the Thread extension). The goal behind these modifications was to enable the creation of stream-like communication paths between threads to complement the message based facilities (thread send). The modifications were only a partial success because an in-depth analysis of the relevant data structures showed that the sharing of a channel between threads is not possible with the current design, only moving. This was implemented to allow at least the dispatcher- / worker-thread pattern for structuring a threaded application.

In further pursuit of the original goal the currently chosen approach is to define a channeltype where two channels are connected internally through in-memory fifo buffers where access to these shared structures is protected by mutexes.

During the implementation of fileevents for this channeltype it was discovered that an efficient implementation of this part is not possible because of the inability to post file events to the eventqueue of the thread the other channel of the pair resides in. An API to post such events is available (Tcl_ThreadQueueEvent), but not the information which thread actually manages the other channel. Because of this the current implementation of the channeltype uses polling based upon timer events posted by each side/thread to itself to manage file events in a rather inefficient way.

Reference implementation

This TIP now proposes to change the internals of the generic I/O layers in the core so that

  1. Channels know the thread they are managed by, and

  2. are able to deliver this information to an extension querying the core.

This then allows the two sides of the channeltype mentioned above to post events to each other, facilitating an efficient implementation of fileevents.

The changes necessary to accomplish this are:

  1. Extend the structure ChannelState in tclIO.h with a new field of type Tcl_ThreadId to hold the id of the thread currently managing all channels with this state. Note: This structure is shared by all channels in a stack of transformations.

  2. Modify the procedure Tcl_CreateChannel to store the Id of the current thread in the ChannelState of the new channel. This information can be obtained with Tcl_GetCurrentThread. It is not necessary to modify Tcl_StackChannel as the thread information is already part of the state when it is called, and won't be changed by the call.

  3. If some sort of NIL/NULL value meaning "No thread" is available for Tcl_ThreadId, then we should modify Tcl_CutChannel to insert this value into the state of the channel it is called with, as this channel will not be managed by any thread afterward (the procedure removes the channel from the list of all channels managed by the current thread).

  4. Modify Tcl_SpliceChannel in the same manner as Tcl_CreateChannel as the channel will be managed by the current thread afterward (The procedure adds the channel to the list of all channels managed by the current thread).

  5. Declare a new API function to retrieve the Id of the managing thread from a channel. Add this declaration to generic/tcl.decls and implement the function in the file generic/tclIO.c. I propose Tcl_GetChannelThread as the name of this new API function.

A patch implementing all of the changes described above and additionally extending the documentation and the test-suite is available here: http://www.cs.man.ac.uk/fellowsd-bin/TIP/10.patch


This document has been placed in the public domain.