TIP 424: Improving [exec]

Bounty program for improvements to Tcl and certain Tcl packages.
Tcl 2017 Conference, Houston/TX, US, Oct 16-20
Send your abstracts to tclconference@googlegroups.com
by Aug 21.
Author:         Alexandre Ferrieux <alexandre.ferrieux@gmail.com>
State:          Draft
Type:           Project
Vote:           Pending
Created:        07-Jul-2013
Keywords:       Tcl,subprocess,execution
Tcl-Version:    8.7


This extension overcomes day-1 limitations of [exec]'s syntax, allowing for unconstrained arguments to commands, and opening the path to more exotic redirections.

Summary Change


   exec foo bar baz > file


   exec | {foo bar baz} > file


For decades people have rightfully complained about the stubborn limitation of exec that prevents it from using commands or args resembling a redirection. It's not just Quoting Hell; it is simply impossible to spawn the equivalent of Bourne Shell's "echo \>" from pure Tcl (i.e., without resorting to another shell).

The reason (excuse?) for this is an unfortunate design choice: stick as closely as possible to the Bourne Shell's syntax, which indeed seamlessly intertwines commands, arguments, and redirects. This is unfortunate, because it overlooks a key difference between the two shells:

In this situation, it would have been possible to add an exec-specific layer of quoting, just for these characters. But as usual, the quoting char itself (typically "\**") would have itself needed quoting ("\\**"), which would have overburdened the backslash density of all but the simplest pipelines...

More importantly, the realization that this was Really Wrong came fairly late in Tcl's life; or at least late enough to consider any incompatible fix out of the question.

So exec can be extended, not fixed.

A few such extensions have been suggested over the years, but none reached critical mass. A possible interpretation of this is that they were considered too "disruptive" - while necessary only for a corner case.

The current proposal addresses all the above concerns. Here are its design goals by decreasing importance:

  1. Current exec's unescapable warts should disappear

    (Yeah, take care of that corner case.)

  2. Current exec's mapping to open | should be carried over

    (This part of exec's design was Good)

  3. Simple pipelines should give easy-to-read lines (like current exec)

    (No disruption, Ma'am)

  4. Shell-ish advanced redirections like "3>&5" should be supported

    (Not just the corner case: you get a free lunch too)



      exec | {echo >} ;# this returns ">"
      exec | {cmd "<funny>xml</funny>"} 2>@ $ch < /dev/null | {cmd2 arg} >&2

Goals reached:

  1. Unescapable warts are gone because the $cmd vs $redir status is positional, not content-based: each command-and args is a separate sublist, with no in-band encoding of redirections.

  2. The above mapping is consistent with the existing open |[list foo bar] logic. It respects the invariant saying, for open |, that [string range $openarg 1 end] is always the list that would be passed, expanded, to exec. And it is handy to type open "|| {foo >} > file"

  3. Simple pipelines are simple.

    exec | $cmd1 | $cmd2 | $cmd3 > file

  4. Advanced redirections are imaginable since the redirection subsyntax now lives on its own. For example, with a putative "NUMBER>@" family of operators, one could define a nonlinear pipe graph:

          lassign [chan pipe] pr pw
          exec | {demuxer ...} 3>@ $pw | {filter ...} | {muxer ...} 3<@ $pr

    The definition of these advanced operators will be hosted by another TIP.


This very conservative syntax, in addition to preserving the overall style and density of current exec, overcomes all the limitations and reaches Bourne Shell power.

Moreover, it leverages the existing internals, so a nearly free side-effect is that it works with pid and close just like current exec does.

Rejected Alternatives

Reference Implementation

Branch "tip-improve-exec" on core.tcl.tk holds the implementation.


This document has been placed in the public domain.