TIP 495: Tcl Based Build System for TEA projects

Login
Home Timeline Branches Tags Help 
Author:         Sean Woods <[email protected]>
State:          Draft
Type:           Project
Vote:           Pending
Created:        14-Jan-2018
Post-History:
Keywords:       Tcl,build system,extension building
Tcl-Version:    9.1

Abstract

This tip proposes that we switch extension development away from implementing automation in autoconf and nmake and towards using Tcl.

Rationale

A present day application developer has to use several more languages than simply Tcl and C to develop extensions for the language. Binary packages require a developer to master autoconf, Makefile and nmake. The reference Sample extension is really only useful for very, very trivial extensions. And even there requires some knowledge of the process to do the appropriate cutting and pasting. Most TEA packages have no concept of how they can populate a VFS for a kit, or bundle themselves as a zip file for proper packaging in a TEAPOT.

As one of the developers in charge of fixing day to day problems in TEA, I find myself writing a lot more sh than Tcl. And most of that writing is working around limitations in the sh language itself.

In this project SH and autoconf are not completely going away. We are just changing their role from "building our make system" to "finding all the platform specific quirks so that Tcl can be our make system". Though for simple projects we do open the door to restricting their role to building the Tcl core, and then allow Tcl scripts to leverage what autoconf discovered.

The reforms to TEA are several fold, but designed in such a way that present packages can continue to use their current build systems, unmodified. New projects (or old projects that upgrade to the TEA 4 standard) will be able to utilize the new features of TEA.

The principle change is that the process of building binaries is being moved out of the Makefile and Nmake, and into a Tcl script. Vestigal batch files for each can be left which call the new mechanisms in the old way.

Instead of the prime mover for packages being:

./configure
make all

New TEA packages will use the convention of:

tclsh make.tcl all

The make.tcl file does not have to implement bread and butter compiler, integration and installation functions. They will be provided by a new package "practcl" which will be distributed as a single file in the /tclconfig directory that TEA packages include in source distributions. A version will also be provided as a tcllib module. The practcl module is a self contained library with no external dependencies, and runs in Tcl 8.6, or 8.5 with the TclOO extension.

Specification

make.tcl

All Tcl extensions will have a make.tcl file. That file will contain all of the instructions needed to build, install, package, and integrate that extension.

make.tcl will accept the following commands:

  1. all - Replicate the behavior of the previous TEA system, generate a dynamic library
  2. info - Return a dict containing the following information: name - The name of the project version - The version of the project teapot_file - The name of the file generated by teapot library_file - The name of the dynamic library file static_file - The name of the static library file stubs_file - The name of the stubs library file
  3. install ?destination? - Install the package to local environment or to the optional directory
  4. packages - Returns a list of packages and versions of those packages this project generates
  5. dynamic - Generate a dynamic library
  6. static - Generate a static library
  7. stubs - Generate a stubs library
  8. shell - Optional - Generate an executable with the package statically linked
  9. teapot - Generate a .zip or .tm file containing an installation image of the package
  10. test - Run the test suite for the package

tcllib (and other large libraries)

Projects such as tcllib which contain many modules, and are unlikely to be installed whole inside of a tclkit, will be given additional make directives:

  1. module-list - Return a list of modules
  2. module-install destination ?all? ?module? ?module?...- Install named modules to the destination. Unlike the standard installation, this method does not place the module inside of a versioned package/module file structure. The module is simply dropped as either a .tm file for directory with the module's name and a pkgIndex.tcl file, within the directory specified.

config.tcl

Instead of having autoconf write all over Makefiles and pkgIndex, the new TEA performs all of its substitutions in one file: config.tcl. That file contains a key/value list readable by a Tcl script. nmake environments will perform identical subsitutions to the same file.

practcl

A standard library of tools (provisinally named "practcl") will be provided in tcllib and as a single file distribution in the same repository as the TEA reference files. (https://core.tcl-lang.org/tclconfig)

This library includes implementations for all build, installation, and integration tasks. The file can be directly sourced, or invoked with package require practcl.

    ## Example make.tcl file

    set CWD [pwd]
    set ::SRCDIR   [file dirname [file normalize [info script]]]
    set ::SANDBOX  [file dirname $::SRCDIR]
    package require practcl

    # Build and configure and object named "LIBRARY"
    array set ::project [::practcl::config.tcl $CWD]
    ::practcl::library create LIBRARY [array get ::project]
    LIBRARY define set builddir $CWD
    LIBRARY define set srcdir $SRCDIR
    LIBRARY meta set author {{Tcl Core}}
    LIBRARY meta set license BSD
    LIBRARY add [file join $::SRCDIR generic sample.c]
    LIBRARY add [file join $::SRCDIR generic sample.tcl]
    LIBRARY define add public-verbatim [file join $::SRCDIR generic sample.h]

    # Run ./configure or nmakehlp to generate the config.tcl file
    if {![LIBRARY define exists TCL_SRC_DIR]} {
      # Common interactions will have pre-canned implementations
      LIBRARY make detect
    }

    # Define a target which generates the dynamic C code
    LIBRARY make add implement {
      filename sample.c
    } {
      # This script evaluates with in the LIBRARY
      # object's namespace
      my go
      my implement $::project(builddir)
      set fout [open pkgIndex.tcl w]
      puts $fout [LIBRARY package-ifneeded]
      close $fout
    }

    # Define the "all" target, and create aliases for "libraries"
    # and "binaries"
    LIBRARY make add all {
      aliases {binaries libraries}
      depends library
    }

    # Define the "library" target
    LIBRARY make add library {
      triggers implement
      filename [LIBRARY define get libfile]
    } {
      puts "BUILDING [my define get libfile]"
      my build-library [my define get libfile] [self]
    }

    # Process command line
    switch [lindex $argv 0] {
      install {
        # Ensure the library is compiled
        LIBRARY make depends library doc
        # Make do acts on all of the steps computed by
        # make depends
        LIBRARY make do
        # Project specific installation procedure
      }
      info {
        # Output information that integrators need
        set dat [LIBRARY target pkginfo]
        foreach {field value} $dat {
          puts [list $field: $value]
        }
        exit 0
      }
      teapot {
        # Ensure the library is compiled
        LIBRARY make depends library
        LIBRARY make do
        # Code to generate teapot distribution
      }
      default {
        # Work like a makefile and trigger build targets
        # Unlike a standard makefile "trigger" will re-kick
        # off any indicated steps
        LIBRARY make trigger {*}$argv
        LIBRARY make do
      }
    }

More Info

The Practcl module is now in Tcllib Practcl project.

A version of the Tcl Sample Extension has been re-worked to utilize practcl. Sample Extension

A version of the Tcl Sample Extension has been re-worked to utilize practcl and autosetup. Sample Extension