The Simple Development Library white paper 

Table of contents

1. The need
2. The answer
3. SimpleDevLib in a nutshell
   3.1 SimpleDevLib programs
   3.2 SimpleDevLib packages
   3.3 SimpleDevLib classes
   3.4 SimpleDevLib procedures and methods
   3.5 SimpleDevLib errors
   3.6 SimpleDevLib types
   3.7 SimpleDevLib variables and attributes
4. SimpleDevLib packages and utilities
   4.1 SimpleDevLib packages
   4.2 SimpleDevLib utilities
5. Copyright

1. The need

One of the problems I face as a senior architect for complex systems is that my customers perceive Tcl software as clumsy and expensive to maintain. And sometimes not without reason. We have evidence that, in some cases, classical Tcl code tends to either get rusty or require a significant maintenance to be kept up to date when compared to that written in higher level languages.

Why is this?

One of the reasons might be that it is so easy to put together a small application doing something useful that Tcl is very often used for quick and dirty workarounds. But the truth is that working software persists in complex systems and acquires the status of the solution for a particular task.

Sometimes a prototype is written in Tcl. But the prototype works and features much more functionality than expected from the investment! So that the prototype ends up being operational.

As for larger components, Tcl is so expressive, dynamic and flexible that, in my experience, maintainability tends to be neglected. Tcl code has a tendency to be worse documented that equivalent code written in higher level languages, simply because the code can be understood. But complex corrective maintenance and refactoring is more difficult. The lack of both the write-compile-execute cycle (one can patch Tcl code within the operational environment!), static typing and object orientation does not promote maintainability either. One can also argue that Tcl software works with less testing, so that the testing plan is less comprehensive, so that less defects are caught when the testing plan is reexecuted after a modification.

But we want to develop in Tcl! Tcl is undoubtedly more productive and better suited for adapting to changing requirements than other languages, not to mention its portability.

Those are the motivations which prompted me to write The Simple Development Library.

2. The answer

The Simple Development Library (SimpleDevLib) is a collection of packages and utilities aimed towards assisting software development in Tcl, the great language designed by John Ousterhout, and [incr Tcl], the popular object oriented extension to Tcl.

SimpleDevLib provides essential elements for good software engineering and/or medium to big projects development such as:

Improved package handling

Homogeneous, easily parseable, fully introspectable code organization

A type system

A framework for regression testing

Object orientation

SimpleDevLib is written in pure Tcl itself, so that no compilation is needed to run it on Unix, Windows or Mac environments. Just unpack into one of the directories that Tcl searches for packages and you are done; Tcl version 8.1 or later is required (this release has been tested using Tcl versions 8.1.1, 8.2.3, 8.3.3 and 8.4.9, both in Unix and Windows; contact the author should you want to use SimpleDevLib with Tcl 8.0). The SimpleClass package requires [incr Tcl], so that a version of this extension compatible with the Tcl one being used is needed.

SimpleDevLib is distributed under the BSD license. In essence, this license allows you to do whatever you want with the software provided the copyright notices are left untouched.

3. SimpleDevLib in a nutshell

SimpleDevLib allows to declare code items (such as packages and procedures). Items contain components. Components can be either functional (required for the item to work, such as a procedure argument default value and body) or informational (metadata, such as the procedure argument descriptions).

All items and components can be introspected using the many information commands available. For example, ::Simple::Package::information classes returns a package list of classes, while ::Simple::Variable::information type returns a variable type. Metadata is usually not stored (so that it is not available for introspection) unless the -storemetadata SimpleDevLib option is set (see the SimpleBase package).

The following is an example of a procedure declaration:

Some items (programs, packages and classes) can embed other items. The following table lists all SimpleDevLib items. The table specifies for each item whether it can be declared standalone and whether as part of a program, package or class declaration.

Standalone Program Package Class
Program X
Package X
Classs X X
Type X X X
Procedure X X X X
Error X X X X
Variable X X X X
Program option X
Package option X
Method X
Attribute X
[incr Tk] option X
[incr Tk] component X

Table 1: Standalone and embedded item declarations

For example, a class can either be declared standalone (using the declare-class command) or as part of a package declaration (using the -class flag of the declare-package command).

In the following sections, the declaration of SimpleDevLib programs, packages, classes, procedures, methods, errors, types, variables and attributes are covered at high level. Refer to the various packages for a more in-depth explanation of each item; The Simple Development Library item components offers a full listing of SimpleDevLiv items and components.

3.1 SimpleDevLib programs

Implemented by

Declaration

Can embed

A SimpleDevLib program is Tcl code intended to be invoked from the operating system.

Program declarations specify the program required packages (-requisites) as well as their errors (-error), types (-type), variable (-variable), commands (-command) and program options (-option and -optionusage).

The most useful feature added by SimpleDevLib for program handling is the command line options parser. From the program body (-body) ::Simple::Program::parse-command-line may be invoked to parse the program command line. This procedure ensures the arguments given to the program are compatible with those specified in its declaration. Otherwise, a message informing about the error is sent to the standard error channel and the program exits unsuccessfully. See the SimpleOption package for a full explanation of the command line options parser

See the SimpleProgram and SimpleOption packages for a full explanation of SimpleDevLib programs.

The following is a simple program declaration example (taken from the simpleprogramexample program distributed with SimpleDevLib); the example does not include program errors, types nor variable.

Lines 1 to 3 are the usual bootstraping code. At line 4 the SimpleDevLibProgram package is requested. This package is actually a bundle that loads the SimpleProgram package and those required by it, so that declare-program, the command needed for program declaration, becomes available. In fact declare-program is used in line 5 and extends up to the end of the example.

Up to line 12, the program is described; the program name and -synopsis are mandatory, but other metadata flags such as -version and -overview are not.

The -option flag at line 13 declares a command line option for the program. If the program accepts multiple command line options, several -option flags would appear in its declaration. This option can be given at the command line as either -n or --number and takes a single argument of type float. Its presence in the command line is mandatory as specified by the -optionsusage flag at lines 15 and 16; see the ::Simple::Option::declare and ::Simple::Option::declare-usage procedures from the SimpleOption package for a full description of the features provided for command line options.

Lines 17 to 19 provide some examples on how to use the program; these examples are displayed to the user when the program is invoked with the --examples command line option.

The -requisites flag at line 20 specifies the packages required by the program; those are automatically loaded by SimpleDevLib.

A program command is declared from lines 22 to 33; see the section on commands for a description of their declaration.

Finally, lines 34 and following are the actual program code. The first thing to do is to parse the command line with the code at line 37. After that, the command line options can be accessed with the various ::Simple::Option::information subcommands provided by the SimpleOption package.

Assuming the program above is placed within a file named simpleprogramexample and the usual conditions to allow for program execution are met (the file has executable permissions in Unix or an .exe extension in Windows ...) the following depicts its use:

At line 1 the program is invoked with no arguments, but as the -n (or --number) flag is mandatory, the program informs of this fact and exits unsuccessfully.

The execution at line 5 uses the --help argument, so that the help notice informing on the various options is displayed to the user and the program exits successfully.

At line 18 the --examples flag is used; the program displays the examples provided in the -examples flag to declare-program and exits successfully.

At line 24 the user mistakenly types -v instead of -n: the program chokes that there is no such option and exits unsuccessfully. After that, the user corrects the flag and the program works as expected at line 28.

Finally, the user provides a value of zero for the option at line 31 that throws a division by zero error; SimpleDevLib catches the error, displays the error at line 32 and exits unsuccessfully.

3.2 SimpleDevLib packages

Implemented by

Declaration

Can embed

A SimpleDevLib package is a Tcl package intended to be required from SimpleDevLib programs and other packages.

Package declarations specify the package required packages (-requisites), as well as their classes (-class), errors (-error), types (-type), variable (-variable), commands (-command) and package options (-option). Other prominent functional components are the namespaces used by the package (-namespaces) and its exported commands (-exportedcommands) and aliases (-aliases).

While regular Tcl packages can just be either installed (loaded) or not, SimpleDevLib defines an intermediate state for packages named declared. The key point is that declaring a package does not modify the status of the Tcl interpreter except within the package namespaces, but declared packages are fully available for comprehensive introspection. For example, the ::Simple::Package::information collisions command returns the list of collisions which would occur should the package be actually installed (collisions may occur as SimpleDevLib packages may define interpreter-wide command aliases as well as namespace-wide exported commands).

SimpleDevLib packages may feature package options. Declared via the -option flag, package options ar similar to variable but are intended for configuring the package. Package options should be configured using the configure and cget commands in the package main namespace, in a similar way to Tk widgets. Nevertheless this is currently just a convention as the user must create these two commands if desired.

SimpleDevLib also introduces the concept of extra packages. An extra package works as a companion to a regular package. It is intended to contain the package functionality less frequently used. That way, when an application requires a package, only the regular package is actually loaded, leaving the functionality in the extra package to be autoloaded in case it becomes necessary.

Finally, SimpleDevLib packages also introduce the concept of access mode through the use of private namespaces. Namespaces whose last element matches ::Priv* are regarded as private namespaces. Items placed in private namespaces are considered private, and are not reported by the various ::Simple::Package::information procedures (except for ::Simple::Package::information privatecommands). Notice that most items declared by SimpleDevLib are ultimately Tcl items (all but [incr Tcl] classes), and Tcl itself lacks the concept of access mode, so that private namespaces are private only by convention.

See the SimpleDeclare (declare-package command) and SimplePackage packages for a full explanation of SimpleDevLib packages.

The following is a simple package declaration example (taken from the simplepackageexample package distributed with SimpleDevLib) that implements a simple stack. The example does not include package classes, types, package options, command aliases nor exported commands.

Line 1 requests the SimpleDevLibPackage package. This package is actually a bundle that loads the SimpleDeclare package and those required by it, so that declare-package, the command needed for package declaration, becomes available. In fact declare-package is used in line 2 and extends up to the end of the example.

Up to line 13 the package is described and the package requisites are given. The -namespaces flag specifies the package namespaces, where the package errors, variables, commands and such are to be placed. Notice the ::Stack::Priv private namespace.

At lines 14, 22, 28, 45 and 66 a package variable, an error and three command declarations appear; see the sections on variable, errors and commands for a description of their declaration, respectively.

Once the above package has been published to Tcl via the usual pkgIndex.tcl file, it can be used as follows from within the Tcl shell:

Lines 1 and 2 request the simplepackageexample package. The package public API is displayed at line 5; notice how the ::Stack::Priv::dump private command does not appear. Another example of introspection is given at line 9 where the package variables are displayed.

Lines 11 and 12 push two elements into the stack. The stack is then dumped at line 13 using the private procedure; notice that Tcl has no concept of private or public procedures, so that one can access all of them irrespectively of their access modes.

Lines 16, 18 and 20 invoke the ::Stack::pop command, but the last one fails as the stack is empty. Notice that the errorCode Tcl global variable is set to the code or name of the SimpleDevLib error thrown, as shown at line 22.

3.3 SimpleDevLib classes

Implemented by

Declaration

Can embed

A SimpleDevLib class is an [incr Tcl] class or [incr Tk] widget. SimpleDevLib wraps most of the functionalities provided by [incr Tcl] and presents them in the form of class declarations, in a similar way to the declarations for other items.

Class declarations specify the class errors, variable, attributes, commands and methods. Also, for [incr Tk] classes the declaration may include [incr Tk] options and [incr Tk] components. Other prominent functional components are the class type (-classtype), the class stereotype (-stereotype) and the classes this one extends (-extends).

The most significant features added by SimpleDevLib to [incr Tcl] classes, apart from wrapping via the structured declaration concept, are the ones for class variables and attributes (typing, as for package variables; see the variable section) and the ones for class commands and methods (extended arguments and subcommand handling, as for package commands; see the commands section). On top of that, SimpleDevLib classes provide support for [incr Tk] widgets via [incr Tk] options and [incr Tk] components.

See the SimpleClass package for a full explanation of SimpleDevLib classes.

The following is the canonical toaster example. The example does not include class variable, class commands, itkoptions nor itkcomponents.

There are two classes in this example, Toaster and SmartToaster.

The Toaster class declaration starts at line 3. At lines 12, 18, 26 and 50 several item declarations appear: an error, an attribute and two methods; see the sections on errors, attributes and commands for a description of their declaration, respectively. Notice how item names within classes are given unqualified, while those for packages (see the package declaration example above) are qualified. The reason is that class items live within the class, which in this respect is equivalent to a namespace, while packages can encompass more than one namespace.

The SmartToaster class starts at line 66. At line 70, the -extends flag specifies that this class extends the Toaster class. At line 77 the toast method is redefined. Notice how the base class toast method is invoked at line 96; this is pure [incr Tcl] code.

Once the above classes have been declared within a Tcl interpreter, it can be used as follows from within the same Tcl shell:

As examples of class introspection the Toaster class public API is displayed at line 1 and the SmartToaster class attributes at line 5.

At line 7 a SmartToaster object is created. We toast 15 slices at line 9, which brings the number of crumbs up to 60. At line 11 we toast yet another slice. This triggers the autoclean feature, so that the final number of crumbs is just 4.

At line 13 we instantiate a regular toaster. Toasting 15 slices at line 15 and another one at line 17 results in the toaster catching fire!

3.4 SimpleDevLib procedures and methods

Implemented by

Declaration

Can embed

A SimpleDevLib procedure or command is a Tcl command procedure. A SimpleDevLib method is an [incr Tcl] method.

Command declarations specify their access mode (-access), alias (-alias; not supported for methods), arguments (-arguments), whether they override the SimpleProc package -checktype flag, and whether the flag arguments parsing is done automatically or is delegated to the user (-parseflags).

Additionally, class constructors can specify a script used to invoke base class constructors (-init).

SimpleDevLib adds two useful features to Tcl commands and [incr Tcl] methods: extended arguments and subcommand handling.

SimpleDevLib command and method arguments have an extended argument syntax. This syntax includes typing (with optional run-time type checking) and flag arguments with or without parameters which may be automatically parsed or left to the user.

Each extended argument is a list of three or four elements. The first is the argument name, the second its type and the last its description; if there are four elements, the third is the default value.

Argument names with a leading dash (such as -flag) are flags, whose presence in the actual arguments list is optional. If present they require a parameter, and the flag value is set to that; otherwise flags default to the flag default value which is mandatory in their definition. Boolean flags are special in that their type is boolflag and have no default value: their value is set to either 1 or 0 depending on whether they appear in the actual arguments list. The user refers to flag arguments removing the leading dash (such as in $flag).

Default values must conform to the argument type but conformance is not assessed for the actual arguments in run-time by default. The user can switch on run-time type checking individually per command or method (by using the -checktype flag in its definition) or globally via the -checktype SimpleProc option.

For example, consider the following list of arguments:

This list of arguments specifies an initial optional flag -boolflag with no parameter, followed by an optional flag -flag with a float parameter whose default value is 1.0, followed by two integers (arguments int1 and int2), optionally followed by an integer whose default value is 999, optionally followed by an arbitrary number of arguments with any type. A procedure named foo defined with the above list of arguments accepts all the following invokations:

Additionally, several ::Simple::Argument::information subcommands can be used within the command body to obtain information about the arguments; for example, the flaggiven subcommand returns whether a certain flag appears in the actual arguments list, while the isdefault subcommand returns whether an argument value is equal to its default value.

Subcommands is SimpleDevLib term for the concept named ensemble commands, command families or composite commands in other contexts. Basically one can define commands (or methods) with two words names, such as the commands foo bar and foo gee corresponding to subcommands bar and gee for the foo base command. This facility not only automates the definition of subcommands, but it automates the error handling as well. If the base command is invoked with an incorrect subcommand, an error is thrown informing of the allowed subcommands; also, when a subcommand is invoked with a bad number of arguments, an error is thrown informing of the argument list for that subcommand. Further, the ::Simple::Subcommand::information subcommands allows to obtain the list of subcommands for a given base command.

See the SimpleDeclare (declare-proc command), SimpleClass (declare-class command, -method flag) and SimpleProc packages for a full explanation of SimpleDevLib procedures and methods.

Some simple examples of SimpleDevLib commands and methods can be found above.

3.5 SimpleDevLib errors

Implemented by

Declaration

Can embed

SimpleDevLib errors are Tcl errors indexed by name. The name is also the error code.

Error declarations include, as a minimum, the error code and error message (-message). Additionally, errors can specify an explanation (-explanation), corrective action (-corrective) a formatted error message (-explained) and a translation string (-translation).

SimpleDevLib errors can be thrown by name (using the error code) and accept error parameters when thrown (using the error message, which is a format string in the sense of the format Tcl command).

Consider the following example:

The above code snippet would throw a Tcl error with the message no file named foo.tcl in directory /usr/home and error code ::FILE-NOT-FOUND.

Errors can also be explained to the user. This mechanism uses the -explanation, -corrective, -explained and -translation error declaration flags; see the ::Simple::Error::explain and ::Simple::Error::catch-explain procedures for details. As an example:

The above would display the following:

See the SimpleError package for a full explanation of SimpleDevLib errors.

3.6 SimpleDevLib types

Implemented by

Declaration

Can embed

SimpleDevLib types associate a type name to a matching script used to assess values type conformance. These types can then be used in the declaration of other SimpleDevLib typed items (such as commands and method arguments, variables and attributes) so that it is possible to check and/or force the conformance of those items values to the type.

SimpleDevLib ships with some 50 predefined types, and more can be declared. There are two kind of types: basic and derived. Derived types are derived from basic types and are denoted as basic type-derivation where basic type is the basic type name (such as integer) and derivation is the kind of derivation. All basic types can be derived but boolflag. Four derivations are currently supported: arrays, pairs, lists and ranges. See the SimpleType package for details.

See the SimpleError package for a full explanation of SimpleDevLib errors.

Consider the following example:

The above delares a new type named bartype. Items with this type can only have values of either bar or BAR. In this way, the code

throws an error invalid value "foo" for argument "-n" of type "bartype".

The following example illustrates the use of derived types:

The above code line displays 0, as the given list is not a proper list of bartype's because of its last element.

3.7 SimpleDevLib variables and attributes

Implemented by

Declaration

Can embed

SimpleDevLib variables and attributes are (non-local) Tcl variables and [incr Tcl] attributes, respectively.

Both variable and attribute declarations specify their type (-type), choices (-choices, used for the type choice or derived from it) and initial value (-value). Initial values must conform to the variable or attribute type but conformance is not assessed when changing the value in run-time by default. The user can switch on run-time type checking individually per variable or attribute (by using the -monitortype flag in its definition) or globally via the -monitortype SimpleVariable option.

Additionall, class variables and attributes support [incr Tcl]'s access modes (-access), and attributes can specify a script which gets evaluated upon its configuration (-configbody).

The most significant feature added by SimpleDevLib to variables and attributes is typing (with optional run-time type checking).

See the SimpleVariable and SimpleClass (declare-class command, -attribute flag) packages for a full explanation of SimpleDevLib variables and attributes.

The following example illustrates the concept of run-time type monitoring:

The above code throws an error can't set "foobar": invalid value "gee" for variable "::foobar" of type "choice": must be foo or bar.

4. SimpleDevLib packages and utilities

4.1 SimpleDevLib packages

SimpleDevLib is organised as a series of Tcl packages, each with its own documentation and and test suite. The list of packages is the following:

As some SimpleDevLib packages depend on others, it is possible to use the most core ones without loading the ones not needed. For example, a user may just load the SimpleError package in order to benefit from the concept of indexed errors, or handle subcommands by loading SimpleSubcommand. The following sketches the dependencies between packages:

4.2 SimpleDevLib utilities

SimpleDevLib also contains a number of utilities as follows:

5. Copyright

Copyright (C) 1999-2005, Juan C. Gil mailto:jgil@gmv.es.