Next Previous Contents

3. The new flow for Tcl

(Sorry for the terminology - it is my own. Suggestions for improvement are most welcome ...)

This subsection intends to describe the basic ideas in as simple a manner as I manage to do it. If it looks way to abstract, do skip ahead to the next section where this things are implemented in code.

Let us keep for the moment Tcl's linear flow of execution: everything is set up for sequential evaluation, all function calls nest nicely. The explanation of the ideas is simpler within this framework, although many of them look like they may not be worth the trouble. Bear with me for a while, we will describe the new possibilities opened by this code design soon enough!

3.1 EvalFrames and Flow Managers

We define a new EvalFrame structure that contains all the information required for:

We maintain a stack of such EvalFrames. When a function needs to eval a script, it must set up and queue the corresponding EvalFrame, indicate which function should evaluate the EvalFrame, and finally call a FlowManager to perform the evaluation.

The FlowManager is responsible for calling the script evaluation, and when that evaluation returns it calls the return function of the EvalFrame that requested the evaluation. The manager itself returns after executing the return function of the first EvalFrame it evaled - i.e., the one responsible for the creation of the manager. We say that that EvalFrame owns the manager.

What this structure permits is to eliminate from the C-stack the calling function before evaluating the script, with obvious gains when there are deeply nested evaluations. We will see in the next section another new possibility opened by the independence of the flow of execution and the C-stack.

We implement two ways of requesting an evaluation: the standard one (ST) and the new non-recursive (NR). The way this is implemented is: a standard evaluation calls a new manager to perform the evaluation and awaits the return of the manager; the NR call schedules the evaluation to be performed by the last not-yet-returned manager.

Let us consider again the previous example : A calls B, which in turn calls C (which calls F)and then D. We assume that the call of B from A is a standard call, and let the other calls be all either ST or NR. The stack effects are (M represents here a manager; we also show for comparison the original Tcl implementation OR):

               ------- C-STACK -------
  FLOW         OR        NR       ST
  ---------------------------------------------
  start        A         A        A
  A calls B    A:B       A:M:B    A:M:B
  B calls C    A:B:C     A:M:C    A:M:B:M:C
  C calls F    A:B:C:F   A:M:F    A:M:B:M:C:M:F
  F returns    A:B:C     A:M:C    A:M:B:M:C
  C returns    A:B       A:M:B    A:M:B
  B calls D    A:B:D     A:M:D    A:M:B:M:D
  D returns    A:B       A:M:B    A:M:B
  B returns    A         A        A

3.2 Micro-threads

The manager has now the possibility to maintain different EvalFrame stacks (muThreads), and assign execution time to one or the other. The muThreads provide a very simple framework for collaborative multitasking and/or co-routines.

It should be very simple to create a muThread framework and solve the coordination problems between tasks if we maintain the TclThread convention that to each interp corresponds just one muThread.

In this case, the creation of a new muThread is as lightweight as the creation of a new interp. I think it should also be easy to define safe muThreads using the existing infrastructure for safe interpreters.

The existing code defines only the very basic infrastructure for muThreads with round-robin scheduling. In order to exploit them, it will be necessary to write a muThread extension or add the corresponding functionality to the core.

3.3 New additional API

The idea is to keep the Tcl library API as is, and to provide additional functions that provide the NR functionality. As an example, there will be a new Tcl_EvalObjNR that provides similar functionality to Tcl_EvalObjEx but does unwind the C-stack before running.

An NR-enabled caller (one that is prepared to be evicted from the C-stack) will call Tcl_EvalObjNR. A non-NR caller calls the old interface Tcl_EvalObjEx, which in turns actually calls Tcl_EvalObjNR in a new manager. This new manager protects the caller from eviction from the C-stack.

We will see that users of the old interface are in a sense " bad citizens" in terms of the new functionality, but they are also the main sufferers from bad citizenship:

3.4 Implementation in C

Why C?

We could reprogram Tcl on top of (Java, ML, CommonLisp, Oberon, ... ) instead of C. Some of these languages' standard implementations may provide an easier access to the needed functionality than C. So, why C?

The advantages of C are (AFAIU):

setjmp/longjmp

Given the C flow conventions this might seem an impossible task. POSIX however provides a mechanism for non-local function exits, usually associated with error handling: setjmp/longjmp.

The main idea is that a function signals a spot in its code as a possible non-local return target with setjmp. If another function then calls a longjmp to that target, the C-stack will be unwound and execution will continue just as if the original setjmp function were returning. Everything that went into the C-stack later than the setjmp is wiped out.

The main constraints for the use of setjmp/longjmp are:

This provides precisely what we need to implement the mechanism described previously: a manager calls setjmp and thus provides a barrier to the unwinding of the C-stack. The call to evaluate is then simply a longjmp to the last manager in the C-stack, which resumes evaling the EvalFrames in the stack.

Alternative implementations

We propose to use longjmp to signal a request for NR evaluation. What alternatives are there?

In any case, setjmp/longjmp does more or less exactly what we want and, assuming it really is widely available, there is no real need for an alternative.


Next Previous Contents