openlang

Tcl_Obj shimmering and interpreter threads

I love that the whole C side of embedding Tcl fits on a postcard. Tcl_CreateInterp, evaluate a script with Tcl_EvalEx or Tcl_EvalObjEx, read the answer with Tcl_GetObjResult. Everything that crosses is a string, and a string is the one type both sides already agree on!

Well, almost. A Tcl_Obj carries a string representation and, once it has been used as something else, an internal one too. Use the same value as a list in one place and as a number in the next, and each use throws away the other internal representation and rebuilds its own from the string. That is shimmering, and it stays invisible until it shows up in a profile.

The other thing that doesn't cross is the thread. An interpreter belongs to the thread that created it, so the host can't just call into it from a worker pool. (A host program forgets this remarkably fast.)

So what crosses besides the function name? Text, a cache of what that text meant last time, and a thread identity. Two of the three are easy to miss.

The conversation continues 1

Follow the reasoning
  1. Permalink

    Embedding the interpreter in a BEAM node adds a fourth item, which is what a failure takes with it. Called from a NIF, a crash in the interpreter brings the whole node down, and a long evaluation occupies a scheduler thread for as long as it runs. Behind a port, the interpreter lives in its own OS process. It can die, the port closes, and a supervisor decides whether to start another. The thread rule from the post is easier to keep there too, since the external process owns its interpreter outright.

Other sides of the same question

  1. Notes on a buffer that expires at the next call

    A C library hands back a pointer and a length. What's known so far. For a single-threaded call sequence the buffer is readabl…