openlang

Braces around expr decide what the bytecode compiler sees

Unbraced expr is the one Tcl habit I'd ban if I could! It looks like a style nit and it isn't.

set qty {[puts "side effect"; list 3]}
puts [expr {$qty == 3}]
puts [expr $qty == 3]

The braced line prints 0. The bytecode compiler sees $qty == 3 as an expression, reads the variable once, and compares a weird string with 3. The unbraced line prints side effect and then 1. Tcl substitutes $qty while building the command words, expr gets the text [puts "side effect"; list 3] == 3, and then does its own round of substitution, which runs the bracketed command. Two substitutions, and the second one executes whatever the variable held.

That's the fun part. The quieter part is speed, because the unbraced form can't be compiled ahead and gets parsed each time it runs. Checked on 9.1 here, but this has been true for a very long time.

The conversation continues 13

Follow the reasoning
  1. Permalink

    Worth reading the braced line once more, because == is doing something too. It compares numerically when both sides look like numbers and falls back to string comparison otherwise, which is why the weird string just gives 0 instead of an error. That fallback bites on its own. expr {"1.0" == "1"} is 1, and expr {"1.0" eq "1"} is 0. If the intent is comparing text, eq says so and never tries the numeric reading.

  2. Permalink

    Does the same thing happen with if and while? Their conditions are expressions too, and I see if $done in old scripts more often than I'd like.

  3. Permalink

    Counted it with tcl::unsupported::disassemble on two one-line procs. expr {$a + $b} compiles to four instructions, two loadScalar, one add, done. expr $a + $b compiles to eight. It loads a, pushes the spaces and the + as literals, loads b, joins all five pieces with strcat, and hands the result to exprStk, which parses and runs the expression at run time on every call. Same answer for numbers, and the second one does a string build and a parse per call.

  4. Permalink

    Yes, both! if $done gets the double substitution, same as expr. while is worse, because the condition is substituted once, before the loop starts.

    set i 0
    set guard 0
    while "$i < 3" {
        incr i
        if {[incr guard] > 10} {
            puts "still looping, i is $i"
            break
        }
    }

    The quotes turn the condition into 0 < 3 before while ever runs, so while re-evaluates a constant and the loop never ends (the guard is only there so the demo stops). With {$i < 3} it re-reads i on every pass. The same goes for for and the test expression it takes.

  5. Permalink

    The same double substitution reaches C hosts that build scripts by concatenation. A command assembled with sprintf and passed to Tcl_Eval is parsed as script text, so a value containing brackets runs. Tcl_EvalObjv takes the command as an array of words that are not parsed or substituted again, which means a value stays one word whatever characters it holds. That is the C equivalent of bracing. It does not help if one of those words is itself a script, for instance the body passed to eval or an unbraced expr, so the rule still has to hold inside the Tcl code that receives the words.

  6. Permalink

    This is the version that would actually happen to me. A little Tk form has a quantity entry, the total is computed with expr $qty * $price, and somebody pastes [clock seconds] into the quantity box to see what happens (people do paste weird things into boxes). The total turns into a huge number and nobody can tell where it came from. Bracing fixes the substitution. Should the entry also refuse anything that isn't a number before the value gets anywhere near expr, or is the braced form enough on its own?

  7. Permalink

    The usual comparison is to SQL bind parameters, and it holds better than most analogies. Braced, the expression text is fixed when the procedure is compiled and $qty arrives at run time as a value, never as expression syntax. Unbraced is string-built SQL, reparsed on every call, as max8's count shows.

    One qualification. A bound value is typed by the parameter. A braced operand is still interpreted by expr according to what the string looks like. With qty set to 0x10, expr {$qty + 0} gives 16, and with 1e3 it gives 1000.0. So bracing removes the injection, and the numeric reading still needs the entry check kim_dh asked about.

  8. Permalink

    Of the wordings, "braces hand expr the expression as written" is the one I'd put in a review comment. "Braces make expr faster" is true and gets them treated as an optimisation you can skip.

  9. Permalink

    PHP had this as create_function, which ran its string body through eval behind the scenes, so anything interpolated became code. It was removed in PHP 8, and closures cover what people used it for.

  10. Permalink

    Went through my own scripts for unbraced expr after all this, and most were plain laziness. One wasn't! A tiny calculator proc did expr $a $op $b with the arithmetic sign kept in a variable, and bracing it breaks, because expr {$a $op $b} is a syntax error, a substitution can only give an operand. The fix I liked is the tcl::mathop command family.

    set a 3
    set b 4
    set op +
    puts [tcl::mathop::$op $a $b]

    That's a normal command call, so $a and $b are passed as words and nothing gets parsed twice. Prints 7. A switch on $op with an allowed list in front would be the stricter version, since tcl::mathop:: also has things like ** and << that the calculator probably shouldn't offer.

  11. Permalink

    Both, in order. The entry check comes first and decides whether the value is ready to be a quantity at all. string is double -strict $qty is the one to use, because without -strict the empty string passes, and an empty quantity box would then reach expr and fail there with a less friendly message. Once the check passes, the braced expr computes the total. Bracing protects the calculation from the text, and the check tells the user what was wrong before any calculation starts. tally6's 0x10 example passes string is double -strict too, so if hex shouldn't count as a quantity, that's one more rule for the entry.

  12. Permalink

    I'd push back a little on the qualification. Hex and exponent forms are numbers, so reading 0x10 as 16 is the value being typed, just with a wider idea of number than the user has. That isn't the same kind of problem as injection, and I'd rather not file them together. The entry check decides which numbers the form accepts. The braces decide whether text can become code. Mixing them is how people end up thinking a numeric check is what made the unbraced version safe.

Other sides of the same question

  1. D's pure lets a function write through its arguments

    D's pure is weaker than the word suggests, and I only noticed because an evaluator experiment wanted to cache calls. pure in…

    Replies 4d
  2. A macro can't make a newline end a form

    A macro can't make a newline end a form. I keep wanting it to. The notation is tiny. paint red blue on its own line should re…

  3. How do you replay a rule that reads the clock?

    Tried to replay a pricing decision and couldn't. The rule, eligible(item) in Lispex, read the host clock and whatever catalog…