openlang

Swapping the config language won't fix drift

The usual pitch for replacing HCL files with a general-purpose language is loops and functions. Sometimes that's a fair point. But the complaint that tends to ride along with it is drift, and drift isn't a language problem.

HCL is a structured configuration syntax. Blocks, attributes, expressions, plus a JSON form that maps onto the same structure. The tool reading it decides what a block means. The file states what you want. Drift is the real resource changing behind the file's back, a disk resized by hand in a web console, a tag edited by a script nobody remembers. A Python program that emits the same desired state drifts in exactly the same way.

What a general-purpose language does change is how much a reviewer has to execute in their head. With plain blocks I read the diff and know which resource moves. With a loop over a map built by a helper, I have to run it first.

For files that mostly list resources, keep the boring syntax and spend the effort on catching drift. A scheduled plan that fails loudly when reality and the file disagree does more than any rewrite.

The conversation continues 4

Follow the reasoning
  1. Permalink

    Not convinced. Drift isn't the language's fault, agreed, as far as it goes. But HCL files grow a programming language anyway. for expressions, conditionals, functions, modules calling modules. You end up writing real logic, just with much thinner tooling for stepping through it or testing one piece alone.

    At that point I'd rather have a real language and a rule that the program stays dumb. Flat data in, resource list out, no cleverness in between.

  2. Permalink

    A plan check fails loudly only if the policy reading it can fail. A rule whose body tests input.resource.encrypted == false is undefined when the field is absent, so the rule produces nothing and the resource passes. Remove that field. What does the query return? Try a missing attribute and an explicit false as inputs, because only one of them is denied.

  3. Permalink

    A real language does buy you editor types, and catching a misspelled attribute before anything runs is worth something. It still only describes what the program meant to send. What happens if the provider hands back a field our type never declared? The plan is the first place the real state shows up, and nothing on our side has inspected it.

  4. Permalink

    Fair, as far as the editor goes. Type the generated config with a Python TypedDict and the checker flags a misspelled key at the call site, before anything runs. That's genuinely useful. But a TypedDict is a plain dict at runtime, so whatever you load back from the provider is whatever came over the wire unless you validate it yourself.

    I keep those as two jobs, typed construction and checked input. Only the first comes free with annotations, and I'm not sure the second ever gets cheap.

Other sides of the same question

  1. asyncio.TaskGroup when one fetch fails

    Concurrency comparisons between languages usually start with syntax. I'd start with what the calculation needs when something…