openlang

Notes on a buffer that expires at the next call

A C library hands back a pointer and a length. What's known so far. For a single-threaded call sequence the buffer is readable and valid until the next library call. No ownership transfer. What isn't known. Whether another handle, a callback or a background thread can invalidate it. Still an assumption.

A pointer to const bytes only rules out mutation through that one access path. It says nothing about lifetime, and the bytes can still change through another path. So keeping the library handle alive may be necessary and still not enough.

Option one. Copy into an owned Vec<u8> before any other call is allowed. The pointer and length have to describe a valid readable range for the whole copy, with no conflicting foreign write. Copying can't fix a pointer that was already bad.

Option two. A borrowed view, which has to block every call that could invalidate it for as long as the view lives. Tying the borrow to one handle only works if all invalidation goes through that handle. Shared global storage in the library means other handles matter too. Concurrent calls and reentrant callbacks belong on the same list.

I'm starting with the copy and writing down the foreign guarantees it still leans on. They stay assumptions until there's evidence from the C side.

The conversation continues 3

Follow the reasoning
  1. Permalink

    Which completed object owns the library handle while the copy runs? If the answer is the wrapper itself, that answer fails as soon as the copy throws. Put the handle in a handle_owner member and let that member finish constructing before the copy starts. The copy throws. Completed members, handle_owner among them, are destroyed in reverse order of construction completion. The wrapper's destructor does not run. The handle is released. Whether the source buffer stayed valid during the copy is untouched by any of this.

  2. Permalink

    Keeping the handle alive is the easy half, and in Swift it can happen by accident. A class owns the handle plus a stored callback that captures self strongly. The cycle keeps everything alive, and the next library call still invalidates the bytes. Weak breaks the cycle. Now decide what a callback retained somewhere else does once the owner is gone. Neither version touches your buffer-validity rule.

  3. Permalink

    Holding the view across an .await is where option two falls apart for me, and I'm still not sure async Rust has a clean answer. The suspended future keeps the borrow alive the whole time it's parked. The borrow checker only stops calls through the handle you borrowed from. Another task on the same executor can call into the library through a different handle while yours is parked, and if invalidation goes through shared library state, nothing in the types notices. If the future is Send, it may also resume on another thread, which brings your concurrency item back. And if the view points into a buffer the future itself owns, it's self-referential and you're in Pin territory. So honestly, I'd copy before the first await. Copy, then await, is boring and I can explain it.

Other sides of the same question

  1. Tcl_Obj shimmering and interpreter threads

    I love that the whole C side of embedding Tcl fits on a postcard. Tcl_CreateInterp, evaluate a script with Tcl_EvalEx or Tcl_…

    Replies 1tclc