openlang

Callbacks from C lose the arena I set up

A C library calls back into my Odin code, and the callback can't see the context I set up before registering it. A proc "c" gets no implicit context, so it has to build one, and runtime.default_context() gives the default allocators, not the arena the registering code had in context.allocator. Anything the callback allocates goes to the heap and is never part of the frame clear.

What I do now is pass the arena through the library's user_data pointer and put it back by hand.

import "base:runtime"
import "core:c"
import "core:mem"

on_event :: proc "c" (user: rawptr, code: c.int) {
	context = runtime.default_context()
	context.allocator = mem.arena_allocator((^mem.Arena)(user))
	handle_event(int(code))
}

It works. The arena pointer has to outlive the registration, which nothing checks. And every callback repeats those two lines. Is there a tidier pattern, or is this just the price of crossing into C?

The conversation continues 8

Follow the reasoning
  1. Permalink

    The user_data pointer is lent to the library, and the library does not own the arena behind it. The C language says nothing about how long the library keeps that pointer, so the header or documentation has to say it, and the usual answer is until the matching unregister call. The arena must therefore outlive the unregister call, not merely the last callback you happened to observe.

    Practically, find the unregister function and read what it promises about a callback that is already running when it is called. If the documentation is silent, assume it returns without waiting.

  2. Permalink

    It's the calling convention. A default Odin proc takes the context as a hidden extra pointer argument, and a C caller passes only what the C prototype lists, so default_context() builds a fresh context rather than recovering yours.

  3. Permalink

    Before the allocator question, one ordering question. Does the library invoke the callback only from inside a call you make, such as a poll or dispatch function, or from threads of its own? If it is the first, the callback runs on your thread at a point you chose, and the arena pattern is fine. If it is the second, every assumption about the arena changes.

  4. Permalink

    Then put the unregister where the arena's owner can't skip it. Register right after the arena is initialised and write defer unregister(handle) on the next line. Odin runs defers in reverse order, so the unregister runs before the arena's own cleanup on every exit from that scope, early returns included. That fixes the order. It does nothing about a callback still running inside the library when unregister returns, which is the case nullbyte said to look up.

  5. Permalink

    Its own thread. The header has one sentence about it, callbacks run on an internal worker. So the arena trick was wrong in a way that happened not to show. mem.Arena isn't safe for concurrent use, and my main loop allocates from the same arena while the callback runs.

    Changed it. The callback now allocates nothing. It pushes the event code into a fixed ring guarded by a mutex, and the frame loop drains the ring at the start of each frame on the main thread, where the normal context and the arena are both valid. The two lines from my first post are gone. The ring has a fixed size, so the open question moves to what the callback does when it is full.

  6. Permalink

    Oh, that's the shape I trust most. A worker thread hands over a small fact and the owning thread does the real work at a point it picked. Tcl ended up with the same rule for interpreters, other threads post an event with Tcl_ThreadQueueEvent and the interpreter's own thread runs it later.

    On the full ring, dropping the newest event and counting the drops is the gentle option, and the frame loop can log the count. Blocking the library's worker until a frame drains it is the scary one, since the library may hold its own locks while it calls you.

  7. Permalink

    The ring removes the arena from the callback, and the lifetime question from my first reply now applies to the ring and its mutex instead. Both are reached through user_data, so both must stay valid until no callback can still be running. Whether that point is the return of the unregister call is the one thing the header has to tell you. If it says nothing, the ring cannot be freed safely before the library itself is shut down.

  8. Permalink

    There are two separate milestones hiding in "no callback can still be running". One is that unregister has returned, so the library will start no new calls. The other is that every call already started has finished. A counter inside the callback can't establish the second one by itself, because a call can be inside the library's dispatch code, past the decision to call you, and not yet at your first line where the counter goes up.

    So the library has to supply that second milestone, as a waiting unregister or a shutdown that drains its worker. If it supplies neither, keeping the ring alive until the process exits is a perfectly fine answer for a ring this small.

Other sides of the same question

  1. Unwrapping a C pointer with .? is an assertion

    .? on a pointer that came back from C is an assertion, not a check. It says null is impossible here. In a safety-checked buil…

    Replies 5zig
  2. Clearing one status flag with |= cleared all of them

    |= on a write-1-to-clear status register clears more than the bit you named. The overrun flag clears by writing 1 to it. Writ…

    Replies 5c
  3. What the canary check looks like at -O2

    Three small C functions, gcc -O2 -fstack-protector-strong -S -masm=intel, x86-64 Linux. read_line and read_count get the same…