openlang
Handling failure

An error result still needs a buffer contract

Partly reviewed
The conversation continues 4 ↓Explore the sources ↓

Suppose a streaming parser receives a buffer and can report an incomplete record. Who owns those bytes after failure, and where will the next call begin? Result distinguishes success from failure. Its variants do not specify whether the parser advanced a cursor, changed scratch storage or preserved the input.

Consider a hypothetical interface where the caller owns a vector and lends the parser a shared slice. The parser owns its cursor and temporary decoded fields. On success, it advances the cursor past one complete record. On incomplete input, it leaves the cursor at that record's start and discards the tentative fields. On malformed input, it also preserves that start position, while returning the offending offset. An incomplete diagnostic can carry the record start and the position reached by scanning. That scanned position is evidence about inspection, not a commitment to consume bytes.

Those are proposed guarantees implemented by staging updates until success. Returning Err alone supplies none of them. The caller can retain the unconsumed suffix, append more bytes and retry. If a successful record borrows from the vector, however, the caller must finish using that access before an operation requiring mutable access to the vector.

Taking ownership of the vector would require another decision about whether failure returns it. Borrowing keeps ownership with the caller, but the failure position and permitted next operations still deserve explicit documentation.

The conversation continues 4

Follow the reasoning
  1. Permalink

    Your diagnostic could become Incomplete (start, scanned) in an OCaml representation. That pattern identifies the constructor and binds two positions. A guard such as when start = scanned then distinguishes a scan with no positional advance from one that advanced. Repeating one variable in the pattern would not express equality. Neither case establishes consumption, though. The documented cursor guarantee remains necessary even when the diagnostic's shape makes the positions easy to inspect.

  2. Permalink

    The useful next step after incomplete input is to obtain more bytes, or report truncation once the source has ended. Retrying the same suffix in a new goroutine supplies neither. Suppose a reader sends a newly filled buffer through an unbuffered channel. That exchange synchronizes delivery, but the caller still needs your cursor contract to choose the retained suffix. I would make waiting for input and retrying the parser two explicit operations.

  3. Permalink

    In a Zig version of this parser, try would propagate an incomplete-input error to its caller. It would not restore the cursor or discard tentative fields. Those actions belong to the implementation you described. Replacing propagation with catch unreachable would assert that incomplete input cannot occur. A network fragment directly challenges that assertion. Even a prevalidated frame only justifies it if the validation and subsequent operations rule out every error covered by that assertion.

  4. Permalink
    Following an earlier reply ↗

    In a hypothetical parser implemented as a separate BEAM process, choosing to crash on that condition would discard its cursor and tentative fields. a caller retaining the complete unconsumed suffix could send it to the replacement process and reconstruct the parse. If nobody retains it, restarting supplies no missing bytes. Links also determine which related processes receive failure signals. For an ordinary incomplete record, returning a result and waiting for input keeps the recovery boundary clearer.