Eight queue slots can still leave 88 goroutines waiting

Put this admission check behind a Java API and someone will ask for a checked exception for saturation. It's only worth declaring if callers can do different things with it. An interactive caller might abandon the request, and a batch caller might defer it under its own bound. If both can only propagate it, the declaration is obligation with no recovery behind it. The throws clause isn't an inventory of failures anyway, since RuntimeException and Error subclasses stay unchecked.

Four running plus eight queued is twelve jobs. How many bytes is that?

It depends on what a queue entry holds. If the entry owns its payload, you can at least add up payload sizes. If it borrows data a caller still owns, a short slice can require a much larger buffer to remain available. That is what borrowing in Rust gives you, access without a transfer of ownership. A count of twelve bounds no bytes unless payload sizes and the retained backing data are bounded too.

Then the caller times out while a worker is still reading the borrowed data. The data has to stay valid until that borrow ends, whatever the caller now wants, so those bytes stay counted after the caller is gone.

There's one gap in the count. Four workers bound how much runs at once. They say nothing about the job a worker was holding when it stopped partway, after sending an external write and before reporting completion.

Whoever picks that request up again can't tell whether to resume, retry or report the write as unknown, unless something recorded the intent before the write went out.

Go isn't my usual language, but move the timeout one step earlier than the worker's reply and see what's left. The caller's ctx.Done() fires, it stops reading the reply channel and leaves. The worker finishes reading the borrowed data and sends its result. On an unbuffered reply channel that send blocks for good, so the worker never returns to the pool and its place never comes free. Give the reply channel a buffer of one and the send completes with nobody listening, and the result goes away with the channel. Neither version tells the worker the caller left. It only learns that by checking the context itself.

The simplest version I can see. At the request entry, try to take one of twelve places before launching anything. Nothing free, return saturation right there. A buffered channel with twelve tokens is enough for the places. The job keeps its place until it has really let go of whatever the place stands for, cancelled or not. Admitted work then shows up as places taken, and nobody needs another coordinator to see it. Alden is right that a place only counts jobs. Payload sizes and retained backing storage need a byte budget of their own next to it.

Agreed on the byte budget, and it has to be reserved before the body goes into work storage, not checked afterwards. Eleven admitted jobs holding 1 MiB each plus a twelfth that keeps a 200 MiB buffer alive through a short slice comes to 211 MiB behind your twelve places.

So admission takes two reservations, a job place and a conservative byte charge. An unknown body size needs an enforced cap to reserve against. If either reservation fails, release whatever was taken and return saturation.

When the caller's deadline passes while the worker still uses the buffer, both stay held. The worker drops the byte charge once nothing it keeps still points into the buffer, a slice copied into the result included, and the place when the job finishes.

Wrote the burst from the root as a test, with admission the way Vadim put it. Twelve tokens, a non-blocking send, default means saturation, and no handler gives its token back during the burst.

tokens := make(chan struct{}, 12)
var admitted, saturated atomic.Int32
var wg sync.WaitGroup
for i := 0; i < 100; i++ {
	wg.Add(1)
	go func() {
		defer wg.Done()
		select {
		case tokens <- struct{}{}:
			admitted.Add(1)
		default:
			saturated.Add(1)
		}
	}()
}
wg.Wait()
fmt.Println(admitted.Load(), saturated.Load())

Prints 12 88 on every run. The goroutine peak is where I was wrong. I sampled runtime.NumGoroutine while it ran and got a different peak each time, from under twenty to the high seventies. The rejected handlers still exist until they return, and how many of them overlap is up to the scheduler. Admission caps the waiting. The rejected goroutines I'd only avoid by checking before spawning, which in this test means doing the select in the loop itself.

Max, the twelve and the eighty-eight came out right and the peak didn't, but both of those are goroutine counts. What did each job in that test carry, a few bytes? The 211 MiB case from your earlier reply was about bytes, and nothing in this test touches them.

Give one admitted job a short slice into a large buffer and read the heap after the burst. What still holds that slice when the job finishes? In Go the slice keeps the whole backing array alive for as long as anything references it, and if the result keeps the slice, the buffer outlives the place.