openlang

canceled context but worker goroutines still alive after first error (always workers + 2)

Hi all,

I am building a small Go CLI, shotcheck, that checks every PNG in a folder against the exact size the App Store slot wants. A pool of workers (8 by default) reads names from jobs, checks them and sends a Result into an unbuffered results channel. collect in main reads them, and on the first bad one calls cancel() and returns an error. Visible behaviour is right: it prints checked 11 of 40, then FAIL with the file name, exit code 1.

My problem is the -debug flag. After a failure I print runtime.NumGoroutine() after a short sleep and expected 1. I get 10 with 8 workers, and 6 with -workers 4, so always workers + 2. A longer sleep (2s, then 5s) changes nothing. I am writing this from memory of my last run, as I can't run Go on this machine right now.

The producer goroutine does stop and jobs is closed, so I don't understand why the canceled context did not stop the rest. I tried moving cancel() around, no change. I suspect the WaitGroup, or defer cancel() running too late.

In the CLI it is harmless because the process exits. But I want this pool inside a long-running CI helper, and there I think these goroutines would pile up on every failed run and slow the machine a lot.

The worker part:

for i := 0; i < n; i++ {
	wg.Add(1)
	go func() {
		defer wg.Done()
		for path := range jobs {
			r := checkShot(path)
			results <- r
		}
	}()
}

go func() {
	wg.Wait()
	close(results)
}()

Is it possible that I am missing something simple here? Thanks for reading))

The conversation continues 4

Follow the reasoning
  1. Permalink

    Pretty much sure the workers are stuck on results <- r. After collect returns, who is reading that channel?

  2. Permalink

    Thanks, fenwick40, I think you are right. After collect returns nobody reads results, and the channel is unbuffered, so every worker that already has a result blocks on results <- r forever. The WaitGroup never reaches zero, so close(results) never runs. That would be the "+ 2": the goroutine waiting on wg.Wait() and probably one more helper. And cancel did nothing because the workers never look at ctx while they are blocked on the send.

    So the fix is to make the send respect the context:

    select {
    case results <- r:
    case <-ctx.Done():
        return
    }

    Also check ctx.Done() in the loop before checkShot. I can't run it now, will try tomorrow and report back.

    About "slow the machine a lot": probably I overstated it, a parked goroutine is cheap, but in a CI helper they would still leak every failed run, so worth fixing))

Other sides of the same question

  1. Marking the filter async did nothing for the freeze

    ledgerscan search over 400k entries. i made run async and await it from main, thinking the 100 ms heartbeat Timer would keep…

    Replies 2Latest activity dart
  2. await for over ReceivePort never finishes after isolates are done

    hi all, small dart tool here, it summarises temperature CSVs and I spawn one isolate per file so the big ones don't wait for…

    Replies 2Latest activity dart
  3. outliers.log has more lines than the :outliers counter, 8 futures

    I fold 20000 simulated tide readings into one atom from 8 futures. Every reading above the threshold also gets appended to ou…

    Replies 1Latest activity clojure