我在写 hostprobe,一个内部的小服务,cron 和几个脚本会以 /probe?host=NAME 的形式 POST 过来。4 个 worker,队列长度 8,探测目前就是 sleep 20 秒(ssh 检查以后再加)。我希望脚本失控的时候它还能保持小体量。
测试用的是 hey -n 100 -c 100。100 个请求几毫秒内全部返回 202。我本来预期从第 9 个开始会被拖慢或者被拒绝。/stats 显示队列 len=8 cap=8,4 个在运行,所以 channel 和 worker 看起来没问题。但 runtime.NumGoroutine() 大约是 100,/debug/pprof/goroutine?debug=1 里有一大组 goroutine 停在同一个位置。请求 1000 个的时候 goroutine 数涨到约 1000,而队列还是 8。Activity Monitor 里的内存每次突发之后也会涨一点。
handler 和池子我好久没动过了,基本是在盯着自己忘掉的代码看。handler 代码如下
job := Job{Host: host, Enqueued: time.Now()}
go func() {
q.jobs <- job
}()
atomic.AddInt64(&q.accepted, 1)
w.WriteHeader(http.StatusAccepted)
fmt.Fprintf(w, "queued %s\n", host)
池子就是常见的写法,4 个 goroutine 里 for job := range q.jobs,channel 是 make(chan Job, cfg.QueueDepth)。
多出来的 goroutine 是哪来的?我怀疑是 worker 或者 channel,但数字上两边都没问题。
从英语翻译·查看原文I'm building hostprobe, a small internal service that cron and a few scripts POST to as /probe?host=NAME. Four workers, queue of 8, and the probe is a 20s sleep for now (the ssh check comes later). I want it to stay small when a script goes wild.
Test: hey -n 100 -c 100 against it. All 100 came back 202 in a few ms. I expected the 9th onward to be slowed or rejected. /stats says queue len=8 cap=8 and 4 running, so the channel and workers look right. But runtime.NumGoroutine() is about 100, and /debug/pprof/goroutine?debug=1 shows one big group sitting at the same spot. With 1000 requests it went to about 1000 while the queue stayed at 8. Activity Monitor memory also climbs a bit with each burst.
I haven't touched the handler or pool in a while, so I'm basically staring at code I forgot. Handler:
job := Job{Host: host, Enqueued: time.Now()}
go func() {
q.jobs <- job
}()
atomic.AddInt64(&q.accepted, 1)
w.WriteHeader(http.StatusAccepted)
fmt.Fprintf(w, "queued %s\n", host)
The pool is the usual for job := range q.jobs in 4 goroutines, and make(chan Job, cfg.QueueDepth).
Where are the extra goroutines coming from? I suspect the workers or the channel, but the counts say both are fine.