我用 8 个 future 把 20000 条模拟潮汐读数折叠进同一个 atom。每条超过阈值的读数还会被追加到 outliers.log。汇总结果没问题,:n 始终是 20000,:outliers 每次运行都一样,因为种子是固定的。但日志的行数比计数器多。上次运行日志大约 250 行,计数器比这个略低。有些 id 出现了两次。用 --workers 1 时两边是一致的。
(defn- log-line [& parts]
(spit (:log-file config)
(str (str/join " " parts) "\n")
:append true))
(defn fold-reading [st r]
(let [lvl (:level r)]
(when (outlier? r)
(log-line "outlier" (:id r) (format "%.3f" lvl)))
(cond-> (-> st
(update :n inc)
(update :sum + lvl)
(update :sumsq + (* lvl lvl)))
(outlier? r) (update :outliers inc))))
(defn record! [r]
(swap! stats fold-reading r))
我原以为是多个线程 spit :append 导致重复写入,但那样的话计数器也应该是错的。我也检查过 partition-all 的分块,它们没有重叠,因为 :n 正好是 20000。每次运行前我都会删掉日志。同样的种子,下一次运行的行数又不一样。
统计值明明完全正确,日志怎么会不对?我要的是每个离群值正好一行。
从英语翻译·查看原文I fold 20000 simulated tide readings into one atom from 8 futures. Every reading above the threshold also gets appended to outliers.log. The summary is fine: :n is always 20000, and :outliers is the same every run because the seed is fixed. But the log has more lines than the counter. Last run it was around 250 lines against a counter a little lower. Some ids appear twice. With --workers 1 they agree.
(defn- log-line [& parts]
(spit (:log-file config)
(str (str/join " " parts) "\n")
:append true))
(defn fold-reading [st r]
(let [lvl (:level r)]
(when (outlier? r)
(log-line "outlier" (:id r) (format "%.3f" lvl)))
(cond-> (-> st
(update :n inc)
(update :sum + lvl)
(update :sumsq + (* lvl lvl)))
(outlier? r) (update :outliers inc))))
(defn record! [r]
(swap! stats fold-reading r))
I thought spit :append from many threads double-writes, but then the counter would be wrong too. I also checked the partition-all chunks and they don't overlap, since :n is exactly 20000. I delete the log before each run. The line count is different again on the next run with the same seed.
How can the stats be exactly right while the log is not? I want exactly one line per outlier.