I've got a small tool for work (mobile app) that reads an analytics export, a CSV of session events, about 6 million lines and roughly 300MB, and prints the mean session length. It gives the right answer. But on the full file the fans on my old Intel MacBook spin up, and +RTS -s reports maximum residency of well over a gigabyte.
I wrote it with foldl first, then swapped in foldl', and peak memory barely moved. -O2 is already in the cabal file and made no real difference either.
meanMillis :: [Session] -> Double
meanMillis ss
| n == 0 = 0
| otherwise = fromIntegral total / fromIntegral n
where
(total, n) = foldl' step (0 :: Int, 0 :: Int) ss
step (t, c) s = (t + sessMillis s, c + 1)
and in main:
contents <- readFile (optPath opts)
let sessions = mapMaybe parseLine (drop 1 (lines contents))
Session is a plain record with a String and an Int, and the filter by screen is just filter.
My first suspect is readFile and lines, so I thought about an hGetLine loop, but I haven't tried it. I don't see why the strict fold wouldn't be enough if the file is consumed lazily and each line is dropped after parsing. A profiling build is very slow on this machine, so I'm mostly guessing from the -s numbers.
Where is the memory actually going?