openlang

COUNT(col) changed what a ratio meant

A report showed the share of orders with a shipping date as COUNT(shipped_at) * 1.0 / COUNT(*). It was right for a while. Then pending orders started arriving in the same table with shipped_at NULL, and the share dropped. Nothing in the query changed.

COUNT(*) counts input rows. COUNT(shipped_at) counts rows where that value is not null. The ratio was always "non-null dates among all rows", and the set of rows moved under it.

Three things to check, and they are not equivalent.

  1. Name the denominator. COUNT(*) FILTER (WHERE status <> 'pending') says which rows the share is about.
  2. Watch the division. In PostgreSQL count returns bigint, so COUNT(a) / COUNT(*) truncates to zero for any share below one. Hence the * 1.0.
  3. Ask what NULL means in that column. Not shipped yet and shipping date unknown are different facts, and a count can't tell them apart.

With two shipped orders and two pending ones the versions disagree in plain sight.

SELECT COUNT(shipped_at) * 1.0 / COUNT(*) AS naive,
       COUNT(shipped_at) * 1.0
         / COUNT(*) FILTER (WHERE status <> 'pending')
         AS shipped_share
FROM orders;
-- naive 0.5, shipped_share 1.0

R has this split too. length(x) counts the NA entries, sum(!is.na(x)) doesn't. When the report is built in R I print sum(is.na(x)) beside the ratio, so the reader sees how many rows the numerator skipped.

The conversation continues 1

Follow the reasoning
  1. Permalink

    Print COUNT(*) - COUNT(shipped_at) next to the ratio. A shift like this then arrives as a number, not a mystery.

Other sides of the same question

  1. count(*) arrived as a string and 12 + 1 made 121

    A todo list screen. The header says 12 items, I add one, and to bump the number before the server answers I do total + 1. The…

  2. rows.Next said false and the error went nowhere

    Small tool on database/sql returned half a list. No error. The code. Context with a deadline. The deadline fired in the middl…

    Replies 3sqlgo
  3. UPDATE rechecks its WHERE after the wait

    Two sessions under PostgreSQL Read Committed, one account with a balance of 15, and both sessions run the same statement. Ses…

    Replies 0sql