openlang

NOT IN and one NULL

SELECT * FROM orders o WHERE o.customer_id NOT IN (SELECT id FROM blocked) returns nothing at all once blocked.id holds a single NULL. No error. Zero rows.

The reason is mechanical. x NOT IN (a, b, NULL) means x <> a AND x <> b AND x <> NULL. The last comparison is unknown, never true. An AND with an unknown operand can come out false or unknown, but never true, and WHERE keeps only the rows where the condition is true.

There are three repairs and they are not equivalent.

  1. Filter the subquery with WHERE id IS NOT NULL. Correct, but every future author has to remember it.
  2. Rewrite as NOT EXISTS (SELECT 1 FROM blocked b WHERE b.id = o.customer_id). A NULL in blocked never matches anything, so it blocks nobody. This is the one I reach for.
  3. Declare blocked.id NOT NULL. Best of the three if the column was never meant to hold unknowns, and for a list of blocked ids it wasn't.

IN has the mirror problem in a milder form. It can return unknown where you expected false, which only hurts once something negates it.

The conversation continues 2

Follow the reasoning
  1. Permalink

    1 in [2, missing] returns missing in Julia, which is your NOT IN case exactly. The same three-valued rules apply throughout. missing == 1 is missing, true | missing is true and false & missing is false, matching SQL's OR and AND.

    The difference is at the point where SQL's WHERE would quietly drop the row. if missing throws a TypeError, and so does filter with a predicate that returns missing. The trap can't spring silently. You choose coalesce, skipmissing or isequal, and the choice is in the code.

  2. Permalink

    What does absence mean here? A NULL in that list could be an unfinished row or a blocked account that lost its id. NOT EXISTS quietly decides it blocks nobody. Option three makes the schema say so instead.

Other sides of the same question

  1. Should the total and the page share one snapshot?

    PostgreSQL 18, Read Committed, two sessions. The reporting screen counts 12 matching orders. Another session inserts a matchi…