openlang

count(*) arrived as a string and 12 + 1 made 121

A todo list screen. The header says 12 items, I add one, and to bump the number before the server answers I do total + 1. The header now says 121 items. (For a second I thought a hundred todos had appeared out of nowhere.)

Here's why. PostgreSQL's count(*) is a bigint, and the Node pg driver returns bigint as a string by default, since a JavaScript number can't hold every 64-bit integer. And the row type is something I wrote by hand.

type CountRow = { total: number };

const { rows } = await pool.query<CountRow>(
  'SELECT count(*) AS total FROM todos WHERE done = false',
);
setTotal(rows[0].total + 1);

The type says number, so the editor stays quiet, and at run time '12' + 1 is string concatenation.

I can see two fixes. Cast in SQL with count(*)::int, or parse the value in code. With only a few queries the cast looks easier. Where do you all put it?

The conversation continues 7

Follow the reasoning
  1. Permalink

    The hand-written CountRow is the real bug, and it stays whichever fix you pick. pool.query<CountRow> is an assertion with nicer syntax. Nothing checked that total is a number, the generic only told the compiler to believe it.

    The cast is fine for a count. count(*)::int fails loudly with an out of range error somewhere past two billion, which a todo list will not reach. But it fixes this one column, and the next bigint column arrives as a string again under a type that says otherwise. A small parser at the boundary covers both.

    function toCount(value: unknown): number {
      if (typeof value !== 'string' || !/^\d+$/.test(value)) {
        throw new Error(`not a count: ${String(value)}`);
      }
      const n = Number(value);
      if (!Number.isSafeInteger(n)) {
        throw new Error(`count too large: ${value}`);
      }
      return n;
    }

    Run the row through it and build CountRow from what comes back. This early I'd still cast in SQL and add the parser when a second query needs it.

  2. Permalink

    There's a third option worth putting beside the two. Parse to bigint and keep it. Then total + 1 doesn't compile, because TypeScript refuses to mix bigint and number in arithmetic, and you have to write total + 1n. That error is the one the screen needed. The cost is everywhere else. Formatting, strict equality with plain numbers, and anything that crosses into JSON all need a conversion. For a count that will never leave the safe integer range, number is the cheaper representation, and the parser tess.p posted is the right place to make that decision.

  3. Permalink

    Went with the SQL cast for now, as tess.p suggested, count(*)::int. The header goes up to 13 like it should. But further down the same screen there's an estimated cost total. It's sum(price), and price is numeric, so that one arrives as a string too. Nobody noticed because it only ever gets printed as is. I was about to slap a parseFloat on it so it could also bump when a todo is added, and then stopped. It's money. Is that okay?

  4. Permalink

    No. The driver hands numeric over as a string precisely because the value is decimal and a binary float is not. parseFloat('19.99') is already an approximation, and adding a few of them gives a total that prints with a long tail of digits or rounds the wrong way at the last cent. Keep the string for display, since the database computed that sum exactly. If the screen must add to it before the server answers, convert to integer cents first, add integers, and let the server's figure replace yours when it arrives. A money column should be added up where its scale is declared.

  5. Permalink

    The JSON part bites sooner than it sounds. JSON.stringify throws a TypeError on a bigint, so the first time that total goes into a response body or a saved state blob, it fails at run time and the compiler never said a word. You can give BigInt.prototype a toJSON, but then every bigint in the program serializes the same way, whether that suits the receiver or not. Agreed that number is the right call here, and the conversion belongs in the parser rather than at each place a value leaves.

  6. Permalink

    Okay, parseFloat is gone. I started on the integer cents version, then asked myself whether bumping the cost estimate before the server answers was worth having at all, and dropped it. The cost total now changes when the response arrives, and for the half second in between the old value stays up. (Better than half a second of a wrong number.) The optimistic bump stays on the count only.

  7. Permalink

    Slightly off the type question. The optimistic total + 1 assumes nobody else added a todo since the count was read, and having the insert return the new count replaces the guess a moment later.

Other sides of the same question

  1. rows.Next said false and the error went nowhere

    Small tool on database/sql returned half a list. No error. The code. Context with a deadline. The deadline fired in the middl…

    Replies 3sqlgo
  2. UPDATE rechecks its WHERE after the wait

    Two sessions under PostgreSQL Read Committed, one account with a balance of 15, and both sessions run the same statement. Ses…

    Replies 0sql
  3. count(*) OVER () or Repeatable Read for a paged report

    Read Committed lets a paged report disagree with itself. The reporting screen counts 12 matching orders. Another session comm…

    Replies 9sql