openlang

A loop that allocated a column at a time

Slicing in Julia copies, which I knew and then forgot inside a loop. Take a function that walks the columns of a matrix and does s += sum(A[:, j]) for each j. Every A[:, j] builds a fresh vector, sums it, and throws it away. @time showed the allocations growing with the number of columns, and the performance tips call unexpected allocation almost always a sign of a problem.

view(A, :, j), or @views in front of the loop, makes the slice a SubArray that points into A, and the copying goes away.

The part that made me curious is what else @views changes. It turns every slice in the expression it wraps into a view, including ones on the right of an assignment. So x = A[:, j] inside a @views block gives you a view, and writing into x afterwards writes into A. Perfectly reasonable once you see it. Easy to miss if you put @views on a whole function to fix one loop and the function also copies a column on purpose somewhere further down.

The conversation continues 4

Follow the reasoning
  1. Permalink

    The first @time call includes compilation, so its allocations are not the loop's. The second call is the one to read. A global A allocates on its own, so pass it in as an argument. Then compare two matrix widths. A count that grows with the column count points at the loop.

  2. Permalink

    liv_n's method settles it, it's the loop and not compilation. Inside a function, with A passed as an argument and only the second call read, the allocations scale with the column count without views and stop scaling with @views on the loop.

    One warning for anyone tempted to move @views up to the whole function. It walks straight into the thing my own post warned about.

    @views function colsums!(out, A)
        for j in axes(A, 2)
            out[j] = sum(A[:, j])
        end
        backup = A[:, 1]
        A[:, 1] .= 0
        return backup
    end

    backup is meant to be a copy. Under @views it's a view, so after the zeroing line it's all zeros too. The allocation numbers look great and the function returns the wrong thing. @views on the loop only, then.

  3. Permalink

    There's a middle path between the loop and the whole function. Leave @views wherever it is and spell the deliberate copy as copy(A[:, 1]). Inside the macro the slice turns into a view first, and copy then builds a fresh array from it, so backup survives the zeroing. Outside the macro the same line copies once more than it needs to, which is harmless. Either way the line means the same thing to someone who later moves the macro around, and that was the part I cared about.

    Then rerun the two-width comparison from liv_n's post with the macro on the whole function. One extra column-sized allocation per call should show up for the backup, and nothing that grows with the width.

  4. Permalink

    Read the bytes as well as the count. If @allocated on the second call divided by the column count lands near one Float64 column, about eight bytes per row, it's the slices and nothing else.

Other sides of the same question

  1. sum(X) on a matrix with one row

    A function takes a matrix of observations, rows are observations and columns are features, and normalizes each column with X…

    Replies 6matlab
  2. na.rm = TRUE on a group that is all NA

    Per-group totals with tapply(x, g, sum, na.rm = TRUE), and one group came back as 0. Every value in that group is NA. Not a s…

    Replies 4r
  3. f2py copies my C-ordered array on every call

    Passing a NumPy array into a Fortran subroutine through f2py. The array is created the usual way, np.zeros((n, m)), so it's C…

    Replies 3fortran