Compile error "SmallVector unable to grow" when working with large vectors

[limber musk]

I've been toying with SIMD and my very first example crashed when the vector had around 100k elements. It works when NUM_POINTS is decreased to 10k.

Given:

const std = @import("std");

const NUM_POINTS: usize = 100_000;

pub fn main() void {
    var prng = std.rand.DefaultPrng.init(0);
    const random = prng.random();

    var points: [NUM_POINTS]i16 = undefined;
    for (&points) |*pos| {
        pos.* = random.intRangeAtMost(i16, 0, 1000);
    }

    const result = isGreaterThan(&points, 3);
    std.debug.print("{any}\n", .{result});
}

pub fn isGreaterThan(points: []i16, gte: i16) @Vector(NUM_POINTS, bool) {
    const vgte: @Vector(NUM_POINTS, i16) = @splat(gte);
    const vpoints: @Vector(NUM_POINTS, i16) = points[0..NUM_POINTS].*;
    return vpoints > vgte;
}

Results in:

LLVM Emit Object... LLVM ERROR: SmallVector unable to grow. Requested capacity (18446744073709486080) is larger than maximum value for size type (4294967295)

Is this a compiler bug or am I missing something?

[edgy galleon]

You should keep Vector size to a small multiple that can fit in your target's SIMD registers. Remember that a Vector is a SIMD abstraction foremost, and Zig uses value types here so const v: @Vector(128, u64) = @splat(0) is 1 KiB of data which is operated on using only the 16 simd registers (on x86_64) so most of that must be spilled to the stack.

[limber musk]

So I guess I would need process the data in batches, where the batch size is determined by my target.

How do people usually handle the case where the input is not a multiple of the batch size? Would I just pad the vector and ignore the padding when reassembling the result?

[limber musk]

I've tried something like this:

const std = @import("std");

const NUM_POINTS: usize = 20;

pub fn main() void {
    var prng = std.rand.DefaultPrng.init(0);
    const random = prng.random();

    var points: [NUM_POINTS]i16 = undefined;
    for (&points) |*pos| {
        pos.* = random.intRangeAtMost(i16, 0, 1000);
    }

    var result: [NUM_POINTS]bool = undefined;
    isGreaterThan(&points, 3, &result);
    std.debug.print("{any}\n", .{result});
}

pub fn isGreaterThan(points: []i16, gte: i16, result: []bool) void {
    const batch_size: u32 = 8;
    const n_batches: usize = ((points.len - 1) / batch_size) + 1;
    const gte_vector: @Vector(batch_size, i16) = @splat(gte);

    var b: usize = 0;
    while (b < n_batches) : (b += 1) {
        const batch = points[b * batch_size .. b * batch_size + batch_size];
        const batch_vector: @Vector(batch_size, i16) = batch.*;
        result[b * batch_size .. b * batch_size + batch_size].* = batch_vector > gte_vector;
    }
}

But this does not work because b is not comptime.

[limber musk]

Figured out a possible solution, with the help of the following blog post:
https://www.openmymind.net/SIMD-With-Zig/

pub fn isGreaterThan(points: []i16, gte: i16, result: []bool) void {
    const batch_size: u32 = 8;
    const gte_vector: @Vector(batch_size, i16) = @splat(gte);

    var pos: usize = 0;
    var remaining = points.len;
    while (remaining > 0) {
        if (remaining < batch_size) {
            for (points[pos..]) |p| {
                result[pos] = p > gte;
            }
            return;
        }

        const batch_vector: @Vector(batch_size, i16) = points[pos..][0..batch_size].*;
        result[pos..][0..batch_size].* = batch_vector > gte_vector;

        remaining -= batch_size;
        pos += batch_size;
    }
}

Could someone explain the difference of the following two examples to me?

Works as expected:

const batch_vector: @Vector(batch_size, i16) = points[pos..][0..batch_size].*;

Compile error index syntax required for slice type '[]i16'

const batch_vector: @Vector(batch_size, i16) = points[pos .. pos + batch_size].*;