#how to achieve paralleism?

1 messages · Page 1 of 1 (latest)

prisma umbra
#

how do i achieve paralleism in the new Io interface? previously i used std.Thread.Pool which was enough
i tried using the following, but this only offloads the work to another thread, not multiple ones.

    var pool = std.Io.Threaded.init(.failing);
    const io = pool.io();
    var g: std.Io.Group = .init;
    g.async(io, foo, .{});
grizzled parrot
#

ummm, maybe try initializing pool with a working memory allocator?

hard hornet
#

use concurrent, and it needs a working allocator

prisma umbra
#

but i don't allocate memory though

grizzled parrot
#

the threaded runtime does

hard hornet
#

Threaded needs to in order to keep track of tasks, the old thread pool didnt because it didnt track tasks

prisma umbra
#

hmm alright

hard hornet
#

pedantic but you probably meant 'concurrency', paralelism cannot reliably be achieved on a modern gerneric OS.

hard hornet
jagged acorn
prisma umbra
#

tried using concurrent with a provided allocator

#

assert(prev_state & GroupClosure.sync_is_waiting == 0); is false

# 
var tsa = std.heap.ThreadSafeAllocator{ .child_allocator = allocator };
const thread_alloc = tsa.allocator();
var pool = std.Io.Threaded.init(thread_alloc);
const io = pool.io();
var g: std.Io.Group = .init;
defer pool.deinit();
#

im not sure what that assertion means

#

and this is my child allocator

var dba: std.heap.DebugAllocator(.{}) = .{};
var allocator: std.mem.Allocator = dba.allocator();
hard hornet
#

Threaded doesnt need a thread safe allocator as it already protects its state accross threads.

prisma umbra
#

I'm confused.
The init function requires a thread safe allocator, but you're saying it doesn't need one?

uE41OO6.png
prisma umbra
#

If i could understand what the assertion means, this would probably be easier

hard hornet
#

well then I am corrected.
the assert is checking if any remaining group tasks are being waited on. You are required to wait or cancel all tasks before you deinit the io implementation.

teal gull
#

what's the exact code you are running?

prisma umbra
#

give me a second, trying to simplify it, didnt originally post it as its quite a big one

# 

fn generateChunks() !void {
    var tsa = std.heap.ThreadSafeAllocator{ .child_allocator = allocator };
    const thread_alloc = tsa.allocator();
    
    var pool = std.Io.Threaded.init(thread_alloc);
    defer pool.deinit();

    const io = pool.io();
    var g: std.Io.Group = .init;

    var y
    
    loop through y
        var x
        loop through x
        var z
            loop through z
                const chunk_pos = int3.new(@intCast(x), @intCast(y), @intCast(z));
                const chunk = getChunk(chunk_pos).?;

                try chunk.init(chunk_pos, allocator);
                try MeshManager.markForMeshing(chunk.position);
            }
        }
    }
    try workOnChunks(io, &g, Chunk.Stage.terrain);
    try workOnChunks(io, &g, Chunk.Stage.decoration);
}

fn workOnChunks(io: std.Io, g: *std.Io.Group, stage: Chunk.Stage) !void {
    var y
    loop through y
        var x 
        loop through x
        var z
            loop through z
                const chunk_pos = int3.new(x, y, z);
                const c = getChunk(chunk_pos).?;

                try g.concurrent(io, ChunkGenerator.generateChunk, .{ chunk, stage });
            }
        }
        g.wait(io);
    }
}
#

here

teal gull
#

I'm not 100% sure, but from what I remember, Io.Group doesn't like being waited on more than once

#

create a new group and always have defer group.cancel(io) in there

prisma umbra
#

basically, call concurrent a bunch of times. then call wait once. repeat for every layer (y)

prisma umbra
teal gull
#

yeah, every group.wait() sets a flag on the group's state, and the assert hits on the second one