#arraylist out of memory

1 messages · Page 1 of 1 (latest)

thin vapor
# 
var gpa: std.heap.GeneralPurposeAllocator(.{}) = .{};

    const alloc = gpa.allocator();
    defer _ = gpa.deinit();
    var k = HardForks.default(alloc);
    k.register(Slot.init(1));
    std.debug.print("hard forks test", .{});
image.png
warped stream
#

that isn't an OOM error, this is a type error

thin vapor
#

so it looks like when it appends it doesnt have mem

#

it says out of mem

warped stream
#

look closely and you'll see the compiler is telling you it's a compilation error

thin vapor
#

oh right yea

#

i need to check the type

#

if its an err

#

right?

warped stream
#

not the type, you need to properly unwrap errors and if you propogate them, ensure the return type is correct

#

what's the function that this happens in look like

charred quartz
#

it looks like you are trying to assign the result of append to a field, but append returns !void so it's probably not what you want

thin vapor
# 
const std = @import("std");
const ArrayList = std.ArrayList;
const allocator = std.mem.Allocator;
const Slot = @import("slot.zig").Slot;

pub const HardFork = struct { Slot, usize };
pub const HardForks = struct {
    hard_forks: ArrayList(HardFork),

    const Self = @This();
    pub fn default(alloc: allocator) Self {
        return .{ .hard_forks = ArrayList(HardFork).init(alloc) };
    }
    pub fn register(self: *Self, new_slot: Slot) void {
        //        const index = std.mem.indexOfScalar(HardFork, self.hard_forks, .{} );
        const index = for (self.hard_forks.items, 0..) |hf, i| {
            if (hf[0].value == new_slot.value) break i;
        } else null;

        if (index != null) {
            self.hard_forks = self.hard_forks.append(.{ new_slot, self.hard_forks.items[index.?][1] +| 1 });
        } else {
            self.hard_forks.append(.{ new_slot, 1 });
        }
        std.mem.sort(HardFork, self.hard_forks);
    }
};

test "hard_forks" {
    var gpa: std.heap.GeneralPurposeAllocator(.{}) = .{};

    const alloc = gpa.allocator();
    defer _ = gpa.deinit();
    var k = HardForks.default(alloc);
    k.register(Slot.init(1));
    std.debug.print("hard forks test", .{});
}

``` its the register function
warped stream
#

right, append doesn't return a new arraylist

thin vapor
warped stream
#

it mutates the arraylist in place

thin vapor
#

the out of memory part made it look like its a memory issue😅

charred quartz
#

so you can simply remove the assignment in this line ```rust
self.hard_forks = /* remove until here */ self.hard_forks.append(.{ new_slot, self.hard_forks.items[index.?][1] +| 1 });

warped stream
#

for future reference: runtime error stack traces don't usually include direct references to types, rather they trace function calls

charred quartz
thin vapor
#

but now i need to change the function return type

charred quartz
#

yep, thats how it is

thin vapor
#

where can i find the Error type for this

#

if i recall i need to write it as ErrorType!void

warped stream
#

you don't need to do that

thin vapor
#

or is there a generic error type

warped stream
#

you can just write !void

thin vapor
#

oh

warped stream
#

that will infer the error type

#

(it's not generic, it's the specific set of errors that are returned from the function)

charred quartz
#

but if you want to specify the error set explicitly you could use Allocator.Error

warped stream
#

there is a runtime-generic error type called anyerror, however you should not use this unless you specifically know that you need to

thin vapor
#

am i using the correct function for sorting

warped stream
#

depends on what you want to do

#

for structs, that's likely not the correct one

charred quartz
thin vapor
thin vapor
warped stream
#

I would probably write

std.mem.sortContext(0, self.hard_forks.items.len, struct {
    items: []HardFork
    pub fn lessThan(ctx: @This(), a: usize, b: usize) bool {
        return ctx.items[a].slot < ctx.items[b].slot;
    }
    pub fn swap(ctx: @This(), a: usize, b: usize) void {
        std.mem.swap(HardFork, &self.items[a], &self.items[b]);
    }
}{ .items = self.hard_forks.items });
charred quartz
#

why sortContext?

warped stream
#

could also write

std.mem.sort(HardFork, self.items, {}, struct{
    pub fn lessThan(_: void, a: HardFork, b: HardFork) bool {
        return a.slot < b.slot;
    }
}.lessThan)
charred quartz
#

yeah thats the simpler version, and you can also just put the function outside of the call itself

warped stream
# charred quartz why `sortContext`?

I mostly use context functions because I usually end up doing something more complex with the sorting which requires I use the context variants anyway

thin vapor
warped stream
#

especially with MultiArrayList

thin vapor
#

oh its the actual sort function

charred quartz
thin vapor
warped stream
#

no

#

in that case you'd need to access the field and compare that

thin vapor
#

but theres no way to tell it to compare

#

oh wait

#

eql

#

nvm

warped stream
#

a.slot.value < b.slot.value

thin vapor
#

eql cn be used to compare entire structs as we prev discussed

warped stream
#

only for equality

#

not order

thin vapor
#

there shud be something like eql for order too

warped stream
#

don't really see the use case. The scope is much more limited

charred quartz
#

specifically in most cases there is no natural order for structs

thin vapor
#

i seem to have got a long list of errors now😐

warped stream
#

it would appear you are not freeing your memory

thin vapor
#

i deferred though

warped stream
#

ensure that at some point you call .deinit() on your arraylist

warped stream
#

defer doesn't do anything by itself

thin vapor
charred quartz
warped stream
#

that doesn't free the list

thin vapor
warped stream
#

that deinitialises the gpa state and tells you you leaked memory

#

you still need to free your allocations

charred quartz
#

if you wanted that behaviour, you'd need to use an ArenaAllocator

warped stream
#

recommendation: add a deinit method to your HardForks type

thin vapor
warped stream
#

inside which you do self.hard_forks.deinit()

#

also

thin vapor
#

i did it from outside

warped stream
#

make sure you do defer k.deinit() immediately after you initialise it

#

otherwise you will leak memory if you encounter an error before you reach the deinit call

thin vapor
#

we can hv two defers?

warped stream
#

you can have however many you want

#

the defer furthest down in the scope will be the first to run

#

and the first in scope will be the last to run

#

very much alike the behaviour RAII objects

charred quartz
warped stream
charred quartz
#

yep, just thought I'd add that in case the question comes up

thin vapor
#

so just so i get this right, i should defer the gpa first in order of the code and then the arraylist so that the array list is freed first and then the alloc is deinit?

charred quartz
#

yes

thin vapor
#

makes sense

charred quartz
#

most of the times you simply defer right after initializing/allocating something

warped stream
#

can think of it like manual RAII

thin vapor
#

in tests its recommended to use the testing allocator right?

warped stream
#

unless you're testing behaviour with a specific allocator, or testing your own allocator implementation, generally yes

thin vapor
#

thanks @warped stream and @charred quartz my code works now 🥂

thin vapor
#

Hey @charred quartz i tried to set a change an index in my array list like

self.hard_forks[index] = ...
``` but it says it doesnt support indexing so is there a method for this?
#

or are we expected to do

self.hard_forks.items[index] = ...
charred quartz
#

of course assuming that index already exists, i.e. items.len > index

thin vapor
#

@warped stream i saw a discord message where u used the older version of @mulWithOverflow where it takes 4 params, do u recall the types of that?

warped stream
#

Now it infers the operand types, and now returns a tuple containing the overflow bool/bool vector, and the result

thin vapor
#

yea so now its just mulwithOverflow(a: anytype, b:anytype)

#

is this how it used to be mulWithOverflow(type: type, a: anytype, b: anytype, result: *anytype)

warped stream
#

More like (comptime T: type, a: T, b: T, result: *T) bool

thin vapor
#

ah ohk thanks

thin vapor
#

how is this a bool

#

is u1 a bool here

little cargo
#

that's weird

#

that makes sense for add with overflow

#

not mul with overflow

thin vapor
little cargo
#

mul overflow can be as big as the input numbers

thin vapor
#

feel like the older return was simpler

warped stream
thin vapor
#

so any idea how we get the bool result from this?

#

should it be @mulWithOverflow(a,b)[0] == @TypeOf(a,b)

warped stream
#

ye

thin vapor
#

gotcha

warped stream
#

it uses PTR

thin vapor
#

?

warped stream
#

peer type resolution

thin vapor
#

which is ptr

#

oh

warped stream
#

pardon the jargon

#

basically when there are two or more expressions in an operation which will evaluate to another value assigned to some location (such as if (cond) a else b, a + b, switch (v) { .foo => a, .bar => b, .baz => c }, etc), peer type resolution is invoked to determine what the "resolved type" between the types of all the expressions will be

thin vapor
#

@warped stream in the older function sig what was the exact return type

warped stream
#

bool

thin vapor
#

trying to modify this legacy code to work again

warped stream
#

'least I believe it was bool

#

might have been u1

thin vapor
# 
const mul_result = @mulWithOverflow(octs[octets_index], 10);
                        if (mul_result[1] == 1) return error.InvalidIpv4;
warped stream
#

seems it was u1 then

thin vapor
#

like just u1 ? not struct?

#

or tuple

warped stream
#

no, originally it returned the result through the last out parameter

#

and returned the overflow state through the return value

#

you can always read the older docs

thin vapor
#

thats what ive been trying to find

warped stream
#

upper left corner

thin vapor
#

is there any way to know which version of zig this code was using

warped stream
#

depends on the code

thin vapor
#

like rust has rustoolchain.toml

warped stream
#

nothin like that

thin vapor
#

k

warped stream
#

well you can see it was last commited 4 years ago

#

might be the version after the one 4 years ago if they were following master