error from FixedBufferStream: error.ProcessFdQuotaExceeded | Zig Programming Language | Page 1

tall kernel Mar 3, 2024, 11:53 PM

#

no idea how in the world FixedBufferStream is giving me this error as everything is in memory

error: ProcessFdQuotaExceeded
/home/bulby/Documents/github/sillyplatformer/wasmp_parser/main.zig:176:30: 0x112dfda in parse (build)
                const tres = try self.til.parse(allocator, src);
                             ^
/home/bulby/Documents/github/sillyplatformer/wasmp_parser/main.zig:210:17: 0x10d9de7 in strip_maps (build)
    const res = try many_til_map.parse(alloc, &fbs) orelse return error.BadParse;
                ^
/home/bulby/Documents/github/sillyplatformer/build.zig:41:19: 0x10da0eb in build (build)
    const funny = try wasmp_parser.strip_maps(alloc, wasmp);
                  ^
/home/bulby/zig-0.12.0-dev/lib/std/Build.zig:1885:24: 0x10bf137 in runBuild__anon_8607 (build)
        .ErrorUnion => try build_zig.build(b),
                       ^
/home/bulby/zig-0.12.0-dev/lib/build_runner.zig:319:9: 0x10baee7 in main (build)
        try builder.runBuild(root);
        ^

1st message is at this which calls into
this and like... how in the world did it get ProcessFdQuoteExceeded. It's all in memory???

#

this isn't just an OOM, fba does this if I give it a LOT of memory too

idle vortex Mar 3, 2024, 11:59 PM

#

I imagine you're opening a file somewhere

#

And just not realising you're doing it much more than you wanted

#

But indeed - I'm not seeing anything obvious in that file

round crane Mar 4, 2024, 12:03 AM

#

are you opening a file in response to a catch somewhere, the printed error is the final error not the initial error

#

also why is everyone doing build logic during the configure stage

tall kernel Mar 4, 2024, 12:05 AM

#

what

round crane Mar 4, 2024, 12:06 AM

#

you shouldn't be effectively performing build steps in fn build

tall kernel Mar 4, 2024, 12:06 AM

#

p. sure that bc it's an embedded file it doesn't really matter but meh. this is a hack anyway

round crane Mar 4, 2024, 12:07 AM

#

I mean there's no point in using the build system if you are just going to run a program and ignore the parallelism and caching

idle vortex Mar 4, 2024, 12:08 AM

#

The build function in the build.zig is only executed in order to figure out what needs to be done

tall kernel Mar 4, 2024, 12:08 AM

#

i'll use a tmp dir when I finally fix it ok

idle vortex Mar 4, 2024, 12:08 AM

#

It isn't run in order to do the things that need to be done

tall kernel Mar 4, 2024, 12:09 AM

#

i did try to do this stuff in comptime but it didn't work so I moved it to build.zig. anyways, I think I would like to know about the FD quota thing

round crane Mar 4, 2024, 12:11 AM

#

I mean the build system needs fds too, you should be compiling and running a build tool that performs the logic in a separate process

tall kernel Mar 4, 2024, 12:11 AM

#

hm k

#

the parser SHOULDN'T need fds tho

#

I don't think i explicitly open one

#

i'll try pulling it out with a main tho

round crane Mar 4, 2024, 12:15 AM

#

I'm getting the crash in save_json which definitely opens file descriptors

tall kernel Mar 4, 2024, 12:15 AM

#

what

#

ok I'll pull it out. thanks for bothering to build it

round crane Mar 4, 2024, 12:17 AM

#

oh that's something else

#

I guess idk what compiler version you are using

tall kernel Mar 4, 2024, 12:17 AM

#

it's 0.12.0 i forgot to tag it

round crane Mar 4, 2024, 12:17 AM

#

zig version

tall kernel Mar 4, 2024, 12:18 AM

#

[bulby@bulby-g707 ~]% ~/zig-0.12.0-dev/zig version
0.12.0-dev.2543+9eda6ccef

#

it's a different branch you have to pull down btw

round crane Mar 4, 2024, 12:21 AM

#

oh

#

I definitely missed that

#

I think it's just printing a random error, maybe there's memory corruption

#

(I get a completely unrelated error in the same place)

tall kernel Mar 4, 2024, 12:25 AM

#

ofc it's memory corruption!

#

i'll handle it

round crane Mar 4, 2024, 12:27 AM

#

yeah I made a random change and it succeeded

tall kernel Mar 4, 2024, 12:27 AM

#

wha

#

what change

#

i mean

#

it prob is garbage data regardless

round crane Mar 4, 2024, 12:40 AM

#

yeah the error appears out of thin air during the indirect call return

#

let's see who's writing to it

tall kernel Mar 4, 2024, 12:41 AM

#

i've gotten segfaults from the error print out thing

round crane Mar 4, 2024, 12:42 AM

#

oh wait it's not being written at all lol

tall kernel Mar 4, 2024, 12:42 AM

#

oh

round crane Mar 4, 2024, 12:42 AM

#

oh I'm blind

tall kernel Mar 4, 2024, 12:45 AM

#

is it @fieldParentPtr?

round crane Mar 4, 2024, 12:47 AM

#

oh wow I just printed out a large chunk of memory

tall kernel Mar 4, 2024, 12:48 AM

#

omg I must have really borked something when writing parser combinators 😭

round crane Mar 4, 2024, 12:49 AM

#

oh yeah you say init().parser not &init().parser, @fieldParentPtr requires the original pointer not a pointer to a copy

tall kernel Mar 4, 2024, 12:49 AM

#

omg

round crane Mar 4, 2024, 12:49 AM

#

err that still wouldn't work, just don't say .parser

tall kernel Mar 4, 2024, 12:50 AM

#

interesting memory corruption from that

#

v. cursed as well

round crane Mar 4, 2024, 12:50 AM

#

@@ -192,18 +195,18 @@ pub fn strip_maps(alloc: Allocator, input: []const u8) ![]u8 {
     var map_end_lit = Literal(FbaType).init("-- </MAP>");
     var map7_lit = Literal(FbaType).init("-- <MAP7>");
     var map7_end_lit = Literal(FbaType).init("-- </MAP7>");
-    var many_til_map = ManyCharTillVoid.init(&anychar, &map_lit.parser).parser;
-    var skip_map_end = ManyCharTillVoid.init(&anychar, &map_end_lit.parser).parser;
+    var many_til_map = ManyCharTillVoid.init(&anychar, &map_lit.parser);
+    var skip_map_end = ManyCharTillVoid.init(&anychar, &map_end_lit.parser);
     _ = &skip_map_end;
-    var many_til_map7 = ManyCharTillVoid.init(&anychar, &map7_lit.parser).parser;
+    var many_til_map7 = ManyCharTillVoid.init(&anychar, &map7_lit.parser);
     _ = &many_til_map7;
-    var skip_map7_end = ManyCharTillVoid.init(&anychar, &map7_end_lit.parser).parser;
+    var skip_map7_end = ManyCharTillVoid.init(&anychar, &map7_end_lit.parser);
     _ = &skip_map7_end;
 
     // var final = Sequence(struct { []u8, []u8, []u8, []u8 }, FbaType).init(.{ &many_til_map, &skip_map_end, &many_til_map7, &skip_map7_end });
 
     // const res = (try final.parser.parse(alloc, &fbs)) orelse return error.BadParse;
-    const res = try many_til_map.parse(alloc, &fbs) orelse return error.BadParse;
+    const res = try many_til_map.parser.parse(alloc, &fbs) orelse return error.BadParse;
     // const rest = try fbs.reader().readAllAlloc(alloc, 65565);
     // const out = try std.mem.join(alloc, "", &.{ res[0], res[2] });

tall kernel Mar 4, 2024, 12:51 AM

#

that is something!

round crane Mar 4, 2024, 12:51 AM

#

before you were loading the parser field from a temporary and storing a copy on the stack

#

and then trying to get back to a pointer to that temporary starting at a pointer to the copy on the stack

tall kernel Mar 4, 2024, 12:53 AM

#

that

#

wow

#

thanks ig, it builds now

idle vortex Mar 4, 2024, 1:20 AM

#

Ah yes - that's also why std.mem.Allocator -- which used to use the @fieldParentPtr approach -- also moved away from it to use what it does now.

#

Or at least that was part of the reason

#

Very easy to make that mistake and then end up with nonsense

round crane Mar 4, 2024, 1:23 AM

#

I'm not aware of accessing allocator fields directly being an issue, but the main reason for that change was to help llvm devirtualize allocators

idle vortex Mar 4, 2024, 1:24 AM

#

round crane I'm not aware of accessing allocator fields directly being an issue, but the mai...

I saw a nontrivial number of people copy the allocator field out, just as the parser field was copied here, and then ask for help about it

round crane Mar 4, 2024, 1:25 AM

#

interesting, probably before my time anyway

idle vortex Mar 4, 2024, 1:25 AM

#

Might've been

#

I have been here a while 🤣

#error from FixedBufferStream: error.ProcessFdQuotaExceeded