#Segmentation fault when parsing json

1 messages · Page 1 of 1 (latest)

river kindle
#

This only happens when my code is run in debug mode

I am trying to parse a json file that is about 56kb into a data structure. however it always segmentation faults in debug mode.

I think it has something to do with stack overflows, however I am not sure what I can do to fix this

pub fn readLevel(a: std.mem.Allocator, save_id: []const u8, level_id: []const u8) !level.Level {
    const path = try getLevelPath(a, save_id, level_id);
    const max_bytes = 10000000; //maximum bytes in a level save file
    const string = try std.fs.cwd().readFileAlloc(a, path, max_bytes);
    if (!try std.json.validate(a, string)) {
        return error.invalid_json;
    }

    //this line segfaults
    return try std.json.parseFromSliceLeaky(level.Level, a, string, .{ .allocate = .alloc_always });
}

The debugger backtrace is included in the attached file

📎 message.txt
#

The code works fine in release compilation mode

#

This is really a major problem for me because I counted on being able to easily convert my Level data structure to and from json in my entire program

versed rose
#

what allocator are you passing to that function?

#

and whats your zig version?

river kindle
#

general purpose allocator wrapped in an arena

#

0.11.0

versed rose
#

have you tried running with latest master instead of 0.11.0? I have no idea what's causing this tbh

river kindle
tulip steeple
#

do you have tests for this function? I wonder if running it in a test with the test allocator would surface more information 🤔

tulip steeple
#

alsoalso, it looks like there's an alternate function parseFromSlice which is less mysterious in how it's deallocated (the result has a deinit() method) - https://ziglang.org/documentation/0.11.0/std/#A;std:json.parseFromSlice

so theoretically you should be able to:

  1. use parseFromSlice instead,
  2. call deinit on the return value of readLevel elsewhere in your program, AND
  3. use a general purpose allocator (GPA), in place of your GPA wrapped in an arena allocator

any reason not to do ^that?

river kindle
river kindle
#

I changed it to the leaky version see if that would fix the issue

#

It did not

#

I've also tried using a normal GPA, instead of an arena and that didn't fix it

#

I ended up having to change some of the fields in level.Level to pointers. That seemed to stop the stack overflow during parsing for some reason

#

I guess the combination of a large stack allocated struct, the recursion in json parsing, and the extra embedded debug information in debug mode will overflow the stack somehow

#

Or at least causes stack_probe.zig to segfault

river kindle
#

And I haven't really learned how to add tests for the whole project yet in build.zig

#

Some of the docs for that sort of thing aren't really great rn

tulip steeple
#

hmm. well this isn't something I've seen off the top of my head, and since I don't have a full code/data example I could run and experiment with, all I can do is guess at the issue

#

can you try to make a minimal reproducible example? something like what SO describes here - https://stackoverflow.com/help/minimal-reproducible-example
in your case, I imagine it would comprise 1) a single code file w/ minimal features, 2) an example json file, and 3) a build command, like zig run example.json -O ReleaseSafe. platform details like RAM size and OS probably wouldn't hurt either

Stack Overflow
How to create a Minimal, Reproducible Example - Help Center

Stack Overflow | The World’s Largest Online Community for Developers

river kindle
#

stack_probe.zig causes a segmentation fault when the stack overflows

tulip steeple
river kindle
#

the first issue may be unrelated

#

I just remembered seeing that earlier today

#

and there being a segfault from the same bit of code

tulip steeple
#

Gotcha, makes sense

#

Sounds like “no dice” ☹️ sorry friend

river kindle
#

seems like a language issue

#

however, changing some fields of the struct to pointers seemed to fix it for now

#

thanks!

tulip steeple
river kindle
#

Non-segmentation fault version

pub const Level = struct {
    name: []const u8,
    entities: *entity.EntityState,
    map: *map.MapState,
    exits: []const Exit,
    player_id: usize,
};

Original

pub const Level = struct {
    name: []const u8,
    entities: entity.EntityState,
    map: map.MapState,
    exits: []const Exit,
    player_id: usize,
};