FFI string corrupted? | Deno | Page 1

idle silo Jul 7, 2023, 8:36 PM

#

Is there anything i'm doing wrong here? I'm confused on why it's crashing on setVerisonUE5 in particular. The code without FFI in C++ works fine

import { Dataminer } from "./mod.ts";

Dataminer.withLogs(true);
Dataminer.setOodle("...");

const core = new Dataminer("...");
console.log(core); // printed
core.setVersionUE5(1008); // crashes here?
console.log("set version"); // not printed```
```ts
const DLL_PATH = "...";

const library = Deno.dlopen(DLL_PATH, {
    "dataminer_options_with_oodle": {
        parameters: ["buffer"],
        result: "void"
    },
    "dataminer_with_logging": {
        parameters: ["bool"],
        result: "void"
    },
    "dataminer_construct": {
        parameters: ["buffer"],
        result: "pointer"
    },
    "dataminer_set_version_ue5": {
        parameters: ["pointer", "i32"],
        result: "void"
    }
});

function encode(text: string) {
    return new TextEncoder().encode(text);
}

export class Dataminer {
    pointer: Deno.PointerValue;
    
    constructor(paksDir: string) {
        this.pointer = library.symbols.dataminer_construct(encode(paksDir));
    }
    
    checkPointer() {
        if (!this.pointer) throw new Error("pointer is null");
    }
    
    static setOodle(oodlePath: string) {
        library.symbols.dataminer_options_with_oodle(encode(oodlePath));
    }
    
    static withLogs(shouldLog: boolean) {
        library.symbols.dataminer_with_logging(shouldLog);
    }
    
    setVersionUE5(version: number) {
        this.checkPointer();
        library.symbols.dataminer_set_version_ue5(this.pointer, version);
    }
}```

#

#include "Dataminer/Dataminer.h"

#define CPAKPARSER_API extern "C" __declspec(dllexport)

CPAKPARSER_API void dataminer_options_with_oodle(const char* OodleDllPath)
{
    Dataminer::Options::WithOodleDecompressor(OodleDllPath);
}

CPAKPARSER_API void dataminer_with_logging(bool bEnableLogging)
{
    Dataminer::Options::WithLogging(bEnableLogging);
}

CPAKPARSER_API Dataminer* dataminer_construct(const char* PaksFolderDir)
{
    auto Result = Dataminer(PaksFolderDir);
    return &Result;
}

CPAKPARSER_API void dataminer_set_version_ue5(Dataminer* This, int Version)
{
    This->SetVersionUE5(Version);
}

idle silo Jul 8, 2023, 12:02 AM

#

~~I think it only happens when dealing with numbers? what's up with that~~

idle silo Jul 11, 2023, 11:28 PM

#

I managed to fix it but now my strings end up corrupted?

CPAKPARSER_API bool dataminer_load_type_mappings(Dataminer* This, const char* UsmapFilePath)
{
    return This->LoadTypeMappings(UsmapFilePath);
}```

[CPakParser] ERR Invalid paks directory Content\Paksď`☻```

const library = Deno.dlopen(DLL_PATH, {
    // other FFI exports here
    "dataminer_load_type_mappings": {
        parameters: ["pointer", "buffer"],
        result: "bool"
    }
});

function encode(text: string) {
    return new TextEncoder().encode(text);
}

// function in the class that wraps it
loadTypeMappings(usmapFileDir: string) {
    return library.symbols.dataminer_load_type_mappings(this.pointer, encode(usmapFileDir));
}

#

FFI string corrupted?

#

Got it to work by doing uint32_t UsmapLength, const char* UsmapFilePath but idk if that's the way to go without leaking memory

raven light Jul 12, 2023, 5:43 AM

#

You at least don't seem to be null-terminating your strings. That's going to be an issue since then C++ won't know when the strings end and odd things happen.

#

Giving the length explicitly will then fix that issue since now C++ can rely on the length parameter to stop reading the string and doesn't need to look for a null byte.

idle silo Jul 12, 2023, 8:49 AM

#

raven light You at least don't seem to be null-terminating your strings. That's going to be ...

How do I pass a null terminated string to C++ from Deno?

#

do i just add a null byte at the end myself?

raven light Jul 12, 2023, 8:51 AM

#

Yeah, easiest and probably fastest is to do

new TextEncoder().encode(`${myString}\0`)

idle silo Jul 12, 2023, 8:51 AM

#

alr ty will try

raven light Jul 12, 2023, 8:55 AM

#

If you know you're dealing with ASCII strings then you could also use encodeInto with a buffer you create yourself with its length being one longer than the length of your string. The extra byte is then your null byte. That could lead to a faster call, but it could also be slower since the buffer is initially zeroes which is not needed in the above version.

idle silo Jul 12, 2023, 8:55 AM

#

raven light Yeah, easiest and probably fastest is to do ```ts new TextEncoder().encode(`${m...

seems like this worked

#

i never expected to deal with pointers in javascript lmfao

#

but it's pretty nice

raven light Jul 12, 2023, 9:05 AM

#

Yeah 🙂 FFI pretty much drew me into Deno in the first place, and by god it's been fun to deal with bytes and pointers and whatnot in JS.

It feels so dirty but oh so fun 🙂

#

Just in case you haven't seen this, do check out Denonomicon: https://denonomicon.deno.dev/introduction

I've written up at least some of the internals and intricacies of FFI there.

idle silo Jul 12, 2023, 9:08 AM

#

never heard about the denomicon

#

thank you!

raven light Jul 12, 2023, 9:09 AM

#

Also in YouTube we did a three part podcasty video series on FFI with Andy. It's on Deno's channel with the name "WTF is FFI"

idle silo Jul 12, 2023, 9:23 AM

#

raven light Also in YouTube we did a three part podcasty video series on FFI with Andy. It's...

last question, the "V8 Fast API" in the denomicon is just used internally within Deno and is not usable via the FFI APIs right?

raven light Jul 12, 2023, 9:48 AM

#

Yes. It gets used automatically with all FFI calls except calls that use struct-by-value parameters or return values ({ struct: [...] })

#FFI string corrupted?