Export llvm-c to wasm using zig? | Zig Programming Language | Page 1

peak gulch Feb 16, 2023, 8:36 PM

#

Hey! Let's say I want to export every (almost) llvm-c function inside a web assembly module using zig. Does anyone have any advice on how someone might do this?

frank pasture Feb 16, 2023, 8:52 PM

#

I can't be of much help on the JS/wasm side yet, but in case you haven't seen it, the Zig code base has a pretty convenient handwritten Zig layer on top of llvm-c. I copied this approach in my latest project and it's much nicer than using LLVM via @cImport. https://github.com/ziglang/zig/blob/master/src/codegen/llvm/bindings.zig

Not sure if it will help you with the wasm side but thought I would share that hidden gem

peak gulch Feb 16, 2023, 8:52 PM

#

WOW

#

Okay

frank pasture Feb 16, 2023, 8:53 PM

#

It's also a nice showcase of an interesting use case for opaque as a namespace

peak gulch Feb 16, 2023, 8:55 PM

#

I'm currently in the process of building llvm from source and lld

frank pasture Feb 16, 2023, 9:02 PM

#

Cool, good luck — if this is your first time, make sure you follow the steps from option A in the wiki as closely as you can: https://github.com/ziglang/zig/wiki/Building-Zig-From-Source#option-a-use-your-system-installed-build-tools

Don't use option B!

peak gulch Feb 16, 2023, 9:02 PM

#

got it

peak gulch Feb 17, 2023, 6:10 PM

#

@frank pasture Okay I got zig installed today. Can I ask for some help?

frank pasture Feb 17, 2023, 7:27 PM

#

Go for it 🙂

#

Always ok to just ask and someone will come around to answer asynchronously

peak gulch Feb 17, 2023, 8:44 PM

#

Okay. How can I iterate over all the header file contents and export each function?

#

(and enum)

#

Do I have to write a layer of functions over the @cImports?

#

I'm sure it will involve @typeInfo

#

Better. Should I just copy and paste the codegen for llvm?

#

https://ziglang.org/documentation/master/#export

#

I assume I need to use this?

frank pasture Feb 17, 2023, 9:07 PM

#

There are a few different directions you could take here I think. @cImport is one way, and what this does is automatically call zig translate-c on the headers (puts it in zig-cache) and declare a bunch of extern fn s etc. Then you could use pub usingnamespace to make those symbols all top-level.

You could also just call zig translate-c directly, then hand-edit the resulting file. This gives you a chance to improve the API, especially turning [*c]T into [*]T, ?*T etc.

frank pasture Feb 17, 2023, 9:09 PM

#

peak gulch Okay. How can I iterate over all the header file contents and export each functi...

But TLDR, this is where pub usingnamespace shines

peak gulch Feb 17, 2023, 9:11 PM

#

Can you show me an example of what the imports would look like?

I think I also might like an example of what each function would look like.

I'm a zig beginner.

#

I apologize for need such granular help, but I plan on compiling all these functions to web assembly and publishing it.

#

Oh wait!

#

pub usingnamespace @cImport({
    @cInclude("epoxy/gl.h");
    @cInclude("GLFW/glfw3.h");
    @cDefine("STBI_ONLY_PNG", "");
    @cDefine("STBI_NO_STDIO", "");
    @cInclude("stb_image.h");
});

frank pasture Feb 17, 2023, 9:15 PM

#

No problem, I might be unavailable for the next hour but then happy to jump back in

peak gulch Feb 17, 2023, 9:15 PM

#

Thank you so much

peak gulch Feb 17, 2023, 9:46 PM

#

src/main.zig:1:20: error: C import failed
pub usingnamespace @cImport({
                   ^~~~~~~~

#

Looks like the cimport failed

frank pasture Feb 17, 2023, 10:02 PM

#

Ok, I'm back. Is there more error information about the failed cImport?

frank pasture Feb 17, 2023, 10:03 PM

#

peak gulch ```zig pub usingnamespace @cImport({ @cInclude("epoxy/gl.h"); @cInclude(...

This looks generally correct, have you set your include paths in build.zig? If you're using the build system, that is

peak gulch Feb 17, 2023, 10:06 PM

#

Thank you for the quick responses. I have all the header files in my /usr/include/llvm-c-15/llvm-c/ directory

#

I have no idea what I'm doing lol

frank pasture Feb 17, 2023, 10:08 PM

#

You'll get there. So when you include a C header, zig needs to know where to find them. Same as if you are compiling a C program with gcc/clang. By default all of these will know to look relative to the C (or Zig) file that does the include. And they usually have some system-wide defaults like /usr/include. But LLVM is at a special path as you can see.

peak gulch Feb 17, 2023, 10:08 PM

#

Okay. I can use the -I flag

#

Is it better to configure the builder?

frank pasture Feb 17, 2023, 10:09 PM

#

So for example in my build.zig, I've added a special case for Homebrew-installed LLVM:

    if (builtin.os.tag == .macos) {
        exe.addIncludePath("/usr/local/opt/llvm/include");
        exe.addLibraryPath("/usr/local/opt/llvm/lib");
    }

peak gulch Feb 17, 2023, 10:09 PM

#

b.addIncludePath()?

frank pasture Feb 17, 2023, 10:10 PM

#

peak gulch Okay. I can use the -I flag

The -I flag is definitely what you'd use if invoking zig build-exe directly, but if you use a build.zig, you can use the functions I pasted above, which internally is translated to those -I / -L flags

frank pasture Feb 17, 2023, 10:10 PM

#

peak gulch `b.addIncludePath()`?

Yep, that adds another search path for C headers

peak gulch Feb 17, 2023, 10:10 PM

#

Perfect! That's exactly what I need

frank pasture Feb 17, 2023, 10:10 PM

#

an "include path" == a search path for headers

peak gulch Feb 17, 2023, 10:11 PM

#

I've had a massive crash course in nearly 4 programming languages this week outside of my job, so it's been a nightmare

#

You've been an awesome help.

frank pasture Feb 17, 2023, 10:11 PM

#

Holy crap, that's a lot of stuff to learn in a week

#

Glad to help! Build tool stuff like this is hard in its own way, separate from languages and programming, heh

peak gulch Feb 17, 2023, 10:15 PM

#

I added:

 b.addIncludePath("/usr/include/llvm-c-15/");

#

and now everything works

#

it compiles at least

#

Here's my zig command

#

zig build-obj \
  -O ReleaseSmall \
  -target wasm32-wasi \
  --library c \
  -freference-trace \
  -I /usr/include/llvm-c-15/ \
  src/main.zig
wasm-ld main.o -o llvm.wasm -O2 --no-entry --allow-undefined

#

It looks like the wasm file is empty

#

pub usingnamespace @cImport({
  @cInclude("llvm-c/Analysis.h");

So this is what I'm exporting ... just Analysis.h

Will this result in exports?

#

(module
  (memory (;0;) 2)
  (global (;0;) (mut i32) (i32.const 66560))
  (export "memory" (memory 0)))

frank pasture Feb 17, 2023, 10:18 PM

#

Yes, it should. Is the pub usingnamespace part in main.zig?

peak gulch Feb 17, 2023, 10:19 PM

#

Yeah it's outputting nothing 😦

frank pasture Feb 17, 2023, 10:19 PM

#

I have no idea how wasm-ld works, in case that's a factor

#

Let me see here

peak gulch Feb 17, 2023, 10:20 PM

#

should I be using a .so file instead?

frank pasture Feb 17, 2023, 10:21 PM

#

Is llvm.wasm all of LLVM built to wasm?

peak gulch Feb 17, 2023, 10:21 PM

#

Heh. I'm just trying to expose the C api

#

those are c bindings afaicr

frank pasture Feb 17, 2023, 10:22 PM

#

in llvm.wasm?

peak gulch Feb 17, 2023, 10:22 PM

#

I'm trying to do as you describe

#

but the wasm file contains nothing

frank pasture Feb 17, 2023, 10:22 PM

#

OHH I get it, llvm.wasm is the out file name

peak gulch Feb 17, 2023, 10:23 PM

#

Yeah sorry I didn't understand your question

frank pasture Feb 17, 2023, 10:23 PM

#

I think what you want to do is statically link against the LLVM libs

#

(I was being unclear)

peak gulch Feb 17, 2023, 10:23 PM

#

I think you're right

#

Should I build llvm first by hand?

frank pasture Feb 17, 2023, 10:26 PM

#

Bear with me as I wrap my head around how this would work. I think you need to build LLVM with llvm-c, from source, with wasm32 as the target.

peak gulch Feb 17, 2023, 10:26 PM

#

I can't count the number of times I had to do this lol

pliant peak Feb 17, 2023, 10:26 PM

#

You want zig build-lib -dynamic ...

#

.wasm files are .sos basically.

#

I'm also fairly certain that Zig does the WASM linking already, so you don't need to do that part separately.

#

If you want something that you can run with wastime or the like, then zig build-exe might actually just work for you.

peak gulch Feb 17, 2023, 10:27 PM

#

@pliant peak

zig build-obj \
  -dynamic\
  -O ReleaseSmall \
  -target wasm32-wasi \
  --library c \
  -freference-trace \
  -I /usr/include/llvm-c-15/ \
  src/main.zig

pliant peak Feb 17, 2023, 10:27 PM

#

-dynamic only works with build-lib AFAIK.

peak gulch Feb 17, 2023, 10:28 PM

#

I don't need a wasmtime build. I forgot to change the command

#

OH it's working now

frank pasture Feb 17, 2023, 10:29 PM

#

Are you planning to target web? Downloading a full llvm.wasm from a website might take a while 🙃

#

Maybe not as bad if it only has the wasm target backend built in

peak gulch Feb 17, 2023, 10:29 PM

#

No I'm trying to build a CLI in node.js

frank pasture Feb 17, 2023, 10:30 PM

#

I see

peak gulch Feb 17, 2023, 10:30 PM

#

@pliant peak it looks like it built something big, but it didn't export any of the llvm functions

pliant peak Feb 17, 2023, 10:30 PM

#

peak gulch <@201094465815969794> it looks like it built something big, but it didn't export...

If you want the functions to be exported, then you must ask them to be, I think.

#

ie. export fn.

#

(Not to be confused with extern fn.)

peak gulch Feb 17, 2023, 10:31 PM

#

Can I use @export() with a loop over the type?

pliant peak Feb 17, 2023, 10:31 PM

#

I believe so.

peak gulch Feb 17, 2023, 10:31 PM

#

May I have some help with that?

pliant peak Feb 17, 2023, 10:31 PM

#

Sure.

peak gulch Feb 17, 2023, 10:31 PM

#

I just started my zig journey today

pliant peak Feb 17, 2023, 10:31 PM

#

Jumping in at the ocean-end, eh. 😄

peak gulch Feb 17, 2023, 10:31 PM

#

😂

pliant peak Feb 17, 2023, 10:31 PM

#

I admire your verasity.

peak gulch Feb 17, 2023, 10:32 PM

#

Should I use @Type or @typeInfo

pliant peak Feb 17, 2023, 10:32 PM

#

Obviously you have balls the size of skyscrapers.

pliant peak Feb 17, 2023, 10:32 PM

#

peak gulch Should I use @Type or @typeInfo

If you want to iterate over the fields of a type, then the latter.

#

First though, I should see the code you're using.

peak gulch Feb 17, 2023, 10:32 PM

#

Okay what would the loop look like?

pliant peak Feb 17, 2023, 10:33 PM

#

peak gulch Okay what would the loop look like?

Iterating over fields:

inline for (std.meta.fields(T)) |field| {

}

peak gulch Feb 17, 2023, 10:33 PM

#

pub usingnamespace @cImport({
  @cInclude("llvm-c/Analysis.h");
  @cInclude("llvm-c/Core.h");
  @cInclude("llvm-c/Disassembler.h");
  @cInclude("llvm-c/ExecutionEngine.h");
  @cInclude("llvm-c/LLJIT.h");
  @cInclude("llvm-c/OrcEE.h");
  @cInclude("llvm-c/TargetMachine.h");
  @cInclude("llvm-c/lto.h");
  @cInclude("llvm-c/BitReader.h");
  @cInclude("llvm-c/DataTypes.h");
  @cInclude("llvm-c/DisassemblerTypes.h");
  @cInclude("llvm-c/ExternC.h");
  @cInclude("llvm-c/Linker.h");
  @cInclude("llvm-c/Remarks.h");
  @cInclude("llvm-c/BitWriter.h");
  @cInclude("llvm-c/DebugInfo.h");
  @cInclude("llvm-c/Error.h");
  @cInclude("llvm-c/IRReader.h");
  @cInclude("llvm-c/Object.h");
  @cInclude("llvm-c/Support.h");
  @cInclude("llvm-c/Types.h");
  @cInclude("llvm-c/Comdat.h");
  @cInclude("llvm-c/Deprecated.h");
  @cInclude("llvm-c/ErrorHandling.h");
  @cInclude("llvm-c/Initialization.h");
  @cInclude("llvm-c/Orc.h");
  @cInclude("llvm-c/Target.h");
  @cInclude("llvm-c/blake3.h");
});

#

this is what I have so far

pliant peak Feb 17, 2023, 10:33 PM

#

AH. 🤣

peak gulch Feb 17, 2023, 10:33 PM

#

🙃

pliant peak Feb 17, 2023, 10:34 PM

#

So.

#

Here's the crash course.

#

cImport generates extern declarations. (Think forward declarations in C.)

peak gulch Feb 17, 2023, 10:34 PM

#

Oh that makes sense

pliant peak Feb 17, 2023, 10:34 PM

#

These are just how you declare the existence of things that aren't in your compilation unit.

#

It also does its best to declare Zig types that have the same layout as every C type in the headers.

#

These externs only require that the other compilation units are linked in the end.

#

If those symbols are exported from C -- i.e with extern "C" -- then they'll also be available IIRC from the final WASM -- at least if it works like an SO.
(If it's C code, and not C++, then extern "C" is unnecessary, because everything is exported by default.)

#

At least, I think that's how it works - been a while since I juggled this kinda things 🤣

peak gulch Feb 17, 2023, 10:37 PM

#

Thank you so much for the info.

#

So I need to export each function?

#

or rather, I can @export each one.

pliant peak Feb 17, 2023, 10:37 PM

#

So, it depends. 😄

#

export means, "I want this thing to be publically exported from the Zig compilation unit, and ultimately, the final output - whatever that is."

peak gulch Feb 17, 2023, 10:38 PM

#

Yes of course.

#

I think I understand that.

pliant peak Feb 17, 2023, 10:38 PM

#

So, if you make a SO, then these will be things that can be accessed by anything that links that SO.

#

Likewise, I would imagine with .wasm, since they are the WASM equiv to an SO.

#

I'm not sure if you actually need to do this or not in this case, but if you do, then you do. 😄

peak gulch Feb 17, 2023, 10:39 PM

#

Okay.

pliant peak Feb 17, 2023, 10:39 PM

#

If you're not seeing the symbols being exported, then I'm guessing that you do.

peak gulch Feb 17, 2023, 10:39 PM

#

How can I loop over all the functions in a cImport

#

inline for (std.meta.fields(llvm)) |field| {

}

pliant peak Feb 17, 2023, 10:40 PM

#

Each @cImport, just like @import, will return a struct that is full of declarations.

#

You'll want const c = @cImport(...);.

peak gulch Feb 17, 2023, 10:41 PM

#

Yep llvm in my case

pliant peak Feb 17, 2023, 10:41 PM

#

And then inline for (std.meta.declarations(c)) |decl| I think.

#

Fields are a different thing.

peak gulch Feb 17, 2023, 10:42 PM

#

Yep! I was copying what @frank pasture did. But yeah declarations seems correct

#

Next I need to @export

frank pasture Feb 17, 2023, 10:42 PM

#

Silly question but what will the new Zig code add over just using LLVM built for wasm?

peak gulch Feb 17, 2023, 10:42 PM

#

The example on the website shows:

comptime {
    @export(internalName, .{ .name = "foo", .linkage = .Strong });
}

frank pasture Feb 17, 2023, 10:43 PM

#

Since llvm-c already exports the symbols and defines the code

pliant peak Feb 17, 2023, 10:43 PM

#

Then, you can do something like:

@export(@field(c, decl.name), .{ .name = decl.name, .linkage = .Strong });

pliant peak Feb 17, 2023, 10:44 PM

#

frank pasture Silly question but what will the new Zig code add over just using LLVM built for...

The ability to access these LLVM functions from outside the WASM module.

peak gulch Feb 17, 2023, 10:44 PM

#

Thank you so much! I'll give it a shot

#

const std = @import("std");
const llvm = @cImport({
  @cInclude("llvm-c/Analysis.h");
  @cInclude("llvm-c/Core.h");
  @cInclude("llvm-c/Disassembler.h");
  @cInclude("llvm-c/ExecutionEngine.h");
  @cInclude("llvm-c/LLJIT.h");
  @cInclude("llvm-c/OrcEE.h");
  @cInclude("llvm-c/TargetMachine.h");
  @cInclude("llvm-c/lto.h");
  @cInclude("llvm-c/BitReader.h");
  @cInclude("llvm-c/DataTypes.h");
  @cInclude("llvm-c/DisassemblerTypes.h");
  @cInclude("llvm-c/ExternC.h");
  @cInclude("llvm-c/Linker.h");
  @cInclude("llvm-c/Remarks.h");
  @cInclude("llvm-c/BitWriter.h");
  @cInclude("llvm-c/DebugInfo.h");
  @cInclude("llvm-c/Error.h");
  @cInclude("llvm-c/IRReader.h");
  @cInclude("llvm-c/Object.h");
  @cInclude("llvm-c/Support.h");
  @cInclude("llvm-c/Types.h");
  @cInclude("llvm-c/Comdat.h");
  @cInclude("llvm-c/Deprecated.h");
  @cInclude("llvm-c/ErrorHandling.h");
  @cInclude("llvm-c/Initialization.h");
  @cInclude("llvm-c/Orc.h");
  @cInclude("llvm-c/Target.h");
  @cInclude("llvm-c/blake3.h");
});

comptime {
    inline for (std.meta.declarations(llvm)) |decl| {
        @export(@field(c, decl.name), .{ .name = decl.name, .linkage = .Strong });
    }
}

#

Looks like

src/main.zig:35:17: error: symbol to export must identify a declaration
        @export(@field(c, decl.name), .{ .name = decl.name, .linkage = .Strong });
                ^~~~~~~~~~~~~~~~~~~~

frank pasture Feb 17, 2023, 10:46 PM

#

pliant peak The ability to access these LLVM functions from outside the WASM module.

I see, I think I'm still trying to wrap my head around the end product here

peak gulch Feb 17, 2023, 10:46 PM

#

The end product is LLVM functions that node.js can call

#

I can do something like rust's inkwell

#

@pliant peak almost there. Just need to fix the identify this here

pliant peak Feb 17, 2023, 10:48 PM

#

peak gulch Looks like ``` src/main.zig:35:17: error: symbol to export must identify a dec...

Ah yes. Seems like you want something like this instead:

inline for (std.meta.declarations(@This())) |decl| {
    const d = @field(llvm, decl.name);
    if (@typeInfo(@TypeOf(d)) == .Fn) {
        @export(d, .{ .name = decl.name, .linkage = .Strong });
    }
}

peak gulch Feb 17, 2023, 10:48 PM

#

@this?

pliant peak Feb 17, 2023, 10:49 PM

#

llvm in your case 😄

peak gulch Feb 17, 2023, 10:49 PM

#

shoudl @This be llvm?

#

Oh

#

I might like to export the enums too

pliant peak Feb 17, 2023, 10:49 PM

#

Remember I said before how @cImport returns a struct full of decls?
Well, that's because a file is actually just a struct. 😄

peak gulch Feb 17, 2023, 10:49 PM

#

not sure how that works

pliant peak Feb 17, 2023, 10:49 PM

#

@This() just returns the current container - whatever struct etc that is.

#

Which is what I was using for testing what you needed on my side 😄

peak gulch Feb 17, 2023, 10:50 PM

#

Oh okay

#

See that makes sense

pliant peak Feb 17, 2023, 10:50 PM

#

pliant peak `@This()` just returns the current container - whatever struct etc that is.

The current container type that is.

pliant peak Feb 17, 2023, 10:50 PM

#

peak gulch See that makes sense

Maximum brain. 🧠

pliant peak Feb 17, 2023, 10:51 PM

#

peak gulch I might like to export the enums too

Well, you cannot export a type, because.. well.. types don't exist at runtime 😄

peak gulch Feb 17, 2023, 10:51 PM

#

src/main.zig:37:13: error: unable to export type 'fn(u16) callconv(.Inline) u16'
            @export(d, .{ .name = decl.name, .linkage = .Strong });
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/main.zig:37:13: note: inline function cannot be extern

#

Now this is really cool

pliant peak Feb 17, 2023, 10:52 PM

#

Remember how I said that importing just generates Zig types that have the same layout as the C types? That's why. Because the C types are not accessible across the linking barrier.

peak gulch Feb 17, 2023, 10:52 PM

#

okay how can I identify inline functions?

pliant peak Feb 17, 2023, 10:53 PM

#

peak gulch ``` src/main.zig:37:13: error: unable to export type 'fn(u16) callconv(.Inline) ...

That's probably a macro that cImport translated.
You can work around it for now with typeinfo, same as I was doing in my example for functions as well.

#

So:

#

inline for (std.meta.declarations(@This())) |decl| {
    const d = @field(llvm, decl.name);
    const ti = @typeInfo(@TypeOf(d));
    if (ti == .Fn) {
        switch (ti.Fn.calling_convention) {
            // NOTE: may only need to check .C, but still.
            .C, .Stdcall, .SysV, .Win64 => {
                @export(d, .{ .name = decl.name, .linkage = .Strong });
            },
        }
    }
}

peak gulch Feb 17, 2023, 10:56 PM

#

the switch needs a default case...

pliant peak Feb 17, 2023, 10:56 PM

#

else => {},

#

Serves me right for not trying to run it 🤣

peak gulch Feb 17, 2023, 10:57 PM

#

src/main.zig:41:29: error: TODO implement exporting arbitrary Value objects
                    @export(d, .{ .name = decl.name, .linkage = .Strong });

#

🤦‍♂️

#

You weren't kidding. this is right off the deep end LOL

pliant peak Feb 17, 2023, 10:59 PM

#

I'm unsure exactly what type of value it is complaining about, but you basically just have to only export things with types that it can do for now 😄

#

This means adding more filters until it works 😄

#

Here is what the typeinfo for a function looks like:

    pub const Fn = struct {
        calling_convention: CallingConvention,
        alignment: comptime_int,
        is_generic: bool,
        is_var_args: bool,
        /// TODO change the language spec to make this not optional.
        return_type: ?type,
        params: []const Param,

        /// This data structure is used by the Zig language code generation and
        /// therefore must be kept in sync with the compiler implementation.
        pub const Param = struct {
            is_generic: bool,
            is_noalias: bool,
            type: ?type,
        };
    };

peak gulch Feb 17, 2023, 10:59 PM

#

yeah I just have no idea what to filter, what properties to inspect at all

pliant peak Feb 17, 2023, 11:00 PM

#

I would probably skip anything where is_generic is true, or is_var_args is true. 😄

#

Or where return_type == null.

#

Likewise for any of the parameters, probably.

#

Something like this may work for you:

outer: inline for (...) |decl| {
    const d = @field(llvm, decl.name);
    const ti = @typeInfo(@TypeOf(d));
    if (ti != .Fn) continue;

    const fi = ti.Fn;
    if (fi.return_type == null) continue;
    if (fi.is_generic) continue;
    if (fi.is_var_args) continue;
    for (fi.params) |param| {
        if (param.is_generic) continue :outer;
        if (param.type == null) continue :outer;
    }

    @export(...);
}

#

There used to be a bug with continue/break in an inline for loop. If that's still present, then you'll have to just use nested ifs instead.

peak gulch Feb 17, 2023, 11:04 PM

#

oh my god

vivid hemlock Feb 17, 2023, 11:04 PM

#

control flow in inline for works fine now afaik

peak gulch Feb 17, 2023, 11:04 PM

#

and

#

not &&

pliant peak Feb 17, 2023, 11:04 PM

#

vivid hemlock control flow in `inline for` works fine now afaik

Hope so 🤣

pliant peak Feb 17, 2023, 11:04 PM

#

peak gulch not `&&`

That's correct. 😛

vivid hemlock Feb 17, 2023, 11:04 PM

#

peak gulch `and`

oh yeah keyword logical ops feel weird at first but there's logic behind it

pliant peak Feb 17, 2023, 11:04 PM

#

There's or too, if you need it.

vivid hemlock Feb 17, 2023, 11:04 PM

#

the idea is that in zig, only keywords can do control flow

#

and and, or are short-circuiting, thus they are control flow

peak gulch Feb 17, 2023, 11:05 PM

#

okay

vivid hemlock Feb 17, 2023, 11:05 PM

#

takes a bit to get used to but rest assured you'll start accidentally doing it in c++ eventually :):)

peak gulch Feb 17, 2023, 11:05 PM

#


comptime {
    inline for (std.meta.declarations(llvm)) |decl| {
        const d = @field(llvm, decl.name);
        const ti = @typeInfo(@TypeOf(d));
        if (ti == .Fn) {
            switch (ti.Fn.calling_convention) {
                // NOTE: may only need to check .C, but still.
                .C, .Stdcall, .SysV, .Win64 => {
                    if (!ti.Fn.is_generic and !ti.Fn.is_var_args and ti.Fn.return_type != null) {
                        @export(d, .{ .name = decl.name, .linkage = .Strong });
                    }
                },
                else => {}
            }
        }
    }
}

pliant peak Feb 17, 2023, 11:06 PM

#

A good start.

peak gulch Feb 17, 2023, 11:06 PM

#

src/main.zig:42:33: error: TODO implement exporting arbitrary Value objects
                        @export(d, .{ .name = decl.name, .linkage = .Strong });

pliant peak Feb 17, 2023, 11:06 PM

#

You're not doing that for loop that I had in my example though 😉

vivid hemlock Feb 17, 2023, 11:06 PM

#

you can do an @compileLog(d) above that line to try and figure out what it's exporting

pliant peak Feb 17, 2023, 11:06 PM

#

Ah yes! The compile-time printf!

#

Well - ish.

#

More like println.

vivid hemlock Feb 17, 2023, 11:07 PM

#

@compileLog is weird but handy

#

it does its darn best to output any value in a comprehensible format, which is very handy! what's less handy is that it doesn't print computed strings as strings so you can't use std.fmt.comptimePrint with it

peak gulch Feb 17, 2023, 11:08 PM

#


comptime {
    inline for (std.meta.declarations(llvm)) |decl| {
        const d = @field(llvm, decl.name);
        const ti = @typeInfo(@TypeOf(d));
        if (ti != .Fn) continue;

        const fi = ti.Fn;
        if (fi.return_type == null) continue;
        if (fi.is_generic) continue;
        if (fi.is_var_args) continue;
        for (fi.params) |param| {
            if (param.is_generic) continue :outer;
            if (param.type == null) continue :outer;
        }
        @compileLog(d);
        @export(d, .{ .name = decl.name, .linkage = .Strong });
    }
}

vivid hemlock Feb 17, 2023, 11:08 PM

#

but yeah here it'll hopefully be able to give you some insight as to where it's breaking

pliant peak Feb 17, 2023, 11:09 PM

#

peak gulch ```rs comptime { inline for (std.meta.declarations(llvm)) |decl| { ...

You're missing the outer: inline for, but yeah - otherwise seems reasonable.

peak gulch Feb 17, 2023, 11:09 PM

#

Oh yeah of course

#

So apparently the struct doesn't have a params field?

#

pub const Fn = struct {
        calling_convention: CallingConvention,
        alignment: comptime_int,
        is_generic: bool,
        is_var_args: bool,
        /// TODO change the language spec to make this not optional.
        return_type: ?type,
        args: []const Param,

        /// This data structure is used by the Zig language code generation and
        /// therefore must be kept in sync with the compiler implementation.
        pub const Param = struct {
            is_generic: bool,
            is_noalias: bool,
            arg_type: ?type,
        }

#

args

#

lol

vivid hemlock Feb 17, 2023, 11:11 PM

#

ah yeah

pliant peak Feb 17, 2023, 11:11 PM

#

Huh. Mine says params 😄

#

Maybe that was changed recently.

vivid hemlock Feb 17, 2023, 11:11 PM

#

mine also says params actually

pliant peak Feb 17, 2023, 11:12 PM

#

I'm on 0.11.0-dev.1575+289e8fab7, ftr.

vivid hemlock Feb 17, 2023, 11:12 PM

#

just updated and still does

peak gulch Feb 17, 2023, 11:12 PM

#

Compile Log Output:
@as(fn(u16) callconv(.Inline) u16, (function '__builtin_bswap16'))

#

heh

#

I should probably filter by name

#

if the name begins with LLVM chances are I should export it

pliant peak Feb 17, 2023, 11:13 PM

#

🤣

#

I suppose --- considering what you're trying to do ---- that would make sense 😄

vivid hemlock Feb 17, 2023, 11:13 PM

#

vivid hemlock just updated and still does

@peak gulch It seems like that was changed in december, so your compiler is a fair bit out of date - might be worth updating
unless you're intentionally using 0.10.1 or something (haven't read backlog)

peak gulch Feb 17, 2023, 11:13 PM

#

I happen to be using the 0.10.1 binary

vivid hemlock Feb 17, 2023, 11:14 PM

#

ah okay

peak gulch Feb 17, 2023, 11:14 PM

#

I haven't built from source because my computer is a potato

vivid hemlock Feb 17, 2023, 11:14 PM

#

if you hit any actual bugs it might be worth trying a nightly binary, since progress is faiiirly fast

vivid hemlock Feb 17, 2023, 11:14 PM

#

peak gulch I haven't built from source because my computer is a potato

you don't need to! there are master binaries on the website

peak gulch Feb 17, 2023, 11:14 PM

#

Oh great

vivid hemlock Feb 17, 2023, 11:15 PM

#

if you're on linux i can give you a handy dandy little script that just installs the latest zig tarball to ~/.opt/zig/

peak gulch Feb 17, 2023, 11:15 PM

#

Yeah that would be awesome but not now

#

how do I do the compile time check for startsWith

vivid hemlock Feb 17, 2023, 11:16 PM

#

peak gulch ``` Compile Log Output: @as(fn(u16) callconv(.Inline) u16, (function '__builtin_...

on this: your latest thing doesn't actually filter on callconv, you should probably do that

peak gulch Feb 17, 2023, 11:16 PM

#

ti.Fn.name

vivid hemlock Feb 17, 2023, 11:16 PM

#

peak gulch how do I do the compile time check for startsWith

if (comptime std.mem.startsWith(u8, fi.name, "LLVM")) { ... }

peak gulch Feb 17, 2023, 11:17 PM

#

        if (fi.calling_convention == .Inline) continue;

pliant peak Feb 17, 2023, 11:17 PM

#

if (!std.mem.startsWith(u8, decl.name, "LLVM")) continue;

vivid hemlock Feb 17, 2023, 11:17 PM

#

pliant peak ```ts if (!std.mem.startsWith(u8, decl.name, "LLVM")) continue; ```

needs to have comptime specified

#

wait no

#

we're in a comptime block

#

ignore me!

pliant peak Feb 17, 2023, 11:17 PM

#

Beat me to it 😛

vivid hemlock Feb 17, 2023, 11:18 PM

#

peak gulch ```rs if (fi.calling_convention == .Inline) continue; ```

sure, or to cover all bases, if (fi.calling_convention != .C) continue;

peak gulch Feb 17, 2023, 11:19 PM

#

src/main.zig:35:19: error: 'union_unnamed_1' is not marked 'pub'
        const d = @field(llvm, decl.name);
                  ^~~~~~~~~~~~~~~~~~~~~~~
/home/jtenner/llvm-zig/zig-cache/o/b98892c84fe7eeec1f62f307de5034c5/cimport.zig:136:1: note: declared here
const union_unnamed_1 = extern union {
^~~~~

pliant peak Feb 17, 2023, 11:19 PM

#

peak gulch ```zig src/main.zig:35:19: error: 'union_unnamed_1' is not marked 'pub' ...

Ah.

peak gulch Feb 17, 2023, 11:19 PM

#

lolwat

pliant peak Feb 17, 2023, 11:19 PM

#

You'll want to add this too:

#

if (!decl.is_pub) continue;

#

😄

peak gulch Feb 17, 2023, 11:19 PM

#

hang on

pliant peak Feb 17, 2023, 11:20 PM

#

This is the difference between these things:

fn LLVMFooBar();
pub fn LLVMFooBar();

vivid hemlock Feb 17, 2023, 11:20 PM

#

it's arguably a bug that you can see the non-pub decls at all, but for now you can, so yeah just gotta skip em

pliant peak Feb 17, 2023, 11:20 PM

#

(pub means "Can be accessed outside the current file", but only semantics-wise in Zig source code. i.e: it's not a linking thing.)

peak gulch Feb 17, 2023, 11:20 PM

#

Okay so I added that

#

/home/jtenner/llvm-zig/zig-cache/o/b98892c84fe7eeec1f62f307de5034c5/cimport.zig:320:33: error: unable to translate macro: undefined identifier `LL`
pub const __INTMAX_C_SUFFIX__ = @compileError("unable to translate macro: undefined identifier `LL`"); // (no file):82:9
                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
referenced by:
    comptime_0: src/main.zig:37:19

pliant peak Feb 17, 2023, 11:21 PM

#

Ah.

#

This may break you. 😄

#

So...

#

The way that cImport handles macros that it cannot make sense of, is to make them @compileError calls.

#

This means that they cause a compile error if they are touched.

#

Guess what - you're touching em. 🤣

peak gulch Feb 17, 2023, 11:22 PM

#

🙃

pliant peak Feb 17, 2023, 11:22 PM

#

Problem is... you can't really not touch them if you're iterating over them. 🤔

peak gulch Feb 17, 2023, 11:22 PM

#

So I imagine I should probably filter out the name

pliant peak Feb 17, 2023, 11:23 PM

#

Something like that may be necessary, yeah.

#

Honestly, you can probably remove anything beginning with __.

peak gulch Feb 17, 2023, 11:23 PM

#

const d = @field(llvm, decl.name);

#

this is the problem

#

lol

#

if (!std.mem.startsWith(u8, decl.name, "LLVM")) continue;

pliant peak Feb 17, 2023, 11:24 PM

#

Yeah - you'll have to filter it out before that.

peak gulch Feb 17, 2023, 11:24 PM

#

Okay the next problem we hit

pliant peak Feb 17, 2023, 11:24 PM

#

GIVE IT TO ME

peak gulch Feb 17, 2023, 11:24 PM

#

/home/jtenner/llvm-zig/zig-cache/o/b98892c84fe7eeec1f62f307de5034c5/cimport.zig:325:44: error: unable to translate macro: undefined identifier `_Pragma`
pub const LLVM_C_STRICT_PROTOTYPES_BEGIN = @compileError("unable to translate macro: undefined identifier `_Pragma`"); // /usr/include/llvm-c-15/llvm-c/ExternC.h:18:9
                                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
referenced by:
    comptime_0: src/main.zig:37:19

#

is there any way to determine if it's a const?

pliant peak Feb 17, 2023, 11:26 PM

#

I don't think so(?)

peak gulch Feb 17, 2023, 11:26 PM

#

😦

pliant peak Feb 17, 2023, 11:27 PM

#

Oh hold on a minute... 👀

#

In your build command, you didn't link LLVM...

peak gulch Feb 17, 2023, 11:28 PM

#

zig build-lib \
  -dynamic\
  -O ReleaseSmall \
  -target wasm32-wasi \
  --library c \
  -freference-trace \
  -I /usr/include/llvm-c-15/ \
  src/main.zig

pliant peak Feb 17, 2023, 11:28 PM

#

Yeah - -I just provides a path for cImport to look in for the headers.

peak gulch Feb 17, 2023, 11:29 PM

#

do I need the .o files?

pliant peak Feb 17, 2023, 11:29 PM

#

It doesn't actually ask it to link or build anything, I don't think.

#

So

#

What you need is to build LLVM with Clang, or Zig.

#

Then link the result here.

peak gulch Feb 17, 2023, 11:29 PM

#

so it would result in me having to grab the .o files built for wasm

pliant peak Feb 17, 2023, 11:30 PM

#

You'd have to build LLVM targeting WASM, yeah.

peak gulch Feb 17, 2023, 11:31 PM

#

Okay I need to stop here.

pliant peak Feb 17, 2023, 11:31 PM

#

Now, you can do that with build.zig -- but that would mean translating LLVM's build system to it.
The alternative is to build LLVM using CMake etc, targeting WASM, and then figure out how to link all that when you zig build-lib.

peak gulch Feb 17, 2023, 11:31 PM

#

Nah I've been practicing building llvm from scratch to wasm

pliant peak Feb 17, 2023, 11:31 PM

#

🤣

peak gulch Feb 17, 2023, 11:31 PM

#

practicing

#

lol

#

@pliant peak you're awesome. thank you so much for your help

#

you too @vivid hemlock

vivid hemlock Feb 17, 2023, 11:34 PM

#

np

pliant peak Feb 17, 2023, 11:37 PM

#

I think you'd have to do is something like:

sudo apt-get install cmake make
# clone LLVM
cd llvm
mkdir build
cd build
cmake .. -DCMAKE_C_COMPILER='zig cc' -DCMAKE_CXX_COMPILER='zig c++' -DCMAKE_TOOLCHAIN_FILE=FOOBAR
make

Not sure about the -DCMAKE_TOOLCHAIN_FILE, but I suspect that LLVM would have one of those somewhere in its repo. In which case, asuming it's what I think it is--you'd just find the one for WASM32-WASI if there is one.

pliant peak Feb 17, 2023, 11:43 PM

#

peak gulch <@201094465815969794> you're awesome. thank you so much for your help

Happy to help o7 😄

peak gulch Feb 17, 2023, 11:44 PM

#

I'm using emscripten

#

can I target wasm-unknown instead?

#

I don't think I need wasi

pliant peak Feb 17, 2023, 11:46 PM

#

You can do wasm32-freestanding if that helps.

#

WASI is just the OS; the interface with the environment.

#

Freestanding just means all of that is entirely up to you, instead of having WASI's setup to fall back on.

peak gulch Feb 17, 2023, 11:52 PM

#

I'm not accessing the filesystem and emscripten is the compiler targeting wasm

#

So

#

Maybe emscripten can target wasi anyway

pliant peak Feb 18, 2023, 12:42 AM

#

I believe Emscripten generates the glue code required that WASI would normally do on the WASM side, but it implements a bunch of the stuff in JS, which of course isn't fast.
And it has to use a bunch of tricks in JS to make things work.
I've never used it successfully, but presumably that can work to some extent.

peak gulch Feb 18, 2023, 1:37 AM

#

@pliant peak I have a ton of experience with web assembly from emscripten to AssemblyScript, so using those tools has been easier for me to do.

peak gulch Feb 18, 2023, 4:42 AM

#

@vivid hemlock mind if I have that command lol

vivid hemlock Feb 18, 2023, 4:46 AM

#

oh yeah, this is the script:

#!/bin/sh -e
case "$(uname -s)" in
    Linux) os=linux;;
    Darwin) os=macos;;
    FreeBSD) os=freebsd;;
    *) echo 'Unsupported OS' >&2; exit 1;;
esac
curl -sSL 'https://ziglang.org/download/index.json' | jq -r ".master.\"$(uname -m)-$os\" | .tarball" | {
    read -r url
    tmpdir="$(mktemp -d)"
    cd "$tmpdir"

    curl -Lo _zig.tar.xz "$url"
    tar xf _zig.tar.xz
    rm -rf "$HOME/.opt/zig"
    mv zig*/ "$HOME/.opt/zig"

    cd
    rm -rf "$tmpdir"
}

#

i just have that in my PATH and run it every few days

peak gulch Feb 18, 2023, 4:46 AM

#

zig.sh lol

#

looks like it requires jq

vivid hemlock Feb 18, 2023, 4:54 AM

#

oh yeah, it has to parse a small json manifest using jq

peak gulch Feb 18, 2023, 5:39 AM

#

tyvm

peak gulch Feb 18, 2023, 6:23 AM

#

jtenner@CTWS140:~/zig-llvm-c$ ./build.sh
error: wasm-ld: /home/jtenner/.cache/zig/o/59c0f4a779642e16c63da8374aaf3ca8/libc.a(/home/jtenner/.cache/zig/o/e734c541c5d6455c42b069419261e554/__main_void.o): undefined symbol: main

#

hey guys

#

I'm really close

#

looks like the linker can't find a main function

#

@pliant peak I'm still in the ocean lol

#

@vivid hemlock it looks like the script we wrote doesn't work

#

Yeah every function in core.h has the c calling convention

#

src/main.zig:60:17: error: TODO implement exporting arbitrary Value objects
        @export(d, .{ .name = decl.name, .linkage = .Strong });
                ^

Compile Log Output:
@as(builtin.Type.Fn, .{.calling_convention = .C, .alignment = 1, .is_generic = false, .is_var_args = false, .return_type = void, .params = .{ .{.is_generic = false, .is_noalias = false, .type = ?*const fn([*c]const u8) callconv(.C) void} }})

peak gulch Feb 18, 2023, 7:04 AM

#

Can we dynamically create functions that call the c function?

frank pasture Feb 18, 2023, 7:50 AM

#

There's no way to procedurally create code (function bodies) at comptime, only types and simple values

vivid hemlock Feb 18, 2023, 7:56 AM

#

I'm taking a look at the TODO in Sema and it looks deceptively simple

vivid hemlock Feb 18, 2023, 8:47 AM

#

peak gulch ```rs src/main.zig:60:17: error: TODO implement exporting arbitrary Value object...

could i see all the code for this?

#

i might have implemented the thing in sema (literally 5 lines) but idk how to trigger the case in order to test it

peak gulch Feb 18, 2023, 4:21 PM

#

Just need a minute to get my bearings just woke up

#


comptime {
    @setEvalBranchQuota(1000000);

    outer: inline for (std.meta.declarations(llvm)) |decl| {
        if (!decl.is_pub) continue;
        if (std.mem.eql(u8, decl.name, "LLVM_ATTRIBUTE_C_DEPRECATED")) continue;
        if (std.mem.eql(u8, decl.name, "LLVM_C_STRICT_PROTOTYPES_BEGIN")) continue;
        if (std.mem.eql(u8, decl.name, "LLVM_C_STRICT_PROTOTYPES_END")) continue;
        if (std.mem.eql(u8, decl.name, "LLVM_FOR_EACH_VALUE_SUBCLASS")) continue;
        if (std.mem.eql(u8, decl.name, "LLVM_DECLARE_VALUE_CAST")) continue;
        if (std.mem.eql(u8, decl.name, "LLVM_C_EXTERN_C_BEGIN")) continue;
        if (std.mem.eql(u8, decl.name, "LLVM_C_EXTERN_C_END")) continue;
        if (!std.mem.startsWith(u8, decl.name, "LLVM")) continue;
        const d = @field(llvm, decl.name);
        const ti = @typeInfo(@TypeOf(d));
        if (ti != .Fn) continue;

        const fi = ti.Fn;
        if (fi.return_type == null) continue;
        if (fi.is_generic) continue;
        if (fi.is_var_args) continue;
        if (fi.calling_convention == .Inline) continue;
        // if (fi.calling_convention == .C) continue;
        for (fi.params) |param| {
            if (param.is_generic) continue :outer;
        }
        @compileLog(fi);

        @export(d, .{ .name = decl.name, .linkage = .Strong });
    }
}

export fn main() i32 {
    return 0;
}

#

You can ignore the LLVM stuff obviously

#

@vivid hemlock Thank you so much for your help

peak gulch Feb 18, 2023, 5:04 PM

#

You just need a fi.calling_convention == .C

peak gulch Feb 18, 2023, 7:22 PM

#

@pliant peak or @frank pasture can I make a file at compile time?

#

Using a comptime block

frank pasture Feb 18, 2023, 7:24 PM

#

Comptime can't do I/O as far as I know, so I think no. You could do it as a step in build.zig but if from what I've seen in this thread I assume you don't plan to use the zig build system

peak gulch Feb 18, 2023, 7:32 PM

#

@frank pasture I need all the types from the c library at build time to generate typescript bindings

frank pasture Feb 18, 2023, 7:34 PM

#

That's fine at build time, just not at comptime

peak gulch Feb 18, 2023, 7:36 PM

#

Is there a way to gather all the exports from main.zig? (At build time)

vivid hemlock Feb 18, 2023, 7:39 PM

#

No, build-time is before comptime so none of the file is analyzed or anything

peak gulch Feb 18, 2023, 7:41 PM

#

@vivid hemlock suppose the c function re-export works. Would it be possible to make some kind of file with the reflected type info?

pliant peak Feb 18, 2023, 7:47 PM

#

peak gulch <@220655631059648512> suppose the c function re-export works. Would it be possib...

Could write a program that just generates that. 🤔

peak gulch Feb 18, 2023, 7:50 PM

#

I could try to use clang to dump the ast and the generate the bindings that way

pliant peak Feb 18, 2023, 7:50 PM

#

peak gulch I could try to use clang to dump the ast and the generate the bindings that way

You do like to jump in at the deep at end don't ya.

#

I respect it.

peak gulch Feb 18, 2023, 7:51 PM

#

Bahahaha

pliant peak Feb 18, 2023, 7:51 PM

#

But if you can get that inline for loop working, that's probably the easier choice.

#

Use the resulting data to write out a file at runtime.

peak gulch Feb 18, 2023, 7:51 PM

#

I can't generate a file at comptime

pliant peak Feb 18, 2023, 7:51 PM

#

Then you can use that file at comptime in a future compilation.

#

No, I mean that you use it temporarily to generate what you need.

peak gulch Feb 18, 2023, 7:52 PM

#

So 2 compilations lol

pliant peak Feb 18, 2023, 7:52 PM

#

This data's not changing.

#

Why bother making a whole thing if a quick and dirty approach is perfectly fine 😄

peak gulch Feb 18, 2023, 7:53 PM

#

Yeah I don't mind quick and dirty.

pliant peak Feb 18, 2023, 7:53 PM

#

Besides, you can always make a separate program that imports the same things, does the inline loop, and generates it, and have that be a separate program.

peak gulch Feb 18, 2023, 7:53 PM

#

Yeah. Then actually execute itm

pliant peak Feb 18, 2023, 7:53 PM

#

zig run generate.zig
zig build-exe main.zig -target wasm32-wasi

#

You can use @embedFile to read a file at comptime.

#

You just can't write one.

peak gulch Feb 18, 2023, 7:54 PM

#

vivid hemlock i might have implemented the thing in sema (literally 5 lines) but idk how to tr...

I don't mean to sound impatient. I'm just doing some project planning. Will this be available on the main branch at some point when it gets tested?

peak gulch Feb 18, 2023, 7:55 PM

#

pliant peak ``` zig run generate.zig zig build-exe main.zig -target wasm32-wasi ```

This is perfect thank you

pliant peak Feb 18, 2023, 7:57 PM

#

vivid hemlock i might have implemented the thing in sema (literally 5 lines) but idk how to tr...

It trips up if the file contains const std = @import("std");, if that helps.

#

(And then use @This() at the iteration target of course.)

vivid hemlock Feb 18, 2023, 8:01 PM

#

Yeah sorry, haven't got around to testing yet, will do so soon

#

If it works I'll PR it but it might take a while to get in since the PR backlog is quite big rn

crimson nebula Feb 18, 2023, 8:05 PM

#

peak gulch ```rs comptime { @setEvalBranchQuota(1000000); outer: inline for (std....

Why are we exporting them again? We only need to generate bindings with this, right?

vivid hemlock Feb 18, 2023, 8:11 PM

#

Ah okay, so this issue is specifically caused by re-exporting extern fns

#

The question of why we're exporting them again is.... actually a good one lol. What's the end goal here?

peak gulch Feb 18, 2023, 8:16 PM

#

We want to compile all the llvm-c headers to wasm

#

Then we can use zig to generate d.ts files so that things can be moderately type safe.

This is an alternative to use emscripten's embind

vivid hemlock Feb 18, 2023, 8:19 PM

#

peak gulch We want to compile all the llvm-c headers to wasm

i'm not really at all familiar with wasm as a target, so forgive me being slow here; does re-exporting them in a wasm binary Just Work for achieving that?

crimson nebula Feb 18, 2023, 8:19 PM

#

vivid hemlock i'm not really at all familiar with wasm as a target, so forgive me being slow h...

Compiling LLVM to Wasm is the easy part

#

but using the raw Wasm exports isn't good DX

#

And there are too many C functions for us to manually write bindings for (and we'd have to update them manually too)

#

So we opted for some sort of bindings generation

#

As @peak gulch can tell you, embind isn't all that cooperative

peak gulch Feb 18, 2023, 8:23 PM

#

@crimson nebula perhaps we could generate the zig bindings automatically by creating a zig file too?

crimson nebula Feb 18, 2023, 8:24 PM

#

In short, we need a way to discover all the functions, enums, and typedefs in a set of llvm-c headers and use that information to write JS/TS wrappers over llvm-c compiled to Wasm

crimson nebula Feb 18, 2023, 8:24 PM

#

peak gulch <@670766281246638080> perhaps we could generate the zig bindings automatically b...

What do you mean?

peak gulch Feb 18, 2023, 8:25 PM

#

Thank you for helping me explain that.

#

Currently zig doesn't like exporting the c functions via @export

#

However, we can generate a function in a zig file that actually calls those exported functions

crimson nebula Feb 18, 2023, 8:26 PM

#

Isn't our primary issue actually writing the bindings themselves into a file?

peak gulch Feb 18, 2023, 8:26 PM

#

We need to generate two files. I'm on it 🙂

crimson nebula Feb 18, 2023, 8:26 PM

#

peak gulch However, we can generate a function in a zig file that actually calls those expo...

We don't need to export anything

peak gulch Feb 18, 2023, 8:26 PM

#

One for the exported zig functions to wasm

#

One for the d.ts definitions

crimson nebula Feb 18, 2023, 8:26 PM

#

We just need to take all those headers, libLLVM*.a, and compile them together into a .wasm/.mjs file

#

...right?

peak gulch Feb 18, 2023, 8:27 PM

#

Well we could stop at the wasm level

crimson nebula Feb 18, 2023, 8:27 PM

#

We can set the functions we need to export via Emscripten

peak gulch Feb 18, 2023, 8:27 PM

#

I think you should have a little faith in me 🙂

crimson nebula Feb 18, 2023, 8:27 PM

#

but we need Zig to accumulate all that delicious type info

crimson nebula Feb 18, 2023, 8:28 PM

#

peak gulch I think you should have a little faith in me 🙂

Okay, fineeeee

pliant peak Feb 18, 2023, 9:06 PM

#

peak gulch However, we can generate a function in a zig file that actually calls those expo...

Let me just confirm: You want to compile a WASM module that contains LLVM compiled for WASM, and then call those LLVM functions from outside the WASM module?

#

And the point of generating these bindings is because you do not want to have to bind all those LLVM functions manually.

peak gulch Feb 18, 2023, 9:07 PM

#

Bingo

#

Heh that whole sentence made my head spin lol

pliant peak Feb 18, 2023, 9:07 PM

#

Okay, cool.

#

🤣

#

FTR, you may find a complication about using @cImport from generate.zig, because that program will not be targeting WASM, however the cImport system takes that into account when translating macros and types, and things.

#

You can run cImport "manually" though, and specify the target to interpret it with.

peak gulch Feb 18, 2023, 9:09 PM

#

My game plan so far:

make a file called src/llvm.wasm which has all the cImports
use type reflection like before to write src/lib.zig src/lib.mjs and src/lib.d.ts from a native program that simply loops over the function names and the parameters called src/build-bindings.zig

pliant peak Feb 18, 2023, 9:09 PM

#

See, const llvm = @cImport("foo.h"); is the same as:

zig translate-c foo.h -target TARGET > zig-cache/cimport.zig

const llvm = @import("zig-cache/cimport.zig");

peak gulch Feb 18, 2023, 9:10 PM

#

alright so we can use zig translate-c foo.j -target wasm32-unknown-unknown

pliant peak Feb 18, 2023, 9:10 PM

#

(Well - it actually also passes -I ... and whathaveyou to translate-c too, but you get the idea.)

#

That's my thinking.

#

You can then try to import that file into generate.zig.

peak gulch Feb 18, 2023, 9:11 PM

#

yes

#

what are all the targets for zig?

pliant peak Feb 18, 2023, 9:12 PM

#

Now, that may not work; it depends on exactly what the file contains, etc. Things that are available on WASM are not on x64, and vice-versa. So you may find yourself having to manually delete some stuff from the translated file to make it work or whatever.

pliant peak Feb 18, 2023, 9:12 PM

#

peak gulch what are all the targets for zig?

zig targets gets you some JSON output from which you can determine all valid combinations.

peak gulch Feb 18, 2023, 9:13 PM

#

@pliant peak can I target wasm32-emscripten... i don't know what the target would be

pliant peak Feb 18, 2023, 9:15 PM

#

peak gulch <@201094465815969794> can I target wasm32-emscripten... i don't know what the ta...

I don't think so, as Emscripten's job is the same as what WASI is meant to do, only more jank 😄

peak gulch Feb 18, 2023, 9:15 PM

#

so I have to target wasi?

#

I just want a library 😭

pliant peak Feb 18, 2023, 9:17 PM

#

It depends largely on what LLVM wants to be able to do.
If that wants to be opening files and the like, then wasm32-freestanding doesn't provide anything for that, so it won't work.
WASI's job is that it standardizes what symbols are provided by the external environment for accessing files and whatnot, hence why I suggest it.

peak gulch Feb 18, 2023, 9:17 PM

#

I don't want to access files

#

I am writing stuff from node.js which is reading the files and parsing AST

pliant peak Feb 18, 2023, 9:18 PM

#

Well - maybe you don't - but LLVM normally provides procs that do, I think.

#

Like LLVMEmitObjectToFile, or whatever its called.

peak gulch Feb 18, 2023, 9:18 PM

#

Yeah we won't be calling that function lol

#

we will emit the object to bitcode and link it manually using wasm-ld

pliant peak Feb 18, 2023, 9:19 PM

#

If LLVM defines it, and it gets compiled to WASM when you compile LLVM, then it probably cause you problems 😄

peak gulch Feb 18, 2023, 9:19 PM

#

okay so wasi it is!

pliant peak Feb 18, 2023, 9:20 PM

#

It's possible that LLVM just omits that if it's compiled to WASM, but you'd have to examine what LLVM does there to find out definitively 😄

pliant peak Feb 18, 2023, 9:20 PM

#

peak gulch we will emit the object to bitcode and link it manually using wasm-ld

How so?

pliant peak Feb 18, 2023, 9:21 PM

#

pliant peak Well - maybe __you__ don't - but LLVM normally provides procs that do, I think.

Also again - just to be clear - this problem about talking to the filesystem is something that the compiled WASM module that contains LLVM will want/need, if you then try to use that module in a program (or make it into a WASM exe) later.

peak gulch Feb 18, 2023, 9:32 PM

#

pliant peak How so?

our goal is to make a compiler using JS

#

We already have a parser/ast, and we want to use llvm to generate bitcode

#

In the meantime, compiling LLVM to wasm as a library will help everyone who can consume wasm and wants to use llvm

#Export llvm-c to wasm using zig?