CIL2C - CIL to C compiler | Operating System Development | Page 1

modest crescent Sep 10, 2023, 12:06 AM

#

Hello! Today I wanted to experiment on a CIL to C compiler... so I did it. It doesn't support much right now, but I'm making progress. (I'm writing this at 2 AM please forgive me) https://github.com/AnErrupTion/CIL2C

GitHub

GitHub - AnErrupTion/CIL2C: A CIL to C code compiler made in C#.

A CIL to C code compiler made in C#. Contribute to AnErrupTion/CIL2C development by creating an account on GitHub.

foggy hedge Sep 10, 2023, 7:14 AM

#

nice!

kindred sorrel Sep 10, 2023, 8:07 AM

#

Woah

#

Looks nice

modest crescent Sep 10, 2023, 9:04 AM

#

Thanks!

stark elk Sep 10, 2023, 8:03 PM

#

3sKannaSus

modest crescent Sep 10, 2023, 9:26 PM

#

stark elk <:3sKannaSus:641865318964264967>

👀

modest crescent Sep 17, 2023, 7:50 AM

#

A lot of work has gone towards CIL2C since then! I've just done a huge type system refactoring, and a bunch of opcodes have been implemented. For example, you can now allocate structs (on the stack, not on the heap though), do all sorts of loops and jumps, methods, fields, etc!

stark elk Sep 17, 2023, 11:10 AM

#

KurisuSip

modest crescent Sep 17, 2023, 4:17 PM

#

Now you can import C functions inside your CIL code! For example, with C#:

[CFunctionImport(IncludeFile = "utils.h", FunctionName = "outb")]
private static extern void Out8(ushort port, byte value);

stark elk Sep 17, 2023, 4:27 PM

#

why not use a normal DllImporter or whatever its called in C#

modest crescent Sep 17, 2023, 4:27 PM

#

stark elk why not use a normal DllImporter or whatever its called in C#

Because I plan to use that for actual DLL imports

stark elk Sep 17, 2023, 4:28 PM

#

you can use that with both static and dynamic linking

modest crescent Sep 17, 2023, 4:28 PM

#

Also that attribute only has one string, I need two

stark elk Sep 17, 2023, 4:28 PM

#

like, staticly link your c code and statically link the outputed C#->C code and then link em together into a final binary

modest crescent Sep 17, 2023, 4:28 PM

#

I plan to have a CStructImportAttribute as well so that you can use C structs in your CIL code

modest crescent Sep 17, 2023, 4:29 PM

#

stark elk like, staticly link your c code and statically link the outputed C#->C code and ...

That's already what I do on my test project

#

Each C file is generated to an ELF object file which is then assembled into a final ELF binary

stark elk Sep 17, 2023, 4:29 PM

#

wait, so in theory, you don't even need an attribute, just an extern

#

you can just have it as a prototype

modest crescent Sep 17, 2023, 4:30 PM

#

Yes but the function name may not be in accordance with your CIL language's naming scheme or whatever

stark elk Sep 17, 2023, 4:30 PM

#

and keep it as is

modest crescent Sep 17, 2023, 4:30 PM

#

Like private static extern void outb() is ugly

stark elk Sep 17, 2023, 4:30 PM

#

well, you don't need the IncludeFile

#

also coreclr does alot of ugly naming for the sake of imports

modest crescent Sep 17, 2023, 4:30 PM

#

stark elk well, you don't need the IncludeFile

Yeah I know, I could just add an extern declaration to the function name

#

Buuuut I don't like that, it requires adding more code to handle the CIL type => C type conversions and stuff

stark elk Sep 17, 2023, 4:31 PM

#

hmmmm

modest crescent Sep 17, 2023, 4:31 PM

#

Plus that could be achieved with DllImport anyway

stark elk Sep 17, 2023, 4:31 PM

#

you don't need to put an extern in c function prototype

modest crescent Sep 17, 2023, 4:31 PM

#

I mean, why would you add an extern declaration to a function that's in your C code when you can just specify the header file

stark elk Sep 17, 2023, 4:31 PM

#

just don't add a body

modest crescent Sep 17, 2023, 4:31 PM

#

Same thing

modest crescent Sep 17, 2023, 4:32 PM

#

modest crescent I mean, why would you add an extern declaration to a function that's in your C c...

Doesn't change that

stark elk Sep 17, 2023, 4:32 PM

#

modest crescent I mean, why would you add an extern declaration to a function that's in your C c...

because then its generic for both static libraries already compiled and normal code you link it

modest crescent Sep 17, 2023, 4:32 PM

#

But why

#

Just why

#

You could simply change to a DllImportAttribute

stark elk Sep 17, 2023, 4:32 PM

#

or for example if you want to import something from another native language

#

like, you don't need the duplication of code here

#

since you would need it for DllImport anyways

#

(the code to have a prorotype)

#

and then you can also more easily import C++ stuff code

modest crescent Sep 17, 2023, 4:33 PM

#

I already generate function prototypes (I wish I didn't have to, fuck C for that and I'm also having issues with structs as well but anyway)

#

(Because of forward declarations and stuff)

stark elk Sep 17, 2023, 4:34 PM

#

yeah it makes sense you have to do that anyways

#

alot of time when I generate C code I just forward declare everything in the top of the file so it will sort itself out lol

modest crescent Sep 17, 2023, 4:34 PM

#

Except you can't do that with structs apparently

#

I mean you can but if you need to specify that struct in another struct the compiler explodes

#

And I'm having that exact problem

stark elk Sep 17, 2023, 4:35 PM

#

well, you can forward declare structs, but if you want to have them inlined you need that

modest crescent Sep 17, 2023, 4:35 PM

#

Especially when my code is multi-threaded

#

The structs are generated pretty much in random order

stark elk Sep 17, 2023, 4:35 PM

#

that is now a graph problem :)

#

is it actually worth it having that part multithreaded?

#

it would simplify alot if you just did it recursively

modest crescent Sep 17, 2023, 4:35 PM

#

Well even if it isn't

#

Sometimes a struct in the type system at index 0 will refer to a struct at index 2

#

If you see what I mean

modest crescent Sep 17, 2023, 4:36 PM

#

stark elk it would simplify alot if you just did it recursively

Huh? How would I do that?

#

I mean I'm loading a lot of the dnlib module into my own type system

stark elk Sep 17, 2023, 4:37 PM

#

I do the struct size calculation as a separate pass from metadata loading

#

and then when I want to fill the size of a type I recursive go over its fields and fill them as well

#

and I have separate passes for heap size and stack size

#

(in structs its the same, in objects its different)

modest crescent Sep 17, 2023, 4:38 PM

#

But that's the thing: I already theoretically know the size of the structs at compile-time

#

If only I could, for example, specify it in the struct prototype...

#

So that the compiler is happy and it can compile the code correctly

stark elk Sep 17, 2023, 4:39 PM

#

yeah but you still have the struct ordering, since C does it the simple way, if it doesn't know the size yet you can't use it

#

so what you would do when declaring structs is recursively go over the fields of the struct you define and emit them as well

modest crescent Sep 17, 2023, 4:39 PM

#

That's why I said to specifiy it in the struct prototype

stark elk Sep 17, 2023, 4:39 PM

#

depth first

modest crescent Sep 17, 2023, 4:39 PM

#

Just like when you specify arguments in a function prototype

modest crescent Sep 17, 2023, 4:40 PM

#

stark elk so what you would do when declaring structs is recursively go over the fields of...

That's already what I do

#

When I emit the struct I emit its fields at the same time

stark elk Sep 17, 2023, 4:40 PM

#

modest crescent If only I could, for example, specify it in the struct prototype...

that would be too error prone

stark elk Sep 17, 2023, 4:41 PM

#

modest crescent When I emit the struct I emit its fields at the same time

so all you need to do is delay the actual emitting, concat all the struct content until you are done with it

#

if you do like that then it will naturally go in the correct order

modest crescent Sep 17, 2023, 4:41 PM

#

Huh?

#

I don't get it

stark elk Sep 17, 2023, 4:41 PM

#

something like

modest crescent Sep 17, 2023, 4:42 PM

#

I'm not sure how that would solve my issue, which is that structs are used in other structs before they're declared

stark elk Sep 17, 2023, 4:42 PM

#

that is when a two pass system is needed

#

you first parse all of the metadata

#

and only then you emit stuff

modest crescent Sep 17, 2023, 4:43 PM

#

Well that's technically already what I do

#

dnlib => TypeSystem => Emit types (with non-static fields) => Emit function prototypes => Emit static fields => Emit methods => Emit main method

#

That's what's happening, in the specified order

stark elk Sep 17, 2023, 4:44 PM

#

so now you just need to emit it recursively, go over all the types you want to emit, concat into a string all the contents, generating the dependant structs on the way

modest crescent Sep 17, 2023, 4:44 PM

#

modest crescent dnlib => TypeSystem => Emit types (with non-static fields) => Emit function prot...

Each emittion you see here is its own separate loop

stark elk Sep 17, 2023, 4:44 PM

#

you would want for struct types that Emit fields will call into Emit type essentially

#

and as long as you only dump into the final buffer at the end of each emit field iteration you will have the order of structs in c be correct

modest crescent Sep 17, 2023, 4:45 PM

#

By Emit fields I mean the static fields

#

The non-static fields go inside the struct itself

stark elk Sep 17, 2023, 4:45 PM

#

I am talking about non-static fields yeah

#

ok so you want to essentially have Emit type call itself to emit the field types of value types

modest crescent Sep 17, 2023, 4:48 PM

#

So, if I understand correctly, I just go:

EmitType:
    for each field in type.Fields:
        EmitTypeAtTop(field.FieldType)

#

Or something like this

stark elk Sep 17, 2023, 4:49 PM

#

essentially yeah

#

just make sure to not do non-value types

#

because otherwise you can get into nasty cyclic loops

modest crescent Sep 17, 2023, 4:49 PM

#

So I just do this for CIL structs and not CIL classes?

stark elk Sep 17, 2023, 4:50 PM

#

so ```py
EmitType:
if already emitted:
return

s = 'struct name {'
for each field in type.Fields:
s += field.type + ' ' + field.name + ';'
if field.type is value type:
EmitType(field.type)
s += '};'

emit_text(s)

modest crescent Sep 17, 2023, 4:52 PM

#

But then how do I start the recursion? I mean, how to get all types emitted at the end?

#

Oh wait nevermind

#

I can just loop over them all since there's a condition at the start

stark elk Sep 17, 2023, 4:53 PM

#

exactly

modest crescent Sep 17, 2023, 5:12 PM

#

Well now, I'm running into the issue but with non-value types as well :v

#

Oh wait

#

Awesome it works!

#

I just forgot to apply the original recursion to non-value types as well

#

I wonder if I can also apply the recursion to function prototypes and static fields as well

stark elk Sep 17, 2023, 5:23 PM

#

nice!

#

static fields maybe

#

function prototypes not really

#

recursion is possible for functions

#

like, you can do it lazy

modest crescent Sep 17, 2023, 5:23 PM

#

Yeah but I'd need to make that code single-threaded and it's really not worth it since it also emits the method bodies

stark elk Sep 17, 2023, 5:24 PM

#

for functions I would forward declare everything

modest crescent Sep 17, 2023, 5:24 PM

#

Yeah

#

I was adding comments before each struct declaration to indicate which type it was, well now it's all a bit backwards 😅

#

However I'll probably remove those before they clutter quite a lot the code for not much reason

#

Yeah it's much more readable without them

#

Well, thanks @stark elk for this worthy help! 😄

stark elk Sep 17, 2023, 5:29 PM

#

glad I could help :)

modest crescent Sep 17, 2023, 5:30 PM

#

Just a note for the DllImport stuff- that attribute would only serve for linking to other external DLLs, whatever they may be, while CFunctionImportAttribute (and soon CStructImportAttribute) only serve for including other external C code

#

So the use cases are different

stark elk Sep 17, 2023, 5:32 PM

#

personally I just don't see it as a difference since both are library, be it static or dynamic 3sGudaEhehe

#

but you do you

modest crescent Sep 17, 2023, 5:32 PM

#

But then, how would you do it? I'm not sure I fully understood it

stark elk Sep 17, 2023, 5:33 PM

#

I would use extern + DllImport optionally and let the linker do the rest of the work

#

so extern without anything just matches the function name as is, DllImport allows to change the name

modest crescent Sep 17, 2023, 5:33 PM

#

So basically have the method name as the function name you want to import?

modest crescent Sep 17, 2023, 5:33 PM

#

stark elk so extern without anything just matches the function name as is, DllImport allow...

But DllImport normally serves for, well, importing a function from a DLL

modest crescent Sep 17, 2023, 5:34 PM

#

modest crescent So basically have the method name as the function name you want to import?

I guess that could work

#

You know what I'll try to implement that

#

Though my code would be more complex

stark elk Sep 17, 2023, 5:34 PM

#

yeah using DllImport is a bit of meh, so you could still keep the FunctionImport attribute

#

but I don't think the IncludeFile is needed

#

since you can let the linker do the job

#

and then you don't depend on external headers

modest crescent Sep 17, 2023, 5:35 PM

#

Fair

#

My idea is to be also able to import C structs directly

stark elk Sep 17, 2023, 5:36 PM

#

that is a cool feature tbf, so no need for codegen

#

even tho you still need the prototype in C# so the C# compiler will be happy

modest crescent Sep 17, 2023, 5:37 PM

#

For now actually I'll leave it as is, I'll focus on making my compiler generate a bit less code first

stark elk Sep 17, 2023, 5:44 PM

#

yeah no need to really change it if you already have it lel

#

just make stuff compile and work

modest crescent Sep 17, 2023, 5:44 PM

#

Yeah

stark elk Sep 17, 2023, 5:44 PM

#

good luck when you get to try-catch smug_face_FB

modest crescent Sep 17, 2023, 5:44 PM

#

I still need to handle the heap, abstracted classes, etc

#

For the heap I only need to implement newobj and newarr though

#

(I think)

#

Of course the GC

stark elk Sep 17, 2023, 5:45 PM

#

I just told myself I will only handle the GC when I run out of memory

#

and use malloc until then

#

with no freeing

modest crescent Sep 17, 2023, 5:45 PM

#

I mean... that's one way of doing it

#

Lol I was thinking about how technically now you could use C# to program an Arduino...

stark elk Sep 17, 2023, 5:47 PM

#

arduino has 2kb of ram

#

and like 2kb of rom too

modest crescent Sep 17, 2023, 5:47 PM

#

But if I compile with -Ofast

#

It should eliminate most of the CIL "bloat"

stark elk Sep 17, 2023, 5:47 PM

#

you want Oz and Os

#

but yeah

modest crescent Sep 17, 2023, 5:47 PM

#

Yeah right

stark elk Sep 17, 2023, 5:48 PM

#

if you don't use too much of the corelib it will be finneee

modest crescent Sep 17, 2023, 5:48 PM

#

xD

grand stag Sep 19, 2023, 12:51 PM

#

its means what we can write os without tysila?

#

hmm

modest crescent Sep 25, 2023, 7:56 AM

#

grand stag its means what we can write os without tysila?

Yeah

#CIL2C - CIL to C compiler