Understanding the bootstrapping system

[young mulch]

I'm trying to understand how the bootstrap build for zip works. In particular, it seems like stage1 should contain a C implmentation of zig but I only see very short files that create wasm conversions. Is there anywhere I should look documentation wise to better understand the bootstrapping process for stage1?

[tiny geyser]

https://ziglang.org/news/goodbye-cpp/ explains it, look for The New Build Process section for the tldr

[young mulch]

so interpreter from stage1 only includes the features necessary to build stage2 right? (ie zig1 and zig2 and full zig)

[tiny geyser]

yeah

[young mulch]

are the standards/semantics for zig1, zig2 and zig listed anywhere

[tiny geyser]

well its not an interpreter, its the same compiler

[young mulch]

so it's a zig compiler that runs in the wasi interpreter

do zig1, zig2, zig have different syntax/features or do they just support different compiler optimization/runtime semantics

[tiny geyser]

it translates the wasm into C and compiles/runs that like a normal program

[tiny geyser]

theyre the exact same, zig1 is just a bit of an older version of the compiler since its only comitted when the compiler or stdlib need to use a new feature

[young mulch]

so this 4000 line c program supports the entire zig standard? https://github.com/andrewrk/zig-wasi/blob/main/src/main.c

[tiny geyser]

thats a wasi interpreter its not related to zig itself

and its not used

[young mulch]

oh so where do they get the zig1.wasm from?

[tiny geyser]

a member of the zig team builds it locally and commits it to the zig repo

[young mulch]

from the article: We provide a minimal WASI interpreter implementation that is built from C source, and then used to translate the Zig self-hosted compiler source code into C code.

so is the zig1.wasm from a previous version of the zig language?

with the first one being from a concrete C implmentation of zig

[tiny geyser]

the one you sent is just an interpreter, it looks like it was an experiment before they landed on outputting C

[tiny geyser]

no it was never in C, always been turning the zig written compiler to wasm

[young mulch]

oh so how was the first zig compiler made then

im assuming since the article is called "goodbye-cpp" that it was written in cpp

[tiny geyser]

sorry i think i misunderstood your previous question, yeah it was originally in c++

c++ written compiler compiled zig written compiler, zig written compiler compiled itself to wasm for bootstrap

[young mulch]

ig this is a problem with all bootstrapping but wouldnt which zig version the current repo is being bootstrapped on need to be maintained

in order for it to be clear what the semantics of the language is at any given time

[tiny geyser]

no zig1.wasm is just updated when needed

when making a breaking change the process is

1. implement the change in the compiler
2. recompile zig1.wasm
3. update the compiler to use the change

the important bit is that 1. implements the change/feature but doesnt use it yet

so the old zig1.wasm can still compile it

[young mulch]

but when teh zig1.wasm is updated how do we know which change it's updated based on

[tiny geyser]

its just the previous commit, the update is done in its own commit

[young mulch]

ah ok so it's like an artifact style update

[tiny geyser]

bootstrapping like this is an inherently opaque process but its been proven to be correct by 3rd parties

[young mulch]

you'll never have a master commit that doesnt update the .wasm

[young mulch]

wdym by this

as in it works for other projects?

[tiny geyser]

no like other people have gone thru the entire bootstrap chain from before the c++ compiler was dropped and verified that the zig1.wasm is what its supposed to be

i think it was the guix people, ill see if i can find it

[young mulch]

ah ok makes sense

[young mulch]

so is stage2 the final zig version that gets shipped or is there another stage after that?

[strange condor]

nope, the process is:
compile bootstrap.c
run bootsrap, which does:
compile wasm2c.c
run wasm2c on zig1.wasm
compile zig1.c
use zig1 to compile zig compiler
zig1 only has the c backend, so this outputs zig2.c
compile zig2.c
end of bootstrap

zig2 is not fit to release, it was compiled by a c compiler, it will not be the same as if compiled by a full zig compiler

use zig2 to compile zig compiler again, this is zig3

zig3 is compiled by a full zig compiler, but to ensure the detour through c didnt affect change any logic:
use zig3 to compile zig compiler again again, this is zig4

zig3 and zig4 should be byte for byte identical
one of these, doesnt matter which, is the release.

when necessary, use a full zig compiler to update zig1.wasm,

[young mulch]

wdym by zig3 is compiled by a full zig compiler?

isnt zig3 a build result

what does it mean to compile it

[lament wolf]

"zig3 is compiled by a full zig compiler"
as in
"zig3 is the result of compiling with a full zig compiler"

[young mulch]

but what is the full zig compiler from?

zig2?

cause zig2 is a full zig compiler but it only supports the C backend right?

which is sufficient for compiling zig3?

cause if you're really using an older version of zig for cocmpiling zig3 then what was the point of doing zig1.wasm boostrapping

[abstract steeple]

zig2 doesn't only support the C backend, that's zig1.

zig1 - output of C compiler on zig1.c which is based on zig1.wasm, only includes support for C backend
zig2 - output of C compiler on zig2.c, which is based on zig1's C backend using the Zig codebase
zig3 - output of zig2 (a full Zig compiler - since its source is the Zig codebase) on the zig codebase

zig4 is the output of zig3 on the zig codebase. it should be identical to zig3 since zig4 and zig3 are the outputs of zig3 and zig2 respectively which are not identical but should be logically the same

[young mulch]

so what's the point of zig4?

a roudntrip sanity check?

[abstract steeple]

yep