I made a new programming language which currently compiles to BEAM bytecode. Should i continue my bytecode approach or switch to creating an abstract format instead. What about gleam which compiles to erlang source? Im not sure what the correct approach is here so any breakdown of each decision would be very appreciated. Many thanks!
#Should i compile to bytecode or to abstract format?
quiet zodiac
dry atlas
There was recently discussion of this on Gleam side, I think it was about getting better error messages (line numbers) if they targeted abstract format. Elixir and LFE also target abstract format.
My understanding is that Gleam targets Erlang source to let the Erlang compiler do the most optimisations possible on it, and it's also handy that you can read the source and see what your code builds to (for those that know Erlang)
What's your language? 🙂
torpid yoke
Hey hey
torpid yoke
I've quite a lot of experience here!
quiet zodiac
gleam member spotted
torpid yoke
Are you going directly to BEAM bytecode or are you targeting Core or such?
quiet zodiac
thats what im going to find out now hopefully
i am currently writing my own binaries though
torpid yoke
The best option is largely believed to be Erlang Abstract Forms
Core Erlang was popular but it's a moving target, and I think all its users have stopped now.
and it lacks compatibility with some tooling
Gleam targets Erlang source. This is great as it's easy to debug and easy to work with outside of BEAM languages, but you lose things like accurate line numbers in stack traces, coverage tooling, etc
quiet zodiac
i mean BEAM bytecode itself has not changed so much over the years except maybe more opcodes but the ISA number is still 0. no breaking changes so far to it
torpid yoke
If you can easily generate Abstract Forms (like if your compiler is written in a BEAM language) then it's a really good choice
BEAM bytecode changes a lot
The problems with BEAM bytecode is that you need to update your code generator for each new release of the BEAM
And since you won't be using the Erlang compiler at all you'll need to implement all your own optimisations, so it can be quite challenging to match the performance you'd get with other targets
So there's quite a large development and maintainence cost to using it
quiet zodiac
yeah it does feel like re-inventing the wheel. not to mention the horrendous algorithm for packing operands (it has like ~5 cases where it does completely different things)
torpid yoke
If you have a larger and very skilled team you may be able to output bytecode that is superiour to the bytecode the Erlang compiler outputs while also keeping up-to-date with new BEAM releases, but it's a lot of extra work.
quiet zodiac
(i think im feeling a bit of a sunk cost fallacy regarding that).
You raise some good points, however my compiler is written in rust. Id have to generate the ast and somehow run it though compile:forms functions
torpid yoke
You'd likely want to make a build tool that runs the Erlang compiler
To start you could just use make
quiet zodiac
probably, unless i want to re-implement the entire thing again which at that point i might aswell just go directly to bytecode
torpid yoke
Gleam implements a build tool from scratch and has a persistent BEAM process for Erlang compilation