Some kind of HTML Export for Typst

[wheat dune]

Claim: This is neither official supported nor exact the HTML export. I just find it cool so I'm going to share it.
Context: The format of documentation comments was discussed and we confirm that it should support typst syntax. However, LSP clients only supports docs in markdown format, but rarely support html format. Therefore, a "Markdown Export" is needed here. And since we know that it will be quite easy to get somewhat a "HTML Export" by exporting converted markdown then.

Since we suppose that the converting documents are all typst documentation, conforming some restricted syntaxes, it seems that they are usually not heavily styled or styles are separated from content. Even if there are some mixed styles, we should be able to drop them safely (looks sensible). By this assumption, a typst-to-markdown converter can be quite simple.

Besides converting documentation, I find it could be abused for publishing your (simple) documents to those platforms only accepting markdowns. For example, converting your README.typ to README.md. This could also be lossy, since people can continue reading styled PDF output of the documents if they would like to.

As a showcase, I run the tool on some GPT-generated typst document.

Source code: https://github.com/Myriad-Dreamin/tinymist/tree/main/crates/typlite

Ideas:

• convert the document as is at syntax-level.
• rendering some content if we have chances, like math equations or contextual expressions.
• drop styles and rest content of complex scripting.

Why not pandoc:

• I don't want to embed a pandoc in LSP or let LSP binary depend on some external pandoc binary, so I write a pure rust one.
• I want to customize the renderer a bit to fit in typst documentation perfectly.

Note: The current goal is only for converting documentation comments or some deadly simple documents. if someone is interested, they can send PRs to improve it.

Ref: #1260973804637786224 message

Q: btw do you plan on fully supporting the tidy documentation format? e.g. I'd like to hover over a parameter and see the description of it
See https://github.com/Mc-Zen/tidy/blob/main/README.md?plain=1#L44

A: This is the plan. We will add support to parse all user documentation from typst to markdown as soon as possible, before people getting used to use markdown format.
*not fully support. The converter will ignore all show/set rules first. I feel the full support will come easily after typst team launching their HTML export. where they will have an IR, and we can convert the IR to markdown format more easily, where the show/set rules are already computed.

[fluid portal]

Nice

[mellow cedar]

Is math svg based or mathml based?

[wheat dune]

svg based

[errant flax]

Oh that may be a issue for hovering then ? Afaik having images show in hover documentation isn't the most widely supported thing

Vscode will show some of them, but only if they're not stored locally, only url link would work (in the markdown syntax)

Though not sure mathml will fare better here (usually the set of html tags that are accepted is pretty small)

[wheat dune]

There should be more clients supporting images showing than mathml tags. Despite whether it supports or not, in the scope of documentation, the "text part" of the converted markdown should already completely describe the documenting items. For example:

// A function to help determine whether `num` is odd.
// 
// == Example
//
// `example
// #is-odd(1);
// ·
#let is-odd(num);

We may get converted markdown:

A function to help determine whether `num` is odd.

## Example
`typ
// #is-odd(1);
`
![rendered-example](data-url:...)

Note the rendered result is attached after the example code. On vscode we will see both the example code and the rendered result. On neovim, some users can see the rendered result if they have installed some image support. Many people will not see the rendered example, but a provided example code is already enough. Everything is just served in best efforts.

[fluid portal]

Yeah I think that's great

[errant flax]

Yeah, I guess that's more of a general issue for all IDEs

Short of doing completly custom things (like using a webview instead of the default tooltip for vscode) or general changes/interest in supporting mathml/images in these context it's the best thing you can do today for a general ide-agnostic LSP

[sullen pelican]

That's incredible! I was thinking about using html export to generate javadoc comments lately. May be it's time for me to reconsider 🙂

Although I'd ask, which GPT did you use? None of those I tried understood Typst...

[mellow cedar]

Do people really have to use gpt for absolutely everything?

[sullen pelican]

Well it's useful fairly often

I use it for writing some annoying parts of code, when it's quite obvious what to write, but it would take a long time. I would very much appreciate the ability to do so with Typst code as well

[open plank]

Claude has a recent cuttof edge and hence knows typst

[clear kernel]

Very cool! @wheat dune is the markdown export from the screenshot available from the vscode extension / tinymist cli yet?

[wheat dune]

not yet. I'll maintain it for LSP uses but not for real markdown export, since it is not clear on how to convert any documents. You know this should be add some restricts to use scripting, which is sensible for converting the documentation but may not for any documents.