Tinymist LSP

[heady kelp]

Can this be fixed?

[foggy sorrel]

Could be a font issue perhaps?

I'm not sure what font the preview panel uses

[blissful merlin]

I’m not sure, it’s blank both on my desktop and laptop

[sand plinth]

It's empty for me too, I found this error in the extension host log:

2024-05-06 18:45:07.317 [error] InvalidCharacterError: Invalid character
    at new DOMException (node:internal/per_context/domexception:53:5)
    at __node_internal_ (node:internal/util:526:10)
    at btoa (node:buffer:1281:13)
    at bv (c:\Users\ericb\.vscode\extensions\myriad-dreamin.tinymist-0.11.7-win32-x64\out\extension.js:44:52883)
    at runNextTicks (node:internal/process/task_queues:60:5)
    at processImmediate (node:internal/timers:447:9)

[foggy sorrel]

I can't even open symbol view at all

@midnight isle

I think it may possibly be because it requests the font 微軟正黑體 which has a latin name on my compute?

just guessing

[midnight isle]

the current implementation is bug prone, so I would like to reimplement one instead of fixing current one. I may start to reimplement it after fixing tons of bugs and rough edges reported by andrew.

[midnight isle]

The path completion changes to "absolute style" if you prepend a slash. Did you find it?

I can't reproduce it

when does it filter list incorrectly?

I find two issue.

• to trigger completion in string, you need to input / or . or " or ctrl space (explicit complete), this is expected, that means if you edit some string in the middle, and input a-zA-Zother-unicode, no completion request is made. This is probably some limitation of lsp.
• it has different filter behavior between abs/rel path completion, but I don't know why.

lsp server isn't aware of editor.inlayHints.enabled, but vscode (lsp client) is, and neovim probably as well.

[blissful merlin]

Got it

[midnight isle]

Now the unspaced field (Variable) does show up, but when I highlight it, it still doesn't have the space added, but when I do select it, it not only adds space, it also removes the leading field name. A bug?
Improved
After opening the file it doesn't show the error unless I save the file (no-op).
vscode invokes lsp's focus command but neovim doesn't. should add an issue, and discuss a suitable behavior to change focus file if the client is neovim.
Hmm, if I type the variable name it won't show it, but if I first trigger autocompletion dropdown menu then I can see it and search/filter it.
Fixed
In this case, just typing y by itself triggers the menu.
Isn't it expected? See the attached Figure 1.
Double forward slash.
Improved
Field selection doesn't seem to work.
I need an approach to invoke editor.action.triggerSuggest and editor.action.triggerParameterHints at the same time, after a field completion, but I don't find a way other than letting client provide an extra commands, https://github.com/Myriad-Dreamin/tinymist/blob/f167ad152449c631e5946187efd2ba1772612f02/editors/vscode/src/extension.ts#L161. This is because https://github.com/microsoft/language-server-protocol/issues/1117.
I looked at rust-analyzer, and find that the parameter hints (signature help) keeps until exiting a function autocompletion. That is, if tinymist does same thing as ra, when I type text, tinymist will complete text(${content-arg}, fill: ${fill-arg}, stroke: ${stroke-arg}, ..), and I will continue finishing my function call for tabs and go to ${content-arg}, ${fill-arg}, and so on. however it is probably not what we would like to have, as we usually write text(fill: blue) and then add a content arguments by []. I may look at python lsps then.
How do you specify the argument types?
tinymist doesn't look your js-doc style comments. that's not typst style and hard to handle.

Or just use the snippet from here: https://github.com/lvimuser/lsp-inlayhints.nvim
Do you mean that I should add information in editors/neovim for users?

[grim berry]

Could you please add an ability to pass --inputs?

In my use case, I have some parts of the document, that are enabled conditionally, and I want to detect, when doc is compiled with LSP, and enable everything

[midnight isle]

here is the way of passing arguments that are not well modeled in scope of lsp.
https://github.com/Myriad-Dreamin/tinymist/tree/main/editors/vscode#passing-extra-cli-arguments

[grim berry]

Thanks!

[midnight isle]

"not well modeled in scope of lsp" means that we may find a better way to expose inputs for lsp clients, but I didn't find one. I guess we can use it intuitively if our extension provides task settings, https://code.visualstudio.com/docs/editor/tasks. But looks like task is only a vscode concept.

[dawn chasm]

Yeah, this is complicated. You might have to resort to some exotic ways of doing this.

[midnight isle]

Tracking remaining reported issues here: https://github.com/Myriad-Dreamin/tinymist/issues/257
https://github.com/Myriad-Dreamin/tinymist/releases/tag/v0.11.8

[heady kelp]

Oh, I didn't know that, thanks. But after second char it all disappears instead of starting the filtration.

[heady kelp]

I think always.

[heady kelp]

Hmm. I will try changing client settings. I don't get the filtration at all (at least in some cases).

[heady kelp]

Oh, so you can't disable inlay hints in server, only in client. I thought this setting was for the server.

[midnight isle]

Strange. I should play a bit in my neovim tomorrow.

[heady kelp]

Now the unspaced field (Variable) does show up, but when I highlight it, it still doesn't have the space added, but when I do select it, it not only adds space, it also removes the leading field name. A bug?

Improved

Hmm, if I type the variable name it won't show it, but if I first trigger autocompletion dropdown menu then I can see it and search/filter it.

Fixed

Double forward slash.

Improved

Awesome! Will check it out later.

[heady kelp]

In this case, just typing y by itself triggers the menu.

Isn't it expected? See the attached Figure 1.

Yes. It was a comparison (working vs. not working).

[heady kelp]

After opening the file it doesn't show the error unless I save the file (no-op).

vscode invokes lsp's focus command but neovim doesn't. should add an issue, and discuss a suitable behavior to change focus file if the client is neovim.

An issue to what project? In other LSP servers initial diagnostics always show up by themself just by opening a file. So it should be an issue with tinymist, I think.

[heady kelp]

Field selection doesn't seem to work.

I need an approach to invoke editor.action.triggerSuggest and editor.action.triggerParameterHints at the same time, after a field completion, but I don't find a way other than letting client provide an extra commands, https://github.com/Myriad-Dreamin/tinymist/blob/f167ad152449c631e5946187efd2ba1772612f02/editors/vscode/src/extension.ts#L161. This is because https://github.com/microsoft/language-server-protocol/issues/1117.

Aren't editor.action.triggerSuggest and editor.action.triggerParameterHints specific to VS Code or are they defined in the LSP spec?

I looked at rust-analyzer, and find that the parameter hints (signature help) keeps until exiting a function autocompletion. That is, if tinymist does same thing as ra, when I type text, tinymist will complete text(${content-arg}, fill: ${fill-arg}, stroke: ${stroke-arg}, ..), and I will continue finishing my function call for tabs and go to ${content-arg}, ${fill-arg}, and so on. however it is probably not what we would like to have, as we usually write text(fill: blue) and then add a content arguments by []. I may look at python lsps then.

I am not sure if any of the other LSP servers use snippets/templates, but it seems that tinymist is the only one that does. Normally, when I type a function name it will show it in the list from where I can select it. After that there is a client setting (IIRC) that automatically adds parenthesis when function was selected. After that at least with some servers the signature help indeed shows up and stays (I don't remember the exact behavior of other servers).

[heady kelp]

How do you specify the argument types?

tinymist doesn't look your js-doc style comments. that's not typst style and hard to handle.

But I remember that in some screenshot it did say that the variable's or function argument's type was int. I can't recreate that. I only have inlay hints that only work with function signature (and not with variables, like rust-analyzer). Is this possible or did I misremember?

[midnight isle]

undoubtedly add an issue to tinymist. Originally, typst-lsp actively changes entry (main file) when any didOpen or didChange event is received from client. But It is just weird to change entry by didOpen event, as some editor may hold the file (keep open) and not emit another didOpen event when you switch back from other file. To fix that, I was letting vscode client keep updating its activated document (the one user focusing) by triggering focus commands to drive entry changes. This becomes broken as we now haven't done anything at neovim's client side, which I forgot to consider. I am thinking of when I can do better otherwise I 'll back to typst-lsp's behavior when no focus command is invoked.

[midnight isle]

editor actions is vscode specific, however neovim starts to support it since 0.9. these two commands are also made used by rust analyzer.

[midnight isle]

The weird is that I find when I put a character after adding parentheses, the signature help doesn't stay. I observed ra's signature help, it is gone after the completion started by parentheses finished. I may learn some knowledge from python lsps processing their completion on keyed arguments.

[midnight isle]

These types depend on what you write in code. But I haven't checked pattern like type(x) == y so you got any type. I can add the pattern since it is easy.

Note It is also not quite necessary to hint type everywhere for lsp by assertions, as it slightly degrades performance. You can report any imperfect of annotated type as well. The current type checker is actually simple and missing a lot of things tho.

[heady kelp]

I mean, if someone already used this plugin (for Rust) then it just works with tinymist. But either way, it wouldn't hurt to add this info to the documentation. For the unreleased v0.10.0 it should work out of the box.

[heady kelp]

There has to be a non-client-dependent way of doing that. It should nicely integrate with many popular editors by itself, without writing specific things for vscode or neovim. It adds an unnecessary burden on the maintainer (you).

[midnight isle]

Oh, I think I know one of your old report. That, you found some diagnostics were missing. I didn't catch possible cause. Now I'm not sure but isn't that caused by weird entry update by didOpen events? It was a long time ago that I just used same strategy as typst-lsp to update entry.

[midnight isle]

I know it, we should always prefer to not assume anything and any client and try to bring every feature out of box. I just get used to analyze our situation client by client.

[heady kelp]

Oh, yeah, I think Python is one example where it does work. I even checked just now (pyright). Well, it kinda works. It still disappears when writing an argument and only appears after pressing ( or ,/, . It changes the selected argument, but it's not useful, at least with named arguments. But it will be 100% useful for positional arguments.

[heady kelp]

Please do!) It will definitely enhance the UX (DX).

[heady kelp]

Sure. I was just checking if assertion will affect the showing type. It does in Python, IIRC. I have assertions in my project for the end user (readable and helpful error messages).

[midnight isle]

Perhaps that will be our best experience without help of clients. Ideally I personally want to keep showing signature when I'm making on some function call, as its blinking annoys me.

[heady kelp]

I don't remember. The only relative thing that I know of is this: https://github.com/typst/typst/issues/2633 (and this https://github.com/DioxusLabs/dioxus/issues/2312).

[midnight isle]

Possibly fix: https://github.com/Myriad-Dreamin/tinymist/actions/runs/9012413043

[foggy sorrel]

@midnight isle I think I've asked this before, but is it possible to have subpixel rendering in typst preview?

in general I just don't find the text rendering to be very pleasing

[foggy sorrel]

for some reason "label" isn't shown as an alternative for autocompletion

[foggy sorrel]

@stable turret can we have indentation in equations in typstyle?

it's a lot more readable as

$
  m(xi) =
$

[heady kelp]

This is not a typstyle thread. Normally it goes to #off-topic.

[foggy sorrel]

This is not a typstyle thread
it's integrated in tinymist

[heady kelp]

Which means it's indirectly related. And the question itself isn't an integration question, therefore it's #off-topic.

[foggy sorrel]

no

As in, all these projects are so closely intertwined that it doesn't make sense to split it up

[heady kelp]

Not as close as you think. And the question isn't about "close intertwinement", so I stay by my statement above.

[foggy sorrel]

Fine, but I'll still leave it here

[grim berry]

What's the font here?

[foggy sorrel]

comic code

[stable turret]

it was indented (incorrectly) before v0.11.10 - [2024-04-02]. and i set it not indented in that version. i cannot remember exactly why but i think it's because i find some tricky case that is hard to handle. but maybe it's already fixed. i will revisit this.

about the indentation: i think the readability of them looks similar. math blocks typically cannot be nested so i think it would be good to not indent them, to save horizontal space. i'm open to this and i will see how latex formatters format math equations

[foggy sorrel]

I can assure you that the indentation helps a lot when you have hundreds of equations in a document

[stable turret]

i'm not knowledgable about this. currently we simply use svgs and canvas so i think advanced text rendering is missing. like hinting.

[foggy sorrel]

the closest thing latex has to a formatter is latexindent, which is extremely non-opinionated (just about anything is configurable)

[midnight isle]

the type checker hasn't checked number-mode in ("line", "label") pattern.

[untold elbow]

This is some weird syntax highlighting

add a space before the . and you don't have this blue highlighting

[dawn chasm]

Isn't this big news for you @midnight isle https://www.reddit.com/r/rust/comments/1colxqb/using_webassembly_for_extension_development_vs/?

[heady kelp]

I can confirm that at least in a current project with the latest version now the #includes also work with go to definition (also with spaces). Perfect.

[midnight isle]

we prefer to use native binary to reach full 🚀 performance. It probably helps web (browser) extension, but we have webapp...

[midnight isle]

Will take a look 😉

[stable turret]

https://github.com/Enter-tainer/typstyle/pull/66

[dawn chasm]

Again, I know so little about this. I'm happy that you're writing it in "binary". I thought you were writing typescript/javascript 😛

[midnight isle]

recognized.

[foggy sorrel]

How does this work? Surely you're not manually adding them

[midnight isle]

by type checking. The underlying techniques are not easy to understand, but I can provide some links here:

1. bidirectional type checking: https://jaked.org/blog/2021-09-15-Reconstructing-TypeScript-part-1
2. type system borrowed here: https://github.com/hkust-taco/mlscript

[foggy sorrel]

Ok so you changed the algorithm to improve detection for this function?

[midnight isle]

yes. The algorithm should recognize containing relationship (a in b / a not in b) to infer the type of number-mode. I implemented some simple cases of containing relationship to get it work.

[dawn chasm]

This is so cool.

[midnight isle]

type(..) works. It also works with combining in operation. I don't really check the logical expression tho. The more complex and accurate type inference should be done after I taking a rest.

[foggy sorrel]

What about for typst preview? Could you have a lot more control over rendering that way?

@stable turret

[stable turret]

about the performance: i'm discussing a new design for typst-preview with @midnight isle mainly to solve performance problem in browsers. Also, most part of computering already happens in native binary. only the renderer runs in wasm.
about rendering: i havent think of this before. afaik high quality glyph rendering using subpixel aa and hinting requires os-specific features. like directtext on windows, coretext on macos, etc. i think it would not happen in near future.

maybe we can see how zed render its texts among all these platforms.

https://github.com/RazrFalcon/tiny-skia/issues/1

okay... number 1 issue in tiny skia

[foggy sorrel]

You can still do better than currently, even without subpixel aa. Text rendering is quite a bit better in the web app than in typst preview, even if neither of them support subpixel aa

I would create a comparison, but the web app isn't working at the moment....

presumably you don't necessarily need os-specific features if you're rendering to an image and not relying on text rendering from the os? though I'm not sure what the performance would be like...

[foggy sorrel]

People mention https://github.com/dfrg/swash in the comments to that issue

and https://github.com/notgull/piet-tiny-skia

[stable turret]

i dont get why webapp is better. is it because tinyskia is better than browser canvas/svg path ?

[stable turret]

okay. let's wait it back online😇

[foggy sorrel]

I think Laurenz said something about this at some point

[foggy sorrel]

#discussions message

You were even participating 😂

[stable turret]

oh i completely forget that.

i'm a little bit concerned since it seems that it need the full font data to run. but the impl seems only takes the outline path of the glyph

i think it is possible to use it in preview. thank you for letting me know this!

[stable turret]

another take is that we are switching to canvas based rendering and deprecating svg. so subpixel rendering becomes possible now.

[foggy sorrel]

I thought you already used png

[manic shadow]

I'm in a contributing mood, anything you want to draw my attention to? ❤️

[midnight isle]

welcome~ @karmic bear is working on tinymist/crates/tinymist, the lsp framework. I'm refactoring and documenting tinymist/crates/tinymist-query and an overview document of tinymist. Many work requires some context to understand and will be documented in the overview document. But I think there is some good start points to contribute for you, that requires little context. That is improving the lsp formatting. https://github.com/Myriad-Dreamin/tinymist/blob/fff227f3aed38dacf671532d9a1daf8650d72904/crates/tinymist/src/actor/format.rs three points are here:

• read typstfmt config from file system.
• poolize the format threads, like the rust analyzer does.
• support general range formatter api based on existing formatters. I remember some neovim format plugin has provided a such function.
  You can start to improve formatting right now or wait for my overview document. Happy coding 🐱

[karmic bear]

I think we can postpone pooling. Once the migration to async lsp is done, we can easily utilize the existing async runtime.

[sand plinth]

Just a nit, but boolean | false seems redundant

[midnight isle]

The enumerated values are distinguished from their types. Like: str | "enum1" | "enum2" | ...
Idea? I think at least boolean | false is really redundant.

[stable turret]

that's black magic🔮 we literally have type(typst)script now

[midnight isle]

typ(e)s(crip)t

@heady kelp we may have module.d.typ to type parameters. (joking)

[foggy sorrel]

A citation browser would be nice, but I think that has been mentioned before

[midnight isle]

this was mentioned. We may have a label browser and filters by bib/files/referenced

[foggy sorrel]

Another idea: Table editor @midnight isle

[untold elbow]

yay let's recreate a jabref concurrent as a side project

[foggy sorrel]

I meant something much simpler. Just a list of references in the bib file, and some indication of which ones have been referenced

Though a full editor is an idea for the far-flung future I guess 😛

[dawn chasm]

I mean these guys can do anything.... I'd just ask for it I guess. 😛

[heady kelp]

It should be val: boolean = false. Like in any other language where you can add a default value (Python).

[untold elbow]

Yeah ...
I honestly wonder for these things if having more integration isn't better than a separate application

[midnight isle]

even if we have power and bandwidth to make a full editor, it is not a good idea to make it full. First, There should be too many details to tune for nice experience, and those are not typst specific. Second, it takes up my time to be a charm typst amateur 😇

[midnight isle]

Having had the idea. We can attach tables with a codelens that call a visualized table editor.

[upbeat pike]

would be nice if you could also edit existing tables

so not just for table creation

[midnight isle]

A code lens is used for floating some code action on some existing syntax unit.

[upbeat pike]

ahh okay

[slate smelt]

The lsp command to compile the file to pdf in helix is not working for me anymore... any tips on that?

[midnight isle]

The all lsp commands don't work in helix, unfortunately, as you cannot invoke commands requiring arguments in helix. you should use typst-cli to compile your file to pdf instead.

[slate smelt]

Thanks

[foggy sorrel]

that sounds like an odd restriction

[midnight isle]

We've tried to overcome this but we have no good idea on playing lsp commands or alternatives in helix.

[midnight isle]

That's real :(. https://github.com/helix-editor/helix/discussions/3933
Perhaps helix's extension for typst should bind some commands for helix users, as suggested.

[foggy sorrel]

By the way, is there some workaround to get the symbol view to show up?

or is it just broken?

[midnight isle]

possbly fixed by https://github.com/Myriad-Dreamin/tinymist/pull/269. You can download the lastest built extension on github action to check it or just wait for next release, https://github.com/Myriad-Dreamin/tinymist/actions/runs/9077951956.

[foggy sorrel]

it's working now, nice 😄

lol

[midnight isle]

I know it is better to show some visible hints, but let's make it perfect step by step..

[foggy sorrel]

I wasn't complaining, just found it funny 😂

[midnight isle]

Also this should be "Control Characters or Spaces" probably. Poor English here.

[foggy sorrel]

Controls and Spaces sounds good to me

Control characters is very long

but I'm not a native speaker either

[heady kelp]

There is probably a standard name for them which should be used.

[midnight isle]

There should be some symbol taxonomists to help us categorize symbols in wild (misc).

[heady kelp]

Are these the first 32 chars from ASCII?

https://www.ascii-code.com/

[midnight isle]

not, they are zwj, zwnj, zws, hairsp, thinsp, etc.

[heady kelp]

The Unicode Standard, Version 15.0:
p. 74 General Structure > 2.9 Details of Allocation > Plane 0 (BMP) > Figure 2-14. Allocation on the BMP
p. 291 Writing Systems and Punctuation > 6.2 General Punctuation > Blocks Devoted to Punctuation

"Format Control Characters" and "Space Characters"

https://www.unicode.org/charts/ > Layout Controls:

Spaces [...] Format characters

So "Format and space characters" or something.

[midnight isle]

Thanks. to shorten format control and space characters, "controls and spaces" looks shorter and better than "format and space characters", and doesn't mislead us. But I doubt whether "controls" is formal. I perhaps pick format and space characters.

[heady kelp]

Controls is used in a wrong way, because you supposed to say "control characters". So the "control" contraction doesn't make sense (it's not a game, there are no "controls"). It is more noticeable if you say "formats" instead of "format characters" (it's not a file, there are no "formats").

Besides, the 5 squares in a row are way wider than the current heading, so making it longer shouldn't be a problem, IMO.

You can always shorten "characters" to "chars", which is a common thing, at least in the programming world (char type, isn't that right, C?).

[foggy sorrel]

eh..

They are characters that control

I don't see any issue with Controls

[foggy sorrel]

Full words are better than abbrevations

[heady kelp]

Says who? #1216655435663409152 message

[foggy sorrel]

I'm allowed to have an opinion, just like you

so chill

[heady kelp]

I'm saying that you first say that full form is too long, and I address that, now you are saying that full form is better. And I didn't insist on shorten version either.

[heady kelp]

tsserver also can properly highlight positional arguments in the signature help floating window.

[heady kelp]

Can we have more semantic tokens? @lsp.type.pol.typst Is almost on everything, which significantly limits the way you can color the code.

I should be able to copy this:

Ok, bad example...

Actually, not entirely. The . should be white, while ^ or := is cyan. Currently, they are all the same.

At least there should be a module token or something similar instead of pol.

What does pol mean? And why := in math mode is escape? Shouldn't it only be applied to \?

I can later show the tree-sitter/semantic token mappings that I made to mimic the default Typst coloring as much as possible.

But the module token is one of the most obvious that is missing.

[heady kelp]

How does tinymist executes typstfmt when formatterMode = "typstfmt"?

It doesn't see either local or global typstfmt config. Moreover, it has formatterPrintWidth which will invalidate any setting from the config file. This setting should at least have "auto" by default or something.

And for some reason with "Default: 120" it mashes everything together in a thin line, which means that it probably uses 50 lines instead (I think it is or was the default value in the typstfmt). This means that documentation is misleading.

Also I don't get the compileStatus:

Since neovim and helix don't have a such feature, it is disabled by default at the language server lebel.

Default: "enable"

Is it disabled by default or not?

I'm pretty sure rust-analyzer has this, and you can show this in Neovim. So it at least should be configurable otherwise why there is an option that is useless?

[midnight isle]

We cannot determine whether an identifier is a module instance without running the code in some cases, so they are all assumed to be an (interpolating) variable, or pol.

[midnight isle]

Enabled in vscode but disabled in other clients.

Should generate different default value in documentation for different clients.

[heady kelp]

So you can't even hardcode sym, emoji and some other core modules?

Does that mean that LSP server can't run code to determine some additional semantic tokens?

[heady kelp]

Where is this condition? I want to test this.

[midnight isle]

Could it show a status bar item in neovim? How does ra's compileStatus look like? tinymist's compileStatus object contains a boolean to indicate success and multiple numbers about word counts.

[heady kelp]

Then maybe it's different.

[midnight isle]

just checked, typstfmt in tinymist uses the 120 width with default config, if there is no bug.

[midnight isle]

I was thinking about a good fallback strategy like eslint, that a typstfmt config in ancestor directories is used. I have not done this since I want to find a way to read it with cache correctly, while typst-lsp actually reads it from fs per formatting request.

[midnight isle]

hardcoded highlight works on core modules, but it also introduces false positives. We can run code but that means we will run a compilation for each identifier for correct highlighting 💀. We can perhaps have that highlighting if our analyzer is fast enough.

[midnight isle]

If I get it correctly, it is the work progress feature which is in lsp standard.

[heady kelp]

1. We don't have to run a compilation for each identifier to make them available, only for a subset of pol ones.
2. Won't the sent semantic tokens be cached on the client? Can't they be cached on the server too?

[heady kelp]

Ok.

[midnight isle]

Even though a subset, you might go crazy someday when there are too many identifiers to analyze in your document.
The sema tokens themselves are already cached for delta rendering but not for faster computing. They are recalculated on source changed and your each key stroke makes a source change.

[midnight isle]

If you would like to, we can rework on design on semantic token kind. It is inherited from typst-lsp and might already be used by many people. It is okay to break them once but we are not better to break them for times.

[heady kelp]

yeah

but the problem is that this one (:=) is incorrect, it's not just any breaking change.

[midnight isle]

Maybe nvarner just wanted to make shorthands having same color as escape. I also like that, but it is semantic invalid as you said. I think another kind of semantic token could be introduced for shorthands. We can introduce more, so we can think of them all before starting.
https://github.com/nvarner/typst-lsp/blob/d76054776d50bade665b82d1f0308615f11d2580/src/server/semantic_tokens/mod.rs#L176

[heady kelp]

Does tinymist support range formatting?

[midnight isle]

https://github.com/Enter-tainer/typstyle/issues/47

[heady kelp]

I think it doesn't have to be module-only semantic token. It looks like if you use #var pattern, it always colors purple.

So it's variable/module access.

[heady kelp]

I think they all supposed to be identified as functions.

[heady kelp]

What do other editors produce?

#import "@preview/fletcher:0.4.3": *
#let node(position, ..args) = {}
#let e1 = (0, 0)
#let e2 = (1, 0)
#let e3 = (2, 0)
#diagram({
  node(e1)[A ]
  node(e2)[B ]
  node(e3)[C ]
})

[midnight isle]

Same. I think I have disallowed it to show before content arguments.

[heady kelp]

For table it doesn't add sink inlay hint, but for custom functions it does.

[slate smelt]

How the formatter line length can be set to disabled?

[foggy sorrel]

Which formatter? Tinymist has two

[slate smelt]

Typstyle

[foggy sorrel]

It has a cli option for column width, but I don't know if it's adjustable through tinymist. @stable turret or @midnight isle would know

[slate smelt]

Thanks!

[midnight isle]

I've forgotten the name XxxWidth but you can see the documentation to learn it.

[slate smelt]

I saw this, but couldnt understand how this can be disabled...

[midnight isle]

by a large enough number? @stable turret

[stable turret]

yes

may i know why you want to disable it? bad case?

[foggy sorrel]

Does the width limit apply to text, or only scripting @stable turret ?

[stable turret]

scripting

[slate smelt]

Beacuse in the past I used to have files with long text for definitions something like:

/ example: #lorem(100)

and the formatter would break this to few lines, this can happend here?

[stable turret]

lorem(100) means actual text?

[slate smelt]

yeah

[stable turret]

is there an example? i think at least the part before lorem will never be breaked into multiple lines

[slate smelt]

/ Statistical Thermodynamics: (also known as Statistical Mechanics), emerging in the late 19th and early 20th centuries, complements classical thermodynamics by explaining macroscopic properties through microscopic interactions among atoms and molecules. It links individual particle behavior with observable bulk properties, incorporating insights from classical mechanics and quantum theory, notably influenced by the Bohr model.

[slate smelt]

BTW how the package can be installed using cargo?

[stable turret]

maybe simply cargo install?

[stable turret]

https://enter-tainer.github.io/typstyle/ you can also try it online

it looks like it will not be breaked into lines?(at least for this example) i'm not at computer at this moment. i test it using the online one.

[slate smelt]

Thanks!

[stable turret]

wait! i still wonder what is the underlying bug😭

[slate smelt]

what bug?

[stable turret]

the "break term list into multiple lines" bug

it shouldnt happen, and i failed to reproduce it at this moment.

[slate smelt]

It happend to me before with other formatter, not yours
I didnt tried yours yet

[stable turret]

oh that explains it. i got it👍

[slate smelt]

Sorry for that 😐
Maybe adding install instructions in the readme with cargo can be good

[stable turret]

updated https://github.com/Enter-tainer/typstyle#installation

[dawn chasm]

Awesome! Is it strictly necessary to define a column-width? Is there any way to make that just be.. untouched?

[stable turret]

There is a default for this though(120 for tinymist, 80 for typstyle-cli).
It is needed, if column width is unlimited, the formatter will tend to put complex nested function call/array/dict into a single line. (As stated, column width is unlimited 😜 )

[dawn chasm]

Interesting. Good to know!

[slate smelt]

Amazing thanks!

[grave moat]

I am using neovim and the exportpdf on save doesn't work anymore

here is my config

require 'lspconfig'.tinymist.setup {
  on_attach = on_attach,
  capabilities = capabilities,
  single_file_support = true,
  settings = {
    exportPdf = "onSave", -- Choose onType, onSave or never.
    -- serverPath = "" -- Normally, there is no need to uncomment it.
  }
}

[midnight isle]

You can downgrade the version first, since there is not remarkable feature but bug fixes recently. I'll pin you once I've found something.

[grave moat]

I don't know which version I used originally. It's been quite some time since the last time I updated.

As a workaround, I just run typst watch in another terminal.

also, it seems that snippets don't work anymore, in the sense that when it doesn't autocomplete the parameters when I call a function

[heady kelp]

Can you share your Neovim and tinymist version from the screenshot?

I remember that before I had the signature+description help window popup when entering (), but now it doesn't work in 0.11.4 up to latest. I tried Neovim 0.9.4 and 0.10.0

What I really miss is that starting from 0.11.9 after autocompleting a function it stops showing autocomplete dropdown menu.
Previously I can open file, then write #tabl, select table and then write fil and select fill. Now I have to re-trigger the autocompletion menu manually.

So the problem is with so snippet or something. If I use table() snippet, then it removes the menu.

[midnight isle]

I suspended notify feature doesn't work, but I had no idea to verify and test it.

@heady kelp. I have been freed from my busy life, now I'm going to make correct hooks to trigger completion and signature help totally, https://github.com/Myriad-Dreamin/tinymist/issues/293.

I remember that before I had the signature+description help window popup when entering (), but now it doesn't work in 0.11.4 up to latest.

The related code haven't changed for a long time. It should work and I tested it in neovim with tinymist 0.11.10 and still works will.

Now I have to re-trigger the autocompletion menu manually.

Same.

There might be some weird bug, I see 4 reports about no response from lsp till now, but I haven't received any concrete (reproducible) report from neovim users :(.

[heady kelp]

Hopefully I will be able to cook up some Docker containers later this month.

[dawn chasm]

It is really amazing work you guys are doing...

Every time I contribute to my project, and I see how slow I'm to make new features, I'm always amazed at the speed tinymist is going.

[midnight isle]

some devcontainer for neovim? 🔥

[heady kelp]

Yes.

[midnight isle]

interesting

[heady kelp]

That's where I start flexing that I can easily whip up a Neovim dockerized environment for reproducible builds unlike those GUI editors/IDEs or whatever.

You just have to install Docker, Docker Compose (and just).

[vale tiger]

Why is just necessary?

[heady kelp]

for your comfort of just typing just to run the container.

Or it's j for me.

[vale tiger]

And if I'm more comfortable just typing docker compose up?

[heady kelp]

Then do that.

It's not mandatory, but too useful to be optional at this day and age. Otherwise you would have to manually copy whatever the "default" recipe is doing.

[slate smelt]

There is a problem I think when trying to install tinymist using cargo

This is the output:
multiple packages with binaries found: tinymist, typst-preview. When installing a git repository, cargo will always search the entire repo for any Cargo.toml.
Please specify a package, e.g. cargo install --git https://github.com/Myriad-Dreamin/tinymist tinymist.

[midnight isle]

It says you should specify a binary to install, which is the solution.

[slate smelt]

I tried but it failed to compile

[midnight isle]

Does this work?

cargo install --git https://github.com/Myriad-Dreamin/tinymist tinymist --no-default-features

I should publish a tinymist-assets for you, but I didn't make it.

[slate smelt]

It would be fixed in the future?

[midnight isle]

would be fixed soon, during this week.

[slate smelt]

thanks a lot!!

[midnight isle]

I tried it, this following command install a nightly version:

cargo install --git https://github.com/Myriad-Dreamin/tinymist tinymist --no-default-features --features cli

    Finished `release` profile [optimized + debuginfo] target(s) in 3m 55s
  Installing tinymist.exe
   Installed package `tinymist v0.11.11 (https://github.com/Myriad-Dreamin/tinymist#bb9f30fc)` (executable `tinymist.exe`)

[slate smelt]

Also fails, just letting you

[midnight isle]

💀

[slate smelt]

I would try right now
Thanks!!

[midnight isle]

sorry it is broken.

[slate smelt]

I am sorry for bringing this up

[midnight isle]

We are managing to merge typst-preview into tinymist, and something didn't set up correctly.

[blissful merlin]

Is the typst-preview and tinymist merge not enabled yet on 11.11?

[foggy sorrel]

I mean, the commit happened after the release of 0.11.11

so yes

[blissful merlin]

Ah yeah right I didn’t notice that is was merged aftwards

Was following the PR but didn’t notice when the bump was committed

[midnight isle]

It is intended, to bring more time window to move things. The first step is just moving the code and no code change is made. The second step is just putting the preview command in a same binary. Further internal merge may happen step by step across several future minor/patch versions.

[dawn chasm]

https://tenor.com/view/exactly-as-i-have-foreseen-star-wars-everything-is-proceeding-gif-15796872

[proven walrus]

Hello, I have been using Tinymist and it has been awesome !

I ran into this problem of big file recently where tinymist became unresponsive and slow (it has been growing because I write everything in it). I split it into multiple files and it was fine, except for references.

I quote stuff across the internet and books. I wrote a helper function that basically links to a label, where that label is added at the end of the document in the template, where the content of each quote is shown to the reader. Spliting into files made it impossible for tinymist to find some references because they are not added locally (i.e. in the template), there are two cases.

quotes

I rewrote my quote system yesterday using the state type (monad?). Instead of inserting quotes in a yml file included by the template in the end of the document, I made it possible to mutate the state (appending to the list of quotes in the state), and cite it directly. This made it possible to insert the quote locally while citing it, so the referencing-another-file issue is gone. I was able to reference quote inserted in another subfile without tinymist complaining, oddly.

(^ This mechanism is inspired by footnote, you can write the footnote locally but it would only show at the page foot. Here I made the same thing but instead of page foot it's the end of the document. )

bibliography

I could just not use bibliography, actually. But I did because I wanted to try it out. The problem is that it only reads from an external file, I can't no longer do the local insertion when stuff refers to it, and tinymist can't find them currently.

What would be a good way to solve this (if the problem is not between the keyboard and the chair to begin with) ?
Thanks :)

[midnight isle]

where tinymist became unresponsive and slow

How many pages does it have? I expect a smooth experience will have if you keep for about < 50 pages, an acceptable experience will have if you keep for about < 100 pages, otherwise we think of it can be slow currently. This also depends on the complexity of show/set/query in your document. We may ensure performance on larger document in future.

What would be a good way to solve this (if the problem is not between the keyboard and the chair to begin with) ?
Thanks 🙂

I think it is some more general question beyond lsp, and you can go #1088372909111783525, search existing posts, or open a new post and ask for more people to join this discussion.

[proven walrus]

How many pages does it have? I expect a smooth experience will have if you keep for about < 50 pages, an acceptable experience will have if you keep for about < 100 pages, otherwise we think of it can be slow currently. This also depends on the complexity of show/set/query in your document. We may ensure performance on larger document in future.
The compiled document has like 210 pages lol, that explains it. I'm looking forward to see the improvements in the future !

Thanks :)

[foggy sorrel]

@midnight isle the multi threading on main should help quite a bit right?

[midnight isle]

It can help a lot. Another big blocker is handling lsp requests concurrently.

, which is working on.

[heady kelp]

It looks like the LSP features don't work (or not all of them) if opening a symlink to an outside file.

Like I have a few setting files between projects and in them practically nothing work or almost nothing.

What's even more concerning is that the LSP client in many cases don't even see some words that a in the same file. So I don't know who is to blame.

In theory server shouldn't change other autocompletion providers, but it almost feels like it does.

[proven walrus]

Though in another scenario where I have to split the files, can I still somehow make tinymist aware of the main file and make references/links work ?

[midnight isle]

see readme in editors/

[midnight isle]

i noticed it. It is hard to keep testimg function working well on symlink files so i don't prioritize it currently. In principle, you should not link sourcr files into workspace but only link a packages to @local or @preview.

[midnight isle]

When in markup mode, do you observe that it doesn't change? If so, server does leave completion on plain content (by design) for other providers.

[heady kelp]

I can instead use absolute path to the same file, but this requires setting a specific root directory. I don't remember if the server provides such action. This would be much harder than just pinning the main file and that's it.

It's just because I'm using a complex multi-document structure. Which is a rare case.

[heady kelp]

Sometimes in markup mode I only see some in-file keywords. LSP server didn't show any autocompletions. And even after adding something like #let keyword = ... in the same file, it then (LSP client) didn't show it in the autocompletion menu. Very weird and annoying. But I think that it is also hard to reproduce.

[midnight isle]

why do variables in code (script) involve markup mode? I mean when we are writing the real content of the article, it should not make completion but leave for lsp clients for completing lorem, ipsum, or other words.

lorem ipsum |// your cursor is here and you are writing lorem(N)

[heady kelp]

not variables, I pretty sure any keywords.

[midnight isle]

oh, you mean you just hint the client that hey you can complete a keyword for me?

[heady kelp]

I just write siehanscthea scithe anstieh word aisnsiteha cniseta insetnisteah and then wo and trigger autocompletion. It wouldn't show the word. While normally it always does (when not editing Typst projects or if they are small).

[midnight isle]

ok, in neovim, I can see it doesn't show the word

[heady kelp]

you do?

It's hard to reproduce and pretty much all simple examples will work as expected:

[midnight isle]

god, I reopen the file and it starts to complete word 🙂

I cannot reproduce it

There should be some points I must catch.

[heady kelp]

One logical explanation would be that there is some variable/constant which is responsible for the amount of keywords that can be remembered and autocompleted or how big the file is.

So like maybe file contains too much keywords or multiple opened files do combined.

But I haven't heard about it and it would be weird if such restrictions are being applied by default.

[heady kelp]

Like here is a simple example where everything is kinda working, but server doesn't show anything in the autocompletion list:

It knows the type at that point, so it supposed to show dictionary's methods. Normally it would do that, but in this situation it doesn't.

Restarting and pinning the main results in the same thing.

ok, it doesn't autocomplete anything.

I can only choose items provided by the LSP client.

[heady kelp]

Oh, right. It's probably because of symlink:

But for me as a user it's weird that in-place errors show up, but autocompletion — doesn't.

[heady kelp]

go to definition works into symlink files, but not from symlink files.

I'm surprised that LSP treats symlinks differently from normal dirs/files. Or does this depend on a specific server?

[heady kelp]

I have something interesting. If I open a file that is in the current project, which is a symlink to a file outside of the current Typst root/project dir, then things like "go to definittion" and LSP-based autocompletion work.
But if I open main file and pin it, then that first symlinked file will lose the (said) LSP features, but will gain the (only?) feature of relevant/actual in-file error diagnostics (because the correct main file was pinned).

So in this case the question is what you need/value more: correct errors on everything else. If, for example, I have some references to insert in the main file, then I have to pin it. So the quetion also expands to: in which file which features do you need more?

[heady kelp]

Why continue is a snippet that doesn't do anything and not just a keyword of some sort?

[heady kelp]

I think this is already documented, but at least some items are being repeated...

[midnight isle]

a field completion still uses typst-ide/lsp's implementation yet.

[midnight isle]

this explains why you lose lsp feature in the file, since it is imported by the linked path by pinning file. when you open the file it as the origin path, lsp think it is unrelated to the pinning file, since lsp doesn't follow links.

[midnight isle]

how do you trigger it?

note if you would like to send some issues to be fixed, you may at least create an issue on github and attach a link to some discord message. I don't really take responsibility to collect all issues from internet (and here). i will just fix it when i remember it.

[heady kelp]

Hmm, I don't think I actually can open the non-origin path. But it does kinda make sense. Then why it would still show errors in that file?

[heady kelp]

a normal autocompletion trigger.

Even without pressing anything special:

[midnight isle]

the screenshot only shows a dot, so we can only judge that there are actually some repeated items.

[heady kelp]

same #().

[midnight isle]

since lsp client follows links 😂? i just guess it.

[heady kelp]

I don't know how to categorize them/how to name them. Only if "feature x doesn't work" or something. And I don't always have MRE.

[midnight isle]

I will check it.

[heady kelp]

So LSP completion and error diagnostics are somehow separate. You would probably know what separates them. maybe.
Maybe the path is somehow supplied differently?

[midnight isle]

you can even named it "unnamed bug from discord message". we can name a title latter or ask gpt. i think this would encourage "lazy" people to send issues.

[heady kelp]

okey dokey

[midnight isle]

first, server doesn't follow links (or canonicalize paths) by design. i asked laurenz.
next, lsp completion doesn't work when some file is pinned, as the origin path is used by server and server doesn't follow link. in comparison, server pushes diagnostics with linked paths but client follows links and report an error in your buffer.

i think some smart improvement can still be made when it handles lsp requests, but i have no nice idea to make fast one, as canonicalization is very expensive.

[heady kelp]

https://github.com/Myriad-Dreamin/tinymist/issues/335

[heady kelp]

Ahh, so it's because the client is responsible for handling the path where to show diagnostics...

So... the solution would be to...hmm, if the client can't specifically open the file at the symlinked location, then it's kinda impossible to fix... at least without any crazy ideas.

[midnight isle]

could you go to the linked directory and edit them?

[heady kelp]

yes

https://github.com/Myriad-Dreamin/tinymist/issues/336

ohhh, I think I got it...

kind of...

It looks like both nvim-tree/nvim-tree.lua and ThePrimeagen/harpoon use absolute path at least for the symlinked path at least by default.

But I can manually one relatively file from the symlinked dir and then it will focus symlinked file in the nvim-tree. But harpoon resolve the symlink path before saving it. But I can cut of/edit the entry manually.

So it looks like on the surface the client sends the in-project file path. And yet after pinning the main file the LSP in the symlinked file stops working once again...

[midnight isle]

@heady kelp I find it, maybe someone can remind me to push a same fix to typst/typst after I fixed the duplicated entries on accessing objects.

[heady kelp]

I can create an issue there.

Is is in the typst-ide?

[midnight isle]

correct

[heady kelp]

But how did you fixed it if you are depend on the not-yet-fixed typst-ide? Can you somehow override stuff or something?

In the commit you just removed one function. Which is weird as it is not referenced?

[midnight isle]

I copied the code

[heady kelp]

You mean the whole typst-ide or some parts of it?

[midnight isle]

I copied the whole typst-ide, to use/modify private functions in the crate.

[heady kelp]

daang

ok

communism all the way

[midnight isle]

It is still easy to diff and send patches to typst/typst, so I think it is okay.

Also there is no license issue.

[dawn chasm]

All of this work, is it solely confined to one repository? the tinymist one? or do you work on multiple repositories simultanously?

[midnight isle]

only tinymist

[blissful merlin]

Now that tinymist was updated to 0.11.12, do we still need the typst preview extension? I don’t see a way to show the preview without it

[midnight isle]

not yet. we need time.

preview command is now available by command, meaning that we are ready for replacing binary:

• Moved typst-preview to tinymist and combined the binary and compiler in https://github.com/Myriad-Dreamin/tinymist/pull/323, https://github.com/Myriad-Dreamin/tinymist/pull/332, and https://github.com/Myriad-Dreamin/tinymist/pull/337

cargo run --bin tinymist -- preview main.typ

Note: However, the built typst-preview extension along with tinymist is not tested. We can do it in next round of developing.

[dawn chasm]

Awesome.

[blissful merlin]

Very cool, looking forward to it! Awesome work by you both!

[midnight isle]

The last PR to merge preview: https://github.com/Myriad-Dreamin/tinymist/pull/368

[dawn chasm]

That's so awesome.

In the r world, Positron has been pre-released. It is an ide built in vs code technology. Tiny mist plus positron could be an insane combo...

[lethal wadi]

Looks exciting, I'm looking forward to this a lot

[blissful merlin]

Awesome!

[slate smelt]

How I can use typst preview on vscode?

[dawn chasm]

It must be in next release.. the PR was merged ❤️

[slate smelt]

Thanks!

[heady kelp]

Can confirm that it now compiles without any issue and has preview subcommand.

[little cove]

Just for me to understand what I have to do in the next release

Is typst-preview obsolete now?

(as far as I understand, it has been merged in tinymist?)

[heady kelp]

tinymist is a central tool of convenience. Every embedded project will stay separate.

[midnight isle]

The typst-preview binary is dead, as we don't want to maintain two cli interfaces, considering that typst having more and more cli arguments to also work for typst-preview binary or tinymist binary. The typst-preview extension is deprecated but still working and will keep alive for a long time. The good thing is that people who are still use some other solutions will be unaware of any change: who use typst-lsp plus typst-preview, or treesitter plus typst-preview. This is the best way to respect people, I believe.

This means that, for nix packaging, you only need to change the binary in typst-preview extension to the path to tinymist binary.

[upbeat pike]

it would probably make sense to add a notice to the typst-preview window, too

I don't think everyone will check the Github README or marketplace page

like this maybe

will keep alive for a long time
Oh, do you mean that it will be updated for future typst versions as well?

[heady kelp]

I also don't understand what exactly that meant.

[midnight isle]

There is already one, whose code was unashamedly copied from typst-companion, but your notification looks better. https://github.com/Myriad-Dreamin/tinymist/blob/adfc15f017e35aba3d2f946c5769bb1aaddae689/editors/vscode/src/extension.ts#L49-L56

[midnight isle]

yes. and the binary of typst-preview in next version will be replaced with tinymist.exe. All bugs/features are shared. The extension code will not be updated though.

[dawn chasm]

Can I ask, from next release, we'll only need tinymist, right? No typst lsp, no typst-preview, correct?

[midnight isle]

correct

Since we have achieved fully concurrent LSP, you will not need to get two compilers to run in backend anymore, and still get both lsp and preview feature with same performance as before.

[dawn chasm]

I should be writing my phd thesis, so I hope to take full advantage of this soon.

[midnight isle]

For thesis authors, this is a huge improvement as GBs of memory usage are eliminated.

It is challenging to preview thesis instantly tho. I mean you will see visual latency but I believe it is still faster than latex. You can report bugs or inperfects if any then.

[dawn chasm]

What I need more than anything, is someone to make a template in typst, for my university 😂

[untold elbow]

why, you don't want to spent two days creating the latex template ?

[midnight isle]

In china, students from different universities usually copy and edit the template from NJU (some university) for their uses. So a good start is to find some template for start.

[dawn chasm]

My university provides the Word and LaTeX template: https://designguide.ku.dk/skabeloner/word/specialer/

[untold elbow]

wasn't the separation the cause of some large memory usage as well ? I remember I had a large ram consumption in the lsp at some point when doing synctex from the typst preview (though that was last year)

[midnight isle]

I remember it as well, but I didn't reproduce it (likely). You can check it again and if you don't like it you can disable it by "tinymist.preview": "disable".

[foggy sorrel]

that someone could be you!

[foggy sorrel]

It should be vastly faster in 0.12 with multithreading and various other improvements

[untold elbow]

I think I'll wait for the next version (after the merge to retest)

I feel you, unreproducible bugs are the worst

Though unreproducible bugs that are also erratic is worse

[little cove]

OK thank you. I'll deprecate typst-preview at the next update.

[foggy sorrel]

@stable turret you may want to archive the typst preview repo

[heady kelp]

If typst-preview will get regular updates, then it shouldn't be archived.

[foggy sorrel]

#1216655435663409152 message

[heady kelp]

I still don't get how this merge thing will play out.

[midnight isle]

ok. the preview repository still runs a ci for publishing deprecated extension. I don't own mgt's personal access token, so..

[stable turret]

for ci & release purpose we will still run it for a while

[dawn chasm]

can I ask, https://github.com/Myriad-Dreamin/tinymist/pull/367, is this a bug that needs to fixed upstream (in typst-repo) as well?

[little cove]

Nix PR: https://github.com/NixOS/nixpkgs/pull/325275

[midnight isle]

obviously not. this only fixes some bug about lsp protocol.

[little cove]

Argh, we have a hash mismatch in Nix

Did you recreated the 0.11.14 tag ?

=> https://github.com/NixOS/nixpkgs/pull/325643

[midnight isle]

oh no, you found the embarrassed fact. the first tag didn't publish successfully so i recreated it. To avoid it, I may takr some effort to create my canary release before real release in future.

[little cove]

yeah... this is somehow critical to recreate a tag.

But OK the issue has been fixed upstream.

Could you comment in the Github issue to clear doubts other people might have?

[untold elbow]

I was trying the new version to re test if I had memory issues on 0.11.14 but I found other issues with having only tinymist installed (and used for the preview)

In particular, clicking on the preview doesn't make the cursor go to the corresponding part of the code

Or only sometimes

[midnight isle]

https://github.com/Myriad-Dreamin/tinymist/pull/395