Distinguish paragraphs and text

[night bloom]

We need a way to distinguish real paragraphs from just "inline flows". This would, among other things, allow us to support unconditional first line indent without everything being affected (captions, ...)

Some relevant discussion starts here: #discussions message

@lucid sable What if the text in question would just not be justified unless you wrap it in par explicitly?

I'm not sure whether this is really what we want, but it's an idea at least

[snow escarp]

I think I also had issues because of this with a table of contents being indented
but I'm too bad with the syntax yet to say if that was really the issue

[night bloom]

I am also not sure what LaTeX actually does

[snow escarp]

imo it would be better if you could opt-out of it

[night bloom]

It should have the same problem conceptually

[snow escarp]

or wouldn't you then have to wrap everything you want to be justified into par?

[night bloom]

I mean, you can already with show table: set par(..)

[night bloom]

only if the automatic distinction doesn't consider it a paragraph

it's of course still unclear how the automatic dinstinction would work, so it's highly dependant on that

[lucid sable]

So you mean text that you write should not be wrapped in par automatically? Like

// These are not two pars
blalkdfskjalkjdsalk

jblkadkjfdslkjfd

// But these are
#par[asdj;ldsjaljf]

#par[alfdskj;skj]

Or do you mean in like tables and notes and stuff?

[snow escarp]

but can't you quite easily fix the issues by just doing #show table: set par(justify: false) or

#margin-note[
  #set par(justify: false)
  Text
]

?

[lucid sable]

Well again, that's not a fix, that's a workaround.

[snow escarp]

imo it's a fix

what woudl a fix be then in your opinion?

[night bloom]

Top-level paragraphs would still be paragraph

I just mean in captions etc. where it would default to non-paragraph

[snow escarp]

automatic detection for some things?
and then just unset justify?

imo that would just be unexpected behavior

[night bloom]

And I've left undefined so far where and how it would detect it

I'm not sure about justify

But first-line-indent should definitely not apply to headings and captions

[lucid sable]

How so. When you set par(justify:true) and it applies it to several places in the document you don't want it to that's wrong in my opinion. And correcting every place like tables and notes separatly seems like a workaround to me.

[night bloom]

So there is some form of text that's different

[snow escarp]

when I set par(justify: true) and then it just isn't applied at some places that seems like a bug to me

[lucid sable]

So basically It would make "normal text" in the document different from content []. I don't know if that's a good idea.

[night bloom]

There isn't really such a thing as normal text versus content

A full file is pretty much the same as a content block

I don't think we're looking for a syntactical solution

[lucid sable]

that's why I don't think it would be a good idea

[night bloom]

I do agree to some extent. But at the same time, there's headings which clearly aren't paragraphs, but still text.

[lucid sable]

My adhoc and not thought out idea was to have a main-par element on which we can use a show rule to set stuff like indent and justification.

[night bloom]

A table cell's content is much more a paragraph than a heading imo

So sort of document > par in CSS speak?

[snow escarp]

yes
but this isn‘t the case for table fields imo

[night bloom]

that may very well be true

[lucid sable]

I think that's probably what makes it hard. These concepts hard not very hard. Maybe we should make a table with cases and discuss where we would expect what behaviour?

[night bloom]

That sounds like a good idea

[lucid sable]

I don't have time right now, but I could make something like that tomorrow.

Where can I post it so we can iterate on the table?

[snow escarp]

here ig

[night bloom]

yeah here is good

the #1175885362065834105 is the canonical place for early discussions now and only once a design has been ironed out, we move to GitHub

[lucid sable]

?r

#show link: li => {text(fill: aqua, underline(li))}

#figure(
  table(
  columns: 3,
  [*Conceptual element*], [*justify*], [*first-line-indent*],
  ["normal" text flow], [yes], [yes],
  [text in table], [yes/no?], [no],
  link("https://github.com/typst/typst/issues/1050")[term list], [yes/no?], [no],
  link("https://github.com/typst/typst/issues/1955")[list inside term list], [yes/no?], [no],
  [headings], [no], [no],
  [captions], [yes/no?], [no]

  ),
  caption: [Which conceptual elements should be affected by `#set par()`?],
)

[lucid sable]

Ok I quickly threw this together. I tried to reflect the current state of discussion. Please give feedback and possible addition in both dimensions for the table 🙂

I also linked relevant issues on github

Maybe someone could go an check how latex does this?

[vast hatch]

maybe this but top level text or text in a parsed enum/list gets parsed as a par instead?

and maybe similarly to maths mode, [text] is a flow, [ text ] is a par

[minor zephyr]

LaTeX has the concepts of vertical (block) and horizontal (paragraph) modes for text. Encountering text characters in vertical mode causes the compiler to transition to horizontal mode, in which raw text is collected, until a paragraph break is encountered, which results in the generation of a new block of text (a paragraph).

There are also commands like \leavevmode, which explicitly causes LaTeX to transition to horizontal mode in a situation, where there are no text characters available. These can be used in command definitions to achieve certain tasks, like entering horizontal mode when a list \item is called.

[vast hatch]

Another thing to note here is that currently par is basically the same as text but with different options — the goal here should be to figure out what is not a paragraph and thus should not be considered par

When we do make that distinction, I believe some options of par would then have to be moved either into text or another sort of intermediate object (~flow?)

[minor zephyr]

In my mind, text is something that is either a leaf node, or a direct parent of a leaf node in a parse tree, in which case the leaf would be a single "grapheme". A paragraph is a non-leaf node that contains a sequence of text nodes as its children (if we ignore things like inline math for now).

Thinking in LL parser terms, if the parser was in block mode and encountered a function call, it would first check what kind of element the function call generates. If it's just plain text (not a function call) or a par function, it would insert a paragraph node into the parse tree, and any plain text until the next paragraph break or a block construct would be inserted as a child of the paragraph.

Or something like that.

The settings of a text node would only allow changing the appearance of the text and maybe positioning in relation to its siblings. A paragraph node would only have settings related to the positioning and shaping of the entire paragraph, such as first line indent.

[knotty flare]

Hi, friends! Here are my two cents: a paragraph, as it is currently designed, isn't abstract enough.

content.is::<SpaceElem>()
            || content.is::<TextElem>()
            || content.is::<HElem>()
            || content.is::<LinebreakElem>()
            || content.is::<SmartQuoteElem>()
            || content.to_packed::<EquationElem>()
                .is_some_and(|elem| !elem.block(styles))
            || content.is::<BoxElem>()

These are currently the only things allowed to be in a paragraph. Basically, only things that can be written inline. However, I think the solution is to expand the "definition" a bit, and include "contiguous thoughts". What this means, to me, is to allow block content to be included.

The distinction can be made between text and paragraphs as paragraphs vs inline content. That is, the "definition" of inline content would be what is the current definition of paragraph content. Where a paragraph can contain additional objects.

[quick crescent]

At least a paragraph could contain invisible objects, so that place or figure does not interrupt the paragraph if set between text...

[knotty flare]

I would also argue that a paragraph needs to start with a text element.

[night bloom]

I think what you're talking about basically aims to solve a problem one lever further than what we're currently trying to solve. It is about handling indent when a paragraph conceptually continues through an equation. But we're still at the problem that currently everything with text is a paragraph, such that first line indent also affects headings, captions, etc. To solve that, we need to only turn inline stuff into paragraphs in certain contexts.

[knotty flare]

I would argue that you shouldn't turn anything into paragraphs unless it's on its own in a content block (e.g., typst is detecting a heading, so, when the heading layouts itself, it should layout inline content). Basically, keep everything the same as it is currently. Rename the current paragraph item to inline. Add a new item for paragraphs that takes inline items and other block content.

As for styling, move first-line-indent to the new paragraph item. Add always-indent-first-line option to paragraph. Probably a few other options can be moved from inline to paragraph, as well.

As far as I can tell, an element lays itself out, and the parser, evaluation, and realization all work to determine what each piece of content is. The changes that would be made for this process, I think, could be to change what typst does with content outside of any special syntax.

I'm not telling anyone what to do. I would, in fact, like to do this all myself, I would just need some advice.