Tables

[wary knot]

well

it will probably just disappear if you try to do that

:p

cuz the algorithm skips colspans / etc

[bold bluff]

i was just trying to think of what it would do.

[wary knot]

yea

erroring wouldnt make sense

cuz you can have a line spanning more than just the colspan

and then it would disappear just for that small part

[bold bluff]

I would expect it to draw a line to the far right of the colspan. if i had previously drawn a colspan. (I don't know if that confuses things)

is the mental model ment to be cell wise?

[wary knot]

that would be a bit too overly complicated yeah :p

[wary knot]

for hlines / vlines no

you can consider them as overrides for sections of the default table lines

the ones which appear when you write stroke: red or smth

hlines / vlines can cross gutter spacing between cells, just like them

so they're not cell-wise

[bold bluff]

Well im excited to see the outcome. but its late in states.

[wary knot]

dw

it's late here too

lol

im just trying to figure out where i screwed up

usually lines dont disappear lol

after im done with that ill log off 😎

ok i see

lol

im dividing the row index by 2 when theres gutter

but that doesnt work specifically for the last row

same for column

gotta add 1

yayyyyy

🚀

[vernal herald]

It's a great day for tables!

[wary knot]

ok nice

i think todo list is pretty short now

for line customization

things left:

• hline(stroke: none) does something
• sorting lines based on thickness

and also the usual

• tests
• cleanup

[wary knot]

ok the first item was a one-liner so i just did it

cool

#grid(
  stroke: red,
  columns: 3,
  inset: 4pt,
  [a], [b], grid.vline(stroke: none, end: 1), [c],
  grid.hline(stroke: none, end: 2),
  [d], [e], [f],
)

applying cell stroke overrides that

ok now im done for today 👍

[vernal herald]

It's 8 am here.

[bold bluff]

Did position: bottom. Become end: 1?

[wary knot]

No

start, end are how you specify where the line starts and ends

By default it starts at 0 and goes all the way to the bottom/to the right

Note that it has stroke: none so it's doing the opposite of adding a line

It's removing the default line between 0 (top) and 1 (above second row)

So between b and c there are no lines

Note that x: and position: affect in which column the vertical line is, while start and end affect which rows the vertical line spans

[keen crescent]

[Fluff] If timezones werent a thing this would be a much funnier message

[frozen oyster]

Maybe the flat earthers have a point: we need to make the earth flat!

[vernal herald]

I wrote that because I felt pgsuper was on the cusp of crashing. Wanted to give the old, I'm in the metro omw to work, good night buddy.

[frozen oyster]

on your way to work

wtf

you're in Denmark, the same time zone as me

[grand haven]

At 8 am. Not exactly strange

[frozen oyster]

Ah right

okay

I'm dumb

don't mind me

[upbeat pine]

exposed! /s

[wary knot]

so

I did this pretty cursed thing

https://github.com/PgBiel/typst/commit/f6ec7624e7bad590786f2c86e7600d0a1026cbf4

but it's all in the name of performance 😎

cc @frozen oyster

why for loop + vector when iterator adapters are enough 😎

tbh it would be nice to use gen functions here

cant wait for rust 2
i mean rust 2024 edition ok thx

[wary knot]

yayyy

sorting lines works

isnt it beautiful

cc @frozen oyster @robust viper should I use Rayon's par_sort_by rather than sort_by? seems attractive but who knows 😂

anyway cool

ill be opening draft PR soon ™️

[bold bluff]

It’s a very pretty table. 10/10 would view again.

[wary knot]

how about this? 😎

well i forgot a line there

kk better

[bold bluff]

Still good. Name choice seems a bit 1980s

[wary knot]

lol

makes sense

credits to @thorny coral for the double line #quick-questions message

[bold bluff]

Is that a double hline!

[wary knot]

#let double-line = pattern(size: (1.5pt, 1.5pt), {
  place(line(stroke: .6pt, start: (0%, 50%), end: (100%, 50%)))
})

#table(
  stroke: (_, y) => if y == 0 or y > 1 { (bottom: black) },
  columns: 3,
  table.cell(colspan: 3, align: center)[*Epic Table*],
  align(center)[*Name*], align(center)[*Age*], align(center)[*Data*],
  table.hline(stroke: (paint: double-line, thickness: 2pt)),
  [John], [30], [None],
  [Martha], [20], [A],
  [Joseph], [35], [D]
)

[bold bluff]

So good

Well it’s so much I forget what is left for native tables.

Your pretty close to your 0.11 goals right?

[wary knot]

rowspans (part 3b), line customization (4, above) and headers (5)

line customization is almost done p much

rowspans have a lot of work done but theres a roadblock that we need to clear

headers are 0% done so far

[bold bluff]

Oh headers . I saw you mentioned having replaceable headers. But I didn’t know what you would do with them.

[wary knot]

yea repeatable headers are a common request

in principle they shouldnt be too hard to implement

but that might change once rowspans are merged

:p

but at least i have some general ideas about it in my head rn

[bold bluff]

I think it was this. The sticky headers/ footers

[wary knot]

yeah

those are the ideas i have rn for the API

[bold bluff]

I guess I could see a more extensive header followed by more basic ones. But yeah basic repeating headers would be great for usability.

[wary knot]

yeah im not 100% sure if it will be possible to implement all of my ideas + subheaders and stuff

i want to implement as much as possible but time constraints exist

in general, as long as we get the ability to repeat a single header until the end of the table, we're fine

:p

[bold bluff]

High bar.

[wary knot]

Draft PR created: https://github.com/typst/typst/pull/3393

🚀

(not meant for review yet but we're close)

@robust viper two final design decisions, tell me what you think:

1. right now, table.vline(x: 2, position: left) places a vline to the left (default) of column 2, while table.vline(x: 2, position: right) places a vline to the right of column 2 (this is only needed when gutter is used, otherwise you can't add lines to the position to the right).
   But it is not left and right when using RTL, but rather inverted. Should we change those to start and end instead? I guess it would be consistent with the rest of typst, but not sure if that's strictly necessary
   (We'd remove left and right as possibilities then, i think. Making them auto invert depending on RTL sounds a bit convoluted here.)

2. I was thinking of making it so you can specify #table(stroke: (..., inside: (horizontal: red, vertical: none))) for example, to override all horizontal or vertical global lines. Theoretically they could also accept functions (taking y and x), but those functions would need to take two parameters in the case of gutter (otherwise just y and x would be ambiguous), so I'd stay away from functions for now. But at least just that could be useful when all people want to do is to disable vertical lines (they'd write #table(stroke: (vertical: none))). Wdyt?
   (yeah I know theres the (x, y) => ... deal for that, but it's not the same when there's gutter so there's that. And also i believe that makes your intention clearer ig lol)

[random mica]

I would expect left to always be left, right to always be right; start and end to adjust based on language.

[robust viper]

Why do we need position: right? Isn't that just x: 3?

I also just tested the branch and in my opinion, it is strange that hline/vline don't have a stroke by default. I.e. if I disable the table's default stroke and add a hline, nothing is visible. It should probably default to 1pt + black.

[random mica]

As I understood it there may be two vlines between adjacent columns if they have a gutter between them

[robust viper]

ah, the gutter strikes again!

[robust viper]

then, same

Regarding inside and outside, I am not sure whether we are on the same page of what they should do. I had thought of them as purely symmetrical to left, right, etc. Just have left | right | top | bottom | x | y | inside | outside. x and y are a shorthand for left/right and top/bottom and inside/outside only apply to lines that are contain in the table vs. at the outer border. So basically what I thought (in pseudo types) was:

#type s = stroke | none
#type d = s | (left?: s, right?: s, top?: s, bottom?: s, x?: s, y?: s, inside?: s, outside?: s)
#type table(stroke: d | (int, int) => d)

@wary knot

In particular table(stroke: (inside: 1pt), ..) would not render a full set of lines, but only the lines that are not around the table

I thought that easy control over inside and outside would be useful, but we can also scrap them if people disagree with that.

[robust viper]

Regarding 2. here, that would be x and y in this proposal, in line with how sides dictionaries always work.

For reference, here is the relevant code from my old branch.

[upbeat pine]

I think the removal of outermost border strokes is a regional thing, that's why it can be less useful in some regions. And since in Russia we/I normally only use tables where all strokes are drawn, it doesn't look very useful to me. But maybe math/research/study tables' styling is universal, which means that it would be useful in such fields around the world.

I mean, the functionality itself seems useful, but for me, it's not practical at all, maybe in very rare cases, like tic-tac-toe or something.

[pallid surge]

Regarding all the new features shown here for tables: It would great to also have the ability to end hlines slightly before reaching the midpoint to the next element similar to this graphic (from: https://tex.stackexchange.com/questions/345062/how-to-create-a-statistical-table).

[robust viper]

What might be more useful is a combination of inside/y if you just want lines between your rows, but not above the first and below the last.

Which isn't directly accomodated for in my design

It would need something like stroke: (y: (inside: black)), but it starts getting crazy.

[upbeat pine]

is using stroke: y + inside too crazy of an idea? but idk about further customization: stroke: ((y + inside): black)

the problem is that normally + is OR and for AND we have to do & or *. and that this is a new syntax...

that's what first came to mind when I read "inside/y", but then remembered that these are not like left/start/center etc.

[robust viper]

inside-y would be possible, but if we add one more thing the possibilities explode

[grand haven]

To play devils advocate, is gutter truly necessary? Except for the whole stroke around cells things, the functionality overlaps with inset. And the same effect could be achieved by wrapping cells in a rect

(if there is a desire for simplification)

[robust viper]

I'm sort of wondering the same thing. Gutter is very useful for grids but less so for tables. However PgSuper has already put a lot of effort into making it work properly now. And there might be use cases I'm not aware of.

[lone nova]

I think we can do amazing things with gutters. Rare but beautiful

[grand haven]

But can those things also be done some other way?

[grand haven]

True, although that's sort of a sunken cost fallacy

[wary knot]

That’s probably bug

Will fix 👍

[wary knot]

Great, will change then. 👍

[robust viper]

True

[wary knot]

Hmmmm... idk, I'm confused lol
What are left right top bottom for then?

I was working with outside and inside before and I was told the outside stuff had to be on top level so i changed that

Now brain is glitching

[wary knot]

I think you can make some pretty cool effects with it using the new per-cell stroke

I'm not in my PC right now but if you set gutter and per-cell stroke you can have many "floating boxes "

And they come with consistent stroke lengths and stuff thanks to the table (looks better than using a bunch of rects)

So that's one thing I'd use it for at least

And yeah for grids in general it's useful to just add some gutter between things

Also I just realized I pasted the wrong code for the second example in the PR description lol

That's clearly not aqua stroke 😂

[robust viper]

left, right, top, bottom are the most specific

If they are present, they have precedence

x/y is more general

inside/outside is a third option

it would just be just a convenient way to configure whatever sides are on the inside/outside for a particular cell

[wary knot]

Hmmm, yeah I'm still confused
Would those be equivalent to today's (inside : (left: stroke, right: stroke, ...)) ?

[robust viper]

I can't really tell because I don't understand how your inside/outside works 😂

[wary knot]

I mean

#set table.cell(stroke: (left: ..., ...))

[robust viper]

This dictionary would also take inside/outside

Outside would only be relevant for cells at the edges of the table

[wary knot]

Hmmm

So it would be exclusively a per-cell thing?

[robust viper]

conceptually yes

but #set table(stroke: (inside: red, left: blue)) should also work

[upbeat pine]

will inside only be accessible/assignable for table/cell strokes, or this key will be ignored elsewhere?

[robust viper]

the key would be invalid elsewhere

basically if none | stroke | (left/right/top/bottom/x/y/inside/outside dict) is T then grid.cell stroke takes T and table stroke takes T | (int, int) => T.

and as said before we can also still decide to scrap inside/outside altogether, I just thought that it might be useful

[wary knot]

I think it's useful, I'm just not really sure regarding what'd be the best API here

Using #1194703458482262087(stroke: (left: red)) to make the left border red did seem to click in general

So maybe it's not that bad

And making it per-cell only seems a bit limiting, it would remove the ability to cross gutters with lines

But that doesn't mean we couldn't just add some extra key to control that

[robust viper]

what is the current API in pseudo type system?

[wary knot]

it's basically like

stroke / func / array becomes (inside: that)

The full form is
(left: stroke, right: stroke, top: stroke, bottom: stroke, (and x y rest), inside: stroke / sides(stroke) / func (x, y) -> sides(stroke) / array of sides(stroke) )

When inside is just stroke it overrides table lines and not cells

Otherwise inside: (left: ... Etc) works exactly like in your API

The array thing is more of a consequence of using celled

:p

[robust viper]

what confused me most is that in my testing "inside" didn't render only the inner table lines

which was the whole idea of it

[wary knot]

i guess you could say that the outside ones just "default " to match the inside ones

But I'm not sure how else I'd call it

[robust viper]

I feel like at this point we are both pretty confused by each other :p if you have time we could hop on a call in a few hours

[wary knot]

yea sure but I think I understand your proposal now

My concern right now was mostly that it felt like we lost the ability to have lines crossing gutters

But there must be some way to conciliate the two things

[grand haven]

Is that something that's actually desirable though?

[wary knot]

Basically my proposal defaults to changing global table lines (irrespective of cells, so they cross gutters) unless you explicitly use something that applies only per-cell , i.e. use a function or (inside: (left ... Etc))

[wary knot]

yeah. For example for borders, you don't want your table border to be interrupted on every row gutter

I'm not saying it's desirable on every situation. Really what we're missing right now is mostly the ability to choose between the two things

Tablex handles this by just giving you the line objects and you construct stuff by yourself. (That is, you can either add a single horizontal line spanning an entire row, or you can add one above every cell, for example.) But we can probably be smarter than that here

With that said, an alternative is to split the two options

For example, we could have something like

(lines: stroke, cells: stroke)
Or that kind of thing

Though in the future we probably won't have per-cell config when we have more powerful selectors :p

But yeah

[robust viper]

I find that a bit confusing though tbh. I would expect red and (_,_) => red to be equivalent.

[robust viper]

let's still do it, I also want to understand all your gutter-related plans and concerns more

[wary knot]

That's fair enough, though I'm not rlly sure how to change this

Perhaps we could make the gutter spanning config opt-in rather than opt-out

It would break all tables with gutter ATM but I'm not sure if that's too much of a problem 😛

[robust viper]

I wouldn't be concerned about changing table defaults at all

[wary knot]

maybe something like

(lines: red | (horizontal: red, vertical: none)) could work

But that's just an idea

And then red / (_, _) => red would do the same thing

argh discord

[wary knot]

Ok 👍

[lone nova]

Even in future, per-cell customization will be were useful.

[wary knot]

yea i meant like show-set on cell elements will supplant all features of per-cell stroke customization

right now show-set is a bit limited in its filtering capabilities so yea

[upbeat pine]

P. S. while skimming, I was very surprised that い caught my attention and that you even used it, but it was just (, )... it actually surprisingly similar.

[robust viper]

@wary knot I'd be ready

[wary knot]

ok

gimme 5 min

[upbeat pine]

can I sneak in?

[robust viper]

sure

[wary knot]

Kk I'm ready

Where are we doing it?

[robust viper]

https://discord.com/channels/1054443721975922748/1054443722592497797

[wary knot]

Ok

[robust viper]

Result from discussion:

• stroke on table.stroke and cell.stroke behave equivalently
• stroke directly vs stroke function behaves equivalently
• cell strokes never cross gutters (unless there are spans)
• hline/vline have max priority vs. strokes

[wary knot]

so, just to showcase what this means, the following tables and grids changed in tests:

before/after

i think this is a good default at least for grids, im kinda torn on tables but I think we're fine

also part of me wants to keep the outside stuff in there somehow... felt kinda sadge removing code for it 😂

[lone nova]

Does this mean that it won't be possible to create strokes that cross the gutter?

[wary knot]

you will be able to use hlines and vlines for that purpose

but it will be a bit manual for now

we havent decided on the best API to configure them "en masse"

[lone nova]

Oh, that's great👍

[wary knot]

that means that this table ^ will be possible (thanks @pallid surge for the example)

you'd use some column gutter and three hlines for the lines which cross gutters

[lone nova]

Five hlines?😁

[wary knot]

no those would be per-cell stroke

in the header

so the gaps are preserved

i mean sure you can add a bunch of hlines there but nobody wants to do that

:p

[lone nova]

There are two small lines that also cross the gutters.😅

[wary knot]

yea but

in that case it's a colspan

the entire cell crosses gutters :p

[wary knot]

well now, i´d say i got pretty close

bar the absolutely horrifying alignment choices

which caused pain

the full code: (click to see)
||```rs
#set text(font: "New Computer Modern")
#set page("a4")
#show table.cell: it => {
let inset = (:)
if it.y > 3 and it.y < 7 {
inset.bottom = -6pt
}
if it.y >= 3 and it.x in (2, 5) {
inset.right = -3.5pt
}
pad(..inset, it)
}
#table(
columns: 7,
stroke: (_, y) => if y in range(0, 3) { (bottom: black + 0.5pt) } else { none },
align: (x, y) => if y in range(0, 3) { center + top } else { (start, center, end, center, center, end, center).at(x) },
column-gutter: 5pt,
inset: (left: 0pt, bottom: 7pt, right: 5pt, top: 5pt),
table.cell(colspan: 7, align: center, stroke: (bottom: 2pt))[Unit Root Test Results],
table.cell(stroke: none, align: left)[Variables], table.cell(colspan: 3)[ADF Test], table.cell(colspan: 3)[KPSS Test],
table.cell(stroke: none, hide[Variables]), table.cell(colspan: 2)[At Level], [At First\ Difference], table.cell(colspan: 2)[At Level], [At First\ Difference],
hide[Variables], [C], [C&T], [C], box(width: 37.5pt)[C], [C&T], [C],
table.hline(stroke: black),
[GDS], [-1.05], [-2.1], [-7.01***], [0.73], [0.13], [0.07],
[NSP], [-0.74], [-2.16], [-6.98***], [0.77], [0.16], [0.1],
[Inflation], [-4.75***], [NR], [NR], [0.1], [NR], [NR],
[GDP], [4.831], [1.07], [-4.67***], [0.79], [0.2], [0.73],
table.hline(stroke: 1pt)
)

the show table.cell and stuff are really an attempt at fixing alignment :p

regarding alignment on the number sign, idk theres probably a way to do that manually but i didnt bother

also i used some cells with hidden content to emulate a rowspan on "Variables" (as they arent implemented for now, or at least they are in a separate branch)

[bold bluff]

this is fantastic. such powerful native tables will be un-paralleled in markup.

[wary knot]

hehe

im really excited about it as well 🙂

[lone nova]

Yah, this amazing! 🥳

Can we add this code to the documentation as an example?

Maybe with some polishing, if any😂

[vernal herald]

People will go nuts for this

Finally, it might be a great day for typst (because of tables)

[grand haven]

Aligning numbers with awareness of - is a pretty common thing to do.

Though I don't use actual numbers myself, so I don't know how to do it in LaTeX 😛

[random mica]

In matrices I just put &s

[grand haven]

We might need some more clever alignment options in the future

[grand haven]

Ideally it shouldn't be necessary to manually specify alignment point

[upbeat pine]

1. Damn, this looks more and more like latex, but at least you actually read what the hell is going on...
2. very impressive
3. hold on, what about raw numbers in cells? This is one of the reasons I used tablex — to not constantly type [] around numbers. will this be a feature in 0.11 or at all?

[grand haven]

Numbers should ideally be typeset in a math context

[grand haven]

Few, or none, of those numbers are representable as floats

[upbeat pine]

wdym? there are 20 floats

also, since the default font is the math mode, using [] is ok, I think, but without it you would use $$ which is less comfortable, and probably more noisy (visually), but at least I have snippet to make m into $$, so that's not that big of a deal to me. but not using any wrapper at all would be much better.

I also don't remember if you had to use $$ or something for numbers to appear in math font when using "normal" tables in latex. I only remember that you don't need to use $$ for every cell when using arraytable or something, that wraps in one pair of $ and everything inside is in math mode/font.

[grand haven]

No, there are 20 decimal numbers. There is no float that is exactly 1.05 for instance.

[upbeat pine]

they are clearly floats in typst

[upbeat pine]

but why are you even saying that? my point is to not wrap around numbers, any numbers in [] or $$.

if you have to add *** then you don't have a choice, but that's not my case.

[grand haven]

I'm saying that because if you're not wrapping them then they'll be interpreted as floats instead of strings of digits, and information is lost

[upbeat pine]

which information is lost when inserting a float?

?r

#import "@preview/tablex:0.0.8": tablex
#tablex(1.05, 0.73, -1.05, -0.73)

[upbeat pine]

I'm saying that because if you're not wrapping them then they'll be interpreted as floats instead of strings of digits
This is obvious, and I don't see any problem here.

and information is lost
This I don't understand completely.

[robust viper]

https://stackoverflow.com/questions/1089018/why-cant-decimal-numbers-be-represented-exactly-in-binary

[upbeat pine]

no, that's a completely separate topic.

?r

#import "@preview/tablex:0.0.8": tablex
#tablex(columns: 5, 0.1, 1.05, 0.73, -1.05, -0.73)
#tablex(columns: 7, 8/1, 4/1, 2/1, 1/1, 1/2, 1/4, 1/8)
#tablex(columns: 2, 0.1 + 0.2, 1/3)

[upbeat pine]

I am the author of my document and I know when things go wrong when inserting certain numbers. So usually it's fine. And if not, then I use something like:

#let round(number) = calc.round(digits: 2, number)
// or
#let r(number) = ...

[robust viper]

that's exactly what @grand haven was talking about

[upbeat pine]

and usually all my numbers are floats from the start, and that's why it is useless for me to make them "strings".

[grand haven]

they're not actually floats though

[upbeat pine]

typst says they are floats. idk what to tell ya

[upbeat pine]

that's fine, but again, that's not my point. also, if author wants to "preserve" decimal/floats or whatever, then they can always pass a string or a content as a table cell. but I want to pass ints and floats as well. so what I want is an opt-in feature, because built-in tables doesn't allow for stuff other than content (and maybe str). but tablex allow all that, and that's why I used it (without even using any of its advanced features). and I didn't have any problem with how floats were represented in my tables.

[robust viper]

with the type rework that will sort of happen automatically. I'm not sure whether I would really want to add a special case for tables before that.

but what Enivex is trying to say I think is that encouraging direct use of floats over numbers in math context is a bit dangerous because it can lead to a different rendering than expected.

[upbeat pine]

with the type rework that will sort of happen automatically.
wh...it will what? automatically? can you elaborate on this, please? non-table examples? I don't understand what will change with that update for it to be "automatic". I mean, if it will be possible, then I'm all for it.

but what Enivex is trying to say I think is that encouraging direct use of floats over numbers in math context is a bit dangerous because it can lead to a different rendering than expected.
Ohhh!!! That is, yeah, I'm 100% agree with that. But one thing is to encourage, and the other thing is to make it possible, you know? they are not exactly the same. and also a warning about unexpected output can be added for good measures (in the docs). and if a user sees an unexpected output, then they will know that they probably should've used $$ or [].

[robust viper]

what I meant was that if we remove the content type in favor of every element function being its own type, then anything that currently takes content will just take any conceptually and thus also allow integers and all other computational types.

[lone nova]

What about non-English speaking countries? For example, in Russian the comma is used as a decimal separator.

[upbeat pine]

I know that "we" use comma, but (1) I hate that we do use comma (because it also used when you list things and numbers; that's why I use period myself, even at school, IIRC), and (2) in all programming languages you use period for floats and it is only natural to use period here as well.

(1,34, 45,28, 654,65 — like this is hideous! that's why we have a "hack" of sometime using ; instead of , for listing: 1,34; 45,28; 654,65. and it's a hack, because in Russian grammar a way to list things is to use comma and not semicolon.)

[upbeat pine]

what? that's huge! I didn't know that you want to remove content. Did you say about that in you blog? or somewhere else? that's definitely a more that I didn't expect.

[lone nova]

The use of a comma as a decimal separator is very common in the world. For example in France, Germany and the rest of Europe. For example, you can look at this article: https://en.m.wikipedia.org/wiki/Decimal_separator.

The traditions of book printing, nothing can be done about it.

And the lists of numerical numbers you provided are very rare. I've never seen anything like this anywhere. Basically, all information containing more than two numbers is presented in the form of a table.

[grand haven]

The decimal separator used in code doesn't have to be the same one that is displayed

[upbeat pine]

interesting. I thought europe also uses period.

And the lists of numerical numbers you provided are very rare. I've never seen anything like this anywhere. Basically, all information containing more than two numbers is presented in the form of a table.

Listing numbers is not a rare thing, definitely not for integers and not in programming languages. I also remember that for answers to a problem in school, we had to use ;. so the answer would look like: 1,5; 45,64; 0,0045.

Yeah, I think that it would be nice if typst can print comma in this case:

#set text(lang: "ru")
$1.5$

anyway, this sort of problem/discussion should go into a separate thread in #contributors, not in #1182377294682128445, because it doesn't specifically concern only tables.

[wary knot]

Regarding 3, you'll be able to do the following:

#let mytable(..args) = table(..args.pos().map(x => [#x]), ..args.named())

Which is more or less what tablex does under the hood

[wary knot]

Sure I can try to do so

But not in this PR (i will add more stuff to docs later)

[upbeat pine]

@robust viper said that it should be possible after content type will be removed.

[wary knot]

yeah i know

That was spoken about in #1175895383600275516 before

But it will take a while

Like probably multiple months

So

In the meantime you have that

[wary knot]

We certainly need at least baseline alignment for enums and lists

Number alignment would be cool as well though, just don't know how exactly we'd hash out the details

[upbeat pine]

will this break vlines and stuff like that?

[wary knot]

not sure, if it does you can just add a guard

[upbeat pine]

yeah, add an if statement for vlines, hlines and cells, and that should be enough.

[wary knot]

ye

[upbeat pine]

nice

yeah, thanks. now, if this would work, I will have one less reason to opt-in for the tablex.

[bold bluff]

Floats are tricky. And I don’t think they are discussed unless someone runs into an issue. Most numbers are not representable in floats.

[hoary tartan]

It would be great if the if condition could be set as the inset value in the table function, similar to how it works in fill.

#table(
  columns: 5,
  inset:  (col, row) => if row==0 { .5em } else { 1em },
  fill: (col, row) => if row==0 { luma(225) } else { white },
  ...

Suitable for feauture request?

[wary knot]

yea, i was planning to add this in one of the table PRs but i ran into a rather fundamental implementation issue

but i think that issue is mostly gone now so it shouldnt be that hard to add it later

either way it's already possible with the new table stuff, it's just a bit more manual

you'd set the global inset to 0pt and then do something like

#show table.cell: it => {
  let inset = if it.y == 0 {
    (rest: .5em)
  } else {
    (rest: 1em)
  }
  pad(..inset, it)
}

(note: this will only be possible in the next typst version)

anyway. but yeah your request is reasonable. and simpler

[hoary tartan]

Oh, thanks

[hoary tartan]

Okay, then I think I will send in the repository

[wary knot]

so

i was working on fixing this, which is what happens when u set stroke for entire columns of cells

u can see the hlines to the left appear on top of the hlines to the right even though those to the right have priority

and the fix was pretty simple , just had to reverse the order of the line segments so that they are drawn from left to right and top to bottom

instead of the opposite

so now it looks like that which i think makes more sense

but there are some consequences elsewhere , which im not sure about

as we can see here for example

before it used to be a fully blue square

perhaps im just used to vertical lines being on top or something

lol

[upbeat pine]

maybe left should have a priority if they override the default styling?

but if right also defines a custom style, then right has a priority

at least for that example this should work

[upbeat pine]

if "d"'s ||nuts|| style is the only one which is non-default, then all 4 strokes should appear on top.
if b or e also have non-default style, then they should take a priority because they are right.
but if d and c/f are non-default, then d again has a priority, because no other adjacent cells on the right have non-default styling.

[robust viper]

I don't think a reliable concept of "default" styling really exists

Just draw a small triangle in the middle :D

[upbeat pine]

yeah, I also don't know about that. everyone has their own preferences. so we either have to go by some "standards" or by the rule of 51%.

[wary knot]

we could perhaps give more priority to explicit lines by default

at least

cuz rn we have this

but technically you can control priority by abusing thickness sorting

lol

by adding like 0.01 of thickness

:p

which then goes like this

well anyway whatever

something has to win

😂

yeah we could add little triangles everywhere but i dont speak graphics so

😂

well, hlines stay on top either way though

well w/e

i'll keep it like this for now :p

[wary knot]

Ok well

I think this is one of the things which actual real usage feedback will be useful for

So yeah I'll keep it like this

Now I should probably focus on getting the PR ready for review

At this point there isn't much further for me to do independently

[bold bluff]

I approve of abusing line thickness.

[wary knot]

ok so i took a bit longer than i should have nitpicking myself

lol

for the remaining matters ill just add a review comment instead , things are otherwise in an OK state though

[wary knot]

well

i couldnt stop nitpicking myself

lol

and while doing so i was writing a doc comment for something when i realized "hmmm wait"

and found a bug lol

uhhhhh what

wth did I do 😂

I'm innocent I swear 😂

great i cant reproduce it anymore

hip hip hooray for non-strictly-reproducible tests

[upbeat pine]

maybe caching? or something.

[bold bluff]

arrgh? sounds like the pirates life is calling.

[upbeat pine]

same

I thought it was a pirate joke of some sort.

[wary knot]

ok guys

PR is now ready for review 🎉

crossing almost-ready stuff through the finish line is really annoying lol

but there it is

[upbeat pine]

Great job!

[wary knot]

thanks 👍

[wary knot]

btw @robust viper (idk why that happened and only happened once so pinging just in case lol)

also btw, regarding https://github.com/typst/typst/issues/3401 (making inset take (x, y) => ... and so on), I have a commit ready for it but it depends on the latest PR (line customization), as it implements Fold on Celled
should I add it as part of the PR, or create a separate PR?

[robust viper]

as you wish

[robust viper]

are you really sure that nothing changed in between? this does not look like a thing that would happen spuriously.

[wary knot]

I'm like 85% sure

I say that cuz i don't remember fully , and i was testing a bug fix

So it's possible I changed something but unlikely

And even if I did change something , i really don't see at all how it would have affected that

[wary knot]

Cuz all i was doing was apply a fix to this bug

And bibliography doesn't use hlines / vlines , let alone CellGrid::resolve , so yeah lol

Anyway, worth keeping in mind

Maybe i was just unlucky. cosmic rays and all that

[robust viper]

bibliography uses introspection

if anything is slightly off with measurement and layout order, things like this can happen

because location assignment is then broken

[wary knot]

i see yeah

well anyway it hasnt happened again

if it does i'll tell ya

[robust viper]

@wary knot is the PR ready for review?

[upbeat pine]

#1182377294682128445 message ?

[wary knot]

Yes

[robust viper]

done!

[wary knot]

yay ty

sorry in advance if i went a bit too overboard on the comments

or maybe not in advance

lol

but i guess it's better to have too much than too little before a review 😄

[robust viper]

no, they were very helpful

but still one hell of a big PR

took me more than two hours to review

[wary knot]

hehe, whoops 😄

but hey, at least we got that going

🚀

[robust viper]

but still a good ratio compared to the time you put in, I assume :)

[wary knot]

perhaps haha

took me about a week to get all the functionality going, and then a few more days just making the final touches

to be honest those final touches easily felt like they were half of the total time I had spent

😂

[robust viper]

btw, regarding the next release

do you think end of month is realistic for rowspan?

[wary knot]

yes, we just need to figure out the algorithm

i have everything else implemented though in theory

[robust viper]

any input you need from my side or just more experimentation about what works best?

[wary knot]

well ok ill also need to fix the frame layout order thing (to not layout all rowspans at the end but as soon as they end or whatever)

[wary knot]

right now im still idealizing it, so some input can still be helpful

ive thought of some possible approaches but im not 100% sure

i can share more later probs

worth saying that, in parallel, I also plan on starting work on repeatable headers, since thats also something that I want to see going as soon as possible

but ill probably trim down on the features, since very basic functionality is probably enough for now

lol

if it doesnt make it into the next release i think its not the end of the world either though, just an interesting goal to have

anyway, just wanted to declare my intention explicitly at least

[vernal herald]

Repeated headers would be nuts.

[wary knot]

i have a rough idea of how they could be implemented but ill take some time to experiment with it

[wary knot]

ok well ill probably dump my current thoughts here later

ive decided it's probably better

lol

cuz i think ill need some help 😄

and if we cant figure something out over the next few days, then perhaps a call might be in order next week :p

but let's see

maybe if i ping enough people someone will have a brilliant idea

😄

(ok but give me some time while I write it down and stuff 🙂 )

[upbeat pine]

Oh, wow, that's a long PR review. Good job to you too!

[random mica]

If it's still relevant, regarding the order of drawing table lines, I wonder how feasible it is to draw the strokes in order of the scope their defined in

so each set rule, each show-set rule in order, each stroke from tables' arguments, and each cells own arguments

[wary knot]

the more general idea of drawing more prioritized stuff on top is in the right direction I believe

but I wouldnt go as far as involving set rules here, that'd be way too complex

both to implement and to grasp

Laurenz did point out a problem in the current design which is the fact that overriding a single cell wont work in the way you could want it to

as in, doing something like

#table(
  stroke: red,
  table.cell(stroke: blue)[a], [b],
  [c], [d],
)

will draw blue lines to the top and to the left of "a"

but red to the right and below

cuz rn , internally, that's exactly the same as the following

#table(
  stroke: none,
  table.cell(stroke: blue)[a], table.cell(stroke: red)[b],
  table.cell(stroke: red)[c], table.cell(stroke: red)[d],
)

which explains why it does that

im still not sure what's the best way to tackle this

one possible approach is to have a simple boolean

for each cell, if it gave a non-none stroke then it will have priority

another possible approach is to delay folding between per-cell stroke and the global stroke until we actually draw the lines

but note that we still have to fold early anyway so that show rules recognize the final stroke value

so we'd now fold twice

fold here being the process being done when you merge the global stroke with the per-cell stroke (the latter having priority)

by delaying it, we'd know exactly which cells have asked for priority

but it does feel a bit inconsistent with the rest of things

maybe it's a small price to pay for salvation though

but boolean approach is so simple ;-;

i-have-priority: true (internal field of course)

also note that none of this matters to the final judge: The Great Thickness

you can just write table.cell(stroke: blue + 1.1pt)[a] and suddenly it's fixed 😂

conversely, even with this priority "fix" larger strokes would still be on top

but imo that's for the best

[wary knot]

ok

quick status update:

I took a first look through all reviews, I can see @robust viper made a lot of effort to provide high-quality reviews, so thanks for that 👍

I answered what I could (currently away from PC so cant fix stuff etc), I'll answer the rest in the coming days

[random mica]

I think it should work just as if the strokes were drawn eagerly

so whatever would conceptually be evaluated last will have priority

[wary knot]

i mean, i understand where you're coming from, but i think this would be incompatible with how set rules work

a set rule means that each element's fields are replaced by whatever you have in the set rule if it doesnt specify it

in other words elements behave exactly as if they had specified their own stroke identically to the set rule

it's true that the same is done for the table's global stroke in the current design, but that's not by necessity, just one design

so we can change it for the table's global stroke

[random mica]

perhaps then cells should have an internal priority field that gets ticked on each set rule application

[robust viper]

I don't think it makes sense for cell set rules to somehow be magic

The problem here is just really with table.stroke vs table.cell.stroke, not set rule vs non-set rule I think.

[wary knot]

yea, by changing it for table.stroke this will automatically mean that
#set table(stroke: red)
will have less priority than
#set table.cell(stroke: blue)

so you can still simulate that effect through set rules

with that said, the internal priority field is one way to go around table.stroke vs table.cell.stroke

but it wouldnt be a counter, just a bool

[robust viper]

hlines/vlines are always on top, right?

[wary knot]

they have priority, so they override the strokes of cells beneath them, but they arent always on top (thats the first review thread I opened in the PR)

[robust viper]

ah right

[wary knot]

you commented that it makes sense for them to be on top, and I think that sounds more consistent in the end so I'll change that

but it's worth noting that, at least in my envisioned fix, they'd still be below larger cell strokes

[robust viper]

I mean, larger is ok I guess

just same width, they should have priority

[wary knot]

yeah, sounds fair enough to me

and either way you can always override it by subtracting a bit of stroke...

nobody will notice...

😄

[robust viper]

but that's so hacky!!

[wary knot]

that's what we get for the lack of an explicit priority field

😂

well but i think it's fine for now, i doubt anyone will actually do that

[random mica]

we need some magic universal z-control

[wary knot]

Btw, i think in one of the reviews you mentioned the possibility adding left / right to vline's position (or maybe someone somewhere else did and i forgor), so i wanted to briefly comment on that

I intentionally had decided against that cuz i considered that it isn't an exclusively visual thing

In that , depending on your choice, you might or might not get an error

To be more explicit, if you used left or right at the end border, you'd get an error in one text direction but not the other

Which idk doesn't sound ok to me

I wanted people to get used to start / end cuz it's easy to just throw in left / right and be done with it without thinking about our fellow RTL mates

Anyway, but that's just what I considered. Maybe there's some stronger reasoning in favor of being more permissive either way

[robust viper]

While I agree that start / end is preferrable, it's inconsistent to disallow it

Since start and end are generally preferrable, but left and right are still allowed everywhere else

[wary knot]

yeah ok

I think we can add a warning in the docs then

I'd want to add an actual warning as well but

Without warning suppression we can't

:p

[wicked totem]

Yes, that is definitely the easier mental model for the users -
priority cascades like: Cell > Row > Table
Please take a look at here -
https://wiki.contextgarden.net/TABLE#Cell_Addressing_and_Frames

[wary knot]

i assume by Row here you mean that the stroke / frame of a cell from a later row (downwards) has priority over a cell from an earlier row (above), yeah?

i mean, considering the current proposal

[wicked totem]

May be my mental model is not very accurate about stroke - does the cell stroke covers - (top, bottom, left, right) ?

[wary knot]

correct

as such we kind of only have Cell > Table

in the high-level story

but in practice there are conflicts

so in that case a row from below wins

and a later column also wins

[wicked totem]

So you don't have a notion of row - row is an array of cells with x = <rownum> ?
Below is an example of the table (similar to HTML table) -

\setupTABLE[row][each][rulethickness=.25pt,offset=\dimexpr1mm+1.75pt] 
\bTABLE
\bTR \bTD 1 \eTD \bTD 1 \eTD \bTD 2/3 \eTD \eTR
\bTR \bTD 2 \eTD \bTD 2 \eTD \bTD 3 \eTD \eTR
\bTR \bTD 2 \eTD \bTD[rulethickness=2pt,offset=1mm] 2 \eTD \bTD 3 \eTD \eTR
\bTR \bTD 1 \eTD \bTD 1 \eTD \bTD 4 \eTD \eTR
\bTR \bTD 1 \eTD \bTD 1 \eTD \bTD 2 \eTD \eTR
\eTABLE

[wary knot]

yeah pretty much, setting the stroke for a row = setting the stroke for each cell with that row's y index

and yea I think I see what you mean

[wary knot]

this would be at the Table level

cuz you can use #table(stroke: (x, y) => if y == 1 { red }) for example

[wicked totem]

I dunno - can you "detect" that someone is trying set a sroke at the row level - if yes, it gets a priority (for all sides)...

[wary knot]

no, not at the moment
but I think it's fine

there's also the mechanism of explicit lines

so you'd get Explicit lines > Cells > Table

writing table.hline(stroke: red) above a row will place a red line above the row , overriding any cell strokes there

so you can use that to have further customization options

[wicked totem]

I think if we can document the rules of the stroke - then it is fine. I can help you with that..

[wary knot]

yeah we surely should

i'll probably have a PR just for docs lol

so we can discuss further there

though in principle i plan on writing something about that yeah

[wicked totem]

On rowspans - do you need any help with brainstorming or you have that under control ?

[wary knot]

i'll definitely need some help i think :p

right now the main blocker is mostly a matter of mathematics

lol

ive been consistently running out of time to think deeply about it though

which sucks

im pushing some stuff to the current PR but I'll try to summarize my problem in a bit

[wary knot]

ok

there was a considerable amount of stuff to tackle but most of the smaller stuff is done now

basically my current problem is regarding the algorithm to determine how much to expand the last auto row spanned by a rowspan such that the rowspan gets all the height it needs

and i spoke about this a bit before, so ill grab the context

here #1182377294682128445 message

in particular here #1182377294682128445 message

Context / motivation

basically the problem is that we need to somewhat "predict the future"

which is hard

actually, at the moment, we have all the tools to predict very accurately , but we have to use some maths

cuz consider for example

rowspan has a height of 10em

and it spans the following rows (assume it is the only cell in the table):

4em auto 4em

now

remember that our priority is that the rowspan gets all the height it needs

so

right now it has 8em, so what's missing? 2em

so auto there will be 2em

why didnt auto expand to the rowspan content's full 10em? cuz that'd make the auto row much larger than it needs to be

we just need to make sure the rowspan has the height it needs, but we shouldnt exaggerate

now let's twist it up a bit (and this is the case of interest)

add a row gutter of 1em to the mix. note that, since rowspans also span the gutters between the rows they span, they also contribute to its total height.

we'd have something like

4em 1em auto 1em 4em (bold = row gutter)

now, you might think. If you add all of those up you get 10em, so the rowspan (whose contents have a height of exactly 10em) already has all the height it needs and auto doesnt have to expand. right?

well yeah, normally

so auto would become 0em there

but theres a problem

if the table is placed at the very end of the page, it's possible that the last row (the second one with 4em) is moved to the next page

and then you get

4em 1em auto (pagebreak) 4em

why does this matter? well, the pagebreak "replaced" the gutter of 1em, but a rowspan cannot "span the pagebreak" - it must stay within the table's boundaries

this means that, in this case, the auto row has to expand (by 1em) to compensate

but note that, in principle, we only know if this pagebreak happened after it happened

this means that, when we're determining the height of the auto row, we dont know yet (in principle) whether or not there will be a pagebreak right after us

we need to predict that

note that anything before the auto row is already laid out and the heights are final

but anything after the auto row has to be predicted

so that's my problem

that's what I have to find an algorithm for

a way to calculate how much extra height the auto row needs, already predicting which row gutter spacing will be available or replaced by a pagebreak

Available tools

• We know how much height the rowspan needs in each page (how much height it needs in the auto row's page, in the page after, etc.).
• At any moment, we can tell how much space is left in the auto row's page.
• At any moment, we can predict how much total space will be available in the pages after the auto row's page.
• At any moment, we can access the list of rows which come after the auto row spanned by the rowspan cell, and their cells.
  • We can predict their heights easily , since they aren't auto, so they either have a fixed height or they are 1fr-like rows (consider that they always have 0pt height to simplify things, so they are also fixed height).

also for simplicity we are considering the auto row is just a single rowspan, we should be able to easily extend that into multiple rowspans i think.

How we could approach this

In principle we'd have to run a small simulation of laying out each upcoming row, subtracting from the available size, and then replacing gutter spacing between rows with pagebreaks as soon as we run out of space. We'd then simulate the next page, repeating the cycle, and keep doing that until we have simulated all spanned rows. We'd then be able to tell which instances of row gutter are removed or not, and thus prompt the auto row to expand based on the total missing gutter spacing.

The problem with that

So... the thing is that expanding the auto row causes the entire configuration to change, which could invalidate the results of our simulation.

In other words, depending on how much we expand the auto row, we'd push rows downward enough to cause some rows to (perhaps) unexpectedly move to the next page and make some particular row gutter be replaced with a pagebreak, while also reintroducing some other row gutter that was there before.

Note that row gutter isnt uniform - you can have a gutter of 1em between the first and the second row, and a gutter of 2em between the second and the third for example, so the gutter that "disappeared" could have been big / significant, and the one that reappeared, small. Therefore, the total height provided to the rowspan by the gutters would decrease, and thus we would have expanded the auto row less than we should have. In other words, the problem isnt solved.

The thing is, is this fear of mine actually realistic? I think I just need a simple counter-example. But, in principle, Im failing to logically refute my thesis above

So we'd need to increase the auto row just enough such that, even if it causes the table configuration to change and rows to move to other pages (despite its prior simulation which didnt take that into account), it'd still have expanded the correct amount

really, feels like we're looking for some sort of mathematical tool here, like a system of equations or some calculus magic or something. but im not rlly sure lol.
(Edit: I'm saying this cuz math is cool and math solves problems. But those examples of math tools are likely way off the mark. But saying that is fun so why not.)

[wicked totem]

I am trying to understand the problem statement little better - to me this looks like some kind of priority - the usecase describes a rowspan with height with some rows that is a mix of fixed + auto... something like below (ignoring the pagebreak issue for the moment) -

[wary knot]

ok so

i should clarify

when I say "rowspan height" or "height of the rowspan" I mean the height of the content inside the rowspan cell

for example, the rowspan height of table.cell(rowspan: 9832948, block(height: 5em)[a]) is 5em

(in principle)

cuz the content inside it demands 5em

so our task is to make sure the last auto row it spans expands enough that the content of the rowspan will get the height it needs

(we can simplify and assume that it only spans a single auto row, anything before that isnt relevant)

[wary knot]

so yeah thats more or less it

since it only spans a single auto row (by our simplification), anything after that auto will be fixed

(let's ignore fractional height rows, assume they are fixed with a height of 0pt)

[wicked totem]

if the rowspan cell gets what is needed (it is higher priority), then we adjust the height of auto row as you have suggested - it got 8 from the 2 other fixed rows and auto gave it the rest ?

[wary knot]

Correct

(assuming no gutter)

[wicked totem]

And as Laurenz suggested, you layout as if rowpspans do not exist first and then "layerin" the rowspan in the subsequent pass ?

[wary knot]

yeah sort of

here im simplifying cuz, assuming everything is fixed, we dont have to layout

we'd layout only to figure out the heights, but we already know the heights

so we can just use the heights directly

but the idea is the same

what changes is like instead of row.layout().get_height() at each step we use row.get_height_cuz_we_already_know_it()

(in practice not everything will be fixed when repeatable headers are thrown into the mix. but let's ignore that for now 😄 )
(edit: actually repeatable headers only influence the size available, but they obviously arent spanned by the rowspan. so just ignore that entirely)

[wicked totem]

I am thinking from a majority usecase point of view and as this it he 1st implementation - something simple first... most of the time rowspans are of some grouping of the rows - so you give it what it wants and then divy up the rows it spanned for the individual rows

[wary knot]

i mean

the current implementation i have is pretty simple in that sense

i just sum everything that comes after the auto row, and subtract it from the height we need to expand

very simple

the problem is that some of the things that come after the auto row are gutter, and gutter can disappear when it would otherwise appear at the bottom of the page (it has to appear between two rows in the same page)

so thats the problem..

we could also only enable unbreakable / single-page rowspans for now

that'd be a solution until we figure that out

but i present the problem anyway in case something is figured out

i dont wanna just give up on it :p

but of course it's acceptable to delay the solution to it if it means getting at least some rowspans into the next release

noting that tablex never had to handle this problem cuz all of its rowspans (and all of its cells, in fact) are unbreakable

[wicked totem]

To me single-page row-spans looks like a acceptable compromise - for the 1st release - to get some initial feedback...

as usually the content needs to be repeated every page for multi-page row spans... as a reader in the 2nd page the rows are grouped but the content is in the 1st page - reader needs to flip back etc

and use the time for heading row repeat - that would be my 2c on this ...

[wary knot]

yea

i think that if we dont find a solution soon we should probably just head in that direction

note also that theres an alternative, which is to kind of intentionally botch it up a bit

make rowspans breakable, but predict sizes properly up to the first page

[wicked totem]

Of even if you find a solution but it would take a lot to explain to the user - then that is a sign that something is not very correct...

[wary knot]

after the first page we'd just ignore all gutter rows and thus have the auto row expand more than it needs

it'd be ugly to have the huge auto row yeah but it'd work

😂

so it's probably not the best idea for now

[wary knot]

i think we wouldnt have to explain much to the user, since it'd be the expected behavior

assuming we solve it in some ideal way

anyway but that was helpful feedback

thanks for staying along

:p

i'll wait for some feedback from @robust viper

[wicked totem]

Thank you - ok, let's wait to hear from Laurenz

To me having rowspan + gutter - not sure what is the usecase - may be we can ignore gutten altogether when we have rows with rowspan ?

[wary knot]

it'd be basically this approach

we make it look "a tad worse" when theres gutter

we can also just error and say "dont"

lol

but my approach works just fine when theres no gutter

even with repeatable headers in the mix

well maybe some small simulation would still be good for the most precise results

but it would still not face the gutter problem

[wary knot]

bruh

re-running (without any changes) fixed it

i can attest that I did not tamper with anything this time!

i mean yes my git tree is dirty but my changes dont even affect that file 🤔

anyway cc @robust viper cuz the thing happened again lol

[robust viper]

I think this sounds fine. It will be a very rare case and we can look into improving it later.

[robust viper]

since this has never happened on main, I would assume that something is up with introspection and grid layout (some bibliography's are grids). I will look extra carefully at the things that could affect introspection when reviewing.

[turbid granite]

My leetcode alert went off reading this. It sounds like a dynamic programming problem? Like making the best change with certain values of coins?

Or the knapsack problem where you need to optimize choices of weights to maximize value?

[wary knot]

Yeah, if we don't manage to do it properly then let's at least have something

Though it does mean it will be a "breaking change" of sorts to fix it later

But anyway, shouldn't matter that much

As you said, it's gonna be rare

[wary knot]

Sounds like you might be onto something

You have to find the shortest amount possible you can expand the auto row by

With the condition that the rowspan cell gets at least the space it needs... And preferably not more than that

So sounds a bit like some sort of shortest path in a graph problem

But I don't really know how to model it

[robust viper]

I wouldn't call it that, it's just a bug fix / improvement.

It's generally not at all clear that we should really consider layout improvements breaking changes. SILE for example doesn't as far as I know.

[wary knot]

Fair enough

In the coming days I'll try to make a sketch of the "sorta botched up so not perfect" algorithm , if it works out I'll head towards cleaning up stuff to make it a PR

[wary knot]

lol

@robust viper im not sure but i think your call to replace tons of prepends with prepend_multiple actually sped up tables a bit lol

[robust viper]

haha nice

[wary knot]

my benchmark insane documents are now faster on my PR than on main

lol

[robust viper]

yeah, it was already botched on main

is the PR merge ready by the way?

[wary knot]

almost, im taking a look at the rayon thing now (hence why I'm running benchmarks haha)

[robust viper]

the folding with default and (x: none, y: red) feels a bit unfortunate to me, but I see now why it happens here and not elsewhere

[wary knot]

yea i see where you are coming from

im not sure how to best fix that

we could add auto and do some manual synthesis but idk

[robust viper]

it'd be kinda dumb to add auto just to disable folding

[wary knot]

yea

maybe it's something we have to change in the proc macro

[robust viper]

when we disable folding on the arg altogether then user stuff wouldn't accumulate anymore, right?

[wary knot]

yeah

[robust viper]

I mean, it is consistent, it's just weird because it's not the way things work for other elements

[wary knot]

multiple #set table(...) would probs break i think

[robust viper]

but that's just a coincidence with them using auto

[wary knot]

yea

we could have some #[fold(non-default)] kinda thing

is there any place where we actually use this fold with default thing though?

lol

[robust viper]

I was just about to ask whether you're aware of such a place

[wary knot]

i guess inset would also fall victim to this

[robust viper]

But I guess being aware of that is my job

[wary knot]

and anywhere with stroke basically

...and not auto

[robust viper]

yeah, if you do (inset: 10pt), it'd get rid of the 5pt default

but in this case I don't feel like that'd actually be desirable

[wary knot]

yea

[robust viper]

I am inconsistent

noo!

[wary knot]

😂

requirements are complicated

[robust viper]

what's up with this insane desire for consistency that we computer people have?

[wary knot]

yeah i cant explain that either

[robust viper]

I guess it's fine as it is for now. Tell me when it's ready to merge

[wary knot]

ok well, i didnt implement the grid.cell errors yet but i think i can bundle that up with inset: (x, y) => ... in a follow-up PR

i'll ping ya

[wary knot]

also obligatory @frozen oyster

😎

okay so i added strokes to the 10000 tables benchmark

and wow

i got really scared lol (edit: this is RAM)

i was almost Ctrl+C'ing lol

but it went through

61.7s (with a stroke for each side for each cell) compared to 47.06s (default stroke options)

on main the latter is 49s

lol

okay

the benchmarks hath spoken

rayon is Slower

lol

i was flabbergasted

went from ~61s to ~63s

on the 10000 tables benchmark

and on the 400 tables benchmark went from ~1.6s to ~1.7s

bad rayon

[random mica]

I find something so elegant about emergent simplicity through abstraction. Is that not what programming is all about?

[robust viper]

It is

[wary knot]

@robust viper lol i was reviewing again and i found a small problem

whoopsie

well i did write the wrong span in the error but point was made

😂

ok fixed

very tiny and inconsequential thing but getting it right is nice

will keep taking a look now

[grand haven]

Time to buy download more ram

[wary knot]

@robust viper

[robust viper]

yes, sir

[wary knot]

It's ready!

I made two commits after my final review

fixed the thing I said above and removed some useless #[allow(dead_code)]

and that's it

[robust viper]

Okay, I clicked the button

I trust that the comments are adressed, I can't go through those 2000 lines again ^^

[wary knot]

yeah dont worry haha

[robust viper]

very nice work!

[wary knot]

thanks 😄

this week is gonna be a true battle for me, cuz there's just so much demand at once 😂

gotta do rowspans, do internship stuff, go to the doctor, and other stuff

it do be like that

😂

but we can do this 💪

[wary knot]

well, obligatory announcement then

Line customization merged!!! https://github.com/typst/typst/pull/3393

🚀 🎉

[robust viper]

two small bugs:

• #table.hline(stroke: none) to override cell stroke doesn't work
• show table.cell: set table.cell(..) doesn't work. but that's the same as with figure.caption and probably on me to fix.

[random mica]

can't you diff the latest commits since the review?

[vernal herald]

No! We'll test it live damn it!

https://tenor.com/view/we-will-do-it-live-do-it-live-angry-anchorman-anchorman-gif-9159361

[robust viper]

I could, but it's not always helpful. The comments were also mostly minor and I trust that PgSuper resolved them.

[wary knot]

the first one isnt really a bug , I intentionally made it that way

as in, that's the requirement I had understood

we can change it easily

just lmk

the second one i have no idea 😄

[robust viper]

I'm not entirely sure but I'd have thought that the idea was that hlines always override normal strokes and that's useful.

is one more consistent in the code than the oder? that's typically a sign of general consistency.

[wary knot]

yeah, i guess a hline with stroke of none is also a bit useless rn

other than to override other hlines

well then it's not useless i guess

😂

so it could be thought as a way to "disable overrides" for a particular portion ig

[wary knot]

Well tbh

At that point you might as well just split your lines then

lol

so i think it makes more sense for them to have proper purpose beyond that

That is, to remove cell strokes across a certain part

[wary knot]

Though

I think the current system is the way it is cuz cell strokes are similar in the sense that a cell stroke of none doesnt remove anything

[wary knot]

Important update

I spoke to the doctor just now and guess what

They have successfully learned about Typst

😎

#RIIT must live

[random mica]

cultcultcultcult

I mean–

[upbeat pine]

We do have some characteristics of that, but with such definitions, a ton of communities can be treated as a cult. Maybe cult doesn't have to have a bad meaning. But if I will proudly say that I'm in the Typst cult, something bad will happen, I think. Like sirens and stuff.

[random mica]

no no, I think cult does have to have a bad meaning.

[upbeat pine]

In that case...

leaves the server

[wary knot]

It's not that serious lol

You don't have to pretend you're not in the cult

We all know what you are

[frozen oyster]

https://tenor.com/view/christine-sydelko-i-am-shooketh-shooketh-shook-iamshook-gif-9804999

about rayon being slower *

[grand haven]

Or just skill issues

[wary knot]

Maybe we should just all_elements.par_iter()

Whoops i just invented multi-threaded typst

Bow down before me mortals

[wary knot]

sadge

i will go back to the stone age

where hints didnt exist

cc @robust viper we should probably do something about this at some point lol

but either way a simple string should do it for now

[wary knot]

well, this is interesting

right now we just repeat the strokes at the bottom border on every page

so the result is that it looks a bit funny there cuz of the gutter interrupting the lines

but i wonder if folding the bottom border with the bottom stroke of the last cell in the page would be more appropriate

otherwise the cell's bottom stroke becomes useless

i guess the same could somehow apply to a repeated top stroke and a cell's top stroke

but at the same idk what the best thing to do here would be

maybe we could fold by default unless you specify a custom outside: stroke

[bold bluff]

Looks like a great corner case. I’m guessing it is not expected behavior.

It looks different when not on a page break doesn’t it.

[wary knot]

yeah

like if the cell had stroke: (bottom: blue) it would be ignored afaik

[bold bluff]

It looks like it could cause some stress during use. Are you wanting fix it? Or something else?

[wary knot]

im not sure rn

i can see how this would be annoying

to the point that im considering if we could restrict this behavior to just explicit hlines placed on the bottom border

but im not gonna concern myself too much with this atm

if it has to ship like this, it's not that bad, just possibly mildly inconvenient for some cases

[wary knot]

btw @robust viper is there anything i can do to help fix the #show grid.cell: set grid.cell problem you had? like anything i should know or implement

[wary knot]

Regarding rowspans:

I think I have managed to make an initial version of the "semi-botched" algorithm

It's probably totally broken right now, but it's the first thing I could make that compiles

Woohoo!

Now gotta proceed to a long session of debugging and cleanup haha

(Tomorrow!)

[wary knot]

first test results with the new algorithm are out

an auto row completely disappeared

when it shouldnt have

great sign!

😂

ok fixed that. nice

[wary knot]

so

i couldnt help myself

and I did a tiny little bit of debugging okay I swear I will stop now

and uh

so far it appears i have fixed the box example I gave, and it doesn't even run my simulation lol

[wary knot]

nvm i tested the wrong thing

sadge

[vernal herald]

If anything, it is the Saga that I cannot quit. Keep going or take rest oh Knight.

[robust viper]

The problem is that you populate the cell's fields before show-set runs (which is when the cell itself is layouted), so by the time that happens it can't do anything anymore. To fix this, we'd need some sort of eager styling API that you could call.

[wary knot]

ah right

we'd have to somehow run the show/set rules earlier

i was gonna say that we could postpone the field population but we cant cuz we need to synthesize stuff for Cell { }

so yea

[wary knot]

okay i managed to make the simulation actually run

and the result is kinda funny

basically right now im trying to make some sort of "retry up to 5 times" thing

lol

and it entered an infinite loop

as a disclaimer, this is the second approach im experimenting with, the first one was limiting the amount of pages we'd simulate for, but it didnt go very far

lol

eh

ill just play with it more

lol

ok i think i have a clearer idea of what to do

basically the simulation will only expand the auto row further and further, never goes back

which will result in a bit of imprecision at times but thats ok

but, to compensate, we can just stop the simulation earlier when we detect that the rowspan has already been fully covered by the rows we have analyzed so far

that is, future changes in gutter wont make a difference

so it's certain that the simulation will eventually stop (though i'd still keep an upper bound of iterations just in case)

[wary knot]

omg

you guys have no idea how long this took to debug

i was almost freaking out

😂

i swear i read the algorithm's code like 5 times trying to find the flaw in my logic

i even refactored it and stuff

but that miserable line

was the bane of my existence

😂

[vernal herald]

Listen... I can only offer a virtual hug.

[wary knot]

I'll take it 🫂

i added so many print calls 😂

truly amazing

sure i could use a debugger but im too lazy to download one (im not in my main PC rn, which does have one)

anyway...

with that

the block example works.

top one doesnt use the algorithm (no gutter so who cares)

bottom does, and it used to expand too much or too little

now it's just right 👍

[wary knot]

regarding rowspan layout order: in principle it seems that laying them out at the end isn't that bad regarding introspection; see this example with a counter stepping for every cell

(there are two rowspans there: 4 and 14)

i guess the thing is that the rowspan is added at the top of everything else

but still we cant lay it out before we actually reach its end

so we'd necessarily have to add some .inserts if we wanted things to be laid out correctly

...

unless

we push some placeholder empty content and later just replace it

:p

[robust viper]

Looking good!

@wary knot do you think you could finish 0.11 tables until Friday? I really don't want to put any pressure on you, but the release has got to come at some point.

we can also release without rowspans if need be, but so far you seemed to be optimistic, so maybe we don't have to

[upbeat pine]

Classic "no pressure" — proceeds to put pressure.

[robust viper]

I didn't say "no pressure", I said I didn't want to put pressure ^^

[upbeat pine]

I didn't say you said "no pressure," I said it's a classic situation which follows the same pattern. ^^

[plucky haven]

could we do a summary on incoming 0.11 table api? I'm sorry that I cannot follow thousands of messages in this forge. 😵‍💫

[sly sparrow]

this itself should be a good overview https://github.com/typst/typst/issues/3001

[plucky haven]

I find that I can also search "flexible tables" in the page to check landed efforts by examples: https://github.com/PgBiel/typst/tree/main/tests/typ/layout

[lone nova]

Just a small request - can we also include repeating headers in the upcoming release?

[robust viper]

That will likely take too long. The last release is already very long ago.

[lone nova]

🥲

[wary knot]

i think so but let's stay in touch

I'm pouring all my energy into getting this going, but it's also possible that a buffer of like a few days / the weekend could be helpful

I think rowspans are pretty much almost ready, bar unbreakable stuff (working on that rn), but they really need some cleanup and testing

Regarding repeatable headers, I'm gonna study what the plan will be
I definitely wont be able to have a full-featured thing with multiple repeated headers plus footers and whatnot by 0.11

But it's possible that a simple version with just a single header repeated all across the table (like in tablex) could be viable, but I'll confirm that later

[wary knot]

refactoring some stuff right now and it seems there was a tiny and extremely specific bug with the colspans implementation

for the heuristic of having a colspan not expand auto columns if it spans all fractional columns, I was using the pre-gutter colspan to check if a fractional column was spanned , meaning it would be wrong with gutter enabled

now i introduced a grid.effective_colspan_of_cell helper method and am using it everywhere so that wont happen now : )

[wary knot]

hmmm

i was thinking here

it's possible that i'll always need to simulate an auto row with rowspans (unless the rowspans are all unbreakable)

as in, gutter isnt the only problem

because relative lengths are also a problem

it's rare in practice, but it's possible that the table might span regions of different sizes

and then a row with size 50% would have a different height depending on which region it ends up in

thus i might not be able to take the cheap exit of just adding up all of the non-gutter relative lengths, resolving them at the current region instead of simulating the possibility of them appearing in further regions

(does this check out / is it valid to be concerned about this? cc @robust viper )

[robust viper]

I think it's a valid concern, but not necessarily for 0.11. It doesn't have to be perfect from the get-go. Since tablex can't do rowspans across pages, it's already a step up anyway.

[wary knot]

fair enough

[wary knot]

ok

[wary knot]

i had posted some messages about breakability stuff earlier but i will postpone that decision until the PR is up

[wary knot]

ok i think i actually made it work by refactoring the algorithm a bit

not that I needed that feature, but it came as a side-effect...

but I only trigger it when gutter is enabled anyway

for now the non-gutter case should be as simple as possible

but basically i think the algorithm is more intuitive now

it will:

• measure each upcoming row (excluding removed gutters) and sum
• subtract the measured amount from the sizes of the rowspans, this will be how much we have to expand
• then simulate again, if nothing changes OK, otherwise try again (up to 5 times rn)

the first point has a bit of complexity cuz there are some extra variables when measuring upcoming rows, in particular sneaky unbreakable rowspans contained within larger breakable rowspans (lol)

but thats about it

i think the algorithm should be mostly good to go now

i just pity you cuz it's a lot of code to review 😂

to the point I made a separate rowspans.rs file to help (it can be easily undone if needed)

but in the PR description i will lay out my mental model more clearly

cuz i think there are basically four main points to the changes:

1. User API and CellGrid (as usual)
2. Basic rowspan layout code (rendering fills and strokes, actually placing them in the finished frames)
3. Unbreakable rowspans (concept of unbreakable row group, which must be laid out in the same region)
4. Breakable rowspans (simulation required)

so i made the rowspans.rs file where i grouped up methods for the last three things (bar changes in the already existing functions)

anyway

PR should be up in the coming days

👍

just need to add like 100 tests 😂

[wary knot]

this was the first test I ran after refactoring the algorithm for the last time lol

it passed on first try 😎

[wary knot]

okay

i managed to make it so rowspans are drawn as soon as their last row is reached

that way , counters (and anything depending on cell order) behave properly

the main problem here is that the last row might be removed if it's a fully empty auto row. I made it so it wasn't removed, but that generated some ugly visual glitches with empty rows. So the solution was to draw at the last row or the row immediately after, and then do a final pass at the end (draw any missing rowspans, though a rowspan which got this far has to be the edge of edge edge cases!).

anyway. yeah

oh yeah theres also the fact that a row might cross multiple pages

so the rowspan was drawn at the first page

lol

i made it so those "multi rows" indicate when they're "done"

👍

[wary knot]

Draft PR is up: https://github.com/typst/typst/pull/3501

Still gotta fill out a few TODOs before undrafting

but yeah