Tables

[wary knot]

that seems to be in monospaced font

or something of the sort

[grand haven]

it's just whatever font is the default

[wary knot]

hmm

?r #set page("a7") #array.range(9812,9824).map(it => eval("\\u{"+str(it,base: 16)+"}",mode:"markup")).map(x => [#x]).join([, ])

[wary knot]

there it is

though i think i have thought of another example already

i could make a table of student grades

and then fill a column for some exam where everyone got 100% score

or something like that

lol

[wary knot]

weird lol

[grand haven]

I wonder what font it's falling back to in the first place

[wary knot]

right

that makes more sense

the CLI doesnt have it by default

[grand haven]

if you set the font manually to dejavu sans it'll use it for all the symbols instead of randomly falling back on the last symbol for whatever reason

it seems like the emoji font has priority by default?

that being said, noto sans symbols 2 looks a lot better

stix two math also has them

[upbeat pine]

I mean, the idea of playing real chess in Typst using chess moves, I guess (instead of wasd).

[wary knot]

ok i think i'll use this example

a little bad but whatever lol

we can always replace later

[wary knot]

Part 2b is ready for review 🎉 (cc @robust viper )
https://github.com/typst/typst/pull/3050

This should hopefully be faster to review since the changes are more localized.

Docs can take a bit longer to review ofc, but if that becomes too big of a blocker we can just remove what's bad rn and fix later (dont want to slow down table work too much)

[keen crescent]

Ooo, I like

[trail oar]

Someone’s on fire 🔥

[robust viper]

Martin and I always have strong opinions on docs and examples, so I would suggest to focus on functionality first and get things merged more quickly and then have one final PR for the docs where we can then also just make some adjustments by pushing ourselves.

The docs are always hard to judge without viewing them built of course.

[wary knot]

Yeah that's fair enough

I was mostly worried about maybe having some fundamental blocker or something which we could just leave for later (like we did in the previous PR)

But I think that approach is probably the best

Either way please take your time, didn't mean to rush you or anything, I guess I didn't word that in the best way :p

Perhaps we can conciliate both approaches - of per-PR docs and one large docs PR - by having a docs PR every time we're nearing release?

That way I think it'd make me less worried about things being underdocumented 😂

I think starting from the next PR I'll try to focus on such an approach

[robust viper]

Yeah that sounds good!

[wary knot]

Great 🚀

[robust viper]

No worries, I didn't feel rushed

[wary knot]

Thanks for the confirmation 🙂

[wary knot]

oof

unscientific benchmarks reveal that my totally illegitimate usage scenario of tables with 5000 cells takes 70%-80% longer to compile with Part 2b
(same for a... maybe more realistic document with 2 tables of 400 cells each)

I guess there isnt much we can do about that 😛

[upbeat pine]

70–80% is substantial. if the code is slim/optimized, then there is really nothing we can do about it.
You should've made a few tests to draw a graph "cells count/compile time" to get the idea if it is an issue (for smaller tables).

[wary knot]

well i mean

it's not "catastrophic" in that the two documents already take very little time to compile

so the one with 2 tables of 400 cells went from like 5.4ms to about 9.8-10ms for example

while the one with 2 tables of 5000 cells went from 700ms to 1.22s or so

although those "went from" numbers are relative to current main (so after Parts 1 and 2a), so the accumulated increase since 0.10.0 is somewhat higher

either way Part 2b has been the most significant so far in this regard

i was expecting it to be bad but not that bad 😂

but yeah i mean idk if theres much we can do about it, theres much more code running in general so

[upbeat pine]

oh, well I thought we are talking seconds...

[wary knot]

Well, we would be in the case of 10000 tables with 5000 cells each

Which is one of my test documents

I haven't measured it yet but it would likely go from 32s to almost a minute

But I mean, I guess 10000 tables of 5000 cells isn't a very common scenario...

😅

[upbeat pine]

Unless you're a mad scientist.

[frozen oyster]

Well what I can tell you is that I've seen my dad having excel sheets of customers of around 10M rows each with ~20 cells

so it's not entirely impossible if you're doing spreedsheet-like functionality in Typst

[wary knot]

wow

well that'd likely take long in Typst anyway 😂

for various reasons...

i wonder how much RAM that would take

lol

a "simple" table with 20 columns and 10M rows filled with empty cells

[upbeat pine]

That phenomena is called "Excel Database", and a lot of corporate people like this or held hostages of "the legacy way". I of course hate this, because it is stupid, and it slows down the UI significantly. I read stories about such a "database" where each action takes several minutes and is considered normal. It can be used "in normal amounts", but a lot of examples show that people crank this to the extreme.

[frozen oyster]

that kind of stuff will likely improve with your rework in pure rust, right? RIGHT??

[wary knot]

depends on what you mean by "improve" 😂

if you mean compared to tablex, then without a doubt

if you mean compared to old tables, well, tables will be slower now (noooo)

but on the upside they will have more features

i mean

tablex cant even compile a single table with like 4000 cells

without taking forever and OOM'ing

while new default tables can compile 10000 tables with 5000 cells each in ~30s (maybe ~1 min after this PR :p)

so i dont think tablex would have been very useful for your dad

[frozen oyster]

https://tenor.com/view/i-can-fix-that-let-me-fix-it-sad-worried-gif-13459848

[wary knot]

whoops

accidentally used a Rust 1.73.0 feature lol

would have been very handy though

[grand haven]

Is that a problem?

[sly sparrow]

Yea aren't we on 1.75.0 or did you mean a different version

[wary knot]

yea, CI failed cuz Typst MSRV is 1.70.0

[frozen oyster]

I have been on a break for like a month now

I only got like 5 PRs merged in that time 😭

people are catching up 💀

[grand haven]

Live a little, use nightly features

[wary knot]

Right i didn't think of that

I'll enable all nightly features for Typst now

Including yeet_expr

Thanks !

[frozen oyster]

I wish we could, there are some crazy optimizations you can do now that traits can return impl Trait and with GATs

[grand haven]

I like new shiny things

[frozen oyster]

same

[grand haven]

Is there a policy to be x number of versions behind?

[frozen oyster]

no afaik we can stay up to date with stable rustc

but we need to ask @robust viper to be sure

[wary knot]

well, some packages use the debian version as a reference

debian sid has 1.70.0 rn

so it's fair enough

and it also takes a bit for new versions to fan out to non-rustup places

with that said, i think we shouldnt bump unless theres something we need

like Laurenz used Rust 1.74.0 [lints] for example cuz it's useful

although that doesnt bump MSRV so it's fine

[frozen oyster]

my best friend's company recently introduced rust for linting SystemVerilog code (they use an open source linter + their own lints added which they don't contribute back :/) and they don't have rustup, just cargo and rustc

and it's a bloody mess

[wary knot]

yea

the nixpkgs stable version is also a good reference

image died for some reason but it's 1.73.0

[frozen oyster]

why image no load?

[wary knot]

;lol

idk

[frozen oyster]

discord broken

[wary knot]

on nixpkgs unstable it's 1.74.0

on stable it's 1.73.0

with that said

regarding RPITIT

i think we should wait until they add the return-type bounds checking stuff

otherwise we might run into papercuts

[frozen oyster]

kesako????

[wary knot]

🤔

also 1.76.0 will add dyn Trait upcasting

nice

[frozen oyster]

(a slang for what does it mean in early 2000s French forums)

[frozen oyster]

what do you mean?

[wary knot]

like

if A is supertrait of B

then you will be able to use A stuff in dyn B

or smth like that

[frozen oyster]

oh right

that's neat

Might cleanup some of the trait objects we use!

[wary knot]

this test should sum it up

[frozen oyster]

lol

[wary knot]

gahhhh discord

[frozen oyster]

baka images don't work

[wary knot]

link is https://github.com/rust-lang/rust/pull/118133/files
scroll to the very bottom

[frozen oyster]

fn into_any(self: Arc<Self>) -> Arc<dyn Any + Send + Sync>;

Actually right now we use this fact

and it's disgusting

so good to know we'll be able to remove it

[wary knot]

lol

nice ™️

[frozen oyster]

    /// Downcast the element into an owned value.
    #[inline]
    pub fn unpack<T: NativeElement>(self) -> Option<Arc<T>> {
        // Early check for performance.
        if !self.is::<T>() {
            return None;
        }

        Arc::downcast(self.0.into_any()).ok()
    }

We actually use it to do downcasting lmao

Glad to know we'll just need to do Arc::downcast(self.0) in the future

anyway, sorry for the #off-topic stuff 😄

[wary knot]

you were forgiven

[frozen oyster]

I just realized that method is not used anymore 😂

[wary knot]

@robust viper is this more or less what you were thinking of?

cuz it doesnt seem to add the grid call

unless im misinterpreting the purpose of this

[upbeat pine]

Aren't you supposed to change the number of columns first? This is how tablex works now.

[wary knot]

oh okay

this is what i was looking for

so I think that works

[robust viper]

the tracepoint is intentionally omitted if overlapping

[robust viper]

this is what I meant. as mentioned in the comment, we could also skip that and write a more general solution for "error occured while layouting this X".

[wary knot]

yeah fair enough

i think we can add this for now, it's useful and changing it later wont hurt

[robust viper]

sounds good

[wary knot]

👀

gonna be experimenting a bit for now

I was thinking of potentially splitting the merging cells PR in two (colspans and rowspans) if rowspans become too complex

but we'll see

honestly i'd be fine with making an initial release of table features with just colspans (if deemed necessary) since that'd already be a huge improvement over the status quo

let's see though, maybe my ideas will work...

[frozen oyster]

https://tenor.com/view/sussy-baka-breaking-bad-meme-gif-24570545

[wary knot]

Part 2b merged 🚀

now you can do these

or even this

next up: merged cells

ive started preliminary work on it, but rn im gonna focus on colspans first, rowspans later

[wary knot]

👀

well, this is almost PR material (if we consider splitting Part 3 into colspans and rowspans)

however I havent fixed strokes yet

i also find this one to be pretty cool

hmmm

it appears i screwed up somewhere , given that this (below) is the equivalent tablex output lol

will investigate

ok

investigation concluded

mistake found ⚠️

huzzah 🚀

[upbeat pine]

Can you also show the code? To get the "full immersion".

[wary knot]

yeah sure

#grid(
  columns: 4,
  fill: (x, y) => if calc.odd(x + y) { blue.lighten(50%) } else { blue.lighten(10%) },
  inset: 5pt,
  align: center,
  grid.cell(colspan: 4)[*Full Header*],
  grid.cell(colspan: 2, fill: orange)[*Half*],
  grid.cell(colspan: 2, fill: orange.darken(10%))[*Half*],
  [*A*], [*B*], [*C*], [*D*],
  [1], [2], [3], [4],
  [5], grid.cell(colspan: 3, fill: orange.darken(10%))[6],
  grid.cell(colspan: 2, fill: orange)[7], [8], [9],
  [10], grid.cell(colspan: 2, fill: orange.darken(10%))[11], [12]
)

the bottom one is the same but with gutter: 3pt

[upbeat pine]

Man, the grid.cell starts to annoy me just from looking at it. It's soo much useless visual junk (and it also highlighted in 2 colors with js).

[wary knot]

well idk

we can bikeshed later

this is at least the bare minimum we need for it to work

whether we should get something like grid.colspan(4)[*Full Header*] is debatable so , to speed things up, im leaving it out for now

[upbeat pine]

I'm just saying, that in the end it should look much cleaner and more readable.

[wary knot]

well

problem is that it's a bit hard to determine what that entails :p

at least for me it's saying clearly what it is (it's a grid cell spanning 2 columns and with an orange fill with the text *Half*)

though I guess you can write some helpers if you write a lot of tables

[upbeat pine]

Yes, but some stuff is simply too obvious/verbose. At least the second time you will use it.

[wary knot]

well, ig you can do something like #let cell = grid.cell in that context

though we had to split them into grid.cell and table.cell to have different show rules

[upbeat pine]

I think you can import it, right? Or not yet?

[wary knot]

yeah that should work too

[upbeat pine]

I'm thinking that you can't add all aliases to std, so in tablex or wherever you can add stuff like gcell, gc, tcell, tc, gcs (grid column span), grs, tcs, trs. It will substantially shrink the amount of stuff written inside tables, therefore it will be easier to focus just on cell content. Everyone would want to just import "std" aliases without defining them for each document. So, either you will have to add something like this, or I will. Yes, that's a threat. /s

[wary knot]

i think that would be a bit out of scope for tablex

what I'll probably do instead is allow you to use native tables as a backend for tablex

so you can use tablex syntax at will

[upbeat pine]

hmm, so you will keep the non-native backend for tablex? Isn't it will be slower and just pointless (unless it has cooler features)?

[wary knot]

it's slower but at least i can control what it does :p

with the native backend some tablex features wouldnt work

in particular map-cols / map-rows if i make it a frontend as well

[upbeat pine]

Well, right now, you are also controlling what native tables can do. So what's the difference.

[wary knot]

well

that's only half true

there's another person involved

:p

[upbeat pine]

you mean @robust viper? I guess you're right.

[wary knot]

ye

either way dont worry too much about it

i'll probably have a "half-slow" native backend at first

tablex generates the grid and then just dumps it to the table

that would allow map-cols and map-rows to remain working

[upbeat pine]

I'm just looking from an optimization/file size perspective. I mean, if native tables will have everything the tablex have, then there will be no point in tablex, maybe other than playing with new stuff (that are built on idk which backend in Typst).

[wary knot]

oh yea it wouldnt make much sense, but it also wouldnt make much sense to not make the option available :p

though tablex will also gain a CeTZ backend in 0.1.0 so it will remain useful

[upbeat pine]

In the end, will tablex just be archived or maintained for bugs?

[wary knot]

i'll probs still maintain it

just not as much as now

[wary knot]

anyway, here's the latest progress for the rest of people 👆

colspans soon™️

[wary knot]

welp

I'm on track to have proper line splitting and thus finish colspans

i think i made the algorithms work, at least I tested them on the Rust playground lol, will add unit tests later

fellow Rustaceans, please roast my code (sorry if it's alien, it's 2 am okay, but it werks !1111!!): https://github.com/PgBiel/typst/commit/cebb7f4bce5904e175b86108843005710e906c41

I'll sleep now

but today was a very productive day 🚀

[upbeat pine]

Nothing like going to sleep late at night after a productive (and fun) programming session. I miss that.

[wary knot]

I was thinking and I realized the first algorithm really sucks

I'll probably simplify it a lot tomorrow

So. Preemptive roasting

😂

[wary knot]

Yep definitely nice

Though I can't stop thinking about changes i want to make

😆

If everything goes well though... We should have a PR by Friday... (Maybe even tomorrow depending on how much time available i get)

[upbeat pine]

But it is even nicer to go past 4:00 and just forget about sleep. /s (why you are not sleeping?)

[wary knot]

(for colspans only)

[wary knot]

lol

Well the true answer is that there's some stuff I have to do before sleeping

So I was spending the past ~1h doing that

Health and hygiene stuff ya know

Now I shall sleep

. So yeah gn

[robust viper]

very nice!

[keen crescent]

Thats a really neat effect

[wary knot]

lol

after spending hours perfecting the line splitting algorithm (to split vertical lines so they don't go above colspans)

ive hooked the algorithm onto the line drawing stuff

and it seems i caused a microscopic change in a crap ton of tests

lol

gotta love it

[wary knot]

well

I've managed to turn on my microscopic lenses and found that I missed the mark by a bit on some lines due to an epic blunder

still got a bit to go though 😂

hmmm

[wary knot]

well ok

it seems that summing the resolved heights of each row in the region wont necessarily give you the total height of the current internal frame (which was the height being used earlier)

that's a bit of a bummer

though i find it interesting that it's excluding exactly one row

on all tests

i probs messed up some math somewhere 🤔

[upbeat pine]

the problem is you 99% of the time. tested, can confirm.

[wary knot]

yeah i think i just realized something

:p

[wary knot]

YEEEE

i was doing a bit of a dumby dumb

if ya get me

i asked each line to go from row y = 0 to row y = (amount of rows in the current page)

instead of row y = 0 to (amount of rows in total)

lol

not very smart

[vernal herald]

KEEP GOING

Apparently, #1176509648707256370 was still on-going, but it got removed from my discord list.. so I haven't been playing my cheerleader role at all..

[wary knot]

it is time for the truth

it is time to create a table and see if the lines will overlap

we will find out if ill have to spend 5 more hours debugging

aaaaand...

VICTORY

ladies and gentlemen

the goal has been achieved

[upbeat pine]

hmmm

[wary knot]

thats normal

also happens with current tables

all lines are expanded by thickness / 2 on both ends so hlines and vlines properly join

[upbeat pine]

never seen this with my tables. well, I never played with thickness that much.

[wary knot]

it depends on your viewer

with larger thicknesses it's not visible

[upbeat pine]

that means I use superior viewer.

[wary knot]

lol

anyway yeah

huge day for tables

this means we have all functionality needed for colspans now

just have to clean stuff up

[frozen oyster]

Congrats 🎉

[upbeat pine]

Great day for tables, and therefore, for the whole world.

(South Park reference)

[vernal herald]

HUGE DAY FOR TABLES

[wary knot]

thanks 🎉

i'll probably make rowspans a separate PR after all, there's already quite a bit to review lol

[trail oar]

grats!!

[wary knot]

thanks 😄

i forgot to check the output with gutter too but seems to be working smoothly as well ™️

it does look kinda funny though lol

but well, it is what it is

:^)

[upbeat pine]

classic

it's what it's

[vernal herald]

that's literally provocative.

loooook at that gap

[wary knot]

actually a huge chunk of the debugging took was cuz of gutter lol

and i didnt write a single table to test until now, all debugging was done through unit tests and breakpoints lol

[vernal herald]

https://tenor.com/view/you-dont-say-frowning-coffee-gif-14597430

[wary knot]

gutters have like at least 4 edge cases to consider

it's a pain

lol

[vernal herald]

europeans are sleepy now dude.. i have to log off, my brain is only meme'ing.

[wary knot]

cya 👋

[bold bluff]

It’s looking really nice. Great work @wary knot

[wary knot]

Thanks!!

[wary knot]

you know what

i might actually have messed up there too 🤔

definitely longer than it should be 😂

[upbeat pine]

my skills of nitpicking finally paying off!

[wary knot]

it's funny cuz i was doing this to test for another potential bug

and found a second one

lol

the potential bug is that im only applying the thickness offset at the start and the end of each line, not on each segment (e.g. you dont see it going slightly lower above the 6, only above 11)

but now im see that whatever im doing is just wrong

:p

still, nothing too bad, probably just some math fixes here and there

what matters is that we got a nice initial result

huzzah

[wary knot]

ok yeah

[wary knot]

it's precisely because of this

the first segment is offset by -thickness/2

the last segment is expanded by thickness

so what you get is a long last segment, and a short first segment

[upbeat pine]

wait, we can't change radius on table. will we be able to later?

[wary knot]

there we go

[wary knot]

no more artifacts

yey

i mean if you look very close you might see something (probably due to anti-aliasing or w/e) but if you do, then well, don't

😂

[wary knot]

?

you mean on the table border?

[upbeat pine]

block(radius)

[wary knot]

yeah so

you mean only on the table border or a per-cell kind of thing?

[upbeat pine]

I think you can add a per-cell rounded block in the grid

but you can't round the table

[wary knot]

well

technically you can

but only for a single-page table

and only if you disable strokes

with line customization that'll probably be easier since u'd be able to just remove border lines

and then you could clip

lemme see if i can reproduce that with tablex

[upbeat pine]

?r

#let cell = square(radius: 0.5em, size: 2em, stroke: white)
#grid(columns: 3, ..(cell,) * 9)

[upbeat pine]

hell yeah, I have a speed cube!

[wary knot]

?r t=l ```
#import "@preview/tablex:0.0.8": tablex, vlinex, hlinex, cellx
#block(stroke: black, radius: 0.5em, clip: true, tablex(
columns: 3,
vlinex(stroke: none), (), (), vlinex(stroke: none),
hlinex(stroke: none),
[a], [b], [c],
[d], [e], [f],
[g], [h], cellx(inset: 0pt, rect(width: 100%, height: 100% + 5pt, fill: red)),
hlinex(stroke: none)
))

[wary knot]

not bad

[upbeat pine]

but that's tablex

[wary knot]

yea

with line customization you'll probably be able to do the same on default tables

that'd be part 4

[upbeat pine]

?r

#set block(below: 0.2em)
#let cell(color) = square(radius: 0.4em, size: 2em, stroke: 1pt, fill: color)
#let face(color) = box(grid(columns: 3, ..(cell(color),) * 9))

#face(white)

#face(green)
#face(red)
#face(blue)
#face(orange)

#face(yellow)

[upbeat pine]

Great!

[wary knot]

just imagine the magic that will happen when you get access to #show grid.cell

[upbeat pine]

I cannot imagine. I will need a lot of examples that will try to flex the power of grid.cell as hard as they can. Then everyone will know how awesome the table 2.0 is.

[wary knot]

heres a sample

[upbeat pine]

but show grid.cell: set align(center + horizon) is the same as set grid(align: center + horizon) and both of them can only run once before typesetting.

the show grid.cell: cell(color) is pretty neat. but right now for it me only makes sense when you need it as a decoration and not as a content.

[wary knot]

well kind of

there is a subtle difference , though in this case it probably wouldnt change the output at all

but with those show rules, the alignment is applied before cell(color)

with grid it'd be applied after

but for alignment it probs doesnt matter

for other properties it could

[upbeat pine]

also, I forgot that currently I can't align in grid...yet.

Here I'm trying to show the set grid(align: center + horizon) which is applied at start and also the "i don't wanna insert 9 different elements/cells" function (over-optimization):

?r

// Don't look at this! Otherwise it (source code) wouldn't look as clean!
#let alph(offset) = str.from-unicode("a".to-unicode() + offset)
#let grid = table.with(align: center + horizon, stroke: none, inset: 0pt)
#set text(black)

#set block(below: 0.2em)
#let cell(color, body) = square(radius: 0.4em, size: 2em, stroke: 1pt, fill: color, body)
#let face(color) = {
  show: box // Never thought I can and would do this! Feels like preemptive `if-return` statement.
  grid(columns: 3, ..range(9).map(i => cell(color, alph(i))))
}

#face(white)

#face(green)
#face(red)
#face(blue)
#face(orange)

#face(yellow)

[grand haven]

Is it what the user expects though? It makes gutter useless with strokes. I'd expect the strokes to jump over the gutters

[wary knot]

It's how it works today at least

In tablex it's configurable (although a bit hidden)

So we could add some config on the line customization PR

[grand haven]

is that intentional though?

[wary knot]

Well idk

But I'm not gonna change it in the next PR at least :p

At least with my line splitting algorithm it should become trivial to add a setting for it

[bold bluff]

I think it should be typically hidden as well. It hurts my eyes with the optical illusion squares.

[grand haven]

I didn't expect you to, just wanted to get the conversation started

[wary knot]

Yeah dw

Just wanted to point that out

But I'm afraid the work I'm doing for colspans will directly impact that

Cuz you'd need to split vlines to make sure they don't span gutters, so , since I made an algorithm to split vlines (to not overlap with colspans), it can be trivially reused

Split vlines = ok so, i want a vline from row A to row B, but please make sure to only draw it where it makes sense

And "where it makes sense" can vary

In the algorithm it's just a function "should_draw_vline_at_row"

lol

So it can just be like
if this is a gutter row {
don't draw
}

Right now it's just like

if there is a colspan here {
don't draw
}

So yea

Perhaps the gutters look like this not necessarily intentionally, but because the algorithm didn't exist before. Not sure

[grand haven]

I'm just wondering if it would make more sense if the strokes were properties of the cells instead

[wary knot]

It could, and this discussion will probably become the main topic of this thread after I'm done with part 3 (merging cells)

In general though, I can see the benefits of specifying lines in a more general manner as well

E.g. i want a line under row 3

you can do that with per-cell stroke of course, it just could perhaps be made a bit easier like tablex does

There's also a bit of a concern here regarding inset

Should large strokes increase cell inset?

In Laurenz's preliminary work towards strokes, it appears he was working on something of the sort (increase inset in containers based on strokes)

While it makes sense to me, I'm not sure if:

1. That should be the default;
2. That should be customizable

By Not sure I mean I don't have much of an idea (not that I'm against anything)

So there's some room for discussion here

[grand haven]

You mentioned that table performance has reduced, but how does the new native tables compare to the old tablex?

[wary knot]

Well, if this helps, i have some test documents with like 2 tables with 200 cells each

And tablex can't even fathom those

with native tables they take about 1.2s

At least so far

Performance will probably decrease as more PRs get merged, but what can I do , more code is being run :p

Just the latest PR had a larger impact on performance cuz we started building the cell grid manually with a for loop instead of just mapping a vector once

[wary knot]

Fellow Rustaceans,

Should I

for (offset, slot) in resolved_cells[resolved_index..][..colspan]
                .iter_mut()
                .enumerate()
                .skip(1)
            {

or

for (offset, slot) in resolved_cells
                [resolved_index + 1..resolved_index + colspan]
                .iter_mut()
                .enumerate()
            {

?

i find the latter kinda ugly cuz thats how rustfmt wants it 😦

so i found the first syntax online and found it interesting

I'd prefer something like (1..colspan).offset(resolved_index) but that doesnt exist apparently

[vernal herald]

Slicing twice is crazy.

[wary knot]

lol

[vernal herald]

I'd do that.

[wary knot]

i mean it makes sense if you think about it, but a .offset method would make much more sense 😂

i guess i could also just roll out my own extension trait on ranges

but meh

[vernal herald]

All the pointers have offset method. You're just not trying hard enough /s

[wary knot]

fair enough

[vernal herald]

That's also an option.

[wary knot]

we goin' back to C

[upbeat pine]

(1st looks more sane)

[wary knot]

Colspan PR is up 🎉

https://github.com/typst/typst/pull/3239

[upbeat pine]

Is it me or we gonna have new tables in the next version? (or was this the goal?)

[wary knot]

yeah lol

at least the goal is to have as much as possible in the next version

[upbeat pine]

Is table 1.0 has something that table 2.0 doesn't ATM?

[wary knot]

no

i didnt remove any features at least

[upbeat pine]

then it's perfect

[wary knot]

well

table 1.0 is faster

but thats about it

lol

like 2x faster

[upbeat pine]

kinda marginally, right?

oh

nvm then

[wary knot]

:p

but i mean there's like

a whole lot more code now

so it's kinda... yeah

[vernal herald]

IT IS A GREAT DAY FOR TABLES

[upbeat pine]

nothin a good optimization can't fix.

[wary knot]

the true optimization is to make an HTTP request to a typst endpoint with supercomputer CPUs and GPUs to render for us

or to install a TPU unit*

*Product sold separately.

[wary knot]

hell yea 🚀

[wary knot]

Hmmm I should have added some tests for RTL

I knew I was forgetting something lol

Will add those later

I hope that doesn't cause chaos 😅

[wary knot]

welp

whoopsie

ok fixed the panic but it's not looking very good 😂

ok, a bit better now guys 🚀

[bold bluff]

https://tenor.com/view/confused-confused-face-confused-look-im-confused-im-so-confused-gif-16067865999736184254

[bold bluff]

Is that what it’s supposed to look like?

[wary knot]

what's your guess? 😂

[wary knot]

use this as a reference

:p

very important

we're close!!!

(just fill is wrong now i think)

(actually strokes will probably be wrong too)

winner winner chicken dinner

let's check strokes...

yeah adjustments will be needed 😂

but we'll get there

[wary knot]

WE DID IT

*at the cost of ugly code 😭

but it's THERE

it EXISTS

[bold bluff]

How ugly. I mean you can still love it right?

[wary knot]

yeah true...

but i mean..

look how they massacred my boy...

cleaning this mess up will be a task for future me

present me could not care less!

[upbeat pine]

It looks like a small fraction of borders sticks out (barely visible, maybe PDF viewer) and some borders are thicker than others (maybe an illusion or viewer).

[wary knot]

i think it's a bit of an optical illusion

im zooming in and they seem to have the same thickness

maybe anti-aliasing is to blame

[random mica]

You could make a small puzzle that happens to not use pawns

[upbeat pine]

you can share the file and I can figure out quickly if it is or not.

[random mica]

On behalf of rtl languages, I'm sorry

[random mica]

Yeah definitely looks like it

[wary knot]

Not a bad idea, though in the end i settled for a simple table with one usage of each feature

With exam grades or smth

[wary knot]

The file is in the PR

one of the test images

I think you'll need to compile my branch and export to PDF (Or wait for me to wake up tomorrow and have me export to pdf) to be able to discard anti-aliasing

[wary knot]

It's fine it's fine

[upbeat pine]

yeah, maybe I'll wait. I thought it was already PDF.

[wary knot]

To be fair it's good I'm doing this later cuz it's easier to undo :p

But the panic was pretty dumb

Should have foreseen that

Either way it's working now, just need to clean it up

So we'll be fine!!

[wary knot]

oh ok, yea it's a PNG

If it were a pdf at max zoom I'd be concerned

[upbeat pine]

#1182377294682128445 message
Looks good in PDF: #discussions message

[wary knot]

well

looking to un-ugly my code, I decided to make an iterator adapter

for .rev_if()

little did I know that , in order to make it fully transparent, I have to reimplement like every single iterator method to call the stored iterator's method instead

lol

it would be very nice if i could just

fn rev_if<I>(iter: impl Iterator<Item = I>, cond: bool) -> impl Iterator<Item = I> {
  if cond {
    iter.rev()
  } else {
    iter
  }
}

but the types are different

it's very tempting to just do this...

[frozen oyster]

I think you could create your own Iterator struct that is generic like:

enum RevIfIterator<I, Item> where I: Iterator<Item = Item> {
    Base(I),
    Reversed(Rev<I>)
}

And then impleting iterator on it:

impl<I, Item> Iterator for RevIfIterator<I, Item> where I: DoubleEndedIterator<Item = Item> {
  type Item = Item;

  fn next(&mut self) -> Option<Item> {
      match self {
          Self::Base(i) => i.next(),
          Self::Reversed(i) => i.next(),
      }
   }
}

And then you can just create a trait like:

trait RevOpt: DoubleEndedIterator {
  fn rev_if(self, condition: bool) -> RevIfIterator<Self, Self::Item>;
}

impl<I, Item> RevOpt for I where I: DoubleEndedIterator<Item = Item> {
  fn rev_if(self, condition: bool) -> RevIfIterator<Self, Self::Item> {
    if condition {
      RevIfIterator::Reversed(self)
    } else {
      RevIfIterator::Base(self)
    }
  }
}

This is close to (untested) a proper implementation

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=6eacda2a66df02a7099b6f5764d0c2c3

There you go, a link to the playground with a correct implementation @wary knot

Hope it helps ❤️

It should be basically zero cost too since the CPU will likely be able to predict the right branch after a few iterations at worse 😉

You can make it even shorter with less generics:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=e96bf329762169266cf9e1a3026f1bd8

[robust viper]

I think you do lose some of the optimizations though (for try_fold and such) because it falls back to the default impl.

[frozen oyster]

Probably, but overall it's a good way of having nice clean functional code 😄

Could also just manually implement them 🤷‍♂️

[wary knot]

yeah thats what i did

but like

i had to reimplement a ton of stuff

not cool

[wary knot]

and yeah specifically try_fold is a problem

cuz it requires an unstable trait

i need to add a bound for Try<...>

and that sucks

lol

[frozen oyster]

oof

💀

Could you specialize it for Result<T, E>?

[wary knot]

note : not reimplementing tons of stuff lead to panics

lol

it clearly wasnt respecting what Rev did

[frozen oyster]

💀 💀 💀

[wary knot]

so yea

here reimplementing = calling the Rev impl

so it's not that bad

just annoying

but still it's not complete cuz Rev can implement try_fold cuz it's in stdlib

😠

[keen crescent]

Watching the rubber ducky for this really makes me wish I'd have watched the rubber ducky for gradients

[frozen oyster]

The rubber ducky was being screamed at for all of PDF's woes, the rubber ducky was either laughing a lot or crying because I was mean to it 😭

[wary knot]

well it has no highlighting but

https://paste.opensuse.org/pastes/f5a616d2f54a

this is what i have currently

[wary knot]

man

i swear fixing things for RTL is draining my energy 😂

im gonna finish this tomorrow

for sure that yak has to be fully shaved by this point

(edit: link is fully safe)

[wary knot]

ok i had an insight

let's see if it will work

but basically i shouldnt do rev_if to try to "normalize" column order from RTL to LTR

rather

I should embrace RTL

and provide different code paths for it

[wary knot]

wow just found out about this

okay

i think i finally understood the problem

lol

when calculating the sizes of auto cols, we are always going left to right

in LTR that's ok since cells expand to the right, so when we reach the rightmost position of the colspan, we have already resolved earlier columns so we can determine how much space the cell needs

in RTL however that's bad cuz the position to expand is the leftmost auto column, which will be analyzed before the ones after it

in other words colspans will always assign their full widths to the last spanned column (leftmost in RTL, rightmost in LTR)

making tables look off

honestly im not sure but im thinking of just doing the same thing tablex did for RTL

which is to just consider RTL / LTR when actually rendering the cells, at the final stage

we do all calculations ignoring RTL and then later just flip the table

sounds fair enough to me in principle

cuz sprinkling stuff for RTL across table code is causing more problem than it's worth lol

[wary knot]

also you know what

screw it

😂

[wary knot]

nah i cant do that everywhere though

but it's ok

the workaround has been planted

😂

uhh ignore the struct

it was my first attempt to make it work 😂

and the huge associated type there is just to not increase MSRV to 1.75.0 (otherwise i'd return impl Iterator<Item = ...>)

[wary knot]

...also ignore the obvious logic error

😂

i was wondering why I made 100 tests fail

[wary knot]

ladies and gentlemen

I am proud to announce

the RTL is real

https://tenor.com/view/walter-white-walter-crying-walter-dying-gif-20964719

it has been a tough battle

but the beast has been tamed

[upbeat pine]

nice

[frozen oyster]

https://tenor.com/view/thanos-infinity-war-gamora-what-did-it-cost-gif-14001403

Congrats @wary knot

Looks like you’re making awesome progress

[wary knot]

Thanks 🎉

This particular bit costed a bit of my sanity but it's okay 😂

my experience in tablex saved me

sometimes the solution , after all, is to put everything down and start over 😂

[robust viper]

@wary knot is the PR still WIP or ready for review?

[wary knot]

should be ready for review

most i would do now is add more tests if i deem necessary

but i think its fine for now

hmmm

[wary knot]

ok hold on, i think i found a bug with RTL 😂

LTR vs RTL

something is off 😂

at least seems to be a problem exclusively with vlines (the cells appear to be in the correct positions)

[wary knot]

yea ok

i understood the problem

ok i think i have the fix

my previous solution was too naive

I was taking each (x, dx) vline position info pair and reversing the order of "x" such that you'd have (0, width), (1, width - final column), ..., (n, 0)

but that doesnt work well with the vline splitting algorithm cuz it always checks the cell at x + 1

it assumes LTR

so

this simple line seems to have done the job

ok well

i have no idea what changed here lol

kk

@robust viper ready 👍

took another look and stuff looks fine now

[frozen oyster]

So, can you just give me the TL;DR of where we're at with new tables?

[upbeat pine]

Isn't there an image diff tool? Would be nice to have.

[upbeat pine]

At the finish line, almost.

But there are a lot of features left to do in the future.

[wary knot]

i wouldnt say that lol

i mean we've gone very far but theres still a bit to go until we have parity with tablex

[upbeat pine]

But there are colspan and rowspan and that's it?

[wary knot]

no

theres parts 4 and 5 still

line customization and repeatable headers

well, and 3b (rowspans)

[upbeat pine]

Will they be in 0.11?

[wary knot]

probably

[upbeat pine]

Oh wow

Cool

[wary knot]

i mean depends on the release date, but if my predictions are correct then it's likely

[vernal herald]

That would be....

A great day for tables.

[wary knot]

indeed

[wary knot]

i think this is a good tl;dr: https://github.com/typst/typst/issues/3001

[frozen oyster]

So colspan is done but not rowspan, gotcha 😄

Congrats on the progress!!!!

I'm surprised how "short" the code ends up being

[wary knot]

lol

[wary knot]

tyty

yeah, what matters most is what the code is doing, not the amount of code :p

which is why tablex experience was very important

also (compared to the current PR) when i add rowspans ill probably have to change more stuff, colspans are relatively "simpler"

[frozen oyster]

Nice!

[wary knot]

So

regarding line customization, which would be part 4, just wanted to share some initial thoughts regarding what I was discussing with Enivex earlier

I agree that lines crossing gutter aren't always desired and stuff, and while tablex has an option to disable that (which we could take inspiration from and would be trivial to implement), we could take an alternative approach

we could consider:

1. having grid.line and table.line stuff (or hline / vline whatever) like tablex which would span multiple cells and thus cross gutters (that can still be made optional though)

2. in parallel, adding a stroke field to grid.cell / table.cell would add lines around cells which dont cross gutters, thus making the lines "individual"

though this also means that changing per-cell stroke wouldnt affect stroke for lines on gutters so that could be weird if we e.g. added some stroke: (x, y) => (left: ..., right: ... etc) thing

either way i think it might be better to create a distinction between both ("global" table lines and per-cell table lines)

not sure of the exact semantics here though

[robust viper]

yeah, that seems sensible

what is your opinion on allowing stroke: (outside: 1pt) in addition to things like left and right, which would only add strokes around the entire table?

[bold bluff]

It looks pretty similar im guessing that its aliasing but the characters on the right seem to have less weight. (im looking at you 7)

[wary knot]

lol, that's probably just my screenshot being bad i think, but good eye lol

[wary knot]

well, doesnt sound bad, though we'd have to see the possible interactions between this and headers and footers

and "global custom lines" would also play a role here as well

not sure if we should take inspiration from tablex so as to disable a default global line when you specify one on top at the same position

[bold bluff]

It looks like the middle collumn on the bottom has the bigges difference between the two sides. But i can tell by looking at it that its just aliasing.

[wary knot]

hmm nice

it could definitely have moved by like a very small amount with my changes in that commit

should likely be ok anyways

[wary knot]

actually

i think it should be fine (re: headers and footers) like

if u add a line at the top of the header then naturally it should override the normal top line probs, but if u dont then it doesnt

im actually not 100% sure about how headers (and footers) are gonna look like implementation-wise

the way im thinking rn is that they'd become so feature-packed that they'd be basically a subtable lol

somethihng like

#table(
  stroke: red,
  table.header(
    // this one is in the header, is repeated
    table.hline(stroke: blue),
    [a], [b], [c]
  ),
  // this one is outside the header, is not repeated
  table.hline(),
  [d], [e], [f]
)

that could also help with the dilemma of "header-hlines-have-priority" in tablex

though more generally we could probably implement some z-index mechanism for lines at some point, but let's not go there rn lol

[wary knot]

an interesting consequence of this design is that headers wouldnt have to be at the top of the table

headers would basically become "sticky rows"

what's more, ideally you'd be able to have more than one header

i've seen some examples out in the wild which use "subheaders", i.e. headers which are replaced along the table

[bold bluff]

I’d like to say table continued on next page. But idk how it would work.

[wary knot]

that'd be more related to figure captions

this has been suggested before, and I'm positive it will be implemented eventually

but it's unlikely it'd arrive on time for 0.11

[bold bluff]

Figures….

[vernal herald]

Finally I can say it: Figures are Off-topic!!!

[frozen oyster]

You're on an #off-topic in #1108789303124967514 #1175885362065834105s rampage today 😄

[bold bluff]

Do these tables work with cetz. I thought I heard mention of this.

[wary knot]

no

Tablex will work with CeTZ in the future

Not native tables

[wary knot]

@robust viper is self.regions.in_last() supposed to be true for this grid's layouter? (And/or can the value of in_last() change during the layout process?)

was thinking of that bug from earlier

and i cant seem to be able to detect whether or not we're in an unbreakable block

hmmm i see

in_last() seems to be changing between before and inside a call to layout_auto_row

well that's definitely gonna be one pesky bug to fix 😂

i'll prob focus on my stuff, just wanted to use some spare time to take a look at this

[wary knot]

Part 3a merged 🎉🚀

https://github.com/typst/typst/pull/3239

🦀We have colspans🦀

Lol

Rowspans next...

Those might take a bit cuz I'm still not sure of the best algorithm, I'll be experimenting with it thoufh

[vernal herald]

It's a great day for tables!

[frozen oyster]

I saw, really awesome stuff 😄

I can't wait to optimize the heck out of it 😈

[wary knot]

Hehe, I'll be waiting for it

[frozen oyster]

#1182377294682128445 and #1176509648707256370 crossover 😎

[wary knot]

It'll be legendary

[frozen oyster]

You know what would be funny, if the eval rework made tablex faster than native tables 😂

[wary knot]

Lol nah

That's impossible

I'm sorrt

[frozen oyster]

probably indeed 😄

[wary knot]

You'd have to make eval faster than Rust + LLVM

that's a pretty tough challenge

[robust viper]

compile Typst to wasm and then JIT it with wasmtime

if we ever want to go down the JIT route, I think that'd be the way

wasm can also be nicely bound to a runtime that does stuff, too

but it's still hugely complex

[frozen oyster]

the JIT route would require some pretty chonky library to go around the generated bytecode

[frozen oyster]

I think that will get better over time

[wary knot]

so

theres an important design decision to make here regarding rowspans

consider this table

very badly drawn

assume that all rows have a fixed height

so not auto

if the page would break at that red line (assume the page has 0pt margins)

what should happen?

1. the same as a normal table: the first row would stay in the first page, the following rows would go to the next page. However, this would imply breaking the rowspan cell across both pages.

2. all rows spanned by the rowspan move to the next page.

basically the question is, should a rowspan force its spanned rows to be inseparable / unbreakable?

(except, perhaps , for the last auto row or w/e)

[bold bluff]

It depends on what you’re trying to achieve…. Maybe both ways are right for different situations.

Rowspan(breakable: false)?

I’d imagine that they would be default brakable and possibly even repeat content on break.

[wary knot]

yeah maybe...

it's just that both things would lead to considerably different implementations

or at least in theory

maybe in practice it's not that different

well depends

if rowspans were 100% unbreakable then they'd be like very easy to implement

but there has to be some breakability, e.g. with auto rows as is now

so that wouldnt be very sane

[wary knot]

yeah i think that might make more sense by default? though not sure about the technical viability of this

in particular, i wonder how this would interact with stuff like #block(breakable: false)[...]

theres already an open issue about grids not respecting that for cells

either way making them breakable seems more logical perhaps? but i can see how it can be unexpected

like. to forcefully break the cell when it doesnt need to be broken

especially now that the internal codebase has a clearer distinction of what can and can't be laid out across multiple pages

[upbeat pine]

Wait, rowspans can break?

[random mica]

Okay hear me out.

equation.cell

[wary knot]

I mean this is what we are discussing

But in general any cell can break when in an auto row

But we aren't dealing with auto rows here

[random mica]

Or would this be better as math.rowspan etc.?

[upbeat pine]

if so, then you know my answer

[random mica]

another thing that solves is alignment of a maths cell

[bold bluff]

Well it’s probably better to implement the simple solution (unbreakable) while provisioning for the complex one. ?

[signal breach]

Being breakable looks like the safest default, as the opposite would make some tables unable to be layed out in a page.
Think of a table of html elements with the first column telling if it's a block or inline element, the next one with the name and the last with a description... and both categories have around 30 elements each.

[random mica]

that would make the rowspan elements more like headers; you would want them to repeat on each page

[upbeat pine]

Nooo! You should've used Typst for this! /s

[bold bluff]

forgive me:

[wary knot]

Thanks for summarizing

I think repeatable rowspans would basically be "header columns" so to speak

In principle I think I'll not focus on that in the moment

We'll tackle this in part 5 or later

For the next update it's most important to reach basic feature parity with tablex

[bold bluff]

100% agree.

[wary knot]

I'm still not fully sure on this matter, so I think I'll request some opinions from @robust viper

(Discussion starts at the message I'm replying to)

Mostly opinions in the technical viability side of things (whether or not those approaches seem sensible to have in the compiler in a broad manner)

I've proposed some algorithms for each approach before, but I'm still thinking regarding what makes the most sense to implement

If we make rowspans 100% unbreakable (tablex approach), then rowspans are just cells with larger height than usual, so it's very easy to implement them

But if we allow them to break, then we have to consider where they can break
We could make them unbreakable except at the last spanned auto row, which still wouldn't be very hard to implement, but would cause all previous auto rows to be unbreakable

Alternatively, we could allow them to break every time the table would normally break, and then the current algorithms will need larger changes

So I think the initial image I sent nicely summarizes the matter :p

I think for the user it could be ideal to have both options as discussed, but it could also be better to implement just one of them for now as an MVP

If things turn out to take too long that is

Right now I think I'll try to take a stab at breakable rowspans

Not in code but conceptually

Cuz still gotta figure out the best algorithm for that

[bold bluff]

Wouldn’t they just break wherever the table would break without the rowspan?

[wary knot]

That's the practical effect, yes

But internally things work very differently :p

I'll give more details later

Was gonna write stuff down but I realized maybe I should take a break 😂

So tomorrow I'll explain what I mean

[robust viper]

I think the current table behaviour of allowing breaks on anything auto and forbidding them on anything with fixed size is fairly reasonable and would ideally also be respected in this exact same way by rowspanned cells. However, I can also imagine that a breakable property on cells would be quite useful to override this.

[wary knot]

Yea fair enough

I think there are some technical problems regarding making rowspans breakable

Cuz like, we'd have to say that it has some space available out of a total of (page height) in the first page, and then in the second page it could or could not span the entire page height depending on the spanned rows, and then in the last page it would have a fixed amount of space left

So it seems like we'd have to somehow tell the cell that the "last page" has a different height than the rest. Or something

Whereas, with auto rows, we can just tell the cell that it can grow as much as it wants, since the last auto row will accommodate that

So it makes more sense to allow rowspans through auto rows to be breakable at that point, but not rowspans going exclusively through fixed-size rows

At least from a technical PoV

I think we're bound to have some similar problems with headers too since we'd have to reduce the amount of vertical space available for multi-page cells... (So that they don't overlap with the header)

[random mica]

is a rowspan with auto size but whose size is constrained by other cells on its rows 'fixed-size'?

[wary knot]

So

I think a fixed size rowspan here would be one spanning exclusively fixed size rows

While an auto sized rowspan would be one spanning at least one auto row

Cuz then it will have unlimited space available

I think we can agree with this for now then:

1. Rowspans spanning one or more auto rows are breakable.
2. Rowspans spanning only fixed size or fractional rows are unbreakable.

I think that's a good starting point at least

We could make fixed size rowspans breakable, but that could require changes to the layout engine in order to actually look good...

[wary knot]

well actually

maybe I can use this?

@robust viper would this be abuse? 😂

(I'll make a drawing to show what my idea is)

here "b" would be page height inside the margins

so

if the rowspan spans rows such that their combined height is a + b + c, then maybe, when laying out the rowspan cell, we can just say that the followup regions will have sizes b, c and the first one has available size "a" ?

if we want to add an option to make fixed-size rowspans breakable, that is

[robust viper]

sounds like you want to do exactly what the backlog is intended for. it's already used in grid code so that bottom alignment works properly in multi-page auto rows.

[wary knot]

perfect then

thanks!

i think im good to go now

for now i wont implement the breakable option (i'll focus on the defaults being fixed-size = unbreakable, spans auto rows = breakable)

but im working on making the code generic enough such that the option would be trivial to implement

well, if not trivial then at least easy enough to not require yet another large PR 😂

[wary knot]

👀

baby steps...

this will need a looot of testing still 😂

cant say im fully proud of my solution but...

if it works...

[wary knot]

yay we have lines too

so. in theory all the functionality (other than the breakable option) is implemented

in practice: need to test a lot , clean up etc.

but this means PR will be up within a few days 🙂

[wary knot]

well

just wanted to share a bit of the recent progress

ive been tackling basically two problems:

1. when a rowspan spans all columns, depending on your row configuration, the line above it is not drawn and its fill is also not drawn.
   turns out this is because auto rows are just deleted when they're completely empty (as an optimization), so the first row spanned by the rowspan is deleted , therefore the code fails to detect when the rowspan starts in order to draw its fill etc.

ive managed to fix this just now by having an "effective first row" (keep going through the list of rows in the current page until you find one that is spanned by the rowspan, that will be its new "first row").

still cleaning up my solution though.

2. im still perfecting the code related to measuring the last auto row spanned by a rowspan.
   Right now it uses a bit of a heuristic: the last auto row will layout the cell into multiple pages, calculate the total height of rows spanned in previous pages, and keep deleting pages from the laid out cell until that total height was satisfied. it does a similar thing but in the opposite direction for rows in future pages (which weren't yet laid out so it can only try to guess what their sizes will be).

however, there are several improvements i can make to this algorithm , by using the global list of rowspans i keep (it includes data such as total height spanned in each page etc., so i can just use that data)

so that's what ill work on next

3. (i guess this also counts) i guess that having an unbreakable rowspan and a breakable rowspan have the same last auto row could be a bit weird, I dont handle that yet. though rn thats also impossible to happen cuz rn spanning auto row = breakable (not configurable). but ig it would be nice to guard against that either way

so. those are the three main blockers right now, after I tackle those (should hopefully be done by tomorrow or thursday) I'll do more testing and see how things are doing, if everything is OK i'll open the PR, otherwise figure out what's missing and test more 🙂

well, if that counts, i also found a case where a gutter row can be the first row in a page. no idea if this is already possible with tables, if not it would be a bug too

[wary knot]

items 1 and 2 have been addressed 🫡

item 1 was fairly quick as I managed to figure out the correct algorithm fairly quickly

item 2 , however, took hours of debugging

cuz some examples with rowspans over auto rows weren't working correctly

I even spoke to a friend who has nothing to do with Typst and they suggested there could be a TOCTOU issue in the code

:p

and that hint led me to realize there was some code by Laurenz which I didn't realize was there, which explained a bit of why the examples were going rogue

so, long story short, I switched my strategy around and stuff and all examples were fixed 🚀

i'll tackle the other 2 items tomorrow

[wary knot]

regarding this (gutter rows in wrong spots), I think I might just add a simple check for if (gonna draw a gutter row at the top or bottom) dont(); honestly

lol

aint nobody got time for that

😂

anyway, today was very productive 🚀

this has been the most challenging PR for me so far, no doubt haha

I've already used up a large stock of brain cells here 😂

[frozen oyster]

Congrats 😄

[night python]

great progress! tables are the one large blocker for using Typst for my dissertation for me, so i'm excitedly following your work here!

[frozen oyster]

isn't tablex suitable for your needs? 🤔

[night python]

it probably would be, but i very much prefer a builtin solution

[upbeat pine]

yeah, the lack of powerful built-in tables is probably the major con of Typst. For using tablex you have to make extra steps, and beginners don't like extra steps.

[robust viper]

that all sounds great!

[robust viper]

did whatever I wrote make sense at least? ^^

[wary knot]

Ty ty 👍

[wary knot]

I wouldn't say it's the "major con" given the existence of tablex, but it's definitely something we can improve, and a lot

So that's what we are doing now 👍

[wary knot]

Yeah, it was the thing to expand frames on multi-region auto rows

It seemed to interact in an unexpected way with my previous approach of , after measuring, subtracting height from the left , since expansion would make frames larger and so my calculations would have been slightly imprecise

I switched to simply removing the first N frames after measuring , where N is the amount of previous regions spanned by the rowspan
And it worked :p

[wary knot]

Alright so

I'll keep going on the weekend, so we might see some more updates by then

Hopefully I can get at least a good enough version up for review by Monday

This is taking a bit longer than expected, but I also took a break in the past 2 days, cuz you know, can't stay at full speed forever haha

But worry not, we shall keep going

[vernal herald]

It's gonna be a great weekend for tables everyone!

[wary knot]

alright folks, im gonna need some opinions / suggestions on "the gutter problem" (cc @robust viper )

Basically I'm trying to satisfy the following requirements:

• If a rowspan spans one or more auto rows, only the height of the last (bottommost) auto row is affected.
• The auto row should increase just enough (to its best ability) such that it doesn't expand more than the rowspan will need in terms of height, but, at the same time, the rowspan should get all the height it needs. For example, if a rowspan cell's body has a height of 4.5em, and it spans a row of height 4em, a row of height 1em and an auto row , the auto row wont expand at all , because all of the 4.5em the rowspan needs were already covered by the first two rows spanned (it doesn't need more height). In contrast, if the cell's body had a height of 6em, the auto row would have a minimum height of 1em - ignoring cell inset - to compensate (so the rowspan cell has all the space it needs).
• However, the principle above should also apply forwards. In particular, if a rowspan spans an auto row , followed by a 4em row and an 1em row, the auto row should also not expand with a cell body height of 4.5em, cuz later rows will cover the rowspan's height; in contrast, the auto row would have a min height of 1em with a cell body height of 6em.

so this basically means that an auto row has to consider the height before and after that row to determine how much height it needs to give to the rowspan

Now

the problem is that, at the moment, this calculation includes spanned gutter rows
(btw: gutter rows = gutter space between each row, internally they are just empty rows with the gutter size as height.)

Gutter rows before the last spanned auto row are fine to check the height of cuz they were already laid out at this point, so in particular it's easy to tell the height of a rowspan before the last spanned auto row

But gutter rows after the last spanned auto row are trickier
Well, in fact, anything that isn't a fixed-size non-gutter row is tricky (spanned fractional rows are ignored cuz their sizes are only known later)

For a simple reason: gutter rows might or might not be present

if a gutter row would be at the bottom or at the top of a page, it should just be deleted

gutter rows should only appear strictly between two consecutive rows, visually

but if we delete a gutter row after the last spanned auto row, this means that the rowspan will now have less height available than we predicted (when calculating how much to expand the last spanned auto row, we considered that the gutter row would contribute to its height and thus we exclude that row's height from how much the auto row expands)

this means that it can't expand "as much as it needs" in that case

so we will have to make a choice

1. Ignore spanned gutter rows after the last spanned auto row when calculating how much that row needs to expand to ensure the rowspan has all height it needs. This can, however, look particularly bad when tables use gutter (the larger the gutter, the more apparent it is), since we will expand the auto row much more than it actually needs to be expanded. But here we're prioritizing giving enough space to the rowspan over "looking good".

   • Here, we could perhaps use a fit-spans option - to have users opt into disabling auto rows (and/or columns) expanding due to rowspans(/colspans) - to have users conditionally opt out of this behavior and manually size their rows appropriately.

2. Ensure gutter rows which participate in rowspans after their last spanned auto row aren't deleted. However, this means gutter rows can now appear at the top or bottom of pages. For rowspans which span all columns, that's fine since the gutter itself isnt visible (the rowspan covers the whole thing), but otherwise you can see that theres gutter under the hood.

3. Mix the two approaches - if the gutter row is fully composed of rowspans, it isn't deleted. Otherwise, it is deleted if needed and last spanned auto rows ignore those gutter rows when calculating how much to expand.

4. Ignore - however rowspans might have less height than expected (though they'll be aware of that when being laid out later, so it's not that bad - note that, for this problem, we are just measuring the rowspan's approximate height anyway).

At first I was leaning towards 1, but now im considering 3 might be better

however 3 is definitely a bit harder to implement :p

though i could play with it and see

[wary knot]

just to illustrate the problem, here's a case without gutter vs with gutter:

three rows, first two are auto and the last one is fixed-size

to the right you have a fixed height block spanning all three

the second auto row didnt expand enough to fit the block, cuz the second gutter row was deleted (it would be at the green arrow's position)

the "ZD" is used to mark the height limit given to the cell by the table

[bold bluff]

What’s an auto row?

[wary knot]

auto-sized row

expands based on the height of its content

however, for rowspans, only the last auto row is expanded

[wary knot]

the problem is that it doesnt expand enough here

cuz it assumed there was some gutter below which covered for a bit of height

but the gutter ceased to exist cuz it would otherwise appear at the top of the page whihc is ugly and weird

[bold bluff]

So the case we are discussing is purely that of rowspans + brakes + gutters.

[wary knot]

yeah basically

in a non-gutter world it doesnt matter

or if you're in an unbreakable context ig

well, overall the crux of the problem is that, at the last spanned auto row, we have to try to predict how much space is "covered" after that row

and we can't predict with 100% accuracy without either time travel or two-pass

so it's more a question of whether we're fine with having this inaccuracy causing the rowspan have less space than it needs, or if we should always favor it having more space

I think the latter makes more sense

and like

if we add the fit-spans option to make it possible to have rowspans not affect auto rows' sizes at all, then it'd be mostly ok cuz then the user could be like "yeah i didnt like that" and opt out of the algorithm we choose

(then they'd be opting into the rowspan having (potentially) less height than it needs, rather than (potentially) more)

[bold bluff]

Why can we know the necessary height in the non-gutter case but not the gutter one?

[wary knot]

we actually cant with 100% precision in either case

basically the possible causes of imprecision are:

1. fractional-sized rows (e.g. 1fr height), cuz each fractional row's height is always based on the available height after rows in its page were laid out, so their heights aren't known beforehand
2. gutter rows, cuz even though they are fixed-size rows (which is cool and predictable), they might be deleted later if they happen to appear at the top or bottom of a page (which isn't cool or predictable)

you could argue that auto rows could induce imprecision as well, but that's precisely why rowspans only expand the last spanned auto row (cuz then all auto rows were already laid out, so we have their precise heights)

so, in the non-gutter case, only fractional-sized rows can cause some imprecision

well, i guess either way there will be some amount of imprecision since we're using measure

so whatever

lol

but fractional-sized rows and gutter rows can cause significant/visible amounts of imprecision, thats mostly what i mean

[bold bluff]

It’s a pretty involved question. I don’t think you should have gutters on breaks. But it kinda have a hard time understanding the options. Is 4 the easiest?

[wary knot]

yea 4 is the "do nothing" one

:p

1 is basically ignore spanned gutter rows' sizes when expanding auto rows (much like we ignore fractional rows currently, out of necessity)
2 is basically don't delete relevant gutter rows (which can look odd tho)
3 is basically 1 but with some extra heuristic to not delete some relevant gutter rows
4 is basically keep considering gutter rows' sizes and deleting gutter rows, it's the user's problem

it's definitely not an easy question

that's why I brought it to the public

I myself had trouble answering it

haha

of those options, 4 is the easiest to implement (cuz it's already implemented ig lol), 1 is fairly easy too
2 and 3 are more involved

but that shouldnt matter, the most important thing is what we would expect

correctness, therefore

anyway, let's see though, perhaps Laurenz will have an opinion, and he's mostly aware of the relevant code

rn im leaning towards 1 or 3, they seem the most sensible

either way, this question is for one of the two main blockers left for the PR so im just speeding stuff up by asking here 😂

the other being the breakable / unbreakable rowspan conflict thing

i still havent decided how i'd deal with like, breakable and unbreakable rowspan sharing an auto row
like should such an auto row become breakable and allow the breakable rowspan to run its course, which would however violate the unbreakable rowspan's condition (and potentially make it look ugly and broken), or should it be unbreakable anyway and potentially make the breakable rowspan look very ugly and broken

tbh i think that one is more straightforward, imo we should just make an arbitrary decision cuz that situation is just not supposed to happen

lol

also btw

i have fixed the problem with gutter sometimes appearing on the top or bottom of pages even though it shouldnt

which is cool

one step closer

👍

[bold bluff]

I think it’s for sure easiest to do unbreakable rowspans . A later feature can add breaking . If I had to pick a predictable one. It’s better to do the simple. Because there are no behavioral requirements for the tables. It can be kinda arbitrary guessing behavior. Also, it looks like you’re limiting your behaviors set to ones that allow single pass or efficient processing.

[wary knot]

yea this would be just in case of conflict

ive already made most of the code revolve around breakable rowspans so it's fine :p

and unbreakable rowspans are just a special case , so to speak

basically it's kind of two pass

first all non-rowspan cells are laid out, and then rowspan cells come right after

that way all row sizes will be known