#Table utility

1 messages · Page 1 of 1 (latest)

shut scarab
#

Would anyone be interested in me making a package out of the following table utility I hashed together?

It takes data in an array of records format, then column definitions, including a function used to determine the contents of each cell, which could be just passing through the value of a record property, or whatever function or formatting you can express in Typst.

You can also pass in any arguments you'd pass into the standard table function.

It will also generate column widths for each column, either auto if not defined, or whatever you define in the column definition.

I just threw this together last night, so there's a few things to improve or cleanup if I made it a public package.

#let tablez(data, colDefs, ..tableArgs) = {
  let colWidths = ()
  for colDef in colDefs {
    if colDef.keys().contains("width") {
      colWidths.push(colDef.width);
    } else {
      colWidths.push(auto);
    }
  }

  let colAlignments = ()
  for colDef in colDefs {
    if colDef.keys().contains("align") {
      colAlignments.push(colDef.align);
    } else {
      colAlignments.push(auto);
    }
  }

  let entries = ()
  for colDef in colDefs {
    entries.push(colDef.label);
  }

  let i = 0;

  for record in data {
    record.insert("index", i);
    i = i + 1;
    for colDef in colDefs {
      entries.push([#(colDef.cellValueFunc)(record)])
    }
  }

  set align(center);

  table(
    columns: colWidths,
    align: colAlignments,
    ..tableArgs,
    ..entries
  )
}

#let tableData = (
   (name_first: "Alice", name_last: "Wright", position: "Manager", office_code: 0),
   (name_first: "Bob", name_last: "Smith", position: "Sales Representative", office_code: 1),
   (name_first: "Charlie", name_last: "Brown", position: "System Administrator", office_code: 0),
   (name_first: "Diana", name_last: "Ross", position: "Recruiter", office_code: 2),
   (name_first: "Evan", name_last: "Johnson", position: "Manager", office_code: 1)
)

#let office_locations = (
  "Headquarters",
  "Regional Office",
  "International Branch"
)

#tablez(
  tableData,
  (
    (label: [], cellValueFunc: r => { r.index }), // index is automatically generated for each row
    (label: [*Full Name*], width: 1fr, cellValueFunc: r => { [#r.name_first #r.name_last] }), // concatenation
    (label: [*Office*], width: 1fr, cellValueFunc: r => { office_locations.at(r.office_code) }), // reference external data/enum
  ),
  fill: (_, row) => if calc.odd(row) { luma(30) } else { luma(45) }, // Table arguments
  stroke: none  // Table arguments
)

#tablez(
  tableData.filter(r => r.position == "Manager").sorted(key: r => r.name_last), // filter and sort data, all native array functions
  (
    (label: [*Managers, by Last Name*], width: 1fr, align: left, cellValueFunc: r => { [#r.name_first #r.name_last] }), // alignment
    (label: [*Office*], cellValueFunc: r => { office_locations.at(r.office_code) }), 
  ),
  fill: (_, row) => if calc.odd(row) { luma(30) } else { luma(45) }, 
  stroke: none
)
rendered.png
zenith tapir
shut scarab
# zenith tapir is this a similar idea to https://github.com/ntjess/typst-tada ?

Yes, though I actually made this after my frustrations with Tada. I don't like how expressions are defined in python, and I think this is more consistent and streamlined generally. I think you should also do most table operations like sorting and filtering data using the default array functions, which you can technically do in Tada, but it's complicating to have multiple methods of doing it.

zenith tapir
#

i see

#

sounds interesting

shut scarab
#

I mean I don't want to make this about all the things I dislike about Tada, but like:

#let make-dict(field, expression) = {
  let out = (:)
  out.insert(
    field,
    (expression: expression, type: "currency"),
  )
  out
}

#let td = update-fields(
  td, ..make-dict("total", "price * quantity" )
)

#let tax-expr(total: none, ..rest) = { total * 0.2 }
#let taxed = update-fields(
  td, ..make-dict("tax", tax-expr),
)

#to-tablex(
  subset(taxed, fields: ("name", "total", "tax"))
)

vs

#tablez(
tableData,
(
(label: [Name], cellValueFunc: r => { r.name }),
(label: [Price], align: left, cellValueFunc: r => { [$#r.price] }),
(label: [Tax], align: left, cellValueFunc: r => { [$#{r.price * 0.2}] }),
)
)

zenith tapir
#

yeah fair enough

#

though im not sure i'd name it tablez

#

the table-letter namespace is a bit crowded

#

😂

shut scarab
#

Yeahhh, that's the most difficult task, figuring out a name 😭

zenith tapir
#

dont worry

#

so you're not alone in your pain

shut scarab
#

Yet Another Typist Table Utility
Yattu

zenith tapir
#

not bad just please write Typst instead of Typist thanks

shut scarab
#

lol, aaa I blame autocorrect

zenith tapir
#

fair lol

#

but dont worry i had the same problem recently for my next (WIP) package

shut scarab
#

What are you working on?

zenith tapir
#

i spent a good chunk of time brainstorming with a friend about it

#

😂

zenith tapir
shut scarab
#

Cool! I'll give it a look!

zenith tapir
#

TL;DR though, im working on some sort of standard interface for making custom bibliography styles and "bibliography-like" things

#

the idea is to have different levels of flexibility, each one depending on the previous, so you just choose how much freedom you want

#

ive been liking the idea so far but it's so hard to decide what i want the API (function parameters etc.) to actually look like 😂

#

basically the naming problem but in a smaller scale

shut scarab
#

Very useful, I mean technically api's are a whole bunch of naming problems.
And ugh, I still need to come up with something better than "cellValueFunc"

zenith tapir
#

yea lol

#

i suggest using kebab-case, it's the standard for most typst stuff

shut scarab
#

Cool. I could just have it be "contents" or "value". I could also make it so you can pass in just the string of whatever property you wanna pass through directly, instead of having to do a whole function.

zenith tapir
#

i think the function approach is definitely the most flexible, but you can certainly allow for both

#

one thing i dont like about functions is that you cant really specify the arguments you want to get

#

perhaps something like (functype: () => ...) would be ideal

#

but idk

#

in your case i doubt thats at all needed

#

in mine im considering doing something like that though 😂

shut scarab
#

What do you mean by "specify the arguments" exactly?

zenith tapir
#

like

#

sometimes you accept a function that takes 3 integers as arguments

#

other times you need one that takes a single string

#

and you cant check for that by just doing type(x) == function

#

sure you know you received a function. but you dont know anything about its arguments

shut scarab
#

Sounds like you just need to put more functions in your function 🤷‍♀️
Though at that point you might just want to use rust or whatever.

zenith tapir
#

😂 yeah at some point i just want to go back to strictly typed langs

#

enums would be a godsend...

shut scarab
#

I am currently automatically adding an "index property" to each record, before it's being processed, which could cause a collision, but the alternative would be like forcing an index to be an input to each function, and whatever metadata I want to add going forward so, yeah.

zenith tapir
#

you get it now 😂

shut scarab
#

I mean that's the issue, typst has a good enough scripting language that you immediately start thinking about the features you miss from actual programming languages.
What you need is to be able to directly import functions from rust files 😅

zenith tapir
#

technically speaking you can

#

through wasm plugins

#

but

#

it's not as seamless as having a function literally available

#

:p

#

and either way you cant work with typst markup in wasm plugins so yea

zenith tapir
#

😂

shut scarab
#

Yeah, but you have to go through all the compilation and whatnot, it looks non trivial.

And like if you could streamline the experience, that would be powerful.

Typst is written in Rust, so theoretically you could compile and load up rust modules fairly directly.

zenith tapir
#

yea but u're gonna have to mod the compiler for that :p

#

but would be awesome if everything was just that easy yeah

shut scarab
#

I don't think so, you can compile rust src in rust, and then load up the resulting module functions. You'd have to cache your compiled binaries somewhere, or just recompile them each time, but I don't think that requires doing anything to the compiler?

zenith tapir
#

i meant more in the sense that typst will never support that (since it also has to run on the web and other platforms)

#

but it's still good food for thought

#

:p

#

well, i cant say never cuz i dont make decisions on typst but it's unlikely to say the least

shut scarab
#

Ohhh, the Typst compiler... hmmm, most likely not... but it would be neat, gotta go to Berlin(?) and complain them directly

zenith tapir
#

yeah 😂

#

(or just to Discord, the typst team is here 😄 )

shut scarab
#

Less dramatic, and there are fun things to do in Berlin

zenith tapir
#

fair enough

#

anyway, gotta go now, but that was a fun conversation for sure :p

#

cya 👋

shut scarab
#

See you around!

random jasper
#

btw consider using kebab-case rather than camelCase

dreamy pike
#

I'd love to increase usability in general so if you created some issues about where improvements are most needed, I'm happy to help 🙂

shut scarab
dreamy pike
#

Oh, if you wanted to work directly with the arrays, you can just do

td.data.my-field.map(x => x + 1).sum()

#

The data field holds all the named arrays

dreamy pike
shut scarab
#

I think the fundamental premise I'm operating off of is that there should be one idiomatic way to modify data that will eventually be turned into tables.
You could implement all the native array manipulation functions into tada chain functions, but how does that provide a better experience than just using the native array manipulation functions?
Though tbf, I'm assuming you're data is already a record array, if not It could be useful to make a table object from a different format and manipulate it using table specific functions. But you could also get the same effect, by converting to a record array format. And I also think that a record array is the most understandable way to store and manipulate this sorta data, but that's a different discussion.
I suppose you might also have like edge cases with insane amounts of columns, like thousands, per record could cause issues if they're loaded anytime you load a record, compared to just paying attention to a small amount of columns that you need for a given purpose. But in that hypothetical case... You should most likely not be doing that... And you could role up your own solution.

#

Sorry I'm a bit tired, I had a long day, but I'll definitely look into that tomorrow and give you some examples/clean up some of my thoughts above

dreamy pike
#

how does that provide a better experience than just using the native array manipulation functions?

  • It is essential to provide tada functions for sorting, aggregation, and filtering since otherwise it requires lots of manual work from the end user to e.g. sort all values based on a single field (see examples/titanic.pdf sorting passenger names by the Fare cost)
  • Native array functions are also limited to array => array transformations, whereas chain lets you do things like array => multiple arrays, array => scalar => array, and other options

I'm assuming you're data is already a record array
It depends on what you mean by record array... Currently data is stored as

(
  field-name: (val1, val2, val3),
  field-name2: (val1, val2, val3),
  ...
)

My experience with record array is a list-of-dicts rather than the current format which is dict-of-lists

But you could also get the same effect, by converting to a record array format.
It becomes slightly harder to do things like transposing your table or aggregating if your data is in record array format

I thought a lot about how to store the underlying data (actually changed from list-of-dicts to dict-of-lists), so I can explain more of the reasoning if it helps. But hopefully these explanations make a little sense in the meantime!

I again appreciate you writing out your thoughts and look forward to additional feedback after you have a chance to rest 🙂