#Types

1793 messages · Page 2 of 2 (latest)

ancient wagon
#

well it is generic in a way, typescript allows passing string literals to type parameters for example :)

grim crescent
#

it contains a proposal for show within the type

elder sequoia
#

i see that, i just dont like the special case

grim crescent
#

in the compiler, show is a Rust trait, so it'd probably work similarly

elder sequoia
#

plus, how do you differentiate the show rule of the type with show rules above

#

as in

grim crescent
elder sequoia
#

well i mean

#

that sounds like a type key -> content function value to me!!!

grim crescent
#

how? its not a key, it's an instance

elder sequoia
#

show(type) = fn(instance: type) => content

#

youre specifying a function, associated with a type

grim crescent
#

i just meant to say that counter(heading) doesnt return a new type

#

like it isnt a Counter<T>

elder sequoia
#

catnod yeah thats the key difference i suppose

#

but do you see what im getting at here at least?

ancient wagon
grim crescent
#

i mean, i see that you are proposing generics

elder sequoia
grim crescent
elder sequoia
#

this i believe

#

although no i guess that is still different

#

counter(heading) resolves to a value

#

but if types were first class, theres no reason it couldnt be a type instead, right?

grim crescent
#

to be honest im a bit lost :p, im not sure if there is something additional being proposed

elder sequoia
#

yeah sorry im not explaining Too well

#

essentially

#

counter(heading) implies a type accepting another type as an argument, right?

#

to me this implies that heading and by extension any other type is just another value. it is first-class

#

so theres no reason that a function could not return a type, right?

ancient wagon
#

well it does

#

return a type

#

it's just not generic

elder sequoia
#

?

#

it returns a value

ancient wagon
#

it's dynamically typed

#

yes

#

sorry that's what i meant

elder sequoia
#

theres a difference between returning a value with a type, and returning a type itself

ancient wagon
#

what would the returned type be used for

elder sequoia
#

well for example

#

generics

#

:3

ancient wagon
#

what for though, we have a dynamic language wuithout a distinction between compile and runtime

clever bloom
#

I think you mean kinda like returning a struct type from a function in Zig, right?

elder sequoia
#

something like that, not too familiar with zig

elder sequoia
ancient wagon
#

this snippet alone doesn't really convey any meaning to me

#

how would it be used

elder sequoia
# ancient wagon how would it be used

with some notion of defining specialized types based on argument, this is a replacement for traits

#type showable.where(t: heading) = it => [
  strong(it)
]
#

theres absolutely some confusion on my part in terms of what 'defining a type' entails

#

i suppose itd be more likie

ancient wagon
#

I'm unsure what this would bring over the simple type x { show: it => { ... } } version

#

I still dont' know how to exactly read showable, it feels like a trait but is assigned a function?

elder sequoia
#

its a more generic pattern than show and would be usable for anything using this kind of dispatching-over-types

elder sequoia
clever bloom
#

how would the compiler know about it?

#

like, a trait impl is a global thing

#

but Typst doesn't have the concept of a global thing with an effect on unrelated parts

elder sequoia
#

haha, sorry im taking a bit, just trying to make sure im being coherent here

elder sequoia
#

if it were possible to do something like

#let is_three(n) -> bool = false
#let is_three(3) = true

implementing show for a given type could be done as

#let show(type) -> fn(instance: type) -> content
#let show(heading) = it => {

}
ancient wagon
#

I assume thre former should be interpreted as the haskell equivalent of

is_three :: Int -> Bool
is_three 3 = True
is_three _ = False
#

though it's a weird example

elder sequoia
#

YES i was trying to remember where i had seen that pattern

#

but no really it shouldve been more

elder sequoia
ancient wagon
#

ok i see

#

the issueist hat in haskell you write all these cases at the same declaration site

#

here you'd do specializations all over the place

#

and I'm not sure if this is a good or bad thing

elder sequoia
#

in some sense, it is similar to how set rules can partial-applicate based on selectors

#

actually i dont know if thats true

ancient wagon
#

well, that's the application not the declaration

elder sequoia
#

yeah

#

show rules are already just scoped specializations

#

of the show 'function'

#

like the example i was asking for help with earlier

image.png
#

this is equivalent to

# 
#let show(heading.where(depth: 2)) = it => {

}
ancient wagon
#

the problem i see with this is that it is incoherent from a declaration perspective

#

show is different ebcause it dynamically binds something for the remaining scope, let is not a dynamic binding, it does not propagate though the same way, i.e. a type and it's show rule are not dynamically bound and imo shouldn't be the same way show is

#

I don't see any upside of defining it like that over just having it be part of the typedef

elder sequoia
#

parameterization based on type

ancient wagon
#

but so would type definitions, beacuse that's what elements are

elder sequoia
#

i mean things other than show

#

like

ancient wagon
#

the point of types is to give users the ability to do these things

elder sequoia
#

you could write a plot function

#

like follows

ancient wagon
#

what speaks against it being an element

#

fundamentally that's what it is

#

it gets input data and produces content

elder sequoia
#

an element is just a type which happens to be showable

ancient wagon
#

yes

#

which a plot would be

elder sequoia
#

well

#

what im saying is something like

elder sequoia
# ancient wagon what speaks against it being an element

nothing speaks against it 'being an Element', but what im discussing is how one declares a regular data type over an element. that being, an element has a defined show function.

  • the blog post proposes a special syntax for declaring the show contract on a type. i am against having a specific syntax for this, as its essentially an implementation of interfaces but restricted only to the Element interface.
  • instead, i suggest allowing the definition of show(type), and by overriding this for a specific type, you are implementing the show interface. i.e show(heading) = ... is just implementing show on heading
#

it is basically traits! the show function is a trait, and show(heading) = ... is impl show for heading

ancient wagon
#

the blog post goes into detail how types and elements will be one and the same thing

elder sequoia
#

catnod sounds like i need to do more reading

ancient wagon
#

show just being how a type is displayed as content

elder sequoia
ancient wagon
#

that's why it's #type not #element

elder sequoia
#

absolutely

#

but for example

#

#type point(x: int, y: int)

#

this should be possible, yes?

#

a type without a show function

ancient wagon
#

it would probably get a default show function

elder sequoia
#

sure

ancient wagon
#

similar to those of array and dictionary

elder sequoia
#

mhm

#

but if this were not displayed as content, but just used as a container for data, it would not really be an element, right?

#

perhaps later down the line it would be consumed by an element which does produce content

ancient wagon
#

beacuse elements would be no more

#

just types

elder sequoia
#

yes exactly

#

so the question is: what should the syntax for defining a show function on a specific type be, right?

ancient wagon
#

well i think it should just be like in the blog post

#

it's unclear how additional constraints could be set, but this is unsolved in general

elder sequoia
#

what do you mean by additional constraints? as in like interfaces right

#

because my problem with the proposed syntax in the blog post, is that it is a special case of a pattern which would otherwise be very widely usable

#

'is this function defined on this type'

ancient wagon
elder sequoia
#

yeah

#

but there are things other than show which a type might want to implement, and consumers of the type would like to have a contract for, yknow?

#

like say

ancient wagon
#

Yes I get that

#

That's indeed unclear

#

Although I can't think of any of the top of my head

elder sequoia
ancient wagon
#

mhm

elder sequoia
#

or for example

ancient wagon
#

I'm unsure whether this is really necessary over jsut defining them as member function which happen to have the same signature

elder sequoia
#

sure, that would be a form of implicit interfacing

#

but in this case, distance isnt an arbitrary name, its a pre-declared function, which can be imported for example

ancient wagon
#

yes but the declaration sites of the function don't actually differe from normal functions

#

a simple mismatch would just define a free standing random function

elder sequoia
#

yeah, that is definitely an issue with it

#

overloading syntax would have to be distinct from declaration syntax

#

and at that point its literally Just traits i suppose

#

like i guess its literally just

let element(t) -> fn() -> content

// equivalent
trait element {
  let show() -> content
}
#

and if that was on the table id totally advocate for that instead, but i assume its not

ancient wagon
#

I would prefer that, but I don't know how important this is for typst, perhaps it's integral to it working well

elder sequoia
#

mm yeah

#

i dont see a world where we can get away without non-element types

#

another question: are primitive types like ints considered showable? does that make them elements?

ancient wagon
#

no currently types are sort of special cased

#

with the unification of types and elements they would get show rules and such

#

allowing thins like configuring the precision of floats and base of ints when showing them

elder sequoia
#

so really it's just, an element is any piece of data which can be mapped to drawable content

#

a.k.a any type with a show function

#

okay ive figured out my exact concern

#

if show truly is meant to just be a member of the type representing the default show rule, then it should be fully treated as such imo

ancient wagon
#

it isn't just a simple member

#

it is affected by the style rules in scope

#

or rather

#

at the showsite

elder sequoia
#

is that not the case for other members?

#

say we have

# 
type some_type(x: bool)
#

what would occur here?

# 
#set some_type(x: false)
#x
ancient wagon
#

those are fields

#

#x would fail since it's not defined

elder sequoia
#

oops

#

then

# 
#set some_type(x: false)
#let instance = some_type()
#instance.x
ancient wagon
#

the difference is that fields can be set and get, show has intricate interacitons with other rules and the "previous" rule

#

unlike a regular function like distance

ancient wagon
#

consider:

#

?r ```
#set heading(level: 3)
#heading[].level

humble roostBOT
ancient wagon
#

it's not known here yet

#

because it's now shown here

elder sequoia
#

and what about

#

hmm. is there a meaningful semantic difference between

#show text: emph

and

#set text(show: emph)
ancient wagon
#

to some degree

#

most but not all set rules are absolute in a way

#

they dont' interact with previous rules

#

show rules do (and so do foldable fields when set)

clever bloom
#

also a selector can be more than just a function

ancient wagon
#

i.e. show is not like set text(fill: red) but more like set block(stroke: red) and set block(stroke: 5pt)

clever bloom
#

and not all types which shall be shown have fields

ancient wagon
#

where both apply to some degree, similar to how using it will use the previous rule in show

elder sequoia
ancient wagon
#

well that would do away with the selector mechanism

clever bloom
elder sequoia
#

i mean, in this hypothetical scenario where all elements are types, 'selectors' are really just pattern matching, right?

clever bloom
#

yes

ancient wagon
#

yeah

elder sequoia
#

theres no reason a set rule couldnt set a field on all instances matching a pattern, is there?

ancient wagon
#

in theory nothing speaks against it other than it being maybe confusing

elder sequoia
#

less confusing than 2 rules which operate similarly but are used for different things, imo

ancient wagon
#

we should not forget that typst is used by non-programmers too

elder sequoia
#

itd be a unified styling syntax, only set, no show

elder sequoia
#

whereas a unified system where show is just another value which can be set feels easier to understand, no?

clever bloom
#

I don't see how adding a specific field on every element is unifying things. It's still the same thing, just more verbose.

#

Is that field documented for every element?

elder sequoia
#

hm?

#

i mean

clever bloom
#

Or is it magic?

elder sequoia
#

as it stands, a 'default show rule' is just another field

clever bloom
#

It is not a field

elder sequoia
#

hmm

clever bloom
#

It is a property of the type, not of the instance

elder sequoia
#

right

#

whereas settable fields are properties of the instance?

clever bloom
#

yes

#

the trait analogy is closer in my opinion

#

but full traits are overkill imo

elder sequoia
#

yeah, ofc id love to see them available, but i get that for sure

#

but wait, show rules do take an instance dont they?

ancient wagon
#

yes they do

elder sequoia
#

thank u two again for ur patience btw

#

i still dont quite understand why overriding a field is not the same as overriding a show rule

#

becausee each instance can have its own show rule depending on scope, right

clever bloom
#

a field exists on every instance of an element

#

a show rule exists in a context of styles

#

it is never part of the element

elder sequoia
#

hm, but what about optional fields that arent set per-instance

clever bloom
#

you cannot get it.show-rule. it doesn't make sense

elder sequoia
#

are those not also removed from the individual context

clever bloom
elder sequoia
#

can you do the following

type point(x, y)
set point(x: 0)

let zero = point(y: 0)
clever bloom
#

zero will not have x set during scripting

elder sequoia
#

ah yes!!!

#

as established above

clever bloom
#

only once it's shown, when the style context is known

elder sequoia
clever bloom
#

there is it.x, but only within a show rule for x

#

then the field is materialized

ancient wagon
#

within a context there is

elder sequoia
#

right

#

so both x and show() materialize in the context

ancient wagon
#

sort of with the distinction that show is not a field on an instance

#

or a method on one

elder sequoia
#

but why is that a necessary distinction

ancient wagon
#

same way a non-static method in C++ is not a property of a an isntance but of the class itself it just get s special first parameter that is the instance

elder sequoia
ancient wagon
#

the fact that it can be dynamically rebound

elder sequoia
#

but so can any field....

ancient wagon
#

but it's not a field

clever bloom
#

another thing: even if there was some way to have more flexible selectors with set, a set rule must always operate on a well-known type of element, such that it knows which field exists.

meanwhile show does not have this restriction. you can write show <my-label>: it => .. and it works for any kind of element.

ancient wagon
#

you can't pass a show rule to an instance when creating it

grim crescent
clever bloom
#

PgSuper joined the chat

ancient wagon
#

yes yes

grim crescent
clever bloom
ancient wagon
#

can be thought of as inheritance overriding dynamically withaccess to super()

grim crescent
elder sequoia
#

hahaha

#

evil

grim crescent
#

i dont wanna think that

ancient wagon
#

hahaha

elder sequoia
#

okay so

#

some_func(it) vs show(it)

#

show is special in what respect? it is ill-defined until it is called?

grim crescent
#

😂

#

always nice to take a quick peek back to the realm of normalness

elder sequoia
#

btw i love the term showtime. its great

grim crescent
elder sequoia
#

yeah, im asking why show couldnt simply be a setable function if types were to be a thing

grim crescent
#

i think there is some confusion from how tinger explained it, it isnt something that you "change", it is just rules that are accumulated

#

like you'll have some array of transformation functions in the style chain and they are applied in order until there are none more

#

until you place the element in the document tree, such that there is a style chain in the first place, there are no show rules to be applied in the first place

elder sequoia
#

okay, and set rules are different how? arent they also transformation functions, with hte transformation just being the setting of a specific field?

grim crescent
#

i mean, if you abstract everything far away enough sure, everything is a transformation function, but thats not very helpful for comprehension

elder sequoia
#

right

grim crescent
#

ill also note that set rules dont affect only the element they're setting

#

indeed, you can write #context { if text.lang == "es" [Hola] else [Hello] }

#

text.lang is determined by set rules on text

#

but it isnt affecting a text element here , it is affecting your code inside a context element

elder sequoia
#

mhm

#

so wait

grim crescent
#

so show rules are different in the sense that they only affect the elements they target

elder sequoia
#

youre telling me its modifying a field that is bound to a type, rather than an instance

grim crescent
#

its not modifying anything really, it's just appending properties to a style chain which are later queried

#

so if you write set text(red) and then set text(blue), when you write text.fill it will query those set rules and return the last value (blue)

elder sequoia
#

okay yeah

#

so

#

why cant set be used to accumulate the show function of an element?

#

hence my suggestion earliere

# 
set text(show: emph)
grim crescent
#

why? like i dont see the point

#

the fundamental mechanism here is the style chain, and it accumulates stuff, sure

elder sequoia
#

because then

grim crescent
#

but doesnt mean all the stuff that accumulates has to be one thing

elder sequoia
#

show really is just a member function!!!

#

or

#

wlel not a member function

#

but it operates the same as a field in this way

grim crescent
#

i mean

#

to me this just sounds like you invented a new syntax for show rules

elder sequoia
#

yes!!

#

thats exactly what im doing

grim crescent
#

okay

#

though this syntax is worse cuz it's more restrictive than before

elder sequoia
#

it is?

grim crescent
#

yeah

#

how do you do this? #show selector(heading).or(<some-label>): emph

#

or this #show regex("abc"): emph

#

they arent elements

#

so they dont have fields

elder sequoia
#

ah

#

but

#

i thought all showable things were going to become elements

grim crescent
#

no

#

show rules operate on a selector , the selector can match more than one thing

#

doesnt mean all the possible selectors are going to become distinct elements

elder sequoia
#

right, so show rules currently can target things which are not elements

#

via selectors

grim crescent
#

it will always apply to certain elements - but you dont need to list the elements explicitly, you just need to specify how to match them

elder sequoia
#

yes

grim crescent
#

like #show regex("abc") will operate on text elements , but not any of them

elder sequoia
#

why cant the same functionality apply to set rules?

grim crescent
#

cuz they have a different purpose, they only operate on element fields

#

it doesnt make sense to set level: 5 on an image element

elder sequoia
#

right

#

so you have a selector constraint that there is a level field, or it is a type which has one

grim crescent
#

like #set heading(level: 5) only makes sense on a heading

elder sequoia
#

mhm

grim crescent
#

but you can filter, if thats what you are asking

elder sequoia
#

heading(level: 5) is just a selector, is it not?

grim crescent
#

thats what show-set is for

grim crescent
#

a selector specifies what to search

elder sequoia
#

yep

grim crescent
#

here we're not searching for level 5 headings

elder sequoia
#

sorry

#

i meant

#

heading is the selector here

grim crescent
#

it's the element that owns the field

elder sequoia
#

yep

#

except its saying

#

at least to my understanding

#

set level, on all elements which are heading

grim crescent
#

it's distinct from a selector, it just says which fields you can set
and by default it applies to all of those elements - but you can restrict it as follows

#show heading.where(outlined: false): set heading(level: 5)

elder sequoia
#

yes, so the current syntax is restricted in that you cannot set with a selector - why is that?

grim crescent
#

you can, it's just a different syntax

elder sequoia
#

yeah

ancient wagon
#

i mean they have a point

elder sequoia
#

thats what i mean, is there a reason?

#

why show-set

grim crescent
#

i mean it's just the syntax that was chosen to specify it

elder sequoia
#

because surely #set heading.where is more comprehensible

elder sequoia
ancient wagon
elder sequoia
#

given i think it has merits over the original

grim crescent
clever bloom
elder sequoia
#

yeah

#

but that, again, is just another selector constraint

#

'field exists'

elder sequoia
clever bloom
#

if you change 100% how everything works, then yes, you can change how it works. I'm just trying to argue why I think it does not make sense.

elder sequoia
#

catnod i appreciate that, i know im contradicting how things work currently from a place of less knowledge than you

clever bloom
#

it is good that set rules enforce that the field exists

#

well it is also kinda necessary because it runs type conversions and respects positionalness of parameters etc.

#

but if you're changing everything, might as well change that too :p

elder sequoia
#

and those properties are necessarily lost with a more powerful set syntax?

ancient wagon
#

it's the semantics that come along

clever bloom
#

they are lost if set does not know which element it operates on

#

it is very fundamental to set rules

elder sequoia
#

meaning that the current set syntax encompasses the entire domain of 'selecting a field on a known type' right?

grim crescent
#

thats why it's not a selector - it's specifying which element the field belongs to

elder sequoia
#

right makes sense, because we're doing nominal rather than structural comparison

#

but also in my head that wouldnt be a valid selector, im not insinuating that () should match a field on any instance supplied

#

can show rules do the second one with selectors?

grim crescent
#

you can write #show <abc>: set text(lang: "es") if thats what youre asking

elder sequoia
#

but you couldnt do <abc>(lang: "es")

grim crescent
#

yeah

#

thats why the show-set syntax is necessary

elder sequoia
#

so why would you be able to do it in the set rule?

grim crescent
#

cuz there are two parts

elder sequoia
#

right

grim crescent
#

the selector

elder sequoia
#

i see

grim crescent
#

and the element

#

with <abc>(lang: "es") i have no idea which property you're setting

#

you could think of each property as being a pair (type, field name)

#

without the first part of the pair its not a valid property

elder sequoia
#

to clarify where my head is with this

#

the syntax i feel makes the most sense without regard to technical constraint is

# 
#set text(lang: "es") for <abc>
#

something like that

grim crescent
#

i mean

#

that isnt very different, but i think you're trying to imply something else

elder sequoia
#

yeah, that show rules would be deprecated in general in favour of something like

# 
#set [element selector] field assignment
#

since those are two different kinds of things

grim crescent
#

i mean

#

im sorry but that seems more confusing to me lol

#

the two things are entirely different but have combined syntax

#

syntax is just a design choice, what matters is the implementation under the hood

#

and the implementation would remain the same

#

with this syntax it makes it seem like show rules are a field with a final value that you can change but they are not, they are accumulated

#

when i write #set text(blue) i expect that all previous set rules on fill have no effect

elder sequoia
#

mmmmm

#

override vs application

#

despite my questionss im certainly understanding the decision a lot more

grim crescent
#

although some things in set rules do have a joining behavior, for example you can change just one property in a stroke field and keep other properties the same

#

but that's a bit different, it's really just setting distinct parts of things

#

as if they were separate fields

elder sequoia
#

yeah, more of a compound field situation

elder sequoia
#

woagh... thanks to ur explanations i am coming to Understandings...

#

setting a field is fundamentally different from show rules because show is a shorthand for nesting functions

grim crescent
#

yeah, it's also how templates work

# 
#[
  #show: fff
  Hi
]

is (ignoring the spacing) the same as

#fff[Hi]
elder sequoia
#

yeahhh

grim crescent
#

so in particular, writing #show: templatefunc in the document is the same as replacing your document with #templatefunc(document), so the template can apply set rules and stuff on top of it

elder sequoia
#

with that in mind

#

all i'd really want to make clear the mental model behind things is perhaps an alternative set syntax to reflect the idea of changing a property of quote itself, as opposed to finding quote and setting a default

#set quote.quotes = true
#

or heading.offset = 1 and such

hybrid cairn
#

I can see the reasoning, but I think that gets tedious very quickly. It's very common that you set multiple fields. As an analogy, it's also common to do this:

#show table: set text(...)
#show table: set block(...)

or something like that, i.e. set multiple elements for the same show selector. It's already inconvenient to have to type the show part twice. The assignment syntax you're suggesting would present a similar inconvenience for the set (also applying show/set of course).

elder sequoia
#

right, i see

#

i think my brain just really wants either full seperation or full unification of instance/global data

#

though is there any reason a set statement couldnt support comma seperated values

#

#set text(...), block(...)

#

i also realised, the mental model makes a lot more sense to me if i replace show with select

#select <abc>: set text(...), block(...)
#

my confusion mainly stemmed from show having the role of both Selecting an element and then Applying a function to it

#

so imagining smth like this really helped

#select quote: apply it => {
  set text(color: red)
  apply emph  

  it
}
hybrid cairn
#

honestly I wasn't here for the whole conversation, but from what I skimmed I noticed that you switched a lot between syntax and semantics. for example:

i think my brain just really wants either full seperation or full unification of instance/global data

am I right in reading this as you wanting to have semantics that can be understood via one single underlying concept instead of two? If that's possible it would be huge, but it's also really hard to pull off.

#set text(...), block(...)

that's probably fairly trivial. the reason we don't have that is probably because we already have set ...; set ... (usually on separate lines), so if that could be reused it would save users from learning an extra thing.

elder sequoia
hybrid cairn
# elder sequoia so imagining smth like this really helped ``` #select quote: apply it => { set...

I think Laurenz said separating show: and show ...: is on the table, since they're pretty different (not selecting elements and such). So this would be bikesheddable once the time is coming. Personally I'm not a fan of apply like you use it in the first line, since it's only an additional separator. But I'm generally not "adventurous" with new keywords and new syntactic variations, so that doesn't say much.

clever bloom
#

but I honestly can't remember what my final verdict on this syntax was

elder sequoia
#

HAHAHA

#

okay nice im in good company then

clever bloom
#

I agree that select X: set Y is a bit clearer than show X: set Y. And I also like apply better than bare show:. what I'm not full sure about is whether I prefer select X: apply Y over show X: ... it is a bit more verbose, though I can see how it seems like a somewhat clearer composition of existing concepts

elder sequoia
#

yeahh

#

so really whats happening is elements are defining scoped configuration data, which are being selected for

clever bloom
#

one thing I might do differently nowadays is the last thing, which could just be

#select heading: apply {
  set text(size: 12pt)
  apply smallcaps
}
#

composing rules (set or apply) via some form joining essentially

ancient wagon
#

I see we're back at styles as values

clever bloom
#

which doesn't hide them behind a function, making it easier to deal with them in some kind of revoking thingy

#

I see we're back at styles as values
indeed

#

I can still somewhat roughly remember coming back from a walk and writing down this syntax thinking it's the best thing ever. What I can't remember is what stopped me in the end. Can't really have been backwards compatibility cause it was before even the private beta.

#

But it was only before it by a month or so, so probably I just had other stuff to do (like writing docs!)

gloomy perch
clever bloom
#

haha

#

one problem with the syntax was that selectors were their own thing, not really values

#

like heading(level: 1) was kinda just vibing there without being an expression

#

the .where business was later apparently

#

but it's unrelated to select/apply vs show

elder sequoia
#

makes sense

clever bloom
#

okay I found a never, but still old (pre-private-beta) note with the current syntax, so apparently that happened after the previous note

#

I have no clue what my thoughts in between were though

elder sequoia
#

huh, interesting

clever bloom
#

that's the note

image.png
#

almost today's syntax, except for the lack of a colon in the bare show case

#

that was added later

#

I guess the [x] means what was implemented then

elder sequoia
#

looks like a checklist yeah lol

clever bloom
#

I have a ton of files like this and there all just called styling.typ or layout.typ or layout-ng.typ or ng3.typ....

elder sequoia
#

so real....

gloomy perch
#

typst-syntax-final-v1(1).typ

elder sequoia
#

but yeah, so is selectors being actual values something thats desirable? id imagine so

clever bloom
#

uhhh, yeah, certainly Laurenz ....

image.png
clever bloom
#

cause selectors are also used in query for instance

elder sequoia
#

yeah

clever bloom
#

anybody liking the wrap rule. or the * operator? instead of bare show:

image.png
#

ah, this is nostalgic ^^

elder sequoia
#

hehehehe

#

oh actually

clever bloom
#

ah looks like this one was also on the table once

image.png
elder sequoia
#

was just trying to write smth up, and came across a relevant thing: how come #set page(columns: 1) is adding a pagebreak as opposed to #set page()

elder sequoia
clever bloom
#

cause #set page() doesn't produce any styles and the pagebreak is forced by the precence of styles

#

a set rules produces an array of style properties

#

and if there are no none, it doesn't do anything much

elder sequoia
#

huh, thats a little counterintuitive

#

is it like, not implied to be wrapped in a #page element if #page has no associated styles?

clever bloom
#

if you call #page[..] that's a different thing

#

but in a set rule, it's just not how it works. I can see either way making sense, but that's just how it is.

elder sequoia
#

yeah but, is adding the columns field implicitly adding a call to page? guess im just not sure why itd do that

clever bloom
#

there is no call to page

#

the layout engine is just aware of page styles and inserts pagebreaks as appropriate

elder sequoia
#

fair enough yeah

clever bloom
#

(it's getting kinda late here, so I'll tune out now)

elder sequoia
#

all good! tysm for the discussion, super interesting stuff

#

and also really helping me understand how to use the shit better on my end!!

clever bloom
#

haha, I like that phrasing

elder sequoia
#

:D

elder sequoia
#

thoughts about a potential apply syntax: a seperate select statement is not needed, since selectors and 'style scopes' are semantically equivalent

#apply strong, emph: [
  Hello!
]

#apply strong, emph: <label>
#

also, as established before, set is semantically just another transformation over the style scope. therefore it could be used with apply too

#apply text(lang: "es"), strong, emph: <label>
// or perhaps
#apply set(text(lang = "es")), strong, emph: <label>
// or even
#apply set text(lang = "es"), strong, emph: <label>
# 
#apply(
  set text(lang = "es"), strong, emph
): selector | label
#

inverted the selector and applied function compared to #show as it reads more left to right, apply those rules to the selector, but thats an arbitrary thing that could go either way

#

doing it this way round has the benefit of being able to immediately supply a scope

#

or, perhaps, something like this is even better

#rule (optional selector): {
  apply: strong, emph
  set: text(lang: "es")
}
latent swallow
#

I just came here after reading the types and context Blogpost. This thread is very long, is there a summary anywhere/or rather, what would be the point where I should jump in and start reading?

I see there are types, traits, apply, select, etc.

After reading the Blogpost, I wanted to mention the show rule as described there is similar to pattern matching in Rust etc., right? So you could leverage features like bindings, guards and short-circuiting to have a very expressive syntax on which transformations are applied and on what nodes in the document tree destructing deep into the tree. But I figure this has already come up and apply is doing precisely that so nvm I guess ...

latent swallow
ancient wagon
ancient wagon
latent swallow
latent swallow
#

I cannot come up with an actually practical example but I came up with a general example to proof the point.

Because if one Custom Element A expects its field b to be of a type (say, a block), it expects to be able to access the fields of a block on b.
However, if (for whatever reason) the expected block was transformed into, let's say, a grid by a show rule, then the fields don't match.

So when A accesses a field of b which blocks have but grid's don't, there's an error which was known to have been preventable because the show rule was a transformation into an incompatible type.

For sure there must be cases where a show rule wants to extract information out of an element like a block in a semantical manner.

If not, you can just denote a field as the inbound type. A type system is useless here and custom Elements might as well just be dictionaries which show rules do pattern matching on to check applicability.

But if you need to access some field, you have to specify the type of that field. But in this case, you also need to be able to rely and that type being present and the expected fields to be accessible.

ancient wagon
#

I guess this makes sense from the perspective of a show rule which intrusively looks into the content of the fields, but until now at least they do not really

#

but i see your point

latent swallow
#

I just wrote another pretty long comment, I hope it gets clear what I mean 😅
I think it's important to keep in mind that the Type System and Custom Elements don't need to be same but it's a decision

#

IMHO (and uninformed opinion of course) introducing a type system and introducing Custom Elements are different things if one doesn't even enforce type safety on the show rule transformations.

Because on one hand, types would have helped me for my ordinary functions and their arguments, which often were just dictionaries and arrays instead of Elements.

The custom elements approach could also be constructed without the notion of types by introducing one CustomElement(kind, args, sefault-show-function) as one type.
The show rules would still work by pattern matching (or alternative ways) and instead of a type conglomerating different instances of one Custom Element, the kind argument serves that purpose now. If I'm not making mistakes, this serves the same purpose without types.

That's why I think fusing Types and Custom Elements is only worth it if the types are actually also enforced for the show rules

I think this is a similar but way more educated comment: https://github.com/typst/typst/issues/317#issuecomment-2227343975

GitHub
Type annotations · Issue #317 · typst/typst

As in the title. I think inevitably when a language grows large enough the need for typing will be present. (similar to JavaScript → TypeScript, Python → type hint etc.) Although at the moment it i...

ancient wagon
#

I think enforcing types in show rules will just make it less flexible

latent swallow
latent swallow
ancient wagon
#

that is what they do, they transform some abstract element into simpler content

#

To address point 2 we (i.e. a package author) don't necessarily need to be able to declare a new "type" in a strong sense, but just a new "concept" like address which under the hood refers to say a dict of a certain structure. Most of the time, what is passed to public functions are dicts (with mostly strings, integers, and content as values).

I think this comment kind of misses the point of custom elements, most people care about custom elements because they are fundamentally what can have set rules and show rules, not just some structured data to pass around to functions, although i can see people using it for that too

#

the other points brought up make sense to me

latent swallow
#

I agree that any kind of element should be meant pretty much only for embedding it in the document tree and potentially having rules acting on it

ancient wagon
#

the end idea is to not make a differenciation between the two though, I don't see much of a point either

latent swallow
#

All passing data around through functions should be done only with dicts. For what it's worth, I would love a function to validate the schema of my function arguments but I realize now that this is besides the point. Still, I would be hesitant to call the new Custom Elements as Types because I think they serve different purposes and shouldn't be seen as the same thing

ancient wagon
#

the main concern of an "incorrect" transformation brought up by a show rule could only do the following transformations which have no impact on types of the fields itself, these are already done today and are not problematic

#

All passing data around through functions should be done only with dicts
I don't thingk so there's merit in having types with well defined methods on them especially for scripting heavy packages

latent swallow
#

Are the show rules evaluated leaves-first, root-last or vice versa? Sorry that I don't know

ancient wagon
#

leaves first if i understand that right

latent swallow
ancient wagon
#

?r ```
#show quote: it => [first] + it
#show quote: it => [second] + it

#quote[aa]

humble roostBOT
latent swallow
#

This plugin is crazy wow

#

I'm gonna try writing code which has the problem I'm talking about but probably only tomorrow because I'm on mobile right now

ancient wagon
#

to me fundamentally, elements just happen to be those types that have settable fields and use show rules to define how they're displayed

#

and any type in typst currently can be displayed, some just use a simple debug representation, but it's a show rule in my book no less

latent swallow
#

Are their fields typed for you?

ancient wagon
#

yes

latent swallow
ancient wagon
#

not yet

latent swallow
#

Got it

ancient wagon
#

an example of where this is a problem is locale aware number formatting as they're not elements, but simple primitive types

latent swallow
# ancient wagon to me fundamentally, elements just happen to be those types that have settable f...

I was thinking but this sounds good.
The only things which bugs me a little is that in my head, I have a pretty clear separation between "Data Types" and "Element Types". With Data Types, I mean those passed to functions just to pass information around. For example "MyPackageInitInfo".
Data Types don't need default show rules of course.

As I understand, you would use a single/shared type system for both of those, right?

ancient wagon
#

yes

#

and if you ignore the compount content type we already do

latent swallow
#

Because for instance array and figure live under the same type system, right

#

Sure any other implementation would probably just add complexity

ancient wagon
#

they would then yes, their default representation becoming their show rules

latent swallow
#

So every type would be representable. I understand.

#

Which would be pretty nice for debugging afterall

#

Or I guess one could just leave out the default show rule and if none is defined when trying to represent a type -> runtime error

#

You are completely right this is very elegant and minimal

ancient wagon
#

basically

#

could even be

#

?r ```
#show figure: it => panic("oops")

#figure[]

humble roostBOT
ancient wagon
#

to make types unrepresentable, although i find that a bad idea for debugging

latent swallow
#

Yes very nice 👍

#

Lots of options

# 
type Error {
field message: str 
show: panic(message)
}

type ReprType {
field: subject: ?
show: debugRepr(subject)
}
type DefaultShowRuleImplType {
field subject
show: Error("trying to represent $(subject) which is missing a show rule to use")

// While debugging:
show DefaultShowRuleImplType: it => ReprType(it.subject)

This turns the behavior on and off for debugging

latent swallow
#

I realized this assumes an order of operation where the fields are overridden by the show rules before the show rule for the type is evaluated. This is what I meant when I was asking about show rules being evaluated leave-to-root. I don't know if this is right. By now, I feel like I'm very fishy on the order of operations to be honest. Nevertheless, here goes nothing;

// Base type
type WeddingCard {
field date: datetime,
field image: Image
}

// Action which uses field transparently instead of opaquely
#show: #image + "You've been " + (datetime.today() - date).days() + "married!"
}

// Action at a distant
show datetime: box(stroke: red)

Now this is my example for the other discussion we've been having. I need to add a few things:

  1. This doesn't work today because datetime is not an element. But under you proposal, this ought to work
  2. I want to emphasize that WeddingCard should indeed be an Element because people could want to restyle it to fit their needs
    => This creates precedence for an Element which has semantic arguments that are also sensible to be modified by a show rule
  3. If I'm not mistaken, their occurs an error because the .date field is used as a datetime but before that was replaced with content by a show rule
  4. Datetime serves well as a semantic type which might also want to be placed and have a show rule applied to
#

Also, a question: does the default show rule get overridden or wrapped?
That is, if another is defined does the default one not get executed or does the external show rule get the result of default rule passed instead of the raw type?

latent swallow
#

I honestly don't think such a case is producible in today's version because types are either not show-rulable (string for ex.) or they are used in an opaque way (noone ever fetches for example the content of the caption of a figure in the figures show rule). However, if all types are elements, such types will exist and I think that's gonna lead to problems because the bugs are from random show rules changing types.

Either we keep track of which show rule changed the type or we mark data types as special or we reverse that show-rule-order to be root-to-leave (honestly, that's probably the solution if I come to think about it lol)

In my example above, then the datetime-show-rule would never be called because the datetime was destructured before it made it into the document tree

#

But do you see where I'm coming from? What I mean by root-to-leave and vice versa. Why the latter means (if I'm right) that the fields are replaced before their own show rule is run and the other way around works?

clever bloom
#

The show rule order is already root-to-leaves (using your terminology). Fields are never replaced.

reef sorrel
#

For the type/content rework, is it planned to allow joining non-content with content?
Currently, #{[text]; 1} returns the error: "cannot join content with integer."
Is the plan for this to change? If so, I have some design thoughts.

clever bloom
reef sorrel
#

Ok, my design thought was that it would be neat to define joining of separate types as generating a dictionary with the type names as keys, and merge any values with the same type. This could let us explain why 1pt + red is a valid stroke without needing a special case for it: it would evaluate to the dict (length: 1pt, color: red), and the stroke constructor would accept that as one of its input options.
So now 1 + [text] would become the dict (int: 1, content: [text]).
(And adding none is a no-op)
I'm not sure if this is the best design, but it feels really symmetric, and gives big functional programming vibes.
One issue I can forsee is adding floats and integers. Not sure on that one.

mossy elm
frosty sphinx
#

Has there been much prior thought / discussion about (de)serialisation of custom types? I imagine we might have access to fields() or similar on a type, but it would be useful to easily convert an instance to a dict and vice versa, especially if the serialisation can be deep.
I've been using cbor/ciborium to pass structured data through to wasm plugins, and I can imagine wanting to pass through some nested structured data.

clever bloom
#

There are still a few design blockers that need to be resolved before custom types can become real. Here is a non-exhaustive list of them:

#

Step 1: Unify value and content

  • Do we accept that things like none | auto | content degrade into none | auto | any, which is a slightly weird type or are not all types content after all
  • Handle some naming clashes in compiler codebase that will come with dropping the Elem suffix from types
  • Decide how new merged content + value is represented in memory
  • Design replacements for content methods
    • maybe label.of for .label() and location.of for .location()?
    • maybe type.fields for .fields() and in for .has?
  • Settle joining behaviour of array/dict vs sequence

Step 2: Custom types

  • Finalize syntax
    • Decide on field syntax for types
    • What to do with the type type and type(x) when type becomes a keyword? Or do custom types use a different keyword after all?
  • Is there exactly one default show rule for custom types? That's how it works right now for built-in elements, but it's a bit inflexible for e.g. HTML. Sometimes, I'd like to have different ones depending on a selector.
  • How to do a built in show-set for custom types?
  • Do we want to support hoisting and, if yes, how?
ancient wagon
#

Could you pin that

#

For posterity

clever bloom
earnest reef
# clever bloom **Step 1: Unify value and content** - Do we accept that things like `none | auto...

I may ignore "step 2" first, and put some text under in regard to "step 1".
I have seen discussion about unifying value and content. I remember you discussed it with dherse from somewhere, but I don't remember it concretely. In that discussions, there might have some conclusion about Point 2: "Decide how new merged content + value is represented in memory".
Point 1 and Point 3 should be some design about enriching current type and merging the common type of elements content. First, it is okay to remove content type into any type, but whether remove it or not, the semantics of "content type" is convenient and should not be removed. We should still maintain which ones implement "content". I then recall that a success type design in my mind, the go interfaces. The go interfaces make good abstracting over IO interfaces. They have any and interfaces like io.Reader, fs.Fs. They also have a nice runtime detection about whether a any can be casted to io.Reader or fs.Fs safely and perform further operations. Similarly, we could introduce interface, and migrate content type to some interface, interface.

#interface inspectable {
  has(field) :: (str) -> boolean
  ...
}
#interface content-like extends inspectable {
  element() :: () -> element_func
}

And we can also have some way to cast an any to a content-like value dynamically:

let v: Value;
if let Some(content) = v.dyn_cast::<ContentLike>() {
  // do things about the content.
}

And we can also maintain some cache about interface casting like go to accelerate the checking. I remember they did it.
This could be internal in step 1 and not visible to users.
I also think it is beneficial to expose interfaces in future, where we will have content-like, table-like, block-like, and other interfaces. Point 4 may be resolved if we have interfaces.
I have no clue about Point 5.

clever bloom
#

Interfaces are a possibility, but I'm not sure I like the complexity they introduce. For what it's worth, the concept of anelement_func will be completely gone from the language, it'll just be a type.

#

If we had an inspectable interface with has(..), then almost every type would need to exhibit a has method, wouldn't it?

earnest reef
ancient wagon
#

Maybe Interfaces aren't needed if we just have good reflection

earnest reef
#

If this is a goal, we may ultimately introduce the complexity. I understand typst language should eliminate most unnecessary complexity about typesetting a nice output, but it deserves to introduce some language complexity to overcome the increasing complexity of writing large and collaborative packages that correspondingly.

ancient wagon
#

Since it's a dynamic language anyway

clever bloom
#

Or do the planning fast :)

#

I agree that if we don't do it right, we'll just grow it badly bit by bit

#

There needs to be some vision

#

And mine is still lacking as of yet

ancient wagon
#

Interfaces have the upside of being usable in static analysis

earnest reef
#

I also have some not direct humble suggestion. we can have some editions like rust, like typst2025, typst2028, and so on, to reduce fear of breaking language design.

clever bloom
#

we can definitely consider something in that direction, but I think it's more relevant post 1.0

#

for custom types, e.g. I would try to ship a compatibility layer that magically makes .has(..), .location(), etc. work with warnings for one or two versions and then drop it

ancient wagon
earnest reef
clever bloom
#

so if interface are just TS-like, for that to work, almost every type would have to have that method

earnest reef
# clever bloom not every, but many

I'm not sure whether I really understand the point. We will keep as is. Only the types that currently is a content, will continue to implement a has function. Specific to current implementation, this also doesn't mean we should get rid of struct Content<T>. There will be an internal content-like interface and Content<T> that implements content-like and proxies rest methods to the inner value.

clever bloom
#

I'm just not sure I like that

earnest reef
# clever bloom I'm less talking about the compiler and more how it works on a conceptual level....

okay, tbh, limited to doc pages, the current doc pages don't have to show all "interfaces" explicitly. I saw some people in typst-jp mentioned that "caution: the pages that have content type mean that you can use the value like a content, which have some common methods like has". That means that it is okay to not mention any interfaces on the docs pages, and we can optionally render another version of page like cargo docs, which put a lot of implemented traits under the doc pages of the struct.

ancient wagon
#

Could also just link to interfaces instead of showing the full impl everytime

earnest reef
#

tinger might have some better understand to the sentence. This is also correct. He says it not necessary to list interface methods. I say it not necessary to list all of the interfaces on every pages, but you could have some page like Symbols for not important interfaces like inspectable or encodable page, which list all of the types implementing the interfaces.

clever bloom
earnest reef
ancient wagon
#

Is this still about the docs or about the overall API bloat that comes with having more and more interfaces

clever bloom
#

I didn't mean docs anymore

#

I meant that a single interface bloats the API of all types that implement it

earnest reef
ancient wagon
#

Well but they were there before, just not documented on the "types" themselves

earnest reef
earnest reef
earnest reef
#

👀 I find that to ensure all the code could be migrated to the world having custom types where there will be no content (probably), we should find a satisfying solution to achieve type(value) == content. This is a concrete problem and might not be possible if we don't make interface public (perhaps)...

clever bloom
clever bloom
earnest reef
#

No, there might be no type(value) == content in future, since in Point 4 you said moving content.label() to label.of(value). This should be not great, at least not great on syntax. For example, rust have a.map(f) and python have map(f, a). I personally not like the python's map, comparing with rust's map. Changing the content.label() to label.of() looks like a similar change and we will get it work similar to map in python. We should have some design to allow to call label.of using the method syntax, should it? Then, we could continue allow people to write some-value.label() which are rewritten to label.of(some-value).

clever bloom
#

personally, I prefer methods for things that are type specific

#

and functions for things that are available in all values

#

map e.g. is specific to options or iterators, so it's the first camp

#

label would be available on everything, even an integer

earnest reef
#

yes, some functions work well with explicit call (I mean type(x)), if you are going to make it work on all values, but I generally like method call (I mean x.type()) if it is not commonly work on values. The map is pretty bad as explicit calls in my opinion.

clever bloom
#

yes because map is often chained

earnest reef
#

I feel we should have a good habit to make some summary notes, and find things that we believe can start at once to make progress.

#

I did make a note on the IDE, though I haven't started it...

earnest reef
# clever bloom yes because map is often chained

chained is a point, but I should point out that this is bad in terms of providing a map function but not allowing it to be called by method. This is harmful on both readability and writing experience I think. As a user, I'm loving if I can finish a thing by repeatedly dot on some expression and ask IDE to feed back something that I would like to make. So, label.of() may be good as a function on. I agree with you. typst could do better if I'm allowed to have a.label() legally in some way which will end in calling the label.of, as what we have now.

earnest reef
#

Not sure whether it is good, I'm thinking of what if every value is allowed to map:

let l = label.of(value) // <=>
let l = value.map(label.of) // or unusual design in language
let l = value.(label.of)
let ty = value.(type)
let s = value.(str)

This also help some other cases that I was hurt:

figure(
  align(
    center, 
    table(...)
  )
) // <=>
table(
  ...
).map(align.with(center)).map(figure)

show: it => block(html.frame(it)) // <=>
show: it => it.map(html.frame).map(block)
clever bloom
#

sounds a bit like the pipe operator

#

there have been discussion about that before

hybrid cairn
#
  • Do we accept that things like none | auto | content degrade into none | auto | any, which is a slightly weird type or are not all types content after all
  • Settle joining behaviour of array/dict vs sequence

First, it is okay to remove content type into any type, but whether remove it or not, the semantics of "content type" is convenient and should not be removed

I kinda agree with that sentiment, that not all types would be content after all. I want to add what I was thinking about in that regard.

Maybe there could be a distinction between types with a default show rule and ones without. That would mean, without interfaces or anything, that these types could be joined into a sequence, while others are restricted to being used as values. For example right now for x in (1pt, 2pt) { stroke(x) } gives "cannot join stroke with stroke", and that would continue to be the case because stroke doesn't have a default show rule.

That would also allow us to keep special joining behavior of arrays and dicts, since they're not showable.

#

on the other hand, I'm not sure if that's so ideal. It resembles the implicit connection between named & optional arguments, and prevents users from making types showable if the type's author hadn't planned for it to be. And it at least seems to clash with the idea of multiple default show rules.

crisp fog
crisp fog
sleek tangle
#

From https://github.com/typst/typst/pull/6547

This PR rewrites the foundations of native elements and the #[elem] macro. The goal is to move more of the logic into normal as opposed to macro-generated code, improving maintainability. The whole system would've needed an overhaul with the unification of types and elements at the latest, so this is a good incremental preparation step for this.

One step closer to custom types in Typst <3

GitHub
Rewrite foundations of native elements by laurmaedje · Pull Reques...

This PR rewrites the foundations of native elements and the #[elem] macro. The goal is to move more of the logic into normal as opposed to macro-generated code, improving maintainability. The whole...

deep aspen
#

Its quiet here. Is there any updated timeline for type system unification and custom types issue, now that 0.14 has landed?

clever bloom
twin ivy
#

That's sad, such a cool feature to have (not demanding anything of course). I was wondering if there are some more concrete thoughts about syntax or implementation or something

ancient wagon
#

There are concrete thoughts about syntax in here

#

And they have largely remained the same

#

I can't find the screenshot but it's somewhere in here and on a discord issue

#

Perhaps we should pin it in here

mellow yoke
#

Hi everyone, just joined the server to catch up on the state of custom types in Typst and I’d like to add my two cents’ worth to the discussion:

First of all I have to say that I am very pleased with the design ideas from the original blog post, in particular the idea of implementing former element functions as ordinary product types with a show method. I think this design strikes a really good balance of adding expressivity to Typst’s scripting language while remaining approachable for non-technical people as a typesetting DSL and even making the language more consistent from a certain perspective.

Speaking of Typst as a DSL, I have always liked how the language uses domain specific types like color or alignment. This also includes the type content, which is why I was a bit concerned to see several suggestions here with the goal of unifying content with the all-encompassing any type. I’d therefore like to add my support to those comments that argued for keeping a dedicated type for visible content in Typst.

#

I do, however, agree that the current mechanism to join different elements together should be reworked if we want to properly introduce types to the language, since the current version of this “join” operation needs to combine arbitrary types of showable content.
I might be wrong here but I think there might be a rather simple fix for this: We could just require that elements that should be treated as content are explicitly converted to content before they get combined. With the proposed design this would simply amount to calling the show method on those elements.
Of course this would introduce a lot of noise to content producing code blocks but on the other hand we could require the explicit show conversions only in code mode and have all elements of a content mode block ([ … ]) be converted implicitly. Isn’t this exactly why the dedicated content mode exists in the first place?
If a code mode block contains only a single showable expression it could also be implicitly converted because it would technically also fall into content mode before being shown. This means the explicit calls to show would really only be required when there are multiple elements in a code block with + operators or line breaks in between.

mellow yoke
# clever bloom this is funny cause it looked so familiar. I just dug and found these notes from...

Another thought I had while skimming through the long history of this discussion is that it might actually be a good idea to migrate the current show rule syntax to use select and apply instead (like in the design notes of the comment I replied to).
While I believe that show works great as a name for the method required for showable types/elements in the new design, I think that it might become confusing to have so many semantics associated with the same keyword/name (as exemplified by the discussing shortly before the referenced comment was posted). This is especially true if users are required to explicitly convert elements to content as proposed above.

I am not familiar with Typst’s implementation, so maybe I am missing something and my ideas do not make any sense, but this is just what came to my mind while I was reading through the history of the discussion today.
Any thoughts?

mossy elm
#

Definitely. The + symbol is really overloaded anyways, and often misused as well since mathematically concatenating sequences of anything is really more of a product than it is a sum, when you think about how it behaves algebraically.

But joining should really not be the * symbol (alone) either. A simple suggestion would be a ** or a ***. A fancier one could be something like <*, *>, or <*>, but the last one might annoy Haskellers. I would prefer a symmetric operator shape, though, especially if joining implies an allocation.

crisp fog
#

how about ⨝ U+2A1D JOIN /j

mossy elm
crisp fog
#

For a serious suggestion, I'd say ++, which is already somewhat established as "concat" in some places

runic idol
#

Is ~ still open as an option?

#

Raku uses that and I enjoy it a lot

ancient wagon
#

I don't think Laurenz is particularly interested in adding another operator just for joining.

hybrid cairn
#

?r I'd say ; is the joining operator already, anyway:

#{[a]; [b]}

#{(1,); (2,)}

It's kinda the definition of joining that ; would work for this – only difference to regular operators is that it only works in block, not parenthesized expressions

humble roostBOT
ancient wagon
#

Honestly, what if we did allow it in expressions

#

Ah no that breaks the cases where semicolons are used as stops to code mods

hybrid cairn
#

and #a; b would still mean the same, you'd simply have to write #(a; b) to use the "new" meaning

ancient wagon
#

I was thinking of #foo;bar for example

#

the explicit lack of whitespace here

#

though i wonder

#

?r ```
#let foo = 1
#let bar = 1

#foo+bar

humble roostBOT
ancient wagon
#

well i guess its' no problem but tit woudl be confusing

hybrid cairn
#

not more confusing than that plus, imo. arguments and property access attach stronger, i.e. #foo(), #foo[] and #foo.bar work, but all other operators to my understanding already need to be grouped

ancient wagon
#

Yeah but now it would have two intended uses whereas the previous joining use was unintended i would argue

#

it always meant stop statement/code mode

#

the joining was more so an effect of blocks themselves without the semi colon already

#

this is mosty #syntax at this point anyway

hybrid cairn
#

I wouldn't say that the previous joining use was unintended. Yes, semicolon is just an explicit way of separating statements (rather, expressions). But sequences of expressions are fundamentally joined; semantically there's no difference between explicitly delimiting expressions with ; or joining them. In that way, I think the semicolon really is the joining operator, even if it's sometimes ok to omit it. The only thing that's missing to make it truly so is allowing it in parentheses.

Coming truly back to #1175895383600275516, the issue for me is not so much whether there should be an explicit joining operator (if we want it, ; is available) but what the semantics would be. Right now, we can join contents and we can join arrays (for example). If content is no longer a type and any type can have a show rule, it would make sense that {arr1; arr2} should no longer be the same as (arr1 + arr2) but instead sequence(arr1, arr2), where both children can be separately styled. I don't know what broader consequences that would have; it would probably complicate DSLs like CeTZ a bit (but extracting the underlying array from a sequence is trivial) and could lead to harder to read error messages.

ancient wagon
#

I meant that the semi colon was not intentionally a joining operator

crisp fog
#

there is one expression where joining wouldn't behave like the semocolon: let x = 1

hybrid cairn
#

why not?

crisp fog
#

?r ...turns out let is way weirder than I thought.
I would've expected these to error:

#(let x = 1, [#x]).join()
#(let x = 1, [#x])
humble roostBOT
echo moat
grim lake
mossy elm
#

What about ; within function argument lists? Wouldn't that cause a bit of a headache?

#

Considering what ; currently does in the context of srgument lists.

reef sorrel
mossy elm
#

How would mat work then?

reef sorrel
#

We would say that math is special

#

And just disallow it for code

mossy elm
#

Separate size argument?

reef sorrel
crisp fog
mellow yoke
# mellow yoke I do, however, agree that the current mechanism to join different elements toget...

It seems like my comment from a few days ago has revived the discussion around the semantics of joining content in Typst. However, no one has responded to my proposed solution so far: What if we address the excessive overloading of the join operation by making content a distinct type and limiting the join operation to work on values of type content?

Instead of

#{
  "Hello world!" + image("img.png")
  rect() + str(42)
}

(which mixes text with an image and a rectangle and confusingly doesn't work with 42 as a plain number) we would have

{
  "Hello world!".show() + image("img.png").show()
  rect().show() + 42.show()
}

where joining happens explicitly only between content and we can still reuse the + operator because its overloaded semantics is now unambiguously determined by the type of its operands (number addition when the operands have type num, array concatenation when both operands are arrays, content joining when both operands are of type content, etc.).

Of course this makes the syntax much more verbose but keep in mind that this only applies to code mode. In content mode (between [ and ]) we could say that all expressions are implicitly converted to content (via x.show()) and only then joined together.
This means the previous code block would still be equivalent to

[
  #"Hello world!"#image("img.png")
  #rect()#42
]

which would of course be written more directly as

[
  Hello world!#image("img.png")
  #rect()42
]

Notably, the joining operation remains unambiguous in this notation since we expect [1 + 2] to be displayed as "1 + 2" instead of "3" and would anyways need to rely on code mode to perform other kinds of join operations like number addition.

#

In my opinion this would trade some verbosity in code mode syntax for much better code clarity and language consistency. But it's likely that I am missing something here 👀

clever bloom
echo moat
#

We need to clone laurenz

clever bloom
#

The two mes would probably fight all the time

deep aspen
#

@clever bloom ping?

ancient wagon
#

?

#

What's that for

clever bloom
#

Probably this:

Feel free to ping me if I don't.

#

I still have it in my reminders

clever bloom
# mellow yoke It seems like my comment from a few days ago has revived the discussion around t...

So, it took a little but I found some time (or rather accumulated enough of a bad conscience for not answering :D).

We discussed custom types a bit during our team on-site event and, while nothing final, reached at least a bit of consensus on some things. One of these was that it's kinda bad if we fully lose the notion of content, to the point that anything that takes content just takes any.

That said, content is likely to become a set of types rather than a type by itself. (Or, in other words, a type pattern, which in turn is something that's to be unified with selectors).

Drawing the line between content and non-content is not easy. A heading is definitely content and an array isn't. But what about, for example, an integer? Or about a datetime? With the current type/element split, they aren't (and it shows in that you need to write [#n] for an integer to join with other content), but integers are still among a few blessed types that behave at least in some way like content: They don't show via repr. Instead, they go through a special match statement (in Value::display) because they do have a natural way to display. The full list of types that have this special handling is none, int, float, decimal, str, version, symbol, and module.

Why does this special handling exist? Because they are not elements and yet they have a natural way to be displayed. To me, this kind of implies that these types ought to be content under the new model where all elements are types. This would also make using numbers in tables much more natural. You wouldn't have to add awkward .map calls anymore to convert numbers to strings/content.

#

It would also mean that integers and floats can just have a show rule with local-aware formatting. The same goes for datetime and duration. These are currently not blessed and show via repr but they ought to show in a local-aware fashion via an overridable show rule.

So the verdict is: The current plan is to have a distinction between content and non-content but without content being a type. Rather, a type would statically declare whether it "conforms to the content interface" (likely implicitly via presence of a default show rule). Content types would join into sequences while non-content types (like arrays and dictionaries) would not be accepted where content is expected and could have other custom joining behavior.

What would need to change is the + operator, since it's truly ambiguous in a world where integers are content. My plan would be to reserve addition strictly for numeric operations and have joining occur via blocks, e.g. { a; b }. If we really wanted, we could have another operator for joining, but I don't think it's a big loss to remove support for joining via an operator (since, as previously called out, ; already kind of is that operator). (I would try to have compatibility behavior though to not break everything right away.)

Coming to your suggestion with this in mind: I think a .show() method or something similar isn't really needed in this design (and it would indeed result in a lot of boilerplate) as the ambiguity can be resolved in a different way.

echo moat
#

Discord blog post

mellow yoke
# clever bloom So, it took a little but I found some time (or rather accumulated enough of a ba...

Thank you for taking the time to write such a detailed response! I agree with many aspects of this direction, especially the idea that content should remain meaningfully distinct from arbitrary values while being associated with some kind of interface or capability.

That said, I still find myself preferring a design with an explicit content type and explicit conversion into that type.
The main reason is that it keeps the boundary between data and document content explicit and uniformly typed.
With a dedicated content type, joining and show rules can be described in exactly the same way as other language constructs:

  • joining has type (content, content) -> content
  • a show rule for values of type A has type A -> content
    This makes document construction conceptually very regular: values are first converted into content and content is then combined.
    In the alternative design, the conversion into displayable content becomes implicit. Conceptually, joining would then operate on something like “any value that conforms to the content/display interface” and produce some internal content representation. Likewise, show rules would implicitly return that representation.
    Even if these details are mostly hidden from users in practice, I think there is value in the language model itself remaining explicit and uniform. In particular, I find it appealing if default show behavior can be expressed as ordinary methods or functions rather than as a special implicit mechanism in the evaluator.

At the same time, I think most of the practical advantages you mentioned still remain available in a design with explicit conversion.
For example, types like int, float, datetime, or duration could still provide customizable default conversions to content via show methods or show rules. Numbers in tables could still be conveniently displayed by using content blocks. The main difference is simply that the conversion step remains visible at the language level.

#

To me, the primary advantage of the implicit approach is therefore mostly syntactic: it avoids repeated .show() calls in expressions such as

x.show() + y.show() + z.show()

However, I think this verbosity is somewhat mitigated by content blocks anyway:

[#x #y #z]

While this still introduces a bit of syntax, I also think it has an upside: Code blocks become more clearly focused on computation while content blocks become the canonical way to compose document content, much like math blocks are the preferred place for mathematical notation.
Personally, I find this separation appealing because it makes it more explicit which expressions contribute to the rendered document and which merely compute intermediate values.

Of course, there is clearly a genuine tradeoff here between explicitness and convenience, and I can absolutely understand why others may prefer the more implicit model. I just personally find the explicit content approach slightly more consistent from a language-design perspective.

Either way, though, I am very excited about the future direction of Typst’s type system — this all sounds extremely promising already :D

opal bobcat
#

It seems to me that a content class and a "representable as content" interface should be two different things

rugged cape
#

I just tried implementing a custom-element function, as it does not require any new syntax. Every custom element has a default show rule that just resolves to Content::empty(). The given snippet works without errors. Would this be a feasible way to implement custom elements?


#let foo = custom-element((prefix: none, body) => {})

#show foo: it => [
  #it.prefix: #it.body
]
#show foo: set text(blue)
#set foo(prefix: "ABC")
#show foo.where(prefix: "DEF"): set text(red)

#foo(prefix: "DEF")[Hello World]
ancient wagon
#

Without seeing how it's implemented we can't really judge that

#

But if the implementation is sound then we'd also want it to use the proposed type syntax down the line

rugged cape
#

Well, the implementation is by no means clean yet, but as a first draft I added a CustomElementInstance content element that holds the custom element's field values. Show and Set rules have new enum variants for custom element properties and selectors. The custom-element function returns a CustomElement definition that just holds the field declarations.

My first intuition was to just change the Element struct to be able to hold user defined "vtables" of some sort. But I found that to be rather hard to implement correctly.

clever bloom
rugged cape
#

Got it, stopping experimentation

pastel plover
#

(I don't think this has been suggested before, sorry if it has)
Replacing selectors with pattern matching is an established idea, but I think we can also replace type hinting with pattern matching.

As I understand it, current proposals are along the lines of:

show heading(level: 1): it => ...
let afunction(number: int) = ...

But we could have

show it @ heading(level: 1): ...
let afunction(number @ int) = ...

If the pattern matching syntax is flexible enough.

pastel plover
#

I used @ to be similar to rust and haskell, but the syntax could use : to be less weird and more like type annotations

ancient wagon
#

I don't think : will work because it is used in a bunch of places where it would be ambiguous, but I haven't thought to much about this

pastel plover
ancient wagon
#

Well that's mostly what I was thinking of, like i said thought, I haven't thought too much about this.

#

I think it's more important what the semantic implicaitons of this are

#

Especially with the whole shebang of types as values

pastel plover
#

Types as values isn't worthwhile without making the whole data model much more complex.
The languages that come to mind are Python an JavaScript, which both have a lot of a data model and what one might call a data-driven type system. And then there are dependently typed languages but Typst has no interest in that side of things.

#

If Typst never gets sum types (and in a dynamically typed language there's not much reason to add them) then a type is really just a constructor and an associated scope

pastel plover
ancient wagon
#

I'm not even talking about sum types, I'm saying how does one match on the value of a type in cases like show figure.where(kind: image)

#

This currently works fine, but pattern mathcing on types introduces ambiguity there that needs to be resolved, this has been discussed here before with no clear solution that is both easy to write and read.

pastel plover
#

I can't find the discussion you refer to, I'm guessing this was along the lines of:

  • constructors are values in the current scope
  • which implies that heading in the pattern heading(level: 1) is a reference to a binding
  • which implies we can refer to existing bindings in our patterns
  • which implies that the pattern heading should match only the value heading (the constructor for headings)
    Furthermore, if patterns can reference bindings in the current scope, then things like
let x = 1
let x = 2

Would cause an error not because shadowing is not allowed but because the second pattern is evaluated like let 1 = 2, which fails to match

#

You can't win because in e.g. heading(level: x), x is intended to introduce a new binding but heading is referring to an existing binding.
If you want to have consistency but not ambiguity then you need special syntax for one or the other, but they're both important enough that this is not acceptible?

#

As far as I can tell Lean 4 and Haskell both make the distinction based on whether the identifier appears in constructor position or not. I think Typst sacrifice some flexibility and do the same.

ancient wagon
#

I guess that somewhat sums it up

#

Though the example of a pattern being used in the left hand side of an declaration was not mentioned y et

pastel plover
#

oh, another problem.
Typst's current syntactic sugar for function definitions would clash with basically any pattern matching syntax.
I think it would have to use a different keyword than let or be removed entirely.

#

This might be considered acceptible syntactic churn given this bit of the migration could be automated and there'll be a lot of breakage anyway

#

I assume that other keyword would be fn or fun, I would vote for fun because it's more fun

ancient wagon
pastel plover
#

Well we currently usually write function definitions like
let square(x) = x * x
But that's yet another overloading of the same syntax, because under the pattern matching interpretation that's matching the value of x * x on a constructor square with a field x

#

In this case Haskell copes by recognising the constructor (because it is in pascal case), but Typst doesn't have that luxury

#

So in a statement like let code-block(a, ..rest) = ...
code-block can be interpreted as

  • (in function definition) The name of a new function being defined
  • (in pattern matching) The name of an existing binding which is expected to refer to a constructor to be used in pattern matching
  • (in pattern matching) The name of a new binding to be introduced equal to the constructor of the rhs of the let binding

a and ..rest similarly can be interpreted as

  • (in function definition) Function parameters
  • (in pattern matching) The names of existing bindings which the values of certain fields must be equal to for the pattern to match
  • (in pattern matching) The names of new bindings to be introduced equal to the values of those fields from the rhs of the let binding
#

And I'm advocating for always using option 2 in the first case and option 3 in the second case (because this is how other languages behave) and introducing a new keyword for when option 1 is needed

ancient wagon
#

I generally agree

pastel plover
#

Oh and currently Typst allows destructuring in reassignments, not just variable declarations, which will probably cause even more problems

ancient wagon
#

Oh wow good morning i was just about to write something lol

pastel plover
#

good morning

ancient wagon
#

Is it actually much of a problem? I figure it works very similar to before, you bind whatever the pattern declares as blanket patterns

#

Hmm, actually yes, this would be problematic for loops and constructs which are more imperative than pure

pastel plover
#

Looking at it I think it's probably not ambiguous but it should still be avoided because it requires n-token lookahead to correctly parse

#

e.g. depending on whether there's an = sign way later on duplicate identifiers in function arguments are valid or invalid

ancient wagon
#

I'm guessing you mean something like this?

#fn func(headings: array) = {
  let x = [init]
  for h in headings {
    let heading(body: x) = h
    // does this bind x in here or overwrite it?
  }
}
pastel plover
#

that but without even the let

#

with let it should be creating block-scoped new bindings

ancient wagon
#

but what does let (x,) = (h.body,) currently do? I figure it should have the same effect and it'll be fine

pastel plover
# 
#{
  let a = 1
  (a, ..) = (2, 3)
  a
}
#

currently you can have that

ancient wagon
#

ah

pastel plover
# 
#{
  let a = 1
  heading(a) = h
  a
}```
ancient wagon
#

Yeah I would probably argue for disallowing patterns on the lhs of an assignment

#

it's expressly a feature of certain syntactic constructs such as let bindings and closure arguments

pastel plover
#

tbf actually the current system allowing array and dictionary destructuring does have the same problem of n-token lookahead, but with more powerful pattern matching the problem gets way worse

ancient wagon
#

With the context of the discussion i know what this means, but i would've not known what this was meant to be beforehand i think

ancient wagon
#

and can be changed to work with regular let patterns and some intermediaries

#

in this example no extra intermediate variable is required, this is only the case when this happens inside a construct which can have local side effects i.e. loops and the like

#

Though it sounds tough to detect this reliably for automatic conversion

pastel plover
#

I think removing it is likely the best option but I'm worried there'll be some poor typst user who has used this everywhere

pastel plover
clever bloom
#

Is pattern destructuring á la a(b) = c or let a(b) = c a new proposal here or is there the impression that there are plans to have this? Because I didn't have any.

ancient wagon
#

It was an example of taking the (...) = (...) syntax further which only complicates things

clever bloom
#

I'm asking specifically because of this:

Replacing selectors with pattern matching is an established idea

#

Which is half true. The idea is to unify selectors and type annotations, but that doesn't necessarily involve complex pattern destructuring

pastel plover
#

I assumed that if we were going to have pattern matching syntax to replace selectors it would also be applied to the other places where Typst already has some (weak) pattern matching

clever bloom
#

So far, there weren't plans to have pattern matching syntax, only pattern matching. In my ideas, patterns were always normal values.

#

And type annotations would also be values.

#

Syntactically, I would also say that changing the function syntax away from #let f(x) = x + 1 is not something I would realistically like to do.

#

Not just because of breakage, but because I like the current syntax a lot

pastel plover
#

I'm not sure you can really have "patterns are normal values" without using rather strange syntax for patterns

ancient wagon
#

I think it can work, consider this:

pastel plover
#

if they are normal values wouldn't that imply I can write something like

let my-selector = heading(level: 1)
show my-selector: ...
clever bloom
#

You can write that today if you add .where

pastel plover
#

and that's fine, because the where makes it clear that it's a selector not an actual heading, but the syntax I showed is (I assume) the natural syntax pattern matching would use

ancient wagon
#

Yes but it would introduce more ambiguities than it solves

#

and we could still introduce syntax for this like let pattern = where heading(level: 1) or something

#

if really necessary

pastel plover
#

Would we write show where heading(level: 1)?

#

I don't hate that but I think for consistency it would have to be that way

ancient wagon
#

if we go down the patterns are values route which is an extension with what selectors were before, then changing the syntax for selectors seems like a big change for little use

#

It was just an example, it's not too different from heading.where

#

which is what we use already

clever bloom
ancient wagon
#

I'm just saying that, if we consider them values then obviously we can't use the "natural" pattern mirrors construction syntax

pastel plover
#

If you have patterns/selectors/types unified but keep the existing destructuring, that gets a bit messy --- you'll end up having to implement very similar syntax for destructuring arrays/dictionaries and for patterns over arrays/dictionaries. And all the places where destructuring is used are the same places where you might want type annotations, so they're going to end up in the same places. Having 3 different array related pieces of syntax --- literals, patterns, and destructuring, may cause confusion.

#

Although,.. patterns as values probably is necessary for things like query, so what I had in mind definitely couldn't work

pastel plover
pastel plover
#

Languages like Rust and Haskell with pattern matching but patterns are not values would use general predicates where we might use a pattern.
From a design perspective there's no reason Typst couldn't do this but there may be a performance concern

#

You could reasonably have e.g. query((it @ heading(level: 1)) => it.body) return a list of level 1 heading bodies

pastel plover
#

I was just looking at #3356 (the set stroke(...) issue) and I think it ties in with the future of package configuration, which also ties in here.

afaik the current expectation is that when packages can define custom elements, package configuration can happen with set rules and will no longer be a problem. I agree with this in the majority of cases. However set stroke is an example of configuration for std which applies to multiple elements and hence cannot be done with a simple set rule currently.

If there's no dedicated feature to accommodate this, package authors will probably start using 'void' elements which are not intended to be constructed and only exist so their fields can be set and read in the default show rules of the other elements the package defines.
We can embrace that and call it a design pattern (I advocate for this) or offer an alternative.

If this were applied to the standard library, that would mean creating a non-constructable element default_stroke.

As for the "offer an alternative", as far as I can see settable fields on non-constructable elements do just what is wanted such that a dedicated feature would end up being very similar, which is not a good use of developer time and might (probably not) be confusing for users.

pastel plover
#

This thread has ~1800 messages since 2023, even if I take the time to read all that I'm not going to remember the important bits. Can we have a more organised way to keep track of the probably a few hundred proposals around?

grim lake
#

There is at least one pinned message, but I think this feature is underutilized (from what I can remember).

pastel plover
#

we could have a github repo and use issues fairly effectively

  • An issue is closed as not planned if it's been considered and rejected
  • Left open if it's an option still on the table
  • Closed as completed if it's decided upon
#

and the 'blocking on' feature can be used to mark where a particular proposal depends on particular other proposals

crisp fog
#

issue threads can also get unwieldy tho