#Generic data structure to hold different types of objects

1 messages · Page 1 of 1 (latest)

upbeat wyvern
#

I'm trying to write an HTML template engine that parses HTML like this:

<!doctype html>
<html lang="en-US">
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width" />
    <title>My test page</title>
  </head>
  <body>
    <table1 items="people"></table1>
    <div class="@settings.className">@person.FirstName</div>
    <for items="people" item="person" index="i">
      <for items="person.Nicknames" item="nickname" index="j">
        <div>@i @j @nickname</div>
      </for>
    </for>
  </body>
</html>

and can replace variables like @settings.className with contextual data (e.g. the data could originate from a Dict(String, Dict(String, String)) where the top-level key is settings and the lower-level key is className. I'd love to be able to represent different structures of data (e.g. a List(String) under a key, with different levels of nesting), and am trying to figure out a good data structure in Gleam to represent this. Any ideas?

#

(The idea is that for and table1 could be custom tags that could have accompanying code/HTML in separate files)

mighty tree
#

Would a type with various constructors be helpful for your use case? It is possible to create a generic tree like struct.

upbeat wyvern
#

like this?

pub type Context {
  CString(value: String)
  CInt(value: Int)
  CFloat(value: Float)
  CDict(value: Dict(String, Context))
  CList(value: List(Context))
}
#

I was toying around with having a recursive data structure of some sort

#

but didn't want to create a monster

#

🙂

mighty tree
#

Yup! Just like that

upbeat wyvern
#

constructing the object is not really pretty in code, so I was hoping to come up with something that is elegant, and wouldn't scare people away

mighty tree
#

Seems sensible to me. It is for generating HTML rather than parsing but there is the nakai library.

upbeat wyvern
# 
import nakai
import nakai/html.{type Node}
import nakai/attr.{type Attr}

const header_style = "
  color: #331f26;
  font-family: 'Neuton', serif;
  font-size: 128px;
  font-weight: 400;
"

pub fn header(attrs: List(Attr), text: String) -> Node {
  let attrs = [attr.style(header_style), ..attrs]
  html.h1_text(attrs, text)
}

pub fn app() -> String {
  html.div([],
    [
      html.Head([html.title("Hello!")]),
      header([], "Hello, from Nakai!")
    ]
  )
  |> nakai.to_string()
}
#

(just looking at how they're doing it)

mighty tree
upbeat wyvern
#

true 🙂

#

which uses a string path (similar to the whole @settings.className sort of string path

#

but it's decoding JSON

#

so it's somemwhat similar, but not quite what I need I don't think

mighty tree
upbeat wyvern
#

yeah,

#

I definitely need types, since from what I've read, there's no reflection in Gleam

#

so I couldn't use gleam_json to represent the object, from what I can tell,

#

e.g.

import myapp.{type Cat}
import gleam/json.{object, string, array, int, null}

pub fn cat_to_json(cat: Cat) -> String {
  object([
    #("name", string(cat.name)),
    #("lives", int(cat.lives)),
    #("flaws", null()),
    #("nicknames", array(cat.nicknames, of: string)),
  ])
  |> json.to_string
}
#

since there isn't a type to use in a case expression

#

thanks for the tips 🙂

mighty tree
#

Maybe parsing to nakai types?

upbeat wyvern
#

oh, that's an interesting idea

mighty tree
#

The problem with parsing to a dictionary would be retrieving the data. Since everything would return a Result.

So something more specific than a dictionary would work out. More concrete types can help (like your example or nakai).

upbeat wyvern
#

I think one advantage I'd lose is being able to change templates without needing to rebuild/regenerate code

#

(if I parsed the HTML into nakai types)

mighty tree
#

What do you mean by rebuilding?

upbeat wyvern
#

if I write Gleam code that reads in HTML template files,

#

I could change the HTML and refresh the page,

#

without having to do code generation (rebuild the Nakai code when parsing the HTML)

#

or were you saying that could be done on the fly

#

sorry, just thinking out loud

mighty tree
#

Nah just trying to understand your aim.

upbeat wyvern
#

basically, I want to have a templating system that reads like HTML, formats and syntax-highlights just like HTML, that works kind of like React components,

#

so I can compose my own components

#

the Gleam template library reads in the HTML template files when a user hits a web page, processes the custom tags, and replaces the variables

mighty tree
#

Oh so if you go for a parser to types, I think you will always need to transform back to HTML.

Unless your types work as some kind of DB for HTML.

upbeat wyvern
#

yeah,

#

I'd like to avoid that if I can

mighty tree
upbeat wyvern
#

so far, I'm successfully parsing the custom tags in the document (I don't maintain a tree of any of the other tags, since that HTML will be taken verbatim),

#

e.g. identifying the start and end tags (or self-closing tags) for table and for

#

I'm trying to figure out how to set a context (set of data) so that when I'm generating the final HTML from the template,

#

I can replace all of the variables in the HTML

#

oh, you've created a templating library like this previously?

mighty tree
#

yeah, trying to find it, never published cause in the end nakaid did solve my issue, but your problem is a bit different

upbeat wyvern
#

I'd also like to have a custom tag, e.g. <include src="myscript.js" /> so that I can include some JS

#

(I'd like to write JS via Gleam, and include it)

mighty tree
#

So I didn't implement an HTML parser I used bbmustache and erlang library for interpolating templates. But I had this API for composition that you may find interesting?

The module: https://github.com/chouzar/luster/blob/6b064bd6ffe2caf956fd6339014f4ac30b2bc5b5/src/luster/web/plant.gleam

How it's used:

#

(sorry for the spam)

upbeat wyvern
# 
  <section id="alert-message" class="alerts">
    {{{alert-message}}}
  </section>
mighty tree
#

I'm cheating 👆 😄

upbeat wyvern
# 
  Document(
    mime: HTML,
    template: plant.lay(
      from: "src/luster/web/battleline/component/layout.html",
      with: [
        #("session-id", plant.raw(session_id)),
        #("odd-pile", draw_deck.new(card_back.Clouds, size)),
        #("draw-pile", draw_deck.new(card_back.Diamonds, size)),
        #("player-hand", hand.new(hand)),
      ],
    ),
  )
#

yes, similar concept

mighty tree
#

If what you're looking for is to change the templates on the fly maybe you don't even need the parser, doing erlang/gleam binary interpolation would be enough.

#

(well, you need some kind of parser to know where to interpolate, as you mentioned)

upbeat wyvern
#

yeah, I have to know where the tags start and end,

#

so I can transform their contents

#

cool, I can keep looking into the custom type

#

I was hoping maybe there'd be a library with a type that most Gleam devs love

#

that maybe even had code generation from Records -> the structure

#

(like Glerd and Glerd_json)

#

maybe I could use Glerd to codegen from Records to the structure I use

mighty tree
#

not that I'm aware, there used to be another library similar to mine, with the added feature that it would cache the templates into memory (instead of always reading from the file)

upbeat wyvern
#

yeah, caching would be good too, for prod

#

thanks a ton for all of your help

#

if you think of any more ideas, I'd love to hear them 🙂

#

(I'm a noob to Gleam)

mighty tree
#

are you planning on deploying this with erlang or js?

upbeat wyvern
#

erlang

mighty tree
#

nice! are you familiar with bitstrings in erlang or elixir?

upbeat wyvern
#

no

#

I've seen mention of them, but noticed it was erlang-only

#

haven't looked into them

#

I think I read a bit about them and thought I'd never use them

mighty tree
#

And the gleam stdlib contains many functions to operate on bit arrays and convert from to strings.

upbeat wyvern
#

so it'd make parsing faster?

#

(so far I'm just using pop_grapheme and string matching)

mighty tree
#

Not parsing, but the generation, yeah I think if you're working with graphemes you're already using bitarrays.

upbeat wyvern
#

oh interesting, didn't know that

mighty tree
upbeat wyvern
#

yeah, I am concatenating like this after popping graphemes: username <> "@" <> domain

mighty tree
#

I guess just something to keep in mind, because you're already using those APIs from Gleam lucy

upbeat wyvern
#

cool, yeah

#

I'll have to consider performance once I've made a first pass at this

#

maybe there's an equivalent of this in Gleam: ["Welcome ", name, ", your email is: ", email(username, domain)]

mighty tree
#

should do it for you

upbeat wyvern
#

wow nice: Internally it is represented as tree so to append or prepend to a bytes builder is a constant time operation

#

yeah I'll look into that

mighty tree
#

I don't know if Gleam's <> operator is constant time or just interpolation 🤔

upbeat wyvern
#

I really hope constant time, because I'm using it everywhere

mighty tree
#

maybe something to ask for in #general

upbeat wyvern
#

or do they prefer it asked under "questions"?

#

(or do they not care)

mighty tree
#

oh true questions would be a better place

#

sometimes we do get shorter questions on #general though

hushed slate
#

i.e. for JS it might not hold

#

ive seen a few instances of that iirc, i would submit a PR but i forgot where it was lol

#

i think maybe some stuff in the bit array page

#

but either way on js it's converted to simple + addition, on erlang it becomes bitarray stuff

#

iirc

upbeat wyvern
#

oh nice, so performance shouldn't be an issue 🤞

#

(outside of the fact that it's noob Gleam code)

upbeat wyvern
#

I think I have something that can do it now...it looks like this:

  let context =
    cx.dict()
    |> cx.add("settings", cx.add_string(cx.dict(), "className", "myClass"))
    |> cx.add_list("people", [
      cx.add_strings(cx.dict(), "Nicknames", ["Jane", "Jill"]),
    ])

to populate this:

    <div class="@settings.className">@person.FirstName</div>
    <for items="people" item="person" index="i">
      <for items="person.Nicknames" item="nickname" index="j">
        <div>@i @j @nickname</div>