#New decode API inspired by Toy

1 messages · Page 1 of 1 (latest)

pallid kestrel
small oyster
#

this looks lovely

waxen shale
#

Looks super cool

#

A couple note on the documentation though:
While I like the use of assert to demonstrate the output of the different decoders, I feel it could be potentially confusing as that's not actually valid Gleam syntax.

In the Records section, there's a piece of text which adds:

The ordering of the parameters defined with the parameter function must match the ordering of the decoders used with the field function.
I couldn't make any sense of this since there seems to be no other mention of a parameter function. This seems to be accidentally copied from decode?

mental sentinel
barren oriole
#

I believeparameter is a leftover from the docs of decode

waxen shale
#

Yeah that's what I thought potentially too

waxen shale
mental sentinel
pallid kestrel
#

Maybe not the next release but assert is coming

pallid kestrel
mental sentinel
#

makes sense

autumn kite
#

Another thing, in the last example (the Pokémon trainer one) you talk about the then function but it’s not used in the code and you just use the field function directly

#

Really nice!

#

I think I’ll update squirrel to use this new api

tall dust
#

There are references to zero.from starting from the bit_array section. Did you mean zero.run

mental sentinel
#

also this

image.png
#

maybe zero.then(zero.field(...)) is correct here?

violet root
#

The ordering of the parameters defined with the parameter function must match the ordering of the decoders used with the field function.
I fail to find the parameter function

#

How would one decode NaN? Or Infinity?

#

_ -> zero.failure(Fire, "PocketMonsterType"
What is Fire doing there?

#

Can I run an arbitrary userland callback on the full data or some nesting selector to decode manually?

#

As one doesnt always have control over data coming from the outside:

Is there a way to decode into variants without the data having a tag but by mere “pattern” matching? Given fields a and b => variant n given b and c => variant m?

#

Lovely docs 🙂

pallid kestrel
#

I really did not put enough effort into these docs lmao

#

That’s what I get for publishing late at night

violet root
#

They are awesome, these are just a few questions and typos I suppose

mental sentinel
dense dawn
#

This looks great! In toy, I made a specific decision to add field, optional_field and optional. This is to make it possible to parse PATCH requests, as not specifying a field and specifying it as null is semantically different. Is this somehow possible with zero/decode? Because I couldn't figure out how to do it without custom optional_field function.

autumn kite
#

Yeah you can do it with zero.at(..., zero.optional(...))
For example:

pub fn main() {
  let decoder =
    zero.at(["wibble", "wobble"], zero.optional(zero.int))

  let assert Ok(Some(1)) =
    "{ 'wibble': { 'wobble': 1 } }"
    |> string.replace(each: "'", with: "\"")
    |> json.parse(zero.run(_, decoder))

  let assert Ok(None) =
    "{ 'wibble': { } }"
    |> string.replace(each: "'", with: "\"")
    |> json.decode(zero.run(_, decoder))

  let assert Error(_) =
    "{ 'wibble': { 'wobble': 'not an int' } }"
    |> string.replace(each: "'", with: "\"")
    |> json.decode(zero.run(_, decoder))

  let assert Error(_) =
    "{ 'wibble': 1 }"
    |> string.replace(each: "'", with: "\"")
    |> json.decode(zero.run(_, decoder))
}
dense dawn
#

@autumn kite what I meant is this:

pub type UpdateUser {
  UpdateUser(username: Option(Option(String)), email: Option(Option(String)))
}

pub fn main() {
  let assert Ok(UpdateUser(username: Some(Some("Thomas")), email: None)) =
    "{ 'username': 'Thomas' }"
    |> string.replace(each: "'", with: "\"")
    |> json.decode(zero.run(_, decoder))

  let assert Ok(UpdateUser(username: None, email: None)) =
    "{  }"
    |> string.replace(each: "'", with: "\"")
    |> json.decode(zero.run(_, decoder))

  let assert Ok(UpdateUser(username: Some(None), email: None)) =
    "{ 'username': null }"
    |> string.replace(each: "'", with: "\"")
    |> json.decode(zero.run(_, decoder))

  let assert Error(_) =
    "{ 'username': 1 }"
    |> string.replace(each: "'", with: "\"")
    |> json.decode(zero.run(_, decoder))
}

I want to be able to detect the difference between:

{ }
{ "username": null }

Both are valid, but mean different things.
The first one - do not change the username
The second one - change the username to null

pallid kestrel
#

Neat

#

You could decode a dict and check if the property exists

#

But yeah some better solution for this would be good

#

Could you make an issue please

dense dawn
violet root
#

What do I need to do to replace the old decode2...6 with the new decode here?

fn user_decoder() {
  dynamic.tuple2(dynamic.string, dynamic.int)
}

fn user_decoder_new() {
  use id <- decode.field("id", decode.string)
  use username <- decode.field("username", decode.string)
  use display_name <- decode.field("display_name", decode.string)
  use first_name <- decode.field("first_name", decode.string)
  use last_name <- decode.field("last_name", decode.string)
  // ...
  decode.success(User(
    id: UserId(id),
    inserted_at: "todo",
    updated_at: "todo",
    username:,
    display_name:,
    // ...
  ))
}
# 

Expected type:

    fn(dynamic.Dynamic) -> Result(a, List(dynamic.DecodeError))

Found type:

    decode.Decoder(User)
#

the upper is expected by sqlight and gleam_pgo I think and gmysql

pallid kestrel
#

let dynamic_decoder = decode.run(_, user_decoder)

violet root
#

I have to dig into creating functions off functions with the capture operator again

#

it caught be offguard to not know how to utilize that properly twice now

pallid kestrel
#

fn(x) { decode.run(x, user_decoder) } works too

violet root
#

I'd love to see the queries emitted by sqlight somehow, but that's unrelated

#

SqlightError( GenericError, "Decoder failed, expected Dict, got Tuple of 13 elements in ", -1, ) .. doing something wrong at the queries

pallid kestrel
#

What do you mean queries emitted?

violet root
#

the queries at sqlight client level

#

hm but the error has to do with my understanding of the decoder I think

#

it expects a dict not a tuple

chilly basin
#

check out the "Indexing arrays" section. it mentions tuples

violet root
#

that doesnt work with use and decode.success then AFAIU

#

hm zero.from( ?

pallid kestrel
#

It does.

violet root
#

the docs mention zero.from but it aint in zero module anymore

#

should I use then?

pallid kestrel
#

I don't know what you mean

autumn kite
#

I think zero.success

violet root
#

How do I combine those two:

    let decoder = decode.at([0, 2], decode.int)
    let decoder = decode.at([12], decode.bool)
pallid kestrel
#

This does work. You don't need to do anything different if you're decoding from a tuple

use x <- field(0, int)
decode.succes(Thing(x))
violet root
#

that works 🙂

#

docs say so even:

This function will index into dictionaries with any key type, and if the key is an int then it’ll also index into Erlang tuples

#

How would I apply sqlight.decode_bool to decode a bool

autumn kite
#

Something like this:

case sqlight.decode_bool(your_dinamic_data) {
  Ok(result) -> zero.success(result)
  Error(_) -> zero.failure(True, "bool")
}
violet root
violet root
autumn kite
#

I'm not sure I understand what you mean

violet root
#

I don't know how to apply the mapping above

#

decode.field(12, decode.int) decode.int is just a constant, field does not take a callback

# 
  let sqlite_bool_mapper = fn(x) {
    case sqlight.decode_bool(x) {
      Ok(result) -> decode.success(result)
      Error(_) -> decode.failure(True, "bool")
    }
  }

use _is_admin <- decode.field(12, sqlite_bool_mapper)
expects a decode.Decoder(a) but got a closure: fn(dynamic.Dynamic) -> decode.Decoder(Bool)

autumn kite
#

Oh yeah you can use zero.dynamic and zero.then

violet root
#

I am a bit... disppointed at me... because i thought you had the same problems and I could just run my queries in raw sql against postgres to get some good decoders :D... but was too lazy to do it, and forgot you had snapshots ❤️

violet root
#

Turns out decode.field(n, decoder) returns Nil if n is out of bounds...

#

I'd expect it to crash or log a warning I suppose

pallid kestrel
#

It never crashes or logs by design

violet root
#

how could I enable a logger though? for the execurted queries and the raw erlang terms? no way?

#

instead of returning Nil if someething has not been found, could it return an error about an index out of bounds or a map key not existing?

pallid kestrel
#

No, it is intentional that it returns a null value

#

It greatly reduces boilerplate

violet root
#

except for my counting skill issue not being caught otherwise the decode lib is just sweet while I toy with it atm

#

❤️ thank you!

potent echo
#

Updated gleojson with zero decoders. Looks much cleaner. Though still need to expose at least something like:

import decode/zero.{type Decoder}
import gleam/dynamic.{type Dynamic}

pub fn to_dynamic_decoder(zero_decoder: Decoder(a)) {
  fn(dynamic_value: Dynamic) { zero.run(dynamic_value, zero_decoder) }
}

to simplify passing zero decoders to existing API that accept dynamic.Decoder, like json.decode.

autumn kite
#

You can do that pretty easily with function captures!

json.parse(data, zero.run(_, decoder))
solemn plaza
#

I'm having a go at converting a small project I had using toy over to decode/zero. It's going quite nicely!

I'm a little stuck at one function I had that decoded a time string into a time value. birl.parse obviously returns a result, but I'm not sure the best way to integrate this into a decoder cleanly.

I was previously using toy.try_map(), and constructing an error that "looked like" a decoder error (albeit with no 'path' information). But now decoders that produce results don't really integrate cleanly with other decoders.

I'm sure I'm missing something obvious, my "Gleam brain" for these type transformations is not well developed.

// The old toy decoder produced a fake 'decode' error if time parsing failed.
// This made it compatible with all the rest of the decoders
fn time_decoder_toy() {
  toy.string
  |> toy.try_map(time.now(), fn(val) {
    time.parse(val)
    |> result.map_error(fn(_) {
      [toy.ToyError(toy.InvalidType("DateTime", val), [])]
  })
}

// The new decoder
fn time_decoder() -> Decoder(Result(birl.Time, Nil)) {
  zero.string
  |> zero.map(birl.parse)
}

fn using_time_decoder(data) {
    let my_decoder = zero.list({
      use name <- zero.field("name", decode.string)
      use time_result <- zero.field("time", time_decoder())

      // At the usage site I need to do this case expression. Not ideal.
      case time_result {
        Ok(time) -> zero.success(Something(name:, time:))
        Error(_) -> zero.failure(Something(name: "", time: time.unix_epoch))
      }
    })

    zero.run(data, my_decoder)
}
muted hazel
# 
fn time_decoder() -> Decoder(Result(birl.Time, Nil)) {
  use timestring <- zero.then(zero.string)

  case birl.parse(timestring) {
    Ok(time) -> zero.success(time)
    Error(_) -> zero.failure(time.unix_epoch, "DateTime")
  }
}

is what you want

# 
fn using_time_decoder(dataa) {
  let my_decoder = zero.list({
    use name <- zero.field("name", decode.string)
    use time <- zero.field("time", time_decoder())

    Something(name:, time:)
  })
}
solemn plaza
#

Ah, lovely! It hadn't occurred to me to use the success/failure functions at that level. This is starting to look nice now. Thanks for the amazingly speedy help!

odd python
#

What is the benefit of making Decoder an opaque type?

While using dynamic https://hexdocs.pm/gleam_stdlib/gleam/dynamic.html#Decoder
I found it very useful that a Decoder is just an alias to fn(Dynamic) -> Result(t, DecodeErrors)
This helped me understand what a decoder is.

Sometimes I just have to do something outside of the beaten path, and writing a decoder by following this function signature is straightforward.

For me using an opaque type makes it harder to understand what a decoder is, and how to write one.

gleam/dynamic · gleam_stdlib · v0.41.0
mental sentinel
#

in particular fn run converts it to that type with Result

#

so you can create a fn(Dynamic) -> Result(t, DecodeErrors) with run(_, opaque decoder here)