Streaming data while deserializing with serde, Possible ?

[slow anchor]

now, what if i got an Option<Vec<u8>>, how'd that work

After many failed attempts messing with serde, i still can't make an Iterator<Item=u8> from a [1, 2, 3], or at least call a passed FnMut(u8) for each u8, do i really have to make my own parser ?

I sure can have a Visitor over the sequencial u8s but i can't seem to be able to call my FnMut(u8).

[sand gale]

So you want to deserialize from a byte stream?

[halcyon raven]

why can't you call the function?

[slow anchor]

i can't get it to be part of the visitor

[slow anchor]

well i got a streamed JSON from a /dev port

and i want to avoid Vec<u8> allocations

[halcyon raven]

just make it a field?

[slow anchor]

field where

[halcyon raven]

in the visitor

[slow anchor]

i'll see if i can feed it to serde somehoe

[halcyon raven]

what do you have implemented until now?

[slow anchor]

ill show in a sec, just lemme get home

[sand gale]

If its a stream you can deseriaize it directly off the stream. Serde has functions for that

[halcyon raven]

it does?

from Read yes

if that's a "stream"

[sand gale]

yea

https://docs.serde.rs/serde_json/de/fn.from_reader.html

[halcyon raven]

I forgot that Stream is now AsyncIterator

[slow anchor]

thats not what im asking

and im already doing that

i'm asking if i can avoid collecting into a Vec but instead get the T's individually and call some function on them

i'm already using FromReader on the tty port

[halcyon raven]

https://serde.rs/stream-array.html

that example shows almost what you want

[slow anchor]

still not quite

how could i call from there my FnMut ?

[halcyon raven]

put it in the MaxVisitor struct

[sand gale]

That is cool. Can you parameterize the visitor though?

[halcyon raven]

what do you mean?

[sand gale]

Is there any way to pass in the mutable thing in your FnMut?

[halcyon raven]

"thing"? lol

[sand gale]

Yea, I assume they want to e.g. send a network message or modify program state for each item in the sequence

[halcyon raven]

you have an FnMut, right?

you can capture stuff

you can also hold other arguments in the visitor

[sand gale]

Oh. How becuase deserialize_max takes one argument, the derserializer

[halcyon raven]

well, it won't work trivially with deserialize_with because of that

you'd just call the deserialize_max function with other arguments

[sand gale]

You cant

serde calls that

[halcyon raven]

you can call it yourself

[sand gale]

Well that is pretty cool but I can;t see how

[slow anchor]

yeah thats what im struggling to see

[halcyon raven]

?play ```rs
use serde::de::{SeqAccess, Visitor};
use serde::{Deserialize, Deserializer};

use std::fmt;
use std::marker::PhantomData;

fn deserialize_for_each<'de, T, D>(deserializer: D, f: impl FnMut(T)) -> Result<(), D::Error>
where
T: Deserialize<'de>,
D: Deserializer<'de>,
{
struct ForEachVisitor<T, F>(F, PhantomData<fn(T)>);

impl<'de, T, F> Visitor<'de> for ForEachVisitor<T, F>
where
    T: Deserialize<'de>,
    F: FnMut(T),
{
    type Value = ();

    fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
        formatter.write_str("a nonempty sequence of numbers")
    }

    fn visit_seq<S>(mut self, mut seq: S) -> Result<(), S::Error>
    where
        S: SeqAccess<'de>,
    {
        while let Some(value) = seq.next_element()? {
            (self.0)(value)
        }

        Ok(())
    }
}

// Create the visitor and ask the deserializer to drive it. The
// deserializer will call visitor.visit_seq() if a seq is present in
// the input data.
let visitor = ForEachVisitor(f, PhantomData);
deserializer.deserialize_seq(visitor)

}

fn main() {
let j = r#"[
256,
100,
384,
314,
271
]
"#;

deserialize_for_each(&mut serde_json::Deserializer::from_str(j), |i: i32| {
    println!("visit {i}")
})
.unwrap();

}

[dense saffron]

visit 256
visit 100
visit 384
visit 314
visit 271

[halcyon raven]

adapted mostly from that example

[sand gale]

Very nice!

Didn't know serde was so composable

[halcyon raven]

yeah, you can do cool stuff with Deserializer, but it's not obvious

&mut serde_json::Deserializer::from_str(j) itself is weird

[slow anchor]

i guess i was close but not close enough

i'll retry ig

[halcyon raven]

you don't usually put bounds on the struct itself

only on impls

what you had looks usable

you just need to use that visitor somehow

[slow anchor]

i see