#Deserialize custom map type with different value types

23 messages · Page 1 of 1 (latest)

indigo chasm
#

I am looking to deserialize a nested JSON to a http crate HeaderMap structure with https://serde.rs/field-attrs.html#deserialize_with. How would I handle cases where the values are not strings but nested while still having access to a mutable variable to insert my items (see below)? Alternatively, is there a way to skip non-string entries.

{
    "method": "GET",
    "headers": {
        "host": "test.whatever.app",
        "x-some-key": [
            "0.0.0.0",
            "0.0.0.1"
        ]
}

fn deserialize_headers<'de, D>(deserializer: D) -> Result<HeaderMap<HeaderValue>, D::Error>
where
    D: Deserializer<'de>,
{
    struct HeaderVisitor;

    impl<'de> Visitor<'de> for HeaderVisitor {
        type Value = HeaderMap<HeaderValue>;

        fn expecting(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
            write!(formatter, "a HeaderMap<HeaderValue>")
        }

        fn visit_map<A>(self, mut map: A) -> Result<Self::Value, A::Error>
        where
            A: MapAccess<'de>,
        {
            let mut headers = http::HeaderMap::new();
            // How to handle the case where the header is not a string but a sequence?
            while let Some((key, value)) = map.next_entry::<Cow<'_, str>, Cow<'_, str>>()? {
                let header_name = key
                    .parse::<http::header::HeaderName>()
                    .map_err(A::Error::custom)?;

                let header_value = HeaderValue::from_str(&value).map_err(A::Error::custom)?;

                headers.append(header_name, header_value);
            }
            Ok(headers)
        }
    }

    deserializer.deserialize_map(HeaderVisitor)
}
Field attributes · Serde
regal slate
#

Is the header name enough to determine if it is a string or a list? In that case you can use next_key/next_value instead of next_entry and always choose a matching value type.

If you have a self describing data format, like the JSON example, you can deserialize into a enum of all the correct types. In this case with two variants, one being a String and one being Vec<String>. More variants can be added too. Then you implement Deserialize via deserialize_any and provide both visit_str and visit_seq. An enum with #[serde(untagged)] works too, but is quite a bit slower than hand written Deserialize.

Using untagged enums you can skip non-string entries by creating a String and a IgnoredAny variant.

Deserializing a Cow<'_, str> like this always produces the owned variant which allocates, and can be simplified to just String.

indigo chasm
#

Unfortunately the header name is not enough as any header could possibly be passed with an array value

#

I will try your suggestion and follow up. Thanks for now!

indigo chasm
#

@regal slate I took another look and I am not sure how I would achieve this step: you can deserialize into a enum of all the correct type. I figured I could just then deserialize it directly and tried the following. Using map.next_value::<Vec<String>>() seems to crash the program though. Any leads?

fn deserialize_headers<'de, D>(deserializer: D) -> Result<HeaderMap<HeaderValue>, D::Error>
where
    D: Deserializer<'de>,
{
    struct HeaderVisitor;

    impl<'de> Visitor<'de> for HeaderVisitor {
        type Value = HeaderMap<HeaderValue>;

        fn expecting(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
            write!(formatter, "a HeaderMap<HeaderValue>")
        }

        fn visit_map<A>(self, mut map: A) -> Result<Self::Value, A::Error>
        where
            A: MapAccess<'de>,
        {
            let mut headers = http::HeaderMap::new();

            while let Some(key) = map.next_key::<String>()? {
                let header_name = key
                    .parse::<http::header::HeaderName>()
                    .map_err(A::Error::custom)?;

                if let Ok(string_value) = map.next_value::<String>() {
                    let header_value =
                        HeaderValue::from_str(&string_value).map_err(A::Error::custom)?;
                    headers.insert(header_name, header_value);
                } else {
                    // Crashes here when handling Vec<String> values
                    let vec_value = map.next_value::<Vec<String>>()?;

                    let header_value =
                        HeaderValue::from_str(&vec_value.join(",")).map_err(A::Error::custom)?;

                    headers.insert(header_name, header_value);
                };
            }

            Ok(headers)
        }
    }

    deserializer.deserialize_map(HeaderVisitor)
}
regal slate
#

next_key and next_value always need to alternate, that is why it crashes

#

I meant to create an enum like

enum Value {
    Single(String),
    Multiple(Vec<String>),
}

and then use map.next_value::<Value>()?

#

A hand written Deserialize is easy enough and performant, but you can also use untagged on that.

indigo chasm
#

Alternate as in 1:1 calls between next_key and next_value?

#

I think I can't hand write Deserialize as the HeaderMap struct is exported from another crate?

regal slate
#

The Deserialize is for the enum I just posted

indigo chasm
#

How do I get from my HeaderMap<HeaderValue> to HeaderValue though?

#

Oh I see I'd not import it and do the conversion later

regal slate
#

sorry, should not have called it that. I didn't see you used that name already

indigo chasm
#

Yeah that's also exported from the http crate

regal slate
#

You use it where you currently deserialize a String

indigo chasm
#

Hm that doesn't seem to do the trick

# 
fn deserialize_headers<'de, D>(
    deserializer: D,
) -> Result<HashMap<String, HeaderValueEnum>, D::Error>
where
    D: Deserializer<'de>,
{
    struct HeaderVisitor;

    impl<'de> Visitor<'de> for HeaderVisitor {
        type Value = HashMap<String, HeaderValueEnum>;

        fn expecting(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
            write!(formatter, "a HashMap<HeaderValueEnum>")
        }

        fn visit_map<A>(self, mut map: A) -> Result<Self::Value, A::Error>
        where
            A: MapAccess<'de>,
        {
            let mut headers = HashMap::new();

            while let Some((key, value)) = map.next_entry::<String, HeaderValueEnum>()? {
                dbg!(&key, &value);
                headers.insert(key, value);
            }

            Ok(headers)
        }
    }

    deserializer.deserialize_map(HeaderVisitor)
}
#

expected Single or Multiple

indigo chasm
#

It seems #[serde(flatten)] also doesn't work for this usecase. This means I'm back to square one with the alternate call issue 🌧️

regal slate
indigo chasm
#

Nope

#

I just found a dirty solution (funny enough you on stack overflow) by obtaining the raw serde json value and then trying to parse