Dealing with a somewhat cursed input data format

[jovial wigeon]

Hello!
I've been dealing with this problem for about 2 days now, and the code solution I've come up with looks pretty awful and hacky. So, I was wondering if there was a better way to do this.

The Problem:
I need to accept some data that's being POST-ed to me and do stuff based on it, however the problem is how the data is arriving to me.

The actual data model is supposed to look like this: https://paste.rs/bvD.json
It's got JSON strings nested inside JSON strings.

When this gets POST-ed to me, Axum gets the data in the form of a byte string that looks like this: https://paste.rs/Tpj

What I've Tried So Far:
This is a small snippet of what my current code looks like: https://paste.rs/nhN.rs

It kinda works, but even I can tell that it's probably not a good way to go about this.

So, I was wondering:

1. Is there a better way for me to wrangle that input data?
2. If no, is there a better way for me to structure my code?

[swift slate]

You may be able to flatten this out into a rust type with a moderately low amount of effort. Lemme try some stuff.

[jovial wigeon]

I have these 4 structs going at the moment, and I'm trying to get the data to sit in them:

#[derive(Serialize, Deserialize, Debug)]
pub struct XqueueSubmission {
    xqueue_body: XqueueBody,
    xqueue_file: String,
}
#[derive(Serialize, Deserialize, Debug)]
pub struct XqueueBody {
    student_info: StudentInfo,
    student_response: String,
    grader_payload: GraderPayload,
}

#[derive(Serialize, Deserialize, Debug)]
pub struct StudentInfo {
    anonymous_student_id: String,
    submission_time: String,
    random_seed: u32,
}

#[derive(Serialize, Deserialize, Debug)]
pub struct GraderPayload {
    program_name: String,
    program_output: String,
    program_language: String,
}

[hoary sluice]

Seems like a perfect situation for a custom deserializer imo

[jovial wigeon]

I see

Is there a link you could point me to? I'll try reading and coming up with something

[hoary sluice]

https://serde.rs/impl-deserialize.html

[jovial wigeon]

Gotcha, thank you, I'll take a look

[hoary sluice]

I've never had a situation where it's necessary for myself, I just believe that's your cleanest solution until Seaish produces something

[jovial wigeon]

Alrighty!

Lemme mess around with this and see how far I can get

[swift slate]

That's what I was doing too

[hoary sluice]

Nevermind, there's an attribute after googling

[jovial wigeon]

An attribute?

[hoary sluice]

https://github.com/serde-rs/serde/issues/994

[jovial wigeon]

Ooooh

Daaang, that looks clean o.o

So, then my structs would turn into something like this, right?

#[derive(Serialize, Deserialize, Debug)]
pub struct XqueueSubmission {
    #[serde(with = "serde_json::nested")]
    xqueue_body: XqueueBody,
    xqueue_file: String,
}
#[derive(Serialize, Deserialize, Debug)]
pub struct XqueueBody {
    #[serde(with = "serde_json::nested")]
    student_info: StudentInfo,
    student_response: String,
    #[serde(with = "serde_json::nested")]
    grader_payload: GraderPayload,
}

#[derive(Serialize, Deserialize, Debug)]
pub struct StudentInfo {
    anonymous_student_id: String,
    submission_time: String,
    random_seed: u32,
}

#[derive(Serialize, Deserialize, Debug)]
pub struct GraderPayload {
    program_name: String,
    program_output: String,
    program_language: String,
}

[hoary sluice]

I'm not very sure if it was ever implemented into the crate as much as you can just copy that guy's example

Though I guess you can try

[jovial wigeon]

Yeah, it says it can't resolve it.
I'm checking to see if it's under a feature I need to enable.

Otherwise I'll fall back to the good ol' copy

[hoary sluice]

Following the issues, it fell through among many other features due to "helper crates are now possible, closing main issue"

[jovial wigeon]

Right, just saw that

Thanks for the help!!

[hoary sluice]

No problem!

[jovial wigeon]

I got it to work!!! Thank you once again!! :D

My sanity has been saved! 😄

[swift slate]

Cool! I was trying to get it to work without taking ownership but it didn't wanna go. Copying the module from the github issue worked though.

[jovial wigeon]

I'm reading the serde data model document to better understand the solution now

But this is amazing

[jovial wigeon]

Sorry if I'm asking too much here, but I just wanted to know if I'm understanding the solution right.

So, this is what's doing the deserialization right?

    pub fn deserialize<'de, T, D>(deserializer: D) -> Result<T, D::Error>
        where T: DeserializeOwned,
              D: Deserializer<'de>
    {
        let j = String::deserialize(deserializer)?;
        serde_json::from_str(&j).map_err(de::Error::custom)
    }

If I'm understanding this correctly, all it's doing is that it's accepting the value as a String and then generating a serde Value from it?

[swift slate]

Not exactly. It's deserializing it as serde's builtin String deserializer, then serde_json is deserializing that string as T.

[jovial wigeon]

T being whatever type I put in the struct?

[swift slate]

Yeah

[jovial wigeon]

Oh, okay, I think it just clicked in my head

Thanks again 😄

So it's doing D -> String -> T?

[swift slate]

I think it's more like, consuming one String from inside D

[jovial wigeon]

Ahhhhh

Okay okay