Environment variables and lifetimes

[gritty dust]

Hey, I've got a bit of a weird case & I'm having a bit of a struggle with lifetimes (nothing new...). I know exactly why it isn't working but I can't see a solution.

The end goal is a struct method that given a key, returns an enum variant that represents the value type. The value should either come from a HashMap<&'a str, Value<'a>> that comes from a struct field, or the system environment variables if present.

The problem is that the Value::String variant has to take a &'a str, and cannot take an owned string due to limitations elsewhere. env::vars() meanwhile provides key/value pairs of owned strings, which then need to be borrowed and of course don't live long enough.

Some things I've tried:

1. Pre-populating the inputs with environment variables. This doesn't work because neither k nor v live long enough.

let inputs = env::vars().map(|(k, v)| (k.as_str(), Value::String(v.as_str()))).collect::<Inputs>();

2. Getting the env var inputs at the same time as other inputs. This doesn't work because the env var reference is held and subsequently dropped in the function.

fn get_input(&self, key: &'a str) -> Result<Value<'a>> {
    if key.starts_with("$env_") {
        let env_name = key.replace("$env_", "");
        let var = env::var(env_name); // cannot return value referencing local variable

        if let Ok(var) = var {
            return Ok(Value::String(&var));
        }
    }

    // snip
}

Basically, what pattern or approach or special pointer type or whatever should I be using to make sure the env vars last for 'a?

I'm happy to share more code if required. Cheers!

[crude reef]

you can't; you should be using String not &str

String is that special pointer type you need

[gritty dust]

Right, if that is the case and there's no way around it, here's my problem at the other end of the tunnel (and why I'm using &'a str rather than String:

All of this feeds into a Serde deserializer, which needs to be able to take a &str:

fn deserialize_str<V>(self, visitor: V) -> std::result::Result<V::Value, Self::Error>
    where
        V: Visitor<'de>,
    {
        let value = self.value.take().unwrap();
        match value {
            Value::String(val) => visitor.visit_borrowed_str(val),
           _ => unreachable!(),
        }
    }

If I change Value::String to take a String, is there any way I can then pass a &str to visit_borrowed_str that lives long enough? It's got to live for the deserializer struct's lifetime.

[crude reef]

what error do you get if you try?

[cunning quartz]

I think you need to call visit_string instead

visit_borrowed_str() is for the specific case where the deserializer can provide output that lives for 'de

but it's entirely normal for that not to be the case (depending on the deserialization format)

[gritty dust]

I'd have to refactor quite a bit rn to get the exact error, but it's another lifetime error because val gets dropped after the match arm so you can't keep a borrow, which again makes sense...

[gritty dust]

Hm okay, does that work if you're deserializing to a field that takes a &str? Like?

struct Foo<'a> {
  bar: &'a str
}

[cunning quartz]

no, that's exactly what it prohibits

somebody has to own the characters

and that can be "the input data which outlives the deserialized structure" or it can be "the deserialized structure", but it cannot be neither one

[gritty dust]

Right okay, makes sense

It sounds like the best path forward here may just be to throw some kind of UnsupportedError in the case of deserializing to a &str then

cheers for the help both

[cunning quartz]

I'm not very familiar with serde workings, but I think you don't need to make any kind of error

you just call the visitor with visit_string(), and if the type can't accept that, that's its problem

?play

#[derive(serde::Deserialize)]
struct Foo<'a> {
    bar: &'a str,
}

fn main() {
    let json = r#" {"bar": "HELLO \u2022 WORLD"} "#;
    let foo: Foo<'_> = serde_json::from_str(json).unwrap();
}

[lyric lotus]

     Running `target/debug/playground`
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("invalid type: string \"HELLO • WORLD\", expected a borrowed string", line: 1, column: 29)', src/main.rs:8:51
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

[cunning quartz]

↑ example of this failure in JSON

and if you look at the serde_json source code, you see:
https://docs.rs/serde_json/latest/src/serde_json/de.rs.html

    fn deserialize_str<V>(self, visitor: V) -> Result<V::Value>
    where
        V: de::Visitor<'de>,
    {
        ...
        let value = match peek {
            b'"' => {
                self.eat_char();
                self.scratch.clear();
                match tri!(self.read.parse_str(&mut self.scratch)) {
                    Reference::Borrowed(s) => visitor.visit_borrowed_str(s),
                    Reference::Copied(s) => visitor.visit_str(s),
                }
            }
            _ => Err(self.peek_invalid_type(&visitor)),
        };
        ...
    }

it's just calling whichever of visit_str() or visit_borrowed_str() is possible

not producing an error on its own choice, unless the input JSON contains a non-string value

[gritty dust]

ahhh right okay

in that case actually, since my current Value::String(&str) implementation works apart from these darn env vars

i could probably just add a second Value::OwnedString(String) variant and handle both in deserialize_str, effectively doing what serde_json is doing

i think this has given me enough to think about it i should be able to come up with something, thanks :)