HRTB, closure, requires that it is borrowed for 'static

[hoary scroll]

Hello there, still trying to understand HRTB and especially structs with references and closures. I have the following example: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=1ad2d4175fc1f96d71c9e51b048c179b which uses a function with signature

fn with_pass(&self, f: impl for<'inner> Fn(&mut Pass<'inner>))

and leads finally to an error:

argument requires that `p` is borrowed for `'static`

which I cannot make sense. Maybe I'm trying something impossible here, but I would like to understand, what is happening... is this a self-referential closure struct and a misleading error message? Or something else?

[rocky tusk]

Well changing the signature on start to not have an explicit lifetime fixes the compile error but I'm not sure why

?play

struct Ctxt {}
struct Pipe {}
struct Pass<'inner> {
    inner: &'inner String,
}

impl Ctxt {
    fn with_pass(&self, f: impl for<'inner> Fn(&mut Pass<'inner>)) {
        let inner = String::from("inner");
        let mut pass = Pass { inner: &inner };

        f(&mut pass)
    }
}

impl Pipe {
    fn start(&self, pass: &mut Pass<'_>) {
        println!("rendering:{}", pass.inner);
    }
}

fn main() {

    let c = Ctxt {};
    let p = Pipe {};

    c.with_pass(|pass| {
        p.start(pass)
    })

}

[high portal]

rendering:inner```

[rocky tusk]

Since I thought that

fn start(&self, pass: &mut Pass<'_>)

Is the same as

fn start<'pass>(&'pass self, pass: &mut Pass<'pass>)

Since I thought the lifetime elision rules would cause the '_ to be inferred to the same as &self.....but not sure, I guess not because it's an input lifetime and I think it only binds output lifetimes to &self

Maybe the elided one actually expands to more like

fn start<'1, '2, '3>(&'1 self, pass: &'2 mut Pass<'3>)

Whereas when you use the explicit lifetime you're tying the lifetime of &self to the lifetime inside Pass

And since HRTBs allow any lifetime to be chosen for your 'inner (including 'static) - I think this is why it's ending up assuming it can be Pass<'static>.....which then because you're tying that to &self means that self has to be 'static as well

[atomic herald]

With

fn start<'pass>(&'pass self, pass: &mut Pass<'pass>)
```it's theoretically possible for this function to store `self` inside `pass` (Imagine `pass` had a field which is a `Vec<&'pass dyn Pipe>` or something)

That means that to call p.start(pass), with a pass that lives for 'x, you need p to also live for 'x

Now, what is 'x here? The answer is that you don't know: it's a for<'a> lifetime, so you have to assume it could be any possible lifetime. In practice, this ends up meaning you have to assume it could be the least permissive lifetime of them all, which is usually the shortest or the longest possible lifetime.

That also means that the closure could be called with a &mut Pass<'static>, which implies that, since pass could store self, then you also need &'static self to allow it

Removing the named lifetime causes self and pass to not have compatible lifetimes anymore, and thus makes it impossible for start to store self inside pass

The expansion is in fact

fn start<'a, 'b, 'c>(&'a self, pass: &'b mut Pass<'c>)

[atomic herald]

The elision rules cause return type lifetimes to be derived from the lifetime on the reference to self. Argument lifetimes are still elided as usual: Each is a different lifetime

@hoary scroll

[hoary scroll]

Thanks!
Still trying to wrap my head around that for<'pass>, because it makes absolute sense to say: well, you can put anything here, so even 'static, which means, whenever we let that lifetime be caller chosen, assume it is static?

In other words, a callee chosen (for<'pass>) used in a caller chosen (fn start<'pass>) will be set to 'static? In your last example, @atomic herald , can 'c be anything else then 'static?

When used as above, i.e. from a closue with impl for<'pass> Fn(&mut Pass<'pass>)?

[atomic herald]

For practical purposes, when you see a for<'a> function and you're the one writing it, assume you'll have to satisfy the most restrictive possible lifetime. In this case longer = more restrictive, so 'static is what you end up dealing with.

In other words, a callee chosen (for<'pass>) used in a caller chosen (fn start<'pass>) will be set to 'static?
No, but I can see where you got the idea.

A callee-chosen lifetime can be set to anything the callee chooses. Now, since the entire point of for<'a> closures is that they must work for all possible choices, you end up having to tolerate not just the choices the callee makes, but also the choices they could make.
Usually, this manifests in thinking of 'a as the shortest possible lifetime (think 'infinitesimal, if you will), but this isn't a hard-and-fast rule, it's just that it's more common the restrictive lifetime to be shorter. Here, because of &mut's invariance, you find yourself having the most restrictive lifetime be the longest.

can 'c be anything else then 'static
'c is whatever the caller of start decides that it is 😄

[hoary scroll]

I need a bit to process. ; )

Okay, so I see my mistake here, in f: impl for<'pass> Fn(&mut Pass<'pass>), the f works with any lifetime. So whatever 'c is, this is fine, since f works with it.

An the other hand, having &'c self, it is still caller chosen within fn start<'b, 'c>(&'c self, &'b mut Pass<'c>), but in calling

c.with_pass(|pass| {
   pipe.start(pass) 
}

we have the restriction that the closure |pass| pipe.start(pass) needs to work with any lifetime, this is a specific restriction to that very closure, fulfilling the bounds of f. And now of course, also with 'static?

Is the reason behind the error message, that the compiler just checks for 'static and sees it fails?

Brb, short walk with the dogs. ; )

[atomic herald]

I think you understood it correctly, yes

[hoary scroll]

Thanks a lot! : )

[white pecan]

A "reduction" to justify the error message:

f: impl for<'inner> Fn(&mut Pass<'inner>)
// is the same as the combination of concrete `impl`s,
// for each possible "value" of `'inner`.
// That is, say `'a, 'b, 'c, …` represents the set of all possible lifetimes,
// then we have:
f: impl Fn(&mut Pass<'a>)
      + Fn(&mut Pass<'b>)
      + Fn(&mut Pass<'c>)
      + …

and this enumeration does include:

+ …
+ Fn(&mut Pass<'static>)
+ …

since it's just as valid of a lifetime choice as any other.

Which is where 'static came from.

So we can reduce your code to:

impl Ctxt {
    /// This is not equivalent to your signature, but it is
    /// *more lenient* than your signature, so if this one
    /// does not work, a fortiori the `impl for<'any> Fn(&mut Pass<'any>)` 
    /// won't work either.
    fn with_pass(&self, f: impl Fn(&mut Pass<'static>)) {
        let inner = String::from("inner");
        let mut pass = Pass { inner: &inner };

        f(&mut pass)
    }
}

impl Pipe {
    fn start<'pass>(&'pass self, pass: &mut Pass<'pass>) {
        println!("rendering:{}", pass.inner);
    }
}

let p = Pipe {};
c.with_pass(|pass /*: &mut Pass<'static> */| {
    p.start(pass) // -> fn start<'pass = 'static>(&'static self, …) => Error
})

By the way, sometimes the call-site does need that kind of "repeated 'pass lifetime" situation, and in that case it's up to your for<>-featuring with_pass signature to be more lenient, by avoiding big lifetimes in that infinite enumeration / by using an infinite enumeration of lifetimes that can be smaller than 'p (let's say that's the name of the lifetime of your p variable).
See https://users.rust-lang.org/t/argument-requires-that-is-borrowed-for-static/66503/2?u=yandros for more info

[hoary scroll]

I was about to ask, before writing my notes, if I'm right about from where the 'static arises. But there was already this nice answer from you, @white pecan. I might consider the 'upper_bound "hack" from your forum post, but it also explains a lot. Never thought about the idea that an upper bound can solve this issue. But, yes, of course.