Weird lifetime error with impl Trait, closures, and mutable references

[daring elk]

Sorry for the vague description, here's a playground link with a minimized version of the error: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=aefe04553d8c34011e1aae7c4bd56830

There's a couple things that I think are weird:

1. The need for drop on line 48. This behaves like p takes ownership of &mut s, even though it doesn't. I'm guessing this is a limitation on what rustc can infer about lifetimes. This isn' a big deal, drop is fine as a workaround.
2. It still fails to compile, and the error message implies that p lives until the end of the function, despite the explicit drop before the end of the function.

This smells like a compiler bug, but I could just be missing something about how impl Trait interacts with lifetimes.

[autumn hedge]

So, the need to drop is indeed extremely weird; I think I've sometimes seen that, I'll need to investigate a bit further

Regarding why without drop it fails it's actually legitimate: an impl Traits… is:

• not allowed to be used beyond any lifetime appearing in Traits… (e.g., here, that of the 'borrow of the String in S = &'borrow mut String)
• considered to have significant drop glue, so that the usual NLL nicety does not apply

You're probably surprised by the second bullet, since it's indeed something not that frequent to run into, not that well documented, and rather subtle

?play

fn id(_: impl Sized) -> impl Sized {}

fn main() {
    let _r;
    let local = ();
    _r = id(&local); /*
drop(local); drop(_r: impl 'local + Sized); */ }

[tidal spear]

error[E0597]: `local` does not live long enough
 --> src/main.rs:6:13
  |
5 |     let local = ();
  |         ----- binding `local` declared here
6 |     _r = id(&local); /*
  |             ^^^^^^ borrowed value does not live long enough
7 | drop(local); drop(_r: impl 'local + Sized); */ }
  |                                                -
  |                                                |
  |                                                `local` dropped here while still borrowed
  |                                                borrow might be used here, when `_r` is dropped and runs the destructor for type `impl Sized`
  |
  = note: values in a scope are dropped in the opposite order they are defined

For more information about this error, try `rustc --explain E0597`.```

[autumn hedge]

Ok, got a bit more advanced with my investigation

?play ```rs
fn id(_: impl Sized) -> impl Sized {}

fn main()
{
let r;
let mut local = ();
r = id(&mut local);
drop(r); // <- this one is fine
}

[autumn hedge]

?play ```rs
fn id(_: impl Sized) -> impl Sized {}

fn main()
{
let r;
let mut local = String::new(); // <- but if local has drop glue of its own
r = id(&mut local);
drop(r); // <- then this does not suffice
}

[tidal spear]

error[E0597]: `local` does not live long enough
 --> src/main.rs:7:12
  |
6 |     let mut local = String::new(); // <- but if `local` has drop glue of its own
  |         --------- binding `local` declared here
7 |     r = id(&mut local);
  |            ^^^^^^^^^^ borrowed value does not live long enough
8 |     drop(r); // <- then this does not suffice
9 | }
  | -
  | |
  | `local` dropped here while still borrowed
  | borrow might be used here, when `r` is dropped and runs the destructor for type `impl Sized`
  |
  = note: values in a scope are dropped in the opposite order they are defined

For more information about this error, try `rustc --explain E0597`.```

[autumn hedge]

(you'd imagine I got to the latter snippet from the former, but I actually started from your snipet and started hatching at it until only this remained 😆)

[daring elk]

Okay I think I get it

[daring elk]

Something that's throwing me off is the let r; ...; r = .. Becaus I don't have that (verbatim) in my snippet. But, with the first exaplanation, the p.parse after let s = ..., is equivalent

[autumn hedge]

Yes it is. To be better convinced why, notice how:

let r;
let s = …;
r = …&mut s…;

is not that different from:

let mut r = None;
let s = …;
r.replace(…&mut s);

is not that different from:

let r = some_fn_with_a_fixed_lt_parameter_arg…;
let s = …;
r.call_that_fn(&mut s);

So I do think there is a compiler bug w.r.t. drop-tracking

[daring elk]

Got it, yea that's clearer

[autumn hedge]

(it is known that drop(x) does not handle all of the cases for the compiler to eliminate x's drop glue)

So what I think is happening is that the compiler does successfully eliminate the drop glue of x, but not the scope of its own type:

{
    let r; // if this has drop glue, then its type is required to be valid --+
    …                                                                     // |
} // <------------------- until here ----------------------------------------+

no matter what happens in …, even if drop(r) happens.

so, when the type of r captures S = &'borrow mut T, then 'borrow needs to span beyond that scope.
And when type-checking that a &mut s: &mut T against such a constraint:

• when T = String has drop glue, it considers that the s is "used" when it goes out of scope (which is expected), and that's when it notices that doing so conflicts with the requirement on its 'borrow. This all does make quite a bit of sense.
• the surprising thing is that when T = () does not have drop glue, some NLL magical heuristic seems to be ok with 'borrow spanning beyond the scope of the variable (it's very surprising to say the least; I'd even be worried that it be some unsoundness in the language)

So, the way I see it:

• the "if this has drop glue then its type is required to be valid until here no matter what happens in … even if drop happens" is a bug/limitation of the compiler which is not capable of understanding scopes
• assuming the previous to be ok or at least accepted, the fact that when drop(r) does occur, if s has no drop glue then NLL accepts the snippet is a "counter-bug" which, if the previous point was right to deny it, then it would be unsound ⚠️; but since the previous point is silly to begin with, this counter-bug puts things back into place for this specific case

?godbolt flags="-Zpolonius" ```rs
fn id(_: impl Sized) -> impl Sized {}

fn main()
{
let r;
let mut local = String::new();
r = id(&mut local);
drop(r);
}

[tidal spear]

error[E0597]: `local` does not live long enough
 --> <source>:7:12
  |
6 |     let mut local = String::new();
  |         --------- binding `local` declared here
7 |     r = id(&mut local);
  |            ^^^^^^^^^^ borrowed value does not live long enough
8 |     drop(r);
9 | }
  | -
  | |
  | `local` dropped here while still borrowed
  | borrow might be used here, when `r` is dropped and runs the destructor for type `impl Sized`
  |
  = note: values in a scope are dropped in the opposite order they are defined

error: aborting due to previous error

For more information about this error, try `rustc --explain E0597`.

<Compilation failed>

[daring elk]

but since the previous point is silly to begin with,
Which point are you referring to?

Are you saying if "E0597 happens with drop glue is requried for soundness" then "no E0597 when no drop glue is unsound"?

[autumn hedge]

I meant

"if this has drop glue then its type is required to be valid until here no matter what happens in … even if drop happens"
I don't know how to name this. Let's call it "rust does not always understand the implication of drop"

So let me rephrase the second bullet accordingly:

• Assuming "not undertanding drop correctly / refuting its semantics" to be ok or at least accepted, the fact that, when drop(r) does occur if s has no drop glue then NLL accepts the snippet, is a "counter-bug" which, if "not understanding drop" was [somehow] a legimate thing to do for soundness, then it this NLL heuristic would be unsound ⚠️; but since "not understanding drop" is silly to begin with, this counter-bug puts things back into place for this specific case

[daring elk]

Okay, yea that makes

[autumn hedge]

The only hypothesis I had to justify "not understanding drop" so as to deny the non-NLL case of s: String = impl DropGlue was the possibility of unwinds in the middle, but looking at the control-flow graph Rust does correctly no see any unwind path possible between:

• constructing DropGlue
• dropping it
• with some hypothetical unwind and drop of s in between

As we can see, the only possible unwind path it sees is the drop(r) itself unwinding, but this is fine, because at that point r does not exist anymore (once a value has started being dropped, whether we return from that or unwind does not matter, the value is deemed to have been destroyed and not to exist)

[autumn hedge]

?godbolt flags="-C opt-level=z -Zunpretty=mir-cfg" ```rs
/// Newtype wrapper for a nicer name / readability
struct DropGlue<S>(S);

impl<S> Drop for DropGlue<S> {
fn drop(&mut self) {}
}

fn main()
{
let _2;
let mut _1 = String::new();
// we use braced construction to guarantee lack of unwinds in DropGlue's construction
_2 = DropGlue {
// we use unsafe and a raw pointer to let the snippet actually compile 😅 to get its MIR
0: unsafe { &mut *std::ptr::addr_of_mut!(_1) }
};
drop(_2);
}

[tidal spear]

warning: function `main` is never used
 --> <source>:8:4
  |
8 | fn main()
  |    ^^^^
  |
  = note: `#[warn(dead_code)]` on by default

warning: 1 warning emitted

digraph __crate__ {
subgraph cluster_Mir_0_9 {
    graph [fontname="Courier, monospace"];
    node [fontname="Courier, monospace"];
    edge [fontname="Courier, monospace"];
    label=<fn &lt;DropGlue&lt;S&gt; as Drop&gt;::drop(_1: &amp;mut DropGlue&lt;S&gt;) -&gt; ()<br align="left"/>debug self =&gt; _1;<br align="left"/>>;
    bb0__0_9 [shape="none", label=<<table border="0" cellborder="1" cellspacing="0"><tr><td bgcolor="gray" align="center" colspan="1">0</td></tr><tr><td align="left">return</td></tr></table>>];
}
subgraph cluster_Mir_0_10 {
    graph [fontname="Courier, monospace"];
    node [fontname="Courier, monospace"];
    edge [fontname="Courier, monospace"];
    label=<fn main() -&gt; ()<br align="left"/>let mut _1: std::string::String;<br align="left"/>let mut _2: DropGlue&lt;&amp;mut std::string::String&gt;;<br align="left"/>let mut _3: &amp;mut std::string::String;<br align="left"/>let _4: ();<br align="left"/>debug _2 =&gt; _2;<br align="left"/>debug _1 =&gt; _1;<br align="left"/>>;
    bb0__0_10 [shape="none", label=<<table border="0" cellborder="1" cellspacing="0"><tr><td bgcolor="gray" align="center" colspan="1">0</td></tr><tr><td align="left" balign="left">StorageLive(_1)<br/></td></tr><tr><td align="left">_1 = String::new()</td></tr></table>>];
    bb1__0_10 [shape="none", label=<<table border="0" cellborder="1" cellspacing="0"><tr><td bgcolor="gray" align="center" colspan="1">1</td></tr><tr><td align="left" balign="left">StorageLive(_3)<br/>_3 = &amp;mut _1<br/>_2 = DropGlue::&lt;&amp;mut String&gt;(move _3)<br/>StorageDead(_3)<br/></td></tr><tr><td align="left">_4 = std::mem::drop::&lt;DropGlue&lt;&amp;mut String&gt;&gt;(move _2)</td></tr></table>>];
    bb2__0_10 [shape="none", label=<<table border="0" c
```Output too large. Godbolt link: <https://godbolt.org/z/sj3vxonEj>

[autumn hedge]

Yielding: https://sketchviz.com/@danielhenrymantilla/02a18868013d7a41c58fddb03d316806