Derive macro causing lifetime errors, even though there is no problem | Rust Programming Language Community | Page 1

wind field Jul 4, 2022, 10:00 AM

#

Hi there, I'm using the async-graphql crate and after the last update I started getting compile errors stating lifetime may not live long enough and it originates in the SimpleObject derive macro. The thing is I never got this error before (have used the crate and the exact same macro in several projects before), I only started getting it after updating rust to 1.62. Another thing to consider is that this happens sporadically and only on some structs I use this macro on. The process is that cargo check and cargo build both fail, while cargo build -r works fine, however, this still prevents me from using cargo watch -x run. The reason why I'm posting it here and not in the crate repo is that I believe it might be a bug in the rust compiler, something to do with incremental compilation (frankly, I don't really understand either, but after reading about similar bugs that's the conclusion I came to). Has anyone encountered similar behavior? If so, what did you do in order to fix it?

twilit mortar Jul 4, 2022, 10:02 AM

#

Bugs in incremental often go away if you run cargo clean, or just rename target/

hidden bone Jul 4, 2022, 10:02 AM

#

cargo clean -p <package>

wind field Jul 4, 2022, 10:02 AM

#

running cargo clean helps, but only for a short period of time, after compiling again one or two times the problem reoccurs

wind field Jul 4, 2022, 10:03 AM

#

hidden bone `cargo clean -p <package>`

Tried that, same thing as above, the problem disappears, but comes back, no matter what code change I make

distant ridge Jul 4, 2022, 11:18 AM

#

wind field Hi there, I'm using the `async-graphql` crate and after the last update I starte...

Nit: if incremental compilation causes problems with a macro, it's the macro's fault, not the compiler's.
In the "language contract", macros are supposed to be pure (no side-effect) and thus "consistent" (for a given input, yield the same output no matter when and how many times it is invoked). These requirements are there precisely to allow the compiler to be faster, especially with incremental compilation, by more "aggressively" caching macro invocations.

It is likely that SimpleObject is interacting with the file system in a way that violates these conditions, and that an upgrade of the compiler's caching heuristics is thus finally exposing an always-present bug.

That being said, it's true that in nightly Rust there are tools to let macro authors interact with the filesystem (and thus be inpure) by letting macros tell the compilers about it (to disable there and then the caching optimizations), and that in stable Rust those tools are currently unavailable. So it's a bit unfair to start exploiting these macro mistakes if macro authors don't yet have the tools to write proper code.

With all that being said, I suggest that you do open an issue in the repo, and you can even ping me (@danielhenrymantilla) so that I may chime in and suggest workarounds, once I know what exactly is that crate's macro doing 🙂

#

It is likely that SimpleObject is interacting with the file system in a way that violates these conditions
Hmm, after looking at their code more in detail, I find no trace of that "bad" pattern anywhere, so my assumption was wrong (for those curious I've grepped for fs, static, or [Hh]ash or other sources of randomness, and only found HashSet occurring, but with no .into_iter() on it (which could have been a source of impurity))

#

This invalidates my whole rant, then, since it seems the macro is genuinely pure

#

That, then, shifts the burden of responsibility for cache invalidation to the compiler indeed!

#

@wind field wouldn't you happen to reproduce this in a small contained example, I imagine? 😅 (since incremental compilation bugs are quite hard to produce on demand)

wind field Jul 4, 2022, 11:33 AM

#

I was just looking through the macro code as well and found nothing like you described, but then, I'm not nearly as experienced to fully understand what's happenning under the hood 😄 , there already is an issue open in the repo https://github.com/async-graphql/async-graphql/issues/900 although it was created 2 months ago, and for me and at least one other person (most recent comment) the bug only started to show after update to 1.62

GitHub

SimpleObject: error: lifetime may not live long enough · Issue #900...

I got some weird error after using #[graphql(complex)] and #[ComplexObject] on other struct. So the affected struct doesn't have the "complex" thing. I'm probably ...

wind field Jul 4, 2022, 11:37 AM

#

distant ridge <@285501227322834944> wouldn't you happen to reproduce this in a small contained...

I did manage to get this bug in a small example I made, I will send it in a minute, but it still happens only sporadically, as you said yourself, it's hard to reproduce

#

By the way thanks a lot for the explanation above, even though it may not be the problem in this case

distant ridge Jul 4, 2022, 11:38 AM

#

Ok, with your small example, try to isolate the #[derive(SimpleObject)] struct … to a file or module, and then try to run cargo expand on that module; to see what could be the source of the lifetime error as well

#

(I imagine that isolated it won't trigger the incremental compilation bug, but it may hint towards the code that could be leading to that)

wind field Jul 4, 2022, 11:42 AM

#

ahh, here it is, it happened after I added struct B, before that everything was fine. This is the whole code by the way, nothing else in there (except obligatory fn main() )

#

The fact that the struct is generic over lifetimes is not important, the bug persists after I change Cow<'a, str> to String

#

use async_graphql::SimpleObject;

#[derive(SimpleObject)]
pub struct A {
    field: String,
}

pub struct B {
    field: String,
}```

#

This is all I needed for the error to show, the code still compiles with cargo build -r and now also just with cargo build, but cargo check fails, thus giving me those error lines as seen in the screenshot

distant ridge Jul 4, 2022, 11:49 AM

#

Oh that's nicely small, awesome!! Lemme check

wind field Jul 4, 2022, 11:50 AM

#

It is really weird though, when I make a small change all errors seem to go away, after I make yet another change, they come back

distant ridge Jul 4, 2022, 12:11 PM

#

So this is what that derive generates: https://paste.rs/GHl.rs

#

Ok, got a nicer error message thanks to it:

error: lifetime may not live long enough
 --> /private/var/folders/m_/347zzk7j6nz08tt6z3tnd3z00000gn/T/tmp.qeHHjilha7/target/debug-proc-macros/test-f4020daedf6df8e8.rs:5:95
  |
5 |       async fn field(&self, ctx: &async_graphql::Context<'_>) -> async_graphql::Result<&String> {
  |  ____________________-___________________________________--_____________________________________^
  | |                    |                                   |
  | |                    |                                   let's call the lifetime of this reference `'2`
  | |                    let's call the lifetime of this reference `'1`
6 | |         ::std::result::Result::Ok(&self.field)
7 | |     }
  | |_____^ associated function was supposed to return data with lifetime `'1` but it is returning data with lifetime `'2`

error: lifetime may not live long enough
 --> /private/var/folders/m_/347zzk7j6nz08tt6z3tnd3z00000gn/T/tmp.qeHHjilha7/target/debug-proc-macros/test-f4020daedf6df8e8.rs:6:9
  |
5 |     async fn field(&self, ctx: &async_graphql::Context<'_>) -> async_graphql::Result<&String> {
  |                    -                                   -- let's call the lifetime of this reference `'2`
  |                    |
  |                    let's call the lifetime of this reference `'1`
6 |         ::std::result::Result::Ok(&self.field)
  |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ associated function was supposed to return data with lifetime `'2` but it is returning data with lifetime `'1`

#

That part does not involve async_trait (I was worried it would somehow play a role), which means that the macro expanding to:

#[allow(clippy::all, clippy::pedantic)]
impl A {
    #[inline]
    #[allow(missing_docs)]
    async fn field(&self, ctx: &async_graphql::Context<'_>) -> async_graphql::Result<&String> {
        ::std::result::Result::Ok(&self.field)
    }
}

#

is the problem

wind field Jul 4, 2022, 12:23 PM

#

distant ridge That part does not involve `async_trait` (I was worried it would somehow play a ...

Please don't mind the noob question, but what exactly is the problem here?

distant ridge Jul 4, 2022, 1:17 PM

#

Ok, got a minimal repro:

#

?play

fn main(){let _=std::process::Command::new("/bin/bash").args(&["-c", concat!(r##"{
set -euo pipefail
cd $(mktemp -d)


cargo init -q --name example --lib

cat> src/lib.rs <<'EOF'
    #![allow(unused)]
    struct Foo;
    
    impl Foo {
        async fn f(&self, _: &&()) -> &() {
            &()
        }
    }
EOF

(set -x
    cargo c -q
    { set +x; echo 'enum Bar {}'; } 2>/dev/null | tee -a src/lib.rs
    cargo c -q
)


echo ✅
} 2>&1"##)]).status();}

tight dewBOT Jul 4, 2022, 1:17 PM

#

+ cargo c -q
+ tee -a src/lib.rs
enum Bar {}
+ cargo c -q
error: lifetime may not live long enough
 --> src/lib.rs:5:43
  |
5 |           async fn f(&self, _: &&()) -> &() {
  |  ____________________-__________-___________^
  | |                    |          |
  | |                    |          let's call the lifetime of this reference `'2`
  | |                    let's call the lifetime of this reference `'1`
6 | |             &()
7 | |         }
  | |_________^ associated function was supposed to return data with lifetime `'2` but it is returning data with lifetime `'1`
  |
help: consider introducing a named lifetime parameter and update trait if needed
  |
5 |         async fn f<'a>(&'a self, _: &&'a ()) -> &() {
  |                   ++++  ++            ++

error: lifetime may not live long enough
 --> src/lib.rs:5:43
  |
5 |           async fn f(&self, _: &&()) -> &() {
  |  ____________________-__________-___________^
  | |                    |          |
  | |                    |          let's call the lifetime of this reference `'2`
  | |                    let's call the lifetime of this reference `'1`
6 | |             &()
7 | |         }
  | |_________^ associated function was supposed to return data with lifetime `'1` but it is returning data with lifetime `'2`
  |
help: consider introducing a named lifetime parameter and update trait if needed
  |
5 |         async fn f<'a>(&'a self, _: &&'a ()) -> &() {
  |                   ++++  ++            ++

error: could not compile `example` due to 2 previous errors

wind field Jul 4, 2022, 1:46 PM

#

@distant ridge wow, great work. So if I understand all this correctly it is indeed a bug in the compiler?

distant ridge Jul 4, 2022, 1:46 PM

#

Yes 💯

#

I'm working on a palliative workaround for async-graphql by using https://docs.rs/fix-hidden-lifetime-bug

fix_hidden_lifetime_bug - Rust

::fix_hidden_lifetime_bug

distant ridge Jul 4, 2022, 2:02 PM

#

@wind field see https://github.com/async-graphql/async-graphql/pull/972#issue-1293217161

GitHub

Palliative workaround for #900 by danielhenrymantilla · Pull Reques...

This uses https://docs.rs/fix-hidden-lifetime-bug to replace the compiler's heuristic with lifetimes with a simpler and more straight-forward one, which ought to dodge the regression from rust-...

wind field Jul 4, 2022, 2:13 PM

#

great! ferrisBased I, as a casual user of rust, thank you for the work you've put into this. I've also read the issue you've opened in the rust repo, hope it will get resolved soon

#Derive macro causing lifetime errors, even though there is no problem