how to keep iterators as values in a HashMap without leaking lifetime semantics everywhere

[nova prawn]

hi folks!

I am wondering if this is possible -- what do you think? I would like to be able to save them in one function call and then access them by key later in another function, before using them to iterate

[fossil mica]

What do you have right now?

[nova prawn]

pub type NodeMap = HashMap<u8,Node>;
pub type NodeMapIterator<'a> = hash_map::Iter<'a, u8, Node>;

pub struct Dock<'a> {

    pub data: NodeMap,

    pub itmap: HashMap<Output,HashSet<NodeMapIterator<'a>>>,

    /**
      | nodes in vector for quick
      | random eviction
      |
      |
      */
    pub entry_list: Vec<NodeMapIterator<'a>>,

    /**
      | Index from id into the data
      | to lookup entries using
      | their ids.
      |
      */
    pub id_to_node_it: HashMap<Id,NodeMapIterator<'a>>,
}

basically that ^^

the trouble is there is another struct which contains a Dock<'a> from which closures are scheduled to be run

when I add the lifetime 'a to the impl blocks of this struct, I get an error during the closure scheduling

"cannot infer lifetime due to conflicting requirements"

first, the lifetime cannot outlive 'a

but the lifetime must be valid for the static lifetime

[fossil mica]

I think you're trying to make a self-referential type, which is not simple to do in rust. 'a has to outlive Dock, but 'a refers to data (I'm guessing) which is part of Dock. Why do you need 3 different views into the map though?

[nova prawn]

pub struct Manager<'a> {
  dock: Dock<'a>,

}

impl<'a> Manager<'a> {
  fn do_stuff_and_schedule(self: Arc<Self>, scheduler: &mut Scheduler) {

    // do some stuff

    scheduler.schedule_every(Box::new(||{ /* do some check */ }), CHECK_INTERVAL)

  }

}

yes, 'a refers to data

as for why three different view into map are needed -- i think it is because they represent different logical views of the data

in one case (id_to_node_it), we want them indexed by id

in another case, (entry_list) we just want a small list which may be evicted on the next process call

in the last case (itmap), there is a type Output we use to index, but the mapping points to sets of iterators, not just iterators. and, in this case, we want to do something with the set

it does seem like a bit of an unwieldy design

it is kind of like maintaining multiple indices i suppose

i experimented with just using pub type NodeMapIterator = u8; because this is how the original data is indexed

I hit a problem because the client code actually wants to use it to iterate, not just access

ok ------------- i think i have distilled the situation down a little bit more clearly now:

the problem is showing up here:

impl<'a> Manager<'a> {
  fn do_stuff_and_schedule(self: Arc<Self>, scheduler: &mut Scheduler) {

    // do some stuff
    let s = self.clone();

    scheduler.schedule_every(Box::new(||{
        let s = s.clone();

    }), CHECK_INTERVAL)
  }
}

I am needing to use the self: Arc<Self> syntax so that I can pass a handle to self into this scheduler function

however, when I introduced the lifetime for reasons outlined above, now I hit a compiler error

`

|| cannot infer an appropriate lifetime due to conflicting requirements
||     |
|| 87  |           let maybe_send = |pnode: Amo<Node>| {
||     |  _________________________________^
|| 88  | |
|| 89  | |             let mgr = self.clone();
|| 90  | |
|| ...   |
|| 139 | |             */
|| 140 | |         };
||     | |_________^
||     |
|| first, the lifetime cannot outlive the lifetime `'a` as defined here...
||     |
|| 39  | impl<'a> MyTrait for Manager<'a> {
||     |      ^^
|| ...so that the types are compatible
||     |
|| 87  |           let maybe_send = |pnode: Amo<Node>| {
||     |  _________________________________^
|| 88  | |
|| 89  | |             let mgr = self.clone();
|| 90  | |
|| ...   |
|| 139 | |             */
|| 140 | |         };
||     | |_________^
||     = note: expected `(&proj_imports::Arc<Manager<'_>>, &std::option::Option<proj_imports::Arc<proj::Index>>)`
||                found `(&proj_imports::Arc<Manager<'a>>, &std::option::Option<proj_imports::Arc<proj::Index>>)`
||     = note: but, the lifetime must be valid for the static lifetime...
|| ...so that the expression is assignable
||     |
|| 142 |         self.net.get().for_each_node(&maybe_send_header);
||     |                                          ^^^^^^^^^^^^^^^^^^
||     = note: expected `&(dyn Fn(proj_imports::Amo<proj_net::Node>) + 'static)`
||                found `&dyn Fn(proj_imports::Amo<proj_net::Node>)`
||

Amo<T> is Arc<Mutex<Option<T>>>

[fossil mica]

It's saying 'static is the only lifetime for which this will be valid, which is because it's self-referential.

[nova prawn]

what can be done?

[fossil mica]

You can't use hash_map::Iter. You should probably store a bunch of u8 instead like you said before, but return a custom iterator that references Dock or data whenever it's asked for. The u8 will just store progress.

[nova prawn]

thanks -- i will give it a shot 🙏

i appreciate the energy you spent looking through what i wrote to help me ☺️

[fossil mica]

good luck!