#Return reference to buffer in iterator

17 messages · Page 1 of 1 (latest)

somber heart
#

I'm trying to implement a basic Lexer as a learning experience. I understand that returning a reference to the buffer with lifetime <'s> doesn't work because 's outlives the Lexer, but I'm not sure what to do about it.

pub struct Lexer<'s> {
    src: Peekable<Chars<'s>>,
    buf: String,
}

impl<'s> Iterator for Lexer<'s> {
    type Item = Token<'s>;

    fn next(&mut self) -> Option<Self::Item> {
        self.buf.clear();
        loop {
            match self.src.next()? {
                n if is_ident_start(&n) => {
                    self.src.next_if(is_ident_char).iter().collect_into(&mut self.buf);
                    return Some(kw_or_ident(&*self.buf)); // <-- Error occurs here
                },
                ' ' => continue,
                _ => {}
            }
        }
    }
}

fn kw_or_ident<'t>(buf: &'t str) -> Token<'t> {
    KEYWORDS
        .get(buf)
        .copied()
        .map(|kw| Token::Keyword(kw))
        .unwrap_or_else(|| Token::Literal(buf))
}

error: lifetime may not live long enough
  --> src/lexer.rs:27:28
   |
18 | impl<'s> Iterator for Lexer<'s> {
   |      -- lifetime `'s` defined here
...
21 |     fn next(&mut self) -> Option<Self::Item> {
   |             - let's call the lifetime of this reference `'1`
...
27 |                     return Some(kw_or_ident(&*self.buf));
   |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ method was supposed to return data with lifetime `'s` but it is returning data with lifetime `'1`
pallid meteor
#

maybe try use a 2nd lifetime

# 
pub struct Lexer<'s> {
    src: Peekable<Chars<'s>>,
    buf: String,
}

impl<'s, 'o> Iterator for Lexer<'s> {
    type Item = Token<'o>;

    fn next(&mut self) -> Option<Self::Item> {
        self.buf.clear();
        loop {
            match self.src.next()? {
                n if is_ident_start(&n) => {
                    self.src.next_if(is_ident_char).iter().collect_into(&mut self.buf);
                    return Some(kw_or_ident(&*self.buf)); // <-- Error occurs here
                },
                ' ' => continue,
                _ => {}
            }
        }
    }
}

fn kw_or_ident<'t>(buf: &'t str) -> Token<'t> {
    KEYWORDS
        .get(buf)
        .copied()
        .map(|kw| Token::Keyword(kw))
        .unwrap_or_else(|| Token::Literal(buf))
}
#

like this might work

somber heart
# 
error[E0207]: the lifetime parameter `'t` is not constrained by the impl trait, self type, or predicates
  --> src/lexer.rs:18:10
   |
18 | impl<'s, 't> Iterator for Lexer<'s> {
   |          ^^ unconstrained lifetime parameter
pallid meteor
#

i think u need

impl<'s: 't, 't> ...
somber heart
#

same error

bleak summit
#

This isn't possible with the iterator api

#

The buffer borrowed from has to be outside the iterator

pallid meteor
#

someone who knows more

#

can help

bleak summit
#

This requires what is called lending iterators

somber heart
bleak summit
#

char_indices()

somber heart
#

Didn't realize that was a thing

#

ill try it out, thanks