#Why does this code not compile

65 messages · Page 1 of 1 (latest)

grim harbor
#

I don't quite understand how the return makes the code not compile, it complains about a second mutable borrow but that borrow should never happen if the function returns.

use std::io::BufRead;

struct Foo<R: BufRead> {
    r: R,
}

impl<R: BufRead> Foo<R> {
    fn foo(&mut self) -> &[u8] {
        let buf = self.r.fill_buf().unwrap();
        
        if buf.len() > 1 {
            // compiles if commented out
            return buf;
        }
        
        drop(buf);
        self.r.consume(1);
        &[]
    }
}
light jay
grim harbor
#

ha it works with that yeah

#

i got it working on my end by doing the return last and moving the other code before it. its more messy but works

#

i guess thats all i can do for now

#

thanks for the help

jagged meteor
#

the simple solution is just to put your condition in a variable first

#

oh actually nvm i think i misunderstood the problem

light jay
jagged meteor
#

this isn't the if let situation

grim harbor
#

what i want to do is read a bufread line by line without copying the bytes when possible.

so the flow is call fill_buf(), return a slice of the line if theres a '\n'. otherwise fill my own buffer in a loop until there is a new line and return a slice to my internal buffer.

I have working messy code that does compile. but if there's any established way to do this please let me know

jagged meteor
#

How is this different to BufRead::lines()?

grim harbor
#

lines returns a vec unconditionally. I want this to be zero copy when possible

jagged meteor
#

it doesn't

formal gorge
jagged meteor
#

oh, you mean the lines iterator itself

grim harbor
#

by vec i mean String i guess

#

im working with Vec<u8>

#

or &[u8] ideally rather

jagged meteor
#

I feel like you should just use Read directly and make your own buffer

#

Instead of juggling both your own buffer and the one in BufRead

formal gorge
grim harbor
#

my call to fill_buf() fowards to libarchive's archive_read_data_block which is zero copy and returns the block of bytes it allocated

jagged meteor
#

alr

formal gorge
#

?play ```rust
use std::io::BufRead;

struct Foo<R: BufRead> {
r: R,
}

impl<R: BufRead> Foo<R> {
fn foo(&mut self) -> &[u8] {
let buf = self.r.fill_buf().unwrap();

    if buf.len() > 1 {
        // compiles if commented out
        return self.r.fill_buf().unwrap();
    }
    
    drop(buf);
    self.r.consume(1);
    &[]
}

}

languid yewBOT
jagged meteor
formal gorge
#

There we go

grim harbor
#

this is my messy untested code that compiles

    fn read_line2(&mut self) -> Result<&[u8], ParseError> {
        self.read.consume(self.consume);
        self.buf.clear();
        self.consume = 0;

        let buf = self.read.fill_buf().unwrap();
        let pos = memchr::memchr(b'\n', buf);

        if pos.is_none() {
            self.buf.extend_from_slice(buf);
            self.read.consume(self.buf.len());

            loop {
                let buf = self.read.fill_buf().unwrap();
                if let Some(n) = memchr::memchr(b'\n', buf) {
                    self.buf.extend_from_slice(&buf[..=n]);
                    self.read.consume(n);
                    return Ok(self.buf.as_slice());
                } else {
                    let len = buf.len();
                    self.buf.extend_from_slice(buf);
                    self.read.consume(len);
                }
            }
        }

        let buf = self.read.fill_buf().unwrap();
        let pos = pos.unwrap();
        self.consume = pos;
        return Ok(&buf[..=pos]);
    }

im just double checking there's not an established pattern for this

formal gorge
#

Any well behaved BufRead should do nothing on the second call

jagged meteor
#

but i think this is a logic error, because you should be consuming the bytes when returning them

#

otherwise you'll just keep returning the same line

grim harbor
formal gorge
grim harbor
#

essentially i have a line based ini ish format and i want to return the lines without copying when possible

jagged meteor
formal gorge
#

Yeah all I was trying to solve was your borrowing error

grim harbor
#

ah okay

#

ill diff the two

jagged meteor
formal gorge
grim harbor
jagged meteor
#

does it almost always fit into the internal buffer?

grim harbor
#

yeah so each line is usually like 30 bytes so the case where i have to copy it into my own buffer and join the 2 reads is rare

#

and the file is 33MB

jagged meteor
#

alr

#

have you compared it to just slurping the entire file first? just curious

grim harbor
#

and loading 33MB into memroy?

jagged meteor
#

yeah

grim harbor
#

non starter for me really

#

this is meant to be a reimplementation of pacman in rust if you're familiar

jagged meteor
#

the game?

grim harbor
#

archlinux's package manager

jagged meteor
#

oh the Packet manager

#

right

jagged meteor
grim harbor
#

yeah its rearanged like that to avoid the compile error. but ill see if your solution works in the real code

#

thanks for the help

jagged meteor
#

BufReader::lines() indeed appears slow

#

however, using BufReader::read_line allows to reuse the buffer and is on par with something that should be similar to what you're trying.

#

@grim harbor :)

#

actually, read_line is slower, not sure how conclusive this is though

#

actually, using read_until it's faster than the more complicated approach