Memory leak when printing out values

[sly tartan]

I am working on a program that sets my hue lights to the color of pixels on my screen. While working on this I have added a println! to print the rgb values I send to the hue bridge, but after doing so, it eats up my RAM in less than a minute.

The offending part of a code:```rs
let mut loop_body = async || {
let frame = match capturer.get_next_frame() {
Err(e) => {
dbg_print!("{e}");
return;
}
Ok(frame) => frame,
};

    let Frame::BGRx(frame) = frame else {
        return;
    };

    let middle_pixel_index: usize = (4 * frame.width * (frame.height / 2)) as usize;

    let [r, g, b]: &[u8; 3] = &frame.data[middle_pixel_index..middle_pixel_index + 3]
        .try_into()
        .unwrap();

    let r: u16 = *r as u16 * 255;
    let g: u16 = *g as u16 * 255;
    let b: u16 = *b as u16 * 255;

    println!("{r} {g} {b}"); // IF left uncommented, leaks memory

    let colors = &[Color::new(r, g, b), Color::new(r, g, b)];

    if let Err(e) = hue_entertainment.send_colors(colors).await {
        dbg_print!("{e}");
    }
};

loop {
    select! {
        _ = &mut shutdown => { break }
        _ = ticker.tick() => loop_body().await
    }
}

You can see the whole code base here: https://github.com/kiselina2/mood/tree/master/src there isnt much yet.

[winged sequoia]

1. how are you measuring memory usage?
2. how much memory is it using?

[sly tartan]

btop

its constantly growing

idk by how much

[winged sequoia]

well, run it with and without the println for some amount of time (30 seconds?) and see what btop says

[sly tartan]

I already did

you want some actuall numbers?

[winged sequoia]

yes

[sly tartan]

idle

10s

If you want something more precise I can get some other tools

[winged sequoia]

this is fine. and if you comment out the println it stays at 7.6?

[sly tartan]

yep

[winged sequoia]

ugh, why didn't scap's docs build... My current guess is that println is slowing things down enough that scap is building up frames. Dunno if it behaves like that, though.

[sly tartan]

that is why I reduced framerate to 1 per second

it doesnt seem to change much

It seems to grow at a the same speed whatever framerate a set

I was able to build the scap docs localy

Might be because of

config.toml

[winged sequoia]

it's just a library that docs.rs doesn't have

[sly tartan]

hm, I hought that this meant It wasnt able to build them

Oh, I was right

[winged sequoia]

I mean, this crate depends on a native library that docs.rs doesn't have

[sly tartan]

yeh

[tranquil vector]

have you tried running heaptrack to profile where the allocations are coming from?

[sly tartan]

nope

[tranquil vector]

i'd recommend giving that a go then

[sly tartan]

I have not much idea how to work with this program :D

Guess Ill try some research

[tranquil vector]

there are a few different views

there should be some tabs in different places

one should give you a graph of the total heap usage over time

another should allow you to see a tree of the most allocating functions

but there doesn't seem to be any rust symbols there

[sly tartan]

yea, but unresolved function does not help much

[tranquil vector]

remember to compile with debug = true

in your release profile

or compile in dev mode

[sly tartan]

cargo run doesnt have debug = true by default?

[tranquil vector]

nope

[sly tartan]

ok

[tranquil vector]

at least not in release mode

also when profiling, make sure that you're profiling the actual executable

not the cargo process

[sly tartan]

oh, ok

[tranquil vector]

ie run cargo build rather than cargo run, then profile the executable from the target directory

[sly tartan]

still looks like this

[tranquil vector]

there should be an option to enable symbol demangling i think

and then have a look at the different views

not sure if you're currently looking at the top-down or bottom-up view

[sly tartan]

top down

got somewhere into pipewire :D

[tranquil vector]

if it's the top-down one, you'll just have to keep expanding the function calls until you reach the internals of your actual app

[sly tartan]

This is as deep as it goes

[tranquil vector]

apparently you need rustc_demangle to demangle rust symbols https://invent.kde.org/selimrecep/heaptrack/-/blob/master/README.md#heaptrack-dependencies

but well you can see that something is allocating a bunch of vec data

so look at the parent functions there, see what's allocating the data and where it's supposed to be dropped

i see that you're enabling the async_drop feature, so i wonder if there's a bug there causing some data to not be dropped properly

[sly tartan]

Ill try build without the feature

removing async drop did not help

I have tried installing aur/rustc-demangle but that did not help :D

mabey I need to restart the PC?

Ok, the leak is caused by the combination of the print and the capturer

if I comment out either the capturer.start_capture(); or the print, it stops

hm

I guess I should find an alternative to scap

welp, from heaptrack, it looks like its caused by scap, but I would very much like to know, why does the print have any influence on it

[tranquil vector]

what happens if you pass the values to black_box instead of printing them?

[sly tartan]

nothing

does not leak

I would like to point out, that it does not matter what gets printed out

I have now this code:

    capturer.start_capture();

    let mut loop_body = async || {
        // let frame = match capturer.get_next_frame() {
        //     Err(e) => {
        //         dbg_print!("{e}");
        //         return;
        //     }
        //     Ok(frame) => frame,
        // };

        // let Frame::BGRx(frame) = frame else {
        //     return;
        // };

        // let middle_pixel_index: usize = (4 * frame.width * (frame.height / 2)) as usize;

        // let [r, g, b]: &[u8; 3] = &frame.data[middle_pixel_index..middle_pixel_index + 3]
        //     .try_into()
        //     .unwrap();

        // let r: u16 = *r as u16 * 255;
        // let g: u16 = *g as u16 * 255;
        // let b: u16 = *b as u16 * 255;

        println!(""); // IF left uncommented, leaks memory

        // let colors = &[Color::new(r, g, b), Color::new(r, g, b)];

        // if let Err(e) = hue_entertainment.send_colors(colors).await {
        //     dbg_print!("{e}");
        // }
    };

    loop {
        select! {
            _ = &mut shutdown => { break }
            _ = ticker.tick() => loop_body().await
        }
    }

and it still leaks

[tranquil vector]

also heaptrack should allow you to jump to the corresponding line of code

so you can investigate what the allocating code is doing and see where it should be getting dropped

in most cases a single print causing a difference in observed behaviour tends to be a sign of undefined behaviour

idk if the crate had some unsound unsafe code or something

[winged sequoia]

Another thing you can try is replacing the print with a small sleep (like 10ms). If it's the same, it's more evidence of a timing issue.

[sly tartan]

Used thread sleep instead of the print and no leak

[sour kraken]

Do you have a #[global_allocator]? @sly tartan

Oh, seems like you don't

odd

@sly tartanDoes changing println!() to tokio::task::yield_now().await still leak memory?

[winged sequoia]

don't forget the .await

[sly tartan]

I think I found the place, where the leak happens

https://github.com/CapSoftware/scap/blob/main/src/capturer/engine/linux/mod.rs

line 128

fixing it is a different problem :D

I dont think I have enough rust knowledge to know wtf is happening

[sour kraken]

Why do you think that's a memory leak?

[sly tartan]

its the most similar part of the code#1489656145940316291 message

[sour kraken]

That is where the data was allocated, but not where the data was leaked.

[sly tartan]

oh, ok

[sour kraken]

The user_data.tx.send calls sends the memory over to somewhere else. The memory leak happened on the receiving end there.

[sly tartan]

Well, after that it gets recieved in get_next_frame, not much happening there, and after that its in my program

[sour kraken]

That means the frame variable in your program got leaked somehow

Which is bizarre

[sly tartan]

nope

[sour kraken]

Bizarre

[sly tartan]

although I did notice, that it still grabbed suspiciously a lot of ram

like 3GB

[sly tartan]

new discovery
instead of println I used vscode logpoint
now I can literally turn on the leak by activating the logpoint

[sly tartan]

I might have found the problem

its a theory

notice the loop

[winged sequoia]

nothing here should be leaking memory

[sly tartan]

My theory:

1. the fps set in the options does not change how often does mpsc channel get filled with a frame
2. if your program runs slower, then the channel queue starts getting bigger and bigger
3. since the get_next_frame is in the loop that wont stop until it finds a valid frame, you never know if you actually emptied the queue

[winged sequoia]

that doesn't explain why printing is the only thing that can cause it

[sly tartan]

the reason for my point #1 is that when I set the FPS to 1, the get_next_frame still returned a frame way more ofthen than once a second

[sly tartan]

hmm, true

[winged sequoia]

process_channel_item only returns None on macos, so this is effectively not a loop. You're on linux right? This crate doesn't let you drain the channel yourself without doing hacky timing checks, so you'll probably need to stop using scap if this is really the problem.

[sly tartan]

yeh

its assuming I will need all the frames, but I only need the latest

ok, I fixed it

[winged sequoia]

If you really need to, you could put all the frame stuff in another thread that always consumes frames and sends them to the main thread through another channel.

[sly tartan]

I just cloned scap and rewrote the function

no more leaking

And I can easily get the newest frame

[winged sequoia]

yeah that's fair. Maybe pr them? (make it a new function)

[sly tartan]

Ill write an issue. The repo doesnt seem to be very active. Their newest version isnt even buildable.

Actually even this version isnt, without me putting this into .cargo/config.toml

[env]
LIBCLANG_PATH = "/usr/lib/llvm21/lib"

[winged sequoia]

ahh

[sly tartan]

I also tried xcap, but that didnt even give me the promt to select which screen I want to record

And there doesnt seem to be any other cross platform screen recording rust lib

So I am kinda stuck with this

[sly tartan]

Ill mark it as solved even though I am still not 100% sure what the actual cause of the memory leak is. But I am suspecting the channel queue.

[sour kraken]

@sly tartan Maybe you're running into some compiler bug with the async_drop feature? IIRC, that one still has some bugs.

[sly tartan]

nope, we tried compiling without it already

[sour light]

i'm new to rust, didn't see the issue, was curious, so decided to ask brave ai search.

i would advise you to also go to brave.com and just paste the code and ask it for the same advise. don't dare to paste bot output here as many channels seem to be against it.

its top recommendations, one of 5 or so is to release frame earlier and remove the inner while loop and just call get_next_frame each tick.

[covert quarry]

<@&631915156854538260> spam above

[sturdy jackal]

Not spam; maybe not great advice, but it's an attempt at advice

[covert quarry]

mb, sorry