#Why are the HashMap keys not released from the heap memory?

22 messages · Page 1 of 1 (latest)

shrewd pond
shrewd pond
#

Any comment about it? ferrisPlead

inner spindle
#

free_static_str is undefined behaviour, for starters

#

For many reasons:

  • in Stacked Borrows, the current WIP memory model, you can't turn a & into a Box. It's practically the same operation as converting & to &mut
  • in practice, the deallocate call used to free the memory may write to it, which writes to a &'s referent.
  • also, pointer provenance: A reference only has "access rights" for its exact referent, but the allocator might expect to be allowed to access some "allocator metadata" stores next to the value.
#

Can't you just put an actual Box<str> into the hashmap instead of going through the unsafety dance?

rose junco
#

Um, also isn't the Box from raw the wrong type?

shrewd pond
#

It did not work for me to use these HashMap keys as String or static str. How should I define it to store words like key and the memory is released. ferrisBigBrain

#

It doesn't work for me with Box<str>. I am doing something wrong. Would you give me some simple example? @inner spindle

rose junco
rose junco
# shrewd pond It did not work for me to use these HashMap keys as String or static str. How sh...

What about:

use std::{collections::HashMap, thread, time::Duration};

#[derive(Debug, PartialEq)]
enum Example {
    Vec(Vec<u8>),
    Test(u32),
    None
}

fn test_hashmap_static_str(i: i32) {
    let mut map: HashMap<String, Example> = HashMap::new();
    let mut vec_keys_hashmap = Vec::new();

    println!("load memory");
    for num in 0..100_000 {
        
        let str_num = format!("{}_test_{}", num, i);
        let key_hashmap = str_num;

        map.insert(key_hashmap.clone(), Example::Vec(vec![0; 100_000]));
        
        vec_keys_hashmap.push(key_hashmap);
    }
    thread::sleep(Duration::from_secs(5));

    println!("clear memory");
}

fn main() {
    for i in 0..5 {
        test_hashmap_static_str(i);
        thread::sleep(Duration::from_secs(5));
    }
    
    println!("END");
    loop {
        thread::sleep(Duration::from_secs(5));
    }
}
#

28M -> 300M -> 85M -> 300M -> 137M -> 300M -> 137M is an expected behavior from the system allocator as it is probably detecting the repeated allocation requests and keeping pages available to use

#

If the 300M goes up over time then there would be an issue

rose junco
shrewd pond
#

Thanks, I added: map.clear(); map.shrink_to_fit(); drop(map); and in this case I did not see that the memory was released completely. But it is mostly reused.

use std::{collections::HashMap, thread, time::Duration};

#[derive(Debug, PartialEq)]
enum Example {
    Vec(Vec<u8>),
    Test(u32),
    None
}

fn test_hashmap_static_str(i: i32) {
    let mut map: HashMap<String, Example> = HashMap::new();
    let mut vec_keys_hashmap = Vec::new();

    println!("load memory");
    for num in 0..100_000 {
        
        let str_num = format!("{}_test_{}", num, i);
        let key_hashmap = str_num;

        map.insert(key_hashmap.to_string(), Example::Vec(vec![0; 100_000]));
        
        vec_keys_hashmap.push(key_hashmap);
    }
    thread::sleep(Duration::from_secs(5));

    println!("clear memory");
    map.clear();
    map.shrink_to_fit();
    drop(map);
}

fn main() {
    for i in 0..5 {
        test_hashmap_static_str(i);
        thread::sleep(Duration::from_secs(5));
    }
    
    println!("END");
    loop {
        thread::sleep(Duration::from_secs(5));
    }
}
rose junco
shrewd pond
#

@rose junco
If you remove these lines:

let mut vec keys hashmap = Vec::new();
...
vec_keys_hashmap.push(key_hashmap);

memory release is not complete after Hashmap lifetime.

Add after line println!("clear memory");
Something like this: map = HashMap::new();
Does not help release memory to system! ferrisThonk

rose junco
rose junco
#

The 1024 delta in both is caused by the lazy initialization of the print macro

rose junco