Await Many Futures to Complete Asynchronously

[mortal comet]

I have a ton of futures (~100) and I don't care in what order they're done. I'd like for multiple futures to be "in-flight" at once (around 4 futures to be worked on at the same time).

I'm using tokio and I expected something like FuturesUnordered to poll multiple futures at once. I wrote this:

    let client = ClientBuilder::new().build().ok()?;
    let result = files.iter()
        .map(|file| download_file(&client, url, file.clone())) // Returns Result<(String, Vec<u8>), ...>
        .collect::<FuturesUnordered<_>>()
        .collect::<Vec<_>>() // Collecting into a vec here since I'm not sure how to filter
        .await
        .into_iter()
        .filter_map(|x| x.ok())
        .collect::<HashMap<String, Vec<u8>>>();

But with this code, we can clearly see one async call is being processed at a time. Yikes! How can I ask it to dispatch multiple async calls at once?

I'd expect something more like this if that makes sense. No point in doing these tasks one at a time.

Downloading file ...
Downloading file ...
Downloading file ...
File ... completed!
Downloading file ...
File ... completed!
File ... completed!
Downloading file ...
Downloading file ...

[mortal seal]

I think this will do what you want: https://docs.rs/futures/latest/futures/future/fn.join_all.html

[ancient viper]

FuturesUnordered should also work for this use case

maybe check if download_file is somehow blocking or subject to a 1-request-at-a-time policy

[mortal comet]

iirc i also tried join_all and got similar behaviour, there's no way the futures are completing too fast (i don't think, there's like a couple second pauses between calls)

[mortal comet]

alright... that's interesting

I'm sharing one http client between all the requests

maybe i have to make multiple? 😅

i guess the conus question is how many futures will FuturesUnordered poll at once?

[ancient viper]

if that's reqwest::Client then you shouldn't need to, because it is specified to internally use a connection pool

[ancient viper]

all of them

[mortal comet]

:|

[ancient viper]

there are different tools for limiting

[mortal comet]

can i limit that?

yeah

i dont want to simultaneously make 130 some requests to some poor server

but i dont want one at a time

[ancient viper]

https://docs.rs/futures/latest/futures/stream/trait.StreamExt.html#method.buffer_unordered

I would expect reqwest to have its own limit but I don't know whether it in fact does or where to see it

what is the code of download_file ?

[mortal comet]

This is the whole thing 😅 again the client, HttpRequestBuilder and friends are tauri's http stuff

async fn download_file(
    client: &Client, base_url: &str, file: String
) -> tauri::api::Result<(String, Vec<u8>)> {
    let path = format!("{}{}", base_url, file);
    println!("Downloading file {}...", &path);
    let request = HttpRequestBuilder::new("GET", &path)?;

    let response = client.send(request).await?;
    let bytes = response.bytes().await?.data;

    println!("File {} gave {} bytes!", &path, bytes.len());

    Ok((file, bytes))
}

hmm... even when i instantiate many clients

im getting this sync behaviour

[ancient viper]

are you sure the server isn't giving you one download at a time

oh no then the "Downloading... would be batched

[mortal comet]

potentially, the server is github

yeah

the println should probably be grouped regardless

im tempted to just time::wait(Duration::from_millis(1000)).await or smth

[ancient viper]

well, one option would be to tokio::spawn each request (inside the .map(|file| ...) so you know they'll be started

not dependent on FuturesUnordered

this shouldn't be necessary but it might help pin down where the problem is

[mortal comet]

aye, yeah this seems weird

i guess this means that the first call to poll doesn't return for a long time?

if I sleep, i get the expected behaviour...

Downloading file https://raw.githubusercontent.com/gleitz/midi-js-soundfonts/gh-pages/FatBoy/electric_guitar_jazz-mp3.js...
Downloading file https://raw.githubusercontent.com/gleitz/midi-js-soundfonts/gh-pages/FatBoy/harpsichord-mp3.js...
Downloading file https://raw.githubusercontent.com/gleitz/midi-js-soundfonts/gh-pages/FatBoy/glockenspiel-mp3.js...
Downloading file https://raw.githubusercontent.com/gleitz/midi-js-soundfonts/gh-pages/FatBoy/pad_5_bowed-mp3.js...
Downloading file https://raw.githubusercontent.com/gleitz/midi-js-soundfonts/gh-pages/FatBoy/bagpipe-mp3.js...

literally all of them at once 😰

does tokio not have some kind of batch dispatching (like popping items from a queue to work on?)

Sounds like I'll have to pull reqwest 😅

[ancient viper]

i guess this means that the first call to poll doesn't return for a long time?
if that's true then that's a bug in some future / async fn

poll() should always do whatever updates it needs and return promptly

[mortal comet]

yeah im just wondering how else you could accomplish this behaviour

you would have to coerce FuturesUnordered to somehow