http fetch binary and write to file ends in.. OOM?

[frigid pond]

Some background, dag.http isn't usable. Short story is minecraft.net has the worst WAF known to mankind.
So I'm using a go http client with a custom header (like I said, worst WAF in the world) and I can fetch the binary file I need. I'm then trying to construct and return a new file and I'm having to use the hacky directory and newfile call (can't find any other way?)

I'm not sure where, but at some point... apparently I'm OOMing (unlikely, machine has plenty of RAM and the docker engine does too)

    // return a dagger file with the contents of the response body
    myFile := dag.Directory().WithNewFile("bds.zip", string(content)).File("bds.zip")

│ ✔ directory: Directory! 0.0s

Error: Post "http://dagger/query": command [docker exec -i dagger-engine-v0.17.1 buildctl dial-stdio] has exited with exit status 137, make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=

The file is like 70M and it's returned from a function as a *dagger.File and then being mounted into a container.

Any ideas?

[frigid pond]

Okay I resolved it. Well not entirely, I don't know truly how to fix it, but it's because of the file creation. It really hates the creation of a new directory with a string([]byte)

there really needs to be a way to construct a file from bytes

[cedar nebula]

@frigid pond is there any chance you can fetch the file with wget from a container? I think that'll be better than fiddling with the WithNewFile API. The main reason it's tricky is because since the engine API is GraphQL, it doesn't have any mechanism to stream data from the client to the server

related: https://github.com/dagger/dagger/issues/6093