#How to upload a file to a storage bucket without file.Export?

1 messages · Page 1 of 1 (latest)

dire slate
#

I'm trying to upload a file to GCS without using file.Export(), so we can avoid interacting with the host. When I use file.Contents(), I have one (large) binary that reports the error.

✘ File.contents: String! 0.0s
! file size 257548386 exceeds limit 134217728

Generally, it would be great to have a file.Bytes() function that returned the contents as a bytes slice, rather than a string.

minor zephyr
#

Yeah we do need that... At the moment there's no way to stream the raw contents of the file efficiently in the core API. I recommend mounting the file in a container with the upload client installed, and orchestrating the upload that way

#

In this particular case I would probably recommend doing the upload that way, even if we did support streaming the bytes to the client

dire slate
#

facepalm that was what I had planned to do but forgot about today...

#

Does Dagger Export with a different format when the filesize is over that limit?

#

I'm getting a header like... with file.Export() for the one large (almost 400Mb) file

!<arch>
__.PKGDEF       0           0     0     644     2811757   `
go object linux amd64 go1.22.5 GOAMD64=v1 X:regabiwrappers,regabiargs,coverageredesign,allocheaders,exectracer2
build id "qG5sMaH23_LieBFs7noz/sClDwypCbBk5PD5dU45Z"
#

this one is also built with CGO enabled

#

ok, this header thing is PEBCAK error, was go build'n the wrong directory with the -o flag

minor zephyr
#

btw I'm about to push a major improvement to our own reusable Go builder module. Getting into stdlib material

dire slate
#

Are we able to use 3rd party modules via the SDKs?

#

If you have a link, I'd love to check it out either way