#Difference between `with-exec` and running the same command in `terminal`

1 messages · Page 1 of 1 (latest)

crimson bay
#

Hi all!
I've recently began using dagger for describing our CI workflows and so far am really pleased.
A small thing though drives me nuts as I just can't understand why there should be anything different.

I'm using dagger currently to cross-compile a C++ Meson project on an AMD64 machine by using a docker image with the linux/arm64 platform.
The whole command can be summarized as:

get-build-kit-container --platform linux/arm64 \                                                                                                                                                  
                     | with-directory /src . \                                                                                                                                                                       
                     | with-workdir /src \                                                                                                                                                                           
                     | with-exec meson,setup,build,--default-library=shared \                                                                                                                                        
                     | with-exec meson,compile,-C,build

where get-build-kit-container just returns the container with all required dependencies.

The whole build works just fine when using the linux/amd64 platform instead.
When using the linux/arm64 platform though it depends:

  • In the form above, it fails on build with multiple segfaults like c++: internal compiler error: Segmentation fault signal terminated program cc1plus
  • but when I replace the with-exec meson,compile,-C,build with a terminal and run the exact same meson compile -C build command it builds successfully every single time!
#

I've tried this now so many times that it can't be just luck as it always fails in the above form and always works when running the build command in the terminal.
I've also tried reducing the build processes by appending -j1 to the with-exec.
That interestingly enough worked, but I haven't tested it multiple times due to the build then taking a chilling ~55 min instead of the usual ~4 min.
Going up to even only -j2 failed again though.

So my main question is: Is there any difference between running the build command via with-exec instead of manually in the terminal and especially does anyone have an idea how I could fix this?

Thanks in advance for any help!

crude crane
#

@crimson bay one random thought: in one case dagger is executing the command directly, in the other it's executing a shell; then the shell runs the command. So maybe try with-exec sh,-c,"meson setup ..." to eliminate that variable?

crimson bay
#

I've previously checked that by adding a simple shell script with the compile command and executing that - now also tried it with sh -c but same outcome.
Thank you though 🙂

crude crane
#

@crimson bay does the crashing scenario crash 100% of the time? Does the working scenario work 100% of the time? If not, it could be an external factor maybe

#

Also:

  • what architecture is the dagger CLI running on?
  • what architecture is the dagger engine container running on?
arctic wind
#

Could it be memory? Is there some memory limit in dagger maybe for a single process, even if the engine has enough memory in total?

crimson bay
# crude crane <@628652364915802112> does the crashing scenario crash 100% of the time? Does th...

Yes, it always fails when using with-exec and never fails when using terminal. I've checked this now over the span of 3 days at least ~10 times for each of them in different constellations, even on different machines, because I still don't want to accept this 😄 . It has the exact same behavior when running dagger on my local machine as when using our build server. Both are x86_64 machines and the whole thing works when I use the same as the target platform. It only fails when using linux/arm64 as the platform of the base docker image - but even then it works when just running the build command in the terminal instead so in general it seems to be possible to use arm64 as the image's platform. Still, it most likely is connected to the ARM64 qemu emulation being used then.

crude crane
#

(when you request a container for another architecture, dagger runs qemu under the hood)

crimson bay