So far running Dagger standalone, | Dagger | Page 1

vapid locust May 20, 2025, 2:04 PM

#

Hi, I just finalize the setup you recommend me with xfs and deploy directly the dagger but I can't see any improvement, it's not worse but not better it's the same time per step

junior sparrow May 20, 2025, 2:37 PM

#

Hi!

What kind of disk storage do you have on the instances? Are you using NVMe?

vapid locust May 20, 2025, 4:43 PM

#

the node have this disk:

  blockDeviceMappings:
    - deviceName: /dev/xvda
      ebs:
        volumeSize: 30Gi
        volumeType: gp3
        encrypted: true
        deleteOnTermination: true

the pod with dagger have gp3 formated with xfs mounted at /var/lib/dagger
using this storage class:

allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  creationTimestamp: "2025-05-19T16:51:47Z"
  name: gp3-xfs
  resourceVersion: "102717206"
  uid: 4e26bbe5-19fb-41eb-bb96-9495dfda37c2
parameters:
  encrypted: "true"
  fsType: xfs
  type: gp3
provisioner: ebs.csi.aws.com
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

junior sparrow May 20, 2025, 7:22 PM

#

I see. While its true that SSDs are faster than the old gp2 block storage, they are still annoyingly slow for disk-intensive operations. We are currently using c6id instances that have really fast builtin NVMe storage. This does have a big disadvantage which is long term storage. Since this is ephemeral storage built into the node itself you'll loose it once the node goes away.

You could do a manual process of copying it over and then restoring it, but that sounds like a nightmare to implement and wont be that robust. Were you planning on having long term storage?

vapid locust May 20, 2025, 10:03 PM

#

for the moment no it's only ephemeral storage

vapid locust May 21, 2025, 8:57 AM

#

ok I have a better idea of what to do I think when I was reading you articles + setup I miss something + confusing the xfs on the ssd with the storage class with gp3.
I have a better idea of what to do, I keep you inform when I did the change

vapid locust May 22, 2025, 4:09 PM

#

Hi, FYI now I have something working very well thank you for you help

#

I did a similar setup than in your article + github repo but with terraform instead of argocd

#

with worker size

#

I just a bit the user data script in order to be resilient in case of the device name change:

#!/bin/bash
    set -euxo pipefail
    
    sudo yum install nvme-cli -y
    # Find the NVMe device labeled as "Instance Storage"
    INSTANCE_NVME=$(sudo nvme list | grep "Amazon EC2 NVMe Instance Storage" | awk '{print $1}')
    
    # Exit if not found
    if [ -z "$INSTANCE_NVME" ]; then
      echo "No instance store NVMe device found."
      exit 1
    fi
    
    # Format, mount
    sudo mkfs -t xfs "$INSTANCE_NVME"
    sudo mkdir -p /var/lib/dagger
    sudo mount "$INSTANCE_NVME" /var/lib/dagger

#So far running Dagger standalone,