#How to expose OTEL metrics for collector

1 messages · Page 1 of 1 (latest)

velvet badge
#

Hello I'm trying to expose metrics to otel collector, but only thing im able to receive is traces. I do have dagger 0.19.2 version with env engine env variables:

      - name: _EXPERIMENTAL_DAGGER_METRICS_ADDR
        value: ":9090"
      - name: OTEL_EXPORTER_OTLP_METRICS_PROTOCOL
        value: http/protobuf
      - name: OTEL_EXPORTER_OTLP_ENDPOINT
        value: "<address>:4318"
      - name: OTEL_METRICS_EXPORTER
        value: otlp

Am I missing something ?

velvet badge
#

I have tried multiple different variants with environment variables, but none of them were success

cobalt orchid
#

Hey @velvet badge! I haven't tested this myself, but have you tried configuring OTEL_EXPORTER_OTLP_METRICS_ENDPOINT?

Not all OTLP endpoints support metrics so we have a separate env variable to configure metrics

velvet badge
#

Hey, yes. I did tried that, as well as all other combinations that could be found in source code. I successfully received traces.

#

Although interestingly I found that these metrics are logged as simple logs every now and then, most of the metrics are there but not the full set. So as workaround Im using these for now, but would be better to use normal metrics flow

nimble wing
velvet badge
#

Do I get any metrics? From other services - yes, but not from dagger engine

cobalt orchid
#

I just checked running an otel-collector and making a dagger call, I can see that metrics are arriving (screenshot attached). This is how I ran the collector:

docker run -d \
  --name otel-collector \
  -p 4317:4317 \
  -p 4318:4318 \
  -v $(pwd)/config.yaml:/etc/otelcol-contrib/config.yaml \
  otel/opentelemetry-collector-contrib:latest

The config:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

exporters:
  debug:
    verbosity: detailed 

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [debug]
    metrics:
      receivers: [otlp]
      exporters: [debug]

This is how I setup and ran the dagger call command:

export OTEL_EXPORTER_OTLP_ENDPOINT="http://127.0.0.1:4318"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_EXPORTER_OTLP_METRICS_ENDPOINT="http://127.0.0.1:4318"
export OTEL_EXPORTER_OTLP_METRICS_PROTOCOL="http/protobuf"
dagger call <...>
#

Not sure if those are the metrics you are looking for or if you are looking for something else?