#Permission denied on K3s /var/log/containers

1 messages · Page 1 of 1 (latest)

mighty tinsel
#

Hi all, I am trying to deploy CrowdSec to my K3s cluster that runs Traefik as the ingress controller. I managed to get everything running (lapi and agents), but no signals showed up in the console. After some log reading, I found out that the issue is:

2026-02-20T16:06:09.895123817Z time="2026-02-20T16:06:09Z" level=error msg="Could not create watch on directory /var/log/containers : permission denied" module=acquisition.file type=file

I tried a LOT of things, including mounting the /var/log/containers directory myself, even tried using a /var/log/pods hostPath with a custom file acquisition to /var/log/pods/traefik_traefik-*/traefik/*.log. That did find the logfile still, but also could not create the watch. I tried with and without inotify, but nothing seemed to help. In the Helm chart there is no option to setting the fsGroup or fsUser (securityContext), that might theoretically be the issue, but before trying to create a custom DaemonSet and skipping Helm, I wanted to check-in here.

Current values.yaml for 0.22.0 of the Helm chart:

container_runtime: containerd

agent:
  acquisition:
    - namespace: traefik
      podName: traefik-*
      program: traefik
  env:
    - name: COLLECTIONS
      value: "crowdsecurity/traefik"

... lapi / config stuff ...

Any help would be greatly appreciated.

warm hareBOT
#
Important Information

This post has been marked as resolved. If this is a mistake please press the red button below or type /unresolve

mighty tinsel
#

This section walks you through troubleshooting acquisitions that are not working as expected.

We will add a file-based acquisition. If you need a different source, adjust the instructions to match your setup.

GitHub

CrowdSec community kubernetes helm charts. Contribute to crowdsecurity/helm-charts development by creating an account on GitHub.

#

Oh and Traefik is set to output access logging:

traefik-values.yaml
... other stuff ...
logs:
  general:
    level: INFO
  access:
    enabled: true
    format: json
    fields:
      headers:
        defaultmode: keep
... other stuff ...
mighty tinsel
#

Issue resolved, the problem was SELinux on my nodes not allowing read access to /var/log. This fixed it:

SELINUX_TMP=$(mktemp -d)
printf '%s\n' \
      '(allow container_t var_log_t (dir (open read search watch)))' \
      '(allow container_t var_log_t (file (getattr open read)))' \
      '(allow container_t var_log_t (lnk_file (getattr read)))' \
      '(allow container_t container_log_t (dir (open read search watch)))' \
      '(allow container_t container_log_t (file (getattr open read)))' \
      '(allow container_t container_log_t (lnk_file (getattr read)))' \
> "$SELINUX_TMP/allow_container_varlog.cil"
cat $SELINUX_TMP/allow_container_varlog.cil
semodule -i "$SELINUX_TMP/allow_container_varlog.cil"
rm -rf "$SELINUX_TMP"
warm hareBOT
# warm hare

Resolving Permission denied on K3s /var/log/containers