Appsec : ban decision clarification

[unique garnet]

Hello,

I'm testing Crowdsec. I put crowsed engine (LAPI) + appsec (WAF) on a Vm. In a second VM, I installed nginx with only the bouncer/remediation component and in a third Vm, only the remediation component for IPTABLES.

These 2 VMs access the first VM's LAPI (8080) and WAF (7422).

I'm testing with a http://IP_VM_NGINX/.env and http://IP_VM_NGINX/rpc2.

I have alerts and blocks (code 403) in nginx, but there is no remediation or ban decision.So the iptables bouncer doesn't block the ip.

I've read that you need 2 different alerts, but I've tried using kali to generate alerts, but nothing.I even modified the scenario using ‘trigger’...

Can anyone help me ? Thanks 🙂

[finite walrus]

Important Information

Thank you for getting in touch with your support request. To expedite a swift resolution, could you kindly provide the following information? Rest assured, we will respond promptly, and we greatly appreciate your patience. While you wait, please check the links below to see if this issue has been previously addressed. If you have managed to resolve it, please use run the command /resolve or press the green resolve button below.

Log Files

If you possess any log files that you believe could be beneficial, please include them at this time. By default, CrowdSec logs to /var/log/, where you will discover a corresponding log file for each component.

Guide Followed (CrowdSec Official)

If you have diligently followed one of our guides and hit a roadblock, please share the guide with us. This will help us assess if any adjustments are necessary to assist you further.

Screenshots

Please forward any screenshots depicting errors you encounter. Your visuals will provide us with a clear view of the issues you are facing.

© Created By WhyAydan for CrowdSec ❤️ · Apr 24, 2025, 8:32 AM

[neon gale]

Most likely you IP is internal and by default all internal ips are whitelist, if you check cscli metrics do you see an increment on the private ranges?

[unique garnet]

Thanks for the answers.

I'm testing with VMs (vagrant, etc.) in a private environment.

I've run these commands to avoid simulation:

sudo cscli simulation disable --global
sudo cscli parsers remove crowdsecurity/whitelists
sudo systemctl reload crowdsec

Should this apply the ban (at least have the entry in the list of decisions), even for private IPs?

[neon gale]

as long as sudo cscli parsers remove crowdsecurity/whitelists is removed then yeah the ban should propagate note though the scenario has a fast leakspeed so the request need to be right after reach other

[unique garnet]

On the screenshots, the command used since the second VM to trigger alerts, alerts, parsers, scenarios...

Note that I have not touched the configuration of the scenarios

[neon gale]

and cscli metrics?

[unique garnet]

here their are

[neon gale]

the whitelist is loaded in memory

╭─────────────────────────────────────────────────────────────────────────────╮
│ Whitelist Metrics                                                           │
├──────────────────────────┬─────────────────────────────┬──────┬─────────────┤
│ Whitelist                │ Reason                      │ Hits │ Whitelisted │
├──────────────────────────┼─────────────────────────────┼──────┼─────────────┤
│ crowdsecurity/whitelists │ private ipv4/ipv6 ip/ranges │ 9    │ 9           │
╰──────────────────────────┴─────────────────────────────┴──────┴─────────────╯

did you remove the whitelist of the appsec node cause its local to each node

[unique garnet]

I have

[neon gale]

Reload sometimes doesnt remove parsers, can you sudo systemctl restart

[unique garnet]

I restarted it. Still no descision

[neon gale]

It seems you are missing parsers

can you run sudo cscli collections install crowdsecurity/linux

cause it seems to geoip enricher is missing

[neon gale]

and it needs for the second stage else it never hits scenarios

[unique garnet]

"Nothing to do"

[neon gale]

and just so we can test cscli version says

[unique garnet]

root@crowdsec-deb12:/etc/nginx/sites-enabled# cscli version

version: v1.6.8-debian-pragmatic-amd64-f209766e
Codename: alphaga
BuildDate: 2025-03-25_14:51:10
GoVersion: 1.24.1
Platform: linux
libre2: C++
User-Agent: crowdsec/v1.6.8-debian-pragmatic-amd64-f209766e-linux
Constraint_parser: >= 1.0, <= 3.0
Constraint_scenario: >= 1.0, <= 3.0
Constraint_api: v1
Constraint_acquis: >= 1.0, < 2.0
Built-in optional components: cscli_setup, datasource_appsec, datasource_cloudwatch, datasource_docker, datasource_file, datasource_http, datasource_journalctl, datasource_k8s-audit, datasource_kafka, datasource_kinesis, datasource_loki, datasource_s3, datasource_syslog, datasource_victorialogs, datasource_wineventlog

[small surge]

Hello, catching up 🙂

The initial WAF alerts didn't trigger the scenario because those are private IPs that are discarded by the crowdsecurity/whitelists parser, but seems you removed it. Can you share your modified appsec scenario ? (crowdsecurity/appsec-vpatch)

[unique garnet]

here it is.

root@crowdsec-deb12:/etc/nginx/sites-enabled# cat /etc/crowdsec/scenarios/appsec-vpatch.yaml

type: leaky
format: 3.0
name: crowdsecurity/appsec-vpatch
description: "Identify attacks flagged by CrowdSec AppSec"
## See appsec-native.yaml for reasons why we created a negative startsWith here, we want to ignore is native_rules but catch any of our DSL rules.
filter: "evt.Meta.log_type == 'appsec-block' && evt.Meta.rule_name not startsWith 'native_rule'"
distinct: evt.Meta.rule_name
leakspeed: "60s"
capacity: 1
groupby: evt.Meta.source_ip
blackhole: 1m
labels:
  service: http
  confidence: 3
  spoofable: 0
  classification:
    - attack.T1110
  label: "Blocked by CrowdSec AppSec"
  behavior: "http:exploit"
  remediation: true

[small surge]

ok so it's still a "leaky" bucket, meaning that you need to trigger two distinct rules (within 60s each) to trigger a ban.

let me try to reproduce if something else would prevent private IPs from triggering a ban

do you have the possibility to do it from non-private IPs ? just to check if the setup is broken for some other reason 🙂

on my hand, I tried modifying the whitelist parser to remove the exception on loopback :

# cat /etc/crowdsec/parsers/s02-enrich/whitelists.yaml 
name: crowdsecurity/whitelists
description: "Whitelist events from private ipv4 addresses"
whitelist:
  reason: "private ipv4/ipv6 ip/ranges"
  ip: 
    - "::1"
  cidr:
#    - "127.0.0.0/8"
    - "192.168.0.0/16"
    - "10.0.0.0/8"
    - "172.16.0.0/12"

and when doing so, I manage to trigger the decision

curl  -k https://127.0.0.1/.env
curl  -k https://127.0.0.1/.git/config

time="2025-04-24T13:03:43Z" level=info msg="AppSec block: crowdsecurity/vpatch-env-access from 127.0.0.1 (127.0.0.1)"
time="2025-04-24T13:03:43Z" level=info msg="(3ef52352a7e54c92b4394646a32bc095auADzhivHUYuhyJb) alert : crowdsecurity/vpatch-env-access by ip 127.0.0.1 (/0)"
time="2025-04-24T13:03:48Z" level=info msg="AppSec block: crowdsecurity/vpatch-git-config from 127.0.0.1 (127.0.0.1)"
time="2025-04-24T13:03:48Z" level=info msg="Ip 127.0.0.1 performed 'crowdsecurity/appsec-vpatch' (2 events over 5.126035594s) at 2025-04-24 13:03:48.594838214 +0000 UTC"
time="2025-04-24T13:03:48Z" level=info msg="(3ef52352a7e54c92b4394646a32bc095auADzhivHUYuhyJb) alert : crowdsecurity/vpatch-git-config by ip 127.0.0.1 (/0)"
time="2025-04-24T13:03:48Z" level=info msg="(3ef52352a7e54c92b4394646a32bc095auADzhivHUYuhyJb/crowdsec) crowdsecurity/appsec-vpatch by ip 127.0.0.1 : 4h ban on Ip 127.0.0.1"
time="2025-04-24T13:03:49Z" level=info msg="Signal push: 3 signals to push"

-> time="2025-04-24T13:03:48Z" level=info msg="(3ef52352a7e54c92b4394646a32bc095auADzhivHUYuhyJb/crowdsec) crowdsecurity/appsec-vpatch by ip 127.0.0.1 : 4h ban on Ip 127.0.0.1"

[unique garnet]

I can try to test both (changing the whitelist conf an make a test with a public IP (will took time)

[small surge]

I would tend to believe that for some reason the whitelist parser hasn't been disabled. After removing it, restarting crowdsec and performing some requests, if you run cscli metrics do you still see it appear at the bottom ?

[unique garnet]

No, after "systemctl restart crowdsec", no more metrics #1364881713674584115 message

If a I comment the range "10.0.0.0" int eh whitelist, and restart, and make the curl, the ban appears

I will no try to make a test with a "public" IP

[small surge]

so if the whitelist is still present, it means it wasn't disabled

[unique garnet]

No, I reinstall for this test. The file was not there

[unique garnet]

From a public IP (a secondary IP added to the NIC), it works.

So , it did not works from an private IP, if I remove the whitelist parser and restart the engine

[neon gale]

ohhh

I know why

we fail to set a timestamp so it fails dateparse enrich

and the geoip module is set to ignore private ranges

since it fails all s02 pipeline as a private ip it doesnt reach scenarios

[unique garnet]

@neon gale so It is somthing Crowdsec have to correct ?

[neon gale]

Well for testing yes it sucks, it works in real life (WAN ranges) it just not for appsec itself.

[unique garnet]

@neon gale ok 👍 because I was searching searching.... 🙂

[neon gale]

I know it is confusing and we shall improve

[unique garnet]

@neon gale 👍