#How to get custom URL for GeoLite2-City.mmdb?

1 messages · Page 1 of 1 (latest)

tame solstice
#

Follow up question of https://discord.com/channels/921520481163673640/1484584592483553401.

The test framework works successfully except for this part:

crowdsec-full # [   10.433371] crowdsec-setup[960]: level=warning msg="Failed to check last modified: failed to make HEAD request for https://hub-data.crowdsec.net/mmdb_update/GeoLite2-City.mmdb: Head \"https://hub-data.crowdsec.net/mmdb_update/GeoLite2-City.mmdb\": dial tcp: lookup hub-data.crowdsec.net: no such host" url="https://hub-data.crowdsec.net/mmdb_update/GeoLite2-City.mmdb"
crowdsec-full # [   10.435231] crowdsec-setup[960]: downloading https://hub-data.crowdsec.net/mmdb_update/GeoLite2-City.mmdb
crowdsec-full # [   10.435823] crowdsec-setup[960]: Error: cscli collections install: while downloading data for geoip-enrich.yaml: while getting data: failed http request for https://hub-data.crowdsec.net/mmdb_update/GeoLite2-City.mmdb: Get "https://hub-data.crowdsec.net/mmdb_update/GeoLite2-City.mmdb": dial tcp: lookup hub-data.crowdsec.net: no such host
crowdsec-full # [   10.439044] systemd[1]: crowdsec-setup.service: Main process exited, code=exited, status=1/FAILURE
crowdsec-full # [   10.439766] systemd[1]: crowdsec-setup.service: Failed with result 'exit-code'.

I couldn't find any GeoLite2-City.mmdb in the hub and I couldn't find any option for the cscli to set the URL for this geolite file (https://github.com/crowdsecurity/crowdsec/blob/master/pkg/csconfig/cscli.go#L8-L19).

Any suggestions on how I can fix that?

olive vaporBOT
#
Important Information

This post has been marked as resolved. If this is a mistake please press the red button below or type /unresolve

tame solstice
#

hm... could it be that it's just checking if the file exists at a specific part and if I just put a file there, it won't try to download it?

#
name: crowdsecurity/geoip-enrich
description: "Populate event with geoloc info : as, country, coords, source range."
data:
  - source_url: http://127.0.0.1/geolite/GeoLite2-City.mmdb
    dest_file: GeoLite2-City.mmdb
  - source_url: http://127.0.0.1/geolite/GeoLite2-ASN.mmdb
    dest_file: GeoLite2-ASN.mmd

it's still trying to fetch from hub-data.crowdec.net :(

tame solstice
#

I'm trying to understand how cscli parsers install works

#

there are 3 sub-directories in the parsers directory in the hub

#

if I do cscli parsers install crowdsecurity/geoip-enrich, how does it know which directory to use?

tame solstice
#

Ok... claude helped me out here

#

So the reason why it didn't use the new source_url is because of the .index.json file:
So I bascially needed to do the following:

  1. Apply the changes to each file in the hub. For example in my case here, changing the source_url.
  2. Open up hub/src/cshub/mkindex.py and add main() at the bottom of the file to call the function.
  3. Execute uv run src/cshub/mkindex.py and use the new .index.json.
olive vaporBOT
# olive vapor

Resolving How to get custom URL for GeoLite2-City.mmdb?