Katana Output JSONL : The method request is a POST but in the raw field there is a GET

[obsidian plover]

Hi all !

I performed a crawl with these options:
katana -u https://*** -timeout 30 -headless -js-crawl -xhr -fs dn -depth 1 -aff -fx -j | jq . > crawl.json

I've noticed that in the output file crawl requests using the POST method unfortunately have the GET method in the raw data field.

The problem is that when I use this file as input for nuclei, the request is generated using the GET method and not POST. 😦 (cf. picture)

Is this a misunderstanding on my part or a bug? In addition, these POST requests are generated by the crawl with the -aff option.

Thank you in advance for your help and have a nice weekend! 🙂

[storm steeple]

Hey @obsidian plover ! Welcome to the Discord 😄 checking this out for you with the team, bear with us.

(hope you had a nice weekend too!)

[obsidian plover]

Thank you very much @storm steeple !

Ditto I noticed that if a JSONL file is provided as input to nuclei. Requests containing the body field are not taken into account when creating Nuclei's HTTP request. Only the raw field is used to create the Nuclei request.

To mitigate this problem, I had to create a parser on the JSONL files that concatenates the content present in the body into the raw field before using the file with Nuclei.

In the Nuclei code, in the function func (j *JSONFormat) Parse(input string, resultsCb formats.ParseReqRespCallback) in json.go, the body field is not considered, as shown by the call to this function:
rawRequest, err := types.ParseRawRequestWithURL(request.Request.Raw, request.URL)

I'm not sure whether this is the desired behavior or whether it's possible to add a condition to request.Request.Body. In that case, it would be sufficient to concatenate it to request.Request.Raw if the field is not empty.

Thanks in advance for your feedback.

[storm steeple]

Hey @obsidian plover ! Thanks so much for waiting so far. While we're taking a look, could you go ahead and make a GH issue for this? Or we're happy to make the issue for you if you prefer

[obsidian plover]

Hey @storm steeple, no worries. I will try to create the issue today ! 😉