Issue in scrapping data from this particular url :

[rigid cave]

I was scrapping from this particular url that is : https://training.gov.au/training/details/SHBBHRS007/unitdetails from past few months smoothly to feed my ai system with this data but from past few days i am unable to scrap the data can i know why this is happening to me. I have subscription of firecrawl and still unable to scrape

[pulsar junco]

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

[hexed swan]

Hey! The issue is you're using "timeout": 60000 which isn't a valid parameter. You need "waitFor": 15000 instead.
The training.gov.au site loads content dynamically with JavaScript and needs about 15 seconds to fully render all sections. I tested your exact URL and confirmed this configuration works:

{
  "url": "https://training.gov.au/training/details/SHBBHRS007/unitdetails",
  "skipTlsVerification": true,
  "waitFor": 15000,
  "maxAge": 0
}

This successfully returns the complete page including all the detailed sections (modification history, elements, performance criteria, foundation skills, and assessment requirements) that were previously showing as "Loading...".

[rigid cave]

Thank You So Much Brother ! It really worked from me. I was curious to know how you find this solution so i can have idea for future usage if i stuck in loop

[hexed swan]

Sure! When I scraped the page, I saw the markdown started with "error" and had sections that said:

• "Loading Units of competency Details"
• "Loading Assessment requirements details"
  This told me the page was loading but JavaScript wasn't finishing.

[rigid cave]

I was not using this three parameters which you provided me in my code previously so whats the usage of this and how you found this paramters. When i added this paramters it worked smoothly
"skipTlsVerification": true,
"waitFor": 15000,
"maxAge": 0

[hexed swan]

You can find detailed information about these parameters in our documentation: https://docs.firecrawl.dev/introduction

[rigid cave]

Thank You brother so much ! it really helped me a lot