#Retry after 30 seconds for failed requests

1 messages · Page 1 of 1 (latest)

ebon osprey
#

Is there a way that we can retry the failed requests after 30 seconds for cheerioCrawler

fresh flowerBOT
#

@ebon osprey just advanced to level 1! Thanks for your contributions! 🎉

robust gust
#

You have to set up requestHandlerTimeoutSecs: 30,

ebon osprey
#

@robust gust But the definition says - Timeout in which the function passed as requestHandler needs to finish, in seconds.

#

it is the total timeout for the requestHandler right ?

#

@robust gust Thanks i think we need to use errorHandler function
errorHandler: async ({request}) => {
log.info(Error occurred for URL - ${request.url}: ${request.errorMessages});
log.info(Waiting 30 seconds for URL - ${request.url});
// Wait for 30 seconds before retrying
await new Promise(resolve => setTimeout(resolve, 30000));
// Return true to indicate that the error should be retried
return true;
}