supple marsh
Hi guys, I plan to deploy my crawler (a ParselCrawler one) to AWS Lambda. I'm loosely following this guide which is for JavaScript though. I'd like disable persising the storage. I change the configuration like this:
config = Configuration.get_global_configuration()
config.persist_storage = False
and also tried to supply the configuration to the ParselCrawler constructor, neither of these works though. I'm still getting the storage directory created. Am I doing something wrong here?
Thanks
boreal grove
