Crawl using the same tab and session | Apify & Crawlee | Page 1

soft apex Mar 4, 2023, 10:56 AM

#

🤔 I think you could limit the number of currentbrowser/tabs/crawlers depends of what crawler you are using

#

if not, do it in a simple script with puppeteer or cheerio

#

https://crawlee.dev/docs/guides/scaling-crawlers#minconcurrency-and-maxconcurrency you can set min: 1 and max:1 so that would limit ur crawler to use 1 single tab I think 🤔

Scaling our crawlers | Crawlee

To infinity and beyond! ...within limits

hybrid pulsar Mar 4, 2023, 11:00 AM

#

I'm already using min and max concurrency as 1, it still opens a new browser for every request and a new session according to the debug logs :/

soft apex Mar 4, 2023, 11:00 AM

#

then I would suggest to use puppeteer or playwright in 1 single script if you want to do it now and fast

surreal saffronBOT Mar 4, 2023, 11:00 AM

#

@soft apex just advanced to level 1! Thanks for your contributions! 🎉

hybrid pulsar Mar 4, 2023, 11:02 AM

#

I am in fact using playwright in crawlee /apify

soft apex Mar 4, 2023, 11:04 AM

#

https://crawlee.dev/api/core/class/Session#Constructors 🤔

Session | API | Crawlee

Sessions are used to store information such as cookies and can be used for generating fingerprints and proxy sessions.
You can imagine each session as a specific user, with its own cookies, IP (via proxy) and potentially a unique browser fingerprint.
Session internal state can be enriched with custom user data for example some authorization toke...

#

you can set the sessions and cookies I guess

hybrid pulsar Mar 4, 2023, 11:05 AM

#

already using persistent sessions

surreal saffronBOT Mar 4, 2023, 11:05 AM

#

@hybrid pulsar just advanced to level 1! Thanks for your contributions! 🎉

#Crawl using the same tab and session