slender hawk
Hi,
I use Crawlee for a few months and I greatly appreciate it;-)
I face bot detection problem by Imperva for a few days.
Solutions such as Zenrows or Brightdata are able to pass this protection.
But they are "not economically adapted" in my case 😉
Is there a place to post evolution requests and a description of the problem?
Thanks
ocean spear
What proxy you are using? If you blocked from your own IP its expected case and that's why proxies always recommended for scraping
slender hawk
Using domestic or residential do not change the problem.
It's works just fine to chain manually 2 URLs with Chrome. (first one is homepage to get reese84 cookie (kind of authorization cookie) and the second one to get datas)
But it does not work with Crawlee or Puppeteer stealth although it works fine with brightdata or zenrows.
=> That's why it may be a fingerprint problem?
ocean spear
BrightData is proxy, so if it works with BD means it works with proxy, right? Puppeteer might be detected and blocked, so try with Playwright, see if targeted page reached
slender hawk
As I tried with Crawlee with Playwright, I don't think it will work neither.
waxen bay
How you use BrightData? Playwright + Firefox + good proxies is usually the beset
slender hawk
Thanks Lukas!
I'm testing a new service of BrightData but it's too expensive.
Manually sequencing 2 URLS works with my domestic IP and Chrome but not with Firefox!