frosty kernel
Hey!
I'm trying different proxy providers and I've noticed the issue in the title.
I'm setting the proxy in proxyUrls in the following format:
http://user:pass@host:port as I usually do. But with the current providers I'm testing, the request will fail with either 407 (Proxy Authentication Required) or 422 responses.
Strangely enough, if tried with curl -x 'proxy string from the same providers, in the same format' https://example.com - it works.
Any idea what could be causing it?
green lotus
Can you provide the target URL? The syntax looks correct
frosty kernel
example.com or amazon
green lotus
Can you share more of the code? No one ever reported this so it is unlikely it would be a bug in Crawlee
frosty kernel
Same proxy string works perfectly fine with curl, as mentioned above. Not sure why it "asks for authentication". Let me grab a screenshot with the resi proxies as well
There you go
Ahh, I think I know why it happens. There's a semicolon in the username and/or password in both cases. Let me see how should I escape it
austere elkBOT
@frosty kernel just advanced to level 2! Thanks for your contributions! 🎉
frosty kernel
Can't seem to do so, though. Should not need to escape semicolon as it's not really a special character, but Crawlee is not happy with it :\
And that's with Cheerio. Same formatting works just fine with Playwright
green lotus
I think I remember this issue but don't remember how I solved it 😄