For Web Scraping can i use cloudflare workers as a reverse proxy? Does it really help to prevention of ip block of target server? (want to use it for https://pypi.org/project/instagrapi)
#Cloudflare Workers proxy for Web Scraping?
snow ore
muted cobalt
For Web Scraping can i use cloudflare workers as a reverse proxy? Does it really...
Workers aren't meant for that. They are made the opposite if anything, in the CF-Worker header it identifies the website/zone that send the request.
Also Workers at the moment don't support Python really. You can do some stuff with transpiling to JS or maybe even compiling to WASM but it's not really supported/not going to find any docs on it/etc
snow ore
i just want to use workers as a proxy as i get an option while creating a new project in wrangler
muted cobalt
You can use Workers to Reverse Proxy to stuff you own yea. It can be really helpful with rewriting/dealing with weird stuff
snow ore
but main question is Does it really help to prevention of ip block of target server? (want to use it for https://pypi.org/project/instagrapi)
*target server = main server
muted cobalt
I'm not sure ToS wise but like I said they're just not made for that. Every fetch request you do in a Worker identifies the website it came from in the CF-Worker header, so people could easily see which user is doing it. The IPs they use if the target is a CF is the same 2a06:98c0:3600::103, otherwise they use a smallish set set of Proxy IPs
Some websites like Discord block all workers from fetching their CDN by blocking any request with the CF-Worker header (and you can't remove that)
snow ore
I am not worried about my server's ip because I don't do any illegal stuff just try to prevent my server's ip from being blocked from Instagram servers that's why (for image downloader app)
last question atleast my server ip will be isolated or not if i use it according to you???
are you there??
quiet patrol
I mean your worker will use the same IP as other people that use workers so depends what you mean by isolated