placid shoal
Hello all,
I am trying to create an proxy in spring boot, which I can use my browser and navigate the internet thought this proxy. My spring proxy can successfully interpret http requests though it cannot process https requests. The main purpose of this spring program is to create an proxy which I can later use for web scraping purposes. I use as you can see the spring.cloud.gateway. I wan to consume (if this is the correct term) the https requests that I get from web-browser or curl lets say and not get this errors.
Good request http
curl -x localhost:8080 http://publish.illinois.edu/johnrgallagher/scrape/
<!DOCTYPE html>
<!--[if IE 7]>
<html class="ie ie7" lang="en-US">
...
# Bad request https
curl -x localhost:8080 https://webscraping.ai/
curl: (56) Received HTTP code 400 from proxy after CONNECT
This is the controller I made so far.
@Controller
public class WebProxyController {
@RequestMapping(value = "/**")
public ResponseEntity<?> proxy(ProxyExchange<Object> proxy, HttpServletRequest request) {
final String url = Utils.constructURLFromHttpServletRequest(request);
// Create proxy exchange
ProxyExchange<?> proxyExchange = proxy.uri(url);
// Get the response entity (capture response)
ResponseEntity<?> responseEntity = proxyExchange.get();
return responseEntity;
}
}
Thank you, enjoy the rest !
nocturne bough