public static void main(String[] args) {
new Headers().foo();
}
public void foo() {
try (CloseableHttpClient client = HttpClients.createDefault()) {
HttpGet get = new HttpGet("https://www.scrapethissite.com/pages/advanced/?gotcha=headers");
get.setHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
get.setHeader("Accept-Encoding", "gzip, deflate, br, zstd");
get.setHeader("Host", "www.scrapethissite.com");
get.setHeader("Referer", "https://www.scrapethissite.com/pages/advanced/");
get.setHeader("User-Agent", "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:144.0) Gecko/20100101 Firefox/144.0");
try (ClassicHttpResponse resp = client.executeOpen(null, get, null)) {
int responseCode = resp.getCode();
System.out.println(responseCode);
if (responseCode == 200) {
System.out.println(EntityUtils.toString(resp.getEntity(), "UTF-8"));
}
}
} catch (IOException ex) {
ex.printStackTrace();
} catch (ParseException ex) {
ex.printStackTrace();
}
}
I'm trying to fetch data from https website. When connecting, I get 200 response code, so the connection was succesfull. The thing is that I can't really decode the response.
These are the errors:
"java.security.NoSuchAlgorithmException: XDH AlgorithmParameters not available"
"java.io.EOFException: SSL peer shut down incorrectly"
"javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake"
"java.io.IOException: Stream interrupted" (this one many times)