Set sock4/5/http proxy using org.apache.hc.core5?

[floral hawk]

Hi as the title suggests, I'm wondering how to use sock4/5/http proxies using the apache request library. My code is as follows;

[round forge]

⌛ This post has been reserved for your question.

Hey @floral hawk! Please use /close or the Close Post button above when you're finished. Please remember to follow the help guidelines. This post will be automatically closed after 300 minutes of inactivity.

TIP: Narrow down your issue to simple and precise questions to maximize the chance that others will reply in here.

[floral hawk]

package com.scrapium;

import com.scrapium.proxium.Proxy;
import com.scrapium.utils.DebugLogger;
import org.apache.hc.client5.http.impl.async.CloseableHttpAsyncClient;
import org.apache.hc.client5.http.impl.async.HttpAsyncClients;
import org.apache.hc.client5.http.impl.nio.PoolingAsyncClientConnectionManagerBuilder;
import org.apache.hc.core5.http.HttpHost;
import org.apache.hc.core5.http.message.BasicHttpRequest;
import org.apache.hc.core5.http.nio.support.BasicRequestProducer;
import org.apache.hc.core5.http.support.BasicRequestBuilder;
import org.apache.hc.core5.io.CloseMode;
import org.apache.hc.core5.reactor.IOReactorConfig;
import org.apache.hc.core5.util.Timeout;

import java.util.Iterator;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.Future;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.atomic.AtomicInteger;

public class TweetThreadTaskProcessor {

    private final Scraper scraper;
    private final BlockingQueue<TweetTask> taskQueue;
    private final int threadID;
    private BlockingQueue<TweetThreadRequestConsumer> consumerQueue;
    private volatile boolean  tweetThreadRunning;
    private AtomicInteger coroutineCount;

    private int requestCount;
    final IOReactorConfig ioReactorConfig;
    final CloseableHttpAsyncClient client;

    public TweetThreadTaskProcessor(int threadID, boolean running, Scraper scraper, BlockingQueue<TweetTask> taskQueue, AtomicInteger coroutineCount) {
        this.threadID = threadID;
        this.scraper = scraper;
        this.taskQueue = taskQueue;
        this.coroutineCount = coroutineCount;
        this.tweetThreadRunning = running;
        this.consumerQueue = new LinkedBlockingQueue<>();

        ioReactorConfig = IOReactorConfig.custom()
                .setSoTimeout(Timeout.ofSeconds(this.scraper.conSocketTimeout))
                .build();

        // Create a custom connection manager with custom limits

        PoolingAsyncClientConnectionManagerBuilder connectionManagerBuilder = PoolingAsyncClientConnectionManagerBuilder.create()
                .setMaxConnPerRoute(2000) // may be a bottleneck
                .setMaxConnPerRoute(2000);

        client = HttpAsyncClients.custom()
                .setIOReactorConfig(ioReactorConfig)
                .setConnectionManager(connectionManagerBuilder.build())
                .build();

        client.start();
    }

    /*
        Run Continuously
     */
    public void processNextTask() {
        DebugLogger.log("TweetThreadTask: Before attempting to increase request count.");

        try {
            TweetTask task = this.taskQueue.take();
            Proxy proxy = scraper.proxyService.getNewProxy(0);


            coroutineCount.incrementAndGet();

            final BasicHttpRequest request = BasicRequestBuilder.get()
                    .setHttpHost(new HttpHost("httpforever.com"))
                    .setPath("/")
                    .build();

            //System.out.println("Consumer Count: " + this.consumerQueue.size());

            TweetThreadRequestConsumer consumer = new TweetThreadRequestConsumer(this, coroutineCount, scraper);
            this.consumerQueue.add(consumer);

            final Future<Void> future = client.execute(
                    new BasicRequestProducer(request, null),
                    consumer, null);

        } catch (InterruptedException e) {
            e.printStackTrace();
            throw new RuntimeException(e);
        }
    }

    // needs to be updated to accept different types of queries
    protected void removeConsumerFromQueue(TweetThreadRequestConsumer consumer) {
        this.consumerQueue.remove(consumer);
    }
}

.
.
.

I would like to use the Proxy (proxy) object to set the request proxy before calling execute. If possible, this would be per-function call, but I see that with one client, this may not be possible.

[floral hawk]

.