How to load balance actix using rust scripting?

[acoustic junco]

I've been developing an actix-web application for the last few weeks. Actix being multithreaded is proving to be a complete pain - I'm building a system that uses HTTP to proxy data, and I'd really rather have one thread per session token, but there's no way to do that in Actix. I've almost got it working but it's such a mess I've decided I need to rewrite it.

I'd like to have a separate instance (be it via threading or a whole separate application) of Actix (or some other web framework? Is there one that is faster in single-threaded mode? I don't need many bells and whistles) per session, with requests load-balanced based on that token header. If a token is unknown it should check it's legitimate (via decrypting its payload) then start a new instance for it.

How would you recommend going about this? I really don't want to rewrite it twice.

[acoustic junco]

Putting together some rough notes:

I'm thinking of architecting it as:
- Main function
- This function starts a Actix instance
- The Actix instance receives all requests to /submit because reverse proxy
- It then checks, and if it's clear, it creates a new thread
- This new thread starts a new tokio instance, and then a new single-threaded actix instance, which stores all of its state (message passers etc) local to the thread
- The new thread has a function for bidirectional TCP using message passing for that stream
- And a function for going through a upstream request, spawning tasks of the above/message passing as need be
- And a function (run as a task) for going through the results of all the TCP streams, continuously forming them into our new response format, and when modetime/buffersize says it should go, it sends it on a flume message passer to the awaiting downstreams

Am I missing anything obvious?

The part I'm missing is a way to pass on the requests to the correct port (because each actix instance will run on a separate port) based on a header, in a fast and performant way that can be changed in real-time as new sessions come and go. The perfect thing would be a embeddable rust reverse proxy, but that doesn't seem to exist.

[hidden zenith]

i believe this is not for you to worry about. Actix promotes full tokio support.

tokio is a work-stealing executor which means that it is already putting effort towards balancing load of tasks between all threads.

[acoustic junco]

Thanks for the reply. Sorry about the following wall of text:

Yeah, there's a bit of a problem though - multithreading messes everything up because I can't have references to anything, anywhere... it's making my code a complete mess with all the workarounds. Maybe it's possible to deal with it, but at my current level of rust knowledge I am unable to make a decent program with multithreading per-user. It's quite a lot to explain, though I can send the code if you like. As an example, I would like to have a task running on my TransitSocket (which is pretty much a class), but since there's no guarantee of the lifetime of &self, I can't have a reference to it. &self contains the very thing I actually care about - the vector of message passers for returning data, and the message passers for data from exterior connections. I was hoping with a single threaded system I could just have this in a mutable global and not worry about those problems. I know it'll last long enough, but I have no way to convince the compiler.

Additionally, my code is very data-heavy and I'm a bit worried the amount of data passing between threads will end up being a bottleneck.

I just started trying to implement this and I'm finding that despite starting actix with 1 worker it's still acting like a multi-threaded program, meaning that the benefits I was hoping for aren't there... maybe I should implement my own single-threaded system in hyper? Are there any exclusively single-threaded frameworks? Or am I missing the correct way entirely?

To elaborate on what exactly I'm doing - I'm running socks over HTTP GET/POST requests without request streaming for a niche firewall circumvention method. When implemented in Python it's like 100 lines for the server, the actual thing I'm doing is pretty simple, the problem is multithreading and the borrow checker preventing me from doing it. I'd like to simplify the problem through making it single threaded, with global, mutable data due to it being single threaded (and so that shouldn't be a problem)

[hidden zenith]

with async in general its difficult to pass references around. Its hard to prove that task b will never outlive task a.

The problem is possible to solve with the concept of scopes such that if

a:

• b
• c

b and c will be destroyed if a is destroyed.

This is not possible with work stealing executors like tokio however. You would in this case need an executor that has a per thread context.

potential solutions:

Right now in the world of async, we are not built for structured concurrency but for actors with channels.
Typically, you will dedicate a task as an owner of some data and other tasks can send a request over a channel for some information and will respond over a oneshot.

Wrapping all shared data types in an Arc<Mutex or similar

[acoustic junco]

Thanks for the info. Have you got any specific recommendations for setting up an actor-y system?

[hidden zenith]

i haven't written anything in async for a while. Let me draft out some traits later

[acoustic junco]

Don't worry about it, I'll look into it. Thank you for all the help.

[hidden zenith]

pub struct SomeData { }

pub struct GetUser {
    username: String,
    respond: mpsc::Sender<String>,
}

pub struct Response<R> {
    receiver: mpsc::Receiver<R>
}

impl GetUser {
    fn new(username: String) -> (Self, Response<String>) {
        // use a oneshot in normal cases
        let (sender, receiver) = mpsc::channel(1); 

        (
            Self {
                username,
                respond: sender,
            },
            Response {
                receiver,
            }
        )
    }
}

#[async_trait]
impl RespondFromRef for GetUser {
    type T = SomeData;

    async fn respond_with_ref(&mut self, t: &Self::T) {
        let _ = self.respond.send("Response from actor".to_string());
    }
}

#[async_trait]
pub trait RespondFromRef {
    type T;

    async fn respond_with_ref(&mut self, t: &Self::T);   
}

#[tokio::main]
async fn main() {
    let (request, response): (GetUser, Response<String>) = GetUser::new(String::new());

    /// actor task could have a `Receiver<Box<dyn ....>>` in this case and response in a generic
    /// manner
    let mut boxed: Box<dyn RespondFromRef<T = SomeData>> = Box::new(request);

    let some_data = SomeData { };

    /// actor task would response like this
    boxed.respond_with_ref(&some_data).await;
    
    /// Receive the response in another task
    let response: Option<String> = response.receiver.recv().await;
}

an interface like this is very crude but shows the core concept. It would need to be modified for making changes to SomeData but the core functionality is the same.