Great question — answering as the UphillTech team:
You have N workers with names 1 .. N, and they’re not interchangeable once a workflow has begun. So send work pieces to specific named workers. At the very beginning, client asks a load balancing service to assign an available worker. Thereafter, the queue channel identifiers and the URL structure will explicitly name the single worker that is eligible to handle that item of processing. Other possibilities include DB queries, mapping unique TCP ports to individual workers, and waiting on a Redis key that mentions the worker name. For the initial interaction you can use uvicorn in the usual way, load balancing across a bunch of anonymous interchangeable workers. But on subsequent steps in...
Full answer by J_H: https://stackoverflow.stackexchange.com/a/79786619 (CC BY-SA 4.0).
Need it done for your business? UphillTech.
