Balancing

Langworks implements various utilities allowing for load balancing resources.

class langworks.util.balancing.BalancedPool

Provides a container for similar reusable resources, managing these resources so that they are all utilized equally.

Fundamentals

__init__(resources: Sequence[T])

Initializes BalancedPool.

Parameters

resources

Collection of resources to add to the pool.

Methods

stack: list[Resource[T]]

Internal stack keeping track of the order in which resource where added.

stack_lock: allocate_lock

Lock to manage access to internal stack, providing thread safety.

property workload: int

Current number of tasks handled by the pool

property workload_average: float

Average number of tasks per resource.

access()

Context manager that retrieves one resource from the queue, automatically calling release() after utilization of the resource has finished.

add(resource: T) → None

Adds a new resource to the pool.

Parameters

resource

Resource to add to the pool.

get() → T

Retrieves least burdened, least recently used resource from the queue.

release(resource: T) → None

Lowers workload associated with the resource, improving its availability when a new resource is requested.

Parameters

resource

Resource for which to lower the workload.

Important

All clients that retrieve a resource using get(), must always call release to ensure that the resource remains managed correctly by the pool. This may also be done automatically by using access()’s context manager.

remove(resource: T) → None

Remove given resource from pool, preventing it from being access through the pool.

remove_last() → None

Removes last added resource (using add()) from the pool.