Specialised pipeworks

Pypeworks ships with various pipeworks tailored for specific, commonly recurring tasks.

class pypeworks.pipeworks.Replicator(node: Node[T], replications: int, processes: int = None, nocopy: bool = False)

A Replicator is a procedually generated pipework generated from a single pypeworks.Node, whereby said node is replicated in parallel together with a network of connections that distributes input across these replications. Replicators are ideally suited to scale nodes that have a long execution time.

__init__(node: Node[T], replications: int, processes: int = None, nocopy: bool = False)

Instantiates a new Replicator.

Parameters

node

Node to replicate.

replications

The number of replications to generate.

processes

The number of worker processes to use to operate the underlying pypeworks.Pipework. By default this number is equal to the number of logical CPUs in the system.

nocopy

Whether or not to asign each replication its own copy of the node. By default each replication is assigned a deep copy of the given node. However, if desired this behaviour can be disabled, allowing for data sharing between the replications. Do so only if the given node can operate thread-safe.