push.push

class push.push.PusH(mk_module: Callable, *args, cache_size=4, view_size=4, multi=False)

Bases: Waitable

PusH Distribution.

1. Create a Push Distribution which approximates a distribution on nn’s parameters via particles.

2. Create arbitrary number of particles (pinit). Particles execute concurrently of other particles

num_particles() → int

Returns number of particles.

Returns:

The number of particles.

Return type:

int

p_create(mk_optim: Callable, mk_scheduler: Callable, prior=False, train_key=-1, device=0, receive={}, state={}) → int

Create a particle

Parameters:

• mk_optim (Callable) – Optimizer for updating parameters. Can be None.

• device (int, optional) – Device to put particle on. Defaults to 0.

• receive (dict, optional) – Dictionary containing messages that this particle will respond to. Defaults to {}.

Returns:

Particle identifier.

Return type:

int

p_launch(pid_to: int, msg: str, *args, sync=False) → PFuture

Launch a particle.

Parameters:

• pid_to (int) – Identifier of particle that is the main entry point.

• msg (str) – Message associated with main function

• sync (bool, optional) – Async. Defaults to False.

Returns:

Calling wait will get the result of the computation.

Return type:

PFuture

p_parameters(pid: int, sync=True) → int | Iterable[Tensor]

_summary_

Parameters:

• pid (int) – Identifier of particle to obtain parameters of.

• sync (bool, optional) – Obtain parameters. Defaults to True.

Returns:

Parameters

Return type:

Union[int, Iterable[torch.Tensor]]

p_wait(futures: list[push.pfuture.PFuture]) → dict[int, any]

particle_ids() → List[int]

Returns all particles.

Returns:

List of all particle identifiers visible to current particle.

Return type:

List[int]

save(sync=True)

push.push.init_node_event_loop(mk_module: Callable, args: List[any], in_queues: Dict[int, Queue], out_queues: Dict[int, Queue], rank: int, devices: List[int], cache_size: int, view_size: int) → None