parallel.controller.dependency

Module: parallel.controller.dependency

Inheritance diagram for IPython.parallel.controller.dependency:

Dependency utilities

Authors:

• Min RK

Classes

Dependency

class IPython.parallel.controller.dependency.Dependency(dependencies=[], all=True, success=True, failure=False)

Bases: set

An object for representing a set of msg_id dependencies.

Subclassed from set().

Parameters :

dependencies: list/set of msg_ids or AsyncResult objects or output of Dependency.as_dict() :

The msg_ids to depend on

all : bool [default True]

Whether the dependency should be considered met when all depending tasks have completed or only when any have been completed.

success : bool [default True]

Whether to consider successes as fulfilling dependencies.

failure : bool [default False]

Whether to consider failures as fulfilling dependencies.

If `all=success=True` and `failure=False`, then the task will fail with an ImpossibleDependency :

as soon as the first depended-upon task fails.

__init__(dependencies=[], all=True, success=True, failure=False)

add

Add an element to a set.

This has no effect if the element is already present.

all = True

as_dict()

Represent this dependency as a dict. For json compatibility.

check(completed, failed=None)

check whether our dependencies have been met.

clear

Remove all elements from this set.

copy

Return a shallow copy of a set.

difference

Return the difference of two or more sets as a new set.

(i.e. all elements that are in this set but not the others.)

difference_update

Remove all elements of another set from this set.

discard

Remove an element from a set if it is a member.

If the element is not a member, do nothing.

failure = True

intersection

Return the intersection of two or more sets as a new set.

(i.e. elements that are common to all of the sets.)

intersection_update

Update a set with the intersection of itself and another.

isdisjoint

Return True if two sets have a null intersection.

issubset

Report whether another set contains this set.

issuperset

Report whether this set contains another set.

pop

Remove and return an arbitrary set element. Raises KeyError if the set is empty.

remove

Remove an element from a set; it must be a member.

If the element is not a member, raise a KeyError.

success = True

symmetric_difference

Return the symmetric difference of two sets as a new set.

(i.e. all elements that are in exactly one of the sets.)

symmetric_difference_update

Update a set with the symmetric difference of itself and another.

union

Return the union of sets as a new set.

(i.e. all elements that are in either set.)

unreachable(completed, failed=None)

return whether this dependency has become impossible.

update

Update a set with the union of itself and others.

depend

class IPython.parallel.controller.dependency.depend(f, *args, **kwargs)

Bases: object

Dependency decorator, for use with tasks.

@depend lets you define a function for engine dependencies just like you use apply for tasks.

Examples

@depend(df, a,b, c=5)
def f(m,n,p)

view.apply(f, 1,2,3)

will call df(a,b,c=5) on the engine, and if it returns False or raises an UnmetDependency error, then the task will not be run and another engine will be tried.

__init__(f, *args, **kwargs)

dependent

class IPython.parallel.controller.dependency.dependent(f, df, *dargs, **dkwargs)

Bases: object

A function that depends on another function. This is an object to prevent the closure used in traditional decorators, which are not picklable.

__init__(f, df, *dargs, **dkwargs)

Function

IPython.parallel.controller.dependency.require(*mods)

Simple decorator for requiring names to be importable.

Examples

In [1]: @require(‘numpy’)

...: def norm(a): ...: import numpy ...: return numpy.linalg.norm(a,2)