parallel.controller.dependency

Module: parallel.controller.dependency

Inheritance diagram for IPython.parallel.controller.dependency:

Dependency utilities

Authors:

• Min RK

Classes

Dependency

class IPython.parallel.controller.dependency.Dependency(dependencies=[], all=True, success=True, failure=False)

Bases: set

An object for representing a set of msg_id dependencies.

Subclassed from set().

Parameters :

dependencies: list/set of msg_ids or AsyncResult objects or output of Dependency.as_dict() :

The msg_ids to depend on

all : bool [default True]

Whether the dependency should be considered met when all depending tasks have completed or only when any have been completed.

success : bool [default True]

Whether to consider successes as fulfilling dependencies.

failure : bool [default False]

Whether to consider failures as fulfilling dependencies.

If `all=success=True` and `failure=False`, then the task will fail with an ImpossibleDependency :

as soon as the first depended-upon task fails.

__init__(dependencies=[], all=True, success=True, failure=False)

add()

Add an element to a set.

This has no effect if the element is already present.

all = True

as_dict()

Represent this dependency as a dict. For json compatibility.

check(completed, failed=None)

check whether our dependencies have been met.

clear()

Remove all elements from this set.

copy()

Return a shallow copy of a set.

difference()

Return the difference of two or more sets as a new set.

(i.e. all elements that are in this set but not the others.)

difference_update()

Remove all elements of another set from this set.

discard()

Remove an element from a set if it is a member.

If the element is not a member, do nothing.

failure = True

intersection()

Return the intersection of two or more sets as a new set.

(i.e. elements that are common to all of the sets.)

intersection_update()

Update a set with the intersection of itself and another.

isdisjoint()

Return True if two sets have a null intersection.

issubset()

Report whether another set contains this set.

issuperset()

Report whether this set contains another set.

pop()

Remove and return an arbitrary set element. Raises KeyError if the set is empty.

remove()

Remove an element from a set; it must be a member.

If the element is not a member, raise a KeyError.

success = True

symmetric_difference()

Return the symmetric difference of two sets as a new set.

(i.e. all elements that are in exactly one of the sets.)

symmetric_difference_update()

Update a set with the symmetric difference of itself and another.

union()

Return the union of sets as a new set.

(i.e. all elements that are in either set.)

unreachable(completed, failed=None)

return whether this dependency has become impossible.

update()

Update a set with the union of itself and others.

depend

class IPython.parallel.controller.dependency.depend(f, *args, **kwargs)

Bases: object

Dependency decorator, for use with tasks.

@depend lets you define a function for engine dependencies just like you use apply for tasks.

Examples

@depend(df, a,b, c=5)
def f(m,n,p)

view.apply(f, 1,2,3)

will call df(a,b,c=5) on the engine, and if it returns False or raises an UnmetDependency error, then the task will not be run and another engine will be tried.

__init__(f, *args, **kwargs)

dependent

class IPython.parallel.controller.dependency.dependent(f, df, *dargs, **dkwargs)

Bases: object

A function that depends on another function. This is an object to prevent the closure used in traditional decorators, which are not picklable.

__init__(f, df, *dargs, **dkwargs)

Function

IPython.parallel.controller.dependency.require(*mods)

Simple decorator for requiring names to be importable.

Examples

In [1]: @require(‘numpy’)

...: def norm(a): ...: import numpy ...: return numpy.linalg.norm(a,2)