Inheritance diagram for IPython.kernel.multiengineclient:
General Classes for IMultiEngine clients.
Bases: IPython.kernel.multiengineclient.InteractiveMultiEngineClient
A blocking client to the IMultiEngine controller interface.
This class allows users to use a set of engines for a parallel computation through the IMultiEngine interface. In this interface, each engine has a specific id (an int) that is used to refer to the engine, run code on it, etc.
Synchronize a set of PendingResults.
This method is a synchronization primitive that waits for a set of PendingResult objects to complete. More specifically, barier does the following.
After barrier has been called on a PendingResult, its results can be retrieved by calling get_result again or accesing the r attribute of the instance.
Run performance benchmarks for the current IPython cluster.
This method tests both the latency of sending command and data to the engines as well as the throughput of sending large objects to the engines using push. The latency is measured by having one or more engines execute the command ‘pass’. The throughput is measure by sending an NumPy array of size push_size to one or more engines.
These benchmarks will vary widely on different hardware and networks and thus can be used to get an idea of the performance characteristics of a particular configuration of an IPython controller and engines.
This function is not testable within our current testing framework.
Clear all pending deferreds/results from the controller.
For each PendingResult that is created by this client, the controller holds on to the result for that PendingResult. This can be a problem if there are a large number of PendingResult objects that are created.
Once the result of the PendingResult has been retrieved, the result is removed from the controller, but if a user doesn’t get a result ( they just ignore the PendingResult) the result is kept forever on the controller. This method allows the user to clear out all un-retrieved results on the controller.
Clear out the controller’s queue for an engine.
The controller maintains a queue for each engine. This clear it out.
Parameters : |
|
---|
Execute code on a set of engines.
Parameters : |
|
---|
Clear all pending deferreds/results from the controller.
For each PendingResult that is created by this client, the controller holds on to the result for that PendingResult. This can be a problem if there are a large number of PendingResult objects that are created.
Once the result of the PendingResult has been retrieved, the result is removed from the controller, but if a user doesn’t get a result ( they just ignore the PendingResult) the result is kept forever on the controller. This method allows the user to clear out all un-retrieved results on the controller.
Gather a partitioned sequence on a set of engines as a single local seq.
Returns the ids of currently registered engines.
Get a previous result.
When code is executed in an engine, a dict is created and returned. This method retrieves that dict for previous commands.
Parameters : |
|
---|
Get a list of all the variables in an engine’s namespace.
Parameters : |
|
---|
Kill the engines and controller.
This method is used to stop the engine and controller by calling reactor.stop.
Parameters : |
|
---|
A parallel version of Python’s builtin map function.
This method applies a function to sequences of arguments. It follows the same syntax as the builtin map.
This method creates a mapper objects by calling self.mapper with no arguments and then uses that mapper to do the mapping. See the documentation of mapper for more details.
Create a mapper object that has a map method.
This method returns an object that implements the IMapper interface. This method is a factory that is used to control how the map happens.
Parameters : |
|
---|
A decorator that turns a function into a parallel function.
This can be used as:
@parallel() def f(x, y)
...
f(range(10), range(10))
This causes f(0,0), f(1,1), ... to be called in parallel.
Parameters : |
|
---|
Pull Python objects by key out of engines namespaces.
Parameters : |
|
---|
Pull a Python function from an engine.
This method is used to pull a Python function from an engine. Closures are not supported.
Parameters : |
|
---|
Push a dictionary of keys and values to engines namespace.
Each engine has a persistent namespace. This method is used to push Python objects into that namespace.
The objects in the namespace must be pickleable.
Parameters : |
|
---|
Push a Python function to an engine.
This method is used to push a Python function to an engine. This method can then be used in code on the engines. Closures are not supported.
Parameters : |
|
---|
Get the status of an engines queue.
Parameters : |
|
---|
A parallelized version of Python’s builtin map.
This has a slightly different syntax than the builtin map. This is needed because we need to have keyword arguments and thus can’t use *args to capture all the sequences. Instead, they must be passed in a list or tuple.
raw_map(func, seqs) -> map(func, seqs[0], seqs[1], ...)
Most users will want to use parallel functions or the mapper and map methods for an API that follows that of the builtin map.
Reset an engine.
This method clears out the namespace of an engine.
Parameters : |
|
---|
Run a Python code in a file on the engines.
Parameters : |
|
---|
Partition a Python sequence and send the partitions to a set of engines.
Bases: zope.interface.Interface
A representation of a result that is pending.
This class is similar to Twisted’s Deferred object, but is designed to be used in a synchronous context.
Bases: object
A mixin class that add a few methods to a multiengine client.
The methods in this mixin class are designed for interactive usage.
x.__init__(...) initializes x; see x.__class__.__doc__ for signature
Make this MultiEngineClient active for parallel magic commands.
IPython has a magic command syntax to work with MultiEngineClient objects. In a given IPython session there is a single active one. While there can be many MultiEngineClient created and used by the user, there is only one active one. The active MultiEngineClient is used whenever the magic commands %px and %autopx are used.
The activate() method is called on a given MultiEngineClient to make it active. Once this has been done, the magic commands can be used.
Bases: object
A representation of a result that is not yet ready.
A user should not create a PendingResult instance by hand.
Methods:
Properties:
Create a PendingResult with a result_id and a client instance.
The client should implement _getPendingResult(result_id, block).
Add a callback that is called with the result.
If the original result is result, adding a callback will cause f(result, *args, **kwargs) to be returned instead. If multiple callbacks are registered, they are chained together: the result of one is passed to the next and so on.
Unlike Twisted’s Deferred object, there is no errback chain. Thus any exception raised will not be caught and handled. User must catch these by hand when calling get_result.
Get a result that is pending.
This method will connect to an IMultiEngine adapted controller and see if the result is ready. If the action triggers an exception raise it and record it. This method records the result/exception once it is retrieved. Calling get_result again will get this cached result or will re-raise the exception. The .r attribute is a property that calls get_result with block=True.
Parameters : |
|
---|---|
Returns : | The actual result or the default value. |
This property is a shortcut to a get_result(block=True).