Warning

This documentation is for an old version of IPython. You can find docs for newer versions here.

Module: parallel.client.magics

Magic command interface for interactive parallel work.

Usage

%autopx

Toggles auto parallel mode.

Once this is called, all commands typed at the command line are send to the engines to be executed in parallel. To control which engine are used, the targets attribute of the view before entering %autopx mode.

Then you can do the following:

In [25]: %autopx
%autopx to enabled

In [26]: a = 10
Parallel execution on engine(s): [0,1,2,3]
In [27]: print a
Parallel execution on engine(s): [0,1,2,3]
[stdout:0] 10
[stdout:1] 10
[stdout:2] 10
[stdout:3] 10


In [27]: %autopx
%autopx disabled

%px

Executes the given python command in parallel.

Examples

In [24]: %px a = os.getpid()
Parallel execution on engine(s): all

In [25]: %px print a
[stdout:0] 1234
[stdout:1] 1235
[stdout:2] 1236
[stdout:3] 1237

%pxresult

%result [-o SAVE_NAME] [--group-outputs {engine,order,type}] [-e] [-r]

Print the result of the last asynchronous %px command.

This lets you recall the results of %px computations after asynchronous submission (block=False).

Examples

In [23]: %px os.getpid()
Async parallel execution on engine(s): all

In [24]: %pxresult
Out[8:10]: 60920
Out[9:10]: 60921
Out[10:10]: 60922
Out[11:10]: 60923
optional arguments:
-o SAVE_NAME, --out SAVE_NAME
 store the AsyncResult object for this computation in the global namespace under this name.
--group-outputs <{engine,order,type}>
 Group the outputs in a particular way. Choices are: type: group outputs of all engines by type (stdout, stderr, displaypub, etc.). engine: display all output for each engine together. order: like type, but individual displaypub output from each engine is collated. For example, if multiple plots are generated by each engine, the first figure of each engine will be displayed, then the second of each, etc.
-e group outputs by engine (same as group-outputs=engine)
-r collate outputs in order (same as group-outputs=order)

%pxconfig

%pxconfig [--no-verbose] [--verbose] [--local] [-t TARGETS] [-a] [-b]

configure default targets/blocking for %px magics

optional arguments:
--no-verbose don’t print any messages
--verbose print a message at each execution
--local also execute the cell in the local namespace
-t TARGETS, --targets TARGETS
 specify the targets on which to execute
-a, --noblock use non-blocking (async) execution
-b, --block use blocking (sync) execution

1 Class

class IPython.parallel.client.magics.ParallelMagics(shell, view, suffix='')

Bases: IPython.core.magic.Magics

A set of magics useful when controlling a parallel IPython cluster.

__init__(shell, view, suffix='')
autopx(line='')

Toggles auto parallel mode.

Once this is called, all commands typed at the command line are send to the engines to be executed in parallel. To control which engine are used, the targets attribute of the view before entering %autopx mode.

Then you can do the following:

In [25]: %autopx
%autopx to enabled

In [26]: a = 10
Parallel execution on engine(s): [0,1,2,3]
In [27]: print a
Parallel execution on engine(s): [0,1,2,3]
[stdout:0] 10
[stdout:1] 10
[stdout:2] 10
[stdout:3] 10

In [27]: %autopx
%autopx disabled
cell_px(line='', cell=None)
%cell_px [--no-verbose] [--verbose] [--local] [-t TARGETS] [-a] [-b]
             [-o SAVE_NAME] [--group-outputs {engine,order,type}] [-e] [-r]

Executes the cell in parallel.

Examples

In [24]: %%px --noblock
   ....: a = os.getpid()
Async parallel execution on engine(s): all

In [25]: %%px
   ....: print a
[stdout:0] 1234
[stdout:1] 1235
[stdout:2] 1236
[stdout:3] 1237
optional arguments:
--no-verbose don’t print any messages
--verbose print a message at each execution
--local also execute the cell in the local namespace
-t TARGETS, --targets TARGETS
 specify the targets on which to execute
-a, --noblock use non-blocking (async) execution
-b, --block use blocking (sync) execution
-o SAVE_NAME, --out SAVE_NAME
 store the AsyncResult object for this computation in the global namespace under this name.
--group-outputs <{engine,order,type}>
 Group the outputs in a particular way. Choices are: type: group outputs of all engines by type (stdout, stderr, displaypub, etc.). engine: display all output for each engine together. order: like type, but individual displaypub output from each engine is collated. For example, if multiple plots are generated by each engine, the first figure of each engine will be displayed, then the second of each, etc.
-e group outputs by engine (same as group-outputs=engine)
-r collate outputs in order (same as group-outputs=order)
parallel_execute(cell, block=None, groupby='type', save_name=None)

implementation used by %px and %%parallel

px(line='')

Executes the given python command in parallel.

Examples

In [24]: %px a = os.getpid()
Parallel execution on engine(s): all

In [25]: %px print a
[stdout:0] 1234
[stdout:1] 1235
[stdout:2] 1236
[stdout:3] 1237
pxconfig(line)
%pxconfig [--no-verbose] [--verbose] [--local] [-t TARGETS] [-a] [-b]

configure default targets/blocking for %px magics

optional arguments:
--no-verbose don’t print any messages
--verbose print a message at each execution
--local also execute the cell in the local namespace
-t TARGETS, --targets TARGETS
 specify the targets on which to execute
-a, --noblock use non-blocking (async) execution
-b, --block use blocking (sync) execution
pxrun_cell(raw_cell, store_history=False, silent=False)

drop-in replacement for InteractiveShell.run_cell.

This executes code remotely, instead of in the local namespace.

See InteractiveShell.run_cell for details.

result(line='')
%result [-o SAVE_NAME] [--group-outputs {engine,order,type}] [-e] [-r]

Print the result of the last asynchronous %px command.

This lets you recall the results of %px computations after asynchronous submission (block=False).

Examples

In [23]: %px os.getpid()
Async parallel execution on engine(s): all

In [24]: %pxresult
Out[8:10]: 60920
Out[9:10]: 60921
Out[10:10]: 60922
Out[11:10]: 60923
optional arguments:
-o SAVE_NAME, --out SAVE_NAME
 store the AsyncResult object for this computation in the global namespace under this name.
--group-outputs <{engine,order,type}>
 Group the outputs in a particular way. Choices are: type: group outputs of all engines by type (stdout, stderr, displaypub, etc.). engine: display all output for each engine together. order: like type, but individual displaypub output from each engine is collated. For example, if multiple plots are generated by each engine, the first figure of each engine will be displayed, then the second of each, etc.
-e group outputs by engine (same as group-outputs=engine)
-r collate outputs in order (same as group-outputs=order)

2 Functions

IPython.parallel.client.magics.exec_args(f)

decorator for adding block/targets args for execution

applied to %pxconfig and %%px

IPython.parallel.client.magics.output_args(f)

decorator for output-formatting args

applied to %pxresult and %%px