IPython Documentation

Table Of Contents

Previous topic

parallel.client.client

Next topic

parallel.client.map

This Page

parallel.client.magics

Module: parallel.client.magics

Inheritance diagram for IPython.parallel.client.magics:

Magic command interface for interactive parallel work.

Usage

%autopx

Toggles auto parallel mode.

Once this is called, all commands typed at the command line are send to the engines to be executed in parallel. To control which engine are used, the targets attribute of the view before entering %autopx mode.

Then you can do the following:

In [25]: %autopx
%autopx to enabled

In [26]: a = 10
Parallel execution on engine(s): [0,1,2,3]
In [27]: print a
Parallel execution on engine(s): [0,1,2,3]
[stdout:0] 10
[stdout:1] 10
[stdout:2] 10
[stdout:3] 10


In [27]: %autopx
%autopx disabled

%px

Executes the given python command in parallel.

In [24]: %px a = os.getpid()
Parallel execution on engine(s): all

In [25]: %px print a
[stdout:0] 1234
[stdout:1] 1235
[stdout:2] 1236
[stdout:3] 1237

%pxresult

%result [-o SAVE_NAME] [–group-outputs {engine,order,type}] [-e] [-r]

Print the result of the last asynchronous %px command.

This lets you recall the results of %px computations after asynchronous submission (block=False).

Examples

In [23]: %px os.getpid()
Async parallel execution on engine(s): all

In [24]: %pxresult
Out[8:10]: 60920
Out[9:10]: 60921
Out[10:10]: 60922
Out[11:10]: 60923
optional arguments:
-o SAVE_NAME, --out SAVE_NAME
 store the AsyncResult object for this computation in the global namespace under this name.
–group-outputs {engine,order,type}
Group the outputs in a particular way. Choices are: type: group outputs of all engines by type (stdout, stderr, displaypub, etc.). engine: display all output for each engine together. order: like type, but individual displaypub output from each engine is collated. For example, if multiple plots are generated by each engine, the first figure of each engine will be displayed, then the second of each, etc.
-e group outputs by engine (same as group-outputs=engine)
-r collate outputs in order (same as group-outputs=order)

%pxconfig

%pxconfig [–no-verbose] [–verbose] [-t TARGETS] [-a] [-b]

configure default targets/blocking for %px magics

optional arguments:
--no-verbose don’t print any messages
--verbose print a message at each execution
-t TARGETS, --targets TARGETS
 specify the targets on which to execute
-a, --noblock use non-blocking (async) execution
-b, --block use blocking (sync) execution

Class

ParallelMagics

class IPython.parallel.client.magics.ParallelMagics(shell, view, suffix='')

Bases: IPython.core.magic.Magics

A set of magics useful when controlling a parallel IPython cluster.

__init__(shell, view, suffix='')
arg_err(func)

Print docstring if incorrect arguments were passed

autopx(line='')

Toggles auto parallel mode.

Once this is called, all commands typed at the command line are send to the engines to be executed in parallel. To control which engine are used, the targets attribute of the view before entering %autopx mode.

Then you can do the following:

In [25]: %autopx
%autopx to enabled

In [26]: a = 10
Parallel execution on engine(s): [0,1,2,3]
In [27]: print a
Parallel execution on engine(s): [0,1,2,3]
[stdout:0] 10
[stdout:1] 10
[stdout:2] 10
[stdout:3] 10

In [27]: %autopx
%autopx disabled
cell_px(line='', cell=None)

%cell_px [–no-verbose] [–verbose] [-t TARGETS] [-a] [-b] [-o SAVE_NAME] [–group-outputs {engine,order,type}] [-e] [-r]

Executes the cell in parallel.

Examples

In [24]: %%px --noblock
   ....: a = os.getpid()
Async parallel execution on engine(s): all

In [25]: %%px
   ....: print a
[stdout:0] 1234
[stdout:1] 1235
[stdout:2] 1236
[stdout:3] 1237
optional arguments:
--no-verbose don’t print any messages
--verbose print a message at each execution
-t TARGETS, --targets TARGETS
 specify the targets on which to execute
-a, --noblock use non-blocking (async) execution
-b, --block use blocking (sync) execution
-o SAVE_NAME, --out SAVE_NAME
 store the AsyncResult object for this computation in the global namespace under this name.
–group-outputs {engine,order,type}
Group the outputs in a particular way. Choices are: type: group outputs of all engines by type (stdout, stderr, displaypub, etc.). engine: display all output for each engine together. order: like type, but individual displaypub output from each engine is collated. For example, if multiple plots are generated by each engine, the first figure of each engine will be displayed, then the second of each, etc.
-e group outputs by engine (same as group-outputs=engine)
-r collate outputs in order (same as group-outputs=order)
default_option(fn, optstr)

Make an entry in the options_table for fn, with value optstr

format_latex(strng)

Format a string for latex inclusion.

last_result = None
magics = None
options_table = None
parallel_execute(cell, block=None, groupby='type', save_name=None)

implementation used by %px and %%parallel

parse_options(arg_str, opt_str, *long_opts, **kw)

Parse options passed to an argument string.

The interface is similar to that of getopt(), but it returns back a Struct with the options as keys and the stripped argument string still as a string.

arg_str is quoted as a true sys.argv vector by using shlex.split. This allows us to easily expand variables, glob files, quote arguments, etc.

Options:

-mode: default ‘string’. If given as ‘list’, the argument string is returned as a list (split on whitespace) instead of a string.

-list_all: put all option values in lists. Normally only options appearing more than once are put in a list.

-posix (True): whether to split the input line in POSIX mode or not, as per the conventions outlined in the shlex module from the standard library.

px(line='')

Executes the given python command in parallel.

Examples

In [24]: %px a = os.getpid()
Parallel execution on engine(s): all

In [25]: %px print a
[stdout:0] 1234
[stdout:1] 1235
[stdout:2] 1236
[stdout:3] 1237
pxconfig(line)

%pxconfig [–no-verbose] [–verbose] [-t TARGETS] [-a] [-b]

configure default targets/blocking for %px magics

optional arguments:
--no-verbose don’t print any messages
--verbose print a message at each execution
-t TARGETS, --targets TARGETS
 specify the targets on which to execute
-a, --noblock use non-blocking (async) execution
-b, --block use blocking (sync) execution
pxrun_cell(raw_cell, store_history=False, silent=False)

drop-in replacement for InteractiveShell.run_cell.

This executes code remotely, instead of in the local namespace.

See InteractiveShell.run_cell for details.

registered = True
result(line='')

%result [-o SAVE_NAME] [–group-outputs {engine,order,type}] [-e] [-r]

Print the result of the last asynchronous %px command.

This lets you recall the results of %px computations after asynchronous submission (block=False).

Examples

In [23]: %px os.getpid()
Async parallel execution on engine(s): all

In [24]: %pxresult
Out[8:10]: 60920
Out[9:10]: 60921
Out[10:10]: 60922
Out[11:10]: 60923
optional arguments:
-o SAVE_NAME, --out SAVE_NAME
 store the AsyncResult object for this computation in the global namespace under this name.
–group-outputs {engine,order,type}
Group the outputs in a particular way. Choices are: type: group outputs of all engines by type (stdout, stderr, displaypub, etc.). engine: display all output for each engine together. order: like type, but individual displaypub output from each engine is collated. For example, if multiple plots are generated by each engine, the first figure of each engine will be displayed, then the second of each, etc.
-e group outputs by engine (same as group-outputs=engine)
-r collate outputs in order (same as group-outputs=order)
shell = None
suffix = ''
verbose = False
view = None

Functions

IPython.parallel.client.magics.exec_args(f)

decorator for adding block/targets args for execution

applied to %pxconfig and %%px

IPython.parallel.client.magics.output_args(f)

decorator for output-formatting args

applied to %pxresult and %%px