parallel.client.magics

Module: parallel.client.magics

Inheritance diagram for IPython.parallel.client.magics:

Magic command interface for interactive parallel work.

Usage

%autopx

Toggles auto parallel mode.

Once this is called, all commands typed at the command line are send to the engines to be executed in parallel. To control which engine are used, the targets attribute of the view before entering %autopx mode.

Then you can do the following:

In [25]: %autopx
%autopx to enabled

In [26]: a = 10
Parallel execution on engine(s): [0,1,2,3]
In [27]: print a
Parallel execution on engine(s): [0,1,2,3]
[stdout:0] 10
[stdout:1] 10
[stdout:2] 10
[stdout:3] 10


In [27]: %autopx
%autopx disabled

%px

Executes the given python command in parallel.

In [24]: %px a = os.getpid()
Parallel execution on engine(s): all

In [25]: %px print a
[stdout:0] 1234
[stdout:1] 1235
[stdout:2] 1236
[stdout:3] 1237

%pxresult

%result [-o SAVE_NAME] [–group-outputs {engine,order,type}] [-e] [-r]

Print the result of the last asynchronous %px command.

This lets you recall the results of %px computations after asynchronous submission (block=False).

Examples

In [23]: %px os.getpid()
Async parallel execution on engine(s): all

In [24]: %pxresult
Out[8:10]: 60920
Out[9:10]: 60921
Out[10:10]: 60922
Out[11:10]: 60923

optional arguments:

-o SAVE_NAME, --out SAVE_NAME
	store the AsyncResult object for this computation in the global namespace under this name.

–group-outputs {engine,order,type}

Group the outputs in a particular way. Choices are: type: group outputs of all engines by type (stdout, stderr, displaypub, etc.). engine: display all output for each engine together. order: like type, but individual displaypub output from each engine is collated. For example, if multiple plots are generated by each engine, the first figure of each engine will be displayed, then the second of each, etc.

-e	group outputs by engine (same as group-outputs=engine)
-r	collate outputs in order (same as group-outputs=order)

%pxconfig

%pxconfig [–no-verbose] [–verbose] [-t TARGETS] [-a] [-b]

configure default targets/blocking for %px magics

optional arguments:

--no-verbose	don’t print any messages
--verbose	print a message at each execution
-t TARGETS, --targets TARGETS
	specify the targets on which to execute
-a, --noblock	use non-blocking (async) execution
-b, --block	use blocking (sync) execution

Class

ParallelMagics

class IPython.parallel.client.magics.ParallelMagics(shell, view, suffix='')

Bases: IPython.core.magic.Magics

A set of magics useful when controlling a parallel IPython cluster.

__init__(shell, view, suffix='')

arg_err(func)

Print docstring if incorrect arguments were passed

autopx(line='')

Toggles auto parallel mode.

Once this is called, all commands typed at the command line are send to the engines to be executed in parallel. To control which engine are used, the targets attribute of the view before entering %autopx mode.

Then you can do the following:

In [25]: %autopx
%autopx to enabled

In [26]: a = 10
Parallel execution on engine(s): [0,1,2,3]
In [27]: print a
Parallel execution on engine(s): [0,1,2,3]
[stdout:0] 10
[stdout:1] 10
[stdout:2] 10
[stdout:3] 10

In [27]: %autopx
%autopx disabled

cell_px(line='', cell=None)

%cell_px [–no-verbose] [–verbose] [-t TARGETS] [-a] [-b] [-o SAVE_NAME] [–group-outputs {engine,order,type}] [-e] [-r]

Executes the cell in parallel.

Examples

In [24]: %%px --noblock
   ....: a = os.getpid()
Async parallel execution on engine(s): all

In [25]: %%px
   ....: print a
[stdout:0] 1234
[stdout:1] 1235
[stdout:2] 1236
[stdout:3] 1237

optional arguments:

--no-verbose	don’t print any messages
--verbose	print a message at each execution
-t TARGETS, --targets TARGETS
	specify the targets on which to execute
-a, --noblock	use non-blocking (async) execution
-b, --block	use blocking (sync) execution
-o SAVE_NAME, --out SAVE_NAME
	store the AsyncResult object for this computation in the global namespace under this name.

–group-outputs {engine,order,type}

Group the outputs in a particular way. Choices are: type: group outputs of all engines by type (stdout, stderr, displaypub, etc.). engine: display all output for each engine together. order: like type, but individual displaypub output from each engine is collated. For example, if multiple plots are generated by each engine, the first figure of each engine will be displayed, then the second of each, etc.

-e	group outputs by engine (same as group-outputs=engine)
-r	collate outputs in order (same as group-outputs=order)

default_option(fn, optstr)

Make an entry in the options_table for fn, with value optstr

format_latex(strng)

Format a string for latex inclusion.

last_result = None

magics = None

options_table = None

parallel_execute(cell, block=None, groupby='type', save_name=None)

implementation used by %px and %%parallel

parse_options(arg_str, opt_str, *long_opts, **kw)

Parse options passed to an argument string.

The interface is similar to that of getopt(), but it returns back a Struct with the options as keys and the stripped argument string still as a string.

arg_str is quoted as a true sys.argv vector by using shlex.split. This allows us to easily expand variables, glob files, quote arguments, etc.

Options:

-mode: default ‘string’. If given as ‘list’, the argument string is returned as a list (split on whitespace) instead of a string.

-list_all: put all option values in lists. Normally only options appearing more than once are put in a list.

-posix (True): whether to split the input line in POSIX mode or not, as per the conventions outlined in the shlex module from the standard library.

px(line='')

Executes the given python command in parallel.

Examples

In [24]: %px a = os.getpid()
Parallel execution on engine(s): all

In [25]: %px print a
[stdout:0] 1234
[stdout:1] 1235
[stdout:2] 1236
[stdout:3] 1237

pxconfig(line)

%pxconfig [–no-verbose] [–verbose] [-t TARGETS] [-a] [-b]

configure default targets/blocking for %px magics

optional arguments:

--no-verbose	don’t print any messages
--verbose	print a message at each execution
-t TARGETS, --targets TARGETS
	specify the targets on which to execute
-a, --noblock	use non-blocking (async) execution
-b, --block	use blocking (sync) execution

pxrun_cell(raw_cell, store_history=False, silent=False)

drop-in replacement for InteractiveShell.run_cell.

This executes code remotely, instead of in the local namespace.

See InteractiveShell.run_cell for details.

registered = True

result(line='')

%result [-o SAVE_NAME] [–group-outputs {engine,order,type}] [-e] [-r]

Print the result of the last asynchronous %px command.

This lets you recall the results of %px computations after asynchronous submission (block=False).

Examples

In [23]: %px os.getpid()
Async parallel execution on engine(s): all

In [24]: %pxresult
Out[8:10]: 60920
Out[9:10]: 60921
Out[10:10]: 60922
Out[11:10]: 60923

optional arguments:

-o SAVE_NAME, --out SAVE_NAME
	store the AsyncResult object for this computation in the global namespace under this name.

–group-outputs {engine,order,type}

Group the outputs in a particular way. Choices are: type: group outputs of all engines by type (stdout, stderr, displaypub, etc.). engine: display all output for each engine together. order: like type, but individual displaypub output from each engine is collated. For example, if multiple plots are generated by each engine, the first figure of each engine will be displayed, then the second of each, etc.

-e	group outputs by engine (same as group-outputs=engine)
-r	collate outputs in order (same as group-outputs=order)

shell = None

suffix = ''

verbose = False

view = None

Functions

IPython.parallel.client.magics.exec_args(f)

decorator for adding block/targets args for execution

applied to %pxconfig and %%px

IPython.parallel.client.magics.output_args(f)

decorator for output-formatting args

applied to %pxresult and %%px