Warning
This documentation is for an old version of IPython. You can find docs for newer versions here.
Facilities for launching IPython processes asynchronously.
Authors:
Bases: exceptions.Exception
Bases: IPython.config.configurable.LoggingConfigurable
An asbtraction for starting, stopping and signaling a process.
The string form of the program arguments.
A list of cmd and args that will be used to start the process.
This is what is passed to spawnProcess() and the first element will be the process name.
The .args property calls this to find the args list.
Subcommand should implement this to construct the cmd and args.
Call this to trigger startup actions.
This logs the process startup and sets the state to ‘running’. It is a pass-through so it can be used as a callback.
Call this to trigger process stop actions.
This logs the process stopping and sets the state to ‘after’. Call this to trigger callbacks registered via on_stop().
Register a callback to be called with this Launcher’s stop_data when the process actually finishes.
Am I running.
Signal the process.
Parameters: | sig : str or int
|
---|
Start the process.
Bases: IPython.utils.traitlets.HasTraits
MixIn for cluster args as traits
Bases: IPython.parallel.apps.launcher.BaseLauncher
Start and stop an external process in an asynchronous manner.
This will launch the external process with a working directory of self.work_dir.
Send INT, wait a delay and then send KILL.
Bases: IPython.parallel.apps.launcher.LocalProcessLauncher, IPython.parallel.apps.launcher.ControllerMixin
Launch a controller as a regular external process.
Start the controller by profile_dir.
Bases: IPython.parallel.apps.launcher.LocalProcessLauncher, IPython.parallel.apps.launcher.EngineMixin
Launch a single engine as a regular externall process.
Bases: IPython.parallel.apps.launcher.LocalEngineLauncher
Launch a set of engines as regular external processes.
alias of LocalEngineLauncher
Start n engines by profile or profile_dir.
Bases: IPython.parallel.apps.launcher.LocalProcessLauncher
Launch an external process using mpiexec.
Build self.args using all the fields.
Start n instances of the program using mpiexec.
Bases: IPython.parallel.apps.launcher.MPILauncher, IPython.parallel.apps.launcher.ControllerMixin
Launch a controller using mpiexec.
Start the controller by profile_dir.
Bases: IPython.parallel.apps.launcher.MPILauncher, IPython.parallel.apps.launcher.EngineMixin
Launch engines using mpiexec
Start n engines by profile or profile_dir.
Bases: object
Bases: IPython.parallel.apps.launcher.MPILauncher, IPython.parallel.apps.launcher.DeprecatedMPILauncher
Deprecated, use MPILauncher
Bases: IPython.parallel.apps.launcher.MPIControllerLauncher, IPython.parallel.apps.launcher.DeprecatedMPILauncher
Deprecated, use MPIControllerLauncher
Bases: IPython.parallel.apps.launcher.MPIEngineSetLauncher, IPython.parallel.apps.launcher.DeprecatedMPILauncher
Deprecated, use MPIEngineSetLauncher
Bases: IPython.parallel.apps.launcher.LocalProcessLauncher
A minimal launcher for ssh.
To be useful this will probably have to be extended to use the sshx idea for environment variables. There could be other things this needs as well.
fetch remote files (called after start)
send our files (called before start)
Bases: IPython.parallel.apps.launcher.SSHLauncher, IPython.parallel.apps.launcher.ClusterAppMixin
Bases: IPython.parallel.apps.launcher.SSHClusterLauncher, IPython.parallel.apps.launcher.ControllerMixin
Bases: IPython.parallel.apps.launcher.SSHClusterLauncher, IPython.parallel.apps.launcher.EngineMixin
Bases: IPython.parallel.apps.launcher.LocalEngineSetLauncher
determine engine count from engines dict
Start engines by profile or profile_dir. n is ignored, and the engines config property is used instead.
Bases: IPython.parallel.apps.launcher.SSHClusterLauncher
Launcher for calling ipcluster engines on a remote machine.
Requires that remote profile is already configured.
Bases: IPython.parallel.apps.launcher.BaseLauncher
Take the output of the submit command and return the job id.
Start n copies of the process using the Win HPC job scheduler.
Bases: IPython.parallel.apps.launcher.WindowsHPCLauncher, IPython.parallel.apps.launcher.ClusterAppMixin
Start the controller by profile_dir.
Bases: IPython.parallel.apps.launcher.WindowsHPCLauncher, IPython.parallel.apps.launcher.ClusterAppMixin
Start the controller by profile_dir.
Bases: IPython.parallel.apps.launcher.ClusterAppMixin
ClusterApp mixin that updates the self.context dict, rather than cl-args.
Bases: IPython.parallel.apps.launcher.BaseLauncher
Launch an external process using a batch system.
This class is designed to work with UNIX batch systems like PBS, LSF, GridEngine, etc. The overall model is that there are different commands like qsub, qdel, etc. that handle the starting and stopping of the process.
This class also has the notion of a batch script. The batch_template attribute can be set to a string that is a template for the batch script. This template is instantiated using string formatting. Thus the template can use {n} fot the number of instances. Subclasses can add additional variables to the template dict.
Take the output of the submit command and return the job id.
Start n copies of the process using a batch system.
Instantiate and write the batch script to the work_dir.
Bases: IPython.parallel.apps.launcher.BatchSystemLauncher
A BatchSystemLauncher subclass for PBS.
Bases: IPython.parallel.apps.launcher.PBSLauncher, IPython.parallel.apps.launcher.BatchClusterAppMixin
Launch a controller using PBS.
Start the controller by profile or profile_dir.
Bases: IPython.parallel.apps.launcher.PBSLauncher, IPython.parallel.apps.launcher.BatchClusterAppMixin
Launch Engines using PBS
Bases: IPython.parallel.apps.launcher.PBSLauncher
Sun GridEngine is a PBS clone with slightly different syntax
Bases: IPython.parallel.apps.launcher.SGELauncher, IPython.parallel.apps.launcher.BatchClusterAppMixin
Launch a controller using SGE.
Start the controller by profile or profile_dir.
Bases: IPython.parallel.apps.launcher.SGELauncher, IPython.parallel.apps.launcher.BatchClusterAppMixin
Launch Engines with SGE
Bases: IPython.parallel.apps.launcher.BatchSystemLauncher
A BatchSystemLauncher subclass for LSF.
Start n copies of the process using LSF batch system. This cant inherit from the base class because bsub expects to be piped a shell script in order to honor the #BSUB directives : bsub < script
Bases: IPython.parallel.apps.launcher.LSFLauncher, IPython.parallel.apps.launcher.BatchClusterAppMixin
Launch a controller using LSF.
Start the controller by profile or profile_dir.
Bases: IPython.parallel.apps.launcher.LSFLauncher, IPython.parallel.apps.launcher.BatchClusterAppMixin
Launch Engines using LSF
Bases: IPython.parallel.apps.launcher.BatchSystemLauncher
A BatchSystemLauncher subclass for HTCondor.
HTCondor requires that we launch the ipengine/ipcontroller scripts rather that the python instance but otherwise is very similar to PBS. This is because HTCondor destroys sys.executable when launching remote processes - a launched python process depends on sys.executable to effectively evaluate its module search paths. Without it, regardless of which python interpreter you launch you will get the to built in module search paths.
We use the ip{cluster, engine, controller} scripts as our executable to circumvent this - the mechanism of shebanged scripts means that the python binary will be launched with argv[0] set to the location of the ip{cluster, engine, controller} scripts on the remote node. This means you need to take care that:
- Your remote nodes have their paths configured correctly, with the ipengine and ipcontroller of the python environment you wish to execute code in having top precedence.
- This functionality is untested on Windows.
If you need different behavior, consider making you own template.
Bases: IPython.parallel.apps.launcher.HTCondorLauncher, IPython.parallel.apps.launcher.BatchClusterAppMixin
Launch a controller using HTCondor.
Start the controller by profile or profile_dir.
Bases: IPython.parallel.apps.launcher.HTCondorLauncher, IPython.parallel.apps.launcher.BatchClusterAppMixin
Launch Engines using HTCondor
Bases: IPython.parallel.apps.launcher.LocalProcessLauncher
Launch the ipcluster program in an external process.
Run command with arguments and return its output as a byte string.
If the exit code was non-zero it raises a CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute and output in the output attribute.
The arguments are the same as for the Popen constructor. Example:
>>> check_output(["ls", "-l", "/dev/null"])
'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
The stdout argument is not allowed as it is used internally. To capture standard error in the result, use stderr=STDOUT.
>>> check_output(["/bin/sh", "-c",
... "ls -l non_existent_file ; exit 0"],
... stderr=STDOUT)
'ls: non_existent_file: No such file or directory\n'