IPython as a system shell

Warning

As of the 0.11 version of IPython, most of the APIs used by the shell profile have been changed, so the profile currently does very little beyond changing the IPython prompt. To help restore the shell profile to past functionality described here, the old code is found in IPython/deathrow, which needs to be updated to use the APIs in 0.11.

Overview

The ‘sh’ profile optimizes IPython for system shell usage. Apart from certain job control functionality that is present in unix (ctrl+z does “suspend”), the sh profile should provide you with most of the functionality you use daily in system shell, and more. Invoke IPython in ‘sh’ profile by doing ‘ipython -p sh’, or (in win32) by launching the “pysh” shortcut in start menu.

If you want to use the features of sh profile as your defaults (which might be a good idea if you use other profiles a lot of the time but still want the convenience of sh profile), add import ipy_profile_sh to your $IPYTHON_DIR/ipy_user_conf.py.

The ‘sh’ profile is different from the default profile in that:

• Prompt shows the current directory
• Spacing between prompts and input is more compact (no padding with empty lines). The startup banner is more compact as well.
• System commands are directly available (in alias table) without requesting %rehashx - however, if you install new programs along your PATH, you might want to run %rehashx to update the persistent alias table
• Macros are stored in raw format by default. That is, instead of ‘_ip.system(“cat foo”), the macro will contain text ‘cat foo’)
• Autocall is in full mode
• Calling “up” does “cd ..”

The ‘sh’ profile is different from the now-obsolete (and unavailable) ‘pysh’ profile in that:

• ‘$$var = command’ and ‘$var = command’ syntax is not supported
• anymore. Use ‘var = !command’ instead (incidentally, this is
• available in all IPython profiles). Note that !!command will
• work.

Aliases

All of your $PATH has been loaded as IPython aliases, so you should be able to type any normal system command and have it executed. See %alias? and %unalias? for details on the alias facilities. See also %rehashx? for details on the mechanism used to load $PATH.

Directory management

Since each command passed by ipython to the underlying system is executed in a subshell which exits immediately, you can NOT use !cd to navigate the filesystem.

IPython provides its own builtin ‘%cd’ magic command to move in the filesystem (the % is not required with automagic on). It also maintains a list of visited directories (use %dhist to see it) and allows direct switching to any of them. Type ‘cd?’ for more details.

%pushd, %popd and %dirs are provided for directory stack handling.

Enabled extensions

Some extensions, listed below, are enabled as default in this profile.

envpersist

%env can be used to “remember” environment variable manipulations. Examples:

%env - Show all environment variables
%env VISUAL=jed  - set VISUAL to jed
%env PATH+=;/foo - append ;foo to PATH
%env PATH+=;/bar - also append ;bar to PATH
%env PATH-=/wbin; - prepend /wbin; to PATH
%env -d VISUAL   - forget VISUAL persistent val
%env -p          - print all persistent env modifications

ipy_which

%which magic command. Like ‘which’ in unix, but knows about ipython aliases.

Example:

[C:/ipython]|14> %which st
st -> start .
[C:/ipython]|15> %which d
d -> dir /w /og /on
[C:/ipython]|16> %which cp
cp -> cp
  == c:\bin\cp.exe
c:\bin\cp.exe

ipy_app_completers

Custom tab completers for some apps like svn, hg, bzr, apt-get. Try ‘apt-get install <TAB>’ in debian/ubuntu.

ipy_rehashdir

Allows you to add system command aliases for commands that are not along your path. Let’s say that you just installed Putty and want to be able to invoke it without adding it to path, you can create the alias for it with rehashdir:

[~]|22> cd c:/opt/PuTTY/
[c:opt/PuTTY]|23> rehashdir .
             <23> ['pageant', 'plink', 'pscp', 'psftp', 'putty', 'puttygen', 'unins000']

Now, you can execute any of those commams directly:

[c:opt/PuTTY]|24> cd
[~]|25> putty

(the putty window opens).

If you want to store the alias so that it will always be available, do ‘%store putty’. If you want to %store all these aliases persistently, just do it in a for loop:

[~]|27> for a in _23:
   |..>     %store $a
   |..>
   |..>
Alias stored: pageant (0, 'c:\\opt\\PuTTY\\pageant.exe')
Alias stored: plink (0, 'c:\\opt\\PuTTY\\plink.exe')
Alias stored: pscp (0, 'c:\\opt\\PuTTY\\pscp.exe')
Alias stored: psftp (0, 'c:\\opt\\PuTTY\\psftp.exe')
...

mglob

Provide the magic function %mglob, which makes it easier (than the ‘find’ command) to collect (possibly recursive) file lists. Examples:

[c:/ipython]|9> mglob *.py
[c:/ipython]|10> mglob *.py rec:*.txt
[c:/ipython]|19> workfiles = %mglob !.svn/ !.hg/ !*_Data/ !*.bak rec:.

Note that the first 2 calls will put the file list in result history (_, _9, _10), and the last one will assign it to ‘workfiles’.

Prompt customization

The sh profile uses the following prompt configurations:

o.prompt_in1= r'\C_LightBlue[\C_LightCyan\Y2\C_LightBlue]\C_Green|\#>'
o.prompt_in2= r'\C_Green|\C_LightGreen\D\C_Green>'

You can change the prompt configuration to your liking by editing ipy_user_conf.py.

String lists

String lists (IPython.utils.text.SList) are handy way to process output from system commands. They are produced by var = !cmd syntax.

First, we acquire the output of ‘ls -l’:

[Q:doc/examples]|2> lines = !ls -l
 ==
['total 23',
 '-rw-rw-rw- 1 ville None 1163 Sep 30  2006 example-demo.py',
 '-rw-rw-rw- 1 ville None 1927 Sep 30  2006 example-embed-short.py',
 '-rwxrwxrwx 1 ville None 4606 Sep  1 17:15 example-embed.py',
 '-rwxrwxrwx 1 ville None 1017 Sep 30  2006 example-gnuplot.py',
 '-rwxrwxrwx 1 ville None  339 Jun 11 18:01 extension.py',
 '-rwxrwxrwx 1 ville None  113 Dec 20  2006 seteditor.py',
 '-rwxrwxrwx 1 ville None  245 Dec 12  2006 seteditor.pyc']

Now, let’s take a look at the contents of ‘lines’ (the first number is the list element number):

[Q:doc/examples]|3> lines
                <3> SList (.p, .n, .l, .s, .grep(), .fields() available). Value:

0: total 23
1: -rw-rw-rw- 1 ville None 1163 Sep 30  2006 example-demo.py
2: -rw-rw-rw- 1 ville None 1927 Sep 30  2006 example-embed-short.py
3: -rwxrwxrwx 1 ville None 4606 Sep  1 17:15 example-embed.py
4: -rwxrwxrwx 1 ville None 1017 Sep 30  2006 example-gnuplot.py
5: -rwxrwxrwx 1 ville None  339 Jun 11 18:01 extension.py
6: -rwxrwxrwx 1 ville None  113 Dec 20  2006 seteditor.py
7: -rwxrwxrwx 1 ville None  245 Dec 12  2006 seteditor.pyc

Now, let’s filter out the ‘embed’ lines:

[Q:doc/examples]|4> l2 = lines.grep('embed',prune=1)
[Q:doc/examples]|5> l2
                <5> SList (.p, .n, .l, .s, .grep(), .fields() available). Value:

0: total 23
1: -rw-rw-rw- 1 ville None 1163 Sep 30  2006 example-demo.py
2: -rwxrwxrwx 1 ville None 1017 Sep 30  2006 example-gnuplot.py
3: -rwxrwxrwx 1 ville None  339 Jun 11 18:01 extension.py
4: -rwxrwxrwx 1 ville None  113 Dec 20  2006 seteditor.py
5: -rwxrwxrwx 1 ville None  245 Dec 12  2006 seteditor.pyc

Now, we want strings having just file names and permissions:

[Q:doc/examples]|6> l2.fields(8,0)
                <6> SList (.p, .n, .l, .s, .grep(), .fields() available). Value:

0: total
1: example-demo.py -rw-rw-rw-
2: example-gnuplot.py -rwxrwxrwx
3: extension.py -rwxrwxrwx
4: seteditor.py -rwxrwxrwx
5: seteditor.pyc -rwxrwxrwx

Note how the line with ‘total’ does not raise IndexError.

If you want to split these (yielding lists), call fields() without arguments:

[Q:doc/examples]|7> _.fields()
                <7>
[['total'],
 ['example-demo.py', '-rw-rw-rw-'],
 ['example-gnuplot.py', '-rwxrwxrwx'],
 ['extension.py', '-rwxrwxrwx'],
 ['seteditor.py', '-rwxrwxrwx'],
 ['seteditor.pyc', '-rwxrwxrwx']]

If you want to pass these separated with spaces to a command (typical for lists if files), use the .s property:

[Q:doc/examples]|13> files = l2.fields(8).s
[Q:doc/examples]|14> files
                <14> 'example-demo.py example-gnuplot.py extension.py seteditor.py seteditor.pyc'
[Q:doc/examples]|15> ls $files
example-demo.py  example-gnuplot.py  extension.py  seteditor.py  seteditor.pyc

SLists are inherited from normal python lists, so every list method is available:

[Q:doc/examples]|21> lines.append('hey')

Real world example: remove all files outside version control

First, capture output of “hg status”:

[Q:/ipython]|28> out = !hg status
 ==
['M IPython\\extensions\\ipy_kitcfg.py',
 'M IPython\\extensions\\ipy_rehashdir.py',
...
 '? build\\lib\\IPython\\Debugger.py',
 '? build\\lib\\IPython\\extensions\\InterpreterExec.py',
 '? build\\lib\\IPython\\extensions\\InterpreterPasteInput.py',
...

(lines starting with ? are not under version control).

[Q:/ipython]|35> junk = out.grep(r'^\?').fields(1)
[Q:/ipython]|36> junk
            <36> SList (.p, .n, .l, .s, .grep(), .fields() availab
...
10: build\bdist.win32\winexe\temp\_ctypes.py
11: build\bdist.win32\winexe\temp\_hashlib.py
12: build\bdist.win32\winexe\temp\_socket.py

Now we can just remove these files by doing ‘rm $junk.s’.

The .s, .n, .p properties

The ‘.s’ property returns one string where lines are separated by single space (for convenient passing to system commands). The ‘.n’ property return one string where the lines are separated by ‘n’ (i.e. the original output of the function). If the items in string list are file names, ‘.p’ can be used to get a list of “path” objects for convenient file manipulation.