This section describes the IPython configuration system. Starting with version 0.11, IPython has a completely new configuration system that is quite different from the older ipythonrc or ipy_user_conf.py approaches. The new configuration system was designed from scratch to address the particular configuration needs of IPython. While there are many other excellent configuration systems out there, we found that none of them met our requirements.
Warning
If you are upgrading to version 0.11 of IPython, you will need to migrate your old ipythonrc or ipy_user_conf.py configuration files to the new system. You may want to read the section on configuring IPython. There are also some ideas on the IPython wiki about this.
The discussion that follows is focused on teaching users how to configure IPython to their liking. Developers who want to know more about how they can enable their objects to take advantage of the configuration system should consult our developer guide
There are a number of abstractions that the IPython configuration system uses. Each of these abstractions is represented by a Python class.
An application is a process that does a specific job. The most obvious application is the ipython command line program. Each application reads one or more configuration files and a single set of command line options and then produces a master configuration object for the application. This configuration object is then passed to the configurable objects that the application creates. These configurable objects implement the actual logic of the application and know how to configure themselves given the configuration object.
Applications always have a log attribute that is a configured Logger. This allows centralized logging configuration per-application.
A configurable is a regular Python class that serves as a base class for all main classes in an application. The Configurable base class is lightweight and only does one things.
This Configurable is a subclass of HasTraits that knows how to configure itself. Class level traits with the metadata config=True become values that can be configured from the command line and configuration files.
Developers create Configurable subclasses that implement all of the logic in the application. Each of these subclasses has its own configuration information that controls how instances are created.
Note
Singletons are not strictly enforced - you can have many instances of a given singleton class, but the instance() method will always return the same one.
Having described these main concepts, we can now state the main idea in our configuration system: “configuration” allows the default values of class attributes to be controlled on a class by class basis. Thus all instances of a given class are configured in the same way. Furthermore, if two instances need to be configured differently, they need to be instances of two different classes. While this model may seem a bit restrictive, we have found that it expresses most things that need to be configured extremely well. However, it is possible to create two instances of the same class that have different trait values. This is done by overriding the configuration.
Now, we show what our configuration objects and files look like.
A configuration file is simply a pure Python file that sets the attributes of a global, pre-created configuration object. This configuration object is a Config instance. While in a configuration file, to get a reference to this object, simply call the get_config() function. We inject this function into the global namespace that the configuration file is executed in.
Here is an example of a super simple configuration file that does nothing:
c = get_config()
Once you get a reference to the configuration object, you simply set attributes on it. All you have to know is:
The answers to these two questions are provided by the various Configurable subclasses that an application uses. Let’s look at how this would work for a simple configurable subclass:
# Sample configurable:
from IPython.config.configurable import Configurable
from IPython.utils.traitlets import Int, Float, Unicode, Bool
class MyClass(Configurable):
name = Unicode(u'defaultname', config=True)
ranking = Int(0, config=True)
value = Float(99.0)
# The rest of the class implementation would go here..
In this example, we see that MyClass has three attributes, two of whom (name, ranking) can be configured. All of the attributes are given types and default values. If a MyClass is instantiated, but not configured, these default values will be used. But let’s see how to configure this class in a configuration file:
# Sample config file
c = get_config()
c.MyClass.name = 'coolname'
c.MyClass.ranking = 10
After this configuration file is loaded, the values set in it will override the class defaults anytime a MyClass is created. Furthermore, these attributes will be type checked and validated anytime they are set. This type checking is handled by the IPython.utils.traitlets module, which provides the Unicode, Int and Float types. In addition to these traitlets, the IPython.utils.traitlets provides traitlets for a number of other types.
Note
Underneath the hood, the Configurable base class is a subclass of IPython.utils.traitlets.HasTraits. The IPython.utils.traitlets module is a lightweight version of enthought.traits. Our implementation is a pure Python subset (mostly API compatible) of enthought.traits that does not have any of the automatic GUI generation capabilities. Our plan is to achieve 100% API compatibility to enable the actual enthought.traits to eventually be used instead. Currently, we cannot use enthought.traits as we are committed to the core of IPython being pure Python.
It should be very clear at this point what the naming convention is for configuration attributes:
c.ClassName.attribute_name = attribute_value
Here, ClassName is the name of the class whose configuration attribute you want to set, attribute_name is the name of the attribute you want to set and attribute_value the the value you want it to have. The ClassName attribute of c is not the actual class, but instead is another Config instance.
Note
The careful reader may wonder how the ClassName (MyClass in the above example) attribute of the configuration object c gets created. These attributes are created on the fly by the Config instance, using a simple naming convention. Any attribute of a Config instance whose name begins with an uppercase character is assumed to be a sub-configuration and a new empty Config instance is dynamically created for that attribute. This allows deeply hierarchical information created easily (c.Foo.Bar.value) on the fly.
Let’s say you want to have different configuration files for various purposes. Our configuration system makes it easy for one configuration file to inherit the information in another configuration file. The load_subconfig() command can be used in a configuration file for this purpose. Here is a simple example that loads all of the values from the file base_config.py:
# base_config.py
c = get_config()
c.MyClass.name = 'coolname'
c.MyClass.ranking = 100
into the configuration file main_config.py:
# main_config.py
c = get_config()
# Load everything from base_config.py
load_subconfig('base_config.py')
# Now override one of the values
c.MyClass.name = 'bettername'
In a situation like this the load_subconfig() makes sure that the search path for sub-configuration files is inherited from that of the parent. Thus, you can typically put the two in the same directory and everything will just work.
You can also load configuration files by profile, for instance:
load_subconfig('ipython_config.py', profile='default')
to inherit your default configuration as a starting point.
There is another aspect of configuration where inheritance comes into play. Sometimes, your classes will have an inheritance hierarchy that you want to be reflected in the configuration system. Here is a simple example:
from IPython.config.configurable import Configurable
from IPython.utils.traitlets import Int, Float, Unicode, Bool
class Foo(Configurable):
name = Unicode(u'fooname', config=True)
value = Float(100.0, config=True)
class Bar(Foo):
name = Unicode(u'barname', config=True)
othervalue = Int(0, config=True)
Now, we can create a configuration file to configure instances of Foo and Bar:
# config file
c = get_config()
c.Foo.name = u'bestname'
c.Bar.othervalue = 10
This class hierarchy and configuration file accomplishes the following:
So where should you put your configuration files? IPython uses “profiles” for configuration, and by default, all profiles will be stored in the so called “IPython directory”. The location of this directory is determined by the following algorithm:
On posix systems (Linux, Unix, etc.), IPython respects the $XDG_CONFIG_HOME part of the XDG Base Directory specification. If $XDG_CONFIG_HOME is defined and exists ( XDG_CONFIG_HOME has a default interpretation of $HOME/.config), then IPython’s config directory will be located in $XDG_CONFIG_HOME/ipython. If users still have an IPython directory in $HOME/.ipython, then that will be used. in preference to the system default.
For most users, the default value will simply be something like $HOME/.config/ipython on Linux, or $HOME/.ipython elsewhere.
Once the location of the IPython directory has been determined, you need to know which profile you are using. For users with a single configuration, this will simply be ‘default’, and will be located in <IPYTHON_DIR>/profile_default.
The next thing you need to know is what to call your configuration file. The basic idea is that each application has its own default configuration filename. The default named used by the ipython command line program is ipython_config.py, and all IPython applications will use this file. Other applications, such as the parallel ipcluster scripts or the QtConsole will load their own config files after ipython_config.py. To load a particular configuration file instead of the default, the name can be overridden by the config_file command line flag.
To generate the default configuration files, do:
$> ipython profile create
and you will have a default ipython_config.py in your IPython directory under profile_default. If you want the default config files for the IPython.parallel applications, add --parallel to the end of the command-line args.
A profile is a directory containing configuration and runtime files, such as logs, connection info for the parallel apps, and your IPython command history.
The idea is that users often want to maintain a set of configuration files for different purposes: one for doing numerical computing with NumPy and SciPy and another for doing symbolic computing with SymPy. Profiles make it easy to keep a separate configuration files, logs, and histories for each of these purposes.
Let’s start by showing how a profile is used:
$ ipython --profile=sympy
This tells the ipython command line program to get its configuration from the “sympy” profile. The file names for various profiles do not change. The only difference is that profiles are named in a special way. In the case above, the “sympy” profile means looking for ipython_config.py in <IPYTHON_DIR>/profile_sympy.
The general pattern is this: simply create a new profile with:
ipython profile create <name>
which adds a directory called profile_<name> to your IPython directory. Then you can load this profile by adding --profile=<name> to your command line options. Profiles are supported by all IPython applications.
IPython ships with some sample profiles in IPython/config/profile. If you create profiles with the name of one of our shipped profiles, these config files will be copied over instead of starting with the automatically generated config files.
If you are using the notebook, qtconsole, or parallel code, IPython stores connection information in small JSON files in the active profile’s security directory. This directory is made private, so only you can see the files inside. If you need to move connection files around to other computers, this is where they will be. If you want your code to be able to open security files by name, we have a convenience function IPython.utils.path.get_security_file(), which will return the absolute path to a security file from its filename and [optionally] profile name.
If you want some code to be run at the beginning of every IPython session with a particular profile, the easiest way is to add Python (.py) or IPython (.ipy) scripts to your <profile>/startup directory. Files in this directory will always be executed as soon as the IPython shell is constructed, and before any other code or scripts you have specified. If you have multiple files in the startup directory, they will be run in lexicographical order, so you can control the ordering by adding a ‘00-‘ prefix.
Note
Automatic startup files are new in IPython 0.12. Use the InteractiveShellApp.exec_files configurable for similar behavior in 0.11.
IPython exposes all configurable options on the command-line. The command-line arguments are generated from the Configurable traits of the classes associated with a given Application. Configuring IPython from the command-line may look very similar to an IPython config file
IPython applications use a parser called KeyValueLoader to load values into a Config object. Values are assigned in much the same way as in a config file:
$> ipython --InteractiveShell.use_readline=False --BaseIPythonApplication.profile='myprofile'
Is the same as adding:
c.InteractiveShell.use_readline=False
c.BaseIPythonApplication.profile='myprofile'
to your config file. Key/Value arguments always take a value, separated by ‘=’ and no spaces.
Since the strictness and verbosity of the KVLoader above are not ideal for everyday use, common arguments can be specified as flags or aliases.
Flags and Aliases are handled by argparse instead, allowing for more flexible parsing. In general, flags and aliases are prefixed by --, except for those that are single characters, in which case they can be specified with a single -, e.g.:
$> ipython -i -c "import numpy; x=numpy.linspace(0,1)" --profile testing --colors=lightbg
For convenience, applications have a mapping of commonly used traits, so you don’t have to specify the whole class name:
$> ipython --profile myprofile
# and
$> ipython --profile='myprofile'
# are equivalent to
$> ipython --BaseIPythonApplication.profile='myprofile'
Applications can also be passed flags. Flags are options that take no arguments. They are simply wrappers for setting one or more configurables with predefined values, often True/False.
For instance:
$> ipcontroller --debug
# is equivalent to
$> ipcontroller --Application.log_level=DEBUG
# and
$> ipython --pylab
# is equivalent to
$> ipython --pylab=auto
# or
$> ipython --no-banner
# is equivalent to
$> ipython --TerminalIPythonApp.display_banner=False
Some IPython applications have subcommands. Subcommands are modeled after git, and are called with the form command subcommand [...args]. Currently, the QtConsole is a subcommand of terminal IPython:
$> ipython qtconsole --profile=myprofile
and ipcluster is simply a wrapper for its various subcommands (start, stop, engines).
$> ipcluster start --profile=myprofile --n=4
To see a list of the available aliases, flags, and subcommands for an IPython application, simply pass -h or --help. And to see the full list of configurable options (very long), pass --help-all.
Here are the main requirements we wanted our configuration system to have: