.. _libdoc_config:

=======================================
:mod:`config` -- Theano Configuration 
=======================================

.. module:: config
   :platform: Unix, Windows
   :synopsis: Library configuration attributes.
.. moduleauthor:: LISA


Guide
=====

The config module contains many ``attributes`` that modify Theano's behavior.  Many of these
attributes are consulted during the import of the ``theano`` module and many are assumed to be
read-only.

*As a rule, the attributes in this module should not be modified by user code.*

Theano's code comes with default values for these attributes, but you can
override them from your .theanorc file, and override those values in turn by
the :envvar:`THEANO_FLAGS` environment variable.

The order of precedence is:

1. an assignment to theano.config.<property>
2. an assignment in :envvar:`THEANO_FLAGS`
3. an assignment in the .theanorc file (or the file indicated in :envvar:`THEANORC`)

You can print out the current/effective configuration at any time by printing
theano.config.  For example, to see a list  of all active configuration
variables, type this from the command-line:

.. code-block:: bash

    python -c 'import theano; print theano.config' | less

Environment Variables
=====================


.. envvar:: THEANO_FLAGS

    This is a list of comma-delimited key=value pairs that control
    Theano's behavior.

    For example, in bash, you can override your :envvar:`THEANORC` defaults
    for <myscript>.py by typing this:

    .. code-block:: bash

        THEANO_FLAGS='floatX=float32,device=gpu0,nvcc.fastmath=True'  python <myscript>.py

    If a value is defined several times in ``THEANO_FLAGS``,
    the right-most definition is used. So, for instance, if
    ``THEANO_FLAGS='device=cpu,device=gpu0'``, then gpu0 will be used.

.. envvar:: THEANORC

    The location[s] of the .theanorc file[s] in ConfigParser format.
    It defaults to ``$HOME/.theanorc``. On Windows, it defaults to
    ``$HOME/.theanorc:$HOME/.theanorc.txt`` to make Windows users' life
    easier.

    Here is the .theanorc equivalent to the THEANO_FLAGS in the example above:

    .. code-block:: cfg

        [global]
        floatX = float32
        device = gpu0

        [nvcc]
        fastmath = True

    Configuration attributes that are available directly in ``config``
    (e.g. ``config.device``, ``config.mode``) should be defined in the
    ``[global]`` section.
    Attributes from a subsection of ``config`` (e.g. ``config.nvcc.fastmath``,
    ``config.blas.ldflags``) should be defined in their corresponding section
    (e.g. ``[nvcc]``, ``[blas]``).

    Multiple configuration files can be specified by separating them with ':'
    characters (as in $PATH).  Multiple configuration files will be merged,
    with later (right-most) files taking priority over earlier files in the
    case that multiple files specify values for a common configuration option.
    For example, to override system-wide settings with personal ones,
    set ``THEANORC=/etc/theanorc:~/.theanorc``.

Config Attributes
=====================

The list below describes some of the more common and important flags
that you might want to use. For the complete list (including documentation),
import theano and print the config variable, as in:

.. code-block:: bash

    python -c 'import theano; print theano.config' | less

.. attribute:: device

    String value: either ``'cpu'``, ``'gpu'``, ``'gpu0'``, ``'gpu1'``,
    ``'gpu2'``, or ``'gpu3'``

    Default device for computations. If ``gpu*``, change the default to try
    to move computation to it and to put shared variable of float32 on
    it.
    Choose the default compute device for theano graphs.  Setting this to a
    ``gpu*`` string will make theano to try by default to move computation to it.
    Also it will make theano put by default shared variable of float32 on it.
    ``'gpu'`` lets the driver select the GPU to use, while ``'gpu?'`` makes Theano try
    to use a specific device. If we are not able to use the GPU, either we fall back
    on the CPU, or an error is raised, depending on the :attr:`force_device` flag.

    This flag's value cannot be modified during the program execution.

    Do not use upper case letters, only lower case even if NVIDIA use
    capital letters.

.. attribute:: force_device

    Bool value: either ``True`` or ``False``

    Default: ``False``

    If ``True``, we raise an error if we cannot use the specified :attr:`device`.
    If ``False``, we fall back to the CPU.

    This flag's value cannot be modified during the program execution.

.. attribute:: init_gpu_device

    String value: either ``''``, ``'gpu'``, ``'gpu0'``, ``'gpu1'``, ``'gpu2'``,
    or ``'gpu3'``

    Initialize the gpu device to use.
    When its value is gpu*, the theano flag :attr:`device` must be ``"cpu"``.
    Unlike :attr:`device`, setting this flag to a specific GPU will not
    try to use this device by default, in particular it will **not** move
    computations, nor shared variables, to the specified GPU.

    This flag is useful to run GPU-specific tests on a particular GPU, instead
    of using the default one.

    This flag's value cannot be modified during the program execution.

.. attribute:: config.pycuda.init

    Bool value: either ``True`` or ``False``

    Default: ``False``

    If True, always initialize PyCUDA when Theano want to initialize
    the GPU.  With PyCUDA version 2011.2.2 or earlier, PyCUDA must
    initialize the GPU before Theano does it.  Setting
    this flag to True, ensure that, but always import PyCUDA.  It can
    be done manually by importing theano.misc.pycuda_init before
    Theano initialize the GPU device.  Newer version of PyCUDA
    (currently only in the trunk) don't have this restriction.

.. attribute:: config.print_active_device

    Bool value: either ``True`` or ``False``

    Default: ``True``

    Print active device at when the GPU device is initialized.

.. attribute:: floatX

    String value: either 'float64' or 'float32'

    Default: 'float64'

    This sets the default dtype returned by tensor.matrix(), tensor.vector(),
    and similar functions.  It also sets the default theano bit width for
    arguments passed as Python floating-point numbers.

.. attribute:: openmp

    Bool value: either True or False

    Default: True if the environment variable OMP_NUM_THREADS!=1 or
             if we detect more then 1 CPU core. Otherwise False.

    Enable or not parallel computation on the CPU with OpenMP.
    It is the default value used when creating an Op that support it.
    The best is to define it via Theano configuration
    file or with the environment variable THEANO_FLAGS.

.. attribute:: cast_policy

    String value: either 'numpy+floatX' or 'custom'

    Default: 'custom'

    This specifies how data types are implicitly figured out in Theano, e.g. for
    constants or in the results of arithmetic operations. The 'custom' value
    corresponds to a set of custom rules originally used in
    Theano (which can be partially customized, see e.g. the in-code help of
    ``tensor.NumpyAutocaster``), and will be deprecated in the future.
    The 'numpy+floatX' setting attempts to mimic the numpy casting rules,
    although it prefers to use float32 numbers instead of float64 when
    ``config.floatX`` is set to 'float32' and the user uses data that is not
    explicitly typed as float64 (e.g. regular Python floats).
    Note that 'numpy+floatX' is not currently behaving exactly as planned (it
    is a work-in-progress), and thus you should consider it as experimental.
    At the moment it behaves differently from numpy in the following
    situations:

    * Depending on the value of ``config.int_division``, the resulting type
      of a division of integer types with the ``/`` operator may not match
      that of numpy.
    * On mixed scalar / array operations, numpy tries to prevent the scalar
      from upcasting the array's type unless it is of a fundamentally
      different type. Theano does not attempt to do the same at this point,
      so you should be careful that scalars may upcast arrays when they
      would not when using numpy. This behavior should change in the near
      future.

.. attribute:: int_division

    String value: either 'int', 'floatX' or 'raise'

    Default: 'int'

    Specifies what to do when one tries to compute ``x / y``, where both ``x`` and
    ``y`` are of integer types (possibly unsigned). 'int' means an integer is
    returned (as in Python 2.X), but this behavior is deprecated. 'floatX'
    returns a number of type given by ``config.floatX``. 'raise' is the safest
    choice (and will become default in a future release of Theano) and raises
    an error when one tries to do such an operation, enforcing the use of the
    integer division operator (``//``) (if a float result is intended, either
    cast one of the arguments to a float, or use ``x.__truediv__(y)``).

.. attribute:: mode

    String value: 'Mode', 'ProfileMode', 'DebugMode', 'FAST_RUN', 'FAST_COMPILE'

    Default 'Mode'

    This sets the default compilation mode for theano functions. By default the
    mode Mode is equivalent to FAST_RUN. See Config attribute linker and optimizer.

.. attribute:: config.lib.amdlibm

    Bool value: either True or False

    Default False

    This makes the compilation use the
    `amdlibm <http://developer.amd.com/cpu/libraries/libm/>`__
    library, which is faster than the standard libm.

.. attribute:: linker

    String value: 'c|py', 'py', 'c', 'c|py_nogc', 'c&py'

    Default: 'c|py'

    When the mode is Mode, it sets the default linker used.
    See :ref:`using_modes` for a comparison of the different linkers.

.. attribute:: optimizer

    String value: 'fast_run', 'merge', 'fast_compile', 'None'

    Default: 'fast_run'

    When the mode is Mode, it sets the default optimizer used.

.. attribute:: on_opt_error

    String value: 'warn' or 'raise'

    Default: 'warn'

    When a crash occurs while trying to apply some optimization, either warn
    the user and skip this optimization ('warn'), or raise the exception
    ('raise').

.. attribute:: on_shape_error

    String value: 'warn' or 'raise'

    Default: 'warn'

    When an exception is raised when inferring the shape of some apply
    node, either warn the user and use a default value ('warn'), or
    raise the exception ('raise').

.. attribute:: config.warn.ignore_bug_before

    String value: 'None', 'all', '0.3', '0.4', '0.4.1', '0.5', '0.6'

    Default: 'None'

    When we fix a Theano bug that generated bad results under some
    circumstances, we also make Theano raise a warning when it encounters
    the same circumstances again. This helps to detect if said bug
    had affected your past experiments, as you only need to run your
    experiment again with the new version, and you do not have to
    understand the Theano internal that triggered the bug. A better
    way to detect this will be implemented. See this `ticket
    <http://www.assembla.com/spaces/theano/tickets/514>`__.

    This flag allows new users not to get warnings about old bugs, that were
    fixed before their first checkout of Theano.
    You can set its value to the first version of Theano
    that you used (probably 0.3 or higher)

    `None` means that all warnings will be displayed.
    `all` means all warnings will be ignored.

    It is recommended that you put a version, so that you will see future
    warnings.
    It is also recommended you put this into your .theanorc, so this setting
    will always be used.

    This flag's value cannot be modified during the program execution.

.. attribute:: base_compiledir

    Default: On Windows: $LOCALAPPDATA\\Theano if $LOCALAPPDATA is defined,
    otherwise and on other systems: ~/.theano.

    This directory stores the platform-dependent compilation directories.

    This flag's value cannot be modified during the program execution.

.. attribute:: compiledir_format

    Default: "compiledir_%(platform)s-%(processor)s-%(python_version)s"

    This is a Python format string that specifies the subdirectory
    of ``config.base_compiledir`` in which to store platform-dependent
    compiled modules. To see a list of all available substitution keys,
    run ``python -c "import theano; print theano.config"``, and look
    for compiledir_format.

    This flag's value cannot be modified during the program execution.

.. attribute:: compiledir

    Default: ``config.base_compiledir``/``config.compiledir_format``

    This directory stores dynamically-compiled modules for a particular
    platform.

    This flag's value cannot be modified during the program execution.

.. attribute:: config.blas.ldflags

    Default: '-lblas'

    Link arguments to link against a (Fortran) level-3 blas implementation.

.. attribute:: config.cuda.root

    Default: $CUDA_ROOT or failing that, "/usr/local/cuda"

    A directory with bin/, lib/, include/ folders containing cuda utilities.

.. attribute:: config.gcc.cxxflags

    Default: ""

    Extra parameters to pass to gcc when compiling.  Extra include paths,
    library paths, configuration options, etc.

.. attribute:: config.cxx


    Default: 'g++' if g++ is present. '' Otherwise.

    Tell the c++ compiler to use. If empty, don't compile c++ code.
    We automatically detect if g++ is present and disable it if not
    present.

    We print a warning if we detect that g++ is not present. It is
    recommended to run with c++ compilation as Theano will be much
    slower otherwise.

    Currently only g++ is supported, but supporting others is easy.

.. attribute:: optimizer_excluding

    Default: ""

    A list of optimizer tags that we don't want included in the default Mode.
    If multiple tags, separate them by ':'.
    Ex: to remove the elemwise inplace optimizer(slow for big graph),
    use the flags: optimizer_excluding:inplace_opt, where
    inplace_opt is the name of that optimization.

    This flag's value cannot be modified during the program execution.

.. attribute:: optimizer_including

    Default: ""

    A list of optimizer tags that we want included in the default Mode.
    If multiple tags, separate them by ':'.

    This flag's value cannot be modified during the program execution.

.. attribute:: optimizer_requiring

    Default: ""

    A list of optimizer tags that we require for optimizer in the default Mode.
    If multiple tags, separate them by ':'.

    This flag's value cannot be modified during the program execution.

.. attribute:: nocleanup

    Bool value: either True or False

    Default: False

    If False, source code files are removed when they are not needed anymore.
    This means files whose compilation failed are deleted.
    Set to True to keep those files in order to debug compilation errors.

.. attribute:: config.DebugMode

    This section contains various attributes configuring the behaviour
    of mode :class:`~debugmode.DebugMode`. See directly this section
    for the documentation of more configuration options.

.. attribute:: config.DebugMode.check_preallocated_output

    Default: ``''``

    A list of kinds of preallocated memory to use as output buffers for
    each Op's computations, separated by ``:``. Implemented modes are:

    * ``"initial"``: initial storage present in storage map
      (for instance, it can happen in the inner function of Scan),
    * ``"previous"``: reuse previously-returned memory,
    * ``"c_contiguous"``: newly-allocated C-contiguous memory,
    * ``"f_contiguous"``: newly-allocated Fortran-contiguous memory,
    * ``"strided"``: non-contiguous memory with various stride patterns,
    * ``"wrong_size"``: memory with bigger or smaller dimensions,
    * ``"ALL"``: placeholder for all of the above.

    In order not to test with preallocated memory, use an empty string, ``""``.

.. attribute:: config.DebugMode.check_preallocated_output_ndim

    Positive int value, default: 4.

    When testing with "strided" preallocated output memory, test
    all combinations of strides over that number of (inner-most)
    dimensions. You may want to reduce that number to reduce memory or
    time usage, but it is advised to keep a minimum of 2.

.. attribute:: config.DebugMode.warn_input_not_reused

    Bool value, default: True

    Generate a warning when the destroy_map or view_map tell that an op work
    inplace, but the op did not reuse the input for its output.

.. attribute:: config.numpy

    This section contains different attributes for configuring numpy's
    behaviour, described by `numpy.seterr
    <http://docs.scipy.org/doc/numpy/reference/generated/numpy.seterr.html>`__.

.. attribute:: config.numpy.seterr_all

    String Value: ``'ignore'``, ``'warn'``, ``'raise'``, ``'call'``,
    ``'print'``, ``'log'``, ``'None'``

    Default: ``'ignore'``

    Set the default behaviour described by `numpy.seterr
    <http://docs.scipy.org/doc/numpy/reference/generated/numpy.seterr.html>`__.

    ``'None'`` means that numpy's default behaviour will not be changed (unless
    one of the other `config.numpy.seterr_*` overrides it), but this behaviour
    can change between numpy releases.

    This flag sets the default behaviour for all kinds of floating-pont
    errors, and it can be overriden for specific errors by setting one
    (or more) of the flags below.

    This flag's value cannot be modified during the program execution.

.. attribute:: config.numpy.seterr_divide

    String Value: ``'None'``, ``'ignore'``, ``'warn'``, ``'raise'``,
    ``'call'``, ``'print'``, ``'log'``

    Default: ``'None'``

    Sets numpy's behavior for division by zero. ``'None'`` means using the
    default, defined by config.numpy.seterr_all.

    This flag's value cannot be modified during the program execution.

.. attribute:: config.numpy.seterr_over

    String Value: ``'None'``, ``'ignore'``, ``'warn'``, ``'raise'``,
    ``'call'``, ``'print'``, ``'log'``

    Default: ``'None'``

    Sets numpy's behavior for floating-point overflow. ``'None'`` means
    using the default, defined by config.numpy.seterr_all.

    This flag's value cannot be modified during the program execution.

.. attribute:: config.numpy.seterr_under

    String Value: ``'None'``, ``'ignore'``, ``'warn'``, ``'raise'``,
    ``'call'``, ``'print'``, ``'log'``

    Default: ``'None'``

    Sets numpy's behavior for floating-point underflow. ``'None'`` means
    using the default, defined by config.numpy.seterr_all.

    This flag's value cannot be modified during the program execution.

.. attribute:: config.numpy.seterr_invalid

    String Value: ``'None'``, ``'ignore'``, ``'warn'``, ``'raise'``,
    ``'call'``, ``'print'``, ``'log'``

    Default: ``'None'``

    Sets numpy's behavior for invalid floating-point operation. ``'None'``
    means using the default, defined by :attr:`config.numpy.seterr_all`.

    This flag's value cannot be modified during the program execution.

.. attribute:: config.compute_test_value

    String Value: ``'off'``, ``'ignore'``, ``'warn'``, ``'raise'``.

    Default: ``'off'``

    Setting this attribute to something other than ``'off'`` activates a
    debugging mechanism, where Theano executes the graph on-the-fly, as it is
    being built. This allows the user to spot errors early on (such as
    dimension mis-match), **before** optimizations are applied.
   
    Theano will execute the graph using the Constants and/or shared variables
    provided by the user. Purely symbolic variables (e.g. x = T.dmatrix()) can be
    augmented with test values, by writing to their ``'tag.test_value'``
    attribute (e.g. x.tag.test_value = numpy.random.rand(5,4)).

    When not ``'off'``, the value of this option dictates what happens when
    an Op's inputs do not provide appropriate test values:

        - ``'ignore'`` will silently skip the debug mechanism for this Op
        - ``'warn'`` will raise a UserWarning and skip the debug mechanism for
          this Op
        - ``'raise'`` will raise an Exception
    
.. attribute:: config.exception_verbosity

    String Value: ``'low'``, ``'high'``.

    Default: ``'low'``
  
    If ``'low'``, the text of exceptions will generally refer to apply nodes
    with short names such as ``'Elemwise{add_no_inplace}'``. If ``'high'``,
    some exceptions will also refer to apply nodes with long descriptions like:

    ::

        A. Elemwise{add_no_inplace}
              B. log_likelihood_v_given_h
              C. log_likelihood_h

.. attribute:: config.time_seq_optimizer

    Bool value, default: False

    Should each SeqOptimizer object print the time taken by each of its
    optimizer. Each SeqOptimizer prints something like this::

        SeqOptimizer gpu_opt  time 0.014s for 8/9 nodes before/after optimization
          0.012573s - ('gpu_local_optimizations', 'EquilibriumOptimizer')
          0.001373s - ('gpu_cut_transfers', 'EquilibriumOptimizer')
          0.000441s - ('InputToGpuOptimizer', 'InputToGpuOptimizer')

    This prints the name of the SeqOptimizer (gpu_opt), the number of
    Apply nodes in the graph before (8) and after (9)
    optimizations. Then a list of lines, one per optimization
    in this SeqOptimizer. The first element is the time
    taken by this optimization and then it is a tuple with the name of the
    optimization and this class. This list is sorted from the sub
    optimization that takes the most time to the optimization that takes
    the least time.

.. attribute:: config.time_eq_optimizer

    Bool value, default: False

    Should each EquilibriumOptimizer print the time taken by each of its
    iterations, the total number of times each of its optimizers is applied,
    and informations about the total number of nodes in the graph.
    Here is an example of output::

        EquilibriumOptimizer specialize
         time 4.760s for 4 passes, 3801 nodes max
        0 - 1.961s (0.079s in global opts) - 3797 nodes
        1 - 1.233s (0.080s in global opts) - 3801 nodes
        2 - 0.857s (0.071s in global opts) - 3203 nodes
        3 - 0.710s (0.066s in global opts) - 3095 nodes

        times applied - optimizer:
          384 - dimshuffle_as_view
          262 - constant_folding
          216 - local_subtensor_make_vector
          216 - local_shape_to_shape_i
          4 - local_mul_specialize

.. attribute:: config.cmodule.warn_no_version

    Bool value, default: False

    If True, will print a warning when compiling one or more Op with C
    code that can't be cached because there is no c_code_cache_version()
    function associated to at least one of those Ops.

.. attribute:: config.cmodule.mac_framework_link

    Bool value, default: False

    If set to True, breaks certain MacOS installations with the infamous
    Bus Error.

.. attribute:: config.cmodule.remove_gxx_opt

    Bool value, default: False

    If True, will remove -O* parameter passed to g++.
    This is useful to debug in gdb module compiled by Theano.
    The parameter -g is passed by default to g++.

.. attribute:: cmodule.compilation_warning

    Bool value, default: False

    If True, will print compilation warning.
