Buildouts
=========

The word "buildout" refers to a description of a set of parts and the
software to create and assemble them.  It is often used informally to
refer to an installed system based on a buildout definition.  For
example, if we are creating an application named "Foo", then "the Foo
buildout" is the collection of configuration and application-specific
software that allows an instance of the application to be created.  We
may refer to such an instance of the application informally as "a Foo
buildout".  

This document describes how to define buildouts using buildout
configuration files and recipes.  There are three ways to set up the
buildout software and create a buildout instance:

1. Install the zc.buildout egg with easy_install and use the buildout
   script installed in a Python scripts area.

2. Use the buildout bootstrap script to create a buildout that
   includes both the setuptools and zc.buildout eggs.  This allows you
   to use the buildout software without modifying a Python install.
   The buildout script is installed into your buildout local scripts
   area.

3. Use a buildoput command from an already installed buildout to 
   bootstrap a new buildout.  (See the section on bootstraping later
   in this document.)

Often, a software project will be managed in a software repository,
such as a subversion repository, that includes some software source
directories, buildout configuration files, and a copy of the buildout
bootstrap script,  To work on the project, one would check out the
project from the repository and run the bootstrap script which
installs setuptools and zc.buildout into the checkout as well as any
parts defined.

We have a sample buildout that we created using the bootstrap command
of an existing buildout (method 3 above).  It has the absolute minimum
information.  We have bin, develop-eggs, eggs and parts directories,
and a configuration file:
    
    >>> ls(sample_buildout)
    d  bin
    -  buildout.cfg
    d  develop-eggs
    d  eggs
    d  parts

The bin directory contains scripts.

    >>> ls(sample_buildout, 'bin')
    -  buildout

    >>> ls(sample_buildout, 'eggs')
    -  setuptools-0.6-py2.4.egg
    -  zc.buildout-1.0-py2.4.egg

The develop-eggs and parts directories are initially empty:

    >>> ls(sample_buildout, 'develop-eggs')
    >>> ls(sample_buildout, 'parts')

The develop-eggs directory holds egg links for software being
developed in the buildout.  We separate develop-eggs and other eggs to
allow eggs directories to be shared across multiple buildouts.  For
example, a common developer technique is to define a common eggs
directory in their home that all non-develop eggs are stored in.  This
allows larger buildouts to be set up much more quickly and saves disk
space.

The parts directory provides an area where recipes can install
part data.  For example, if we built a custom Python, we would
install it in the part directory.  Part data is stored in a
sub-directory of the parts directory with the same name as the part.

Buildouts are defined using configuration files.  These are in the
format defined by the Python ConfigParser module, with extensions
that we'll describe later.  By default, when a buildout is run, it
looks for the file buildout.cfg in the directory where the buildout is
run.

The minimal configuration file has a buildout section that defines no
parts:

    >>> cat(sample_buildout, 'buildout.cfg')
    [buildout]
    parts =

A part is simply something to be created by a buildout.  It can be
almost anything, such as a Python package, a program, a directory, or
even a configuration file.  

A part is created by a recipe.  Recipes are always installed as Python
eggs. They can be downloaded from a package server, such as the
Python Package Index, or they can be developed as part of a project.
Let's create a recipe as part of the sample project.  We'll create a
recipe for creating directories.  

First, we'll create a recipes directory for
our local recipes:

    >>> mkdir(sample_buildout, 'recipes')

and then we'll create a source file for our mkdir recipe:

    >>> write(sample_buildout, 'recipes', 'mkdir.py', 
    ... """
    ... import logging, os, zc.buildout
    ...
    ... class Mkdir:
    ...
    ...     def __init__(self, buildout, name, options):
    ...         self.buildout = buildout
    ...         self.name = name
    ...         self.options = options
    ...         options['path'] = os.path.join(
    ...                               buildout['buildout']['directory'],
    ...                               options['path'],
    ...                               )
    ...         if not os.path.isdir(os.path.dirname(options['path'])):
    ...             logging.getLogger(self.name).error(
    ...                 'Cannot create %s. %s is not a directory.',
    ...                 options['path'], os.path.dirname(options['path']))
    ...             raise zc.buildout.UserError('Invalid Path')
    ...             
    ...
    ...     def install(self):
    ...         path = self.options['path']
    ...         if not os.path.isdir(path):
    ...             logging.getLogger(self.name).info(
    ...                 'Creating directory %s', os.path.basename(path))
    ...             os.mkdir(path)
    ...         return path
    ... """)

The recipe defines a constructor that takes a buildout object, a part
name, and an options dictionary. It saves them in instance attributes.

If the path is relative, we'll interpret it as relative to the
buildout directory.  The buildout object passed in is a mapping from
section name to a mapping of options for that section. The buildout
directory is available as the directory option of the buildout
section.  We normalize the path and save it back into the options
directory.  

Any time we use data from another section, it is important to reflect
that data in the recipe's options when the recipe is constructed.

When buildout is run, it saves configuration data for installed parts
in a file named installed.cfg.  In subsequent runs, it compares
part-configuration data stored in the installed.cfg file and the
part-configuration data loaded from the configuration files as
modified by recipe constructors to decide if the configuration of a
part has changed. If the configuration has changed, or if the recipe
has changed, then the part is uninstalled before reinstalling it.  The
buildout only looks at the part's options, so any data used to
configure the part needs to be reflected in the part's options.  It is
the job of a recipe constructor to make sure that the options include
all rel event data.

Of course, parts are also uninstalled if they are no-longer used.

The install method is responsible for creating the part.  In this
case, we need the path of the directory to create.  We'll use a
path option from our options dictionary.

The install method logs what it's doing using the Python logging call.

We return the path that we installed.  If the part is uninstalled or
reinstalled, then the path returned will be removed by the buildout
machinery.  A recipe install method is expected to return None, a
string, or an iterable of strings containing paths to be removed if a
part is uninstalled.  For most recipes, this is all of the uninstall
support needed.  A recipe can provide custom uninstall support as will
be described later.

We need to provide packaging information so that our recipe can be
installed as an egg.  We need to define a setup script for this:

    >>> write(sample_buildout, 'recipes', 'setup.py',
    ... """
    ... from setuptools import setup
    ... 
    ... setup(
    ...     name = "recipes",
    ...     entry_points = {'zc.buildout': ['mkdir = mkdir:Mkdir']},
    ...     )
    ... """)

This setup script is incomplete.  It doesn't describe what is to be 
included in a distribution.  This is fine if we never actually create
a distribution. If recipes are going to be used only internally in a
buildout, then we needn't include distribution information.  If we
wanted to use the same recipes in multiple buildouts, then we'd need
to include proper distribution data.  To find out more about creating
distributions, see the setuptools documentation.

Our setup script defines an entry point. Entry points provide
a way for an egg to define the services it provides.  Here we've said
that we define a zc.buildout entry point named default.  Recipe
classes must be exposed as entry points in the zc.buildout group.  we
give entry points names within the group.  The name "default" is
somewhat special because it allows a recipe to be referenced using a
package name without naming an entry point.

We also need a README.txt for our recipes to avoid an annoying warning
from distutils, on which setuptools and zc.buildout are based:

    >>> write(sample_buildout, 'recipes', 'README.txt', " ")

Now let's update our buildout.cfg:

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... develop = recipes
    ... parts = data-dir
    ...
    ... [data-dir]
    ... recipe = recipes:mkdir
    ... path = mystuff
    ... """)

Let's go through the changes one by one::

    develop = recipes

This tells the buildout to install a development egg for our recipes.
Any number of paths can be listed.  The paths can be relative or
absolute.  If relative, they are treated as relative to the buildout
directory.  They can be directory or file paths.  If a file path is
given, it should point to a Python setup script.  If a directory path
is given, it should point to a directory containing a setup.py file.
Development eggs are installed before building any parts, as they may
provide locally-defined recipes needed by the parts.

::

    parts = data-dir

Here we've named a part to be "built".  We can use any name we want
except that different part names must be unique and recipes will often
use the part name to decide what to do.

::

    [data-dir]
    recipe = recipes:mkdir
    path = mystuff    


When we name a part, we also create a section of the same
name that contains part data.  In this section, we'll define
the recipe to be used to install the part.  In this case, we also
specify the path to be created.

Let's run the buildout.  We do so by running the build script in the
buildout:

    >>> import os
    >>> os.chdir(sample_buildout)
    >>> buildout = os.path.join(sample_buildout, 'bin', 'buildout')
    >>> print system(buildout),
    buildout: Develop: /tmp/sample-buildout/recipes/setup.py
    buildout: Installing data-dir
    data-dir: Creating directory mystuff

We see that the recipe created the directory, as expected:

    >>> ls(sample_buildout)
    -  .installed.cfg
    d  bin
    -  buildout.cfg
    d  develop-eggs
    d  eggs
    d  mystuff
    d  parts
    d  recipes

In addition, .installed.cfg has been created containing information
about the part we installed:

    >>> cat(sample_buildout, '.installed.cfg')
    [buildout]
    parts = data-dir
    <BLANKLINE>
    [data-dir]
    __buildout_installed__ = /tmp/sample-buildout/mystuff
    __buildout_signature__ = recipes-c7vHV6ekIDUPy/7fjAaYjg==
    path = /tmp/sample-buildout/mystuff
    recipe = recipes:mkdir

Note that the directory we installed is included in .installed.cfg.
In addition, the path option includes the actual destination
directory. 

If we change the name of the directory in the configuration file,
we'll see that the directory gets removed and recreated:

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... develop = recipes
    ... parts = data-dir
    ...
    ... [data-dir]
    ... recipe = recipes:mkdir
    ... path = mydata
    ... """)

    >>> print system(buildout),
    buildout: Develop: /tmp/sample-buildout/recipes/setup.py
    buildout: Uninstalling data-dir
    buildout: Installing data-dir
    data-dir: Creating directory mydata

    >>> ls(sample_buildout)
    -  .installed.cfg
    d  bin
    -  buildout.cfg
    d  develop-eggs
    d  eggs
    d  mydata
    d  parts
    d  recipes


Error reporting
---------------

If a user makes an error, an error needs to be printed and work needs
to stop.  This is accomplished by logging a detailed error message and
then raising a (or an instance of a subclass of a)
zc.buildout.UserError exception.  Raising UserError causes the
buildout to print the error and exit without printing a traceback.  In
the sample above, of someone gives a non-existant directory to create
the directory in:


    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... develop = recipes
    ... parts = data-dir
    ...
    ... [data-dir]
    ... recipe = recipes:mkdir
    ... path = /xxx/mydata
    ... """)

We'll get a user error, not a traceback.

    >>> print system(buildout),
    buildout: Develop: /private/tmp/tmp_I5pHasample-buildout/recipes/setup.py
    data-dir: Cannot create /xxx/mydata. /xxx is not a directory.
    Error: Invalid Path


Configuration file syntax
-------------------------

As mentioned earlier, buildout configuration files use the format
defined by the Python ConfigParser module with extensions.  The
extensions are:

- option names are case sensitive

- option values can ue a substitution syntax, described below, to
  refer to option values in specific sections.

The ConfigParser syntax is very flexible.  Section names can contain
any characters other than newlines and right square braces ("]").
Option names can contain any characters other than newlines, colons,
and equal signs, can not start with a space, and don't include
trailing spaces.

It is likely that, in the future, some characters will be given
special buildout-defined meanings.  This is already true of the
characters ":", "$", "%", "(", and ")".  For now, it is a good idea to
keep section and option names simple, sticking to alphanumeric
characters, hyphens, and periods.

Variable substitutions
----------------------

Buildout configuration files support two kinds of substitutions,
standard ConfigParser substitutions, and string-template
substitutions.  To illustrate this, we'll create an debug recipe to
allow us to see interactions with the buildout:

    >>> write(sample_buildout, 'recipes', 'debug.py', 
    ... """
    ... class Debug:
    ...
    ...     def __init__(self, buildout, name, options):
    ...         self.buildout = buildout
    ...         self.name = name
    ...         self.options = options
    ...
    ...     def install(self):
    ...         items = self.options.items()
    ...         items.sort()
    ...         for option, value in items:
    ...             print option, value
    ... """)

In this example, we've used a simple base class that provides a
boilerplate constructor.  This recipe doesn't actually create
anything. The install method doesn't return anything, because it
didn't create any files or directories.

We also have to update our setup script:

    >>> write(sample_buildout, 'recipes', 'setup.py',
    ... """
    ... from setuptools import setup
    ... entry_points = (
    ... '''
    ... [zc.buildout]
    ... mkdir = mkdir:Mkdir
    ... debug = debug:Debug
    ... ''')
    ... setup(name="recipes", entry_points=entry_points)
    ... """)

We've rearranged the script a bit to make the entry points easier to
edit.  In particular, entry points are now defined as a configuration
string, rather than a dictionary.

Let's update our configuration to provide variable substitution
examples:

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... develop = recipes
    ... parts = data-dir debug
    ... log-level = INFO
    ...
    ... [debug]
    ... recipe = recipes:debug
    ... File 1 = ${data-dir:path}/file
    ... File 2 = %(File 1)s.out
    ... File 3 = %(base)s/file3
    ... File 4 = ${debug:File 3}/log
    ...
    ... [data-dir]
    ... recipe = recipes:mkdir
    ... path = mydata
    ...
    ... [DEFAULT]
    ... base = var
    ... """)

In this example, we've used ConfigParser substitutions for file2 and
file3.  This type of substitution uses Python string format syntax.
Valid names are options in the same section and options defined in the
DEFAULT section.  

We used a string-template substitution for file1.  This type of
substitution uses the string.Template syntax.  Names substituted are
qualified option names, consisting of a section name and option name
joined by a colon.

Now, if we run the buildout, we'll see the options with the values
substituted. 

    >>> print system(buildout),
    buildout: Develop: /tmp/sample-buildout/recipes/setup.py
    buildout: Uninstalling data-dir
    buildout: Installing data-dir
    data-dir: Creating directory mydata
    buildout: Installing debug
    File 1 mydata/file
    File 2 mydata/file.out
    File 3 var/file3
    File 4 var/file3/log
    base var
    recipe recipes:debug

It might seem surprising that mydata was created again.  This is
because we changed our recipes package by adding the debug module.
The buildout system didn't know if this module could effect the mkdir
recipe, so it assumed it could and reinstalled mydata.  If we rerun
the buildout:

    >>> print system(buildout),
    buildout: Develop: /tmp/sample-buildout/recipes/setup.py
    buildout: Installing data-dir
    buildout: Installing debug
    File 1 mydata/file
    File 2 mydata/file.out
    File 3 var/file3
    File 4 var/file3/log
    base var
    recipe recipes:debug

We can see that mydata was not recreated.

Note that, in this case, we didn't specify a log level, so
we didn't get output about what the buildout was doing.

Section and option names in variable substitutions are only allowed to
contain alphanumeric characters, hyphens, periods and spaces. This
restriction might be relaxed in future releases.

Multiple configuration files
----------------------------

You can use multiple configuration files.  From your main
configuration file, you can include other configuration files in 2
ways:

- Your configuration file can "extend" another configuration file.
  Option are read from the other configuration file if they aren't
  already defined by your configuration file.

- Your configuration file can be "extended-by" another configuration
  file, In this case, the options in the other configuration file
  override options in your configuration file. 

The configuration files your file extends or is extended by can extend
or be extended by other configuration files.  The same file may be
used more than once although, of course, cycles aren't allowed.

To see how this works, we use an example:

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... extends = base.cfg
    ...
    ... [debug]
    ... op = buildout
    ... """)

    >>> write(sample_buildout, 'base.cfg',
    ... """
    ... [buildout]
    ... develop = recipes
    ... parts = debug
    ...
    ... [debug]
    ... recipe = recipes:debug
    ... op = base
    ... """)

    >>> print system(buildout),
    buildout: Develop: /sample-buildout/recipes/setup.py
    buildout: Uninstalling debug
    buildout: Uninstalling data-dir
    buildout: Installing debug
    op buildout
    recipe recipes:debug

The example is pretty trivial, but the pattern it illustrates is
pretty common.  In a more practical example, the base buildout might
represent a product and the extending buildout might be a
customization. 

Here is a more elaborate example. 

    >>> extensions = mkdtemp()

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... extends = b1.cfg b2.cfg
    ... extended-by = e1.cfg %(e2)s
    ...
    ... [debug]
    ... op = %%(name)s
    ...
    ... [DEFAULT]
    ... name = buildout
    ... """ % dict(e2=os.path.join(extensions, 'e2.cfg')))

    >>> write(sample_buildout, 'b1.cfg',
    ... """
    ... [buildout]
    ... extends = base.cfg
    ...
    ... [debug]
    ... op1 = %(name)s 1
    ... op2 = %(name)s 2
    ... op3 = %(name)s 3
    ...
    ... [DEFAULT]
    ... name = b1
    ... """)

    >>> write(sample_buildout, 'b2.cfg',
    ... """
    ... [buildout]
    ... extends = base.cfg
    ...
    ... [debug]
    ... op3 = %(name)s 3
    ... op4 = %(name)s 4
    ... op5 = %(name)s 5
    ...
    ... [DEFAULT]
    ... name = b2
    ... """)

    >>> write(sample_buildout, 'base.cfg',
    ... """
    ... [buildout]
    ... develop = recipes
    ... parts = debug
    ...
    ... [debug]
    ... recipe = recipes:debug
    ... name = base
    ... """)

    >>> write(sample_buildout, 'e1.cfg',
    ... """
    ... [debug]
    ... op1 = %(name)s 1
    ...
    ... [DEFAULT]
    ... name = e1
    ... """)

    >>> write(extensions, 'e2.cfg',
    ... """
    ... [buildout]
    ... extends = eb.cfg
    ... extended-by = ee.cfg
    ... """)

    >>> write(extensions, 'eb.cfg',
    ... """
    ... [debug]
    ... op5 = %(name)s 5
    ...
    ... [DEFAULT]
    ... name = eb
    ... """)

    >>> write(extensions, 'ee.cfg',
    ... """
    ... [debug]
    ... op6 = %(name)s 6
    ...
    ... [DEFAULT]
    ... name = ee
    ... """)

    >>> print system(buildout),
    buildout: Develop: /sample-buildout/recipes/setup.py
    buildout: Uninstalling debug
    buildout: Installing debug
    name ee
    op buildout
    op1 e1 1
    op2 b1 2
    op3 b2 3
    op4 b2 4
    op5 eb 5
    op6 ee 6
    recipe recipes:debug

There are several things to note about this example:

- We can name multiple files in an extends or extended-by option.

- We can reference files recursively.

- DEFAULT sections only directly affect the configuration file they're
  used in, but they can have secondary effects.  For example, the name
  option showed up in the debug section because it was defined in the
  debug sections in several of the input files by virtue of being in
  their DEFAULT sections.

- Relative file names in extended and extended-by options are
  interpreted relative to the directory containing the referencing
  configuration file.  The files eb.cfg and ee.cfg were found in the
  extensions directory because they were referenced from a file in
  that directory.

User defaults
-------------

If the file $HOME/.buildout/defaults.cfg, exists, it is read before
reading the configuration file.  ($HOME is the value of the HOME
environment variable. The '/' is replaced by the operating system file
delimiter.)

    >>> home = mkdtemp()
    >>> mkdir(home, '.buildout')
    >>> write(home, '.buildout', 'default.cfg',
    ... """
    ... [debug]
    ... op1 = 1
    ... op7 = 7
    ... """)

    >>> os.environ['HOME'] = home
    >>> print system(buildout),
    buildout: Develop: /sample-buildout/recipes/setup.py
    buildout: Uninstalling debug
    buildout: Installing debug
    name ee
    op buildout
    op1 e1 1
    op2 b1 2
    op3 b2 3
    op4 b2 4
    op5 eb 5
    op6 ee 6
    op7 7
    recipe recipes:debug

    >>> del os.environ['HOME']

Log level
---------

We can control the level of logging by specifying a log level in out
configuration file.  For example, so suppress info messages, we can
set the logging level to WARNING

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... log-level = WARNING
    ... extends = b1.cfg b2.cfg
    ... extended-by = e1.cfg
    ... """)

    >>> print system(buildout),
    name e1
    op1 e1 1
    op2 b1 2
    op3 b2 3
    op4 b2 4
    op5 b2 5
    recipe recipes:debug

Command-line usage
------------------

A number of arguments can be given on the buildout command line.  The
command usage is::

  buildout [-h] [-c file] [-q] [-v] [assignments] [command [command arguments]]

The -h (or --help) option causes basic usage information to be
printed.  If this option is used, then all other options are ignored.

The -c option can be used to specify a configuration file, rather than
buildout.cfg in the current directory.  

The -q and -v decrement and increment the verbosity by 10.  The
verbosity is used to adjust the logging level.  The verbosity is
subtracted from the numeric value of the log-level option specified in
the configuration file.

Assignments are of the form::

  section_name:option_name=value

Options and assignments can be given in any order.

Here's an example:

    >>> write(sample_buildout, 'other.cfg',
    ... """
    ... [buildout]
    ... develop = recipes
    ... parts = debug
    ... installed = .other.cfg
    ... log-level = WARNING
    ...
    ... [debug]
    ... name = other
    ... recipe = recipes:debug
    ... """)

Note that we used the installed buildout option to specify an
alternate file to store information about installed parts.
    
    >>> print system(buildout+' -c other.cfg debug:op1=foo -v'),
    buildout: Develop: /tmp/sample-buildout/recipes/setup.py
    buildout: Installing debug
    name other
    op1 foo
    recipe recipes:debug

Here we used the -c option to specify an alternate configuration file, 
and the -v option to increase the level of logging from the default,
WARNING.

Options can also be combined in the usual Unix way, as in:
    
    >>> print system(buildout+' -vcother.cfg debug:op1=foo'),
    buildout: Develop: /tmp/sample-buildout/recipes/setup.py
    buildout: Installing debug
    name other
    op1 foo
    recipe recipes:debug

Here we combined the -v and -c options with the configuration file
name.  Note that the -c option has to be last, because it takes an
argument.

    >>> os.remove(os.path.join(sample_buildout, 'other.cfg'))
    >>> os.remove(os.path.join(sample_buildout, '.other.cfg'))

The most commonly used command is 'install' and it takes a
list of parts to install. if any parts are specified, then they must
be listed in the buildout parts option and only those parts are
installed.  To illustrate this, we'll update our configuration and run
the buildout in the usual way:

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... develop = recipes
    ... parts = debug d1 d2 d3
    ...
    ... [d1]
    ... recipe = recipes:mkdir
    ... path = d1
    ...
    ... [d2]
    ... recipe = recipes:mkdir
    ... path = d2
    ...
    ... [d3]
    ... recipe = recipes:mkdir
    ... path = d3
    ...
    ... [debug]
    ... recipe = recipes:debug
    ... """)

    >>> print system(buildout),
    buildout: Develop: /sample-buildout/recipes/setup.py
    buildout: Uninstalling debug
    buildout: Installing debug
    recipe recipes:debug
    buildout: Installing d1
    d1: Creating directory d1
    buildout: Installing d2
    d2: Creating directory d2
    buildout: Installing d3
    d3: Creating directory d3
    
    >>> ls(sample_buildout)
    -  .installed.cfg
    -  b1.cfg
    -  b2.cfg
    -  base.cfg
    d  bin
    -  buildout.cfg
    d  d1
    d  d2
    d  d3
    d  develop-eggs
    -  e1.cfg
    d  eggs
    d  parts
    d  recipes

    >>> cat(sample_buildout, '.installed.cfg')
    [buildout]
    parts = debug d1 d2 d3
    <BLANKLINE>
    [debug]
    __buildout_installed__ = 
    __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
    recipe = recipes:debug
    <BLANKLINE>
    [d1]
    __buildout_installed__ = /tmp/sample-buildout/d1
    __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
    path = /tmp/sample-buildout/d1
    recipe = recipes:mkdir
    <BLANKLINE>
    [d2]
    __buildout_installed__ = /tmp/sample-buildout/d2
    __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
    path = /tmp/sample-buildout/d2
    recipe = recipes:mkdir
    <BLANKLINE>
    [d3]
    __buildout_installed__ = /tmp/sample-buildout/d3
    __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
    path = /tmp/sample-buildout/d3
    recipe = recipes:mkdir

Now we'll update our configuration file:

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... develop = recipes
    ... parts = debug d2 d3 d4
    ...
    ... [d2]
    ... recipe = recipes:mkdir
    ... path = data2
    ...
    ... [d3]
    ... recipe = recipes:mkdir
    ... path = data3
    ...
    ... [d4]
    ... recipe = recipes:mkdir
    ... path = data4
    ...
    ... [debug]
    ... recipe = recipes:debug
    ... x = 1
    ... """)

and run the buildout specifying just d3 and d4:

    >>> print system(buildout+' install d3 d4'),
    buildout: Develop: /sample-buildout/recipes/setup.py
    buildout: Uninstalling d3
    buildout: Installing d3
    d3: Creating directory data3
    buildout: Installing d4
    d4: Creating directory data4
    
    >>> ls(sample_buildout)
    -  .installed.cfg
    -  b1.cfg
    -  b2.cfg
    -  base.cfg
    d  bin
    -  buildout.cfg
    d  d1
    d  d2
    d  data3
    d  data4
    d  develop-eggs
    -  e1.cfg
    d  eggs
    d  parts
    d  recipes
    
Only the d3 and d4 recipes ran.  d3 was removed and data3 and data4
were created.

The .installed.cfg is only updated for the recipes that ran:

    >>> cat(sample_buildout, '.installed.cfg')
    [buildout]
    parts = debug d2 d3 d4 d1
    <BLANKLINE>
    [debug]
    __buildout_installed__ = 
    __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
    recipe = recipes:debug
    <BLANKLINE>
    [d2]
    __buildout_installed__ = /tmp/sample-buildout/d2
    __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
    path = /tmp/sample-buildout/d2
    recipe = recipes:mkdir
    <BLANKLINE>
    [d3]
    __buildout_installed__ = /tmp/sample-buildout/data3
    __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
    path = /tmp/sample-buildout/data3
    recipe = recipes:mkdir
    <BLANKLINE>
    [d4]
    __buildout_installed__ = /tmp/sample-buildout/data4
    __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
    path = /tmp/sample-buildout/data4
    recipe = recipes:mkdir
    <BLANKLINE>
    [d1]
    __buildout_installed__ = /tmp/sample-buildout/d1
    __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg==
    path = /tmp/sample-buildout/d1
    recipe = recipes:mkdir

Note that the installed data for debug, d1, and d2 haven't changed,
because we didn't install those parts and that the d1 and d2
directories are still there.

Now, if we run the buildout without the install command:

    >>> print system(buildout),
    buildout: Develop: /sample-buildout/recipes/setup.py
    buildout: Uninstalling d1
    buildout: Uninstalling d2
    buildout: Uninstalling debug
    buildout: Installing debug
    recipe recipes:debug
    x 1
    buildout: Installing d2
    d2: Creating directory data2
    buildout: Installing d3
    buildout: Installing d4

We see the output of the debug recipe and that data2 was created.  We
also see that d1 and d2 have gone away:

    >>> ls(sample_buildout)
    -  .installed.cfg
    -  b1.cfg
    -  b2.cfg
    -  base.cfg
    d  bin
    -  buildout.cfg
    d  data2
    d  data3
    d  data4
    d  develop-eggs
    -  e1.cfg
    d  eggs
    d  parts
    d  recipes

Alternate directory and file locations
--------------------------------------

The buildout normally puts the bin, eggs, and parts directories in the
directory in the directory containing the configuration file. You can
provide alternate locations, and even names for these directories.

    >>> alt = mkdtemp('sample-alt')

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... develop = recipes
    ... parts = 
    ... develop-eggs-directory = %(developbasket)s
    ... eggs-directory = %(basket)s
    ... bin-directory = %(scripts)s
    ... parts-directory = %(work)s
    ... """ % dict(
    ...    developbasket = os.path.join(alt, 'developbasket'),
    ...    basket = os.path.join(alt, 'basket'),
    ...    scripts = os.path.join(alt, 'scripts'),
    ...    work = os.path.join(alt, 'work'),
    ... ))

    >>> print system(buildout),
    buildout: Creating directory /tmp/sample-alt/scripts
    buildout: Creating directory /tmp/sample-alt/work
    buildout: Creating directory /tmp/sample-alt/basket
    buildout: Creating directory /sample-alt/developbasket
    buildout: Develop: /tmp/sample-buildout/recipes/setup.py
    buildout: Uninstalling d4
    buildout: Uninstalling d3
    buildout: Uninstalling d2
    buildout: Uninstalling debug

    >>> ls(alt)
    d  basket
    d  developbasket
    d  scripts
    d  work

    >>> ls(alt, 'developbasket')    
    -  recipes.egg-link

You can also specify an alternate buildout directory:

    >>> alt = mkdtemp('sample-alt')

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... directory = %(alt)s
    ... develop = %(recipes)s
    ... parts = 
    ... """ % dict(
    ...    alt=alt,
    ...    recipes=os.path.join(sample_buildout, 'recipes'),
    ...    ))
 
    >>> print system(buildout),
    buildout: Creating directory /tmp/sample-alt/bin
    buildout: Creating directory /tmp/sample-alt/parts
    buildout: Creating directory /tmp/sample-alt/eggs
    buildout: Creating directory /tmp/sample-alt/develop-eggs
    buildout: Develop: /tmp/sample-buildout/recipes/setup.py

    >>> ls(alt)
    -  .installed.cfg
    d  bin
    d  develop-eggs
    d  eggs
    d  parts

    >>> ls(alt, 'develop-eggs')    
    -  recipes.egg-link

Logging control
---------------

Three buildout options are used to control logging:

log-level 
   specifies the log level

verbosity 
   adjusts the log level

log-format
   allows an alternate logging for mat to be specified

We've already seen the log level and verbosity.  Let's look at an example
of changing the format:

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... develop = recipes
    ... parts =
    ... log-level = 25
    ... verbosity = 5
    ... log-format = %%(levelname)s %%(message)s
    ... """)
 
Here, we've changed the format to include the log-level name, rather
than the logger name.  Note that we had to double percent signs,
because configuration options allow ConfigParser variable substitution.

We've also illustrated, with a contrived example, that the log level
can be a numeric value and that the verbosity can be specified in the
configuration file.  Because the verbosity is subtracted from the log
level, we get a final log level of 20, which is the INFO level.

    >>> print system(buildout),
    INFO Develop: /tmp/sample-buildout/recipes/setup.py

Predefined buildout options
---------------------------

Buildouts have a number of predefined options that recipes can use
and that users can override in their configuration files.  To see
these, we'll run a minimal buildout configuration with a debug logging
level.  One of the features of debug logging is that the configuration
database is shown.
         
    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... parts =
    ... """)

    >>> print system(buildout+' -v'),
    Configuration data:
    [buildout]
    bin-directory = /tmp/sample-buildout/bin
    develop-eggs-directory = /tmp/sample-buildout/develop-eggs
    directory = /tmp/sample-buildout
    eggs-directory = /tmp/sample-buildout/eggs
    executable = /usr/local/bin/python2.3
    installed = /tmp/sample-buildout/.installed.cfg
    log-format = %%(name)s: %%(message)s
    log-level = INFO
    parts = 
    parts-directory = /tmp/sample-buildout/parts
    python = buildout
    verbosity = 10
    <BLANKLINE>
    zc.buildout.easy_install: Installing ['zc.buildout', 'setuptools']
    zc.buildout.easy_install: We have a develop egg for zc.buildout
    zc.buildout.easy_install: We have the best distribution that satisfies
    setuptools

All of these options can be overridden by configuration files or by
command-line assignments.  We've discussed most of these options
already, but let's review them and touch on some we haven't discussed:

bin-directory
   The directory path where scripts are written.  This can be a
   relative path, which is interpreted relative to the directory
   option.

develop-eggs-directory
   The directory path where development egg links are created for software
   being created in the local project.  This can be a relative path,
   which is interpreted relative to the directory option.

directory
   The buildout directory.  This is the base for other buildout file
   and directory locations, when relative locations are used.

eggs-directory
   The directory path where downloaded eggs are put.  It is common to share
   this directory across buildouts. Eggs in this directory should
   *never* be modified.  This can be a relative path, which is
   interpreted relative to the directory option.

executable
   The Python executable used to run the buildout.  See the python
   option below.

installed
   The file path where information about the results of the previous
   buildout run is written.  This can be a relative path, which is
   interpreted relative to the directory option.  This file provides
   an inventory of installed parts with information needed to decide
   which if any parts need to be uninstalled.

log-format
   The format used for logging messages.

log-level
   The log level before verbosity adjustment

parts
   A white space separated list of parts to be installed.

parts-directory
   A working directory that parts can used to store data.

python
   The name of a section containing information about the default
   Python interpreter.  Recipes that need a installation
   typically have options to tell them which Python installation to
   use.  By convention, if a section-specific option isn't used, the
   option is looked for in the buildout section.  The option must
   point to a section with an executable option giving the path to a
   Python executable.  By default, the buildout section defines the
   default Python as the Python used to run the buildout.

verbosity
   A log-level adjustment.  Typically, this is set via the -q and -v
   command-line options.


Bootstrapping
-------------

If zc.buildout is installed, you can use it to create a new buildout
with it's own local copies of zc.buildout and setuptools and with
local buildout scripts. 

    >>> sample_bootstrapped = mkdtemp('sample-bootstrapped')

    >>> print system(buildout
    ...              +' -c'+os.path.join(sample_bootstrapped, 'setup.cfg')
    ...              +' bootstrap'),
    Warning: creating /sample-bootstrapped/setup.cfg
    buildout: Creating directory /sample-bootstrapped/bin
    buildout: Creating directory /sample-bootstrapped/parts
    buildout: Creating directory /sample-bootstrapped/eggs
    buildout: Creating directory /sample-bootstrapped/develop-eggs

Note that a basic setup.cfg was created for us.

    >>> ls(sample_bootstrapped)
    d  bin
    d  develop-eggs
    d  eggs
    d  parts
    -  setup.cfg

    >>> ls(sample_bootstrapped, 'bin')
    -  buildout

    >>> ls(sample_bootstrapped, 'eggs')
    -  setuptools-0.6-py2.3.egg
    -  zc.buildout-1.0-py2.3.egg

Note that the buildout script was installed but not run.  To run
the buildout, we'd have to run the installed buildout script.