Buildouts ========= The word "buildout" refers to a description of a set of parts and the software to create and assemble them. It is often used informally to refer to an installed system based on a buildout definition. For example, if we are creating an application named "Foo", then "the Foo buildout" is the collection of configuration and application-specific software that allows an instance of the application to be created. We may refer to such an instance of the application informally as "a Foo buildout". This document describes how to define buildouts using buildout configuration files and recipes. There are three ways to set up the buildout software and create a buildout instance: 1. Install the zc.buildout egg with easy_install and use the buildout script installed in a Python scripts area. 2. Use the buildout bootstrap script to create a buildout that includes both the setuptools and zc.buildout eggs. This allows you to use the buildout software without modifying a Python install. The buildout script is installed into your buildout local scripts area. 3. Use a buildoput command from an already installed buildout to bootstrap a new buildout. (See the section on bootstraping later in this document.) Often, a software project will be managed in a software repository, such as a subversion repository, that includes some software source directories, buildout configuration files, and a copy of the buildout bootstrap script, To work on the project, one would check out the project from the repository and run the bootstrap script which installs setuptools and zc.buildout into the checkout as well as any parts defined. We have a sample buildout that we created using the bootstrap command of an existing buildout (method 3 above). It has the absolute minimum information. We have bin, develop-eggs, eggs and parts directories, and a configuration file: >>> ls(sample_buildout) d bin - buildout.cfg d develop-eggs d eggs d parts The bin directory contains scripts. >>> ls(sample_buildout, 'bin') - buildout >>> ls(sample_buildout, 'eggs') - setuptools-0.6-py2.4.egg - zc.buildout-1.0-py2.4.egg The develop-eggs and parts directories are initially empty: >>> ls(sample_buildout, 'develop-eggs') >>> ls(sample_buildout, 'parts') The develop-eggs directory holds egg links for software being developed in the buildout. We separate develop-eggs and other eggs to allow eggs directories to be shared across multiple buildouts. For example, a common developer technique is to define a common eggs directory in their home that all non-develop eggs are stored in. This allows larger buildouts to be set up much more quickly and saves disk space. The parts directory provides an area where recipes can install part data. For example, if we built a custom Python, we would install it in the part directory. Part data is stored in a sub-directory of the parts directory with the same name as the part. Buildouts are defined using configuration files. These are in the format defined by the Python ConfigParser module, with extensions that we'll describe later. By default, when a buildout is run, it looks for the file buildout.cfg in the directory where the buildout is run. The minimal configuration file has a buildout section that defines no parts: >>> cat(sample_buildout, 'buildout.cfg') [buildout] parts = A part is simply something to be created by a buildout. It can be almost anything, such as a Python package, a program, a directory, or even a configuration file. A part is created by a recipe. Recipes are always installed as Python eggs. They can be downloaded from a package server, such as the Python Package Index, or they can be developed as part of a project. Let's create a recipe as part of the sample project. We'll create a recipe for creating directories. First, we'll create a recipes directory for our local recipes: >>> mkdir(sample_buildout, 'recipes') and then we'll create a source file for our mkdir recipe: >>> write(sample_buildout, 'recipes', 'mkdir.py', ... """ ... import logging, os, zc.buildout ... ... class Mkdir: ... ... def __init__(self, buildout, name, options): ... self.buildout = buildout ... self.name = name ... self.options = options ... options['path'] = os.path.join( ... buildout['buildout']['directory'], ... options['path'], ... ) ... if not os.path.isdir(os.path.dirname(options['path'])): ... logging.getLogger(self.name).error( ... 'Cannot create %s. %s is not a directory.', ... options['path'], os.path.dirname(options['path'])) ... raise zc.buildout.UserError('Invalid Path') ... ... ... def install(self): ... path = self.options['path'] ... if not os.path.isdir(path): ... logging.getLogger(self.name).info( ... 'Creating directory %s', os.path.basename(path)) ... os.mkdir(path) ... return path ... """) The recipe defines a constructor that takes a buildout object, a part name, and an options dictionary. It saves them in instance attributes. If the path is relative, we'll interpret it as relative to the buildout directory. The buildout object passed in is a mapping from section name to a mapping of options for that section. The buildout directory is available as the directory option of the buildout section. We normalize the path and save it back into the options directory. Any time we use data from another section, it is important to reflect that data in the recipe's options when the recipe is constructed. When buildout is run, it saves configuration data for installed parts in a file named installed.cfg. In subsequent runs, it compares part-configuration data stored in the installed.cfg file and the part-configuration data loaded from the configuration files as modified by recipe constructors to decide if the configuration of a part has changed. If the configuration has changed, or if the recipe has changed, then the part is uninstalled before reinstalling it. The buildout only looks at the part's options, so any data used to configure the part needs to be reflected in the part's options. It is the job of a recipe constructor to make sure that the options include all rel event data. Of course, parts are also uninstalled if they are no-longer used. The install method is responsible for creating the part. In this case, we need the path of the directory to create. We'll use a path option from our options dictionary. The install method logs what it's doing using the Python logging call. We return the path that we installed. If the part is uninstalled or reinstalled, then the path returned will be removed by the buildout machinery. A recipe install method is expected to return None, a string, or an iterable of strings containing paths to be removed if a part is uninstalled. For most recipes, this is all of the uninstall support needed. A recipe can provide custom uninstall support as will be described later. We need to provide packaging information so that our recipe can be installed as an egg. We need to define a setup script for this: >>> write(sample_buildout, 'recipes', 'setup.py', ... """ ... from setuptools import setup ... ... setup( ... name = "recipes", ... entry_points = {'zc.buildout': ['mkdir = mkdir:Mkdir']}, ... ) ... """) This setup script is incomplete. It doesn't describe what is to be included in a distribution. This is fine if we never actually create a distribution. If recipes are going to be used only internally in a buildout, then we needn't include distribution information. If we wanted to use the same recipes in multiple buildouts, then we'd need to include proper distribution data. To find out more about creating distributions, see the setuptools documentation. Our setup script defines an entry point. Entry points provide a way for an egg to define the services it provides. Here we've said that we define a zc.buildout entry point named default. Recipe classes must be exposed as entry points in the zc.buildout group. we give entry points names within the group. The name "default" is somewhat special because it allows a recipe to be referenced using a package name without naming an entry point. We also need a README.txt for our recipes to avoid an annoying warning from distutils, on which setuptools and zc.buildout are based: >>> write(sample_buildout, 'recipes', 'README.txt', " ") Now let's update our buildout.cfg: >>> write(sample_buildout, 'buildout.cfg', ... """ ... [buildout] ... develop = recipes ... parts = data-dir ... ... [data-dir] ... recipe = recipes:mkdir ... path = mystuff ... """) Let's go through the changes one by one:: develop = recipes This tells the buildout to install a development egg for our recipes. Any number of paths can be listed. The paths can be relative or absolute. If relative, they are treated as relative to the buildout directory. They can be directory or file paths. If a file path is given, it should point to a Python setup script. If a directory path is given, it should point to a directory containing a setup.py file. Development eggs are installed before building any parts, as they may provide locally-defined recipes needed by the parts. :: parts = data-dir Here we've named a part to be "built". We can use any name we want except that different part names must be unique and recipes will often use the part name to decide what to do. :: [data-dir] recipe = recipes:mkdir path = mystuff When we name a part, we also create a section of the same name that contains part data. In this section, we'll define the recipe to be used to install the part. In this case, we also specify the path to be created. Let's run the buildout. We do so by running the build script in the buildout: >>> import os >>> os.chdir(sample_buildout) >>> buildout = os.path.join(sample_buildout, 'bin', 'buildout') >>> print system(buildout), buildout: Develop: /tmp/sample-buildout/recipes/setup.py buildout: Installing data-dir data-dir: Creating directory mystuff We see that the recipe created the directory, as expected: >>> ls(sample_buildout) - .installed.cfg d bin - buildout.cfg d develop-eggs d eggs d mystuff d parts d recipes In addition, .installed.cfg has been created containing information about the part we installed: >>> cat(sample_buildout, '.installed.cfg') [buildout] parts = data-dir <BLANKLINE> [data-dir] __buildout_installed__ = /tmp/sample-buildout/mystuff __buildout_signature__ = recipes-c7vHV6ekIDUPy/7fjAaYjg== path = /tmp/sample-buildout/mystuff recipe = recipes:mkdir Note that the directory we installed is included in .installed.cfg. In addition, the path option includes the actual destination directory. If we change the name of the directory in the configuration file, we'll see that the directory gets removed and recreated: >>> write(sample_buildout, 'buildout.cfg', ... """ ... [buildout] ... develop = recipes ... parts = data-dir ... ... [data-dir] ... recipe = recipes:mkdir ... path = mydata ... """) >>> print system(buildout), buildout: Develop: /tmp/sample-buildout/recipes/setup.py buildout: Uninstalling data-dir buildout: Installing data-dir data-dir: Creating directory mydata >>> ls(sample_buildout) - .installed.cfg d bin - buildout.cfg d develop-eggs d eggs d mydata d parts d recipes Error reporting --------------- If a user makes an error, an error needs to be printed and work needs to stop. This is accomplished by logging a detailed error message and then raising a (or an instance of a subclass of a) zc.buildout.UserError exception. Raising UserError causes the buildout to print the error and exit without printing a traceback. In the sample above, of someone gives a non-existant directory to create the directory in: >>> write(sample_buildout, 'buildout.cfg', ... """ ... [buildout] ... develop = recipes ... parts = data-dir ... ... [data-dir] ... recipe = recipes:mkdir ... path = /xxx/mydata ... """) We'll get a user error, not a traceback. >>> print system(buildout), buildout: Develop: /private/tmp/tmp_I5pHasample-buildout/recipes/setup.py data-dir: Cannot create /xxx/mydata. /xxx is not a directory. Error: Invalid Path Configuration file syntax ------------------------- As mentioned earlier, buildout configuration files use the format defined by the Python ConfigParser module with extensions. The extensions are: - option names are case sensitive - option values can ue a substitution syntax, described below, to refer to option values in specific sections. The ConfigParser syntax is very flexible. Section names can contain any characters other than newlines and right square braces ("]"). Option names can contain any characters other than newlines, colons, and equal signs, can not start with a space, and don't include trailing spaces. It is likely that, in the future, some characters will be given special buildout-defined meanings. This is already true of the characters ":", "$", "%", "(", and ")". For now, it is a good idea to keep section and option names simple, sticking to alphanumeric characters, hyphens, and periods. Variable substitutions ---------------------- Buildout configuration files support two kinds of substitutions, standard ConfigParser substitutions, and string-template substitutions. To illustrate this, we'll create an debug recipe to allow us to see interactions with the buildout: >>> write(sample_buildout, 'recipes', 'debug.py', ... """ ... class Debug: ... ... def __init__(self, buildout, name, options): ... self.buildout = buildout ... self.name = name ... self.options = options ... ... def install(self): ... items = self.options.items() ... items.sort() ... for option, value in items: ... print option, value ... """) In this example, we've used a simple base class that provides a boilerplate constructor. This recipe doesn't actually create anything. The install method doesn't return anything, because it didn't create any files or directories. We also have to update our setup script: >>> write(sample_buildout, 'recipes', 'setup.py', ... """ ... from setuptools import setup ... entry_points = ( ... ''' ... [zc.buildout] ... mkdir = mkdir:Mkdir ... debug = debug:Debug ... ''') ... setup(name="recipes", entry_points=entry_points) ... """) We've rearranged the script a bit to make the entry points easier to edit. In particular, entry points are now defined as a configuration string, rather than a dictionary. Let's update our configuration to provide variable substitution examples: >>> write(sample_buildout, 'buildout.cfg', ... """ ... [buildout] ... develop = recipes ... parts = data-dir debug ... log-level = INFO ... ... [debug] ... recipe = recipes:debug ... File 1 = ${data-dir:path}/file ... File 2 = %(File 1)s.out ... File 3 = %(base)s/file3 ... File 4 = ${debug:File 3}/log ... ... [data-dir] ... recipe = recipes:mkdir ... path = mydata ... ... [DEFAULT] ... base = var ... """) In this example, we've used ConfigParser substitutions for file2 and file3. This type of substitution uses Python string format syntax. Valid names are options in the same section and options defined in the DEFAULT section. We used a string-template substitution for file1. This type of substitution uses the string.Template syntax. Names substituted are qualified option names, consisting of a section name and option name joined by a colon. Now, if we run the buildout, we'll see the options with the values substituted. >>> print system(buildout), buildout: Develop: /tmp/sample-buildout/recipes/setup.py buildout: Uninstalling data-dir buildout: Installing data-dir data-dir: Creating directory mydata buildout: Installing debug File 1 mydata/file File 2 mydata/file.out File 3 var/file3 File 4 var/file3/log base var recipe recipes:debug It might seem surprising that mydata was created again. This is because we changed our recipes package by adding the debug module. The buildout system didn't know if this module could effect the mkdir recipe, so it assumed it could and reinstalled mydata. If we rerun the buildout: >>> print system(buildout), buildout: Develop: /tmp/sample-buildout/recipes/setup.py buildout: Installing data-dir buildout: Installing debug File 1 mydata/file File 2 mydata/file.out File 3 var/file3 File 4 var/file3/log base var recipe recipes:debug We can see that mydata was not recreated. Note that, in this case, we didn't specify a log level, so we didn't get output about what the buildout was doing. Section and option names in variable substitutions are only allowed to contain alphanumeric characters, hyphens, periods and spaces. This restriction might be relaxed in future releases. Multiple configuration files ---------------------------- You can use multiple configuration files. From your main configuration file, you can include other configuration files in 2 ways: - Your configuration file can "extend" another configuration file. Option are read from the other configuration file if they aren't already defined by your configuration file. - Your configuration file can be "extended-by" another configuration file, In this case, the options in the other configuration file override options in your configuration file. The configuration files your file extends or is extended by can extend or be extended by other configuration files. The same file may be used more than once although, of course, cycles aren't allowed. To see how this works, we use an example: >>> write(sample_buildout, 'buildout.cfg', ... """ ... [buildout] ... extends = base.cfg ... ... [debug] ... op = buildout ... """) >>> write(sample_buildout, 'base.cfg', ... """ ... [buildout] ... develop = recipes ... parts = debug ... ... [debug] ... recipe = recipes:debug ... op = base ... """) >>> print system(buildout), buildout: Develop: /sample-buildout/recipes/setup.py buildout: Uninstalling debug buildout: Uninstalling data-dir buildout: Installing debug op buildout recipe recipes:debug The example is pretty trivial, but the pattern it illustrates is pretty common. In a more practical example, the base buildout might represent a product and the extending buildout might be a customization. Here is a more elaborate example. >>> extensions = mkdtemp() >>> write(sample_buildout, 'buildout.cfg', ... """ ... [buildout] ... extends = b1.cfg b2.cfg ... extended-by = e1.cfg %(e2)s ... ... [debug] ... op = %%(name)s ... ... [DEFAULT] ... name = buildout ... """ % dict(e2=os.path.join(extensions, 'e2.cfg'))) >>> write(sample_buildout, 'b1.cfg', ... """ ... [buildout] ... extends = base.cfg ... ... [debug] ... op1 = %(name)s 1 ... op2 = %(name)s 2 ... op3 = %(name)s 3 ... ... [DEFAULT] ... name = b1 ... """) >>> write(sample_buildout, 'b2.cfg', ... """ ... [buildout] ... extends = base.cfg ... ... [debug] ... op3 = %(name)s 3 ... op4 = %(name)s 4 ... op5 = %(name)s 5 ... ... [DEFAULT] ... name = b2 ... """) >>> write(sample_buildout, 'base.cfg', ... """ ... [buildout] ... develop = recipes ... parts = debug ... ... [debug] ... recipe = recipes:debug ... name = base ... """) >>> write(sample_buildout, 'e1.cfg', ... """ ... [debug] ... op1 = %(name)s 1 ... ... [DEFAULT] ... name = e1 ... """) >>> write(extensions, 'e2.cfg', ... """ ... [buildout] ... extends = eb.cfg ... extended-by = ee.cfg ... """) >>> write(extensions, 'eb.cfg', ... """ ... [debug] ... op5 = %(name)s 5 ... ... [DEFAULT] ... name = eb ... """) >>> write(extensions, 'ee.cfg', ... """ ... [debug] ... op6 = %(name)s 6 ... ... [DEFAULT] ... name = ee ... """) >>> print system(buildout), buildout: Develop: /sample-buildout/recipes/setup.py buildout: Uninstalling debug buildout: Installing debug name ee op buildout op1 e1 1 op2 b1 2 op3 b2 3 op4 b2 4 op5 eb 5 op6 ee 6 recipe recipes:debug There are several things to note about this example: - We can name multiple files in an extends or extended-by option. - We can reference files recursively. - DEFAULT sections only directly affect the configuration file they're used in, but they can have secondary effects. For example, the name option showed up in the debug section because it was defined in the debug sections in several of the input files by virtue of being in their DEFAULT sections. - Relative file names in extended and extended-by options are interpreted relative to the directory containing the referencing configuration file. The files eb.cfg and ee.cfg were found in the extensions directory because they were referenced from a file in that directory. User defaults ------------- If the file $HOME/.buildout/defaults.cfg, exists, it is read before reading the configuration file. ($HOME is the value of the HOME environment variable. The '/' is replaced by the operating system file delimiter.) >>> home = mkdtemp() >>> mkdir(home, '.buildout') >>> write(home, '.buildout', 'default.cfg', ... """ ... [debug] ... op1 = 1 ... op7 = 7 ... """) >>> os.environ['HOME'] = home >>> print system(buildout), buildout: Develop: /sample-buildout/recipes/setup.py buildout: Uninstalling debug buildout: Installing debug name ee op buildout op1 e1 1 op2 b1 2 op3 b2 3 op4 b2 4 op5 eb 5 op6 ee 6 op7 7 recipe recipes:debug >>> del os.environ['HOME'] Log level --------- We can control the level of logging by specifying a log level in out configuration file. For example, so suppress info messages, we can set the logging level to WARNING >>> write(sample_buildout, 'buildout.cfg', ... """ ... [buildout] ... log-level = WARNING ... extends = b1.cfg b2.cfg ... extended-by = e1.cfg ... """) >>> print system(buildout), name e1 op1 e1 1 op2 b1 2 op3 b2 3 op4 b2 4 op5 b2 5 recipe recipes:debug Command-line usage ------------------ A number of arguments can be given on the buildout command line. The command usage is:: buildout [-h] [-c file] [-q] [-v] [assignments] [command [command arguments]] The -h (or --help) option causes basic usage information to be printed. If this option is used, then all other options are ignored. The -c option can be used to specify a configuration file, rather than buildout.cfg in the current directory. The -q and -v decrement and increment the verbosity by 10. The verbosity is used to adjust the logging level. The verbosity is subtracted from the numeric value of the log-level option specified in the configuration file. Assignments are of the form:: section_name:option_name=value Options and assignments can be given in any order. Here's an example: >>> write(sample_buildout, 'other.cfg', ... """ ... [buildout] ... develop = recipes ... parts = debug ... installed = .other.cfg ... log-level = WARNING ... ... [debug] ... name = other ... recipe = recipes:debug ... """) Note that we used the installed buildout option to specify an alternate file to store information about installed parts. >>> print system(buildout+' -c other.cfg debug:op1=foo -v'), buildout: Develop: /tmp/sample-buildout/recipes/setup.py buildout: Installing debug name other op1 foo recipe recipes:debug Here we used the -c option to specify an alternate configuration file, and the -v option to increase the level of logging from the default, WARNING. Options can also be combined in the usual Unix way, as in: >>> print system(buildout+' -vcother.cfg debug:op1=foo'), buildout: Develop: /tmp/sample-buildout/recipes/setup.py buildout: Installing debug name other op1 foo recipe recipes:debug Here we combined the -v and -c options with the configuration file name. Note that the -c option has to be last, because it takes an argument. >>> os.remove(os.path.join(sample_buildout, 'other.cfg')) >>> os.remove(os.path.join(sample_buildout, '.other.cfg')) The most commonly used command is 'install' and it takes a list of parts to install. if any parts are specified, then they must be listed in the buildout parts option and only those parts are installed. To illustrate this, we'll update our configuration and run the buildout in the usual way: >>> write(sample_buildout, 'buildout.cfg', ... """ ... [buildout] ... develop = recipes ... parts = debug d1 d2 d3 ... ... [d1] ... recipe = recipes:mkdir ... path = d1 ... ... [d2] ... recipe = recipes:mkdir ... path = d2 ... ... [d3] ... recipe = recipes:mkdir ... path = d3 ... ... [debug] ... recipe = recipes:debug ... """) >>> print system(buildout), buildout: Develop: /sample-buildout/recipes/setup.py buildout: Uninstalling debug buildout: Installing debug recipe recipes:debug buildout: Installing d1 d1: Creating directory d1 buildout: Installing d2 d2: Creating directory d2 buildout: Installing d3 d3: Creating directory d3 >>> ls(sample_buildout) - .installed.cfg - b1.cfg - b2.cfg - base.cfg d bin - buildout.cfg d d1 d d2 d d3 d develop-eggs - e1.cfg d eggs d parts d recipes >>> cat(sample_buildout, '.installed.cfg') [buildout] parts = debug d1 d2 d3 <BLANKLINE> [debug] __buildout_installed__ = __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== recipe = recipes:debug <BLANKLINE> [d1] __buildout_installed__ = /tmp/sample-buildout/d1 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== path = /tmp/sample-buildout/d1 recipe = recipes:mkdir <BLANKLINE> [d2] __buildout_installed__ = /tmp/sample-buildout/d2 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== path = /tmp/sample-buildout/d2 recipe = recipes:mkdir <BLANKLINE> [d3] __buildout_installed__ = /tmp/sample-buildout/d3 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== path = /tmp/sample-buildout/d3 recipe = recipes:mkdir Now we'll update our configuration file: >>> write(sample_buildout, 'buildout.cfg', ... """ ... [buildout] ... develop = recipes ... parts = debug d2 d3 d4 ... ... [d2] ... recipe = recipes:mkdir ... path = data2 ... ... [d3] ... recipe = recipes:mkdir ... path = data3 ... ... [d4] ... recipe = recipes:mkdir ... path = data4 ... ... [debug] ... recipe = recipes:debug ... x = 1 ... """) and run the buildout specifying just d3 and d4: >>> print system(buildout+' install d3 d4'), buildout: Develop: /sample-buildout/recipes/setup.py buildout: Uninstalling d3 buildout: Installing d3 d3: Creating directory data3 buildout: Installing d4 d4: Creating directory data4 >>> ls(sample_buildout) - .installed.cfg - b1.cfg - b2.cfg - base.cfg d bin - buildout.cfg d d1 d d2 d data3 d data4 d develop-eggs - e1.cfg d eggs d parts d recipes Only the d3 and d4 recipes ran. d3 was removed and data3 and data4 were created. The .installed.cfg is only updated for the recipes that ran: >>> cat(sample_buildout, '.installed.cfg') [buildout] parts = debug d2 d3 d4 d1 <BLANKLINE> [debug] __buildout_installed__ = __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== recipe = recipes:debug <BLANKLINE> [d2] __buildout_installed__ = /tmp/sample-buildout/d2 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== path = /tmp/sample-buildout/d2 recipe = recipes:mkdir <BLANKLINE> [d3] __buildout_installed__ = /tmp/sample-buildout/data3 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== path = /tmp/sample-buildout/data3 recipe = recipes:mkdir <BLANKLINE> [d4] __buildout_installed__ = /tmp/sample-buildout/data4 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== path = /tmp/sample-buildout/data4 recipe = recipes:mkdir <BLANKLINE> [d1] __buildout_installed__ = /tmp/sample-buildout/d1 __buildout_signature__ = recipes-PiIFiO8ny5yNZ1S3JfT0xg== path = /tmp/sample-buildout/d1 recipe = recipes:mkdir Note that the installed data for debug, d1, and d2 haven't changed, because we didn't install those parts and that the d1 and d2 directories are still there. Now, if we run the buildout without the install command: >>> print system(buildout), buildout: Develop: /sample-buildout/recipes/setup.py buildout: Uninstalling d1 buildout: Uninstalling d2 buildout: Uninstalling debug buildout: Installing debug recipe recipes:debug x 1 buildout: Installing d2 d2: Creating directory data2 buildout: Installing d3 buildout: Installing d4 We see the output of the debug recipe and that data2 was created. We also see that d1 and d2 have gone away: >>> ls(sample_buildout) - .installed.cfg - b1.cfg - b2.cfg - base.cfg d bin - buildout.cfg d data2 d data3 d data4 d develop-eggs - e1.cfg d eggs d parts d recipes Alternate directory and file locations -------------------------------------- The buildout normally puts the bin, eggs, and parts directories in the directory in the directory containing the configuration file. You can provide alternate locations, and even names for these directories. >>> alt = mkdtemp('sample-alt') >>> write(sample_buildout, 'buildout.cfg', ... """ ... [buildout] ... develop = recipes ... parts = ... develop-eggs-directory = %(developbasket)s ... eggs-directory = %(basket)s ... bin-directory = %(scripts)s ... parts-directory = %(work)s ... """ % dict( ... developbasket = os.path.join(alt, 'developbasket'), ... basket = os.path.join(alt, 'basket'), ... scripts = os.path.join(alt, 'scripts'), ... work = os.path.join(alt, 'work'), ... )) >>> print system(buildout), buildout: Creating directory /tmp/sample-alt/scripts buildout: Creating directory /tmp/sample-alt/work buildout: Creating directory /tmp/sample-alt/basket buildout: Creating directory /sample-alt/developbasket buildout: Develop: /tmp/sample-buildout/recipes/setup.py buildout: Uninstalling d4 buildout: Uninstalling d3 buildout: Uninstalling d2 buildout: Uninstalling debug >>> ls(alt) d basket d developbasket d scripts d work >>> ls(alt, 'developbasket') - recipes.egg-link You can also specify an alternate buildout directory: >>> alt = mkdtemp('sample-alt') >>> write(sample_buildout, 'buildout.cfg', ... """ ... [buildout] ... directory = %(alt)s ... develop = %(recipes)s ... parts = ... """ % dict( ... alt=alt, ... recipes=os.path.join(sample_buildout, 'recipes'), ... )) >>> print system(buildout), buildout: Creating directory /tmp/sample-alt/bin buildout: Creating directory /tmp/sample-alt/parts buildout: Creating directory /tmp/sample-alt/eggs buildout: Creating directory /tmp/sample-alt/develop-eggs buildout: Develop: /tmp/sample-buildout/recipes/setup.py >>> ls(alt) - .installed.cfg d bin d develop-eggs d eggs d parts >>> ls(alt, 'develop-eggs') - recipes.egg-link Logging control --------------- Three buildout options are used to control logging: log-level specifies the log level verbosity adjusts the log level log-format allows an alternate logging for mat to be specified We've already seen the log level and verbosity. Let's look at an example of changing the format: >>> write(sample_buildout, 'buildout.cfg', ... """ ... [buildout] ... develop = recipes ... parts = ... log-level = 25 ... verbosity = 5 ... log-format = %%(levelname)s %%(message)s ... """) Here, we've changed the format to include the log-level name, rather than the logger name. Note that we had to double percent signs, because configuration options allow ConfigParser variable substitution. We've also illustrated, with a contrived example, that the log level can be a numeric value and that the verbosity can be specified in the configuration file. Because the verbosity is subtracted from the log level, we get a final log level of 20, which is the INFO level. >>> print system(buildout), INFO Develop: /tmp/sample-buildout/recipes/setup.py Predefined buildout options --------------------------- Buildouts have a number of predefined options that recipes can use and that users can override in their configuration files. To see these, we'll run a minimal buildout configuration with a debug logging level. One of the features of debug logging is that the configuration database is shown. >>> write(sample_buildout, 'buildout.cfg', ... """ ... [buildout] ... parts = ... """) >>> print system(buildout+' -v'), Configuration data: [buildout] bin-directory = /tmp/sample-buildout/bin develop-eggs-directory = /tmp/sample-buildout/develop-eggs directory = /tmp/sample-buildout eggs-directory = /tmp/sample-buildout/eggs executable = /usr/local/bin/python2.3 installed = /tmp/sample-buildout/.installed.cfg log-format = %%(name)s: %%(message)s log-level = INFO parts = parts-directory = /tmp/sample-buildout/parts python = buildout verbosity = 10 <BLANKLINE> zc.buildout.easy_install: Installing ['zc.buildout', 'setuptools'] zc.buildout.easy_install: We have a develop egg for zc.buildout zc.buildout.easy_install: We have the best distribution that satisfies setuptools All of these options can be overridden by configuration files or by command-line assignments. We've discussed most of these options already, but let's review them and touch on some we haven't discussed: bin-directory The directory path where scripts are written. This can be a relative path, which is interpreted relative to the directory option. develop-eggs-directory The directory path where development egg links are created for software being created in the local project. This can be a relative path, which is interpreted relative to the directory option. directory The buildout directory. This is the base for other buildout file and directory locations, when relative locations are used. eggs-directory The directory path where downloaded eggs are put. It is common to share this directory across buildouts. Eggs in this directory should *never* be modified. This can be a relative path, which is interpreted relative to the directory option. executable The Python executable used to run the buildout. See the python option below. installed The file path where information about the results of the previous buildout run is written. This can be a relative path, which is interpreted relative to the directory option. This file provides an inventory of installed parts with information needed to decide which if any parts need to be uninstalled. log-format The format used for logging messages. log-level The log level before verbosity adjustment parts A white space separated list of parts to be installed. parts-directory A working directory that parts can used to store data. python The name of a section containing information about the default Python interpreter. Recipes that need a installation typically have options to tell them which Python installation to use. By convention, if a section-specific option isn't used, the option is looked for in the buildout section. The option must point to a section with an executable option giving the path to a Python executable. By default, the buildout section defines the default Python as the Python used to run the buildout. verbosity A log-level adjustment. Typically, this is set via the -q and -v command-line options. Bootstrapping ------------- If zc.buildout is installed, you can use it to create a new buildout with it's own local copies of zc.buildout and setuptools and with local buildout scripts. >>> sample_bootstrapped = mkdtemp('sample-bootstrapped') >>> print system(buildout ... +' -c'+os.path.join(sample_bootstrapped, 'setup.cfg') ... +' bootstrap'), Warning: creating /sample-bootstrapped/setup.cfg buildout: Creating directory /sample-bootstrapped/bin buildout: Creating directory /sample-bootstrapped/parts buildout: Creating directory /sample-bootstrapped/eggs buildout: Creating directory /sample-bootstrapped/develop-eggs Note that a basic setup.cfg was created for us. >>> ls(sample_bootstrapped) d bin d develop-eggs d eggs d parts - setup.cfg >>> ls(sample_bootstrapped, 'bin') - buildout >>> ls(sample_bootstrapped, 'eggs') - setuptools-0.6-py2.3.egg - zc.buildout-1.0-py2.3.egg Note that the buildout script was installed but not run. To run the buildout, we'd have to run the installed buildout script.