merge lp:~gary/zc.buildout/python-support (also available as...

merge lp:~gary/zc.buildout/python-support (also available as svn+ssh://svn.zope.org/repos/main/zc.buildout/branches/gary-launchpad).  This adds support for system Pythons, as well as other changes, such as improved Distribute support and some cleanups.

git-svn-id: http://svn.zope.org/repos/main/zc.buildout/trunk@111588 62d5b8a3-27da-0310-9561-8e5933582275

.bzrignore

+.installed.cfg
+bin
+build
+develop-eggs
+eggs
+parts
+src/zc.buildout.egg-info
+z3c.recipe.scripts_/src/z3c.recipe.scripts.egg-info
+zc.recipe.egg_/src/zc.recipe.egg.egg-info

CHANGES.txt

 Change History
 **************
 
-1.4.4 (?)
-=========
+1.5.0 (201?-??-??)
+==================
+
+New Features:
+
+- Added buildout:socket-timout option so that socket timeout can be configured
+  both from command line and from config files. (gotcha)
+
+- Buildout can be safely used with a system Python (or any Python with code
+  in site-packages), as long as you use the new z3c.recipe.scripts
+  recipe to generate scripts and interpreters, rather than zc.recipe.egg.
+
+  zc.recipe.egg is still a fully supported, and simpler, way of
+  generating scripts and interpreters if you are using a "clean" Python,
+  without code installed in site-packages. It keeps its previous behavior in
+  order to provide backwards compatibility.
+
+  The z3c.recipe.scripts recipe allows you to control how you use the
+  code in site-packages.  You can exclude it entirely; allow eggs in it
+  to fulfill package dependencies declared in setup.py and buildout
+  configuration; allow it to be available but not used to fulfill
+  dependencies declared in setup.py or buildout configuration; or only
+  allow certain eggs in site-packages to fulfill dependencies.
+
+- Added new function, ``zc.buildout.easy_install.sitepackage_safe_scripts``,
+  to generate scripts and interpreter.  It produces a full-featured
+  interpreter (all command-line options supported) and the ability to
+  safely let scripts include site packages, such as with a system
+  Python.  The ``z3c.recipe.scripts`` recipe uses this new function.
+
+- Improve bootstrap.
+
+  * New options let you specify where to find ez_setup.py and where to find
+    a download cache.  These options can keep bootstrap from going over the
+    network.
+
+  * Another new option lets you specify where to put generated eggs.
+
+  * The buildout script generated by bootstrap honors more of the settings
+    in the designated configuration file (e.g., buildout.cfg).
+
+- You can develop zc.buildout using Distribute instead of Setuptools.  Use
+  the --distribute option on the dev.py script.  (Releases should be tested
+  with both Distribute and Setuptools.)
+
+- The ``distribute-version`` now works in the [buildout] section, mirroring
+  the ``setuptools-version`` option (this is for consistency; using the
+  general-purpose ``versions`` option is preferred).
+
+Bugs fixed:
+
+- Using Distribute with the ``allow-picked-versions = false`` buildout
+  option no longer causes an error.
+
+- The handling and documenting of default buildout options was normalized.
+  This means, among other things, that ``bin/buildout -vv`` and
+  ``bin/buildout annotate`` correctly list more of the options.
+
+- Installing a namespace package using a Python that already has a package
+  in the same namespace (e.g., in the Python's site-packages) failed in
+  some cases.  It is now handled correctly.
 
+- Another variation of this error showed itself when at least two
+  dependencies were in a shared location like site-packages, and the
+  first one met the "versions" setting.  The first dependency would be
+  added, but subsequent dependencies from the same location (e.g.,
+  site-packages) would use the version of the package found in the
+  shared location, ignoring the version setting.  This is also now
+  handled correctly.
 
 1.4.3 (2009-12-10)
 ==================

README.txt

@@ -35,7 +35,15 @@ Existing recipes include:
 `zc.recipe.egg <http://pypi.python.org/pypi/zc.recipe.egg>`_
    The egg recipe installes one or more eggs, with their
    dependencies.  It installs their console-script entry points with
-   the needed eggs included in their paths.
+   the needed eggs included in their paths.  It is suitable for use with
+   a "clean" Python: one without packages installed in site-packages.
+
+`z3c.recipe.scripts <http://pypi.python.org/pypi/z3c.recipe.scripts>`_
+  Like zc.recipe.egg, this recipe builds interpreter scripts and entry
+  point scripts based on eggs.  It can be used with a Python that has
+  packages installed in site-packages, such as a system Python.  The
+  interpreter also has more features than the one offered by
+  zc.recipe.egg.
 
 `zc.recipe.testrunner <http://pypi.python.org/pypi/zc.recipe.testrunner>`_
    The testrunner egg creates a test runner script for one or

bootstrap/bootstrap.py

@@ -20,102 +20,187 @@ use the -c option to specify an alternate configuration file.
 $Id$
 """
 
-import os, shutil, sys, tempfile, urllib2
+import os, shutil, sys, tempfile, textwrap, urllib, urllib2
 from optparse import OptionParser
 
-tmpeggs = tempfile.mkdtemp()
+if sys.platform == 'win32':
+    def quote(c):
+        if ' ' in c:
+            return '"%s"' % c # work around spawn lamosity on windows
+        else:
+            return c
+else:
+    quote = str
+
+# In order to be more robust in the face of system Pythons, we want to
+# run without site-packages loaded.  This is somewhat tricky, in
+# particular because Python 2.6's distutils imports site, so starting
+# with the -S flag is not sufficient.  However, we'll start with that:
+if 'site' in sys.modules:
+    # We will restart with python -S.
+    args = sys.argv[:]
+    args[0:0] = [sys.executable, '-S']
+    args = map(quote, args)
+    os.execv(sys.executable, args)
+# Now we are running with -S.  We'll get the clean sys.path, import site
+# because distutils will do it later, and then reset the path and clean
+# out any namespace packages from site-packages that might have been
+# loaded by .pth files.
+clean_path = sys.path[:]
+import site
+sys.path[:] = clean_path
+for k, v in sys.modules.items():
+    if (hasattr(v, '__path__') and
+        len(v.__path__)==1 and
+        not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
+        # This is a namespace package.  Remove it.
+        sys.modules.pop(k)
 
 is_jython = sys.platform.startswith('java')
 
+setuptools_source = 'http://peak.telecommunity.com/dist/ez_setup.py'
+distribute_source = 'http://python-distribute.org/distribute_setup.py'
+
 # parsing arguments
-parser = OptionParser()
+def normalize_to_url(option, opt_str, value, parser):
+    if value:
+        if '://' not in value: # It doesn't smell like a URL.
+            value = 'file://%s' % (
+                urllib.pathname2url(
+                    os.path.abspath(os.path.expanduser(value))),)
+        if opt_str == '--download-base' and not value.endswith('/'):
+            # Download base needs a trailing slash to make the world happy.
+            value += '/'
+    else:
+        value = None
+    name = opt_str[2:].replace('-', '_')
+    setattr(parser.values, name, value)
+
+usage = '''\
+[DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
+
+Bootstraps a buildout-based project.
+
+Simply run this script in a directory containing a buildout.cfg, using the
+Python that you want bin/buildout to use.
+
+Note that by using --setup-source and --download-base to point to
+local resources, you can keep this script from going over the network.
+'''
+
+parser = OptionParser(usage=usage)
 parser.add_option("-v", "--version", dest="version",
                           help="use a specific zc.buildout version")
 parser.add_option("-d", "--distribute",
-                   action="store_true", dest="distribute", default=False,
+                   action="store_true", dest="use_distribute", default=False,
                    help="Use Distribute rather than Setuptools.")
-
+parser.add_option("--setup-source", action="callback", dest="setup_source",
+                  callback=normalize_to_url, nargs=1, type="string",
+                  help=("Specify a URL or file location for the setup file. "
+                        "If you use Setuptools, this will default to " +
+                        setuptools_source + "; if you use Distribute, this "
+                        "will default to " + distribute_source +"."))
+parser.add_option("--download-base", action="callback", dest="download_base",
+                  callback=normalize_to_url, nargs=1, type="string",
+                  help=("Specify a URL or directory for downloading "
+                        "zc.buildout and either Setuptools or Distribute. "
+                        "Defaults to PyPI."))
+parser.add_option("--eggs",
+                  help=("Specify a directory for storing eggs.  Defaults to "
+                        "a temporary directory that is deleted when the "
+                        "bootstrap script completes."))
 parser.add_option("-c", None, action="store", dest="config_file",
                    help=("Specify the path to the buildout configuration "
                          "file to be used."))
 
 options, args = parser.parse_args()
 
-# if -c was provided, we push it back into args for buildout' main function
+# if -c was provided, we push it back into args for buildout's main function
 if options.config_file is not None:
     args += ['-c', options.config_file]
 
-if options.version is not None:
-    VERSION = '==%s' % options.version
+if options.eggs:
+    eggs_dir = os.path.abspath(os.path.expanduser(options.eggs))
 else:
-    VERSION = ''
+    eggs_dir = tempfile.mkdtemp()
+
+if options.setup_source is None:
+    if options.use_distribute:
+        options.setup_source = distribute_source
+    else:
+        options.setup_source = setuptools_source
 
-USE_DISTRIBUTE = options.distribute
 args = args + ['bootstrap']
 
-to_reload = False
+
 try:
+    to_reload = False
     import pkg_resources
-    if not hasattr(pkg_resources, '_distribute'):
     to_reload = True
+    if not hasattr(pkg_resources, '_distribute'):
         raise ImportError
+    import setuptools # A flag.  Sometimes pkg_resources is installed alone.
 except ImportError:
+    ez_code = urllib2.urlopen(
+        options.setup_source).read().replace('\r\n', '\n')
     ez = {}
-    if USE_DISTRIBUTE:
-        exec urllib2.urlopen('http://python-distribute.org/distribute_setup.py'
-                         ).read() in ez
-        ez['use_setuptools'](to_dir=tmpeggs, download_delay=0, no_fake=True)
-    else:
-        exec urllib2.urlopen('http://peak.telecommunity.com/dist/ez_setup.py'
-                             ).read() in ez
-        ez['use_setuptools'](to_dir=tmpeggs, download_delay=0)
-
+    exec ez_code in ez
+    setup_args = dict(to_dir=eggs_dir, download_delay=0)
+    if options.download_base:
+        setup_args['download_base'] = options.download_base
+    if options.use_distribute:
+        setup_args['no_fake'] = True
+    ez['use_setuptools'](**setup_args)
     if to_reload:
         reload(pkg_resources)
     else:
         import pkg_resources
-
-if sys.platform == 'win32':
-    def quote(c):
-        if ' ' in c:
-            return '"%s"' % c # work around spawn lamosity on windows
-        else:
-            return c
+    # This does not (always?) update the default working set.  We will
+    # do it.
+    for path in sys.path:
+        if path not in pkg_resources.working_set.entries:
+            pkg_resources.working_set.add_entry(path)
+
+cmd = [quote(sys.executable),
+       '-c',
+       quote('from setuptools.command.easy_install import main; main()'),
+       '-mqNxd',
+       quote(eggs_dir)]
+
+if options.download_base:
+    cmd.extend(['-f', quote(options.download_base)])
+
+requirement = 'zc.buildout'
+if options.version:
+    requirement = '=='.join((requirement, options.version))
+cmd.append(requirement)
+
+if options.use_distribute:
+    setup_requirement = 'distribute'
 else:
-    def quote (c):
-        return c
-
-cmd = 'from setuptools.command.easy_install import main; main()'
+    setup_requirement = 'setuptools'
 ws = pkg_resources.working_set
-
-if USE_DISTRIBUTE:
-    requirement = 'distribute'
-else:
-    requirement = 'setuptools'
+env = dict(
+    os.environ,
+    PYTHONPATH=ws.find(
+        pkg_resources.Requirement.parse(setup_requirement)).location)
 
 if is_jython:
     import subprocess
-
-    assert subprocess.Popen([sys.executable] + ['-c', quote(cmd), '-mqNxd',
-           quote(tmpeggs), 'zc.buildout' + VERSION],
-           env=dict(os.environ,
-               PYTHONPATH=
-               ws.find(pkg_resources.Requirement.parse(requirement)).location
-               ),
-           ).wait() == 0
-
-else:
-    assert os.spawnle(
-        os.P_WAIT, sys.executable, quote (sys.executable),
-        '-c', quote (cmd), '-mqNxd', quote (tmpeggs), 'zc.buildout' + VERSION,
-        dict(os.environ,
-            PYTHONPATH=
-            ws.find(pkg_resources.Requirement.parse(requirement)).location
-            ),
-        ) == 0
-
-ws.add_entry(tmpeggs)
-ws.require('zc.buildout' + VERSION)
+    exitcode = subprocess.Popen(cmd, env=env).wait()
+else: # Windows prefers this, apparently; otherwise we would prefer subprocess
+    exitcode = os.spawnle(*([os.P_WAIT, sys.executable] + cmd + [env]))
+if exitcode != 0:
+    sys.stdout.flush()
+    sys.stderr.flush()
+    print ("An error occured when trying to install zc.buildout. "
+           "Look above this message for any errors that "
+           "were output by easy_install.")
+    sys.exit(exitcode)
+
+ws.add_entry(eggs_dir)
+ws.require(requirement)
 import zc.buildout.buildout
 zc.buildout.buildout.main(args)
-shutil.rmtree(tmpeggs)
+if not options.eggs: # clean up temporary egg directory
+    shutil.rmtree(eggs_dir)

buildout.cfg

 [buildout]
-develop = zc.recipe.egg_ .
+develop = zc.recipe.egg_ z3c.recipe.scripts_ .
 parts = test oltest py
 
 [py]
-recipe = zc.recipe.egg
+recipe = z3c.recipe.scripts
 eggs = zc.buildout
        zope.testing
 interpreter = py
@@ -13,6 +13,7 @@ recipe = zc.recipe.testrunner
 eggs =
   zc.buildout
   zc.recipe.egg
+  z3c.recipe.scripts
 
 # Tests that can be run wo a network
 [oltest]
@@ -20,6 +21,7 @@ recipe = zc.recipe.testrunner
 eggs =
   zc.buildout
   zc.recipe.egg
+  z3c.recipe.scripts
 defaults =
   [
   '-t',

dev.py

@@ -13,43 +13,126 @@
 ##############################################################################
 """Bootstrap the buildout project itself.
 
-This is different from a normal boostrapping process because the
+This is different from a normal bootstrapping process because the
 buildout egg itself is installed as a develop egg.
 
 $Id$
 """
 
 import os, shutil, sys, subprocess, urllib2
+from optparse import OptionParser
+
+if sys.platform == 'win32':
+    def quote(c):
+        if ' ' in c:
+            return '"%s"' % c # work around spawn lamosity on windows
+        else:
+            return c
+else:
+    quote = str
+
+# In order to be more robust in the face of system Pythons, we want to
+# run without site-packages loaded.  This is somewhat tricky, in
+# particular because Python 2.6's distutils imports site, so starting
+# with the -S flag is not sufficient.  However, we'll start with that:
+if 'site' in sys.modules:
+    # We will restart with python -S.
+    args = sys.argv[:]
+    args[0:0] = [sys.executable, '-S']
+    args = map(quote, args)
+    os.execv(sys.executable, args)
+# Now we are running with -S.  We'll get the clean sys.path, import site
+# because distutils will do it later, and then reset the path and clean
+# out any namespace packages from site-packages that might have been
+# loaded by .pth files.
+clean_path = sys.path[:]
+import site
+sys.path[:] = clean_path
+for k, v in sys.modules.items():
+    if (hasattr(v, '__path__') and
+        len(v.__path__)==1 and
+        not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
+        # This is a namespace package.  Remove it.
+        sys.modules.pop(k)
 
 is_jython = sys.platform.startswith('java')
 
+setuptools_source = 'http://peak.telecommunity.com/dist/ez_setup.py'
+distribute_source = 'http://python-distribute.org/distribute_setup.py'
+
+usage = '''\
+[DESIRED PYTHON FOR DEVELOPING BUILDOUT] dev.py [options]
+
+Bootstraps buildout itself for development.
+
+This is different from a normal bootstrapping process because the
+buildout egg itself is installed as a develop egg.
+'''
+
+parser = OptionParser(usage=usage)
+parser.add_option("-d", "--distribute",
+                   action="store_true", dest="use_distribute", default=False,
+                   help="Use Distribute rather than Setuptools.")
+
+options, args = parser.parse_args()
+
+if args:
+    parser.error('This script accepts no arguments other than its options.')
+
+if options.use_distribute:
+    setup_source = distribute_source
+else:
+    setup_source = setuptools_source
+
 for d in 'eggs', 'develop-eggs', 'bin':
     if not os.path.exists(d):
         os.mkdir(d)
-
 if os.path.isdir('build'):
     shutil.rmtree('build')
 
 try:
+    to_reload = False
     import pkg_resources
+    to_reload = True
+    if not hasattr(pkg_resources, '_distribute'):
+        raise ImportError
+    import setuptools # A flag.  Sometimes pkg_resources is installed alone.
 except ImportError:
+    ez_code = urllib2.urlopen(setup_source).read().replace('\r\n', '\n')
     ez = {}
-    exec urllib2.urlopen('http://peak.telecommunity.com/dist/ez_setup.py'
-                         ).read() in ez
-    ez['use_setuptools'](to_dir='eggs', download_delay=0)
-
+    exec ez_code in ez
+    setup_args = dict(to_dir='eggs', download_delay=0)
+    if options.use_distribute:
+        setup_args['no_fake'] = True
+    ez['use_setuptools'](**setup_args)
+    if to_reload:
+        reload(pkg_resources)
+    else:
         import pkg_resources
+    # This does not (always?) update the default working set.  We will
+    # do it.
+    for path in sys.path:
+        if path not in pkg_resources.working_set.entries:
+            pkg_resources.working_set.add_entry(path)
 
+env = os.environ.copy() # Windows needs yet-to-be-determined values from this.
+env['PYTHONPATH'] = os.path.dirname(pkg_resources.__file__)
 subprocess.Popen(
     [sys.executable] +
     ['setup.py', '-q', 'develop', '-m', '-x', '-d', 'develop-eggs'],
-    env = {'PYTHONPATH': os.path.dirname(pkg_resources.__file__)}).wait()
+    env=env).wait()
 
 pkg_resources.working_set.add_entry('src')
 
 import zc.buildout.easy_install
-zc.buildout.easy_install.scripts(
-    ['zc.buildout'], pkg_resources.working_set , sys.executable, 'bin')
+if not os.path.exists('parts'):
+    os.mkdir('parts')
+partsdir = os.path.join('parts', 'buildout')
+if not os.path.exists(partsdir):
+    os.mkdir(partsdir)
+zc.buildout.easy_install.sitepackage_safe_scripts(
+    'bin', pkg_resources.working_set, sys.executable, partsdir,
+    reqs=['zc.buildout'])
 
 bin_buildout = os.path.join('bin', 'buildout')
 
@@ -57,4 +140,5 @@ if is_jython:
     # Jython needs the script to be called twice via sys.executable
     assert subprocess.Popen([sys.executable] + [bin_buildout]).wait() == 0
 
+
 sys.exit(subprocess.Popen(bin_buildout).wait())

setup.py

@@ -12,7 +12,7 @@
 #
 ##############################################################################
 name = "zc.buildout"
-version = "1.4.4dev"
+version = "1.5.0dev"
 
 import os
 from setuptools import setup

src/zc/buildout/bootstrap.txt

@@ -47,7 +47,7 @@ Make sure the bootstrap script actually works::
     X...
     d  zc.buildout-...egg
 
-Now trying the `--version` option, that let you define a version for
+Now we will try the `--version` option, which lets you define a version for
 `zc.buildout`. If not provided, bootstrap will look for the latest one.
 
 Let's try with an unknown version::
@@ -57,7 +57,7 @@ Let's try with an unknown version::
     ...     'bootstrap.py --version UNKNOWN'); print 'X' # doctest: +ELLIPSIS
     ...
     X
-    No local packages or download links found for zc.buildout==UNKNOWN
+    No local packages or download links found for zc.buildout==UNKNOWN...
     ...
 
 Now let's try with `1.1.2`, which happens to exist::
@@ -119,8 +119,8 @@ Let's make sure the generated `buildout` script uses it::
         zc.buildout.buildout.main()
     <BLANKLINE>
 
-`zc.buildout` now can also run with `Distribute` with the `--distribute` option::
-
+`zc.buildout` now can also run with `Distribute` with the `--distribute`
+option::
 
     >>> print 'X'; print system(
     ...     zc.buildout.easy_install._safe_arg(sys.executable)+' '+
@@ -153,7 +153,8 @@ Make sure both options can be used together::
 
     >>> print 'X'; print system(
     ...     zc.buildout.easy_install._safe_arg(sys.executable)+' '+
-    ...     'bootstrap.py --distribute --version 1.2.1'); print 'X' # doctest: +ELLIPSIS
+    ...     'bootstrap.py --distribute --version 1.2.1'); print 'X'
+    ... # doctest: +ELLIPSIS
     ...
     X
     ...
@@ -161,7 +162,8 @@ Make sure both options can be used together::
     <BLANKLINE>
     X
 
-Let's make sure the generated `buildout` script uses ``Distribute`` *and* ``zc.buildout-1.2.1``::
+Let's make sure the generated `buildout` script uses ``Distribute`` *and*
+``zc.buildout-1.2.1``::
 
     >>> print open(buildout_script).read() # doctest: +ELLIPSIS
     #...
@@ -194,4 +196,70 @@ Last, the -c option needs to work on bootstrap.py::
     <BLANKLINE>
     X
 
+You can specify a location of ez_setup.py or distribute_setup, so you
+can rely on a local or remote location.  We'll write our own ez_setup.py
+that we will also use to test some other bootstrap options.
+
+    >>> write('ez_setup.py', '''\
+    ... def use_setuptools(**kwargs):
+    ...     import sys, pprint
+    ...     pprint.pprint(kwargs, width=40)
+    ...     sys.exit()
+    ... ''')
+    >>> print system(
+    ...     zc.buildout.easy_install._safe_arg(sys.executable)+' '+
+    ...     'bootstrap.py --setup-source=./ez_setup.py')
+    ... # doctest: +ELLIPSIS
+    {'download_delay': 0,
+     'to_dir': '...'}
+    <BLANKLINE>
+
+You can also pass a download-cache, and a place in which eggs should be stored
+(they are normally stored in a temporary directory).
+
+    >>> print system(
+    ...     zc.buildout.easy_install._safe_arg(sys.executable)+' '+
+    ...     'bootstrap.py --setup-source=./ez_setup.py '+
+    ...     '--download-base=./download-cache --eggs=eggs')
+    ... # doctest: +ELLIPSIS
+    {'download_base': '/sample/download-cache/',
+     'download_delay': 0,
+     'to_dir': '/sample/eggs'}
+    <BLANKLINE>
+
+Here's the entire help text.
 
+    >>> print system(
+    ...     zc.buildout.easy_install._safe_arg(sys.executable)+' '+
+    ...     'bootstrap.py --help'),
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    Usage: [DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
+    <BLANKLINE>
+    Bootstraps a buildout-based project.
+    <BLANKLINE>
+    Simply run this script in a directory containing a buildout.cfg, using the
+    Python that you want bin/buildout to use.
+    <BLANKLINE>
+    Note that by using --setup-source and --download-base to point to
+    local resources, you can keep this script from going over the network.
+    <BLANKLINE>
+    <BLANKLINE>
+    Options:
+      -h, --help            show this help message and exit
+      -v VERSION, --version=VERSION
+                            use a specific zc.buildout version
+      -d, --distribute      Use Distribute rather than Setuptools.
+      --setup-source=SETUP_SOURCE
+                            Specify a URL or file location for the setup file. If
+                            you use Setuptools, this will default to
+                            http://peak.telecommunity.com/dist/ez_setup.py; if you
+                            use Distribute, this will default to http://python-
+                            distribute.org/distribute_setup.py.
+      --download-base=DOWNLOAD_BASE
+                            Specify a URL or directory for downloading zc.buildout
+                            and either Setuptools or Distribute. Defaults to PyPI.
+      --eggs=EGGS           Specify a directory for storing eggs.  Defaults to a
+                            temporary directory that is deleted when the bootstrap
+                            script completes.
+      -c CONFIG_FILE        Specify the path to the buildout configuration file to
+                            be used.

src/zc/buildout/buildout.txt

@@ -56,10 +56,9 @@ The bin directory contains scripts.
     -  setuptools-0.6-py2.4.egg
     -  zc.buildout-1.0-py2.4.egg
 
-The develop-eggs and parts directories are initially empty:
+The develop-eggs directory is initially empty:
 
     >>> ls(sample_buildout, 'develop-eggs')
-    >>> ls(sample_buildout, 'parts')
 
 The develop-eggs directory holds egg links for software being
 developed in the buildout.  We separate develop-eggs and other eggs to
@@ -69,6 +68,12 @@ directory in their home that all non-develop eggs are stored in.  This
 allows larger buildouts to be set up much more quickly and saves disk
 space.
 
+The parts directory just contains some helpers for the buildout script
+itself.
+
+    >>> ls(sample_buildout, 'parts')
+    d  buildout
+
 The parts directory provides an area where recipes can install
 part data.  For example, if we built a custom Python, we would
 install it in the part directory.  Part data is stored in a
@@ -576,7 +581,7 @@ When we rerun the buildout:
 .. Wait for the file to really disappear. My linux is weird.
 
     >>> wait_until("foo goes away", lambda : not os.path.exists('foo'),
-    ...            timeout=100)
+    ...            timeout=200)
 
 we get the same error, but we don't get the directory left behind:
 
@@ -724,6 +729,10 @@ COMMAND_LINE_VALUE).
     ==================
     <BLANKLINE>
     [buildout]
+    allow-hosts= *
+        DEFAULT_VALUE
+    allow-picked-versions= true
+        DEFAULT_VALUE
     bin-directory= bin
         DEFAULT_VALUE
     develop= recipes
@@ -736,18 +745,36 @@ COMMAND_LINE_VALUE).
         DEFAULT_VALUE
     executable= ...
         DEFAULT_VALUE
+    find-links=
+        DEFAULT_VALUE
+    install-from-cache= false
+        DEFAULT_VALUE
     installed= .installed.cfg
         DEFAULT_VALUE
     log-format=
         DEFAULT_VALUE
     log-level= INFO
         DEFAULT_VALUE
+    newest= true
+        DEFAULT_VALUE
+    offline= false
+        DEFAULT_VALUE
     parts= data-dir
         /sample-buildout/buildout.cfg
     parts-directory= parts
         DEFAULT_VALUE
+    prefer-final= false
+        DEFAULT_VALUE
     python= buildout
         DEFAULT_VALUE
+    relative-paths= false
+        DEFAULT_VALUE
+    socket-timeout=
+        DEFAULT_VALUE
+    unzip= false
+        DEFAULT_VALUE
+    use-dependency-links= true
+        DEFAULT_VALUE
     <BLANKLINE>
     [data-dir]
     path= foo bins
@@ -2194,17 +2221,21 @@ database is shown.
 
     >>> print system(buildout+' -vv'), # doctest: +NORMALIZE_WHITESPACE
     Installing 'zc.buildout', 'setuptools'.
-    We have a develop egg: zc.buildout 1.0.0.
+    We have a develop egg: zc.buildout X.X.
     We have the best distribution that satisfies 'setuptools'.
-    Picked: setuptools = 0.6
+    Picked: setuptools = V.V
     <BLANKLINE>
     Configuration data:
     [buildout]
+    allow-hosts = *
+    allow-picked-versions = true
     bin-directory = /sample-buildout/bin
     develop-eggs-directory = /sample-buildout/develop-eggs
     directory = /sample-buildout
     eggs-directory = /sample-buildout/eggs
-    executable = /usr/local/bin/python2.3
+    executable = python
+    find-links =
+    install-from-cache = false
     installed = /sample-buildout/.installed.cfg
     log-format =
     log-level = INFO
@@ -2212,7 +2243,12 @@ database is shown.
     offline = false
     parts =
     parts-directory = /sample-buildout/parts
+    prefer-final = false
     python = buildout
+    relative-paths = false
+    socket-timeout =
+    unzip = false
+    use-dependency-links = true
     verbosity = 20
     <BLANKLINE>
 
@@ -2273,6 +2309,33 @@ python
    Python executable.  By default, the buildout section defines the
    default Python as the Python used to run the buildout.
 
+relative-paths
+    The paths generated by zc.buildout are absolute by default, and this
+    option is ``false``.  However, if you set this value to be ``true``,
+    bin/buildout will be generated with code that makes the paths relative.
+    Some recipes, such as zc.recipe.egg and z3c.recipe.scripts, honor this
+    value as well.
+
+unzip
+    By default, zc.buildout doesn't unzip zip-safe eggs ("unzip = false").
+    This follows the policy followed by setuptools itself.  Experience shows
+    this policy to to be inconvenient.  Zipped eggs make debugging more
+    difficult and often import more slowly.  You can include an unzip option in
+    the buildout section to change the default unzipping policy ("unzip =
+    true").
+
+use-dependency-links
+    By default buildout will obey the setuptools dependency_links metadata
+    when it looks for dependencies. This behavior can be controlled with
+    the use-dependency-links buildout option::
+
+      [buildout]
+      ...
+      use-dependency-links = false
+
+    The option defaults to true. If you set it to false, then dependency
+    links are only looked for in the locations specified by find-links.
+
 verbosity
    A log-level adjustment.  Typically, this is set via the -q and -v
    command-line options.
@@ -2319,8 +2382,56 @@ buildout or setuptools egg could be installed in the develop-eggs
 directory if the original buildout had develop eggs for either
 buildout or setuptools.)
 
-Note that the buildout script was installed but not run.  To run
-the buildout, we'd have to run the installed buildout script.
+If relative-paths is ``true``, the buildout script uses relative paths.
+
+    >>> write(sample_bootstrapped, 'setup.cfg',
+    ... '''
+    ... [buildout]
+    ... relative-paths = true
+    ... parts =
+    ... ''')
+
+    >>> print system(buildout
+    ...              +' -c'+os.path.join(sample_bootstrapped, 'setup.cfg')
+    ...              +' bootstrap'),
+    Generated script '/sample-bootstrapped/bin/buildout'.
+
+    >>> buildout_script = join(sample_bootstrapped, 'bin', 'buildout')
+    >>> import sys
+    >>> if sys.platform.startswith('win'):
+    ...     buildout_script += '-script.py'
+    >>> print open(buildout_script).read() # doctest: +ELLIPSIS
+    #!... -S
+    <BLANKLINE>
+    import os
+    <BLANKLINE>
+    join = os.path.join
+    base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
+    base = os.path.dirname(base)
+    <BLANKLINE>
+    import sys
+    sys.path[0:0] = [
+        join(base, 'parts/buildout'),
+        ]
+    <BLANKLINE>
+    <BLANKLINE>
+    import os
+    path = sys.path[0]
+    if os.environ.get('PYTHONPATH'):
+        path = os.pathsep.join([path, os.environ['PYTHONPATH']])
+    os.environ['PYTHONPATH'] = path
+    import site # imports custom buildout-generated site.py
+    <BLANKLINE>
+    import zc.buildout.buildout
+    <BLANKLINE>
+    if __name__ == '__main__':
+        zc.buildout.buildout.main()
+    <BLANKLINE>
+
+
+Note that, in the above two examples, the buildout script was installed
+but not run.  To run the buildout, we'd have to run the installed
+buildout script.
 
 If we have an existing buildout that already has a buildout.cfg, we'll
 normally use the bootstrap command instead of init.  It will complain

src/zc/buildout/testing.py

@@ -28,6 +28,7 @@ import socket
 import subprocess
 import sys
 import tempfile
+import textwrap
 import threading
 import time
 import urllib2
@@ -109,6 +110,16 @@ def system(command, input=''):
     e.close()
     return result
 
+def call_py(interpreter, cmd, flags=None):
+    if sys.platform == 'win32':
+        args = ['"%s"' % arg for arg in (interpreter, flags, cmd) if arg]
+        args.insert(-1, '"-c"')
+        return system('"%s"' % ' '.join(args))
+    else:
+        cmd = repr(cmd)
+        return system(
+            ' '.join(arg for arg in (interpreter, flags, '-c', cmd) if arg))
+
 def get(url):
     return urllib2.urlopen(url).read()
 
@@ -120,7 +131,11 @@ def _runsetup(setup, executable, *args):
     args = [zc.buildout.easy_install._safe_arg(arg)
             for arg in args]
     args.insert(0, '-q')
-    args.append(dict(os.environ, PYTHONPATH=setuptools_location))
+    env = dict(os.environ)
+    if executable == sys.executable:
+        env['PYTHONPATH'] = setuptools_location
+    # else pass an executable that has setuptools! See testselectingpython.py.
+    args.append(env)
 
     here = os.getcwd()
     try:
@@ -139,6 +154,11 @@ def sdist(setup, dest):
 def bdist_egg(setup, executable, dest):
     _runsetup(setup, executable, 'bdist_egg', '-d', dest)
 
+def sys_install(setup, dest):
+    _runsetup(setup, sys.executable, 'install', '--install-purelib', dest,
+              '--record', os.path.join(dest, '__added_files__'),
+              '--single-version-externally-managed')
+
 def find_python(version):
     e = os.environ.get('PYTHON%s' % version)
     if e is not None:
@@ -206,14 +226,64 @@ def wait_until(label, func, *args, **kw):
         time.sleep(0.01)
     raise ValueError('Timed out waiting for: '+label)
 
+def get_installer_values():
+    """Get the current values for the easy_install module.
+
+    This is necessary because instantiating a Buildout will force the
+    Buildout's values on the installer.
+
+    Returns a dict of names-values suitable for set_installer_values."""
+    names = ('default_versions', 'download_cache', 'install_from_cache',
+             'prefer_final', 'include_site_packages',
+             'allowed_eggs_from_site_packages', 'use_dependency_links',
+             'allow_picked_versions', 'always_unzip'
+            )
+    values = {}
+    for name in names:
+        values[name] = getattr(zc.buildout.easy_install, name)()
+    return values
+
+def set_installer_values(values):
+    """Set the given values on the installer."""
+    for name, value in values.items():
+        getattr(zc.buildout.easy_install, name)(value)
+
+def make_buildout(executable=None):
+    """Make a buildout that uses this version of zc.buildout."""
+    # Create a basic buildout.cfg to avoid a warning from buildout.
+    open('buildout.cfg', 'w').write(
+        "[buildout]\nparts =\n"
+        )
+    # Get state of installer defaults so we can reinstate them (instantiating
+    # a Buildout will force the Buildout's defaults on the installer).
+    installer_values = get_installer_values()
+    # Use the buildout bootstrap command to create a buildout
+    config = [
+        ('buildout', 'log-level', 'WARNING'),
+        # trick bootstrap into putting the buildout develop egg
+        # in the eggs dir.
+        ('buildout', 'develop-eggs-directory', 'eggs'),
+        ]
+    if executable is not None:
+        config.append(('buildout', 'executable', executable))
+    zc.buildout.buildout.Buildout(
+        'buildout.cfg', config,
+        user_defaults=False,
+        ).bootstrap([])
+    # Create the develop-eggs dir, which didn't get created the usual
+    # way due to the trick above:
+    os.mkdir('develop-eggs')
+    # Reinstate the default values of the installer.
+    set_installer_values(installer_values)
+
 def buildoutSetUp(test):
 
     test.globs['__tear_downs'] = __tear_downs = []
     test.globs['register_teardown'] = register_teardown = __tear_downs.append
 
-    prefer_final = zc.buildout.easy_install.prefer_final()
+    installer_values = get_installer_values()
     register_teardown(
-        lambda: zc.buildout.easy_install.prefer_final(prefer_final)
+        lambda: set_installer_values(installer_values)
         )
 
     here = os.getcwd()
@@ -259,27 +329,7 @@ def buildoutSetUp(test):
     sample = tmpdir('sample-buildout')
 
     os.chdir(sample)
-
-    # Create a basic buildout.cfg to avoid a warning from buildout:
-    open('buildout.cfg', 'w').write(
-        "[buildout]\nparts =\n"
-        )
-
-    # Use the buildout bootstrap command to create a buildout
-    zc.buildout.buildout.Buildout(
-        'buildout.cfg',
-        [('buildout', 'log-level', 'WARNING'),
-         # trick bootstrap into putting the buildout develop egg
-         # in the eggs dir.
-         ('buildout', 'develop-eggs-directory', 'eggs'),
-         ]
-        ).bootstrap([])
-
-
-
-    # Create the develop-eggs dir, which didn't get created the usual
-    # way due to the trick above:
-    os.mkdir('develop-eggs')
+    make_buildout()
 
     def start_server(path):
         port, thread = _start_server(path, name=path)
@@ -287,6 +337,50 @@ def buildoutSetUp(test):
         register_teardown(lambda: stop_server(url, thread))
         return url
 
+    def make_py(initialization=''):
+        """Returns paths to new executable and to its site-packages.
+        """
+        buildout = tmpdir('executable_buildout')
+        site_packages_dir = os.path.join(buildout, 'site-packages')
+        mkdir(site_packages_dir)
+        old_wd = os.getcwd()
+        os.chdir(buildout)
+        make_buildout()
+        # Normally we don't process .pth files in extra-paths.  We want to
+        # in this case so that we can test with setuptools system installs
+        # (--single-version-externally-managed), which use .pth files.
+        initialization = (
+            ('import sys\n'
+             'import site\n'
+             'known_paths = set(sys.path)\n'
+             'site_packages_dir = %r\n'
+             'site.addsitedir(site_packages_dir, known_paths)\n'
+            ) % (site_packages_dir,)) + initialization
+        initialization = '\n'.join(
+            '  ' + line for line in initialization.split('\n'))
+        install_develop(
+            'zc.recipe.egg', os.path.join(buildout, 'develop-eggs'))
+        install_develop(
+            'z3c.recipe.scripts', os.path.join(buildout, 'develop-eggs'))
+        write('buildout.cfg', textwrap.dedent('''\
+            [buildout]
+            parts = py
+
+            [py]
+            recipe = z3c.recipe.scripts
+            interpreter = py
+            initialization =
+            %(initialization)s
+            extra-paths = %(site-packages)s
+            eggs = setuptools
+            ''') % {
+                'initialization': initialization,
+                'site-packages': site_packages_dir})
+        system(os.path.join(buildout, 'bin', 'buildout'))
+        os.chdir(old_wd)
+        return (
+            os.path.join(buildout, 'bin', 'py'), site_packages_dir)
+
     test.globs.update(dict(
         sample_buildout = sample,
         ls = ls,
@@ -297,6 +391,7 @@ def buildoutSetUp(test):
         tmpdir = tmpdir,
         write = write,
         system = system,
+        call_py = call_py,
         get = get,
         cd = (lambda *path: os.chdir(os.path.join(*path))),
         join = os.path.join,
@@ -305,10 +400,9 @@ def buildoutSetUp(test):
         start_server = start_server,
         buildout = os.path.join(sample, 'bin', 'buildout'),
         wait_until = wait_until,
+        make_py = make_py
         ))
 
-    zc.buildout.easy_install.prefer_final(prefer_final)
-
 def buildoutTearDown(test):
     for f in test.globs['__tear_downs']:
         f()

src/zc/buildout/testselectingpython.py

@@ -11,7 +11,7 @@
 # FOR A PARTICULAR PURPOSE.
 #
 ##############################################################################
-import os, re, sys, unittest
+import os, re, subprocess, sys, textwrap, unittest
 from zope.testing import doctest, renormalizing
 import zc.buildout.tests
 import zc.buildout.testing
@@ -42,6 +42,33 @@ We can specify a specific Python executable.
 
 def multi_python(test):
     other_executable = zc.buildout.testing.find_python(other_version)
+    command = textwrap.dedent('''\
+        try:
+            import setuptools
+        except ImportError:
+            import sys
+            sys.exit(1)
+        ''')
+    if subprocess.call([other_executable, '-c', command],
+                       env=os.environ):
+        # the other executable does not have setuptools.  Get setuptools.
+        # We will do this using the same tools we are testing, for better or
+        # worse.  Alternatively, we could try using bootstrap.
+        executable_dir = test.globs['tmpdir']('executable_dir')
+        executable_parts = os.path.join(executable_dir, 'parts')
+        test.globs['mkdir'](executable_parts)
+        ws = zc.buildout.easy_install.install(
+            ['setuptools'], executable_dir,
+            index='http://www.python.org/pypi/',
+            always_unzip=True, executable=other_executable)
+        zc.buildout.easy_install.sitepackage_safe_scripts(
+            executable_dir, ws, other_executable, executable_parts,
+            reqs=['setuptools'], interpreter='py')
+        original_executable = other_executable
+        other_executable = os.path.join(executable_dir, 'py')
+        assert not subprocess.call(
+            [other_executable, '-c', command], env=os.environ), (
+            'test set up failed')
     sample_eggs = test.globs['tmpdir']('sample_eggs')
     os.mkdir(os.path.join(sample_eggs, 'index'))
     test.globs['sample_eggs'] = sample_eggs

src/zc/buildout/update.txt

@@ -78,25 +78,30 @@ new versions found in new releases:
     zc.buildout 99.99
     setuptools 99.99
 
-Our buildout script has been updated to use the new eggs:
+Our buildout script's site.py has been updated to use the new eggs:
 
-    >>> cat(sample_buildout, 'bin', 'buildout')
-    #!/usr/local/bin/python2.4
+    >>> cat(sample_buildout, 'parts', 'buildout', 'site.py')
+    ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
+    "...
+    def addsitepackages(known_paths):
+        """Add site packages, as determined by zc.buildout.
     <BLANKLINE>
-    import sys
-    sys.path[0:0] = [
-      '/sample-buildout/eggs/zc.buildout-99.99-py2.4.egg',
-      '/sample-buildout/eggs/setuptools-99.99-py2.4.egg',
+        See original_addsitepackages, below, for the original version."""
+        buildout_paths = [
+            '/sample-buildout/eggs/zc.buildout-99.99-pyN.N.egg',
+            '/sample-buildout/eggs/setuptools-99.99-pyN.N.egg'
             ]
-    <BLANKLINE>
-    import zc.buildout.buildout
-    <BLANKLINE>
-    if __name__ == '__main__':
-        zc.buildout.buildout.main()
+        for path in buildout_paths:
+            sitedir, sitedircase = makepath(path)
+            if not sitedircase in known_paths and os.path.exists(sitedir):
+                sys.path.append(sitedir)
+                known_paths.add(sitedircase)
+        return known_paths
+    ...
 
 Now, let's recreate the sample buildout. If we specify constraints on
-the versions of zc.buildout and setuptools to use, running the
-buildout will install earlier versions of these packages:
+the versions of zc.buildout and setuptools (or distribute) to use,
+running the buildout will install earlier versions of these packages:
 
     >>> write(sample_buildout, 'buildout.cfg',
     ... """
@@ -107,6 +112,7 @@ buildout will install earlier versions of these packages:
     ... develop = showversions
     ... zc.buildout-version = < 99
     ... setuptools-version = < 99
+    ... distribute-version = < 99
     ...
     ... [show-versions]
     ... recipe = showversions
@@ -119,7 +125,6 @@ Now we can see that we actually "upgrade" to an earlier version.
       zc.buildout version 1.0.0,
       setuptools version 0.6;
     restarting.
-    Generated script '/sample-buildout/bin/buildout'.
     Develop: '/sample-buildout/showversions'
     Updating show-versions.
     zc.buildout 1.0.0

z3c.recipe.scripts_/CHANGES.txt

+Change History
+**************
+
+1.0.0
+=====
+
+Initial public version.

z3c.recipe.scripts_/README.txt

+********************************
+Buildout Script Recipe
+********************************
+
+.. contents::
+
+The script recipe installs eggs into a buildout eggs directory, exactly
+like zc.recipe.egg, and then generates scripts in a buildout bin
+directory with egg paths baked into them.
+

z3c.recipe.scripts_/setup.py

+##############################################################################
+#
+# Copyright (c) 2007 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Setup for z3c.recipe.scripts package
+
+$Id: setup.py 106736 2009-12-18 02:33:08Z gary $
+"""
+
+version = '1.0.0dev'
+
+import os
+from setuptools import setup, find_packages
+
+def read(*rnames):
+    return open(os.path.join(os.path.dirname(__file__), *rnames)).read()
+
+name = "z3c.recipe.scripts"
+setup(
+    name = name,
+    version = version,
+    author = "Gary Poster",
+    author_email = "gary.poster@canonical.com",
+    description = "Recipe for installing Python scripts",
+    long_description = (
+        read('README.txt')
+        + '\n' +
+        read('CHANGES.txt')
+        + '\n' +
+        'Detailed Documentation\n'
+        '**********************\n'
+        + '\n' +
+        read('src', 'z3c', 'recipe', 'scripts', 'README.txt')
+        + '\n' +
+        'Download\n'
+        '*********\n'
+        ),
+    keywords = "development build",
+    classifiers = [
+       'Development Status :: 5 - Production/Stable',
+       'Framework :: Buildout',
+       'Intended Audience :: Developers',
+       'License :: OSI Approved :: Zope Public License',
+       'Topic :: Software Development :: Build Tools',
+       'Topic :: Software Development :: Libraries :: Python Modules',
+       ],
+    url='http://cheeseshop.python.org/pypi/z3c.recipe.scripts',
+    license = "ZPL 2.1",
+
+    packages = find_packages('src'),
+    package_dir = {'':'src'},
+    namespace_packages = ['z3c', 'z3c.recipe'],
+    install_requires = [
+        'zc.buildout >=1.5.0dev',
+        'zc.recipe.egg >=1.2.3dev',
+        'setuptools'],
+    tests_require = ['zope.testing'],
+    test_suite = name+'.tests.test_suite',
+    entry_points = {'zc.buildout': ['default = %s:Scripts' % name,
+                                    'script = %s:Scripts' % name,
+                                    'scripts = %s:Scripts' % name,
+                                    'interpreter = %s:Interpreter' % name,
+                                    ]
+                    },
+    include_package_data = True,
+    zip_safe=False,
+    )

z3c.recipe.scripts_/src/z3c/__init__.py

+__import__('pkg_resources').declare_namespace(__name__)

z3c.recipe.scripts_/src/z3c/recipe/__init__.py

+__import__('pkg_resources').declare_namespace(__name__)

z3c.recipe.scripts_/src/z3c/recipe/scripts/__init__.py

+from z3c.recipe.scripts.scripts import Scripts, Interpreter

z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py

+##############################################################################
+#
+# Copyright (c) 2009-2010 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Install scripts from eggs.
+"""
+import os
+import zc.buildout
+import zc.buildout.easy_install
+from zc.recipe.egg.egg import ScriptBase
+
+
+class Base(ScriptBase):
+
+    def __init__(self, buildout, name, options):
+        if 'extends' in options:
+            options.update(buildout[options['extends']])
+        super(Base, self).__init__(buildout, name, options)
+        self.default_eggs = '' # Disables feature from zc.recipe.egg.
+        b_options = buildout['buildout']
+        options['parts-directory'] = os.path.join(
+            b_options['parts-directory'], self.name)
+
+        value = options.setdefault(
+            'allowed-eggs-from-site-packages',
+            '*')
+        self.allowed_eggs = tuple(name.strip() for name in value.split('\n'))
+
+        value = options.setdefault(
+            'include-site-packages',
+            b_options.get('include-site-packages', 'false'))
+        if value not in ('true', 'false'):
+            raise zc.buildout.UserError(
+                "Invalid value for include-site-packages option: %s" %
+                (value,))
+        self.include_site_packages = (value == 'true')
+
+        value = options.setdefault(
+            'exec-sitecustomize',
+            b_options.get('exec-sitecustomize', 'false'))
+        if value not in ('true', 'false'):
+            raise zc.buildout.UserError(
+                "Invalid value for exec-sitecustomize option: %s" %
+                (value,))
+        self.exec_sitecustomize = (value == 'true')
+
+
+class Interpreter(Base):
+
+    def __init__(self, buildout, name, options):
+        super(Interpreter, self).__init__(buildout, name, options)
+
+        options.setdefault('name', name)
+
+    def install(self):
+        reqs, ws = self.working_set()
+        options = self.options
+        generated = []
+        if not os.path.exists(options['parts-directory']):
+            os.mkdir(options['parts-directory'])
+            generated.append(options['parts-directory'])
+        generated.extend(zc.buildout.easy_install.sitepackage_safe_scripts(
+            options['bin-directory'], ws, options['executable'],
+            options['parts-directory'],
+            interpreter=options['name'],
+            extra_paths=self.extra_paths,
+            initialization=options.get('initialization', ''),
+            include_site_packages=self.include_site_packages,
+            exec_sitecustomize=self.exec_sitecustomize,
+            relative_paths=self._relative_paths,
+            ))
+        return generated
+
+    update = install
+
+
+class Scripts(Base):
+
+    def _install(self, reqs, ws, scripts):
+        options = self.options
+        generated = []
+        if not os.path.exists(options['parts-directory']):
+            os.mkdir(options['parts-directory'])
+            generated.append(options['parts-directory'])
+        generated.extend(zc.buildout.easy_install.sitepackage_safe_scripts(
+            options['bin-directory'], ws, options['executable'],
+            options['parts-directory'], reqs=reqs, scripts=scripts,
+            interpreter=options.get('interpreter'),
+            extra_paths=self.extra_paths,
+            initialization=options.get('initialization', ''),
+            include_site_packages=self.include_site_packages,
+            exec_sitecustomize=self.exec_sitecustomize,
+            relative_paths=self._relative_paths,
+            script_arguments=options.get('arguments', ''),
+            script_initialization=options.get('script-initialization', '')
+            ))
+        return generated

zc.recipe.egg_/setup.py

@@ -16,7 +16,7 @@
 $Id$
 """
 
-version = '0'
+version = '1.2.3dev'
 
 import os
 from setuptools import setup, find_packages
@@ -66,7 +66,7 @@ setup(
     package_dir = {'':'src'},
     namespace_packages = ['zc', 'zc.recipe'],
     install_requires = [
-        'zc.buildout >=1.2.0',
+        'zc.buildout >=1.5.0dev',
         'setuptools'],
     tests_require = ['zope.testing'],
     test_suite = name+'.tests.test_suite',

zc.recipe.egg_/src/zc/recipe/egg/README.txt

@@ -154,6 +154,8 @@ dependent-scripts
 interpreter
    The name of a script to generate that allows access to a Python
    interpreter that has the path set based on the eggs installed.
+   (See the ``z3c.recipe.scripts`` recipe for a more full-featured
+   interpreter.)
 
 extra-paths
    Extra paths to include in a generated script.
@@ -577,7 +579,7 @@ declare entry points using the entry-points option:
     -  demo
     -  other
 
-    >>> cat(sample_buildout, 'bin', 'other')
+    >>> cat(sample_buildout, 'bin', 'other') # doctest: +NORMALIZE_WHITESPACE
     #!/usr/local/bin/python2.4
     <BLANKLINE>
     import sys
@@ -640,3 +642,4 @@ be made to contact an index server:
     Uninstalling bigdemo.
     Installing demo.
     Generated script '/sample-buildout/bin/foo'.
+

zc.recipe.egg_/src/zc/recipe/egg/api.txt

@@ -117,6 +117,7 @@ computed by the egg recipe by looking at .installed.cfg:
     extras = other
     find-links = http://localhost:27071/
     index = http://localhost:27071/index
+    python = buildout
     recipe = sample
 
 If we use the extra-paths option:

zc.recipe.egg_/src/zc/recipe/egg/custom.txt

@@ -150,6 +150,7 @@ eggs directory can be shared across multiple buildouts.
 
     >>> ls(sample_buildout, 'develop-eggs')
     d  extdemo-1.4-py2.4-unix-i686.egg
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Note that no scripts or dependencies are installed.  To install
@@ -231,6 +232,7 @@ We won't get an update.
     >>> ls(sample_buildout, 'develop-eggs')
     -  demo.egg-link
     d  extdemo-1.4-py2.4-unix-i686.egg
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 But if we run the buildout in the default on-line and newest modes, we
@@ -248,6 +250,7 @@ version is imported:
     -  demo.egg-link
     d  extdemo-1.4-py2.4-linux-i686.egg
     d  extdemo-1.5-py2.4-linux-i686.egg
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Controlling the version used
@@ -287,6 +290,7 @@ We can specify a specific version using the egg option:
     >>> ls(sample_buildout, 'develop-eggs')
     -  demo.egg-link
     d  extdemo-1.4-py2.4-linux-i686.egg
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 
@@ -553,6 +557,7 @@ Our develop-eggs now includes an egg link for extdemo:
     >>> ls('develop-eggs')
     -  demo.egg-link
     -  extdemo.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 and the extdemo now has a built extension:

zc.recipe.egg_/src/zc/recipe/egg/egg.py

@@ -19,14 +19,17 @@ $Id$
 import logging, os, re, zipfile
 import zc.buildout.easy_install
 
+
 class Eggs(object):
 
+    include_site_packages = allowed_eggs = None
+
     def __init__(self, buildout, name, options):
         self.buildout = buildout
-        self.name = name
+        self.name = self.default_eggs = name
         self.options = options
-        links = options.get('find-links',
-                            buildout['buildout'].get('find-links'))
+        b_options = buildout['buildout']
+        links = options.get('find-links', b_options['find-links'])
         if links:
             links = links.split()
             options['find-links'] = '\n'.join(links)
@@ -34,25 +37,25 @@ class Eggs(object):
             links = ()
         self.links = links
 
-        index = options.get('index', buildout['buildout'].get('index'))
+        index = options.get('index', b_options.get('index'))
         if index is not None:
             options['index'] = index
         self.index = index
 
-        allow_hosts = buildout['buildout'].get('allow-hosts', '*')
+        allow_hosts = b_options['allow-hosts']
         allow_hosts = tuple([host.strip() for host in allow_hosts.split('\n')
                                if host.strip()!=''])
         self.allow_hosts = allow_hosts
 
-        options['eggs-directory'] = buildout['buildout']['eggs-directory']
+        options['eggs-directory'] = b_options['eggs-directory']
         options['_e'] = options['eggs-directory'] # backward compat.
-        options['develop-eggs-directory'
-                ] = buildout['buildout']['develop-eggs-directory']
+        options['develop-eggs-directory'] = b_options['develop-eggs-directory']
         options['_d'] = options['develop-eggs-directory'] # backward compat.
 
-        assert options.get('unzip') in ('true', 'false', None)
+        # verify that this is None, 'true' or 'false'
+        get_bool(options, 'unzip')
 
-        python = options.get('python', buildout['buildout']['python'])
+        python = options.setdefault('python', b_options['python'])
         options['executable'] = buildout[python]['executable']
 
     def working_set(self, extra=()):
@@ -61,31 +64,36 @@ class Eggs(object):
         This is intended for reuse by similar recipes.
         """
         options = self.options
+        b_options = self.buildout['buildout']
 
         distributions = [
             r.strip()
-            for r in options.get('eggs', self.name).split('\n')
+            for r in options.get('eggs', self.default_eggs).split('\n')
             if r.strip()]
         orig_distributions = distributions[:]
         distributions.extend(extra)
 
-        if self.buildout['buildout'].get('offline') == 'true':
+        if b_options.get('offline') == 'true':
             ws = zc.buildout.easy_install.working_set(
                 distributions, options['executable'],
-                [options['develop-eggs-directory'], options['eggs-directory']]
+                [options['develop-eggs-directory'],
+                 options['eggs-directory']],
+                include_site_packages=self.include_site_packages,
+                allowed_eggs_from_site_packages=self.allowed_eggs,
                 )
         else:
             kw = {}
-            if options.get('unzip'):
+            if 'unzip' in options:
                 kw['always_unzip'] = get_bool(options, 'unzip')
-
             ws = zc.buildout.easy_install.install(
                 distributions, options['eggs-directory'],
                 links=self.links,
                 index=self.index,
                 executable=options['executable'],
                 path=[options['develop-eggs-directory']],
-                newest=self.buildout['buildout'].get('newest') == 'true',
+                newest=b_options.get('newest') == 'true',
+                include_site_packages=self.include_site_packages,
+                allowed_eggs_from_site_packages=self.allowed_eggs,
                 allow_hosts=self.allow_hosts,
                 **kw)
 
@@ -97,16 +105,19 @@ class Eggs(object):
 
     update = install
 
-class Scripts(Eggs):
+
+class ScriptBase(Eggs):
 
     def __init__(self, buildout, name, options):
-        super(Scripts, self).__init__(buildout, name, options)
+        super(ScriptBase, self).__init__(buildout, name, options)
+
+        b_options = buildout['buildout']
 
-        options['bin-directory'] = buildout['buildout']['bin-directory']
+        options['bin-directory'] = b_options['bin-directory']
         options['_b'] = options['bin-directory'] # backward compat.
 
         self.extra_paths = [
-            os.path.join(buildout['buildout']['directory'], p.strip())
+            os.path.join(b_options['directory'], p.strip())
             for p in options.get('extra-paths', '').split('\n')
             if p.strip()
             ]
@@ -115,11 +126,9 @@ class Scripts(Eggs):
 
 
         relative_paths = options.get(
-            'relative-paths',
-            buildout['buildout'].get('relative-paths', 'false')
-            )
+            'relative-paths', b_options.get('relative-paths', 'false'))
         if relative_paths == 'true':
-            options['buildout-directory'] = buildout['buildout']['directory']
+            options['buildout-directory'] = b_options['directory']
             self._relative_paths = options['buildout-directory']
         else:
             self._relative_paths = ''
@@ -128,12 +137,13 @@ class Scripts(Eggs):
     parse_entry_point = re.compile(
         '([^=]+)=(\w+(?:[.]\w+)*):(\w+(?:[.]\w+)*)$'
         ).match
+
     def install(self):
         reqs, ws = self.working_set()
         options = self.options
 
         scripts = options.get('scripts')
-        if scripts or scripts is None:
+        if scripts or scripts is None or options.get('interpreter'):
             if scripts is not None:
                 scripts = scripts.split()
                 scripts = dict([
@@ -157,7 +167,20 @@ class Scripts(Eggs):
                     name = dist.project_name
                     if name != 'setuptools' and name not in reqs:
                         reqs.append(name)
+            return self._install(reqs, ws, scripts)
+        return ()
 
+    update = install
+
+    def _install(self, reqs, ws, scripts):
+        # Subclasses implement this.
+        raise NotImplementedError()
+
+
+class Scripts(ScriptBase):
+
+    def _install(self, reqs, ws, scripts):
+        options = self.options
         return zc.buildout.easy_install.scripts(
             reqs, ws, options['executable'],
             options['bin-directory'],
@@ -166,12 +189,9 @@ class Scripts(Eggs):
             interpreter=options.get('interpreter'),
             initialization=options.get('initialization', ''),
             arguments=options.get('arguments', ''),
-                relative_paths=self._relative_paths,
+            relative_paths=self._relative_paths
             )
 
-        return ()
-
-    update = install
 
 def get_bool(options, name, default=False):
     value = options.get(name)

zc.recipe.egg_/src/zc/recipe/egg/selecting-python.txt

@@ -35,7 +35,7 @@ install the demo package using Python 2.4.
     ... index = http://www.python.org/pypi/
     ...
     ... [python2.4]
-    ... executable = %(python23)s
+    ... executable = %(python24)s
     ...
     ... [demo]
     ... recipe = zc.recipe.egg
@@ -43,7 +43,7 @@ install the demo package using Python 2.4.
     ... find-links = %(server)s
     ... python = python2.4
     ... interpreter = py-demo
-    ... """ % dict(server=link_server, python23=other_executable))
+    ... """ % dict(server=link_server, python24=other_executable))
 
 Now, if we run the buildout:
 

zc.recipe.egg_/src/zc/recipe/egg/tests.py

@@ -50,9 +50,12 @@ def test_suite():
                zc.buildout.tests.normalize_bang,
                (re.compile('zc.buildout(-\S+)?[.]egg(-link)?'),
                 'zc.buildout.egg'),
-               (re.compile('[-d]  setuptools-[^-]+-'), 'setuptools-X-'),
+               (re.compile('[-d]  (setuptools|distribute)-[^-]+-'),
+                'setuptools-X-'),
                (re.compile(r'eggs\\\\demo'), 'eggs/demo'),
                (re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'),
+               # Distribute unzips eggs by default.
+               (re.compile('\-  demoneeded'), 'd  demoneeded'),
                ])
             ),
         doctest.DocFileSuite(
@@ -64,7 +67,7 @@ def test_suite():
                (re.compile('__buildout_signature__ = '
                            'sample-\S+\s+'
                            'zc.recipe.egg-\S+\s+'
-                           'setuptools-\S+\s+'
+                           '(setuptools|distribute)-\S+\s+'
                            'zc.buildout-\S+\s*'
                            ),
                 '__buildout_signature__ = sample- zc.recipe.egg-\n'),
@@ -104,14 +107,17 @@ def test_suite():
                    zc.buildout.testing.normalize_path,
                    zc.buildout.testing.normalize_endings,
                    zc.buildout.testing.normalize_script,
-                   (re.compile('Got setuptools \S+'), 'Got setuptools V'),
-                   (re.compile('([d-]  )?setuptools-\S+-py'),
+                   (re.compile('Got (setuptools|distribute) \S+'),
+                    'Got setuptools V'),
+                   (re.compile('([d-]  )?(setuptools|distribute)-\S+-py'),
                     'setuptools-V-py'),
                    (re.compile('-py2[.][0-35-9][.]'), 'py2.5.'),
                    (re.compile('zc.buildout-\S+[.]egg'),
                     'zc.buildout.egg'),
                    (re.compile('zc.buildout[.]egg-link'),
                     'zc.buildout.egg'),
+                   # Distribute unzips eggs by default.
+                   (re.compile('\-  demoneeded'), 'd  demoneeded'),
                    ]),
                 ),
             )