merge lp:~gary/zc.buildout/python-support (also available as...

merge lp:~gary/zc.buildout/python-support (also available as svn+ssh://svn.zope.org/repos/main/zc.buildout/branches/gary-launchpad).  This adds support for system Pythons, as well as other changes, such as improved Distribute support and some cleanups.

git-svn-id: http://svn.zope.org/repos/main/zc.buildout/trunk@111588 62d5b8a3-27da-0310-9561-8e5933582275

.bzrignore

+.installed.cfg
+bin
+build
+develop-eggs
+eggs
+parts
+src/zc.buildout.egg-info
+z3c.recipe.scripts_/src/z3c.recipe.scripts.egg-info
+zc.recipe.egg_/src/zc.recipe.egg.egg-info

CHANGES.txt

 Change History
 **************
 
-1.4.4 (?)
-=========
+1.5.0 (201?-??-??)
+==================
+
+New Features:
+
+- Added buildout:socket-timout option so that socket timeout can be configured
+  both from command line and from config files. (gotcha)
+
+- Buildout can be safely used with a system Python (or any Python with code
+  in site-packages), as long as you use the new z3c.recipe.scripts
+  recipe to generate scripts and interpreters, rather than zc.recipe.egg.
+
+  zc.recipe.egg is still a fully supported, and simpler, way of
+  generating scripts and interpreters if you are using a "clean" Python,
+  without code installed in site-packages. It keeps its previous behavior in
+  order to provide backwards compatibility.
+
+  The z3c.recipe.scripts recipe allows you to control how you use the
+  code in site-packages.  You can exclude it entirely; allow eggs in it
+  to fulfill package dependencies declared in setup.py and buildout
+  configuration; allow it to be available but not used to fulfill
+  dependencies declared in setup.py or buildout configuration; or only
+  allow certain eggs in site-packages to fulfill dependencies.
+
+- Added new function, ``zc.buildout.easy_install.sitepackage_safe_scripts``,
+  to generate scripts and interpreter.  It produces a full-featured
+  interpreter (all command-line options supported) and the ability to
+  safely let scripts include site packages, such as with a system
+  Python.  The ``z3c.recipe.scripts`` recipe uses this new function.
+
+- Improve bootstrap.
+
+  * New options let you specify where to find ez_setup.py and where to find
+    a download cache.  These options can keep bootstrap from going over the
+    network.
+
+  * Another new option lets you specify where to put generated eggs.
+
+  * The buildout script generated by bootstrap honors more of the settings
+    in the designated configuration file (e.g., buildout.cfg).
+
+- You can develop zc.buildout using Distribute instead of Setuptools.  Use
+  the --distribute option on the dev.py script.  (Releases should be tested
+  with both Distribute and Setuptools.)
+
+- The ``distribute-version`` now works in the [buildout] section, mirroring
+  the ``setuptools-version`` option (this is for consistency; using the
+  general-purpose ``versions`` option is preferred).
+
+Bugs fixed:
+
+- Using Distribute with the ``allow-picked-versions = false`` buildout
+  option no longer causes an error.
+
+- The handling and documenting of default buildout options was normalized.
+  This means, among other things, that ``bin/buildout -vv`` and
+  ``bin/buildout annotate`` correctly list more of the options.
+
+- Installing a namespace package using a Python that already has a package
+  in the same namespace (e.g., in the Python's site-packages) failed in
+  some cases.  It is now handled correctly.
 
+- Another variation of this error showed itself when at least two
+  dependencies were in a shared location like site-packages, and the
+  first one met the "versions" setting.  The first dependency would be
+  added, but subsequent dependencies from the same location (e.g.,
+  site-packages) would use the version of the package found in the
+  shared location, ignoring the version setting.  This is also now
+  handled correctly.
 
 1.4.3 (2009-12-10)
 ==================

README.txt

@@ -35,7 +35,15 @@ Existing recipes include:
 `zc.recipe.egg <http://pypi.python.org/pypi/zc.recipe.egg>`_
    The egg recipe installes one or more eggs, with their
    dependencies.  It installs their console-script entry points with
-   the needed eggs included in their paths.
+   the needed eggs included in their paths.  It is suitable for use with
+   a "clean" Python: one without packages installed in site-packages.
+
+`z3c.recipe.scripts <http://pypi.python.org/pypi/z3c.recipe.scripts>`_
+  Like zc.recipe.egg, this recipe builds interpreter scripts and entry
+  point scripts based on eggs.  It can be used with a Python that has
+  packages installed in site-packages, such as a system Python.  The
+  interpreter also has more features than the one offered by
+  zc.recipe.egg.
 
 `zc.recipe.testrunner <http://pypi.python.org/pypi/zc.recipe.testrunner>`_
    The testrunner egg creates a test runner script for one or

bootstrap/bootstrap.py

@@ -20,102 +20,187 @@ use the -c option to specify an alternate configuration file.
 $Id$
 """
 
-import os, shutil, sys, tempfile, urllib2
+import os, shutil, sys, tempfile, textwrap, urllib, urllib2
 from optparse import OptionParser
 
-tmpeggs = tempfile.mkdtemp()
+if sys.platform == 'win32':
+    def quote(c):
+        if ' ' in c:
+            return '"%s"' % c # work around spawn lamosity on windows
+        else:
+            return c
+else:
+    quote = str
+
+# In order to be more robust in the face of system Pythons, we want to
+# run without site-packages loaded.  This is somewhat tricky, in
+# particular because Python 2.6's distutils imports site, so starting
+# with the -S flag is not sufficient.  However, we'll start with that:
+if 'site' in sys.modules:
+    # We will restart with python -S.
+    args = sys.argv[:]
+    args[0:0] = [sys.executable, '-S']
+    args = map(quote, args)
+    os.execv(sys.executable, args)
+# Now we are running with -S.  We'll get the clean sys.path, import site
+# because distutils will do it later, and then reset the path and clean
+# out any namespace packages from site-packages that might have been
+# loaded by .pth files.
+clean_path = sys.path[:]
+import site
+sys.path[:] = clean_path
+for k, v in sys.modules.items():
+    if (hasattr(v, '__path__') and
+        len(v.__path__)==1 and
+        not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
+        # This is a namespace package.  Remove it.
+        sys.modules.pop(k)
 
 is_jython = sys.platform.startswith('java')
 
+setuptools_source = 'http://peak.telecommunity.com/dist/ez_setup.py'
+distribute_source = 'http://python-distribute.org/distribute_setup.py'
+
 # parsing arguments
-parser = OptionParser()
+def normalize_to_url(option, opt_str, value, parser):
+    if value:
+        if '://' not in value: # It doesn't smell like a URL.
+            value = 'file://%s' % (
+                urllib.pathname2url(
+                    os.path.abspath(os.path.expanduser(value))),)
+        if opt_str == '--download-base' and not value.endswith('/'):
+            # Download base needs a trailing slash to make the world happy.
+            value += '/'
+    else:
+        value = None
+    name = opt_str[2:].replace('-', '_')
+    setattr(parser.values, name, value)
+
+usage = '''\
+[DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
+
+Bootstraps a buildout-based project.
+
+Simply run this script in a directory containing a buildout.cfg, using the
+Python that you want bin/buildout to use.
+
+Note that by using --setup-source and --download-base to point to
+local resources, you can keep this script from going over the network.
+'''
+
+parser = OptionParser(usage=usage)
 parser.add_option("-v", "--version", dest="version",
                           help="use a specific zc.buildout version")
 parser.add_option("-d", "--distribute",
-                   action="store_true", dest="distribute", default=False,
+                   action="store_true", dest="use_distribute", default=False,
                    help="Use Distribute rather than Setuptools.")
-
+parser.add_option("--setup-source", action="callback", dest="setup_source",
+                  callback=normalize_to_url, nargs=1, type="string",
+                  help=("Specify a URL or file location for the setup file. "
+                        "If you use Setuptools, this will default to " +
+                        setuptools_source + "; if you use Distribute, this "
+                        "will default to " + distribute_source +"."))
+parser.add_option("--download-base", action="callback", dest="download_base",
+                  callback=normalize_to_url, nargs=1, type="string",
+                  help=("Specify a URL or directory for downloading "
+                        "zc.buildout and either Setuptools or Distribute. "
+                        "Defaults to PyPI."))
+parser.add_option("--eggs",
+                  help=("Specify a directory for storing eggs.  Defaults to "
+                        "a temporary directory that is deleted when the "
+                        "bootstrap script completes."))
 parser.add_option("-c", None, action="store", dest="config_file",
                    help=("Specify the path to the buildout configuration "
                          "file to be used."))
 
 options, args = parser.parse_args()
 
-# if -c was provided, we push it back into args for buildout' main function
+# if -c was provided, we push it back into args for buildout's main function
 if options.config_file is not None:
     args += ['-c', options.config_file]
 
-if options.version is not None:
-    VERSION = '==%s' % options.version
+if options.eggs:
+    eggs_dir = os.path.abspath(os.path.expanduser(options.eggs))
 else:
-    VERSION = ''
+    eggs_dir = tempfile.mkdtemp()
+
+if options.setup_source is None:
+    if options.use_distribute:
+        options.setup_source = distribute_source
+    else:
+        options.setup_source = setuptools_source
 
-USE_DISTRIBUTE = options.distribute
 args = args + ['bootstrap']
 
-to_reload = False
+
 try:
+    to_reload = False
     import pkg_resources
+    to_reload = True
     if not hasattr(pkg_resources, '_distribute'):
-        to_reload = True
         raise ImportError
+    import setuptools # A flag.  Sometimes pkg_resources is installed alone.
 except ImportError:
+    ez_code = urllib2.urlopen(
+        options.setup_source).read().replace('\r\n', '\n')
     ez = {}
-    if USE_DISTRIBUTE:
-        exec urllib2.urlopen('http://python-distribute.org/distribute_setup.py'
-                         ).read() in ez
-        ez['use_setuptools'](to_dir=tmpeggs, download_delay=0, no_fake=True)
-    else:
-        exec urllib2.urlopen('http://peak.telecommunity.com/dist/ez_setup.py'
-                             ).read() in ez
-        ez['use_setuptools'](to_dir=tmpeggs, download_delay=0)
-
+    exec ez_code in ez
+    setup_args = dict(to_dir=eggs_dir, download_delay=0)
+    if options.download_base:
+        setup_args['download_base'] = options.download_base
+    if options.use_distribute:
+        setup_args['no_fake'] = True
+    ez['use_setuptools'](**setup_args)
     if to_reload:
         reload(pkg_resources)
     else:
         import pkg_resources
-
-if sys.platform == 'win32':
-    def quote(c):
-        if ' ' in c:
-            return '"%s"' % c # work around spawn lamosity on windows
-        else:
-            return c
-else:
-    def quote (c):
-        return c
-
-cmd = 'from setuptools.command.easy_install import main; main()'
-ws  = pkg_resources.working_set
-
-if USE_DISTRIBUTE:
-    requirement = 'distribute'
+    # This does not (always?) update the default working set.  We will
+    # do it.
+    for path in sys.path:
+        if path not in pkg_resources.working_set.entries:
+            pkg_resources.working_set.add_entry(path)
+
+cmd = [quote(sys.executable),
+       '-c',
+       quote('from setuptools.command.easy_install import main; main()'),
+       '-mqNxd',
+       quote(eggs_dir)]
+
+if options.download_base:
+    cmd.extend(['-f', quote(options.download_base)])
+
+requirement = 'zc.buildout'
+if options.version:
+    requirement = '=='.join((requirement, options.version))
+cmd.append(requirement)
+
+if options.use_distribute:
+    setup_requirement = 'distribute'
 else:
-    requirement = 'setuptools'
+    setup_requirement = 'setuptools'
+ws = pkg_resources.working_set
+env = dict(
+    os.environ,
+    PYTHONPATH=ws.find(
+        pkg_resources.Requirement.parse(setup_requirement)).location)
 
 if is_jython:
     import subprocess
-
-    assert subprocess.Popen([sys.executable] + ['-c', quote(cmd), '-mqNxd',
-           quote(tmpeggs), 'zc.buildout' + VERSION],
-           env=dict(os.environ,
-               PYTHONPATH=
-               ws.find(pkg_resources.Requirement.parse(requirement)).location
-               ),
-           ).wait() == 0
-
-else:
-    assert os.spawnle(
-        os.P_WAIT, sys.executable, quote (sys.executable),
-        '-c', quote (cmd), '-mqNxd', quote (tmpeggs), 'zc.buildout' + VERSION,
-        dict(os.environ,
-            PYTHONPATH=
-            ws.find(pkg_resources.Requirement.parse(requirement)).location
-            ),
-        ) == 0
-
-ws.add_entry(tmpeggs)
-ws.require('zc.buildout' + VERSION)
+    exitcode = subprocess.Popen(cmd, env=env).wait()
+else: # Windows prefers this, apparently; otherwise we would prefer subprocess
+    exitcode = os.spawnle(*([os.P_WAIT, sys.executable] + cmd + [env]))
+if exitcode != 0:
+    sys.stdout.flush()
+    sys.stderr.flush()
+    print ("An error occured when trying to install zc.buildout. "
+           "Look above this message for any errors that "
+           "were output by easy_install.")
+    sys.exit(exitcode)
+
+ws.add_entry(eggs_dir)
+ws.require(requirement)
 import zc.buildout.buildout
 zc.buildout.buildout.main(args)
-shutil.rmtree(tmpeggs)
+if not options.eggs: # clean up temporary egg directory
+    shutil.rmtree(eggs_dir)

buildout.cfg

 [buildout]
-develop = zc.recipe.egg_ .
+develop = zc.recipe.egg_ z3c.recipe.scripts_ .
 parts = test oltest py
 
 [py]
-recipe = zc.recipe.egg
+recipe = z3c.recipe.scripts
 eggs = zc.buildout
        zope.testing
 interpreter = py
 
 [test]
 recipe = zc.recipe.testrunner
-eggs = 
+eggs =
   zc.buildout
   zc.recipe.egg
+  z3c.recipe.scripts
 
 # Tests that can be run wo a network
 [oltest]
 recipe = zc.recipe.testrunner
-eggs = 
+eggs =
   zc.buildout
   zc.recipe.egg
+  z3c.recipe.scripts
 defaults =
   [
   '-t',

dev.py

@@ -13,43 +13,126 @@
 ##############################################################################
 """Bootstrap the buildout project itself.
 
-This is different from a normal boostrapping process because the
+This is different from a normal bootstrapping process because the
 buildout egg itself is installed as a develop egg.
 
 $Id$
 """
 
 import os, shutil, sys, subprocess, urllib2
+from optparse import OptionParser
+
+if sys.platform == 'win32':
+    def quote(c):
+        if ' ' in c:
+            return '"%s"' % c # work around spawn lamosity on windows
+        else:
+            return c
+else:
+    quote = str
+
+# In order to be more robust in the face of system Pythons, we want to
+# run without site-packages loaded.  This is somewhat tricky, in
+# particular because Python 2.6's distutils imports site, so starting
+# with the -S flag is not sufficient.  However, we'll start with that:
+if 'site' in sys.modules:
+    # We will restart with python -S.
+    args = sys.argv[:]
+    args[0:0] = [sys.executable, '-S']
+    args = map(quote, args)
+    os.execv(sys.executable, args)
+# Now we are running with -S.  We'll get the clean sys.path, import site
+# because distutils will do it later, and then reset the path and clean
+# out any namespace packages from site-packages that might have been
+# loaded by .pth files.
+clean_path = sys.path[:]
+import site
+sys.path[:] = clean_path
+for k, v in sys.modules.items():
+    if (hasattr(v, '__path__') and
+        len(v.__path__)==1 and
+        not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))):
+        # This is a namespace package.  Remove it.
+        sys.modules.pop(k)
 
 is_jython = sys.platform.startswith('java')
 
+setuptools_source = 'http://peak.telecommunity.com/dist/ez_setup.py'
+distribute_source = 'http://python-distribute.org/distribute_setup.py'
+
+usage = '''\
+[DESIRED PYTHON FOR DEVELOPING BUILDOUT] dev.py [options]
+
+Bootstraps buildout itself for development.
+
+This is different from a normal bootstrapping process because the
+buildout egg itself is installed as a develop egg.
+'''
+
+parser = OptionParser(usage=usage)
+parser.add_option("-d", "--distribute",
+                   action="store_true", dest="use_distribute", default=False,
+                   help="Use Distribute rather than Setuptools.")
+
+options, args = parser.parse_args()
+
+if args:
+    parser.error('This script accepts no arguments other than its options.')
+
+if options.use_distribute:
+    setup_source = distribute_source
+else:
+    setup_source = setuptools_source
+
 for d in 'eggs', 'develop-eggs', 'bin':
     if not os.path.exists(d):
         os.mkdir(d)
-
 if os.path.isdir('build'):
     shutil.rmtree('build')
 
 try:
+    to_reload = False
     import pkg_resources
+    to_reload = True
+    if not hasattr(pkg_resources, '_distribute'):
+        raise ImportError
+    import setuptools # A flag.  Sometimes pkg_resources is installed alone.
 except ImportError:
+    ez_code = urllib2.urlopen(setup_source).read().replace('\r\n', '\n')
     ez = {}
-    exec urllib2.urlopen('http://peak.telecommunity.com/dist/ez_setup.py'
-                         ).read() in ez
-    ez['use_setuptools'](to_dir='eggs', download_delay=0)
-
-    import pkg_resources
+    exec ez_code in ez
+    setup_args = dict(to_dir='eggs', download_delay=0)
+    if options.use_distribute:
+        setup_args['no_fake'] = True
+    ez['use_setuptools'](**setup_args)
+    if to_reload:
+        reload(pkg_resources)
+    else:
+        import pkg_resources
+    # This does not (always?) update the default working set.  We will
+    # do it.
+    for path in sys.path:
+        if path not in pkg_resources.working_set.entries:
+            pkg_resources.working_set.add_entry(path)
 
+env = os.environ.copy() # Windows needs yet-to-be-determined values from this.
+env['PYTHONPATH'] = os.path.dirname(pkg_resources.__file__)
 subprocess.Popen(
     [sys.executable] +
     ['setup.py', '-q', 'develop', '-m', '-x', '-d', 'develop-eggs'],
-    env = {'PYTHONPATH': os.path.dirname(pkg_resources.__file__)}).wait()
+    env=env).wait()
 
 pkg_resources.working_set.add_entry('src')
 
 import zc.buildout.easy_install
-zc.buildout.easy_install.scripts(
-    ['zc.buildout'], pkg_resources.working_set , sys.executable, 'bin')
+if not os.path.exists('parts'):
+    os.mkdir('parts')
+partsdir = os.path.join('parts', 'buildout')
+if not os.path.exists(partsdir):
+    os.mkdir(partsdir)
+zc.buildout.easy_install.sitepackage_safe_scripts(
+    'bin', pkg_resources.working_set, sys.executable, partsdir,
+    reqs=['zc.buildout'])
 
 bin_buildout = os.path.join('bin', 'buildout')
 
@@ -57,4 +140,5 @@ if is_jython:
     # Jython needs the script to be called twice via sys.executable
     assert subprocess.Popen([sys.executable] + [bin_buildout]).wait() == 0
 
+
 sys.exit(subprocess.Popen(bin_buildout).wait())

setup.py

@@ -12,7 +12,7 @@
 #
 ##############################################################################
 name = "zc.buildout"
-version = "1.4.4dev"
+version = "1.5.0dev"
 
 import os
 from setuptools import setup

src/zc/buildout/bootstrap.txt

@@ -47,7 +47,7 @@ Make sure the bootstrap script actually works::
     X...
     d  zc.buildout-...egg
 
-Now trying the `--version` option, that let you define a version for
+Now we will try the `--version` option, which lets you define a version for
 `zc.buildout`. If not provided, bootstrap will look for the latest one.
 
 Let's try with an unknown version::
@@ -57,7 +57,7 @@ Let's try with an unknown version::
     ...     'bootstrap.py --version UNKNOWN'); print 'X' # doctest: +ELLIPSIS
     ...
     X
-    No local packages or download links found for zc.buildout==UNKNOWN
+    No local packages or download links found for zc.buildout==UNKNOWN...
     ...
 
 Now let's try with `1.1.2`, which happens to exist::
@@ -119,8 +119,8 @@ Let's make sure the generated `buildout` script uses it::
         zc.buildout.buildout.main()
     <BLANKLINE>
 
-`zc.buildout` now can also run with `Distribute` with the `--distribute` option::
-
+`zc.buildout` now can also run with `Distribute` with the `--distribute`
+option::
 
     >>> print 'X'; print system(
     ...     zc.buildout.easy_install._safe_arg(sys.executable)+' '+
@@ -153,7 +153,8 @@ Make sure both options can be used together::
 
     >>> print 'X'; print system(
     ...     zc.buildout.easy_install._safe_arg(sys.executable)+' '+
-    ...     'bootstrap.py --distribute --version 1.2.1'); print 'X' # doctest: +ELLIPSIS
+    ...     'bootstrap.py --distribute --version 1.2.1'); print 'X'
+    ... # doctest: +ELLIPSIS
     ...
     X
     ...
@@ -161,7 +162,8 @@ Make sure both options can be used together::
     <BLANKLINE>
     X
 
-Let's make sure the generated `buildout` script uses ``Distribute`` *and* ``zc.buildout-1.2.1``::
+Let's make sure the generated `buildout` script uses ``Distribute`` *and*
+``zc.buildout-1.2.1``::
 
     >>> print open(buildout_script).read() # doctest: +ELLIPSIS
     #...
@@ -194,4 +196,70 @@ Last, the -c option needs to work on bootstrap.py::
     <BLANKLINE>
     X
 
+You can specify a location of ez_setup.py or distribute_setup, so you
+can rely on a local or remote location.  We'll write our own ez_setup.py
+that we will also use to test some other bootstrap options.
+
+    >>> write('ez_setup.py', '''\
+    ... def use_setuptools(**kwargs):
+    ...     import sys, pprint
+    ...     pprint.pprint(kwargs, width=40)
+    ...     sys.exit()
+    ... ''')
+    >>> print system(
+    ...     zc.buildout.easy_install._safe_arg(sys.executable)+' '+
+    ...     'bootstrap.py --setup-source=./ez_setup.py')
+    ... # doctest: +ELLIPSIS
+    {'download_delay': 0,
+     'to_dir': '...'}
+    <BLANKLINE>
+
+You can also pass a download-cache, and a place in which eggs should be stored
+(they are normally stored in a temporary directory).
+
+    >>> print system(
+    ...     zc.buildout.easy_install._safe_arg(sys.executable)+' '+
+    ...     'bootstrap.py --setup-source=./ez_setup.py '+
+    ...     '--download-base=./download-cache --eggs=eggs')
+    ... # doctest: +ELLIPSIS
+    {'download_base': '/sample/download-cache/',
+     'download_delay': 0,
+     'to_dir': '/sample/eggs'}
+    <BLANKLINE>
+
+Here's the entire help text.
 
+    >>> print system(
+    ...     zc.buildout.easy_install._safe_arg(sys.executable)+' '+
+    ...     'bootstrap.py --help'),
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    Usage: [DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
+    <BLANKLINE>
+    Bootstraps a buildout-based project.
+    <BLANKLINE>
+    Simply run this script in a directory containing a buildout.cfg, using the
+    Python that you want bin/buildout to use.
+    <BLANKLINE>
+    Note that by using --setup-source and --download-base to point to
+    local resources, you can keep this script from going over the network.
+    <BLANKLINE>
+    <BLANKLINE>
+    Options:
+      -h, --help            show this help message and exit
+      -v VERSION, --version=VERSION
+                            use a specific zc.buildout version
+      -d, --distribute      Use Distribute rather than Setuptools.
+      --setup-source=SETUP_SOURCE
+                            Specify a URL or file location for the setup file. If
+                            you use Setuptools, this will default to
+                            http://peak.telecommunity.com/dist/ez_setup.py; if you
+                            use Distribute, this will default to http://python-
+                            distribute.org/distribute_setup.py.
+      --download-base=DOWNLOAD_BASE
+                            Specify a URL or directory for downloading zc.buildout
+                            and either Setuptools or Distribute. Defaults to PyPI.
+      --eggs=EGGS           Specify a directory for storing eggs.  Defaults to a
+                            temporary directory that is deleted when the bootstrap
+                            script completes.
+      -c CONFIG_FILE        Specify the path to the buildout configuration file to
+                            be used.

src/zc/buildout/buildout.py

@@ -112,15 +112,26 @@ def _unannotate(data):
     return data
 
 _buildout_default_options = _annotate_section({
-    'eggs-directory': 'eggs',
-    'develop-eggs-directory': 'develop-eggs',
+    'allow-hosts': '*',
+    'allow-picked-versions': 'true',
     'bin-directory': 'bin',
-    'parts-directory': 'parts',
-    'installed': '.installed.cfg',
-    'python': 'buildout',
+    'develop-eggs-directory': 'develop-eggs',
+    'eggs-directory': 'eggs',
     'executable': sys.executable,
-    'log-level': 'INFO',
+    'find-links': '',
+    'install-from-cache': 'false',
+    'installed': '.installed.cfg',
     'log-format': '',
+    'log-level': 'INFO',
+    'newest': 'true',
+    'offline': 'false',
+    'parts-directory': 'parts',
+    'prefer-final': 'false',
+    'python': 'buildout',
+    'relative-paths': 'false',
+    'socket-timeout': '',
+    'unzip': 'false',
+    'use-dependency-links': 'true',
     }, 'DEFAULT_VALUE')
 
 
@@ -191,7 +202,7 @@ class Buildout(UserDict.DictMixin):
         # provide some defaults before options are parsed
         # because while parsing options those attributes might be
         # used already (Gottfried Ganssauge)
-        buildout_section = data.get('buildout')
+        buildout_section = data['buildout']
 
         # Try to make sure we have absolute paths for standard
         # directories. We do this before doing substitutions, in case
@@ -204,22 +215,28 @@ class Buildout(UserDict.DictMixin):
                 d = self._buildout_path(buildout_section[name+'-directory'])
                 buildout_section[name+'-directory'] = d
 
-        links = buildout_section and buildout_section.get('find-links', '')
+        # Attributes on this buildout object shouldn't be used by
+        # recipes in their __init__.  It can cause bugs, because the
+        # recipes will be instantiated below (``options = self['buildout']``)
+        # before this has completed initializing.  These attributes are
+        # left behind for legacy support but recipe authors should
+        # beware of using them.  A better practice is for a recipe to
+        # use the buildout['buildout'] options.
+        links = buildout_section['find-links']
         self._links = links and links.split() or ()
-
-        allow_hosts = buildout_section and buildout_section.get(
-             'allow-hosts', '*').split('\n')
+        allow_hosts = buildout_section['allow-hosts'].split('\n')
         self._allow_hosts = tuple([host.strip() for host in allow_hosts
                                    if host.strip() != ''])
-
         self._logger = logging.getLogger('zc.buildout')
-        self.offline = False
-        self.newest = True
+        self.offline = (buildout_section['offline'] == 'true')
+        self.newest = (buildout_section['newest'] == 'true')
 
         ##################################################################
         ## WARNING!!!
         ## ALL ATTRIBUTES MUST HAVE REASONABLE DEFAULTS AT THIS POINT
-        ## OTHERWISE ATTRIBUTEERRORS MIGHT HAPPEN ANY TIME
+        ## OTHERWISE ATTRIBUTEERRORS MIGHT HAPPEN ANY TIME FROM RECIPES.
+        ## RECIPES SHOULD GENERALLY USE buildout['buildout'] OPTIONS, NOT
+        ## BUILDOUT ATTRIBUTES.
         ##################################################################
         # initialize some attrs and buildout directories.
         options = self['buildout']
@@ -228,7 +245,7 @@ class Buildout(UserDict.DictMixin):
         links = options.get('find-links', '')
         self._links = links and links.split() or ()
 
-        allow_hosts = options.get('allow-hosts', '*').split('\n')
+        allow_hosts = options['allow-hosts'].split('\n')
         self._allow_hosts = tuple([host.strip() for host in allow_hosts
                                    if host.strip() != ''])
 
@@ -246,44 +263,42 @@ class Buildout(UserDict.DictMixin):
 
         self._setup_logging()
 
-        offline = options.get('offline', 'false')
+        offline = options['offline']
         if offline not in ('true', 'false'):
             self._error('Invalid value for offline option: %s', offline)
-        options['offline'] = offline
-        self.offline = offline == 'true'
+        self.offline = (offline == 'true')
 
         if self.offline:
             newest = options['newest'] = 'false'
         else:
-            newest = options.get('newest', 'true')
+            newest = options['newest']
             if newest not in ('true', 'false'):
                 self._error('Invalid value for newest option: %s', newest)
-            options['newest'] = newest
-        self.newest = newest == 'true'
+        self.newest = (newest == 'true')
 
         versions = options.get('versions')
         if versions:
             zc.buildout.easy_install.default_versions(dict(self[versions]))
 
-        prefer_final = options.get('prefer-final', 'false')
+        prefer_final = options['prefer-final']
         if prefer_final not in ('true', 'false'):
             self._error('Invalid value for prefer-final option: %s',
                         prefer_final)
         zc.buildout.easy_install.prefer_final(prefer_final=='true')
 
-        use_dependency_links = options.get('use-dependency-links', 'true')
+        use_dependency_links = options['use-dependency-links']
         if use_dependency_links not in ('true', 'false'):
             self._error('Invalid value for use-dependency-links option: %s',
                         use_dependency_links)
         zc.buildout.easy_install.use_dependency_links(
             use_dependency_links == 'true')
 
-        allow_picked_versions = options.get('allow-picked-versions', 'true')
+        allow_picked_versions = options['allow-picked-versions']
         if allow_picked_versions not in ('true', 'false'):
             self._error('Invalid value for allow-picked-versions option: %s',
                         allow_picked_versions)
         zc.buildout.easy_install.allow_picked_versions(
-            allow_picked_versions=='true')
+            allow_picked_versions == 'true')
 
         download_cache = options.get('download-cache')
         if download_cache:
@@ -300,22 +315,18 @@ class Buildout(UserDict.DictMixin):
 
             zc.buildout.easy_install.download_cache(download_cache)
 
-        install_from_cache = options.get('install-from-cache')
-        if install_from_cache:
-            if install_from_cache not in ('true', 'false'):
-                self._error('Invalid value for install-from-cache option: %s',
-                            install_from_cache)
-            if install_from_cache == 'true':
-                zc.buildout.easy_install.install_from_cache(True)
+        install_from_cache = options['install-from-cache']
+        if install_from_cache not in ('true', 'false'):
+            self._error('Invalid value for install-from-cache option: %s',
+                        install_from_cache)
+        zc.buildout.easy_install.install_from_cache(
+            install_from_cache=='true')
 
-
-        always_unzip = options.get('unzip')
-        if always_unzip:
-            if always_unzip not in ('true', 'false'):
-                self._error('Invalid value for unzip option: %s',
-                            always_unzip)
-            if always_unzip == 'true':
-                zc.buildout.easy_install.always_unzip(True)
+        always_unzip = options['unzip']
+        if always_unzip not in ('true', 'false'):
+            self._error('Invalid value for unzip option: %s',
+                        always_unzip)
+        zc.buildout.easy_install.always_unzip(always_unzip=='true')
 
         # "Use" each of the defaults so they aren't reported as unused options.
         for name in _buildout_default_options:
@@ -338,11 +349,35 @@ class Buildout(UserDict.DictMixin):
 
         self._setup_directories()
 
+        options = self['buildout']
+
+        # Get a base working set for our distributions that corresponds to the
+        # stated desires in the configuration.
+        distributions = ['setuptools', 'zc.buildout']
+        if options.get('offline') == 'true':
+            ws = zc.buildout.easy_install.working_set(
+                distributions, options['executable'],
+                [options['develop-eggs-directory'],
+                 options['eggs-directory']],
+                include_site_packages=False,
+                )
+        else:
+            ws = zc.buildout.easy_install.install(
+                distributions, options['eggs-directory'],
+                links=self._links,
+                index=options.get('index'),
+                executable=options['executable'],
+                path=[options['develop-eggs-directory']],
+                newest=self.newest,
+                allow_hosts=self._allow_hosts,
+                include_site_packages=False,
+                )
+
         # Now copy buildout and setuptools eggs, and record destination eggs:
         entries = []
         for name in 'setuptools', 'zc.buildout':
             r = pkg_resources.Requirement.parse(name)
-            dist = pkg_resources.working_set.find(r)
+            dist = ws.find(r)
             if dist.precedence == pkg_resources.DEVELOP_DIST:
                 dest = os.path.join(self['buildout']['develop-eggs-directory'],
                                     name+'.egg-link')
@@ -361,9 +396,19 @@ class Buildout(UserDict.DictMixin):
         # Create buildout script
         ws = pkg_resources.WorkingSet(entries)
         ws.require('zc.buildout')
-        zc.buildout.easy_install.scripts(
-            ['zc.buildout'], ws, sys.executable,
-            self['buildout']['bin-directory'])
+        partsdir = os.path.join(options['parts-directory'], 'buildout')
+        if not os.path.exists(partsdir):
+            os.mkdir(partsdir)
+        # (Honor the relative-paths option.)
+        relative_paths = options.get('relative-paths', 'false')
+        if relative_paths == 'true':
+            relative_paths = options['directory']
+        else:
+            assert relative_paths == 'false'
+            relative_paths = ''
+        zc.buildout.easy_install.sitepackage_safe_scripts(
+            options['bin-directory'], ws, options['executable'], partsdir,
+            reqs=['zc.buildout'], relative_paths=relative_paths)
 
     init = bootstrap
 
@@ -533,7 +578,7 @@ class Buildout(UserDict.DictMixin):
                 if installed_files is None:
                     self._logger.warning(
                         "The %s install returned None.  A path or "
-                        "iterable os paths should be returned.",
+                        "iterable of paths should be returned.",
                         part)
                     installed_files = ()
                 elif isinstance(installed_files, str):
@@ -792,16 +837,24 @@ class Buildout(UserDict.DictMixin):
         if not self.newest:
             return
 
+        options = self['buildout']
+
+        specs = ['zc.buildout']
+        if zc.buildout.easy_install.is_distribute:
+            specs.append('distribute')
+        else:
+            specs.append('setuptools')
         ws = zc.buildout.easy_install.install(
             [
-            (spec + ' ' + self['buildout'].get(spec+'-version', '')).strip()
-            for spec in ('zc.buildout', 'setuptools')
+            (spec + ' ' + options.get(spec+'-version', '')).strip()
+            for spec in specs
             ],
-            self['buildout']['eggs-directory'],
-            links = self['buildout'].get('find-links', '').split(),
-            index = self['buildout'].get('index'),
-            path = [self['buildout']['develop-eggs-directory']],
-            allow_hosts = self._allow_hosts
+            options['eggs-directory'],
+            links = options.get('find-links', '').split(),
+            index = options.get('index'),
+            path = [options['develop-eggs-directory']],
+            allow_hosts = self._allow_hosts,
+            include_site_packages=False
             )
 
         upgraded = []
@@ -817,7 +870,7 @@ class Buildout(UserDict.DictMixin):
         __doing__ = 'Upgrading.'
 
         should_run = realpath(
-            os.path.join(os.path.abspath(self['buildout']['bin-directory']),
+            os.path.join(os.path.abspath(options['bin-directory']),
                          'buildout')
             )
         if sys.platform == 'win32':
@@ -849,21 +902,34 @@ class Buildout(UserDict.DictMixin):
 
         # the new dist is different, so we've upgraded.
         # Update the scripts and return True
-        zc.buildout.easy_install.scripts(
-            ['zc.buildout'], ws, sys.executable,
-            self['buildout']['bin-directory'],
-            )
+        partsdir = os.path.join(options['parts-directory'], 'buildout')
+        if os.path.exists(partsdir):
+            # This is primarily for unit tests, in which .py files change too
+            # fast for Python to know to regenerate the .pyc/.pyo files.
+            shutil.rmtree(partsdir)
+        os.mkdir(partsdir)
+        zc.buildout.easy_install.sitepackage_safe_scripts(
+            options['bin-directory'], ws, sys.executable, partsdir,
+            reqs=['zc.buildout'])
 
         # Restart
         args = map(zc.buildout.easy_install._safe_arg, sys.argv)
         if not __debug__:
             args.insert(0, '-O')
-        args.insert(0, zc.buildout.easy_install._safe_arg (sys.executable))
-
+        args.insert(0, zc.buildout.easy_install._safe_arg(sys.executable))
+        # We want to make sure that our new site.py is used for rerunning
+        # buildout, so we put the partsdir in PYTHONPATH for our restart.
+        # This overrides any set PYTHONPATH, but since we generally are
+        # trying to run with a completely "clean" python (only the standard
+        # library) then that should be fine.
+        env = os.environ.copy()
+        env['PYTHONPATH'] = partsdir
         if is_jython:
-            sys.exit(subprocess.Popen([sys.executable] + list(args)).wait())
+            sys.exit(
+                subprocess.Popen(
+                    [sys.executable] + list(args), env=env).wait())
         else:
-            sys.exit(os.spawnv(os.P_WAIT, sys.executable, args))
+            sys.exit(os.spawnve(os.P_WAIT, sys.executable, args, env))
 
     def _load_extensions(self):
         __doing__ = 'Loading extensions.'
@@ -884,7 +950,8 @@ class Buildout(UserDict.DictMixin):
                 working_set=pkg_resources.working_set,
                 links = self['buildout'].get('find-links', '').split(),
                 index = self['buildout'].get('index'),
-                newest=self.newest, allow_hosts=self._allow_hosts)
+                newest=self.newest, allow_hosts=self._allow_hosts,
+                include_site_packages=False)
 
             # Clear cache because extensions might now let us read pages we
             # couldn't read before.
@@ -1001,7 +1068,8 @@ def _install_and_load(spec, group, entry, buildout):
                 path=path,
                 working_set=pkg_resources.working_set,
                 newest=buildout.newest,
-                allow_hosts=buildout._allow_hosts
+                allow_hosts=buildout._allow_hosts,
+                include_site_packages=False,
                 )
 
         __doing__ = 'Loading %s recipe entry %s:%s.', group, spec, entry

src/zc/buildout/buildout.txt

@@ -56,10 +56,9 @@ The bin directory contains scripts.
     -  setuptools-0.6-py2.4.egg
     -  zc.buildout-1.0-py2.4.egg
 
-The develop-eggs and parts directories are initially empty:
+The develop-eggs directory is initially empty:
 
     >>> ls(sample_buildout, 'develop-eggs')
-    >>> ls(sample_buildout, 'parts')
 
 The develop-eggs directory holds egg links for software being
 developed in the buildout.  We separate develop-eggs and other eggs to
@@ -69,6 +68,12 @@ directory in their home that all non-develop eggs are stored in.  This
 allows larger buildouts to be set up much more quickly and saves disk
 space.
 
+The parts directory just contains some helpers for the buildout script
+itself.
+
+    >>> ls(sample_buildout, 'parts')
+    d  buildout
+
 The parts directory provides an area where recipes can install
 part data.  For example, if we built a custom Python, we would
 install it in the part directory.  Part data is stored in a
@@ -576,7 +581,7 @@ When we rerun the buildout:
 .. Wait for the file to really disappear. My linux is weird.
 
     >>> wait_until("foo goes away", lambda : not os.path.exists('foo'),
-    ...            timeout=100)
+    ...            timeout=200)
 
 we get the same error, but we don't get the directory left behind:
 
@@ -724,6 +729,10 @@ COMMAND_LINE_VALUE).
     ==================
     <BLANKLINE>
     [buildout]
+    allow-hosts= *
+        DEFAULT_VALUE
+    allow-picked-versions= true
+        DEFAULT_VALUE
     bin-directory= bin
         DEFAULT_VALUE
     develop= recipes
@@ -736,18 +745,36 @@ COMMAND_LINE_VALUE).
         DEFAULT_VALUE
     executable= ...
         DEFAULT_VALUE
+    find-links=
+        DEFAULT_VALUE
+    install-from-cache= false
+        DEFAULT_VALUE
     installed= .installed.cfg
         DEFAULT_VALUE
     log-format=
         DEFAULT_VALUE
     log-level= INFO
         DEFAULT_VALUE
+    newest= true
+        DEFAULT_VALUE
+    offline= false
+        DEFAULT_VALUE
     parts= data-dir
         /sample-buildout/buildout.cfg
     parts-directory= parts
         DEFAULT_VALUE
+    prefer-final= false
+        DEFAULT_VALUE
     python= buildout
         DEFAULT_VALUE
+    relative-paths= false
+        DEFAULT_VALUE
+    socket-timeout=
+        DEFAULT_VALUE
+    unzip= false
+        DEFAULT_VALUE
+    use-dependency-links= true
+        DEFAULT_VALUE
     <BLANKLINE>
     [data-dir]
     path= foo bins
@@ -2194,17 +2221,21 @@ database is shown.
 
     >>> print system(buildout+' -vv'), # doctest: +NORMALIZE_WHITESPACE
     Installing 'zc.buildout', 'setuptools'.
-    We have a develop egg: zc.buildout 1.0.0.
+    We have a develop egg: zc.buildout X.X.
     We have the best distribution that satisfies 'setuptools'.
-    Picked: setuptools = 0.6
+    Picked: setuptools = V.V
     <BLANKLINE>
     Configuration data:
     [buildout]
+    allow-hosts = *
+    allow-picked-versions = true
     bin-directory = /sample-buildout/bin
     develop-eggs-directory = /sample-buildout/develop-eggs
     directory = /sample-buildout
     eggs-directory = /sample-buildout/eggs
-    executable = /usr/local/bin/python2.3
+    executable = python
+    find-links =
+    install-from-cache = false
     installed = /sample-buildout/.installed.cfg
     log-format =
     log-level = INFO
@@ -2212,7 +2243,12 @@ database is shown.
     offline = false
     parts =
     parts-directory = /sample-buildout/parts
+    prefer-final = false
     python = buildout
+    relative-paths = false
+    socket-timeout =
+    unzip = false
+    use-dependency-links = true
     verbosity = 20
     <BLANKLINE>
 
@@ -2273,6 +2309,33 @@ python
    Python executable.  By default, the buildout section defines the
    default Python as the Python used to run the buildout.
 
+relative-paths
+    The paths generated by zc.buildout are absolute by default, and this
+    option is ``false``.  However, if you set this value to be ``true``,
+    bin/buildout will be generated with code that makes the paths relative.
+    Some recipes, such as zc.recipe.egg and z3c.recipe.scripts, honor this
+    value as well.
+
+unzip
+    By default, zc.buildout doesn't unzip zip-safe eggs ("unzip = false").
+    This follows the policy followed by setuptools itself.  Experience shows
+    this policy to to be inconvenient.  Zipped eggs make debugging more
+    difficult and often import more slowly.  You can include an unzip option in
+    the buildout section to change the default unzipping policy ("unzip =
+    true").
+
+use-dependency-links
+    By default buildout will obey the setuptools dependency_links metadata
+    when it looks for dependencies. This behavior can be controlled with
+    the use-dependency-links buildout option::
+
+      [buildout]
+      ...
+      use-dependency-links = false
+
+    The option defaults to true. If you set it to false, then dependency
+    links are only looked for in the locations specified by find-links.
+
 verbosity
    A log-level adjustment.  Typically, this is set via the -q and -v
    command-line options.
@@ -2319,8 +2382,56 @@ buildout or setuptools egg could be installed in the develop-eggs
 directory if the original buildout had develop eggs for either
 buildout or setuptools.)
 
-Note that the buildout script was installed but not run.  To run
-the buildout, we'd have to run the installed buildout script.
+If relative-paths is ``true``, the buildout script uses relative paths.
+
+    >>> write(sample_bootstrapped, 'setup.cfg',
+    ... '''
+    ... [buildout]
+    ... relative-paths = true
+    ... parts =
+    ... ''')
+
+    >>> print system(buildout
+    ...              +' -c'+os.path.join(sample_bootstrapped, 'setup.cfg')
+    ...              +' bootstrap'),
+    Generated script '/sample-bootstrapped/bin/buildout'.
+
+    >>> buildout_script = join(sample_bootstrapped, 'bin', 'buildout')
+    >>> import sys
+    >>> if sys.platform.startswith('win'):
+    ...     buildout_script += '-script.py'
+    >>> print open(buildout_script).read() # doctest: +ELLIPSIS
+    #!... -S
+    <BLANKLINE>
+    import os
+    <BLANKLINE>
+    join = os.path.join
+    base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
+    base = os.path.dirname(base)
+    <BLANKLINE>
+    import sys
+    sys.path[0:0] = [
+        join(base, 'parts/buildout'),
+        ]
+    <BLANKLINE>
+    <BLANKLINE>
+    import os
+    path = sys.path[0]
+    if os.environ.get('PYTHONPATH'):
+        path = os.pathsep.join([path, os.environ['PYTHONPATH']])
+    os.environ['PYTHONPATH'] = path
+    import site # imports custom buildout-generated site.py
+    <BLANKLINE>
+    import zc.buildout.buildout
+    <BLANKLINE>
+    if __name__ == '__main__':
+        zc.buildout.buildout.main()
+    <BLANKLINE>
+
+
+Note that, in the above two examples, the buildout script was installed
+but not run.  To run the buildout, we'd have to run the installed
+buildout script.
 
 If we have an existing buildout that already has a buildout.cfg, we'll
 normally use the bootstrap command instead of init.  It will complain

src/zc/buildout/easy_install.py

@@ -19,6 +19,7 @@ installed.
 """
 
 import distutils.errors
+import fnmatch
 import glob
 import logging
 import os
@@ -50,6 +51,8 @@ url_match = re.compile('[a-z0-9+.-]+://').match
 
 is_win32 = sys.platform == 'win32'
 is_jython = sys.platform.startswith('java')
+is_distribute = (
+    pkg_resources.Requirement.parse('setuptools').key=='distribute')
 
 if is_jython:
     import java.lang.System
@@ -60,12 +63,75 @@ setuptools_loc = pkg_resources.working_set.find(
     pkg_resources.Requirement.parse('setuptools')
     ).location
 
-# Include buildout and setuptools eggs in paths
-buildout_and_setuptools_path = [
-    setuptools_loc,
-    pkg_resources.working_set.find(
-        pkg_resources.Requirement.parse('zc.buildout')).location,
-    ]
+# Include buildout and setuptools eggs in paths.  We prevent dupes just to
+# keep from duplicating any log messages about them.
+buildout_loc = pkg_resources.working_set.find(
+    pkg_resources.Requirement.parse('zc.buildout')).location
+buildout_and_setuptools_path = [setuptools_loc]
+if os.path.normpath(setuptools_loc) != os.path.normpath(buildout_loc):
+    buildout_and_setuptools_path.append(buildout_loc)
+
+def _get_system_paths(executable):
+    """Return lists of standard lib and site paths for executable.
+    """
+    # We want to get a list of the site packages, which is not easy.
+    # The canonical way to do this is to use
+    # distutils.sysconfig.get_python_lib(), but that only returns a
+    # single path, which does not reflect reality for many system
+    # Pythons, which have multiple additions.  Instead, we start Python
+    # with -S, which does not import site.py and set up the extra paths
+    # like site-packages or (Ubuntu/Debian) dist-packages and
+    # python-support. We then compare that sys.path with the normal one
+    # (minus user packages if this is Python 2.6, because we don't
+    # support those (yet?).  The set of the normal one minus the set of
+    # the ones in ``python -S`` is the set of packages that are
+    # effectively site-packages.
+    #
+    # The given executable might not be the current executable, so it is
+    # appropriate to do another subprocess to figure out what the
+    # additional site-package paths are. Moreover, even if this
+    # executable *is* the current executable, this code might be run in
+    # the context of code that has manipulated the sys.path--for
+    # instance, to add local zc.buildout or setuptools eggs.
+    def get_sys_path(*args, **kwargs):
+        cmd = [executable]
+        cmd.extend(args)
+        cmd.extend([
+            "-c", "import sys, os;"
+            "print repr([os.path.normpath(p) for p in sys.path if p])"])
+        # Windows needs some (as yet to be determined) part of the real env.
+        env = os.environ.copy()
+        # We need to make sure that PYTHONPATH, which will often be set
+        # to include a custom buildout-generated site.py, is not set, or
+        # else we will not get an accurate sys.path for the executable.
+        env.pop('PYTHONPATH', None)
+        env.update(kwargs)
+        _proc = subprocess.Popen(
+            cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)
+        stdout, stderr = _proc.communicate();
+        if _proc.returncode:
+            raise RuntimeError(
+                'error trying to get system packages:\n%s' % (stderr,))
+        res = eval(stdout.strip())
+        try:
+            res.remove('.')
+        except ValueError:
+            pass
+        return res
+    stdlib = get_sys_path('-S') # stdlib only
+    no_user_paths = get_sys_path(PYTHONNOUSERSITE='x')
+    site_paths = [p for p in no_user_paths if p not in stdlib]
+    return (stdlib, site_paths)
+
+def _get_version_info(executable):
+    cmd = [executable, '-Sc', 'import sys; print repr(sys.version_info)']
+    _proc = subprocess.Popen(
+        cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    stdout, stderr = _proc.communicate();
+    if _proc.returncode:
+        raise RuntimeError(
+            'error trying to get system packages:\n%s' % (stderr,))
+    return eval(stdout.strip())
 
 
 class IncompatibleVersionError(zc.buildout.UserError):
@@ -109,7 +175,12 @@ class AllowHostsPackageIndex(setuptools.package_index.PackageIndex):
 
 
 _indexes = {}
-def _get_index(executable, index_url, find_links, allow_hosts=('*',)):
+def _get_index(executable, index_url, find_links, allow_hosts=('*',),
+               path=None):
+    # If path is None, the index will use sys.path.  If you provide an empty
+    # path ([]), it will complain uselessly about missing index pages for
+    # packages found in the paths that you expect to use.  Therefore, this path
+    # is always the same as the _env path in the Installer.
     key = executable, index_url, tuple(find_links)
     index = _indexes.get(key)
     if index is not None:
@@ -118,7 +189,8 @@ def _get_index(executable, index_url, find_links, allow_hosts=('*',)):
     if index_url is None:
         index_url = default_index_url
     index = AllowHostsPackageIndex(
-        index_url, hosts=allow_hosts, python=_get_version(executable)
+        index_url, hosts=allow_hosts, search_path=path,
+        python=_get_version(executable)
         )
 
     if find_links:
@@ -131,15 +203,79 @@ clear_index_cache = _indexes.clear
 
 if is_win32:
     # work around spawn lamosity on windows
-    # XXX need safe quoting (see the subproces.list2cmdline) and test
+    # XXX need safe quoting (see the subprocess.list2cmdline) and test
     def _safe_arg(arg):
         return '"%s"' % arg
 else:
     _safe_arg = str
 
-_easy_install_cmd = _safe_arg(
-    'from setuptools.command.easy_install import main; main()'
-    )
+# The following string is used to run easy_install in
+# Installer._call_easy_install.  It is started with python -S (that is,
+# don't import site at start).  That flag, and all of the code in this
+# snippet above the last two lines, exist to work around a relatively rare
+# problem.  If
+#
+# - your buildout configuration is trying to install a package that is within
+#   a namespace package, and
+#
+# - you use a Python that has a different version of this package
+#   installed in in its site-packages using
+#   --single-version-externally-managed (that is, using the mechanism
+#   sometimes used by system packagers:
+#   http://peak.telecommunity.com/DevCenter/setuptools#install-command ), and
+#
+# - the new package tries to do sys.path tricks in the setup.py to get a
+#   __version__,
+#
+# then the older package will be loaded first, making the setup version
+# the wrong number. While very arguably packages simply shouldn't do
+# the sys.path tricks, some do, and we don't want buildout to fall over
+# when they do.
+#
+# The namespace packages installed in site-packages with
+# --single-version-externally-managed use a mechanism that cause them to
+# be processed when site.py is imported  (see
+# http://mail.python.org/pipermail/distutils-sig/2009-May/011730.html
+# for another description of the problem).  Simply starting Python with
+# -S addresses the problem in Python 2.4 and 2.5, but Python 2.6's
+# distutils imports a value from the site module, so we unfortunately
+# have to do more drastic surgery in the _easy_install_cmd code below.
+#
+# Here's an example of the .pth files created by setuptools when using that
+# flag:
+#
+# import sys,new,os;
+# p = os.path.join(sys._getframe(1).f_locals['sitedir'], *('<NAMESPACE>',));
+# ie = os.path.exists(os.path.join(p,'__init__.py'));
+# m = not ie and sys.modules.setdefault('<NAMESPACE>',new.module('<NAMESPACE>'));
+# mp = (m or []) and m.__dict__.setdefault('__path__',[]);
+# (p not in mp) and mp.append(p)
+#
+# The code, below, then, runs under -S, indicating that site.py should
+# not be loaded initially.  It gets the initial sys.path under these
+# circumstances, and then imports site (because Python 2.6's distutils
+# will want it, as mentioned above). It then reinstates the old sys.path
+# value. Then it removes namespace packages (created by the setuptools
+# code above) from sys.modules.  It identifies namespace packages by
+# iterating over every loaded module.  It first looks if there is a
+# __path__, so it is a package; and then it sees if that __path__ does
+# not have an __init__.py.  (Note that PEP 382,
+# http://www.python.org/dev/peps/pep-0382, makes it possible to have a
+# namespace package that has an __init__.py, but also should make it
+# unnecessary for site.py to preprocess these packages, so it should be
+# fine, as far as can be guessed as of this writing.)  Finally, it
+# imports easy_install and runs it.
+
+_easy_install_cmd = _safe_arg('''\
+import sys,os;\
+p = sys.path[:];\
+import site;\
+sys.path[:] = p;\
+[sys.modules.pop(k) for k, v in sys.modules.items()\
+ if hasattr(v, '__path__') and len(v.__path__)==1 and\
+ not os.path.exists(os.path.join(v.__path__[0],'__init__.py'))];\
+from setuptools.command.easy_install import main;\
+main()''')
 
 
 class Installer:
@@ -151,6 +287,8 @@ class Installer:
     _use_dependency_links = True
     _allow_picked_versions = True
     _always_unzip = False
+    _include_site_packages = True
+    _allowed_eggs_from_site_packages = ('*',)
 
     def __init__(self,
                  dest=None,
@@ -162,7 +300,9 @@ class Installer:
                  newest=True,
                  versions=None,
                  use_dependency_links=None,
-                 allow_hosts=('*',)
+                 allow_hosts=('*',),
+                 include_site_packages=None,
+                 allowed_eggs_from_site_packages=None
                  ):
         self._dest = dest
         self._allow_hosts = allow_hosts
@@ -184,7 +324,25 @@ class Installer:
         self._executable = executable
         if always_unzip is not None:
             self._always_unzip = always_unzip
-        path = (path and path[:] or []) + buildout_and_setuptools_path
+        path = (path and path[:] or [])
+        if include_site_packages is not None:
+            self._include_site_packages = include_site_packages
+        if allowed_eggs_from_site_packages is not None:
+            self._allowed_eggs_from_site_packages = tuple(
+                allowed_eggs_from_site_packages)
+        stdlib, self._site_packages = _get_system_paths(executable)
+        version_info = _get_version_info(executable)
+        if version_info == sys.version_info:
+            # Maybe we can add the buildout and setuptools path.  If we
+            # are including site_packages, we only have to include the extra
+            # bits here, so we don't duplicate.  On the other hand, if we
+            # are not including site_packages, we only want to include the
+            # parts that are not in site_packages, so the code is the same.
+            path.extend(
+                set(buildout_and_setuptools_path).difference(
+                    self._site_packages))
+        if self._include_site_packages:
+            path.extend(self._site_packages)
         if dest is not None and dest not in path:
             path.insert(0, dest)
         self._path = path
@@ -193,13 +351,42 @@ class Installer:
         self._newest = newest
         self._env = pkg_resources.Environment(path,
                                               python=_get_version(executable))
-        self._index = _get_index(executable, index, links, self._allow_hosts)
+        self._index = _get_index(executable, index, links, self._allow_hosts,
+                                 self._path)
 
         if versions is not None:
             self._versions = versions
 
+    _allowed_eggs_from_site_packages_regex = None
+    def allow_site_package_egg(self, name):
+        if (not self._include_site_packages or
+            not self._allowed_eggs_from_site_packages):
+            # If the answer is a blanket "no," perform a shortcut.
+            return False
+        if self._allowed_eggs_from_site_packages_regex is None:
+            pattern = '(%s)' % (
+                '|'.join(
+                    fnmatch.translate(name)
+                    for name in self._allowed_eggs_from_site_packages),
+                )
+            self._allowed_eggs_from_site_packages_regex = re.compile(pattern)
+        return bool(self._allowed_eggs_from_site_packages_regex.match(name))
+
     def _satisfied(self, req, source=None):
-        dists = [dist for dist in self._env[req.project_name] if dist in req]
+        # We get all distributions that match the given requirement.  If we are
+        # not supposed to include site-packages for the given egg, we also
+        # filter those out. Even if include_site_packages is False and so we
+        # have excluded site packages from the _env's paths (see
+        # Installer.__init__), we need to do the filtering here because an
+        # .egg-link, such as one for setuptools or zc.buildout installed by
+        # zc.buildout.buildout.Buildout.bootstrap, can indirectly include a
+        # path in our _site_packages.
+        dists = [dist for dist in self._env[req.project_name] if (
+                    dist in req and (
+                        dist.location not in self._site_packages or
+                        self.allow_site_package_egg(dist.project_name))
+                    )
+                ]
         if not dists:
             logger.debug('We have no distributions for %s that satisfies %r.',
                          req.project_name, str(req))
@@ -301,7 +488,7 @@ class Installer:
         try:
             path = setuptools_loc
 
-            args = ('-c', _easy_install_cmd, '-mUNxd', _safe_arg(tmp))
+            args = ('-Sc', _easy_install_cmd, '-mUNxd', _safe_arg(tmp))
             if self._always_unzip:
                 args += ('-Z', )
             level = logger.getEffectiveLevel()
@@ -400,14 +587,22 @@ class Installer:
             # Nothing is available.
             return None
 
-        # Filter the available dists for the requirement and source flag
-        dists = [dist for dist in index[requirement.project_name]
-                 if ((dist in requirement)
-                     and
-                     ((not source) or
-                      (dist.precedence == pkg_resources.SOURCE_DIST)
-                      )
-                     )
+        # Filter the available dists for the requirement and source flag.  If
+        # we are not supposed to include site-packages for the given egg, we
+        # also filter those out. Even if include_site_packages is False and so
+        # we have excluded site packages from the _env's paths (see
+        # Installer.__init__), we need to do the filtering here because an
+        # .egg-link, such as one for setuptools or zc.buildout installed by
+        # zc.buildout.buildout.Buildout.bootstrap, can indirectly include a
+        # path in our _site_packages.
+        dists = [dist for dist in index[requirement.project_name] if (
+                    dist in requirement and (
+                        dist.location not in self._site_packages or
+                        self.allow_site_package_egg(dist.project_name))
+                    and (
+                        (not source) or
+                        (dist.precedence == pkg_resources.SOURCE_DIST))
+                    )
                  ]
 
         # If we prefer final dists, filter for final and use the
@@ -567,7 +762,7 @@ class Installer:
                         self._links.append(link)
                         self._index = _get_index(self._executable,
                                                  self._index_url, self._links,
-                                                 self._allow_hosts)
+                                                 self._allow_hosts, self._path)
 
         for dist in dists:
             # Check whether we picked a version and, if we did, report it:
@@ -609,6 +804,8 @@ class Installer:
 
 
     def _constrain(self, requirement):
+        if is_distribute and requirement.key == 'setuptools':
+            requirement = pkg_resources.Requirement.parse('distribute')
         version = self._versions.get(requirement.project_name)
         if version:
             if version not in requirement:
@@ -648,35 +845,52 @@ class Installer:
                 self._maybe_add_setuptools(ws, dist)
 
         # OK, we have the requested distributions and they're in the working
-        # set, but they may have unmet requirements.  We'll simply keep
-        # trying to resolve requirements, adding missing requirements as they
-        # are reported.
-        #
-        # Note that we don't pass in the environment, because we want
+        # set, but they may have unmet requirements.  We'll resolve these
+        # requirements. This is code modified from
+        # pkg_resources.WorkingSet.resolve.  We can't reuse that code directly
+        # because we have to constrain our requirements (see
+        # versions_section_ignored_for_dependency_in_favor_of_site_packages in
+        # zc.buildout.tests).
+        requirements.reverse() # Set up the stack.
+        processed = {}  # This is a set of processed requirements.
+        best = {}  # This is a mapping of key -> dist.
+        # Note that we don't use the existing environment, because we want
         # to look for new eggs unless what we have is the best that
         # matches the requirement.
-        while 1:
-            try:
-                ws.resolve(requirements)
-            except pkg_resources.DistributionNotFound, err:
-                [requirement] = err
-                requirement = self._constrain(requirement)
-                if destination:
-                    logger.debug('Getting required %r', str(requirement))
-                else:
-                    logger.debug('Adding required %r', str(requirement))
-                _log_requirement(ws, requirement)
-
-                for dist in self._get_dist(requirement, ws, self._always_unzip
-                                           ):
-
-                    ws.add(dist)
-                    self._maybe_add_setuptools(ws, dist)
-            except pkg_resources.VersionConflict, err:
-                raise VersionConflict(err, ws)
-            else:
-                break
-
+        env = pkg_resources.Environment(ws.entries)
+        while requirements:
+            # Process dependencies breadth-first.
+            req = self._constrain(requirements.pop(0))
+            if req in processed:
+                # Ignore cyclic or redundant dependencies.
+                continue
+            dist = best.get(req.key)
+            if dist is None:
+                # Find the best distribution and add it to the map.
+                dist = ws.by_key.get(req.key)
+                if dist is None:
+                    try:
+                        dist = best[req.key] = env.best_match(req, ws)
+                    except pkg_resources.VersionConflict, err:
+                        raise VersionConflict(err, ws)
+                    if dist is None:
+                        if destination:
+                            logger.debug('Getting required %r', str(req))
+                        else:
+                            logger.debug('Adding required %r', str(req))
+                        _log_requirement(ws, req)
+                        for dist in self._get_dist(req,
+                                                   ws, self._always_unzip):
+                            ws.add(dist)
+                            self._maybe_add_setuptools(ws, dist)
+            if dist not in req:
+                # Oops, the "best" so far conflicts with a dependency.
+                raise VersionConflict(
+                    pkg_resources.VersionConflict(dist, req), ws)
+            requirements.extend(dist.requires(req.extras)[::-1])
+            processed[req] = True
+            if dist.location in self._site_packages:
+                logger.debug('Egg from site-packages: %s', dist)
         return ws
 
     def build(self, spec, build_ext):
@@ -771,6 +985,18 @@ def prefer_final(setting=None):
         Installer._prefer_final = bool(setting)
     return old
 
+def include_site_packages(setting=None):
+    old = Installer._include_site_packages
+    if setting is not None:
+        Installer._include_site_packages = bool(setting)
+    return old
+
+def allowed_eggs_from_site_packages(setting=None):
+    old = Installer._allowed_eggs_from_site_packages
+    if setting is not None:
+        Installer._allowed_eggs_from_site_packages = tuple(setting)
+    return old
+
 def use_dependency_links(setting=None):
     old = Installer._use_dependency_links
     if setting is not None:
@@ -793,19 +1019,27 @@ def install(specs, dest,
             links=(), index=None,
             executable=sys.executable, always_unzip=None,
             path=None, working_set=None, newest=True, versions=None,
-            use_dependency_links=None, allow_hosts=('*',)):
+            use_dependency_links=None, allow_hosts=('*',),
+            include_site_packages=None, allowed_eggs_from_site_packages=None):
     installer = Installer(dest, links, index, executable, always_unzip, path,
                           newest, versions, use_dependency_links,
-                          allow_hosts=allow_hosts)
+                          allow_hosts=allow_hosts,
+                          include_site_packages=include_site_packages,
+                          allowed_eggs_from_site_packages=
+                            allowed_eggs_from_site_packages)
     return installer.install(specs, working_set)
 
 
 def build(spec, dest, build_ext,
           links=(), index=None,
           executable=sys.executable,
-          path=None, newest=True, versions=None, allow_hosts=('*',)):
+          path=None, newest=True, versions=None, allow_hosts=('*',),
+          include_site_packages=None, allowed_eggs_from_site_packages=None):
     installer = Installer(dest, links, index, executable, True, path, newest,
-                          versions, allow_hosts=allow_hosts)
+                          versions, allow_hosts=allow_hosts,
+                          include_site_packages=include_site_packages,
+                          allowed_eggs_from_site_packages=
+                            allowed_eggs_from_site_packages)
     return installer.build(spec, build_ext)
 
 
@@ -900,9 +1134,15 @@ def develop(setup, dest,
         undo.reverse()
         [f() for f in undo]
 
+def working_set(specs, executable, path, include_site_packages=None,
+                allowed_eggs_from_site_packages=None):
+    return install(
+        specs, None, executable=executable, path=path,
+        include_site_packages=include_site_packages,
+        allowed_eggs_from_site_packages=allowed_eggs_from_site_packages)
 
-def working_set(specs, executable, path):
-    return install(specs, None, executable=executable, path=path)
+############################################################################
+# Script generation functions
 
 def scripts(reqs, working_set, executable, dest,
             scripts=None,
@@ -912,20 +1152,95 @@ def scripts(reqs, working_set, executable, dest,
             initialization='',
             relative_paths=False,
             ):
+    """Generate scripts and/or an interpreter.
 
-    path = [dist.location for dist in working_set]
-    path.extend(extra_paths)
-    path = map(realpath, path)
+    See sitepackage_safe_scripts for a version that can be used with a Python
+    that has code installed in site-packages. It has more options and a
+    different approach.
+    """
+    path = _get_path(working_set, extra_paths)
+    if initialization:
+        initialization = '\n'+initialization+'\n'
+    generated = _generate_scripts(
+        reqs, working_set, dest, path, scripts, relative_paths,
+        initialization, executable, arguments)
+    if interpreter:
+        sname = os.path.join(dest, interpreter)
+        spath, rpsetup = _relative_path_and_setup(sname, path, relative_paths)
+        generated.extend(
+            _pyscript(spath, sname, executable, rpsetup))
+    return generated
 
+def sitepackage_safe_scripts(
+    dest, working_set, executable, site_py_dest,
+    reqs=(), scripts=None, interpreter=None, extra_paths=(),
+    initialization='', include_site_packages=False, exec_sitecustomize=False,
+    relative_paths=False, script_arguments='', script_initialization=''):
+    """Generate scripts and/or an interpreter from a system Python.
+
+    This accomplishes the same job as the ``scripts`` function, above,
+    but it does so in an alternative way that allows safely including
+    Python site packages, if desired, and  choosing to execute the Python's
+    sitecustomize.
+    """
     generated = []
+    generated.append(_generate_sitecustomize(
+        site_py_dest, executable, initialization, exec_sitecustomize))
+    generated.append(_generate_site(
+        site_py_dest, working_set, executable, extra_paths,
+        include_site_packages, relative_paths))
+    script_initialization = _script_initialization_template % dict(
+        site_py_dest=site_py_dest,
+        script_initialization=script_initialization)
+    if not script_initialization.endswith('\n'):
+        script_initialization += '\n'
+    generated.extend(_generate_scripts(
+        reqs, working_set, dest, [site_py_dest], scripts, relative_paths,
+        script_initialization, executable, script_arguments, block_site=True))
+    if interpreter:
+        generated.extend(_generate_interpreter(
+            interpreter, dest, executable, site_py_dest, relative_paths))
+    return generated
 
+_script_initialization_template = '''
+import os
+path = sys.path[0]
+if os.environ.get('PYTHONPATH'):
+    path = os.pathsep.join([path, os.environ['PYTHONPATH']])
+os.environ['PYTHONPATH'] = path
+import site # imports custom buildout-generated site.py
+%(script_initialization)s'''
+
+# Utilities for the script generation functions.
+
+# These are shared by both ``scripts`` and ``sitepackage_safe_scripts``
+
+def _get_path(working_set, extra_paths=()):
+    """Given working set and extra paths, return a normalized path list."""
+    path = [dist.location for dist in working_set]
+    path.extend(extra_paths)
+    return map(realpath, path)
+
+def _generate_scripts(reqs, working_set, dest, path, scripts, relative_paths,
+                      initialization, executable, arguments,
+                      block_site=False):
+    """Generate scripts for the given requirements.
+
+    - reqs is an iterable of string requirements or entry points.
+    - The requirements must be findable in the given working_set.
+    - The dest is the directory in which the scripts should be created.
+    - The path is a list of paths that should be added to sys.path.
+    - The scripts is an optional dictionary.  If included, the keys should be
+      the names of the scripts that should be created, as identified in their
+      entry points; and the values should be the name the script should
+      actually be created with.
+    - relative_paths, if given, should be the path that is the root of the
+      buildout (the common path that should be the root of what is relative).
+    """
     if isinstance(reqs, str):
         raise TypeError('Expected iterable of requirements or entry points,'
                         ' got string.')
-
-    if initialization:
-        initialization = '\n'+initialization+'\n'
-
+    generated = []
     entry_points = []
     for req in reqs:
         if isinstance(req, str):
@@ -939,7 +1254,6 @@ def scripts(reqs, working_set, executable, dest,
                     )
         else:
             entry_points.append(req)
-
     for name, module_name, attrs in entry_points:
         if scripts is not None:
             sname = scripts.get(name)
@@ -947,40 +1261,51 @@ def scripts(reqs, working_set, executable, dest,
                 continue
         else:
             sname = name
-
         sname = os.path.join(dest, sname)
         spath, rpsetup = _relative_path_and_setup(sname, path, relative_paths)
-
         generated.extend(
-            _script(module_name, attrs, spath, sname, executable, arguments,
-                    initialization, rpsetup)
-            )
-
-    if interpreter:
-        sname = os.path.join(dest, interpreter)
-        spath, rpsetup = _relative_path_and_setup(sname, path, relative_paths)
-        generated.extend(_pyscript(spath, sname, executable, rpsetup))
-
+            _script(sname, executable, rpsetup, spath, initialization,
+                    module_name, attrs, arguments, block_site=block_site))
     return generated
 
-def _relative_path_and_setup(sname, path, relative_paths):
+def _relative_path_and_setup(sname, path,
+                             relative_paths=False, indent_level=1,
+                             omit_os_import=False):
+    """Return a string of code of paths and of setup if appropriate.
+
+    - sname is the full path to the script name to be created.
+    - path is the list of paths to be added to sys.path.
+    - relative_paths, if given, should be the path that is the root of the
+      buildout (the common path that should be the root of what is relative).
+    - indent_level is the number of four-space indents that the path should
+      insert before each element of the path.
+    """
     if relative_paths:
         relative_paths = os.path.normcase(relative_paths)
         sname = os.path.normcase(os.path.abspath(sname))
-        spath = ',\n  '.join(
+        spath = _format_paths(
             [_relativitize(os.path.normcase(path_item), sname, relative_paths)
-             for path_item in path]
-            )
+             for path_item in path], indent_level=indent_level)
         rpsetup = relative_paths_setup
+        if not omit_os_import:
+            rpsetup = '\n\nimport os\n' + rpsetup
         for i in range(_relative_depth(relative_paths, sname)):
-            rpsetup += "base = os.path.dirname(base)\n"
+            rpsetup += "\nbase = os.path.dirname(base)"
     else:
-        spath = repr(path)[1:-1].replace(', ', ',\n  ')
+        spath = _format_paths((repr(p) for p in path),
+                              indent_level=indent_level)
         rpsetup = ''
     return spath, rpsetup
 
-
 def _relative_depth(common, path):
+    """Return number of dirs separating ``path`` from ancestor, ``common``.
+
+    For instance, if path is /foo/bar/baz/bing, and common is /foo, this will
+    return 2--in UNIX, the number of ".." to get from bing's directory
+    to foo.
+
+    This is a helper for _relative_path_and_setup.
+    """
     n = 0
     while 1:
         dirname = os.path.dirname(path)
@@ -993,6 +1318,11 @@ def _relative_depth(common, path):
     return n
 
 def _relative_path(common, path):
+    """Return the relative path from ``common`` to ``path``.
+
+    This is a helper for _relativitize, which is a helper to
+    _relative_path_and_setup.
+    """
     r = []
     while 1:
         dirname, basename = os.path.split(path)
@@ -1006,6 +1336,11 @@ def _relative_path(common, path):
     return os.path.join(*r)
 
 def _relativitize(path, script, relative_paths):
+    """Return a code string for the given path.
+
+    Path is relative to the base path ``relative_paths``if the common prefix
+    between ``path`` and ``script`` starts with ``relative_paths``.
+    """
     if path == script:
         raise AssertionError("path == script")
     common = os.path.dirname(os.path.commonprefix([path, script]))
@@ -1016,66 +1351,82 @@ def _relativitize(path, script, relative_paths):
     else:
         return repr(path)
 
-
 relative_paths_setup = """
-import os
-
 join = os.path.join
-base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
-"""
+base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))"""
 
-def _script(module_name, attrs, path, dest, executable, arguments,
-            initialization, rsetup):
-    generated = []
-    script = dest
-    if is_win32:
-        dest += '-script.py'
-
-    contents = script_template % dict(
-        python = _safe_arg(executable),
-        path = path,
-        module_name = module_name,
-        attrs = attrs,
-        arguments = arguments,
-        initialization = initialization,
-        relative_paths_setup = rsetup,
-        )
-    changed = not (os.path.exists(dest) and open(dest).read() == contents)
+def _write_script(full_name, contents, logged_type):
+    """Write contents of script in full_name, logging the action.
 
+    The only tricky bit in this function is that it supports Windows by
+    creating exe files using a pkg_resources helper.
+    """
+    generated = []
+    script_name = full_name
     if is_win32:
-        # generate exe file and give the script a magic name:
-        exe = script+'.exe'
+        script_name += '-script.py'
+        # Generate exe file and give the script a magic name.
+        exe = full_name + '.exe'
         new_data = pkg_resources.resource_string('setuptools', 'cli.exe')
         if not os.path.exists(exe) or (open(exe, 'rb').read() != new_data):
             # Only write it if it's different.
             open(exe, 'wb').write(new_data)
         generated.append(exe)
-
+    changed = not (os.path.exists(script_name) and
+                   open(script_name).read() == contents)
     if changed:
-        open(dest, 'w').write(contents)
-        logger.info("Generated script %r.", script)
-
+        open(script_name, 'w').write(contents)
         try:
-            os.chmod(dest, 0755)
+            os.chmod(script_name, 0755)
         except (AttributeError, os.error):
             pass
-
-    generated.append(dest)
+        logger.info("Generated %s %r.", logged_type, full_name)
+    generated.append(script_name)
     return generated
 
+def _format_paths(paths, indent_level=1):
+    """Format paths for inclusion in a script."""
+    separator = ',\n' + indent_level * '    '
+    return separator.join(paths)
+
+def _script(dest, executable, relative_paths_setup, path, initialization,
+            module_name, attrs, arguments, block_site=False):
+    if block_site:
+        dash_S = ' -S'
+    else:
+        dash_S = ''
+    contents = script_template % dict(
+        python=_safe_arg(executable),
+        dash_S=dash_S,
+        path=path,
+        module_name=module_name,
+        attrs=attrs,
+        arguments=arguments,
+        initialization=initialization,
+        relative_paths_setup=relative_paths_setup,
+        )
+    return _write_script(dest, contents, 'script')
+
 if is_jython and jython_os_name == 'linux':
-    script_header = '#!/usr/bin/env %(python)s'
+    script_header = '#!/usr/bin/env %(python)s%(dash_S)s'
 else:
-    script_header = '#!%(python)s'
+    script_header = '#!%(python)s%(dash_S)s'
 
+sys_path_template = '''\
+import sys
+sys.path[0:0] = [
+    %s,
+    ]
+'''
 
 script_template = script_header + '''\
-
 %(relative_paths_setup)s
+
 import sys
 sys.path[0:0] = [
-  %(path)s,
-  ]
+    %(path)s,
+    ]
+
 %(initialization)s
 import %(module_name)s
 
@@ -1083,47 +1434,25 @@ if __name__ == '__main__':
     %(module_name)s.%(attrs)s(%(arguments)s)
 '''
 
+# These are used only by the older ``scripts`` function.
 
 def _pyscript(path, dest, executable, rsetup):
-    generated = []
-    script = dest
-    if is_win32:
-        dest += '-script.py'
-
     contents = py_script_template % dict(
-        python = _safe_arg(executable),
-        path = path,
-        relative_paths_setup = rsetup,
+        python=_safe_arg(executable),
+        dash_S='',
+        path=path,
+        relative_paths_setup=rsetup,
         )
-    changed = not (os.path.exists(dest) and open(dest).read() == contents)
-
-    if is_win32:
-        # generate exe file and give the script a magic name:
-        exe = script + '.exe'
-        open(exe, 'wb').write(
-            pkg_resources.resource_string('setuptools', 'cli.exe')
-            )
-        generated.append(exe)
-
-    if changed:
-        open(dest, 'w').write(contents)
-        try:
-            os.chmod(dest,0755)
-        except (AttributeError, os.error):
-            pass
-        logger.info("Generated interpreter %r.", script)
-
-    generated.append(dest)
-    return generated
+    return _write_script(dest, contents, 'interpreter')
 
 py_script_template = script_header + '''\
-
 %(relative_paths_setup)s
+
 import sys
 
 sys.path[0:0] = [
-  %(path)s,
-  ]
+    %(path)s,
+    ]
 
 _interactive = True
 if len(sys.argv) > 1:
@@ -1151,6 +1480,235 @@ if _interactive:
     __import__("code").interact(banner="", local=globals())
 '''
 
+# These are used only by the newer ``sitepackage_safe_scripts`` function.
+
+def _get_module_file(executable, name):
+    """Return a module's file path.
+
+    - executable is a path to the desired Python executable.
+    - name is the name of the (pure, not C) Python module.
+    """
+    cmd = [executable, "-Sc",
+           "import imp; "
+           "fp, path, desc = imp.find_module(%r); "
+           "fp.close; "
+           "print path" % (name,)]
+    env = os.environ.copy()
+    # We need to make sure that PYTHONPATH, which will often be set to
+    # include a custom buildout-generated site.py, is not set, or else
+    # we will not get an accurate value for the "real" site.py and
+    # sitecustomize.py.
+    env.pop('PYTHONPATH', None)
+    _proc = subprocess.Popen(
+        cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)
+    stdout, stderr = _proc.communicate();
+    if _proc.returncode:
+        logger.info(
+            'Could not find file for module %s:\n%s', name, stderr)
+        return None
+    # else: ...
+    res = stdout.strip()
+    if res.endswith('.pyc') or res.endswith('.pyo'):
+        raise RuntimeError('Cannot find uncompiled version of %s' % (name,))
+    if not os.path.exists(res):
+        raise RuntimeError(
+            'File does not exist for module %s:\n%s' % (name, res))
+    return res
+
+def _generate_sitecustomize(dest, executable, initialization='',
+                            exec_sitecustomize=False):
+    """Write a sitecustomize file with optional custom initialization.
+
+    The created script will execute the underlying Python's
+    sitecustomize if exec_sitecustomize is True.
+    """
+    sitecustomize_path = os.path.join(dest, 'sitecustomize.py')
+    sitecustomize = open(sitecustomize_path, 'w')
+    if initialization:
+        sitecustomize.write(initialization + '\n')
+    if exec_sitecustomize:
+        real_sitecustomize_path = _get_module_file(
+            executable, 'sitecustomize')
+        if real_sitecustomize_path:
+            real_sitecustomize = open(real_sitecustomize_path, 'r')
+            sitecustomize.write(
+                '\n# The following is from\n# %s\n' %
+                (real_sitecustomize_path,))
+            sitecustomize.write(real_sitecustomize.read())
+            real_sitecustomize.close()
+    sitecustomize.close()
+    return sitecustomize_path
+
+def _generate_site(dest, working_set, executable, extra_paths=(),
+                   include_site_packages=False, relative_paths=False):
+    """Write a site.py file with eggs from working_set.
+
+    extra_paths will be added to the path.  If include_site_packages is True,
+    paths from the underlying Python will be added.
+    """
+    path = _get_path(working_set, extra_paths)
+    site_path = os.path.join(dest, 'site.py')
+    egg_path_string, preamble = _relative_path_and_setup(
+        site_path, path, relative_paths, indent_level=2, omit_os_import=True)
+    if preamble:
+        preamble = '\n'.join(
+            [(line and '    %s' % (line,) or line)
+             for line in preamble.split('\n')])
+    original_path_setup = ''
+    if include_site_packages:
+        stdlib, site_paths = _get_system_paths(executable)
+        original_path_setup = original_path_snippet % (
+            _format_paths((repr(p) for p in site_paths), 2),)
+        distribution = working_set.find(
+            pkg_resources.Requirement.parse('setuptools'))
+        if distribution is not None:
+            # We need to worry about namespace packages.
+            if relative_paths:
+                location = _relativitize(
+                    distribution.location,
+                    os.path.normcase(os.path.abspath(site_path)),
+                    relative_paths)
+            else:
+                location = repr(distribution.location)
+            preamble += namespace_include_site_packages_setup % (location,)
+            original_path_setup = (
+                addsitedir_namespace_originalpackages_snippet +
+                original_path_setup)
+    addsitepackages_marker = 'def addsitepackages('
+    enableusersite_marker = 'ENABLE_USER_SITE = '
+    successful_rewrite = False
+    real_site_path = _get_module_file(executable, 'site')
+    real_site = open(real_site_path, 'r')
+    site = open(site_path, 'w')
+    try:
+        for line in real_site.readlines():
+            if line.startswith(enableusersite_marker):
+                site.write(enableusersite_marker)
+                site.write('False # buildout does not support user sites.\n')
+            elif line.startswith(addsitepackages_marker):
+                site.write(addsitepackages_script % (
+                    preamble, egg_path_string, original_path_setup))
+                site.write(line[len(addsitepackages_marker):])
+                successful_rewrite = True
+            else:
+                site.write(line)
+    finally:
+        site.close()
+        real_site.close()
+    if not successful_rewrite:
+        raise RuntimeError(
+            'Buildout did not successfully rewrite %s to %s' %
+            (real_site_path, site_path))
+    return site_path
+
+namespace_include_site_packages_setup = '''
+    setuptools_path = %s
+    sys.path.append(setuptools_path)
+    known_paths.add(os.path.normcase(setuptools_path))
+    import pkg_resources'''
+
+addsitedir_namespace_originalpackages_snippet = '''
+            pkg_resources.working_set.add_entry(sitedir)'''
+
+original_path_snippet = '''
+    sys.__egginsert = len(buildout_paths) # Support distribute.
+    original_paths = [
+        %s
+        ]
+    for path in original_paths:
+        addsitedir(path, known_paths)'''
+
+addsitepackages_script = '''\
+def addsitepackages(known_paths):
+    """Add site packages, as determined by zc.buildout.
+
+    See original_addsitepackages, below, for the original version."""%s
+    buildout_paths = [
+        %s
+        ]
+    for path in buildout_paths:
+        sitedir, sitedircase = makepath(path)
+        if not sitedircase in known_paths and os.path.exists(sitedir):
+            sys.path.append(sitedir)
+            known_paths.add(sitedircase)%s
+    return known_paths
+
+def original_addsitepackages('''
+
+def _generate_interpreter(name, dest, executable, site_py_dest,
+                          relative_paths=False):
+    """Write an interpreter script, using the site.py approach."""
+    full_name = os.path.join(dest, name)
+    site_py_dest_string, rpsetup = _relative_path_and_setup(
+        full_name, [site_py_dest], relative_paths, omit_os_import=True)
+    if rpsetup:
+        rpsetup += "\n"
+    if sys.platform == 'win32':
+        windows_import = '\nimport subprocess'
+        # os.exec* is a mess on Windows, particularly if the path
+        # to the executable has spaces and the Python is using MSVCRT.
+        # The standard fix is to surround the executable's path with quotes,
+        # but that has been unreliable in testing.
+        #
+        # Here's a demonstration of the problem.  Given a Python
+        # compiled with a MSVCRT-based compiler, such as the free Visual
+        # C++ 2008 Express Edition, and an executable path with spaces
+        # in it such as the below, we see the following.
+        #
+        # >>> import os
+        # >>> p0 = 'C:\\Documents and Settings\\Administrator\\My Documents\\Downloads\\Python-2.6.4\\PCbuild\\python.exe'
+        # >>> os.path.exists(p0)
+        # True
+        # >>> os.execv(p0, [])
+        # Traceback (most recent call last):
+        #  File "<stdin>", line 1, in <module>
+        # OSError: [Errno 22] Invalid argument
+        #
+        # That seems like a standard problem.  The standard solution is
+        # to quote the path (see, for instance
+        # http://bugs.python.org/issue436259).  However, this solution,
+        # and other variations, fail:
+        #
+        # >>> p1 = '"C:\\Documents and Settings\\Administrator\\My Documents\\Downloads\\Python-2.6.4\\PCbuild\\python.exe"'
+        # >>> os.execv(p1, [])
+        # Traceback (most recent call last):
+        #   File "<stdin>", line 1, in <module>
+        # OSError: [Errno 22] Invalid argument
+        #
+        # We simply use subprocess instead, since it handles everything
+        # nicely, and the transparency of exec* (that is, not running,
+        # perhaps unexpectedly, in a subprocess) is arguably not a
+        # necessity, at least for many use cases.
+        execute = 'subprocess.call(argv, env=environ)'
+    else:
+        windows_import = ''
+        execute = 'os.execve(sys.executable, argv, environ)'
+    contents = interpreter_template % dict(
+        python=_safe_arg(executable),
+        dash_S=' -S',
+        site_dest=site_py_dest_string,
+        relative_paths_setup=rpsetup,
+        windows_import=windows_import,
+        execute=execute,
+        )
+    return _write_script(full_name, contents, 'interpreter')
+
+interpreter_template = script_header + '''
+import os
+import sys%(windows_import)s
+%(relative_paths_setup)s
+argv = [sys.executable] + sys.argv[1:]
+environ = os.environ.copy()
+path = %(site_dest)s
+if environ.get('PYTHONPATH'):
+    path = os.pathsep.join([path, environ['PYTHONPATH']])
+environ['PYTHONPATH'] = path
+%(execute)s
+'''
+
+# End of script generation code.
+############################################################################
+
 runsetup_template = """
 import sys
 sys.path.insert(0, %(setupdir)r)

src/zc/buildout/easy_install.txt

@@ -89,6 +89,14 @@ use_dependency_links
    for using dependency_links in preference to other
    locations. Defaults to true.
 
+include_site_packages
+    A flag indicating whether Python's non-standard-library packages should
+    be available for finding dependencies.  Defaults to true.
+
+    Paths outside of Python's standard library--or more precisely, those that
+    are not included when Python is started with the -S argument--are loosely
+    referred to as "site-packages" here.
+
 relative_paths
    Adjust egg paths so they are relative to the script path.  This
    allows scripts to work when scripts and eggs are moved, as long as
@@ -214,7 +222,9 @@ dependencies.  We might do this to specify a specific version.
     d  other-1.0-py2.4.egg
 
 We can request that eggs be unzipped even if they are zip safe.  This
-can be useful when debugging.
+can be useful when debugging.  (Note that Distribute will unzip eggs by
+default, so if you are using Distribute, most or all eggs will already be
+unzipped without this flag.)
 
     >>> rmdir(dest)
     >>> dest = tmpdir('sample-install')
@@ -399,6 +409,68 @@ dictionary:
     >>> [d.version for d in ws]
     ['0.3', '1.1']
 
+Dependencies in Site Packages
+-----------------------------
+
+Paths outside of Python's standard library--or more precisely, those that are
+not included when Python is started with the -S argument--are loosely referred
+to as "site-packages" here.  These site-packages are searched by default for
+distributions.  This can be disabled, so that, for instance, a system Python
+can be used with buildout, cleaned of any packages installed by a user or
+system package manager.
+
+The default behavior can be controlled and introspected using
+zc.buildout.easy_install.include_site_packages.
+
+    >>> zc.buildout.easy_install.include_site_packages()
+    True
+
+Here's an example of using a Python executable that includes our dependencies.
+
+Our "py_path" will have the "demoneeded," and "demo" packages available.
+ We'll simply be asking for "demoneeded" here, but without any external
+ index or links.
+
+    >>> from zc.buildout.tests import create_sample_sys_install
+    >>> py_path, site_packages_path = make_py()
+    >>> create_sample_sys_install(site_packages_path)
+
+    >>> example_dest = tmpdir('site-packages-example-install')
+    >>> workingset = zc.buildout.easy_install.install(
+    ...     ['demoneeded'], example_dest, links=[], executable=py_path,
+    ...     index=None)
+    >>> [dist.project_name for dist in workingset]
+    ['demoneeded']
+
+That worked fine.  Let's try again with site packages not allowed.  We'll
+change the policy by changing the default.  Notice that the function for
+changing the default value returns the previous value.
+
+    >>> zc.buildout.easy_install.include_site_packages(False)
+    True
+
+    >>> zc.buildout.easy_install.include_site_packages()
+    False
+
+    >>> zc.buildout.easy_install.clear_index_cache()
+    >>> rmdir(example_dest)
+    >>> example_dest = tmpdir('site-packages-example-install')
+    >>> workingset = zc.buildout.easy_install.install(
+    ...     ['demoneeded'], example_dest, links=[], executable=py_path,
+    ...     index=None)
+    Traceback (most recent call last):
+        ...
+    MissingDistribution: Couldn't find a distribution for 'demoneeded'.
+    >>> zc.buildout.easy_install.clear_index_cache()
+
+Now we'll reset the default.
+
+    >>> zc.buildout.easy_install.include_site_packages(True)
+    False
+
+    >>> zc.buildout.easy_install.include_site_packages()
+    True
+
 Dependency links
 ----------------
 
@@ -521,25 +593,38 @@ To return the dependency_links behavior to normal call the function again.
 Script generation
 -----------------
 
-The easy_install module provides support for creating scripts from
-eggs.  It provides a function similar to setuptools except that it
-provides facilities for baking a script's path into the script.  This
-has two advantages:
+The easy_install module provides support for creating scripts from eggs.
+It provides two competing functions.  One, ``scripts``, is a
+well-established approach to generating reliable scripts with a "clean"
+Python--e.g., one that does not have any packages in its site-packages.
+The other, ``sitepackage_safe_scripts``, is newer, a bit trickier, and is
+designed to work with a Python that has code in its site-packages, such
+as a system Python.
+
+Both are similar to setuptools except that they provides facilities for
+baking a script's path into the script.  This has two advantages:
 
 - The eggs to be used by a script are not chosen at run time, making
   startup faster and, more importantly, deterministic.
 
-- The script doesn't have to import pkg_resources because the logic
-  that pkg_resources would execute at run time is executed at
-  script-creation time.
+- The script doesn't have to import pkg_resources because the logic that
+  pkg_resources would execute at run time is executed at script-creation
+  time.  (There is an exception in ``sitepackage_safe_scripts`` if you
+  want to have your Python's site packages available, as discussed
+  below, but even in that case pkg_resources is only partially
+  activated, which can be a significant time savings.)
 
-The scripts method can be used to generate scripts. Let's create a
-destination directory for it to place them in:
 
-    >>> import tempfile
+The ``scripts`` function
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``scripts`` function is the first way to generate scripts that we'll
+examine. It is the earlier approach that the package offered.  Let's
+create a destination directory for it to place them in:
+
     >>> bin = tmpdir('bin')
 
-Now, we'll use the scripts method to generate scripts in this directory
+Now, we'll use the scripts function to generate scripts in this directory
 from the demo egg:
 
     >>> import sys
@@ -736,8 +821,8 @@ original script names to new script names.
     >>> print system(os.path.join(bin, 'run')),
     3 1
 
-Including extra paths in scripts
---------------------------------
+The ``scripts`` function: Including extra paths in scripts
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 We can pass a keyword argument, extra paths, to cause additional paths
 to be included in the a generated script:
@@ -762,8 +847,8 @@ to be included in the a generated script:
     if __name__ == '__main__':
         eggrecipedemo.main()
 
-Providing script arguments
---------------------------
+The ``scripts`` function: Providing script arguments
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 An "argument" keyword argument can be used to pass arguments to an
 entry point.  The value passed is a source string to be placed between the
@@ -786,8 +871,8 @@ parentheses in the call:
     if __name__ == '__main__':
         eggrecipedemo.main(1, 2)
 
-Passing initialization code
----------------------------
+The ``scripts`` function: Passing initialization code
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 You can also pass script initialization code:
 
@@ -812,8 +897,8 @@ You can also pass script initialization code:
     if __name__ == '__main__':
         eggrecipedemo.main(1, 2)
 
-Relative paths
---------------
+The ``scripts`` function: Relative paths
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Sometimes, you want to be able to move a buildout directory around and
 have scripts still work without having to rebuild them.  We can
@@ -836,7 +921,7 @@ to pass a common base directory of the scripts and eggs:
     ...    interpreter='py',
     ...    relative_paths=bo)
 
-    >>> cat(bo, 'bin', 'run')
+    >>> cat(bo, 'bin', 'run') # doctest: +NORMALIZE_WHITESPACE
     #!/usr/local/bin/python2.4
     <BLANKLINE>
     import os
@@ -868,7 +953,7 @@ Of course, running the script works:
 
 We specified an interpreter and its paths are adjusted too:
 
-    >>> cat(bo, 'bin', 'py')
+    >>> cat(bo, 'bin', 'py') # doctest: +NORMALIZE_WHITESPACE
     #!/usr/local/bin/python2.4
     <BLANKLINE>
     import os
@@ -911,6 +996,570 @@ We specified an interpreter and its paths are adjusted too:
         del _interactive
         __import__("code").interact(banner="", local=globals())
 
+The ``sitepackage_safe_scripts`` function
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The newer function for creating scripts is ``sitepackage_safe_scripts``.
+It has the same basic functionality as the ``scripts`` function: it can
+create scripts to run arbitrary entry points, and to run a Python
+interpreter.  The following are the differences from a user's
+perspective.
+
+- It can be used safely with a Python that has packages installed itself,
+  such as a system-installed Python.
+
+- In contrast to the interpreter generated by the ``scripts`` method, which
+  supports only a small subset of the usual Python executable's options,
+  the interpreter generated by ``sitepackage_safe_scripts`` supports all
+  of them. This makes it possible to use as full Python replacement for
+  scripts that need the distributions specified in your buildout.
+
+- Both the interpreter and the entry point scripts allow you to include the
+  site packages, and/or the sitecustomize, of the Python executable, if
+  desired.
+
+It works by creating site.py and sitecustomize.py files that set up the
+desired paths and initialization.  These must be placed within an otherwise
+empty directory.  Typically this is in a recipe's parts directory.
+
+Here's the simplest example, building an interpreter script.
+
+    >>> interpreter_dir = tmpdir('interpreter')
+    >>> interpreter_parts_dir = os.path.join(
+    ...     interpreter_dir, 'parts', 'interpreter')
+    >>> interpreter_bin_dir = os.path.join(interpreter_dir, 'bin')
+    >>> mkdir(interpreter_bin_dir)
+    >>> mkdir(interpreter_dir, 'eggs')
+    >>> mkdir(interpreter_dir, 'parts')
+    >>> mkdir(interpreter_parts_dir)
+
+    >>> ws = zc.buildout.easy_install.install(
+    ...     ['demo'], join(interpreter_dir, 'eggs'), links=[link_server],
+    ...     index=link_server+'index/')
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py')
+
+Depending on whether the machine being used is running Windows or not, this
+produces either three or four files.  In both cases, we have site.py and
+sitecustomize.py generated in the parts/interpreter directory.  For Windows,
+we have py.exe and py-script.py; for other operating systems, we have py.
+
+    >>> sitecustomize_path = os.path.join(
+    ...     interpreter_parts_dir, 'sitecustomize.py')
+    >>> site_path = os.path.join(interpreter_parts_dir, 'site.py')
+    >>> interpreter_path = os.path.join(interpreter_bin_dir, 'py')
+    >>> if sys.platform == 'win32':
+    ...     py_path = os.path.join(interpreter_bin_dir, 'py-script.py')
+    ...     expected = [sitecustomize_path,
+    ...                 site_path,
+    ...                 os.path.join(interpreter_bin_dir, 'py.exe'),
+    ...                 py_path]
+    ... else:
+    ...     py_path = interpreter_path
+    ...     expected = [sitecustomize_path, site_path, py_path]
+    ...
+    >>> assert generated == expected, repr((generated, expected))
+
+We didn't ask for any initialization, and we didn't ask to use the underlying
+sitecustomization, so sitecustomize.py is empty.
+
+    >>> cat(sitecustomize_path)
+
+The interpreter script is simple.  It puts the directory with the
+site.py and sitecustomize.py on the PYTHONPATH and (re)starts Python.
+
+    >>> cat(py_path)
+    #!/usr/bin/python -S
+    import os
+    import sys
+    <BLANKLINE>
+    argv = [sys.executable] + sys.argv[1:]
+    environ = os.environ.copy()
+    path = '/interpreter/parts/interpreter'
+    if environ.get('PYTHONPATH'):
+        path = os.pathsep.join([path, environ['PYTHONPATH']])
+    environ['PYTHONPATH'] = path
+    os.execve(sys.executable, argv, environ)
+
+The site.py file is a modified version of the underlying Python's site.py.
+The most important modification is that it has a different version of the
+addsitepackages function.  It sets up the Python path, similarly to the
+behavior of the function it replaces.  The following shows the part that
+buildout inserts, in the simplest case.
+
+    >>> sys.stdout.write('#\n'); cat(site_path)
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    #...
+    def addsitepackages(known_paths):
+        """Add site packages, as determined by zc.buildout.
+    <BLANKLINE>
+        See original_addsitepackages, below, for the original version."""
+        buildout_paths = [
+            '/interpreter/eggs/demo-0.3-pyN.N.egg',
+            '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'
+            ]
+        for path in buildout_paths:
+            sitedir, sitedircase = makepath(path)
+            if not sitedircase in known_paths and os.path.exists(sitedir):
+                sys.path.append(sitedir)
+                known_paths.add(sitedircase)
+        return known_paths
+    <BLANKLINE>
+    def original_addsitepackages(known_paths):...
+
+Here are some examples of the interpreter in use.
+
+    >>> print call_py(interpreter_path, "print 16+26")
+    42
+    <BLANKLINE>
+    >>> res = call_py(interpreter_path, "import sys; print sys.path")
+    >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    ['',
+     '/interpreter/parts/interpreter',
+     ...,
+     '/interpreter/eggs/demo-0.3-pyN.N.egg',
+     '/interpreter/eggs/demoneeded-1.1-pyN.N.egg']
+    <BLANKLINE>
+    >>> clean_paths = eval(res.strip()) # This is used later for comparison.
+
+If you provide initialization, it goes in sitecustomize.py.
+
+    >>> def reset_interpreter():
+    ...     # This is necessary because, in our tests, the timestamps of the
+    ...     # .pyc files are not outdated when we want them to be.
+    ...     rmdir(interpreter_bin_dir)
+    ...     mkdir(interpreter_bin_dir)
+    ...     rmdir(interpreter_parts_dir)
+    ...     mkdir(interpreter_parts_dir)
+    ...
+    >>> reset_interpreter()
+
+    >>> initialization_string = """\
+    ... import os
+    ... os.environ['FOO'] = 'bar baz bing shazam'"""
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py', initialization=initialization_string)
+    >>> cat(sitecustomize_path)
+    import os
+    os.environ['FOO'] = 'bar baz bing shazam'
+    >>> print call_py(interpreter_path, "import os; print os.environ['FOO']")
+    bar baz bing shazam
+    <BLANKLINE>
+
+If you use relative paths, this affects the interpreter and site.py.  (This is
+again the UNIX version; the Windows version uses subprocess instead of
+os.execve.)
+
+    >>> reset_interpreter()
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py', relative_paths=interpreter_dir)
+    >>> cat(py_path)
+    #!/usr/bin/python -S
+    import os
+    import sys
+    <BLANKLINE>
+    join = os.path.join
+    base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
+    base = os.path.dirname(base)
+    <BLANKLINE>
+    argv = [sys.executable] + sys.argv[1:]
+    environ = os.environ.copy()
+    path = join(base, 'parts/interpreter')
+    if environ.get('PYTHONPATH'):
+        path = os.pathsep.join([path, environ['PYTHONPATH']])
+    environ['PYTHONPATH'] = path
+    os.execve(sys.executable, argv, environ)
+
+For site.py, we again show only the pertinent parts.  Notice that the egg
+paths join a base to a path, as with the use of this argument in the
+``scripts`` function.
+
+    >>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS
+    #...
+    def addsitepackages(known_paths):
+        """Add site packages, as determined by zc.buildout.
+    <BLANKLINE>
+        See original_addsitepackages, below, for the original version."""
+        join = os.path.join
+        base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
+        base = os.path.dirname(base)
+        base = os.path.dirname(base)
+        buildout_paths = [
+            join(base, 'eggs/demo-0.3-pyN.N.egg'),
+            join(base, 'eggs/demoneeded-1.1-pyN.N.egg')
+            ]...
+
+The paths resolve in practice as you would expect.
+
+    >>> print call_py(interpreter_path,
+    ...               "import sys, pprint; pprint.pprint(sys.path)")
+    ... # doctest: +ELLIPSIS
+    ['',
+     '/interpreter/parts/interpreter',
+     ...,
+     '/interpreter/eggs/demo-0.3-pyN.N.egg',
+     '/interpreter/eggs/demoneeded-1.1-pyN.N.egg']
+    <BLANKLINE>
+
+The ``extra_paths`` argument affects the path in site.py.  Notice that
+/interpreter/other is added after the eggs.
+
+    >>> reset_interpreter()
+    >>> mkdir(interpreter_dir, 'other')
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py', extra_paths=[join(interpreter_dir, 'other')])
+    >>> sys.stdout.write('#\n'); cat(site_path) # doctest: +ELLIPSIS
+    #...
+    def addsitepackages(known_paths):
+        """Add site packages, as determined by zc.buildout.
+    <BLANKLINE>
+        See original_addsitepackages, below, for the original version."""
+        buildout_paths = [
+            '/interpreter/eggs/demo-0.3-pyN.N.egg',
+            '/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
+            '/interpreter/other'
+            ]...
+
+    >>> print call_py(interpreter_path,
+    ...               "import sys, pprint; pprint.pprint(sys.path)")
+    ... # doctest: +ELLIPSIS
+    ['',
+     '/interpreter/parts/interpreter',
+     ...,
+     '/interpreter/eggs/demo-0.3-pyN.N.egg',
+     '/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
+     '/interpreter/other']
+    <BLANKLINE>
+
+The ``sitepackage_safe_scripts`` function: using site-packages
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``sitepackage_safe_scripts`` function supports including site
+packages.  This has some advantages and some serious dangers.
+
+A typical reason to include site-packages is that it is easier to
+install one or more dependencies in your Python than it is with
+buildout.  Some packages, such as lxml or Python PostgreSQL integration,
+have dependencies that can be much easier to build and/or install using
+other mechanisms, such as your operating system's package manager.  By
+installing some core packages into your Python's site-packages, this can
+significantly simplify some application installations.
+
+However, doing this has a significant danger.  One of the primary goals
+of buildout is to provide repeatability.  Some packages (one of the
+better known Python openid packages, for instance) change their behavior
+depending on what packages are available.  If Python curl bindings are
+available, these may be preferred by the library.  If a certain XML
+package is installed, it may be preferred by the library.  These hidden
+choices may cause small or large behavior differences.  The fact that
+they can be rarely encountered can actually make it worse: you forget
+that this might be a problem, and debugging the differences can be
+difficult.  If you allow site-packages to be included in your buildout,
+and the Python you use is not managed precisely by your application (for
+instance, it is a system Python), you open yourself up to these
+possibilities.  Don't be unaware of the dangers.
+
+That explained, let's see how it works.  If you don't use namespace packages,
+this is very straightforward.
+
+    >>> reset_interpreter()
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py', include_site_packages=True)
+    >>> sys.stdout.write('#\n'); cat(site_path)
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    #...
+    def addsitepackages(known_paths):
+        """Add site packages, as determined by zc.buildout.
+    <BLANKLINE>
+        See original_addsitepackages, below, for the original version."""
+        buildout_paths = [
+            '/interpreter/eggs/demo-0.3-pyN.N.egg',
+            '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'
+            ]
+        for path in buildout_paths:
+            sitedir, sitedircase = makepath(path)
+            if not sitedircase in known_paths and os.path.exists(sitedir):
+                sys.path.append(sitedir)
+                known_paths.add(sitedircase)
+        sys.__egginsert = len(buildout_paths) # Support distribute.
+        original_paths = [
+            ...
+            ]
+        for path in original_paths:
+            addsitedir(path, known_paths)
+        return known_paths
+    <BLANKLINE>
+    def original_addsitepackages(known_paths):...
+
+It simply adds the original paths using addsitedir after the code to add the
+buildout paths.
+
+Here's an example of the new script in use.  Other documents and tests in
+this package give the feature a more thorough workout, but this should
+give you an idea of the feature.
+
+    >>> res = call_py(interpreter_path, "import sys; print sys.path")
+    >>> print res # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    ['',
+     '/interpreter/parts/interpreter',
+     ...,
+     '/interpreter/eggs/demo-0.3-py2.4.egg',
+     '/interpreter/eggs/demoneeded-1.1-py2.4.egg',
+     ...]
+    <BLANKLINE>
+
+The clean_paths gathered earlier is a subset of this full list of paths.
+
+    >>> full_paths = eval(res.strip())
+    >>> len(clean_paths) < len(full_paths)
+    True
+    >>> set(os.path.normpath(p) for p in clean_paths).issubset(
+    ...     os.path.normpath(p) for p in full_paths)
+    True
+
+Unfortunately, because of how setuptools namespace packages are implemented
+differently for operating system packages (debs or rpms) as opposed to
+standard setuptools installation, there's a slightly trickier dance if you
+use them.  To show this we'll needs some extra eggs that use namespaces.
+We'll use the ``tellmy.fortune`` package, which we'll need to make an initial
+call to another text fixture to create.
+
+    >>> from zc.buildout.tests import create_sample_namespace_eggs
+    >>> namespace_eggs = tmpdir('namespace_eggs')
+    >>> create_sample_namespace_eggs(namespace_eggs)
+
+    >>> reset_interpreter()
+    >>> ws = zc.buildout.easy_install.install(
+    ...     ['demo', 'tellmy.fortune'], join(interpreter_dir, 'eggs'),
+    ...     links=[link_server, namespace_eggs], index=link_server+'index/')
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py', include_site_packages=True)
+    >>> sys.stdout.write('#\n'); cat(site_path)
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    #...
+    def addsitepackages(known_paths):
+        """Add site packages, as determined by zc.buildout.
+    <BLANKLINE>
+        See original_addsitepackages, below, for the original version."""
+        setuptools_path = '...setuptools...'
+        sys.path.append(setuptools_path)
+        known_paths.add(os.path.normcase(setuptools_path))
+        import pkg_resources
+        buildout_paths = [
+            '/interpreter/eggs/demo-0.3-pyN.N.egg',
+            '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg',
+            '...setuptools...',
+            '/interpreter/eggs/demoneeded-1.1-pyN.N.egg'
+            ]
+        for path in buildout_paths:
+            sitedir, sitedircase = makepath(path)
+            if not sitedircase in known_paths and os.path.exists(sitedir):
+                sys.path.append(sitedir)
+                known_paths.add(sitedircase)
+                pkg_resources.working_set.add_entry(sitedir)
+        sys.__egginsert = len(buildout_paths) # Support distribute.
+        original_paths = [
+            ...
+            ]
+        for path in original_paths:
+            addsitedir(path, known_paths)
+        return known_paths
+    <BLANKLINE>
+    def original_addsitepackages(known_paths):...
+
+    >>> print call_py(interpreter_path, "import sys; print sys.path")
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    ['',
+     '/interpreter/parts/interpreter',
+     ...,
+     '...setuptools...',
+     '/interpreter/eggs/demo-0.3-pyN.N.egg',
+     '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg',
+     '/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
+     ...]
+
+As you can see, the script now first imports pkg_resources.  Then we
+need to process egg files specially to look for namespace packages there
+*before* we process process lines in .pth files that use the "import"
+feature--lines that might be part of the setuptools namespace package
+implementation for system packages, as mentioned above, and that must
+come after processing egg namespaces.
+
+The most complex that this function gets is if you use namespace packages,
+include site-packages, and use relative paths.  For completeness, we'll look
+at that result.
+
+    >>> reset_interpreter()
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     interpreter='py', include_site_packages=True,
+    ...     relative_paths=interpreter_dir)
+    >>> sys.stdout.write('#\n'); cat(site_path)
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    #...
+    def addsitepackages(known_paths):
+        """Add site packages, as determined by zc.buildout.
+    <BLANKLINE>
+        See original_addsitepackages, below, for the original version."""
+        join = os.path.join
+        base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
+        base = os.path.dirname(base)
+        base = os.path.dirname(base)
+        setuptools_path = '...setuptools...'
+        sys.path.append(setuptools_path)
+        known_paths.add(os.path.normcase(setuptools_path))
+        import pkg_resources
+        buildout_paths = [
+            join(base, 'eggs/demo-0.3-pyN.N.egg'),
+            join(base, 'eggs/tellmy.fortune-1.0-pyN.N.egg'),
+            '...setuptools...',
+            join(base, 'eggs/demoneeded-1.1-pyN.N.egg')
+            ]
+        for path in buildout_paths:
+            sitedir, sitedircase = makepath(path)
+            if not sitedircase in known_paths and os.path.exists(sitedir):
+                sys.path.append(sitedir)
+                known_paths.add(sitedircase)
+                pkg_resources.working_set.add_entry(sitedir)
+        sys.__egginsert = len(buildout_paths) # Support distribute.
+        original_paths = [
+            ...
+            ]
+        for path in original_paths:
+            addsitedir(path, known_paths)
+        return known_paths
+    <BLANKLINE>
+    def original_addsitepackages(known_paths):...
+
+    >>> print call_py(interpreter_path, "import sys; print sys.path")
+    ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
+    ['',
+     '/interpreter/parts/interpreter',
+     ...,
+     '...setuptools...',
+     '/interpreter/eggs/demo-0.3-pyN.N.egg',
+     '/interpreter/eggs/tellmy.fortune-1.0-pyN.N.egg',
+     '/interpreter/eggs/demoneeded-1.1-pyN.N.egg',
+     ...]
+
+The ``exec_sitecustomize`` argument does the same thing for the
+sitecustomize module--it allows you to include the code from the
+sitecustomize module in the underlying Python if you set the argument to
+True.  The z3c.recipe.scripts package sets up the full environment necessary
+to demonstrate this piece.
+
+The ``sitepackage_safe_scripts`` function: writing scripts for entry points
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+All of the examples so far for this function have been creating
+interpreters.  The function can also write scripts for entry
+points.  They are almost identical to the scripts that we saw for the
+``scripts`` function except that they ``import site`` after setting the
+sys.path to include our custom site.py and sitecustomize.py files.  These
+files then initialize the Python environment as we have already seen.  Let's
+see a simple example.
+
+    >>> reset_interpreter()
+    >>> ws = zc.buildout.easy_install.install(
+    ...     ['demo'], join(interpreter_dir, 'eggs'), links=[link_server],
+    ...     index=link_server+'index/')
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     reqs=['demo'])
+
+As before, in Windows, 2 files are generated for each script.  A script
+file, ending in '-script.py', and an exe file that allows the script
+to be invoked directly without having to specify the Python
+interpreter and without having to provide a '.py' suffix.  This is in addition
+to the site.py and sitecustomize.py files that are generated as with our
+interpreter examples above.
+
+    >>> if sys.platform == 'win32':
+    ...     demo_path = os.path.join(interpreter_bin_dir, 'demo-script.py')
+    ...     expected = [sitecustomize_path,
+    ...                 site_path,
+    ...                 os.path.join(interpreter_bin_dir, 'demo.exe'),
+    ...                 demo_path]
+    ... else:
+    ...     demo_path = os.path.join(interpreter_bin_dir, 'demo')
+    ...     expected = [sitecustomize_path, site_path, demo_path]
+    ...
+    >>> assert generated == expected, repr((generated, expected))
+
+The demo script runs the entry point defined in the demo egg:
+
+    >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE
+    #!/usr/local/bin/python2.4 -S
+    <BLANKLINE>
+    import sys
+    sys.path[0:0] = [
+        '/interpreter/parts/interpreter',
+        ]
+    <BLANKLINE>
+    <BLANKLINE>
+    import os
+    path = sys.path[0]
+    if os.environ.get('PYTHONPATH'):
+        path = os.pathsep.join([path, os.environ['PYTHONPATH']])
+    os.environ['PYTHONPATH'] = path
+    import site # imports custom buildout-generated site.py
+    <BLANKLINE>
+    import eggrecipedemo
+    <BLANKLINE>
+    if __name__ == '__main__':
+        eggrecipedemo.main()
+
+    >>> demo_call = join(interpreter_bin_dir, 'demo')
+    >>> if sys.platform == 'win32':
+    ...     demo_call = '"%s"' % demo_call
+    >>> print system(demo_call)
+    3 1
+    <BLANKLINE>
+
+There are a few differences from the ``scripts`` function.  First, the
+``reqs`` argument (an iterable of string requirements or entry point
+tuples) is a keyword argument here.  We see that in the example above.
+Second, the ``arguments`` argument is now named ``script_arguments`` to
+try and clarify that it does not affect interpreters. While the
+``initialization`` argument continues to affect both the interpreters
+and the entry point scripts, if you have initialization that is only
+pertinent to the entry point scripts, you can use the
+``script_initialization`` argument.
+
+Let's see ``script_arguments`` and ``script_initialization`` in action.
+
+    >>> reset_interpreter()
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     reqs=['demo'], script_arguments='1, 2',
+    ...     script_initialization='import os\nos.chdir("foo")')
+
+    >>> cat(demo_path) # doctest: +NORMALIZE_WHITESPACE
+    #!/usr/local/bin/python2.4 -S
+    import sys
+    sys.path[0:0] = [
+      '/interpreter/parts/interpreter',
+      ]
+    <BLANKLINE>
+    import os
+    path = sys.path[0]
+    if os.environ.get('PYTHONPATH'):
+        path = os.pathsep.join([path, os.environ['PYTHONPATH']])
+    os.environ['PYTHONPATH'] = path
+    import site # imports custom buildout-generated site.py
+    import os
+    os.chdir("foo")
+    <BLANKLINE>
+    import eggrecipedemo
+    <BLANKLINE>
+    if __name__ == '__main__':
+        eggrecipedemo.main(1, 2)
 
 Handling custom build options for extensions provided in source distributions
 -----------------------------------------------------------------------------

src/zc/buildout/testing.py

@@ -28,6 +28,7 @@ import socket
 import subprocess
 import sys
 import tempfile
+import textwrap
 import threading
 import time
 import urllib2
@@ -109,6 +110,16 @@ def system(command, input=''):
     e.close()
     return result
 
+def call_py(interpreter, cmd, flags=None):
+    if sys.platform == 'win32':
+        args = ['"%s"' % arg for arg in (interpreter, flags, cmd) if arg]
+        args.insert(-1, '"-c"')
+        return system('"%s"' % ' '.join(args))
+    else:
+        cmd = repr(cmd)
+        return system(
+            ' '.join(arg for arg in (interpreter, flags, '-c', cmd) if arg))
+
 def get(url):
     return urllib2.urlopen(url).read()
 
@@ -120,7 +131,11 @@ def _runsetup(setup, executable, *args):
     args = [zc.buildout.easy_install._safe_arg(arg)
             for arg in args]
     args.insert(0, '-q')
-    args.append(dict(os.environ, PYTHONPATH=setuptools_location))
+    env = dict(os.environ)
+    if executable == sys.executable:
+        env['PYTHONPATH'] = setuptools_location
+    # else pass an executable that has setuptools! See testselectingpython.py.
+    args.append(env)
 
     here = os.getcwd()
     try:
@@ -139,6 +154,11 @@ def sdist(setup, dest):
 def bdist_egg(setup, executable, dest):
     _runsetup(setup, executable, 'bdist_egg', '-d', dest)
 
+def sys_install(setup, dest):
+    _runsetup(setup, sys.executable, 'install', '--install-purelib', dest,
+              '--record', os.path.join(dest, '__added_files__'),
+              '--single-version-externally-managed')
+
 def find_python(version):
     e = os.environ.get('PYTHON%s' % version)
     if e is not None:
@@ -206,14 +226,64 @@ def wait_until(label, func, *args, **kw):
         time.sleep(0.01)
     raise ValueError('Timed out waiting for: '+label)
 
+def get_installer_values():
+    """Get the current values for the easy_install module.
+
+    This is necessary because instantiating a Buildout will force the
+    Buildout's values on the installer.
+
+    Returns a dict of names-values suitable for set_installer_values."""
+    names = ('default_versions', 'download_cache', 'install_from_cache',
+             'prefer_final', 'include_site_packages',
+             'allowed_eggs_from_site_packages', 'use_dependency_links',
+             'allow_picked_versions', 'always_unzip'
+            )
+    values = {}
+    for name in names:
+        values[name] = getattr(zc.buildout.easy_install, name)()
+    return values
+
+def set_installer_values(values):
+    """Set the given values on the installer."""
+    for name, value in values.items():
+        getattr(zc.buildout.easy_install, name)(value)
+
+def make_buildout(executable=None):
+    """Make a buildout that uses this version of zc.buildout."""
+    # Create a basic buildout.cfg to avoid a warning from buildout.
+    open('buildout.cfg', 'w').write(
+        "[buildout]\nparts =\n"
+        )
+    # Get state of installer defaults so we can reinstate them (instantiating
+    # a Buildout will force the Buildout's defaults on the installer).
+    installer_values = get_installer_values()
+    # Use the buildout bootstrap command to create a buildout
+    config = [
+        ('buildout', 'log-level', 'WARNING'),
+        # trick bootstrap into putting the buildout develop egg
+        # in the eggs dir.
+        ('buildout', 'develop-eggs-directory', 'eggs'),
+        ]
+    if executable is not None:
+        config.append(('buildout', 'executable', executable))
+    zc.buildout.buildout.Buildout(
+        'buildout.cfg', config,
+        user_defaults=False,
+        ).bootstrap([])
+    # Create the develop-eggs dir, which didn't get created the usual
+    # way due to the trick above:
+    os.mkdir('develop-eggs')
+    # Reinstate the default values of the installer.
+    set_installer_values(installer_values)
+
 def buildoutSetUp(test):
 
     test.globs['__tear_downs'] = __tear_downs = []
     test.globs['register_teardown'] = register_teardown = __tear_downs.append
 
-    prefer_final = zc.buildout.easy_install.prefer_final()
+    installer_values = get_installer_values()
     register_teardown(
-        lambda: zc.buildout.easy_install.prefer_final(prefer_final)
+        lambda: set_installer_values(installer_values)
         )
 
     here = os.getcwd()
@@ -259,27 +329,7 @@ def buildoutSetUp(test):
     sample = tmpdir('sample-buildout')
 
     os.chdir(sample)
-
-    # Create a basic buildout.cfg to avoid a warning from buildout:
-    open('buildout.cfg', 'w').write(
-        "[buildout]\nparts =\n"
-        )
-
-    # Use the buildout bootstrap command to create a buildout
-    zc.buildout.buildout.Buildout(
-        'buildout.cfg',
-        [('buildout', 'log-level', 'WARNING'),
-         # trick bootstrap into putting the buildout develop egg
-         # in the eggs dir.
-         ('buildout', 'develop-eggs-directory', 'eggs'),
-         ]
-        ).bootstrap([])
-
-
-
-    # Create the develop-eggs dir, which didn't get created the usual
-    # way due to the trick above:
-    os.mkdir('develop-eggs')
+    make_buildout()
 
     def start_server(path):
         port, thread = _start_server(path, name=path)
@@ -287,6 +337,50 @@ def buildoutSetUp(test):
         register_teardown(lambda: stop_server(url, thread))
         return url
 
+    def make_py(initialization=''):
+        """Returns paths to new executable and to its site-packages.
+        """
+        buildout = tmpdir('executable_buildout')
+        site_packages_dir = os.path.join(buildout, 'site-packages')
+        mkdir(site_packages_dir)
+        old_wd = os.getcwd()
+        os.chdir(buildout)
+        make_buildout()
+        # Normally we don't process .pth files in extra-paths.  We want to
+        # in this case so that we can test with setuptools system installs
+        # (--single-version-externally-managed), which use .pth files.
+        initialization = (
+            ('import sys\n'
+             'import site\n'
+             'known_paths = set(sys.path)\n'
+             'site_packages_dir = %r\n'
+             'site.addsitedir(site_packages_dir, known_paths)\n'
+            ) % (site_packages_dir,)) + initialization
+        initialization = '\n'.join(
+            '  ' + line for line in initialization.split('\n'))
+        install_develop(
+            'zc.recipe.egg', os.path.join(buildout, 'develop-eggs'))
+        install_develop(
+            'z3c.recipe.scripts', os.path.join(buildout, 'develop-eggs'))
+        write('buildout.cfg', textwrap.dedent('''\
+            [buildout]
+            parts = py
+
+            [py]
+            recipe = z3c.recipe.scripts
+            interpreter = py
+            initialization =
+            %(initialization)s
+            extra-paths = %(site-packages)s
+            eggs = setuptools
+            ''') % {
+                'initialization': initialization,
+                'site-packages': site_packages_dir})
+        system(os.path.join(buildout, 'bin', 'buildout'))
+        os.chdir(old_wd)
+        return (
+            os.path.join(buildout, 'bin', 'py'), site_packages_dir)
+
     test.globs.update(dict(
         sample_buildout = sample,
         ls = ls,
@@ -297,6 +391,7 @@ def buildoutSetUp(test):
         tmpdir = tmpdir,
         write = write,
         system = system,
+        call_py = call_py,
         get = get,
         cd = (lambda *path: os.chdir(os.path.join(*path))),
         join = os.path.join,
@@ -305,10 +400,9 @@ def buildoutSetUp(test):
         start_server = start_server,
         buildout = os.path.join(sample, 'bin', 'buildout'),
         wait_until = wait_until,
+        make_py = make_py
         ))
 
-    zc.buildout.easy_install.prefer_final(prefer_final)
-
 def buildoutTearDown(test):
     for f in test.globs['__tear_downs']:
         f()

src/zc/buildout/tests.py

@@ -53,6 +53,7 @@ We should be able to deal with setup scripts that aren't setuptools based.
 
     >>> ls('develop-eggs')
     -  foo.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
     """
@@ -84,6 +85,7 @@ We should be able to deal with setup scripts that aren't setuptools based.
 
     >>> ls('develop-eggs')
     -  foo.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
     >>> print system(join('bin', 'buildout')+' -vvv'), # doctest: +ELLIPSIS
@@ -383,6 +385,64 @@ buildout will tell us who's asking for something that we can't find.
     Error: Couldn't find a distribution for 'demoneeded'.
     """
 
+def show_eggs_from_site_packages():
+    """
+Sometimes you want to know what eggs are coming from site-packages.  This
+might be for a diagnostic, or so that you can get a starting value for the
+allowed-eggs-from-site-packages option.  The -v flag will also include this
+information.
+
+Our "py_path" has the "demoneeded," "demo"
+packages available.  We'll ask for "bigdemo," which will get both of them.
+
+Here's our set up.
+
+    >>> py_path, site_packages_path = make_py()
+    >>> create_sample_sys_install(site_packages_path)
+
+    >>> write('buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = eggs
+    ... prefer-final = true
+    ... find-links = %(link_server)s
+    ...
+    ... [primed_python]
+    ... executable = %(py_path)s
+    ...
+    ... [eggs]
+    ... recipe = zc.recipe.egg:eggs
+    ... python = primed_python
+    ... eggs = bigdemo
+    ... ''' % globals())
+
+Now here is the output.  The lines that begin with "Egg from site-packages:"
+indicate the eggs from site-packages that have been selected.  You'll see
+we have two: demo 0.3 and demoneeded 1.1.
+
+    >>> print system(buildout+" -v")
+    Installing 'zc.buildout', 'setuptools'.
+    We have a develop egg: zc.buildout V
+    We have the best distribution that satisfies 'setuptools'.
+    Picked: setuptools = V
+    Installing 'zc.recipe.egg'.
+    We have a develop egg: zc.recipe.egg V
+    Installing eggs.
+    Installing 'bigdemo'.
+    We have no distributions for bigdemo that satisfies 'bigdemo'.
+    Getting distribution for 'bigdemo'.
+    Got bigdemo 0.1.
+    Picked: bigdemo = 0.1
+    Getting required 'demo'
+      required by bigdemo 0.1.
+    We have a develop egg: demo V
+    Egg from site-packages: demo 0.3
+    Getting required 'demoneeded'
+      required by demo 0.3.
+    We have a develop egg: demoneeded V
+    Egg from site-packages: demoneeded 1.1
+    <BLANKLINE>
+    """
 
 def test_comparing_saved_options_with_funny_characters():
     """
@@ -668,6 +728,7 @@ Create a develop egg:
 
     >>> ls('develop-eggs')
     -  foox.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Create another:
@@ -692,6 +753,7 @@ Create another:
     >>> ls('develop-eggs')
     -  foox.egg-link
     -  fooy.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Remove one:
@@ -709,6 +771,7 @@ It is gone
 
     >>> ls('develop-eggs')
     -  fooy.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Remove the other:
@@ -723,6 +786,7 @@ Remove the other:
 All gone
 
     >>> ls('develop-eggs')
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
     '''
 
@@ -797,9 +861,11 @@ On the other hand, if we have a regular egg, rather than a develop egg:
     ...            + join(sample_buildout, 'eggs'))
 
     >>> ls('develop-eggs')
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
-    >>> ls('eggs') # doctest: +ELLIPSIS
+    >>> print 'START ->'; ls('eggs') # doctest: +ELLIPSIS
+    START...
     -  foox-0.0.0-py2.4.egg
     ...
 
@@ -1769,6 +1835,658 @@ def bug_105081_Specific_egg_versions_are_ignored_when_newer_eggs_are_around():
     1 2
     """
 
+def versions_section_ignored_for_dependency_in_favor_of_site_packages():
+    r"""
+This is a test for a bugfix.
+
+The error showed itself when at least two dependencies were in a shared
+location like site-packages, and the first one met the "versions" setting.  The
+first dependency would be added, but subsequent dependencies from the same
+location (e.g., site-packages) would use the version of the package found in
+the shared location, ignoring the version setting.
+
+We begin with a Python that has demoneeded version 1.1 installed and a
+demo version 0.3, all in a site-packages-like shared directory.  We need
+to create this.  ``eggrecipedemo.main()`` shows the number after the dot
+(that is, ``X`` in ``1.X``), for the demo package and the demoneeded
+package, so this demonstrates that our Python does in fact have demo
+version 0.3 and demoneeded version 1.1.
+
+    >>> py_path = make_py_with_system_install(make_py, sample_eggs)
+    >>> print call_py(
+    ...     py_path,
+    ...     "import tellmy.version; print tellmy.version.__version__"),
+    1.1
+
+Now here's a setup that would expose the bug, using the
+zc.buildout.easy_install API.
+
+    >>> example_dest = tmpdir('example_dest')
+    >>> workingset = zc.buildout.easy_install.install(
+    ...     ['tellmy.version'], example_dest, links=[sample_eggs],
+    ...     executable=py_path,
+    ...     index=None,
+    ...     versions={'tellmy.version': '1.0'})
+    >>> for dist in workingset:
+    ...     res = str(dist)
+    ...     if res.startswith('tellmy.version'):
+    ...         print res
+    ...         break
+    tellmy.version 1.0
+
+Before the bugfix, the desired tellmy.version distribution would have
+been blocked the one in site-packages.
+"""
+
+def handle_namespace_package_in_both_site_packages_and_buildout_eggs():
+    r"""
+If you have the same namespace package in both site-packages and in
+buildout, we need to be very careful that faux-Python-executables and
+scripts generated by easy_install.sitepackage_safe_scripts correctly
+combine the two. We show this with the local recipe that uses the
+function, z3c.recipe.scripts.
+
+To demonstrate this, we will create three packages: tellmy.version 1.0,
+tellmy.version 1.1, and tellmy.fortune 1.0.  tellmy.version 1.1 is installed.
+
+    >>> py_path = make_py_with_system_install(make_py, sample_eggs)
+    >>> print call_py(
+    ...     py_path,
+    ...     "import tellmy.version; print tellmy.version.__version__")
+    1.1
+    <BLANKLINE>
+
+Now we will create a buildout that creates a script and a faux-Python script.
+We want to see that both can successfully import the specified versions of
+tellmy.version and tellmy.fortune.
+
+    >>> write('buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = eggs
+    ... find-links = %(link_server)s
+    ...
+    ... [primed_python]
+    ... executable = %(py_path)s
+    ...
+    ... [eggs]
+    ... recipe = z3c.recipe.scripts
+    ... python = primed_python
+    ... interpreter = py
+    ... include-site-packages = true
+    ... eggs = tellmy.version == 1.0
+    ...        tellmy.fortune == 1.0
+    ...        demo
+    ... script-initialization =
+    ...     import tellmy.version
+    ...     print tellmy.version.__version__
+    ...     import tellmy.fortune
+    ...     print tellmy.fortune.__version__
+    ... ''' % globals())
+
+    >>> print system(buildout)
+    Installing eggs.
+    Getting distribution for 'tellmy.version==1.0'.
+    Got tellmy.version 1.0.
+    Getting distribution for 'tellmy.fortune==1.0'.
+    Got tellmy.fortune 1.0.
+    Getting distribution for 'demo'.
+    Got demo 0.4c1.
+    Getting distribution for 'demoneeded'.
+    Got demoneeded 1.2c1.
+    Generated script '/sample-buildout/bin/demo'.
+    Generated interpreter '/sample-buildout/bin/py'.
+    <BLANKLINE>
+
+Finally, we are ready to see if it worked.  Prior to the bug fix that
+this tests, the results of both calls below was the following::
+
+    1.1
+    Traceback (most recent call last):
+      ...
+    ImportError: No module named fortune
+    <BLANKLINE>
+
+In other words, we got the site-packages version of tellmy.version, and
+we could not import tellmy.fortune at all.  The following are the correct
+results for the interpreter and for the script.
+
+    >>> print call_py(
+    ...     join('bin', 'py'),
+    ...     "import tellmy.version; " +
+    ...     "print tellmy.version.__version__; " +
+    ...     "import tellmy.fortune; " +
+    ...     "print tellmy.fortune.__version__") # doctest: +ELLIPSIS
+    1.0
+    1.0...
+
+    >>> print system(join('bin', 'demo'))
+    1.0
+    1.0
+    4 2
+    <BLANKLINE>
+    """
+
+def handle_sys_path_version_hack():
+    r"""
+This is a test for a bugfix.
+
+If you use a Python that has a different version of one of your
+dependencies, and the new package tries to do sys.path tricks in the
+setup.py to get a __version__, and it uses namespace packages, the older
+package will be loaded first, making the setup version the wrong number.
+While very arguably packages simply shouldn't do this, some do, and we
+don't want buildout to fall over when they do.
+
+To demonstrate this, we will need to create a distribution that has one of
+these unpleasant tricks, and a Python that has an older version installed.
+
+    >>> py_path, site_packages_path = make_py()
+    >>> for version in ('1.0', '1.1'):
+    ...     tmp = tempfile.mkdtemp()
+    ...     try:
+    ...         write(tmp, 'README.txt', '')
+    ...         mkdir(tmp, 'src')
+    ...         mkdir(tmp, 'src', 'tellmy')
+    ...         write(tmp, 'src', 'tellmy', '__init__.py',
+    ...             "__import__("
+    ...             "'pkg_resources').declare_namespace(__name__)\n")
+    ...         mkdir(tmp, 'src', 'tellmy', 'version')
+    ...         write(tmp, 'src', 'tellmy', 'version',
+    ...               '__init__.py', '__version__=%r\n' % version)
+    ...         write(
+    ...             tmp, 'setup.py',
+    ...             "from setuptools import setup\n"
+    ...             "import sys\n"
+    ...             "sys.path.insert(0, 'src')\n"
+    ...             "from tellmy.version import __version__\n"
+    ...             "setup(\n"
+    ...             " name='tellmy.version',\n"
+    ...             " package_dir = {'': 'src'},\n"
+    ...             " packages = ['tellmy', 'tellmy.version'],\n"
+    ...             " install_requires = ['setuptools'],\n"
+    ...             " namespace_packages=['tellmy'],\n"
+    ...             " zip_safe=True, version=__version__,\n"
+    ...             " author='bob', url='bob', author_email='bob')\n"
+    ...             )
+    ...         zc.buildout.testing.sdist(tmp, sample_eggs)
+    ...         if version == '1.0':
+    ...             # We install the 1.0 version in site packages the way a
+    ...             # system packaging system (debs, rpms) would do it.
+    ...             zc.buildout.testing.sys_install(tmp, site_packages_path)
+    ...     finally:
+    ...         shutil.rmtree(tmp)
+    >>> print call_py(
+    ...     py_path,
+    ...     "import tellmy.version; print tellmy.version.__version__")
+    1.0
+    <BLANKLINE>
+    >>> write('buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = eggs
+    ... find-links = %(sample_eggs)s
+    ...
+    ... [primed_python]
+    ... executable = %(py_path)s
+    ...
+    ... [eggs]
+    ... recipe = zc.recipe.egg:eggs
+    ... python = primed_python
+    ... eggs = tellmy.version == 1.1
+    ... ''' % globals())
+
+Before the bugfix, running this buildout would generate this error:
+
+    Installing eggs.
+    Getting distribution for 'tellmy.version==1.1'.
+    Installing tellmy.version 1.1
+    Caused installation of a distribution:
+    tellmy.version 1.0
+    with a different version.
+    Got None.
+    While:
+      Installing eggs.
+    Error: There is a version conflict.
+    We already have: tellmy.version 1.0
+    <BLANKLINE>
+
+You can see the copiously commented fix for this in easy_install.py (see
+zc.buildout.easy_install.Installer._call_easy_install and particularly
+the comment leading up to zc.buildout.easy_install._easy_install_cmd).
+Now the install works correctly, as seen here.
+
+    >>> print system(buildout)
+    Installing eggs.
+    Getting distribution for 'tellmy.version==1.1'.
+    Got tellmy.version 1.1.
+    <BLANKLINE>
+
+    """
+
+def isolated_include_site_packages():
+    """
+
+This is an isolated test of the include_site_packages functionality, passing
+the argument directly to install, overriding a default.
+
+Our "py_path" has the "demoneeded" and "demo" packages available.  We'll
+simply be asking for "demoneeded" here.
+
+    >>> py_path, site_packages_path = make_py()
+    >>> create_sample_sys_install(site_packages_path)
+    >>> zc.buildout.easy_install.include_site_packages(False)
+    True
+
+    >>> example_dest = tmpdir('site-packages-example-install')
+    >>> workingset = zc.buildout.easy_install.install(
+    ...     ['demoneeded'], example_dest, links=[], executable=py_path,
+    ...     index=None, include_site_packages=True)
+    >>> [dist.project_name for dist in workingset]
+    ['demoneeded']
+
+That worked fine.  Let's try again with site packages not allowed (and
+reversing the default).
+
+    >>> zc.buildout.easy_install.include_site_packages(True)
+    False
+
+    >>> zc.buildout.easy_install.clear_index_cache()
+    >>> rmdir(example_dest)
+    >>> example_dest = tmpdir('site-packages-example-install')
+    >>> workingset = zc.buildout.easy_install.install(
+    ...     ['demoneeded'], example_dest, links=[], executable=py_path,
+    ...     index=None, include_site_packages=False)
+    Traceback (most recent call last):
+        ...
+    MissingDistribution: Couldn't find a distribution for 'demoneeded'.
+
+That's a failure, as expected.
+
+Now we explore an important edge case.
+
+Some system Pythons include setuptools (and other Python packages) in their
+site-packages (or equivalent) using a .egg-info directory.  The pkg_resources
+module (from setuptools) considers a package installed using .egg-info to be a
+develop egg.
+
+zc.buildout.buildout.Buildout.bootstrap will make setuptools and zc.buildout
+available to the buildout via the eggs directory, for normal eggs; or the
+develop-eggs directory, for develop-eggs.
+
+If setuptools or zc.buildout is found in site-packages and considered by
+pkg_resources to be a develop egg, then the bootstrap code will use a .egg-link
+in the local develop-eggs, pointing to site-packages, in its entirety.  Because
+develop-eggs must always be available for searching for distributions, this
+indirectly brings site-packages back into the search path for distributions.
+
+Because of this, we have to take special care that we still exclude
+site-packages even in this case.  See the comments about site packages in the
+Installer._satisfied and Installer._obtain methods for the implementation
+(as of this writing).
+
+In this demonstration, we insert a link to the "demoneeded" distribution
+in our develop-eggs, which would bring the package back in, except for
+the special care we have taken to exclude it.
+
+    >>> zc.buildout.easy_install.clear_index_cache()
+    >>> rmdir(example_dest)
+    >>> example_dest = tmpdir('site-packages-example-install')
+    >>> mkdir(example_dest, 'develop-eggs')
+    >>> write(example_dest, 'develop-eggs', 'demoneeded.egg-link',
+    ...       site_packages_path)
+    >>> workingset = zc.buildout.easy_install.install(
+    ...     ['demoneeded'], example_dest, links=[],
+    ...     path=[join(example_dest, 'develop-eggs')],
+    ...     executable=py_path,
+    ...     index=None, include_site_packages=False)
+    Traceback (most recent call last):
+        ...
+    MissingDistribution: Couldn't find a distribution for 'demoneeded'.
+
+The MissingDistribution error shows that buildout correctly excluded the
+"site-packages" source even though it was indirectly included in the path
+via a .egg-link file.
+
+    """
+
+def allowed_eggs_from_site_packages():
+    """
+Sometimes you need or want to control what eggs from site-packages are used.
+The allowed-eggs-from-site-packages option allows you to specify a whitelist of
+project names that may be included from site-packages.  You can use globs to
+specify the value.  It defaults to a single value of '*', indicating that any
+package may come from site-packages.
+
+This option interacts with include-site-packages in the following ways.
+
+If include-site-packages is true, then allowed-eggs-from-site-packages filters
+what eggs from site-packages may be chosen.  If allowed-eggs-from-site-packages
+is an empty list, then no eggs from site-packages are chosen, but site-packages
+will still be included at the end of path lists.
+
+If include-site-packages is false, allowed-eggs-from-site-packages is
+irrelevant.
+
+This test shows the interaction with the zc.buildout.easy_install API.  Another
+test below (allow_site_package_eggs_option) shows using it with a buildout.cfg.
+
+Our "py_path" has the "demoneeded" and "demo" packages available.  We'll
+simply be asking for "demoneeded" here.
+
+    >>> py_path, site_packages_path = make_py()
+    >>> create_sample_sys_install(site_packages_path)
+
+    >>> example_dest = tmpdir('site-packages-example-install')
+    >>> workingset = zc.buildout.easy_install.install(
+    ...     ['demoneeded'], example_dest, links=[], executable=py_path,
+    ...     index=None,
+    ...     allowed_eggs_from_site_packages=['demoneeded', 'other'])
+    >>> [dist.project_name for dist in workingset]
+    ['demoneeded']
+
+That worked fine.  It would work fine for a glob too.
+
+    >>> zc.buildout.easy_install.clear_index_cache()
+    >>> rmdir(example_dest)
+    >>> example_dest = tmpdir('site-packages-example-install')
+    >>> workingset = zc.buildout.easy_install.install(
+    ...     ['demoneeded'], example_dest, links=[], executable=py_path,
+    ...     index=None,
+    ...     allowed_eggs_from_site_packages=['?emon*', 'other'])
+    >>> [dist.project_name for dist in workingset]
+    ['demoneeded']
+
+But now let's try again with 'demoneeded' not allowed.
+
+    >>> zc.buildout.easy_install.clear_index_cache()
+    >>> rmdir(example_dest)
+    >>> example_dest = tmpdir('site-packages-example-install')
+    >>> workingset = zc.buildout.easy_install.install(
+    ...     ['demoneeded'], example_dest, links=[], executable=py_path,
+    ...     index=None,
+    ...     allowed_eggs_from_site_packages=['demo'])
+    Traceback (most recent call last):
+        ...
+    MissingDistribution: Couldn't find a distribution for 'demoneeded'.
+
+Here's the same, but with an empty list.
+
+    >>> zc.buildout.easy_install.clear_index_cache()
+    >>> rmdir(example_dest)
+    >>> example_dest = tmpdir('site-packages-example-install')
+    >>> workingset = zc.buildout.easy_install.install(
+    ...     ['demoneeded'], example_dest, links=[], executable=py_path,
+    ...     index=None,
+    ...     allowed_eggs_from_site_packages=[])
+    Traceback (most recent call last):
+        ...
+    MissingDistribution: Couldn't find a distribution for 'demoneeded'.
+
+Of course, this doesn't stop us from getting a package from elsewhere.  Here,
+we add a link server.
+
+    >>> zc.buildout.easy_install.clear_index_cache()
+    >>> rmdir(example_dest)
+    >>> example_dest = tmpdir('site-packages-example-install')
+    >>> workingset = zc.buildout.easy_install.install(
+    ...     ['demoneeded'], example_dest, executable=py_path,
+    ...     links=[link_server], index=link_server+'index/',
+    ...     allowed_eggs_from_site_packages=['other'])
+    >>> [dist.project_name for dist in workingset]
+    ['demoneeded']
+    >>> [dist.location for dist in workingset]
+    ['/site-packages-example-install/demoneeded-1.1-py2.6.egg']
+
+Finally, here's an example of an interaction: we say that it is OK to
+allow the "demoneeded" egg to come from site-packages, but we don't
+include-site-packages.
+
+    >>> zc.buildout.easy_install.clear_index_cache()
+    >>> rmdir(example_dest)
+    >>> example_dest = tmpdir('site-packages-example-install')
+    >>> workingset = zc.buildout.easy_install.install(
+    ...     ['demoneeded'], example_dest, links=[], executable=py_path,
+    ...     index=None, include_site_packages=False,
+    ...     allowed_eggs_from_site_packages=['demoneeded'])
+    Traceback (most recent call last):
+        ...
+    MissingDistribution: Couldn't find a distribution for 'demoneeded'.
+
+    """
+
+def subprocesses_have_same_environment_by_default():
+    """
+The scripts generated by sitepackage_safe_scripts set the PYTHONPATH so that,
+if the environment is maintained (the default behavior), subprocesses get
+the same Python packages.
+
+First, we set up a script and an interpreter.
+
+    >>> interpreter_dir = tmpdir('interpreter')
+    >>> interpreter_parts_dir = os.path.join(
+    ...     interpreter_dir, 'parts', 'interpreter')
+    >>> interpreter_bin_dir = os.path.join(interpreter_dir, 'bin')
+    >>> mkdir(interpreter_bin_dir)
+    >>> mkdir(interpreter_dir, 'eggs')
+    >>> mkdir(interpreter_dir, 'parts')
+    >>> mkdir(interpreter_parts_dir)
+    >>> ws = zc.buildout.easy_install.install(
+    ...     ['demo'], join(interpreter_dir, 'eggs'), links=[link_server],
+    ...     index=link_server+'index/')
+    >>> test = (
+    ...     "import subprocess, sys; subprocess.call("
+    ...     "[sys.executable, '-c', "
+    ...     "'import eggrecipedemo; print eggrecipedemo.x'])")
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     reqs=['demo'], interpreter='py',
+    ...     script_initialization=test + '; sys.exit(0)')
+
+This works for the script.
+
+    >>> print system(join(interpreter_bin_dir, 'demo'))
+    3
+    <BLANKLINE>
+
+This also works for the generated interpreter.
+
+    >>> print call_py(join(interpreter_bin_dir, 'py'), test)
+    3
+    <BLANKLINE>
+
+If you have a PYTHONPATH in your environment, it will be honored, after
+the buildout-generated path.
+
+    >>> original_pythonpath = os.environ.get('PYTHONPATH')
+    >>> os.environ['PYTHONPATH'] = 'foo'
+    >>> test = (
+    ...     "import subprocess, sys; subprocess.call("
+    ...     "[sys.executable, '-c', "
+    ...     "'import sys, pprint; pprint.pprint(sys.path)'])")
+    >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts(
+    ...     interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir,
+    ...     reqs=['demo'], interpreter='py',
+    ...     script_initialization=test + '; sys.exit(0)')
+
+This works for the script.  As you can see, /sample_buildout/foo is included
+right after the "parts" directory that contains site.py and sitecustomize.py.
+You can also see, actually more easily than in the other example, that we
+have the desired eggs available.
+
+    >>> print system(join(interpreter_bin_dir, 'demo')), # doctest: +ELLIPSIS
+    ['',
+     '/interpreter/parts/interpreter',
+     '/sample-buildout/foo',
+     ...
+     '/interpreter/eggs/demo-0.3-pyN.N.egg',
+     '/interpreter/eggs/demoneeded-1.1-pyN.N.egg']
+
+This also works for the generated interpreter, with identical results.
+
+    >>> print call_py(join(interpreter_bin_dir, 'py'), test),
+    ... # doctest: +ELLIPSIS
+    ['',
+     '/interpreter/parts/interpreter',
+     '/sample-buildout/foo',
+     ...
+     '/interpreter/eggs/demo-0.3-pyN.N.egg',
+     '/interpreter/eggs/demoneeded-1.1-pyN.N.egg']
+
+    >>> # Cleanup
+    >>> if original_pythonpath:
+    ...     os.environ['PYTHONPATH'] = original_pythonpath
+    ... else:
+    ...     del os.environ['PYTHONPATH']
+    ...
+
+    """
+
+def bootstrap_makes_buildout_that_works_with_system_python():
+    r"""
+In order to work smoothly with a system Python, bootstrapping creates
+the buildout script with
+zc.buildout.easy_install.sitepackage_safe_scripts. If it did not, a
+variety of problems might happen.  For instance, if another version of
+buildout or setuptools is installed in the site-packages than is
+desired, it may cause a problem.
+
+A problem actually experienced in the field is when
+a recipe wants a different version of a dependency that is installed in
+site-packages.  We will create a similar situation, and show that it is now
+handled.
+
+First let's write a dummy recipe.
+
+    >>> mkdir(sample_buildout, 'recipes')
+    >>> write(sample_buildout, 'recipes', 'dummy.py',
+    ... '''
+    ... import logging, os, zc.buildout
+    ... class Dummy:
+    ...     def __init__(self, buildout, name, options):
+    ...         pass
+    ...     def install(self):
+    ...         return ()
+    ...     def update(self):
+    ...         pass
+    ... ''')
+    >>> write(sample_buildout, 'recipes', 'setup.py',
+    ... '''
+    ... from setuptools import setup
+    ...
+    ... setup(
+    ...     name = "recipes",
+    ...     entry_points = {'zc.buildout': ['dummy = dummy:Dummy']},
+    ...     install_requires = 'demoneeded==1.2c1',
+    ...     )
+    ... ''')
+    >>> write(sample_buildout, 'recipes', 'README.txt', " ")
+
+Now we'll try to use it with a Python that has a different version of
+demoneeded installed.
+
+    >>> py_path, site_packages_path = make_py()
+    >>> create_sample_sys_install(site_packages_path)
+    >>> rmdir('develop-eggs')
+    >>> from zc.buildout.testing import make_buildout
+    >>> make_buildout(executable=py_path)
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... develop = recipes
+    ... parts = dummy
+    ... find-links = %(link_server)s
+    ... executable = %(py_path)s
+    ...
+    ... [dummy]
+    ... recipe = recipes:dummy
+    ... ''' % globals())
+
+Now we actually run the buildout.  Before the change, we got the following
+error:
+
+    Develop: '/sample-buildout/recipes'
+    While:
+      Installing.
+      Getting section dummy.
+      Initializing section dummy.
+      Installing recipe recipes.
+    Error: There is a version conflict.
+    We already have: demoneeded 1.1
+    but recipes 0.0.0 requires 'demoneeded==1.2c1'.
+
+Now, it is handled smoothly.
+
+    >>> print system(buildout)
+    Develop: '/sample-buildout/recipes'
+    Getting distribution for 'demoneeded==1.2c1'.
+    Got demoneeded 1.2c1.
+    Installing dummy.
+    <BLANKLINE>
+
+Here's the same story with a namespace package, which has some additional
+complications behind the scenes.  First, a recipe, in the "tellmy" namespace.
+
+    >>> mkdir(sample_buildout, 'ns')
+    >>> mkdir(sample_buildout, 'ns', 'tellmy')
+    >>> write(sample_buildout, 'ns', 'tellmy', '__init__.py',
+    ...       "__import__('pkg_resources').declare_namespace(__name__)\n")
+    >>> mkdir(sample_buildout, 'ns', 'tellmy', 'recipes')
+    >>> write(sample_buildout, 'ns', 'tellmy', 'recipes', '__init__.py', ' ')
+    >>> write(sample_buildout, 'ns', 'tellmy', 'recipes', 'dummy.py',
+    ... '''
+    ... import logging, os, zc.buildout
+    ... class Dummy:
+    ...     def __init__(self, buildout, name, options):
+    ...         pass
+    ...     def install(self):
+    ...         return ()
+    ...     def update(self):
+    ...         pass
+    ... ''')
+    >>> write(sample_buildout, 'ns', 'setup.py',
+    ... '''
+    ... from setuptools import setup
+    ... setup(
+    ...     name="tellmy.recipes",
+    ...     packages=['tellmy', 'tellmy.recipes'],
+    ...     install_requires=['setuptools'],
+    ...     namespace_packages=['tellmy'],
+    ...     entry_points = {'zc.buildout':
+    ...                     ['dummy = tellmy.recipes.dummy:Dummy']},
+    ...     )
+    ... ''')
+
+Now, a buildout that uses it.
+
+    >>> create_sample_namespace_eggs(sample_eggs, site_packages_path)
+    >>> rmdir('develop-eggs')
+    >>> from zc.buildout.testing import make_buildout
+    >>> make_buildout(executable=py_path)
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... develop = ns
+    ...           recipes
+    ... parts = dummy
+    ... find-links = %(link_server)s
+    ... executable = %(py_path)s
+    ...
+    ... [dummy]
+    ... recipe = tellmy.recipes:dummy
+    ... ''' % globals())
+
+Now we actually run the buildout.
+
+    >>> print system(buildout)
+    Develop: '/sample-buildout/ns'
+    Develop: '/sample-buildout/recipes'
+    Uninstalling dummy.
+    Installing dummy.
+    <BLANKLINE>
+
+    """
+
 if sys.version_info > (2, 4):
     def test_exit_codes():
         """
@@ -2367,6 +3085,7 @@ Distribution setup scripts can import modules in the distribution directory:
 
     >>> ls('develop-eggs')
     -  foo.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
     """
@@ -2385,6 +3104,13 @@ honoring our version specification.
     >>> write('foo.py', '')
     >>> _ = system(buildout+' setup . sdist')
 
+    >>> if zc.buildout.easy_install.is_distribute:
+    ...     distribute_version = 'distribute = %s' % (
+    ...         pkg_resources.working_set.find(
+    ...             pkg_resources.Requirement.parse('distribute')).version,)
+    ... else:
+    ...     distribute_version = ''
+    ...
     >>> write('buildout.cfg',
     ... '''
     ... [buildout]
@@ -2396,12 +3122,14 @@ honoring our version specification.
     ... [versions]
     ... setuptools = %s
     ... foo = 1
+    ... %s
     ...
     ... [foo]
     ... recipe = zc.recipe.egg
     ... eggs = foo
-    ... ''' % pkg_resources.working_set.find(
-    ...    pkg_resources.Requirement.parse('setuptools')).version)
+    ... ''' % (pkg_resources.working_set.find(
+    ...         pkg_resources.Requirement.parse('setuptools')).version,
+    ...        distribute_version))
 
     >>> print system(buildout),
     Installing foo.
@@ -2654,24 +3382,100 @@ def increment_on_command_line():
 
 ######################################################################
 
+def make_py_with_system_install(make_py, sample_eggs):
+    py_path, site_packages_path = make_py()
+    create_sample_namespace_eggs(sample_eggs, site_packages_path)
+    return py_path
+
+def create_sample_namespace_eggs(dest, site_packages_path=None):
+    from zc.buildout.testing import write, mkdir
+    for pkg, version in (('version', '1.0'), ('version', '1.1'),
+                         ('fortune', '1.0')):
+        tmp = tempfile.mkdtemp()
+        try:
+            write(tmp, 'README.txt', '')
+            mkdir(tmp, 'src')
+            mkdir(tmp, 'src', 'tellmy')
+            write(tmp, 'src', 'tellmy', '__init__.py',
+                "__import__("
+                "'pkg_resources').declare_namespace(__name__)\n")
+            mkdir(tmp, 'src', 'tellmy', pkg)
+            write(tmp, 'src', 'tellmy', pkg,
+                  '__init__.py', '__version__=%r\n' % version)
+            write(
+                tmp, 'setup.py',
+                "from setuptools import setup\n"
+                "setup(\n"
+                " name='tellmy.%(pkg)s',\n"
+                " package_dir = {'': 'src'},\n"
+                " packages = ['tellmy', 'tellmy.%(pkg)s'],\n"
+                " install_requires = ['setuptools'],\n"
+                " namespace_packages=['tellmy'],\n"
+                " zip_safe=True, version=%(version)r,\n"
+                " author='bob', url='bob', author_email='bob')\n"
+                % locals()
+                )
+            zc.buildout.testing.sdist(tmp, dest)
+            if (site_packages_path and pkg == 'version' and version == '1.1'):
+                # We install the 1.1 version in site packages the way a
+                # system packaging system (debs, rpms) would do it.
+                zc.buildout.testing.sys_install(tmp, site_packages_path)
+        finally:
+            shutil.rmtree(tmp)
+
+def _write_eggrecipedemoneeded(tmp, minor_version, suffix=''):
+    from zc.buildout.testing import write
+    write(tmp, 'README.txt', '')
+    write(tmp, 'eggrecipedemoneeded.py',
+          'y=%s\ndef f():\n  pass' % minor_version)
+    write(
+        tmp, 'setup.py',
+        "from setuptools import setup\n"
+        "setup(name='demoneeded', py_modules=['eggrecipedemoneeded'],"
+        " zip_safe=True, version='1.%s%s', author='bob', url='bob', "
+        "author_email='bob')\n"
+        % (minor_version, suffix)
+        )
+
+def _write_eggrecipedemo(tmp, minor_version, suffix=''):
+    from zc.buildout.testing import write
+    write(tmp, 'README.txt', '')
+    write(
+        tmp, 'eggrecipedemo.py',
+        'import eggrecipedemoneeded\n'
+        'x=%s\n'
+        'def main(): print x, eggrecipedemoneeded.y\n'
+        % minor_version)
+    write(
+        tmp, 'setup.py',
+        "from setuptools import setup\n"
+        "setup(name='demo', py_modules=['eggrecipedemo'],"
+        " install_requires = 'demoneeded',"
+        " entry_points={'console_scripts': "
+             "['demo = eggrecipedemo:main']},"
+        " zip_safe=True, version='0.%s%s')\n" % (minor_version, suffix)
+        )
+
+def create_sample_sys_install(site_packages_path):
+    for creator, minor_version in (
+        (_write_eggrecipedemoneeded, 1),
+        (_write_eggrecipedemo, 3)):
+        # Write the files and install in site_packages_path.
+        tmp = tempfile.mkdtemp()
+        try:
+            creator(tmp, minor_version)
+            zc.buildout.testing.sys_install(tmp, site_packages_path)
+        finally:
+            shutil.rmtree(tmp)
+
 def create_sample_eggs(test, executable=sys.executable):
-    write = test.globs['write']
+    from zc.buildout.testing import write
     dest = test.globs['sample_eggs']
     tmp = tempfile.mkdtemp()
     try:
-        write(tmp, 'README.txt', '')
-
         for i in (0, 1, 2):
-            write(tmp, 'eggrecipedemoneeded.py', 'y=%s\ndef f():\n  pass' % i)
-            c1 = i==2 and 'c1' or ''
-            write(
-                tmp, 'setup.py',
-                "from setuptools import setup\n"
-                "setup(name='demoneeded', py_modules=['eggrecipedemoneeded'],"
-                " zip_safe=True, version='1.%s%s', author='bob', url='bob', "
-                "author_email='bob')\n"
-                % (i, c1)
-                )
+            suffix = i==2 and 'c1' or ''
+            _write_eggrecipedemoneeded(tmp, i, suffix)
             zc.buildout.testing.sdist(tmp, dest)
 
         write(
@@ -2685,22 +3489,8 @@ def create_sample_eggs(test, executable=sys.executable):
         os.remove(os.path.join(tmp, 'eggrecipedemoneeded.py'))
 
         for i in (1, 2, 3, 4):
-            write(
-                tmp, 'eggrecipedemo.py',
-                'import eggrecipedemoneeded\n'
-                'x=%s\n'
-                'def main(): print x, eggrecipedemoneeded.y\n'
-                % i)
-            c1 = i==4 and 'c1' or ''
-            write(
-                tmp, 'setup.py',
-                "from setuptools import setup\n"
-                "setup(name='demo', py_modules=['eggrecipedemo'],"
-                " install_requires = 'demoneeded',"
-                " entry_points={'console_scripts': "
-                     "['demo = eggrecipedemo:main']},"
-                " zip_safe=True, version='0.%s%s')\n" % (i, c1)
-                )
+            suffix = i==4 and 'c1' or ''
+            _write_eggrecipedemo(tmp, i, suffix)
             zc.buildout.testing.bdist_egg(tmp, executable, dest)
 
         write(tmp, 'eggrecipebigdemo.py', 'import eggrecipedemo')
@@ -2776,6 +3566,7 @@ def easy_install_SetUp(test):
         test.globs['sample_eggs'])
     test.globs['update_extdemo'] = lambda : add_source_dist(test, 1.5)
     zc.buildout.testing.install_develop('zc.recipe.egg', test)
+    zc.buildout.testing.install_develop('z3c.recipe.scripts', test)
 
 egg_parse = re.compile('([0-9a-zA-Z_.]+)-([0-9a-zA-Z_.]+)-py(\d[.]\d).egg$'
                        ).match
@@ -2818,7 +3609,8 @@ def updateSetup(test):
         here = os.getcwd()
         os.chdir(os.path.dirname(dist.location))
         assert os.spawnle(
-            os.P_WAIT, sys.executable, zc.buildout.easy_install._safe_arg (sys.executable),
+            os.P_WAIT, sys.executable,
+            zc.buildout.easy_install._safe_arg(sys.executable),
             os.path.join(os.path.dirname(dist.location), 'setup.py'),
             '-q', 'bdist_egg', '-d', eggs,
             dict(os.environ,
@@ -2841,11 +3633,13 @@ def updateSetup(test):
 
     # now let's make the new releases
     makeNewRelease('zc.buildout', ws, new_releases)
-    makeNewRelease('setuptools', ws, new_releases)
-
     os.mkdir(os.path.join(new_releases, 'zc.buildout'))
-    os.mkdir(os.path.join(new_releases, 'setuptools'))
-
+    if zc.buildout.easy_install.is_distribute:
+        makeNewRelease('distribute', ws, new_releases)
+        os.mkdir(os.path.join(new_releases, 'distribute'))
+    else:
+        makeNewRelease('setuptools', ws, new_releases)
+        os.mkdir(os.path.join(new_releases, 'setuptools'))
 
 
 normalize_bang = (
@@ -2869,7 +3663,8 @@ def test_suite():
                 '__buildout_signature__ = recipes-SSSSSSSSSSS'),
                (re.compile('executable = [\S ]+python\S*', re.I),
                 'executable = python'),
-               (re.compile('[-d]  setuptools-\S+[.]egg'), 'setuptools.egg'),
+               (re.compile('[-d]  (setuptools|distribute)-\S+[.]egg'),
+                'setuptools.egg'),
                (re.compile('zc.buildout(-\S+)?[.]egg(-link)?'),
                 'zc.buildout.egg'),
                (re.compile('creating \S*setup.cfg'), 'creating setup.cfg'),
@@ -2884,6 +3679,7 @@ def test_suite():
                            r'when that file already exists: '),
                 '[Errno 17] File exists: '
                 ),
+               (re.compile('distribute'), 'setuptools'),
                ])
             ),
         doctest.DocFileSuite(
@@ -2913,9 +3709,18 @@ def test_suite():
                (re.compile('(zc.buildout|setuptools)-\d+[.]\d+\S*'
                            '-py\d.\d.egg'),
                 '\\1.egg'),
+               (re.compile('distribute-\d+[.]\d+\S*'
+                           '-py\d.\d.egg'),
+                'setuptools.egg'),
                (re.compile('(zc.buildout|setuptools)( version)? \d+[.]\d+\S*'),
                 '\\1 V.V'),
-               (re.compile('[-d]  setuptools'), '-  setuptools'),
+               (re.compile('distribute( version)? \d+[.]\d+\S*'),
+                'setuptools V.V'),
+               (re.compile('[-d]  (setuptools|distribute)'), '-  setuptools'),
+               (re.compile('distribute'), 'setuptools'),
+               (re.compile("\nUnused options for buildout: "
+                           "'(distribute|setuptools)\-version'\."),
+                '')
                ])
             ),
 
@@ -2931,9 +3736,17 @@ def test_suite():
                zc.buildout.testing.normalize_egg_py,
                normalize_bang,
                (re.compile('extdemo[.]pyd'), 'extdemo.so'),
-               (re.compile('[-d]  setuptools-\S+[.]egg'), 'setuptools.egg'),
+               (re.compile('[-d]  (setuptools|distribute)-\S+[.]egg'),
+                'setuptools.egg'),
                (re.compile(r'\\[\\]?'), '/'),
                (re.compile(r'\#!\S+\bpython\S*'), '#!/usr/bin/python'),
+               # Normalize generate_script's Windows interpreter to UNIX:
+               (re.compile(r'\nimport subprocess\n'), '\n'),
+               (re.compile('subprocess\\.call\\(argv, env=environ\\)'),
+                'os.execve(sys.executable, argv, environ)'),
+               (re.compile('distribute'), 'setuptools'),
+               # Distribute unzips eggs by default.
+               (re.compile('\-  demoneeded'), 'd  demoneeded'),
                ]+(sys.version_info < (2, 5) and [
                   (re.compile('.*No module named runpy.*', re.S), ''),
                   (re.compile('.*usage: pdb.py scriptfile .*', re.S), ''),
@@ -2965,7 +3778,7 @@ def test_suite():
                zc.buildout.testing.normalize_egg_py,
                (re.compile("buildout: Running \S*setup.py"),
                 'buildout: Running setup.py'),
-               (re.compile('setuptools-\S+-'),
+               (re.compile('(setuptools|distribute)-\S+-'),
                 'setuptools.egg'),
                (re.compile('zc.buildout-\S+-'),
                 'zc.buildout.egg'),
@@ -2973,7 +3786,7 @@ def test_suite():
                 'File "one.py"'),
                (re.compile(r'We have a develop egg: (\S+) (\S+)'),
                 r'We have a develop egg: \1 V'),
-               (re.compile('Picked: setuptools = \S+'),
+               (re.compile('Picked: (setuptools|distribute) = \S+'),
                 'Picked: setuptools = V'),
                (re.compile(r'\\[\\]?'), '/'),
                (re.compile(
@@ -2981,6 +3794,12 @@ def test_suite():
                    '-q develop -mxN -d /sample-buildout/develop-eggs'
                 ),
                (re.compile(r'^[*]...'), '...'),
+               # for bug_92891_bootstrap_crashes_with_egg_recipe_in_buildout_section
+               (re.compile(r"Unused options for buildout: 'eggs' 'scripts'\."),
+                "Unused options for buildout: 'scripts' 'eggs'."),
+               (re.compile('distribute'), 'setuptools'),
+               # Distribute unzips eggs by default.
+               (re.compile('\-  demoneeded'), 'd  demoneeded'),
                ]),
             ),
         zc.buildout.testselectingpython.test_suite(),
@@ -3040,6 +3859,8 @@ def test_suite():
                zc.buildout.testing.normalize_script,
                normalize_bang,
                (re.compile('Downloading.*setuptools.*egg\n'), ''),
+               (re.compile('options:'), 'Options:'),
+               (re.compile('usage:'), 'Usage:'),
                ]),
             ))
 

src/zc/buildout/testselectingpython.py

@@ -11,7 +11,7 @@
 # FOR A PARTICULAR PURPOSE.
 #
 ##############################################################################
-import os, re, sys, unittest
+import os, re, subprocess, sys, textwrap, unittest
 from zope.testing import doctest, renormalizing
 import zc.buildout.tests
 import zc.buildout.testing
@@ -42,6 +42,33 @@ We can specify a specific Python executable.
 
 def multi_python(test):
     other_executable = zc.buildout.testing.find_python(other_version)
+    command = textwrap.dedent('''\
+        try:
+            import setuptools
+        except ImportError:
+            import sys
+            sys.exit(1)
+        ''')
+    if subprocess.call([other_executable, '-c', command],
+                       env=os.environ):
+        # the other executable does not have setuptools.  Get setuptools.
+        # We will do this using the same tools we are testing, for better or
+        # worse.  Alternatively, we could try using bootstrap.
+        executable_dir = test.globs['tmpdir']('executable_dir')
+        executable_parts = os.path.join(executable_dir, 'parts')
+        test.globs['mkdir'](executable_parts)
+        ws = zc.buildout.easy_install.install(
+            ['setuptools'], executable_dir,
+            index='http://www.python.org/pypi/',
+            always_unzip=True, executable=other_executable)
+        zc.buildout.easy_install.sitepackage_safe_scripts(
+            executable_dir, ws, other_executable, executable_parts,
+            reqs=['setuptools'], interpreter='py')
+        original_executable = other_executable
+        other_executable = os.path.join(executable_dir, 'py')
+        assert not subprocess.call(
+            [other_executable, '-c', command], env=os.environ), (
+            'test set up failed')
     sample_eggs = test.globs['tmpdir']('sample_eggs')
     os.mkdir(os.path.join(sample_eggs, 'index'))
     test.globs['sample_eggs'] = sample_eggs

src/zc/buildout/update.txt

@@ -78,25 +78,30 @@ new versions found in new releases:
     zc.buildout 99.99
     setuptools 99.99
 
-Our buildout script has been updated to use the new eggs:
+Our buildout script's site.py has been updated to use the new eggs:
 
-    >>> cat(sample_buildout, 'bin', 'buildout')
-    #!/usr/local/bin/python2.4
+    >>> cat(sample_buildout, 'parts', 'buildout', 'site.py')
+    ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
+    "...
+    def addsitepackages(known_paths):
+        """Add site packages, as determined by zc.buildout.
     <BLANKLINE>
-    import sys
-    sys.path[0:0] = [
-      '/sample-buildout/eggs/zc.buildout-99.99-py2.4.egg',
-      '/sample-buildout/eggs/setuptools-99.99-py2.4.egg',
-      ]
-    <BLANKLINE>
-    import zc.buildout.buildout
-    <BLANKLINE>
-    if __name__ == '__main__':
-        zc.buildout.buildout.main()
+        See original_addsitepackages, below, for the original version."""
+        buildout_paths = [
+            '/sample-buildout/eggs/zc.buildout-99.99-pyN.N.egg',
+            '/sample-buildout/eggs/setuptools-99.99-pyN.N.egg'
+            ]
+        for path in buildout_paths:
+            sitedir, sitedircase = makepath(path)
+            if not sitedircase in known_paths and os.path.exists(sitedir):
+                sys.path.append(sitedir)
+                known_paths.add(sitedircase)
+        return known_paths
+    ...
 
 Now, let's recreate the sample buildout. If we specify constraints on
-the versions of zc.buildout and setuptools to use, running the
-buildout will install earlier versions of these packages:
+the versions of zc.buildout and setuptools (or distribute) to use,
+running the buildout will install earlier versions of these packages:
 
     >>> write(sample_buildout, 'buildout.cfg',
     ... """
@@ -107,6 +112,7 @@ buildout will install earlier versions of these packages:
     ... develop = showversions
     ... zc.buildout-version = < 99
     ... setuptools-version = < 99
+    ... distribute-version = < 99
     ...
     ... [show-versions]
     ... recipe = showversions
@@ -119,7 +125,6 @@ Now we can see that we actually "upgrade" to an earlier version.
       zc.buildout version 1.0.0,
       setuptools version 0.6;
     restarting.
-    Generated script '/sample-buildout/bin/buildout'.
     Develop: '/sample-buildout/showversions'
     Updating show-versions.
     zc.buildout 1.0.0

z3c.recipe.scripts_/CHANGES.txt

+Change History
+**************
+
+1.0.0
+=====
+
+Initial public version.

z3c.recipe.scripts_/README.txt

+********************************
+Buildout Script Recipe
+********************************
+
+.. contents::
+
+The script recipe installs eggs into a buildout eggs directory, exactly
+like zc.recipe.egg, and then generates scripts in a buildout bin
+directory with egg paths baked into them.
+

z3c.recipe.scripts_/setup.py

+##############################################################################
+#
+# Copyright (c) 2007 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Setup for z3c.recipe.scripts package
+
+$Id: setup.py 106736 2009-12-18 02:33:08Z gary $
+"""
+
+version = '1.0.0dev'
+
+import os
+from setuptools import setup, find_packages
+
+def read(*rnames):
+    return open(os.path.join(os.path.dirname(__file__), *rnames)).read()
+
+name = "z3c.recipe.scripts"
+setup(
+    name = name,
+    version = version,
+    author = "Gary Poster",
+    author_email = "gary.poster@canonical.com",
+    description = "Recipe for installing Python scripts",
+    long_description = (
+        read('README.txt')
+        + '\n' +
+        read('CHANGES.txt')
+        + '\n' +
+        'Detailed Documentation\n'
+        '**********************\n'
+        + '\n' +
+        read('src', 'z3c', 'recipe', 'scripts', 'README.txt')
+        + '\n' +
+        'Download\n'
+        '*********\n'
+        ),
+    keywords = "development build",
+    classifiers = [
+       'Development Status :: 5 - Production/Stable',
+       'Framework :: Buildout',
+       'Intended Audience :: Developers',
+       'License :: OSI Approved :: Zope Public License',
+       'Topic :: Software Development :: Build Tools',
+       'Topic :: Software Development :: Libraries :: Python Modules',
+       ],
+    url='http://cheeseshop.python.org/pypi/z3c.recipe.scripts',
+    license = "ZPL 2.1",
+
+    packages = find_packages('src'),
+    package_dir = {'':'src'},
+    namespace_packages = ['z3c', 'z3c.recipe'],
+    install_requires = [
+        'zc.buildout >=1.5.0dev',
+        'zc.recipe.egg >=1.2.3dev',
+        'setuptools'],
+    tests_require = ['zope.testing'],
+    test_suite = name+'.tests.test_suite',
+    entry_points = {'zc.buildout': ['default = %s:Scripts' % name,
+                                    'script = %s:Scripts' % name,
+                                    'scripts = %s:Scripts' % name,
+                                    'interpreter = %s:Interpreter' % name,
+                                    ]
+                    },
+    include_package_data = True,
+    zip_safe=False,
+    )

z3c.recipe.scripts_/src/z3c/__init__.py

+__import__('pkg_resources').declare_namespace(__name__)

z3c.recipe.scripts_/src/z3c/recipe/__init__.py

+__import__('pkg_resources').declare_namespace(__name__)

z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt

+Script and interpreter generation
+=================================
+
+This recipe is very similar to zc.recipe.egg, and if you are familiar with its
+options, you will be able to use this one easily.
+
+The script and interpreter generation in this recipe are improved from
+those provided by zc.recipe.egg in two basic ways.
+
+- The interpreter generated by the script supports all interpreter
+  options, as opposed to the subset provided by zc.recipe.egg.
+
+- Both scripts and interpreters from this recipe can optionally choose
+  to include site-packages, and even sitecustomize.
+
+The recipe takes several options.  First, here's the list of the options
+that overlap from the standard zc.recipe.eggs scripts recipe.  After
+this, we'll list the new options and describe them.
+
+* eggs
+* find-links
+* index
+* python
+* extra-paths
+* entry-points
+* scripts
+* dependent-scripts
+* interpreter
+* arguments
+* initialization
+* relative-paths
+
+In addition to these, the recipe offers these new options.  They are
+introduced here, and described more in depth below.
+
+include-site-packages
+    You can choose to have the site-packages of the underlying Python
+    available to your script or interpreter, in addition to the packages
+    from your eggs.  See the section on this option for motivations and
+    warnings.
+
+allowed-eggs-from-site-packages
+    Sometimes you need or want to control what eggs from site-packages are
+    used. The allowed-eggs-from-site-packages option allows you to specify a
+    whitelist of project names that may be included from site-packages.  You
+    can use globs to specify the value.  It defaults to a single value of '*',
+    indicating that any package may come from site-packages.
+
+    Here's a usage example::
+
+        [buildout]
+        ...
+
+        allowed-eggs-from-site-packages =
+            demo
+            bigdemo
+            zope.*
+
+    This option interacts with the ``include-site-packages`` option in the
+    following ways.
+
+    If ``include-site-packages`` is true, then
+    ``allowed-eggs-from-site-packages`` filters what eggs from site-packages
+    may be chosen.  Therefore, if ``allowed-eggs-from-site-packages`` is an
+    empty list, then no eggs from site-packages are chosen, but site-packages
+    will still be included at the end of path lists.
+
+    If ``include-site-packages`` is false, the value of
+    ``allowed-eggs-from-site-packages`` is irrelevant.
+
+extends
+    You can extend another section using this value.  It is intended to be
+    used by extending a section that uses this package's scripts recipe.
+    In this manner, you can avoid repeating yourself.
+
+exec-sitecustomize
+    Normally the Python's real sitecustomize module is not processed.
+    If you want it to be processed, set this value to 'true'.  This will
+    be honored irrespective of the setting for include-site-packages.
+
+script-initialization
+    The standard initialization code affects both an interpreter and scripts.
+    The code in script-initialization is used only for the generated scripts.
+
+Finally, the "interpreter" entry point ignores ``script-initialization``,
+``scripts``, and ``arguments``, and provides yet another additional option.
+
+name
+    While, by default, the interpreter recipe takes the name of the
+    section to be the desired interpreter name, you can specify the
+    interpreter name here instead.
+
+Script generation
+-----------------
+
+Generating a basic script looks virtually identical to using zc.recipe.egg.
+
+(Note that the find-links and index values are typically not needed; they
+are included to help make this document run as a test successfully.)
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = demo
+    ...
+    ... [demo]
+    ... recipe = z3c.recipe.scripts
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... """ % dict(server=link_server))
+
+    >>> print system(buildout),
+    Installing demo.
+    Getting distribution for 'demo<0.3'.
+    Got demo 0.2.
+    Getting distribution for 'demoneeded'.
+    Got demoneeded 1.2c1.
+    Generated script '/sample-buildout/bin/demo'.
+
+    >>> print system(join(sample_buildout, 'bin', 'demo')),
+    2 2
+
+Interpreter generation
+----------------------
+
+As with zc.recipe.egg, you can generate an interpreter with the default
+script recipe shown above by supplying the "interpreter" option.
+This example will create both an entry point script and an interpreter.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = demo
+    ...
+    ... [demo]
+    ... recipe = z3c.recipe.scripts
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... interpreter = py
+    ... """ % dict(server=link_server))
+
+    >>> print system(buildout),
+    Uninstalling demo.
+    Installing demo.
+    Generated script '/sample-buildout/bin/demo'.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+You can also generate an interpreter alone with the ``interpreter`` recipe.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... """ % dict(server=link_server))
+
+    >>> print system(buildout),
+    Uninstalling demo.
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+In both cases, the bin/py script works by restarting Python after
+specifying a special path in PYTHONPATH.  This example shows the UNIX version;
+the Windows version actually uses subprocess instead.
+
+    >>> cat(sample_buildout, 'bin', 'py') # doctest: +NORMALIZE_WHITESPACE
+    #!/usr/bin/python2.4 -S
+    <BLANKLINE>
+    import os
+    import sys
+    <BLANKLINE>
+    argv = [sys.executable] + sys.argv[1:]
+    environ = os.environ.copy()
+    path = '/sample-buildout/parts/py'
+    if environ.get('PYTHONPATH'):
+        path = os.pathsep.join([path, environ['PYTHONPATH']])
+    environ['PYTHONPATH'] = path
+    os.execve(sys.executable, argv, environ)
+
+The path is a directory that contains two files: our own site.py and
+sitecustomize.py.  The site.py is modified from the underlying Python's
+site.py, and is responsible for setting up our paths. The
+sitecustomize.py is responsible for running the initialization code
+provided.
+
+    >>> ls(sample_buildout, 'parts', 'py')
+    -  site.py
+    -  sitecustomize.py
+
+Here's an example of using the generated interpreter.
+
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ' -c "import sys, pprint; pprint.pprint(sys.path[-2:])"')
+    ['/sample-buildout/eggs/demo-0.2-pyN.N.egg',
+     '/sample-buildout/eggs/demoneeded-1.2c1-pyN.N.egg']
+    <BLANKLINE>
+
+Including site-packages and sitecustomize
+-----------------------------------------
+
+As introduced above, this recipe supports including site packages.  This has
+some advantages and some serious dangers.
+
+A typical reason to include site-packages is that it is easier to
+install one or more dependencies in your Python than it is with
+buildout.  Some packages, such as lxml or Python PostgreSQL integration,
+have dependencies that can be much easier to build and/or install using
+other mechanisms, such as your operating system's package manager.  By
+installing some core packages into your Python's site-packages, this can
+significantly simplify some application installations.
+
+However, doing this has a significant danger.  One of the primary goals
+of buildout is to provide repeatability.  Some packages (one of the
+better known Python openid packages, for instance) change their behavior
+depending on what packages are available.  If Python curl bindings are
+available, these may be preferred by the library.  If a certain XML
+package is installed, it may be preferred by the library.  These hidden
+choices may cause small or large behavior differences.  The fact that
+they can be rarely encountered can actually make it worse: you forget
+that this might be a problem, and debugging the differences can be
+difficult.  If you allow site-packages to be included in your buildout,
+and the Python you use is not managed precisely by your application (for
+instance, it is a system Python), you open yourself up to these
+possibilities.  Don't be unaware of the dangers.
+
+To show off these features, we need to use buildout with a Python
+executable with some extra paths to show ``include-site-packages``; and one
+guaranteed to have a sitecustomize module to show
+``exec-sitecustomize``.  We'll make one using a test fixture called
+``make_py``. The os.environ change below will go into the sitecustomize,
+and the site_packages_path will be in the Python's path.
+
+    >>> py_path, site_packages_path = make_py(initialization='''\
+    ... import os
+    ... os.environ['zc.buildout'] = 'foo bar baz shazam'
+    ... ''')
+    >>> print site_packages_path
+    /executable_buildout/site-packages
+
+Now let's take a look at include-site-packages.  The default is false,
+so we will set it to true.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = py
+    ... executable = %(py_path)s
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... include-site-packages = true
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... """ % dict(server=link_server, py_path=py_path))
+
+    >>> print system(buildout),
+    Uninstalling py.
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+Now executable_buildout/site-packages is included in sys.path.
+
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import sys, pprint; pprint.pprint(sys.path)"''')
+    ... # doctest: +ELLIPSIS
+    ['',
+     '/sample-buildout/parts/py',
+     ...,
+     '/sample-buildout/eggs/demo-0.2-pyN.N.egg',
+     '/sample-buildout/eggs/demoneeded-1.2c1-pyN.N.egg',
+     '/executable_buildout/eggs/setuptools-X-pyN.N.egg',
+     '/executable_buildout/site-packages']
+    <BLANKLINE>
+
+As described above, the allowed-eggs-from-site-packages option lets us
+control what site-packages eggs zc.buildout will allow to fulfill
+dependencies.  The behavior was described above with an example (and the
+implementation is tested elsewhere), so we'll only look at some simple and
+common use cases here.
+
+Sometimes you may want to allow site-packages to be available but you don't
+want your package to depend on it using setup.py.  For instance, perhaps you
+are writing an application, and you want to depend on your system's packaging
+of the PostgreSQL code, but the system Python does not use eggs to
+package it, so you need to manage the two separately.  In this case, you
+might not want to use any eggs from site-packages, but you want it available.
+In this case, you can use allowed-eggs-from-site-packages with an empty value
+to keep any egg from being used from site-packages.
+
+Here's an example.  Let's say we have a Python with demo and demoneeded
+installed as eggs in the system Python.  Normally, they will be used to
+fulfill dependencies, because allowed-eggs-from-site-packages defaults to
+the value "*" (allow any package).  (We use an empty find-links value to say
+that buildout may not look elsewhere for the package. We use a different
+eggs-directory for isolation, so that eggs obtained other parts of the
+document do not affect this example.)
+
+    >>> from zc.buildout.tests import create_sample_sys_install
+    >>> create_sample_sys_install(site_packages_path)
+    >>> import zc.buildout.easy_install
+    >>> zc.buildout.easy_install.clear_index_cache()
+
+    >>> write('buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = eggs
+    ... eggs-directory = tmpeggs
+    ... find-links =
+    ...
+    ... [primed_python]
+    ... executable = %(py_path)s
+    ...
+    ... [eggs]
+    ... recipe = z3c.recipe.scripts
+    ... include-site-packages = true
+    ... python = primed_python
+    ... eggs = demoneeded
+    ... ''' % globals())
+
+    >>> print system(buildout)
+    Creating directory '/sample-buildout/tmpeggs'.
+    Uninstalling py.
+    Installing eggs.
+    <BLANKLINE>
+
+That succeeds fine, getting demoneeded from the Python site-packages.
+
+However, when allowed-eggs-from-site-packages is an empty value, demoneeded
+is not allowed to come from site-packages, and the buildout fails.
+
+    >>> zc.buildout.easy_install.clear_index_cache()
+    >>> rmdir('tmpeggs')
+    >>> write('buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = eggs
+    ... eggs-directory = tmpeggs
+    ... find-links =
+    ...
+    ... [primed_python]
+    ... executable = %(py_path)s
+    ...
+    ... [eggs]
+    ... recipe = z3c.recipe.scripts
+    ... include-site-packages = true
+    ... allowed-eggs-from-site-packages =
+    ... eggs = demoneeded
+    ... ''' % globals())
+    >>> print system(buildout)
+    Creating directory '/sample-buildout/tmpeggs'.
+    Uninstalling eggs.
+    Installing eggs.
+    Couldn't find index page for 'demoneeded' (maybe misspelled?)
+    Getting distribution for 'demoneeded'.
+    While:
+      Installing eggs.
+      Getting distribution for 'demoneeded'.
+    Error: Couldn't find a distribution for 'demoneeded'.
+    <BLANKLINE>
+
+Remember that you can provide multiple lines to the
+allowed-eggs-from-site-packages option, each specifying a whitelist of
+allowed packages.  Globs (* and ?) are allowed.
+
+Next we will use the exec-sitecustomize option.  It simply copies
+Python's underlying sitecustomize module, if it exists, to the local
+version.  The os.environ change shown above in the make_py call will go
+into the sitecustomize.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = py
+    ... executable = %(py_path)s
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... exec-sitecustomize = true
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... """ % dict(server=link_server, py_path=py_path))
+
+    >>> print system(buildout),
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> cat(sample_buildout, 'parts', 'py', 'sitecustomize.py')
+    ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
+    <BLANKLINE>
+    # The following is from
+    # /executable_buildout/parts/py/sitecustomize.py
+    ...
+    import os
+    os.environ['zc.buildout'] = 'foo bar baz shazam'
+
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import os; print os.environ['zc.buildout']"''')
+    foo bar baz shazam
+    <BLANKLINE>
+
+Options
+-------
+
+We'll focus now on the remaining options that are different than
+zc.recipe.egg.
+
+Let's look at the ``extends`` option first.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = demo python
+    ...
+    ... [demo]
+    ... recipe = z3c.recipe.scripts
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ...
+    ... [python]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... extends = demo
+    ... initialization =
+    ...     import os
+    ...     os.environ['zc.buildout'] = 'foo bar baz shazam'
+    ... """ % dict(server=link_server))
+
+That makes it easier to specify some initialization for the interpreter
+that is different than a script, while duplicating other configuration.
+
+Now let's put it in action.
+
+    >>> print system(buildout),
+    Uninstalling py.
+    Installing demo.
+    Generated script '/sample-buildout/bin/demo'.
+    Installing python.
+    Generated interpreter '/sample-buildout/bin/python'.
+
+    >>> print system(join(sample_buildout, 'bin', 'python') +
+    ...              ' -c "import sys, pprint; pprint.pprint(sys.path[-2:])"')
+    ['/sample-buildout/eggs/demo-0.2-pyN.N.egg',
+     '/sample-buildout/eggs/demoneeded-1.2c1-pyN.N.egg']
+    <BLANKLINE>
+    >>> print system(join(sample_buildout, 'bin', 'python') +
+    ...              ''' -c "import os; print os.environ['zc.buildout']"'''),
+    foo bar baz shazam
+
+Note that the parts/py directory has been cleaned up, and parts/python has
+been created.
+
+    >>> ls(sample_buildout, 'parts')
+    d  buildout
+    d  demo
+    d  python
+
+If you want to have initialization that only affects scripts, not the
+interpreter, you can use script-initialization.  Here's a demonstration.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = demo
+    ...
+    ... [demo]
+    ... recipe = z3c.recipe.scripts
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... interpreter = py
+    ... script-initialization =
+    ...     print "Hi from the script"
+    ... """ % dict(server=link_server))
+
+    >>> print system(buildout),
+    Uninstalling python.
+    Uninstalling demo.
+    Installing demo.
+    Generated script '/sample-buildout/bin/demo'.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "print 'Hi from the interpreter'"'''),
+    Hi from the interpreter
+
+    >>> print system(join(sample_buildout, 'bin', 'demo')),
+    Hi from the script
+    2 2
+
+The last new option is ``name``.  This simply changes the name of the
+interpreter, so that you are not forced to use the name of the section.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... """
+    ... [buildout]
+    ... parts = interpreter
+    ...
+    ... [interpreter]
+    ... name = python2
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... """ % dict(server=link_server))
+
+    >>> print system(buildout),
+    Uninstalling demo.
+    Installing interpreter.
+    Generated interpreter '/sample-buildout/bin/python2'.
+
+    >>> print system(join(sample_buildout, 'bin', 'python2') +
+    ...              ' -c "print 42"')
+    42
+    <BLANKLINE>
+
+The other options all identical to zc.recipe.egg.

z3c.recipe.scripts_/src/z3c/recipe/scripts/__init__.py

+from z3c.recipe.scripts.scripts import Scripts, Interpreter

z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py

+##############################################################################
+#
+# Copyright (c) 2009-2010 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Install scripts from eggs.
+"""
+import os
+import zc.buildout
+import zc.buildout.easy_install
+from zc.recipe.egg.egg import ScriptBase
+
+
+class Base(ScriptBase):
+
+    def __init__(self, buildout, name, options):
+        if 'extends' in options:
+            options.update(buildout[options['extends']])
+        super(Base, self).__init__(buildout, name, options)
+        self.default_eggs = '' # Disables feature from zc.recipe.egg.
+        b_options = buildout['buildout']
+        options['parts-directory'] = os.path.join(
+            b_options['parts-directory'], self.name)
+
+        value = options.setdefault(
+            'allowed-eggs-from-site-packages',
+            '*')
+        self.allowed_eggs = tuple(name.strip() for name in value.split('\n'))
+
+        value = options.setdefault(
+            'include-site-packages',
+            b_options.get('include-site-packages', 'false'))
+        if value not in ('true', 'false'):
+            raise zc.buildout.UserError(
+                "Invalid value for include-site-packages option: %s" %
+                (value,))
+        self.include_site_packages = (value == 'true')
+
+        value = options.setdefault(
+            'exec-sitecustomize',
+            b_options.get('exec-sitecustomize', 'false'))
+        if value not in ('true', 'false'):
+            raise zc.buildout.UserError(
+                "Invalid value for exec-sitecustomize option: %s" %
+                (value,))
+        self.exec_sitecustomize = (value == 'true')
+
+
+class Interpreter(Base):
+
+    def __init__(self, buildout, name, options):
+        super(Interpreter, self).__init__(buildout, name, options)
+
+        options.setdefault('name', name)
+
+    def install(self):
+        reqs, ws = self.working_set()
+        options = self.options
+        generated = []
+        if not os.path.exists(options['parts-directory']):
+            os.mkdir(options['parts-directory'])
+            generated.append(options['parts-directory'])
+        generated.extend(zc.buildout.easy_install.sitepackage_safe_scripts(
+            options['bin-directory'], ws, options['executable'],
+            options['parts-directory'],
+            interpreter=options['name'],
+            extra_paths=self.extra_paths,
+            initialization=options.get('initialization', ''),
+            include_site_packages=self.include_site_packages,
+            exec_sitecustomize=self.exec_sitecustomize,
+            relative_paths=self._relative_paths,
+            ))
+        return generated
+
+    update = install
+
+
+class Scripts(Base):
+
+    def _install(self, reqs, ws, scripts):
+        options = self.options
+        generated = []
+        if not os.path.exists(options['parts-directory']):
+            os.mkdir(options['parts-directory'])
+            generated.append(options['parts-directory'])
+        generated.extend(zc.buildout.easy_install.sitepackage_safe_scripts(
+            options['bin-directory'], ws, options['executable'],
+            options['parts-directory'], reqs=reqs, scripts=scripts,
+            interpreter=options.get('interpreter'),
+            extra_paths=self.extra_paths,
+            initialization=options.get('initialization', ''),
+            include_site_packages=self.include_site_packages,
+            exec_sitecustomize=self.exec_sitecustomize,
+            relative_paths=self._relative_paths,
+            script_arguments=options.get('arguments', ''),
+            script_initialization=options.get('script-initialization', '')
+            ))
+        return generated

z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py

+##############################################################################
+#
+# Copyright (c) 2006 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+import os, re, shutil, sys
+import zc.buildout.tests
+import zc.buildout.testselectingpython
+import zc.buildout.testing
+
+import unittest
+from zope.testing import doctest, renormalizing
+
+# We do not explicitly test the recipe support for the ``eggs``,
+# ``find-links``, and ``index`` options because they are used for most or
+# all of the examples.  The README tests ``extends``,
+# ``include-site-customization`` and ``name``.  That leaves ``python``,
+# ``extra-paths``, ``initialization``, ``relative-paths``, and
+# ``include-site-packages``.
+
+def supports_python_option():
+    """
+This simply shows that the ``python`` option can specify another section to
+find the ``executable``.  (The ``python`` option defaults to looking in the
+``buildout`` section.)  We do this by creating a custom Python that will have
+some initialization that we can look for.
+
+    >>> py_path, site_packages_path = make_py(initialization='''
+    ... import os
+    ... os.environ['zc.buildout'] = 'foo bar baz shazam'
+    ... ''')
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [custom_python]
+    ... executable = %(py_path)s
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... exec-sitecustomize = true
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... python = custom_python
+    ... ''' % dict(server=link_server, py_path=py_path))
+
+    >>> print system(buildout),
+    Installing py.
+    Getting distribution for 'demo<0.3'.
+    Got demo 0.2.
+    Getting distribution for 'demoneeded'.
+    Got demoneeded 1.2c1.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import os; print os.environ['zc.buildout']"'''),
+    foo bar baz shazam
+"""
+
+def interpreter_recipe_supports_extra_paths_option():
+    """
+This shows that specifying extra-paths will affect sys.path.
+
+This recipe will not add paths that do not exist, so we create them.
+
+    >>> mkdir(sample_buildout, 'foo')
+    >>> mkdir(sample_buildout, 'foo', 'bar')
+    >>> mkdir(sample_buildout, 'spam')
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... extra-paths =
+    ...    ${buildout:directory}/foo/bar
+    ...    ${buildout:directory}/spam
+    ... ''' % dict(server=link_server))
+
+    >>> print system(buildout),
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import sys;print 'path' + ' '.join(sys.path)"''')
+    ... # doctest:+ELLIPSIS
+    path.../foo/bar /sample-buildout/spam...
+
+"""
+
+def interpreter_recipe_supports_initialization_option():
+    """
+This simply shows that the ``initialization`` option can specify code to
+run on initialization.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... initialization =
+    ...     import os
+    ...     os.environ['zc.buildout'] = 'foo bar baz shazam'
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... ''' % dict(server=link_server))
+
+    >>> print system(buildout),
+    Installing py.
+    Getting distribution for 'demo<0.3'.
+    Got demo 0.2.
+    Getting distribution for 'demoneeded'.
+    Got demoneeded 1.2c1.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> cat(sample_buildout, 'parts', 'py', 'sitecustomize.py')
+    ... # doctest: +NORMALIZE_WHITESPACE
+    <BLANKLINE>
+    import os
+    os.environ['zc.buildout'] = 'foo bar baz shazam'
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import os; print os.environ['zc.buildout']"'''),
+    foo bar baz shazam
+
+This also works with the exec-sitecustomize option, processing local
+initialization, and then the Python's initialization.  We show this with a
+custom Python.
+
+    >>> py_path, site_packages_path = make_py(initialization='''
+    ... import os
+    ... os.environ['zc.buildout'] = 'foo bar baz shazam'
+    ... ''')
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [custom_python]
+    ... executable = %(py_path)s
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... initialization =
+    ...     import os
+    ...     os.environ['zc.recipe.egg'] = 'baLOOba'
+    ... exec-sitecustomize = true
+    ... eggs = demo<0.3
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... python = custom_python
+    ... ''' % dict(server=link_server, py_path=py_path))
+
+    >>> print system(buildout),
+    Uninstalling py.
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+    >>> cat(sample_buildout, 'parts', 'py', 'sitecustomize.py')
+    ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
+    <BLANKLINE>
+    import os
+    os.environ['zc.recipe.egg'] = 'baLOOba'
+    <BLANKLINE>
+    # The following is from
+    # /executable_buildout/parts/py/sitecustomize.py
+    ...
+    import os
+    os.environ['zc.buildout'] = 'foo bar baz shazam'
+
+    >>> print system(join(sample_buildout, 'bin', 'py') + ' -c ' +
+    ...              '''"import os; print os.environ['zc.recipe.egg']"'''),
+    baLOOba
+    >>> print system(join(sample_buildout, 'bin', 'py') +
+    ...              ''' -c "import os; print os.environ['zc.buildout']"'''),
+    foo bar baz shazam
+
+"""
+
+def interpreter_recipe_supports_relative_paths_option():
+    """
+This shows that the relative-paths option affects the code for inserting
+paths into sys.path.
+
+    >>> write(sample_buildout, 'buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = py
+    ...
+    ... [py]
+    ... recipe = z3c.recipe.scripts:interpreter
+    ... find-links = %(server)s
+    ... index = %(server)s/index
+    ... relative-paths = true
+    ... extra-paths =
+    ...    /foo/bar
+    ...    ${buildout:directory}/spam
+    ... ''' % dict(server=link_server))
+
+    >>> print system(buildout),
+    Installing py.
+    Generated interpreter '/sample-buildout/bin/py'.
+
+Let's look at the site.py that was generated:
+
+    >>> import sys
+    >>> sys.stdout.write('#'); cat(sample_buildout, 'parts', 'py', 'site.py')
+    ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS
+    #...
+    def addsitepackages(known_paths):
+        "..."
+        join = os.path.join
+        base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
+        base = os.path.dirname(base)
+        base = os.path.dirname(base)
+        buildout_paths = [
+            '/foo/bar',
+            join(base, 'spam')
+            ]...
+"""
+
+def include_site_packages_option_reusing_eggs():
+    """
+The include-site-packages buildout option not only controls whether
+site-packages are included in the path, but whether eggs in site-packages
+can be used to fulfill direct and indirect dependencies of your package.  If
+it did not, it might fail to exclude site-packages because one of the
+dependencies actually was supposed to be fulfilled with it.
+
+The default is ``include-site-packages = false``.  This makes it possible to
+easily use a system Python.  As a demonstration, we will start with a
+Python executable that has the "demoneeded" and "demo" eggs installed.
+The eggs are not found.
+
+    >>> from zc.buildout.tests import create_sample_sys_install
+    >>> py_path, site_packages_path = make_py()
+    >>> create_sample_sys_install(site_packages_path)
+    >>> write('buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = eggs
+    ... find-links =
+    ...
+    ... [primed_python]
+    ... executable = %(py_path)s
+    ...
+    ... [eggs]
+    ... recipe = z3c.recipe.scripts
+    ... python = primed_python
+    ... eggs = demoneeded
+    ... ''' % globals())
+    >>> print system(buildout)
+    Installing eggs.
+    Couldn't find index page for 'demoneeded' (maybe misspelled?)
+    Getting distribution for 'demoneeded'.
+    While:
+      Installing eggs.
+      Getting distribution for 'demoneeded'.
+    Error: Couldn't find a distribution for 'demoneeded'.
+    <BLANKLINE>
+
+However, if we set include-site-packages to true, the package will be found.
+Notice we do not set find-links, but the eggs are still found because
+they are in the executable's path.
+
+    >>> write('buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = eggs
+    ... find-links =
+    ...
+    ... [primed_python]
+    ... executable = %(py_path)s
+    ...
+    ... [eggs]
+    ... recipe = z3c.recipe.scripts
+    ... python = primed_python
+    ... include-site-packages = true
+    ... eggs = demoneeded
+    ... ''' % globals())
+
+    >>> print system(buildout)
+    Installing eggs.
+    <BLANKLINE>
+
+We get an error if we specify anything but true or false:
+
+    >>> write('buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = eggs
+    ... find-links = %(link_server)s
+    ...
+    ... [eggs]
+    ... recipe = z3c.recipe.scripts
+    ... include-site-packages = no
+    ... eggs = other
+    ... ''' % globals())
+
+    >>> print system(buildout)
+    While:
+      Installing.
+      Getting section eggs.
+      Initializing part eggs.
+    Error: Invalid value for include-site-packages option: no
+    <BLANKLINE>
+
+    """
+
+def allowed_eggs_from_site_packages_option():
+    """
+The allowed-eggs-from-site-packages option allows you to specify a
+whitelist of project names that may be included from site-packages.
+
+In the test below, our "py_path" has the "demoneeded" and "demo"
+packages available.  We'll simply be asking for "demoneeded" here.  The
+default value of '*' will allow it, as we've seen elsewhere. Here we
+explicitly use a "*" for the same result.  This also shows that we
+correctly parse a single-line value.
+
+
+    >>> from zc.buildout.tests import create_sample_sys_install
+    >>> py_path, site_packages_path = make_py()
+    >>> create_sample_sys_install(site_packages_path)
+    >>> write('buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = eggs
+    ... find-links =
+    ...
+    ... [primed_python]
+    ... executable = %(py_path)s
+    ...
+    ... [eggs]
+    ... recipe = z3c.recipe.scripts
+    ... include-site-packages = true
+    ... allowed-eggs-from-site-packages = *
+    ... python = primed_python
+    ... eggs = demoneeded
+    ... ''' % globals())
+
+    >>> print system(buildout)
+    Installing eggs.
+    <BLANKLINE>
+
+Specifying the egg exactly will work as well.  This shows we correctly
+parse a multi-line value.
+
+    >>> zc.buildout.easy_install.clear_index_cache()
+    >>> write('buildout.cfg',
+    ... '''
+    ... [buildout]
+    ... parts = eggs
+    ... find-links =
+    ...
+    ... [primed_python]
+    ... executable = %(py_path)s
+    ...
+    ... [eggs]
+    ... recipe = z3c.recipe.scripts
+    ... include-site-packages = true
+    ... allowed-eggs-from-site-packages = other
+    ...                                   demoneeded
+    ... python = primed_python
+    ... eggs = demoneeded
+    ... ''' % globals())
+
+    >>> print system(buildout)
+    Uninstalling eggs.
+    Installing eggs.
+    <BLANKLINE>
+
+It will also work if we use a glob ("*" or "?").  (We won't show that here
+because we already tested it in
+zc.buildout.tests.allowed_eggs_from_site_packages.)
+
+However, if we do not include "demoneeded" in the
+"allowed-eggs-from-site-packages" key, we get an error, because the
+packages are not available in any links, and they are not allowed to
+come from the executable's site packages. (We won't show that here
+because we already tested it in the same test mentioned above.)
+
+    """
+
+def setUp(test):
+    zc.buildout.tests.easy_install_SetUp(test)
+    zc.buildout.testing.install_develop('zc.recipe.egg', test)
+    zc.buildout.testing.install_develop('z3c.recipe.scripts', test)
+
+def setUpSelecting(test):
+    zc.buildout.testselectingpython.setup(test)
+    zc.buildout.testing.install_develop('zc.recipe.egg', test)
+    zc.buildout.testing.install_develop('z3c.recipe.scripts', test)
+
+def test_suite():
+    suite = unittest.TestSuite((
+        doctest.DocFileSuite(
+            'README.txt',
+            setUp=setUp, tearDown=zc.buildout.testing.buildoutTearDown,
+            checker=renormalizing.RENormalizing([
+               zc.buildout.testing.normalize_path,
+               zc.buildout.testing.normalize_endings,
+               zc.buildout.testing.normalize_script,
+               zc.buildout.testing.normalize_egg_py,
+               zc.buildout.tests.normalize_bang,
+               (re.compile(r'zc.buildout(-\S+)?[.]egg(-link)?'),
+                'zc.buildout.egg'),
+               (re.compile('[-d]  (setuptools|distribute)-[^-]+-'), 'setuptools-X-'),
+               (re.compile(r'(setuptools|distribute)-[\w.]+-py'), 'setuptools-X-py'),
+               (re.compile(r'eggs\\\\demo'), 'eggs/demo'),
+               (re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'),
+               (re.compile(r'\#!\S+\bpython\S*'), '#!/usr/bin/python'),
+               # Normalize generate_script's Windows interpreter to UNIX:
+               (re.compile(r'\nimport subprocess\n'), '\n'),
+               (re.compile('subprocess\\.call\\(argv, env=environ\\)'),
+                'os.execve(sys.executable, argv, environ)'),
+               (re.compile('distribute'), 'setuptools'),
+               ])
+            ),
+        doctest.DocTestSuite(
+            setUp=setUp,
+            tearDown=zc.buildout.testing.buildoutTearDown,
+            checker=renormalizing.RENormalizing([
+                zc.buildout.testing.normalize_path,
+                zc.buildout.testing.normalize_endings,
+                zc.buildout.testing.normalize_egg_py,
+                (re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'),
+                ]),
+            ),
+
+        ))
+
+    return suite
+
+if __name__ == '__main__':
+    unittest.main(defaultTest='test_suite')
+

zc.recipe.egg_/setup.py

@@ -16,7 +16,7 @@
 $Id$
 """
 
-version = '0'
+version = '1.2.3dev'
 
 import os
 from setuptools import setup, find_packages
@@ -66,7 +66,7 @@ setup(
     package_dir = {'':'src'},
     namespace_packages = ['zc', 'zc.recipe'],
     install_requires = [
-        'zc.buildout >=1.2.0',
+        'zc.buildout >=1.5.0dev',
         'setuptools'],
     tests_require = ['zope.testing'],
     test_suite = name+'.tests.test_suite',

zc.recipe.egg_/src/zc/recipe/egg/README.txt

@@ -154,6 +154,8 @@ dependent-scripts
 interpreter
    The name of a script to generate that allows access to a Python
    interpreter that has the path set based on the eggs installed.
+   (See the ``z3c.recipe.scripts`` recipe for a more full-featured
+   interpreter.)
 
 extra-paths
    Extra paths to include in a generated script.
@@ -577,7 +579,7 @@ declare entry points using the entry-points option:
     -  demo
     -  other
 
-    >>> cat(sample_buildout, 'bin', 'other')
+    >>> cat(sample_buildout, 'bin', 'other') # doctest: +NORMALIZE_WHITESPACE
     #!/usr/local/bin/python2.4
     <BLANKLINE>
     import sys
@@ -640,3 +642,4 @@ be made to contact an index server:
     Uninstalling bigdemo.
     Installing demo.
     Generated script '/sample-buildout/bin/foo'.
+

zc.recipe.egg_/src/zc/recipe/egg/api.txt

@@ -117,6 +117,7 @@ computed by the egg recipe by looking at .installed.cfg:
     extras = other
     find-links = http://localhost:27071/
     index = http://localhost:27071/index
+    python = buildout
     recipe = sample
 
 If we use the extra-paths option:

zc.recipe.egg_/src/zc/recipe/egg/custom.txt

@@ -150,6 +150,7 @@ eggs directory can be shared across multiple buildouts.
 
     >>> ls(sample_buildout, 'develop-eggs')
     d  extdemo-1.4-py2.4-unix-i686.egg
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Note that no scripts or dependencies are installed.  To install
@@ -231,6 +232,7 @@ We won't get an update.
     >>> ls(sample_buildout, 'develop-eggs')
     -  demo.egg-link
     d  extdemo-1.4-py2.4-unix-i686.egg
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 But if we run the buildout in the default on-line and newest modes, we
@@ -248,6 +250,7 @@ version is imported:
     -  demo.egg-link
     d  extdemo-1.4-py2.4-linux-i686.egg
     d  extdemo-1.5-py2.4-linux-i686.egg
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 Controlling the version used
@@ -287,6 +290,7 @@ We can specify a specific version using the egg option:
     >>> ls(sample_buildout, 'develop-eggs')
     -  demo.egg-link
     d  extdemo-1.4-py2.4-linux-i686.egg
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 
@@ -553,6 +557,7 @@ Our develop-eggs now includes an egg link for extdemo:
     >>> ls('develop-eggs')
     -  demo.egg-link
     -  extdemo.egg-link
+    -  z3c.recipe.scripts.egg-link
     -  zc.recipe.egg.egg-link
 
 and the extdemo now has a built extension:

zc.recipe.egg_/src/zc/recipe/egg/egg.py

@@ -19,14 +19,17 @@ $Id$
 import logging, os, re, zipfile
 import zc.buildout.easy_install
 
+
 class Eggs(object):
 
+    include_site_packages = allowed_eggs = None
+
     def __init__(self, buildout, name, options):
         self.buildout = buildout
-        self.name = name
+        self.name = self.default_eggs = name
         self.options = options
-        links = options.get('find-links',
-                            buildout['buildout'].get('find-links'))
+        b_options = buildout['buildout']
+        links = options.get('find-links', b_options['find-links'])
         if links:
             links = links.split()
             options['find-links'] = '\n'.join(links)
@@ -34,25 +37,25 @@ class Eggs(object):
             links = ()
         self.links = links
 
-        index = options.get('index', buildout['buildout'].get('index'))
+        index = options.get('index', b_options.get('index'))
         if index is not None:
             options['index'] = index
         self.index = index
 
-        allow_hosts = buildout['buildout'].get('allow-hosts', '*')
+        allow_hosts = b_options['allow-hosts']
         allow_hosts = tuple([host.strip() for host in allow_hosts.split('\n')
                                if host.strip()!=''])
         self.allow_hosts = allow_hosts
 
-        options['eggs-directory'] = buildout['buildout']['eggs-directory']
+        options['eggs-directory'] = b_options['eggs-directory']
         options['_e'] = options['eggs-directory'] # backward compat.
-        options['develop-eggs-directory'
-                ] = buildout['buildout']['develop-eggs-directory']
+        options['develop-eggs-directory'] = b_options['develop-eggs-directory']
         options['_d'] = options['develop-eggs-directory'] # backward compat.
 
-        assert options.get('unzip') in ('true', 'false', None)
+        # verify that this is None, 'true' or 'false'
+        get_bool(options, 'unzip')
 
-        python = options.get('python', buildout['buildout']['python'])
+        python = options.setdefault('python', b_options['python'])
         options['executable'] = buildout[python]['executable']
 
     def working_set(self, extra=()):
@@ -61,31 +64,36 @@ class Eggs(object):
         This is intended for reuse by similar recipes.
         """
         options = self.options
+        b_options = self.buildout['buildout']
 
         distributions = [
             r.strip()
-            for r in options.get('eggs', self.name).split('\n')
+            for r in options.get('eggs', self.default_eggs).split('\n')
             if r.strip()]
         orig_distributions = distributions[:]
         distributions.extend(extra)
 
-        if self.buildout['buildout'].get('offline') == 'true':
+        if b_options.get('offline') == 'true':
             ws = zc.buildout.easy_install.working_set(
                 distributions, options['executable'],
-                [options['develop-eggs-directory'], options['eggs-directory']]
+                [options['develop-eggs-directory'],
+                 options['eggs-directory']],
+                include_site_packages=self.include_site_packages,
+                allowed_eggs_from_site_packages=self.allowed_eggs,
                 )
         else:
             kw = {}
-            if options.get('unzip'):
+            if 'unzip' in options:
                 kw['always_unzip'] = get_bool(options, 'unzip')
-
             ws = zc.buildout.easy_install.install(
                 distributions, options['eggs-directory'],
                 links=self.links,
                 index=self.index,
                 executable=options['executable'],
                 path=[options['develop-eggs-directory']],
-                newest=self.buildout['buildout'].get('newest') == 'true',
+                newest=b_options.get('newest') == 'true',
+                include_site_packages=self.include_site_packages,
+                allowed_eggs_from_site_packages=self.allowed_eggs,
                 allow_hosts=self.allow_hosts,
                 **kw)
 
@@ -97,16 +105,19 @@ class Eggs(object):
 
     update = install
 
-class Scripts(Eggs):
+
+class ScriptBase(Eggs):
 
     def __init__(self, buildout, name, options):
-        super(Scripts, self).__init__(buildout, name, options)
+        super(ScriptBase, self).__init__(buildout, name, options)
+
+        b_options = buildout['buildout']
 
-        options['bin-directory'] = buildout['buildout']['bin-directory']
+        options['bin-directory'] = b_options['bin-directory']
         options['_b'] = options['bin-directory'] # backward compat.
 
         self.extra_paths = [
-            os.path.join(buildout['buildout']['directory'], p.strip())
+            os.path.join(b_options['directory'], p.strip())
             for p in options.get('extra-paths', '').split('\n')
             if p.strip()
             ]
@@ -115,11 +126,9 @@ class Scripts(Eggs):
 
 
         relative_paths = options.get(
-            'relative-paths',
-            buildout['buildout'].get('relative-paths', 'false')
-            )
+            'relative-paths', b_options.get('relative-paths', 'false'))
         if relative_paths == 'true':
-            options['buildout-directory'] = buildout['buildout']['directory']
+            options['buildout-directory'] = b_options['directory']
             self._relative_paths = options['buildout-directory']
         else:
             self._relative_paths = ''
@@ -128,12 +137,13 @@ class Scripts(Eggs):
     parse_entry_point = re.compile(
         '([^=]+)=(\w+(?:[.]\w+)*):(\w+(?:[.]\w+)*)$'
         ).match
+
     def install(self):
         reqs, ws = self.working_set()
         options = self.options
 
         scripts = options.get('scripts')
-        if scripts or scripts is None:
+        if scripts or scripts is None or options.get('interpreter'):
             if scripts is not None:
                 scripts = scripts.split()
                 scripts = dict([
@@ -157,22 +167,32 @@ class Scripts(Eggs):
                     name = dist.project_name
                     if name != 'setuptools' and name not in reqs:
                         reqs.append(name)
-
-            return zc.buildout.easy_install.scripts(
-                reqs, ws, options['executable'],
-                options['bin-directory'],
-                scripts=scripts,
-                extra_paths=self.extra_paths,
-                interpreter=options.get('interpreter'),
-                initialization=options.get('initialization', ''),
-                arguments=options.get('arguments', ''),
-                relative_paths=self._relative_paths,
-                )
-
+            return self._install(reqs, ws, scripts)
         return ()
 
     update = install
 
+    def _install(self, reqs, ws, scripts):
+        # Subclasses implement this.
+        raise NotImplementedError()
+
+
+class Scripts(ScriptBase):
+
+    def _install(self, reqs, ws, scripts):
+        options = self.options
+        return zc.buildout.easy_install.scripts(
+            reqs, ws, options['executable'],
+            options['bin-directory'],
+            scripts=scripts,
+            extra_paths=self.extra_paths,
+            interpreter=options.get('interpreter'),
+            initialization=options.get('initialization', ''),
+            arguments=options.get('arguments', ''),
+            relative_paths=self._relative_paths
+            )
+
+
 def get_bool(options, name, default=False):
     value = options.get(name)
     if not value:

zc.recipe.egg_/src/zc/recipe/egg/selecting-python.txt

@@ -35,7 +35,7 @@ install the demo package using Python 2.4.
     ... index = http://www.python.org/pypi/
     ...
     ... [python2.4]
-    ... executable = %(python23)s
+    ... executable = %(python24)s
     ...
     ... [demo]
     ... recipe = zc.recipe.egg
@@ -43,7 +43,7 @@ install the demo package using Python 2.4.
     ... find-links = %(server)s
     ... python = python2.4
     ... interpreter = py-demo
-    ... """ % dict(server=link_server, python23=other_executable))
+    ... """ % dict(server=link_server, python24=other_executable))
 
 Now, if we run the buildout:
 

zc.recipe.egg_/src/zc/recipe/egg/tests.py

@@ -50,9 +50,12 @@ def test_suite():
                zc.buildout.tests.normalize_bang,
                (re.compile('zc.buildout(-\S+)?[.]egg(-link)?'),
                 'zc.buildout.egg'),
-               (re.compile('[-d]  setuptools-[^-]+-'), 'setuptools-X-'),
+               (re.compile('[-d]  (setuptools|distribute)-[^-]+-'),
+                'setuptools-X-'),
                (re.compile(r'eggs\\\\demo'), 'eggs/demo'),
                (re.compile(r'[a-zA-Z]:\\\\foo\\\\bar'), '/foo/bar'),
+               # Distribute unzips eggs by default.
+               (re.compile('\-  demoneeded'), 'd  demoneeded'),
                ])
             ),
         doctest.DocFileSuite(
@@ -64,7 +67,7 @@ def test_suite():
                (re.compile('__buildout_signature__ = '
                            'sample-\S+\s+'
                            'zc.recipe.egg-\S+\s+'
-                           'setuptools-\S+\s+'
+                           '(setuptools|distribute)-\S+\s+'
                            'zc.buildout-\S+\s*'
                            ),
                 '__buildout_signature__ = sample- zc.recipe.egg-\n'),
@@ -104,14 +107,17 @@ def test_suite():
                    zc.buildout.testing.normalize_path,
                    zc.buildout.testing.normalize_endings,
                    zc.buildout.testing.normalize_script,
-                   (re.compile('Got setuptools \S+'), 'Got setuptools V'),
-                   (re.compile('([d-]  )?setuptools-\S+-py'),
+                   (re.compile('Got (setuptools|distribute) \S+'),
+                    'Got setuptools V'),
+                   (re.compile('([d-]  )?(setuptools|distribute)-\S+-py'),
                     'setuptools-V-py'),
                    (re.compile('-py2[.][0-35-9][.]'), 'py2.5.'),
                    (re.compile('zc.buildout-\S+[.]egg'),
                     'zc.buildout.egg'),
                    (re.compile('zc.buildout[.]egg-link'),
                     'zc.buildout.egg'),
+                   # Distribute unzips eggs by default.
+                   (re.compile('\-  demoneeded'), 'd  demoneeded'),
                    ]),
                 ),
             )