Commit fbb8c37b authored by Leonardo Rochael Almeida's avatar Leonardo Rochael Almeida Committed by Jim Fulton

Installer: Encapsulate all uses of `._env` and `._path` (#352)

* buildout.cfg: Remove redundancy

Share the eggs definition between the the online and offline tests.

* Remove redundant code

Trying to insert `self._dest` in `self._path`  in `Installer.install()`
is unnecessary since:

 * it's already done in `__init__()`

 * nothing changes either `self._path` or `self._dest` during the
   lifetime of `Installer`

Remove `Installer._load_dist(self, dist)` since it has never been
used in the whole history of Buildout.

Finally, simplify the interface to `._get_dist()` and  `_call_easy_install()`:

All callers to `._get_dist()` add the resulting dist to the WorkingSet. No
reason to do it in `._get_dist()` or to pass the `ws` into
`._call_easy_install()`.

* Be stricter when adding setuptools as requirements

Only do it for namespace package dists that actually need it. I.e. those
that use `pkg_resources.declare_namespace()` instead of
`pkg_util.extend_path()` or PEP 420.

* easy_install: `eggs-directory` may contain more than eggs

Modify the `easy_install.Installer` class to locate not only eggs, but
anything that has a `.dist-info` inside `._dest` (i.e. the Buildout
`eggs-directory`).

This makes it easier for extensions like `buildout.wheel` to add non eggs
inside it: by installing a distribution inside a subdirectory of `._dest` as if
that subdirectory was `site-packages`, you can later locate that
distribution with the Installer.

Conversely, make sure any distribution found in a direct child of `.dest` is
treated as if it was an `egg` distribution (as opposed to a develop
distribution or a site-packages distribution).

All access to `Installer._path` and all access that modifies `._env` is now
done through `Installer` methods, so that Buildout extensions can install
subclasses of `Installer` that alter their behaviour.

* Refactor `_get_dist()` and `_call_easy_install()`

Move the actual invocation of `easy_install` to a module global function
`call_easy_install()`. Simplify its signature so it's simpler to
override in Buildout extensions.

Refactor `._get_dist()` and remove all unpack/install logic into
`_move_to_eggs_dir_and_compile()`, bypassing
`Installer._call_easy_install()`.

`._get_dist()` now calls directly `call_easy_install()` instead. But
only as a fallback to a dictionary lookup of filename extensions to
distribution unpacking methods like:

 - `.egg`: `unpack_egg()`
 - `.whl`: `unpack_wheel()`

This will make it easier for extensions to add support to new dist
formats.
parent ee151de5
...@@ -25,9 +25,7 @@ eggs = ...@@ -25,9 +25,7 @@ eggs =
# Tests that can be run wo a network # Tests that can be run wo a network
[oltest] [oltest]
recipe = zc.recipe.testrunner recipe = zc.recipe.testrunner
eggs = eggs = ${test:eggs}
zc.buildout[test]
zc.recipe.egg
defaults = defaults =
[ [
'-t', '-t',
......
This diff is collapsed.
...@@ -836,12 +836,15 @@ and we will get setuptools included in the working set. ...@@ -836,12 +836,15 @@ and we will get setuptools included in the working set.
... 'zc.buildout.easy_install', level=logging.WARNING) ... 'zc.buildout.easy_install', level=logging.WARNING)
>>> logging.getLogger('zc.buildout.easy_install').propagate = False >>> logging.getLogger('zc.buildout.easy_install').propagate = False
>>> [dist.project_name >>> def get_working_set(*project_names):
... paths = [join(sample_buildout, 'eggs'),
... join(sample_buildout, 'develop-eggs')]
... return [
... dist.project_name
... for dist in zc.buildout.easy_install.working_set( ... for dist in zc.buildout.easy_install.working_set(
... ['foox'], sys.executable, ... project_names, sys.executable, paths)
... [join(sample_buildout, 'eggs'), ... ]
... join(sample_buildout, 'develop-eggs'), >>> get_working_set('foox')
... ])]
['foox', 'setuptools'] ['foox', 'setuptools']
>>> print_(handler) >>> print_(handler)
...@@ -851,13 +854,15 @@ and we will get setuptools included in the working set. ...@@ -851,13 +854,15 @@ and we will get setuptools included in the working set.
>>> handler.clear() >>> handler.clear()
On the other hand, if we have a regular egg, rather than a develop egg: On the other hand, if we have a zipped egg, rather than a develop egg:
>>> os.remove(join('develop-eggs', 'foox.egg-link')) >>> os.remove(join('develop-eggs', 'foox.egg-link'))
>>> _ = system(join('bin', 'buildout') + ' setup foo bdist_egg -d' >>> _ = system(join('bin', 'buildout') + ' setup foo bdist_egg')
... + join(sample_buildout, 'eggs')) >>> foox_dist = join('foo', 'dist')
>>> import glob
>>> [foox_egg] = glob.glob(join(foox_dist, 'foox-*.egg'))
>>> _ = shutil.copy(foox_egg, join(sample_buildout, 'eggs'))
>>> ls('develop-eggs') >>> ls('develop-eggs')
- zc.recipe.egg.egg-link - zc.recipe.egg.egg-link
...@@ -870,17 +875,27 @@ On the other hand, if we have a regular egg, rather than a develop egg: ...@@ -870,17 +875,27 @@ On the other hand, if we have a regular egg, rather than a develop egg:
We do not get a warning, but we do get setuptools included in the working set: We do not get a warning, but we do get setuptools included in the working set:
>>> [dist.project_name >>> get_working_set('foox')
... for dist in zc.buildout.easy_install.working_set( ['foox', 'setuptools']
... ['foox'], sys.executable,
... [join(sample_buildout, 'eggs'), >>> print_(handler, end='')
... join(sample_buildout, 'develop-eggs'),
... ])] Likewise for an unzipped egg:
>>> foox_egg_basename = os.path.basename(foox_egg)
>>> os.remove(join(sample_buildout, 'eggs', foox_egg_basename))
>>> _ = zc.buildout.easy_install.install(
... ['foox'], join(sample_buildout, 'eggs'), links=[foox_dist],
... index='file://' + foox_dist)
>>> ls('develop-eggs')
- zc.recipe.egg.egg-link
>>> get_working_set('foox')
['foox', 'setuptools'] ['foox', 'setuptools']
>>> print_(handler, end='') >>> print_(handler, end='')
We get the same behavior if the it is a dependency that uses a We get the same behavior if it is a dependency that uses a
namespace package. namespace package.
...@@ -903,12 +918,7 @@ namespace package. ...@@ -903,12 +918,7 @@ namespace package.
Develop: '/sample-buildout/foo' Develop: '/sample-buildout/foo'
Develop: '/sample-buildout/bar' Develop: '/sample-buildout/bar'
>>> [dist.project_name >>> get_working_set('bar')
... for dist in zc.buildout.easy_install.working_set(
... ['bar'], sys.executable,
... [join(sample_buildout, 'eggs'),
... join(sample_buildout, 'develop-eggs'),
... ])]
['bar', 'foox', 'setuptools'] ['bar', 'foox', 'setuptools']
>>> print_(handler, end='') >>> print_(handler, end='')
...@@ -916,6 +926,32 @@ namespace package. ...@@ -916,6 +926,32 @@ namespace package.
Develop distribution: foox 0.0.0 Develop distribution: foox 0.0.0
uses namespace packages but the distribution does not require setuptools. uses namespace packages but the distribution does not require setuptools.
On the other hand, if the distribution uses ``pkgutil.extend_path()`` to
implement its namespaces, even if just as fallback from the absence of
``pkg_resources``, then ``setuptools`` shoudn't be added as requirement to
its unzipped egg:
>>> foox_installed_egg = join(sample_buildout, 'eggs', foox_egg_basename)
>>> namespace_init = join(foox_installed_egg, 'stuff', '__init__.py')
>>> write(namespace_init,
... """try:
... __import__('pkg_resources').declare_namespace(__name__)
... except ImportError:
... __path__ = __import__('pkgutil').extend_path(__path__, __name__)
... """)
>>> os.remove(join('develop-eggs', 'foox.egg-link'))
>>> os.remove(join('develop-eggs', 'bar.egg-link'))
>>> get_working_set('foox')
['foox']
The same goes for packages using PEP420 namespaces
>>> os.remove(namespace_init)
>>> get_working_set('foox')
['foox']
Cleanup:
>>> logging.getLogger('zc.buildout.easy_install').propagate = True >>> logging.getLogger('zc.buildout.easy_install').propagate = True
>>> handler.uninstall() >>> handler.uninstall()
...@@ -3155,8 +3191,7 @@ class UnitTests(unittest.TestCase): ...@@ -3155,8 +3191,7 @@ class UnitTests(unittest.TestCase):
def wheel_to_egg(dist, dest): def wheel_to_egg(dist, dest):
newloc = os.path.join(dest, egg_name) newloc = os.path.join(dest, egg_name)
shutil.copy(dist.location, newloc) shutil.copy(dist.location, newloc)
return pkg_resources.Distribution.from_location(newloc, return pkg_resources.Distribution.from_filename(newloc)
'demo-0.3.whl')
zc.buildout.easy_install.wheel_to_egg = wheel_to_egg zc.buildout.easy_install.wheel_to_egg = wheel_to_egg
egg_dir = os.path.join(self.sample_buildout, 'eggs') egg_dir = os.path.join(self.sample_buildout, 'eggs')
self.assertFalse(egg_name in os.listdir(egg_dir)) self.assertFalse(egg_name in os.listdir(egg_dir))
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment