Automatic dependency part selection (#367)

* Added support for special implication syntax ``=>``

* added <part-dependencies> support

doc/reference.rst

@@ -524,9 +524,10 @@ and a value:
     {'buildout': {'parts': 'py'}}
 
 Option names may have any characters other than whitespace, square
-braces, curly braces, equal signs, or colons.  There may be and usually
-is whitespace between the name and the equal sign and the name and
-equal sign must be on the same line.
+braces, curly braces, equal signs, or colons.  There may be and
+usually is whitespace between the name and the equal sign and the name
+and equal sign must be on the same line.  Names starting with ``<``
+are reserved for Buildout's use.
 
 Option values may contain any characters. A consequence of this is
 that there can't be comments in option values.
@@ -591,6 +592,27 @@ no data on the first line
 
        >>> eq(parse(header + option)['buildout']['code'] + '\n', val)
 
+Special "implication" syntax for the ``<part-dependencies>`` option
+____________________________________________________________________
+
+An exception to the normal option syntax is the use of ``=>`` as a
+short-hand for the ``<part-dependencies>`` option:
+
+.. code-block:: ini
+
+   => part1 part2
+      part3
+
+This is equivalent to:
+
+.. code-block:: ini
+
+   <part-dependencies> = part1 part2
+      part3
+
+and declares that the named parts are dependencies of the part in
+which this option appears.
+
 Comments and blank lines
 ------------------------
 

doc/topics/implicit-parts.rst

+=================================================
+Automatic installation of part dependencies
+=================================================
+
+Buildout parts are requested by the ``parts`` option of the
+``buildout`` section, but a buildout may install additional parts that
+are dependencies of the named parts.  For example, in
+
+.. code-block:: ini
+
+   [buildout]
+   develop = .
+   parts = server
+
+   [server]
+   => app
+   recipe = zc.zdaemonrecipe
+   program = $buildout:bin-directory}/app ${config:location}
+
+   [app]
+   recipe = zc.recipe.egg
+   eggs = myapp
+
+   [config]
+   recipe = zc.recipe.deployment:configuration
+   text = port 8080
+
+.. -> src
+
+    >>> write(src, 'buildout.cfg')
+    >>> write("from setuptools import setup; setup(name='myapp')", 'setup.py')
+    >>> run_buildout('buildout annotate')
+    >>> run_buildout()
+    >>> print(read()) # doctest: +ELLIPSIS
+    Creating ...
+    Installing config.
+    Installing app.
+    Installing server.
+    ...
+
+the ``server`` part depends on the ``app`` part to
+install the server software and on the ``config`` part to provide the
+server configuration.
+
+The ``config`` part will be installed before the ``server`` part
+because it's referenced in a value substitution.  The value
+substitution makes the ``config`` part a dependency of the ``server``
+part.
+
+The ``server`` part has the line:
+
+.. code-block:: ini
+
+   => app
+
+This line [#implication-syntax]_, uses a feature that's **new in zc.buildout
+2.9**.  It declares that the ``app`` part is a dependency of the
+``server`` part.  The server part doesn't use any information from the
+``app`` part, so it has to declare the dependency explicitly.  It
+could have declared both dependencies explicitly:
+
+.. code-block:: ini
+
+  => app config
+
+Dependency part selection serves separation of concerns.  The
+buildout ``parts`` option reflects the requirements of a buildout as a
+whole.  If a named part depends on another part, that's the concern of
+the named part, not of the buildout itself.
+
+.. [#implication-syntax] The ``=>`` syntax is a convenience.  It's
+   based on the mathematical symbol for implication.  It's a short
+   hand for:
+
+   .. code-block:: ini
+
+      <part-dependencies> = app
+
+   Multiple parts may be listed and spread over multiple lines, as
+   long as continuation lines are indented.

doc/topics/index.rst

@@ -7,6 +7,7 @@ Buildout Topics
 
    history
    variables-extending-and-substitutions
+   implicit-parts
    optimizing
    bootstrapping
    writing-recipes

doc/topics/writing-recipes.rst

@@ -155,7 +155,7 @@ Some things to note about this example:
   as we've seen here.
 
 - We didn't have to list the ``config`` part in the buildout ``parts``
-  option.  It's :doc:`added implicitly <implicit-parts>` by virtue of
+  option.  It's :doc:`added automatically <implicit-parts>` by virtue of
   its use in the ``server`` part.
 
 - We used the ``develop`` option to specify a ``src`` directory

setup.py

@@ -94,7 +94,8 @@ setup(
     entry_points = entry_points,
     extras_require = dict(
         test=['zope.testing', 'manuel',
-              'bobo ==2.3.0', 'zdaemon', 'zc.zdaemonrecipe']),
+              'bobo ==2.3.0', 'zdaemon', 'zc.zdaemonrecipe',
+              'zc.recipe.deployment']),
     zip_safe=False,
     classifiers = [
        'Intended Audience :: Developers',

src/zc/buildout/buildout.py

@@ -1237,6 +1237,10 @@ class Options(DictMixin):
         if name == 'buildout':
             return # buildout section can never be a part
 
+        for dname in self.get('<part-dependencies>', '').split():
+            # force use of dependencies in buildout:
+            self.buildout[dname]
+
         if self.get('recipe'):
             self.initialize()
             self.buildout._parts.append(name)

src/zc/buildout/configparser.py

@@ -206,6 +206,8 @@ def parse(fp, fpname, exp_globals=dict):
                 # no section header in the file?
                 raise MissingSectionHeaderError(fpname, lineno, line)
             else:
+                if line[:2] == '=>':
+                    line = '<part-dependencies> = ' + line[2:]
                 mo = option_start(line)
                 if mo:
                     if not section_condition:

src/zc/buildout/configparser.test

@@ -39,9 +39,12 @@ First, an example that illustrates a well-formed configuration::
 
     >>> try: import StringIO
     ... except ImportError: import io as StringIO
-    >>> import pprint, zc.buildout.configparser
-    >>> pprint.pprint(zc.buildout.configparser.parse(StringIO.StringIO(
-    ...     text), 'test'))
+    >>> from pprint import pprint
+    >>> import zc.buildout.configparser
+    >>> def parse(config, *args, **kw):
+    ...     return zc.buildout.configparser.parse(StringIO.StringIO(config),
+    ...                                           'test', *args, **kw)
+    >>> pprint(parse(text))
     {'s1': {'a': '1'},
      's2': {'b    +': '1',
             'c': '1',
@@ -53,8 +56,7 @@ First, an example that illustrates a well-formed configuration::
 Here's an example with leading blank lines:
 
     >>> text = '\n\n[buildout]\nz=1\n\n'
-    >>> pprint.pprint(zc.buildout.configparser.parse(StringIO.StringIO(
-    ...     text), 'test'))
+    >>> pprint(parse(text))
     {'buildout': {'z': '1'}}
 
 
@@ -81,8 +83,7 @@ otherwise empty section) is blank.  For example:"
 
 .. -> text
 
-    >>> pprint.pprint(zc.buildout.configparser.parse(StringIO.StringIO(
-    ...     text), 'test'))
+    >>> pprint(parse(text))
     {'buildout': {'parts': 'hello', 'versions': 'versions'},
      'hello': {'cmds': 'echo Hello',
                'on_install': 'true',
@@ -111,11 +112,7 @@ conditional exclusion of sections::
 
 .. -> text
 
-    >>> try: import StringIO
-    ... except ImportError: import io as StringIO
-    >>> import pprint, zc.buildout.configparser
-    >>> pprint.pprint(zc.buildout.configparser.parse(StringIO.StringIO(
-    ...     text), 'test'))
+    >>> pprint(parse(text))
     {'s1': {'a': '1'}, 's2': {'long': 'b'}, 's3': {'long': 'c'}}
 
 
@@ -159,11 +156,7 @@ some cases. For example, valid sections lines include::
 
 .. -> text
 
-    >>> try: import StringIO
-    ... except ImportError: import io as StringIO
-    >>> import pprint, zc.buildout.configparser
-    >>> pprint.pprint(zc.buildout.configparser.parse(StringIO.StringIO(
-    ...     text), 'test'))
+    >>> pprint(parse(text))
     {'a': {'a': '1'},
      'b': {'b': '1'},
      'c': {'c': '1'},
@@ -197,11 +190,7 @@ character. The following are valid semicolon-separated comments::
 
 .. -> text
 
-    >>> try: import StringIO
-    ... except ImportError: import io as StringIO
-    >>> import pprint, zc.buildout.configparser
-    >>> pprint.pprint(zc.buildout.configparser.parse(StringIO.StringIO(
-    ...     text), 'test'))
+    >>> pprint(parse(text))
     {'a': {'a': '1'},
      'b': {'b': '1'},
      'c': {'c': '1'},
@@ -229,11 +218,7 @@ The following sections with hash comment separators are valid too::
 
 .. -> text
 
-    >>> try: import StringIO
-    ... except ImportError: import io as StringIO
-    >>> import pprint, zc.buildout.configparser
-    >>> pprint.pprint(zc.buildout.configparser.parse(StringIO.StringIO(
-    ...     text), 'test'))
+    >>> pprint(parse(text))
     {'a': {'a': '1'},
      'b': {'b': '1'},
      'c': {'c': '1'},
@@ -256,11 +241,7 @@ These expressions are valid and use escaped hash and semicolons in literals::
 
 .. -> text
 
-    >>> try: import StringIO
-    ... except ImportError: import io as StringIO
-    >>> import pprint, zc.buildout.configparser
-    >>> pprint.pprint(zc.buildout.configparser.parse(StringIO.StringIO(
-    ...     text), 'test'))
+    >>> pprint(parse(text))
     {'a': {'a': '1'}, 'b': {'b': '1'}}
 
 
@@ -271,10 +252,7 @@ And using unescaped semicolon and hash characters in expressions triggers an err
 
 .. -> text
 
-    >>> try: import StringIO
-    ... except ImportError: import io as StringIO
-    >>> import zc.buildout.configparser
-    >>> try: zc.buildout.configparser.parse(StringIO.StringIO(text), 'test')
+    >>> try: parse(text)
     ... except zc.buildout.configparser.MissingSectionHeaderError: pass # success
 
 
@@ -296,13 +274,9 @@ platform and sys modules functions and objects in our expressions ::
 
 .. -> text
 
-    >>> try: import StringIO
-    ... except ImportError: import io as StringIO
-    >>> import pprint, zc.buildout.configparser
     >>> import platform, sys
     >>> globs = lambda: {'platform': platform, 'sys': sys}
-    >>> pprint.pprint(zc.buildout.configparser.parse(StringIO.StringIO(
-    ...     text), 'test', exp_globals=globs))
+    >>> pprint(parse(text, exp_globals=globs))
     {'s1': {'a': '1'}, 's2': {'long': 'b'}}
 
 
@@ -334,15 +308,21 @@ called by buildout::
 
 .. -> text
 
-    >>> try: import StringIO
-    ... except ImportError: import io as StringIO
-    >>> import pprint, zc.buildout.configparser
     >>> import zc.buildout.buildout
-    >>> pprint.pprint(zc.buildout.configparser.parse(StringIO.StringIO(
-    ...     text), 'test', zc.buildout.buildout._default_globals))
+    >>> pprint(parse(text, zc.buildout.buildout._default_globals))
     {'s1': {'a': '1'},
      's2': {'b': '1'},
      's3': {'c': '1'},
      's4': {'d': '1'},
      's5': {'e': '1'}}
 
+
+Preprocessing of implication and unicode cuteness::
+
+  [foo]
+  => part1 part2
+
+.. -> text
+
+    >>> pprint(parse(text))
+    {'foo': {'<part-dependencies>': 'part1 part2'}}

src/zc/buildout/tests.py

@@ -3835,6 +3835,7 @@ def test_suite():
                 os.path.join(docdir, 'getting-started.rst'),
                 os.path.join(docdir, 'reference.rst'),
                 os.path.join(docdir, 'topics', 'bootstrapping.rst'),
+                os.path.join(docdir, 'topics', 'implicit-parts.rst'),
                 os.path.join(
                     docdir,
                     'topics', 'variables-extending-and-substitutions.rst'),