Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
S
slapos.buildout
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
Xavier Thompson
slapos.buildout
Commits
4f32ff9e
Commit
4f32ff9e
authored
Feb 28, 2017
by
Jim Fulton
Committed by
GitHub
Feb 28, 2017
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Added the writing-recipes topic (#359)
parent
9b6dfd40
Changes
5
Hide whitespace changes
Inline
Side-by-side
Showing
5 changed files
with
392 additions
and
19 deletions
+392
-19
doc/reference.rst
doc/reference.rst
+20
-0
doc/topics/index.rst
doc/topics/index.rst
+1
-3
doc/topics/writing-recipes.rst
doc/topics/writing-recipes.rst
+355
-0
src/zc/buildout/buildout.py
src/zc/buildout/buildout.py
+11
-13
src/zc/buildout/tests.py
src/zc/buildout/tests.py
+5
-3
No files found.
doc/reference.rst
View file @
4f32ff9e
...
@@ -138,6 +138,22 @@ It then runs the resulting buildout.
...
@@ -138,6 +138,22 @@ It then runs the resulting buildout.
See :ref:`Bootstrapping <init-generates-buildout.cfg>` for examples.
See :ref:`Bootstrapping <init-generates-buildout.cfg>` for examples.
.. _install-command:
install
_______
Install the parts specified in the buildout configuration. This is
the default command if no command is specified.
.. We'
re
not
documenting
arguments
.
Passing
arguments
to
install
is
an
attractive
nuisance
,
since
it
can
lead
to
parts
being
installed
inconsistently
.
The
feature
exists
for
backward
compatibility
,
but
may
be
dropped
in
the
future
.
setup
PATH
SETUP
-
COMMANDS
setup
PATH
SETUP
-
COMMANDS
_________________________
_________________________
...
@@ -301,6 +317,8 @@ extends-cache
...
@@ -301,6 +317,8 @@ extends-cache
substitutions, and the result is a relative path, then it will be
substitutions, and the result is a relative path, then it will be
interpreted relative to the buildout directory.)
interpreted relative to the buildout directory.)
.. _find-links-option:
find-links, default: ''
find-links, default: ''
Extra locations to search for distributions to download.
Extra locations to search for distributions to download.
...
@@ -337,6 +355,8 @@ install-from-cache, default: 'false'
...
@@ -337,6 +355,8 @@ install-from-cache, default: 'false'
This
mode
may
only
be
used
if
a
:
ref
:`
download
-
cache
This
mode
may
only
be
used
if
a
:
ref
:`
download
-
cache
<
download
-
cache
>`
is
specified
.
<
download
-
cache
>`
is
specified
.
..
_installed
-
option
:
installed
,
default
:
'.installed.cfg'
installed
,
default
:
'.installed.cfg'
The
name
of
the
file
used
to
store
information
about
what
's installed.
The
name
of
the
file
used
to
store
information
about
what
's installed.
...
...
doc/topics/index.rst
View file @
4f32ff9e
...
@@ -9,8 +9,6 @@ Buildout Topics
...
@@ -9,8 +9,6 @@ Buildout Topics
variables-extending-and-substitutions
variables-extending-and-substitutions
optimizing
optimizing
bootstrapping
bootstrapping
.. todo:
writing-recipes
writing-recipes
.. todo:
doc/topics/writing-recipes.rst
0 → 100644
View file @
4f32ff9e
========================
Writing Buildout recipes
========================
There are two kinds of buildout recipes: *install* and
*uninstall*. Install recipes are by far the most common. Uninstall
recipes are very rarely needed because most install recipes add files and
directories that can be removed by Buildout.
Install recipes
===============
Install recipes are typically implemented with classes and have 3
important parts:
- A constructor (typically, ``__init__``) initializes a recipe object.
The constructor plays a very important role, because it may update
the configuration data it's passed, making information available to
other parts and controlling whether a part will need to be
re-installed.
The constructor performs the first of two phases of recipe work, the
second phase being the responsibility of either the ``install`` or
``update`` methods.
- The ``install`` method installs new parts.
- The ``update`` method updates previously installed parts. It's
often an empty method or an alias for ``install``.
Buildout phases
---------------
When buildout is run using the default :ref:`install command
<install-command>`, parts are installed in several phases:
1. Parts are initialized by calling their recipe constructors. This may
cause part configuration options to be updated, as described below.
2. Part options are compared to part options from previous runs
[#installed]_.
- Parts from previous runs that are no longer part of the buildout
are uninstalled.
- Parts from previous runs whose options have changed are also
uninstalled.
3. Parts are either installed or updated.
``install()`` is called on new parts or old parts that were uninstalled.
``update()`` is called on old parts whose configuration hasn't changed.
Initialization phase: the constructor
-------------------------------------
The constructor is passed 3 arguments:
``buildout``
The buildout configuration
The buildout configuration is a mapping from section names to
sections. Sections are mappings from option names to values. The
buildout configuration allows the recipe to access configuration
data in much the same way as configuration files use :ref:`value
substitutions <value-substitutions>`.
``name``
The name of the section the recipe was used for
``options``
The part options
This is a mapping object and may be written to to save derived
configuration, to provide information for use by other part
recipes, or for :ref:`value substitutions <value-substitutions>`.
Nothing should be installed in this phase.
If the part being installed isn't new, options after calling the
constructor are compared to the options from the previous Buildout
run. If they are different, then the part will be uninstalled and then
re-installed by calling the ``install`` method, otherwise, the ``update``
method will be called.
Install or update phase
-----------------------
In this phase, ``install()`` or ``update()`` is called, depending on
whether the part is new or has new configuration.
This is the phase in which the part does its work. In addition to
affecting changes, these methods have some responsibilities that can
be a little delicate:
- If an error is raised, it is the responsibility of the recipe to
undo any partial changes.
- If the recipe created any files or directories, the recipe should
return their paths. Doing so allows Buildout to take care of
removing them if the part is uninstalled, making a separate
uninstall recipe unnecessary.
To make these responsibilities easier to cope with, the ``option``
object passed to the constructor has a helper function, ``created``.
It should be passed one or more paths just before they are created and
returns a list of all of the paths passed as well as any earlier paths
created. If an exception is raised, any files or directories created
will be removed automatically. When the recipe returns, it can just
return the result of calling ``created()`` with no arguments.
Example: configuration from template recipe
-------------------------------------------
In this example, we'll show a recipe that creates a configuration file
based on a configuration string computed using value substitutions
[#templaterecipes]_. A sample usage:
.. code-block:: ini
[buildout]
develop = src
parts = server
[config]
recipe = democonfigrecipe
port = 8080
contents =
<zeo>
address ${:port}
</zeo>
<mappingstorage>
</mappingstorage>
[server]
recipe = zc.zdaemonrecipe
program = runzeo -C ${config:path}
.. -> src
>>> write(src, 'buildout.cfg')
Some things to note about this example:
- The ``config`` part uses the recipe whose source code we'll show
below. It has a ``port`` option, which it uses in its ``contents``
option. It could as easily have used options from other sections.
- The ``server`` part uses ``${config:path}`` to get the path to the
configuration file generated by the ``config`` part. The ``path``
option value will be computed by the recipe for use in other parts,
as we've seen here.
- We didn't have to list the ``config`` part in the buildout ``parts``
option. It's :doc:`added implicitly <implicit-parts>` by virtue of
its use in the ``server`` part.
- We used the ``develop`` option to specify a ``src`` directory
containing our recipe. This allows us to use the recipe locally
without having to build a distribution file.
If we were to run this buildout, a ``parts/config`` file would be
generated:
.. code-block:: xml
<zeo>
address 8080
</zeo>
<mappingstorage>
</mappingstorage>
.. -> config
as would a zdaemon configuration file, ``parts/server/zdaemon.conf``, like:
.. code-block:: xml
<runner>
daemon on
directory /sample/parts/server
program runzeo -C /sample/parts/config
socket-name /sample/parts/server/zdaemon.sock
transcript /sample/parts/server/transcript.log
</runner>
<eventlog>
<logfile>
path /sample/parts/server/transcript.log
</logfile>
</eventlog>
.. -> server
>>> server = server.replace('/sample', os.getcwd())
Here's the recipe source, ``src/democonfigrecipe.py``::
import os
class Recipe:
def __init__(self, buildout, name, options):
options['path'] = os.path.join(
buildout['buildout']['parts-directory'],
name,
)
self.options = options
def install(self):
self.options.created(self.options['path'])
with open(self.options['path'], 'w') as f:
f.write(self.options['contents'])
return self.options.created()
updated = install
.. -> src
>>> os.mkdir('src')
>>> write(src, 'src', 'democonfigrecipe.py')
The constructor computes the ``path`` option. This is then available
for use by the ``server`` part above. It's also used later in the
``install`` method. We use
``buildout['buildout']['parts-directory']`` to get the buildout parts
directory. This is equivalent to using ``${buildout:parts-directory}``
in the configuration. The parts directory is the standard place for
recipes to create files or directories. If a recipe uses the parts
directory, it should create only one file or directory whose name is
the part name, which is passed in as the ``name`` argument to the
constructor.
The constructor saves the options so that the data and ``created``
method are available in ``install``.
The ``install`` method calls the option object's ``created`` method
**before** creating a file. The order is important, because if the
file-creation fails partially, the file will be removed automatically.
The recipe itself doesn't need an exception handler. The configuration
file is then written out. Finally, the ``created`` method is called
again [#boilerplate]_ to return the list of created files (one, in
this case).
The ``update`` method is just an alias for the ``install`` method. We
could have used an empty method, however running install again makes
sure the file contents are as expected, overwriting manual changes, if
any.
Like the ``install`` method, the ``update`` method returns any paths
it created. These are merged with values returned by the ``install`` or
``update`` in previous runs.
For this recipe to be usable, we need to make it available as a
distribution [#even-though-develop]_, so we need to create a setup
script, ``src/setup.py``::
from setuptools import setup
setup(
name='democonfigrecipe',
version='0.1.0',
py_modules = ['democonfigrecipe'],
entry_points = {"zc.buildout": ["default=democonfigrecipe:Recipe"]},
)
.. -> src
>>> write(src, 'src', 'setup.py')
>>> run_buildout()
>>> eq(config.strip(), read('parts', 'config'))
>>> eq(server, read('parts', 'server', 'zdaemon.conf'))
Run again, nothing changes:
>>> run_buildout()
>>> eq(config.strip(), read('parts', 'config'))
>>> eq(server, read('parts', 'server', 'zdaemon.conf'))
The setup script specifies a name and version and lists the module to
be included.
The setup script also uses an ``entry_points`` option. Entry points
provide a `miniature component systems for setuptools
<https://setuptools.readthedocs.io/en/latest/setuptools.html#extensible-applications-and-frameworks>`_.
A project can supply named components of given types. In the example
above, the type of the component is ``"zc.buildout"``, which is the
type used for Buildout recipes. A single components named ``default``
is provided. The component is named as the ``Recipe`` attribute of
the ``democonfigrecipe`` module. When you specify a recipe in the
``recipe`` option, you name a recipe requirement, which names a
project, and optionally provide a recipe name. The default name is
``default``. Most recipe projects provide a single recipe component
named ``default``.
If we removed the ``server`` part from the configuration, the
two configuration files would be removed, because Buildout recorded
their paths and would remove them automatically.
.. Oh yeah?
>>> write(read('buildout.cfg').replace('parts = server', 'parts ='),
... 'buildout.cfg')
>>> run_buildout()
>>> eqs(ls('parts'))
Uninstall recipes
=================
Uninstall recipes are very rarely needed, because most recipes just
install files and Buildout can handle those automatically.
An uninstall recipe is just a function that takes a name and an
options mapping. One of the few packages with an uninstall recipe is
`zc.recipe.rhrc
<https://github.com/zopefoundation/zc.recipe.rhrc/blob/master/src/zc/recipe/rhrc/__init__.py#L183>`_.
The ``uninstall`` function there provides the uninstall recipe.
Here's a **highly simplified** version::
def uninstall(name, options):
os.system('/sbin/chkconfig --del ' + name)
.. -> src
This was used with a recipe that installed services on older Red Hat
Linux servers. When the part was uninstalled, it needed to run
``/sbin/chkconfig`` to disable the service. Uninstall recipes don't
need to return anything.
Like uninstall recipes, install recipes need to be registered using
entry points, using the type ``zc.buildout.uninstall`` as can be seen
in the `zc.recipe.rhrc setup script
<https://github.com/zopefoundation/zc.recipe.rhrc/blob/master/setup.py#L23>`_.
.. [#installed] Configuration data from previous runs are saved in a
buildout's installed database, :ref:`typically saved in
<installed-option>` a generated ``.installed.cfg`` file.
.. [#templaterecipes] There are a variety of template recipes that
provide different features, like using template files and
supporting various template engines. Don't re-use the example here.
.. [#boilerplate] Unfortunately, returning the result of calling
``created()`` is boilerplate. Future versions of buildout `won't
require this return <https://github.com/buildout/buildout/issues/357>`_.
.. [#even-though-develop] Even though we aren't distributing the
recipe in this example, we still need to create a :ref:`develop
distribution <python-development-projects>` so that Buildout can
find the recipe and its meta data.
src/zc/buildout/buildout.py
View file @
4f32ff9e
...
@@ -1829,17 +1829,10 @@ Options and assignments can be interspersed.
...
@@ -1829,17 +1829,10 @@ Options and assignments can be interspersed.
Commands:
Commands:
install
[parts]
install
Install parts. If no command arguments are given, then the parts
Install the parts specified in the buildout configuration. This is
definition from the configuration file is used. Otherwise, the
the default command if no command is specified.
arguments specify the parts to be installed.
Note that the semantics differ depending on whether any parts are
specified. If parts are specified, then only those parts will be
installed. If no parts are specified, then the parts specified by
the buildout parts option will be installed along with all of
their dependencies.
bootstrap
bootstrap
...
@@ -1847,12 +1840,17 @@ Commands:
...
@@ -1847,12 +1840,17 @@ Commands:
the buildout and setuptools eggs and, creating a basic directory
the buildout and setuptools eggs and, creating a basic directory
structure and a buildout-local buildout script.
structure and a buildout-local buildout script.
init
init
[requirements]
Initialize a buildout, creating a buildout.cfg file if it doesn't
Initialize a buildout, creating a
minimal
buildout.cfg file if it doesn't
exist and then performing the same actions as for the b
uildout
exist and then performing the same actions as for the b
ootstrap
command.
command.
If requirements are supplied, then the generated configuration
will include an interpreter script that requires them. This
provides an easy way to quickly set up a buildout to experiment
with some packages.
setup script [setup command and options]
setup script [setup command and options]
Run a given setup script arranging that setuptools is in the
Run a given setup script arranging that setuptools is in the
...
...
src/zc/buildout/tests.py
View file @
4f32ff9e
...
@@ -3760,8 +3760,8 @@ def test_suite():
...
@@ -3760,8 +3760,8 @@ def test_suite():
if
process
.
is_alive
()
or
process
.
exitcode
:
if
process
.
is_alive
()
or
process
.
exitcode
:
print
(
read
())
print
(
read
())
def
read
(
path
=
'out'
):
def
read
(
path
=
'out'
,
*
rest
):
with
open
(
path
)
as
f
:
with
open
(
os
.
path
.
join
(
path
,
*
rest
)
)
as
f
:
return
f
.
read
()
return
f
.
read
()
def
write
(
text
,
*
path
):
def
write
(
text
,
*
path
):
...
@@ -3789,7 +3789,8 @@ def test_suite():
...
@@ -3789,7 +3789,8 @@ def test_suite():
write
=
write
,
write
=
write
,
ls
=
lambda
d
=
'.'
,
*
rest
:
os
.
listdir
(
os
.
path
.
join
(
d
,
*
rest
)),
ls
=
lambda
d
=
'.'
,
*
rest
:
os
.
listdir
(
os
.
path
.
join
(
d
,
*
rest
)),
join
=
os
.
path
.
join
,
join
=
os
.
path
.
join
,
clear_here
=
clear_here
clear_here
=
clear_here
,
os
=
os
,
)
)
setupstack
.
setUpDirectory
(
test
)
setupstack
.
setUpDirectory
(
test
)
...
@@ -3802,6 +3803,7 @@ def test_suite():
...
@@ -3802,6 +3803,7 @@ def test_suite():
os
.
path
.
join
(
os
.
path
.
join
(
docdir
,
docdir
,
'topics'
,
'variables-extending-and-substitutions.rst'
),
'topics'
,
'variables-extending-and-substitutions.rst'
),
os
.
path
.
join
(
docdir
,
'topics'
,
'writing-recipes.rst'
),
os
.
path
.
join
(
docdir
,
'topics'
,
'optimizing.rst'
),
os
.
path
.
join
(
docdir
,
'topics'
,
'optimizing.rst'
),
setUp
=
docSetUp
,
tearDown
=
setupstack
.
tearDown
setUp
=
docSetUp
,
tearDown
=
setupstack
.
tearDown
))
))
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment