Copy in current versions of ZConfig and zdaemon.

trunk/src/ZConfig/NEWS.txt

+Changes since ZConfig 2.1:
+
+- More documentation has been written.
+
+- Added a timedelta datatype function; the input is the same as for
+  the time-interval datatype, but the resulting value is a
+  datetime.timedelta object.
+
+- Make sure keys specified as attributes of the <default> element are
+  converted by the appropriate key type, and are re-checked for
+  derived sections.
+
+- Refactored the ZConfig.components.logger schema components so that a
+  schema can import just one of the "eventlog" or "logger" sections if
+  desired.  This can be helpful to avoid naming conflicts.
+
+- Added a reopen() method to the logger factories.
+
+- Always use an absolute pathname when opening a FileHandler.
+
+- A fix to the logger 'format' key to allow the %(process)d expansion variable
+  that the logging package supports.
+
+- A new timedelta built-in datatype was added.  Similar to time-interval
+  except that it returns a datetime.timedelta object instead.
+
+Changes since ZConfig 2.0:
+
+- Removed compatibility with Python 2.1 and 2.2.
+
+- Schema components must really be in Python packages; the directory
+  search has been modified to perform an import to locate the package
+  rather than incorrectly implementing the search algorithm.
+
+- The default objects use for section values now provide a method
+  getSectionAttributes(); this returns a list of all the attributes of
+  the section object which store configuration-defined data (including
+  information derived from the schema).
+
+- Default information can now be included in a schema for <key
+  name="+"> and <multikey name="+"> by using <default key="...">.
+
+- More documentation has been added to discuss schema extension.
+
+- Support for a Unicode-free Python has been fixed.
+
+- Derived section types now inherit the datatype of the base type if
+  no datatype is identified explicitly.
+
+- Derived section types can now override the keytype instead of always
+  inheriting from their base type.
+
+- <import package='...'/> makes use of the current prefix if the
+  package name begins witha dot.
+
+- Added two standard datatypes:  dotted-name and dotted-suffix.
+
+- Added two standard schema components: ZConfig.components.basic and
+  ZConfig.components.logger.
+
+Changes since ZConfig 1.0:
+
+- Configurations can import additional schema components using a new
+  "%import" directive; this can be used to integrate 3rd-party
+  components into an application.
+
+- Schemas may be extended using a new "extends" attribute on the
+  <schema> element.
+
+- Better error messages when elements in a schema definition are
+  improperly nested.
+
+- The "zconfig" script can now simply verify that a schema definition
+  is valid, if that's all that's needed.

trunk/src/ZConfig/PACKAGE.cfg

+# Load the license from an external source, so we don't have to keep a
+# copy of it sitting around:
+<load>
+  LICENSE.txt  http://cvs.zope.org/Zope3/ZopePublicLicense.txt?rev=HEAD
+</load>
+
+# Add a few things to the distribution root.
+<distribution>
+  doc
+  LICENSE.txt
+  NEWS.txt
+  README.txt
+</distribution>
+
+# Specify what is included in the component.
+<collection>
+  # Python modules from the package:
+  *.py
+
+  # Child packages:
+  components
+  tests
+
+  # Other files and directories needed when distutils runs:
+  scripts
+</collection>

trunk/src/ZConfig/PUBLICATION.cfg

+Metadata-Version: 1.1
+Name: ZConfig
+Summary: Structured Configuration Library
+Home-page: http://www.zope.org/Members/fdrake/zconfig/
+Author: Fred L. Drake, Jr.
+Author-email: fred@zope.com
+License: ZPL 2
+Description: ZConfig is a configuration library intended for general use.  It
+        supports a hierarchical schema-driven configuration model that allows
+        a schema to specify data conversion routines written in Python.
+        ZConfig's model is very different from the model supported by the
+        ConfigParser module found in Python's standard library, and is more
+        suitable to configuration-intensive applications.
+        
+        ZConfig schema are written in an XML-based language and are able to
+        "import" schema components provided by Python packages.  Since
+        components are able to bind to conversion functions provided by Python
+        code in the package (or elsewhere), configuration objects can be
+        arbitrarily complex, with values that have been verified against
+        arbitrary constraints.  This makes it easy for applications to
+        separate configuration support from configuration loading even with
+        configuration data being defined and consumed by a wide range of
+        separate packages.
+Platform: POSIX
+Platform: Windows
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: System Administrators
+Classifier: License :: OSI Approved :: Zope Public License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python
+Classifier: Topic :: Software Development :: Libraries :: Python Modules

trunk/src/ZConfig/README.txt

+This is ZConfig.
+
+ZConfig is a configuration library intended for general use.  It
+supports a hierarchical schema-driven configuration model that allows
+a schema to specify data conversion routines written in Python.
+ZConfig's model is very different from the model supported by the
+ConfigParser module found in Python's standard library, and is more
+suitable to configuration-intensive applications.
+
+ZConfig schema are written in an XML-based language and are able to
+"import" schema components provided by Python packages.  Since
+components are able to bind to conversion functions provided by Python
+code in the package (or elsewhere), configuration objects can be
+arbitrarily complex, with values that have been verified against
+arbitrary constraints.  This makes it easy for applications to
+separate configuration support from configuration loading even with
+configuration data being defined and consumed by a wide range of
+separate packages.
+
+ZConfig is licensed under the Zope Public License, version 2.1.  See
+the file LICENSE.txt in the distribution for the full license text.
+
+Reference documentation is available in the doc/ directory.
+
+Information on the latest released version of the ZConfig package is
+available at
+
+  http://www.zope.org/Members/fdrake/zconfig/
+
+You may either create an RPM and install this, or install directly from
+the source distribution.
+
+
+Creating RPMS:
+
+  python setup.py bdist_rpm
+
+  If you need to force the Python interpreter to, for example, python2:
+
+    python2 setup.py bdist_rpm --python=python2
+
+
+Installation from the source distribution:
+
+    python setup.py install
+
+  To install to a user's home-dir:
+    python setup.py install --home=<dir>
+
+  To install to another prefix (eg. /usr/local)
+    python setup.py install --prefix=/usr/local
+
+  If you need to force the python interpreter to e.g. python2:
+    python2 setup.py install
+
+  For more information please refer to
+    http://www.python.org/doc/current/inst/inst.html

trunk/src/ZConfig/SETUP.cfg

+# Metadata used by zpkg.
+
+#documentation  doc/zconfig.pdf
+#documentation  doc/schema.dtd
+
+script scripts/zconfig*

trunk/src/ZConfig/__init__.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Configuration data structures and loader for the ZRS.
+
+$Id: __init__.py,v 1.18 2004/04/15 20:33:32 fdrake Exp $
+"""
+version_info = (2, 2)
+__version__ = ".".join([str(n) for n in version_info])
+
+from ZConfig.loader import loadConfig, loadConfigFile
+from ZConfig.loader import loadSchema, loadSchemaFile
+
+
+class ConfigurationError(Exception):
+    """Base class for ZConfig exceptions."""
+
+    def __init__(self, msg, url=None):
+        self.message = msg
+        self.url = url
+        Exception.__init__(self, msg)
+
+    def __str__(self):
+        return self.message
+
+
+class _ParseError(ConfigurationError):
+    def __init__(self, msg, url, lineno, colno=None):
+        self.lineno = lineno
+        self.colno = colno
+        ConfigurationError.__init__(self, msg, url)
+
+    def __str__(self):
+        s = self.message
+        if self.url:
+            s += "\n("
+        elif (self.lineno, self.colno) != (None, None):
+            s += " ("
+        if self.lineno:
+            s += "line %d" % self.lineno
+            if self.colno is not None:
+                s += ", column %d" % self.colno
+            if self.url:
+                s += " in %s)" % self.url
+            else:
+                s += ")"
+        elif self.url:
+            s += self.url + ")"
+        return s
+
+
+class SchemaError(_ParseError):
+    """Raised when there's an error in the schema itself."""
+
+    def __init__(self, msg, url=None, lineno=None, colno=None):
+        _ParseError.__init__(self, msg, url, lineno, colno)
+
+
+class SchemaResourceError(SchemaError):
+    """Raised when there's an error locating a resource required by the schema.
+    """
+
+    def __init__(self, msg, url=None, lineno=None, colno=None,
+                 path=None, package=None, filename=None):
+        self.filename = filename
+        self.package = package
+        if path is not None:
+            path = path[:]
+        self.path = path
+        SchemaError.__init__(self, msg, url, lineno, colno)
+
+    def __str__(self):
+        s = SchemaError.__str__(self)
+        if self.package is not None:
+            s += "\n  Package name: " + repr(self.package)
+        if self.filename is not None:
+            s += "\n  File name: " + repr(self.filename)
+        if self.package is not None:
+            s += "\n  Package path: " + repr(self.path)
+        return s
+
+
+class ConfigurationSyntaxError(_ParseError):
+    """Raised when there's a syntax error in a configuration file."""
+
+
+class DataConversionError(ConfigurationError, ValueError):
+    """Raised when a data type conversion function raises ValueError."""
+
+    def __init__(self, exception, value, position):
+        ConfigurationError.__init__(self, str(exception))
+        self.exception = exception
+        self.value = value
+        self.lineno, self.colno, self.url = position
+
+    def __str__(self):
+        s = "%s (line %s" % (self.message, self.lineno)
+        if self.colno is not None:
+            s += ", %s" % self.colno
+        if self.url:
+            s += ", in %s)" % self.url
+        else:
+            s += ")"
+        return s
+
+
+class SubstitutionSyntaxError(ConfigurationError):
+    """Raised when interpolation source text contains syntactical errors."""
+
+
+class SubstitutionReplacementError(ConfigurationSyntaxError, LookupError):
+    """Raised when no replacement is available for a reference."""
+
+    def __init__(self, source, name, url=None, lineno=None):
+        self.source = source
+        self.name = name
+        ConfigurationSyntaxError.__init__(
+            self, "no replacement for " + `name`, url, lineno)

trunk/src/ZConfig/cfgparser.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Configuration parser."""
+
+import ZConfig
+import ZConfig.url
+
+from ZConfig.substitution import isname, substitute
+
+
+class ZConfigParser:
+    __metaclass__ = type
+    __slots__ = ('resource', 'context', 'lineno',
+                 'stack', 'defs', 'file', 'url')
+
+    def __init__(self, resource, context, defines=None):
+        self.resource = resource
+        self.context = context
+        self.file = resource.file
+        self.url = resource.url
+        self.lineno = 0
+        self.stack = []   # [(type, name, prevmatcher), ...]
+        if defines is None:
+            defines = {}
+        self.defs = defines
+
+    def nextline(self):
+        line = self.file.readline()
+        if line:
+            self.lineno += 1
+            return False, line.strip()
+        else:
+            return True, None
+
+    def parse(self, section):
+        done, line = self.nextline()
+        while not done:
+            if line[:1] in ("", "#"):
+                # blank line or comment
+                pass
+
+            elif line[:2] == "</":
+                # section end
+                if line[-1] != ">":
+                    self.error("malformed section end")
+                section = self.end_section(section, line[2:-1])
+
+            elif line[0] == "<":
+                # section start
+                if line[-1] != ">":
+                    self.error("malformed section start")
+                section = self.start_section(section, line[1:-1])
+
+            elif line[0] == "%":
+                self.handle_directive(section, line[1:])
+
+            else:
+                self.handle_key_value(section, line)
+
+            done, line = self.nextline()
+
+        if self.stack:
+            self.error("unclosed sections not allowed")
+
+    def start_section(self, section, rest):
+        isempty = rest[-1:] == "/"
+        if isempty:
+            text = rest[:-1].rstrip()
+        else:
+            text = rest.rstrip()
+        # parse section start stuff here
+        m = _section_start_rx.match(text)
+        if not m:
+            self.error("malformed section header")
+        type, name = m.group('type', 'name')
+        type = type.lower()
+        if name:
+            name = name.lower()
+        try:
+            newsect = self.context.startSection(section, type, name)
+        except ZConfig.ConfigurationError, e:
+            self.error(e[0])
+
+        if isempty:
+            self.context.endSection(section, type, name, newsect)
+            return section
+        else:
+            self.stack.append((type, name, section))
+            return newsect
+
+    def end_section(self, section, rest):
+        if not self.stack:
+            self.error("unexpected section end")
+        type = rest.rstrip().lower()
+        opentype, name, prevsection = self.stack.pop()
+        if type != opentype:
+            self.error("unbalanced section end")
+        try:
+            self.context.endSection(
+                prevsection, type, name, section)
+        except ZConfig.ConfigurationError, e:
+            self.error(e[0])
+        return prevsection
+
+    def handle_key_value(self, section, rest):
+        m = _keyvalue_rx.match(rest)
+        if not m:
+            self.error("malformed configuration data")
+        key, value = m.group('key', 'value')
+        if not value:
+            value = ''
+        else:
+            value = self.replace(value)
+        try:
+            section.addValue(key, value, (self.lineno, None, self.url))
+        except ZConfig.ConfigurationError, e:
+            self.error(e[0])
+
+    def handle_directive(self, section, rest):
+        m = _keyvalue_rx.match(rest)
+        if not m:
+            self.error("missing or unrecognized directive")
+        name, arg = m.group('key', 'value')
+        if name not in ("define", "import", "include"):
+            self.error("unknown directive: " + `name`)
+        if not arg:
+            self.error("missing argument to %%%s directive" % name)
+        if name == "include":
+            self.handle_include(section, arg)
+        elif name == "define":
+            self.handle_define(section, arg)
+        elif name == "import":
+            self.handle_import(section, arg)
+        else:
+            assert 0, "unexpected directive for " + `"%" + rest`
+
+    def handle_import(self, section, rest):
+        pkgname = self.replace(rest.strip())
+        self.context.importSchemaComponent(pkgname)
+
+    def handle_include(self, section, rest):
+        rest = self.replace(rest.strip())
+        newurl = ZConfig.url.urljoin(self.url, rest)
+        self.context.includeConfiguration(section, newurl, self.defs)
+
+    def handle_define(self, section, rest):
+        parts = rest.split(None, 1)
+        defname = parts[0].lower()
+        defvalue = ''
+        if len(parts) == 2:
+            defvalue = parts[1]
+        if self.defs.has_key(defname):
+            self.error("cannot redefine " + `defname`)
+        if not isname(defname):
+            self.error("not a substitution legal name: " + `defname`)
+        self.defs[defname] = self.replace(defvalue)
+
+    def replace(self, text):
+        try:
+            return substitute(text, self.defs)
+        except ZConfig.SubstitutionReplacementError, e:
+            e.lineno = self.lineno
+            e.url = self.url
+            raise
+
+    def error(self, message):
+        raise ZConfig.ConfigurationSyntaxError(message, self.url, self.lineno)
+
+
+import re
+# _name_re does not allow "(" or ")" for historical reasons.  Though
+# the restriction could be lifted, there seems no need to do so.
+_name_re = r"[^\s()]+"
+_keyvalue_rx = re.compile(r"(?P<key>%s)\s*(?P<value>[^\s].*)?$"
+                          % _name_re)
+_section_start_rx = re.compile(r"(?P<type>%s)"
+                               r"(?:\s+(?P<name>%s))?"
+                               r"$"
+                               % (_name_re, _name_re))
+del re

trunk/src/ZConfig/cmdline.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""Support for command-line provision of settings.
+
+This module provides an extension of the ConfigLoader class which adds
+a way to add configuration settings from an alternate source.  Each
+setting is described by a string of the form::
+
+    some/path/to/key=value
+"""
+
+import ZConfig
+import ZConfig.loader
+import ZConfig.matcher
+
+
+class ExtendedConfigLoader(ZConfig.loader.ConfigLoader):
+    def __init__(self, schema):
+        ZConfig.loader.ConfigLoader.__init__(self, schema)
+        self.clopts = []   # [(optpath, value, source-position), ...]
+
+    def addOption(self, spec, pos=None):
+        if pos is None:
+            pos = "<command-line option>", -1, -1
+        if "=" not in spec:
+            e = ZConfig.ConfigurationSyntaxError(
+                "invalid configuration specifier", *pos)
+            e.specifier = spec
+            raise e
+        # For now, just add it to the list; not clear that checking
+        # against the schema at this point buys anything.
+        opt, val = spec.split("=", 1)
+        optpath = opt.split("/")
+        if "" in optpath:
+            # // is not allowed in option path
+            e = ZConfig.ConfigurationSyntaxError(
+                "'//' is not allowed in an option path", *pos)
+            e.specifier = spec
+            raise e
+        self.clopts.append((optpath, val, pos))
+
+    def createSchemaMatcher(self):
+        if self.clopts:
+            sm = ExtendedSchemaMatcher(self.schema)
+            sm.set_optionbag(self.cook())
+        else:
+            sm = ZConfig.loader.ConfigLoader.createSchemaMatcher(self)
+        return sm
+
+    def cook(self):
+        if self.clopts:
+            return OptionBag(self.schema, self.schema, self.clopts)
+        else:
+            return None
+
+
+class OptionBag:
+    def __init__(self, schema, sectiontype, options):
+        self.sectiontype = sectiontype
+        self.schema = schema
+        self.keypairs = {}
+        self.sectitems = []
+        self._basic_key = schema.registry.get("basic-key")
+        for item in options:
+            optpath, val, pos = item
+            name = sectiontype.keytype(optpath[0])
+            if len(optpath) == 1:
+                self.add_value(name, val, pos)
+            else:
+                self.sectitems.append(item)
+
+    def basic_key(self, s, pos):
+        try:
+            return self._basic_key(s)
+        except ValueError:
+            raise ZConfig.ConfigurationSyntaxError(
+                "could not convert basic-key value", *pos)
+
+    def add_value(self, name, val, pos):
+        if self.keypairs.has_key(name):
+            L = self.keypairs[name]
+        else:
+            L = []
+            self.keypairs[name] = L
+        L.append((val, pos))
+
+    def has_key(self, name):
+        return self.keypairs.has_key(name)
+
+    def get_key(self, name):
+        """Return a list of (value, pos) items for the key 'name'.
+
+        The returned list may be empty.
+        """
+        L = self.keypairs.get(name)
+        if L:
+            del self.keypairs[name]
+            return L
+        else:
+            return []
+
+    def keys(self):
+        return self.keypairs.keys()
+
+    def get_section_info(self, type, name):
+        L = []  # what pertains to the child section
+        R = []  # what we keep
+        for item in self.sectitems:
+            optpath, val, pos = item
+            s = optpath[0]
+            bk = self.basic_key(s, pos)
+            if name and s.lower() == name:
+                L.append((optpath[1:], val, pos))
+            elif bk == type:
+                L.append((optpath[1:], val, pos))
+            else:
+                R.append(item)
+        if L:
+            self.sectitems[:] = R
+            return OptionBag(self.schema, self.schema.gettype(type), L)
+        else:
+            return None
+
+    def finish(self):
+        if self.sectitems or self.keypairs:
+            raise ZConfig.ConfigurationError(
+                "not all command line options were consumed")
+
+
+class MatcherMixin:
+    def set_optionbag(self, bag):
+        self.optionbag = bag
+
+    def addValue(self, key, value, position):
+        try:
+            realkey = self.type.keytype(key)
+        except ValueError, e:
+            raise ZConfig.DataConversionError(e, key, position)
+        if self.optionbag.has_key(realkey):
+            return
+        ZConfig.matcher.BaseMatcher.addValue(self, key, value, position)
+
+    def createChildMatcher(self, type, name):
+        sm = ZConfig.matcher.BaseMatcher.createChildMatcher(self, type, name)
+        bag = self.optionbag.get_section_info(type.name, name)
+        if bag is not None:
+            sm = ExtendedSectionMatcher(
+                sm.info, sm.type, sm.name, sm.handlers)
+            sm.set_optionbag(bag)
+        return sm
+
+    def finish_optionbag(self):
+        for key in self.optionbag.keys():
+            for val, pos in self.optionbag.get_key(key):
+                ZConfig.matcher.BaseMatcher.addValue(self, key, val, pos)
+        self.optionbag.finish()
+
+
+class ExtendedSectionMatcher(MatcherMixin, ZConfig.matcher.SectionMatcher):
+    def finish(self):
+        self.finish_optionbag()
+        return ZConfig.matcher.SectionMatcher.finish(self)
+
+class ExtendedSchemaMatcher(MatcherMixin, ZConfig.matcher.SchemaMatcher):
+    def finish(self):
+        self.finish_optionbag()
+        return ZConfig.matcher.SchemaMatcher.finish(self)

trunk/src/ZConfig/components/__init__.py

+# This is a Python package.

trunk/src/ZConfig/components/basic/__init__.py

+# This is a Python package.

trunk/src/ZConfig/components/basic/component.xml

+<component>
+  <description>
+    Convenient loader which causes all the "basic" components to be
+    loaded.
+  </description>
+
+  <import package="ZConfig.components.basic" file="mapping.xml"/>
+
+</component>

trunk/src/ZConfig/components/basic/mapping.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""Python datatype for the ZConfig.components.basic.mapping section type."""
+
+def mapping(section):
+    return section.mapping

trunk/src/ZConfig/components/basic/mapping.xml

+<component>
+
+  <sectiontype name="ZConfig.basic.mapping"
+               datatype="ZConfig.components.basic.mapping.mapping">
+    <description>
+      Section that provides a simple mapping implementation.  An
+      application should derive a more specific section type for use
+      in configuration files:
+
+        &lt;import package="ZConfig.components.basic"
+                     file="mapping.xml"
+                     /&gt;
+
+        &lt;sectiontype name="mapping"
+                     extends="ZConfig.basic.mapping"
+                     /&gt;
+
+      If a non-standard keytype is needed, it can be overridden as
+      well:
+
+        &lt;sectiontype name="system-map"
+                     extends="ZConfig.basic.mapping"
+                     keytype="mypkg.datatypes.system_name"
+                     /&gt;
+
+    </description>
+
+    <key name="+"
+         attribute="mapping"
+         required="no"
+         />
+  </sectiontype>
+
+</component>

trunk/src/ZConfig/components/basic/tests/__init__.py

+# This is a Python package.

trunk/src/ZConfig/components/basic/tests/test_mapping.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""Tests of the 'basic' section types provided as part of
+ZConfig.components.basic."""
+
+import unittest
+
+from ZConfig.tests import support
+
+
+SIMPLE_SCHEMA = '''\
+<schema>
+  <import package="ZConfig.components.basic" file="mapping.xml" />
+
+  <sectiontype name="dict"
+               extends="ZConfig.basic.mapping" />
+
+  <sectiontype name="intkeys"
+               extends="ZConfig.basic.mapping"
+               keytype="integer" />
+
+  <section name="*"
+           type="dict"
+           attribute="simple_dict" />
+
+  <section name="*"
+           type="intkeys"
+           attribute="int_dict" />
+
+</schema>
+'''
+
+
+class BasicSectionTypeTestCase(support.TestBase):
+    schema = None
+
+    def setUp(self):
+        if self.schema is None:
+            self.__class__.schema = self.load_schema_text(SIMPLE_SCHEMA)
+
+    def test_simple_empty_dict(self):
+        conf = self.load_config_text(self.schema, "<dict/>")
+        self.assertEqual(conf.simple_dict, {})
+        conf = self.load_config_text(self.schema, """\
+            <dict foo>
+            # comment
+            </dict>
+            """)
+        self.assertEqual(conf.simple_dict, {})
+
+    def test_simple_dict(self):
+        conf = self.load_config_text(self.schema, """\
+           <dict foo>
+           key-one value-one
+           key-two value-two
+           </dict>
+           """)
+        L = conf.simple_dict.items()
+        L.sort()
+        self.assertEqual(L, [("key-one", "value-one"),
+                             ("key-two", "value-two")])
+
+    def test_derived_dict(self):
+        conf = self.load_config_text(self.schema, """\
+            <intkeys>
+            1 foo
+            2 bar
+            42 question?
+            </intkeys>
+            """)
+        L = conf.int_dict.items()
+        L.sort()
+        self.assertEqual(L, [(1, "foo"), (2, "bar"), (42, "question?")])
+
+
+def test_suite():
+    return unittest.makeSuite(BasicSectionTypeTestCase)

trunk/src/ZConfig/components/logger/__init__.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""ZConfig schema component package for logging configuration."""
+
+# Make sure we can't import this if "logging" isn't available; we
+# don't want partial imports to appear to succeed.
+
+try:
+    import logging
+except ImportError:
+    import sys
+    del sys.modules[__name__]

trunk/src/ZConfig/components/logger/abstract.xml

+<component>
+  <description>
+  </description>
+
+  <abstracttype name="ZConfig.logger.handler"/>
+  <abstracttype name="ZConfig.logger.log"/>
+</component>

trunk/src/ZConfig/components/logger/base-logger.xml

+<component prefix="ZConfig.components.logger.logger">
+
+  <import package="ZConfig.components.logger" file="abstract.xml"/>
+
+  <sectiontype name="ZConfig.logger.base-logger">
+    <description>
+      Base definition for the logger types defined by
+      ZConfig.components.logger.  This exists entirely to provide
+      shared key definitions and documentation.
+    </description>
+
+    <key name="level"
+         datatype="ZConfig.components.logger.datatypes.logging_level"
+         default="info">
+      <description>
+        Verbosity setting for the logger.  Values must be a name of
+        a level, or an integer in the range [0..50].  The names of the
+        levels, in order of increasing verbosity (names on the same
+        line are equivalent):
+
+            critical, fatal
+            error
+            warn, warning
+            info
+            blather
+            debug
+            trace
+            all
+
+        The special name "notset", or the numeric value 0, indicates
+        that the setting for the parent logger should be used.
+
+        It is strongly recommended that names be used rather than
+        numeric values to ensure that configuration files can be
+        deciphered more easily.
+      </description>
+    </key>
+
+    <multisection type="ZConfig.logger.handler"
+                  attribute="handlers" name="*">
+      <description>
+        Handlers to install on this logger.  Each handler describes
+        how logging events should be presented.
+      </description>
+    </multisection>
+  </sectiontype>
+
+</component>

trunk/src/ZConfig/components/logger/component.xml

+<component prefix="ZConfig.components.logger.datatypes">
+  <description>
+  </description>
+
+  <import package="ZConfig.components.logger" file="abstract.xml"/>
+  <import package="ZConfig.components.logger" file="handlers.xml"/>
+  <import package="ZConfig.components.logger" file="logger.xml"/>
+  <import package="ZConfig.components.logger" file="eventlog.xml"/>
+
+</component>

trunk/src/ZConfig/components/logger/datatypes.py

+##############################################################################
+#
+# Copyright (c) 2002 Zope Corporation and Contributors. All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""ZConfig datatypes for logging support."""
+
+
+_logging_levels = {
+    "critical": 50,
+    "fatal": 50,
+    "error": 40,
+    "warn": 30,
+    "warning": 30,
+    "info": 20,
+    "blather": 15,
+    "debug": 10,
+    "trace": 5,
+    "all": 1,
+    "notset": 0,
+    }
+
+def logging_level(value):
+    s = str(value).lower()
+    if _logging_levels.has_key(s):
+        return _logging_levels[s]
+    else:
+        v = int(s)
+        if v < 0 or v > 50:
+            raise ValueError("log level not in range: " + `v`)
+        return v

trunk/src/ZConfig/components/logger/eventlog.xml

+<component prefix="ZConfig.components.logger.logger">
+
+  <import package="ZConfig.components.logger" file="abstract.xml"/>
+  <import package="ZConfig.components.logger" file="base-logger.xml"/>
+
+  <sectiontype name="eventlog"
+               datatype=".EventLogFactory"
+               extends="ZConfig.logger.base-logger"
+               implements="ZConfig.logger.log">
+    <description>
+      Configuration for the root logger.
+    </description>
+  </sectiontype>
+
+</component>

trunk/src/ZConfig/components/logger/factory.py

+##############################################################################
+#
+# Copyright (c) 2002 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+_marker = object()
+
+class Factory:
+    """Generic wrapper for instance construction.
+
+    Calling the factory causes the instance to be created if it hasn't
+    already been created, and returns the object.  Calling the factory
+    multiple times returns the same object.
+
+    The instance is created using the factory's create() method, which
+    must be overriden by subclasses.
+    """
+    def __init__(self):
+        self.instance = _marker
+
+    def __call__(self):
+        if self.instance is _marker:
+            self.instance = self.create()
+        return self.instance
+
+    def create(self):
+        raise NotImplementedError("subclasses need to override create()")

trunk/src/ZConfig/components/logger/handlers.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""ZConfig factory datatypes for log handlers."""
+
+import sys
+
+from ZConfig.components.logger.factory import Factory
+
+
+_log_format_variables = {
+    'name': '',
+    'levelno': '3',
+    'levelname': 'DEBUG',
+    'pathname': 'apath',
+    'filename': 'afile',
+    'module': 'amodule',
+    'lineno': 1,
+    'created': 1.1,
+    'asctime': 'atime',
+    'msecs': 1,
+    'relativeCreated': 1,
+    'thread': 1,
+    'message': 'amessage',
+    'process': 1,
+    }
+
+def log_format(value):
+    value = ctrl_char_insert(value)
+    try:
+        # Make sure the format string uses only names that will be
+        # provided, and has reasonable type flags for each, and does
+        # not expect positional args.
+        value % _log_format_variables
+    except (ValueError, KeyError):
+        raise ValueError, 'Invalid log format string %s' % value
+    return value
+
+_control_char_rewrites = {r'\n': '\n', r'\t': '\t', r'\b': '\b',
+                          r'\f': '\f', r'\r': '\r'}.items()
+
+def ctrl_char_insert(value):
+    for pattern, replacement in _control_char_rewrites:
+        value = value.replace(pattern, replacement)
+    return value
+
+
+class HandlerFactory(Factory):
+    def __init__(self, section):
+        Factory.__init__(self)
+        self.section = section
+
+    def create_loghandler(self):
+        raise NotImplementedError(
+            "subclasses must override create_loghandler()")
+
+    def create(self):
+        import logging
+        logger = self.create_loghandler()
+        logger.setFormatter(logging.Formatter(self.section.format,
+                                              self.section.dateformat))
+        logger.setLevel(self.section.level)
+        return logger
+
+    def getLevel(self):
+        return self.section.level
+
+class FileHandlerFactory(HandlerFactory):
+    def create_loghandler(self):
+        from ZConfig.components.logger import loghandler
+        path = self.section.path
+        if path == "STDERR":
+            handler = loghandler.StreamHandler(sys.stderr)
+        elif path == "STDOUT":
+            handler = loghandler.StreamHandler(sys.stdout)
+        else:
+            handler = loghandler.FileHandler(path)
+        return handler
+
+_syslog_facilities = {
+    "auth": 1,
+    "authpriv": 1,
+    "cron": 1,
+    "daemon": 1,
+    "kern": 1,
+    "lpr": 1,
+    "mail": 1,
+    "news": 1,
+    "security": 1,
+    "syslog": 1,
+    "user": 1,
+    "uucp": 1,
+    "local0": 1,
+    "local1": 1,
+    "local2": 1,
+    "local3": 1,
+    "local4": 1,
+    "local5": 1,
+    "local6": 1,
+    "local7": 1,
+    }
+
+def syslog_facility(value):
+    value = value.lower()
+    if not _syslog_facilities.has_key(value):
+        L = _syslog_facilities.keys()
+        L.sort()
+        raise ValueError("Syslog facility must be one of " + ", ".join(L))
+    return value
+
+class SyslogHandlerFactory(HandlerFactory):
+    def create_loghandler(self):
+        from ZConfig.components.logger import loghandler
+        return loghandler.SysLogHandler(self.section.address.address,
+                                        self.section.facility)
+
+class Win32EventLogFactory(HandlerFactory):
+    def create_loghandler(self):
+        from ZConfig.components.logger import loghandler
+        return loghandler.Win32EventLogHandler(self.section.appname)
+
+def http_handler_url(value):
+    import urlparse
+    scheme, netloc, path, param, query, fragment = urlparse.urlparse(value)
+    if scheme != 'http':
+        raise ValueError, 'url must be an http url'
+    if not netloc:
+        raise ValueError, 'url must specify a location'
+    if not path:
+        raise ValueError, 'url must specify a path'
+    q = []
+    if param:
+        q.append(';')
+        q.append(param)
+    if query:
+        q.append('?')
+        q.append(query)
+    if fragment:
+        q.append('#')
+        q.append(fragment)
+    return (netloc, path + ''.join(q))
+
+def get_or_post(value):
+    value = value.upper()
+    if value not in ('GET', 'POST'):
+        raise ValueError('method must be "GET" or "POST", instead received: '
+                         + repr(value))
+    return value
+
+class HTTPHandlerFactory(HandlerFactory):
+    def create_loghandler(self):
+        from ZConfig.components.logger import loghandler
+        host, selector = self.section.url
+        return loghandler.HTTPHandler(host, selector, self.section.method)
+
+class SMTPHandlerFactory(HandlerFactory):
+    def create_loghandler(self):
+        from ZConfig.components.logger import loghandler
+        host, port = self.section.smtp_server
+        if not port:
+            mailhost = host
+        else:
+            mailhost = host, port
+        return loghandler.SMTPHandler(mailhost,
+                                      self.section.fromaddr,
+                                      self.section.toaddrs,
+                                      self.section.subject)

trunk/src/ZConfig/components/logger/handlers.xml

+<component prefix="ZConfig.components.logger.handlers">
+  <description>
+  </description>
+
+  <import package="ZConfig.components.logger" file="abstract.xml"/>
+
+  <sectiontype name="ZConfig.logger.base-log-handler">
+    <description>
+      Base type for most log handlers.  This is cannot be used as a
+      loghandler directly since it doesn't implement the loghandler
+      abstract section type.
+    </description>
+    <key name="dateformat"
+         default="%Y-%m-%dT%H:%M:%S"/>
+    <key name="level"
+         default="notset"
+         datatype="ZConfig.components.logger.datatypes.logging_level"/>
+  </sectiontype>
+
+  <sectiontype name="logfile"
+               datatype=".FileHandlerFactory"
+               implements="ZConfig.logger.handler"
+               extends="ZConfig.logger.base-log-handler">
+    <key name="path" required="yes"/>
+    <key name="format"
+         default="------\n%(asctime)s %(levelname)s %(name)s %(message)s"
+         datatype=".log_format"/>
+  </sectiontype>
+
+  <sectiontype name="syslog"
+               datatype=".SyslogHandlerFactory"
+               implements="ZConfig.logger.handler"
+               extends="ZConfig.logger.base-log-handler">
+    <key name="facility" default="user" datatype=".syslog_facility"/>
+    <key name="address" datatype="socket-address" default="localhost:514"/>
+    <key name="format"
+         default="%(name)s %(message)s"
+         datatype=".log_format"/>
+  </sectiontype>
+
+  <sectiontype name="win32-eventlog"
+               datatype=".Win32EventLogFactory"
+               implements="ZConfig.logger.handler"
+               extends="ZConfig.logger.base-log-handler">
+    <key name="appname" default="Zope"/>
+    <key name="format"
+         default="%(levelname)s %(name)s %(message)s"
+         datatype=".log_format"/>
+  </sectiontype>
+
+  <sectiontype name="http-logger"
+               datatype=".HTTPHandlerFactory"
+               implements="ZConfig.logger.handler"
+               extends="ZConfig.logger.base-log-handler">
+    <key name="url" default="http://localhost/" datatype=".http_handler_url"/>
+    <key name="method" default="GET" datatype=".get_or_post"/>
+    <key name="format"
+         default="%(asctime)s %(levelname)s %(name)s %(message)s"
+         datatype=".log_format"/>
+  </sectiontype>
+
+  <sectiontype name="email-notifier"
+               datatype=".SMTPHandlerFactory"
+               implements="ZConfig.logger.handler"
+               extends="ZConfig.logger.base-log-handler">
+    <key name="from" required="yes" attribute="fromaddr"/>
+    <multikey name="to" required="yes" attribute="toaddrs"/>
+    <key name="subject" default="Message from Zope"/>
+    <key name="smtp-server" default="localhost" datatype="inet-address"/>
+    <key name="format"
+         default="%(asctime)s %(levelname)s %(name)s %(message)s"
+         datatype=".log_format"/>
+  </sectiontype>
+
+</component>

trunk/src/ZConfig/components/logger/logger.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""ZConfig factory datatypes for loggers."""
+
+from ZConfig.components.logger.factory import Factory
+
+
+class LoggerFactoryBase(Factory):
+    """Base class for logger factories.
+
+    Factory used to create loggers while delaying actual logger
+    instance construction.  We need to do this because we may want to
+    reference a logger before actually instantiating it (for example,
+    to allow the app time to set an effective user).  An instance of
+    this wrapper is a callable which, when called, returns a logger
+    object.
+    """
+
+    def __init__(self, section):
+        Factory.__init__(self)
+        self.level = section.level
+        self.handler_factories = section.handlers
+
+    def create(self):
+        # set the logger up
+        import logging
+        logger = logging.getLogger(self.name)
+        logger.setLevel(self.level)
+        if self.handler_factories:
+            for handler_factory in self.handler_factories:
+                handler = handler_factory()
+                logger.addHandler(handler)
+        else:
+            from ZConfig.components.logger import loghandler
+            logger.addHandler(loghandler.NullHandler())
+        return logger
+
+    def startup(self):
+        # make sure we've instantiated the logger
+        self()
+
+    def getLowestHandlerLevel(self):
+        """Return the lowest log level provided by any configured handler.
+
+        If all handlers and the logger itself have level==NOTSET, this
+        returns NOTSET.
+        """
+        import logging
+        lowest = self.level
+        for factory in self.handler_factories:
+            level = factory.getLevel()
+            if level != logging.NOTSET:
+                if lowest == logging.NOTSET:
+                    lowest = level
+                else:
+                    lowest = min(lowest, level)
+        return lowest
+
+    def reopen(self):
+        """Re-open any handlers for which this is a meaningful operation.
+
+        This only works on handlers on the logger provided by this
+        factory directly; handlers for child loggers are not affected.
+        (This can be considered a bug, but is sufficient at the
+        moment.)
+        """
+        logger = self()
+        for handler in logger.handlers:
+            reopen = getattr(handler, "reopen", None)
+            if reopen is not None and callable(reopen):
+                reopen()
+
+
+class EventLogFactory(LoggerFactoryBase):
+    """Logger factory that returns the root logger."""
+
+    name = None
+
+
+class LoggerFactory(LoggerFactoryBase):
+    """Logger factory that returns the named logger."""
+
+    def __init__(self, section):
+        LoggerFactoryBase.__init__(self, section)
+        self.name = section.name
+        self.propagate = section.propagate
+
+    def create(self):
+        logger = LoggerFactoryBase.create(self)
+        logger.propagate = self.propagate
+        return logger

trunk/src/ZConfig/components/logger/logger.xml

+<component prefix="ZConfig.components.logger.logger">
+  <description>
+  </description>
+
+  <import package="ZConfig.components.logger" file="abstract.xml"/>
+  <import package="ZConfig.components.logger" file="base-logger.xml"/>
+  <import package="ZConfig.components.logger" file="eventlog.xml"/>
+
+  <sectiontype name="logger"
+               datatype=".LoggerFactory"
+               extends="ZConfig.logger.base-logger"
+               implements="ZConfig.logger.log">
+    <key name="propagate"
+         datatype="boolean"
+         default="true">
+      <description>
+        Indicates whether events that reach this logger should be
+        propogated toward the root of the logger hierarchy.  If true
+        (the default), events will be passed to the logger's parent
+        after being handled.  If false, events will be handled and the
+        parent will not be informed.  There is not a way to control
+        propogation by the severity of the event.
+      </description>
+    </key>
+
+    <key name="name"
+         datatype="dotted-name"
+         required="yes">
+      <description>
+        The dotted name of the logger.  This give it a location in the
+        logging hierarchy.  Most applications provide a specific set
+        of subsystem names for which logging is meaning; consult the
+        application documentation for the set of names that are
+        actually interesting for the application.
+      </description>
+    </key>
+  </sectiontype>
+
+</component>

trunk/src/ZConfig/components/logger/loghandler.py

+##############################################################################
+#
+# Copyright (c) 2001 Zope Corporation and Contributors. All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""Handlers which can plug into a PEP 282 logger."""
+
+import os.path
+import sys
+
+from logging import Handler, StreamHandler
+from logging.handlers import SysLogHandler, BufferingHandler
+from logging.handlers import HTTPHandler, SMTPHandler
+from logging.handlers import NTEventLogHandler as Win32EventLogHandler
+
+
+class FileHandler(StreamHandler):
+    """File handler which supports reopening of logs.
+
+    Re-opening should be used instead of the 'rollover' feature of
+    the FileHandler from the standard library's logging package.
+    """
+
+    def __init__(self, filename, mode="a"):
+        filename = os.path.abspath(filename)
+        StreamHandler.__init__(self, open(filename, mode))
+        self.baseFilename = filename
+        self.mode = mode
+
+    def close(self):
+        self.stream.close()
+
+    def reopen(self):
+        self.close()
+        self.stream = open(self.baseFilename, self.mode)
+
+
+class NullHandler(Handler):
+    """Handler that does nothing."""
+
+    def emit(self, record):
+        pass
+
+    def handle(self, record):
+        pass
+
+
+class StartupHandler(BufferingHandler):
+    """Handler which stores messages in a buffer until later.
+
+    This is useful at startup before we can know that we can safely
+    write to a configuration-specified handler.
+    """
+
+    def __init__(self):
+        BufferingHandler.__init__(self, sys.maxint)
+
+    def shouldFlush(self, record):
+        return False
+
+    def flushBufferTo(self, target):
+        while self.buffer:
+            target.handle(self.buffer.pop(0))

trunk/src/ZConfig/components/logger/tests/__init__.py

+# This is a Python package.

trunk/src/ZConfig/components/logger/tests/test_logger.py

+##############################################################################
+#
+# Copyright (c) 2002 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""Tests for logging configuration via ZConfig."""
+
+import cStringIO as StringIO
+import logging
+import sys
+import tempfile
+import unittest
+
+import ZConfig
+
+from ZConfig.components.logger import datatypes
+from ZConfig.components.logger import handlers
+from ZConfig.components.logger import loghandler
+
+
+class LoggingTestBase(unittest.TestCase):
+
+    # XXX This tries to save and restore the state of logging around
+    # the test.  Somewhat surgical; there may be a better way.
+
+    name = None
+
+    def setUp(self):
+        self._old_logger = logging.getLogger(self.name)
+        self._old_level = self._old_logger.level
+        self._old_handlers = self._old_logger.handlers[:]
+        self._old_logger.handlers[:] = []
+        self._old_logger.setLevel(logging.WARN)
+
+    def tearDown(self):
+        for h in self._old_logger.handlers:
+            self._old_logger.removeHandler(h)
+        for h in self._old_handlers:
+            self._old_logger.addHandler(h)
+        self._old_logger.setLevel(self._old_level)
+
+    _schema = None
+
+    def get_schema(self):
+        if self._schema is None:
+            sio = StringIO.StringIO(self._schematext)
+            self.__class__._schema = ZConfig.loadSchemaFile(sio)
+        return self._schema
+
+    def get_config(self, text):
+        conf, handler = ZConfig.loadConfigFile(self.get_schema(),
+                                               StringIO.StringIO(text))
+        self.assert_(not handler)
+        return conf
+
+
+class TestConfig(LoggingTestBase):
+
+    _schematext = """
+      <schema>
+        <import package='ZConfig.components.logger'/>
+        <section type='eventlog' name='*' attribute='eventlog'/>
+      </schema>
+    """
+
+    def test_logging_level(self):
+        # Make sure the expected names are supported; it's not clear
+        # how to check the values in a meaningful way.
+        # Just make sure they're case-insensitive.
+        convert = datatypes.logging_level
+        for name in ["notset", "all", "trace", "debug", "blather",
+                     "info", "warn", "warning", "error", "fatal",
+                     "critical"]:
+            self.assertEqual(convert(name), convert(name.upper()))
+        self.assertRaises(ValueError, convert, "hopefully-not-a-valid-value")
+
+    def test_http_method(self):
+        convert = handlers.get_or_post
+        self.assertEqual(convert("get"), "GET")
+        self.assertEqual(convert("GET"), "GET")
+        self.assertEqual(convert("post"), "POST")
+        self.assertEqual(convert("POST"), "POST")
+        self.assertRaises(ValueError, convert, "")
+        self.assertRaises(ValueError, convert, "foo")
+
+    def test_syslog_facility(self):
+        convert = handlers.syslog_facility
+        for name in ["auth", "authpriv", "cron", "daemon", "kern",
+                     "lpr", "mail", "news", "security", "syslog",
+                     "user", "uucp", "local0", "local1", "local2",
+                     "local3", "local4", "local5", "local6", "local7"]:
+            self.assertEqual(convert(name), name)
+            self.assertEqual(convert(name.upper()), name)
+        self.assertRaises(ValueError, convert, "hopefully-never-a-valid-value")
+
+    def test_config_without_logger(self):
+        conf = self.get_config("")
+        self.assert_(conf.eventlog is None)
+
+    def test_config_without_handlers(self):
+        logger = self.check_simple_logger("<eventlog/>")
+        # Make sure there's a NullHandler, since a warning gets
+        # printed if there are no handlers:
+        self.assertEqual(len(logger.handlers), 1)
+        self.assert_(isinstance(logger.handlers[0],
+                                loghandler.NullHandler))
+
+    def test_with_logfile(self):
+        import os
+        fn = tempfile.mktemp()
+        logger = self.check_simple_logger("<eventlog>\n"
+                                          "  <logfile>\n"
+                                          "    path %s\n"
+                                          "    level debug\n"
+                                          "  </logfile>\n"
+                                          "</eventlog>" % fn)
+        logfile = logger.handlers[0]
+        self.assertEqual(logfile.level, logging.DEBUG)
+        self.assert_(isinstance(logfile, loghandler.FileHandler))
+        logfile.close()
+        os.remove(fn)
+
+    def test_with_stderr(self):
+        self.check_standard_stream("stderr")
+
+    def test_with_stdout(self):
+        self.check_standard_stream("stdout")
+
+    def check_standard_stream(self, name):
+        old_stream = getattr(sys, name)
+        conf = self.get_config("""
+            <eventlog>
+              <logfile>
+                level info
+                path %s
+              </logfile>
+            </eventlog>
+            """ % name.upper())
+        self.assert_(conf.eventlog is not None)
+        # The factory has already been created; make sure it picks up
+        # the stderr we set here when we create the logger and
+        # handlers:
+        sio = StringIO.StringIO()
+        setattr(sys, name, sio)
+        try:
+            logger = conf.eventlog()
+        finally:
+            setattr(sys, name, old_stream)
+        logger.warn("woohoo!")
+        self.assert_(sio.getvalue().find("woohoo!") >= 0)
+
+    def test_with_syslog(self):
+        logger = self.check_simple_logger("<eventlog>\n"
+                                          "  <syslog>\n"
+                                          "    level error\n"
+                                          "    facility local3\n"
+                                          "  </syslog>\n"
+                                          "</eventlog>")
+        syslog = logger.handlers[0]
+        self.assertEqual(syslog.level, logging.ERROR)
+        self.assert_(isinstance(syslog, loghandler.SysLogHandler))
+
+    def test_with_http_logger_localhost(self):
+        logger = self.check_simple_logger("<eventlog>\n"
+                                          "  <http-logger>\n"
+                                          "    level error\n"
+                                          "    method post\n"
+                                          "  </http-logger>\n"
+                                          "</eventlog>")
+        handler = logger.handlers[0]
+        self.assertEqual(handler.host, "localhost")
+        # XXX The "url" attribute of the handler is misnamed; it
+        # really means just the selector portion of the URL.
+        self.assertEqual(handler.url, "/")
+        self.assertEqual(handler.level, logging.ERROR)
+        self.assertEqual(handler.method, "POST")
+        self.assert_(isinstance(handler, loghandler.HTTPHandler))
+
+    def test_with_http_logger_remote_host(self):
+        logger = self.check_simple_logger("<eventlog>\n"
+                                          "  <http-logger>\n"
+                                          "    method get\n"
+                                          "    url http://example.com/log/\n"
+                                          "  </http-logger>\n"
+                                          "</eventlog>")
+        handler = logger.handlers[0]
+        self.assertEqual(handler.host, "example.com")
+        # XXX The "url" attribute of the handler is misnamed; it
+        # really means just the selector portion of the URL.
+        self.assertEqual(handler.url, "/log/")
+        self.assertEqual(handler.level, logging.NOTSET)
+        self.assertEqual(handler.method, "GET")
+        self.assert_(isinstance(handler, loghandler.HTTPHandler))
+
+    def test_with_email_notifier(self):
+        logger = self.check_simple_logger("<eventlog>\n"
+                                          "  <email-notifier>\n"
+                                          "    to sysadmin@example.com\n"
+                                          "    to sa-pager@example.com\n"
+                                          "    from zlog-user@example.com\n"
+                                          "    level fatal\n"
+                                          "  </email-notifier>\n"
+                                          "</eventlog>")
+        handler = logger.handlers[0]
+        self.assertEqual(handler.toaddrs, ["sysadmin@example.com",
+                                           "sa-pager@example.com"])
+        self.assertEqual(handler.fromaddr, "zlog-user@example.com")
+        self.assertEqual(handler.level, logging.FATAL)
+
+    def check_simple_logger(self, text, level=logging.INFO):
+        conf = self.get_config(text)
+        self.assert_(conf.eventlog is not None)
+        self.assertEqual(conf.eventlog.level, level)
+        logger = conf.eventlog()
+        self.assert_(isinstance(logger, logging.Logger))
+        self.assertEqual(len(logger.handlers), 1)
+        return logger
+
+
+def test_suite():
+    return unittest.makeSuite(TestConfig)
+
+if __name__ == '__main__':
+    unittest.main(defaultTest="test_suite")

trunk/src/ZConfig/datatypes.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Selection of standard datatypes for ZConfig."""
+
+import os
+import re
+import sys
+import datetime
+
+# types.StringTypes was added in Python 2.2; basestring in 2.3
+try:
+    unicode
+except NameError:
+    have_unicode = False
+else:
+    have_unicode = True
+
+
+class MemoizedConversion:
+    """Conversion helper that caches the results of expensive conversions."""
+
+    def __init__(self, conversion):
+        self._memo = {}
+        self._conversion = conversion
+
+    def __call__(self, value):
+        try:
+            return self._memo[value]
+        except KeyError:
+            v = self._conversion(value)
+            self._memo[value] = v
+            return v
+
+
+class RangeCheckedConversion:
+    """Conversion helper that range checks another conversion."""
+
+    def __init__(self, conversion, min=None, max=None):
+        self._min = min
+        self._max = max
+        self._conversion = conversion
+
+    def __call__(self, value):
+        v = self._conversion(value)
+        if self._min is not None and v < self._min:
+            raise ValueError("%s is below lower bound (%s)"
+                             % (`v`, `self._min`))
+        if self._max is not None and v > self._max:
+            raise ValueError("%s is above upper bound (%s)"
+                             % (`v`, `self._max`))
+        return v
+
+
+class RegularExpressionConversion:
+    reason = "value did not match regular expression"
+
+    def __init__(self, regex):
+        self._rx = re.compile(regex)
+
+    def __call__(self, value):
+        m = self._rx.match(value)
+        if m and m.group() == value:
+            return value
+        else:
+            raise ValueError("%s: %s" % (self.reason, repr(value)))
+
+
+def check_locale(value):
+    import locale
+    prev = locale.setlocale(locale.LC_ALL)
+    try:
+        try:
+            locale.setlocale(locale.LC_ALL, value)
+        finally:
+            locale.setlocale(locale.LC_ALL, prev)
+    except locale.Error:
+        raise ValueError(
+            'The specified locale "%s" is not supported by your system.\n'
+            'See your operating system documentation for more\n'
+            'information on locale support.' % value)
+    else:
+        return value
+
+
+class BasicKeyConversion(RegularExpressionConversion):
+    def __init__(self):
+        RegularExpressionConversion.__init__(self, "[a-zA-Z][-._a-zA-Z0-9]*")
+
+    def __call__(self, value):
+        value = str(value)
+        return RegularExpressionConversion.__call__(self, value).lower()
+
+
+class ASCIIConversion(RegularExpressionConversion):
+    def __call__(self, value):
+        value = RegularExpressionConversion.__call__(self, value)
+        if have_unicode and isinstance(value, unicode):
+            value = value.encode("ascii")
+        return value
+
+
+_ident_re = "[_a-zA-Z][_a-zA-Z0-9]*"
+
+class IdentifierConversion(ASCIIConversion):
+    reason = "not a valid Python identifier"
+
+    def __init__(self):
+        ASCIIConversion.__init__(self, _ident_re)
+
+
+class DottedNameConversion(ASCIIConversion):
+    reason = "not a valid dotted name"
+
+    def __init__(self):
+        ASCIIConversion.__init__(self,
+                                 r"%s(?:\.%s)*" % (_ident_re, _ident_re))
+
+
+class DottedNameSuffixConversion(ASCIIConversion):
+    reason = "not a valid dotted name or suffix"
+
+    def __init__(self):
+        ASCIIConversion.__init__(self,
+                                 r"(?:%s)(?:\.%s)*|(?:\.%s)+"
+                                 % (_ident_re, _ident_re, _ident_re))
+
+
+def integer(value):
+    try:
+        return int(value)
+    except ValueError:
+        return long(value)
+    except OverflowError:
+        return long(value)
+
+
+def null_conversion(value):
+    return value
+
+
+def asBoolean(s):
+    """Convert a string value to a boolean value."""
+    ss = str(s).lower()
+    if ss in ('yes', 'true', 'on'):
+        return True
+    elif ss in ('no', 'false', 'off'):
+        return False
+    else:
+        raise ValueError("not a valid boolean value: " + repr(s))
+
+
+def string_list(s):
+    """Convert a string to a list of strings using .split()."""
+    return s.split()
+
+
+port_number = RangeCheckedConversion(integer, min=1, max=0xffff).__call__
+
+
+if sys.platform[:3] == "win":
+    DEFAULT_HOST = "localhost"
+else:
+    DEFAULT_HOST = ""
+
+def inet_address(s):
+    # returns (host, port) tuple
+    host = ''
+    port = None
+    if ":" in s:
+        host, s = s.split(":", 1)
+        if s:
+            port = port_number(s)
+        host = host.lower()
+    else:
+        try:
+            port = port_number(s)
+        except ValueError:
+            if len(s.split()) != 1:
+                raise ValueError("not a valid host name: " + repr(s))
+            host = s.lower()
+    if not host:
+        host = DEFAULT_HOST
+    return host, port
+
+
+class SocketAddress:
+    def __init__(self, s):
+        # returns (family, address) tuple
+        import socket
+        if "/" in s or s.find(os.sep) >= 0:
+            self.family = getattr(socket, "AF_UNIX", None)
+            self.address = s
+        else:
+            self.family = socket.AF_INET
+            self.address = inet_address(s)
+
+def float_conversion(v):
+    if isinstance(v, basestring):
+        if v.lower() in ["inf", "-inf", "nan"]:
+            raise ValueError(`v` + " is not a portable float representation")
+    return float(v)
+
+class IpaddrOrHostname(RegularExpressionConversion):
+    def __init__(self):
+        # IP address regex from the Perl Cookbook, Recipe 6.23 (revised ed.)
+        # We allow underscores in hostnames although this is considered
+        # illegal according to RFC1034.
+        expr = (r"(^(\d|[01]?\d\d|2[0-4]\d|25[0-5])\." #ipaddr
+                r"(\d|[01]?\d\d|2[0-4]\d|25[0-5])\." #ipaddr cont'd
+                r"(\d|[01]?\d\d|2[0-4]\d|25[0-5])\." #ipaddr cont'd
+                r"(\d|[01]?\d\d|2[0-4]\d|25[0-5])$)" #ipaddr cont'd
+                r"|([A-Za-z_][-A-Za-z0-9_.]*[-A-Za-z0-9_])") # or hostname
+        RegularExpressionConversion.__init__(self, expr)
+
+    def __call__(self, value):
+        return RegularExpressionConversion.__call__(self, value).lower()
+
+def existing_directory(v):
+    nv = os.path.expanduser(v)
+    if os.path.isdir(nv):
+        return nv
+    raise ValueError, '%s is not an existing directory' % v
+
+def existing_path(v):
+    nv = os.path.expanduser(v)
+    if os.path.exists(nv):
+        return nv
+    raise ValueError, '%s is not an existing path' % v
+
+def existing_file(v):
+    nv = os.path.expanduser(v)
+    if os.path.exists(nv):
+        return nv
+    raise ValueError, '%s is not an existing file' % v
+
+def existing_dirpath(v):
+    nv = os.path.expanduser(v)
+    dir = os.path.dirname(nv)
+    if not dir:
+        # relative pathname with no directory component
+        return nv
+    if os.path.isdir(dir):
+        return nv
+    raise ValueError, ('The directory named as part of the path %s '
+                       'does not exist.' % v)
+
+
+class SuffixMultiplier:
+    # d is a dictionary of suffixes to integer multipliers.  If no suffixes
+    # match, default is the multiplier.  Matches are case insensitive.  Return
+    # values are in the fundamental unit.
+    def __init__(self, d, default=1):
+        self._d = d
+        self._default = default
+        # all keys must be the same size
+        self._keysz = None
+        for k in d.keys():
+            if self._keysz is None:
+                self._keysz = len(k)
+            else:
+                assert self._keysz == len(k)
+
+    def __call__(self, v):
+        v = v.lower()
+        for s, m in self._d.items():
+            if v[-self._keysz:] == s:
+                return int(v[:-self._keysz]) * m
+        return int(v) * self._default
+
+
+def timedelta(s):
+    # Unlike the standard time-interval data type, which returns a float
+    # number of seconds, this datatype takes a wider range of syntax and
+    # returns a datetime.timedelta
+    #
+    # Accepts suffixes:
+    #    w - weeks
+    #    d - days
+    #    h - hours
+    #    m - minutes
+    #    s - seconds
+    #
+    # and all arguments may be integers or floats, positive or negative.
+    # More than one time interval suffix value may appear on the line, but
+    # they should all be separated by spaces, e.g.:
+    #
+    # sleep_time 4w 2d 7h 12m 0.00001s
+    weeks = days = hours = minutes = seconds = 0
+    for part in s.split():
+        val = float(part[:-1])
+        suffix = part[-1]
+        if suffix == 'w':
+            weeks = val
+        elif suffix == 'd':
+            days = val
+        elif suffix == 'h':
+            hours = val
+        elif suffix == 'm':
+            minutes = val
+        elif suffix == 's':
+            seconds = val
+        else:
+            raise TypeError('bad part %s in %s' % (part, s))
+    return datetime.timedelta(weeks=weeks, days=days, hours=hours,
+                              minutes=minutes, seconds=seconds)
+
+
+stock_datatypes = {
+    "boolean":           asBoolean,
+    "dotted-name":       DottedNameConversion(),
+    "dotted-suffix":     DottedNameSuffixConversion(),
+    "identifier":        IdentifierConversion(),
+    "integer":           integer,
+    "float":             float_conversion,
+    "string":            str,
+    "string-list":       string_list,
+    "null":              null_conversion,
+    "locale":            MemoizedConversion(check_locale),
+    "port-number":       port_number,
+    "basic-key":         BasicKeyConversion(),
+    "inet-address":      inet_address,
+    "socket-address":    SocketAddress,
+    "ipaddr-or-hostname":IpaddrOrHostname(),
+    "existing-directory":existing_directory,
+    "existing-path":     existing_path,
+    "existing-file":     existing_file,
+    "existing-dirpath":  existing_dirpath,
+    "byte-size":         SuffixMultiplier({'kb': 1024,
+                                           'mb': 1024*1024,
+                                           'gb': 1024*1024*1024L,
+                                           }),
+    "time-interval":     SuffixMultiplier({'s': 1,
+                                           'm': 60,
+                                           'h': 60*60,
+                                           'd': 60*60*24,
+                                           }),
+    "timedelta":         timedelta,
+    }
+
+
+class Registry:
+    def __init__(self, stock=None):
+        if stock is None:
+            stock = stock_datatypes.copy()
+        self._stock = stock
+        self._other = {}
+        self._basic_key = None
+
+    def get(self, name):
+        if '.' not in name:
+            if self._basic_key is None:
+                self._basic_key = self._other.get("basic-key")
+                if self._basic_key is None:
+                    self._basic_key = self._stock.get("basic-key")
+                if self._basic_key is None:
+                    self._basic_key = stock_datatypes["basic-key"]
+            name = self._basic_key(name)
+        t = self._stock.get(name)
+        if t is None:
+            t = self._other.get(name)
+            if t is None:
+                t = self.search(name)
+        return t
+
+    def register(self, name, conversion):
+        if self._stock.has_key(name):
+            raise ValueError("datatype name conflicts with built-in type: "
+                             + `name`)
+        if self._other.has_key(name):
+            raise ValueError("datatype name already registered: " + `name`)
+        self._other[name] = conversion
+
+    def search(self, name):
+        if not "." in name:
+            raise ValueError("unloadable datatype name: " + `name`)
+        components = name.split('.')
+        start = components[0]
+        g = {}
+        package = __import__(start, g, g)
+        modulenames = [start]
+        for component in components[1:]:
+            modulenames.append(component)
+            try:
+                package = getattr(package, component)
+            except AttributeError:
+                n = '.'.join(modulenames)
+                package = __import__(n, g, g, component)
+        self._other[name] = package
+        return package

trunk/src/ZConfig/doc/Makefile

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+# Rules to convert the documentation to a single PDF file.
+#
+# PostScript, HTML, and plain text output are also supported, though
+# PDF is the default.
+#
+# See the README.txt file for information on the mkhowto program used
+# to generate the formatted versions of the documentation.
+
+.PHONY:	default all html pdf ps text
+
+default:  pdf
+all:	  html pdf ps text
+
+html:   zconfig/zconfig.html
+pdf:	zconfig.pdf
+ps:	zconfig.ps
+text:   zconfig.txt
+
+zconfig/zconfig.html:  zconfig.tex schema.dtd xmlmarkup.perl
+	mkhowto --html $<
+
+zconfig.pdf:  zconfig.tex schema.dtd xmlmarkup.sty
+	mkhowto --pdf $<
+
+zconfig.ps:  zconfig.tex schema.dtd xmlmarkup.sty
+	mkhowto --postscript $<
+
+zconfig.txt: zconfig.tex schema.dtd xmlmarkup.sty
+	mkhowto --text $<
+
+clean:
+	rm -f zconfig.l2h zconfig.l2h~
+
+clobber:  clean
+	rm -f zconfig.pdf zconfig.ps zconfig.txt
+	rm -rf zconfig

trunk/src/ZConfig/doc/README.txt

+The zconfig.tex document in this directory contains the reference
+documentation for the ZConfig package.  This documentation is written
+using the Python LaTeX styles.
+
+To format the documentation, get a copy of the Python documentation
+tools (the Doc/ directory from the Python sources), and create a
+symlink to the tools/mkhowto script from some convenient bin/
+directory.  You will need to have a fairly complete set of
+documentation tools installed on your platform; see
+
+    http://www.python.org/doc/current/doc/doc.html
+
+for more information on the tools.
+
+This documentation requires the latest version of the Python
+documentation tools from CVS.

trunk/src/ZConfig/doc/schema.dtd

+<!--
+  *************************************************************************
+  Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+  All Rights Reserved.
+
+  This software is subject to the provisions of the Zope Public License,
+  Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+  THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+  WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+  WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+  FOR A PARTICULAR PURPOSE.
+  *************************************************************************
+
+  Please note that not all documents that conform to this DTD are
+  legal ZConfig schema.  The ZConfig reference manual describes many
+  constraints that are important to understanding ZConfig schema.
+  -->
+
+<!-- DTD for ZConfig schema documents. -->
+
+<!ELEMENT schema (description?, metadefault?, example?,
+                  import*,
+                  (sectiontype | abstracttype)*,
+                  (section | key | multisection | multikey)*)>
+<!ATTLIST schema
+          extends    NMTOKEN  #IMPLIED
+          prefix     NMTOKEN  #IMPLIED
+          handler    NMTOKEN  #IMPLIED
+          keytype    NMTOKEN  #IMPLIED
+          datatype   NMTOKEN  #IMPLIED>
+
+<!ELEMENT component (description?, (sectiontype | abstracttype)*)>
+<!ATTLIST component
+          prefix     NMTOKEN  #IMPLIED>
+
+<!ELEMENT import EMPTY>
+<!ATTLIST import
+          file       CDATA    #IMPLIED
+          package    NMTOKEN  #IMPLIED
+          src        CDATA    #IMPLIED>
+
+<!ELEMENT description (#PCDATA)*>
+<!ATTLIST description
+          format     NMTOKEN  #IMPLIED>
+
+<!ELEMENT metadefault (#PCDATA)*>
+<!ELEMENT example     (#PCDATA)*>
+
+<!ELEMENT sectiontype (description?, 
+                       (section | key | multisection | multikey)*)>
+<!ATTLIST sectiontype
+          name       NMTOKEN  #REQUIRED
+          prefix     NMTOKEN  #IMPLIED
+          keytype    NMTOKEN  #IMPLIED
+          datatype   NMTOKEN  #IMPLIED
+          implements NMTOKEN  #IMPLIED
+          extends    NMTOKEN  #IMPLIED>
+
+<!ELEMENT abstracttype (description?)>
+<!ATTLIST abstracttype
+          name       NMTOKEN  #REQUIRED
+          prefix     NMTOKEN  #IMPLIED>
+
+<!ELEMENT default    (#PCDATA)*>
+<!ATTLIST default
+          key        CDATA    #IMPLIED>
+
+<!ELEMENT key (description?, metadefault?, example?, default*)>
+<!ATTLIST key
+          name       CDATA    #REQUIRED
+          attribute  NMTOKEN  #IMPLIED
+          datatype   NMTOKEN  #IMPLIED
+          handler    NMTOKEN  #IMPLIED
+          required   (yes|no) "no"
+          default    CDATA    #IMPLIED>
+
+<!ELEMENT multikey (description?, metadefault?, example?, default*)>
+<!ATTLIST multikey
+          name       CDATA    #REQUIRED
+          attribute  NMTOKEN  #IMPLIED
+          datatype   NMTOKEN  #IMPLIED
+          handler    NMTOKEN  #IMPLIED
+          required   (yes|no) "no">
+
+<!ELEMENT section (description?)>
+<!ATTLIST section
+          name       CDATA    #REQUIRED
+          attribute  NMTOKEN  #IMPLIED
+          type       NMTOKEN  #REQUIRED
+          handler    NMTOKEN  #IMPLIED
+          required   (yes|no) "no">
+
+<!ELEMENT multisection (description?)>
+<!ATTLIST multisection
+          name       CDATA    #REQUIRED
+          attribute  NMTOKEN  #IMPLIED
+          type       NMTOKEN  #REQUIRED
+          handler    NMTOKEN  #IMPLIED
+          required   (yes|no) "no">

trunk/src/ZConfig/doc/xmlmarkup.perl

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+# LaTeX2HTML support for the xmlmarkup package.  Doesn't do indexing.
+
+package main;
+
+
+sub do_cmd_element{
+    local($_) = @_;
+    my $name = next_argument();
+    return "<tt class='element'>$name</tt>" . $_;
+}
+
+sub do_cmd_attribute{
+    local($_) = @_;
+    my $name = next_argument();
+    return "<tt class='attribute'>$name</tt>" . $_;
+}
+
+sub do_env_attributedesc{
+    local($_) = @_;
+    my $name = next_argument();
+    my $valuetype = next_argument();
+    return ("\n<dl class='macrodesc'>"
+            . "\n<dt><b><tt class='macro'>$name</tt></b>"
+            . "&nbsp;&nbsp;&nbsp;($valuetype)"
+            . "\n<dd>"
+            . $_
+            . "</dl>");
+}
+
+sub do_env_elementdesc{
+    local($_) = @_;
+    my $name = next_argument();
+    my $contentmodel = next_argument();
+    return ("\n<dl class='elementdesc'>"
+            . "\n<dt class='start-tag'><tt>&lt;"
+            . "<b class='element'>$name</b>&gt;</tt>"
+            . "\n<dd class='content-model'>$contentmodel"
+            . "\n<dt class='endtag'><tt>&lt;/"
+            . "<b class='element'>$name</b>&gt;</tt>"
+            . "\n<dd class='descrition'>"
+            . $_
+            . "</dl>");
+}
+
+1;				# Must end with this, because Perl is bogus.

trunk/src/ZConfig/doc/xmlmarkup.sty

+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%
+% Copyright (c) 2003 Zope Corporation and Contributors.
+% All Rights Reserved.
+%
+% This software is subject to the provisions of the Zope Public License,
+% Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+% THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+% WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+% WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+% FOR A PARTICULAR PURPOSE.
+%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+% Define some simple markup for the LaTeX command documentation:
+
+\ProvidesPackage{xmlmarkup}
+\RequirePackage{python}      % fulllineitems environment
+
+\newcommand{\element}[1]{\code{#1}}
+\newcommand{\attribute}[1]{\code{#1}}
+
+% \begin{elementdesc}{type}{content-model}
+\newenvironment{elementdesc}[2]{
+  \begin{fulllineitems}
+    \item[\code{\textless{\bfseries #1}\textgreater}]
+    \code{#2}
+    \item[\code{\textless/{\bfseries #1}\textgreater}]
+    \index{#1 element@\py@idxcode{#1} element}
+    \index{elements!#1@\py@idxcode{#1}}
+}{\end{fulllineitems}}
+
+% \begin{attributedesc}{name}{content-type}
+\newenvironment{attributedesc}[2]{
+  \begin{fulllineitems}
+    \item[\code{\bfseries#1}{\quad(#2)}]
+    \index{#1@\py@idxcode{#1}}
+}{\end{fulllineitems}}

trunk/src/ZConfig/doc/zconfig.tex

+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%
+% Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+% All Rights Reserved.
+%
+% This software is subject to the provisions of the Zope Public License,
+% Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+% THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+% WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+% WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+% FOR A PARTICULAR PURPOSE.
+%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\documentclass{howto}
+\usepackage{xmlmarkup}
+
+\newcommand{\datatype}[1]{\strong{#1}}
+
+\title{ZConfig Package Reference}
+
+%\date{27 October 2003}
+\release{2.2}
+\setshortversion{2.2}
+
+\author{Zope Corporation}
+\authoraddress{
+    Lafayette Technology Center\\
+    513 Prince Edward Street\\
+    Fredericksburg, VA 22401\\
+    \url{http://www.zope.com/}
+}
+
+\begin{document}
+\maketitle
+
+\begin{abstract}
+\noindent
+This document describes the syntax and API used in configuration files
+for components of a Zope installation written by Zope Corporation.  This
+configuration mechanism is itself configured using a schema specification
+written in XML.
+\end{abstract}
+
+\tableofcontents
+
+
+\section{Introduction \label{intro}}
+
+Zope uses a common syntax and API for configuration files designed for
+software components written by Zope Corporation.  Third-party software
+which is also part of a Zope installation may use a different syntax,
+though any software is welcome to use the syntax used by Zope
+Corporation.  Any software written in Python is free to use the
+\module{ZConfig} software to load such configuration files in order to
+ensure compatibility.  This software is covered by the Zope Public
+License, version 2.0.
+
+The \module{ZConfig} package has been tested with Python 2.3.  Older
+versions of Python are not supported.
+\module{ZConfig} only relies on the Python standard library.
+
+Configurations which use \module{ZConfig} are described using
+\dfn{schema}.  A schema is a specification for the allowed structure
+and content of the configuration.  \module{ZConfig} schema are written
+using a small XML-based language.  The schema language allows the
+schema author to specify the names of the keys allowed at the top
+level and within sections, to define the types of sections which may
+be used (and where), the types of each values, whether a key or
+section must be specified or is optional, default values for keys, and
+whether a value can be given only once or repeatedly.
+
+
+\section{Configuration Syntax \label{syntax}}
+
+Like the \ulink{\module{ConfigParser}}
+{http://docs.python.org/lib/module-ConfigParser.html}
+format, this format supports key-value pairs arranged in sections.
+Unlike the \module{ConfigParser} format, sections are typed and can be
+organized hierarchically.
+Additional files may be included if needed.  Schema components not
+specified in the application schema can be imported from the
+configuration file.  Though both formats are substantially
+line-oriented, this format is more flexible.
+
+The intent of supporting nested section is to allow setting up the
+configurations for loosely-associated components in a container.  For
+example, each process running on a host might get its configuration
+section from that host's section of a shared configuration file.
+
+The top level of a configuration file consists of a series of
+inclusions, key-value pairs, and sections.
+
+Comments can be added on lines by themselves.  A comment has a
+\character{\#} as the first non-space character and extends to the end
+of the line:
+
+\begin{verbatim}
+# This is a comment
+\end{verbatim}
+
+An inclusion is expressed like this:
+
+\begin{verbatim}
+%include defaults.conf
+\end{verbatim}
+
+The resource to be included can be specified by a relative or absolute
+URL, resolved relative to the URL of the resource the
+\keyword{\%include} directive is located in.
+
+
+A key-value pair is expressed like this:
+
+\begin{verbatim}
+key value
+\end{verbatim}
+
+The key may include any non-white characters except for parentheses.
+The value contains all the characters between the key and the end of
+the line, with surrounding whitespace removed.
+
+Since comments must be on lines by themselves, the \character{\#}
+character can be part of a value:
+
+\begin{verbatim}
+key value # still part of the value
+\end{verbatim}
+
+Sections may be either empty or non-empty.  An empty section may be
+used to provide an alias for another section.
+
+A non-empty section starts with a header, contains configuration
+data on subsequent lines, and ends with a terminator.
+
+The header for a non-empty section has this form (square brackets
+denote optional parts):
+
+\begin{alltt}
+<\var{section-type} \optional{\var{name}} >
+\end{alltt}
+
+\var{section-type} and \var{name} all have the same syntactic
+constraints as key names.
+
+The terminator looks like this:
+
+\begin{alltt}
+</\var{section-type}>
+\end{alltt}
+
+The configuration data in a non-empty section consists of a sequence
+of one or more key-value pairs and sections.  For example:
+
+\begin{verbatim}
+<my-section>
+    key-1 value-1
+    key-2 value-2
+
+    <another-section>
+        key-3 value-3
+    </another-section>
+</my-section>
+\end{verbatim}
+
+(The indentation is used here for clarity, but is not required for
+syntactic correctness.)
+
+The header for empty sections is similar to that of non-empty
+sections, but there is no terminator:
+
+\begin{alltt}
+<\var{section-type} \optional{\var{name}} />
+\end{alltt}
+
+
+\subsection{Extending the Configuration Schema}
+
+As we'll see in section~\ref{writing-schema}, ``Writing Configuration
+Schema,'' what can be written in a configuration is controlled by
+schemas which can be built from \emph{components}.  These components
+can also be used to extend the set of implementations of objects the
+application can handle.  What this means when writing a configuration
+is that third-party implementations of application object types can be
+used wherever those application types are used in the configuration,
+if there's a \module{ZConfig} component available for that
+implementation.
+
+The configuration file can use an \keyword{\%import} directive to load
+a named component:
+
+\begin{verbatim}
+%import Products.Ape
+\end{verbatim}
+
+The text to the right of the \keyword{\%import} keyword must be the
+name of a Python package; the \module{ZConfig} component provided by
+that package will be loaded and incorporated into the schema being
+used to load the configuration file.  After the import, section types
+defined in the component may be used in the configuration.
+
+More detail is needed for this to really make sense.
+
+A schema may define section types which are \emph{abstract}; these
+cannot be used directly in a configuration, but multiple concrete
+section types can be defined which \emph{implement} the abstract
+types.  Wherever the application allows an abstract type to be used,
+any concrete type which implements that abstract type can be used in
+an actual configuration.
+
+The \keyword{\%import} directive allows loading schema components
+which provide alternate concrete section types which implement the
+abstract types defined by the application.  This allows third-party
+implementations of abstract types to be used in place of or in
+addition to implementations provided with the application.
+
+Consider an example application application which supports logging in
+the same way Zope 2 does.  There are some parameters which configure
+the general behavior of the logging mechanism, and an arbitrary number
+of \emph{log handlers} may be specified to control how the log
+messages are handled.  Several log handlers are provided by the
+application.  Here is an example logging configuration:
+
+\begin{verbatim}
+<eventlog>
+  level verbose
+
+  <logfile>
+    path /var/log/myapp/events.log
+  </logfile>
+</eventlog>
+\end{verbatim}
+
+A third-party component may provide a log handler to send
+high-priority alerts the system administrator's text pager or
+SMS-capable phone.  All that's needed is to install the implementation
+so it can be imported by Python, and modify the configuration:
+
+\begin{verbatim}
+%import my.pager.loghandler
+
+<eventlog>
+  level verbose
+
+  <logfile>
+    path /var/log/myapp/events.log
+  </logfile>
+
+  <pager>
+    number   1-800-555-1234
+    message  Something broke!
+  </pager>
+</eventlog>
+\end{verbatim}
+
+
+\subsection{Textual Substitution in Values}
+
+\module{ZConfig} provides a limited way to re-use portions of a value
+using simple string substitution.  To use this facility, define named
+bits of replacement text using the \keyword{\%define} directive, and
+reference these texts from values.
+
+The syntax for \keyword{\%define} is:
+
+\begin{alltt}
+%define \var{name} \optional{\var{value}}
+\end{alltt}
+
+The value of \var{name} must be a sequence of letters, digits, and
+underscores, and may not start with a digit; the namespace for these
+names is separate from the other namespaces used with
+\module{ZConfig}, and is case-insensitive.  If \var{value} is
+omitted, it will be the empty string.  If given, there must be
+whitespace between \var{name} and \var{value}; \var{value} will not
+include any whitespace on either side, just like values from key-value
+pairs.
+
+Names must be defined before they are used, and may not be
+re-defined.  All resources being parsed as part of a configuration
+share a single namespace for defined names.  This means that resources
+which may be included more than once should not define any names.
+
+References to defined names from configuration values use the syntax
+described for the \refmodule{ZConfig.substitution} module.
+Configuration values which include a \character{\$} as part of the
+actual value will need to use \code{\$\$} to get a single
+\character{\$} in the result.
+
+The values of defined names are processed in the same way as
+configuration values, and may contain references to named
+definitions.
+
+For example, the value for \code{key} will evaluate to \code{value}:
+
+\begin{verbatim}
+%define name value
+key $name
+\end{verbatim} %$ <-- bow to font-lock
+
+
+\section{Writing Configuration Schema \label{writing-schema}}
+
+\module{ZConfig} schema are written as XML documents.
+
+Data types are searched in a special namespace defined by the data
+type registry.  The default registry has slightly magical semantics:
+If the value can be matched to a standard data type when interpreted
+as a \datatype{basic-key}, the standard data type will be used.  If
+that fails, the value must be a \datatype{dotted-name} containing at
+least one dot, and a conversion function will be sought using the
+\method{search()} method of the data type registry used to load the
+schema.
+
+
+\subsection{Schema Elements \label{elements}}
+
+For each element, the content model is shown, followed by a
+description of how the element is used, and then a list of the
+available attributes.  For each attribute, the type of the value is
+given as either the name of a \module{ZConfig} datatype or an XML
+attribute value type.  Familiarity with XML's Document Type Definition
+language is helpful.
+
+The following elements are used to describe a schema:
+
+\begin{elementdesc}{schema}{description?, metadefault?, example?,
+                            import*,
+                            (sectiontype | abstracttype)*,
+                            (section | key | multisection |
+                            multikey)*}
+  Document element for a \module{ZConfig} schema.
+
+  \begin{attributedesc}{extends}{\datatype{space-separated-url-references}}
+  A list of URLs of base schemas from which this section type will inherit key,
+  section, and section type declarations.  If omitted, this schema
+  is defined using only the keys, sections, and section types contained within
+  the \element{schema} element.
+  \end{attributedesc}
+
+  \begin{attributedesc}{datatype}{\datatype{basic-key}
+                                  or \datatype{dotted-name}}
+    The data type converter which will be applied to the value of this
+    section.  If the value is a \datatype{dotted-name} that begins
+    with a period, the value of \attribute{prefix} will be pre-pended,
+    if set.  If any base schemas are listed in the \attribute{extends}
+    attribute, the default value for this attribute comes from the base
+    schemas.  If the base schemas all use the same \attribute{datatype}, then
+    that data type will be the default value for the extending schema.  If
+    there are no base schemas, the default value is \datatype{null}, which
+    means that the \module{ZConfig} section object will be used unconverted.
+    If the base schemas have different \attribute{datatype} definitions, you
+    must explicitly define the \attribute{datatype} in the extending schema.
+  \end{attributedesc}
+
+  \begin{attributedesc}{handler}{\datatype{basic-key}}
+  \end{attributedesc}
+
+  \begin{attributedesc}{keytype}{\datatype{basic-key}
+                                  or \datatype{dotted-name}}
+    The data type converter which will be applied to keys found in
+    this section.  This can be used to constrain key values in
+    different ways; two data types which may be especially useful are
+    the \datatype{identifier} and \datatype{ipaddr-or-hostname}
+    types.  If the value is a \datatype{dotted-name} that begins
+    with a period, the value of \attribute{prefix} will be pre-pended,
+    if set.  If any base schemas are listed in the \attribute{extends}
+    attribute, the default value for this attribute comes from the base
+    schemas.  If the base schemas all use the same \attribute{keytype}, then
+    that key type will be the default value for the extending schema.  If there
+    are no base schemas, the default value is \datatype{basic-key}.  If the
+    base schemas have different \attribute{keytype} definitions, you must
+    explicitly define the \attribute{keytype} in the extending schema.
+  \end{attributedesc}
+
+  \begin{attributedesc}{prefix}{\datatype{dotted-name}}
+    Prefix to be pre-pended in front of partial dotted-names that
+    start with a period.  The value of this attribute is used in all
+    contexts with the \element{schema} element if it hasn't been
+    overridden by an inner element with a \attribute{prefix}
+    attribute.
+  \end{attributedesc}
+\end{elementdesc}
+
+\begin{elementdesc}{description}{PCDATA}
+  Descriptive text explaining the purpose the container of the
+  \element{description} element.  Most other elements can contain
+  a \element{description} element as their first child.
+  At most one \element{description} element may appear in a given
+  context.
+
+  \begin{attributedesc}{format}{NMTOKEN}
+    Optional attribute that can be added to indicate what conventions
+    are used to mark up the contained text.  This is intended to serve
+    as a hint for documentation extraction tools.  Suggested values
+    are:
+
+    \begin{tableii}{l|l}{code}{Value}{Content Format}
+      \lineii{plain}{\mimetype{text/plain}; blank lines separate paragraphs}
+      \lineii{rest}{reStructuredText}
+      \lineii{stx}{Classic Structured Text}
+    \end{tableii}
+  \end{attributedesc}
+\end{elementdesc}
+
+\begin{elementdesc}{example}{PCDATA}
+  An example value.  This serves only as documentation.
+\end{elementdesc}
+
+\begin{elementdesc}{metadefault}{PCDATA}
+  A description of the default value, for human readers.  This may
+  include information about how a computed value is determined when
+  the schema does not specify a default value.
+\end{elementdesc}
+
+\begin{elementdesc}{abstracttype}{description?}
+  Define an abstract section type.
+
+  \begin{attributedesc}{name}{\datatype{basic-key}}
+    The name of the abstract section type; required.
+  \end{attributedesc}
+\end{elementdesc}
+
+\begin{elementdesc}{sectiontype}{description?, (section | key |
+                                  multisection | multikey)*}
+  Define a concrete section type.
+
+  \begin{attributedesc}{datatype}{\datatype{basic-key}
+                                  or \datatype{dotted-name}}
+    The data type converter which will be applied to the value of this
+    section.  If the value is a \datatype{dotted-name} that begins
+    with a period, the value of \attribute{prefix} will be pre-pended,
+    if set.  If \attribute{datatype} is omitted and
+    \attribute{extends} is used, the \attribute{datatype} from the
+    section type identified by the \attribute{extends} attribute is
+    used.
+  \end{attributedesc}
+
+  \begin{attributedesc}{extends}{\datatype{basic-key}}
+    The name of a concrete section type from which this section type
+    acquires all key and section declarations.  This type does
+    \emph{not} automatically implement any abstract section type
+    implemented by the named section type.  If omitted, this section
+    is defined with only the keys and sections contained within the
+    \element{sectiontype} element.  The new section type is called a
+    \emph{derived} section type, and the type named by this attribute
+    is called the \emph{base} type.  Values for the
+    \attribute{datatype} and \attribute{keytype} attributes are
+    acquired from the base type if not specified.
+  \end{attributedesc}
+
+  \begin{attributedesc}{implements}{\datatype{basic-key}}
+    The name of an abstract section type which this concrete section
+    type implements.  If omitted, this section type does not implement
+    any abstract type, and can only be used if it is specified
+    directly in a schema or other section type.
+  \end{attributedesc}
+
+  \begin{attributedesc}{keytype}{\datatype{basic-key}}
+    The data type converter which will be applied to keys found in
+    this section.  This can be used to constrain key values in
+    different ways; two data types which may be especially useful are
+    the \datatype{identifier} and \datatype{ipaddr-or-hostname}
+    types.  If the value is a \datatype{dotted-name} that begins
+    with a period, the value of \attribute{prefix} will be pre-pended,
+    if set.  The default value is \datatype{basic-key}.  If
+    \attribute{keytype} is omitted and \attribute{extends} is used,
+    the \attribute{keytype} from the section type identified by the
+    \attribute{extends} attribute is used.
+  \end{attributedesc}
+
+  \begin{attributedesc}{name}{\datatype{basic-key}}
+    The name of the section type; required.
+  \end{attributedesc}
+
+  \begin{attributedesc}{prefix}{\datatype{dotted-name}}
+    Prefix to be pre-pended in front of partial dotted-names that
+    start with a period.  The value of this attribute is used in all
+    contexts in the \element{sectiontype} element.  If omitted, the
+    prefix specified by a containing context is used if specified.
+  \end{attributedesc}
+\end{elementdesc}
+
+\begin{elementdesc}{import}{EMPTY}
+  Import a schema component.  Exactly one of the attributes
+  \attribute{package} and \attribute{src} must be specified.
+
+  \begin{attributedesc}{file}{file name without directory information}
+    Name of the component file within a package; if not specified,
+    \file{component.xml} is used.  This may only be given when
+    \attribute{package} is used.  (The \file{component.xml} file is
+    always used when importing via \keyword{\%import} from a
+    configuration file.)
+  \end{attributedesc}
+
+  \begin{attributedesc}{package}{\datatype{dotted-suffix}}
+    Name of a Python package that contains the schema component being
+    imported.  The component will be loaded from the file identified
+    by the \attribute{file} attribute, or \file{component.xml} if
+    \attribute{file} is not specified.  If the package name given
+    starts with a dot (\character{.}), the name used will be the
+    current prefix and the value of this attribute concatenated.
+  \end{attributedesc}
+
+  \begin{attributedesc}{src}{\datatype{url-reference}}
+    URL to a separate schema which can provide useful types.  The
+    referenced resource must contain a schema, not a schema
+    component.  Section types defined or imported by the referenced
+    schema are added to the schema containing the \element{import};
+    top-level keys and sections are ignored.
+  \end{attributedesc}
+\end{elementdesc}
+
+\begin{elementdesc}{key}{description?, example?, metadefault?, default*}
+  A \element{key} element is used to describe a key-value pair which
+  may occur at most once in the section type or top-level schema in
+  which it is listed.
+
+  \begin{attributedesc}{attribute}{\datatype{identifier}}
+    The name of the Python attribute which this key should be the
+    value of on a \class{SectionValue} instance.  This must be unique
+    within the immediate contents of a section type or schema.  If
+    this attribute is not specified, an attribute name will be
+    computed by converting hyphens in the key name to underscores.
+  \end{attributedesc}
+
+  \begin{attributedesc}{datatype}{\datatype{basic-key}
+                                  or \datatype{dotted-name}}
+    The data type converter which will be applied to the value of this
+    key.  If the value is a \datatype{dotted-name} that begins
+    with a period, the value of \attribute{prefix} will be pre-pended,
+    if set.
+  \end{attributedesc}
+
+  \begin{attributedesc}{default}{\datatype{string}}
+    If the key-value pair is optional and this attribute is specified,
+    the value of this attribute will be converted using the appropriate
+    data type converter and returned to the application as the
+    configured value.  This attribute may not be specified if the
+    \attribute{required} attribute is \code{yes}.
+  \end{attributedesc}
+
+  \begin{attributedesc}{handler}{\datatype{basic-key}}
+  \end{attributedesc}
+
+  \begin{attributedesc}{name}{\datatype{basic-key}}
+    The name of the key, as it must be given in a configuration
+    instance, or `\code{*}'.  If the value is `\code{*}', any name not
+    already specified as a key may be used, and the configuration
+    value for the key will be a dictionary mapping from the key name
+    to the value.  In this case, the \attribute{attribute} attribute
+    must be specified, and the data type for the key will be applied
+    to each key which is found.
+  \end{attributedesc}
+
+  \begin{attributedesc}{required}{\code{yes|no}}
+    Specifies whether the configuration instance is required to
+    provide the key.  If the value is \code{yes}, the
+    \attribute{default} attribute may not be specified and an error
+    will be reported if the configuration instance does not specify a
+    value for the key.  If the value is \code{no} (the default) and
+    the configuration instance does not specify a value, the value
+    reported to the application will be that specified by the
+    \attribute{default} attribute, if given, or \code{None}.
+  \end{attributedesc}
+\end{elementdesc}
+
+
+\begin{elementdesc}{multikey}{description?, example?, metadefault?, default*}
+  A \element{multikey} element is used to describe a key-value pair
+  which may occur any number of times in the section type or top-level
+  schema in which it is listed.
+
+  \begin{attributedesc}{attribute}{\datatype{identifier}}
+    The name of the Python attribute which this key should be the
+    value of on a \class{SectionValue} instance.  This must be unique
+    within the immediate contents of a section type or schema.  If
+    this attribute is not specified, an attribute name will be
+    computed by converting hyphens in the key name to underscores.
+  \end{attributedesc}
+
+  \begin{attributedesc}{datatype}{\datatype{basic-key}
+                                  or \datatype{dotted-name}}
+    The data type converter which will be applied to the value of this
+    key.  If the value is a \datatype{dotted-name} that begins
+    with a period, the value of \attribute{prefix} will be pre-pended,
+    if set.
+  \end{attributedesc}
+
+  \begin{attributedesc}{handler}{\datatype{basic-key}}
+  \end{attributedesc}
+
+  \begin{attributedesc}{name}{\datatype{basic-key}}
+    The name of the key, as it must be given in a configuration
+    instance, or `\code{+}'.  If the value is `\code{+}', any name not
+    already specified as a key may be used, and the configuration
+    value for the key will be a dictionary mapping from the key name
+    to the value.  In this case, the \attribute{attribute} attribute
+    must be specified, and the data type for the key will be applied
+    to each key which is found.
+  \end{attributedesc}
+
+  \begin{attributedesc}{required}{\code{yes|no}}
+    Specifies whether the configuration instance is required to
+    provide the key.  If the value is \code{yes}, no \element{default}
+    elements may be specified and an error will be reported if the
+    configuration instance does not specify at least one value for the
+    key.  If the value is \code{no} (the default) and the
+    configuration instance does not specify a value, the value
+    reported to the application will be a list containing one element
+    for each \element{default} element specified as a child of the
+    \element{multikey}.  Each value will be individually converted
+    according to the \attribute{datatype} attribute.
+  \end{attributedesc}
+\end{elementdesc}
+
+
+\begin{elementdesc}{default}{PCDATA}
+  Each \element{default} element specifies a single default value for
+  a \element{multikey}.  This element can be repeated to produce a
+  list of individual default values.  The text contained in the
+  element will be passed to the datatype conversion for the
+  \element{multikey}.
+
+  \begin{attributedesc}{key}{key type of the containing sectiontype}
+    Key to associate with the default value.  This is only used for
+    defaults of a \element{key} or \element{multikey} with a
+    \attribute{name} of \code{+}; in that case this attribute is
+    required.  It is an error to use the \attribute{key} attribute
+    with a \element{default} element for a \element{multikey} with a
+    name other than \code{+}.
+
+    \begin{notice}[warning]
+      The datatype of this attribute is that of the section type
+      \emph{containing} the actual keys, not necessarily that of the
+      section type which defines the key.  If a derived section
+      overrides the key type of the base section type, the actual
+      key type used is that of the derived section.
+
+      This can lead to confusing errors in schemas, though the
+      \refmodule{ZConfig} package checks for this when the schema is
+      loaded.  This situation is particularly likely when a derived
+      section type uses a key type which collapses multiple default
+      keys which were not collapsed by the base section type.
+
+      Consider this example schema:
+
+\begin{verbatim}
+<schema>
+  <sectiontype name="base" keytype="identifier">
+    <key name="+" attribute="mapping">
+      <default key="foo">some value</default>
+      <default key="FOO">some value</default>
+    </key>
+  </sectiontype>
+
+  <sectiontype name="derived" keytype="basic-key"
+               extends="base"/>
+
+  <section type="derived" name="*" attribute="section"/>
+</schema>
+\end{verbatim}
+
+      When this schema is loaded, a set of defaults for the
+      \datatype{derived} section type is computed.  Since
+      \datatype{basic-key} is case-insensitive (everything is
+      converted to lower case), \samp{foo} and \samp{Foo} are both
+      converted to \samp{foo}, which clashes since \element{key} only
+      allows one value for each key.
+    \end{notice}
+  \end{attributedesc}
+\end{elementdesc}
+
+
+\begin{elementdesc}{section}{description?}
+  A \element{section} element is used to describe a section which may
+  occur at most once in the section type or top-level schema in which
+  it is listed.
+
+  \begin{attributedesc}{attribute}{\datatype{identifier}}
+    The name of the Python attribute which this section should be the
+    value of on a \class{SectionValue} instance.  This must be unique
+    within the immediate contents of a section type or schema.  If
+    this attribute is not specified, an attribute name will be
+    computed by converting hyphens in the section name to underscores,
+    in which case the \attribute{name} attribute may not be \code{*}
+    or \code{+}.
+  \end{attributedesc}
+
+  \begin{attributedesc}{handler}{\datatype{basic-key}}
+  \end{attributedesc}
+
+  \begin{attributedesc}{name}{\datatype{basic-key}}
+    The name of the section, as it must be given in a configuration
+    instance, \code{*}, or \code{+}.  If the value is \code{*}, any
+    name not already specified as a key may be used.  If the value is
+    \code{*} or \code{+}, the \attribute{attribute} attribute must be
+    specified.  If the value is \code{*}, any name is allowed, or the
+    name may be omitted.  If the value is \code{+}, any name is
+    allowed, but some name must be provided.
+  \end{attributedesc}
+
+  \begin{attributedesc}{required}{\code{yes|no}}
+    Specifies whether the configuration instance is required to
+    provide the section.  If the value is \code{yes}, an error will be
+    reported if the configuration instance does not include the
+    section.  If the value is \code{no} (the default) and the
+    configuration instance does not include the section, the value
+    reported to the application will be \code{None}.
+  \end{attributedesc}
+
+  \begin{attributedesc}{type}{\datatype{basic-key}}
+    The section type which matching sections must implement.  If the
+    value names an abstract section type, matching sections in the
+    configuration file must be of a type which specifies that it
+    implements the named abstract type.  If the name identifies a
+    concrete type, the section type must match exactly.
+  \end{attributedesc}
+\end{elementdesc}
+
+
+\begin{elementdesc}{multisection}{description?}
+  A \element{multisection} element is used to describe a section which
+  may occur any number of times in the section type or top-level
+  schema in which it is listed.
+
+  \begin{attributedesc}{attribute}{\datatype{identifier}}
+    The name of the Python attribute which matching sections should be
+    the value of on a \class{SectionValue} instance.  This is required
+    and must be unique within the immediate contents of a section type
+    or schema.  The \class{SectionValue} instance will contain a list
+    of matching sections.
+  \end{attributedesc}
+
+  \begin{attributedesc}{handler}{\datatype{basic-key}}
+  \end{attributedesc}
+
+  \begin{attributedesc}{name}{\datatype{basic-key}}
+    For a \element{multisection}, any name not already specified as a
+    key may be used.  If the value is \code{*} or \code{+}, the
+    \attribute{attribute} attribute must be specified.  If the value
+    is \code{*}, any name is allowed, or the name may be omitted.  If
+    the value is \code{+}, any name is allowed, but some name must be
+    provided.  No other value for the \attribute{name} attribute is
+    allowed for a \element{multisection}.
+  \end{attributedesc}
+
+  \begin{attributedesc}{required}{\code{yes|no}}
+    Specifies whether the configuration instance is required to
+    provide at least one matching section.  If the value is
+    \code{yes}, an error will be reported if the configuration
+    instance does not include the section.  If the value is \code{no}
+    (the default) and the configuration instance does not include the
+    section, the value reported to the application will be
+    \code{None}.
+  \end{attributedesc}
+
+  \begin{attributedesc}{type}{\datatype{basic-key}}
+    The section type which matching sections must implement.  If the
+    value names an abstract section type, matching sections in the
+    configuration file must be of types which specify that they
+    implement the named abstract type.  If the name identifies a
+    concrete type, the section type must match exactly.
+  \end{attributedesc}
+\end{elementdesc}
+
+
+\subsection{Schema Components \label{schema-components}}
+
+XXX need more explanation
+
+\module{ZConfig} supports schema components that can be
+provided by disparate components, and allows them to be knit together
+into concrete schema for applications.  Components cannot add
+additional keys or sections in the application schema.
+
+A schema \dfn{component} is allowed to define new abstract and
+section types.
+Components are identified using a dotted-name, similar to a Python
+module name.  For example, one component may be \code{zodb.storage}.
+
+Schema components are stored alongside application code since they
+directly reference datatype code.  Schema components are provided by
+Python packages.  The component definition is normally stored in the
+file \file{component.xml}; an alternate filename may be specified
+using the \attribute{file} attribute of the \element{import} element.
+Components imported using the \keyword{\%import} keyword from a
+configuration file must be named \file{component.xml}.
+The component defines the types provided by that component; it must
+have a \element{component} element as the document element.
+
+The following element is used as the document element for schema
+components.  Note that schema components do not allow keys and
+sections to be added to the top-level of a schema; they serve only to
+provide type definitions.
+
+\begin{elementdesc}{component}{description?, (abstracttype | sectiontype)*}
+  The top-level element for schema components.
+
+  \begin{attributedesc}{prefix}{\datatype{dotted-name}}
+    Prefix to be pre-pended in front of partial dotted-names that
+    start with a period.  The value of this attribute is used in all
+    contexts within the \element{component} element if it hasn't been
+    overridden by an inner element with a \attribute{prefix}
+    attribute.
+  \end{attributedesc}
+\end{elementdesc}
+
+
+\section{Standard \module{ZConfig} Datatypes\label{standard-datatypes}}
+
+There are a number of data types which can be identified using the
+\attribute{datatype} attribute on \element{key},
+\element{sectiontype}, and \element{schema} elements.
+Applications may extend the set of datatypes by calling the
+\method{register()} method of the data type registry being used or by
+using Python dotted-names to refer to conversion routines defined in
+code.
+
+The following data types are provided by the default type registry.
+
+\begin{definitions}
+\term{\datatype{basic-key}}
+  The default data type for a key in a ZConfig configuration file.
+  The result of conversion is always lower-case, and matches the
+  regular expression \regexp{[a-z][-._a-z0-9]*}.
+
+\term{\datatype{boolean}}
+  Convert a human-friendly string to a boolean value.  The names
+  \code{yes}, \code{on}, and \code{true} convert to \constant{True},
+  while \code{no}, \code{off}, and \code{false} convert to
+  \constant{False}.  Comparisons are case-insensitive.  All other
+  input strings are disallowed.
+
+\term{\datatype{byte-size}}
+  A specification of a size, with byte multiplier suffixes (for
+  example, \samp{128MB}).  Suffixes are case insensitive and may be
+  \samp{KB}, \samp{MB}, or \samp{GB}
+
+\term{\datatype{dotted-name}}
+  A string consisting of one or more \datatype{identifier} values
+  separated by periods (\character{.}).
+
+\term{\datatype{dotted-suffix}}
+  A string consisting of one or more \datatype{identifier} values
+  separated by periods (\character{.}), possibly prefixed by a
+  period.  This can be used to indicate a dotted name that may be
+  specified relative to some base dotted name.
+
+\term{\datatype{existing-dirpath}}
+  Validates that the directory portion of a pathname exists.  For
+  example, if the value provided is \file{/foo/bar}, \file{/foo} must
+  be an existing directory.  No conversion is performed.
+
+\term{\datatype{existing-directory}}
+  Validates that a directory by the given name exists on 
+  the local filesystem.  No conversion is performed. 
+
+\term{\datatype{existing-file}}
+  Validates that a file by the given name exists.  No conversion 
+  is performed. 
+
+\term{\datatype{existing-path}}
+  Validates that a path (file, directory, or symlink) by the
+  given name exists on the local filesystem.  No conversion
+  is performed.
+
+\term{\datatype{float}}
+  A Python float.  \code{Inf}, \code{-Inf}, and \code{NaN} are not
+  allowed.
+
+\term{\datatype{identifier}}
+  Any valid Python identifier.
+
+\term{\datatype{inet-address}}
+  An Internet address expressed as a \code{(\var{hostname},
+  \var{port})} pair.  If only the port is specified, the default host
+  will be returned for \var{hostname}.  The default host is
+  \code{localhost} on Windows and the empty string on all other
+  platforms.  If the port is omitted, \code{None} will be returned for
+  \var{port}.
+
+\term{\datatype{integer}}
+  Convert a value to an integer.  This will be a Python \class{int} if
+  the value is in the range allowed by \class{int}, otherwise a Python
+  \class{long} is returned.
+
+\term{\datatype{ipaddr-or-hostname}}
+  Validates a valid IP address or hostname.  If the first 
+  character is a digit, the value is assumed to be an IP 
+  address.  If the first character is not a digit, the value 
+  is assumed to be a hostname.  Hostnames are converted to lower
+  case.
+
+\term{\datatype{locale}}
+  Any valid locale specifier accepted by the available
+  \function{locale.setlocale()} function.  Be aware that only the
+  \code{'C'} locale is supported on some platforms.
+
+\term{\datatype{null}}
+  No conversion is performed; the value passed in is the value
+  returned.  This is the default data type for section values.
+
+\term{\datatype{port-number}}
+  Returns a valid port number as an integer.  Validity does not imply
+  that any particular use may be made of the port, however.  For
+  example, port number lower than 1024 generally cannot be bound by
+  non-root users.
+
+\term{\datatype{socket-address}}
+  An address for a socket.  The converted value is an object providing
+  two attributes.  \member{family} specifies the address family
+  (\constant{AF_INET} or \constant{AF_UNIX}), with \code{None} instead
+  of \constant{AF_UNIX} on platforms that don't support it.  The
+  \member{address} attribute will be the address that should be passed
+  to the socket's \method{bind()} method.  If the family is
+  \constant{AF_UNIX}, the specific address will be a pathname; if the
+  family is \constant{AF_INET}, the second part will be the result of
+  the \datatype{inet-address} conversion.
+
+\term{\datatype{string}}
+  Returns the input value as a string.  If the source is a Unicode
+  string, this implies that it will be checked to be simple 7-bit
+  \ASCII.  This is the default data type for values in
+  configuration files.
+
+\term{\datatype{time-interval}}
+  A specification of a time interval in seconds, with multiplier
+  suffixes (for example, \code{12h}).  Suffixes are case insensitive
+  and may be \samp{s} (seconds), \samp{m} (minutes), \samp{h} (hours),
+  or \samp{d} (days).
+
+\term{\datatype{timedelta}}
+  Similar to the \datatype{time-interval}, this data type returns a Python
+  datetime.timedelta object instead of a float.  The set of suffixes
+  recognized by \datatype{timedelta} are: \samp{w} (weeks), \samp{d} (days),
+  \samp{h} (hours), \samp{m} (minutes), \samp{s} (seconds).  Values may be
+  floats, for example: \code{4w 2.5d 7h 12m 0.001s}.
+
+\end{definitions}
+
+
+\section{Standard \module{ZConfig} Schema Components
+         \label{standard-components}}
+
+\module{ZConfig} provides a few convenient schema components as part
+of the package.  These may be used directly or can server as examples
+for creating new components.
+
+
+\subsection{\module{ZConfig.components.basic}}
+
+The \module{ZConfig.components.basic} package provides small
+components that can be helpful in composing application-specific
+components and schema.  There is no large functionality represented by
+this package.  The default component provided by this package simply
+imports all of the smaller components.  This can be imported using
+
+\begin{verbatim}
+<import package="ZConfig.components.basic"/>
+\end{verbatim}
+
+Each of the smaller components is documented directly; importing these
+selectively can reduce the time it takes to load a schema slightly,
+and allows replacing the other basic components with alternate
+components (by using different imports that define the same type
+names) if desired.
+
+
+\subsubsection{The Mapping Section Type \label{basic-mapping}}
+
+There is a basic section type that behaves like a simple Python
+mapping; this can be imported directly using
+
+\begin{verbatim}
+<import package="ZConfig.components.basic" file="mapping.xml"/>
+\end{verbatim}
+
+This defines a single section type, \datatype{ZConfig.basic.mapping}.
+When this is used, the section value is a Python dictionary mapping
+keys to string values.
+
+This type is intended to be used by extending it in simple ways.  The
+simplest is to create a new section type name that makes more sense
+for the application:
+
+\begin{verbatim}
+<import package="ZConfig.components.basic" file="mapping.xml"/>
+
+<sectiontype name="my-mapping"
+             extends="ZConfig.basic.mapping"
+             />
+
+<section name="*"
+         type="my-mapping"
+         attribute="map"
+         />
+\end{verbatim}
+
+This allows a configuration to contain a mapping from
+\datatype{basic-key} names to string values like this:
+
+\begin{verbatim}
+<my-mapping>
+  This that
+  and the other
+</my-mapping>
+\end{verbatim}
+
+The value of the configuration object's \member{map} attribute would
+then be the dictionary
+
+\begin{verbatim}
+{'this': 'that',
+ 'and': 'the other',
+ }
+\end{verbatim}
+
+(Recall that the \datatype{basic-key} data type converts everything to
+lower case.)
+
+Perhaps a more interesting application of
+\datatype{ZConfig.basic.mapping} is using the derived type to override
+the \attribute{keytype}.  If we have the conversion function:
+
+\begin{verbatim}
+def email_address(value):
+    userid, hostname = value.split("@", 1)
+    hostname = hostname.lower()  # normalize what we know we can
+    return "%s@%s" % (userid, hostname)
+\end{verbatim}
+
+then we can use this as the key type for a derived mapping type:
+
+\begin{verbatim}
+<import package="ZConfig.components.basic" file="mapping.xml"/>
+
+<sectiontype name="email-users"
+             extends="ZConfig.basic.mapping"
+             keytype="mypkg.datatypes.email_address"
+             />
+
+<section name="*"
+         type="email-users"
+         attribute="email_users"
+         />
+\end{verbatim}
+
+
+\subsection{\module{ZConfig.components.logger}}
+
+The \module{ZConfig.components.logger} package provides configuration
+support for the \ulink{\module{logging} package}
+{http://docs.python.org/lib/module-logging.html} in
+Python's standard library.  This component can be imported using
+
+\begin{verbatim}
+<import package="ZConfig.components.logger"/>
+\end{verbatim}
+
+This component defines two abstract types and several concrete section
+types.  These can be imported as a unit, as above, or as four smaller
+components usable in creating alternate logging packages.
+
+The first of the four smaller components contains the abstract types,
+and can be imported using
+
+\begin{verbatim}
+<import package="ZConfig.components.logger" file="abstract.xml"/>
+\end{verbatim}
+
+The two abstract types imported by this are:
+
+\begin{definitions}
+
+\term{\datatype{ZConfig.logger.log}}
+  Logger objects are represented by this abstract type.
+
+\term{\datatype{ZConfig.logger.handler}}
+  Each logger object can have one or more ``handlers'' associated with
+  them.  These handlers are responsible for writing logging events to
+  some form of output stream using appropriate formatting.  The output
+  stream may be a file on a disk, a socket communicating with a server
+  on another system, or a series of \code{syslog} messages.  Section
+  types which implement this type represent these handlers.
+
+\end{definitions}
+
+
+The second and third of the smaller components provides section types
+that act as factories for \class{logging.Logger} objects.  These can be
+imported using
+
+\begin{verbatim}
+<import package="ZConfig.components.logger" file="eventlog.xml"/>
+<import package="ZConfig.components.logger" file="logger.xml"/>
+\end{verbatim}
+
+The types defined in these components implement the
+\datatype{ZConfig.logger.log} abstract type.  The \file{eventlog.xml}
+component defines an \datatype{eventlog} type which represents the
+root logger from the the \module{logging} package (the return value of
+\function{logging.getLogger()}), while the \file{logger.xml} component
+defines a \datatype{logger} section type which represents a named
+logger (as returned by \function{logging.getLogger(\var{name})}).
+
+
+The third of the smaller components provides section types that are
+factories for \class{logging.Handler} objects.  This can be imported
+using
+
+\begin{verbatim}
+<import package="ZConfig.components.logger" file="handlers.xml"/>
+\end{verbatim}
+
+The types defined in this component implement the
+\datatype{ZConfig.logger.handler} abstract type.
+
+
+
+The configuration objects provided by both the logger and handler
+types are factories for the finished loggers and handlers.  These
+factories should be called with no arguments to retrieve the logger or
+log handler objects.  Calling the factories repeatedly will cause the
+same objects to be returned each time, so it's safe to simply call
+them to retrieve the objects.
+
+The factories for the logger objects, whether the \datatype{eventlog}
+or \datatype{logger} section type is used, provide a \method{reopen()}
+method which may be called to close any log files and re-open them.
+This is useful when using a \UNIX{} signal to effect log file
+rotation: the signal handler can call this method, and not have to
+worry about what handlers have been registered for the logger.
+
+Building an application that uses the logging components is fairly
+straightforward.  The schema needs to import the relevant components
+and declare their use:
+
+\begin{verbatim}
+<schema>
+  <import package="ZConfig.components.logger" file="eventlog.xml"/>
+  <import package="ZConfig.components.logger" file="handlers.xml"/>
+
+  <section type="eventlog" name="*" attribute="eventlog"
+           required="yes"/>
+</schema>
+\end{verbatim}
+
+In the application, the schema and configuration file should be loaded
+normally.  Once the configuration object is available, the logger
+factory should be called to configure Python's \module{logging} package:
+
+\begin{verbatim}
+import os
+import ZConfig
+
+def run(configfile):
+    schemafile = os.path.join(os.path.dirname(__file__), "schema.xml")
+    schema = ZConfig.loadSchema(schemafile)
+    config, handlers = ZConfig.loadConfig(schema, configfile)
+
+    # configure the logging package:
+    config.eventlog()
+
+    # now do interesting things
+\end{verbatim}
+
+An example configuration file for this application may look like this:
+
+\begin{verbatim}
+<eventlog>
+  level  info
+
+  <logfile>
+    path        /var/log/myapp
+    format      %(asctime)s %(levelname)s %(name)s %(message)s
+    # locale-specific date/time representation
+    dateformat  %c
+  </logfile>
+
+  <syslog>
+    level    error
+    address  syslog.example.net:514
+    format   %(levelname)s %(name)s %(message)s
+  </syslog>
+</eventlog>
+\end{verbatim}
+
+Refer to the \module{logging} package documentation for the names
+available in the message format strings (the \code{format} key in the
+log handlers).  The date format strings (the \code{dateformat} key in
+the log handlers) are the same as those accepted by the
+\function{time.strftime()} function.
+
+
+\begin{seealso}
+  \seepep{282}{A Logging System}
+         {The proposal which described the logging feature for
+          inclusion in the Python standard library.}
+  \seelink{http://docs.python.org/lib/module-logging.html}
+          {\module{logging} --- Logging facility for Python}
+          {Python's \module{logging} package documentation, from the
+           \citetitle[http://docs.python.org/lib/lib.html]
+           {Python Library Reference}.}
+  \seelink{http://www.red-dove.com/python_logging.html}
+          {Original Python \module{logging} package}
+          {This is the original source for the \module{logging}
+           package.  This is mostly of historical interest.}
+\end{seealso}
+
+
+\section{Using Components to Extend Schema}
+
+% XXX This section needs a lot of work, but should get people started
+% who really want to add new pieces to ZConfig-configured applications.
+
+It is possible to use schema components and the \keyword{\%import}
+construct to extend the set of section types available for a specific
+configuration file, and allow the new components to be used in place
+of standard components.
+
+The key to making this work is the use of abstract section types.
+Wherever the original schema accepts an abstract type, it is possible
+to load new implementations of the abstract type and use those instead
+of, or in addition to, the implementations loaded by the original
+schema.
+
+Abstract types are generally used to represent interfaces.  Sometimes
+these are interfaces for factory objects, and sometimes not, but
+there's an interface that the new component needs to implement.  What
+interface is required should be documented in the
+\element{description} element in the \element{abstracttype} element;
+this may be by reference to an interface specified in a Python module
+or described in some other bit of documentation.
+
+The following things need to be created to make the new component
+usable from the configuration file:
+
+\begin{enumerate}
+  \item An implementation of the required interface.
+
+  \item A schema component that defines a section type that contains
+        the information needed to construct the component.
+
+  \item A ``datatype'' function that converts configuration data to an
+        instance of the component.
+\end{enumerate}
+
+For simplicity, let's assume that the implementation is defined by a
+Python class.
+
+The example component we build here will be in the \module{noise}
+package, but any package will do.  Components loadable using
+\keyword{\%import} must be contained in the \file{component.xml} file;
+alternate filenames may not be selected by the \keyword{\%import}
+construct.
+
+Create a ZConfig component that provides a section type to support
+your component.  The new section type must declare that it implements
+the appropriate abstract type; it should probably look something like
+this:
+
+\begin{verbatim}
+<component prefix="noise.server">
+  <import package="ZServer"/>
+
+  <sectiontype name="noise-generator"
+               implements="ZServer.server"
+               datatype=".NoiseServerFactory">
+
+    <!-- specific configuration data should be described here -->
+
+    <key name="port"
+         datatype="port-number"
+         required="yes">
+      <description>
+        Port number to listen on.
+      </description>
+    </key>
+
+    <key name="color"
+         datatype=".noise_color"
+         default="white">
+      <description>
+        Silly way to specify a noise generation algorithm.
+      </description>
+    </key>
+
+  </sectiontype>
+</component>
+\end{verbatim}
+
+This example uses one of the standard ZConfig datatypes,
+\datatype{port-number}, and requires two additional types to be
+provided by the \module{noise.server} module:
+\class{NoiseServerFactory} and \function{noise_color()}.
+
+The \function{noise_color()} function is a datatype conversion for a
+key, so it accepts a string and returns the value that should be used:
+
+\begin{verbatim}
+_noise_colors = {
+    # color -> r,g,b
+    'white': (255, 255, 255),
+    'pink':  (255, 182, 193),
+    }
+
+def noise_color(string):
+    if string in _noise_colors:
+        return _noise_colors[string]
+    else:
+        raise ValueError('unknown noise color: %r' % string)
+\end{verbatim}
+
+\class{NoiseServerFactory} is a little different, as it's the datatype
+function for a section rather than a key.  The parameter isn't a
+string, but a section value object with two attributes, \member{port}
+and \member{color}.
+
+Since the \datatype{ZServer.server} abstract type requires that the
+component returned is a factory object, the datatype function can be
+implemented at the constructor for the class of the factory object.
+(If the datatype function could select different implementation
+classes based on the configuration values, it makes more sense to use
+a simple function that returns the appropriate implementation.)
+
+A class that implements this datatype might look like this:
+
+\begin{verbatim}
+from ZServer.datatypes import ServerFactory
+from noise.generator import WhiteNoiseGenerator, PinkNoiseGenerator
+
+class NoiseServerFactory(ServerFactory):
+
+    def __init__(self, section):
+        # host and ip will be initialized by ServerFactory.prepare()
+        self.host = None
+        self.ip = None
+        self.port = section.port
+        self.color = section.color
+
+    def create(self):
+        if self.color == 'white':
+            generator = WhiteNoiseGenerator()
+        else:
+            generator = PinkNoiseGenerator()
+        return NoiseServer(self.ip, self.port, generator)
+\end{verbatim}
+
+You'll need to arrange for the package containing this component is
+available on Python's \code{sys.path} before the configuration file is
+loaded; this is mostly easily done by manipulating the
+\envvar{PYTHONPATH} environment variable.
+
+Your configuration file can now include the following to load and use
+your new component:
+
+\begin{verbatim}
+%import noise
+
+<noise-generator>
+  port 1234
+  color white
+</noise-generator>
+\end{verbatim}
+
+
+\section{\module{ZConfig} --- Basic configuration support}
+
+\declaremodule{}{ZConfig}
+\modulesynopsis{Configuration package.}
+
+The main \module{ZConfig} package exports these convenience functions:
+
+\begin{funcdesc}{loadConfig}{schema, url\optional{, overrides}}
+  Load and return a configuration from a URL or pathname given by
+  \var{url}.  \var{url} may be a URL, absolute pathname, or relative
+  pathname.  Fragment identifiers are not supported.  \var{schema} is
+  a reference to a schema loaded by \function{loadSchema()} or
+  \function{loadSchemaFile()}.
+  The return value is a tuple containing the configuration object and
+  a composite handler that, when called with a name-to-handler
+  mapping, calls all the handlers for the configuration.
+
+  The optional \var{overrides} argument represents information derived
+  from command-line arguments.  If given, it must be either a sequence
+  of value specifiers, or \code{None}.  A \dfn{value specifier} is a
+  string of the form \code{\var{optionpath}=\var{value}}.  The
+  \var{optionpath} specifies the ``full path'' to the configuration
+  setting: it can contain a sequence of names, separated by
+  \character{/} characters. Each name before the last names a section
+  from the configuration file, and the last name corresponds to a key
+  within the section identified by the leading section names.  If
+  \var{optionpath} contains only one name, it identifies a key in the
+  top-level schema.  \var{value} is a string that will be treated
+  just like a value in the configuration file.
+\end{funcdesc}
+
+\begin{funcdesc}{loadConfigFile}{schema, file\optional{,
+                                 url\optional{, overrides}}}
+  Load and return a configuration from an opened file object.  If
+  \var{url} is omitted, one will be computed based on the
+  \member{name} attribute of \var{file}, if it exists.  If no URL can
+  be determined, all \keyword{\%include} statements in the
+  configuration must use absolute URLs.  \var{schema} is a reference
+  to a schema loaded by \function{loadSchema()} or
+  \function{loadSchemaFile()}.
+  The return value is a tuple containing the configuration object and
+  a composite handler that, when called with a name-to-handler
+  mapping, calls all the handlers for the configuration.
+  The \var{overrides} argument is the same as for the
+  \function{loadConfig()} function.
+\end{funcdesc}
+
+\begin{funcdesc}{loadSchema}{url}
+  Load a schema definition from the URL \var{url}.
+  \var{url} may be a URL, absolute pathname, or relative pathname.
+  Fragment identifiers are not supported.
+  The resulting
+  schema object can be passed to \function{loadConfig()} or
+  \function{loadConfigFile()}.  The schema object may be used as many
+  times as needed.
+\end{funcdesc}
+
+\begin{funcdesc}{loadSchemaFile}{file\optional{, url}}
+  Load a schema definition from the open file object \var{file}.  If
+  \var{url} is given and not \code{None}, it should be the URL of
+  resource represented by \var{file}.  If \var{url} is omitted or
+  \code{None}, a URL may be computed from the \member{name} attribute
+  of \var{file}, if present.  The resulting schema object can
+  be passed to \function{loadConfig()} or \function{loadConfigFile()}.
+  The schema object may be used as many times as needed.
+\end{funcdesc}
+
+The following exceptions are defined by this package:
+
+\begin{excdesc}{ConfigurationError}
+  Base class for exceptions specific to the \module{ZConfig} package.
+  All instances provide a \member{message} attribute that describes
+  the specific error, and a \member{url} attribute that gives the URL
+  of the resource the error was located in, or \constant{None}.
+\end{excdesc}
+
+\begin{excdesc}{ConfigurationSyntaxError}
+  Exception raised when a configuration source does not conform to the
+  allowed syntax.  In addition to the \member{message} and
+  \member{url} attributes, exceptions of this type offer the
+  \member{lineno} attribute, which provides the line number at which
+  the error was detected.
+\end{excdesc}
+
+\begin{excdesc}{DataConversionError}
+  Raised when a data type conversion fails with
+  \exception{ValueError}.  This exception is a subclass of both
+  \exception{ConfigurationError} and \exception{ValueError}.  The
+  \function{str()} of the exception provides the explanation from the
+  original \exception{ValueError}, and the line number and URL of the
+  value which provoked the error.  The following additional attributes
+  are provided:
+
+  \begin{tableii}{l|l}{member}{Attribute}{Value}
+    \lineii{colno}
+           {column number at which the value starts, or \code{None}}
+    \lineii{exception}
+           {the original \exception{ValueError} instance}
+    \lineii{lineno}
+           {line number on which the value starts}
+    \lineii{message}
+           {\function{str()} returned by the original \exception{ValueError}}
+    \lineii{value}
+           {original value passed to the conversion function}
+    \lineii{url}
+           {URL of the resource providing the value text}
+  \end{tableii}
+\end{excdesc}
+
+\begin{excdesc}{SchemaError}
+  Raised when a schema contains an error.  This exception type
+  provides the attributes \member{url}, \member{lineno}, and
+  \member{colno}, which provide the source URL, the line number, and
+  the column number at which the error was detected.  These attributes
+  may be \code{None} in some cases.
+\end{excdesc}
+
+\begin{excdesc}{SchemaResourceError}
+  Raised when there's an error locating a resource required by the
+  schema.  This is derived from \exception{SchemaError}.  Instances of
+  this exception class add the attributes \member{filename},
+  \member{package}, and \member{path}, which hold the filename
+  searched for within the package being loaded, the name of the
+  package, and the \code{__path__} attribute of the package itself (or
+  \constant{None} if it isn't a package or could not be imported).
+\end{excdesc}
+
+\begin{excdesc}{SubstitutionReplacementError}
+  Raised when the source text contains references to names which are
+  not defined in \var{mapping}.  The attributes \member{source} and
+  \member{name} provide the complete source text and the name
+  (converted to lower case) for which no replacement is defined.
+\end{excdesc}
+
+\begin{excdesc}{SubstitutionSyntaxError}
+  Raised when the source text contains syntactical errors.
+\end{excdesc}
+
+
+\subsection{Basic Usage}
+
+The simplest use of \refmodule{ZConfig} is to load a configuration
+based on a schema stored in a file.  This example loads a
+configuration file specified on the command line using a schema in the
+same directory as the script:
+
+\begin{verbatim}
+import os
+import sys
+import ZConfig
+
+try:
+    myfile = __file__
+except NameError:
+    myfile = os.path.realpath(sys.argv[0])
+
+mydir = os.path.dirname(myfile)
+
+schema = ZConfig.loadSchema(os.path.join(mydir, 'schema.xml'))
+conf, handler = ZConfig.loadConfig(schema, sys.argv[1])
+\end{verbatim}
+
+If the schema file contained this schema:
+
+\begin{verbatim}
+<schema>
+  <key name='server' required='yes'/>
+  <key name='attempts' datatype='integer' default='5'/>
+</schema>
+\end{verbatim}
+
+and the file specified on the command line contained this text:
+
+\begin{verbatim}
+# sample configuration
+
+server www.example.com
+\end{verbatim}
+
+then the configuration object \code{conf} loaded above would have two
+attributes:
+
+\begin{tableii}{l|l}{member}{Attribute}{Value}
+  \lineii{server}{\code{'www.example.com'}}
+  \lineii{attempts}{\code{5}}
+\end{tableii}
+
+
+\section{\module{ZConfig.datatypes} --- Default data type registry}
+
+\declaremodule{}{ZConfig.datatypes}
+\modulesynopsis{Default implementation of a data type registry}
+
+The \module{ZConfig.datatypes} module provides the implementation of
+the default data type registry and all the standard data types
+supported by \module{ZConfig}.  A number of convenience classes are
+also provided to assist in the creation of additional data types.
+
+A \dfn{datatype registry} is an object that provides conversion
+functions for data types.  The interface for a registry is fairly
+simple.
+
+A \dfn{conversion function} is any callable object that accepts a
+single argument and returns a suitable value, or raises an exception
+if the input value is not acceptable.  \exception{ValueError} is the
+preferred exception for disallowed inputs, but any other exception
+will be properly propagated.
+
+\begin{classdesc}{Registry}{\optional{stock}}
+  Implementation of a simple type registry.  If given, \var{stock}
+  should be a mapping which defines the ``built-in'' data types for
+  the registry; if omitted or \code{None}, the standard set of data
+  types is used (see section~\ref{standard-datatypes}, ``Standard
+  \module{ZConfig} Datatypes'').
+\end{classdesc}
+
+\class{Registry} objects have the following methods:
+
+\begin{methoddesc}{get}{name}
+  Return the type conversion routine for \var{name}.  If the
+  conversion function cannot be found, an (unspecified) exception is
+  raised.  If the name is not provided in the stock set of data types
+  by this registry and has not otherwise been registered, this method
+  uses the \method{search()} method to load the conversion function.
+  This is the only method the rest of \module{ZConfig} requires.
+\end{methoddesc}
+
+\begin{methoddesc}{register}{name, conversion}
+  Register the data type name \var{name} to use the conversion
+  function \var{conversion}.  If \var{name} is already registered or
+  provided as a stock data type, \exception{ValueError} is raised
+  (this includes the case when \var{name} was found using the
+  \method{search()} method).
+\end{methoddesc}
+
+\begin{methoddesc}{search}{name}
+  This is a helper method for the default implementation of the
+  \method{get()} method.  If \var{name} is a Python dotted-name, this
+  method loads the value for the name by dynamically importing the
+  containing module and extracting the value of the name.  The name
+  must refer to a usable conversion function.
+\end{methoddesc}
+
+
+The following classes are provided to define conversion functions:
+
+\begin{classdesc}{MemoizedConversion}{conversion}
+  Simple memoization for potentially expensive conversions.  This
+  conversion helper caches each successful conversion for re-use at a
+  later time; failed conversions are not cached in any way, since it
+  is difficult to raise a meaningful exception providing information
+  about the specific failure.
+\end{classdesc}
+
+\begin{classdesc}{RangeCheckedConversion}{conversion\optional{,
+                                          min\optional{, max}}}
+  Helper that performs range checks on the result of another
+  conversion.  Values passed to instances of this conversion are
+  converted using \var{conversion} and then range checked.  \var{min}
+  and \var{max}, if given and not \code{None}, are the inclusive
+  endpoints of the allowed range.  Values returned by \var{conversion}
+  which lay outside the range described by \var{min} and \var{max}
+  cause \exception{ValueError} to be raised.
+\end{classdesc}
+
+\begin{classdesc}{RegularExpressionConversion}{regex}
+  Conversion that checks that the input matches the regular expression
+  \var{regex}.  If it matches, returns the input, otherwise raises
+  \exception{ValueError}.
+\end{classdesc}
+
+
+\section{\module{ZConfig.loader} --- Resource loading support}
+
+\declaremodule{}{ZConfig.loader}
+\modulesynopsis{Support classes for resource loading}
+
+This module provides some helper classes used by the primary APIs
+exported by the \module{ZConfig} package.  These classes may be useful
+for some applications, especially applications that want to use a
+non-default data type registry.
+
+\begin{classdesc}{Resource}{file, url\optional{, fragment}}
+  Object that allows an open file object and a URL to be bound
+  together to ease handling.  Instances have the attributes
+  \member{file}, \member{url}, and \member{fragment} which store the
+  constructor arguments.  These objects also have a \method{close()}
+  method which will call \method{close()} on \var{file}, then set the
+  \member{file} attribute to \code{None} and the \member{closed} to
+  \constant{True}.
+\end{classdesc}
+
+\begin{classdesc}{BaseLoader}{}
+  Base class for loader objects.  This should not be instantiated
+  directly, as the \method{loadResource()} method must be overridden
+  for the instance to be used via the public API.
+\end{classdesc}
+
+\begin{classdesc}{ConfigLoader}{schema}
+  Loader for configuration files.  Each configuration file must
+  conform to the schema \var{schema}.  The \method{load*()} methods
+  return a tuple consisting of the configuration object and a
+  composite handler.
+\end{classdesc}
+
+\begin{classdesc}{SchemaLoader}{\optional{registry}}
+  Loader that loads schema instances.  All schema loaded by a
+  \class{SchemaLoader} will use the same data type registry.  If
+  \var{registry} is provided and not \code{None}, it will be used,
+  otherwise an instance of \class{ZConfig.datatypes.Registry} will be
+  used.
+\end{classdesc}
+
+
+\subsection{Loader Objects}
+
+Loader objects provide a general public interface, an interface which
+subclasses must implement, and some utility methods.
+
+The following methods provide the public interface:
+
+\begin{methoddesc}[loader]{loadURL}{url}
+  Open and load a resource specified by the URL \var{url}.
+  This method uses the \method{loadResource()} method to perform the
+  actual load, and returns whatever that method returns.
+\end{methoddesc}
+
+\begin{methoddesc}[loader]{loadFile}{file\optional{, url}}
+  Load from an open file object, \var{file}.  If given and not
+  \code{None}, \var{url} should be the URL of the resource represented
+  by \var{file}.  If omitted or \code{None}, the \member{name}
+  attribute of \var{file} is used to compute a \code{file:} URL, if
+  present.
+  This method uses the \method{loadResource()} method to perform the
+  actual load, and returns whatever that method returns.
+\end{methoddesc}
+
+The following method must be overridden by subclasses:
+
+\begin{methoddesc}[loader]{loadResource}{resource}
+  Subclasses of \class{BaseLoader} must implement this method to
+  actually load the resource and return the appropriate
+  application-level object.
+\end{methoddesc}
+
+The following methods can be used as utilities:
+
+\begin{methoddesc}[loader]{isPath}{s}
+  Return true if \var{s} should be considered a filesystem path rather
+  than a URL.
+\end{methoddesc}
+
+\begin{methoddesc}[loader]{normalizeURL}{url-or-path}
+  Return a URL for \var{url-or-path}.  If \var{url-or-path} refers to
+  an existing file, the corresponding \code{file:} URL is returned.
+  Otherwise \var{url-or-path} is checked for sanity: if it
+  does not have a schema, \exception{ValueError} is raised, and if it
+  does have a fragment identifier, \exception{ConfigurationError} is
+  raised.
+  This uses \method{isPath()} to determine whether \var{url-or-path}
+  is a URL of a filesystem path.
+\end{methoddesc}
+
+\begin{methoddesc}[loader]{openResource}{url}
+  Returns a resource object that represents the URL \var{url}.  The
+  URL is opened using the \function{urllib2.urlopen()} function, and
+  the returned resource object is created using
+  \method{createResource()}.  If the URL cannot be opened,
+  \exception{ConfigurationError} is raised.
+\end{methoddesc}
+
+\begin{methoddesc}[loader]{createResource}{file, url}
+  Returns a resource object for an open file and URL, given as
+  \var{file} and \var{url}, respectively.  This may be overridden by a
+  subclass if an alternate resource implementation is desired.
+\end{methoddesc}
+
+
+\section{\module{ZConfig.cmdline} --- Command-line override support}
+
+\declaremodule{}{ZConfig.cmdline}
+\modulesynopsis{Support for command-line overrides for configuration
+                settings.}
+
+This module exports an extended version of the \class{ConfigLoader}
+class from the \refmodule{ZConfig.loader} module.  This provides
+support for overriding specific settings from the configuration file
+from the command line, without requiring the application to provide
+specific options for everything the configuration file can include.
+
+\begin{classdesc}{ExtendedConfigLoader}{schema}
+  Construct a \class{ConfigLoader} subclass that adds support for
+  command-line overrides.
+\end{classdesc}
+
+The following additional method is provided, and is the only way to
+provide position information to associate with command-line
+parameters:
+
+\begin{methoddesc}{addOption}{spec\optional{, pos}}
+  Add a single value to the list of overridden values.  The \var{spec}
+  argument is a value specified, as described for the
+  \function{\refmodule{ZConfig}.loadConfig()} function.  A source
+  position for the specifier may be given as \var{pos}.  If \var{pos}
+  is specified and not \code{None}, it must be a sequence of three
+  values.  The first is the URL of the source (or some other
+  identifying string).  The second and third are the line number and
+  column of the setting.  These position information is only used to
+  construct a \exception{DataConversionError} when data conversion
+  fails.
+\end{methoddesc}
+
+
+\section{\module{ZConfig.substitution} --- String substitution}
+
+\declaremodule{}{ZConfig.substitution}
+\modulesynopsis{Shell-style string substitution helper.}
+
+This module provides a basic substitution facility similar to that
+found in the Bourne shell (\program{sh} on most \UNIX{} platforms).  
+
+The replacements supported by this module include:
+
+\begin{tableiii}{l|l|c}{code}{Source}{Replacement}{Notes}
+  \lineiii{\$\$}{\code{\$}}{(1)}
+  \lineiii{\$\var{name}}{The result of looking up \var{name}}{(2)}
+  \lineiii{\$\{\var{name}\}}{The result of looking up \var{name}}{}
+\end{tableiii}
+
+\noindent
+Notes:
+\begin{description}
+  \item[(1)]  This is different from the Bourne shell, which uses
+              \code{\textbackslash\$} to generate a \character{\$} in
+              the result text.  This difference avoids having as many
+              special characters in the syntax.
+
+  \item[(2)]  Any character which immediately follows \var{name} may
+              not be a valid character in a name.
+\end{description}
+
+In each case, \var{name} is a non-empty sequence of alphanumeric and
+underscore characters not starting with a digit.  If there is not a
+replacement for \var{name}, the exception
+\exception{SubstitutionReplacementError} is raised.
+Note that the lookup is expected to be case-insensitive; this module
+will always use a lower-case version of the name to perform the query.
+
+This module provides these functions:
+
+\begin{funcdesc}{substitute}{s, mapping}
+  Substitute values from \var{mapping} into \var{s}.  \var{mapping}
+  can be a \class{dict} or any type that supports the \method{get()}
+  method of the mapping protocol.  Replacement
+  values are copied into the result without further interpretation.
+  Raises \exception{SubstitutionSyntaxError} if there are malformed
+  constructs in \var{s}.
+\end{funcdesc}
+
+\begin{funcdesc}{isname}{s}
+  Returns \constant{True} if \var{s} is a valid name for a substitution
+  text, otherwise returns \constant{False}.
+\end{funcdesc}
+
+
+\subsection{Examples}
+
+\begin{verbatim}
+>>> from ZConfig.substitution import substitute
+>>> d = {'name': 'value',
+...      'top': '$middle',
+...      'middle' : 'bottom'}
+>>>
+>>> substitute('$name', d)
+'value'
+>>> substitute('$top', d)
+'$middle'
+\end{verbatim}
+
+
+\appendix
+\section{Schema Document Type Definition \label{schema-dtd}}
+
+The following is the XML Document Type Definition for \module{ZConfig}
+schema:
+
+\verbatiminput{schema.dtd}
+
+\end{document}

trunk/src/ZConfig/info.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Objects that can describe a ZConfig schema."""
+
+import copy
+
+import ZConfig
+
+
+class UnboundedThing:
+    __metaclass__ = type
+    __slots__ = ()
+
+    def __lt__(self, other):
+        return False
+
+    def __le__(self, other):
+        return isinstance(other, self.__class__)
+
+    def __gt__(self, other):
+        return True
+
+    def __ge__(self, other):
+        return True
+
+    def __eq__(self, other):
+        return isinstance(other, self.__class__)
+
+    def __ne__(self, other):
+        return not isinstance(other, self.__class__)
+
+    def __repr__(self):
+        return "<Unbounded>"
+
+Unbounded = UnboundedThing()
+
+
+class ValueInfo:
+    __metaclass__ = type
+    __slots__ = 'value', 'position'
+
+    def __init__(self, value, position):
+        self.value = value
+        # position is (lineno, colno, url)
+        self.position = position
+
+    def convert(self, datatype):
+        try:
+            return datatype(self.value)
+        except ValueError, e:
+            raise ZConfig.DataConversionError(e, self.value, self.position)
+
+
+class BaseInfo:
+    """Information about a single configuration key."""
+
+    description = None
+    example = None
+    metadefault = None
+
+    def __init__(self, name, datatype, minOccurs, maxOccurs, handler,
+                 attribute):
+        if maxOccurs is not None and maxOccurs < 1:
+            if maxOccurs < 1:
+                raise ZConfig.SchemaError(
+                    "maxOccurs must be at least 1")
+            if minOccurs is not None and minOccurs < maxOccurs:
+                raise ZConfig.SchemaError(
+                    "minOccurs must be at least maxOccurs")
+        self.name = name
+        self.datatype = datatype
+        self.minOccurs = minOccurs
+        self.maxOccurs = maxOccurs
+        self.handler = handler
+        self.attribute = attribute
+
+    def __repr__(self):
+        clsname = self.__class__.__name__
+        return "<%s for %s>" % (clsname, `self.name`)
+
+    def isabstract(self):
+        return False
+
+    def ismulti(self):
+        return self.maxOccurs > 1
+
+    def issection(self):
+        return False
+
+
+class BaseKeyInfo(BaseInfo):
+
+    _rawdefaults = None
+
+    def __init__(self, name, datatype, minOccurs, maxOccurs, handler,
+                 attribute):
+        assert minOccurs is not None
+        BaseInfo.__init__(self, name, datatype, minOccurs, maxOccurs,
+                          handler, attribute)
+        self._finished = False
+
+    def finish(self):
+        if self._finished:
+            raise ZConfig.SchemaError(
+                "cannot finish KeyInfo more than once")
+        self._finished = True
+
+    def adddefault(self, value, position, key=None):
+        if self._finished:
+            raise ZConfig.SchemaError(
+                "cannot add default values to finished KeyInfo")
+        # Check that the name/keyed relationship is right:
+        if self.name == "+" and key is None:
+            raise ZConfig.SchemaError(
+                "default values must be keyed for name='+'")
+        elif self.name != "+" and key is not None:
+            raise ZConfig.SchemaError(
+                "unexpected key for default value")
+        self.add_valueinfo(ValueInfo(value, position), key)
+
+    def add_valueinfo(self, vi, key):
+        """Actually add a ValueInfo to this key-info object.
+
+        The appropriate value of None-ness of key has already been
+        checked with regard to the name of the key, and has been found
+        permissible to add.
+
+        This method is a requirement for subclasses, and should not be
+        called by client code.
+        """
+        raise NotImplementedError(
+            "add_valueinfo() must be implemented by subclasses of BaseKeyInfo")
+
+    def prepare_raw_defaults(self):
+        assert self.name == "+"
+        if self._rawdefaults is None:
+            self._rawdefaults = self._default
+        self._default = {}
+
+
+class KeyInfo(BaseKeyInfo):
+
+    _default = None
+
+    def __init__(self, name, datatype, minOccurs, handler, attribute):
+        BaseKeyInfo.__init__(self, name, datatype, minOccurs, 1,
+                             handler, attribute)
+        if self.name == "+":
+            self._default = {}
+
+    def add_valueinfo(self, vi, key):
+        if self.name == "+":
+            if self._default.has_key(key):
+                # not ideal: we're presenting the unconverted
+                # version of the key
+                raise ZConfig.SchemaError(
+                    "duplicate default value for key %s" % `key`)
+            self._default[key] = vi
+        elif self._default is not None:
+            raise ZConfig.SchemaError(
+                "cannot set more than one default to key with maxOccurs == 1")
+        else:
+            self._default = vi
+
+    def computedefault(self, keytype):
+        self.prepare_raw_defaults()
+        for k, vi in self._rawdefaults.iteritems():
+            key = ValueInfo(k, vi.position).convert(keytype)
+            self.add_valueinfo(vi, key)
+
+    def getdefault(self):
+        # Use copy.copy() to make sure we don't allow polution of
+        # our internal data without having to worry about both the
+        # list and dictionary cases:
+        return copy.copy(self._default)
+
+
+class MultiKeyInfo(BaseKeyInfo):
+
+    def __init__(self, name, datatype, minOccurs, maxOccurs, handler,
+                 attribute):
+        BaseKeyInfo.__init__(self, name, datatype, minOccurs, maxOccurs,
+                             handler, attribute)
+        if self.name == "+":
+            self._default = {}
+        else:
+            self._default = []
+
+    def add_valueinfo(self, vi, key):
+        if self.name == "+":
+            # This is a keyed value, not a simple value:
+            if key in self._default:
+                self._default[key].append(vi)
+            else:
+                self._default[key] = [vi]
+        else:
+            self._default.append(vi)
+
+    def computedefault(self, keytype):
+        self.prepare_raw_defaults()
+        for k, vlist in self._rawdefaults.iteritems():
+            key = ValueInfo(k, vlist[0].position).convert(keytype)
+            for vi in vlist:
+                self.add_valueinfo(vi, key)
+
+    def getdefault(self):
+        return copy.copy(self._default)
+
+
+class SectionInfo(BaseInfo):
+    def __init__(self, name, sectiontype, minOccurs, maxOccurs, handler,
+                 attribute):
+        # name        - name of the section; one of '*', '+', or name1
+        # sectiontype - SectionType instance
+        # minOccurs   - minimum number of occurances of the section
+        # maxOccurs   - maximum number of occurances; if > 1, name
+        #               must be '*' or '+'
+        # handler     - handler name called when value(s) must take effect,
+        #               or None
+        # attribute   - name of the attribute on the SectionValue object
+        if maxOccurs > 1:
+            if name not in ('*', '+'):
+                raise ZConfig.SchemaError(
+                    "sections which can occur more than once must"
+                    " use a name of '*' or '+'")
+            if not attribute:
+                raise ZConfig.SchemaError(
+                    "sections which can occur more than once must"
+                    " specify a target attribute name")
+        if sectiontype.isabstract():
+            datatype = None
+        else:
+            datatype = sectiontype.datatype
+        BaseInfo.__init__(self, name, datatype,
+                          minOccurs, maxOccurs, handler, attribute)
+        self.sectiontype = sectiontype
+
+    def __repr__(self):
+        clsname = self.__class__.__name__
+        return "<%s for %s (%s)>" % (
+            clsname, self.sectiontype.name, `self.name`)
+
+    def issection(self):
+        return True
+
+    def allowUnnamed(self):
+        return self.name == "*"
+
+    def isAllowedName(self, name):
+        if name == "*" or name == "+":
+            return False
+        elif self.name == "+":
+            return name and True or False
+        elif self.name == "*":
+            return True
+        else:
+            return name == self.name
+
+    def getdefault(self):
+        # sections cannot have defaults
+        if self.maxOccurs > 1:
+            return []
+        else:
+            return None
+
+
+class AbstractType:
+    __metaclass__ = type
+    __slots__ = '_subtypes', 'name', 'description'
+
+    def __init__(self, name):
+        self._subtypes = {}
+        self.name = name
+        self.description = None
+
+    def addsubtype(self, type):
+        self._subtypes[type.name] = type
+
+    def getsubtype(self, name):
+        try:
+            return self._subtypes[name]
+        except KeyError:
+            raise ZConfig.SchemaError("no sectiontype %s in abstracttype %s"
+                                      % (`name`, `self.name`))
+
+    def hassubtype(self, name):
+        """Return true iff this type has 'name' as a concrete manifestation."""
+        return name in self._subtypes.keys()
+
+    def getsubtypenames(self):
+        """Return the names of all concrete types as a sorted list."""
+        L = self._subtypes.keys()
+        L.sort()
+        return L
+
+    def isabstract(self):
+        return True
+
+
+class SectionType:
+    def __init__(self, name, keytype, valuetype, datatype, registry, types):
+        # name      - name of the section, or '*' or '+'
+        # datatype  - type for the section itself
+        # keytype   - type for the keys themselves
+        # valuetype - default type for key values
+        self.name = name
+        self.datatype = datatype
+        self.keytype = keytype
+        self.valuetype = valuetype
+        self.handler = None
+        self.description = None
+        self.registry = registry
+        self._children = []    # [(key, info), ...]
+        self._attrmap = {}     # {attribute: info, ...}
+        self._keymap = {}      # {key: info, ...}
+        self._types = types
+
+    def gettype(self, name):
+        n = name.lower()
+        try:
+            return self._types[n]
+        except KeyError:
+            raise ZConfig.SchemaError("unknown type name: " + `name`)
+
+    def gettypenames(self):
+        return self._types.keys()
+
+    def __len__(self):
+        return len(self._children)
+
+    def __getitem__(self, index):
+        return self._children[index]
+
+    def _add_child(self, key, info):
+        # check naming constraints
+        assert key or info.attribute
+        if key and self._keymap.has_key(key):
+            raise ZConfig.SchemaError(
+                "child name %s already used" % key)
+        if info.attribute and self._attrmap.has_key(info.attribute):
+            raise ZConfig.SchemaError(
+                "child attribute name %s already used" % info.attribute)
+        # a-ok, add the item to the appropriate maps
+        if info.attribute:
+            self._attrmap[info.attribute] = info
+        if key:
+            self._keymap[key] = info
+        self._children.append((key, info))
+
+    def addkey(self, keyinfo):
+        self._add_child(keyinfo.name, keyinfo)
+
+    def addsection(self, name, sectinfo):
+        assert name not in ("*", "+")
+        self._add_child(name, sectinfo)
+
+    def getinfo(self, key):
+        if not key:
+            raise ZConfig.ConfigurationError(
+                "cannot match a key without a name")
+        try:
+            return self._keymap[key]
+        except KeyError:
+            raise ZConfig.ConfigurationError("no key matching " + `key`)
+
+    def getrequiredtypes(self):
+        d = {}
+        if self.name:
+            d[self.name] = 1
+        stack = [self]
+        while stack:
+            info = stack.pop()
+            for key, ci in info._children:
+                if ci.issection():
+                    t = ci.sectiontype
+                    if not d.has_key(t.name):
+                        d[t.name] = 1
+                        stack.append(t)
+        return d.keys()
+
+    def getsectioninfo(self, type, name):
+        for key, info in self._children:
+            if key:
+                if key == name:
+                    if not info.issection():
+                        raise ZConfig.ConfigurationError(
+                            "section name %s already in use for key" % key)
+                    st = info.sectiontype
+                    if st.isabstract():
+                        try:
+                            st = st.getsubtype(type)
+                        except ZConfig.ConfigurationError:
+                            raise ZConfig.ConfigurationError(
+                                "section type %s not allowed for name %s"
+                                % (`type`, `key`))
+                    if not st.name == type:
+                        raise ZConfig.ConfigurationError(
+                            "name %s must be used for a %s section"
+                            % (`name`, `st.name`))
+                    return info
+            # else must be a sectiontype or an abstracttype:
+            elif info.sectiontype.name == type:
+                if not (name or info.allowUnnamed()):
+                    raise ZConfig.ConfigurationError(
+                        `type` + " sections must be named")
+                return info
+            elif info.sectiontype.isabstract():
+                st = info.sectiontype
+                if st.name == type:
+                    raise ZConfig.ConfigurationError(
+                        "cannot define section with an abstract type")
+                try:
+                    st = st.getsubtype(type)
+                except ZConfig.ConfigurationError:
+                    # not this one; maybe a different one
+                    pass
+                else:
+                    return info
+        raise ZConfig.ConfigurationError(
+            "no matching section defined for type='%s', name='%s'" % (
+            type, name))
+
+    def isabstract(self):
+        return False
+
+
+class SchemaType(SectionType):
+    def __init__(self, keytype, valuetype, datatype, handler, url,
+                 registry):
+        SectionType.__init__(self, None, keytype, valuetype, datatype,
+                             registry, {})
+        self._components = {}
+        self.handler = handler
+        self.url = url
+
+    def addtype(self, typeinfo):
+        n = typeinfo.name
+        if self._types.has_key(n):
+            raise ZConfig.SchemaError("type name cannot be redefined: "
+                                      + `typeinfo.name`)
+        self._types[n] = typeinfo
+
+    def allowUnnamed(self):
+        return True
+
+    def isAllowedName(self, name):
+        return False
+
+    def issection(self):
+        return True
+
+    def getunusedtypes(self):
+        alltypes = self.gettypenames()
+        reqtypes = self.getrequiredtypes()
+        for n in reqtypes:
+            alltypes.remove(n)
+        if self.name and self.name in alltypes:
+            alltypes.remove(self.name)
+        return alltypes
+
+    def createSectionType(self, name, keytype, valuetype, datatype):
+        t = SectionType(name, keytype, valuetype, datatype,
+                        self.registry, self._types)
+        self.addtype(t)
+        return t
+
+    def deriveSectionType(self, base, name, keytype, valuetype, datatype):
+        if isinstance(base, SchemaType):
+            raise ZConfig.SchemaError(
+                "cannot derive sectiontype from top-level schema")
+        t = self.createSectionType(name, keytype, valuetype, datatype)
+        t._attrmap.update(base._attrmap)
+        t._keymap.update(base._keymap)
+        t._children.extend(base._children)
+        for i in range(len(t._children)):
+            key, info = t._children[i]
+            if isinstance(info, BaseKeyInfo) and info.name == "+":
+                # need to create a new info object and recompute the
+                # default mapping based on the new keytype
+                info = copy.copy(info)
+                info.computedefault(t.keytype)
+                t._children[i] = (key, info)
+        return t
+
+    def addComponent(self, name):
+        if self._components.has_key(name):
+            raise ZConfig.SchemaError("already have component %s" % name)
+        self._components[name] = name
+
+    def hasComponent(self, name):
+        return self._components.has_key(name)
+
+
+def createDerivedSchema(base):
+    new = SchemaType(base.keytype, base.valuetype, base.datatype,
+                     base.handler, base.url, base.registry)
+    new._components.update(base._components)
+    new.description = base.description
+    new._children[:] = base._children
+    new._attrmap.update(base._attrmap)
+    new._keymap.update(base._keymap)
+    new._types.update(base._types)
+    return new

trunk/src/ZConfig/loader.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Schema loader utility."""
+
+import os.path
+import sys
+import urllib
+import urllib2
+
+import ZConfig
+import ZConfig.cfgparser
+import ZConfig.datatypes
+import ZConfig.info
+import ZConfig.matcher
+import ZConfig.schema
+import ZConfig.url
+
+
+def loadSchema(url):
+    return SchemaLoader().loadURL(url)
+
+def loadSchemaFile(file, url=None):
+    return SchemaLoader().loadFile(file, url)
+
+def loadConfig(schema, url, overrides=()):
+    return _get_config_loader(schema, overrides).loadURL(url)
+
+def loadConfigFile(schema, file, url=None, overrides=()):
+    return _get_config_loader(schema, overrides).loadFile(file, url)
+
+
+def _get_config_loader(schema, overrides):
+    if overrides:
+        from ZConfig import cmdline
+        loader = cmdline.ExtendedConfigLoader(schema)
+        for opt in overrides:
+            loader.addOption(opt)
+    else:
+        loader = ConfigLoader(schema)
+    return loader
+
+
+class BaseLoader:
+    def __init__(self):
+        pass
+
+    def createResource(self, file, url):
+        return Resource(file, url)
+
+    def loadURL(self, url):
+        url = self.normalizeURL(url)
+        r = self.openResource(url)
+        try:
+            return self.loadResource(r)
+        finally:
+            r.close()
+
+    def loadFile(self, file, url=None):
+        if not url:
+            url = _url_from_file(file)
+        r = self.createResource(file, url)
+        try:
+            return self.loadResource(r)
+        finally:
+            r.close()
+
+    # utilities
+
+    def loadResource(self, resource):
+        raise NotImplementedError(
+            "BaseLoader.loadResource() must be overridden by a subclass")
+
+    def openResource(self, url):
+        # ConfigurationError exceptions raised here should be
+        # str()able to generate a message for an end user.
+        #
+        # XXX This should be replaced to use a local cache for remote
+        # resources.  The policy needs to support both re-retrieve on
+        # change and provide the cached resource when the remote
+        # resource is not accessible.
+        url = str(url)
+        try:
+            file = urllib2.urlopen(url)
+        except urllib2.URLError, e:
+            # urllib2.URLError has a particularly hostile str(), so we
+            # generally don't want to pass it along to the user.
+            self._raise_open_error(url, e.reason)
+        except (IOError, OSError), e:
+            # Python 2.1 raises a different error from Python 2.2+,
+            # so we catch both to make sure we detect the situation.
+            self._raise_open_error(url, str(e))
+        return self.createResource(file, url)
+
+    def _raise_open_error(self, url, message):
+        if url[:7].lower() == "file://":
+            what = "file"
+            ident = urllib.url2pathname(url[7:])
+        else:
+            what = "URL"
+            ident = url
+        raise ZConfig.ConfigurationError(
+            "error opening %s %s: %s" % (what, ident, message),
+            url)
+
+    def normalizeURL(self, url):
+        if self.isPath(url):
+            url = "file://" + urllib.pathname2url(os.path.abspath(url))
+        newurl, fragment = ZConfig.url.urldefrag(url)
+        if fragment:
+            raise ZConfig.ConfigurationError(
+                "fragment identifiers are not supported",
+                url)
+        return newurl
+
+    def isPath(self, s):
+        """Return True iff 's' should be handled as a filesystem path."""
+        if ":" in s:
+            # XXX This assumes that one-character scheme identifiers
+            # are always Windows drive letters; I don't know of any
+            # one-character scheme identifiers.
+            scheme, rest = urllib.splittype(s)
+            return len(scheme) == 1
+        else:
+            return True
+
+
+
+def _url_from_file(file):
+    name = getattr(file, "name", None)
+    if name and name[0] != "<" and name[-1] != ">":
+        return "file://" + urllib.pathname2url(os.path.abspath(name))
+    else:
+        return None
+
+
+class SchemaLoader(BaseLoader):
+    def __init__(self, registry=None):
+        if registry is None:
+            registry = ZConfig.datatypes.Registry()
+        BaseLoader.__init__(self)
+        self.registry = registry
+        self._cache = {}
+
+    def loadResource(self, resource):
+        if resource.url and self._cache.has_key(resource.url):
+            schema = self._cache[resource.url]
+        else:
+            schema = ZConfig.schema.parseResource(resource, self)
+            self._cache[resource.url] = schema
+        return schema
+
+    # schema parser support API
+
+    def schemaComponentSource(self, package, file):
+        parts = package.split(".")
+        if not parts:
+            raise ZConfig.SchemaError(
+                "illegal schema component name: " + `package`)
+        if "" in parts:
+            # '' somewhere in the package spec; still illegal
+            raise ZConfig.SchemaError(
+                "illegal schema component name: " + `package`)
+        file = file or "component.xml"
+        try:
+            __import__(package)
+        except ImportError, e:
+            raise ZConfig.SchemaResourceError(
+                "could not load package %s: %s" % (package, str(e)),
+                filename=file,
+                package=package)
+        pkg = sys.modules[package]
+        if not hasattr(pkg, "__path__"):
+            raise ZConfig.SchemaResourceError(
+                "import name does not refer to a package",
+                filename=file, package=package)
+        for dir in pkg.__path__:
+            dirname = os.path.abspath(dir)
+            fn = os.path.join(dirname, file)
+            if os.path.exists(fn):
+                return "file://" + urllib.pathname2url(fn)
+        else:
+            raise ZConfig.SchemaResourceError("schema component not found",
+                                              filename=file,
+                                              package=package,
+                                              path=pkg.__path__)
+
+
+class ConfigLoader(BaseLoader):
+    def __init__(self, schema):
+        if schema.isabstract():
+            raise ZConfig.SchemaError(
+                "cannot check a configuration an abstract type")
+        BaseLoader.__init__(self)
+        self.schema = schema
+        self._private_schema = False
+
+    def loadResource(self, resource):
+        sm = self.createSchemaMatcher()
+        self._parse_resource(sm, resource)
+        result = sm.finish(), CompositeHandler(sm.handlers, self.schema)
+        return result
+
+    def createSchemaMatcher(self):
+        return ZConfig.matcher.SchemaMatcher(self.schema)
+
+    # config parser support API
+
+    def startSection(self, parent, type, name):
+        t = self.schema.gettype(type)
+        if t.isabstract():
+            raise ZConfig.ConfigurationError(
+                "concrete sections cannot match abstract section types;"
+                " found abstract type " + `type`)
+        return parent.createChildMatcher(t, name)
+
+    def endSection(self, parent, type, name, matcher):
+        sectvalue = matcher.finish()
+        parent.addSection(type, name, sectvalue)
+
+    def importSchemaComponent(self, pkgname):
+        schema = self.schema
+        if not self._private_schema:
+            # replace the schema with an extended schema on the first %import
+            self._loader = SchemaLoader(self.schema.registry)
+            schema = ZConfig.info.createDerivedSchema(self.schema)
+            self._private_schema = True
+            self.schema = schema
+        url = self._loader.schemaComponentSource(pkgname, '')
+        if schema.hasComponent(url):
+            return
+        resource = self.openResource(url)
+        schema.addComponent(url)
+        try:
+            ZConfig.schema.parseComponent(resource, self._loader, schema)
+        finally:
+            resource.close()
+
+    def includeConfiguration(self, section, url, defines):
+        url = self.normalizeURL(url)
+        r = self.openResource(url)
+        try:
+            self._parse_resource(section, r, defines)
+        finally:
+            r.close()
+
+    # internal helper
+
+    def _parse_resource(self, matcher, resource, defines=None):
+        parser = ZConfig.cfgparser.ZConfigParser(resource, self, defines)
+        parser.parse(matcher)
+
+
+class CompositeHandler:
+
+    def __init__(self, handlers, schema):
+        self._handlers = handlers
+        self._convert = schema.registry.get("basic-key")
+
+    def __call__(self, handlermap):
+        d = {}
+        for name, callback in handlermap.items():
+            n = self._convert(name)
+            if d.has_key(n):
+                raise ZConfig.ConfigurationError(
+                    "handler name not unique when converted to a basic-key: "
+                    + `name`)
+            d[n] = callback
+        L = []
+        for handler, value in self._handlers:
+            if not d.has_key(handler):
+                L.append(handler)
+        if L:
+            raise ZConfig.ConfigurationError(
+                "undefined handlers: " + ", ".join(L))
+        for handler, value in self._handlers:
+            f = d[handler]
+            if f is not None:
+                f(value)
+
+    def __len__(self):
+        return len(self._handlers)
+
+
+class Resource:
+    def __init__(self, file, url):
+        self.file = file
+        self.url = url
+
+    def close(self):
+        if self.file is not None:
+            self.file.close()
+            self.file = None
+            self.closed = True
+
+    def __getattr__(self, name):
+        return getattr(self.file, name)

trunk/src/ZConfig/matcher.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Utility that manages the binding of configuration data to a section."""
+
+import ZConfig
+
+from ZConfig.info import ValueInfo
+
+
+class BaseMatcher:
+    def __init__(self, info, type, handlers):
+        self.info = info
+        self.type = type
+        self._values = {}
+        for key, info in type:
+            if info.name == "+" and not info.issection():
+                v = {}
+            elif info.ismulti():
+                v = []
+            else:
+                v = None
+            assert info.attribute is not None
+            self._values[info.attribute] = v
+        self._sectionnames = {}
+        if handlers is None:
+            handlers = []
+        self.handlers = handlers
+
+    def __repr__(self):
+        clsname = self.__class__.__name__
+        extra = "type " + `self.type.name`
+        return "<%s for %s>" % (clsname, extra)
+
+    def addSection(self, type, name, sectvalue):
+        if name:
+            if self._sectionnames.has_key(name):
+                raise ZConfig.ConfigurationError(
+                    "section names must not be re-used within the"
+                    " same container:" + `name`)
+            self._sectionnames[name] = name
+        ci = self.type.getsectioninfo(type, name)
+        attr = ci.attribute
+        v = self._values[attr]
+        if ci.ismulti():
+            v.append(sectvalue)
+        elif v is None:
+            self._values[attr] = sectvalue
+        else:
+            raise ZConfig.ConfigurationError(
+                "too many instances of %s section" % `ci.sectiontype.name`)
+
+    def addValue(self, key, value, position):
+        try:
+            realkey = self.type.keytype(key)
+        except ValueError, e:
+            raise ZConfig.DataConversionError(e, key, position)
+        arbkey_info = None
+        for i in range(len(self.type)):
+            k, ci = self.type[i]
+            if k == realkey:
+                break
+            if ci.name == "+" and not ci.issection():
+                arbkey_info = k, ci
+        else:
+            if arbkey_info is None:
+                raise ZConfig.ConfigurationError(
+                    `key` + " is not a known key name")
+            k, ci = arbkey_info
+        if ci.issection():
+            if ci.name:
+                extra = " in %s sections" % `self.type.name`
+            else:
+                extra = ""
+            raise ZConfig.ConfigurationError(
+                "%s is not a valid key name%s" % (`key`, extra))
+
+        ismulti = ci.ismulti()
+        attr = ci.attribute
+        assert attr is not None
+        v = self._values[attr]
+        if v is None:
+            if k == '+':
+                v = {}
+            elif ismulti:
+                v = []
+            self._values[attr] = v
+        elif not ismulti:
+            if k != '+':
+                raise ZConfig.ConfigurationError(
+                    `key` + " does not support multiple values")
+        elif len(v) == ci.maxOccurs:
+            raise ZConfig.ConfigurationError(
+                "too many values for " + `name`)
+
+        value = ValueInfo(value, position)
+        if k == '+':
+            if ismulti:
+                if v.has_key(realkey):
+                    v[realkey].append(value)
+                else:
+                    v[realkey] = [value]
+            else:
+                if v.has_key(realkey):
+                    raise ZConfig.ConfigurationError(
+                        "too many values for " + `key`)
+                v[realkey] = value
+        elif ismulti:
+            v.append(value)
+        else:
+            self._values[attr] = value
+
+    def createChildMatcher(self, type, name):
+        ci = self.type.getsectioninfo(type.name, name)
+        assert not ci.isabstract()
+        if not ci.isAllowedName(name):
+            raise ZConfig.ConfigurationError(
+                "%s is not an allowed name for %s sections"
+                % (`name`, `ci.sectiontype.name`))
+        return SectionMatcher(ci, type, name, self.handlers)
+
+    def finish(self):
+        """Check the constraints of the section and convert to an application
+        object."""
+        values = self._values
+        for key, ci in self.type:
+            if key:
+                key = repr(key)
+            else:
+                key = "section type " + `ci.sectiontype.name`
+            assert ci.attribute is not None
+            attr = ci.attribute
+            v = values[attr]
+            if ci.name == '+' and not ci.issection():
+                # v is a dict
+                if ci.minOccurs > len(v):
+                    raise ZConfig.ConfigurationError(
+                        "no keys defined for the %s key/value map; at least %d"
+                        " must be specified" % (attr, ci.minOccurs))
+            if v is None and ci.minOccurs:
+                default = ci.getdefault()
+                if default is None:
+                    raise ZConfig.ConfigurationError(
+                        "no values for %s; %s required" % (key, ci.minOccurs))
+                else:
+                    v = values[attr] = default[:]
+            if ci.ismulti():
+                if not v:
+                    default = ci.getdefault()
+                    if isinstance(default, dict):
+                        v.update(default)
+                    else:
+                        v[:] = default
+                if len(v) < ci.minOccurs:
+                    raise ZConfig.ConfigurationError(
+                        "not enough values for %s; %d found, %d required"
+                        % (key, len(v), ci.minOccurs))
+            if v is None and not ci.issection():
+                if ci.ismulti():
+                    v = ci.getdefault()[:]
+                else:
+                    v = ci.getdefault()
+                values[attr] = v
+        return self.constuct()
+
+    def constuct(self):
+        values = self._values
+        for name, ci in self.type:
+            assert ci.attribute is not None
+            attr = ci.attribute
+            if ci.ismulti():
+                if ci.issection():
+                    v = []
+                    for s in values[attr]:
+                        if s is not None:
+                            st = s.getSectionDefinition()
+                            try:
+                                s = st.datatype(s)
+                            except ValueError, e:
+                                raise ZConfig.DataConversionError(
+                                    e, s, (-1, -1, None))
+                        v.append(s)
+                elif ci.name == '+':
+                    v = values[attr]
+                    for key, val in v.items():
+                        v[key] = [vi.convert(ci.datatype) for vi in val]
+                else:
+                    v = [vi.convert(ci.datatype) for vi in values[attr]]
+            elif ci.issection():
+                if values[attr] is not None:
+                    st = values[attr].getSectionDefinition()
+                    try:
+                        v = st.datatype(values[attr])
+                    except ValueError, e:
+                        raise ZConfig.DataConversionError(
+                            e, values[attr], (-1, -1, None))
+                else:
+                    v = None
+            elif name == '+':
+                v = values[attr]
+                if not v:
+                    for key, val in ci.getdefault().items():
+                        v[key] = val.convert(ci.datatype)
+                else:
+                    for key, val in v.items():
+                        v[key] = val.convert(ci.datatype)
+            else:
+                v = values[attr]
+                if v is not None:
+                    v = v.convert(ci.datatype)
+            values[attr] = v
+            if ci.handler is not None:
+                self.handlers.append((ci.handler, v))
+        return self.createValue()
+
+    def createValue(self):
+        return SectionValue(self._values, None, self)
+
+
+class SectionMatcher(BaseMatcher):
+    def __init__(self, info, type, name, handlers):
+        if name or info.allowUnnamed():
+            self.name = name
+        else:
+            raise ZConfig.ConfigurationError(
+                `type.name` + " sections may not be unnamed")
+        BaseMatcher.__init__(self, info, type, handlers)
+
+    def createValue(self):
+        return SectionValue(self._values, self.name, self)
+
+
+class SchemaMatcher(BaseMatcher):
+    def __init__(self, schema):
+        BaseMatcher.__init__(self, schema, schema, [])
+
+    def finish(self):
+        # Since there's no outer container to call datatype()
+        # for the schema, we convert on the way out.
+        v = BaseMatcher.finish(self)
+        v = self.type.datatype(v)
+        if self.type.handler is not None:
+            self.handlers.append((self.type.handler, v))
+        return v
+
+
+class SectionValue:
+    """Generic 'bag-of-values' object for a section.
+
+    Derived classes should always call the SectionValue constructor
+    before attempting to modify self.
+    """
+
+    def __init__(self, values, name, matcher):
+        self.__dict__.update(values)
+        self._name = name
+        self._matcher = matcher
+        self._attributes = tuple(values.keys())
+
+    def __repr__(self):
+        if self._name:
+            # probably unique for a given config file; more readable than id()
+            name = `self._name`
+        else:
+            # identify uniquely
+            name = "at %#x" % id(self)
+        clsname = self.__class__.__name__
+        return "<%s for %s %s>" % (clsname, self._matcher.type.name, name)
+
+    def __str__(self):
+        l = []
+        attrnames = [s for s in self.__dict__.keys() if s[0] != "_"]
+        attrnames.sort()
+        for k in attrnames:
+            v = getattr(self, k)
+            l.append('%-40s: %s' % (k, v))
+        return '\n'.join(l)
+
+    def getSectionName(self):
+        return self._name
+
+    def getSectionType(self):
+        return self._matcher.type.name
+
+    def getSectionDefinition(self):
+        return self._matcher.type
+
+    def getSectionMatcher(self):
+        return self._matcher
+
+    def getSectionAttributes(self):
+        return self._attributes

trunk/src/ZConfig/schema.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Parser for ZConfig schemas."""
+
+import os
+import xml.sax
+
+import ZConfig
+
+from ZConfig import info
+from ZConfig import url
+
+
+def parseResource(resource, loader):
+    parser = SchemaParser(loader, resource.url)
+    xml.sax.parse(resource.file, parser)
+    return parser._schema
+
+
+def parseComponent(resource, loader, schema):
+    parser = ComponentParser(loader, resource.url, schema)
+    xml.sax.parse(resource.file, parser)
+
+
+def _srepr(ob):
+    if isinstance(ob, type(u'')):
+        # drop the leading "u" from a unicode repr
+        return `ob`[1:]
+    else:
+        return `ob`
+
+
+class BaseParser(xml.sax.ContentHandler):
+
+    _cdata_tags = "description", "metadefault", "example", "default"
+    _handled_tags = ("import", "abstracttype", "sectiontype",
+                     "key", "multikey", "section", "multisection")
+
+    _allowed_parents = {
+        "description": ["key", "section", "multikey", "multisection",
+                        "sectiontype", "abstracttype",
+                        "schema", "component"],
+        "example": ["key", "section", "multikey", "multisection"],
+        "metadefault": ["key", "section", "multikey", "multisection"],
+        "default": ["key", "multikey"],
+        "import": ["schema", "component"],
+        "abstracttype": ["schema", "component"],
+        "sectiontype": ["schema", "component"],
+        "key": ["schema", "sectiontype"],
+        "multikey": ["schema", "sectiontype"],
+        "section": ["schema", "sectiontype"],
+        "multisection": ["schema", "sectiontype"],
+        }
+
+    def __init__(self, loader, url):
+        self._registry = loader.registry
+        self._loader = loader
+        self._basic_key = self._registry.get("basic-key")
+        self._identifier = self._registry.get("identifier")
+        self._cdata = None
+        self._locator = None
+        self._prefixes = []
+        self._schema = None
+        self._stack = []
+        self._url = url
+        self._elem_stack = []
+
+    # SAX 2 ContentHandler methods
+
+    def setDocumentLocator(self, locator):
+        self._locator = locator
+
+    def startElement(self, name, attrs):
+        attrs = dict(attrs)
+        if self._elem_stack:
+            parent = self._elem_stack[-1]
+            if not self._allowed_parents.has_key(name):
+                self.error("Unknown tag " + name)
+            if parent not in self._allowed_parents[name]:
+                self.error("%s elements may not be nested in %s elements"
+                           % (_srepr(name), _srepr(parent)))
+        elif name != self._top_level:
+            self.error("Unknown document type " + name)
+        self._elem_stack.append(name)
+        if name == self._top_level:
+            if self._schema is not None:
+                self.error("schema element improperly nested")
+            getattr(self, "start_" + name)(attrs)
+        elif name in self._handled_tags:
+            if self._schema is None:
+                self.error(name + " element outside of schema")
+            getattr(self, "start_" + name)(attrs)
+        elif name in self._cdata_tags:
+            if self._schema is None:
+                self.error(name + " element outside of schema")
+            if self._cdata is not None:
+                self.error(name + " element improperly nested")
+            self._cdata = []
+            self._position = None
+            self._attrs = attrs
+
+    def characters(self, data):
+        if self._cdata is not None:
+            if self._position is None:
+                self._position = self.get_position()
+            self._cdata.append(data)
+        elif data.strip():
+            self.error("unexpected non-blank character data: "
+                       + `data.strip()`)
+
+    def endElement(self, name):
+        del self._elem_stack[-1]
+        if name in self._handled_tags:
+            getattr(self, "end_" + name)()
+        else:
+            data = ''.join(self._cdata).strip()
+            self._cdata = None
+            getattr(self, "characters_" + name)(data)
+
+    def endDocument(self):
+        if self._schema is None:
+            self.error("no %s found" % self._top_level)
+
+    # helper methods
+
+    def get_position(self):
+        if self._locator:
+            return (self._locator.getLineNumber(),
+                    self._locator.getColumnNumber(),
+                    (self._locator.getSystemId() or self._url))
+        else:
+            return None, None, self._url
+
+    def get_handler(self, attrs):
+        v = attrs.get("handler")
+        if v is None:
+            return v
+        else:
+            return self.basic_key(v)
+
+    def push_prefix(self, attrs):
+        name = attrs.get("prefix")
+        if name:
+            if self._prefixes:
+                convert = self._registry.get("dotted-suffix")
+            else:
+                convert = self._registry.get("dotted-name")
+            try:
+                name = convert(name)
+            except ValueError, err:
+                self.error("not a valid prefix: %s (%s)"
+                           % (_srepr(name), str(err)))
+            if name[0] == ".":
+                prefix = self._prefixes[-1] + name
+            else:
+                prefix = name
+        elif self._prefixes:
+            prefix = self._prefixes[-1]
+        else:
+            prefix = ''
+        self._prefixes.append(prefix)
+
+    def pop_prefix(self):
+        del self._prefixes[-1]
+
+    def get_classname(self, name):
+        name = str(name)
+        if name.startswith("."):
+            return self._prefixes[-1] + name
+        else:
+            return name
+
+    def get_datatype(self, attrs, attrkey, default, base=None):
+        if attrs.has_key(attrkey):
+            dtname = self.get_classname(attrs[attrkey])
+        else:
+            convert = getattr(base, attrkey, None)
+            if convert is not None:
+                return convert
+            dtname = default
+
+        try:
+            return self._registry.get(dtname)
+        except ValueError, e:
+            self.error(e[0])
+
+    def get_sect_typeinfo(self, attrs, base=None):
+        keytype = self.get_datatype(attrs, "keytype", "basic-key", base)
+        valuetype = self.get_datatype(attrs, "valuetype", "string")
+        datatype = self.get_datatype(attrs, "datatype", "null", base)
+        return keytype, valuetype, datatype
+
+    def get_required(self, attrs):
+        if attrs.has_key("required"):
+            v = attrs["required"]
+            if v == "yes":
+                return True
+            elif v == "no":
+                return False
+            self.error("value for 'required' must be 'yes' or 'no'")
+        else:
+            return False
+
+    def get_ordinality(self, attrs):
+        min, max = 0, info.Unbounded
+        if self.get_required(attrs):
+            min = 1
+        return min, max
+
+    def get_sectiontype(self, attrs):
+        type = attrs.get("type")
+        if not type:
+            self.error("section must specify type")
+        return self._schema.gettype(type)
+
+    def get_key_info(self, attrs, element):
+        any, name, attribute = self.get_name_info(attrs, element)
+        if any == '*':
+            self.error(element + " may not specify '*' for name")
+        if not name and any != '+':
+            self.error(element + " name may not be omitted or empty")
+        datatype = self.get_datatype(attrs, "datatype", "string")
+        handler = self.get_handler(attrs)
+        return name or any, datatype, handler, attribute
+
+    def get_name_info(self, attrs, element):
+        name = attrs.get("name")
+        if not name:
+            self.error(element + " name must be specified and non-empty")
+        aname = attrs.get("attribute")
+        if aname:
+            aname = self.identifier(aname)
+            if aname.startswith("getSection"):
+                # reserved; used for SectionValue methods to get meta-info
+                self.error("attribute names may not start with 'getSection'")
+        if name in ("*", "+"):
+            if not aname:
+                self.error(
+                    "container attribute must be specified and non-empty"
+                    " when using '*' or '+' for a section name")
+            return name, None, aname
+        else:
+            # run the keytype converter to make sure this is a valid key
+            try:
+                name = self._stack[-1].keytype(name)
+            except ValueError, e:
+                self.error("could not convert key name to keytype: " + str(e))
+            if not aname:
+                aname = self.basic_key(name)
+                aname = self.identifier(aname.replace('-', '_'))
+            return None, name, aname
+
+    # schema loading logic
+
+    def characters_default(self, data):
+        key = self._attrs.get("key")
+        self._stack[-1].adddefault(data, self._position, key)
+
+    def characters_description(self, data):
+        if self._stack[-1].description is not None:
+            self.error(
+                "at most one <description> may be used for each element")
+        self._stack[-1].description = data
+
+    def characters_example(self, data):
+        self._stack[-1].example = data
+
+    def characters_metadefault(self, data):
+        self._stack[-1].metadefault = data
+
+    def start_import(self, attrs):
+        src = attrs.get("src", "").strip()
+        pkg = attrs.get("package", "").strip()
+        file = attrs.get("file", "").strip()
+        if not (src or pkg):
+            self.error("import must specify either src or package")
+        if src and pkg:
+            self.error("import may only specify one of src or package")
+        if src:
+            if file:
+                self.error("import may not specify file and src")
+            src = url.urljoin(self._url, src)
+            src, fragment = url.urldefrag(src)
+            if fragment:
+                self.error("import src many not include"
+                           " a fragment identifier")
+            schema = self._loader.loadURL(src)
+            for n in schema.gettypenames():
+                self._schema.addtype(schema.gettype(n))
+        else:
+            if os.path.dirname(file):
+                self.error("file may not include a directory part")
+            pkg = self.get_classname(pkg)
+            src = self._loader.schemaComponentSource(pkg, file)
+            if not self._schema.hasComponent(src):
+                self._schema.addComponent(src)
+                self.loadComponent(src)
+
+    def loadComponent(self, src):
+        r = self._loader.openResource(src)
+        parser = ComponentParser(self._loader, src, self._schema)
+        try:
+            xml.sax.parse(r.file, parser)
+        finally:
+            r.close()
+
+    def end_import(self):
+        pass
+
+    def start_sectiontype(self, attrs):
+        name = attrs.get("name")
+        if not name:
+            self.error("sectiontype name must not be omitted or empty")
+        name = self.basic_key(name)
+        self.push_prefix(attrs)
+        if attrs.has_key("extends"):
+            basename = self.basic_key(attrs["extends"])
+            base = self._schema.gettype(basename)
+            if base.isabstract():
+                self.error("sectiontype cannot extend an abstract type")
+            keytype, valuetype, datatype = self.get_sect_typeinfo(attrs, base)
+            sectinfo = self._schema.deriveSectionType(
+                base, name, keytype, valuetype, datatype)
+        else:
+            keytype, valuetype, datatype = self.get_sect_typeinfo(attrs)
+            sectinfo = self._schema.createSectionType(
+                name, keytype, valuetype, datatype)
+        if attrs.has_key("implements"):
+            ifname = self.basic_key(attrs["implements"])
+            interface = self._schema.gettype(ifname)
+            if not interface.isabstract():
+                self.error(
+                    "type specified by implements is not an abstracttype")
+            interface.addsubtype(sectinfo)
+        self._stack.append(sectinfo)
+
+    def end_sectiontype(self):
+        self.pop_prefix()
+        self._stack.pop()
+
+    def start_section(self, attrs):
+        sectiontype = self.get_sectiontype(attrs)
+        handler = self.get_handler(attrs)
+        min = self.get_required(attrs) and 1 or 0
+        any, name, attribute = self.get_name_info(attrs, "section")
+        if any and not attribute:
+            self.error(
+                "attribute must be specified if section name is '*' or '+'")
+        section = info.SectionInfo(any or name, sectiontype,
+                                   min, 1, handler, attribute)
+        self._stack[-1].addsection(name, section)
+        self._stack.append(section)
+
+    def end_section(self):
+        self._stack.pop()
+
+    def start_multisection(self, attrs):
+        sectiontype = self.get_sectiontype(attrs)
+        min, max = self.get_ordinality(attrs)
+        any, name, attribute = self.get_name_info(attrs, "multisection")
+        if any not in ("*", "+"):
+            self.error("multisection must specify '*' or '+' for the name")
+        handler = self.get_handler(attrs)
+        section = info.SectionInfo(any or name, sectiontype,
+                                   min, max, handler, attribute)
+        self._stack[-1].addsection(name, section)
+        self._stack.append(section)
+
+    def end_multisection(self):
+        self._stack.pop()
+
+    def start_abstracttype(self, attrs):
+        name = attrs.get("name")
+        if not name:
+            self.error("abstracttype name must not be omitted or empty")
+        name = self.basic_key(name)
+        abstype = info.AbstractType(name)
+        self._schema.addtype(abstype)
+        self._stack.append(abstype)
+
+    def end_abstracttype(self):
+        self._stack.pop()
+
+    def start_key(self, attrs):
+        name, datatype, handler, attribute = self.get_key_info(attrs, "key")
+        min = self.get_required(attrs) and 1 or 0
+        key = info.KeyInfo(name, datatype, min, handler, attribute)
+        if attrs.has_key("default"):
+            if min:
+                self.error("required key cannot have a default value")
+            key.adddefault(str(attrs["default"]).strip(),
+                           self.get_position())
+        if name != "+":
+            key.finish()
+        self._stack[-1].addkey(key)
+        self._stack.append(key)
+
+    def end_key(self):
+        key = self._stack.pop()
+        if key.name == "+":
+            key.computedefault(self._stack[-1].keytype)
+            key.finish()
+
+    def start_multikey(self, attrs):
+        if attrs.has_key("default"):
+            self.error("default values for multikey must be given using"
+                       " 'default' elements")
+        name, datatype, handler, attribute = self.get_key_info(attrs,
+                                                               "multikey")
+        min, max = self.get_ordinality(attrs)
+        key = info.MultiKeyInfo(name, datatype, min, max, handler, attribute)
+        self._stack[-1].addkey(key)
+        self._stack.append(key)
+
+    def end_multikey(self):
+        multikey = self._stack.pop()
+        if multikey.name == "+":
+            multikey.computedefault(self._stack[-1].keytype)
+        multikey.finish()
+
+    # datatype conversion wrappers
+
+    def basic_key(self, s):
+        try:
+            return self._basic_key(s)
+        except ValueError, e:
+            self.error(e[0])
+
+    def identifier(self, s):
+        try:
+            return self._identifier(s)
+        except ValueError, e:
+            self.error(e[0])
+
+    # exception setup helpers
+
+    def initerror(self, e):
+        if self._locator is not None:
+            e.colno = self._locator.getColumnNumber()
+            e.lineno = self._locator.getLineNumber()
+            e.url = self._locator.getSystemId()
+        return e
+
+    def error(self, message):
+        raise self.initerror(ZConfig.SchemaError(message))
+
+
+class SchemaParser(BaseParser):
+
+    # needed by startElement() and endElement()
+    _handled_tags = BaseParser._handled_tags + ("schema",)
+    _top_level = "schema"
+
+    def __init__(self, loader, url, extending_parser=None):
+        BaseParser.__init__(self, loader, url)
+        self._extending_parser = extending_parser
+        self._base_keytypes = []
+        self._base_datatypes = []
+
+    def start_schema(self, attrs):
+        self.push_prefix(attrs)
+        handler = self.get_handler(attrs)
+        keytype, valuetype, datatype = self.get_sect_typeinfo(attrs)
+
+        if self._extending_parser is None:
+            # We're not being inherited, so we need to create the schema
+            self._schema = info.SchemaType(keytype, valuetype, datatype,
+                                           handler, self._url, self._registry)
+        else:
+            # Parse into the extending ("subclass") parser's schema
+            self._schema = self._extending_parser._schema
+
+        self._stack = [self._schema]
+
+        if attrs.has_key("extends"):
+            sources = attrs["extends"].split()
+            sources.reverse()
+
+            for src in sources:
+                src = url.urljoin(self._url, src)
+                src, fragment = url.urldefrag(src)
+                if fragment:
+                    self.error("schema extends many not include"
+                               " a fragment identifier")
+                self.extendSchema(src)
+
+            # Inherit keytype from bases, if unspecified and not conflicting
+            if self._base_keytypes and not attrs.has_key("keytype"):
+                keytype = self._base_keytypes[0]
+                for kt in self._base_keytypes[1:]:
+                    if kt is not keytype:
+                        self.error("base schemas have conflicting keytypes,"
+                                   " but no keytype was specified in the"
+                                   " extending schema")
+
+            # Inherit datatype from bases, if unspecified and not conflicting
+            if self._base_datatypes and not attrs.has_key("datatype"):
+                datatype = self._base_datatypes[0]
+                for dt in self._base_datatypes[1:]:
+                    if dt is not datatype:
+                        self.error("base schemas have conflicting datatypes,"
+                                   " but no datatype was specified in the"
+                                   " extending schema")
+
+        # Reset the schema types to our own, while we parse the schema body
+        self._schema.keytype = keytype
+        self._schema.valuetype = valuetype
+        self._schema.datatype = datatype
+
+        # Update base key/datatypes for the "extending" parser
+        if self._extending_parser is not None:
+            self._extending_parser._base_keytypes.append(keytype)
+            self._extending_parser._base_datatypes.append(datatype)
+
+
+    def extendSchema(self,src):
+        parser = SchemaParser(self._loader, src, self)
+        r = self._loader.openResource(src)
+        try:
+            xml.sax.parse(r.file, parser)
+        finally:
+            r.close()
+
+    def end_schema(self):
+        del self._stack[-1]
+        assert not self._stack
+        self.pop_prefix()
+        assert not self._prefixes
+
+
+class ComponentParser(BaseParser):
+
+    _handled_tags = BaseParser._handled_tags + ("component",)
+    _top_level = "component"
+
+    def __init__(self, loader, url, schema):
+        BaseParser.__init__(self, loader, url)
+        self._parent = schema
+
+    def characters_description(self, data):
+        if self._stack:
+            self._stack[-1].description = data
+
+    def start_key(self, attrs):
+        self._check_not_toplevel("key")
+        BaseParser.start_key(self, attrs)
+
+    def start_multikey(self, attrs):
+        self._check_not_toplevel("multikey")
+        BaseParser.start_multikey(self, attrs)
+
+    def start_section(self, attrs):
+        self._check_not_toplevel("section")
+        BaseParser.start_section(self, attrs)
+
+    def start_multisection(self, attrs):
+        self._check_not_toplevel("multisection")
+        BaseParser.start_multisection(self, attrs)
+
+    def start_component(self, attrs):
+        self._schema = self._parent
+        self.push_prefix(attrs)
+
+    def end_component(self):
+        self.pop_prefix()
+
+    def _check_not_toplevel(self, what):
+        if not self._stack:
+            self.error("cannot define top-level %s in a schema %s"
+                       % (what, self._top_level))

trunk/src/ZConfig/scripts/zconfig

+#!/usr/bin/env python
+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""zconfig: Script to check validity of a configuration file.
+
+Usage:
+
+    zconfig [options] [file...]
+
+Options:
+
+    -h
+    --help          Print this help text.
+
+    -s file
+    --schema file   Use the schema in 'file' to validate the configuration;
+                    this must be specified.
+
+Each file named on the command line is checked for syntactical errors
+and schema conformance.  The schema must be specified.  If no files
+are specified and standard input is not a TTY, standard in is treated
+as a configuration file.  Specifying a schema and no configuration
+files causes the schema to be checked.
+
+"""
+
+import optparse
+import sys
+
+import ZConfig
+
+
+def main():
+    optparser = optparse.OptionParser(
+        usage="usage: %prog [-s FILE] [file ...]")
+    optparser.add_option(
+        "-s", "--schema", dest="schema",
+        help="use the schema in FILE (can be a URL)",
+        metavar="FILE")
+    options, args = optparser.parse_args()
+
+    if not options.schema:
+        print >>sys.stderr, "No schema specified, but is required."
+        usage(sys.stderr)
+        return 2
+    schema = ZConfig.loadSchema(options.schema)
+
+    if not args:
+        if sys.stdin.isatty():
+            # just checking the schema
+            return 0
+        else:
+            # stdin is a pipe
+            args = ["-"]
+
+    errors = 0
+    for fn in args:
+        try:
+            if fn == "-":
+                ZConfig.loadConfigFile(schema, sys.stdin)
+            else:
+                ZConfig.loadConfig(schema, fn)
+        except ZConfig.ConfigurationError, e:
+            print >>sys.stderr, str(e)
+            errors += 1
+
+    if errors:
+        return 1
+    else:
+        return 0
+
+
+def usage(fp):
+    print >>fp, __doc__
+
+
+if __name__ == "__main__":
+    sys.exit(main())

trunk/src/ZConfig/scripts/zconfig_schema2html

+#!/usr/bin/env python
+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+__version__ = '$Revision: 1.3 $'[11:-2]
+
+import ZConfig.loader
+from ZConfig.info import *
+import sys, cgi
+
+def esc(x): return cgi.escape(str(x))
+def dt(x):
+    tn = type(x).__name__
+    if tn == 'instance':
+        return '%s %s'%(tn, x.__class__.__module__ + '.' + x.__class__.__name__)
+    elif tn == 'class':
+        return '%s %s'%(tn, x.__module__ + '.' + x.__name__)
+    else:
+        return '%s %s'%(tn, x.__name__)
+
+class explain:
+    done = []
+    def __call__(self, st):
+        if st.name in self.done:
+            return
+        self.done.append(st.name)
+
+        if st.description:
+            print st.description
+        for sub in st.getsubtypenames():
+            print '<dl>'
+            printContents(None, st.getsubtype(sub))
+            print '</dl>'
+explain = explain()
+
+def printSchema(schema):
+    print '<dl>'
+    for child in schema:
+        printContents(*child)
+    print '</dl>'
+
+def printContents(name, info):
+    if isinstance(info, SectionType):
+        print '<dt><b><i>', info.name, '</i></b> (%s)</dt>'%dt(info.datatype)
+        print '<dd>'
+        if info.description:
+            print info.description
+        print '<dl>'
+        for sub in info:
+            printContents(*sub)
+        print '</dl></dd>'
+    elif isinstance(info, SectionInfo):
+        st = info.sectiontype
+        if st.isabstract():
+            print '<dt><b><i>', st.name, '</i>', info.name, '</b></dt>'
+            print '<dd>'
+            if info.description:
+                print info.description
+            explain(st)
+            print '</dd>'
+        else:
+            print '<dt><b>', info.attribute, info.name, '</b>'
+            print '(%s)</dt>'%dt(info.datatype)
+            print '<dd><dl>'
+            for sub in info.sectiontype:
+                printContents(*sub)
+            print '</dl></dd>'
+    else:
+        print '<dt><b>',info.name, '</b>', '(%s)'%dt(info.datatype)
+        default = info.getdefault()
+        if isinstance(default, ValueInfo):
+            print '(default: %r)'%esc(default.value)
+        elif default is not None:
+            print '(default: %r)'%esc(default)
+        if info.metadefault:
+            print '(metadefault: %s)' % info.metadefault
+        print '</dt>'
+        if info.description:
+            print '<dd>',info.description,'</dd>'
+
+schema = ZConfig.loader.loadSchemaFile(sys.argv[1])
+
+print '''<html><body>
+<style>
+dl {margin: 0 0 1em 0;}
+</style>
+'''
+printSchema(schema)
+print '</body></html>'
+
+# vim: set filetype=python ts=4 sw=4 et si
+

trunk/src/ZConfig/substitution.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Substitution support for ZConfig values."""
+
+import ZConfig
+
+
+def substitute(s, mapping):
+    """Interpolate variables from `mapping` into `s`."""
+    if "$" in s:
+        result = ''
+        rest = s
+        while rest:
+            p, name, namecase, rest = _split(rest)
+            result += p
+            if name:
+                v = mapping.get(name)
+                if v is None:
+                    raise ZConfig.SubstitutionReplacementError(s, namecase)
+                result += v
+        return result
+    else:
+        return s
+
+
+def isname(s):
+    """Return True iff s is a valid substitution name."""
+    m = _name_match(s)
+    if m:
+        return m.group() == s
+    else:
+        return False
+
+
+def _split(s):
+    # Return a four tuple:  prefix, name, namecase, suffix
+    # - prefix is text that can be used literally in the result (may be '')
+    # - name is a referenced name, or None
+    # - namecase is the name with case preserved
+    # - suffix is trailling text that may contain additional references
+    #   (may be '' or None)
+    if "$" in s:
+        i = s.find("$")
+        c = s[i+1:i+2]
+        if c == "":
+            raise ZConfig.SubstitutionSyntaxError(
+                "illegal lone '$' at end of source")
+        if c == "$":
+            return s[:i+1], None, None, s[i+2:]
+        prefix = s[:i]
+        if c == "{":
+            m = _name_match(s, i + 2)
+            if not m:
+                raise ZConfig.SubstitutionSyntaxError(
+                    "'${' not followed by name")
+            name = m.group(0)
+            i = m.end() + 1
+            if not s.startswith("}", i - 1):
+                raise ZConfig.SubstitutionSyntaxError(
+                    "'${%s' not followed by '}'" % name)
+        else:
+            m = _name_match(s, i+1)
+            if not m:
+                raise ZConfig.SubstitutionSyntaxError(
+                    "'$' not followed by '$' or name")
+            name = m.group(0)
+            i = m.end()
+        return prefix, name.lower(), name, s[i:]
+    else:
+        return s, None, None, None
+
+
+import re
+_name_match = re.compile(r"[a-zA-Z_][a-zA-Z0-9_]*").match
+del re

trunk/src/ZConfig/tests/__init__.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Tests for the configuration data structures and loader.
+
+$Id: __init__.py,v 1.2 2003/01/03 21:05:56 fdrake Exp $
+"""

trunk/src/ZConfig/tests/input/base-datatype1.xml

+<schema datatype="ZConfig.tests.test_schema.MySection">
+  <sectiontype name="type-1"/>
+</schema>
+

trunk/src/ZConfig/tests/input/base-datatype2.xml

+<schema datatype="ZConfig.tests.test_schema.appsection">
+  <sectiontype name="type-2"/>
+</schema>

trunk/src/ZConfig/tests/input/base-keytype1.xml

+<schema keytype="basic-key">
+  <sectiontype name="type-1"/>
+</schema>

trunk/src/ZConfig/tests/input/base-keytype2.xml

+<schema keytype="ZConfig.tests.test_schema.uppercase">
+  <sectiontype name="type-2"/>
+</schema>

trunk/src/ZConfig/tests/input/base.xml

+<schema>
+  <sectiontype name="type-X"/>
+</schema>

trunk/src/ZConfig/tests/input/include.conf

+var2 value2
+%include simple.conf
+var3 value3
+var4 $name

trunk/src/ZConfig/tests/input/inner.conf

+refouter $outervar
+%define innervar inner

trunk/src/ZConfig/tests/input/library.xml

+<schema>
+  <description>
+    Sample library of reusable data types.
+  </description>
+  <sectiontype name="type-a"/>
+  <sectiontype name="type-b"/>
+</schema>

trunk/src/ZConfig/tests/input/logger.xml

+<schema>
+  <sectiontype name="loghandler">
+    <key name="constructor" datatype="constructor" required="yes"/>
+    <key name="formatter" datatype="constructor"
+         default="logging.Formatter()"/>
+  </sectiontype>
+
+  <sectiontype name="logger">
+     <key name="level" datatype="integer" default="info"/>
+     <multisection type="loghandler" attribute="handlers" name="*"/>
+  </sectiontype>
+</schema>

trunk/src/ZConfig/tests/input/outer.conf

+%define outervar outer
+%include inner.conf
+refinner $innervar

trunk/src/ZConfig/tests/input/simple.conf

+empty
+
+var1 abc
+int-var  12
+float-var	12.02
+neg-int -2
+
+true-var-1 true
+true-var-2 on
+true-var-3 yes
+
+false-var-1 false
+false-var-2 off
+false-var-3 no
+
+list-1
+list-2 abc
+list-3 abc def ghi
+list-4 	[  what	now? 	]
+
+# These test the %define mechanism:
+
+%define dollars $$$$
+%define empty
+%define name value
+%define twowords two words
+
+getname $name
+getnametwice $name${name}
+getdollars $dollars
+getempty x${empty}y
+getwords abc $twowords def

trunk/src/ZConfig/tests/input/simple.xml

+<schema>
+  <key name="empty"        />
+  <key name="var1"         />
+  <key name="getname"      />
+  <key name="getnametwice" />
+  <key name="getdollars"   />
+  <key name="getempty"     />
+  <key name="getwords"     />
+
+  <key name="int-var"     datatype="integer" />
+  <key name="float-var"   datatype="float"   />
+  <key name="neg-int"     datatype="integer" />
+  <key name="true-var-1"  datatype="boolean" />
+  <key name="true-var-2"  datatype="boolean" />
+  <key name="true-var-3"  datatype="boolean" />
+  <key name="false-var-1" datatype="boolean" />
+  <key name="false-var-2" datatype="boolean" />
+  <key name="false-var-3" datatype="boolean" />
+
+  <key name="list-1" datatype="string-list" />
+  <key name="list-2" datatype="string-list" />
+  <key name="list-3" datatype="string-list" />
+  <key name="list-4" datatype="string-list" />
+
+  <!-- added by include.conf -->
+  <key name="var2" />
+  <key name="var3" />
+  <key name="var4" />
+</schema>

trunk/src/ZConfig/tests/input/simplesections.conf

+var foo
+var-0 foo-0
+
+<section name>
+  var bar
+  var-one  splat
+</section>
+
+var-1 foo-1
+
+<section delegate>
+  var spam
+  var-two  stuff
+</section>
+
+var-2 foo-2
+
+<section another>
+  var  quack!
+  var-three  yet
+</section>
+
+var-3 foo-3
+
+# An anonymous empty section:
+<section />
+
+var-4 foo-4
+
+# A fairly trivial section:
+<trivial>
+  var triv
+</trivial>
+
+var-5 foo-5
+
+# A minimal section:
+<minimal/>
+
+var-6 foo-6

trunk/src/ZConfig/tests/input/simplesections.xml

+<schema>
+  <sectiontype name="section">
+    <key name="var" />
+    <key name="var-one" />
+    <key name="var-two" />
+    <key name="var-three" />
+  </sectiontype>
+  <sectiontype name="minimal" />
+  <sectiontype name="trivial">
+    <key name="var" />
+  </sectiontype>
+
+  <multisection type="section" name="*" attribute="sections" />
+  <section type="minimal" name="*" attribute="minimal" />
+  <section type="trivial" name="*" attribute="trivial" />
+
+  <key name="var" />
+  <key name="var-0" />
+  <key name="var-1" />
+  <key name="var-2" />
+  <key name="var-3" />
+  <key name="var-4" />
+  <key name="var-5" />
+  <key name="var-6" />
+</schema>

trunk/src/ZConfig/tests/library/README.txt

+This is a sample library of configuration schema components.  This is
+used for testing.

trunk/src/ZConfig/tests/library/__init__.py

+# Make this a package.

trunk/src/ZConfig/tests/library/thing/__init__.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Example of a package that extends its __path__.
+
+$Id: __init__.py,v 1.2 2003/10/03 17:11:33 fdrake Exp $
+"""
+
+import os
+
+here = os.path.dirname(__file__)
+__path__.append(os.path.join(here, "extras"))

trunk/src/ZConfig/tests/library/thing/component.xml

+<?xml version="1.0" encoding="us-ascii"?>
+<component>
+  <abstracttype name='thing'/>
+  <sectiontype name='thing-a' implements='thing'>
+    <key name='thing-a-key' default='thing-a-default'/>
+  </sectiontype>
+  <sectiontype name='thing-b' implements='thing'>
+    <key name='thing-b-key' default='thing-b-default'/>
+  </sectiontype>
+</component>

trunk/src/ZConfig/tests/library/thing/extras/extras.xml

+<component>
+  <sectiontype name='extra-thing'>
+    <key name='some-key'/>
+  </sectiontype>
+</component>

trunk/src/ZConfig/tests/library/widget/__init__.py

+# Make this a package.

trunk/src/ZConfig/tests/library/widget/component.xml

+<component>
+  <abstracttype name='widget'/>
+  <sectiontype name='widget-a' implements='widget'>
+    <key name='widget-a-key' default='widget-a-default'/>
+  </sectiontype>
+  <sectiontype name='widget-b' implements='widget' extends='widget-a'/>
+</component>

trunk/src/ZConfig/tests/library/widget/extra.xml

+<component>
+  <sectiontype name='extra-type'>
+    <key name='some-key'/>
+  </sectiontype>
+</component>

trunk/src/ZConfig/tests/runtests.bat

+@echo off
+
+rem  Simple script to run the tests on Windows.
+
+rem  The paths to different versions of Python need to be
+rem  edited for the system this is being run on; comment
+rem  out lines that aren't needed or wanted.
+
+\Python213\python runtests.py
+\Python221\python runtests.py
+\Python222\python runtests.py
+\Python230\python runtests.py

trunk/src/ZConfig/tests/runtests.py

+#! /usr/bin/env python
+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Script to run all the regression tests for the ZConfig package."""
+
+import os
+import sys
+import unittest
+
+if __name__ == "__main__":
+    __file__ = sys.argv[0]
+
+TESTDIR = os.path.dirname(os.path.abspath(__file__))
+
+PKGDIR = os.path.dirname(TESTDIR) # the ZConfig package directory
+TOPDIR = os.path.dirname(PKGDIR)
+
+COMPONENTS = os.path.join(PKGDIR, "components")
+
+TESTDIRS = {
+    "ZConfig.tests": TESTDIR,
+    "ZConfig.components.basic.tests": os.path.join(COMPONENTS,
+                                                   "basic", "tests"),
+    "ZConfig.components.logger.tests": os.path.join(COMPONENTS,
+                                                    "logger", "tests"),
+    }
+
+
+def load_tests(pkgname, name):
+    name = "%s.%s" % (pkgname, name)
+    __import__(name)
+    mod = sys.modules[name]
+    return mod.test_suite()
+
+def test_suite():
+    L = []
+    for pkgname, path in TESTDIRS.items():
+        for fn in os.listdir(path):
+            name, ext = os.path.splitext(fn)
+            if name[:4] == "test" and ext == ".py":
+                L.append(load_tests(pkgname, name))
+    if len(L) == 1:
+        return L[0]
+    else:
+        suite = unittest.TestSuite()
+        for t in L:
+            suite.addTest(t)
+        return suite
+
+if __name__ == "__main__":
+    if TOPDIR not in sys.path:
+        sys.path.insert(0, TOPDIR)
+    unittest.main(defaultTest="test_suite")

trunk/src/ZConfig/tests/support.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""Support code shared among the tests."""
+
+import os
+import StringIO
+import unittest
+import urllib
+
+import ZConfig
+
+from ZConfig.loader import ConfigLoader
+from ZConfig.url import urljoin
+
+
+try:
+    __file__
+except NameError:
+    import sys
+    __file__ = sys.argv[0]
+
+d = os.path.abspath(os.path.join(os.path.dirname(__file__), "input"))
+CONFIG_BASE = "file://%s/" % urllib.pathname2url(d)
+
+
+class TestBase(unittest.TestCase):
+    """Utility methods which can be used with the schema support."""
+
+    def load_both(self, schema_url, conf_url):
+        schema = self.load_schema(schema_url)
+        conf = self.load_config(schema, conf_url)
+        return schema, conf
+
+    def load_schema(self, relurl):
+        self.url = urljoin(CONFIG_BASE, relurl)
+        self.schema = ZConfig.loadSchema(self.url)
+        self.assert_(self.schema.issection())
+        return self.schema
+
+    def load_schema_text(self, text, url=None):
+        sio = StringIO.StringIO(text)
+        self.schema = ZConfig.loadSchemaFile(sio, url)
+        return self.schema
+
+    def load_config(self, schema, conf_url, num_handlers=0):
+        conf_url = urljoin(CONFIG_BASE, conf_url)
+        loader = self.create_config_loader(schema)
+        self.conf, self.handlers = loader.loadURL(conf_url)
+        self.assertEqual(len(self.handlers), num_handlers)
+        return self.conf
+
+    def load_config_text(self, schema, text, num_handlers=0, url=None):
+        sio = StringIO.StringIO(text)
+        loader = self.create_config_loader(schema)
+        self.conf, self.handlers = loader.loadFile(sio, url)
+        self.assertEqual(len(self.handlers), num_handlers)
+        return self.conf
+
+    def create_config_loader(self, schema):
+        return ConfigLoader(schema)

trunk/src/ZConfig/tests/test_cfgimports.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Tests of the %import mechanism.
+
+$Id: test_cfgimports.py,v 1.1 2003/10/03 20:01:57 fdrake Exp $
+"""
+
+import unittest
+
+from StringIO import StringIO
+
+import ZConfig
+import ZConfig.tests.support
+
+
+class TestImportFromConfiguration(ZConfig.tests.support.TestBase):
+
+    def test_simple_import(self):
+        schema = self.load_schema_text("<schema/>")
+        loader = self.create_config_loader(schema)
+        config, _ = loader.loadFile(
+            StringIO("%import ZConfig.tests.library.widget\n"))
+        # make sure we now have a "private" schema object; the only
+        # way to get it is from the loader itself
+        self.assert_(schema is not loader.schema)
+        # make sure component types are only found on the private schema:
+        loader.schema.gettype("widget-b")
+        self.assertRaises(ZConfig.SchemaError, schema.gettype, "widget-b")
+
+    def test_repeated_import(self):
+        schema = self.load_schema_text("<schema/>")
+        loader = self.create_config_loader(schema)
+        config, _ = loader.loadFile(
+            StringIO("%import ZConfig.tests.library.widget\n"
+                     "%import ZConfig.tests.library.widget\n"))
+
+    def test_missing_import(self):
+        schema = self.load_schema_text("<schema/>")
+        loader = self.create_config_loader(schema)
+        self.assertRaises(ZConfig.SchemaError, loader.loadFile,
+                          StringIO("%import ZConfig.tests.missing\n"))
+
+
+def test_suite():
+    return unittest.makeSuite(TestImportFromConfiguration)

trunk/src/ZConfig/tests/test_cmdline.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""Tests of the command-line integration."""
+
+import unittest
+
+import ZConfig
+
+from ZConfig.cmdline import ExtendedConfigLoader
+from ZConfig.tests.support import TestBase
+
+
+class CommandLineTest(TestBase):
+
+    def create_config_loader(self, schema):
+        loader = ExtendedConfigLoader(schema)
+        for item in self.clopts:
+            loader.addOption(*item)
+        return loader
+
+    def test_loading(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='st'>
+                <key name='innerkey'/>
+              </sectiontype>
+              <key name='mykey'/>
+              <section name='*' type='st' attribute='sect'/>
+            </schema>
+            """)
+        self.clopts = [("mykey=splat!", None),
+                       ("section/innerkey=spoogey", None)]
+        bag = self.create_config_loader(schema).cook()
+        # Test a variety of queries on the OptionBag:
+        self.assert_(bag.has_key("mykey"))
+        self.assert_(not bag.has_key("another"))
+        self.assertEqual(bag.get_section_info("st", None), None)
+        self.assertEqual(bag.get_section_info("st", "missing-sect"), None)
+        # Consume everything in the OptionBag:
+        L = bag.get_key("mykey")
+        s, pos = L[0]
+        self.assertEqual(len(L), 1)
+        self.assertEqual(s, "splat!")
+        bag2 = bag.get_section_info("st", "section")
+        self.assert_(bag2.has_key("innerkey"))
+        self.assert_(not bag2.has_key("another"))
+        L = bag2.get_key("innerkey")
+        s, pos = L[0]
+        self.assertEqual(len(L), 1)
+        self.assertEqual(s, "spoogey")
+        # "Finish" to make sure everything has been consumed:
+        bag2.finish()
+        bag.finish()
+
+    def test_named_sections(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <abstracttype name='at'/>
+              <sectiontype name='st1' implements='at'>
+                <key name='k1'/>
+              </sectiontype>
+              <sectiontype name='st2' implements='at'>
+                <key name='k2'/>
+              </sectiontype>
+              <section name='foo' type='at'/>
+              <section name='bar' type='st2'/>
+            </schema>
+            """)
+        self.clopts = [("foo/k1=v1", None), ("bar/k2=v2", ("someurl", 2, 3))]
+        bag = self.create_config_loader(schema).cook()
+        foo = bag.get_section_info("st2", "foo")
+        bar = bag.get_section_info("st2", "bar")
+        bag.finish()
+        self.assertEqual(bar.get_key("k2"), [("v2", ("someurl", 2, 3))])
+        bar.finish()
+        # Ignore foo for now; it's not really important *when* it fails.
+
+    simple_schema = None
+
+    def get_simple_schema(self):
+        if self.simple_schema is None:
+            self.__class__.simple_schema = self.load_schema_text("""\
+                <schema>
+                  <key name='k0'/>
+                  <key name='k1'/>
+                  <key name='k2' datatype='integer'/>
+                  <key name='k3' datatype='integer' default='19'/>
+                </schema>
+                """)
+        return self.simple_schema
+
+    def test_reading_config(self):
+        self.clopts = [("k1=stringvalue", None), ("k2=12", None)]
+        schema = self.get_simple_schema()
+        conf = self.load_config_text(schema, """\
+            k0 stuff
+            k1 replaced-stuff
+            k2 42
+            """)
+        self.assertEqual(conf.k0, "stuff")
+        self.assertEqual(conf.k1, "stringvalue")
+        self.assertEqual(conf.k2, 12)
+        self.assertEqual(conf.k3, 19)
+
+    def test_unknown_key(self):
+        self.clopts = [("foo=bar", None)]
+        schema = self.get_simple_schema()
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "")
+
+    def test_too_many_keys(self):
+        self.clopts = [("k1=v1", None), ("k1=v2", None)]
+        schema = self.get_simple_schema()
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "")
+
+    def test_bad_datatype(self):
+        self.clopts = [("k2=42.0", None)]
+        schema = self.get_simple_schema()
+        self.assertRaises(ZConfig.DataConversionError,
+                          self.load_config_text, schema, "")
+
+    def test_without_clopts(self):
+        self.clopts = []
+        schema = self.get_simple_schema()
+        conf = self.load_config_text(schema, "k3 42")
+        self.assertEqual(conf.k0, None)
+        self.assertEqual(conf.k1, None)
+        self.assertEqual(conf.k2, None)
+        self.assertEqual(conf.k3, 42)
+
+    def test_section_contents(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='st'>
+                <key name='k1'/>
+                <key name='k2' default='3' datatype='integer'/>
+                <multikey name='k3'>
+                  <default>k3-v1</default>
+                  <default>k3-v2</default>
+                  <default>k3-v3</default>
+                </multikey>
+              </sectiontype>
+              <section name='s1' type='st'/>
+              <section name='s2' type='st'/>
+            </schema>
+            """)
+        self.clopts = [("s1/k1=foo", None),
+                       ("s2/k3=value1", None),
+                       ("s2/k3=value2", None),
+                       ("s1/k2=99", None),
+                       ("s2/k3=value3", None),
+                       ("s2/k3=value4", None),
+                       ]
+        conf = self.load_config_text(schema, "<st s1/>\n<st s2/>")
+        self.assertEqual(conf.s1.k1, "foo")
+        self.assertEqual(conf.s1.k2, 99)
+        self.assertEqual(conf.s1.k3, ["k3-v1", "k3-v2", "k3-v3"])
+        self.assertEqual(conf.s2.k1, None)
+        self.assertEqual(conf.s2.k2, 3)
+        self.assertEqual(conf.s2.k3, ["value1", "value2", "value3", "value4"])
+
+
+def test_suite():
+    return unittest.makeSuite(CommandLineTest)
+
+if __name__ == "__main__":
+    unittest.main(defaultTest="test_suite")

trunk/src/ZConfig/tests/test_config.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Tests of the configuration data structures and loader."""
+
+import os
+import StringIO
+import tempfile
+import unittest
+
+import ZConfig
+
+from ZConfig.tests.support import CONFIG_BASE
+
+
+class ConfigurationTestCase(unittest.TestCase):
+
+    schema = None
+
+    def get_schema(self):
+        if self.schema is None:
+            ConfigurationTestCase.schema = ZConfig.loadSchema(
+                CONFIG_BASE + "simple.xml")
+        return self.schema
+
+    def load(self, relurl, context=None):
+        url = CONFIG_BASE + relurl
+        self.conf, self.handlers = ZConfig.loadConfig(self.get_schema(), url)
+        conf = self.conf
+        #self.assertEqual(conf.url, url)
+        self.assert_(conf.getSectionName() is None)
+        self.assert_(conf.getSectionType() is None)
+        #self.assert_(conf.delegate is None)
+        return conf
+
+    def loadtext(self, text):
+        sio = StringIO.StringIO(text)
+        return self.loadfile(sio)
+
+    def loadfile(self, file):
+        schema = self.get_schema()
+        self.conf, self.handlers = ZConfig.loadConfigFile(schema, file)
+        return self.conf
+
+    def check_simple_gets(self, conf):
+        self.assertEqual(conf.empty, '')
+        self.assertEqual(conf.int_var, 12)
+        self.assertEqual(conf.neg_int, -2)
+        self.assertEqual(conf.float_var, 12.02)
+        self.assertEqual(conf.var1, 'abc')
+        self.assert_(conf.true_var_1)
+        self.assert_(conf.true_var_2)
+        self.assert_(conf.true_var_3)
+        self.assert_(not conf.false_var_1)
+        self.assert_(not conf.false_var_2)
+        self.assert_(not conf.false_var_3)
+        self.assertEqual(conf.list_1, [])
+        self.assertEqual(conf.list_2, ['abc'])
+        self.assertEqual(conf.list_3, ['abc', 'def', 'ghi'])
+        self.assertEqual(conf.list_4, ['[', 'what', 'now?', ']'])
+
+    def test_simple_gets(self):
+        conf = self.load("simple.conf")
+        self.check_simple_gets(conf)
+
+    def test_type_errors(self):
+        Error = ZConfig.DataConversionError
+        raises = self.assertRaises
+        raises(Error, self.loadtext, "int-var true")
+        raises(Error, self.loadtext, "float-var true")
+        raises(Error, self.loadtext, "neg-int false")
+        raises(Error, self.loadtext, "true-var-1 0")
+        raises(Error, self.loadtext, "true-var-1 1")
+        raises(Error, self.loadtext, "true-var-1 -1")
+
+    def test_simple_sections(self):
+        self.schema = ZConfig.loadSchema(CONFIG_BASE + "simplesections.xml")
+        conf = self.load("simplesections.conf")
+        self.assertEqual(conf.var, "foo")
+        # check each interleaved position between sections
+        for c in "0123456":
+            self.assertEqual(getattr(conf, "var_" +c), "foo-" + c)
+        sect = [sect for sect in conf.sections
+                if sect.getSectionName() == "name"][0]
+        self.assertEqual(sect.var, "bar")
+        self.assertEqual(sect.var_one, "splat")
+        self.assert_(sect.var_three is None)
+        sect = [sect for sect in conf.sections
+                if sect.getSectionName() == "delegate"][0]
+        self.assertEqual(sect.var, "spam")
+        self.assertEqual(sect.var_two, "stuff")
+        self.assert_(sect.var_three is None)
+
+    def test_include(self):
+        conf = self.load("include.conf")
+        self.assertEqual(conf.var1, "abc")
+        self.assertEqual(conf.var2, "value2")
+        self.assertEqual(conf.var3, "value3")
+        self.assertEqual(conf.var4, "value")
+
+    def test_includes_with_defines(self):
+        self.schema = ZConfig.loadSchemaFile(StringIO.StringIO("""\
+            <schema>
+              <key name='refinner' />
+              <key name='refouter' />
+            </schema>
+            """))
+        conf = self.load("outer.conf")
+        self.assertEqual(conf.refinner, "inner")
+        self.assertEqual(conf.refouter, "outer")
+
+    def test_define(self):
+        conf = self.load("simple.conf")
+        self.assertEqual(conf.getname, "value")
+        self.assertEqual(conf.getnametwice, "valuevalue")
+        self.assertEqual(conf.getdollars, "$$")
+        self.assertEqual(conf.getempty, "xy")
+        self.assertEqual(conf.getwords, "abc two words def")
+
+    def test_define_errors(self):
+        self.assertRaises(ZConfig.ConfigurationSyntaxError,
+                          self.loadtext, "%define\n")
+        self.assertRaises(ZConfig.ConfigurationSyntaxError,
+                          self.loadtext, "%define abc-def\n")
+        self.assertRaises(ZConfig.ConfigurationSyntaxError,
+                          self.loadtext, "%define a value\n%define a value\n")
+
+    def test_fragment_ident_disallowed(self):
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load, "simplesections.conf#another")
+
+    def test_load_from_fileobj(self):
+        sio = StringIO.StringIO("%define name value\n"
+                                "getname x $name y \n")
+        cf = self.loadfile(sio)
+        self.assertEqual(cf.getname, "x value y")
+
+    def test_load_from_abspath(self):
+        fn = self.write_tempfile()
+        try:
+            self.check_load_from_path(fn)
+        finally:
+            os.unlink(fn)
+
+    def test_load_from_relpath(self):
+        fn = self.write_tempfile()
+        dir, name = os.path.split(fn)
+        pwd = os.getcwd()
+        try:
+            os.chdir(dir)
+            self.check_load_from_path(name)
+        finally:
+            os.chdir(pwd)
+            os.unlink(fn)
+
+    def write_tempfile(self):
+        fn = tempfile.mktemp()
+        fp = open(fn, "w")
+        fp.write("var1 value\n")
+        fp.close()
+        return fn
+
+    def check_load_from_path(self, path):
+        schema = self.get_schema()
+        ZConfig.loadConfig(schema, path)
+
+
+def test_suite():
+    return unittest.makeSuite(ConfigurationTestCase)
+
+if __name__ == '__main__':
+    unittest.main(defaultTest='test_suite')

trunk/src/ZConfig/tests/test_cookbook.py

+##############################################################################
+#
+# Copyright (c) 2004 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Tests of examples from the online cookbook, so we don't break them
+down the road.  Unless we really mean to.
+
+The ZConfig Cookbook is available online at:
+
+    http://dev.zope.org/Zope3/ZConfig
+
+"""
+
+import unittest
+
+from ZConfig.tests.support import TestBase
+
+
+def basic_key_mapping_password_to_passwd(key):
+    # Lower-case the key since that's what basic-key does:
+    key = key.lower()
+    # Now map password to passwd:
+    if key == "password":
+        key = "passwd"
+    return key
+
+def user_info_conversion(section):
+    return section
+
+
+class CookbookTestCase(TestBase):
+
+    def test_rewriting_key_names(self):
+        schema = self.load_schema_text("""
+            <schema prefix='%s'>
+              <sectiontype name='userinfo' datatype='.user_info_conversion'
+                           keytype='.basic_key_mapping_password_to_passwd'>
+                <key name='userid' datatype='integer'/>
+                <key name='username' datatype='identifier'/>
+                <key name='password'/>
+              </sectiontype>
+              <section type='userinfo' name='*' attribute='userinfo'/>
+            </schema>
+            """ % __name__)
+        config = self.load_config_text(schema, """\
+            <userinfo>
+              USERID 42
+              USERNAME foouser
+              PASSWORD yeah-right
+            </userinfo>
+            """)
+        self.assertEqual(config.userinfo.userid, 42)
+        self.assertEqual(config.userinfo.username, "foouser")
+        self.assertEqual(config.userinfo.passwd, "yeah-right")
+        self.assert_(not hasattr(config.userinfo, "password"))
+
+
+def test_suite():
+    return unittest.makeSuite(CookbookTestCase)
+
+if __name__ == "__main__":
+    unittest.main(defaultTest="test_suite")

trunk/src/ZConfig/tests/test_datatypes.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Tests of standard ZConfig datatypes."""
+
+import os
+import sys
+import shutil
+import socket
+import datetime
+import tempfile
+import unittest
+
+import ZConfig.datatypes
+
+try:
+    here = __file__
+except NameError:
+    here = sys.argv[0]
+
+here = os.path.abspath(here)
+
+try:
+    unicode
+except NameError:
+    have_unicode = False
+else:
+    have_unicode = True
+
+
+class DatatypeTestCase(unittest.TestCase):
+    types = ZConfig.datatypes.Registry()
+
+    def test_datatype_basickey(self):
+        convert = self.types.get("basic-key")
+        eq = self.assertEqual
+        raises = self.assertRaises
+
+        eq(convert("abc"), "abc")
+        eq(convert("ABC_DEF.123"), "abc_def.123")
+        eq(convert("Abc-Def-456"), "abc-def-456")
+        eq(convert("Abc.Def"), "abc.def")
+
+        raises(ValueError, convert, "_abc")
+        raises(ValueError, convert, "-abc")
+        raises(ValueError, convert, "123")
+        raises(ValueError, convert, "")
+
+    def test_datatype_boolean(self):
+        convert = self.types.get("boolean")
+        check = self.assert_
+        raises = self.assertRaises
+
+        check(convert("on"))
+        check(convert("true"))
+        check(convert("yes"))
+        check(not convert("off"))
+        check(not convert("false"))
+        check(not convert("no"))
+        raises(ValueError, convert, '0')
+        raises(ValueError, convert, '1')
+        raises(ValueError, convert, '')
+        raises(ValueError, convert, 'junk')
+
+    def test_datatype_float(self):
+        convert = self.types.get("float")
+        eq = self.assertEqual
+        raises = self.assertRaises
+
+        eq(convert("1"), 1.0)
+        self.assert_(type(convert(1)) is type(1.0))
+        eq(convert("1.1"), 1.1)
+        eq(convert("50.50"), 50.50)
+        eq(convert("-50.50"), -50.50)
+        eq(convert(0), 0.0)
+        eq(convert("0"), 0.0)
+        eq(convert("-0"), 0.0)
+        eq(convert("0.0"), 0.0)
+
+        raises(ValueError, convert, "junk")
+        raises(ValueError, convert, "0x234.1.9")
+        raises(ValueError, convert, "0.9-")
+
+        # These are not portable representations; make sure they are
+        # disallowed everywhere for consistency.
+        raises(ValueError, convert, "inf")
+        raises(ValueError, convert, "-inf")
+        raises(ValueError, convert, "nan")
+
+        if have_unicode:
+            raises(ValueError, convert, unicode("inf"))
+            raises(ValueError, convert, unicode("-inf"))
+            raises(ValueError, convert, unicode("nan"))
+
+    def test_datatype_identifier(self):
+        convert = self.types.get("identifier")
+        raises = self.assertRaises
+        self.check_names(convert)
+        self.check_never_namelike(convert)
+        raises(ValueError, convert, ".abc")
+
+    def check_names(self, convert):
+        eq = self.assert_ascii_equal
+        eq(convert, "AbcDef")
+        eq(convert, "a________")
+        eq(convert, "abc_def")
+        eq(convert, "int123")
+        eq(convert, "_abc")
+        eq(convert, "_123")
+        eq(convert, "__dict__")
+
+    def assert_ascii_equal(self, convert, value):
+        v = convert(value)
+        self.assertEqual(v, value)
+        self.assert_(isinstance(v, str))
+        if have_unicode:
+            unicode_value = unicode(value)
+            v = convert(unicode_value)
+            self.assertEqual(v, value)
+            self.assert_(isinstance(v, str))
+
+    def check_never_namelike(self, convert):
+        raises = self.assertRaises
+        raises(ValueError, convert, "2345")
+        raises(ValueError, convert, "23.45")
+        raises(ValueError, convert, ".45")
+        raises(ValueError, convert, "23.")
+        raises(ValueError, convert, "abc.")
+        raises(ValueError, convert, "-abc")
+        raises(ValueError, convert, "-123")
+        raises(ValueError, convert, "abc-")
+        raises(ValueError, convert, "123-")
+        raises(ValueError, convert, "-")
+        raises(ValueError, convert, ".")
+        raises(ValueError, convert, "&%$*()")
+        raises(ValueError, convert, "")
+
+    def test_datatype_dotted_name(self):
+        convert = self.types.get("dotted-name")
+        raises = self.assertRaises
+        self.check_names(convert)
+        self.check_dotted_names(convert)
+        self.check_never_namelike(convert)
+        raises(ValueError, convert, "abc.")
+        raises(ValueError, convert, ".abc.")
+        raises(ValueError, convert, "abc.def.")
+        raises(ValueError, convert, ".abc.def.")
+        raises(ValueError, convert, ".abc.def")
+
+    def test_datatype_dotted_suffix(self):
+        convert = self.types.get("dotted-suffix")
+        eq = self.assert_ascii_equal
+        raises = self.assertRaises
+        self.check_names(convert)
+        self.check_dotted_names(convert)
+        self.check_never_namelike(convert)
+        eq(convert, ".a")
+        eq(convert, ".a.b")
+        eq(convert, ".a.b.c.d.e.f.g.h.i.j.k.l.m.n.o")
+        raises(ValueError, convert, "abc.")
+        raises(ValueError, convert, ".abc.")
+        raises(ValueError, convert, "abc.def.")
+        raises(ValueError, convert, ".abc.def.")
+
+    def check_dotted_names(self, convert):
+        eq = self.assert_ascii_equal
+        eq(convert, "abc.def")
+        eq(convert, "abc.def.ghi")
+        eq(convert, "a.d.g.g.g.g.g.g.g")
+
+    def test_datatype_inet_address(self):
+        convert = self.types.get("inet-address")
+        eq = self.assertEqual
+        defhost = ZConfig.datatypes.DEFAULT_HOST
+        eq(convert("Host.Example.Com:80"), ("host.example.com", 80))
+        eq(convert(":80"),                 (defhost, 80))
+        eq(convert("80"),                  (defhost, 80))
+        eq(convert("host.EXAMPLE.com"),    ("host.example.com", None))
+        self.assertRaises(ValueError, convert, "40 # foo")
+
+    def test_datatype_integer(self):
+        convert = self.types.get("integer")
+        eq = self.assertEqual
+        raises = self.assertRaises
+
+        eq(convert('-100'), -100)
+        eq(convert('-1'), -1)
+        eq(convert('-0'), 0)
+        eq(convert('0'), 0)
+        eq(convert('1'), 1)
+        eq(convert('100'), 100)
+        eq(convert('65535'), 65535)
+        eq(convert('65536'), 65536)
+
+        big = sys.maxint + 1L  # Python 2.1 needs the L suffix here
+        s = str(big)           # s won't have the suffix
+        eq(convert(s), big)
+        eq(convert("-" + s), -big)
+
+        raises(ValueError, convert, 'abc')
+        raises(ValueError, convert, '-0xabc')
+        raises(ValueError, convert, '')
+        raises(ValueError, convert, '123 456')
+        raises(ValueError, convert, '123-')
+
+    def test_datatype_locale(self):
+        convert = self.types.get("locale")
+        # Python supports "C" even when the _locale module is not available
+        self.assertEqual(convert("C"), "C")
+        self.assertRaises(ValueError, convert, "locale-does-not-exist")
+
+    def test_datatype_port(self):
+        convert = self.types.get("port-number")
+        eq = self.assertEqual
+        raises = self.assertRaises
+
+        raises(ValueError, convert, '-1')
+        raises(ValueError, convert, '0')
+        eq(convert('1'), 1)
+        eq(convert('80'), 80)
+        eq(convert('1023'), 1023)
+        eq(convert('1024'), 1024)
+        eq(convert('60000'), 60000)
+        eq(convert('65535'), 0xffff)
+        raises(ValueError, convert, '65536')
+
+    def test_datatype_socket_address(self):
+        convert = self.types.get("socket-address")
+        eq = self.assertEqual
+        AF_INET = socket.AF_INET
+        defhost = ZConfig.datatypes.DEFAULT_HOST
+
+        def check(value, family, address, self=self, convert=convert):
+            a = convert(value)
+            self.assertEqual(a.family, family)
+            self.assertEqual(a.address, address)
+
+        check("Host.Example.Com:80", AF_INET, ("host.example.com", 80))
+        check(":80",                 AF_INET, (defhost, 80))
+        check("80",                  AF_INET, (defhost, 80))
+        check("host.EXAMPLE.com",    AF_INET, ("host.example.com",None))
+        a1 = convert("/tmp/var/@345.4")
+        a2 = convert("/tmp/var/@345.4:80")
+        self.assertEqual(a1.address, "/tmp/var/@345.4")
+        self.assertEqual(a2.address, "/tmp/var/@345.4:80")
+        if hasattr(socket, "AF_UNIX"):
+            self.assertEqual(a1.family, socket.AF_UNIX)
+            self.assertEqual(a2.family, socket.AF_UNIX)
+        else:
+            self.assert_(a1.family is None)
+            self.assert_(a2.family is None)
+
+    def test_ipaddr_or_hostname(self):
+        convert = self.types.get('ipaddr-or-hostname')
+        eq = self.assertEqual
+        raises = self.assertRaises
+        eq(convert('hostname'),          'hostname')
+        eq(convert('hostname.com'),      'hostname.com')
+        eq(convert('www.hostname.com'),  'www.hostname.com')
+        eq(convert('HOSTNAME'),          'hostname')
+        eq(convert('HOSTNAME.COM'),      'hostname.com')
+        eq(convert('WWW.HOSTNAME.COM'),  'www.hostname.com')
+        eq(convert('127.0.0.1'),         '127.0.0.1')
+        raises(ValueError, convert,  '1hostnamewithleadingnumeric')
+        raises(ValueError, convert,  '255.255')
+        raises(ValueError, convert,  '12345678')
+        raises(ValueError, convert,  '999.999.999.999')
+        raises(ValueError, convert,  'a!badhostname')
+
+    def test_existing_directory(self):
+        convert = self.types.get('existing-directory')
+        eq = self.assertEqual
+        raises = self.assertRaises
+        eq(convert('.'), '.')
+        eq(convert(os.path.dirname(here)), os.path.dirname(here))
+        raises(ValueError, convert, tempfile.mktemp())
+
+    def test_existing_file(self):
+        convert = self.types.get('existing-file')
+        eq = self.assertEqual
+        raises = self.assertRaises
+        eq(convert('.'), '.')
+        eq(convert(here), here)
+        raises(ValueError, convert, tempfile.mktemp())
+
+    def test_existing_path(self):
+        convert = self.types.get('existing-path')
+        eq = self.assertEqual
+        raises = self.assertRaises
+        eq(convert('.'), '.')
+        eq(convert(here), here)
+        eq(convert(os.path.dirname(here)), os.path.dirname(here))
+        raises(ValueError, convert, tempfile.mktemp())
+
+    def test_existing_dirpath(self):
+        convert = self.types.get('existing-dirpath')
+        eq = self.assertEqual
+        raises = self.assertRaises
+        eq(convert('.'), '.')
+        eq(convert(here), here)
+        raises(ValueError, convert, '/a/hopefully/nonexistent/path')
+        raises(ValueError, convert, here + '/bogus')
+
+    def test_byte_size(self):
+        eq = self.assertEqual
+        raises = self.assertRaises
+        convert = self.types.get('byte-size')
+        eq(convert('128'), 128)
+        eq(convert('128KB'), 128*1024)
+        eq(convert('128MB'), 128*1024*1024)
+        eq(convert('128GB'), 128*1024*1024*1024L)
+        raises(ValueError, convert, '128TB')
+        eq(convert('128'), 128)
+        eq(convert('128kb'), 128*1024)
+        eq(convert('128mb'), 128*1024*1024)
+        eq(convert('128gb'), 128*1024*1024*1024L)
+        raises(ValueError, convert, '128tb')
+
+    def test_time_interval(self):
+        eq = self.assertEqual
+        raises = self.assertRaises
+        convert = self.types.get('time-interval')
+        eq(convert('120'), 120)
+        eq(convert('120S'), 120)
+        eq(convert('120M'), 120*60)
+        eq(convert('120H'), 120*60*60)
+        eq(convert('120D'), 120*60*60*24)
+        raises(ValueError, convert, '120W')
+        eq(convert('120'), 120)
+        eq(convert('120s'), 120)
+        eq(convert('120m'), 120*60)
+        eq(convert('120h'), 120*60*60)
+        eq(convert('120d'), 120*60*60*24)
+        raises(ValueError, convert, '120w')
+
+    def test_timedelta(self):
+        eq = self.assertEqual
+        raises = self.assertRaises
+        convert = self.types.get('timedelta')
+        eq(convert('4w'), datetime.timedelta(weeks=4))
+        eq(convert('2d'), datetime.timedelta(days=2))
+        eq(convert('7h'), datetime.timedelta(hours=7))
+        eq(convert('12m'), datetime.timedelta(minutes=12))
+        eq(convert('14s'), datetime.timedelta(seconds=14))
+        eq(convert('4w 2d 7h 12m 14s'),
+           datetime.timedelta(2, 14, minutes=12, hours=7, weeks=4))
+
+
+class RegistryTestCase(unittest.TestCase):
+
+    def test_registry_does_not_mask_toplevel_imports(self):
+        old_sys_path = sys.path[:]
+        tmpdir = tempfile.mkdtemp(prefix="test_datatypes_")
+        fn = os.path.join(tmpdir, "datatypes.py")
+        f = open(fn, "w")
+        f.write(TEST_DATATYPE_SOURCE)
+        f.close()
+        registry = ZConfig.datatypes.Registry()
+
+        # we really want the temp area to override everything else:
+        sys.path.insert(0, tmpdir)
+        try:
+            datatype = registry.get("datatypes.my_sample_datatype")
+        finally:
+            shutil.rmtree(tmpdir)
+            sys.path[:] = old_sys_path
+        self.assertEqual(datatype, 42)
+
+TEST_DATATYPE_SOURCE = """
+# sample datatypes file
+
+my_sample_datatype = 42
+"""
+
+
+def test_suite():
+    suite = unittest.makeSuite(DatatypeTestCase)
+    suite.addTest(unittest.makeSuite(RegistryTestCase))
+    return suite
+
+
+if __name__ == '__main__':
+    unittest.main(defaultTest='test_suite')

trunk/src/ZConfig/tests/test_loader.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Tests of ZConfig.loader classes and helper functions."""
+
+import os.path
+import sys
+import tempfile
+import unittest
+import urllib2
+
+from StringIO import StringIO
+
+import ZConfig
+import ZConfig.loader
+import ZConfig.url
+
+from ZConfig.tests.support import CONFIG_BASE, TestBase
+
+
+try:
+    myfile = __file__
+except NameError:
+    myfile = sys.argv[0]
+
+myfile = os.path.abspath(myfile)
+LIBRARY_DIR = os.path.join(os.path.dirname(myfile), "library")
+
+
+class LoaderTestCase(TestBase):
+
+    def test_schema_caching(self):
+        loader = ZConfig.loader.SchemaLoader()
+        url = ZConfig.url.urljoin(CONFIG_BASE, "simple.xml")
+        schema1 = loader.loadURL(url)
+        schema2 = loader.loadURL(url)
+        self.assert_(schema1 is schema2)
+
+    def test_simple_import_with_cache(self):
+        loader = ZConfig.loader.SchemaLoader()
+        url1 = ZConfig.url.urljoin(CONFIG_BASE, "library.xml")
+        schema1 = loader.loadURL(url1)
+        sio = StringIO("<schema>"
+                       "  <import src='library.xml'/>"
+                       "  <section type='type-a' name='section'/>"
+                       "</schema>")
+        url2 = ZConfig.url.urljoin(CONFIG_BASE, "stringio")
+        schema2 = loader.loadFile(sio, url2)
+        self.assert_(schema1.gettype("type-a") is schema2.gettype("type-a"))
+
+    def test_simple_import_using_prefix(self):
+        self.load_schema_text("""\
+            <schema prefix='ZConfig.tests.library'>
+              <import package='.thing'/>
+            </schema>
+            """)
+
+    def test_import_errors(self):
+        # must specify exactly one of package or src
+        self.assertRaises(ZConfig.SchemaError, ZConfig.loadSchemaFile,
+                          StringIO("<schema><import/></schema>"))
+        self.assertRaises(ZConfig.SchemaError, ZConfig.loadSchemaFile,
+                          StringIO("<schema>"
+                                   "  <import src='library.xml'"
+                                   "          package='ZConfig'/>"
+                                   "</schema>"))
+        # cannot specify src and file
+        self.assertRaises(ZConfig.SchemaError, ZConfig.loadSchemaFile,
+                          StringIO("<schema>"
+                                   "  <import src='library.xml'"
+                                   "          file='other.xml'/>"
+                                   "</schema>"))
+        # cannot specify module as package
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.test_loader'/>"
+                       "</schema>")
+        try:
+            ZConfig.loadSchemaFile(sio)
+        except ZConfig.SchemaResourceError, e:
+            self.assertEqual(e.filename, "component.xml")
+            self.assertEqual(e.package, "ZConfig.tests.test_loader")
+            self.assert_(e.path is None)
+            # make sure the str() doesn't raise an unexpected exception
+            str(e)
+        else:
+            self.fail("expected SchemaResourceError")
+
+    def test_import_from_package(self):
+        loader = ZConfig.loader.SchemaLoader()
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.library.widget'/>"
+                       "</schema>")
+        schema = loader.loadFile(sio)
+        self.assert_(schema.gettype("widget-a") is not None)
+
+    def test_import_from_package_with_file(self):
+        loader = ZConfig.loader.SchemaLoader()
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.library.widget'"
+                       "          file='extra.xml' />"
+                       "</schema>")
+        schema = loader.loadFile(sio)
+        self.assert_(schema.gettype("extra-type") is not None)
+
+    def test_import_from_package_extra_directory(self):
+        loader = ZConfig.loader.SchemaLoader()
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.library.thing'"
+                       "          file='extras.xml' />"
+                       "</schema>")
+        schema = loader.loadFile(sio)
+        self.assert_(schema.gettype("extra-thing") is not None)
+
+    def test_import_from_package_with_missing_file(self):
+        loader = ZConfig.loader.SchemaLoader()
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.library.widget'"
+                       "          file='notthere.xml' />"
+                       "</schema>")
+        try:
+            loader.loadFile(sio)
+        except ZConfig.SchemaResourceError, e:
+            self.assertEqual(e.filename, "notthere.xml")
+            self.assertEqual(e.package, "ZConfig.tests.library.widget")
+            self.assert_(e.path)
+            # make sure the str() doesn't raise an unexpected exception
+            str(e)
+        else:
+            self.fail("expected SchemaResourceError")
+
+    def test_import_from_package_with_directory_file(self):
+        loader = ZConfig.loader.SchemaLoader()
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.library.widget'"
+                       "          file='really/notthere.xml' />"
+                       "</schema>")
+        self.assertRaises(ZConfig.SchemaError, loader.loadFile, sio)
+
+    def test_import_two_components_one_package(self):
+        loader = ZConfig.loader.SchemaLoader()
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.library.widget' />"
+                       "  <import package='ZConfig.tests.library.widget'"
+                       "          file='extra.xml' />"
+                       "</schema>")
+        schema = loader.loadFile(sio)
+        schema.gettype("widget-a")
+        schema.gettype("extra-type")
+
+    def test_import_component_twice_1(self):
+        # Make sure we can import a component twice from a schema.
+        # This is most likely to occur when the component is imported
+        # from each of two other components, or from the top-level
+        # schema and a component.
+        loader = ZConfig.loader.SchemaLoader()
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.library.widget' />"
+                       "  <import package='ZConfig.tests.library.widget' />"
+                       "</schema>")
+        schema = loader.loadFile(sio)
+        schema.gettype("widget-a")
+
+    def test_import_component_twice_2(self):
+        # Make sure we can import a component from a config file even
+        # if it has already been imported from the schema.
+        loader = ZConfig.loader.SchemaLoader()
+        sio = StringIO("<schema>"
+                       "  <import package='ZConfig.tests.library.widget' />"
+                       "</schema>")
+        schema = loader.loadFile(sio)
+        loader = ZConfig.loader.ConfigLoader(schema)
+        sio = StringIO("%import ZConfig.tests.library.widget")
+        loader.loadFile(sio)
+
+    def test_urlsplit_urlunsplit(self):
+        # Extracted from Python's test.test_urlparse module:
+        for url, parsed, split in [
+            ('http://www.python.org',
+             ('http', 'www.python.org', '', '', '', ''),
+             ('http', 'www.python.org', '', '', '')),
+            ('http://www.python.org#abc',
+             ('http', 'www.python.org', '', '', '', 'abc'),
+             ('http', 'www.python.org', '', '', 'abc')),
+            ('http://www.python.org/#abc',
+             ('http', 'www.python.org', '/', '', '', 'abc'),
+             ('http', 'www.python.org', '/', '', 'abc')),
+            ("http://a/b/c/d;p?q#f",
+             ('http', 'a', '/b/c/d', 'p', 'q', 'f'),
+             ('http', 'a', '/b/c/d;p', 'q', 'f')),
+            ('file:///tmp/junk.txt',
+             ('file', '', '/tmp/junk.txt', '', '', ''),
+             ('file', '', '/tmp/junk.txt', '', '')),
+            ]:
+            result = ZConfig.url.urlsplit(url)
+            self.assertEqual(result, split)
+            result2 = ZConfig.url.urlunsplit(result)
+            self.assertEqual(result2, url)
+
+    def test_file_url_normalization(self):
+        self.assertEqual(
+            ZConfig.url.urlnormalize("file:/abc/def"),
+            "file:///abc/def")
+        self.assertEqual(
+            ZConfig.url.urlunsplit(("file", "", "/abc/def", "", "")),
+            "file:///abc/def")
+        self.assertEqual(
+            ZConfig.url.urljoin("file:/abc/", "def"),
+            "file:///abc/def")
+        self.assertEqual(
+            ZConfig.url.urldefrag("file:/abc/def#frag"),
+            ("file:///abc/def", "frag"))
+
+    def test_isPath(self):
+        assert_ = self.assert_
+        isPath = ZConfig.loader.BaseLoader().isPath
+        assert_(isPath("abc"))
+        assert_(isPath("abc/def"))
+        assert_(isPath("/abc"))
+        assert_(isPath("/abc/def"))
+        assert_(isPath(r"\abc"))
+        assert_(isPath(r"\abc\def"))
+        assert_(isPath(r"c:\abc\def"))
+        assert_(not isPath("http://www.example.com/"))
+        assert_(not isPath("http://www.example.com/sample.conf"))
+        assert_(not isPath("file:///etc/zope/zope.conf"))
+        assert_(not isPath("file:///c|/foo/bar.conf"))
+
+
+class TestNonExistentResources(unittest.TestCase):
+
+    # XXX Not sure if this is the best approach for these.  These
+    # tests make sure that the error reported by ZConfig for missing
+    # resources is handled in a consistent way.  Since ZConfig uses
+    # urllib2.urlopen() for opening all resources, what we do is
+    # replace that function with one that always raises an exception.
+    # Since urllib2.urlopen() can raise either IOError or OSError
+    # (depending on the version of Python), we run test for each
+    # exception.  urllib2.urlopen() is restored after running the
+    # test.
+
+    def setUp(self):
+        self.urllib2_urlopen = urllib2.urlopen
+        urllib2.urlopen = self.fake_urlopen
+
+    def tearDown(self):
+        urllib2.urlopen = self.urllib2_urlopen
+
+    def fake_urlopen(self, url):
+        raise self.error()
+
+    def test_nonexistent_file_ioerror(self):
+        self.error = IOError
+        self.check_nonexistent_file()
+
+    def test_nonexistent_file_oserror(self):
+        self.error = OSError
+        self.check_nonexistent_file()
+
+    def check_nonexistent_file(self):
+        fn = tempfile.mktemp()
+        schema = ZConfig.loadSchemaFile(StringIO("<schema/>"))
+        self.assertRaises(ZConfig.ConfigurationError,
+                          ZConfig.loadSchema, fn)
+        self.assertRaises(ZConfig.ConfigurationError,
+                          ZConfig.loadConfig, schema, fn)
+        self.assertRaises(ZConfig.ConfigurationError,
+                          ZConfig.loadConfigFile, schema,
+                          StringIO("%include " + fn))
+        self.assertRaises(ZConfig.ConfigurationError,
+                          ZConfig.loadSchema,
+                          "http://www.zope.org/no-such-document/")
+        self.assertRaises(ZConfig.ConfigurationError,
+                          ZConfig.loadConfig, schema,
+                          "http://www.zope.org/no-such-document/")
+
+
+def test_suite():
+    suite = unittest.makeSuite(LoaderTestCase)
+    suite.addTest(unittest.makeSuite(TestNonExistentResources))
+    return suite
+
+if __name__ == '__main__':
+    unittest.main(defaultTest='test_suite')

trunk/src/ZConfig/tests/test_schema.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Tests of ZConfig schemas."""
+
+import unittest
+
+import ZConfig
+
+from ZConfig.tests.support import TestBase, CONFIG_BASE
+
+
+def uppercase(value):
+    return str(value).upper()
+
+def appsection(value):
+    return MySection(value)
+
+def get_foo(section):
+    return section.foo
+
+class MySection:
+    def __init__(self, value):
+        self.conf = value
+
+
+def get_section_attributes(section):
+    L = list(section.getSectionAttributes())
+    L.sort()
+    return L
+
+
+class SchemaTestCase(TestBase):
+    """Tests of the basic schema support itself."""
+
+    def test_minimal_schema(self):
+        schema = self.load_schema_text("<schema/>")
+        self.assertEqual(len(schema), 0)
+        self.assertRaises(IndexError,
+                          lambda schema=schema: schema[0])
+        self.assertRaises(ZConfig.ConfigurationError,
+                          schema.getinfo, "foo")
+
+    def test_simple(self):
+        schema, conf = self.load_both("simple.xml", "simple.conf")
+        self._verifySimpleConf(conf)
+
+    def _verifySimpleConf(self,conf):
+        eq = self.assertEqual
+        eq(conf.var1, 'abc')
+        eq(conf.int_var, 12)
+        eq(conf.float_var, 12.02)
+        eq(conf.neg_int, -2)
+
+        check = self.assert_
+        check(conf.true_var_1)
+        check(conf.true_var_2)
+        check(conf.true_var_3)
+        check(not conf.false_var_1)
+        check(not conf.false_var_2)
+        check(not conf.false_var_3)
+
+    def test_app_datatype(self):
+        dtname = __name__ + ".uppercase"
+        schema = self.load_schema_text("""\
+            <schema>
+              <key name='a' datatype='%s'/>
+              <key name='b' datatype='%s' default='abc'/>
+              <multikey name='c' datatype='%s'>
+                <default>abc</default>
+                <default>abc</default>
+                </multikey>
+              <multikey name='d' datatype='%s'>
+                <default>not</default>
+                <default>lower</default>
+                <default>case</default>
+                </multikey>
+            </schema>
+            """ % (dtname, dtname, dtname, dtname))
+        conf = self.load_config_text(schema, """\
+                                     a qwerty
+                                     c upp
+                                     c er
+                                     c case
+                                     """)
+        eq = self.assertEqual
+        eq(conf.a, 'QWERTY')
+        eq(conf.b, 'ABC')
+        eq(conf.c, ['UPP', 'ER', 'CASE'])
+        eq(conf.d, ['NOT', 'LOWER', 'CASE'])
+        eq(get_section_attributes(conf),
+           ["a", "b", "c", "d"])
+
+    def test_app_sectiontype(self):
+        schema = self.load_schema_text("""\
+            <schema datatype='.appsection' prefix='%s'>
+              <sectiontype name='foo' datatype='.MySection'>
+                <key name='sample' datatype='integer' default='345'/>
+                </sectiontype>
+              <section name='sect' type='foo' />
+            </schema>
+            """ % __name__)
+        conf = self.load_config_text(schema, """\
+                                     <foo sect>
+                                       sample 42
+                                     </foo>
+                                     """)
+        self.assert_(isinstance(conf, MySection))
+        o1 = conf.conf.sect
+        self.assert_(isinstance(o1, MySection))
+        self.assertEqual(o1.conf.sample, 42)
+
+    def test_empty_sections(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='section'/>
+              <section type='section' name='s1'/>
+              <section type='section' name='s2'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     <section s1>
+                                     </section>
+                                     <section s2/>
+                                     """)
+        self.assert_(conf.s1 is not None)
+        self.assert_(conf.s2 is not None)
+        self.assertEqual(get_section_attributes(conf),
+                         ["s1", "s2"])
+
+    def test_deeply_nested_sections(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='type1'>
+                <key name='key' default='type1-value'/>
+              </sectiontype>
+              <sectiontype name='type2'>
+                <key name='key' default='type2-value'/>
+                <section name='sect' type='type1'/>
+              </sectiontype>
+              <sectiontype name='type3'>
+                <key name='key' default='type3-value'/>
+                <section name='sect' type='type2'/>
+              </sectiontype>
+              <section name='sect' type='type3'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     <type3 sect>
+                                       key sect3-value
+                                       <type2 sect>
+                                         key sect2-value
+                                         <type1 sect/>
+                                       </type2>
+                                     </type3>
+                                     """)
+        eq = self.assertEqual
+        eq(conf.sect.sect.sect.key, "type1-value")
+        eq(conf.sect.sect.key, "sect2-value")
+        eq(conf.sect.key, "sect3-value")
+        eq(get_section_attributes(conf),
+           ["sect"])
+        eq(get_section_attributes(conf.sect),
+           ["key", "sect"])
+        eq(get_section_attributes(conf.sect.sect),
+           ["key", "sect"])
+        eq(get_section_attributes(conf.sect.sect.sect),
+           ["key"])
+
+    def test_multivalued_keys(self):
+        schema = self.load_schema_text("""\
+            <schema handler='def'>
+              <multikey name='a' handler='ABC' />
+              <multikey name='b' datatype='integer'>
+                <default>1</default>
+                <default>2</default>
+              </multikey>
+              <multikey name='c' datatype='integer'>
+                <default>3</default>
+                <default>4</default>
+                <default>5</default>
+              </multikey>
+              <multikey name='d' />
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     a foo
+                                     a bar
+                                     c 41
+                                     c 42
+                                     c 43
+                                     """, num_handlers=2)
+        L = []
+        self.handlers({'abc': L.append,
+                       'DEF': L.append})
+        self.assertEqual(L, [['foo', 'bar'], conf])
+        L = []
+        self.handlers({'abc': None,
+                       'DEF': L.append})
+        self.assertEqual(L, [conf])
+        self.assertEqual(conf.a, ['foo', 'bar'])
+        self.assertEqual(conf.b, [1, 2])
+        self.assertEqual(conf.c, [41, 42, 43])
+        self.assertEqual(conf.d, [])
+        self.assertEqual(get_section_attributes(conf),
+                         ["a", "b", "c", "d"])
+
+    def test_multikey_required(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <multikey name='k' required='yes'/>
+            </schema>
+            """)
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "")
+
+    def test_multisection_required(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='s'/>
+              <multisection name='*' attribute='s' type='s' required='yes'/>
+            </schema>
+            """)
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "")
+
+    def test_key_required_but_missing(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <key name='k' required='yes'/>
+            </schema>
+            """)
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "")
+
+    def test_section_required_but_missing(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='k'/>
+              <section name='k' type='k' required='yes'/>
+            </schema>
+            """)
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "")
+
+    def test_key_default_element(self):
+        self.assertRaises(
+            ZConfig.SchemaError, self.load_schema_text, """\
+            <schema>
+              <key name='name'>
+                <default>text</default>
+              </key>
+            </schema>
+            """)
+
+    def test_bad_handler_maps(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <key name='a' handler='abc'/>
+              <key name='b' handler='def'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     a foo
+                                     b bar
+                                     """, num_handlers=2)
+        self.assertEqual(get_section_attributes(conf),
+                         ["a", "b"])
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.handlers, {'abc': id, 'ABC': id, 'def': id})
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.handlers, {})
+
+    def test_handler_ordering(self):
+        schema = self.load_schema_text("""\
+            <schema handler='c'>
+              <sectiontype name='inner'>
+              </sectiontype>
+              <sectiontype name='outer'>
+                <section type='inner' name='sect-inner' handler='a'/>
+              </sectiontype>
+              <section type='outer' name='sect-outer' handler='b'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     <outer sect-outer>
+                                       <inner sect-inner/>
+                                     </outer>
+                                     """, num_handlers=3)
+        L = []
+        self.handlers({'a': L.append,
+                       'b': L.append,
+                       'c': L.append})
+        outer = conf.sect_outer
+        inner = outer.sect_inner
+        self.assertEqual(L, [inner, outer, conf])
+
+    def test_duplicate_section_names(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='sect'/>
+              <sectiontype name='nesting'>
+                <section name='a' type='sect'/>
+              </sectiontype>
+              <section name='a' type='nesting'/>
+            </schema>
+            """)
+        self.assertRaises(ZConfig.ConfigurationError, self.load_config_text,
+                          schema, """\
+                          <sect a/>
+                          <sect a/>
+                          """)
+        conf = self.load_config_text(schema, """\
+                                     <nesting a>
+                                       <sect a/>
+                                     </nesting>
+                                     """)
+
+    def test_disallowed_duplicate_attribute(self):
+        self.assertRaises(ZConfig.SchemaError, self.load_schema_text, """\
+                          <schema>
+                            <key name='a'/>
+                            <key name='b' attribute='a'/>
+                          </schema>
+                          """)
+
+    def test_unknown_datatype_name(self):
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, "<schema datatype='foobar'/>")
+
+    def test_load_abstracttype(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <abstracttype name='group'>
+                <description>This is an abstract section type.</description>
+              </abstracttype>
+              <sectiontype name='t1' implements='group'>
+                <key name='k1' default='default1'/>
+              </sectiontype>
+              <sectiontype name='t2' implements='group'>
+                <key name='k2' default='default2'/>
+              </sectiontype>
+              <multisection name='*' type='group' attribute='g'/>
+            </schema>
+            """)
+        # check the types that get defined
+        t = schema.gettype("group")
+        self.assert_(t.isabstract())
+        t1 = schema.gettype("t1")
+        self.assert_(not t1.isabstract())
+        self.assert_(t.getsubtype("t1") is t1)
+        t2 = schema.gettype("t2")
+        self.assert_(not t2.isabstract())
+        self.assert_(t.getsubtype("t2") is t2)
+        self.assertRaises(ZConfig.ConfigurationError, t.getsubtype, "group")
+        self.assert_(t1 is not t2)
+        # try loading a config that relies on this schema
+        conf = self.load_config_text(schema, """\
+                                     <t1/>
+                                     <t1>
+                                       k1 value1
+                                     </t1>
+                                     <t2/>
+                                     <t2>
+                                       k2 value2
+                                     </t2>
+                                     """)
+        eq = self.assertEqual
+        eq(get_section_attributes(conf), ["g"])
+        eq(len(conf.g), 4)
+        eq(conf.g[0].k1, "default1")
+        eq(conf.g[1].k1, "value1")
+        eq(conf.g[2].k2, "default2")
+        eq(conf.g[3].k2, "value2")
+
+        # white box:
+        self.assert_(conf.g[0].getSectionDefinition() is t1)
+        self.assert_(conf.g[1].getSectionDefinition() is t1)
+        self.assert_(conf.g[2].getSectionDefinition() is t2)
+        self.assert_(conf.g[3].getSectionDefinition() is t2)
+
+    def test_abstracttype_extension(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <abstracttype name='group'/>
+              <sectiontype name='extra' implements='group'/>
+              <section name='thing' type='group'/>
+            </schema>
+            """)
+        abstype = schema.gettype("group")
+        self.assert_(schema.gettype("extra") is abstype.getsubtype("extra"))
+
+        # make sure we can use the extension in a config:
+        conf = self.load_config_text(schema, "<extra thing/>")
+        self.assertEqual(conf.thing.getSectionType(), "extra")
+        self.assertEqual(get_section_attributes(conf), ["thing"])
+        self.assertEqual(get_section_attributes(conf.thing), [])
+
+    def test_abstracttype_extension_errors(self):
+        # specifying a non-existant abstracttype
+        self.assertRaises(ZConfig.SchemaError, self.load_schema_text, """\
+                          <schema>
+                            <sectiontype name='s' implements='group'/>
+                          </schema>
+                          """)
+        # specifying something that isn't an abstracttype
+        self.assertRaises(ZConfig.SchemaError, self.load_schema_text, """\
+                          <schema>
+                            <sectiontype name='t1'/>
+                            <sectiontype name='t2' implements='t1'/>
+                          </schema>
+                          """)
+
+    def test_arbitrary_key(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <key name='+' required='yes' attribute='keymap'
+                   datatype='integer'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, "some-key 42")
+        self.assertEqual(conf.keymap, {'some-key': 42})
+        self.assertEqual(get_section_attributes(conf), ["keymap"])
+
+    def test_arbitrary_multikey_required(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <multikey name='+' required='yes' attribute='keymap'
+                        datatype='integer'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     some-key 42
+                                     some-key 43
+                                     """)
+        self.assertEqual(conf.keymap, {'some-key': [42, 43]})
+
+    def test_arbitrary_multikey_optional(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='sect'>
+                <multikey name='+' attribute='keymap'/>
+              </sectiontype>
+              <section name='+' type='sect' attribute='stuff'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     <sect foo>
+                                       some-key 42
+                                       some-key 43
+                                     </sect>
+                                     """)
+        self.assertEqual(conf.stuff.keymap, {'some-key': ['42', '43']})
+        self.assertEqual(get_section_attributes(conf), ["stuff"])
+
+    def test_arbitrary_multikey_optional_empty(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='sect'>
+                <multikey name='+' attribute='keymap'/>
+              </sectiontype>
+              <section name='+' type='sect' attribute='stuff'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, "<sect foo/>")
+        self.assertEqual(conf.stuff.keymap, {})
+
+    def test_arbitrary_multikey_with_defaults(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <multikey name='+' attribute='keymap'>
+                <default key='a'>value-a1</default>
+                <default key='a'>value-a2</default>
+                <default key='b'>value-b</default>
+              </multikey>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, "")
+        self.assertEqual(conf.keymap, {'a': ['value-a1', 'value-a2'],
+                                       'b': ['value-b']})
+
+    def test_arbitrary_multikey_with_unkeyed_default(self):
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, """\
+                          <schema>
+                            <multikey name='+' attribute='keymap'>
+                              <default>value-a1</default>
+                            </multikey>
+                          </schema>
+                          """)
+
+    def test_arbitrary_key_with_defaults(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <key name='+' attribute='keymap'>
+                <default key='a'>value-a</default>
+                <default key='b'>value-b</default>
+              </key>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, "")
+        self.assertEqual(conf.keymap, {'a': 'value-a', 'b': 'value-b'})
+
+    def test_arbitrary_key_with_unkeyed_default(self):
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, """\
+                          <schema>
+                            <key name='+' attribute='keymap'>
+                              <default>value-a1</default>
+                            </key>
+                          </schema>
+                          """)
+
+    def test_arbitrary_keys_with_others(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <key name='k1' default='v1'/>
+              <key name='k2' default='2' datatype='integer'/>
+              <key name='+' required='yes' attribute='keymap'
+                   datatype='integer'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     some-key 42
+                                     k2 3
+                                     """)
+        self.assertEqual(conf.k1, 'v1')
+        self.assertEqual(conf.k2, 3)
+        self.assertEqual(conf.keymap, {'some-key': 42})
+        self.assertEqual(get_section_attributes(conf),
+                         ["k1", "k2", "keymap"])
+
+    def test_arbitrary_key_missing(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <key name='+' required='yes' attribute='keymap' />
+            </schema>
+            """)
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "# empty config file")
+
+    def test_arbitrary_key_bad_schema(self):
+        self.assertRaises(ZConfig.SchemaError, self.load_schema_text, """\
+                          <schema>
+                            <key name='+' attribute='attr1'/>
+                            <key name='+' attribute='attr2'/>
+                          </schema>
+                          """)
+
+    def test_getrequiredtypes(self):
+        schema = self.load_schema("library.xml")
+        self.assertEqual(schema.getrequiredtypes(), [])
+
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='used'/>
+              <sectiontype name='unused'/>
+              <section type='used' name='a'/>
+            </schema>
+            """)
+        L = schema.getrequiredtypes()
+        L.sort()
+        self.assertEqual(L, ["used"])
+
+    def test_getunusedtypes(self):
+        schema = self.load_schema("library.xml")
+        L = schema.getunusedtypes()
+        L.sort()
+        self.assertEqual(L, ["type-a", "type-b"])
+
+        schema = self.load_schema_text("""\
+            <schema type='top'>
+              <sectiontype name='used'/>
+              <sectiontype name='unused'/>
+              <section type='used' name='a'/>
+            </schema>
+            """)
+        self.assertEqual(schema.getunusedtypes(), ["unused"])
+
+    def test_section_value_mutation(self):
+        schema, conf = self.load_both("simple.xml", "simple.conf")
+        orig = conf.empty
+        new = []
+        conf.empty = new
+        self.assert_(conf.empty is new)
+
+    def test_simple_anonymous_section(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='sect'>
+                <key name='key' default='value'/>
+              </sectiontype>
+              <section name='*' type='sect' attribute='attr'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, "<sect/>")
+        self.assertEqual(conf.attr.key, "value")
+
+    def test_simple_anynamed_section(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='sect'>
+                <key name='key' default='value'/>
+              </sectiontype>
+              <section name='+' type='sect' attribute='attr'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, "<sect name/>")
+        self.assertEqual(conf.attr.key, "value")
+        self.assertEqual(conf.attr.getSectionName(), "name")
+
+        # if we omit the name, it's an error
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "<sect/>")
+
+    def test_nested_abstract_sectiontype(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <abstracttype name='abstract'/>
+              <sectiontype name='t1' implements='abstract'/>
+              <sectiontype name='t2' implements='abstract'>
+                <section type='abstract' name='s1'/>
+              </sectiontype>
+              <section type='abstract' name='*' attribute='s2'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     <t2>
+                                       <t1 s1/>
+                                     </t2>
+                                     """)
+
+    def test_reserved_attribute_prefix(self):
+        template = """\
+            <schema>
+              <sectiontype name='s'/>
+                %s
+            </schema>
+            """
+        def check(thing, self=self, template=template):
+            text = template % thing
+            self.assertRaises(ZConfig.SchemaError,
+                              self.load_schema_text, text)
+
+        check("<key name='a' attribute='getSection'/>")
+        check("<key name='a' attribute='getSectionThing'/>")
+        check("<multikey name='a' attribute='getSection'/>")
+        check("<multikey name='a' attribute='getSectionThing'/>")
+        check("<section type='s' name='*' attribute='getSection'/>")
+        check("<section type='s' name='*' attribute='getSectionThing'/>")
+        check("<multisection type='s' name='*' attribute='getSection'/>")
+        check("<multisection type='s' name='*' attribute='getSectionThing'/>")
+
+    def test_sectiontype_as_schema(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='s'>
+                <key name='skey' default='skey-default'/>
+              </sectiontype>
+              <sectiontype name='t'>
+                <key name='tkey' default='tkey-default'/>
+                <section name='*' type='s' attribute='section'/>
+              </sectiontype>
+            </schema>
+            """)
+        t = schema.gettype("t")
+        conf = self.load_config_text(t, "<s/>")
+        self.assertEqual(conf.tkey, "tkey-default")
+        self.assertEqual(conf.section.skey, "skey-default")
+        self.assertEqual(get_section_attributes(conf), ["section", "tkey"])
+        self.assertEqual(get_section_attributes(conf.section), ["skey"])
+
+    def test_datatype_conversion_error(self):
+        schema_url = "file:///tmp/fake-url-1.xml"
+        config_url = "file:///tmp/fake-url-2.xml"
+        schema = self.load_schema_text("""\
+             <schema>
+               <key name='key' default='bogus' datatype='integer'/>
+             </schema>
+             """, url=schema_url)
+        e = self.get_data_conversion_error(
+            schema, "", config_url)
+        self.assertEqual(e.url, schema_url)
+        self.assertEqual(e.lineno, 2)
+
+        e = self.get_data_conversion_error(schema, """\
+                                           # comment
+
+                                           key splat
+                                           """, config_url)
+        self.assertEqual(e.url, config_url)
+        self.assertEqual(e.lineno, 3)
+
+    def get_data_conversion_error(self, schema, src, url):
+        try:
+            self.load_config_text(schema, src, url=url)
+        except ZConfig.DataConversionError, e:
+            return e
+        else:
+            self.fail("expected ZConfig.DataConversionError")
+
+    def test_numeric_section_name(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='sect'/>
+              <multisection name='*' type='sect' attribute='things'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, "<sect 1 />")
+        self.assertEqual(len(conf.things), 1)
+
+    def test_sectiontype_extension(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='t1'>
+                <key name='k1'/>
+              </sectiontype>
+              <sectiontype name='t2' extends='t1'>
+                <key name='k2'/>
+              </sectiontype>
+              <section name='s' type='t2'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+                                     <t2 s>
+                                       k1 k1-value
+                                       k2 k2-value
+                                     </t2>
+                                     """)
+        eq = self.assertEqual
+        eq(conf.s.k1, "k1-value")
+        eq(conf.s.k2, "k2-value")
+        eq(get_section_attributes(conf), ["s"])
+        eq(get_section_attributes(conf.s), ["k1", "k2"])
+
+    def test_sectiontype_extension_errors(self):
+        # cannot override key from base
+        self.assertRaises(ZConfig.SchemaError, self.load_schema_text, """\
+                          <schema>
+                            <sectiontype name='t1'>
+                              <key name='k1'/>
+                            </sectiontype>
+                            <sectiontype name='t2' extends='t1'>
+                              <key name='k1'/>
+                            </sectiontype>
+                          </schema>
+                          """)
+        # cannot extend non-existing section
+        self.assertRaises(ZConfig.SchemaError, self.load_schema_text, """\
+                          <schema>
+                            <sectiontype name='t2' extends='t1'/>
+                          </schema>
+                          """)
+        # cannot extend abstract type
+        self.assertRaises(ZConfig.SchemaError, self.load_schema_text, """\
+                          <schema>
+                            <abstracttype name='t1'/>
+                            <sectiontype name='t2' extends='t1'/>
+                          </schema>
+                          """)
+
+    def test_sectiontype_derived_keytype(self):
+        # make sure that a derived section type inherits the keytype
+        # of its base
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='sect' keytype='identifier'/>
+              <sectiontype name='derived' extends='sect'>
+                <key name='foo' attribute='foo'/>
+                <key name='Foo' attribute='Foo'/>
+              </sectiontype>
+              <section name='foo' type='derived'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+            <derived foo>
+              foo bar
+              Foo BAR
+            </derived>
+            """)
+        self.assertEqual(conf.foo.foo, "bar")
+        self.assertEqual(conf.foo.Foo, "BAR")
+        self.assertEqual(get_section_attributes(conf.foo), ["Foo", "foo"])
+
+    def test_sectiontype_override_keytype(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='base' keytype='identifier' >
+                <key name='+' attribute='map' />
+              </sectiontype>
+              <sectiontype name='derived' keytype='ipaddr-or-hostname'
+                           extends='base' />
+              <section name='*' type='base' attribute='base' />
+              <section name='*' type='derived' attribute='derived' />
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+            <base>
+              ident1 foo
+              Ident2 bar
+            </base>
+            <derived>
+              EXAMPLE.COM foo
+            </derived>
+            """)
+        L = conf.base.map.items()
+        L.sort()
+        self.assertEqual(L, [("Ident2", "bar"), ("ident1", "foo")])
+        L = conf.derived.map.items()
+        L.sort()
+        self.assertEqual(L, [("example.com", "foo")])
+        self.assertEqual(get_section_attributes(conf), ["base", "derived"])
+
+    def test_keytype_applies_to_default_key(self):
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='sect'>
+                <key name='+' attribute='mapping'>
+                  <default key='foo'>42</default>
+                  <default key='BAR'>24</default>
+                </key>
+              </sectiontype>
+              <section type='sect' name='*' attribute='sect'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, "<sect/>")
+        items = conf.sect.mapping.items()
+        items.sort()
+        self.assertEqual(items, [("bar", "24"), ("foo", "42")])
+
+    def test_duplicate_default_key_checked_in_schema(self):
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, """\
+            <schema>
+              <sectiontype name='sect'>
+                <key name='+' attribute='mapping'>
+                  <default key='foo'>42</default>
+                  <default key='Foo'>24</default>
+                </key>
+              </sectiontype>
+              <section type='sect' name='*' attribute='sect'/>
+            </schema>
+            """)
+
+    def test_default_keys_rechecked_clash_in_derived_sectiontype(self):
+        # If the default values associated with a <key name="+"> can't
+        # be supported by a new keytype for a derived sectiontype, an
+        # error should be indicated.
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, """\
+            <schema>
+              <sectiontype name='base' keytype='identifier'>
+                <key name='+' attribute='mapping'>
+                  <default key='foo'>42</default>
+                  <default key='Foo'>42</default>
+                </key>
+              </sectiontype>
+              <sectiontype name='sect' keytype='basic-key'
+                           extends='base'>
+                <!-- should cry foul here -->
+              </sectiontype>
+              <section type='sect' name='*' attribute='sect'/>
+            </schema>
+            """)
+
+    def test_default_keys_rechecked_dont_clash_in_derived_sectiontype(self):
+        # If the default values associated with a <key name="+"> can't
+        # be supported by a new keytype for a derived sectiontype, an
+        # error should be indicated.
+        schema = self.load_schema_text("""\
+            <schema>
+              <sectiontype name='base' keytype='identifier'>
+                <multikey name='+' attribute='mapping'>
+                  <default key='foo'>42</default>
+                  <default key='Foo'>42</default>
+                </multikey>
+              </sectiontype>
+              <sectiontype name='sect' keytype='basic-key'
+                           extends='base'>
+                <!-- should cry foul here -->
+              </sectiontype>
+              <section type='base' name='*' attribute='base'/>
+              <section type='sect' name='*' attribute='sect'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+            <base/>
+            <sect/>
+            """)
+        base = conf.base.mapping.items()
+        base.sort()
+        self.assertEqual(base, [("Foo", ["42"]), ("foo", ["42"])])
+        sect = conf.sect.mapping.items()
+        sect.sort()
+        self.assertEqual(sect, [("foo", ["42", "42"])])
+
+    def test_sectiontype_inherited_datatype(self):
+        schema = self.load_schema_text("""\
+            <schema prefix='ZConfig.tests.test_schema'>
+              <sectiontype name='base' datatype='.get_foo'>
+                <key name="foo"/>
+              </sectiontype>
+              <sectiontype name='derived' extends='base'/>
+              <section name='*' type='derived' attribute='splat'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema, """\
+            <derived>
+              foo bar
+            </derived>
+            """)
+        self.assertEqual(conf.splat, "bar")
+
+    def test_schema_keytype(self):
+        schema = self.load_schema_text("""\
+            <schema keytype='ipaddr-or-hostname'>
+              <key name='+' attribute='table' datatype='ipaddr-or-hostname'/>
+            </schema>
+            """)
+        conf = self.load_config_text(schema,
+                                     "host.example.com 127.0.0.1\n"
+                                     "www.example.org 127.0.0.2\n")
+        table = conf.table
+        self.assertEqual(len(table), 2)
+        L = table.items()
+        L.sort()
+        self.assertEqual(L, [("host.example.com", "127.0.0.1"),
+                             ("www.example.org", "127.0.0.2")])
+
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "abc.  127.0.0.1")
+
+    def test_keytype_identifier(self):
+        schema = self.load_schema_text("""\
+           <schema keytype='identifier'>
+             <key name='foo' attribute='foo'/>
+             <key name='Foo' attribute='Foo'/>
+           </schema>
+           """)
+        conf = self.load_config_text(schema,
+                                     "Foo Foo-value\n"
+                                     "foo foo-value\n")
+        self.assertEqual(conf.foo, "foo-value")
+        self.assertEqual(conf.Foo, "Foo-value")
+        self.assertEqual(get_section_attributes(conf), ["Foo", "foo"])
+        # key mis-match based on case:
+        self.assertRaises(ZConfig.ConfigurationError,
+                          self.load_config_text, schema, "FOO frob\n")
+        # attribute names conflict, since the keytype isn't used to
+        # generate attribute names
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, """\
+                          <schema keytype='identifier'>
+                            <key name='foo'/>
+                            <key name='Foo'/>
+                          </schema>
+                          """)
+
+    def test_datatype_casesensitivity(self):
+        self.load_schema_text("<schema datatype='NULL'/>")
+
+    def test_simple_extends(self):
+        schema = self.load_schema_text("""\
+           <schema extends='%s/simple.xml %s/library.xml'>
+             <section name='A' type='type-a' />
+           </schema>
+           """ % (CONFIG_BASE, CONFIG_BASE))
+        self._verifySimpleConf(self.load_config(schema, "simple.conf"))
+
+    def test_extends_fragment_failure(self):
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text,
+            "<schema extends='%s/library.xml#foo'/>" % CONFIG_BASE)
+
+    def test_multi_extends_implicit_OK(self):
+        self.load_schema_text("""\
+           <schema extends='%s/base.xml %s/library.xml'>
+             <section name='A' type='type-a' />
+             <section name='X' type='type-X' />
+           </schema>
+           """ % (CONFIG_BASE, CONFIG_BASE))
+
+    def test_multi_extends_explicit_datatype_OK(self):
+        self.load_schema_text("""\
+           <schema extends='%s/base-datatype1.xml %s/base-datatype2.xml'
+                   datatype='null'>
+             <section name='One' type='type-1' />
+             <section name='Two' type='type-2' />
+           </schema>
+           """ % (CONFIG_BASE, CONFIG_BASE))
+
+    def test_multi_extends_explicit_keytype_OK(self):
+        self.load_schema_text("""\
+           <schema extends='%s/base-keytype1.xml %s/base-keytype2.xml'
+                   keytype='%s.uppercase'>
+             <section name='One' type='type-1' />
+             <section name='Two' type='type-2' />
+           </schema>
+           """ % (CONFIG_BASE, CONFIG_BASE, __name__))
+
+    def test_multi_extends_datatype_conflict(self):
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, """\
+            <schema extends='%s/base-datatype1.xml %s/base-datatype2.xml'/>
+            """ % (CONFIG_BASE, CONFIG_BASE))
+
+    def test_multi_extends_keytype_conflict(self):
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, """\
+            <schema extends='%s/base-keytype1.xml %s/base-keytype2.xml'/>
+            """ % (CONFIG_BASE, CONFIG_BASE))
+
+    def test_multiple_descriptions_is_error(self):
+        self.assertRaises(ZConfig.SchemaError,
+                          self.load_schema_text, """\
+            <schema>
+              <description>  foo  </description>
+              <description>  bar  </description>
+            </schema>
+            """)
+
+
+def test_suite():
+    return unittest.makeSuite(SchemaTestCase)
+
+if __name__ == '__main__':
+    unittest.main(defaultTest='test_suite')

trunk/src/ZConfig/tests/test_subst.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Tests of the string interpolation module."""
+
+# This is needed to support Python 2.1.
+from __future__ import nested_scopes
+
+import unittest
+
+from ZConfig import SubstitutionReplacementError, SubstitutionSyntaxError
+from ZConfig.substitution import isname, substitute
+
+
+class SubstitutionTestCase(unittest.TestCase):
+    def test_simple_names(self):
+        d = {"name": "value",
+             "name1": "abc",
+             "name_": "def",
+             "_123": "ghi"}
+        def check(s, v):
+            self.assertEqual(substitute(s, d), v)
+        check("$name", "value")
+        check(" $name ", " value ")
+        check("${name}", "value")
+        check(" ${name} ", " value ")
+        check("$name$name", "valuevalue")
+        check("$name1$name", "abcvalue")
+        check("$name_$name", "defvalue")
+        check("$_123$name", "ghivalue")
+        check("$name $name", "value value")
+        check("$name1 $name", "abc value")
+        check("$name_ $name", "def value")
+        check("$_123 $name", "ghi value")
+        check("splat", "splat")
+        check("$$", "$")
+        check("$$$name$$", "$value$")
+
+    def test_undefined_names(self):
+        d = {"name": "value"}
+        self.assertRaises(SubstitutionReplacementError,
+                          substitute, "$splat", d)
+        self.assertRaises(SubstitutionReplacementError,
+                          substitute, "$splat1", d)
+        self.assertRaises(SubstitutionReplacementError,
+                          substitute, "$splat_", d)
+
+    def test_syntax_errors(self):
+        d = {"name": "${next"}
+        def check(s):
+            self.assertRaises(SubstitutionSyntaxError,
+                              substitute, s, d)
+        check("${")
+        check("${name")
+        check("${1name}")
+        check("${ name}")
+
+    def test_edge_cases(self):
+        # It's debatable what should happen for these cases, so we'll
+        # follow the lead of the Bourne shell here.
+        def check(s):
+            self.assertRaises(SubstitutionSyntaxError,
+                              substitute, s, {})
+        check("$1")
+        check("$")
+        check("$ stuff")
+
+    def test_non_nesting(self):
+        d = {"name": "$value"}
+        self.assertEqual(substitute("$name", d), "$value")
+
+    def test_isname(self):
+        self.assert_(isname("abc"))
+        self.assert_(isname("abc_def"))
+        self.assert_(isname("_abc"))
+        self.assert_(isname("abc_"))
+        self.assert_(not isname("abc-def"))
+        self.assert_(not isname("-def"))
+        self.assert_(not isname("abc-"))
+        self.assert_(not isname(""))
+
+
+def test_suite():
+    return unittest.makeSuite(SubstitutionTestCase)
+
+if __name__ == '__main__':
+    unittest.main(defaultTest='test_suite')

trunk/src/ZConfig/url.py

+##############################################################################
+#
+# Copyright (c) 2002, 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""urlparse-like helpers that normalize file: URLs.
+
+ZConfig and urllib2 expect file: URLs to consistently use the '//'
+hostpart seperator; the functions here enforce this constraint.
+"""
+
+import urlparse as _urlparse
+
+try:
+    from urlparse import urlsplit
+except ImportError:
+    def urlsplit(url):
+        # Check for the fragment here, since Python 2.1.3 didn't get
+        # it right for things like "http://www.python.org#frag".
+        if '#' in url:
+            url, fragment = url.split('#', 1)
+        else:
+            fragment = ''
+        parts = list(_urlparse.urlparse(url))
+        parts[-1] = fragment
+        param = parts.pop(3)
+        if param:
+            parts[2] += ";" + param
+        return tuple(parts)
+
+
+def urlnormalize(url):
+    lc = url.lower()
+    if lc.startswith("file:/") and not lc.startswith("file:///"):
+        url = "file://" + url[5:]
+    return url
+
+
+def urlunsplit(parts):
+    parts = list(parts)
+    parts.insert(3, '')
+    url = _urlparse.urlunparse(tuple(parts))
+    if (parts[0] == "file"
+        and url.startswith("file:/")
+        and not url.startswith("file:///")):
+        url = "file://" + url[5:]
+    return url
+
+
+def urldefrag(url):
+    url, fragment = _urlparse.urldefrag(url)
+    return urlnormalize(url), fragment
+
+
+def urljoin(base, relurl):
+    url = _urlparse.urljoin(base, relurl)
+    if url.startswith("file:/") and not url.startswith("file:///"):
+        url = "file://" + url[5:]
+    return url

trunk/src/zdaemon/DEPENDENCIES.cfg

+ZConfig

trunk/src/zdaemon/SETUP.cfg

+script  zdctl.py
+script  zdrun.py

trunk/src/zdaemon/__init__.py

+##############################################################################
+#
+# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""zdaemon -- a package to manage a daemon application."""

trunk/src/zdaemon/component.xml

+<component>
+
+  <!-- Note on logging configuration:
+
+       This schema component expects to use a section type named
+       "eventlog"; this type needs to be provided by some other
+       component that the top-level schema needs to import.
+
+       The import is not performed here to allow applications to
+       load the type from different components.
+    -->
+
+  <sectiontype name="runner">
+
+    <description>
+      This section describes the options for zdctl.py and zdrun.py.
+      The only required option is "program".  Many other options have
+      no default value specified in the schema; in some cases, the
+      program calculates a dynamic default, in others, the feature
+      associated with the option is disabled.
+
+      For those options that also have corresponding command-line
+      options, the command line option (short and long form) are given
+      here too.
+    </description>
+
+    <section name="*" type="ZConfig.logger.log"
+             attribute="eventlog"
+             required="no">
+      <description>
+        Log configuration for zdctl.py and zdrun.py.  These
+        applications will normally use the eventlog section at the top
+        level of the configuration, but will use this eventlog section
+        if it exists.
+
+        (This is done so that the combined schema for the runner and
+        the controlled application will write to the same logs by
+        default, but a separation of logs can be achieved if desired.)
+      </description>
+    </section>
+
+    <key name="program" datatype="string-list"
+         required="yes">
+      <description>
+        Command-line option: -p or --program (zdctl.py only).
+
+        This option gives the command used to start the subprocess
+        managed by zdrun.py.  This is currently a simple list of
+        whitespace-delimited words. The first word is the program
+        file, subsequent words are its command line arguments.  If the
+        program file contains no slashes, it is searched using $PATH.
+        (XXX There is no way to to include whitespace in the program
+        file or an argument, and under certain circumstances other
+        shell metacharacters are also a problem, e.g. the "foreground"
+        command of zdctl.py.)
+
+        NOTE: zdrun.py doesn't use this option; it uses its positional
+        arguments.  Rather, zdctl.py uses this option to determine the
+        positional argument with which to invoke zdrun.py.  (XXX This
+        could be better.)
+      </description>
+    </key>
+
+    <key name="python" datatype="existing-path"
+         required="no">
+      <description>
+        Path to the Python interpreter.  Used by zdctl.py to start the
+        zdrun.py process.  Defaults to sys.executable.
+      </description>
+    </key>
+
+    <key name="zdrun" datatype="existing-path"
+         required="no">
+      <description>
+        Path to the zdrun.py script.  Used by zdctl.py to start the
+        zdrun.py process.  Defaults to a file named "zdrun.py" in the
+        same directory as zdctl.py.
+      </description>
+    </key>
+
+    <key name="socket-name" datatype="existing-dirpath"
+         required="no"
+         default="zdsock">
+      <description>
+        Command-line option: -s or --socket-name.
+
+        The pathname of the Unix domain socket used for communication
+        between zdctl.py and zdrun.py.  The default is relative to the
+        current directory in which zdctl.py and zdrun.py are started.
+        You want to specify an absolute pathname here.
+      </description>
+    </key>
+
+    <key name="daemon" datatype="boolean"
+         required="no"
+         default="false">
+      <description>
+        Command-line option: -d or --daemon.
+
+        If this option is true, zdrun.py runs in the background as a
+        true daemon.  It forks a child process which becomes the
+        subprocess manager, while the parent exits (making the shell
+        that started it believe it is done).  The child process also
+        does the following:
+
+        - if the directory option is set, change into that directory
+
+        - redirect stdin, stdout and stderr to /dev/null
+
+        - call setsid() so it becomes a session leader
+
+        - call umask() with specified value
+      </description>
+    </key>
+
+    <key name="directory" datatype="existing-directory"
+         required="no">
+      <description>
+        Command-line option: -z or --directory.
+
+        If the daemon option is true, this option can specify a
+        directory into which zdrun.py changes as part of the
+        "daemonizing".  If the daemon option is false, this option is
+        ignored.
+      </description>
+    </key>
+
+    <key name="backoff-limit" datatype="integer"
+         required="no"
+         default="10">
+      <description>
+        Command-line option: -b or --backoff-limit.
+
+        When the subprocess crashes, zdrun.py inserts a one-second
+        delay before it restarts it.  When the subprocess crashes
+        again right away, the delay is incremented by one second, and
+        so on.  What happens when the delay has reached the value of
+        backoff-limit (in seconds), depends on the value of the
+        forever option.  If forever is false, zdrun.py gives up at
+        this point, and exits.  An always-crashing subprocess will
+        have been restarted exactly backoff-limit times in this case.
+        If forever is true, zdrun.py continues to attempt to restart
+        the process, keeping the delay at backoff-limit seconds.
+
+        If the subprocess stays up for more than backoff-limit
+        seconds, the delay is reset to 1 second.
+      </description>
+    </key>
+
+    <key name="forever" datatype="boolean"
+         required="no"
+         default="false">
+      <description>
+        Command-line option: -f or --forever.
+
+        If this option is true, zdrun.py will keep restarting a
+        crashing subprocess forever.  If it is false, it will give up
+        after backoff-limit crashes in a row.  See the description of
+        backoff-limit for details.
+      </description>
+    </key>
+
+    <key name="exit-codes" datatype="zdaemon.zdoptions.list_of_ints"
+         required="no"
+         default="0,2">
+      <description>
+        Command-line option: -x or --exit-codes.
+
+        If the subprocess exits with an exit status that is equal to
+        one of the integers in this list, zdrun.py will not restart
+        it.  The default list requires some explanation.  Exit status
+        0 is considered a willful successful exit; the ZEO and Zope
+        server processes use this exit status when they want to stop
+        without being restarted.  (Including in response to a
+        SIGTERM.)  Exit status 2 is typically issued for command line
+        syntax errors; in this case, restarting the program will not
+        help!
+
+        NOTE: this mechanism overrides the backoff-limit and forever
+        options; i.e. even if forever is true, a subprocess exit
+        status code in this list makes zdrun.py give up.  To disable
+        this, change the value to an empty list.
+      </description>
+    </key>
+
+    <key name="user" datatype="string"
+         required="no">
+      <description>
+        Command-line option: -u or --user.
+
+        When zdrun.py is started by root, this option specifies the
+        user as who the the zdrun.py process (and hence the daemon
+        subprocess) will run.  This can be a user name or a numeric
+        user id.  Both the user and the group are set from the
+        corresponding password entry, using setuid() and setgid().
+        This is done before zdrun.py does anything else besides
+        parsing its command line arguments.
+
+        NOTE: when zdrun.py is not started by root, specifying this
+        option is an error.  (XXX This may be a mistake.)
+
+        XXX The zdrun.py event log file may be opened *before*
+        setuid() is called.  Is this good or bad?
+      </description>
+    </key>
+
+    <key name="umask" datatype="zdaemon.zdoptions.octal_type"
+         required="no"
+         default="022">
+      <description>
+        Command-line option: -m or --umask.
+
+        When daemon mode is used, this option specifies the octal umask
+        of the subprocess.
+      </description>
+    </key>
+
+    <key name="hang-around" datatype="boolean"
+         required="no"
+         default="false">
+      <description>
+        If this option is true, the zdrun.py process will remain even
+        when the daemon subprocess is stopped.  In this case, zdctl.py
+        will restart zdrun.py as necessary.  If this option is false,
+        zdrun.py will exit when the daemon subprocess is stopped
+        (unless zdrun.py intends to restart it).
+      </description>
+    </key>
+
+    <key name="default-to-interactive" datatype="boolean"
+         required="no"
+         default="true">
+      <description>
+        If this option is true, zdctl.py enters interactive mode
+        when it is invoked without a positional command argument.  If
+        it is false, you must use the -i or --interactive command line
+        option to zdctl.py to enter interactive mode.
+      </description>
+    </key>
+
+    <key name="logfile" datatype="existing-dirpath"
+         required="no">
+      <description>
+        This option specifies a log file that is the default target of
+        the "logtail" zdctl.py command.
+
+        NOTE: This is NOT the log file to which zdrun.py writes its
+        logging messages!  That log file is specified by the
+        &lt;eventlog&gt; section.
+      </description>
+    </key>
+
+    <key name="transcript" datatype="existing-dirpath"
+         required="no">
+      <description>
+        The name of a file in which a transcript of all output from
+        the command being run will be written to when daemonized.
+
+        If not specified, output from the command will be discarded.
+
+        This only takes effect when the "daemon" option is enabled.
+      </description>
+    </key>
+
+    <key name="prompt" datatype="string"
+         required="no">
+       <description>
+         The prompt shown by the controller program.  The default must
+         be provided by the application.
+       </description>
+    </key>
+
+  </sectiontype>
+
+</component>

trunk/src/zdaemon/sample.conf

+# Sample config file for zdctl.py and zdrun.py (which share a schema).
+
+<runner>
+  # Harmless example
+  program       sleep 100
+  # Repeat the defaults
+  backoff-limit 10
+  daemon	True
+  forever	True
+  socket-name	zdsock
+  exit-codes	0,2
+  # user has no default
+  umask		022
+  directory	.
+  default-to-interactive True
+  hang-around   False
+</runner>
+
+<eventlog>
+  level info
+  <logfile>
+    path /tmp/zdrun.log
+  </logfile>
+</eventlog>

trunk/src/zdaemon/schema.xml

+<schema>
+
+  <description>
+    This schema describes various options that control zdctl.py and
+    zdrun.py.  zdrun.py is the "daemon process manager"; it runs a
+    subprocess in the background and restarts it when it crashes.
+    zdctl.py is the user interface to zdrun.py; it can tell zdrun.py
+    to start, stop or restart the subprocess, send it a signal, etc.
+
+    There are two sections: &lt;runner&gt; defines options unique
+    zdctl.py and zdrun.py, and &lt;eventlog&gt; defines a standard
+    event logging section used by zdrun.py.
+
+    More information about zdctl.py and zdrun.py can be found in the
+    file Doc/zdctl.txt.  This all is specific to Unix/Linux.
+  </description>
+
+  <import package="ZConfig.components.logger"/>
+
+  <import package="zdaemon"/>
+
+  <section name="*" type="runner" attribute="runner" required="yes" />
+
+  <section name="*" type="eventlog" attribute="eventlog" required="no" />
+
+</schema>

trunk/src/zdaemon/tests/__init__.py

+# This file is needed to make this a package.

trunk/src/zdaemon/tests/donothing.sh

+#!/bin/sh
+while [ "1" -ne "2" ]; do
+   sleep 10
+done
+
+    

trunk/src/zdaemon/tests/nokill.py

+#! /usr/bin/env python
+
+import signal
+
+signal.signal(signal.SIGTERM, signal.SIG_IGN)
+
+while 1:
+    signal.pause()

trunk/src/zdaemon/tests/parent.py

+import time
+import os
+import sys
+
+def main():
+    # dummy zdctl startup of zdrun
+    shutup()
+    file = os.path.normpath(os.path.abspath(sys.argv[0]))
+    dir = os.path.dirname(file)
+    zctldir = os.path.dirname(dir)
+    zdrun = os.path.join(zctldir, 'zdrun.py')
+    args = [sys.executable, zdrun]
+    args += ['-d', '-b', '10', '-s', os.path.join(dir, 'testsock'),
+             '-x', '0,2', '-z', dir, os.path.join(dir, 'donothing.sh')]
+    flag = os.P_NOWAIT
+    #cmd = ' '.join([sys.executable] + args)
+    #print cmd
+    os.spawnvp(flag, args[0], args)
+    while 1:
+        # wait to be signaled
+        time.sleep(1)
+
+def shutup():
+    os.close(0)
+    sys.stdin = sys.__stdin__ = open("/dev/null")
+    os.close(1)
+    sys.stdout = sys.__stdout__ = open("/dev/null", "w")
+    os.close(2)
+    sys.stderr = sys.__stderr__ = open("/dev/null", "w")
+
+if __name__ == '__main__':
+    main()

trunk/src/zdaemon/tests/testzdoptions.py

+##############################################################################
+#
+# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE
+#
+##############################################################################
+
+"""Test suite for zdaemon.zdoptions."""
+
+import os
+import sys
+import tempfile
+import unittest
+from StringIO import StringIO
+
+import ZConfig
+import zdaemon
+from zdaemon.zdoptions import ZDOptions
+
+class ZDOptionsTestBase(unittest.TestCase):
+
+    OptionsClass = ZDOptions
+
+    def save_streams(self):
+        self.save_stdout = sys.stdout
+        self.save_stderr = sys.stderr
+        sys.stdout = self.stdout = StringIO()
+        sys.stderr = self.stderr = StringIO()
+
+    def restore_streams(self):
+        sys.stdout = self.save_stdout
+        sys.stderr = self.save_stderr
+
+    def check_exit_code(self, options, args):
+        save_sys_stderr = sys.stderr
+        try:
+            sys.stderr = StringIO()
+            try:
+                options.realize(args)
+            except SystemExit, err:
+                self.assertEqual(err.code, 2)
+            else:
+                self.fail("SystemExit expected")
+        finally:
+            sys.stderr = save_sys_stderr
+
+
+class TestZDOptions(ZDOptionsTestBase):
+
+    input_args = ["arg1", "arg2"]
+    output_opts = []
+    output_args = ["arg1", "arg2"]
+
+    def test_basic(self):
+        progname = "progname"
+        doc = "doc"
+        options = self.OptionsClass()
+        options.positional_args_allowed = 1
+        options.schemadir = os.path.dirname(zdaemon.__file__)
+        options.realize(self.input_args, progname, doc)
+        self.assertEqual(options.progname, "progname")
+        self.assertEqual(options.doc, "doc")
+        self.assertEqual(options.options, self.output_opts)
+        self.assertEqual(options.args, self.output_args)
+
+    def test_configure(self):
+        configfile = os.path.join(os.path.dirname(zdaemon.__file__),
+                                  "sample.conf")
+        for arg in "-C", "--c", "--configure":
+            options = self.OptionsClass()
+            options.realize([arg, configfile])
+            self.assertEqual(options.configfile, configfile)
+
+    def test_help(self):
+        for arg in "-h", "--h", "--help":
+            options = self.OptionsClass()
+            try:
+                self.save_streams()
+                try:
+                    options.realize([arg])
+                finally:
+                    self.restore_streams()
+            except SystemExit, err:
+                self.assertEqual(err.code, 0)
+            else:
+                self.fail("%s didn't call sys.exit()" % repr(arg))
+
+    def test_unrecognized(self):
+        # Check that we get an error for an unrecognized option
+        self.check_exit_code(self.OptionsClass(), ["-/"])
+
+
+class TestBasicFunctionality(TestZDOptions):
+
+    def test_no_positional_args(self):
+        # Check that we get an error for positional args when they
+        # haven't been enabled.
+        self.check_exit_code(self.OptionsClass(), ["A"])
+
+    def test_positional_args(self):
+        options = self.OptionsClass()
+        options.positional_args_allowed = 1
+        options.realize(["A", "B"])
+        self.assertEqual(options.args, ["A", "B"])
+
+    def test_positional_args_empty(self):
+        options = self.OptionsClass()
+        options.positional_args_allowed = 1
+        options.realize([])
+        self.assertEqual(options.args, [])
+
+    def test_positional_args_unknown_option(self):
+        # Make sure an unknown option doesn't become a positional arg.
+        options = self.OptionsClass()
+        options.positional_args_allowed = 1
+        self.check_exit_code(options, ["-o", "A", "B"])
+
+    def test_conflicting_flags(self):
+        # Check that we get an error for flags which compete over the
+        # same option setting.
+        options = self.OptionsClass()
+        options.add("setting", None, "a", flag=1)
+        options.add("setting", None, "b", flag=2)
+        self.check_exit_code(options, ["-a", "-b"])
+
+    def test_handler_simple(self):
+        # Test that a handler is called; use one that doesn't return None.
+        options = self.OptionsClass()
+        options.add("setting", None, "a:", handler=int)
+        options.realize(["-a2"])
+        self.assertEqual(options.setting, 2)
+
+    def test_handler_side_effect(self):
+        # Test that a handler is called and conflicts are not
+        # signalled when it returns None.
+        options = self.OptionsClass()
+        L = []
+        options.add("setting", None, "a:", "append=", handler=L.append)
+        options.realize(["-a2", "--append", "3"])
+        self.assert_(options.setting is None)
+        self.assertEqual(L, ["2", "3"])
+
+    def test_handler_with_bad_value(self):
+        options = self.OptionsClass()
+        options.add("setting", None, "a:", handler=int)
+        self.check_exit_code(options, ["-afoo"])
+
+    def test_raise_getopt_errors(self):
+        options = self.OptionsClass()
+        # note that we do not add "a" to the list of options;
+        # if raise_getopt_errors was true, this test would error
+        options.realize(["-afoo"], raise_getopt_errs=False)
+        # check_exit_code realizes the options with raise_getopt_errs=True
+        self.check_exit_code(options, ['-afoo'])
+
+
+class EnvironmentOptions(ZDOptionsTestBase):
+
+    saved_schema = None
+
+    class OptionsClass(ZDOptions):
+        def __init__(self):
+            ZDOptions.__init__(self)
+            self.add("opt", "opt", "o:", "opt=",
+                     default=42, handler=int, env="OPT")
+
+        def load_schema(self):
+            # Doing this here avoids needing a separate file for the schema:
+            if self.schema is None:
+                if EnvironmentOptions.saved_schema is None:
+                    schema = ZConfig.loadSchemaFile(StringIO("""\
+                        <schema>
+                          <key name='opt' datatype='integer' default='12'/>
+                        </schema>
+                        """))
+                    EnvironmentOptions.saved_schema = schema
+                self.schema = EnvironmentOptions.saved_schema
+
+        def load_configfile(self):
+            if getattr(self, "configtext", None):
+                self.configfile = tempfile.mktemp()
+                f = open(self.configfile, 'w')
+                f.write(self.configtext)
+                f.close()
+                try:
+                    ZDOptions.load_configfile(self)
+                finally:
+                    os.unlink(self.configfile)
+            else:
+                ZDOptions.load_configfile(self)
+
+    # Save and restore the environment around each test:
+
+    def setUp(self):
+        self._oldenv = os.environ
+        env = {}
+        for k, v in os.environ.items():
+            env[k] = v
+        os.environ = env
+
+    def tearDown(self):
+        os.environ = self._oldenv
+
+    def create_with_config(self, text):
+        options = self.OptionsClass()
+        zdpkgdir = os.path.dirname(os.path.abspath(zdaemon.__file__))
+        options.schemadir = os.path.join(zdpkgdir, 'tests')
+        options.schemafile = "envtest.xml"
+        # configfile must be set for ZDOptions to use ZConfig:
+        if text:
+            options.configfile = "not used"
+            options.configtext = text
+        return options
+
+
+class TestZDOptionsEnvironment(EnvironmentOptions):
+
+    def test_with_environment(self):
+        os.environ["OPT"] = "2"
+        self.check_from_command_line()
+        options = self.OptionsClass()
+        options.realize([])
+        self.assertEqual(options.opt, 2)
+
+    def test_without_environment(self):
+        self.check_from_command_line()
+        options = self.OptionsClass()
+        options.realize([])
+        self.assertEqual(options.opt, 42)
+
+    def check_from_command_line(self):
+        for args in (["-o1"], ["--opt", "1"]):
+            options = self.OptionsClass()
+            options.realize(args)
+            self.assertEqual(options.opt, 1)
+
+    def test_with_bad_environment(self):
+        os.environ["OPT"] = "Spooge!"
+        # make sure the bad value is ignored if the command-line is used:
+        self.check_from_command_line()
+        options = self.OptionsClass()
+        try:
+            self.save_streams()
+            try:
+                options.realize([])
+            finally:
+                self.restore_streams()
+        except SystemExit, e:
+            self.assertEqual(e.code, 2)
+        else:
+            self.fail("expected SystemExit")
+
+    def test_environment_overrides_configfile(self):
+        options = self.create_with_config("opt 3")
+        options.realize([])
+        self.assertEqual(options.opt, 3)
+
+        os.environ["OPT"] = "2"
+        options = self.create_with_config("opt 3")
+        options.realize([])
+        self.assertEqual(options.opt, 2)
+
+
+class TestCommandLineOverrides(EnvironmentOptions):
+
+    def test_simple_override(self):
+        options = self.create_with_config("# empty config")
+        options.realize(["-X", "opt=-2"])
+        self.assertEqual(options.opt, -2)
+
+    def test_error_propogation(self):
+        self.check_exit_code(self.create_with_config("# empty"),
+                             ["-Xopt=1", "-Xopt=2"])
+        self.check_exit_code(self.create_with_config("# empty"),
+                             ["-Xunknown=foo"])
+
+
+def test_suite():
+    suite = unittest.TestSuite()
+    for cls in [TestBasicFunctionality,
+                TestZDOptionsEnvironment,
+                TestCommandLineOverrides]:
+        suite.addTest(unittest.makeSuite(cls))
+    return suite
+
+if __name__ == "__main__":
+    unittest.main(defaultTest='test_suite')

trunk/src/zdaemon/tests/testzdrun.py

+"""Test suite for zdrun.py."""
+
+import os
+import sys
+import time
+import signal
+import tempfile
+import unittest
+import socket
+
+from StringIO import StringIO
+
+import ZConfig
+
+from zdaemon import zdrun, zdctl
+
+
+class ConfiguredOptions:
+    """Options class that loads configuration from a specified string.
+
+    This always loads from the string, regardless of any -C option
+    that may be given.
+    """
+
+    def set_configuration(self, configuration):
+        self.__configuration = configuration
+        self.configfile = "<preloaded string>"
+
+    def load_configfile(self):
+        sio = StringIO(self.__configuration)
+        cfg = ZConfig.loadConfigFile(self.schema, sio, self.zconfig_options)
+        self.configroot, self.confighandlers = cfg
+
+
+class ConfiguredZDRunOptions(ConfiguredOptions, zdrun.ZDRunOptions):
+
+    def __init__(self, configuration):
+        zdrun.ZDRunOptions.__init__(self)
+        self.set_configuration(configuration)
+
+
+class ZDaemonTests(unittest.TestCase):
+
+    python = os.path.abspath(sys.executable)
+    assert os.path.exists(python)
+    here = os.path.abspath(os.path.dirname(__file__))
+    assert os.path.isdir(here)
+    nokill = os.path.join(here, "nokill.py")
+    assert os.path.exists(nokill)
+    parent = os.path.dirname(here)
+    zdrun = os.path.join(parent, "zdrun.py")
+    assert os.path.exists(zdrun)
+
+    ppath = os.pathsep.join(sys.path)
+
+    def setUp(self):
+        self.zdsock = tempfile.mktemp()
+        self.new_stdout = StringIO()
+        self.save_stdout = sys.stdout
+        sys.stdout = self.new_stdout
+        self.expect = ""
+
+    def tearDown(self):
+        sys.stdout = self.save_stdout
+        for sig in (signal.SIGTERM,
+                    signal.SIGHUP,
+                    signal.SIGINT,
+                    signal.SIGCHLD):
+            signal.signal(sig, signal.SIG_DFL)
+        try:
+            os.unlink(self.zdsock)
+        except os.error:
+            pass
+        output = self.new_stdout.getvalue()
+        self.assertEqual(self.expect, output)
+
+    def quoteargs(self, args):
+        for i in range(len(args)):
+            if " " in args[i]:
+                args[i] = '"%s"' % args[i]
+        return " ".join(args)
+
+    def rundaemon(self, args):
+        # Add quotes, in case some pathname contains spaces (e.g. Mac OS X)
+        args = self.quoteargs(args)
+        cmd = ('PYTHONPATH="%s" "%s" "%s" -d -s "%s" %s' %
+               (self.ppath, self.python, self.zdrun, self.zdsock, args))
+        os.system(cmd)
+        # When the daemon crashes, the following may help debug it:
+        ##os.system("PYTHONPATH=%s %s %s -s %s %s &" %
+        ##    (self.ppath, self.python, self.zdrun, self.zdsock, args))
+
+    def run(self, args):
+        if type(args) is type(""):
+            args = args.split()
+        try:
+            zdctl.main(["-s", self.zdsock] + args)
+        except SystemExit:
+            pass
+
+    def testSystem(self):
+        self.rundaemon(["echo", "-n"])
+        self.expect = ""
+
+##     def testInvoke(self):
+##         self.run("echo -n")
+##         self.expect = ""
+
+##     def testControl(self):
+##         self.rundaemon(["sleep", "1000"])
+##         time.sleep(1)
+##         self.run("stop")
+##         time.sleep(1)
+##         self.run("exit")
+##         self.expect = "Sent SIGTERM\nExiting now\n"
+
+##     def testStop(self):
+##         self.rundaemon([self.python, self.nokill])
+##         time.sleep(1)
+##         self.run("stop")
+##         time.sleep(1)
+##         self.run("exit")
+##         self.expect = "Sent SIGTERM\nSent SIGTERM; will exit later\n"
+
+    def testHelp(self):
+        self.run("-h")
+        import __main__
+        self.expect = __main__.__doc__
+
+    def testOptionsSysArgv(self):
+        # Check that options are parsed from sys.argv by default
+        options = zdrun.ZDRunOptions()
+        save_sys_argv = sys.argv
+        try:
+            sys.argv = ["A", "B", "C"]
+            options.realize()
+        finally:
+            sys.argv = save_sys_argv
+        self.assertEqual(options.options, [])
+        self.assertEqual(options.args, ["B", "C"])
+
+    def testOptionsBasic(self):
+        # Check basic option parsing
+        options = zdrun.ZDRunOptions()
+        options.realize(["B", "C"], "foo")
+        self.assertEqual(options.options, [])
+        self.assertEqual(options.args, ["B", "C"])
+        self.assertEqual(options.progname, "foo")
+
+    def testOptionsHelp(self):
+        # Check that -h behaves properly
+        options = zdrun.ZDRunOptions()
+        try:
+            options.realize(["-h"], doc=zdrun.__doc__)
+        except SystemExit, err:
+            self.failIf(err.code)
+        else:
+            self.fail("SystemExit expected")
+        self.expect = zdrun.__doc__
+
+    def testSubprocessBasic(self):
+        # Check basic subprocess management: spawn, kill, wait
+        options = zdrun.ZDRunOptions()
+        options.realize(["sleep", "100"])
+        proc = zdrun.Subprocess(options)
+        self.assertEqual(proc.pid, 0)
+        pid = proc.spawn()
+        self.assertEqual(proc.pid, pid)
+        msg = proc.kill(signal.SIGTERM)
+        self.assertEqual(msg, None)
+        wpid, wsts = os.waitpid(pid, 0)
+        self.assertEqual(wpid, pid)
+        self.assertEqual(os.WIFSIGNALED(wsts), 1)
+        self.assertEqual(os.WTERMSIG(wsts), signal.SIGTERM)
+        proc.setstatus(wsts)
+        self.assertEqual(proc.pid, 0)
+
+    def testEventlogOverride(self):
+        # Make sure runner.eventlog is used if it exists
+        options = ConfiguredZDRunOptions("""\
+            <runner>
+              program /bin/true
+              <eventlog>
+                level 42
+              </eventlog>
+            </runner>
+
+            <eventlog>
+              level 35
+            </eventlog>
+            """)
+        options.realize(["/bin/true"])
+        self.assertEqual(options.config_logger.level, 42)
+
+    def testEventlogWithoutOverride(self):
+        # Make sure eventlog is used if runner.eventlog doesn't exist
+        options = ConfiguredZDRunOptions("""\
+            <runner>
+              program /bin/true
+            </runner>
+
+            <eventlog>
+              level 35
+            </eventlog>
+            """)
+        options.realize(["/bin/true"])
+        self.assertEqual(options.config_logger.level, 35)
+
+    def testRunIgnoresParentSignals(self):
+        # Spawn a process which will in turn spawn a zdrun process.
+        # We make sure that the zdrun process is still running even if
+        # its parent process receives an interrupt signal (it should
+        # not be passed to zdrun).
+        zdrun_socket = os.path.join(self.here, 'testsock')
+        zdctlpid = os.spawnvp(
+            os.P_NOWAIT,
+            sys.executable,
+            [sys.executable, os.path.join(self.here, 'parent.py')]
+            )
+        time.sleep(2) # race condition possible here
+        os.kill(zdctlpid, signal.SIGINT)
+        try:
+            response = send_action('status\n', zdrun_socket) or ''
+        except socket.error, msg:
+            response = ''
+        params = response.split('\n')
+        self.assert_(len(params) > 1, repr(response))
+        # kill the process
+        send_action('exit\n', zdrun_socket)
+
+    def testUmask(self):
+        path = tempfile.mktemp()
+        # With umask 666, we should create a file that we aren't able
+        # to write.  If access says no, assume that umask works.
+        try:
+            touch_cmd = "/bin/touch"
+            if not os.path.exists(touch_cmd):
+                touch_cmd = "/usr/bin/touch" # Mac OS X
+            self.rundaemon(["-m", "666", touch_cmd, path])
+            for i in range(5):
+                if not os.path.exists(path):
+                    time.sleep(0.1)
+            self.assert_(os.path.exists(path))
+            self.assert_(not os.access(path, os.W_OK))
+        finally:
+            if os.path.exists(path):
+                os.remove(path)
+
+def send_action(action, sockname):
+    """Send an action to the zdrun server and return the response.
+
+    Return None if the server is not up or any other error happened.
+    """
+    sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+    try:
+        sock.connect(sockname)
+        sock.send(action + "\n")
+        sock.shutdown(1) # We're not writing any more
+        response = ""
+        while 1:
+            data = sock.recv(1000)
+            if not data:
+                break
+            response += data
+        sock.close()
+        return response
+    except socket.error, msg:
+        return None
+
+def test_suite():
+    suite = unittest.TestSuite()
+    if os.name == "posix":
+        suite.addTest(unittest.makeSuite(ZDaemonTests))
+    return suite
+
+if __name__ == '__main__':
+    __file__ = sys.argv[0]
+    unittest.main(defaultTest='test_suite')

trunk/src/zdaemon/zdctl.py

+#!python
+##############################################################################
+#
+# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""zdctl -- control an application run by zdaemon.
+
+Usage: python zdctl.py [-C URL] [-S schema.xml] [-h] [-p PROGRAM]
+       [zdrun-options] [action [arguments]]
+
+Options:
+-C/--configure URL -- configuration file or URL
+-S/--schema XML Schema -- XML schema for configuration file
+-h/--help -- print usage message and exit
+-b/--backoff-limit SECONDS -- set backoff limit to SECONDS (default 10)
+-d/--daemon -- run as a proper daemon; fork a subprocess, close files etc.
+-f/--forever -- run forever (by default, exit when backoff limit is exceeded)
+-h/--help -- print this usage message and exit
+-i/--interactive -- start an interactive shell after executing commands
+-l/--logfile -- log file to be read by logtail command
+-p/--program PROGRAM -- the program to run
+-s/--socket-name SOCKET -- Unix socket name for client (default "zdsock")
+-u/--user USER -- run as this user (or numeric uid)
+-m/--umask UMASK -- use this umask for daemon subprocess (default is 022)
+-x/--exit-codes LIST -- list of fatal exit codes (default "0,2")
+-z/--directory DIRECTORY -- directory to chdir to when using -d (default off)
+action [arguments] -- see below
+
+Actions are commands like "start", "stop" and "status".  If -i is
+specified or no action is specified on the command line, a "shell"
+interpreting actions typed interactively is started (unless the
+configuration option default_to_interactive is set to false).  Use the
+action "help" to find out about available actions.
+"""
+
+import os
+import re
+import cmd
+import sys
+import time
+import signal
+import socket
+import stat
+
+if __name__ == "__main__":
+    # Add the parent of the script directory to the module search path
+    # (but only when the script is run from inside the zdaemon package)
+    from os.path import dirname, basename, abspath, normpath
+    scriptdir = dirname(normpath(abspath(sys.argv[0])))
+    if basename(scriptdir).lower() == "zdaemon":
+        sys.path.append(dirname(scriptdir))
+
+from zdaemon.zdoptions import RunnerOptions
+
+
+def string_list(arg):
+    return arg.split()
+
+
+class ZDCtlOptions(RunnerOptions):
+
+    positional_args_allowed = 1
+
+    def __init__(self):
+        RunnerOptions.__init__(self)
+        self.add("schemafile", short="S:", long="schema=",
+                 default="schema.xml",
+                 handler=self.set_schemafile)
+        self.add("interactive", None, "i", "interactive", flag=1)
+        self.add("default_to_interactive", "runner.default_to_interactive",
+                 default=1)
+        self.add("program", "runner.program", "p:", "program=",
+                 handler=string_list,
+                 required="no program specified; use -p or -C")
+        self.add("logfile", "runner.logfile", "l:", "logfile=")
+        self.add("python", "runner.python")
+        self.add("zdrun", "runner.zdrun")
+        programname = os.path.basename(sys.argv[0])
+        base, ext = os.path.splitext(programname)
+        if ext == ".py":
+            programname = base
+        self.add("prompt", "runner.prompt", default=(programname + ">"))
+
+    def realize(self, *args, **kwds):
+        RunnerOptions.realize(self, *args, **kwds)
+
+        # Maybe the config file requires -i or positional args
+        if not self.args and not self.interactive:
+            if not self.default_to_interactive:
+                self.usage("either -i or an action argument is required")
+            self.interactive = 1
+
+        # Where's python?
+        if not self.python:
+            self.python = sys.executable
+
+        # Where's zdrun?
+        if not self.zdrun:
+            if __name__ == "__main__":
+                file = sys.argv[0]
+            else:
+                file = __file__
+            file = os.path.normpath(os.path.abspath(file))
+            dir = os.path.dirname(file)
+            self.zdrun = os.path.join(dir, "zdrun.py")
+
+    def set_schemafile(self, file):
+        self.schemafile = file
+
+
+
+class ZDCmd(cmd.Cmd):
+
+    def __init__(self, options):
+        self.options = options
+        self.prompt = self.options.prompt + ' '
+        cmd.Cmd.__init__(self)
+        self.get_status()
+        if self.zd_status:
+            m = re.search("(?m)^args=(.*)$", self.zd_status)
+            if m:
+                s = m.group(1)
+                args = eval(s, {"__builtins__": {}})
+                if args != self.options.program:
+                    print "WARNING! zdrun is managing a different program!"
+                    print "our program   =", self.options.program
+                    print "daemon's args =", args
+
+    def emptyline(self):
+        # We don't want a blank line to repeat the last command.
+        # Showing status is a nice alternative.
+        self.do_status()
+
+    def send_action(self, action):
+        """Send an action to the zdrun server and return the response.
+
+        Return None if the server is not up or any other error happened.
+        """
+        sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+        try:
+            sock.connect(self.options.sockname)
+            sock.send(action + "\n")
+            sock.shutdown(1) # We're not writing any more
+            response = ""
+            while 1:
+                data = sock.recv(1000)
+                if not data:
+                    break
+                response += data
+            sock.close()
+            return response
+        except socket.error, msg:
+            return None
+
+    def get_status(self):
+        self.zd_up = 0
+        self.zd_pid = 0
+        self.zd_status = None
+        resp = self.send_action("status")
+        if not resp:
+            return
+        m = re.search("(?m)^application=(\d+)$", resp)
+        if not m:
+            return
+        self.zd_up = 1
+        self.zd_pid = int(m.group(1))
+        self.zd_status = resp
+
+    def awhile(self, cond, msg):
+        try:
+            self.get_status()
+            while not cond():
+                sys.stdout.write(". ")
+                sys.stdout.flush()
+                time.sleep(1)
+                self.get_status()
+        except KeyboardInterrupt:
+            print "^C"
+        else:
+            print msg % self.__dict__
+
+    def help_help(self):
+        print "help          -- Print a list of available actions."
+        print "help <action> -- Print help for <action>."
+
+    def do_EOF(self, arg):
+        print
+        return 1
+
+    def help_EOF(self):
+        print "To quit, type ^D or use the quit command."
+
+    def do_start(self, arg):
+        self.get_status()
+        if not self.zd_up:
+            args = [
+                self.options.python,
+                self.options.zdrun,
+                ]
+            args += self._get_override("-S", "schemafile")
+            args += self._get_override("-C", "configfile")
+            args += self._get_override("-b", "backofflimit")
+            args += self._get_override("-d", "daemon", flag=1)
+            args += self._get_override("-f", "forever", flag=1)
+            args += self._get_override("-s", "sockname")
+            args += self._get_override("-u", "user")
+            args += self._get_override("-m", "umask")
+            args += self._get_override(
+                "-x", "exitcodes", ",".join(map(str, self.options.exitcodes)))
+            args += self._get_override("-z", "directory")
+            args.extend(self.options.program)
+            if self.options.daemon:
+                flag = os.P_NOWAIT
+            else:
+                flag = os.P_WAIT
+            os.spawnvp(flag, args[0], args)
+        elif not self.zd_pid:
+            self.send_action("start")
+        else:
+            print "daemon process already running; pid=%d" % self.zd_pid
+            return
+        self.awhile(lambda: self.zd_pid,
+                    "daemon process started, pid=%(zd_pid)d")
+
+    def _get_override(self, opt, name, svalue=None, flag=0):
+        value = getattr(self.options, name)
+        if value is None:
+            return []
+        configroot = self.options.configroot
+        if configroot is not None:
+            for n, cn in self.options.names_list:
+                if n == name and cn:
+                    v = configroot
+                    for p in cn.split("."):
+                        v = getattr(v, p, None)
+                        if v is None:
+                            break
+                    if v == value: # We didn't override anything
+                        return []
+                    break
+        if flag:
+            if value:
+                args = [opt]
+            else:
+                args = []
+        else:
+            if svalue is None:
+                svalue = str(value)
+            args = [opt, svalue]
+        return args
+
+    def help_start(self):
+        print "start -- Start the daemon process."
+        print "         If it is already running, do nothing."
+
+    def do_stop(self, arg):
+        self.get_status()
+        if not self.zd_up:
+            print "daemon manager not running"
+        elif not self.zd_pid:
+            print "daemon process not running"
+        else:
+            self.send_action("stop")
+            self.awhile(lambda: not self.zd_pid, "daemon process stopped")
+
+    def help_stop(self):
+        print "stop -- Stop the daemon process."
+        print "        If it is not running, do nothing."
+
+    def do_restart(self, arg):
+        self.get_status()
+        pid = self.zd_pid
+        if not pid:
+            self.do_start(arg)
+        else:
+            self.send_action("restart")
+            self.awhile(lambda: self.zd_pid not in (0, pid),
+                        "daemon process restarted, pid=%(zd_pid)d")
+
+    def help_restart(self):
+        print "restart -- Stop and then start the daemon process."
+
+    def do_kill(self, arg):
+        if not arg:
+            sig = signal.SIGTERM
+        else:
+            try:
+                sig = int(arg)
+            except: # int() can raise any number of exceptions
+                print "invalid signal number", `arg`
+                return
+        self.get_status()
+        if not self.zd_pid:
+            print "daemon process not running"
+            return
+        print "kill(%d, %d)" % (self.zd_pid, sig)
+        try:
+            os.kill(self.zd_pid, sig)
+        except os.error, msg:
+            print "Error:", msg
+        else:
+            print "signal %d sent to process %d" % (sig, self.zd_pid)
+
+    def help_kill(self):
+        print "kill [sig] -- Send signal sig to the daemon process."
+        print "              The default signal is SIGTERM."
+
+    def do_wait(self, arg):
+        self.awhile(lambda: not self.zd_pid, "daemon process stopped")
+        self.do_status()
+
+    def help_wait(self):
+        print "wait -- Wait for the daemon process to exit."
+
+    def do_status(self, arg=""):
+        if arg not in ["", "-l"]:
+            print "status argument must be absent or -l"
+            return
+        self.get_status()
+        if not self.zd_up:
+            print "daemon manager not running"
+        elif not self.zd_pid:
+            print "daemon manager running; daemon process not running"
+        else:
+            print "program running; pid=%d" % self.zd_pid
+        if arg == "-l" and self.zd_status:
+            print self.zd_status
+
+    def help_status(self):
+        print "status [-l] -- Print status for the daemon process."
+        print "               With -l, show raw status output as well."
+
+    def do_show(self, arg):
+        if not arg:
+            arg = "options"
+        try:
+            method = getattr(self, "show_" + arg)
+        except AttributeError, err:
+            print err
+            self.help_show()
+            return
+        method()
+
+    def show_options(self):
+        print "zdctl/zdrun options:"
+        print "schemafile:  ", repr(self.options.schemafile)
+        print "configfile:  ", repr(self.options.configfile)
+        print "interactive: ", repr(self.options.interactive)
+        print "default_to_interactive:",
+        print                  repr(self.options.default_to_interactive)
+        print "zdrun:       ", repr(self.options.zdrun)
+        print "python:      ", repr(self.options.python)
+        print "program:     ", repr(self.options.program)
+        print "backofflimit:", repr(self.options.backofflimit)
+        print "daemon:      ", repr(self.options.daemon)
+        print "forever:     ", repr(self.options.forever)
+        print "sockname:    ", repr(self.options.sockname)
+        print "exitcodes:   ", repr(self.options.exitcodes)
+        print "user:        ", repr(self.options.user)
+        print "umask:       ", oct(self.options.umask)
+        print "directory:   ", repr(self.options.directory)
+        print "logfile:     ", repr(self.options.logfile)
+        print "hang_around: ", repr(self.options.hang_around)
+
+    def show_python(self):
+        print "Python info:"
+        version = sys.version.replace("\n", "\n              ")
+        print "Version:     ", version
+        print "Platform:    ", sys.platform
+        print "Executable:  ", repr(sys.executable)
+        print "Arguments:   ", repr(sys.argv)
+        print "Directory:   ", repr(os.getcwd())
+        print "Path:"
+        for dir in sys.path:
+            print "    " + repr(dir)
+
+    def show_all(self):
+        self.show_options()
+        print
+        self.show_python()
+
+    def help_show(self):
+        print "show options -- show zdctl options"
+        print "show python -- show Python version and details"
+        print "show all -- show all of the above"
+
+    def complete_show(self, text, *ignored):
+        options = ["options", "python", "all"]
+        return [x for x in options if x.startswith(text)]
+
+    def do_logreopen(self, arg):
+        self.do_kill(str(signal.SIGUSR2))
+
+    def help_logreopen(self):
+        print "logreopen -- Send a SIGUSR2 signal to the daemon process."
+        print "             This is designed to reopen the log file."
+
+    def do_logtail(self, arg):
+        if not arg:
+            arg = self.options.logfile
+            if not arg:
+                print "No default log file specified; use logtail <logfile>"
+                return
+        try:
+            helper = TailHelper(arg)
+            helper.tailf()
+        except KeyboardInterrupt:
+            print
+        except IOError, msg:
+            print msg
+        except OSError, msg:
+            print msg
+
+    def help_logtail(self):
+        print "logtail [logfile] -- Run tail -f on the given logfile."
+        print "                     A default file may exist."
+        print "                     Hit ^C to exit this mode."
+
+    def do_shell(self, arg):
+        if not arg:
+            arg = os.getenv("SHELL") or "/bin/sh"
+        try:
+            os.system(arg)
+        except KeyboardInterrupt:
+            print
+
+    def help_shell(self):
+        print "shell [command] -- Execute a shell command."
+        print "                   Without a command, start an interactive sh."
+        print "An alias for this command is ! [command]"
+
+    def do_reload(self, arg):
+        if arg:
+            args = arg.split()
+            if self.options.configfile:
+                args = ["-C", self.options.configfile] + args
+        else:
+            args = None
+        options = ZDCtlOptions()
+        options.positional_args_allowed = 0
+        try:
+            options.realize(args)
+        except SystemExit:
+            print "Configuration not reloaded"
+        else:
+            self.options = options
+            if self.options.configfile:
+                print "Configuration reloaded from", self.options.configfile
+            else:
+                print "Configuration reloaded without a config file"
+
+    def help_reload(self):
+        print "reload [options] -- Reload the configuration."
+        print "    Without options, this reparses the command line."
+        print "    With options, this substitutes 'options' for the"
+        print "    command line, except that if no -C option is given,"
+        print "    the last configuration file is used."
+
+    def do_foreground(self, arg):
+        self.get_status()
+        pid = self.zd_pid
+        if pid:
+            print "To run the program in the foreground, please stop it first."
+            return
+        program = " ".join(self.options.program)
+        print program
+        try:
+            os.system(program)
+        except KeyboardInterrupt:
+            print
+
+    def do_fg(self, arg):
+        self.do_foreground(arg)
+
+    def help_foreground(self):
+        print "foreground -- Run the program in the forground."
+        print "fg -- an alias for foreground."
+
+    def help_fg(self):
+        self.help_foreground()
+
+    def do_quit(self, arg):
+        self.get_status()
+        if not self.zd_up:
+            print "daemon manager not running"
+        elif not self.zd_pid:
+            print "daemon process not running; stopping daemon manager"
+            self.send_action("exit")
+            self.awhile(lambda: not self.zd_up, "daemon manager stopped")
+        else:
+            print "daemon process and daemon manager still running"
+        return 1
+
+    def help_quit(self):
+        print "quit -- Exit the zdctl shell."
+        print "        If the daemon process is not running,"
+        print "        stop the daemon manager."
+
+
+class TailHelper:
+
+    MAX_BUFFSIZE = 1024
+
+    def __init__(self, fname):
+        self.f = open(fname, 'r')
+
+    def tailf(self):
+        sz, lines = self.tail(10)
+        for line in lines:
+            sys.stdout.write(line)
+            sys.stdout.flush()
+        while 1:
+            newsz = self.fsize()
+            bytes_added = newsz - sz
+            if bytes_added < 0:
+                sz = 0
+                print "==> File truncated <=="
+                bytes_added = newsz
+            if bytes_added > 0:
+                self.f.seek(-bytes_added, 2)
+                bytes = self.f.read(bytes_added)
+                sys.stdout.write(bytes)
+                sys.stdout.flush()
+                sz = newsz
+            time.sleep(1)
+
+    def tail(self, max=10):
+        self.f.seek(0, 2)
+        pos = sz = self.f.tell()
+
+        lines = []
+        bytes = []
+        num_bytes = 0
+
+        while 1:
+            if pos == 0:
+                break
+            self.f.seek(pos)
+            byte = self.f.read(1)
+            if byte == '\n':
+                if len(lines) == max:
+                    break
+                bytes.reverse()
+                line = ''.join(bytes)
+                line and lines.append(line)
+                bytes = []
+            bytes.append(byte)
+            num_bytes = num_bytes + 1
+            if num_bytes > self.MAX_BUFFSIZE:
+                break
+            pos = pos - 1
+        lines.reverse()
+        return sz, lines
+
+    def fsize(self):
+        return os.fstat(self.f.fileno())[stat.ST_SIZE]
+
+def main(args=None, options=None):
+    if options is None:
+        options = ZDCtlOptions()
+    options.realize(args)
+    c = ZDCmd(options)
+    if options.args:
+        c.onecmd(" ".join(options.args))
+    if options.interactive:
+        try:
+            import readline
+        except ImportError:
+            pass
+        print "program:", " ".join(options.program)
+        c.do_status()
+        c.cmdloop()
+
+if __name__ == "__main__":
+    main()

trunk/src/zdaemon/zdoptions.py

+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""Option processing for zdaemon and related code."""
+
+import os
+import sys
+import getopt
+
+import ZConfig
+
+class ZDOptions:
+
+    doc = None
+    progname = None
+    configfile = None
+    schemadir = None
+    schemafile = "schema.xml"
+    schema = None
+    confighandlers = None
+    configroot = None
+
+    # Class variable to control automatic processing of an <eventlog>
+    # section.  This should be the (possibly dotted) name of something
+    # accessible from configroot, typically "eventlog".
+    logsectionname = None
+    config_logger = None # The configured event logger, if any
+
+    # Class variable deciding whether positional arguments are allowed.
+    # If you want positional arguments, set this to 1 in your subclass.
+    positional_args_allowed = 0
+
+    def __init__(self):
+        self.names_list = []
+        self.short_options = []
+        self.long_options = []
+        self.options_map = {}
+        self.default_map = {}
+        self.required_map = {}
+        self.environ_map = {}
+        self.zconfig_options = []
+        self.add(None, None, "h", "help", self.help)
+        self.add("configfile", None, "C:", "configure=")
+        self.add(None, None, "X:", handler=self.zconfig_options.append)
+
+    def help(self, dummy):
+        """Print a long help message (self.doc) to stdout and exit(0).
+
+        Occurrences of "%s" in self.doc are replaced by self.progname.
+        """
+        doc = self.doc
+        if doc.find("%s") > 0:
+            doc = doc.replace("%s", self.progname)
+        print doc,
+        sys.exit(0)
+
+    def usage(self, msg):
+        """Print a brief error message to stderr and exit(2)."""
+        sys.stderr.write("Error: %s\n" % str(msg))
+        sys.stderr.write("For help, use %s -h\n" % self.progname)
+        sys.exit(2)
+
+    def remove(self,
+               name=None,               # attribute name on self
+               confname=None,           # name in ZConfig (may be dotted)
+               short=None,              # short option name
+               long=None,               # long option name
+               ):
+        """Remove all traces of name, confname, short and/or long."""
+        if name:
+            for n, cn in self.names_list[:]:
+                if n == name:
+                    self.names_list.remove((n, cn))
+            if self.default_map.has_key(name):
+                del self.default_map[name]
+            if self.required_map.has_key(name):
+                del self.required_map[name]
+        if confname:
+            for n, cn in self.names_list[:]:
+                if cn == confname:
+                    self.names_list.remove((n, cn))
+        if short:
+            key = "-" + short[0]
+            if self.options_map.has_key(key):
+                del self.options_map[key]
+        if long:
+            key = "--" + long
+            if key[-1] == "=":
+                key = key[:-1]
+            if self.options_map.has_key(key):
+                del self.options_map[key]
+
+    def add(self,
+            name=None,                  # attribute name on self
+            confname=None,              # name in ZConfig (may be dotted)
+            short=None,                 # short option name
+            long=None,                  # long option name
+            handler=None,               # handler (defaults to string)
+            default=None,               # default value
+            required=None,              # message if not provided
+            flag=None,                  # if not None, flag value
+            env=None,                   # if not None, environment variable
+            ):
+        """Add information about a configuration option.
+
+        This can take several forms:
+
+        add(name, confname)
+            Configuration option 'confname' maps to attribute 'name'
+        add(name, None, short, long)
+            Command line option '-short' or '--long' maps to 'name'
+        add(None, None, short, long, handler)
+            Command line option calls handler
+        add(name, None, short, long, handler)
+            Assign handler return value to attribute 'name'
+
+        In addition, one of the following keyword arguments may be given:
+
+        default=...  -- if not None, the default value
+        required=... -- if nonempty, an error message if no value provided
+        flag=...     -- if not None, flag value for command line option
+        env=...      -- if not None, name of environment variable that
+                        overrides the configuration file or default
+        """
+
+        if flag is not None:
+            if handler is not None:
+                raise ValueError, "use at most one of flag= and handler="
+            if not long and not short:
+                raise ValueError, "flag= requires a command line flag"
+            if short and short.endswith(":"):
+                raise ValueError, "flag= requires a command line flag"
+            if long and long.endswith("="):
+                raise ValueError, "flag= requires a command line flag"
+            handler = lambda arg, flag=flag: flag
+
+        if short and long:
+            if short.endswith(":") != long.endswith("="):
+                raise ValueError, "inconsistent short/long options: %r %r" % (
+                    short, long)
+
+        if short:
+            if short[0] == "-":
+                raise ValueError, "short option should not start with '-'"
+            key, rest = short[:1], short[1:]
+            if rest not in ("", ":"):
+                raise ValueError, "short option should be 'x' or 'x:'"
+            key = "-" + key
+            if self.options_map.has_key(key):
+                raise ValueError, "duplicate short option key '%s'" % key
+            self.options_map[key] = (name, handler)
+            self.short_options.append(short)
+
+        if long:
+            if long[0] == "-":
+                raise ValueError, "long option should not start with '-'"
+            key = long
+            if key[-1] == "=":
+                key = key[:-1]
+            key = "--" + key
+            if self.options_map.has_key(key):
+                raise ValueError, "duplicate long option key '%s'" % key
+            self.options_map[key] = (name, handler)
+            self.long_options.append(long)
+
+        if env:
+            self.environ_map[env] = (name, handler)
+
+        if name:
+            if not hasattr(self, name):
+                setattr(self, name, None)
+            self.names_list.append((name, confname))
+            if default is not None:
+                self.default_map[name] = default
+            if required:
+                self.required_map[name] = required
+
+    def realize(self, args=None, progname=None, doc=None,
+                raise_getopt_errs=True):
+        """Realize a configuration.
+
+        Optional arguments:
+
+        args     -- the command line arguments, less the program name
+                    (default is sys.argv[1:])
+
+        progname -- the program name (default is sys.argv[0])
+
+        doc      -- usage message (default is __main__.__doc__)
+        """
+
+         # Provide dynamic default method arguments
+        if args is None:
+            args = sys.argv[1:]
+        if progname is None:
+            progname = sys.argv[0]
+        if doc is None:
+            import __main__
+            doc = __main__.__doc__
+        self.progname = progname
+        self.doc = doc
+
+        self.options = []
+        self.args = []
+
+        # Call getopt
+        try:
+            self.options, self.args = getopt.getopt(
+                args, "".join(self.short_options), self.long_options)
+        except getopt.error, msg:
+            if raise_getopt_errs:
+                self.usage(msg)
+
+        # Check for positional args
+        if self.args and not self.positional_args_allowed:
+            self.usage("positional arguments are not supported")
+
+        # Process options returned by getopt
+        for opt, arg in self.options:
+            name, handler = self.options_map[opt]
+            if handler is not None:
+                try:
+                    arg = handler(arg)
+                except ValueError, msg:
+                    self.usage("invalid value for %s %r: %s" % (opt, arg, msg))
+            if name and arg is not None:
+                if getattr(self, name) is not None:
+                    self.usage("conflicting command line option %r" % opt)
+                setattr(self, name, arg)
+
+        # Process environment variables
+        for envvar in self.environ_map.keys():
+            name, handler = self.environ_map[envvar]
+            if name and getattr(self, name, None) is not None:
+                continue
+            if os.environ.has_key(envvar):
+                value = os.environ[envvar]
+                if handler is not None:
+                    try:
+                        value = handler(value)
+                    except ValueError, msg:
+                        self.usage("invalid environment value for %s %r: %s"
+                                   % (envvar, value, msg))
+                if name and value is not None:
+                    setattr(self, name, value)
+
+        if self.configfile is None:
+            self.configfile = self.default_configfile()
+        if self.zconfig_options and self.configfile is None:
+            self.usage("configuration overrides (-X) cannot be used"
+                       " without a configuration file")
+        if self.configfile is not None:
+            # Process config file
+            self.load_schema()
+            try:
+                self.load_configfile()
+            except ZConfig.ConfigurationError, msg:
+                self.usage(str(msg))
+
+        # Copy config options to attributes of self.  This only fills
+        # in options that aren't already set from the command line.
+        for name, confname in self.names_list:
+            if confname and getattr(self, name) is None:
+                parts = confname.split(".")
+                obj = self.configroot
+                for part in parts:
+                    if obj is None:
+                        break
+                    # Here AttributeError is not a user error!
+                    obj = getattr(obj, part)
+                setattr(self, name, obj)
+
+        # Process defaults
+        for name, value in self.default_map.items():
+            if getattr(self, name) is None:
+                setattr(self, name, value)
+
+        # Process required options
+        for name, message in self.required_map.items():
+            if getattr(self, name) is None:
+                self.usage(message)
+
+        if self.logsectionname:
+            self.load_logconf(self.logsectionname)
+
+    def default_configfile(self):
+        """Return the name of the default config file, or None."""
+        # This allows a default configuration file to be used without
+        # affecting the -C command line option; setting self.configfile
+        # before calling realize() makes the -C option unusable since
+        # then realize() thinks it has already seen the option.  If no
+        # -C is used, realize() will call this method to try to locate
+        # a configuration file.
+        return None
+
+    def load_schema(self):
+        if self.schema is None:
+            # Load schema
+            if self.schemadir is None:
+                self.schemadir = os.path.dirname(__file__)
+            self.schemafile = os.path.join(self.schemadir, self.schemafile)
+            self.schema = ZConfig.loadSchema(self.schemafile)
+
+    def load_configfile(self):
+        self.configroot, self.confighandlers = \
+            ZConfig.loadConfig(self.schema, self.configfile,
+                               self.zconfig_options)
+
+    def load_logconf(self, sectname="eventlog"):
+        parts = sectname.split(".")
+        obj = self.configroot
+        for p in parts:
+            if obj == None:
+                break
+            obj = getattr(obj, p)
+        self.config_logger = obj
+        if obj is not None:
+            obj.startup()
+
+
+class RunnerOptions(ZDOptions):
+
+    uid = gid = None
+
+    def __init__(self):
+        ZDOptions.__init__(self)
+        self.add("backofflimit", "runner.backoff_limit",
+                 "b:", "backoff-limit=", int, default=10)
+        self.add("daemon", "runner.daemon", "d", "daemon", flag=1, default=0)
+        self.add("forever", "runner.forever", "f", "forever",
+                 flag=1, default=0)
+        self.add("sockname", "runner.socket_name", "s:", "socket-name=",
+                 ZConfig.datatypes.existing_dirpath, default="zdsock")
+        self.add("exitcodes", "runner.exit_codes", "x:", "exit-codes=",
+                 list_of_ints, default=[0, 2])
+        self.add("user", "runner.user", "u:", "user=")
+        self.add("umask", "runner.umask", "m:", "umask=", octal_type,
+                 default=022)
+        self.add("directory", "runner.directory", "z:", "directory=",
+                 ZConfig.datatypes.existing_directory)
+        self.add("hang_around", "runner.hang_around", default=0)
+
+    def realize(self, *args, **kwds):
+        ZDOptions.realize(self, *args, **kwds)
+
+        # Additional checking of user option; set uid and gid
+        if self.user is not None:
+            import pwd
+            try:
+                uid = int(self.user)
+            except ValueError:
+                try:
+                    pwrec = pwd.getpwnam(self.user)
+                except KeyError:
+                    self.usage("username %r not found" % self.user)
+                uid = pwrec[2]
+            else:
+                try:
+                    pwrec = pwd.getpwuid(uid)
+                except KeyError:
+                    self.usage("uid %r not found" % self.user)
+            gid = pwrec[3]
+            self.uid = uid
+            self.gid = gid
+
+
+# ZConfig datatype
+
+def list_of_ints(arg):
+    if not arg:
+        return []
+    else:
+        return map(int, arg.split(","))
+
+def octal_type(arg):
+    return int(arg, 8)
+
+
+def _test():
+    # Stupid test program
+    z = ZDOptions()
+    z.add("program", "zdctl.program", "p:", "program=")
+    print z.names_list
+    z.realize()
+    names = z.names_list[:]
+    names.sort()
+    for name, confname in names:
+        print "%-20s = %.56r" % (name, getattr(z, name))
+
+if __name__ == "__main__":
+    __file__ = sys.argv[0]
+    _test()

trunk/src/zdaemon/zdrun.py

+#!python
+##############################################################################
+#
+# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE
+#
+##############################################################################
+"""zrdun -- run an application as a daemon.
+
+Usage: python zrdun.py [zrdun-options] program [program-arguments]
+
+Options:
+-C/--configure URL -- configuration file or URL
+-S/--schema XML Schema -- XML schema for configuration file
+-b/--backoff-limit SECONDS -- set backoff limit to SECONDS (default 10)
+-d/--daemon -- run as a proper daemon; fork a subprocess, setsid(), etc.
+-f/--forever -- run forever (by default, exit when backoff limit is exceeded)
+-h/--help -- print this usage message and exit
+-s/--socket-name SOCKET -- Unix socket name for client (default "zdsock")
+-u/--user USER -- run as this user (or numeric uid)
+-m/--umask UMASK -- use this umask for daemon subprocess (default is 022)
+-t/--transcript FILE -- transript of output from daemon-mode program
+-x/--exit-codes LIST -- list of fatal exit codes (default "0,2")
+-z/--directory DIRECTORY -- directory to chdir to when using -d (default off)
+program [program-arguments] -- an arbitrary application to run
+
+This daemon manager has two purposes: it restarts the application when
+it dies, and (when requested to do so with the -d option) it runs the
+application in the background, detached from the foreground tty
+session that started it (if any).
+
+Exit codes: if at any point the application exits with an exit status
+listed by the -x option, it is not restarted.  Any other form of
+termination (either being killed by a signal or exiting with an exit
+status not listed in the -x option) causes it to be restarted.
+
+Backoff limit: when the application exits (nearly) immediately after a
+restart, the daemon manager starts slowing down by delaying between
+restarts.  The delay starts at 1 second and is increased by one on
+each restart up to the backoff limit given by the -b option; it is
+reset when the application runs for more than the backoff limit
+seconds.  By default, when the delay reaches the backoff limit, the
+daemon manager exits (under the assumption that the application has a
+persistent fault).  The -f (forever) option prevents this exit; use it
+when you expect that a temporary external problem (such as a network
+outage or an overfull disk) may prevent the application from starting
+but you want the daemon manager to keep trying.
+"""
+
+"""
+XXX TO DO
+
+- Finish OO design -- use multiple classes rather than folding
+  everything into one class.
+
+- Add unit tests.
+
+- Add doc strings.
+
+"""
+
+import os
+import sys
+import time
+import errno
+import logging
+import socket
+import select
+import signal
+from stat import ST_MODE
+
+if __name__ == "__main__":
+    # Add the parent of the script directory to the module search path
+    # (but only when the script is run from inside the zdaemon package)
+    from os.path import dirname, basename, abspath, normpath
+    scriptdir = dirname(normpath(abspath(sys.argv[0])))
+    if basename(scriptdir).lower() == "zdaemon":
+        sys.path.append(dirname(scriptdir))
+
+from zdaemon.zdoptions import RunnerOptions
+
+
+class ZDRunOptions(RunnerOptions):
+
+    positional_args_allowed = 1
+    logsectionname = "runner.eventlog"
+    program = None
+
+    def __init__(self):
+        RunnerOptions.__init__(self)
+        self.add("schemafile", short="S:", long="schema=",
+                 default="schema.xml",
+                 handler=self.set_schemafile)
+        self.add("transcript", "runner.transcript", "t:", "transcript=",
+                 default="/dev/null")
+
+    def set_schemafile(self, file):
+        self.schemafile = file
+
+    def realize(self, *args, **kwds):
+        RunnerOptions.realize(self, *args, **kwds)
+        if self.args:
+            self.program = self.args
+        if not self.program:
+            self.usage("no program specified (use -C or positional args)")
+        if self.sockname:
+            # Convert socket name to absolute path
+            self.sockname = os.path.abspath(self.sockname)
+        if self.config_logger is None:
+            # This doesn't perform any configuration of the logging
+            # package, but that's reasonable in this case.
+            self.logger = logging.getLogger()
+        else:
+            self.logger = self.config_logger()
+
+    def load_logconf(self, sectname):
+        """Load alternate eventlog if the specified section isn't present."""
+        RunnerOptions.load_logconf(self, sectname)
+        if self.config_logger is None and sectname != "eventlog":
+            RunnerOptions.load_logconf(self, "eventlog")
+
+
+class Subprocess:
+
+    """A class to manage a subprocess."""
+
+    # Initial state; overridden by instance variables
+    pid = 0 # Subprocess pid; 0 when not running
+    lasttime = 0 # Last time the subprocess was started; 0 if never
+
+    def __init__(self, options, args=None):
+        """Constructor.
+
+        Arguments are a ZDRunOptions instance and a list of program
+        arguments; the latter's first item must be the program name.
+        """
+        if args is None:
+            args = options.args
+        if not args:
+            options.usage("missing 'program' argument")
+        self.options = options
+        self.args = args
+        self._set_filename(args[0])
+
+    def _set_filename(self, program):
+        """Internal: turn a program name into a file name, using $PATH."""
+        if "/" in program:
+            filename = program
+            try:
+                st = os.stat(filename)
+            except os.error:
+                self.options.usage("can't stat program %r" % program)
+        else:
+            path = get_path()
+            for dir in path:
+                filename = os.path.join(dir, program)
+                try:
+                    st = os.stat(filename)
+                except os.error:
+                    continue
+                mode = st[ST_MODE]
+                if mode & 0111:
+                    break
+            else:
+                self.options.usage("can't find program %r on PATH %s" %
+                                   (program, path))
+        if not os.access(filename, os.X_OK):
+            self.options.usage("no permission to run program %r" % filename)
+        self.filename = filename
+
+    def spawn(self):
+        """Start the subprocess.  It must not be running already.
+
+        Return the process id.  If the fork() call fails, return 0.
+        """
+        assert not self.pid
+        self.lasttime = time.time()
+        try:
+            pid = os.fork()
+        except os.error:
+            return 0
+        if pid != 0:
+            # Parent
+            self.pid = pid
+            self.options.logger.info("spawned process pid=%d" % pid)
+            return pid
+        else:
+            # Child
+            try:
+                # Close file descriptors except std{in,out,err}.
+                # XXX We don't know how many to close; hope 100 is plenty.
+                for i in range(3, 100):
+                    try:
+                        os.close(i)
+                    except os.error:
+                        pass
+                try:
+                    os.execv(self.filename, self.args)
+                except os.error, err:
+                    sys.stderr.write("can't exec %r: %s\n" %
+                                     (self.filename, err))
+            finally:
+                os._exit(127)
+            # Does not return
+
+    def kill(self, sig):
+        """Send a signal to the subprocess.  This may or may not kill it.
+
+        Return None if the signal was sent, or an error message string
+        if an error occurred or if the subprocess is not running.
+        """
+        if not self.pid:
+            return "no subprocess running"
+        try:
+            os.kill(self.pid, sig)
+        except os.error, msg:
+            return str(msg)
+        return None
+
+    def setstatus(self, sts):
+        """Set process status returned by wait() or waitpid().
+
+        This simply notes the fact that the subprocess is no longer
+        running by setting self.pid to 0.
+        """
+        self.pid = 0
+
+
+class Daemonizer:
+
+    def main(self, args=None):
+        self.options = ZDRunOptions()
+        self.options.realize(args)
+        self.logger = self.options.logger
+        self.set_uid()
+        self.run()
+
+    def set_uid(self):
+        if self.options.uid is None:
+            return
+        uid = os.geteuid()
+        if uid != 0 and uid != self.options.uid:
+            self.options.usage("only root can use -u USER to change users")
+        os.setgid(self.options.gid)
+        os.setuid(self.options.uid)
+
+    def run(self):
+        self.proc = Subprocess(self.options)
+        self.opensocket()
+        try:
+            self.setsignals()
+            if self.options.daemon:
+                self.daemonize()
+            self.runforever()
+        finally:
+            try:
+                os.unlink(self.options.sockname)
+            except os.error:
+                pass
+
+    mastersocket = None
+    commandsocket = None
+
+    def opensocket(self):
+        sockname = self.options.sockname
+        tempname = "%s.%d" % (sockname, os.getpid())
+        self.unlink_quietly(tempname)
+        while 1:
+            sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+            try:
+                sock.bind(tempname)
+                os.chmod(tempname, 0700)
+                try:
+                    os.link(tempname, sockname)
+                    break
+                except os.error:
+                    # Lock contention, or stale socket.
+                    self.checkopen()
+                    # Stale socket -- delete, sleep, and try again.
+                    msg = "Unlinking stale socket %s; sleep 1" % sockname
+                    sys.stderr.write(msg + "\n")
+                    self.logger.warn(msg)
+                    self.unlink_quietly(sockname)
+                    sock.close()
+                    time.sleep(1)
+                    continue
+            finally:
+                self.unlink_quietly(tempname)
+        sock.listen(1)
+        sock.setblocking(0)
+        self.mastersocket = sock
+
+    def unlink_quietly(self, filename):
+        try:
+            os.unlink(filename)
+        except os.error:
+            pass
+
+    def checkopen(self):
+        s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+        try:
+            s.connect(self.options.sockname)
+            s.send("status\n")
+            data = s.recv(1000)
+            s.close()
+        except socket.error:
+            pass
+        else:
+            while data.endswith("\n"):
+                data = data[:-1]
+            msg = ("Another zrdun is already up using socket %r:\n%s" %
+                   (self.options.sockname, data))
+            sys.stderr.write(msg + "\n")
+            self.logger.critical(msg)
+            sys.exit(1)
+
+    def setsignals(self):
+        signal.signal(signal.SIGTERM, self.sigexit)
+        signal.signal(signal.SIGHUP, self.sigexit)
+        signal.signal(signal.SIGINT, self.sigexit)
+        signal.signal(signal.SIGCHLD, self.sigchild)
+
+    def sigexit(self, sig, frame):
+        self.logger.critical("daemon manager killed by %s" % signame(sig))
+        sys.exit(1)
+
+    waitstatus = None
+
+    def sigchild(self, sig, frame):
+        try:
+            pid, sts = os.waitpid(-1, os.WNOHANG)
+        except os.error:
+            return
+        if pid:
+            self.waitstatus = pid, sts
+
+    def daemonize(self):
+
+        # To daemonize, we need to become the leader of our own session
+        # (process) group.  If we do not, signals sent to our
+        # parent process will also be sent to us.   This might be bad because
+        # signals such as SIGINT can be sent to our parent process during
+        # normal (uninteresting) operations such as when we press Ctrl-C in the
+        # parent terminal window to escape from a logtail command.
+        # To disassociate ourselves from our parent's session group we use
+        # os.setsid.  It means "set session id", which has the effect of
+        # disassociating a process from is current session and process group
+        # and setting itself up as a new session leader.
+        #
+        # Unfortunately we cannot call setsid if we're already a session group
+        # leader, so we use "fork" to make a copy of ourselves that is
+        # guaranteed to not be a session group leader.
+        #
+        # We also change directories, set stderr and stdout to null, and
+        # change our umask.
+        #
+        # This explanation was (gratefully) garnered from
+        # http://www.hawklord.uklinux.net/system/daemons/d3.htm
+
+        pid = os.fork()
+        if pid != 0:
+            # Parent
+            self.logger.debug("daemon manager forked; parent exiting")
+            os._exit(0)
+        # Child
+        self.logger.info("daemonizing the process")
+        if self.options.directory:
+            try:
+                os.chdir(self.options.directory)
+            except os.error, err:
+                self.logger.warn("can't chdir into %r: %s"
+                                 % (self.options.directory, err))
+            else:
+                self.logger.info("set current directory: %r"
+                                 % self.options.directory)
+        os.close(0)
+        sys.stdin = sys.__stdin__ = open("/dev/null")
+        os.close(1)
+        sys.stdout = sys.__stdout__ = open(self.options.transcript, "a", 0)
+        os.close(2)
+        sys.stderr = sys.__stderr__ = open(self.options.transcript, "a", 0)
+        os.setsid()
+        os.umask(self.options.umask)
+        # XXX Stevens, in his Advanced Unix book, section 13.3 (page
+        # 417) recommends calling umask(0) and closing unused
+        # file descriptors.  In his Network Programming book, he
+        # additionally recommends ignoring SIGHUP and forking again
+        # after the setsid() call, for obscure SVR4 reasons.
+
+    mood = 1 # 1: up, 0: down, -1: suicidal
+    delay = 0 # If nonzero, delay starting or killing until this time
+    killing = 0 # If true, send SIGKILL when delay expires
+    proc = None # Subprocess instance
+
+    def runforever(self):
+        self.logger.info("daemon manager started")
+        min_mood = not self.options.hang_around
+        while self.mood >= min_mood or self.proc.pid:
+            if self.mood > 0 and not self.proc.pid and not self.delay:
+                pid = self.proc.spawn()
+                if not pid:
+                    # Can't fork.  Try again later...
+                    self.delay = time.time() + self.backofflimit
+            if self.waitstatus:
+                self.reportstatus()
+            r, w, x = [self.mastersocket], [], []
+            if self.commandsocket:
+                r.append(self.commandsocket)
+            timeout = self.options.backofflimit
+            if self.delay:
+                timeout = max(0, min(timeout, self.delay - time.time()))
+                if timeout <= 0:
+                    self.delay = 0
+                    if self.killing and self.proc.pid:
+                        self.proc.kill(signal.SIGKILL)
+                        self.delay = time.time() + self.options.backofflimit
+            try:
+                r, w, x = select.select(r, w, x, timeout)
+            except select.error, err:
+                if err[0] != errno.EINTR:
+                    raise
+                r = w = x = []
+            if self.waitstatus:
+                self.reportstatus()
+            if self.commandsocket and self.commandsocket in r:
+                try:
+                    self.dorecv()
+                except socket.error, msg:
+                    self.logger.exception("socket.error in dorecv(): %s"
+                                          % str(msg))
+                    self.commandsocket = None
+            if self.mastersocket in r:
+                try:
+                    self.doaccept()
+                except socket.error, msg:
+                    self.logger.exception("socket.error in doaccept(): %s"
+                                          % str(msg))
+                    self.commandsocket = None
+        self.logger.info("Exiting")
+        sys.exit(0)
+
+    def reportstatus(self):
+        pid, sts = self.waitstatus
+        self.waitstatus = None
+        es, msg = decode_wait_status(sts)
+        msg = "pid %d: " % pid + msg
+        if pid != self.proc.pid:
+            msg = "unknown " + msg
+            self.logger.warn(msg)
+        else:
+            killing = self.killing
+            if killing:
+                self.killing = 0
+                self.delay = 0
+            else:
+                self.governor()
+            self.proc.setstatus(sts)
+            if es in self.options.exitcodes and not killing:
+                msg = msg + "; exiting now"
+                self.logger.info(msg)
+                sys.exit(es)
+            self.logger.info(msg)
+
+    backoff = 0
+
+    def governor(self):
+        # Back off if respawning too frequently
+        now = time.time()
+        if not self.proc.lasttime:
+            pass
+        elif now - self.proc.lasttime < self.options.backofflimit:
+            # Exited rather quickly; slow down the restarts
+            self.backoff += 1
+            if self.backoff >= self.options.backofflimit:
+                if self.options.forever:
+                    self.backoff = self.options.backofflimit
+                else:
+                    self.logger.critical("restarting too frequently; quit")
+                    sys.exit(1)
+            self.logger.info("sleep %s to avoid rapid restarts" % self.backoff)
+            self.delay = now + self.backoff
+        else:
+            # Reset the backoff timer
+            self.backoff = 0
+            self.delay = 0
+
+    def doaccept(self):
+        if self.commandsocket:
+            # Give up on previous command socket!
+            self.sendreply("Command superseded by new command")
+            self.commandsocket.close()
+            self.commandsocket = None
+        self.commandsocket, addr = self.mastersocket.accept()
+        self.commandbuffer = ""
+
+    def dorecv(self):
+        data = self.commandsocket.recv(1000)
+        if not data:
+            self.sendreply("Command not terminated by newline")
+            self.commandsocket.close()
+            self.commandsocket = None
+        self.commandbuffer += data
+        if "\n" in self.commandbuffer:
+            self.docommand()
+            self.commandsocket.close()
+            self.commandsocket = None
+        elif len(self.commandbuffer) > 10000:
+            self.sendreply("Command exceeds 10 KB")
+            self.commandsocket.close()
+            self.commandsocket = None
+
+    def docommand(self):
+        lines = self.commandbuffer.split("\n")
+        args = lines[0].split()
+        if not args:
+            self.sendreply("Empty command")
+            return
+        command = args[0]
+        methodname = "cmd_" + command
+        method = getattr(self, methodname, None)
+        if method:
+            method(args)
+        else:
+            self.sendreply("Unknown command %r; 'help' for a list" % args[0])
+
+    def cmd_start(self, args):
+        self.mood = 1 # Up
+        self.backoff = 0
+        self.delay = 0
+        self.killing = 0
+        if not self.proc.pid:
+            self.proc.spawn()
+            self.sendreply("Application started")
+        else:
+            self.sendreply("Application already started")
+
+    def cmd_stop(self, args):
+        self.mood = 0 # Down
+        self.backoff = 0
+        self.delay = 0
+        self.killing = 0
+        if self.proc.pid:
+            self.proc.kill(signal.SIGTERM)
+            self.sendreply("Sent SIGTERM")
+            self.killing = 1
+            self.delay = time.time() + self.options.backofflimit
+        else:
+            self.sendreply("Application already stopped")
+
+    def cmd_restart(self, args):
+        self.mood = 1 # Up
+        self.backoff = 0
+        self.delay = 0
+        self.killing = 0
+        if self.proc.pid:
+            self.proc.kill(signal.SIGTERM)
+            self.sendreply("Sent SIGTERM; will restart later")
+            self.killing = 1
+            self.delay = time.time() + self.options.backofflimit
+        else:
+            self.proc.spawn()
+            self.sendreply("Application started")
+
+    def cmd_exit(self, args):
+        self.mood = -1 # Suicidal
+        self.backoff = 0
+        self.delay = 0
+        self.killing = 0
+        if self.proc.pid:
+            self.proc.kill(signal.SIGTERM)
+            self.sendreply("Sent SIGTERM; will exit later")
+            self.killing = 1
+            self.delay = time.time() + self.options.backofflimit
+        else:
+            self.sendreply("Exiting now")
+            self.logger.info("Exiting")
+            sys.exit(0)
+
+    def cmd_kill(self, args):
+        if args[1:]:
+            try:
+                sig = int(args[1])
+            except:
+                self.sendreply("Bad signal %r" % args[1])
+                return
+        else:
+            sig = signal.SIGTERM
+        if not self.proc.pid:
+            self.sendreply("Application not running")
+        else:
+            msg = self.proc.kill(sig)
+            if msg:
+                self.sendreply("Kill %d failed: %s" % (sig, msg))
+            else:
+                self.sendreply("Signal %d sent" % sig)
+
+    def cmd_status(self, args):
+        if not self.proc.pid:
+            status = "stopped"
+        else:
+            status = "running"
+        self.sendreply("status=%s\n" % status +
+                       "now=%r\n" % time.time() +
+                       "mood=%d\n" % self.mood +
+                       "delay=%r\n" % self.delay +
+                       "backoff=%r\n" % self.backoff +
+                       "lasttime=%r\n" % self.proc.lasttime +
+                       "application=%r\n" % self.proc.pid +
+                       "manager=%r\n" % os.getpid() +
+                       "backofflimit=%r\n" % self.options.backofflimit +
+                       "filename=%r\n" % self.proc.filename +
+                       "args=%r\n" % self.proc.args)
+
+    def cmd_help(self, args):
+        self.sendreply(
+            "Available commands:\n"
+            "  help -- return command help\n"
+            "  status -- report application status (default command)\n"
+            "  kill [signal] -- send a signal to the application\n"
+            "                   (default signal is SIGTERM)\n"
+            "  start -- start the application if not already running\n"
+            "  stop -- stop the application if running\n"
+            "          (the daemon manager keeps running)\n"
+            "  restart -- stop followed by start\n"
+            "  exit -- stop the application and exit\n"
+            )
+
+    def sendreply(self, msg):
+        try:
+            if not msg.endswith("\n"):
+                msg = msg + "\n"
+            if hasattr(self.commandsocket, "sendall"):
+                self.commandsocket.sendall(msg)
+            else:
+                # This is quadratic, but msg is rarely more than 100 bytes :-)
+                while msg:
+                    sent = self.commandsocket.send(msg)
+                    msg = msg[sent:]
+        except socket.error, msg:
+            self.logger.warn("Error sending reply: %s" % str(msg))
+
+
+# Helpers for dealing with signals and exit status
+
+def decode_wait_status(sts):
+    """Decode the status returned by wait() or waitpid().
+
+    Return a tuple (exitstatus, message) where exitstatus is the exit
+    status, or -1 if the process was killed by a signal; and message
+    is a message telling what happened.  It is the caller's
+    responsibility to display the message.
+    """
+    if os.WIFEXITED(sts):
+        es = os.WEXITSTATUS(sts) & 0xffff
+        msg = "exit status %s" % es
+        return es, msg
+    elif os.WIFSIGNALED(sts):
+        sig = os.WTERMSIG(sts)
+        msg = "terminated by %s" % signame(sig)
+        if hasattr(os, "WCOREDUMP"):
+            iscore = os.WCOREDUMP(sts)
+        else:
+            iscore = sts & 0x80
+        if iscore:
+            msg += " (core dumped)"
+        return -1, msg
+    else:
+        msg = "unknown termination cause 0x%04x" % sts
+        return -1, msg
+
+_signames = None
+
+def signame(sig):
+    """Return a symbolic name for a signal.
+
+    Return "signal NNN" if there is no corresponding SIG name in the
+    signal module.
+    """
+
+    if _signames is None:
+        _init_signames()
+    return _signames.get(sig) or "signal %d" % sig
+
+def _init_signames():
+    global _signames
+    d = {}
+    for k, v in signal.__dict__.items():
+        k_startswith = getattr(k, "startswith", None)
+        if k_startswith is None:
+            continue
+        if k_startswith("SIG") and not k_startswith("SIG_"):
+            d[v] = k
+    _signames = d
+
+def get_path():
+    """Return a list corresponding to $PATH, or a default."""
+    path = ["/bin", "/usr/bin", "/usr/local/bin"]
+    if os.environ.has_key("PATH"):
+        p = os.environ["PATH"]
+        if p:
+            path = p.split(os.pathsep)
+    return path
+
+# Main program
+def main(args=None):
+    assert os.name == "posix", "This code makes many Unix-specific assumptions"
+
+    d = Daemonizer()
+    d.main(args)
+
+if __name__ == "__main__":
+    main()