Skip to content

Z
Zope
Commit b795bc99 authored by Chris McDonough's avatar Chris McDonough
Browse files
Options

Work to simplify mounting: 

- Remove the DBTab package (moved code into Zope2.Startup.datatypes)

- Consolidate Mount.py code into MountedObject.py in ZODBMountPoint.

- Remove cant-work-now autoClassFactory.

- Removed test that was dependent on old mounting implementation.
parent 84ae4ea7
Hide whitespace changes
Inline Side-by-side
Showing with 193 additions and 499 deletions
lib/python/DBTab/CHANGES.txt deleted 100644 → 0
View file @ 84ae4ea7
HEAD
- Merged into Zope 2.7+. Split into two pieces: the ZODBMountPoint
Product and the DBTab package. Neither custom_zodb.py nor dbtab.conf
is now required (all configuration is performed via zope.conf).
Version 1.2.1
- Began unit tests.
- Fixed a race condition on connection close. The symptom was
spurious "Should not load state when connection closed" errors under
high load.
- DemoStorage configurations can now include a base_type option,
taking advantage of DemoStorage's layering feature.
- The mount status page (visible by selecting "Add DBTab Mount Point")
sometimes indicated there were objects mounted that really were not.
Fixed.
Version 1.2
- Fixed activity monitoring for mounted databases.
- Removed import of AdaptableStorage. Argument converters now work
when you specify the full module of a storage class.
- You can now specify a container_class to generate folderish
objects other than standard folders when mounting a new database.
See dbtab.conf.in.
Version 1.1
- Changed DBTab's mounting strategy so that mounted connections stay
bound to a root connection. This change is designed to:
- eliminate issues with volatile attributes in application code
that cross mount boundaries. Now it's safe to use cross-database
volatile attributes.
- eliminate the global registry of open connections, which seemed
to have a rare race condition (ugh!)
- go faster. :-) The mount point traversal penalty is much lower
now, since the mount point can keep a long-lived reference to the
mounted object.
Version 1.0.2
- Updated to work with the latest BDBStorage and AdaptableStorage.
Version 1.0.1
- Deferred startup until after MainConfiguration has been imported.
Needed for ZRS.
- Added AdaptableStorage and BerkeleyStorage to the list of
easily-configured storage types.
- Fixed bug reported by Robert Boulanger:
If the Storage/Databasename is the same like the mountpath it is
not possible to access the database management screens in the
Control Panel. Instead getting Admin screens for Cache and
Activity you will be redirected to the manage workspace of the
folder.
- Arranged for Zope to properly close open database connections on
clean shutdown.
lib/python/DBTab/ClassFactories.py deleted 100644 → 0
View file @ 84ae4ea7
##############################################################################
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE
#
##############################################################################
"""Available ZODB class factories.
$Id$"""
import OFS.Uninstalled
class_factories = {}
def minimalClassFactory(jar, module, name,
_silly=('__doc__',), _globals={},
):
"""Minimal class factory.
If any class is not found, this class factory will propagate
the exception to the application, unlike the other class factories.
"""
m = __import__(module, _globals, _globals, _silly)
return getattr(m, name)
class_factories['minimal'] = minimalClassFactory
def simpleClassFactory(jar, module, name,
_silly=('__doc__',), _globals={},
):
"""Class factory without ZClass support.
"""
try:
m = __import__(module, _globals, _globals, _silly)
return getattr(m, name)
except:
return OFS.Uninstalled.Broken(jar, None, (module, name))
class_factories['simple'] = simpleClassFactory
def zopeClassFactory(jar, module, name,
_silly=('__doc__',), _globals={},
):
"""Class factory with ZClass support.
"""
try:
if module[:1]=='*':
# ZCLass! Yee ha!
return jar.root()['ZGlobals'][module]
else:
m=__import__(module, _globals, _globals, _silly)
return getattr(m, name)
except:
return OFS.Uninstalled.Broken(jar, None, (module, name))
class_factories['zope'] = zopeClassFactory
def autoClassFactory(jar, module, name):
"""Class factory with ZClasses and support for central class definitions.
"""
# If not the root connection, use the class factory from
# the root database, otherwise use the Zope class factory.
root_conn = getattr(jar, '_root_connection', None)
root_db = getattr(root_conn, '_db', None)
if root_db is not None:
return root_db.classFactory(root_conn, module, name)
else:
return zopeClassFactory(jar, module, name)
class_factories['auto'] = autoClassFactory
lib/python/DBTab/DBTab.py deleted 100644 → 0
View file @ 84ae4ea7
##############################################################################
#
# Copyright (c) 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE
#
##############################################################################
"""DBTab and DatabaseFactory classes.
$Id$
"""
import sys
from ZODB.ActivityMonitor import ActivityMonitor
import Globals
from Exceptions import DBTabConfigurationError
class DBTab:
"""A Zope database configuration, similar in purpose to /etc/fstab.
"""
def __init__(self, db_factories, mount_paths):
self._started = 0
self.db_factories = db_factories # { name -> DatabaseFactory }
self.mount_paths = mount_paths # { virtual path -> name }
self.databases = {}
def startup(self):
"""Opens the databases set to open_at_startup."""
if self._started:
return
self._started = 1
for name, factory in self.db_factories.items():
if factory.getOpenAtStartup():
self.getDatabase(name=name)
def listMountPaths(self):
"""Returns a sequence of (virtual_mount_path, database_name).
"""
return self.mount_paths.items()
def listDatabaseNames(self):
"""Returns a sequence of names.
"""
return self.db_factories.keys()
def hasDatabase(self, name):
"""Returns true if name is the name of a configured database."""
return self.db_factories.has_key(name)
def _mountPathError(self, mount_path):
if mount_path == '/':
raise DBTabConfigurationError(
"No root database configured")
else:
raise DBTabConfigurationError(
"No database configured for mount point at %s"
% mount_path)
def getDatabase(self, mount_path=None, name=None, is_root=0):
"""Returns an opened database. Requires either mount_path or name.
"""
self.startup() # XXX get rid of this
if name is None:
name = self.getName(mount_path)
db = self.databases.get(name, None)
if db is None:
factory = self.getDatabaseFactory(name=name)
db = factory.open(name, self.databases)
return db
def getDatabaseFactory(self, mount_path=None, name=None):
if name is None:
name = self.getName(mount_path)
if not self.db_factories.has_key(name):
raise KeyError('%s is not a configured database' % repr(name))
return self.db_factories[name]
def getName(self, mount_path):
name = self.mount_paths.get(mount_path)
if name is None:
self._mountPathError(mount_path)
return name
lib/python/DBTab/Exceptions.py deleted 100644 → 0
View file @ 84ae4ea7
##############################################################################
#
# Copyright (c) 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE
#
##############################################################################
"""DBTab exception classes.
$Id$
"""
class DBTabConfigurationError (Exception):
"""Error in dbtab configuration"""
args = ()
class DBTabOverrideError (Exception):
"""DBTab has taken over some piece of functionality"""
args = ()
lib/python/DBTab/__init__.py deleted 100644 → 0
View file @ 84ae4ea7
##############################################################################
#
# Copyright (c) 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE
#
##############################################################################
"""DBTab product.
$Id$
"""
# importing ThreadedAsync has the side effect of patching asyncore so
# that loop callbacks get invoked. You need this to
# mount a ZEO client connection if the main database is not a ZEO client.
# Otherwise ZEO never receives the message telling it to start using the
# main async loop.
import ThreadedAsync
lib/python/Products/ZODBMountPoint/Mount.py deleted 100644 → 0
View file @ 84ae4ea7
##############################################################################
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE
#
##############################################################################
"""ZODB Mounted database support, simplified for DBTab.
$Id$"""
import sys
from logging import getLogger
try:
from cStringIO import StringIO
except:
from StringIO import StringIO
import traceback
import Persistence, Acquisition
from Acquisition import aq_base
from ZODB.POSException import MountedStorageError, ConnectionStateError
LOG = getLogger('Zope.ZODBMountPoint')
class MountPoint(Persistence.Persistent, Acquisition.Implicit):
'''The base class for a Zope object which, when traversed,
accesses a different database.
'''
# Default values for non-persistent variables.
_v_data = None # An object in an open connection
_v_connect_error = None
def __init__(self, id):
self.id = id
def _getDB(self):
"""Hook for getting the DB object for this mount point.
"""
raise NotImplementedError
def _getDBName(self):
"""Hook for getting the name of the database for this mount point.
"""
raise NotImplementedError
def _getRootDBName(self):
"""Hook for getting the name of the root database.
"""
raise NotImplementedError
def _traverseToMountedRoot(self, root, mount_parent):
"""Hook for getting the object to be mounted.
"""
raise NotImplementedError
def __repr__(self):
return "%s(id=%s)" % (self.__class__.__name__, repr(self.id))
def _getMountedConnection(self, anyjar):
db_name = self._getDBName()
conn = anyjar.get_connection(db_name)
return conn
def _getOrOpenObject(self, parent):
t = self._v_data
if t is not None:
data = t[0]
else:
self._v_connect_error = None
conn = None
try:
anyjar = self._p_jar
if anyjar is None:
anyjar = parent._p_jar
conn = self._getMountedConnection(anyjar)
root = conn.root()
obj = self._traverseToMountedRoot(root, parent)
data = aq_base(obj)
# Store the data object in a tuple to hide from acquisition.
self._v_data = (data,)
except:
# Possibly broken database.
self._logConnectException()
raise
try:
# XXX This method of finding the mount point is deprecated.
# Do not use the _v_mount_point_ attribute.
data._v_mount_point_ = (aq_base(self),)
except:
# Might be a read-only object.
pass
return data.__of__(parent)
def __of__(self, parent):
# Accesses the database, returning an acquisition
# wrapper around the connected object rather than around self.
try:
return self._getOrOpenObject(parent)
except:
return Acquisition.ImplicitAcquisitionWrapper(self, parent)
def _test(self, parent):
'''Tests the database connection.
'''
self._getOrOpenObject(parent)
return 1
def _logConnectException(self):
'''Records info about the exception that just occurred.
'''
try:
from cStringIO import StringIO
except:
from StringIO import StringIO
import traceback
exc = sys.exc_info()
LOG.error('Failed to mount database. %s (%s)' % exc[:2], exc_info=exc)
f=StringIO()
traceback.print_tb(exc[2], 100, f)
self._v_connect_error = (exc[0], exc[1], f.getvalue())
exc = None
lib/python/Products/ZODBMountPoint/MountedObject.py
View file @ b795bc99
......@@ -11,23 +11,29 @@
# FOR A PARTICULAR PURPOSE
#
##############################################################################
"""DBTab mount point (stored in ZODB).
"""Mount point (stored in ZODB).
$Id$
"""
import os
import sys
import traceback
from cStringIO import StringIO
from logging import getLogger
import transaction
import Globals
import Acquisition
from Acquisition import aq_base, aq_inner, aq_parent
from AccessControl.ZopeGuards import guarded_getattr
from OFS.SimpleItem import SimpleItem
from OFS.Folder import Folder
from Products.PageTemplates.PageTemplateFile import PageTemplateFile
from Mount import MountPoint
from ZODB.POSException import MountedStorageError, ConnectionStateError
LOG = getLogger('Zope.ZODBMountPoint')
_www = os.path.join(os.path.dirname(__file__), 'www')
......@@ -111,8 +117,8 @@ class CustomTrailblazer (SimpleTrailblazer):
return obj
class MountedObject(MountPoint, SimpleItem):
'''A MountPoint with a basic interface for displaying the
class MountedObject(SimpleItem):
'''A database mount point with a basic interface for displaying the
reason the database did not connect.
'''
meta_type = 'ZODB Mount Point'
......@@ -124,6 +130,8 @@ class MountedObject(MountPoint, SimpleItem):
icon = 'p_/broken'
manage_options = ({'label':'Traceback', 'action':'manage_traceback'},)
_v_mount_params = None
_v_data = None
_v_connect_error = None
manage_traceback = PageTemplateFile('mountfail.pt', _www)
......@@ -131,7 +139,7 @@ class MountedObject(MountPoint, SimpleItem):
path = str(path)
self._path = path
id = path.split('/')[-1]
MountPoint.__init__(self, id)
self.id = id
def _getMountedConnection(self, anyjar):
db_name = self._getDBName()
......@@ -205,6 +213,73 @@ class MountedObject(MountPoint, SimpleItem):
raise
return obj
def _logConnectException(self):
'''Records info about the exception that just occurred.
'''
try:
from cStringIO import StringIO
except:
from StringIO import StringIO
import traceback
exc = sys.exc_info()
LOG.error('Failed to mount database. %s (%s)' % exc[:2], exc_info=exc)
f=StringIO()
traceback.print_tb(exc[2], 100, f)
self._v_connect_error = (exc[0], exc[1], f.getvalue())
exc = None
def __of__(self, parent):
# Accesses the database, returning an acquisition
# wrapper around the connected object rather than around self.
try:
return self._getOrOpenObject(parent)
except:
return Acquisition.ImplicitAcquisitionWrapper(self, parent)
def _test(self, parent):
'''Tests the database connection.
'''
self._getOrOpenObject(parent)
return 1
def _getOrOpenObject(self, parent):
t = self._v_data
if t is not None:
data = t[0]
else:
self._v_connect_error = None
conn = None
try:
anyjar = self._p_jar
if anyjar is None:
anyjar = parent._p_jar
conn = self._getMountedConnection(anyjar)
root = conn.root()
obj = self._traverseToMountedRoot(root, parent)
data = aq_base(obj)
# Store the data object in a tuple to hide from acquisition.
self._v_data = (data,)
except:
# Possibly broken database.
self._logConnectException()
raise
try:
# XXX This method of finding the mount point is deprecated.
# Do not use the _v_mount_point_ attribute.
data._v_mount_point_ = (aq_base(self),)
except:
# Might be a read-only object.
pass
return data.__of__(parent)
def __repr__(self):
return "%s(id=%s)" % (self.__class__.__name__, repr(self.id))
Globals.InitializeClass(MountedObject)
......@@ -239,7 +314,7 @@ def setMountPoint(container, id, mp):
manage_addMountsForm = PageTemplateFile('addMountsForm.pt', _www)
def manage_getMountStatus(dispatcher):
"""Returns the status of each mount point specified by dbtab.conf.
"""Returns the status of each mount point specified by zope.conf
"""
res = []
conf = getConfiguration()
......@@ -324,3 +399,4 @@ def manage_addMounts(dispatcher, paths=(), create_mount_points=True,
REQUEST['URL1'] + ('/manage_main?manage_tabs_message='
'Added %d mount points.' % count))
lib/python/Products/ZODBMountPoint/tests/testMountPoint.py
View file @ b795bc99
......@@ -24,7 +24,7 @@ from OFS.Application import Application
from OFS.Folder import Folder
import App.config
from Products.ZODBMountPoint.MountedObject import manage_addMounts, getMountPoint
from DBTab.DBTab import DBTab
from Zope2.Startup.datatypes import DBTab
try:
__file__
......@@ -64,8 +64,6 @@ original_config = None
class DBTabTests (unittest.TestCase):
def setUp(self):
global original_config
if original_config is None:
......@@ -129,41 +127,6 @@ class DBTabTests (unittest.TestCase):
self.assertEqual(app.mount2._p_changed, 0)
self.assertEqual(app._p_changed, 0)
def testRaceOnClose(self):
# There used to be a race condition in
# ConnectionPatches.close(). The root connection was returned
# to the pool before the mounted connections were closed. If
# another thread pulled the root connection out of the pool
# before the original thread finished closing mounted
# connections, when the original thread got control back it
# closed the mounted connections even though the new thread
# was using them.
# Test by patching to watch for a vulnerable moment.
from ZODB.DB import DB
def _closeConnection(self, connection):
self._real_closeConnection(connection)
mc = connection._mounted_connections
if mc is not None:
for c in mc.values():
if c._storage is not None:
raise AssertionError, "Connection remained partly open"
DB._real_closeConnection = DB._closeConnection
DB._closeConnection = _closeConnection
try:
conn = self.db.open()
conn.root()['Application']['mount1']
conn.root()['Application']['mount2']
conn.close()
finally:
DB._closeConnection = DB._real_closeConnection
del DB._real_closeConnection
def testGetMountPoint(self):
self.assert_(getMountPoint(self.app) is None)
self.assert_(getMountPoint(self.app.mount1) is not None)
......
lib/python/Zope2/Startup/datatypes.py
View file @ b795bc99
......@@ -18,6 +18,7 @@ import os
from ZConfig.components.logger import logger
from ZODB.config import ZODBDatabase
import OFS.Uninstalled
# generic datatypes
......@@ -109,7 +110,7 @@ def python_dotted_path(name):
# Datatype for the root configuration object
# (adds the softwarehome and zopehome fields; default values for some
# computed paths, configures dbtab)
# computed paths, configures the dbtab)
def root_config(section):
from ZConfig import ConfigurationError
......@@ -147,9 +148,7 @@ def root_config(section):
raise ConfigurationError(dup_err % (mount_points[point],
name, point))
mount_points[point] = name
from DBTab.DBTab import DBTab
section.dbtab = DBTab(mount_factories, mount_points)
return section
class ZopeDatabase(ZODBDatabase):
......@@ -173,10 +172,6 @@ class ZopeDatabase(ZODBDatabase):
def getName(self):
return self.name
def getOpenAtStartup(self):
# XXX implement
return 0
def computeMountPaths(self):
mps = []
for part in self.config.mount_points:
......@@ -211,3 +206,110 @@ class ZopeDatabase(ZODBDatabase):
return (real_root, real_path, container_class)
raise LookupError('Nothing known about mount path %s' % mount_path)
class DBTab:
"""A Zope database configuration, similar in purpose to /etc/fstab.
"""
def __init__(self, db_factories, mount_paths):
self.db_factories = db_factories # { name -> DatabaseFactory }
self.mount_paths = mount_paths # { virtual path -> name }
self.databases = {} # { name -> DB instance }
def listMountPaths(self):
"""Returns a sequence of (virtual_mount_path, database_name).
"""
return self.mount_paths.items()
def listDatabaseNames(self):
"""Returns a sequence of names.
"""
return self.db_factories.keys()
def hasDatabase(self, name):
"""Returns true if name is the name of a configured database."""
return self.db_factories.has_key(name)
def _mountPathError(self, mount_path):
from ZConfig import ConfigurationError
if mount_path == '/':
raise ConfigurationError(
"No root database configured")
else:
raise ConfigurationError(
"No database configured for mount point at %s"
% mount_path)
def getDatabase(self, mount_path=None, name=None, is_root=0):
"""Returns an opened database. Requires either mount_path or name.
"""
if name is None:
name = self.getName(mount_path)
db = self.databases.get(name, None)
if db is None:
factory = self.getDatabaseFactory(name=name)
db = factory.open(name, self.databases)
return db
def getDatabaseFactory(self, mount_path=None, name=None):
if name is None:
name = self.getName(mount_path)
if not self.db_factories.has_key(name):
raise KeyError('%s is not a configured database' % repr(name))
return self.db_factories[name]
def getName(self, mount_path):
name = self.mount_paths.get(mount_path)
if name is None:
self._mountPathError(mount_path)
return name
# class factories (potentially) used by the class-factory parameter in
# zopeschema.xml
def minimalClassFactory(jar, module, name,
_silly=('__doc__',), _globals={},
):
"""Minimal class factory.
If any class is not found, this class factory will propagate
the exception to the application, unlike the other class factories.
"""
m = __import__(module, _globals, _globals, _silly)
return getattr(m, name)
def simpleClassFactory(jar, module, name,
_silly=('__doc__',), _globals={},
):
"""Class factory without ZClass support.
"""
try:
m = __import__(module, _globals, _globals, _silly)
return getattr(m, name)
except:
return OFS.Uninstalled.Broken(jar, None, (module, name))
def zopeClassFactory(jar, module, name,
_silly=('__doc__',), _globals={},
):
"""Class factory with ZClass support.
"""
try:
if module[:1]=='*':
# ZCLass! Yee ha!
return jar.root()['ZGlobals'][module]
else:
m=__import__(module, _globals, _globals, _silly)
return getattr(m, name)
except:
return OFS.Uninstalled.Broken(jar, None, (module, name))
# There used to be an "autoClassFactory" whose docstring read "If not the root
# connection, use the class factory from the root database, otherwise use the
# Zope class factory." This no longer works with the implementation of
# mounted databases, so we just use the zopeClassFactory as the default
lib/python/Zope2/Startup/zopeschema.xml
View file @ b795bc99
......@@ -217,7 +217,7 @@
</key>
<key name="class-factory" datatype=".importable_name"
default="DBTab.ClassFactories.autoClassFactory">
default="Zope2.Startup.datatypes.zopeClassFactory">
<description>
Change the class factory function a database uses on a
per-database basis to support different class factory policy.
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7