Skip to content

Z
ZODB
Commit aa195678 authored by Jeremy Hylton's avatar Jeremy Hylton
Browse files
Options

Merge changes from Zope-2_7-branch to the trunk.

parent b765fc5a
Hide whitespace changes
Inline Side-by-side
Showing with 515 additions and 433 deletions
trunk/doc/ZEO/howto.txt
View file @ aa195678
......@@ -207,13 +207,6 @@ then this command will install the new ZEO and ZODB:
The install command should create a /home/zope/lib/python/ZEO directoy.
Simple configuration
--------------------
mkzeoinst.py
Or, do it step-by-step.
Configuring server
------------------
......@@ -407,13 +400,9 @@ Running the ZEO server as a daemon
In an operational setting, you will want to run the ZEO server a
daemon process that is restarted when it dies. The zdaemon package
provides two tools for running daemons: zdrun.py and zdctl.py.
The document "Using zdctl and zdrun to manage server processes"
explains how to use these scripts to manage daemons.
XXX example of how to use zdrun
XXX mkzeoinst.py docs should probably go here
provides two tools for running daemons: zdrun.py and zdctl.py. The
document "Using zdctl and zdrun to manage server processes"
(Doc/zdctl.txt) explains how to use these scripts to manage daemons.
Rotating log files
~~~~~~~~~~~~~~~~~~
......@@ -437,10 +426,6 @@ manages a ZEO servers password database.
Diagnosing problems
-------------------
How to use the debug logs.
Common gotchas.
If an exception occurs on the server, the server will log a traceback
and send an exception to the client. The traceback on the client will
show a ZEO protocol library as the source of the error. If you need
......
trunk/doc/ZEO/trace.txt
View file @ aa195678
Client Cache Tracing
====================
ZEO Client Cache Tracing
========================
An important question for ZEO users is: how large should the ZEO
client cache be? ZEO 2 (as of ZEO 2.0b2) has a new feature that lets
......
trunk/doc/guide/chatter.py
View file @ aa195678
......@@ -16,8 +16,8 @@ class ChatSession(Persistent):
add_message(message) -- add a message to the channel
new_messages() -- return new messages since the last call to
this method
"""
def __init__(self, name):
......@@ -27,15 +27,15 @@ class ChatSession(Persistent):
self.name = name
# Internal attribute: _messages holds all the chat messages.
# Internal attribute: _messages holds all the chat messages.
self._messages = BTree.BTree()
def new_messages(self):
"Return new messages."
# self._v_last_time is the time of the most recent message
# returned to the user of this class.
# returned to the user of this class.
if not hasattr(self, '_v_last_time'):
self._v_last_time = 0
......@@ -48,7 +48,7 @@ class ChatSession(Persistent):
self._v_last_time = T2
return new
def add_message(self, message):
"""Add a message to the channel.
message -- text of the message to be added
......@@ -82,7 +82,7 @@ def get_chat_session(conn, channelname):
print 'Creating chat_sessions B-tree'
root['chat_sessions'] = BTree.BTree()
get_transaction().commit()
sessions = root['chat_sessions']
# Get a session object corresponding to the channel name, creating
......@@ -94,7 +94,7 @@ def get_chat_session(conn, channelname):
session = sessions[ channelname ]
return session
if __name__ == '__main__':
if len(sys.argv) != 2:
......@@ -119,6 +119,5 @@ if __name__ == '__main__':
print msg
# Wait for a few seconds
pause = random.randint( 1, 4 )
pause = random.randint( 1, 4 )
time.sleep( pause )
trunk/doc/guide/links.tex
View file @ aa195678
......@@ -25,7 +25,4 @@ and less in others.
\\
\url{http://www.zope.org/Members/bwarsaw/ipc10-slides}
Download link for ZEO: \\
\url{http://www.zope.org/Products/ZEO/}
trunk/doc/guide/prog-zodb.tex
View file @ aa195678
......@@ -9,36 +9,28 @@
\subsection{Installing ZODB}
The ZODB forms part of Zope, but it's difficult and somewhat painful
to extract the bits from Zope needed to support just the ZODB.
Therefore I've assembled a distribution containing only the packages
required to use the ZODB and ZEO, so you can install it and start
programming right away.
To download the distribution, go to my ZODB page at
\url{http://www.amk.ca/zodb/}.
The distribution is still experimental, so don't be surprised if the
installation process runs into problems. Please inform me of any
difficulties you encounter.
ZODB is packaged using the standard distutils tools.
\subsubsection{Requirements}
You will need Python 2.1 or higher. The code is packaged using
Distutils. So it is simply a matter of untarring or unzipping the
release package, and then running \code{python setup.py install}.
You will need Python 2.2 or higher. Since the code is packaged using
distutils, it is simply a matter of untarring or unzipping the release
package, and then running \code{python setup.py install}.
You'll need a C compiler to build the packages, because there are
various C extension modules. At the moment no one is making Windows
binaries available, so you'll need a Windows development environment
to build ZODB.
various C extension modules. Binary installers are provided for
Windows users.
\subsubsection{Installing the Packages}
Download the ZODB tarball containing all the packages for both ZODB
and ZEO from \url{http://www.zope.org/Products/StandaloneZODB}. See
and ZEO from \url{http://www.zope.org/Products/ZODB3.2}. See
the \file{README.txt} file in the top level of the release directory
for details on building, testing, and installing.
You can find information about ZODB and the most current releases in
the ZODB Wiki at \url{http://www.zope.org/Wikis/ZODB}.
\subsection{How ZODB Works}
The ZODB is conceptually simple. Python classes subclass a
......@@ -59,8 +51,10 @@ don't get corrupted by software or hardware crashes, and most database
software offers protection against such corruption by supporting four
useful properties, Atomicity, Consistency, Isolation, and Durability.
In computer science jargon these four terms are collectively dubbed
the ACID properties, forming an acronym from their names. The
definitions of the ACID properties are:
the ACID properties, forming an acronym from their names.
The ZODB provides all of the ACID properties. Definitions of the
ACID properties are:
\begin{itemize}
......@@ -73,18 +67,13 @@ forgotten. That's bad, but it's better than having a
partially-applied modification put the database into an inconsistent
state.
\item[Consistency] means that the data cannot be placed into a
logically invalid state; sanity checks can be written and enforced.
Usually this is done by defining a database schema, and requiring
the data always matches the schema. There are two typical
approaches to consistency. One is to enforce rules about the types
of objects and attribute; for example, enforce that the
\code{order_number} attribute is always an integer, and not a
string, tuple, or other object. Another is to guarantee consistency
across data structures; for example, that any object with an
\code{order_number} attribute must also appear in the
\code{orders_table} object. In general, atomicity and isolation make
it possible for applications to provide consistency.
\item[Consistency] means that each transaction executes a valid
transformation of the database state. Some databases, but not ZODB,
provide a variety of consistency checks in the database or language;
for example, a relational database constraint columns to be of
particular types and can enforce relations across tables. Viewed more
generally, atomicity and isolation make it possible for applications
to provide consistency.
\item[Isolation] means that two programs or threads running in two
different transactions cannot see each other's changes until they
......@@ -95,10 +84,6 @@ a subsequent crash will not cause any data to be lost or corrupted.
\end{itemize}
The ZODB provides 3 of the ACID properties. Only Consistency is not
supported; the ZODB has no notion of a database schema, and therefore
has no way of enforcing consistency with a schema.
\subsection{Opening a ZODB}
There are 3 main interfaces supplied by the ZODB:
......@@ -132,9 +117,9 @@ implement the \class{Storage} interface.
\end{itemize}
Preparing to use a ZODB requires 3 steps: you have to open the
\class{Storage}, then create a \class{DB} instance that uses the \class{Storage}, and then get
a \class{Connection} from the \class{DB instance}. All this is only a few lines of
code:
\class{Storage}, then create a \class{DB} instance that uses the
\class{Storage}, and then get a \class{Connection} from the \class{DB
instance}. All this is only a few lines of code:
\begin{verbatim}
from ZODB import FileStorage, DB
......@@ -189,7 +174,7 @@ retrieving a \class{User} object given the user's ID. First, we
retrieve the primary root object of the ZODB using the \method{root()}
method of the \class{Connection} instance. The root object behaves
like a Python dictionary, so you can just add a new key/value pair for
your application's root object. We'll insert a \class{BTree} object
your application's root object. We'll insert an \class{OOBTree} object
that will contain all the \class{User} objects. (The
\class{BTree} module is also included as part of Zope.)
......@@ -199,8 +184,8 @@ dbroot = conn.root()
# Ensure that a 'userdb' key is present
# in the root
if not dbroot.has_key('userdb'):
import BTree
dbroot['userdb'] = BTree.BTree()
from BTrees.OOBTree import OOBTree
dbroot['userdb'] = OOBTree()
userdb = dbroot['userdb']
\end{verbatim}
......@@ -426,13 +411,11 @@ different class instances, then there's no longer any easy way to find
them all, short of writing a generalized object traversal function
that would walk over every single object in a ZODB, checking each one
to see if it's an instance of \class{User}.
\footnote{XXX is there a convenience method for walking the object graph hiding
somewhere inside DC's code? Should there be a utility method for
doing this? Should I write one and include it in this section?}
Some OODBs support a feature called extents, which allow quickly
finding all the instances of a given class, no matter where they are
in the object graph; unfortunately the ZODB doesn't offer extents as a
feature.
XXX Rest of section not written yet: __getstate__/__setstate__
% XXX Rest of section not written yet: __getstate__/__setstate__
trunk/doc/guide/zodb.tex
View file @ aa195678
\documentclass{howto}
\title{ZODB/ZEO Programming Guide}
\release{0.1}
\release{0.2}
\date{\today}
\author{A.M.\ Kuchling}
......@@ -23,9 +23,6 @@
\input{prog-zodb}
\input{zeo}
\input{transactions}
%\input{storages}
%\input{indexing}
%\input{admin}
\input{modules}
\appendix
......
trunk/src/BTrees/tests/testBTrees.py
View file @ aa195678
......@@ -33,7 +33,7 @@ class Base:
""" Tests common to all types: sets, buckets, and BTrees """
db = None
def tearDown(self):
self.t = None
del self.t
......
trunk/src/ThreadedAsync/LoopCallback.py
View file @ aa195678
......@@ -25,7 +25,7 @@ register_loop_callback() to register interest. When the mainloop
thread calls loop(), each registered callback will be called with the
socket map as its first argument.
"""
__version__ = '$Revision: 1.8 $'[11:-2]
__version__ = '$Revision: 1.9 $'[11:-2]
import asyncore
import select
......@@ -57,7 +57,17 @@ def register_loop_callback(callback, args=(), kw=None):
_loop_callbacks.append((callback, args, kw))
finally:
_loop_lock.release()
def remove_loop_callback(callback):
"""Remove a callback function registered earlier.
This is useful if loop() was never called.
"""
for i in range(len(_loop_callbacks)):
if _loop_callbacks[i][0] == callback:
del _loop_callbacks[i]
return
def _start_loop(map):
_loop_lock.acquire()
try:
......
trunk/src/ThreadedAsync/__init__.py
View file @ aa195678
......@@ -13,7 +13,7 @@
##############################################################################
"""Manage the asyncore mainloop in a multi-threaded app.
$Id: __init__.py,v 1.6 2003/02/25 15:17:09 fdrake Exp $
$Id: __init__.py,v 1.7 2003/10/02 18:17:26 jeremy Exp $
"""
from LoopCallback import register_loop_callback, loop
from LoopCallback import register_loop_callback, loop, remove_loop_callback
trunk/src/ZEO/ClientStorage.py
View file @ aa195678
......@@ -99,7 +99,7 @@ class ClientStorage(object):
name='', client=None, debug=0, var=None,
min_disconnect_poll=5, max_disconnect_poll=300,
wait_for_server_on_startup=None, # deprecated alias for wait
wait=None, # defaults to 1
wait=None, wait_timeout=None,
read_only=0, read_only_fallback=0,
username='', password='', realm=None):
"""ClientStorage constructor.
......@@ -152,6 +152,9 @@ class ClientStorage(object):
wait -- A flag indicating whether to wait until a connection
with a server is made, defaulting to true.
wait_timeout -- Maximum time to wait for a connection before
giving up. Only meaningful if wait is True.
read_only -- A flag indicating whether this should be a
read-only storage, defaulting to false (i.e. writing is
allowed by default).
......@@ -302,7 +305,7 @@ class ClientStorage(object):
tmax=max_disconnect_poll)
if wait:
self._wait()
self._wait(wait_timeout)
else:
# attempt_connect() will make an attempt that doesn't block
# "too long," for a very vague notion of too long. If that
......@@ -313,7 +316,9 @@ class ClientStorage(object):
if not self._ready.isSet():
self._cache.open()
def _wait(self):
def _wait(self, timeout=None):
if timeout is not None:
deadline = time.time() + timeout
# Wait for a connection to be established.
self._rpc_mgr.connect(sync=1)
# When a synchronous connect() call returns, there is
......@@ -326,6 +331,9 @@ class ClientStorage(object):
self._ready.wait(30)
if self._ready.isSet():
break
if timeout and time.time() > deadline:
log2(PROBLEM, "Timed out waiting for connection")
break
log2(INFO, "Waiting for cache verification to finish")
else:
self._wait_sync()
......@@ -434,8 +442,10 @@ class ClientStorage(object):
auth = stub.getAuthProtocol()
log2(INFO, "Server authentication protocol %r" % auth)
if auth:
if self.doAuth(auth, stub):
skey = self.doAuth(auth, stub)
if skey:
log2(INFO, "Client authentication successful")
conn.setSessionKey(skey)
else:
log2(ERROR, "Authentication failed")
raise AuthError, "Authentication failed"
......
trunk/src/ZEO/CommitLog.py
View file @ aa195678
......@@ -42,3 +42,8 @@ class CommitLog:
self.read = 1
self.file.seek(0)
return self.stores, cPickle.Unpickler(self.file)
def close(self):
if self.file:
self.file.close()
self.file = None
trunk/src/ZEO/DebugServer.py
View file @ aa195678
......@@ -28,7 +28,7 @@ DATA = 3
ERROR = 4
class DebugManagedServerConnection(ManagedServerConnection):
def __init__(self, sock, addr, obj, mgr):
# mgr is the DebugServer instance
self.mgr = mgr
......@@ -67,7 +67,7 @@ class DebugServer(StorageServer):
ZEOStorageClass = DebugZEOStorage
ManagedServerConnectionClass = DebugManagerConnection
def __init__(self, *args, **kwargs):
StorageServer.__init__(*args, **kwargs)
self._setup_record(kwargs["record"])
......
trunk/src/ZEO/ServerStub.py
View file @ aa195678
......@@ -32,9 +32,13 @@ class StorageServer:
zrpc.connection.Connection class.
"""
self.rpc = rpc
if self.rpc.peer_protocol_version == 'Z200':
# Wait until we know what version the other side is using.
while rpc.peer_protocol_version is None:
rpc.pending()
if rpc.peer_protocol_version == 'Z200':
self.lastTransaction = lambda: None
self.getInvalidations = lambda tid: None
self.getAuthProtocol = lambda: None
def extensionMethod(self, name):
return ExtensionMethodWrapper(self.rpc, name).call
......@@ -47,7 +51,7 @@ class StorageServer:
def getAuthProtocol(self):
return self.rpc.call('getAuthProtocol')
def lastTransaction(self):
# Not in protocol version 2.0.0; see __init__()
return self.rpc.call('lastTransaction')
......@@ -150,6 +154,6 @@ class ExtensionMethodWrapper:
def __init__(self, rpc, name):
self.rpc = rpc
self.name = name
def call(self, *a, **kwa):
return self.rpc.call(self.name, *a, **kwa)
trunk/src/ZEO/StorageServer.py
View file @ aa195678
......@@ -43,7 +43,7 @@ from ZODB.POSException import StorageError, StorageTransactionError
from ZODB.POSException import TransactionError, ReadOnlyError, ConflictError
from ZODB.referencesf import referencesf
from ZODB.Transaction import Transaction
from ZODB.utils import u64
from ZODB.utils import u64, oid_repr
_label = "ZSS" # Default label used for logging.
......@@ -90,7 +90,7 @@ class ZEOStorage:
self._extensions = {}
for func in self.extensions:
self._extensions[func.func_name] = None
def finish_auth(self, authenticated):
if not self.auth_realm:
return 1
......@@ -99,7 +99,7 @@ class ZEOStorage:
def set_database(self, database):
self.database = database
def notifyConnected(self, conn):
self.connection = conn # For restart_other() below
self.client = self.ClientStorageStubClass(conn)
......@@ -189,7 +189,7 @@ class ZEOStorage:
if not protocol or protocol == 'none':
return None
return protocol
def register(self, storage_id, read_only):
"""Select the storage that this client will use
......@@ -398,6 +398,7 @@ class ZEOStorage:
def _clear_transaction(self):
# Common code at end of tpc_finish() and tpc_abort()
self.transaction = None
self.txnlog.close()
if self.locked:
self.locked = 0
self.timeout.end(self)
......@@ -483,6 +484,8 @@ class ZEOStorage:
self.store_failed = 1
if isinstance(err, ConflictError):
self.stats.conflicts += 1
self.log("conflict error oid=%s msg=%s" %
(oid_repr(oid), str(err)), zLOG.BLATHER)
if not isinstance(err, TransactionError):
# Unexpected errors are logged and passed to the client
exc_info = sys.exc_info()
......@@ -506,6 +509,7 @@ class ZEOStorage:
self.invalidated.append((oid, version))
if newserial == ResolvedSerial:
self.stats.conflicts_resolved += 1
self.log("conflict resolved oid=%s" % oid_repr(oid), zLOG.BLATHER)
self.serials.append((oid, newserial))
return err is None
......@@ -680,7 +684,7 @@ class StorageServer:
It should be in a format compatible with the authentication
protocol used; for instance, "sha" and "srp" require different
formats.
Note that to implement an authentication protocol, a server
and client authentication mechanism must be implemented in a
auth_* module, which should be stored inside the "auth"
......@@ -728,7 +732,7 @@ class StorageServer:
self.monitor = StatsServer(monitor_address, self.stats)
else:
self.monitor = None
def _setup_auth(self, protocol):
# Can't be done in global scope, because of cyclic references
from ZEO.auth import get_module
......@@ -739,9 +743,9 @@ class StorageServer:
if not module:
log("%s: no such an auth protocol: %s" % (name, protocol))
return
storage_class, client, db_class = module
if not storage_class or not issubclass(storage_class, ZEOStorage):
log(("%s: %s isn't a valid protocol, must have a StorageClass" %
(name, protocol)))
......@@ -750,7 +754,7 @@ class StorageServer:
self.ZEOStorageClass = storage_class
log("%s: using auth protocol: %s" % (name, protocol))
# We create a Database instance here for use with the authenticator
# modules. Having one instance allows it to be shared between multiple
# storages, avoiding the need to bloat each with a new authenticator
......@@ -762,7 +766,7 @@ class StorageServer:
"does not match storage realm %r"
% (self.database.realm, self.auth_realm))
def new_connection(self, sock, addr):
"""Internal: factory to create a new connection.
......@@ -776,7 +780,7 @@ class StorageServer:
zstorage.set_database(self.database)
else:
zstorage = self.ZEOStorageClass(self, self.read_only)
c = self.ManagedServerConnectionClass(sock, addr, zstorage, self)
log("new connection %s: %s" % (addr, `c`))
return c
......@@ -846,12 +850,12 @@ class StorageServer:
if not self.invq:
log("invq empty")
return None, []
earliest_tid = self.invq[0][0]
if earliest_tid > tid:
log("tid to old for invq %s < %s" % (u64(tid), u64(earliest_tid)))
return None, []
oids = {}
for tid, L in self.invq:
for key in L:
......@@ -928,6 +932,7 @@ class TimeoutThread(threading.Thread):
self._cond.acquire()
try:
assert self._client is not None
assert self._client is client
self._client = None
self._deadline = None
finally:
......@@ -953,7 +958,6 @@ class TimeoutThread(threading.Thread):
self._trigger.pull_trigger(lambda: client.connection.close())
else:
time.sleep(howlong)
self.trigger.close()
def run_in_thread(method, *args):
t = SlowMethodThread(method, args)
......
trunk/src/ZEO/__init__.py
View file @ aa195678
......@@ -15,10 +15,10 @@
See the file README.txt in this directory for an overview.
ZEO is now part of ZODB; ZODB's home on the web is
ZEO is now part of ZODB; ZODB's home on the web is
http://www.zope.org/Wikis/ZODB
"""
version = "2.1b2"
version = "2.2c1"
trunk/src/ZEO/auth/__init__.py
View file @ aa195678
......@@ -28,4 +28,3 @@ def register_module(name, storage_class, client, db):
if _auth_modules.has_key(name):
raise TypeError, "%s is already registred" % name
_auth_modules[name] = storage_class, client, db
trunk/src/ZEO/auth/auth_digest.py
View file @ aa195678
......@@ -61,7 +61,7 @@ def hexdigest(s):
class DigestDatabase(Database):
def __init__(self, filename, realm=None):
Database.__init__(self, filename, realm)
# Initialize a key used to build the nonce for a challenge.
# We need one key for the lifetime of the server, so it
# is convenient to store in on the database.
......@@ -134,7 +134,7 @@ class DigestClient(Client):
raise AuthError("expected realm %r, got realm %r"
% (_realm, realm))
h_up = hexdigest("%s:%s:%s" % (username, realm, password))
resp_dig = hexdigest("%s:%s" % (h_up, challenge))
result = self.stub.auth_response((username, challenge, resp_dig))
if result:
......
trunk/src/ZEO/auth/base.py
View file @ aa195678
......@@ -34,19 +34,19 @@ def sort(L):
"""Sort a list in-place and return it."""
L.sort()
return L
class Database:
"""Abstracts a password database.
This class is used both in the authentication process (via
get_password()) and by client scripts that manage the password
database file.
database file.
The password file is a simple, colon-separated text file mapping
usernames to password hashes. The hashes are SHA hex digests
produced from the password string.
"""
def __init__(self, filename, realm=None):
"""Creates a new Database
......@@ -61,7 +61,7 @@ class Database:
self.filename = filename
self.realm = realm
self.load()
def save(self, fd=None):
filename = self.filename
......@@ -72,7 +72,7 @@ class Database:
for username in sort(self._users.keys()):
print >> fd, "%s: %s" % (username, self._users[username])
def load(self):
filename = self.filename
if not filename:
......@@ -80,7 +80,7 @@ class Database:
if not os.path.exists(filename):
return
fd = open(filename)
L = fd.readlines()
......@@ -90,7 +90,7 @@ class Database:
if L[0].startswith("realm "):
line = L.pop(0).strip()
self.realm = line[len("realm "):]
for line in L:
username, hash = line.strip().split(":", 1)
self._users[username] = hash.strip()
......@@ -103,10 +103,10 @@ class Database:
Callers must check for LookupError, which is raised in
the case of a non-existent user specified."""
if not self._users.has_key(username):
if not self._users.has_key(username):
raise LookupError, "No such user: %s" % username
return self._users[username]
def hash(self, s):
return sha.new(s).hexdigest()
......@@ -116,7 +116,7 @@ class Database:
self._store_password(username, password)
def del_user(self, username):
if not self._users.has_key(username):
if not self._users.has_key(username):
raise LookupError, "No such user: %s" % username
del self._users[username]
......
trunk/src/ZEO/monitor.py
View file @ aa195678
......@@ -13,7 +13,7 @@
##############################################################################
"""Monitor behavior of ZEO server and record statistics.
$Id: monitor.py,v 1.3 2003/01/15 21:23:16 jeremy Exp $
$Id: monitor.py,v 1.4 2003/10/02 18:17:22 jeremy Exp $
"""
import asyncore
......@@ -54,7 +54,7 @@ class StorageStats:
elif field == "Active transactions":
self.active_txns = int(value)
elif field == "Commit lock held for":
# This assumes
# This assumes
self.lock_time = time.time() - int(value)
elif field == "Commits":
self.commits = int(value)
......@@ -112,7 +112,7 @@ class StatsClient(asyncore.dispatcher):
n = self.socket.send(s)
if n < len(s):
self.buf.append(s[:n])
if self.closed and not self.buf:
asyncore.dispatcher.close(self)
......
trunk/src/ZEO/tests/CommitLockTests.py
View file @ aa195678
......@@ -27,7 +27,7 @@ from ZEO.tests.TestThread import TestThread
ZERO = '\0'*8
class DummyDB:
def invalidate(self, *args):
def invalidate(self, *args, **kwargs):
pass
class WorkerThread(TestThread):
......@@ -116,7 +116,7 @@ class CommitLockTests:
self._dostore()
self._cleanup()
def checkCommitLockVoteAbort(self):
oid, txn = self._start_txn()
self._storage.tpc_vote(txn)
......@@ -129,7 +129,7 @@ class CommitLockTests:
self._dostore()
self._cleanup()
def checkCommitLockVoteClose(self):
oid, txn = self._start_txn()
self._storage.tpc_vote(txn)
......@@ -153,7 +153,7 @@ class CommitLockTests:
def _finish_undo(self, msgid):
return self._storage._server.rpc._deferred_wait(msgid)
def checkCommitLockUndoFinish(self):
trans_id = self._get_trans_id()
oid, txn = self._start_txn()
......@@ -170,7 +170,7 @@ class CommitLockTests:
self._dostore()
self._cleanup()
def checkCommitLockUndoAbort(self):
trans_id = self._get_trans_id()
oid, txn = self._start_txn()
......@@ -186,7 +186,7 @@ class CommitLockTests:
self._dostore()
self._cleanup()
def checkCommitLockUndoClose(self):
trans_id = self._get_trans_id()
oid, txn = self._start_txn()
......@@ -201,23 +201,24 @@ class CommitLockTests:
self._finish_threads()
self._cleanup()
def _begin_threads(self):
# Start a second transaction on a different connection without
# blocking the test thread.
# blocking the test thread. Returns only after each thread has
# set it's ready event.
self._storages = []
self._threads = []
for i in range(self.NUM_CLIENTS):
storage = self._duplicate_client()
txn = Transaction()
tid = self._get_timestamp()
t = WorkerThread(self, storage, txn)
self._threads.append(t)
t.start()
t.ready.wait()
# Close on the connections abnormally to test server response
if i == 0:
storage.close()
......
trunk/src/ZEO/tests/ConnectionTests.py
View file @ aa195678
......@@ -170,7 +170,7 @@ class CommonSetupTearDown(StorageTestBase):
if ro_svr:
zconf.read_only = 1
if self.monitor:
zconf.monitor_address = ("", 42000)
zconf.monitor_address = ("", 42000)
if self.invq:
zconf.invalidation_queue_size = self.invq
if self.timeout:
......
trunk/src/ZEO/tests/InvalidationTests.py
View file @ aa195678
......@@ -347,8 +347,8 @@ class InvalidationTests:
# Run two threads that update the BTree
cd = {}
t1 = self.StressThread(self, db1, stop, 1, cd, 1, sleep=0.001)
t2 = self.StressThread(self, db1, stop, 2, cd, 2, sleep=0.001)
t1 = self.StressThread(self, db1, stop, 1, cd, 1, sleep=0.01)
t2 = self.StressThread(self, db1, stop, 2, cd, 2, sleep=0.01)
self.go(stop, cd, t1, t2)
cn.sync()
......@@ -375,8 +375,8 @@ class InvalidationTests:
cd = {}
t1 = self.StressThread(self, db1, stop, 1, cd, 1, 3)
t2 = self.StressThread(self, db2, stop, 2, cd, 2, 3, 0.001)
t3 = self.StressThread(self, db2, stop, 3, cd, 3, 3, 0.001)
t2 = self.StressThread(self, db2, stop, 2, cd, 2, 3, 0.01)
t3 = self.StressThread(self, db2, stop, 3, cd, 3, 3, 0.01)
self.go(stop, cd, t1, t2, t3)
cn.sync()
......@@ -404,8 +404,8 @@ class InvalidationTests:
cd = {}
t1 = VersionStressThread(self, db1, stop, 1, cd, 1, 3)
t2 = VersionStressThread(self, db2, stop, 2, cd, 2, 3, 0.001)
t3 = VersionStressThread(self, db2, stop, 3, cd, 3, 3, 0.001)
t2 = VersionStressThread(self, db2, stop, 2, cd, 2, 3, 0.01)
t3 = VersionStressThread(self, db2, stop, 3, cd, 3, 3, 0.01)
self.go(stop, cd, t1, t2, t3)
cn.sync()
......@@ -435,9 +435,9 @@ class InvalidationTests:
# at the same time.
cd = {}
t1 = LargeUpdatesThread(self, db1, stop, 1, cd, 1, 3, 0.001)
t2 = LargeUpdatesThread(self, db2, stop, 2, cd, 2, 3, 0.001)
t3 = LargeUpdatesThread(self, db2, stop, 3, cd, 3, 3, 0.001)
t1 = LargeUpdatesThread(self, db1, stop, 1, cd, 1, 3, 0.02)
t2 = LargeUpdatesThread(self, db2, stop, 2, cd, 2, 3, 0.01)
t3 = LargeUpdatesThread(self, db2, stop, 3, cd, 3, 3, 0.01)
self.go(stop, cd, t1, t2, t3)
cn.sync()
......
trunk/src/ZEO/tests/auth_plaintext.py
View file @ aa195678
......@@ -13,11 +13,11 @@
##############################################################################
"""Implements plaintext password authentication. The password is stored in
an SHA hash in the Database. The client sends over the plaintext
password, and the SHA hashing is done on the server side.
password, and the SHA hashing is done on the server side.
This mechanism offers *no network security at all*; the only security
is provided by not storing plaintext passwords on disk. (See the
auth_srp module for a secure mechanism)"""
is provided by not storing plaintext passwords on disk.
"""
import sha
......@@ -25,20 +25,31 @@ from ZEO.StorageServer import ZEOStorage
from ZEO.auth import register_module
from ZEO.auth.base import Client, Database
def session_key(username, realm, password):
return sha.new("%s:%s:%s" % (username, realm, password)).hexdigest()
class StorageClass(ZEOStorage):
def auth(self, username, password):
try:
dbpw = self.database.get_password(username)
except LookupError:
return 0
password = sha.new(password).hexdigest()
return self.finish_auth(dbpw == password)
password_dig = sha.new(password).hexdigest()
if dbpw == password_dig:
self.connection.setSessionKey(session_key(username,
self.database.realm,
password))
return self.finish_auth(dbpw == password_dig)
class PlaintextClient(Client):
extensions = ["auth"]
def start(self, username, realm, password):
return self.stub.auth(username, password)
if self.stub.auth(username, password):
return session_key(username, realm, password)
else:
return None
register_module("plaintext", StorageClass, PlaintextClient, Database)
trunk/src/ZEO/tests/forker.py
View file @ aa195678
......@@ -70,20 +70,20 @@ def start_zeo_server(storage_conf, zeo_conf, port, keep=0):
Returns the ZEO port, the test server port, the pid, and the path
to the config file.
"""
# Store the config info in a temp file.
tmpfile = tempfile.mktemp(".conf")
fp = open(tmpfile, 'w')
zeo_conf.dump(fp)
fp.write(storage_conf)
fp.close()
# Find the zeoserver script
import ZEO.tests.zeoserver
script = ZEO.tests.zeoserver.__file__
if script.endswith('.pyc'):
script = script[:-1]
# Create a list of arguments, which we'll tuplify below
qa = _quote_arg
args = [qa(sys.executable), qa(script), '-C', qa(tmpfile)]
......
trunk/src/ZEO/tests/testAuth.py
View file @ aa195678
......@@ -18,8 +18,11 @@ import tempfile
import time
import unittest
import zLOG
from ThreadedAsync import LoopCallback
from ZEO.ClientStorage import ClientStorage
from ZEO.Exceptions import ClientDisconnected
from ZEO.StorageServer import StorageServer
from ZEO.tests.ConnectionTests import CommonSetupTearDown
......@@ -30,7 +33,7 @@ class AuthTest(CommonSetupTearDown):
__super_getServerConfig = CommonSetupTearDown.getServerConfig
__super_setUp = CommonSetupTearDown.setUp
__super_tearDown = CommonSetupTearDown.tearDown
realm = None
def setUp(self):
......@@ -74,7 +77,9 @@ class AuthTest(CommonSetupTearDown):
self.assert_(self._storage._connection)
self._storage._connection.poll()
self.assert_(self._storage.is_connected())
# Make a call to make sure the mechanism is working
self._storage.versions()
def testNOK(self):
self._storage = self.openClientStorage(wait=0, username="foo",
password="noogie",
......@@ -83,6 +88,20 @@ class AuthTest(CommonSetupTearDown):
# If the test established a connection, then it failed.
self.failIf(self._storage._connection)
def testUnauthenticatedMessage(self):
# Test that an unauthenticated message is rejected by the server
# if it was sent after the connection was authenticated.
# Sleep for 0.2 seconds to give the server some time to start up
# seems to be needed before and after creating the storage
self._storage = self.openClientStorage(wait=0, username="foo",
password="bar", realm=self.realm)
self.wait()
self._storage.versions()
# Manually clear the state of the hmac connection
self._storage._connection._SizedMessageAsyncConnection__hmac_send = None
# Once the client stops using the hmac, it should be disconnected.
self.assertRaises(ClientDisconnected, self._storage.versions)
class PlainTextAuth(AuthTest):
import ZEO.tests.auth_plaintext
protocol = "plaintext"
......@@ -108,4 +127,3 @@ def test_suite():
if __name__ == "__main__":
unittest.main(defaultTest='test_suite')
trunk/src/ZEO/tests/testClientCache.py
View file @ aa195678
......@@ -396,7 +396,7 @@ class PersistentClientCacheTests(unittest.TestCase):
version = "folio"
vdata = "Mend your speech a little, lest you may mar your fortunes."
vserial = "12346789"
self.cache.store(self._oid, nvdata, nvserial, version, vdata, vserial)
self.cache.checkSize(10 * self.cachesize) # force a cache flip
......@@ -421,7 +421,7 @@ class PersistentClientCacheTests(unittest.TestCase):
data, serial = self.cache.load(self._oid, xversion)
self.assertEqual(data, xdata)
self.assertEqual(serial, xserial)
class ClientCacheLongOIDTests(ClientCacheTests):
_oid = 'abcdefghijklmnop' * 2
_oid2 = 'bcdefghijklmnopq' * 2
......
trunk/src/ZEO/tests/testMonitor.py
View file @ aa195678
......@@ -13,7 +13,7 @@
##############################################################################
"""Test that the monitor produce sensible results.
$Id: testMonitor.py,v 1.6 2003/05/30 19:20:56 jeremy Exp $
$Id: testMonitor.py,v 1.7 2003/10/02 18:17:21 jeremy Exp $
"""
import socket
......@@ -70,7 +70,7 @@ class MonitorTests(CommonSetupTearDown):
s.parse("\n".join(sect[1:]))
return d
def getConfig(self, path, create, read_only):
return """<mappingstorage 1/>"""
......
trunk/src/ZEO/tests/testZEO.py
View file @ aa195678
......@@ -132,7 +132,8 @@ class GenericTests(
self._servers = [adminaddr]
self._conf_path = path
self._storage = ClientStorage(zport, '1', cache_size=20000000,
min_disconnect_poll=0.5, wait=1)
min_disconnect_poll=0.5, wait=1,
wait_timeout=60)
self._storage.registerDB(DummyDB(), None)
def tearDown(self):
......
trunk/src/ZEO/tests/zeoserver.py
View file @ aa195678
......@@ -122,8 +122,11 @@ class Suicide(threading.Thread):
self._adminaddr = addr
def run(self):
# If this process doesn't exit in 300 seconds, commit suicide
time.sleep(300)
# If this process doesn't exit in 330 seconds, commit suicide.
# The client threads in the ConcurrentUpdate tests will run for
# as long as 300 seconds. Set this timeout to 330 to minimize
# chance that the server gives up before the clients.
time.sleep(330)
log("zeoserver", "suicide thread invoking shutdown")
from ZEO.tests.forker import shutdown_zeo_server
# XXX If the -k option was given to zeoserver, then the process will
......@@ -134,7 +137,7 @@ class Suicide(threading.Thread):
def main():
label = 'zeoserver:%d' % os.getpid()
log(label, 'starting')
# We don't do much sanity checking of the arguments, since if we get it
# wrong, it's a bug in the test suite.
keep = 0
......@@ -150,11 +153,11 @@ def main():
zo = ZEOOptions()
zo.realize(["-C", configfile])
zeo_port = int(zo.address[1])
# XXX a hack
if zo.auth_protocol == "plaintext":
import ZEO.tests.auth_plaintext
# Open the config file and let ZConfig parse the data there. Then remove
# the config file, otherwise we'll leave turds.
# The rest of the args are hostname, portnum
......@@ -186,7 +189,7 @@ def main():
storage.close()
cleanup(storage)
sys.exit(2)
t.register_socket(server.dispatcher)
# Create daemon suicide thread
d = Suicide(test_addr)
......
trunk/src/ZEO/zeopasswd.py
View file @ aa195678
......@@ -17,16 +17,12 @@
usage: python zeopasswd.py [options] username [password]
-C/--configuration URL -- configuration file or URL
-p/--protocol -- authentication protocol name
-f/--filename -- authentication database filename
-r/--realm -- authentication database realm
-d/--delete -- delete user instead of updating password
"""
import getopt
import getpass
import sys
import os
import ZConfig
import ZEO
......@@ -39,41 +35,22 @@ def usage(msg):
def options(args):
"""Password-specific options loaded from regular ZEO config file."""
schema = ZConfig.loadSchema(os.path.join(os.path.dirname(ZEO.__file__),
"schema.xml"))
try:
options, args = getopt.getopt(args, "dr:p:f:C:", ["configure=",
"protocol=",
"filename=",
"realm"])
options, args = getopt.getopt(args, "C:", ["configure="])
except getopt.error, msg:
usage(msg)
config = None
delete = 0
auth_protocol = None
auth_db = ""
auth_realm = None
delete = False
for k, v in options:
if k == '-C' or k == '--configure':
schemafile = os.path.join(os.path.dirname(ZEO.__file__),
"schema.xml")
schema = ZConfig.loadSchema(schemafile)
config, nil = ZConfig.loadConfig(schema, v)
if k == '-d' or k == '--delete':
delete = 1
if k == '-p' or k == '--protocol':
auth_protocol = v
if k == '-f' or k == '--filename':
auth_db = v
if k == '-r' or k == '--realm':
auth_realm = v
if config is not None:
if auth_protocol or auth_db:
usage("Conflicting options; use either -C *or* -p and -f")
auth_protocol = config.zeo.authentication_protocol
auth_db = config.zeo.authentication_database
auth_realm = config.zeo.authentication_realm
elif not (auth_protocol and auth_db):
usage("Must specifiy configuration file or protocol and database")
delete = True
if config is None:
usage("Must specifiy configuration file")
password = None
if delete:
......@@ -91,22 +68,21 @@ def options(args):
username = args[0]
else:
username, password = args
return auth_protocol, auth_db, auth_realm, delete, username, password
return config.zeo, delete, username, password
def main(args=None):
p, auth_db, auth_realm, delete, username, password = options(args)
options, delete, username, password = options(args)
p = options.authentication_protocol
if p is None:
usage("ZEO configuration does not specify authentication-protocol")
if p == "digest":
from ZEO.auth.auth_digest import DigestDatabase as Database
elif p == "srp":
from ZEO.auth.auth_srp import SRPDatabase as Database
else:
raise ValueError, "Unknown database type %r" % p
if auth_db is None:
if options.authentication_database is None:
usage("ZEO configuration does not specify authentication-database")
db = Database(auth_db, auth_realm)
db = Database(options.authentication_database)
if delete:
db.del_user(username)
else:
......@@ -116,5 +92,4 @@ def main(args=None):
db.save()
if __name__ == "__main__":
main(sys.argv[1:])
main(sys.argv)
trunk/src/ZEO/zrpc/client.py
View file @ aa195678
......@@ -105,6 +105,7 @@ class ConnectionManager(object):
if self.trigger is not None:
self.trigger.close()
self.trigger = None
ThreadedAsync.remove_loop_callback(self.set_async)
def set_async(self, map):
# This is the callback registered with ThreadedAsync. The
......@@ -406,7 +407,7 @@ class ConnectThread(threading.Thread):
del wrappers[wrap]
# XXX should check deadline
class ConnectWrapper:
"""An object that handles the connection procedure for one socket.
......
trunk/src/ZEO/zrpc/connection.py
View file @ aa195678
......@@ -126,11 +126,15 @@ class Connection(smac.SizedMessageAsyncConnection, object):
# Protocol history:
#
# Z200 -- original ZEO 2.0 protocol
# Z200 -- Original ZEO 2.0 protocol
#
# Z201 -- added invalidateTransaction() to client;
# renamed several client methods;
# added lastTransaction() to server
# Z201 -- Added invalidateTransaction() to client.
# Renamed several client methods.
# Added several sever methods:
# lastTransaction()
# getAuthProtocol() and scheme-specific authentication methods
# getExtensionMethods().
# getInvalidations().
def __init__(self, sock, addr, obj=None):
self.obj = None
......
trunk/src/ZEO/zrpc/server.py
View file @ aa195678
......@@ -31,7 +31,6 @@ class Dispatcher(asyncore.dispatcher):
self.__super_init()
self.addr = addr
self.factory = factory
self.clients = []
self._open_socket()
def _open_socket(self):
......@@ -58,4 +57,3 @@ class Dispatcher(asyncore.dispatcher):
return
c = self.factory(sock, addr)
log("connect from %s: %s" % (repr(addr), c))
self.clients.append(c)
trunk/src/ZEO/zrpc/smac.py
View file @ aa195678
......@@ -64,7 +64,7 @@ del tmp_dict
# that we could pass to send() without blocking.
SEND_SIZE = 60000
MAC_BIT = 0x80000000
MAC_BIT = 0x80000000L
class SizedMessageAsyncConnection(asyncore.dispatcher):
__super_init = asyncore.dispatcher.__init__
......@@ -96,12 +96,33 @@ class SizedMessageAsyncConnection(asyncore.dispatcher):
self.__output_lock = threading.Lock() # Protects __output
self.__output = []
self.__closed = 0
self.__hmac = None
# Each side of the connection sends and receives messages. A
# MAC is generated for each message and depends on each
# previous MAC; the state of the MAC generator depends on the
# history of operations it has performed. So the MACs must be
# generated in the same order they are verified.
# Each side is guaranteed to receive messages in the order
# they are sent, but there is no ordering constraint between
# message sends and receives. If the two sides are A and B
# and message An indicates the nth message sent by A, then
# A1 A2 B1 B2 and A1 B1 B2 A2 are both legitimate total
# orderings of the messages.
# As a result, there must be seperate MAC generators for each
# side of the connection. If not, the generator state would
# be different after A1 A2 B1 B2 than it would be after
# A1 B1 B2 A2; if the generator state was different, the MAC
# could not be verified.
self.__hmac_send = None
self.__hmac_recv = None
self.__super_init(sock, map)
def setSessionKey(self, sesskey):
log("set session key %r" % sesskey)
self.__hmac = hmac.HMAC(sesskey, digestmod=sha)
self.__hmac_send = hmac.HMAC(sesskey, digestmod=sha)
self.__hmac_recv = hmac.HMAC(sesskey, digestmod=sha)
def get_addr(self):
return self.addr
......@@ -150,16 +171,18 @@ class SizedMessageAsyncConnection(asyncore.dispatcher):
inp = "".join(inp)
offset = 0
expect_mac = 0
has_mac = 0
while (offset + msg_size) <= input_len:
msg = inp[offset:offset + msg_size]
offset = offset + msg_size
if not state:
msg_size = struct.unpack(">i", msg)[0]
expect_mac = msg_size & MAC_BIT
if expect_mac:
msg_size = struct.unpack(">I", msg)[0]
has_mac = msg_size & MAC_BIT
if has_mac:
msg_size ^= MAC_BIT
msg_size += 20
elif self.__hmac_send:
raise ValueError("Received message without MAC")
state = 1
else:
msg_size = 4
......@@ -174,12 +197,12 @@ class SizedMessageAsyncConnection(asyncore.dispatcher):
# incoming call to be handled. During all this
# time, the __input_lock is held. That's a good
# thing, because it serializes incoming calls.
if expect_mac:
if has_mac:
mac = msg[:20]
msg = msg[20:]
if self.__hmac:
self.__hmac.update(msg)
_mac = self.__hmac.digest()
if self.__hmac_recv:
self.__hmac_recv.update(msg)
_mac = self.__hmac_recv.digest()
if mac != _mac:
raise ValueError("MAC failed: %r != %r"
% (_mac, mac))
......@@ -245,8 +268,9 @@ class SizedMessageAsyncConnection(asyncore.dispatcher):
def message_output(self, message):
if __debug__:
if self._debug:
log('message_output %d bytes: %s' %
(len(message), short_repr(message)),
log("message_output %d bytes: %s hmac=%d" %
(len(message), short_repr(message),
self.__hmac_send and 1 or 0),
level=zLOG.TRACE)
if self.__closed:
......@@ -255,12 +279,12 @@ class SizedMessageAsyncConnection(asyncore.dispatcher):
self.__output_lock.acquire()
try:
# do two separate appends to avoid copying the message string
if self.__hmac:
self.__output.append(struct.pack(">i", len(message) | MAC_BIT))
self.__hmac.update(message)
self.__output.append(self.__hmac.digest())
if self.__hmac_send:
self.__output.append(struct.pack(">I", len(message) | MAC_BIT))
self.__hmac_send.update(message)
self.__output.append(self.__hmac_send.digest())
else:
self.__output.append(struct.pack(">i", len(message)))
self.__output.append(struct.pack(">I", len(message)))
if len(message) <= SEND_SIZE:
self.__output.append(message)
else:
......
trunk/src/ZEO/zrpc/trigger.py
View file @ aa195678
......@@ -167,6 +167,15 @@ else:
self.lock = thread.allocate_lock()
self.thunks = []
self._trigger_connected = 0
self._closed = 0
def close(self):
if not self._closed:
self._closed = 1
self.del_channel()
# self.socket is a, self.trigger is w from __init__
self.socket.close()
self.trigger.close()
def __repr__(self):
return '<select-trigger (loopback) at %x>' % id(self)
......
trunk/src/ZODB/BaseStorage.py
View file @ aa195678
......@@ -13,7 +13,7 @@
##############################################################################
"""Handy standard storage machinery
$Id: BaseStorage.py,v 1.35 2003/09/15 16:29:15 jeremy Exp $
$Id: BaseStorage.py,v 1.36 2003/10/02 18:17:19 jeremy Exp $
"""
import cPickle
import time
......
trunk/src/ZODB/Connection.py
View file @ aa195678
......@@ -13,7 +13,7 @@
##############################################################################
"""Database connection support
$Id: Connection.py,v 1.99 2003/09/15 16:29:15 jeremy Exp $"""
$Id: Connection.py,v 1.100 2003/10/02 18:17:19 jeremy Exp $"""
from __future__ import nested_scopes
......@@ -308,12 +308,12 @@ class Connection(ExportImport.ExportImport, object):
method_name, args, kw = self.__onCommitActions.pop(0)
getattr(self, method_name)(transaction, *args, **kw)
return
oid = object._p_oid
if self._conflicts.has_key(oid):
self.getTransaction().register(object)
raise ReadConflictError(object=object)
invalid = self._invalid
if oid is None or object._p_jar is not self:
# new object
......@@ -625,7 +625,7 @@ class Connection(ExportImport.ExportImport, object):
else:
self.getTransaction().register(obj)
raise ReadConflictError(object=obj)
def oldstate(self, object, serial):
oid=object._p_oid
p = self._storage.loadSerial(oid, serial)
......@@ -717,7 +717,7 @@ class Connection(ExportImport.ExportImport, object):
# the connection does not match what is written to the
# database. Invalidate the object here to guarantee that
# the new state is read the next time the object is used.
if not store_return:
return
if isinstance(store_return, StringType):
......@@ -757,7 +757,7 @@ class Connection(ExportImport.ExportImport, object):
def callback():
d = {}
for oid in self._modified:
d[oid] = 1
d[oid] = 1
self._db.invalidate(d, self)
self._storage.tpc_finish(transaction, callback)
......
trunk/src/ZODB/DB.py
View file @ aa195678
......@@ -13,8 +13,8 @@
##############################################################################
"""Database objects
$Id: DB.py,v 1.54 2003/09/15 16:29:15 jeremy Exp $"""
__version__='$Revision: 1.54 $'[11:-2]
$Id: DB.py,v 1.55 2003/10/02 18:17:19 jeremy Exp $"""
__version__='$Revision: 1.55 $'[11:-2]
import cPickle, cStringIO, sys, POSException, UndoLogCompatible
from Connection import Connection
......@@ -145,9 +145,9 @@ class DB(UndoLogCompatible.UndoLogCompatible, object):
# We need to break circular refs to make it really go.
# XXX What objects are involved in the cycle?
connection.__dict__.clear()
return
pool.append(connection)
if len(pool)==1:
# Pool now usable again, unlock it.
......
trunk/src/ZODB/DemoStorage.py
View file @ aa195678
......@@ -79,7 +79,7 @@ method::
and call it to monitor the storage.
"""
__version__='$Revision: 1.20 $'[11:-2]
__version__='$Revision: 1.21 $'[11:-2]
import base64, time, string
from ZODB import POSException, BaseStorage, utils
......
trunk/src/ZODB/FileStorage.py
View file @ aa195678
......@@ -115,7 +115,7 @@
# may have a back pointer to a version record or to a non-version
# record.
#
__version__='$Revision: 1.137 $'[11:-2]
__version__='$Revision: 1.138 $'[11:-2]
import base64
from cPickle import Pickler, Unpickler, loads
......@@ -293,7 +293,7 @@ class FileStorage(BaseStorage.BaseStorage,
self._records_before_save = max(self._records_before_save,
len(self._index))
self._ltid = tid
# self._pos should always point just past the last
# transaction. During 2PC, data is written after _pos.
# invariant is restored at tpc_abort() or tpc_finish().
......@@ -607,7 +607,7 @@ class FileStorage(BaseStorage.BaseStorage,
self._file.seek(u64(pnv))
h_pnv = self._file.read(DATA_VERSION_HDR_LEN)
newserial = h_pnv[8:16]
if self._index.get(oid) == srcpos:
# This is a current record!
self._tindex[oid] = here
......@@ -981,7 +981,7 @@ class FileStorage(BaseStorage.BaseStorage,
else:
warn("restore could not find previous non-version data "
"at %d or %d" % (prev, bp))
return pnv
def supportsUndo(self):
......@@ -1075,7 +1075,7 @@ class FileStorage(BaseStorage.BaseStorage,
if fsync is not None: fsync(file.fileno())
self._pos = nextpos
self._index.update(self._tindex)
self._vindex.update(self._tvindex)
self._oid2serial.update(self._toid2serial)
......@@ -1084,16 +1084,16 @@ class FileStorage(BaseStorage.BaseStorage,
del self._oid2serial[oid]
except KeyError:
pass
# Update the number of records that we've written
# +1 for the transaction record
self._records_written += len(self._tindex) + 1
self._records_written += len(self._tindex) + 1
if self._records_written >= self._records_before_save:
self._save_index()
self._records_written = 0
self._records_before_save = max(self._records_before_save,
len(self._index))
self._ltid = tid
def _abort(self):
......@@ -1538,7 +1538,7 @@ class FileStorage(BaseStorage.BaseStorage,
# If the storage is empty, there's nothing to do.
if not self._index:
return
# Record pack time so we don't undo while packing
self._lock_acquire()
try:
......
trunk/src/ZODB/POSException.py
View file @ aa195678
......@@ -13,7 +13,7 @@
##############################################################################
"""ZODB-defined exceptions
$Id: POSException.py,v 1.20 2003/06/10 15:46:31 shane Exp $"""
$Id: POSException.py,v 1.21 2003/10/02 18:17:19 jeremy Exp $"""
from types import StringType, DictType
from ZODB.utils import oid_repr, serial_repr
......@@ -173,7 +173,7 @@ class UndoError(POSError):
class MultipleUndoErrors(UndoError):
"""Several undo errors occured during a single transaction."""
def __init__(self, errs):
# provide a reason and oid for clients that only look at that
UndoError.__init__(self, *errs[0])
......
trunk/src/ZODB/TmpStore.py
View file @ aa195678
......@@ -42,7 +42,7 @@ class TmpStore:
def getName(self):
return self._db.getName()
def getSize(self):
return self._pos
......
trunk/src/ZODB/Transaction.py
View file @ aa195678
......@@ -13,7 +13,7 @@
##############################################################################
"""Transaction management
$Id: Transaction.py,v 1.49 2003/06/10 15:46:31 shane Exp $
$Id: Transaction.py,v 1.50 2003/10/02 18:17:19 jeremy Exp $
"""
import time, sys, struct, POSException
......@@ -43,7 +43,7 @@ def jar_cmp(j1, j2):
except:
LOG("TM", WARNING, "jar missing sortKey() method: %s" % j2)
k2 = id(j2)
return cmp(k1, k2)
class Transaction:
......@@ -271,12 +271,12 @@ class Transaction:
def _get_jars(self, objects, subtransaction):
# Returns a list of jars for this transaction.
# Find all the jars and sort them in a globally consistent order.
# objects is a list of persistent objects and jars.
# If this is a subtransaction and a jar is not subtransaction aware,
# it's object gets delayed until the parent transaction commits.
d = {}
for o in objects:
jar = getattr(o, '_p_jar', o)
......@@ -298,7 +298,7 @@ class Transaction:
if self._non_st_objects is None:
self._non_st_objects = []
self._non_st_objects.append(o)
jars = d.values()
jars.sort(jar_cmp)
......@@ -406,7 +406,7 @@ class Transaction:
# After the tpc_abort(), call abort_sub() on all the
# subtrans-aware jars to *really* abort the subtransaction.
# Example: For Connection(), the tpc_abort() will abort the
# subtransaction TmpStore() and abort_sub() will remove the
# TmpStore.
......
trunk/src/ZODB/__init__.py
View file @ aa195678
......@@ -12,7 +12,7 @@
#
##############################################################################
__version__ = '3.2b2'
__version__ = '3.2c1'
import sys
import cPersistence, Persistence
......
trunk/src/ZODB/cPickleCache.c
View file @ aa195678
......@@ -90,7 +90,7 @@ process must skip such objects, rather than deactivating them.
static char cPickleCache_doc_string[] =
"Defines the PickleCache used by ZODB Connection objects.\n"
"\n"
"$Id: cPickleCache.c,v 1.85 2003/05/30 19:20:55 jeremy Exp $\n";
"$Id: cPickleCache.c,v 1.86 2003/10/02 18:17:19 jeremy Exp $\n";
#define ASSIGN(V,E) {PyObject *__e; __e=(E); Py_XDECREF(V); (V)=__e;}
#define UNLESS(E) if(!(E))
......@@ -499,6 +499,17 @@ cc_clear(ccobject *self, PyObject *args)
Py_DECREF(o);
}
self->ring_lock = 0;
/* Free two references to the Connection, which can't be discovered
via garbage collection.
*/
Py_XDECREF(self->setklassstate);
self->setklassstate = NULL;
Py_XDECREF(self->jar);
self->jar = NULL;
Py_INCREF(Py_None);
return Py_None;
}
......
trunk/src/ZODB/config.py
View file @ aa195678
......@@ -13,7 +13,7 @@
##############################################################################
"""Open database and storage from a configuration.
$Id: config.py,v 1.14 2003/09/15 16:29:15 jeremy Exp $"""
$Id: config.py,v 1.15 2003/10/02 18:17:19 jeremy Exp $"""
import os
from cStringIO import StringIO
......
trunk/src/ZODB/fspack.py
View file @ aa195678
......@@ -192,7 +192,7 @@ class FileStorageFormatter:
if dh.plen:
self.fail(pos, "data record has back pointer and data")
def DataHeaderFromString(s):
def DataHeaderFromString(s):
return DataHeader(*struct.unpack(DATA_HDR, s))
class DataHeader:
......@@ -338,7 +338,7 @@ class DataCopier(FileStorageFormatter):
return pos
pos += h.recordlen()
return 0
def _restore_pnv(self, oid, prev, version, bp):
# Find a valid pnv (previous non-version) pointer for this version.
......@@ -415,7 +415,7 @@ class DataCopier(FileStorageFormatter):
self._tfile.write(z64)
else:
self._tfile.write(data)
class GC(FileStorageFormatter):
def __init__(self, file, eof, packtime):
......@@ -437,7 +437,7 @@ class GC(FileStorageFormatter):
# second is a dictionary mapping objects to lists of
# positions; it is used to handle the same number of objects
# for which we must keep multiple revisions.
self.reachable = fsIndex()
self.reach_ex = {}
......@@ -460,7 +460,7 @@ class GC(FileStorageFormatter):
self.findReachableFromFuture()
# These mappings are no longer needed and may consume a lot
# of space.
del self.oid2verpos
del self.oid2verpos
del self.oid2curpos
def buildPackIndex(self):
......@@ -528,7 +528,7 @@ class GC(FileStorageFormatter):
# non-current revision could refer to objects that were
# otherwise unreachable at the packtime.
extra_roots = []
pos = self.packpos
while pos < self.eof:
th = self._read_txn_header(pos)
......@@ -558,7 +558,7 @@ class GC(FileStorageFormatter):
extra_roots.append(dh.pnv)
else:
self.reachable[dh.oid] = dh.back
pos += dh.recordlen()
tlen = self._read_num(pos)
......@@ -631,7 +631,7 @@ class FileStoragePacker(FileStorageFormatter):
self._file.seek(0, 2)
self.file_end = self._file.tell()
self._file.seek(0)
self.gc = GC(self._file, self.file_end, self._stop)
# The packer needs to acquire the parent's commit lock
......@@ -672,7 +672,7 @@ class FileStoragePacker(FileStorageFormatter):
# Txn and data records contain pointers to previous records.
# Because these pointers are stored as file offsets, they
# must be updated when we copy data.
# XXX Need to add sanity checking to pack
self.gc.findReachable()
......@@ -730,7 +730,7 @@ class FileStoragePacker(FileStorageFormatter):
self._tfile.seek(new_pos - 8)
self._tfile.write(p64(tlen))
tlen = self._read_num(pos)
if tlen != th.tlen:
self.fail(pos, "redundant transaction length does not "
......@@ -757,7 +757,7 @@ class FileStoragePacker(FileStorageFormatter):
Returns position of txn header in output file and position
of next record in the input file.
If any data records are copied, also write txn header (th).
"""
copy = 0
......@@ -878,4 +878,3 @@ class FileStoragePacker(FileStorageFormatter):
if self._lock_counter % 20 == 0:
self._commit_lock_acquire()
return ipos
trunk/src/ZODB/fsrecover.py
View file @ aa195678
......@@ -227,7 +227,7 @@ def main():
except getopt.error:
die()
print __doc__ % argv[0]
force = partial = verbose = 0
pack = None
for opt, v in opts:
......@@ -321,7 +321,7 @@ def recover(inp, outp, verbose=0, partial=0, force=0, pack=0):
l = "bp"
else:
l = len(r.data)
print "%7d %s %s" % (u64(r.oid), l, r.version)
s = ofs.restore(r.oid, r.serial, r.data, r.version,
r.data_txn, txn)
......@@ -372,4 +372,3 @@ def recover(inp, outp, verbose=0, partial=0, force=0, pack=0):
if __name__ == "__main__":
main()
trunk/src/ZODB/tests/BasicStorage.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""Run the basic tests for a storage as described in the official storage API
......
trunk/src/ZODB/tests/ConflictResolution.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""Tests for application-level conflict resolution."""
......
trunk/src/ZODB/tests/Corruption.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""Do some minimal tests of data corruption"""
......
trunk/src/ZODB/tests/HistoryStorage.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""Run the history() related tests for a storage.
......
trunk/src/ZODB/tests/IteratorStorage.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""Run tests against the iterator() interface for storages.
......
trunk/src/ZODB/tests/LocalStorage.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
class LocalStorage:
"""A single test that only make sense for local storages.
......
trunk/src/ZODB/tests/MTStorage.py
View file @ aa195678
......@@ -96,7 +96,6 @@ class ZODBClientThread(TestThread):
return root.get(name)
except ConflictError:
get_transaction().abort()
root._p_jar.sync()
class StorageClientThread(TestThread):
......
trunk/src/ZODB/tests/MinPO.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""A minimal persistent object to use for tests"""
......
trunk/src/ZODB/tests/PackableStorage.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""Run some tests relevant for storages that support pack()."""
......@@ -472,13 +472,13 @@ class PackableStorage(PackableStorageBase):
obj2 = self._newobj()
oid2 = obj2.getoid()
obj2.value = 2
# Commit the first revision of each of them
revid11 = self._dostoreNP(oid1, data=pickle.dumps(obj1),
description="1-1")
revid22 = self._dostoreNP(oid2, data=pickle.dumps(obj2),
description="2-2")
# remember the time. everything above here will be packed away
snooze()
packtime = time.time()
......@@ -509,7 +509,7 @@ class ClientThread(threading.Thread):
def __init__(self, db):
threading.Thread.__init__(self)
self.root = db.open().root()
def run(self):
for j in range(50):
try:
......
trunk/src/ZODB/tests/PersistentStorage.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""Test that a storage's values persist across open and close."""
......
trunk/src/ZODB/tests/ReadOnlyStorage.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
from ZODB.POSException import ReadOnlyError
from ZODB.Transaction import Transaction
......
trunk/src/ZODB/tests/RecoveryStorage.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""More recovery and iterator tests."""
......@@ -71,7 +71,7 @@ class RecoveryStorage(IteratorDeepCompare):
self.assertRaises(IndexError, lambda i, t=trans: t[i], 1)
self.assertEqual(data.oid, oid)
self.assertEqual(data.data, None)
def checkRecoverUndoInVersion(self):
oid = self._storage.new_oid()
version = "aVersion"
......@@ -128,7 +128,7 @@ class RecoveryStorage(IteratorDeepCompare):
self._dst = self.new_dest()
self._dst.copyTransactionsFrom(self._storage)
self.compare(self._storage, self._dst)
def checkRestoreAcrossPack(self):
db = DB(self._storage)
c = db.open()
......
trunk/src/ZODB/tests/RevisionStorage.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""Check loadSerial() on storages that support historical revisions."""
......
trunk/src/ZODB/tests/StorageTestBase.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""Provide a mixin base class for storage tests.
......@@ -210,7 +210,7 @@ class StorageTestBase(unittest.TestCase):
def _dostoreNP(self, oid=None, revid=None, data=None, version=None,
user=None, description=None):
return self._dostore(oid, revid, data, version, 1, user, description)
# The following methods depend on optional storage features.
def _undo(self, tid, oid=None):
......
trunk/src/ZODB/tests/Synchronization.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""Test the storage's implemenetation of the storage synchronization spec.
......
trunk/src/ZODB/tests/TransactionalUndoStorage.py
View file @ aa195678
......@@ -42,7 +42,7 @@ def snooze():
now = time.time()
while now == time.time():
time.sleep(0.1)
def listeq(L1, L2):
"""Return True if L1.sort() == L2.sort()"""
c1 = L1[:]
......@@ -608,11 +608,11 @@ class TransactionalUndoStorage:
txn.commit()
set_pack_time()
root._p_deactivate()
cn.sync()
self.assert_(listeq(root.keys(), ["key0", "key2"]))
L = db.undoInfo()
db.undo(L[0]["id"])
txn = get_transaction()
......@@ -624,7 +624,7 @@ class TransactionalUndoStorage:
root._p_deactivate()
cn.sync()
self.assert_(listeq(root.keys(), ["key0", "key1", "key2"]))
for t in pack_times:
self._storage.pack(t, referencesf)
......@@ -663,7 +663,7 @@ class TransactionalUndoStorage:
self.assertEqual(rt["test"].value, i % 2 and 3 or 1)
self.assertEqual(rt["test2"].value, 2)
packtimes.append(time.time())
snooze()
......@@ -723,7 +723,7 @@ class TransactionalUndoStorage:
s.transactionalUndo(tid, t)
s.tpc_vote(t)
s.tpc_finish(t)
for i in range(BATCHES):
undo(i)
......@@ -740,14 +740,14 @@ class TransactionalUndoStorage:
for i in range(BATCHES):
txn = iter[offset]
offset += 1
tid = p64(i + 1)
eq(txn.tid, tid)
L1 = [(rec.oid, rec.serial, rec.data_txn) for rec in txn]
L2 = [(oid, revid, None) for _tid, oid, revid in orig
if _tid == tid]
eq(L1, L2)
for i in range(BATCHES * OBJECTS):
......@@ -789,5 +789,3 @@ class TransactionalUndoStorage:
self.assertEqual(d['description'],'t1')
self.assertEqual(d['k2'],'this is transaction metadata')
self.assertEqual(d['user_name'],'p3 u3')
trunk/src/ZODB/tests/TransactionalUndoVersionStorage.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
from __future__ import nested_scopes
......@@ -60,7 +60,7 @@ class TransactionalUndoVersionStorage:
self.assertEqual(zodb_unpickle(data), MinPO(versiondata))
data, revid = self._storage.load(oid, '')
self.assertEqual(zodb_unpickle(data), MinPO(nonversiondata))
oid = self._storage.new_oid()
version = 'one'
revid_a = self._dostore(oid, data=MinPO(91))
......@@ -68,17 +68,17 @@ class TransactionalUndoVersionStorage:
version=version)
revid_c = self._dostore(oid, revid=revid_b, data=MinPO(93),
version=version)
info = self._storage.undoInfo()
self._undo(info[0]['id'], oid)
data, revid = self._storage.load(oid, '')
eq(revid, revid_a)
eq(zodb_unpickle(data), MinPO(91))
data, revid = self._storage.load(oid, version)
unless(revid > revid_b and revid > revid_c)
eq(zodb_unpickle(data), MinPO(92))
# Now commit the version...
t = Transaction()
self._storage.tpc_begin(t)
......@@ -89,7 +89,7 @@ class TransactionalUndoVersionStorage:
eq(oids[0], oid)
check_objects(92, 92)
# ...and undo the commit
info = self._storage.undoInfo()
self._undo(info[0]['id'], oid)
......@@ -101,11 +101,11 @@ class TransactionalUndoVersionStorage:
assert oids[0] == oid
check_objects(91, 91)
# Now undo the abort
info=self._storage.undoInfo()
self._undo(info[0]['id'], oid)
check_objects(91, 92)
def checkUndoCommitVersion(self):
......
trunk/src/ZODB/tests/VersionStorage.py
View file @ aa195678
......@@ -493,7 +493,7 @@ class VersionStorage:
self._storage.pack(time.time(), referencesf)
cn.sync()
cn._cache.clear()
# make sure all the non-version data is there
for name, obj in root.items():
self.assertEqual(name, obj.value)
......@@ -516,16 +516,15 @@ class VersionStorage:
t = time.time()
snooze()
L = db.undoInfo()
db.undo(L[0]["id"])
txn = get_transaction()
txn.note("undo abort")
txn.commit()
self._storage.pack(t, referencesf)
cn2 = db.open(version="b")
rt2 = cn2.root()
self.assertEqual(rt2["b"].value.value, "still version")
trunk/src/ZODB/tests/dangle.py
View file @ aa195678
......@@ -4,14 +4,14 @@
#
# Copyright (c) 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE
#
#
##############################################################################
"""Functional test to produce a dangling reference."""
......@@ -61,6 +61,6 @@ def main():
db = DB(fs)
create_dangling_ref(db)
db.close()
if __name__ == "__main__":
main()
trunk/src/ZODB/tests/testCache.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""A few simple tests of the public cache API.
......
trunk/src/ZODB/tests/testConfig.py
View file @ aa195678
......@@ -56,6 +56,7 @@ class ZODBConfigTest(ConfigTestBase):
""")
def test_file_config1(self):
import ZODB.FileStorage
path = tempfile.mktemp()
self._test(
"""
......@@ -65,9 +66,10 @@ class ZODBConfigTest(ConfigTestBase):
</filestorage>
</zodb>
""" % path)
os.unlink(path)
ZODB.FileStorage.cleanup(path)
def test_file_config2(self):
import ZODB.FileStorage
path = tempfile.mktemp()
cfg = """
<zodb>
......@@ -79,6 +81,7 @@ class ZODBConfigTest(ConfigTestBase):
</zodb>
""" % path
self.assertRaises(ReadOnlyError, self._test, cfg)
ZODB.FileStorage.cleanup(path)
def test_zeo_config(self):
# We're looking for a port that doesn't exist so a connection attempt
......
trunk/src/ZODB/tests/testDB.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
import os
import time
......@@ -57,14 +57,14 @@ class DBTests(unittest.TestCase):
# Test that we can remove a version pool
# This is white box because we check some internal data structures
self.dowork()
self.dowork('v2')
c1 = self.db.open('v1')
c1.close() # return to pool
c12 = self.db.open('v1')
c12.close() # return to pool
self.assert_(c1 is c12) # should be same
self.assert_(c1 is c12) # should be same
pools, pooll = self.db._pools
......@@ -78,7 +78,7 @@ class DBTests(unittest.TestCase):
c12 = self.db.open('v1')
c12.close() # return to pool
self.assert_(c1 is not c12) # should be different
self.assert_(c1 is not c12) # should be different
self.assertEqual(len(pools), 3)
self.assertEqual(len(pooll), 3)
......@@ -95,13 +95,13 @@ class DBTests(unittest.TestCase):
# Test that we can remove a version pool
# This is white box because we check some internal data structures
self.dowork()
self.dowork('v2')
c1 = self.db.open('v1')
c1.close() # return to pool
c12 = self.db.open('v1')
self.assert_(c1 is c12) # should be same
self.assert_(c1 is c12) # should be same
pools, pooll = self.db._pools
......@@ -120,7 +120,7 @@ class DBTests(unittest.TestCase):
c12 = self.db.open('v1')
c12.close() # return to pool
self.assert_(c1 is not c12) # should be different
self.assert_(c1 is not c12) # should be different
self.assertEqual(len(pools), 3)
self.assertEqual(len(pooll), 3)
......
trunk/src/ZODB/tests/testFileStorage.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
from __future__ import nested_scopes
......@@ -92,21 +92,21 @@ class FileStorageTests(
from ZODB.fsIndex import fsIndex
# Hack FileStorage to create dictionary indexes
self._storage = OldFileStorage('FileStorageTests.fs')
self.assertEqual(type(self._storage._index), type({}))
for i in range(10):
self._dostore()
# Should save the index
self._storage.close()
self._storage = ZODB.FileStorage.FileStorage(
'FileStorageTests.fs', read_only=1)
self.assertEqual(type(self._storage._index), type({}))
def check_conversion_to_fsIndex(self):
self.tearDown()
......@@ -117,16 +117,16 @@ class FileStorageTests(
from ZODB.fsIndex import fsIndex
# Hack FileStorage to create dictionary indexes
self._storage = OldFileStorage('FileStorageTests.fs')
self.assertEqual(type(self._storage._index), type({}))
for i in range(10):
self._dostore()
oldindex = self._storage._index.copy()
# Should save the index
self._storage.close()
......@@ -151,7 +151,7 @@ class FileStorageTests(
# This would make the unit tests too slow
# check_save_after_load_that_worked_hard(self)
# check_save_after_load_that_worked_hard(self)
def check_periodic_save_index(self):
......
trunk/src/ZODB/tests/testPersistentList.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""Test the list interface to PersistentList
"""
......
trunk/src/ZODB/tests/testPersistentMapping.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""Verify that PersistentMapping works with old versions of Zope.
......
trunk/src/ZODB/tests/testRecover.py
View file @ aa195678
......@@ -151,5 +151,3 @@ class RecoverTest(unittest.TestCase):
def test_suite():
return unittest.makeSuite(RecoverTest)
trunk/src/ZODB/tests/testTimeStamp.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""Test the TimeStamp utility type"""
......
trunk/src/ZODB/tests/testTransaction.py
View file @ aa195678
......@@ -14,7 +14,7 @@
"""
Revision information:
$Id: testTransaction.py,v 1.12 2003/01/27 20:29:50 bwarsaw Exp $
$Id: testTransaction.py,v 1.13 2003/10/02 18:17:17 jeremy Exp $
"""
"""
......@@ -599,7 +599,7 @@ class HoserJar(BasicJar):
# The HoserJars coordinate their actions via the class variable
# committed. The check() method will only raise its exception
# if committed > 0.
# if committed > 0.
committed = 0
......@@ -615,7 +615,7 @@ class HoserJar(BasicJar):
self.check('tpc_finish')
self.ctpc_finish += 1
HoserJar.committed += 1
def test_suite():
......
trunk/src/ZODB/tests/testUtils.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
"""Test the routines to convert between long and 64-bit strings"""
......
trunk/src/ZODB/tests/testZODB.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
from __future__ import nested_scopes
......@@ -213,7 +213,7 @@ class ZODBTests(unittest.TestCase):
r2 = cn2.root()
self.assertEqual(r1._p_serial, r2._p_serial)
self.obj.child2 = P()
conn.getTransaction().commit()
......
trunk/src/ZODB/tests/testfsIndex.py
View file @ aa195678
......@@ -2,14 +2,14 @@
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
#
##############################################################################
import unittest, sys
from ZODB.fsIndex import fsIndex
......
trunk/src/ZODB/transact.py
View file @ aa195678
......@@ -34,7 +34,7 @@ def transact(f, note=None, retries=5):
"""
# XXX deal with ZEO disconnected errors?
def g(*args, **kwargs):
n = retries
while n:
......
trunk/src/ZODB/utils.py
View file @ aa195678
......@@ -96,4 +96,3 @@ def oid_repr(oid):
return repr(oid)
serial_repr = oid_repr
trunk/src/scripts/README.txt
View file @ aa195678
This directory contains a collect of utilities for managing ZODB
This directory contains a collection of utilities for managing ZODB
databases. Some are more useful than others. If you install ZODB
using distutils ("python setup.py install"), fsdump.py, fstest.py,
repozo.py, and zeopack.py will be installed in /usr/local/bin.
......@@ -7,27 +7,27 @@ Unless otherwise noted, these scripts are invoked with the name of the
Data.fs file as their only argument. Example: checkbtrees.py data.fs.
analyze.py -- A transaction analyzer for FileStorage
analyze.py -- a transaction analyzer for FileStorage
Reports on the data in a FileStorage. The report is organized by
class. It shows total data, as well as separate reports for current
and historical revisions of objects.
checkbtrees.py -- Checks BTrees in a FileStorage for corruption.
checkbtrees.py -- checks BTrees in a FileStorage for corruption
Attempts to find all the BTrees contained in a Data.fs and calls their
_check() methods.
Attempts to find all the BTrees contained in a Data.fs, calls their
_check() methods, and runs them through BTrees.check.check().
fsdump.py -- Summarize FileStorage contents, one line per revision.
fsdump.py -- summarize FileStorage contents, one line per revision
Prints a report of FileStorage contents, with one line for each
transaction and one line for each data record in that transaction.
Includes time stamps, file positions, and class names.
fstest.py -- Simple consistency checker for FileStorage
fstest.py -- simple consistency checker for FileStorage
usage: fstest.py [-v] data.fs
......@@ -46,7 +46,14 @@ possible for the damage to occur only in the part of the file that
stores object pickles. Those errors will go undetected.
netspace.py -- Hackish attempt to report on size of objects
space.py -- report space used by objects in a FileStorage
usage: space.py [-v] data.fs
This ignores revisions and versions.
netspace.py -- hackish attempt to report on size of objects
usage: netspace.py [-P | -v] data.fs
......@@ -57,7 +64,7 @@ Traverses objects from the database root and attempts to calculate
size of object, including all reachable subobjects.
parsezeolog.py -- Parse BLATHER logs from ZEO server.
parsezeolog.py -- parse BLATHER logs from ZEO server
This script may be obsolete. It has not been tested against the
current log output of the ZEO server.
......@@ -66,12 +73,12 @@ Reports on the time and size of transactions committed by a ZEO
server, by inspecting log messages at BLATHER level.
repozo.py -- Incremental backup utility for FileStorage.
repozo.py -- incremental backup utility for FileStorage
Run the script with the -h option to see usage details.
timeout.py -- Script to test transaction timeout
timeout.py -- script to test transaction timeout
usage: timeout.py address delay [storage-name]
......@@ -80,13 +87,13 @@ and tpc_vote(), and then sleeps forever. This should trigger the
transaction timeout feature of the server.
zeopack.py -- Script to pack a ZEO server.
zeopack.py -- pack a ZEO server
The script connects to a server and calls pack() on a specific
storage. See the script for usage details.
zeoreplay.py -- Experimental script to replay transactions from a ZEO log.
zeoreplay.py -- experimental script to replay transactions from a ZEO log
Like parsezeolog.py, this may be obsolete because it was written
against an earlier version of the ZEO server. See the script for
......@@ -95,7 +102,7 @@ usage details.
zeoup.py
Usage: zeoup.py [options]
usage: zeoup.py [options]
The test will connect to a ZEO server, load the root object, and
attempt to update the zeoup counter in the root. It will report
......@@ -106,13 +113,34 @@ start a transaction.
See the script for details about the options.
zodbload.py - exercise ZODB under a heavy synthesized Zope-like load
zodbload.py -- exercise ZODB under a heavy synthesized Zope-like load
See the module docstring for details. Note that this script requires
Zope. New in ZODB3 3.1.4.
zeoserverlog.py - analyze ZEO server log for performance statistics
zeoserverlog.py -- analyze ZEO server log for performance statistics
See the module docstring for details; there are a large number of
options. New in ZODB3 3.1.4.
\ No newline at end of file
options. New in ZODB3 3.1.4.
fsrefs.py -- check FileStorage for dangling references
fstail.py -- display the most recent transactions in a FileStorage
usage: fstail.py [-n nxtn] data.fs
The most recent ntxn transactions are displayed, to stdout.
Optional argument -n specifies ntxn, and defaults to 10.
migrate.py -- do a storage migration and gather statistics
See the module docstring for details.
zeoqueue.py -- report number of clients currently waiting in the ZEO queue
See the module docstring for details.
trunk/src/scripts/tests/testfstest.py
View file @ aa195678
......@@ -160,7 +160,7 @@ class TestCorruptedFS(unittest.TestCase):
newlen = struct.pack(">II", 0, tl - (len(data) - 24))
self._file.write(newlen)
self.detectsError("truncated at")
def testBadDataLength(self):
self.copyTransactions(1)
tl, data = self.getHeader()
......@@ -180,4 +180,3 @@ class TestCorruptedFS(unittest.TestCase):
if __name__ == "__main__":
unittest.main()
trunk/src/scripts/tests/testzeopack.py
View file @ aa195678
......@@ -26,7 +26,7 @@ class PackerTests(StorageTestBase):
self.started = 0
def start(self):
self.started =1
self.started =1
self.path = tempfile.mktemp(suffix=".fs")
self._storage = FileStorage(self.path)
self.db = ZODB.DB(self._storage)
......@@ -105,7 +105,7 @@ class PackerTests(StorageTestBase):
assert os.path.exists(self.path + ".old")
class UpTest(unittest.TestCase):
def testUp(self):
status = os.system("zeoup.py -p 19")
# There is no ZEO server on port 19, so we should see non-zero
......
trunk/src/scripts/zeoserverlog.py
View file @ aa195678
......@@ -140,7 +140,7 @@ Commands:
- wall time to verify
- average miliseconds to verify per object.
$Id: zeoserverlog.py,v 1.2 2003/09/15 16:29:19 jeremy Exp $
$Id: zeoserverlog.py,v 1.3 2003/10/02 18:17:26 jeremy Exp $
"""
import datetime, sys, re, os
......
trunk/src/scripts/zodbload.py
View file @ aa195678
......@@ -88,7 +88,7 @@ Usage: loadmail2 [options]
Specify the mailbox for getting input data.
$Id: zodbload.py,v 1.2 2003/09/15 16:29:19 jeremy Exp $
$Id: zodbload.py,v 1.3 2003/10/02 18:17:26 jeremy Exp $
"""
import mailbox
......
trunk/src/zLOG/BaseLogger.py
View file @ aa195678
......@@ -21,4 +21,3 @@ class BaseLogger:
for handler in self.logger.handlers:
if hasattr(handler, 'reopen') and callable(handler.reopen):
handler.reopen()
trunk/src/zLOG/LogHandlers.py
View file @ aa195678
......@@ -75,8 +75,3 @@ class StartupHandler(Handler):
for record in self.buffer:
target.handle(record)
self.buffer = []
trunk/src/zLOG/tests/testzLog.py
View file @ aa195678
......@@ -17,6 +17,7 @@ import sys
import tempfile
import unittest
import zLOG
import logging
severity_string = {
-300: 'TRACE',
......@@ -50,13 +51,24 @@ class StupidLogTest(unittest.TestCase):
self.wipeEnvironment()
self.path = tempfile.mktemp()
self._severity = 0
# Windows cannot remove a file that's open. The logging code
# keeps the log file open, and I can't find an advertised API
# to tell the logger to close a log file. So here we cheat:
# tearDown() will close and remove all the handlers that pop
# into existence after setUp() runs. This breaks into internals,
# but I couldn't find a sane way to do it.
self.handlers = logging._handlers.keys() # capture current handlers
def tearDown(self):
try:
os.remove(self.path)
except os.error:
pass
# Close and remove all the handlers that came into existence
# since setUp ran.
for h in logging._handlers.keys():
if h not in self.handlers:
h.close()
del logging._handlers[h]
os.remove(self.path)
self.wipeEnvironment()
zLOG.initialize()
def setLog(self, severity=0):
os.environ['%s_LOG_FILE' % self.prefix] = self.path
......@@ -111,14 +123,20 @@ class StupidLogTest(unittest.TestCase):
self.setLog()
zLOG.LOG("basic", zLOG.INFO, "summary")
f = self.getLogFile()
self.verifyEntry(f, subsys="basic", summary="summary")
try:
self.verifyEntry(f, subsys="basic", summary="summary")
finally:
f.close()
def checkDetail(self):
self.setLog()
zLOG.LOG("basic", zLOG.INFO, "xxx", "this is a detail")
f = self.getLogFile()
self.verifyEntry(f, subsys="basic", detail="detail")
try:
self.verifyEntry(f, subsys="basic", detail="detail")
finally:
f.close()
def checkError(self):
self.setLog()
......@@ -131,9 +149,13 @@ class StupidLogTest(unittest.TestCase):
zLOG.LOG("basic", zLOG.ERROR, "raised exception", error=err)
f = self.getLogFile()
self.verifyEntry(f, subsys="basic", summary="summary")
self.verifyEntry(f, subsys="basic", severity=zLOG.ERROR,
error=err)
try:
self.verifyEntry(f, subsys="basic", summary="summary")
self.verifyEntry(f, subsys="basic", severity=zLOG.ERROR,
error=err)
finally:
f.close()
class EventLogTest(StupidLogTest):
""" Test alternate envvars EVENT_LOG_FILE and EVENT_LOG_SEVERITY """
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7