dont close the storages mid-transaction. need to check whether we need to close them at the end of z2.py

New `catdel' command which deletes the entire catalog in one
transaction.

Add a new global VERBOSE flag instead of passing around the printstat
argument everywhere.  This also lets us use getopt for proper option
parsing, and add a -v/--verbose flag for setting that global.

Added a usage() function.

two values are tainted failed because no implicit conversion to a list took
place.

between objects which are unreachable from the root -- and thus might
be collected by autopack -- and cycles which are unreachable from the
root -- which can only be collected by a full pack.

cycle deletion tests.

collect cycles.

Also, some code updating and re-org to factor code into the base
class.

_dostore(): Support conflict resolution.

__init__(), _withtxn(): Removed.

_decrefPickle(): Fixed a typo.

BerkeleyConfig: Clarify in the docstring the semantics of the various
autopack variables, w.r.t. the differences between Full and Minimal
autopacking (the latter always does a gc pack because there's nothing
else it /could/ do :).

__init__(): We can create the packlock here, allowing us to get rid of
the __init__'s in the derived classes.

_withtxn(): Turns out to be useful for both storages.

ThreadedAsync/LoopCallback.py: when select() is interrupted by a
signal handler, don't attempt to read all file descriptors.

in the setup.py script.

cache size, and other useful BerkeleyDB parameters.

attributes for controlling autopack behavior.

three new configuration variables for controlling how autopacking
works.  Basically, you set an autopack frequency, a "packtime" -- the
point in the past you want to pack to -- and a counter for
automatically doing a classic pack.

Specific changes here include:

_setupDBs(): If autopacking is enabled, create the autopacking thread
object and get it started.

close(): When shutting down the storage, we need to stop and join the
autopacking thread, otherwise I think we have a high possibility of
corrupting our database (requiring recovery).

_dopack(): Add a flag for whether full gc should be done or not.
That's about the only difference between classic pack and autopack
(the latter does not do gc).

autopack(): The method that the autopacking thread calls to start an
autopack.  It takes a pack time with the same semantics as pack(), but
it also takes a flag specifying whether to do garbage collection of
unreachable objects or not.

_Autopack: A derived class of threading.Thread to handing the
background autopacking.

Here's how it works:

- On every store(), we write an entry to a objrev table containing the
  tuple of information (newserial, oid, oldserial).  We don't write
  this entry if the store is the first revision of an object on a new
  version.

  We do basically the same thing on restore() and transactionalUndo().

- On an abortVersion(), we write two entries to the objrev table, one
  that has (newserial, oid, oldserial) -- which points to the old
  serial in the version, and (newserial, oid, nvserial) -- which
  points to the non-version revision of the version revision.

- On commitVersion(), we do the same as abortVersion() except that we
  don't write the non-version data if we're committing to a different
  version.

- Now, when we pack, all we need to do is cruise from the beginning of
  the objrev table until we find an entry with a newserial > packtime.
  If the oldserial is ZERO, it's an object creation event which we
  don't need to worry about because there's no previous revision.  But
  otherwise, we can delete the oid+oldserial revision because we know
  it's not current.  We do this, updating pickle refcounts and then
  collecting any objects that are left unreferenced.

The cute thing is that autopacking will use the same algorithm.  The
main difference between autopack and classic pack, is that the latter
does a mark and sweep garbage collection phase after the normal objrev
collection phase.  Also, this algorithm means autopack needs only
three pieces of information:

- How often the thread should run (e.g. once per hour)

- How far in the past it should pack (e.g. pack to 4 hours ago).  We
  don't need a start time for the autopack window, because we'll
  always just start at the beginning of the objrev table.

- How often should autopack also do a classic pack (e.g. do a classic
  pack once per day).

Autopack isn't implemented in this checkin, but I believe it will be
nearly trivial to add.  That comes next.

checkTransactionalUndoAfterPackWithObjectUnlinkFromRoot()

for ZODB 3.2.

(__init__.py still makes format_exception available; is this needed?)

must've used this to disable the exception handling temporarily to
debug the exception formatting machinery, and forgotten to re-enable
it before checking in.

Unfortunately, this made it into the ZODB 3.1 release, making this a
3.1.1 backport candidate.

Fixes #651

Remade the test code for rfc822() to provide more stable tests.

Make TOC items and values sloppier.  These are utility methods used for custom introspection on the TOC.  Under concurrent use, they tended to cause a KeyError
to be raised out of __getitem__ due to a synchronization problem.  Making them more tolerant of desynchronization between _items and _data makes them useful.

Delayed Zope startup based on changes from Zope-2_7-development-branch
(with corrections), with the intent of merging this into the 2_6
branch and the head.

Until now, "import Zope" always opened the database automatically.
Unfortunately, that strategy caused the Python import lock to be held
by the main thread during database initialization, which led to a
deadlock if other threads required something to be imported before
completing initialization.  This can be a big problem for ZEO.

The only foreseen risk is that external scripts which "import Zope"
may depend in some way on the database being opened immediately.  Most
scripts just use "Zope.app()", which still works well.  If you have a
script that breaks with these changes, either add a call to the new
Zope.startup() function or set the ZOPE_COMPATIBLE_STARTUP environment
variable to a non-empty value.