Remove unnecessary Py_FatalError() calls on failed module import.

Merged remaining changes from toby-clean-shutdown-branch. This resolves the overnight problem where storages were not closed if Zope is shut down using the ZMI, and the long-standing problem where storages were closed too soon when shut down using a signal

        the Zope profiling support

#include <string.h> is implied by ExtensionClass.h, because it includes
Python.h.  Also, it is an error to include standard headers before
Python.h.

#include <stdio.h> is implied by ExtensionClass.h, because it includes
Python.h.  Also, it is an error to include standard headers before
Python.h.

Bugfix candidate.

dont close the storages mid-transaction. need to check whether we need to close them at the end of z2.py

New `catdel' command which deletes the entire catalog in one
transaction.

Add a new global VERBOSE flag instead of passing around the printstat
argument everywhere.  This also lets us use getopt for proper option
parsing, and add a -v/--verbose flag for setting that global.

Added a usage() function.

two values are tainted failed because no implicit conversion to a list took
place.

between objects which are unreachable from the root -- and thus might
be collected by autopack -- and cycles which are unreachable from the
root -- which can only be collected by a full pack.

cycle deletion tests.

collect cycles.

Also, some code updating and re-org to factor code into the base
class.

_dostore(): Support conflict resolution.

__init__(), _withtxn(): Removed.

_decrefPickle(): Fixed a typo.

BerkeleyConfig: Clarify in the docstring the semantics of the various
autopack variables, w.r.t. the differences between Full and Minimal
autopacking (the latter always does a gc pack because there's nothing
else it /could/ do :).

__init__(): We can create the packlock here, allowing us to get rid of
the __init__'s in the derived classes.

_withtxn(): Turns out to be useful for both storages.

ThreadedAsync/LoopCallback.py: when select() is interrupted by a
signal handler, don't attempt to read all file descriptors.

in the setup.py script.

cache size, and other useful BerkeleyDB parameters.

attributes for controlling autopack behavior.

three new configuration variables for controlling how autopacking
works.  Basically, you set an autopack frequency, a "packtime" -- the
point in the past you want to pack to -- and a counter for
automatically doing a classic pack.

Specific changes here include:

_setupDBs(): If autopacking is enabled, create the autopacking thread
object and get it started.

close(): When shutting down the storage, we need to stop and join the
autopacking thread, otherwise I think we have a high possibility of
corrupting our database (requiring recovery).

_dopack(): Add a flag for whether full gc should be done or not.
That's about the only difference between classic pack and autopack
(the latter does not do gc).

autopack(): The method that the autopacking thread calls to start an
autopack.  It takes a pack time with the same semantics as pack(), but
it also takes a flag specifying whether to do garbage collection of
unreachable objects or not.

_Autopack: A derived class of threading.Thread to handing the
background autopacking.

Here's how it works:

- On every store(), we write an entry to a objrev table containing the
  tuple of information (newserial, oid, oldserial).  We don't write
  this entry if the store is the first revision of an object on a new
  version.

  We do basically the same thing on restore() and transactionalUndo().

- On an abortVersion(), we write two entries to the objrev table, one
  that has (newserial, oid, oldserial) -- which points to the old
  serial in the version, and (newserial, oid, nvserial) -- which
  points to the non-version revision of the version revision.

- On commitVersion(), we do the same as abortVersion() except that we
  don't write the non-version data if we're committing to a different
  version.

- Now, when we pack, all we need to do is cruise from the beginning of
  the objrev table until we find an entry with a newserial > packtime.
  If the oldserial is ZERO, it's an object creation event which we
  don't need to worry about because there's no previous revision.  But
  otherwise, we can delete the oid+oldserial revision because we know
  it's not current.  We do this, updating pickle refcounts and then
  collecting any objects that are left unreferenced.

The cute thing is that autopacking will use the same algorithm.  The
main difference between autopack and classic pack, is that the latter
does a mark and sweep garbage collection phase after the normal objrev
collection phase.  Also, this algorithm means autopack needs only
three pieces of information:

- How often the thread should run (e.g. once per hour)

- How far in the past it should pack (e.g. pack to 4 hours ago).  We
  don't need a start time for the autopack window, because we'll
  always just start at the beginning of the objrev table.

- How often should autopack also do a classic pack (e.g. do a classic
  pack once per day).

Autopack isn't implemented in this checkin, but I believe it will be
nearly trivial to add.  That comes next.

checkTransactionalUndoAfterPackWithObjectUnlinkFromRoot()

for ZODB 3.2.

(__init__.py still makes format_exception available; is this needed?)

must've used this to disable the exception handling temporarily to
debug the exception formatting machinery, and forgotten to re-enable
it before checking in.

Unfortunately, this made it into the ZODB 3.1 release, making this a
3.1.1 backport candidate.

Fixes #651