bogus config object.

sleep based.  We create two ends of a pipe in the parent thread, with
the child thread polling/reading one end and the parent writing to the
other.  The only thing written to this pipe is a "stop marker" -- just
a string that wakes the thread up immediately when we're closing the
storage.

The primary reason for this is to speed up shutdown so that we don't
have to wait for the next tick of the sleep counter before we'll
trigger the thread shutdown.  Makes shutting down Zope using this
storage much quicker.

Specific changes include:

BerkeleyBase.py

    SLEEP_TIME -> JOIN_TIME since it's now just the interval we'll
    wait for the thread.join to complete.

    __init__():	Create both the checkpointer thread and the autopacker
    thread, set up the pipes, and get them both rolling.  We refactor
    creation of the autopacker instance into a separate overrideable
    method since this is the one bit that's different between the two
    storages.

    _make_autopacker(): Intended to be overridden.

    close(), _doclose(): Move the thread shutdown code out of the
    lock, since we don't want to potentially deadlock the shutting
    down of the background thread.  This could happen if close() was
    entered when the autopacker thread was between lock acquisitions
    in _dopack().  Also, be sure to write to the pipe to wake the
    child threads up immediately.

    env_from_string(): Wrap the actual creation and opening of the
    environment in a try/except, so that if there's a failure, we can
    be sure to give up the file lock.

    _WorkThread class: Accept a poll object which wraps the read end
    of the pipe.  Rework run() to do the poll-with-timeout instead of
    sleep.  I don't think we strictly need to do the fd read given the
    simplicity (and mono-commandity) of the protocol, but it can't
    hurt.  The _dowork() method's signature no longer contains the
    `now' variable.

Full.py

    _make_autopacker(): Override base class to return storage specific
    _Autopack instance.

Minimal.py

    Same, but also include some code cleanup.


Also, get rid of some unnecessary imports.

Major refactor of the catalog search engine centered around optimizing sort by index operations. The resulting code greatly outperforms the previous version and uses less memory.

Exposed a new ZCatalog method "search" which has a better interface for programmatic searches. Updated documentation as well.

Implemented a sort limit option which allows you to inform the catalog that you are only interested in a certain number of results. In the common case this allows the ZCatalog machinery to use a different sorting algorithm (N-Best) which scales much better then a full sort.

Also more tightly integrated the merge option which allows you to tell the catalog that you would like raw and unsorted intermediate results returned rather than sorted and lazified results. This can be used to efficiently merge search results across multiple catalogs.

list that was being modified as we went).
[Backport from Zope3; I'm not actually sure if the list returned by
**Btree.keys() is lazy, but I assume it is.]

        # DB 4.1.24 requires that operations happening in a transaction must
        # be performed on a database that was opened in a transaction.  Since
        # we do the former, we must do the latter.  However, earlier DB
        # versions don't transactionally protect database open, so this is the
        # most portable way to write the code.

i.e. we use the DB_AUTO_COMMIT flag for the open, if it's defined.

right thing to do anyway, and should work just find for earlier
versions of the wrapper.  Specifically,

_dorecovery(), _collect_objs(), _mark(): transactionally protect
.truncate() and .consume()

_docommit(): Move the setting of the _pending flag to here from
_finish() so we can transactionally protect it more conveniently.
This does't change the semantics and the recovery code in _setupDBs()
doesn't care since the flag will already be set to COMMIT.

right thing to do anyway, and should work just find for earlier
versions of the wrapper.  Specifically,

_doabort(), _docommit(), _mark(), _sweep(), _collect_objs():
Transactionally protect .truncate() and .consume()

_docommit(): Move the setting of the _pending flag to here from
_finish() so we can transactionally protect it more conveniently.
This does't change the semantics and the recovery code in _setupDBs()
doesn't care since the flag will already be set to COMMIT.

_begin(): Transactionally protect the setting of the _pending flag to
ABORT.

I couldn't find any references to cPickle in any of the setup/build
tools, but I couldn't actually verify that the Windows build was
unaffected.

Added ZPL header to stopper test module

Python 2.2.X version

- logging an error message instead of raising an exception in case of
  unindex_object() failure

      - Collector #699: MailHosts created in 2.5 breaks in 2.6.

      - Collector #694: dtml-sendmail mailto specification replaces "To:" header.

Normalize whitespace (horizontal and vertical).

confusion caused by having the instance and the class have the same
name.

ChrisM's changes here) it is no longer necessary to call
zLOG.initialize() in order to see logging output.

least 2.2.2.)

Add -v1 to TESTOPTS, so running the entire test suite is a little less
verbose.

Stringify the object in the exception in case its a list or tuple (an inscrutable error occurs if this is not done).

_doclose() twice (runsvr.py had a bug that caused the storages to be
closed twice).

Allowing DA to have an option connection hook which allows you to change the database connection at run time for a sqlmethod.

close(): The base class can now do all the work of closing, including
stopping the background threads.

log(): Much more convenient.

do the build in *this* version of Zope.

  severity level.  This allows the common names to be used for
  severities, not just integers.  This can make configuration data
  less obscure.

- Refactor initialize():
  get_environment_info() pulls information from the environment
  variables, and this gets passed to initialize_log(), which does
  nothing to determine the source of the configuration data.

- New method initialize_with_config():  Load information not provided
  by the environment from a ZConfig section.  This allows the
  environment to override a config file, and avoids having the
  application deal with the specific information needed to configure
  logging.