Casey, you have the baton now.

merged toby-xmlrpclib099-branch. Changed ZPublisher xmlrpc support to allow published methods to return None nested inside a data structure

and code accordingly.

Also rewrote the code to never create a new IIBucket unless one is
really needed (for example, if the input list has only one (bucket, weight)
pair, and the weight is 1, the input bucket can be returned as-is; and
this case will be common with score maps produced by an Okapi index).

    so the import machinery is not invoked on each call.

profile with a warm database cache instead of the cold cache.

    right.  Added a comment explaining what it represents in the
    function.

CosineIndex.query_weight():  rewrote to squash code duplication.  No
change in what it returns (it's always returned an upper bound on
possible doc scores, although people probably haven't thought of it
that way before).

Elsewhere:  consequent changes.

Problems:

+ mhindex.py needs repair, but I can't run it.  Note that its current
  use of query_weight isn't legitimate (the usage doesn't conform to
  the IIndex interface -- passing a string is passing "a sequence",
  but not the intended sequence <wink>).

+ ZCTextIndex doesn't pass query_weight() on.

+ We've defined no methods to help clients compute what needs to be
  passed to query_weight (a sequence of only the positive terms).

  I changed mailtest.py to cheat, but it's doing a wrong thing for
  negative terms.

+ I expect it will be impossible to shake people from the belief that
  100.0 * score / query_weight is some kind of "relevance score".  It
  isn't.  So perhaps better not to expose this in ZCTextIndex.

len(IIBTree) in these guys, so don't.

expensive.

to use both after the test to ensure they were the same.

guidelines.  Added some conformance tests.

Fixed problem where external methods did not setup the func_defaults properly on first load. This resulted in ZPublisher being unable to properly arguments to Ex Methods on the first try.

calling _add_wordinfo in a loop.  This is a simple way to save oodles
of functions calls.  In a brief but non-trivial test, this boosted
overall indexing rate by 12% (so huge bang for the buck).

Persistent class.

Add -w and -W option to dump the word list (by word and by wid,
respectively).

Except KeyboardInterrupt from unqualified except clauses.

- Use slightly more portable values for the Data.fs and
  Zope/lib/python.

- Add -t NNN option to specify how often to commit a transaction;
  default 20,000.

- Change -p into -p NNN to specify how often (counted in commits) to
  pack (default 0 -- never pack).

- Reworked the commit and pack logic to maintain the various counters
  across folders.

- Store relative paths (e.g. "inbox/1").

- Store the mtime of indexed messages in doctimes[docid].

- Store the mtime of indexed folders in watchfolders[folder] (unused).

- Refactor updatefolder() to:

  (a) Avoid indexing messages it's already indexed and whose mtime
      hasn't changed.  (This probably needs an override just in case.)

  (b) Unindex messages that no longer exist in the folder.

- Include the folder name and the message header fields from, to, cc,
  bcc, and subject in the text to be indexed.