goahead().  This rejects most invalid constructs: a lone & or <
without proper syntax following it is now an error.  Even at EOF.

Added logic to use the value of 'textindex_operator' as passed in the request to determine which query operator to use (and, near, andnot, or).  Valid values to pass in to textindex_operator are 'and', 'or', 'near', and 'andnot' (capitalization is ignored).  This is a near-term workaround for the inability to specify a default text index query operator on a per-index basis.  It provides the ability to override the currently module-defined default 'Or' operator for textindexes on a per-search basis.

An example of the utility of textindex_operator used with a ZCatalog instance:

zcatalog.searchResults(PrincipiaSearchSource='foo', textindex_operator='and')

these are really complaints about the intgrity of our own code.
HTMLParseError should be raised only for invalid input.  (Should we
turn more unrecognized constructs into exceptions, e.g. '&' not
followed by an entity or character reference?)

Also added an 'r' prefix to a regex string containing a backslash.

- NestingError derives from HTMLParseError, and is hence simplified.

- Thread the input position through all the code generation routines;
  all compile-time exceptions now possess lineno and offset.

- Restructured the code that inserts implied end-tags, and made it
  generate output that is the same as the input more often.  This was
  the hardest to get right, and I expect to be working more on it.

<foo></foo>.  (This fails until the next checkins.)

Add test for finish_startendtag().

Style nits:

Add blank lines between class header and first method definition.

Move _parse_error() to earlier in the class.

as that of TALError.

Add finish_startendtag(), which works like finish_starttag() but is
called instead of that for tags of the form <foo.../>.  The default
implementation simply calls finish_starttag(), so the interface is
still compatible.

Changed comments indicating that finish_*tag() are Overridable
methods.

Got rid of the documented and implemented return value of
finish_starttag() -- it wasn't used.

2.2. Unfortunately, there are old products (Confera) that depend on
it. One brokenness is that this module depends on 'query' from
Lexicon that isn't there anymore.  Worse, the new 'query' in
UnTextIndex is not compatible. I've copied the old 'query' from an old
version of Lexicon here to try to make things better.

It should really be possible to pass in special word characters.

depends on a pre-catalog architecture and the post-catalog
architecture isn't fully cooked. To get positions, we need to either
store the positions during indexing, which is way too expensive, or we
need to be able to get to the original text when searching, which we
can't do.

For now we'll return something bogus, which will make 'near' searches
behave effectively as 'and' searches.

STX improvements to the DTML Reference. Also slightly improved the documentation of dtml-var entity syntax (again).

Simplify the path math used to make sure we can pick up unittest.

The module from Python was sgmllib, not nsgmllib.

(Should we still include the standard Zope license?)