Improve thread-death error reporting in tests.

Reworked the way some of the ZEO tests use threads, so that unittest is
more likely to notice the real cause of a failure (which usually occurs in
a thread), and less likely to latch on to spurious problems resulting from
the real failure.

Detail:  The TestThread class got the (unittest) test case as an argument,
and remembered it.  If the run() method saw the thread die with an
exception, it called testcase.fail() with the formatted exception.  But
this can't work as apparently intended:  unittest's fail() raises an
AssertionError then, and the piece of unittest running the test expects
to catch that exception and append the message to the result object's
list of failures.  But the piece of unittest running the test isn't on
*this* thread's call stack!  It's on the main thread's call stack.
So unittest simply raised an exception that wasn't noticed by anything,
and the thread died then, leaving a traceback somewhere in the middle
of the output.  unittest didn't know anything about that, and sometimes
thought the test passed despite the thread failure.

Alas, when a thread dies like this it's also got a decent chance of
leaving things in a state where the test's final checking code can't
succeed, but the checking code doesn't know the thread died either,
and spurious unittest failures and errors could get reported then.

Reworked things so that only the main thread ever tries to tell
unittest that a test failed.  A TestThread remembers an exception-
death now, but doesn't report it until the main thread tries to join
it.  This way unittest knows the test failed; records and reports
the true cause of failure; and, because tests generally join all their
threads before doing their final checking, unittest stops the test
before final checking if a thread death occurred, and so there's
less chance of seeing reports of spurious errors and failures.

Reworked the way some of the ZEO tests use threads, so that unittest is
more likely to notice the real cause of a failure (which usually occurs in
a thread), and less likely to latch on to spurious problems resulting from
the real failure.

Detail:  The TestThread class got the (unittest) test case as an argument,
and remembered it.  If the run() method saw the thread die with an
exception, it called testcase.fail() with the formatted exception.  But
this can't work as apparently intended:  unittest's fail() raises an
AssertionError then, and the piece of unittest running the test expects
to catch that exception and append the message to the result object's
list of failures.  But the piece of unittest running the test isn't on
*this* thread's call stack!  It's on the main thread's call stack.
So unittest simply raised an exception that wasn't noticed by anything,
and the thread died then, leaving a traceback somewhere in the middle
of the output.  unittest didn't know anything about that, and sometimes
thought the test passed despite the thread failure.

Alas, when a thread dies like this it's also got a decent chance of
leaving things in a state where the test's final checking code can't
succeed, but the checking code doesn't know the thread died either,
and spurious unittest failures and errors could get reported then.

Reworked things so that only the main thread ever tries to tell
unittest that a test failed.  A TestThread remembers an exception-
death now, but doesn't report it until the main thread tries to join
it.  This way unittest knows the test failed; records and reports
the true cause of failure; and, because tests generally join all their
threads before doing their final checking, unittest stops the test
before final checking if a thread death occurred, and so there's
less chance of seeing reports of spurious errors and failures.

Repair type in pack() docstring.

repozo improvement suggested by Toby Dickenson.

Write backups to a temp file first; flush and fsync when done;
only then rename; in case of a crash, no relevant damaged file
will be left behind then.  And on Windows, Python's os.fsync()
is necessary to convince the OS to write anything to disk.

Write backups to a temp file first; flush and fsync when done;
only then rename; in case of a crash, no relevant damaged file
will be left behind then.  And on Windows, Python's os.fsync()
is necessary to convince the OS to write anything to disk.

Repaired massive lies I introduced about whether fsrefs requires
application code to be available.  Noted that fsrefs also partly
verifies the sanity of the .fs.index file.

Repaired massive lies I introduced about wether fsrefs requires
application code to be available.  Noted that fsrefs also partly
verifies the sanity of the .fs.index file.

Repaired a bug wherein spurious error msgs could be produced after
reporting a problem with an unloadable object (discovered by eyeball,
while staring at the code to figure out what it actually does).

Vastly expanded the module docstring, with a slimmed-down version of
the new fsrefs docs on the ZODB Wiki.

Repaired a bug wherein spurious error msgs could be produced after
reporting a problem with an unloadable object (discovered by eyeball,
while staring at the code to figure out what it actually does).

Vastly expanded the module docstring, with a slimmed-down version of
the new fsrefs docs on the ZODB Wiki.

Zope3-dev Collector #139: Memory leak involving buckets and connections

Connection objects were typically immortal because the threaded
transaction manager kept them in ever-growing lists.  Reworked the
transaction manager internals to use a simple implementation of weak sets
instead.  This plugs all leaks in the test program attached to the
collector report (which was leaking about 100KB/sec on my box).

Connection objects were typically immortal because the threaded
transaction manager kept them in an ever-growing lists.  Reworked the
transaction manager internals to use a simple implementation of weak sets
instead.  This plugs all leaks in the test program attached to the
collector report (which was leaking about 100KB/sec on my box).

Forward-porting from Zope 2.7.
Merged from 3.3 branch, revision 26085.

The checkFullTimeStamp() test was sensitive to unique mktime() behavior
on FreeBSD.  See:

http://lists.freebsd.org/pipermail/freebsd-standards/2003-November/000268.html 

The purpose of this test is to exercise ZODB's TimeStamp object, so got
rid of dependence on platform mktime() and time.timezone quirks --
TimeStamp works in GMT, so how mktime() treats tm_isdst should be
irrelevant in all TimeStamp tests.

Also added a comment about the highly non-obvious numeric characteristics
of TimeStamp's treatment of seconds (round-tripping is surprisingly
inaccurate, but for a real reason).

Forward-porting from Zope 2.7.

The checkFullTimeStamp() test was sensitive to unique mktime() behavior
on FreeBSD.  See:

http://lists.freebsd.org/pipermail/freebsd-standards/2003-November/000268.html 

The purpose of this test is to exercise ZODB's TimeStamp object, so got
rid of dependence on platform mktime() and time.timezone quirks --
TimeStamp works in GMT, so how mktime() treats tm_isdst should be
irrelevant in all TimeStamp tests.

Also added a comment about the highly non-obvious numeric characteristics
of TimeStamp's treatment of seconds (round-tripping is surprisingly
inaccurate, but for a real reason).

intend real implementations to work, and beefed up the tests
to match.  The collaboration also got a little simpler.

New interface IResourceManager intends to combine the
best aspects of IDataManager and IDataManagerOriginal,
and will become "the" interface.
New interface ISavePoint captures (along with models in
collaborations.txt) what should be a clean design for a
savepoint/rollback facility that actually works.

Updated an old collaboration to reflect the change.

are in now (partly ZODB 3, partly ZODB 4, partly transitional).
More is needed.
Added new collaborations.txt, which spells out some dynamics
of the system in a compact way.  This is in a format Jim used
for Zope 3, essentially a flat-text and less-formal representation
of UML sequence diagrams.

No commit message

and include that in the ZODB project.  While we're at it,
include zope.testing too, as zope.interface depends on that,
and we had copied zope.testing.loggingsupport.py into ZODB
earlier anyway (and delete that unique copy here now).

This also merges some changes made to the ZODB copy in
Zope3 back into ZODB.

Collector 1327:  FileStorage init confused by time travel

If the system clock moved back in time more than 30 days between
a FileStorage close and (re)open, new txn ids could be smaller
than txn ids already in the database.  Plugged that hole.

If the system clock moved back in time more than 30 days between
a FileStorage close and (re)open, new txn ids could be smaller
than txn ids already in the database.  Plugged that hole.

New test checkRestoreWithMultipleObjectsInUndoRedo from ZODB 3.2.2.
3.3 didn't have the bug, and this confirms it.

3.3 didn't have the bug, and this confirms it.

(identical to revision 25416 on the ZODB-3.3 branch, but svn would not merge
the change)