Collector #1517: History tab for ZPT does not work

FileStorage.history() was reading the user, description,
and extension fields out of the object pickle, due to
starting the read at a wrong location.

Restore the Connection._opened attribute, actually used by
DB.connectionDebugInfo().

Typo repair.

os.listdir() now prints errors on stdout

When raising DeprecationWarning, point the warning machinery at
the caller, not at Transaction.begin() itself.  As is (voice
of experience <wink>), these warnings are darned near useless to
track down code that needs to be changed on Zope trunk.  With the
change, the warnings point directly at the deprecated uses.

Rev 26842 changed cache-size parameters to type byte-size.
That's a good change, but it was checked in from a wrong place
(from the Zope trunk's copy of ZODB).  Since we want Zope 2.8 to
be usable with ZODB 3.3 too, I need to slam this change into
3.3 now.

Alas, I don't understand how ZODB+ZEO got stitched into the
Zope trunk (I do understand how it got stitched into the
Zope3 trunk, and into the X3 branch), and it looks like that
was left in a poor state.

So that if Zope X3-3.0 final is released while I'm on vacation,
it will have the right ZODB stuff in it (although I still need
to stitch this in to the Zope 3 and X3 trees).

This confirms that 3.3 didn't have the 3.2.4c1 bug wherein
an explict transaction abort() failed to clear Connections'
memory of ReadConflictErrors.

be.  Besides "the usual" release fiddling, repaired many
out-of-date pieces of the ZODB programming guide; I'm sure
many remain, though.

transaction when a commit() fails.  One test here provoked a
failing commit(), which caused a later test to fail.  Repaired
by simply abort()'ing the failed transaction.

After a commit fails (raises an exception), all subsequent attempts
to commit, join, or register with the transaction now raise the new
TransactionFailedError.  The failed transaction must be explicitly
discarded now, via abort() on the transaction or begin() on its
transaction manager.

more was incomprehensible.  Tried to repair that.  Plus
random improvements to ReST markup.

The ConflictError.get_{old,new}_serial() methods each did what the other
was supposed to do.

Collector #1488 (TemporaryStorage -- going backward in time).

This confusion was really due to that the detail on a ConflictError
exception didn't make sense.

Forward port from Zope 2.7 branch.

_handle_independent():  Failed to record that a ReadConflictError
was raised for an object with a _p_independent() method that
returned false.

is made while the connection has a pending subtransaction.

begin() is supposed to abort the current transaction, but
Transaction.begin() did not.  Calling begin() on a transaction
*manager* worked fine, and is the intended way to do a begin()
in 3.3.  But calling begin() on a Transaction object is still
very easy to do (e.g., the older get_transaction().begin()
spelling still works), and shouldn't be a subtle disaster.

creating problems.

started with a pickle GLOBAL opcode.  Curiously, it handled the
case of starting with a MARK GLOBAL opcode *sequence* fine.

FileIterator.next():  the code for reading the user, description,
and extension fields from a transaction was fatally confused,
usually reading them out of the object pickle by mistake.  This
caused several tools to display binary gibberish.

analyze_rec():  This produced spurious "len of unsized object" messages
when a data record had a backpointer instead of a pickle.  Repaired.

analyze(), analyze_trans():  Simplified overly elaborate iteration code.

Many changes to fsrefs.py, based on problems I hit using it in real life.

oid_repr():  Make it obvious which base is intended.  The output was
3-way ambiguous.

This includes:

r26945 | jim | 2004-08-06 18:30:44 -0400 (Fri, 06 Aug 2004) | 6 lines
   M /Zope3/trunk/src/persistent/tests/test_persistent.py
   M /Zope3/trunk/src/zope/event/tests.py

Updated to work with the versions of doctest from Python with versions
  greater than or equal to 2.3.0 and less than 2.4.0.a2 and with
  versions greater than 2.4.0a2.

r26482 | srichter | 2004-07-13 13:07:03 -0400 (Tue, 13 Jul 2004) | 2 lines
   M /Zope3/trunk/src/transaction/__init__.py
   M /Zope3/trunk/src/transaction/_transaction.py
   M /Zope3/trunk/src/transaction/tests/test_transaction.py

Converted XXX to TODO.

r25953 | sidnei | 2004-06-23 13:14:20 -0400 (Wed, 23 Jun 2004) | 1 line
   M /Zope3/trunk/src/ZODB/interfaces.py
   M /Zope3/trunk/src/transaction/_transaction.py
   M /Zope3/trunk/src/zope/app/rdb/__init__.py
   M /Zope3/trunk/src/zope/app/rdb/tests/test_zopedbtransactionmanager.py

Add missing sortKey method to ZopeDBTransactionManager and DataManagerAdapter, and also add it to the interface.

r25330 | alga | 2004-06-10 10:07:45 -0400 (Thu, 10 Jun 2004) | 4 lines
   M /Zope3/trunk/src/ZODB/Connection.py
   M /Zope3/trunk/src/ZODB/interfaces.py
   M /Zope3/trunk/src/ZODB/tests/testConnection.py

Added an IConnection interface declaration on ZODB Connection.

Jim says Tim Peters will be mad at him, but that's the right way to go.

r25273 | philikon | 2004-06-06 04:43:57 -0400 (Sun, 06 Jun 2004) | 4 lines
Changed paths:
   M /Zope3/trunk/src/BTrees
   M /Zope3/trunk/src/ZODB
   M /Zope3/trunk/src/persistent

Ignore .so files.

This fix should probably be merged to the original ZODB tree.

Improve thread-death error reporting in tests.

Reworked the way some of the ZEO tests use threads, so that unittest is
more likely to notice the real cause of a failure (which usually occurs in
a thread), and less likely to latch on to spurious problems resulting from
the real failure.

Detail:  The TestThread class got the (unittest) test case as an argument,
and remembered it.  If the run() method saw the thread die with an
exception, it called testcase.fail() with the formatted exception.  But
this can't work as apparently intended:  unittest's fail() raises an
AssertionError then, and the piece of unittest running the test expects
to catch that exception and append the message to the result object's
list of failures.  But the piece of unittest running the test isn't on
*this* thread's call stack!  It's on the main thread's call stack.
So unittest simply raised an exception that wasn't noticed by anything,
and the thread died then, leaving a traceback somewhere in the middle
of the output.  unittest didn't know anything about that, and sometimes
thought the test passed despite the thread failure.

Alas, when a thread dies like this it's also got a decent chance of
leaving things in a state where the test's final checking code can't
succeed, but the checking code doesn't know the thread died either,
and spurious unittest failures and errors could get reported then.

Reworked things so that only the main thread ever tries to tell
unittest that a test failed.  A TestThread remembers an exception-
death now, but doesn't report it until the main thread tries to join
it.  This way unittest knows the test failed; records and reports
the true cause of failure; and, because tests generally join all their
threads before doing their final checking, unittest stops the test
before final checking if a thread death occurred, and so there's
less chance of seeing reports of spurious errors and failures.

Write backups to a temp file first; flush and fsync when done;
only then rename; in case of a crash, no relevant damaged file
will be left behind then.  And on Windows, Python's os.fsync()
is necessary to convince the OS to write anything to disk.

Repaired massive lies I introduced about wether fsrefs requires
application code to be available.  Noted that fsrefs also partly
verifies the sanity of the .fs.index file.

Repaired a bug wherein spurious error msgs could be produced after
reporting a problem with an unloadable object (discovered by eyeball,
while staring at the code to figure out what it actually does).

Vastly expanded the module docstring, with a slimmed-down version of
the new fsrefs docs on the ZODB Wiki.

Connection objects were typically immortal because the threaded
transaction manager kept them in an ever-growing lists.  Reworked the
transaction manager internals to use a simple implementation of weak sets
instead.  This plugs all leaks in the test program attached to the
collector report (which was leaking about 100KB/sec on my box).