The mystery of the Win98 hangs in the checkReconnectSwitch() test

until I added an is_connected() test to testConnection() is solved.

After the ConnectThread has switched the client to the new, read-write
connection, it closes the read-only connection(s) that it was saving
up in case there was no read-write connection.  But closing a
ManagedConnection calls notify_closed() on the manager, which
disconnected the manager and the client from its brand new read-write
connection.  The mistake here is that this should only be done when
closing the manager's current connection!

The fix was to add an argument to notify_closed() that passes the
connection object being closed; notify_closed() returns without doing
a thing when that is not the current connection.

I presume this didn't happen on Linux because there the sockets
happened to connect in a different order, and there was no read-only
connection to close yet (just a socket trying to connect).

I'm taking out the previous "fix" to ClientStorage, because that only
masked the problem in this relatively simple test case.  The problem
could still occur when both a read-only and a read-write server are up
initially, and the read-only server connects first; once the
read-write server connects, the read-write connection is installed,
and then the saved read-only connection is closed which would again
mistakenly disconnect the read-write connection.

Another (related) fix is not to call self.mgr.notify_closed() but to
call self.mgr.connection.close() when reconnecting.  (Hmm, I wonder if
it would make more sense to have an explicit reconnect callback to the
manager and the client?  Later.)

src/ZEO/ClientStorage.py

@@ -13,7 +13,7 @@
 ##############################################################################
 """Network ZODB storage client
 
-$Id: ClientStorage.py,v 1.61 2002/09/18 21:17:48 gvanrossum Exp $
+$Id: ClientStorage.py,v 1.62 2002/09/19 03:51:22 gvanrossum Exp $
 """
 
 # XXX TO DO
@@ -199,7 +199,7 @@ class ClientStorage:
             stub.register(str(self._storage), self._is_read_only)
             return (stub, 1)
         except POSException.ReadOnlyError:
-            if not self._read_only_fallback or self.is_connected():
+            if not self._read_only_fallback:
                 raise
             log2(INFO, "Got ReadOnlyError; trying again with read_only=1")
             stub.register(str(self._storage), read_only=1)

src/ZEO/zrpc/client.py

@@ -174,7 +174,11 @@ class ConnectionManager:
             finally:
                 self.thread_lock.release()
 
-    def notify_closed(self):
+    def notify_closed(self, conn):
+        if conn is not self.connection:
+            # Closing a non-current connection
+            log("CM.notify_closed() non-current", level=zLOG.BLATHER)
+            return
         log("CM.notify_closed()")
         self.connected = 0
         self.connection = None
@@ -425,7 +429,7 @@ class ConnectWrapper:
         if self.mgr.connected:
             assert self.preferred
             log("CW: reconnecting client to preferred stub")
-            self.mgr.notify_closed()
+            self.mgr.connection.close()
         try:
             self.client.notifyConnected(self.stub)
         except:

src/ZEO/zrpc/connection.py

@@ -470,4 +470,4 @@ class ManagedConnection(Connection):
 
     def close(self):
         self.__super_close()
-        self.__mgr.notify_closed()
+        self.__mgr.notify_closed(self)