MDEV-3861: Assertion in TC_LOG_MMAP.

Root cause was that number of entries in commit checkpoint buffer
was bigger than total available entries in the mmap()'ed score
file. This causes TC_LOG_MMAP to run out of entries before even
the first checkpoint is started, which causes a hang.

Fixed by making sure we have fewer entries within one commit
checkpoint than total available scorefile entries.

Another part of this bug was discovery of severel unrelated bugs
in TC_LOG_MMAP dating back to 5.1. These were fixed in 5.1 and
will be merged up (the problem this patch fixes exists only in
10.0).

sql/log.h

@@ -118,7 +118,7 @@ class TC_LOG_MMAP: public TC_LOG
   struct pending_cookies {
     uint count;
     uint pending_count;
-    ulong cookies[TC_LOG_PAGE_SIZE];
+    ulong cookies[TC_LOG_PAGE_SIZE/sizeof(my_xid)];
   };
 
   private: