1. 17 Nov, 2014 2 commits
    • Kristian Nielsen's avatar
      MDEV-7079: rpl.rpl_parallel_temptable fails in valgrind builder · f9760507
      Kristian Nielsen authored
      The test case rpl.rpl_parallel_temptable deliberately crashes the master
      server as part of the testing. This makes it unsuitable for Valgrind
      testing. So make sure that it will be skipped when testing with Valgrind.
      f9760507
    • Kristian Nielsen's avatar
      MDEV-7080: rpl.rpl_gtid_crash fails sporadically in buildbot · 7671fd70
      Kristian Nielsen authored
      The real problem here was inconsistent handling of entry->commit_errno in
      MYSQL_BIN_LOG::write_transaction_or_stmt(). Some return paths were setting it
      to the value of errno, some where not. And the setting was redundant anyway,
      as it is set consistently by the caller.
      
      Fix by consistently setting it in the caller, and not in each return path in
      the function.
      
      The test failure happened because a DBUG_EXECUTE_IF() used in the test case
      set an entry->commit_errno that was immediately overwritten in the caller with
      whatever happened to be the value of errno. This could lead to different error
      message in the .result file.
      7671fd70
  2. 14 Nov, 2014 1 commit
  3. 13 Nov, 2014 13 commits
    • Kristian Nielsen's avatar
      MDEV-6917: Parallel replication: "Commit failed due to failure of an earlier... · 26b11130
      Kristian Nielsen authored
      MDEV-6917: Parallel replication: "Commit failed due to failure of an earlier commit on which this one depends", but no prior failure seen
      
      This bug was seen when parallel replication experienced a deadlock between
      transactions T1 and T2, where T2 has reached the commit phase and is waiting
      for T1 to commit first. In this case, the deadlock is broken by sending a kill
      to T2; that kill error is then later detected and converted to a deadlock
      error, which causes T2 to be rolled back and retried.
      
      The problem was that the kill caused ha_commit_trans() to errorneously call
      wakeup_subsequent_commits() on T3, signalling it to abort because T2 failed
      during commit. This is incorrect, because the error in T2 is only a temporary
      error, which will be resolved by normal transaction retry. We should not
      signal error to the next transaction until we have executed the code that
      handles such temporary errors.
      
      So this patch just removes the calls to wakeup_subsequent_commits() from
      ha_commit_trans(). They are incorrect in this case, and they are not needed in
      general, as wakeup_subsequent_commits() must in any case be called in
      finish_event_group() to wakeup any transactions that may have started to wait
      after ha_commit_trans(). And normally, wakeup will in fact have happened
      earlier, either from the binlog group commit code, or (in case of no
      binlogging) after the fast part of InnoDB/XtraDB group commit.
      
      The symptom of this bug was that replication would break on some transaction
      with "Commit failed due to failure of an earlier commit on which this one
      depends", but with no such failure of an earlier commit visible anywhere.
      
      26b11130
    • Kristian Nielsen's avatar
      MDEV-7065: Incorrect relay log position in parallel replication after retry of transaction · 3dcd01e5
      Kristian Nielsen authored
      The retry of an event group in parallel replication set the wrong value for
      the end log position of the event that was retried
      (qev->future_event_relay_log_pos). It was too large by the size of the event,
      so it pointed into the middle of the following event.
      
      If the retry happened in the very last event of the event group, _and_ the SQL
      thread was stopped just after successfully retrying that event, then the SQL
      threads's relay log position would be left incorrect. Restarting the SQL
      thread could then try to read events from a garbage offset in the relay log,
      usually leading to an error about not being able to read the event.
      3dcd01e5
    • Kristian Nielsen's avatar
      MDEV-6775: Wrong binlog order in parallel replication: Intermediate commit · d08b893b
      Kristian Nielsen authored
      The code in binlog group commit around wait_for_commit that controls commit
      order, did the wakeup of subsequent commits early, as soon as a following
      transaction is put into the group commit queue, but before any such commit has
      actually taken place. This causes problems with too early wakeup of
      transactions that need to wait for prior to commit, but do not take part in
      the binlog group commit for one reason or the other.
      
      This patch solves the problem, by moving the wakeup to happen only after the
      binlog group commit is completed.
      
      This requires a new solution to ensure that transactions that arrive later
      than the leader are still able to participate in group commit. This patch
      introduces a flag wait_for_commit::commit_started. When this is set, a waiter
      can queue up itself in the group commit queue.
      
      This way, effectively the wait_for_prior_commit() is skipped only for
      transactions that participate in group commit, so that skipping the wait is
      safe. Other transactions still wait as needed for correctness.
      d08b893b
    • Kristian Nielsen's avatar
      MDEV-6680: Performance of domain_parallel replication is disappointing · eec04fb4
      Kristian Nielsen authored
      The code that handles free lists of various objects passed to worker threads
      in parallel replication handles freeing in batches, to avoid taking and
      releasing LOCK_rpl_thread too often. However, it was possible for freeing to
      be delayed to the point where one thread could stall the SQL driver thread due
      to full queue, while other worker threads might be idle. This could
      significantly degrade possible parallelism and thus performance.
      
      Clean up the batch freeing code so that it is more robust and now able to
      regularly free batches of object, so that normally the queue will not run full
      unless the SQL driver thread is really far ahead of the worker threads.
      eec04fb4
    • Kristian Nielsen's avatar
      MDEV-6718: Server crashed in Gtid_log_event::Gtid_log_event with parallel replication · 8a3e2f29
      Kristian Nielsen authored
      The bug occured in parallel replication when re-trying transactions that
      failed due to deadlock. In this case, the relay log file is re-opened and the
      events are read out again. This reading requires a format description event of
      the appropriate version. But the code was using a description event stored in
      rli, which is not thread-safe. This could lead to various rare races if the
      format description event was replaced by the SQL driver thread at the exact
      moment where a worker thread was trying to use it.
      
      The fix is to instead make the retry code create and maintain its own format
      description event. When the relay log file is opened, we first read the format
      description event from the start of the file, before seeking to the current
      position. This now uses the same code as when the SQL driver threads starts
      from a given relay log position. This also makes sure that the correct format
      description event version will be used in cases where the version of the
      binlog could change during replication.
      8a3e2f29
    • Kristian Nielsen's avatar
      MDEV-7102: Incorrect PSI_stage_info message in SHOW PROCESSLIST during parallel replication · a98a034c
      Kristian Nielsen authored
      In parallel replication, threads can do two different waits for a prior
      transaction. One is for the prior transaction to start commit, the other is
      for it to complete commit.
      
      It turns out that the same PSI_stage_info message was errorneously used in
      both cases (probably a merge error), causing SHOW PROCESSLIST to be
      misleading.
      
      Fix by using correct, distinct message in each case.
      a98a034c
    • Kristian Nielsen's avatar
      ecc33da2
    • Kristian Nielsen's avatar
      MDEV-6775: Wrong binlog order in parallel replication · 684715a2
      Kristian Nielsen authored
      In parallel replication, the wait_for_commit facility is used to ensure that
      events are written into the binlog in the correct order. This is handled in an
      optimised way in the binlogging group commit code.
      
      However, some statements, for example GRANT, are written directly into the
      binlog, outside of the group commit code. There was a bug that this direct
      write does not correctly wait for the prior transactions to have been written
      first, which allows f.ex. GRANT to be written ahead of earlier transactions.
      
      This patch adds the missing wait_for_prior_commit() before writing directly to
      the binlog.
      
      However, the problem is still there, although the race is much less likely to
      occur now. The problem is that the optimised group commit code does wakeup of
      following transactions early, before the binlog write is actually done. A
      woken-up following transaction is then allowed to run ahead and queue up for
      the group commit, which will ensure that binlog write happens in correct order
      in the end. However, the code for directly written events currently bypass
      this mechanism, so they get woken up and written too early.
      
      This will be fixed properly in a later patch.
      684715a2
    • Kristian Nielsen's avatar
      Revert incorrect/redundant fix for old BUG#34656 · 55791c1a
      Kristian Nielsen authored
      The real bug was that open_tables() returned error in case of
      thd->killed() without properly calling thd->send_kill_message()
      to set the correct error. This was fixed some time ago.
      
      So remove the, now redundant, extra checks for thd->is_error(),
      possibly allowing to catch in debug builds more incorrect
      error handling cases.
      55791c1a
    • Kristian Nielsen's avatar
      MDEV-7101: SAFE_MUTEX lock order warning when reusing wait_for_commit mutex · fbc8768c
      Kristian Nielsen authored
      In SAFE_MUTEX builds, reset the wait_for_commit mutex (destroy and
      re-initialise), so that SAFE_MUTEX lock order check does not become
      confused when the mutex is re-used for a different purpose.
      fbc8768c
    • Jan Lindström's avatar
      MDEV-7035: Remove innodb_io_capacity setting depending on · 0f322994
      Jan Lindström authored
      setting of innodb_io_capacity_max
      
      (a) Changed the behaviour so that if you set innodb_io_capacity to a 
      value > innodb_io_capacity_max that the value is accepted AND 
      that innodb_io_capacity_max = innodb_io_capacity * 2.
      
      (b) If someone wants to reduce innodb_io_capacity_max and 
      reduce it below innodb_io_capacity then innodb_io_capacity 
      should be reduced to the same level as innodb_io_capacity_max.
      
      In both cases give a warning to user.
      0f322994
    • Jan Lindström's avatar
      MDEV-7100: InnoDB error monitor might unnecessary wait log_sys mutex · bff2d46b
      Jan Lindström authored
      Analysis: InnoDB error monitor is responsible to call every second
      sync_arr_wake_threads_if_sema_free() to wake up possible hanging
      threads if they are missed in mutex_signal_object. This is not
      possible if error monitor itself is on mutex/semaphore wait. We
      should avoid all unnecessary mutex/semaphore waits on error monitor.
      Currently error monitor calls function buf_flush_stat_update()
      that calls log_get_lsn() function and there we will try to get
      log_sys mutex. Better, solution for error monitor is that in
      buf_flush_stat_update() we will try to get lsn with
      mutex_enter_nowait() and if we did not get mutex do not update
      the stats.
      
      Fix: Use log_get_lsn_nowait() function on buf_flush_stat_update()
      function. If returned lsn is 0, we do not update flush stats.
      log_get_lsn_nowait() will use mutex_enter_nowait() and if
      we get mutex we return a correct lsn if not we return 0.
      bff2d46b
    • Jan Lindström's avatar
      MDEV-7083: sys_vars.innodb_sched_priority* tests fail in buildbot · 84f3f3fa
      Jan Lindström authored
      on work-amd64-valgrind.
      
      Fixed issue by finding out first the current used priority
      for both treads and using that seeing did we really change
      the priority or not.
      84f3f3fa
  4. 12 Nov, 2014 5 commits
  5. 11 Nov, 2014 1 commit
  6. 10 Nov, 2014 1 commit
  7. 03 Nov, 2014 2 commits
  8. 02 Nov, 2014 2 commits
  9. 01 Nov, 2014 1 commit
  10. 31 Oct, 2014 2 commits
    • unknown's avatar
      Cleanup. · ee309b10
      unknown authored
      ee309b10
    • Kristian Nielsen's avatar
      Fix sporadic test failure in main.processlist · bad5fdec
      Kristian Nielsen authored
      The test runs a query in one thread, then in another queries the processlist
      and expects to find the first thread in the COM_SLEEP state. The problem is
      that the thread signals completion to the client before changing to COM_SLEEP
      state, so there is a window where the other thread can see the wrong state.
      
      A previous attempt to fix this was ineffective. It set a DEBUG_SYNC to handle
      proper waiting, but unfortunately that DEBUG_SYNC point ended up triggering
      already at the end of SET DEBUG_SYNC=xxx, so the wait was ineffective.
      
      Fix it properly now (hopefully) by ensuring that we wait for the DEBUG_SYNC
      point to trigger at the end of the SELECT SLEEP(), not just at the end of
      SET DEBUG_SYNC=xxx.
      bad5fdec
  11. 30 Oct, 2014 2 commits
  12. 29 Oct, 2014 4 commits
  13. 28 Oct, 2014 4 commits