• unknown's avatar
    MDEV-5262, MDEV-5914, MDEV-5941, MDEV-6020: Deadlocks during parallel · bd4153a8
    unknown authored
    replication causing replication to fail.
    
    Remove the temporary fix for MDEV-5914, which used READ COMMITTED for parallel
    replication worker threads. Replace it with a better, more selective solution.
    
    The issue is with certain edge cases of InnoDB gap locks, for example between
    INSERT and ranged DELETE. It is possible for the gap lock set by the DELETE to
    block the INSERT, if the DELETE runs first, while the record lock set by
    INSERT does not block the DELETE, if the INSERT runs first. This can cause a
    conflict between the two in parallel replication on the slave even though they
    ran without conflicts on the master.
    
    With this patch, InnoDB will ask the server layer about the two involved
    transactions before blocking on a gap lock. If the server layer tells InnoDB
    that the transactions are already fixed wrt. commit order, as they are in
    parallel replication, InnoDB will ignore the gap lock and allow the two
    transactions to proceed in parallel, avoiding the conflict.
    
    Improve the fix for MDEV-6020. When InnoDB itself detects a deadlock, it now
    asks the server layer for any preferences about which transaction to roll
    back. In case of parallel replication with two transactions T1 and T2 fixed to
    commit T1 before T2, the server layer will ask InnoDB to roll back T2 as the
    deadlock victim, not T1. This helps in some cases to avoid excessive deadlock
    rollback, as T2 will in any case need to wait for T1 to complete before it can
    itself commit.
    
    Also some misc. fixes found during development and testing:
    
     - Remove thd_rpl_is_parallel(), it is not used or needed.
    
     - Use KILL_CONNECTION instead of KILL_QUERY when a parallel replication
       worker thread is killed to resolve a deadlock with fixed commit
       ordering. There are some cases, eg. in sql/sql_parse.cc, where a KILL_QUERY
       can be ignored if the query otherwise completed successfully, and this
       could cause the deadlock kill to be lost, so that the deadlock was not
       correctly resolved.
    
     - Fix random test failure due to missing wait_for_binlog_checkpoint.inc.
    
     - Make sure that deadlock or other temporary errors during parallel
       replication are not printed to the the error log; there were some places
       around the replication code with extra error logging. These conditions can
       occur occasionally and are handled automatically without breaking
       replication, so they should not pollute the error log.
    
     - Fix handling of rgi->gtid_sub_id. We need to be able to access this also at
       the end of a transaction, to be able to detect and resolve deadlocks due to
       commit ordering. But this value was also used as a flag to mark whether
       record_gtid() had been called, by being set to zero, losing the value. Now,
       introduce a separate flag rgi->gtid_pending, so rgi->gtid_sub_id remains
       valid for the entire duration of the transaction.
    
     - Fix one place where the code to handle ignored errors called reset_killed()
       unconditionally, even if no error was caught that should be ignored. This
       could cause loss of a deadlock kill signal, breaking deadlock detection and
       resolution.
    
     - Fix a couple of missing mysql_reset_thd_for_next_command(). This could
       cause a prior error condition to remain for the next event executed,
       causing assertions about errors already being set and possibly giving
       incorrect error handling for following event executions.
    
     - Fix code that cleared thd->rgi_slave in the parallel replication worker
       threads after each event execution; this caused the deadlock detection and
       handling code to not be able to correctly process the associated
       transactions as belonging to replication worker threads.
    
     - Remove useless error code in slave_background_kill_request().
    
     - Fix bug where wfc->wakeup_error was not cleared at
       wait_for_commit::unregister_wait_for_prior_commit(). This could cause the
       error condition to wrongly propagate to a later wait_for_prior_commit(),
       causing spurious ER_PRIOR_COMMIT_FAILED errors.
    
     - Do not put the binlog background thread into the processlist. It causes
       too many result differences in mtr, but also it probably is not useful
       for users to pollute the process list with a system thread that does not
       really perform any user-visible tasks...
    bd4153a8
log.cc 286 KB