1. 28 Apr, 2021 3 commits
    • Marko Mäkelä's avatar
      MDEV-25491: Race condition between DROP TABLE and purge of SYS_INDEXES record · f619d79b
      Marko Mäkelä authored
      btr_free_if_exists(): Always use the BUF_GET_POSSIBLY_FREED mode
      when accessing pages, because due to MDEV-24589 the function
      fil_space_t::set_stopping(true) can be called at any time during
      the execution of this function.
      
      mtr_t::m_freeing_tree: New data member for debugging purposes.
      
      buf_page_get_low(): Assert that the BUF_GET mode is not being used
      anywhere during the execution of btr_free_if_exists().
      
      In all code related to freeing or allocating pages, we will add some
      robustness, by making more use of BUF_GET_POSSIBLY_FREED and by
      reporting an error instead of crashing in some cases of corruption.
      f619d79b
    • Marko Mäkelä's avatar
      MDEV-25491 preparation: Clean up tablespace destruction · a81aec15
      Marko Mäkelä authored
      fil_check_pending_ops(), fil_check_pending_io(): Remove.
      These functions were actually duplicating each other ever since
      commit 118e258a (MDEV-23855).
      
      fil_space_t::check_pending_operations(): Replaces
      fil_check_pending_operations() and incorporates the logic of
      fil_check_pending_ops(). Avoid unnecessary lookups for the tablespace.
      Just wait for the reference count to drop to zero.
      
      fil_space_t::io(): Remove an unnecessary condition. We can (and
      probably better should) refuse asynchronous reads of undo tablespaces
      that are being truncated.
      
      fil_truncate_prepare(): Remove.
      
      trx_purge_truncate_history(): Implement the necessary steps that used
      to be in fil_truncate_prepare().
      a81aec15
    • Kentoku SHIBA's avatar
      MDEV-22265 Connect string character limit too small for full 64 character... · b3d963fe
      Kentoku SHIBA authored
      MDEV-22265 Connect string character limit too small for full 64 character InnoDB table-name limit when using ad-hoc Spider server definitions.
      
      Fix length for getting default table name.
      b3d963fe
  2. 26 Apr, 2021 2 commits
  3. 25 Apr, 2021 3 commits
    • Marko Mäkelä's avatar
      MDEV-25512 Deadlock between sux_lock::u_x_upgrade() and sux_lock::u_lock() · b8138288
      Marko Mäkelä authored
      In the SUX_LOCK_GENERIC implementation, we can remember at most
      one pending exclusive lock request. If multiple exclusive lock
      requests are pending, the WRITER_WAITING flag will be cleared when
      the first waiting writer acquires the exclusive lock.
      
      ssux_lock_low::update_lock(): If WRITER_WAITING is set, wake up
      the writer even if the UPDATER flag is set, because the waiting
      writer may be in the process of upgrading its U lock to X.
      
      rw_lock::read_unlock(): Also indicate that an X lock waiter must
      be woken up if an U lock exists.
      
      This fix may cause unnecessary wake-ups and system calls, but this
      is the best that we can do. Ideally we would use the MDEV-25404
      idea of a separate 'writer' mutex, but there is no portable way to
      request that a non-recursive mutex be created, and InnoDB requires
      the ability to transfer buf_block_t::lock ownership to an I/O thread.
      
      To allow problems like this to be caught more reliably in the future,
      we add a unit test for srw_mutex, srw_lock, ssux_lock, sux_lock.
      b8138288
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · 4725792b
      Marko Mäkelä authored
      4725792b
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · e4394cc5
      Marko Mäkelä authored
      e4394cc5
  4. 24 Apr, 2021 4 commits
    • Igor Babaev's avatar
      This commit adds the same call of st_select_lex::set_unique_exclude() that · 2c9bf0ae
      Igor Babaev authored
      complemented the fix for MDEV-24823 in 10.2. As it is the only call of
      this function in 10.3 the commit also has added the code of the function.
      2c9bf0ae
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · c425d93b
      Marko Mäkelä authored
      except commit 1288dfff
      c425d93b
    • Marko Mäkelä's avatar
      MDEV-23026/MDEV-25474 fixup: Assertion ib_table->stat_initialized · 14a18d7d
      Marko Mäkelä authored
      It is possible that an object that was originally created by
      open_purge_table() will remain cached and reused for SQL execution.
      Our previous fix wrongly assumed that ha_innobase::open() would
      always be called before SQL execution starts. Therefore, we must
      invoke dict_stats_init() in ha_innobase::info_low() instead of
      only doing it in ha_innobase::open().
      
      Note: Concurrent execution of dict_stats_init() on the same table
      is possible, but it also was possible between two calls to
      ha_innobase::open(), with no ill effects observed.
      
      This should fix the assertion failure on stat_initialized.
      A possibly easy way to reproduce it would have been
      to run the server with innodb_force_recovery=2 (disable the purge of
      history), update a table so that an indexed virtual column will be
      affected, and finally restart the server normally (purge enabled),
      to observe a crash when the table is accessed from SQL.
      
      The problem was first observed and this fix verified by
      Elena Stepanova. Also Thirunarayanan Balathandayuthapani
      repeated the problem.
      14a18d7d
    • Marko Mäkelä's avatar
      MDEV-25459 MVCC read from index on CHAR or VARCHAR wrongly omits rows · 25ed665a
      Marko Mäkelä authored
      row_sel_sec_rec_is_for_clust_rec(): If the field in the
      clustered index record stored off page, always fetch it,
      also when the secondary index field has been built on the
      entire column. This was broken ever since the InnoDB Plugin
      for MySQL Server 5.1 introduced ROW_FORMAT=DYNAMIC and
      ROW_FORMAT=COMPRESSED for InnoDB tables. That code was first
      introduced in this tree in
      commit 3945d5e5.
      
      For the original ROW_FORMAT=REDUNDANT and the MySQL 5.0.3
      ROW_FORMAT=COMPRESSED, there was no problem, because for
      those tables we always stored at least a 768-byte prefix of
      each column in the clustered index record.
      
      row_sel_sec_rec_is_for_blob(): Allow prefix_len==0 for matching
      the full column.
      25ed665a
  5. 23 Apr, 2021 7 commits
    • Igor Babaev's avatar
      1288dfff
    • Aleksey Midenkov's avatar
      MDEV-25091 CREATE TABLE: field references qualified by a wrong table name succeed · 42f8548f
      Aleksey Midenkov authored
      Before FRM is written walk vcol expressions through
      check_table_name_processor() and check if field items match (db,
      table_name) qualifier.
      
      We cannot do this in check_vcol_func_processor() as there is already
      no table name qualifiers in expressions of written and loaded FRM.
      42f8548f
    • Aleksey Midenkov's avatar
      MDEV-23455 Hangs + Sig11 in unknown location(s) due to single complex FK query · 4649ba74
      Aleksey Midenkov authored
      Buffer overflow in ib_push_warning() fixed by using vsnprintf().
      
      InnoDB parser was obsoleted by MDEV-16417.
      
      Thanks to Nikita Malyavin for review and suggestion.
      4649ba74
    • Sergei Golubchik's avatar
      remove EXCEPTIONS-CLIENT · 018d7440
      Sergei Golubchik authored
      It's Oracle libmysqlclient license exception, we no longer include,
      build or ship libmysqlclient
      018d7440
    • Marko Mäkelä's avatar
      MDEV-25487 Assertion failed in lock_rec_move · 1a647b70
      Marko Mäkelä authored
      row_ins_clust_index_entry_low(): Do not enable bulk insert if
      any record locks exist on the table. Bulk insert is assumed to
      be covered only by an exclusive table lock, with no row-level
      locking or undo logging.
      1a647b70
    • Sujatha's avatar
      MDEV-20220 post push fix · ee20e26d
      Sujatha authored
      Remove garbage files.
      ee20e26d
    • Igor Babaev's avatar
      MDEV-24823 Crash with invalid multi-table update of view in 2nd execution of SP · e3a25793
      Igor Babaev authored
      Before this patch mergeable derived tables / view used in a multi-table
      update / delete were merged before the preparation stage.
      When the merge of a derived table / view is performed the on expression
      attached to it is fixed and ANDed with the where condition of the select S
      containing this derived table / view. It happens after the specification of
      the derived table / view has been merged into S. If the ON expression refers
      to a non existing field an error is reported and some other mergeable derived
      tables / views remain unmerged. It's not a problem if the multi-table
      update / delete statement is standalone. Yet if it is used in a stored
      procedure the select with incompletely merged derived tables / views may
      cause a problem for the second call of the procedure. This does not happen
      for select queries using derived tables / views, because in this case their
      specifications are merged after the preparation stage at which all ON
      expressions are fixed.
      This patch makes sure that merging of the derived tables / views used in a
      multi-table update / delete statement is performed after the preparation
      stage.
      
      Approved by Oleksandr Byelkin <sanja@mariadb.com>
      e3a25793
  6. 22 Apr, 2021 18 commits
    • Igor Babaev's avatar
      MDEV-24823 Crash with invalid multi-table update of view in 2nd execution of SP · b3b5d57e
      Igor Babaev authored
      Before this patch mergeable derived tables / view used in a multi-table
      update / delete were merged before the preparation stage.
      When the merge of a derived table / view is performed the on expression
      attached to it is fixed and ANDed with the where condition of the select S
      containing this derived table / view. It happens after the specification of
      the derived table / view has been merged into S. If the ON expression refers
      to a non existing field an error is reported and some other mergeable derived
      tables / views remain unmerged. It's not a problem if the multi-table
      update / delete statement is standalone. Yet if it is used in a stored
      procedure the select with incompletely merged derived tables / views may
      cause a problem for the second call of the procedure. This does not happen
      for select queries using derived tables / views, because in this case their
      specifications are merged after the preparation stage at which all ON
      expressions are fixed.
      This patch makes sure that merging of the derived tables / views used in a
      multi-table update / delete statement is performed after the preparation
      stage.
      
      Approved by Oleksandr Byelkin <sanja@mariadb.com>
      b3b5d57e
    • Sergei Golubchik's avatar
    • Sergei Golubchik's avatar
      MYSQL_MAINTAINER_MODE fixes · a40f29ab
      Sergei Golubchik authored
      * MSVC check was lost in a bad merge
      * add a comment
      * move AIX check where it belongs and where it would still
        allow -Werror, if desired
      a40f29ab
    • Marko Mäkelä's avatar
      Null-merge the revert of MDEV-24589 · bfc0110b
      Marko Mäkelä authored
      While MDEV-24589 causes the regression
      MDEV-25491 Race condition between DROP TABLE and purge of
      SYS_INDEXES record
      we must not revert it from 10.6, because MDEV-24589 is a prerequisite for
      MDEV-25180 Atomic ALTER TABLE
      that is targeting 10.6. The regression will be dealt with later.
      bfc0110b
    • Marko Mäkelä's avatar
      Merge 10.5 into 10.6 · 18f35a86
      Marko Mäkelä authored
      18f35a86
    • Vladislav Vaintroub's avatar
      5c5d24c7
    • Vladislav Vaintroub's avatar
      MDEV-25456 MariaBackup logs "[ERROR]" on Invalid log block checksum · 78bb9533
      Vladislav Vaintroub authored
      Fix is to changed message to be [WARNING] for backup
      78bb9533
    • Vladislav Vaintroub's avatar
      Update timezone data on Windows · f6542a7a
      Vladislav Vaintroub authored
      There is new Yukon Standard time Windows timezone.
      
      Also fix the powershell script that generates the Windows locale mapping,
      tell powershell to use TLSv1.2 to access the github (on some reason it is
      TLS1.1 that powershell is using by default, and it does no work)
      f6542a7a
    • Marko Mäkelä's avatar
      MDEV-25404 fixup: Fix ssux_lock_low::u_wr_upgrade() · 8121d03f
      Marko Mäkelä authored
      The U-to-X upgrade turned out to be incorrect. A debug assertion
      failed in wr_wait(), called from mtr_defer_drop_ahi() in a stress
      test with innodb_adaptive_hash_index=ON.
      
      A correct upgrade procedure ought to be readers.fetch_add(WRITER-1)
      to register ourselves as a WRITER (or waiting writer) and to release
      the reference that was being held for the U lock.
      
      Thanks to Matthias Leich for catching the problem.
      8121d03f
    • Marko Mäkelä's avatar
      Revert "MDEV-24589 DROP TABLE is not crash-safe" · 1636db54
      Marko Mäkelä authored
      This reverts commit e731a283.
      
      A crash occurred during the test stress.ddl_innodb when
      fil_delete_tablespace() for DROP TABLE was waiting in
      fil_check_pending_operations() and a purge thread for handling
      an earlier DROP INDEX was attempting to load the index root page
      in btr_free_if_exists() and btr_free_root_check(). The function
      buf_page_get_gen() would write out several times
      "trying to read...being-dropped tablespace"
      before giving up and committing suicide.
      
      It turns out that during any page access in btr_free_if_exists(),
      fil_space_t::set_stopping() could have been invoked by
      fil_check_pending_operations(), as part of dropping the tablespace.
      Preventing this race condition would require extensive changes
      to the allocation code or some locking mechanism that would ensure
      that we only set the flag if btr_free_if_exists() is not in progress.
      
      Either way, that could be a too risky change in a GA release.
      Because MDEV-24589 is not strictly necessary in the 10.5 release
      series and it only is a requirement for MDEV-25180 in a later
      major release, we will revert the change from 10.5.
      1636db54
    • sachin's avatar
      MDEV-22953 main.flush_read_lock failed in buildbot with XAER_NOTA: Unknown XID · 21973d0d
      sachin authored
      Issue:- Since there is no waiting for the actual disconnection of the con_tmp
      (which does XA prepare of test1), We can have a issue when test1 is not
      prepared and we are calling rollback on test1 , giving XAER_NOTA: Unknown XID
      error
      
      Solution:- Wait for the complete disconnection of con_tmp
      21973d0d
    • Marko Mäkelä's avatar
      Merge 10.5 into 10.6 · 54c460ac
      Marko Mäkelä authored
      54c460ac
    • Marko Mäkelä's avatar
      fixup dd6ad380: remove dead code · b728c3db
      Marko Mäkelä authored
      b728c3db
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · df33b719
      Marko Mäkelä authored
      df33b719
    • Marko Mäkelä's avatar
      MDEV-25483: Shutdown crash during innodb.innodb_buffer_pool_resize_temporary · cbbca7ed
      Marko Mäkelä authored
      buf_resize_callback(): Correct an invalid assertion, and enable
      the assertion in debug builds only.
      
      Between buf_resize_start() and buf_resize_shutdown(),
      srv_shutdown_state must be less than SRV_SHUTDOWN_CLEANUP.
      
      The incorrect assertion had been introduced in
      commit 5e62b6a5 (MDEV-16264).
      As a result, the server could crash if shutdown was initiated
      concurrently with initiating a change of innodb_buffer_pool_size.
      cbbca7ed
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · ee455e6f
      Marko Mäkelä authored
      ee455e6f
    • Marko Mäkelä's avatar
      0d267f7c
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 6f271302
      Marko Mäkelä authored
      6f271302
  7. 21 Apr, 2021 3 commits
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-25474 Background thread returns uninitialized statistics · fb96ac0a
      Thirunarayanan Balathandayuthapani authored
      		to mysql interpreter
      
      InnoDB returns uninitialized statistics to mysql interpreter
      when background thread is opening the table. So it leads to
      assertion failure. In that case, InnoDB avoid sending
      innodb statistics information to mysql interpreter.
      fb96ac0a
    • Eugene Kosov's avatar
      MDEV-25457 Server crashes in row_undo_mod_clust_low upon rollback of read-only transaction · 64eeb250
      Eugene Kosov authored
      node->index was NULL.
      But it's possible to get dict_table_t* from another source.
      64eeb250
    • Andrei Elkin's avatar
      MDEV-24526 binlog rotate via FLUSH LOGS may obsolate binlog file for recovery too eary · 2a7dd644
      Andrei Elkin authored
      There was race between a committing transaction and the following in binlog
      order FLUSH LOGS that could create a 2nd Binlog checkpoint (BCP) event
      in the new file *before* the first logged-in-old-binlog transaction gets committed in
      Innodb. That would cause the transaction loss at recovery, should
      the server stop right after the BCP.
      
      The race is tackled by enforcing the necessary set of mutexes to be acquired
      by FLUSH-LOGS handler in the correct order (of the group commit leader
      pattern).
      
      Note, there remain two cases where a similar race is still possible:
        - the above race as it is when the server is run with ("unlikely")
          non-default `--binlog-optimize-thread-scheduling=0` (MDEV-24530), and
        - at unlikely event of bin-logging of Incident event (MDEV-24531) that
          also triggers binlog rotation,
          in both cases though with lesser chances after the current fixes.
      2a7dd644