1. 28 Sep, 2021 3 commits
    • Marko Mäkelä's avatar
      MDEV-26467: Actually use spinloop on block_lock · e79fa9f5
      Marko Mäkelä authored
      In commit 277ba134
      we accidentally omitted this.
      e79fa9f5
    • Marko Mäkelä's avatar
      MDEV-26467: Universally implement spin loop · d0d4ade9
      Marko Mäkelä authored
      Previously, neither our wrapper of Microsoft Windows SRWLOCK
      nor the futex-less implementation SUX_LOCK_GENERIC supported spin loops.
      
      This was suggested by Vladislav Vaintroub.
      d0d4ade9
    • Marko Mäkelä's avatar
      MDEV-26467: More cache friendliness · 35f59bc4
      Marko Mäkelä authored
      srw_mutex_impl<bool>::wait_and_lock(): In
      commit a73eedbf we introduced
      an std::atomic::fetch_or() in a loop. Alas, on the IA-32 and AMD64,
      that was being translated into a loop around LOCK CMPXCHG.
      To avoid a nested loop, it is better to explicitly invoke
      std::atomic::compare_exchange_weak() in the loop, but only if
      the attempt has a chance to succeed (the HOLDER flag is not set).
      
      It is even more efficient to use LOCK BTS, but contemporary compilers
      fail to translate std::atomic::fetch_or(x) & x into that when x is
      a single-bit constant. On GCC-compatible compilers, we will use
      inline assembler to achieve that.
      
      On other ISA than IA-32 and AMD64, we will continue to use
      std::atomic::fetch_or().
      
      ssux_lock_impl<spinloop>::rd_wait(): Use rd_lock_try().
      A loop around std::atomic::compare_exchange_weak() should be
      cheaper than fetch_add(), fetch_sub() and a wakeup system call.
      
      These deficiencies were pointed out and the use of LOCK BTS was
      suggested by Thiago Macieira.
      35f59bc4
  2. 27 Sep, 2021 1 commit
  3. 24 Sep, 2021 11 commits
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-26631 InnoDB fails to fetch page from doublewrite buffer · 76972163
      Thirunarayanan Balathandayuthapani authored
      Problem:
      ========
      InnoDB fails to fetch the page0 from dblwr if page0 is
      corrupted.In that case, InnoDB defers the tablespace
      and doesn't find the INIT_PAGE redo log record for page0
      and it leads to failure.
      
      Solution:
      =========
       InnoDB should recover page0 from dblwr if space_id can
      be found for deferred tablespace.
      76972163
    • Marko Mäkelä's avatar
      Merge 10.5 into 10.6 · d9536110
      Marko Mäkelä authored
      d9536110
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · 88f38661
      Marko Mäkelä authored
      88f38661
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · d5bd704f
      Marko Mäkelä authored
      d5bd704f
    • Marko Mäkelä's avatar
      MDEV-26672 innodb_undo_log_truncate may reset transaction ID sequence · 4bfdba2e
      Marko Mäkelä authored
      trx_rseg_header_create(): Add a parameter for the value that is
      to be written to TRX_RSEG_MAX_TRX_ID. If we omit this write, then
      the updated test innodb.undo_truncate will fail for the 4k, 8k, 16k
      page sizes. This was broken ever since
      commit 947efe17 (MDEV-15158)
      removed the writes of transaction identifiers to the TRX_SYS page.
      
      srv_do_purge(): Truncate undo tablespaces also during slow shutdown
      (innodb_fast_shutdown=0).
      
      Thanks to Krunal Bauskar for noticing this problem.
      4bfdba2e
    • Marko Mäkelä's avatar
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · 7e2b4232
      Marko Mäkelä authored
      7e2b4232
    • Marko Mäkelä's avatar
      MDEV-26445 innodb_undo_log_truncate is unnecessarily slow · f5794e1d
      Marko Mäkelä authored
      trx_purge_truncate_history(): Do not force a write of the undo tablespace
      that is being truncated. Instead, prevent page writes by acquiring
      an exclusive latch on all dirty pages of the tablespace.
      
      fseg_create(): Relax an assertion that could fail if a dirty undo page
      is being initialized during undo tablespace truncation (and
      trx_purge_truncate_history() already acquired an exclusive latch on it).
      
      fsp_page_create(): If we are truncating a tablespace, try to reuse
      a page that we may have already latched exclusively (because it was
      in buf_pool.flush_list). To some extent, this helps the test
      innodb.undo_truncate,16k to avoid running out of buffer pool.
      
      mtr_t::commit_shrink(): Mark as clean all pages that are outside the
      new bounds of the tablespace, and only add the newly reinitialized pages
      to the buf_pool.flush_list.
      
      buf_page_create(): Do not unnecessarily invoke change buffer merge on
      undo tablespaces.
      
      buf_page_t::clear_oldest_modification(bool temporary): Move some
      assertions to the caller buf_page_write_complete().
      
      innodb.undo_truncate: Use a bigger innodb_buffer_pool_size=24M.
      On my system, it would otherwise hang 1 out of 1547 attempts
      (on the 40th repeat of innodb.undo_truncate,16k).
      Other page sizes were not affected.
      f5794e1d
    • Marko Mäkelä's avatar
      MDEV-26450: Corruption due to innodb_undo_log_truncate · f5fddae3
      Marko Mäkelä authored
      At least since commit 055a3334
      (MDEV-13564) the undo log truncation in InnoDB did not work correctly.
      
      The main issue is that during the execution of
      trx_purge_truncate_history() some pages of the newly truncated
      undo tablespace could be discarded.
      
      This is improved from commit 1cb218c3
      which was applied to earlier-version branches.
      
      fsp_try_extend_data_file(): Apply the peculiar rounding of
      fil_space_t::size_in_header only to the system tablespace,
      whose size can be expressed in megabytes in a configuration parameter.
      Other files may freely grow by a number of pages.
      
      fseg_alloc_free_page_low(): Do allow the extension of undo tablespaces,
      and mention the file name in the error message.
      
      mtr_t::commit_shrink(): Implement crash-safe shrinking of a tablespace:
      (1) durably write the log
      (2) release the page latches of the rebuilt tablespace
      (3) release the mutexes
      (4) truncate the file
      (5) release the tablespace latch
      This is refactored from trx_purge_truncate_history().
      
      log_write_and_flush_prepare(), log_write_and_flush(): New functions
      to durably write log during mtr_t::commit_shrink().
      f5fddae3
    • Marko Mäkelä's avatar
      MDEV-26626 fixup: Do not advance checkpoint during startup · 15efb7ed
      Marko Mäkelä authored
      While the redo log is being resized in srv_start(),
      we must not write checkpoint information to the old log.
      
      Thanks to Matthias Leich for noticing this.
      15efb7ed
    • Jan Lindström's avatar
      Update wsrep-lib submodule · 9c2d9236
      Jan Lindström authored
      9c2d9236
  4. 23 Sep, 2021 2 commits
  5. 22 Sep, 2021 6 commits
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · 9024498e
      Marko Mäkelä authored
      9024498e
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · b46cf33a
      Marko Mäkelä authored
      b46cf33a
    • Marko Mäkelä's avatar
      MDEV-26450: Corruption due to innodb_undo_log_truncate · 1cb218c3
      Marko Mäkelä authored
      At least since commit 055a3334
      (MDEV-13564) the undo log truncation in InnoDB did not work correctly.
      
      The main issue is that during the execution of
      trx_purge_truncate_history() some pages of the newly truncated
      undo tablespace could be discarded.
      
      fsp_try_extend_data_file(): Apply the peculiar rounding of
      fil_space_t::size_in_header only to the system tablespace,
      whose size can be expressed in megabytes in a configuration parameter.
      Other files may freely grow by a number of pages.
      
      fseg_alloc_free_page_low(): Do allow the extension of undo tablespaces,
      and mention the file name in the error message.
      
      mtr_t::commit_shrink(): Implement crash-safe shrinking of a tablespace
      file. First, durably write the log, then shrink the file, and finally
      release the page latches of the rebuilt tablespace. Refactored from
      trx_purge_truncate_history().
      
      log_write_and_flush_prepare(), log_write_and_flush(): New functions
      to durably write log during mtr_t::commit_shrink().
      1cb218c3
    • Marko Mäkelä's avatar
      21d19ed4
    • Daniel Ye's avatar
      MDEV-26545 Spider does not correctly handle UDF and stored function in where conds · 9fc1ef93
      Daniel Ye authored
      - Handle stored function conditions correctly, with the same logic as with UDFs.
      - When running queries on Spider SE, by default, we do not push down WHERE conditions containing usage of UDFs/stored functions to remote data nodes, unless the user demands (by setting spider_use_pushdown_udf).
      - Disable direct update/delete when a udf condition is skipped.
      9fc1ef93
    • Daniel Ye's avatar
      MDEV-26545 Spider does not correctly handle UDF and stored function in where conds · ac1c6738
      Daniel Ye authored
      - Handle stored function conditions correctly, with the same logic as with UDFs.
      - When running queries on Spider SE, by default, we do not push down WHERE conditions containing usage of UDFs/stored functions to remote data nodes, unless the user demands (by setting spider_use_pushdown_udf).
      ac1c6738
  6. 21 Sep, 2021 2 commits
  7. 22 Sep, 2021 2 commits
  8. 21 Sep, 2021 5 commits
  9. 20 Sep, 2021 1 commit
    • Julius Goryavsky's avatar
      MDEV-26441: Linux-dependent construct in SST scripts · f4d6d017
      Julius Goryavsky authored
      SST scripts currently use Linux-specific construction
      to create a temporary directory if the path prefix for
      that directory is specified by the user. This does not
      work with FreeBSD. This commit adds support for FreeBSD.
      
      No separate test required.
      f4d6d017
  10. 18 Sep, 2021 3 commits
    • Marko Mäkelä's avatar
      MDEV-26636: InnoDB defragmentation statistics cause races on TEMPORARY TABLE · 3209bc66
      Marko Mäkelä authored
      btr_defragment_save_defrag_stats_if_needed(): Do not save
      defragmentation statistics for temporary tables.
      They are exempt of defragmentation anyway
      (ha_innobase::optimize() never invokes defragmentation for them),
      and the user-visible names are not available inside InnoDB.
      
      Furthermore, InnoDB assumes that temporary tables are never accessed
      by other threads than the one that handles the session with which
      the temporary table is associated with.
      
      Furthermore, we simplify the test innodb.innodb_defrag_stats
      and include a test case that demonstrates that defragmentation
      statistics are no longer being saved for temporary tables.
      3209bc66
    • Marko Mäkelä's avatar
      MDEV-25919 fixup: Acquire MDL also in defragmentation · b740b235
      Marko Mäkelä authored
      dict_stats_process_entry_from_defrag_pool(): Acquire MDL on the table
      for which we are invoking dict_stats_save_defrag_stats(), to avoid
      any race condition with DROP TABLE or similar operations.
      b740b235
    • Marko Mäkelä's avatar
      MDEV-24258 fixup: Correct a condition · 56843d62
      Marko Mäkelä authored
      dict_stats_process_entry_from_defrag_pool(): Restore a condition as it was
      before commit 82b7c561.
      56843d62
  11. 17 Sep, 2021 4 commits