1. 28 Jan, 2022 3 commits
  2. 27 Jan, 2022 4 commits
    • Alexander Barkov's avatar
      MDEV-24487 Error after update to 10.5.8 on CentOS-8: DBD::mysql::st execute... · 430d60d1
      Alexander Barkov authored
      MDEV-24487 Error after update to 10.5.8 on CentOS-8: DBD::mysql::st execute failed: Unknown MySQL error
      
      The problem happened because the the new client capability flag
      CLIENT_EXTENDED_METADATA was not put into the cache entry key.
      So results cached by a new client were sent to the old client (and vica versa)
      with a mis-matching metadata, which made the client abort the connection on
      an unexpected result set metadata packet format.
      
      The problem was caused by the patch for:
        MDEV-17832 Protocol: extensions for Pluggable types and JSON, GEOMETRY
      which forgot to adjust the query cache code.
      
      Fix:
      
      - Adding a new member Query_cache_query_flags::client_extended_metadata,
        so only clients with equal CLIENT_EXTENDED_METADATA flag values can
        reuse results.
      
      - Adding a new column CLIENT_EXTENDED_METADATA into
        INFORMATION_SCHEMA.QUERY_CACHE_INFO (privided by the qc_info plugin).
      430d60d1
    • Vladislav Vaintroub's avatar
    • Oleksandr Byelkin's avatar
      new pcre 10.39 · a73acf6c
      Oleksandr Byelkin authored
      a73acf6c
    • mkaruza's avatar
      MDEV-26223 Galera cluster node consider old server_id value even after... · 53173709
      mkaruza authored
      MDEV-26223 Galera cluster node consider old server_id value even after modification of server_id [wsrep_gtid_mode=ON]
      
      Variable `wsrep_new_cluster` now will be TRUE also when there is only `gcomm://` used
      in configuration. This configuration, even without --wsrep-new-cluster,
      is considered to bootstrap new cluster.
      
      Updated galera GTID test to ignore warning message when non bootstrap
      node have server-id different thant one cluster is initialized with.
      Reviewed-by: default avatarJan Lindström <jan.lindstrom@mariadb.com>
      53173709
  3. 26 Jan, 2022 1 commit
    • Marko Mäkelä's avatar
      MDEV-27610 Unnecessary wait in InnoDB crash recovery · 56f5599f
      Marko Mäkelä authored
      In recv_sys_t::apply(), we were unnecessarily looking up pages
      in buf_pool.page_hash and potentially waiting for exclusive page latches.
      
      Before buf_page_get_low() would return an x-latched page,
      that page will have to be read and buf_page_read_complete() would
      have invoked recv_recover_page() to apply the log to the page.
      
      Therefore, it suffices to invoke recv_read_in_area() to trigger
      a transition from RECV_NOT_PROCESSED.
      
      recv_read_in_area(): Take the iterator as a parameter, and remove
      page_id lookups. Should the page already be in buf_pool.page_hash,
      buf_page_init_for_read() will return nullptr to buf_read_page_low()
      and buf_read_page_background().
      
      recv_sys_t::apply(): Replace goto, remove dead code, and add assertions
      to guarantee that the iteration will make progress.
      
      Reviewed by: Vladislav Lesin
      56f5599f
  4. 25 Jan, 2022 4 commits
  5. 21 Jan, 2022 2 commits
    • Alexander Barkov's avatar
      MDEV-27018 IF and COALESCE lose "json" property · e4b302e4
      Alexander Barkov authored
      Hybrid functions (IF, COALESCE, etc) did not preserve the JSON property
      from their arguments. The same problem was repeatable for single row subselects.
      
      The problem happened because the method Item::is_json_type() was inconsistently
      implemented across the Item hierarchy. For example, Item_hybrid_func
      and Item_singlerow_subselect did not override is_json_type().
      
      Solution:
      
      - Removing Item::is_json_type()
      
      - Implementing specific JSON type handlers:
        Type_handler_string_json
        Type_handler_varchar_json
        Type_handler_tiny_blob_json
        Type_handler_blob_json
        Type_handler_medium_blob_json
        Type_handler_long_blob_json
      
      - Reusing the existing data type infrastructure to pass JSON
        type handlers across all item types, including classes Item_hybrid_func
        and Item_singlerow_subselect. Note, these two classes themselves do not
        need any changes!
      
      - Extending the data type infrastructure so data types can inherit
        their properties (e.g. aggregation rules) from their base data types.
        E.g. VARCHAR/JSON acts as VARCHAR, LONGTEXT/JSON acts as LONGTEXT
        when mixed to a non-JSON data type. This is done by:
          - adding virtual method Type_handler::type_handler_base()
          - adding a helper class Type_handler_pair
          - refactoring Type_handler_hybrid_field_type methods
            aggregate_for_result(), aggregate_for_min_max(),
            aggregate_for_num_op() to use Type_handler_pair.
      
      This change also fixes:
      
        MDEV-27361 Hybrid functions with JSON arguments do not send format metadata
      
      Also, adding mtr tests for JSON replication. It was not covered yet.
      And the current patch changes the replication code slightly.
      e4b302e4
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-26784 [Warning] InnoDB: Difficult to find free blocks in the buffer pool · 28e166d6
      Thirunarayanan Balathandayuthapani authored
      Problem:
      =======
        InnoDB ran out of memory during recovery and it fails to
      flush the dirty LRU blocks. The reason is that buffer pool
      can ran out before the LRU list length reaches
      BUF_LRU_OLD_MIN_LEN(256) threshold.
      
      Fix:
      ====
      During recovery, InnoDB should write out and evict all
      dirty blocks.
      28e166d6
  6. 20 Jan, 2022 2 commits
  7. 19 Jan, 2022 1 commit
    • Sergei Petrunia's avatar
      MDEV-27382: OFFSET is ignored when combined with DISTINCT · 7259b299
      Sergei Petrunia authored
      A query in form
      
        SELECT DISTINCT expr_that_is_inferred_to_be_const LIMIT 0 OFFSET n
      
      produces one row when it should produce none. The issue was in
      JOIN_TAB::remove_duplicates() in the piece of logic that tried to
      avoid duplicate removal for such cases but didn't account for possible
      "LIMIT 0".
      
      Fixed by making Select_limit_counters::set_limit() change OFFSET to 0
      when LIMIT is 0.
      7259b299
  8. 18 Jan, 2022 2 commits
    • Vlad Lesin's avatar
      MDEV-27025 insert-intention lock conflicts with waiting ORDINARY lock · be811386
      Vlad Lesin authored
      The code was backported from 10.6 bd03c0e5
      commit. See that commit message for details.
      
      Apart from the above commit trx_lock_t::wait_trx was also backported from
      MDEV-24738. trx_lock_t::wait_trx is protected with lock_sys.wait_mutex
      in 10.6, but that mutex was implemented only in MDEV-24789. As there is no
      need to backport MDEV-24789 for MDEV-27025,
      trx_lock_t::wait_trx is protected with the same mutexes as
      trx_lock_t::wait_lock.
      
      This fix should not break innodb-lock-schedule-algorithm=VATS. This
      algorithm uses an Eldest-Transaction-First (ETF) heuristic, which prefers
      older transactions over new ones. In this fix we just insert granted lock
      just before the last granted lock of the same transaction, what does not
      change transactions execution order.
      
      The changes in lock_rec_create_low() should not break Galera Cluster,
      there is a big "if" branch for WSREP. This branch is necessary to provide
      the correct transactions execution order, and should not be changed for
      the current bug fix.
      be811386
    • Marko Mäkelä's avatar
      MDEV-27499 Performance regression in log_checkpoint_margin() · e44439ab
      Marko Mäkelä authored
      In commit 4c3ad244 (MDEV-27416)
      an unnecessarily strict wait condition was introduced in the
      function buf_flush_wait(). Most callers actually only care that
      the pages have been flushed, not that a checkpoint has completed.
      
      Only in the buf_flush_sync() call for log resizing, we might care
      about the log checkpoint. But, in fact,
      srv_prepare_to_delete_redo_log_file() is explicitly disabling
      checkpoints. So, we can simply remove the unnecessary wait loop.
      
      Thanks to Krunal Bauskar for reporting this performance regression
      that we failed to repeat in our testing.
      e44439ab
  9. 17 Jan, 2022 3 commits
  10. 15 Jan, 2022 3 commits
  11. 14 Jan, 2022 2 commits
    • Marko Mäkelä's avatar
      Remove FIXME comments that refer to an early MDEV-14425 plan · 8535c260
      Marko Mäkelä authored
      In MDEV-14425, an early plan was to introduce a separate log file
      for file-level records and checkpoint information. The reasoning was
      that fil_system.mutex contention would be reduced by not having to
      maintain fil_system.named_spaces. The mutex contention was actually
      fixed in MDEV-23855 by making some data fields in fil_space_t and
      fil_node_t use std::atomic.
      
      Using a single circular log file simplifies recovery and backup.
      8535c260
    • Marko Mäkelä's avatar
      MDEV-27500 buf_page_free() fails to drop the adaptive hash index · c104a01b
      Marko Mäkelä authored
      The function buf_page_free() that was introduced
      in commit a35b4ae8 (MDEV-15528)
      failed to remove any adaptive hash index entries for the page
      before freeing the page.
      
      This caused an assertion failure on shutdown of 10.6 server of
      in the function buf_pool_t::clear_hash_index() with the expression:
      (s >= buf_page_t::UNFIXED || s == buf_page_t::REMOVE_HASH).
      The assertion would fail for a block that is in the freed state.
      
      The failing assertion was added in
      commit aaef2e1d
      in the 10.6 branch.
      
      Thanks to Matthias Leich for finding the bug and testing the fix.
      c104a01b
  12. 12 Jan, 2022 2 commits
  13. 11 Jan, 2022 1 commit
    • Eugene Kosov's avatar
      MDEV-27022 Buffer pool is being flushed during recovery · f443cd11
      Eugene Kosov authored
      The problem was introduced by the removal of buf_pool.flush_rbt
      in commit 46b1f500 (MDEV-23399)
      
      recv_sys_t::apply(): don't write to disc and fsync() the last batch.
      Insead, sort it by oldest_modification for MariaDB server and some
      mariabackup operations.
      
      log_sort_flush_list(): a thread-safe function which sorts buf_pool::flush_list
      f443cd11
  14. 10 Jan, 2022 1 commit
    • Rucha Deodhar's avatar
      MDEV-23836: Assertion `! is_set() || m_can_overwrite_status' in · 81e00485
      Rucha Deodhar authored
      Diagnostics_area::set_error_status (interrupted ALTER TABLE under LOCK)
      
      Analysis: KILL_QUERY is not ignored when local memory used exceeds maximum
      session memory. Hence the query proceeds, OK is sent and we end up
      reopening tables that are marked for reopen. During this, kill status is
      eventually checked and assertion failure happens during trying to send error
      message because OK has already been sent.
      Fix: Ok is already sent so statement has already executed. It is too
      late to give error. So ignore kill.
      81e00485
  15. 09 Jan, 2022 1 commit
  16. 04 Jan, 2022 1 commit
    • Marko Mäkelä's avatar
      MDEV-27416 InnoDB hang in buf_flush_wait_flushed(), on log checkpoint · 4c3ad244
      Marko Mäkelä authored
      InnoDB could sometimes hang when triggering a log checkpoint. This is
      due to commit 7b1252c0 (MDEV-24278),
      which introduced an untimed wait to buf_flush_page_cleaner().
      
      The hang was noticed by occasional failures of IMPORT TABLESPACE tests,
      such as innodb.innodb-wl5522, which would (unnecessarily) invoke
      log_make_checkpoint() from row_import_cleanup().
      
      The reason of the hang was that buf_flush_page_cleaner() would enter
      untimed sleep despite buf_flush_sync_lsn being set. The exact failure
      scenario is unclear, because buf_flush_sync_lsn should actually be
      protected by buf_pool.flush_list_mutex. We prevent the hang by
      invoking buf_pool.page_cleaner_set_idle(false) whenever we are
      setting buf_flush_sync_lsn and signaling buf_pool.do_flush_list.
      
      The bulk of these changes was originally developed as a preparation
      for MDEV-26827, to invoke buf_flush_list() from fewer threads,
      and tested on 10.6 by Matthias Leich.
      
      This fix was tested by running 100 repetitions of 100 concurrent instances
      of the test innodb.innodb-wl5522 on a RelWithDebInfo build, using ext4fs
      and innodb_flush_method=O_DIRECT on a SATA SSD with 4096-byte block size.
      During the test, the call to log_make_checkpoint() in row_import_cleanup()
      was present.
      
      buf_flush_list(): Make static.
      
      buf_flush_wait(): Wait for buf_pool.get_oldest_modification()
      to reach a target, by work done in the buf_flush_page_cleaner.
      If buf_flush_sync_lsn is going to be set, we will invoke
      buf_pool.page_cleaner_set_idle(false).
      
      buf_flush_ahead(): If buf_flush_sync_lsn or buf_flush_async_lsn
      is going to be set and the page cleaner woken up, we will invoke
      buf_pool.page_cleaner_set_idle(false).
      
      buf_flush_wait_flushed(): Invoke buf_flush_wait().
      
      buf_flush_sync(): Invoke recv_sys.apply() at the start in case
      crash recovery is active. Invoke buf_flush_wait().
      
      buf_flush_sync_batch(): A lower-level variant of buf_flush_sync()
      that is only called by recv_sys_t::apply().
      
      buf_flush_sync_for_checkpoint(): Do not trigger log apply
      or checkpoint during recovery.
      
      buf_dblwr_t::create(): Only initiate a buffer pool flush, not
      a checkpoint.
      
      row_import_cleanup(): Do not unnecessarily invoke log_make_checkpoint().
      Invoking buf_flush_list_space() before starting to generate redo log
      for the imported tablespace should suffice.
      
      srv_prepare_to_delete_redo_log_file():
      Set recv_sys.recovery_on in order to prevent
      buf_flush_sync_for_checkpoint() from initiating a checkpoint
      while the log is inaccessible. Remove a wait loop that is already
      part of buf_flush_sync().
      Do not invoke fil_names_clear() if the log is being upgraded,
      because the FILE_MODIFY record is specific to the latest format.
      
      create_log_file(): Clear recv_sys.recovery_on only after calling
      log_make_checkpoint(), to prevent buf_flush_page_cleaner from
      invoking a checkpoint.
      
      innodb_shutdown(): Simplify the logic in mariadb-backup --prepare.
      
      os_aio_wait_until_no_pending_writes(): Update the function comment.
      Apart from row_quiesce_table_start() during FLUSH TABLES...FOR EXPORT,
      this is being called by buf_flush_list_space(), which is invoked
      by ALTER TABLE...IMPORT TABLESPACE as well as some encryption operations.
      4c3ad244
  17. 03 Jan, 2022 4 commits
  18. 28 Dec, 2021 1 commit
  19. 27 Dec, 2021 2 commits