1. 22 Jul, 2021 4 commits
    • Marko Mäkelä's avatar
      MDEV-26110: Do not rely on alignment on static allocation · 82d59945
      Marko Mäkelä authored
      It is implementation-defined whether alignment requirements
      that are larger than std::max_align_t (typically 8 or 16 bytes)
      will be honored by the compiler and linker.
      
      It turns out that on IBM AIX, both alignas() and MY_ALIGNED()
      only guarantees alignment up to 16 bytes.
      
      For some data structures, specifying alignment to the CPU
      cache line size (typically 64 or 128 bytes) is a mere performance
      optimization, and we do not really care whether the requested
      alignment is guaranteed.
      
      But, for the correct operation of direct I/O, we do require that
      the buffers be aligned at a block size boundary.
      
      field_ref_zero: Define as a pointer, not an array.
      For innochecksum, we can make this point to unaligned memory;
      for anything else, we will allocate an aligned buffer from the heap.
      This buffer will be used for overwriting freed data pages when
      innodb_immediate_scrub_data_uncompressed=ON. And exactly that code
      hit an assertion failure on AIX, in the test innodb.innodb_scrub.
      
      log_sys.checkpoint_buf: Define as a pointer to aligned memory
      that is allocated from heap.
      
      log_t::file::write_header_durable(): Reuse log_sys.checkpoint_buf
      instead of trying to allocate an aligned buffer from the stack.
      82d59945
    • Marko Mäkelä's avatar
      MDEV-26200 buf_pool.flush_list corrupted by buffer pool resizing or ROW_FORMAT=COMPRESSED · bf435a3f
      Marko Mäkelä authored
      The lazy deletion of clean blocks from buf_pool.flush_list that was
      introduced in commit 6441bc61 (MDEV-25113)
      introduced a race condition around the function
      buf_flush_relocate_on_flush_list().
      
      The test innodb_zip.wl5522_debug_zip as well as the buffer pool
      resizing tests would occasionally fail in debug builds due to
      buf_pool.flush_list.count disagreeing with the actual length of the
      doubly-linked list.
      
      The safe procedure for relocating a block in buf_pool.flush_list should be
      as follows, now that we implement lazy deletion from buf_pool.flush_list:
      
      1. Acquire buf_pool.mutex.
      2. Acquire the exclusive buf_pool.page_hash.latch.
      3. Acquire buf_pool.flush_list_mutex.
      4. Copy the block descriptor.
      5. Invoke buf_flush_relocate_on_flush_list().
      6. Release buf_pool.flush_list_mutex.
      
      buf_flush_relocate_on_flush_list(): Assert that
      buf_pool.flush_list_mutex is being held. Invoke
      buf_page_t::oldest_modification() only once, using
      std::memory_order_relaxed, now that the mutex protects us.
      
      buf_LRU_free_page(), buf_LRU_block_remove_hashed(): Avoid
      an unlock-lock cycle on hash_lock. (We must not acquire hash_lock
      while already holding buf_pool.flush_list_mutex, because that
      could lead to a deadlock due to latching order violation.)
      bf435a3f
    • nia's avatar
      Fix building crc32_arm64 on NetBSD/aarch64 · 316a8ceb
      nia authored
      pmull_supported is not necessarily defined before crc32c_aarch64
      Signed-off-by: default avatarNia Alarie <nia@NetBSD.org>
      316a8ceb
    • Daniel Black's avatar
      mtr: aix - stack-trace is optional · 1519013f
      Daniel Black authored
      1519013f
  2. 21 Jul, 2021 1 commit
    • Daniel Black's avatar
      debian/salsa: Show complete auth and plugin situtation · 8642f592
      Daniel Black authored
      SHOW PLUGINS has a more complete view of the installed
      plugins into the server.
      
      The mysql.user is a compatibility view that doesn't
      show the complete authentication picture. Use global_priv.
      
      Add `show create user` for default users to more clearly
      represent its contents.
      8642f592
  3. 20 Jul, 2021 2 commits
  4. 19 Jul, 2021 2 commits
  5. 16 Jul, 2021 3 commits
  6. 15 Jul, 2021 2 commits
  7. 14 Jul, 2021 1 commit
    • Nayuta Yanagisawa's avatar
      MDEV-26139 Spider crashes with segmentation fault (signal 11) on CREATE TABLE... · e3814a74
      Nayuta Yanagisawa authored
      MDEV-26139 Spider crashes with segmentation fault (signal 11) on CREATE TABLE when COMMENT does not contain embedded double quotes
      
      The root cause of the bug MDEV-26139 is the lack of NULL checking
      on the variable `dq`.
      
      Comments on if (dq && (!sq || sq > dq)) {...} else {...}:
      
        * The if block corresponds to the case where parameters are
          quoted by double quotes. In that case, a single quote doesn't
          appear at all or only appears in the middle of double quotes.
      
        * The else block corresponds to the case where parameters are
          quoted by single quotes. In that case, a double quote doesn't
          appear at all or only appears in the middle of single quotes.
      
        * If the program reaches the if-else statement, `sq || dq` holds.
          Thus, the negation of `dq && (!sq || sq > dq)` is equivalent to
          `sq && (!dq || sq <= dq)`.
      e3814a74
  8. 12 Jul, 2021 1 commit
  9. 09 Jul, 2021 1 commit
    • Igor Babaev's avatar
      MDEV-26108 Crash with query referencing twice CTE that uses embedded recursive CTE · 78735dca
      Igor Babaev authored
      This bug could affect queries that had at least two references to a CTE that
      used an embedded recursive CTE.
      Starting from version 10.4 some code in With_element::clone_parsed_spec()
      that assumed a certain order of selects after parsing the specification of
      a CTE became not valid anymore. It could lead to global select lists where
      some selects were missing. If a missing CTE happened to belong to the
      recursive part of a recursive CTE some recursive table references were not
      set as references to materialized derived tables and this caused a crash of
      the server.
      
      Approved by Oleksandr Byelkin <sanja@mariadb.com>
      78735dca
  10. 07 Jul, 2021 1 commit
    • Sergei Petrunia's avatar
      MDEV-26106: [ERROR] InnoDB: Unlock row could not find a 2 mode lock on the record · 35294053
      Sergei Petrunia authored
      Port the following patch from MySQL:
      
        commit 1b2e8ea269c80cb93cc79d8be934c40b1c58e947
        Author: Kailasnath Nagarkar <kailasnath.nagarkar@oracle.com>
        Date:   Fri Nov 30 16:43:13 2018 +0530
      
          Bug #20939184: INNODB: UNLOCK ROW COULD NOT FIND A 2 MODE
                         LOCK ON THE RECORD
      
          Issue:
          ------
          Consdier tables t1 and t2 such that t1 has multiple rows
          and join condition for t1 left join t2 results in only
          single row from t2.
      
          In this case, access to table t2 is const since there
          is a single row that qualifies the join condition.
      
          However, while executing the query, attempt is made to
          unlock t2's row multiple times.
      
          The current algorithm to fetch rows approximates to:
          1) Retrieve the row for t1.
          2) Retrieve the row for t2.
          3) Apply the join conditions.
             a) If condition evaluates to true:
                Project the row to the result.
             b) If condition evaluates to false:
                i) If t2's qep_tab->not_null_complement is true,
                   unlock t2's row.
                ii) Null-complement the row by calling
                    "evaluate_null_complemented_join_record()". In
                    this function qep_tab->not_null_complement is
                    set to false.
      
          The t2's only one row, that qualifies join condition,
          is unlocked in Step i) when t1's row is evaluated to
          false.
          When t1's next row is also evaluated to false, another
          attempt is made to unlock t2's already unlocked row.
      
          This results in following error being logged in error.log:
      
          "[ERROR] InnoDB: Unlock row could not find a 3 mode lock on
          the record. Current statement:
          select * from t1 left join t2 ......"
      
          Solution:
          ---------
          When a table's access method is "const", set record unlock
          method for this table to do no operation.
      35294053
  11. 06 Jul, 2021 7 commits
  12. 03 Jul, 2021 2 commits
    • Marko Mäkelä's avatar
      MDEV-26017 fixup · f0f47cbc
      Marko Mäkelä authored
      buf_flush_relocate_on_flush_list(): Use dpage->physical_size()
      because bpage->zip.ssize may already have been zeroed in
      page_zip_set_size() invoked by buf_pool_t::realloc().
      
      This would cause occasional failures of the test
      innodb.innodb_buffer_pool_resize, which creates a
      ROW_FORMAT=COMPRESSED table.
      f0f47cbc
    • Marko Mäkelä's avatar
      MDEV-26033: Race condition between buf_pool.page_hash and resize() · bd5a6403
      Marko Mäkelä authored
      The replacement of buf_pool.page_hash with a different type of
      hash table in commit 5155a300 (MDEV-22871)
      introduced a race condition with buffer pool resizing.
      
      We have an execution trace where buf_pool.page_hash.array is changed
      to point to something else while page_hash_latch::read_lock() is
      executing. The same should also affect page_hash_latch::write_lock().
      
      We fix the race condition by never resizing (and reallocating) the
      buf_pool.page_hash. We assume that resizing the buffer pool is
      a rare operation. Yes, there might be a performance regression if a
      server is first started up with a tiny buffer pool, which is later
      enlarged. In that case, the tiny buf_pool.page_hash.array could cause
      increased use of the hash bucket lists. That problem can be worked
      around by initially starting up the server with a larger buffer pool
      and then shrinking that, until changing to a larger size again.
      
      buf_pool_t::resize_hash(): Remove.
      
      buf_pool_t::page_hash_table::lock(): Do not attempt to deal with
      hash table resizing. If we really wanted that in a safe manner,
      we would probably have to introduce a global rw-lock around the
      operation, or at the very least, poll buf_pool.resizing, both of
      which would be detrimental to performance.
      bd5a6403
  13. 02 Jul, 2021 13 commits
    • Sergei Golubchik's avatar
      MDEV-23004 When using GROUP BY with JSON_ARRAYAGG with joint table, the square... · 77926284
      Sergei Golubchik authored
      MDEV-23004 When using GROUP BY with JSON_ARRAYAGG with joint table, the square brackets are not included
      
      make test results stable
      
      followup for 98c7916f
      77926284
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · 8c029d42
      Marko Mäkelä authored
      8c029d42
    • Marko Mäkelä's avatar
      MDEV-25236 Online log apply fails for ROW_FORMAT=REDUNDANT tables · a635588b
      Marko Mäkelä authored
      In other ROW_FORMAT than REDUNDANT, the InnoDB record header
      size calculation depends on dict_index_t::n_core_null_bytes.
      
      In ROW_FORMAT=REDUNDANT, the record header always is 6 bytes
      plus n_fields or 2*n_fields bytes, depending on the maximum
      record size. But, during online ALTER TABLE, the log records
      in the temporary file always use a format similar to
      ROW_FORMAT=DYNAMIC, even omitting the 5-byte fixed-length part
      of the header.
      
      While creating a temporary file record for a ROW_FORMAT=REDUNDANT
      table, InnoDB must refer to dict_index_t::n_nullable.
      The field dict_index_t::n_core_null_bytes is only valid for
      other than ROW_FORMAT=REDUNDANT tables.
      
      The bug does not affect MariaDB 10.3, because only
      commit 7a27db77 (MDEV-15563)
      allowed an ALGORITHM=INSTANT change of a NOT NULL column to
      NULL in a ROW_FORMAT=REDUNDANT table.
      
      The fix was developed by Thirunarayanan Balathandayuthapani
      and tested by Matthias Leich. The test case was simplified by me.
      a635588b
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · 372ea882
      Marko Mäkelä authored
      372ea882
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · f9194d02
      Marko Mäkelä authored
      f9194d02
    • Marko Mäkelä's avatar
      Fixup 586870f9 · a6adefad
      Marko Mäkelä authored
      One more result was affected by merging
      768c5188.
      a6adefad
    • Eugene Kosov's avatar
      submodules.cmake: add missing --depth=1 · ffe744e7
      Eugene Kosov authored
      ffe744e7
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · 15dcb8bd
      Marko Mäkelä authored
      15dcb8bd
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · c294443b
      Marko Mäkelä authored
      c294443b
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 05f7fd57
      Marko Mäkelä authored
      05f7fd57
    • Marko Mäkelä's avatar
      MDEV-26077 Assertion err != DB_DUPLICATE_KEY or unexpected ER_TABLE_EXISTS_ERROR · 2bf6f2c0
      Marko Mäkelä authored
      This is a backport of 161e4bfa.
      
      trans_rollback_to_savepoint(): Only release metadata locks (MDL)
      if the storage engines agree, after the changes were already rolled back.
      
      Ever since commit 3792693f
      and mysql/mysql-server@55ceedbc3feb911505dcba6cee8080d55ce86dda
      we used to cheat here and always release MDL if the binlog is disabled.
      
      MDL are supposed to prevent race conditions between DML and DDL also
      when no replication is in use. MDL are supposed to be a superset of
      InnoDB table locks: InnoDB table lock may only exist if the thread
      also holds MDL on the table name.
      
      In the included test case, ROLLBACK TO SAVEPOINT would wrongly release
      the MDL on both tables and let ALTER TABLE proceed, even though the DML
      transaction is actually holding locks on the table.
      
      Until commit 1bd681c8 (MDEV-25506)
      in MariaDB 10.6, InnoDB would often work around the locking violation
      in a blatantly non-ACID way: If locks exist on a table that is being
      dropped (in this case, actually a partition of a table that is being
      rebuilt by ALTER TABLE), InnoDB could move the table (or partition)
      into a queue, to be dropped after the locks and references had been
      released. If the lock is not released and the original copy of the
      table not dropped quickly enough, a name conflict could occur on
      a subsequent ALTER TABLE.
      
      The scenario of commit 3792693f
      is unaffected by this fix, because mysqldump
      would use non-locking reads, and the transaction would not be holding
      any InnoDB locks during the execution of ROLLBACK TO SAVEPOINT.
      MVCC reads inside InnoDB are only covered by MDL and page latches,
      not by any table or record locks.
      
      FIXME: It would be nice if storage engines were specifically asked
      which MDL can be released, instead of only offering a choice
      between all or nothing. InnoDB should be able to release any
      locks for tables that are no longer in trx_t::mod_tables, except
      if another transaction had converted some implicit record locks
      to explicit ones, before the ROLLBACK TO SAVEPOINT had been completed.
      
      Reviewed by: Sergei Golubchik
      2bf6f2c0
    • Marko Mäkelä's avatar
      MDEV-25129 fixup: Adjust test result · 5a2b6258
      Marko Mäkelä authored
      Fixup for commit 768c5188
      5a2b6258
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-25971 Instant ADD COLUMN fails to issue truncation warnings · e34877ab
      Thirunarayanan Balathandayuthapani authored
      A table rebuild that would truncate the default value of a
      DATE column is expected to issue data truncation warnings.
      But, these warnings are not being issued if the ADD COLUMN
      is being executed with ALGORITHM=INSTANT. InnoDB sets the
      warning of the field while assigning the default value
      of the field during check_if_supported_inplace_alter().
      e34877ab