1. 05 Sep, 2023 2 commits
    • Monty's avatar
      Fix compression tests for s390x · b0847443
      Monty authored
      The problem is that s390x is not using the default bzip library we use
      on other platforms, which causes compressed string lengths to be differnt
      than what mtr tests expects.
      
      Fixed by:
      - Added have_normal_bzip.inc, which checks if compress() returns the
        expected length.
      - Adjust the results to match the expected one
        - main.func_compress.test & archive.archive
      - Don't print lengths that depends on compression library
        - mysqlbinlog compress tests & connect.zip
      - Don't print DATA_LENGTH for SET column_compression_zlib_level=1
        - main.column_compression
      b0847443
    • Dmitry Shulga's avatar
      Merge branch '10.4' into 10.5 · 68a925b3
      Dmitry Shulga authored
      68a925b3
  2. 04 Sep, 2023 1 commit
  3. 02 Sep, 2023 5 commits
    • Dmitry Shulga's avatar
      MDEV-14959: Fixed memory leak relating with view and IS · d0a872c2
      Dmitry Shulga authored
      Fixed memory leak taken place on executing a prepared statement or
      a stored routine that querying a view and this view constructed
      on an information schema table. For example,
      
      Lets consider the following definition of the view 'v1'
      CREATE VIEW v1 AS SELECT table_name FROM information_schema.views
      ORDER BY table_name;
      
      Querying this view in PS mode result in hit of assert.
      PREPARE stmt FROM "SELECT * FROM v1";
      EXECUTE stmt;
      EXECUTE stmt; (*)
      
      Running the statement marked with (*) leads to a crash in case
      server build with mode to control allocation of a memory from SP/PS
      memory root on the second and following executions of PS/SP.
      
      The reason of leaking the memory is that a memory allocated on
      processing of FRM file for the view requested from a PS/PS memory
      root meaning that this memory be released only when a stored routine
      be evicted from SP-cache or a prepared statement be deallocated
      that typically happens on termination of a user session.
      
      To fix the issue switch to a memory root specially created for
      allocation of short-lived objects that requested on parsing FRM.
      d0a872c2
    • Dmitry Shulga's avatar
      MDEV-14959: Fixed memory leak happened on re-parsing a view that substitutes a table · be023562
      Dmitry Shulga authored
      In case a table accessed by a PS/SP is dropped after the first execution of
      PS/SP and a view created with the same name as a table just dropped then
      the second execution of PS/SP leads to allocation of a memory on SP/PS
      memory root already marked as read only on first execution.
      
      For example, the following test case:
      CREATE TABLE t1 (a INT);
      PREPARE stmt FROM "INSERT INTO t1 VALUES (1)";
      EXECUTE stmt;
      DROP TABLE t1;
      CREATE VIEW t1 S SELECT 1;
      --error ER_NON_INSERTABLE_TABLE
      EXECUTE stmt; # (*)
      DROP VIEW t1;
      
      will hit assert on running the statement 'EXECUTE stmt' marked with (*)
      when allocation of a memory be performed on parsing the view.
      
      Memory allocation is requested inside the function mysql_make_view
      when a view definition being parsed. In order to avoid an assertion
      failure, call of the function mysql_make_view() must be moved after
      invocation of the function check_and_update_table_version().
      It will result in re-preparing the whole PS statement or current
      SP instruction that will free currently allocated items and reset
      read_only flag for the memory root.
      be023562
    • Dmitry Shulga's avatar
      MDEV-14959: Fixed possible memory leaks that could happen on running PS/SP depending on a trigger · 1d502a29
      Dmitry Shulga authored
      Moved call of the function check_and_update_table_version() just
      before the place where the function extend_table_list() is invoked
      in order to avoid allocation of memory on a PS/SP memory root
      marked as read only. It happens by the reason that the function
      extend_table_list() invokes sp_add_used_routine() to add a trigger
      created for the table in time frame between execution the statement
      EXECUTE `stmt_id` .
      
      For example, the following test case
      create table t1 (a int);
      
      prepare stmt from "insert into t1 (a) value (1)";
      execute stmt;
      
      create trigger t1_bi before insert on t1 for each row
        set @message= new.a;
      
      execute stmt; # (*)
      
      adds the trigger t1_bi to a list of used routines that involves
      allocation of a memory on PS memory root that has been already marked
      as read only on first run of the statement 'execute stmt'.
      In result, when the statement marked with (*) is executed it results in
      assert hit.
      
      To fix the issue call the function check_and_update_table_version()
      before invocation of extend_table_list() to force re-compilation of
      PS/SP that resets read-only flag of its memory root.
      1d502a29
    • Dmitry Shulga's avatar
      MDEV-14959: Moved calculation the number of items reserved for exists to in transformation · d8574dbb
      Dmitry Shulga authored
      It is done now before call of select_lex->setup_ref_array()
      in order to avoid allocation of SP/PS's memory on its second invocation.
      d8574dbb
    • Dmitry Shulga's avatar
      MDEV-14959: Control over memory allocated for SP/PS · 0d4be10a
      Dmitry Shulga authored
      This patch adds support for controlling of memory allocation
      done by SP/PS that could happen on second and following executions.
      As soon as SP or PS has been executed the first time its memory root
      is marked as read only since no further memory allocation should
      be performed on it. In case such allocation takes place it leads to
      the assert hit for invariant that force no new memory allocations
      takes place as soon as the SP/PS has been marked as read only.
      
      The feature for control of memory allocation made on behalf SP/PS
      is turned on when both debug build is on and the cmake option
      -DWITH_PROTECT_STATEMENT_MEMROOT is set.
      
      The reason for introduction of the new cmake option
        -DWITH_PROTECT_STATEMENT_MEMROOT
      to control memory allocation of second and following executions of
      SP/PS is that for the current server implementation there are too many
      places where such memory allocation takes place. As soon as all such
      incorrect allocations be fixed the cmake option
       -DWITH_PROTECT_STATEMENT_MEMROOT
      can be removed and control of memory allocation made on second and
      following executions can be turned on only for debug build. Before
      every incorrect memory allocation be fixed it makes sense to guard
      the checking of memory allocation on read only memory by extra cmake
      option else we would get a lot of failing test on buildbot.
      
      Moreover, fixing of all incorrect memory allocations could take pretty
      long period of time, so for introducing the feature without necessary
      to wait until all places throughout the source code be fixed it makes
      sense to add the new cmake option.
      0d4be10a
  4. 01 Sep, 2023 1 commit
  5. 31 Aug, 2023 2 commits
    • Marko Mäkelä's avatar
      MDEV-32049 Deadlock due to log_free_check() in trx_purge_truncate_history() · 2db5f1b2
      Marko Mäkelä authored
      The function log_free_check() is not supposed to be invoked while
      the caller is holding any InnoDB synchronization objects, such as
      buffer page latches, tablespace latches, index tree latches, or
      in this case, rseg->mutex (rseg->latch in 10.6 or later).
      
      A hang was reported in 10.6 where several threads were waiting for
      an rseg->latch that had been exclusively acquired in
      trx_purge_truncate_history(), which invoked log_free_check() inside
      trx_purge_truncate_rseg_history(). Because the threads that were
      waiting for the rseg->latch were holding exclusive latches on some
      index pages, log_free_check() was unable to advance the checkpoint
      because those index pages could not be written out.
      
      trx_purge_truncate_history(): Invoke log_free_check() before
      acquiring the rseg->mutex and invoking trx_purge_free_segment().
      
      trx_purge_free_segment(): Do not invoke log_free_check() in order
      to avoid a deadlock.
      2db5f1b2
    • Marko Mäkelä's avatar
      MDEV-23974 fixup: Use standard quotes in have_innodb.inc · 3c86765e
      Marko Mäkelä authored
      This fixes the following test:
      set sql_mode=ORACLE;
      --source include/have_innodb.inc
      3c86765e
  6. 29 Aug, 2023 1 commit
    • Alexander Barkov's avatar
      MDEV-31303 Key not used when IN clause has both signed and usigned values · 53499cd1
      Alexander Barkov authored
      Summary:
      
      This patch enables possible index optimization when
      the WHERE clause has an IN condition of the form:
      
      signed_or_unsigned_column IN (signed_or_unsigned_constant,
                                    signed_or_unsigned_constant
                                    [,signed_or_unsigned_constant]*)
      
      when the IN list constants are of different signess, e.g.:
        WHERE signed_column   IN (signed_constant, unsigned_constant ...)
        WHERE unsigned_column IN (signed_constant, unsigned_constant ...)
      
      Details:
      
      In a condition like:
         WHERE unsigned_predicant IN (1, LONGLONG_MAX + 1)
      
      comparison handlers for individual (predicant,value) pairs are
      calculated as follows:
      
      * unsigned_predicant and 1 produce &type_handler_newdecimal
      * unsigned_predicant and (LONGLONG_MAX + 1) produce &type_handler_slonglong
      
      The old code decided that it could not use bisection because
      the two pairs had different comparison handlers.
      As a result, bisection was not allowed, and, in case of
      an indexed integer column predicant the index on the column was not used.
      
      The new code catches special cases like:
          signed_predicant   IN (signed_constant, unsigned_constant)
          unsigned_predicant IN (signed_constant, unsigned_constant)
      
      It enables bisection using in_longlong, which supports a mixture
      of predicant and values of different signess.
      In case when the predicant is an indexed column this change
      automatically enables index range optimization.
      
      Thanks to Vicențiu Ciorbaru for proposing the idea and for preparing MTR tests.
      53499cd1
  7. 28 Aug, 2023 2 commits
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-32028 InnoDB scrubbing doesn't write zero while freeing the extent · e938d7c1
      Thirunarayanan Balathandayuthapani authored
      Problem:
      ========
      InnoDB fails to mark the page status as FREED during
      freeing of an extent of a segment. This behaviour affects
      scrubbing and doesn't write all zeroes in file even though
      pages are freed.
      
      Solution:
      ========
      InnoDB should mark the page status as FREED before
      reinitialize the extent descriptor entry.
      e938d7c1
    • Dmitry Shulga's avatar
      MDEV-31890: Compilation failing on MacOS (unknown warning option -Wno-unused-but-set-variable) · 1fde7853
      Dmitry Shulga authored
      For clang compiler the compiler's flag -Wno-unused-but-set-variable
      was set based on compiler version. This approach could result in
      false positive detection for presence of compiler option since
      only first three groups of digits in compiler version taken into account
      and it could lead to inaccuracy in determining of supported compiler's
      features.
      
      Correct way to detect options supported by a compiler is to use
      the macros  MY_CHECK_CXX_COMPILER_FLAG and to check the result of
      variable with prefix have_CXX__
      So, to check whether compiler does support the option
       -Wno-unused-but-set-variable
      the macros
       MY_CHECK_CXX_COMPILER_FLAG(-Wno-unused-but-set-variable)
      should be called and the result variable
       have_CXX__Wno_unused_but_set_variable
      be tested for assigned value.
      1fde7853
  8. 24 Aug, 2023 2 commits
  9. 23 Aug, 2023 2 commits
    • Yuchen Pei's avatar
      Merge 10.4 into 10.5 · 0d88365b
      Yuchen Pei authored
      0d88365b
    • Yuchen Pei's avatar
      MDEV-31117 Fix spider connection info parsing · e9f3ca61
      Yuchen Pei authored
      Spider connection string is a comma-separated parameter definitions,
      where each definition is of the form "<param_title> <param_value>",
      where <param_value> is quote delimited on both ends, with backslashes
      acting as an escaping prefix.
      
      Despite the simple syntax, the existing spider connection string
      parser was poorly-written, complex, hard to reason and error-prone,
      causing issues like the one described in MDEV-31117. For example it
      treated param title the same way as param value when assigning, and
      have nonsensical fields like delim_title_len and delim_title.
      
      Thus as part of the bugfix, we clean up the spider comment connection
      string parsing, including:
      
      - Factoring out some code from the parsing function
      - Simplify the struct `st_spider_param_string_parse`
      - And any necessary changes caused by the above changes
      e9f3ca61
  10. 22 Aug, 2023 2 commits
  11. 21 Aug, 2023 2 commits
    • Oleksandr Byelkin's avatar
      Make vgdb call more universal. · c062b351
      Oleksandr Byelkin authored
      c062b351
    • Marko Mäkelä's avatar
      Remove a stale comment · be5fd3ec
      Marko Mäkelä authored
      buf_LRU_block_remove_hashed(): Remove a comment that had been added
      in mysql/mysql-server@aad1c7d0dd8a152ef6bb685356c68ad9978d686a
      and apparently referring to buf_LRU_invalidate_tablespace(),
      which was later replaced with buf_LRU_flush_or_remove_pages() and
      ultimately with buf_flush_remove_pages() and buf_flush_list_space().
      All that code is covered by buf_pool.mutex. The note about releasing
      the hash_lock for the buf_pool.page_hash slice would actually apply to
      the last reference to hash_lock in buf_LRU_free_page(), for the
      case zip=false (retaining a ROW_FORMAT=COMPRESSED page while
      discarding the uncompressed one).
      be5fd3ec
  12. 17 Aug, 2023 4 commits
  13. 16 Aug, 2023 1 commit
    • Kristian Nielsen's avatar
      MDEV-29974: Missed kill waiting for worker queues to drain · 34e85854
      Kristian Nielsen authored
      When the SQL driver thread goes to wait for room in the parallel slave
      worker queue, there was a race where a kill at the right moment could
      be ignored and the wait proceed uninterrupted by the kill.
      
      Fix by moving the THD::check_killed() to occur _after_ doing ENTER_COND().
      
      This bug was seen as sporadic failure of the testcase rpl.rpl_parallel
      (rpl.rpl_parallel_gco_wait_kill since 10.5), with "Slave stopped with
      wrong error code".
      Signed-off-by: default avatarKristian Nielsen <knielsen@knielsen-hq.org>
      34e85854
  14. 15 Aug, 2023 9 commits
    • Kristian Nielsen's avatar
      Merge 10.4 into 10.5 · 7c9837ce
      Kristian Nielsen authored
      Signed-off-by: default avatarKristian Nielsen <knielsen@knielsen-hq.org>
      7c9837ce
    • Kristian Nielsen's avatar
      MDEV-31655: Parallel replication deadlock victim preference code errorneously removed · 900c4d69
      Kristian Nielsen authored
      Restore code to make InnoDB choose the second transaction as a deadlock
      victim if two transactions deadlock that need to commit in-order for
      parallel replication. This code was erroneously removed when VATS was
      implemented in InnoDB.
      
      Also add a test case for InnoDB choosing the right deadlock victim.
      Also fixes this bug, with testcase that reliably reproduces:
      
      MDEV-28776: rpl.rpl_mark_optimize_tbl_ddl fails with timeout on sync_with_master
      
      Note: This should be null-merged to 10.6, as a different fix is needed
      there due to InnoDB locking code changes.
      Signed-off-by: default avatarKristian Nielsen <knielsen@knielsen-hq.org>
      900c4d69
    • Kristian Nielsen's avatar
      MDEV-31482: Lock wait timeout with INSERT-SELECT, autoinc, and statement-based replication · 920789e9
      Kristian Nielsen authored
      Remove the exception that InnoDB does not report auto-increment locks waits
      to the parallel replication.
      
      There was an assumption that these waits could not cause conflicts with
      in-order parallel replication and thus need not be reported. However, this
      assumption is wrong and it is possible to get conflicts that lead to hangs
      for the duration of --innodb-lock-wait-timeout. This can be seen with three
      transactions:
      
      1. T1 is waiting for T3 on an autoinc lock
      2. T2 is waiting for T1 to commit
      3. T3 is waiting on a normal row lock held by T2
      
      Here, T3 needs to be deadlock killed on the wait by T1.
      
      Note: This should be null-merged to 10.6, as a different fix is needed
      there due to InnoDB lock code changes.
      Signed-off-by: default avatarKristian Nielsen <knielsen@knielsen-hq.org>
      920789e9
    • Marko Mäkelä's avatar
      Remove the often-hanging test innodb.alter_rename_files · b4ace139
      Marko Mäkelä authored
      The test innodb.alter_rename_files rather frequently hangs in
      checkpoint_set_now. The test was removed in MariaDB Server 10.5
      commit 37e7bde1 when the code that
      it aimed to cover was simplified. Starting with MariaDB Server 10.5
      the page flushing and log checkpointing is much simpler, handled
      by the single buf_flush_page_cleaner() thread.
      
      Let us remove the test to avoid occasional failures. We are not going
      to fix the cause of the failure in MariaDB Server 10.4.
      b4ace139
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · 599c4d9a
      Marko Mäkelä authored
      599c4d9a
    • Marko Mäkelä's avatar
      Merge mariadb-10.4.31 into 10.4 · 6fdc6846
      Marko Mäkelä authored
      6fdc6846
    • Marko Mäkelä's avatar
      Merge mariadb-10.5.22 into 10.5 · 2e78465d
      Marko Mäkelä authored
      2e78465d
    • Alexander Barkov's avatar
      MDEV-24797 Column Compression - ERROR 1265 (01000): Data truncated for column · 9c8ae6dc
      Alexander Barkov authored
      Fix issue was earlier fixed by MDEV-31724. Only adding MTR tests.
      9c8ae6dc
    • Alexander Barkov's avatar
      MDEV-31724 Compressed varchar values lost on joins when sorting on columns from joined table(s) · 1fa7c9a3
      Alexander Barkov authored
      Field_varstring::get_copy_func() did not take into account
      that functions do_varstring1[_mb], do_varstring2[_mb] do not support
      compressed data.
      
      Changing the return value of Field_varstring::get_copy_func()
      to `do_field_string` if there is a compresion and truncation
      at the same time. This fixes the problem, so now it works as follows:
      - val_str() uncompresses the data
      - The prefix is then calculated on the uncompressed data
      
      Additionally, introducing two new copying functions
      - do_varstring1_no_truncation()
      - do_varstring2_no_truncation()
      
      Using new copying functions in cases when:
      - a Field_varstring with length_bytes==1 is changing to a longer
          Field_varstring with length_bytes==1
      - a Field_varstring with length_bytes==2 is changing to a longer
          Field_varstring with length_bytes==2
      
      In these cases we don't care neither of compression nor
      of multi-byte prefixes: the entire data gets fully copied
      from the source column to the target column as is.
      
      This is a kind of new optimization, but this also was needed
      to preserve existing MTR test results.
      1fa7c9a3
  15. 14 Aug, 2023 2 commits
  16. 11 Aug, 2023 1 commit
  17. 10 Aug, 2023 1 commit