1. 16 Apr, 2021 3 commits
    • Sujatha's avatar
      MDEV-16437: merge 5.7 P_S replication instrumentation and tables · 767648cc
      Sujatha authored
      Merge 'replication_applier_configuration' table.
      
      This table captures SQL_THREAD configuration parameters.
      767648cc
    • Sujatha's avatar
      MDEV-16437: merge 5.7 P_S replication instrumentation and tables · 70642871
      Sujatha authored
      Merge 'replication_applier_status_by_coordinator' table.
      
      This table captures SQL_THREAD status in case of both single threaded and
      multi threaded slave configuration. When multi_source replication is enabled
      this table will display each source specific SQL_THREAD status.
      
      Added new columns for:
       - LAST_SEEN_TRANSACTION
       - LAST_TRANS_RETRY_COUNT
      70642871
    • Sujatha's avatar
      MDEV-16437: merge 5.7 P_S replication instrumentation and tables · 2674365c
      Sujatha authored
      Merge 'replication_connection_configuration' table.
      
      Replaced following column:
        - AUTO_POSITION with USING_GTID
      Added new columns for:
        - IGNORE_SERVER_IDS
        - DO_DOMAIN_IDS
        - IGNORE_SERVER_IDS
      Removed following columns as they are not part of mariadb replication
      connection configuration:
        - NETWORK_INTERFACE
        - TLS_VERSION
      
      @sql/mysqld.cc
        Changed "master-retry-count" default value to 100000.
      2674365c
  2. 15 Apr, 2021 6 commits
  3. 14 Apr, 2021 15 commits
  4. 13 Apr, 2021 9 commits
    • Julius Goryavsky's avatar
      MDEV-25307: The value of the auto-increment variables changes during the test · 9ff737b2
      Julius Goryavsky authored
      Part #2, specifically for the 10.5+ branch:
      
      The auto-increment parameters can change sporadically during the
      execution of the mtr test "galera_vote_rejoin_ddl", causing it to
      fail. This patch creates an environment where unpredictable changes
      to these auto-increment settings do not occur during the test.
      9ff737b2
    • Sergei Golubchik's avatar
      -DMYSQL_MAINTAINER_MODE=NO · 55a7682a
      Sergei Golubchik authored
      also add =WARN as an alias for =OFF
      and clarify the help text
      55a7682a
    • Marko Mäkelä's avatar
      MDEV-24745 Generic CRC-32C computation wrongly uses SSE4.2 instructions · 58f184a4
      Marko Mäkelä authored
      In commit d25f806d (MDEV-22749)
      the CRC-32C implementation of MariaDB was broken on some
      IA-32 and AMD64 builds, depending on the compiler version and
      build options. This was verified for IA-32 on GCC 10.2.1.
      
      Even though we try to identify the SSE4.2 extensions and the
      availaibility of the PCLMULQDQ instruction by executing CPUID,
      the fall-back code could be generated with extended instructions,
      because the entire file mysys/crc32/crc32c.c was being compiled
      with -msse4.2 -mpclmul. This would cause SIGILL on a PINSRD
      instruction on affected IA-32 targets (such as some Intel Atom
      processors). This might also affect old AMD64 processors
      (predating the 2007 Intel Nehalem microarchitecture), if some
      compiler chose to emit the offending instructions.
      
      While it is fine to pass a target-specific option to a target-specific
      compilation unit (like -mpclmul to a PCLMUL-specific compilation unit),
      that is not safe for mixed-architecture compilation units.
      
      For mixed-architecture compilation units, the correct way is to set
      target attributes on the target-specific functions.
      
      There does not seem to be a way to pass target attributes to
      function template instantiation. Hence, we must replace the
      ExtendImpl template with plain functions crc32_sse42() and
      crc32_slow().
      
      We will also remove some inconsistency between
      my_crc32_implementation() and mysys_namespace::crc32::Choose_Extend().
      
      The function crc32_pclmul_enabled() will be moved to mysys/crc32/crc32c.cc
      so that the detection code will be compiled without -msse4.2 -mpclmul.
      
      The AMD64 PCLMUL accelerated crc32c_3way() will be moved to a new
      file crc32c_amd64.cc. In this way, only a few functions that depend
      on -msse4.2 in mysys/crc32/crc32c.cc can be declared with
      __attribute__((target("sse4.2"))), and most of the file can be compiled
      for the generic target.
      
      Last, the file mysys/crc32ieee.cc will be omitted on 64-bit POWER,
      because it was dead code (no symbols were exported).
      
      Reviewed by: Vladislav Vaintroub
      58f184a4
    • sjaakola's avatar
      MDEV-24966 Galera multi-master regression · a1e70388
      sjaakola authored
      After the merging of MDEV-24915, 10.6 branch has regressions with handling of
      concurrent write load against two or more cluster nodes. These regressions may
      surface as cluster hanging, node crashes or data inconsistency. With some test
      scenarios, the only visible symptom could be that the BF victim aborting happens
      only by innodb lock wait timeout expiration. This would result only to poor
      performance (by default 50 sec hang for each BF conflict), and could be somewhat
      difficult to diagnose.
      
      This pull request has following fixes to handle concurrent write load from
      multiple nodes:
      
      In lock_wait_wsrep_kill(), the victim trx was expected to be only in
      TRX_STATE_ACTIVE state. With the delayed BF conflict handling, it can happen
      that victim has advanced into pre commit state. This was fixed by choosing
      victim both in TRX_STATE_ACTIVE and TRX_STATE_PREPARED states.
      
      Victim transaction may be in several different states at the time of detected
      lock conflict, and due to delayed BF aborting practice in MDEV-24915, the victim
      may advance further before the actual BF aborting takes place. The BF aborting
      in MDEV-24915 did not wake the victim, if it was in the state of waiting for
      some other lock (than the one that was blocking the high priority thread).
      This anomaly caused the innodb lock wait timeout expiration delays and poor
      performance symptom. To fix this, lock_wait_wsrep_kill() now looks if
      victim is in lock waiting state, and uses lock_cancel_waiting_and_release()
      to cancel this lock wait.
      
      wsrep_bf_abort() checks if the victim has active transaction (in wsrep-lib),
      and starts a new transaction if there was no active transaction before.
      Due to late BF aborting, the victim may have e.g. failed in certification
      and is already aborting or has aborted at this stage. This has caused
      problems in testing where BF aborter tries to BF abort himself.
      The fix in wsrep_bf_abort() now skips the BF abort, if victim is aborting
      or has aborted. Victim may not have started transaction yet in wsrep context,
      but it may have acquired MDL locks (due to DDL execution), and this has
      caused BF conflict. Such case does not require aborting in wsrep or
      replication provider state.
      
      BF aborting could cause BF-BF conflict scenario, if victim was already aborted
      and changed to replayer having high priority as well. This BF-BF conflict
      scenario is now avoided in lock_wait_wsrep() where we now check if blocking
      lock holder is also high priority and is ordered before, caller should wait
      for the lock in this situation.
      
      The natural innodb deadlock resolving algorithm could pick BF thread as
      deadlock victim. This is fixed by giving max weigh to BF threads in
      Deadlock::report().
      
      MDEV-24341 has changed excution paths in do_command() and this affects BF
      aborted victim execution. This PR fixes one assert in do_command():
       DBUG_ASSERT(!thd->async_state.pending_ops())
      Which fired if the thd was BF aborted earlier. This assert is now changed
      to allow pending_ops() if thd was BF aborted before.
      
      With these fixes, long term highly conflicting write load could be run against
      to node cluster. If binlogging is configured, log_slave_updates should be
      also set.
      a1e70388
    • Anel Husakovic's avatar
      MDEV-23015: mariadb-setpermission seems incompatible with DBI:MariaDB · 18a82901
      Anel Husakovic authored
      - Commit 5cc2096f introduced in `10.5` changed DBI:DBD to DBD:MariaDB in this case with redudant `mysql` option.
      - According to database handle (dbh) and `connect` method one should follow
        https://metacpan.org/pod/DBD::MariaDB#Class-Methods with proper created data source name (dsn).
      - Adding socket precedance over port.
      - Adding skipping the comments when reading the `my.cnf` file.
      - MDEV-23016: mariadb-setpermission included
      18a82901
    • Marko Mäkelä's avatar
      MDEV-24620 ASAN heap-buffer-overflow in btr_pcur_restore_position() · b8c8692f
      Marko Mäkelä authored
      Between btr_pcur_store_position() and btr_pcur_restore_position()
      it is possible that purge empties a table and enlarges
      index->n_core_fields and index->n_core_null_bytes.
      Therefore, we must cache index->n_core_fields in
      btr_pcur_t::old_n_core_fields so that btr_pcur_t::old_rec can be
      parsed correctly.
      
      Unfortunately, this is a huge change, because we will replace
      "bool leaf" parameters with "ulint n_core"
      (passing index->n_core_fields, or 0 for non-leaf pages).
      For special cases where we know that index->is_instant() cannot hold,
      we may also pass index->n_fields.
      b8c8692f
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 6e6318b2
      Marko Mäkelä authored
      6e6318b2
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-24971 InnoDB access freed virtual column after rollback of secondary index · e262eb16
      Thirunarayanan Balathandayuthapani authored
      - Fixing post-fix failure. In clean_new_vcol_index(), InnoDB has the wrong
      offset to store the virtual column
      e262eb16
    • Dmitry Shulga's avatar
      MDEV-25197: The statement set password=password('') executed in PS mode fails... · 61f84bba
      Dmitry Shulga authored
      MDEV-25197: The statement set password=password('') executed in PS mode fails in case it is run by a user with expired password
      
      A user connected to a server with an expired password
      can't change password with the statement "SET password=..."
      if this statement is run in PS mode. In mentioned use case a user
      gets the error ER_MUST_CHANGE_PASSWORD on attempt to run
      the statement  PREPARE stmt FOR "SET password=...";
      
      The reason of failure to reset password by a locked user using the
      statement PREPARE stmt FOR "SET password=..." is that PS-related
      statements are not listed among the commands allowed for execution
      by a user with expired password. However, simple adding of PS-related
      statements (PREPARE FOR/EXECUTE/DEALLOCATE PREPARE ) to the list of
      statements allowed for execution by a locked user is not enough
      to solve problems, since it opens the opportunity for a locked user
      to execute any statement in the PS mode.
      
      To exclude this opportunity, additional checking that the statement
      being prepared for execution in PS-mode is the SET statement has to be added.
      This extra checking has been added by this patch into the method
      Prepared_statement::prepared() that executed on preparing any statement
      for execution in PS-mode.
      61f84bba
  5. 12 Apr, 2021 7 commits
    • Dmitriy Karpovskiy's avatar
      MDEV-24135: Print warnings in XML, save test retries in XML, save the... · f776fa96
      Dmitriy Karpovskiy authored
      MDEV-24135: Print warnings in XML, save test retries in XML, save the combinations in XML, replace the special symbols in the XML comment
      f776fa96
    • Alexander Barkov's avatar
    • Monty's avatar
      Fixed assert in WSREP if one started with --wsrep_provider=.. --wsrep_on=OFF · e14b6826
      Monty authored
      Assert was:
      mariadbd: /my/maria-10.6/wsrep-lib/src/client_state.cpp:256:
      int wsrep::client_state::after_statement(): Assertion `state() == s_exec'
      
      The reason was because of two faults:
      - A missing test for WSREP(thd) when checking wsrep_after_statement(()
      - THD->wsrep_cs().state was set to s_idle instead of s_none
      e14b6826
    • Oleksandr Byelkin's avatar
      MDEV-25182 Complex query in Store procedure corrupts results · 68e0defc
      Oleksandr Byelkin authored
      At the second execution of the PS
      1. mark_as_dependent() is called with the same parameters as at the first
         execution (select#4 and select#3)
      2. as outer_select (select#3) has been already merged at the first
         execution of PS it cannot be reached using the outer_select() function
         anymore (and so can not stop iteration).
      3. as a result all selects towards the top level select including the
         select for 'ca' are marked as uncacheable.
      4. Marked uncacheable it executed incorrectly triggering filling its
         temporary table several times and using freed memory at the end.
      
      To avoid the problem we use name resolution context to go "up".
      
      NOTE: problem also exists in 10.2 but has no visible effect on execution.
      That is why the problem is fixed in 10.2.
      
      The patch also add debug logging of important procedures and
      better specify parameters types of st_select_lex::mark_as_dependent.
      68e0defc
    • Dmitry Shulga's avatar
      MDEV-25108: Running of the EXPLAIN EXTENDED statement produces extra warning... · f8bf2a01
      Dmitry Shulga authored
      MDEV-25108: Running of the EXPLAIN EXTENDED statement produces extra warning in case it is executed in PS (prepared statement) mode
      
      The EXPLAIN EXTENDED statement run as a prepared statement can produce extra
      warning comparing with a case when EXPLAIN EXTENDED statement is run as
      a regular statement. For example, the following test case
        CREATE TABLE t1 (c int);
        CREATE TABLE t2 (d int);
        EXPLAIN EXTENDED SELECT (SELECT 1 FROM t2 WHERE d = c) FROM t1;
      
      produces the extra warning
        "Field or reference 'c' of SELECT #2 was resolved in SELECT #1"
      in case the above mentioned "EXPLAIN EXTENDED" statement is executed
      in PS mode, that is by submitting the following statements:
         PREPARE stmt FROM "EXPLAIN EXTENDED SELECT (SELECT 1 FROM t2 WHERE d = c) FROM t1";
         EXECUTE stmt;
      
      The reason of the extra warning emittion is in a way items
      are handled (being fixed) during execution of the JOIN::prepare() method.
      The method Item_field::fix_fields() calls the find_field_in_tables()
      function in case a field hasn't been associated yet with the item.
      Implementation of the find_field_in_tables() function first checks whether
      a table containing the required field was already opened and cached.
      It is done by checking the data member item->cached_table. This data member
      is set on handling the PRERARE FROM statement and checked on executing
      the EXECUTE statement. If the data member item->cached_table is set
      the find_field_in_tables() function invoked and the
      mark_select_range_as_dependent() function called if the field
      is an outer referencee. The mark_select_range_as_dependent() function
      calls the mark_as_dependent() function that finally invokes
      the push_warning_printf() function that produces extra warning.
      
      To fix the issue, calling of push_warning_printf() is elimited in case
      it was run indirectly in result of hanlding already opened table from
      the Item_field::fix_fields() method.
      f8bf2a01
    • Julius Goryavsky's avatar
      MDEV-21484: galera_sst_mariabackup_encrypt_with_key test failed · e95cdc45
      Julius Goryavsky authored
      This commit removes the mtr test galera_sst_mariabackup_encrypt_with_key
      from the list of disabled tests because the problem with it has already
      been fixed.
      e95cdc45
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-24971 InnoDB access freed virtual column after rollback of secondary index · cf2c6b7f
      Thirunarayanan Balathandayuthapani authored
      Problem:
      ========
       InnoDB fails to clean the index stub if it fails to add the
      virtual index which contains new virtual column. But it clears
      the newly virtual column from index in clear_added_indexes()
      during inplace_alter_table. On commit, InnoDB evicts and
      reload the table. In case of rollback, it doesn't happen.
      InnoDB clears the ABORTED index while opening the table
      or doing the DDL. In the mean time, InnoDB can access
      the dropped virtual index columns while creating prebuilt
      or rollback of concurrent DML.
      
      Solution:
      ==========
      (1) InnoDB should maintain newly added virtual column while
      rollbacking the newly added virtual index.
      (2) InnoDB must not defer the index removal
      if the alter table is executed with LOCK=EXCLUSIVE.
      (3) For LOCK=SHARED, InnoDB should check whether the table
      has any other transaction lock other than alter transaction
      before deferring the index stub.
      
      Replaced has_new_v_col with dict_add_vcol_info in dict_index_t to
      indicate whether the index has any new virtual column.
      
      dict_index_t::has_new_v_col(): Returns whether the index has
      newly added virtual column, it doesn't say which columns are
      newly added virtual column
      
      ha_innobase_inplace_ctx::is_new_vcol(): Return whether the
      given column is added as a part of the current alter.
      
      ha_innobase_inplace_ctx::clean_new_vcol_index(): Copy the newly
      added virtual column to new_vcol_info in dict_index_t. Replace
      the column in the index fields with virtual column stored
      in new_vcol_info.
      
      dict_index_t::assign_new_v_col(): Store the number of virtual
      column added in index as a part of alter table.
      
      dict_index_t::get_n_new_vcol(): Get the number of newly added
      virtual column
      
      dict_index_t::assign_drop_v_col(): Allocate the memory for
      adding new virtual column in new_vcol_info.
      
      dict_index_t::add_drop_v_col(): Add the newly added virtual
      column in new_vcol_info.
      
      dict_table_t::has_lock_for_other_trx(): Whether the table has
      any other transaction lock than given transaction.
      
      row_merge_drop_indexes(): Add parameter alter_trx and check
      whether the table has any other lock than alter transaction.
      cf2c6b7f