1. 07 Jun, 2021 2 commits
    • Marko Mäkelä's avatar
      MDEV-25783: Change buffer records are lost under heavy load · 310dff5d
      Marko Mäkelä authored
      ibuf_read_merge_pages(): Disable some code that was added in MDEV-20394
      in order to avoid a server hang if the change buffer is corrupted,
      presumably due to the race condition in recovery that was later fixed in
      MDEV-24449. The code will still be available in debug builds when
      the command line option --debug=d,ibuf_merge_corruption is specified.
      
      Due to MDEV-19514, the impact of this code is much worse starting
      with the 10.5 series. In older versions, the code was only enabled
      during a shutdown with innodb_fast_shutdown=0, but in 10.5 it was
      active during the normal operation of the server.
      310dff5d
    • Monty's avatar
      Make maria.repair more resiliant for different failures · f456e716
      Monty authored
      This is because on different compilation and server configurations the
      memory usage is different and the query can get killed in different places
      with different error messages as a result.
      
      Reviewer: None (trival change)
      f456e716
  2. 06 Jun, 2021 1 commit
    • Otto Kekäläinen's avatar
      Deb: Misc cleanup and autobake-deb.sh and Salsa-CI fixes · d4a6e3a6
      Otto Kekäläinen authored
      * Clean up autobake-deb.sh
      
        - No need to define any TokuDB rules, there is no such package
        - No need to define RocksDB arch, it already has "Architecture:" line
        - No need to dh-systemd backwards compat stanza, neither Debian Jessie
          nor Ubuntu Xenial has any new MariaDB 10.5 releases anymore
        - Minor spelling fixes
      
      * Ensure dch runs non-interactively so builds pass with new dch version
      
        A recent version of dch (available in Ubuntu Hirsute and Debian Bullseye)
        had a change in behaviour that it started prompting if the DEBEMAIL or
        EMAIL variable as unset, asking for confirmation. We can't have anything
        interactive in our build scripts, so prevent this prompt by giving
        --controlmaint to the command, so it always uses the name and email from
        the debian/control file and does not prompt anything.
      
        The command-line argument has been around for a long time, so it is safe
        to use on all Debian/Ubuntu builds we have.
      
        See https://manpages.debian.org/jessie/devscripts/dch.1.en.html
      
        Since MariaDB 10.5 is the oldest release we still release for Ubuntu Hisute
        and Debian Bullseye, merge this on 10.5 and from there merge up to latest.
        No need to consider 10.2, 10.3 and 10.4 as those will not be released for
        Ubuntu Bullseye or Ubuntu Hirsute.
      
      * Minor Salsa-CI cleanup
      
        - Fix spelling (synced from downstream Debian)
      
      * Many minor spelling fixes (synced from downstream Debian)
      d4a6e3a6
  3. 04 Jun, 2021 4 commits
  4. 03 Jun, 2021 2 commits
    • Igor Babaev's avatar
      MDEV-25714 Join using derived with aggregation returns incorrect results · 0b797130
      Igor Babaev authored
      If a join query uses a derived table (view / CTE) with GROUP BY clause then
      the execution plan for such join may employ split optimization. When this
      optimization is employed the derived table is not materialized. Rather only
      some partitions of the derived table are subject to grouping. Split
      optimization can be applied only if:
      - there are some indexes over the tables used in the join specifying the
        derived table whose prefixes partially cover the field items used in the
        GROUP BY list (such indexes are called splitting indexes)
      - the WHERE condition of the join query contains conjunctive equalities
        between columns of the derived table that comprise major parts of
        splitting indexes and columns of the other join tables.
      
      When the optimizer evaluates extending of a partial join by the rows of the
      derived table it always considers a possibility of using split optimization.
      Different splitting indexes can be used depending on the extended partial
      join. At some rare conditions, for example, when there is a non-splitting
      covering index for a table joined in the join specifying the derived table
      usage of a splitting index to produce rows needed for grouping may be still
      less beneficial than usage of such covering index without any splitting
      technique. The function JOIN_TAB::choose_best_splitting() must take this
      into account.
      
      Approved by Oleksandr Byelkin <sanja@mariadb.com>
      0b797130
    • Igor Babaev's avatar
      MDEV-25714 Join using derived with aggregation returns incorrect results · 663bc849
      Igor Babaev authored
      If a join query uses a derived table (view / CTE) with GROUP BY clause then
      the execution plan for such join may employ split optimization. When this
      optimization is employed the derived table is not materialized. Rather only
      some partitions of the derived table are subject to grouping. Split
      optimization can be applied only if:
      - there are some indexes over the tables used in the join specifying the
        derived table whose prefixes partially cover the field items used in the
        GROUP BY list (such indexes are called splitting indexes)
      - the WHERE condition of the join query contains conjunctive equalities
        between columns of the derived table that comprise major parts of
        splitting indexes and columns of the other join tables.
      
      When the optimizer evaluates extending of a partial join by the rows of the
      derived table it always considers a possibility of using split optimization.
      Different splitting indexes can be used depending on the extended partial
      join. At some rare conditions, for example, when there is a non-splitting
      covering index for a table joined in the join specifying the derived table
      usage of a splitting index to produce rows needed for grouping may be still
      less beneficial than usage of such covering index without any splitting
      technique. The function JOIN_TAB::choose_best_splitting() must take this
      into account.
      
      Approved by Oleksandr Byelkin <sanja@mariadb.com>
      663bc849
  5. 02 Jun, 2021 7 commits
  6. 01 Jun, 2021 8 commits
  7. 31 May, 2021 5 commits
    • Julius Goryavsky's avatar
      MDEV-25818: RSYNC SST failed due to busy port · 2fb44078
      Julius Goryavsky authored
      This commit reduces the likelihood of getting a busy port on
      quick restarts with rsync SST (problem MDEV-25818) and fixes
      a number of other flaws in SST scripts, adds new functionality,
      and also synchronizes the xtrabackup-v2 script with the
      mariabackup script (the latter applies only to the 10.2 branch):
      
       1) SST via rsync: rsync and stunnel does not always get the right
          time to complete by correctly handling SIGTERM. These utilities
          are now given more time to complete normally (via normal SIGTERM
          processing) before we move on to using "kill -9";
       2) SST via rsync: attempts to terminate an rsync or stunnel process
          (via "kill" utility) are only made if it did not terminated on
          its own;
       3) SST via rsync: if a combination of stunnel and rsync is used,
          then we need to wait for both utilities to finish or stop, not
          just one of them;
       4) The config file and pid file for stunnel are now deleted after
          successful completion of SST on the donor node;
       5) The configs and pid files from rsync and stunnel should not be
          deleted unless these utilities succeed (or are sucessfully
          terminated) on the joiner node;
       6) The configs and pid files now excluded from transfer via rsync;
       7) Spaces in paths are now valid for config files as well (when
          used with SST via rsync or mariabackup / xtrabackup[-v2]);
       8) SST via mariabackup: added preliminary verification of keys and
          certificates that are used when establishing a connection using
          SSL (to avoid long timeouts and improve diagnostics) - by analogy
          with how it is done for the xtrabackup-v2 (plus check for CA file),
          while that check is skipped if the user does not have openssl
          installed (or does not have diff utility);
       9) Added backup-threads=<n> configuration option which adds
          "--parallel=<n>" for mariabackup / xtrabackup at backup and
          move-back stages;
      10) Added encrypt-threads and encrypt-chunk-size configuration
          options for xbcrypt management (when xbcrypt is used);
      11) Small optimization: checking the socat version and adding
          a file with parameters for 2048-bit Diffie-Hellman (if necessary)
          is done only if the user has not specified "dhparam=" in the
          "sockopt" option value;
      12) SST via rsync now supports "backup-threads" configuration option
          (in server-related sections or in the "[sst]");
      13) Determining the number of available processors is now supported
          for FreeBSD + mariabackup/xtrabackup: before that we might have
          problems with "--compact" (rebuild indexes) or qpress on FreeBSD;
      14) The check_pid() function should not raise an error state in
          the rare cases when the pid file was created, but it is empty,
          or if it is deleted right during the check, or when zero is read
          from the pid file;
      15) Iproved templates that are used to check if a requested socket
          is "listening" when using the ss utility;
      16) Shortened some other templates for socket state utilities;
      17) Temporary files created by mariabackup / xtrabackup are moved
          to a separate subdirectory inside tmpdir (so they don't get
          mixed with other temporary files, which can make debugging
          more difficult);
      18) 10.2 only: the script for SST via xtrabackup-v2 has been brought
          in full compliance with all the bugfixes made for mariabackup (as
          it previously contained many flaws compared to the updated script
          for mariabackup).
      2fb44078
    • Marko Mäkelä's avatar
      MDEV-25745: Not applying INSERT_REUSE_REDUNDANT · 139333a6
      Marko Mäkelä authored
      page_apply_insert_redundant(): Correct a condition that would
      occasionally fail when recovering changes for the change buffer tree
      (where extra_size and data_size can vary wildly).
      
      This was broken in commit 138cbec5
      (MDEV-21724).
      139333a6
    • Marko Mäkelä's avatar
    • Marko Mäkelä's avatar
      Cleanup: deduplicate code · 601eb411
      Marko Mäkelä authored
      601eb411
    • Vladislav Vaintroub's avatar
  8. 30 May, 2021 1 commit
    • Dmitry Shulga's avatar
      MDEV-25576: The statement EXPLAIN running as regular statement and as prepared... · 91bde0fb
      Dmitry Shulga authored
      MDEV-25576: The statement EXPLAIN running as regular statement and as prepared statement produces different results for UPDATE with subquery
      
      Both EXPLAIN and EXPLAIN EXTENDED statements produce different results set
      in case it is run in normal way and in PS mode for the statements
      UPDATE/DELETE with subquery.
      
      The use case below reproduces the issue:
      MariaDB [test]> CREATE TABLE t1 (c1 INT KEY) ENGINE=MyISAM;
      Query OK, 0 rows affected (0,128 sec)
      
      MariaDB [test]> CREATE TABLE t2 (c2 INT) ENGINE=MyISAM;
      Query OK, 0 rows affected (0,023 sec)
      
      MariaDB [test]> CREATE TABLE t3 (c3 INT) ENGINE=MyISAM;
      Query OK, 0 rows affected (0,021 sec)
      
      MariaDB [test]> EXPLAIN EXTENDED UPDATE t3 SET c3 =
          -> ( SELECT COUNT(d1.c1) FROM ( SELECT a11.c1 FROM t1 AS a11
          -> STRAIGHT_JOIN t2 AS a21 ON a21.c2 = a11.c1 JOIN t1 AS a12
          -> ON a12.c1 = a11.c1 ) d1 );
      +------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
      | id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra                          |
      +------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
      |    1 | PRIMARY     | t3    | ALL  | NULL          | NULL | NULL    | NULL |    0 |   100.00 |                                |
      |    2 | SUBQUERY    | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL |     NULL | Impossible WHERE noticed after reading const tables
      +------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
      2 rows in set (0,002 sec)
      
      MariaDB [test]> PREPARE stmt FROM
          -> EXPLAIN EXTENDED UPDATE t3 SET c3 =
          -> ( SELECT COUNT(d1.c1) FROM ( SELECT a11.c1 FROM t1 AS a11
          -> STRAIGHT_JOIN t2 AS a21 ON a21.c2 = a11.c1 JOIN t1 AS a12
          -> ON a12.c1 = a11.c1 ) d1 );
      Query OK, 0 rows affected (0,000 sec)
      Statement prepared
      
      MariaDB [test]>  EXECUTE stmt;
      +------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
      | id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra                          |
      +------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
      |    1 | PRIMARY     | t3    | ALL  | NULL          | NULL | NULL    | NULL |    0 |   100.00 |                                |
      |    2 | SUBQUERY    | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL |     NULL | no matching row in const table |
      +------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
      2 rows in set (0,000 sec)
      
      The reason by that different result sets are produced is that on execution
      of the statement 'EXECUTE stmt' the flag SELECT_DESCRIBE not set
      in the data member SELECT_LEX::options for instances of SELECT_LEX that
      correspond to subqueries used in the UPDTAE/DELETE statements.
      
      Initially, these flags were set on parsing the statement
        PREPARE stmt FROM "EXPLAIN EXTENDED UPDATE t3 SET ..."
      but latter they were reset before starting real execution of
      the parsed query during handling the statement 'EXECUTE stmt';
      
      So, to fix the issue the functions mysql_update()/mysql_delete()
      have been modified to set the flag SELECT_DESCRIBE forcibly
      in the data member SELECT_LEX::options for the primary SELECT_LEX
      of the UPDATE/DELETE statement.
      91bde0fb
  9. 29 May, 2021 1 commit
  10. 28 May, 2021 1 commit
  11. 27 May, 2021 6 commits
  12. 26 May, 2021 2 commits