1. 16 Mar, 2020 4 commits
  2. 15 Mar, 2020 7 commits
  3. 14 Mar, 2020 9 commits
    • Andrei Elkin's avatar
      MDEV-742 XA PREPAREd transaction survive disconnect/server restart · c8ae3573
      Andrei Elkin authored
      Lifted long standing limitation to the XA of rolling it back at the
      transaction's
      connection close even if the XA is prepared.
      
      Prepared XA-transaction is made to sustain connection close or server
      restart.
      The patch consists of
      
          - binary logging extension to write prepared XA part of
            transaction signified with
            its XID in a new XA_prepare_log_event. The concusion part -
            with Commit or Rollback decision - is logged separately as
            Query_log_event.
            That is in the binlog the XA consists of two separate group of
            events.
      
            That makes the whole XA possibly interweaving in binlog with
            other XA:s or regular transaction but with no harm to
            replication and data consistency.
      
            Gtid_log_event receives two more flags to identify which of the
            two XA phases of the transaction it represents. With either flag
            set also XID info is added to the event.
      
            When binlog is ON on the server XID::formatID is
            constrained to 4 bytes.
      
          - engines are made aware of the server policy to keep up user
            prepared XA:s so they (Innodb, rocksdb) don't roll them back
            anymore at their disconnect methods.
      
          - slave applier is refined to cope with two phase logged XA:s
            including parallel modes of execution.
      
      This patch does not address crash-safe logging of the new events which
      is being addressed by MDEV-21469.
      
      CORNER CASES: read-only, pure myisam, binlog-*, @@skip_log_bin, etc
      
      Are addressed along the following policies.
      1. The read-only at reconnect marks XID to fail for future
         completion with ER_XA_RBROLLBACK.
      
      2. binlog-* filtered XA when it changes engine data is regarded as
         loggable even when nothing got cached for binlog.  An empty
         XA-prepare group is recorded. Consequent Commit-or-Rollback
         succeeds in the Engine(s) as well as recorded into binlog.
      
      3. The same applies to the non-transactional engine XA.
      
      4. @@skip_log_bin=OFF does not record anything at XA-prepare
         (obviously), but the completion event is recorded into binlog to
         admit inconsistency with slave.
      
      The following actions are taken by the patch.
      
      At XA-prepare:
         when empty binlog cache - don't do anything to binlog if RO,
         otherwise write empty XA_prepare (assert(binlog-filter case)).
      
      At Disconnect:
         when Prepared && RO (=> no binlogging was done)
           set Xid_cache_element::error := ER_XA_RBROLLBACK
           *keep* XID in the cache, and rollback the transaction.
      
      At XA-"complete":
         Discover the error, if any don't binlog the "complete",
         return the error to the user.
      
      Kudos
      -----
      Alexey Botchkov took to drive this work initially.
      Sergei Golubchik, Sergei Petrunja, Marko Mäkelä provided a number of
      good recommendations.
      Sergei Voitovich made a magnificent review and improvements to the code.
      They all deserve a bunch of thanks for making this work done!
      c8ae3573
    • Monty's avatar
      5754ea2e
    • Sergei Golubchik's avatar
      41cba6c9
    • Sergei Golubchik's avatar
      Merge branch 'github/10.5' into 10.5 · 91d1588d
      Sergei Golubchik authored
      91d1588d
    • Sergei Golubchik's avatar
    • Sergei Golubchik's avatar
      fix a race condition in the main.grant_kill test · 92c05a39
      Sergei Golubchik authored
      also remove redundant --connection commands
      92c05a39
    • Sergei Golubchik's avatar
      df25d67d
    • Sergei Golubchik's avatar
      MDEV-21222 mariabackup.incremental_backup failed with memory allocation failure · 57de4def
      Sergei Golubchik authored
      mariabackup tries to allocate a buffer of page_size*page_size/4 size.
      for 64k page it means 1Gb, which doesn't work very well on 32-bit builders.
      
      Skip the 64k page test on 32bit.
      57de4def
    • Sergei Golubchik's avatar
      mtr sets MYSQLTEST_REAL_VARDIR when MYSQLTEST_VARDIR is a symlink · 422ba205
      Sergei Golubchik authored
      this fixes main.mysqld--defaults-file failure with --mem
      422ba205
  4. 13 Mar, 2020 9 commits
    • Eugene Kosov's avatar
      cleanup redo log · 774fe896
      Eugene Kosov authored
      774fe896
    • Sergey Vojtovich's avatar
      Pre-MDEV-742 InnoDB fixes · 78cc9c9e
      Sergey Vojtovich authored
      1. Refactored innobase_close_connection(). Transaction must've already
      been rolled back by this time. We should expect only transactions in the
      PREPARED state when MDEV-742 is done.
      
      2. Added missing put_pins() to trx_disconnect_prepared(). Missing
      put_pins() wasn't a problem because trx_disconnect_prepared() is a dead
      code. But it will get revived in the main MDEV-742 patch.
      
      3. Fixed missing reset of trx->mysql_log_file_name when RW transaction
      didn't emit any log records (zero-modification RW). The problem was
      detected by ASAN when disconnected XA transaction was trying to make
      use of inherited mysql_log_file_name pointing into binlog data of
      detached THD.
      
      This missing reset also had user-visible side effect, when
      trx_sys_print_mysql_binlog_offset() would report binlog position
      not of the most recently committed transaction.
      
      One of possible scenarios that is expected to misbehave is as following:
      
        thr1> CREATE TABLE t1(a INT) ENGINE=InnoDB;
        thr1> INSERT INTO t1 VALUES(1);
        thr1> BEGIN;
        thr1> UPDATE t1 SET a=1
        thr1> COMMIT; -- zero-modification, misses to reset mysql_log_file_name
      
        thr2> BEGIN;
        thr2> INSERT INTO t1 VALUES(2);
        thr2> COMMIT;
      
        thr1> BEGIN;
        thr1> do-some-real-changes;
        thr1> ROLLBACK; -- will store binlog pos from previous COMMIT in thr1?
      
      In this case it means if binlog is replayed from position reported by
      trx_sys_print_mysql_binlog_offset(), t1 will end up with two records
      containing '2'.
      
      Part of
      MDEV-742 - XA PREPAREd transaction survive disconnect/server restart
      78cc9c9e
    • Sergey Vojtovich's avatar
      Extended debug_sync_control life time · 662d8a86
      Sergey Vojtovich authored
      It now lives from THD constructor to THD destructor. Reset before THD is
      released to a cache. Change user doesn't reset debug_sync_control anymore.
      
      Needed to be able to make use of DEBUG_SYNC() at later stages like
      ha_close_connection().
      662d8a86
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-21903 FTS optimize thread aborts during shutdown · c5868644
      Thirunarayanan Balathandayuthapani authored
      - This issue was caused by 5e62b6a5.
      fts_optimize_callback() should free fts_optimize_wq and make it as NULL
      when it receives FTS_MSG_STOP message. So that subsequent
      fts_optimize_callback() doesn't fail with segmentation fault.
      c5868644
    • Marko Mäkelä's avatar
      e74c1c9e
    • Marko Mäkelä's avatar
      MDEV-15058: Remove buf_pool_get_dirty_pages_count() · fbe662a5
      Marko Mäkelä authored
      Starting with commit 1a6f708e
      the function buf_pool_get_dirty_pages_count() is only used
      in a debug check. It was dead code for non-debug builds.
      
      buf_flush_dirty_pages(): Perform the debug check inline,
      and replace the assertion
      	ut_ad(first || buf_pool_get_dirty_pages_count(id) == 0);
      with another one that is executed while holding the mutexes:
      	ut_ad(id != bpage->id.space());
      fbe662a5
    • Alexander Barkov's avatar
      MDEV-21920 binlog_encryption.rpl_gtid_basic test failure with --ps · 2bbcf9a1
      Alexander Barkov authored
      Updating libmariadb to use 3be5897c3346639fa6d7195480d93108798c4917,
      which fixed the problem.
      2bbcf9a1
    • Marko Mäkelä's avatar
      MDEV-20632: Fix -Wmaybe-uninitialized · f48718be
      Marko Mäkelä authored
      Create_tmp_table::add_fields(): Initialize uneven_delta= 0
      to suppress the warning from GCC 9.2.1 and 10.0.1,
      and consistently indent the code with spaces.
      f48718be
    • Marko Mäkelä's avatar
      MDEV-21907: Fix or disable -Wconversion on GCC 5.3.0 i386 · c57b2079
      Marko Mäkelä authored
      Fix or disable those -Wconversion that were missed by GCC 5.4.0
      targeting AMD64.
      c57b2079
  5. 12 Mar, 2020 6 commits
    • Marko Mäkelä's avatar
      MDEV-21907: InnoDB: Enable -Wconversion on clang and GCC · f2245252
      Marko Mäkelä authored
      The -Wconversion in GCC seems to be stricter than in clang.
      GCC at least since version 4.4.7 issues truncation warnings for
      assignments to bitfields, while clang 10 appears to only issue
      warnings when the sizes in bytes rounded to the nearest integer
      powers of 2 are different.
      
      Before GCC 10.0.0, -Wconversion required more casts and would not
      allow some operations, such as x<<=1 or x+=1 on a data type that
      is narrower than int.
      
      GCC 5 (but not GCC 4, GCC 6, or any later version) is complaining
      about x|=y even when x and y are compatible types that are narrower
      than int.  Hence, we must rewrite some x|=y as
      x=static_cast<byte>(x|y) or similar, or we must disable -Wconversion.
      
      In GCC 6 and later, the warning for assigning wider to bitfields
      that are narrower than 8, 16, or 32 bits can be suppressed by
      applying a bitwise & with the exact bitmask of the bitfield.
      For older GCC, we must disable -Wconversion for GCC 4 or 5 in such
      cases.
      
      The bitwise negation operator appears to promote short integers
      to a wider type, and hence we must add explicit truncation casts
      around them. Microsoft Visual C does not allow a static_cast to
      truncate a constant, such as static_cast<byte>(1) truncating int.
      Hence, we will use the constructor-style cast byte(~1) for such cases.
      
      This has been tested at least with GCC 4.8.5, 5.4.0, 7.4.0, 9.2.1, 10.0.0,
      clang 9.0.1, 10.0.0, and MSVC 14.22.27905 (Microsoft Visual Studio 2019)
      on 64-bit and 32-bit targets (IA-32, AMD64, POWER 8, POWER 9, ARMv8).
      f2245252
    • Marko Mäkelä's avatar
      MDEV-21907: Fix some -Wconversion outside InnoDB · d82ac8d3
      Marko Mäkelä authored
      Some .c and .cc files are compiled as part of Mariabackup.
      Enabling -Wconversion for InnoDB would also enable it for
      Mariabackup. The .h files are being included during
      InnoDB or Mariabackup compilation.
      
      Notably, GCC 5 (but not GCC 4 or 6 or later versions)
      would report -Wconversion for x|=y when the type is
      unsigned char. So, we will either write x=(uchar)(x|y)
      or disable the -Wconversion warning for GCC 5.
      
      bitmap_set_bit(), bitmap_flip_bit(), bitmap_clear_bit(), bitmap_is_set():
      Always implement as inline functions.
      d82ac8d3
    • Marko Mäkelä's avatar
    • Marko Mäkelä's avatar
      MDEV-21924 Clean up InnoDB GIS record comparison · 8be3794b
      Marko Mäkelä authored
      The extension of the record comparison functions for SPATIAL INDEX in
      mysql/mysql-server@b66ad511b61fffe75c58d0a607cdb837c6e6c821
      was suboptimal for multiple reasons:
      
      Some functions used unnecessary temporary variables of the int type,
      instead of the more appropriate size_t, causing type mismatch.
      
      Many functions unnecessarily required rec_get_offsets() to be
      computed, or a parameter for length, although the size of the
      minimum bounding rectangle (MBR) is hard-coded as
      SPDIMS * 2 * sizeof(double), or 32 bytes.
      
      In InnoDB SPATIAL INDEX records, there always is a 32-byte key
      followed by either a 4-byte child page number or the PRIMARY KEY value.
      
      The length parameters were not properly validated.
      The function cmp_geometry_field() was making an incorrect attempt
      at checking that the lengths are at least sizeof(double) (8 bytes),
      even though the function is accessing up to 32 bytes in both MBR.
      
      Functions that are called from only one compilation unit are defined
      in another compilation unit, making the code harder to follow and
      potentially slower to execute.
      
      cmp_dtuple_rec_with_gis(): FIXME: Correct the debug assertion
      and possibly the function TABLE_SHARE::init_from_binary_frm_image()
      or related code, which causes an unexpected length of
      DATA_MBR_LEN + 2 bytes to be passed to this function.
      8be3794b
    • Marko Mäkelä's avatar
      MDEV-13362: Fix -Wset-but-unused · 0d767778
      Marko Mäkelä authored
      0d767778
    • Julius Goryavsky's avatar
      MDEV-13362: implement --require_secure_transport option · 28fabc86
      Julius Goryavsky authored
      Currently, if a user wants to require TLS for every connection made
      over the network, then every user account on the system needs to be
      created with "REQUIRE SSL" or one of the other TLS options.
      
      Implementing a require_secure_transport system varuable (which,
      in particular, can be set using the --require_secure_transport=ON
      command line option) in the MariaDB Server would make it a lot
      easier to require TLS (or other secure transport) system-wide.
      
      This patch implements this new system variable, adds the ability
      to set it with SQL statements, from the command line and from the
      configuration file, and also contains improvements for mtr that allow
      the user to establish non-secure TCP/IP connections (for example,
      to verify the operation of the new option).
      28fabc86
  6. 11 Mar, 2020 5 commits