1. 09 Sep, 2020 2 commits
    • Marko Mäkelä's avatar
      MDEV-23456 fixup: Fix mtr_t::get_fix_count() · c26eae0c
      Marko Mäkelä authored
      Before commit 05fa4558 (MDEV-22110)
      we have slot->type == MTR_MEMO_MODIFY that are unrelated to
      incrementing the buffer-fix count.
      
      FindBlock::operator(): In debug builds, skip MTR_MEMO_MODIFY entries.
      
      Also, simplify the code a little.
      
      This fixes an infinite loop in the tests
      innodb.innodb_defragment and innodb.innodb_wl6326_big.
      c26eae0c
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-23456 fil_space_crypt_t::write_page0() is accessing an uninitialized page · b1009ae5
      Thirunarayanan Balathandayuthapani authored
      buf_page_create() is invoked when page is initialized. So that
      previous contents of the page ignored. In few cases, it calls
      buf_page_get_gen() is called to fetch the page from buffer pool.
      It should take x-latch on the page. If other thread uses the block
      or block io state is different from BUF_IO_NONE then release the
      mutex and check the state and buffer fix count again. For compressed
      page, use the existing free block from LRU list to create new page.
      Retry to fetch the compressed page if it is in flush list
      
      fseg_create(), fseg_create_general(): Introduce block as a parameter
      where segment header is placed. It is used to avoid repetitive
      x-latch on the same page
      
      Change the assert to check whether the page has SX latch and
      X latch in all callee function of buf_page_create()
      
      mtr_t::get_fix_count(): Get the buffer fix count of the given
      block added by the mtr
      
      FindBlock is added to find the buffer fix count of the given
      block acquired by the mini-transaction
      b1009ae5
  2. 07 Sep, 2020 2 commits
    • Marko Mäkelä's avatar
      MDEV-22924 Corruption in MVCC read via secondary index · f99cace7
      Marko Mäkelä authored
      An unsafe optimization was introduced by
      commit 2347ffd8 (MDEV-20301)
      which is based on
      mysql/mysql-server@3f3136188f1bd383f77f97823cf6ebd72d5e4d7e or
      mysql/mysql-server@647a3814a91c3d3bffc70ddff5513398e3f37bd4
      in MySQL 8.0.12 or MySQL 8.0.13
      (which in turn is based on the contribution in MySQL Bug #84958).
      
      Row_sel_get_clust_rec_for_mysql::operator(): In addition to checking
      that the pointer to the record matches, also check the latest
      modification of the page (FIL_PAGE_LSN) as well as the page identifier.
      Only if all three match, it is safe to reuse cached_old_vers.
      
      Row_sel_get_clust_rec_for_mysql::check_eq(): Assert that the PRIMARY KEY
      of the cached old version of the record corresponds to the latest version.
      
      We got a test case where CHECK TABLE, UPDATE and purge would be
      hammering on the same table (with only 6 rows) and a pointer that
      was originally pointing to a record pk=2 would match a cached_clust_rec
      that was pointing to a record pk=1. In the diagnosed `rr replay` trace,
      we would wrongly return an old cached version of the pk=1 record,
      instead of retrieving the correct version of the pk=2 record. Because
      of this, CHECK TABLE would fail to count one of the records in a
      secondary index, and report failure.
      
      This bug appears to affect MVCC reads via secondary indexes only.
      The purge of history in secondary indexes uses a different code path,
      and so do checks for implicit record locks.
      f99cace7
    • Kentoku SHIBA's avatar
      MDEV-7098 spider/bg.spider_fixes failed in buildbot with safe_mutex: Trying to... · 9dedba16
      Kentoku SHIBA authored
      MDEV-7098 spider/bg.spider_fixes failed in buildbot with safe_mutex: Trying to unlock mutex conn->mta_conn_mutex that wasn't locked at storage/spider/spd_db_conn.cc, line 671
      9dedba16
  3. 04 Sep, 2020 2 commits
  4. 03 Sep, 2020 4 commits
    • Sergei Petrunia's avatar
      MDEV-23661: RocksDB produces "missing initializer for member" warnings · d63fcbc2
      Sergei Petrunia authored
      Add -Wno-missing-field-initializers for MyRocks and gcc version below 5.0
      d63fcbc2
    • Marko Mäkelä's avatar
      Merge 10.1 into 10.2 · 2a93e632
      Marko Mäkelä authored
      2a93e632
    • Marko Mäkelä's avatar
      MDEV-22387: Do not pass null pointer to some memcpy() · 94a520dd
      Marko Mäkelä authored
      Passing a null pointer to a nonnull argument is not only undefined
      behaviour, but it also grants the compiler the permission to optimize
      away further checks whether the pointer is null. GCC -O2 at least
      starting with version 8 may do that, potentially causing SIGSEGV.
      
      These problems were caught in a WITH_UBSAN=ON build with the
      Bug#7024 test in main.view.
      94a520dd
    • Marko Mäkelä's avatar
      MDEV-7110 follow-up fix: Do not pass NULL as nonnull parameter · a256070e
      Marko Mäkelä authored
      Passing a null pointer to the "%s" argument of a printf-like
      function is undefined behaviour. In the GNU libc implementation
      of the printf() family of functions, it happens to work.
      
      GCC 10.2.0 would diagnose this with -Wformat-overflow -Og.
      In -fsanitize=undefined (WITH_UBSAN=ON) builds, a runtime error
      would be generated. In some other builds, GCC 8 or later might infer
      that the parameter is nonnull and optimize away further checks whether
      the parameter is null, leading to SIGSEGV.
      a256070e
  5. 02 Sep, 2020 4 commits
  6. 01 Sep, 2020 6 commits
  7. 31 Aug, 2020 3 commits
    • Andrei Elkin's avatar
      MDEV-16372 ER_BASE64_DECODE_ERROR upon replaying binary log via mysqlbinlog --verbose · feac078f
      Andrei Elkin authored
      (This commit is exclusively for 10.1 branch, do not merge it to upper ones)
      
      In case of a pattern of non-STMT_END-marked Rows-log-event (A) followed by
      a STMT_END marked one (B) mysqlbinlog mixes up the base64 encoded rows events
      with their pseudo sql representation produced by the verbose option:
            BINLOG '
              base64 encoded data for A
              ### verbose section for A
              base64 encoded data for B
              ### verbose section for B
            '/*!*/;
      In effect the produced BINLOG '...' query is not valid and is rejected with the error.
      Examples of this way malformed BINLOG could have been found in binlog_row_annotate.result
      that gets corrected with the patch.
      
      The issue is fixed with introduction an auxiliary IO_CACHE to hold on the verbose
      comments until the terminal STMT_END event is found. The new cache is emptied
      out after two pre-existing ones are done at that time.
      The correctly produced output now for the above case is as the following:
            BINLOG '
              base64 encoded data for A
              base64 encoded data for B
            '/*!*/;
              ### verbose section for A
              ### verbose section for B
      
      Thanks to Alexey Midenkov for the problem recognition and attempt to tackle,
      Venkatesh Duggirala who produced a patch for the upstream whose
      idea is exploited here, as well as to MDEV-23077 reporter LukeXwang who
      also contributed a piece of a patch aiming at this issue.
      
      Extra: mysqlbinlog_row_minimal refined to not produce mutable numeric values into the result file.
      feac078f
    • Andrei Elkin's avatar
      MDEV-16372 ER_BASE64_DECODE_ERROR upon replaying binary log via mysqlbinlog --verbose · 6112a0f9
      Andrei Elkin authored
      (This commit is exclusively for 10.2 branch. Do not merge it to 10.3)
      
      In case of a pattern of non-STMT_END-marked Rows-log-event (A) followed by
      a STMT_END marked one (B) mysqlbinlog mixes up the base64 encoded rows events
      with their pseudo sql representation produced by the verbose option:
            BINLOG '
              base64 encoded data for A
              ### verbose section for A
              base64 encoded data for B
              ### verbose section for B
            '/*!*/;
      In effect the produced BINLOG '...' query is not valid and is rejected with the error.
      Examples of this way malformed BINLOG could have been found in binlog_row_annotate.result
      that gets corrected with the patch.
      
      The issue is fixed with introduction an auxiliary IO_CACHE to hold on the verbose
      comments until the terminal STMT_END event is found. The new cache is emptied
      out after two pre-existing ones are done at that time.
      The correctly produced output now for the above case is as the following:
            BINLOG '
              base64 encoded data for A
              base64 encoded data for B
            '/*!*/;
              ### verbose section for A
              ### verbose section for B
      
      Thanks to Alexey Midenkov for the problem recognition and attempt to tackle,
      and to Venkatesh Duggirala who produced a patch for the upstream whose
      idea is exploited here, as well as to MDEV-23077 reporter LukeXwang who
      also contributed a piece of a patch aiming at this issue.
      6112a0f9
    • Eugene Kosov's avatar
      fix clang build · 9bb17ecf
      Eugene Kosov authored
      FAILED: sql/CMakeFiles/sql.dir/sql_test.cc.o
      /home/kevgs/bin/clang++ -DHAVE_CONFIG_H -DHAVE_EVENT_SCHEDULER -DHAVE_POOL_OF_THREADS -DMYSQL_SERVER -D_FILE_OFFSET_BITS=64 -Iinclude -I../include -I../sql -Ipcre -I../pcre -I../zlib -Izlib -I../extra/yassl/include -I../extra/yassl/taocrypt/include -Isql -I../wsrep -O2 -fdiagnostics-color=always -fno-omit-frame-pointer -gsplit-dwarf -march=native -mtune=native -fPIC -fno-rtti -g -DENABLED_DEBUG_SYNC -ggdb3 -DSAFE_MUTEX -Wall -Wdeclaration-after-statement -Wextra -Wformat-security -Wno-init-self -Wno-null-conversion -Wno-unused-parameter -Wno-unused-private-field -Woverloaded-virtual -Wvla -Wwrite-strings -Werror   -DHAVE_YASSL -DYASSL_PREFIX -DHAVE_OPENSSL -DMULTI_THREADED -MD -MT sql/CMakeFiles/sql.dir/sql_test.cc.o -MF sql/CMakeFiles/sql.dir/sql_test.cc.o.d -o sql/CMakeFiles/sql.dir/sql_test.cc.o -c ../sql/sql_test.cc
      ../sql/sql_test.cc:390:20: error: '::' and '*' tokens forming pointer to member type are separated by whitespace [-Werror,-Wcompound-token-split-by-space]
      Item* (List<Item>:: *dbug_list_item_elem_ptr)(int)= &List<Item>::elem;
                       ~~^~
      ../sql/sql_test.cc:391:32: error: '::' and '*' tokens forming pointer to member type are separated by whitespace [-Werror,-Wcompound-token-split-by-space]
      Item_equal* (List<Item_equal>:: *dbug_list_item_equal_elem_ptr)(int)=
                                   ~~^~
      ../sql/sql_test.cc:393:32: error: '::' and '*' tokens forming pointer to member type are separated by whitespace [-Werror,-Wcompound-token-split-by-space]
      TABLE_LIST* (List<TABLE_LIST>:: *dbug_list_table_list_elem_ptr)(int) =
                                   ~~^~
      3 errors generated.
      9bb17ecf
  8. 28 Aug, 2020 3 commits
    • Jan Lindström's avatar
      MDEV-21578 : CREATE OR REPLACE TRIGGER in Galera cluster not replicating · c710c450
      Jan Lindström authored
      While doing TOI buffer OR REPLACE option was not added to replicated
      string.
      c710c450
    • sjaakola's avatar
      MDEV-23557 Galera heap-buffer-overflow in wsrep_rec_get_foreign_key · df07ea0b
      sjaakola authored
      This commit contains a fix and extended test case for a ASAN failure
      reported during galera.fk mtr testing.
      The reported heap buffer overflow happens in test case where a cascading
      foreign key constraint is defined for a column of varchar type, and
      galera.fk.test has such vulnerable test scenario.
      
      Troubleshoting revealed that erlier fix for MDEV-19660 has made a fix
      for cascading delete handling to append wsrep keys from pcur->old_rec,
      in row_ins_foreign_check_on_constraint(). And, the ASAN failuer comes from
      later scanning of this old_rec reference.
      
      The fix in this commit, moves the call for wsrep_append_foreign_key() to happen
      somewhat earlier, and inside ongoing mtr, and using clust_rec which is set
      earlier in the same mtr for both update and delete cascade operations.
      for wsrep key populating, it does not matter when the keys are populated,
      all keys just have to be appended before wsrep transaction replicates.
      
      Note that I also tried similar fix for earlier wsrep key append, but using
      the old implementation with pcur->old_rec (instead of clust_rec), and same
      ASAN failure was reported. So it appears that pcur->old_rec is not properly
      set, to be used for wsrep key appending.
      
      galera.galera_fk_cascade_delete test has been extended by two new test scenarios:
      * FK cascade on varchar column.
        This test case reproduces same scenario as galera.fk, and this test scenario
        will also trigger ASAN failure with non fixed MariaDB versions.
      * multi-master conflict with FK cascading.
        this scenario causes a conflict between a replicated FK cascading transaction
        and local transaction trying to modify the cascaded child table row.
        Local transaction should be aborted and get deadlock error.
        This test scenario is passing both with old MariaDB version and with this
        commit as well.
      df07ea0b
    • Jan Lindström's avatar
  9. 27 Aug, 2020 9 commits
  10. 26 Aug, 2020 1 commit
  11. 25 Aug, 2020 4 commits
    • Sergei Golubchik's avatar
      MDEV-23569 temporary tables can overwrite existing files · 62d1e3bf
      Sergei Golubchik authored
      for internal temporary tables: don't use realpath(),
      and let them overwrite whatever orphan temp files might've
      left in the tmpdir (see main.error_simulation test).
      
      for user created temporary tables: we have to use realpath(),
      (see 3a726ab6, remember DATA/INDEX DIRECTORY). don't allow
      them to overwrite existing files.
      
      This bug was reported by RACK911 LABS
      62d1e3bf
    • Marko Mäkelä's avatar
      MDEV-23547 InnoDB: Failing assertion: *len in row_upd_ext_fetch · 8cf8ad86
      Marko Mäkelä authored
      This bug was originally repeated on 10.4 after defining a UNIQUE KEY
      on a TEXT column, which is implemented by MDEV-371 by creating the
      index on a hidden virtual column.
      
      While row_vers_vc_matches_cluster() is executing in a purge thread
      to find out if an index entry may be removed in a secondary index
      that comprises a virtual column, another purge thread may process
      the undo log record that this check is interested in, and write
      a null BLOB pointer in that record. This would trip the assertion.
      
      To prevent this from occurring, we must propagate the 'missing BLOB'
      error up the call stack.
      
      row_upd_ext_fetch(): Return NULL when the error occurs.
      
      row_upd_index_replace_new_col_val(): Return whether the previous
      version was built successfully.
      
      row_upd_index_replace_new_col_vals_index_pos(): Check the error
      result. Yes, we would intentionally crash on this error if it
      occurs outside the purge thread.
      
      row_upd_index_replace_new_col_vals(): Check for the error condition,
      and simplify the logic.
      
      trx_undo_prev_version_build(): Check for the error condition.
      8cf8ad86
    • Marko Mäkelä's avatar
    • Jan Lindström's avatar
      MDEV-23483: Set Galera SST thd as system thread · 0be70a1b
      Jan Lindström authored
      Revert change to MDL and set SST donor thread as a system thread.
      Joiner thread was already a system thread.
      0be70a1b