1. 25 Aug, 2017 3 commits
  2. 24 Aug, 2017 3 commits
    • Kenny John Jacob's avatar
      Update README.md · a544225d
      Kenny John Jacob authored
      Fix minor typo.
      a544225d
    • Marko Mäkelä's avatar
      MDEV-13534 InnoDB STATS_PERSISTENT fails to ignore garbage delete-mark flag on node pointer pages · e7bf8bca
      Marko Mäkelä authored
      This bug was a regression caused by MDEV-12698.
      
      On non-leaf pages, the delete-mark flag in the node pointer records is
      basically garbage. (Delete-marking only makes sense at the leaf level
      anyway. The purpose of the delete-mark is to tell MVCC, locking and purge
      that a leaf-level record does not exist in the READ UNCOMMITTED view,
      but it used to exist.)
      Node pointer records and non-leaf pages are glue that attaches multiple
      leaf pages to an index. This glue is supposed to be transparent to the
      transactional layer.
      
      When a page is split, InnoDB creates a node pointer record out of the
      child page record that the cursor is positioned on. The node pointer record
      for the parent page will be a copy of the child page record, amended with
      the child page number. If the child page record happened to carry the
      delete-mark flag, then the node pointer record would also carry this flag
      (even though the flag makes no sense outside child pages).
      
      (On a related note, for the first node pointer record in the first
      node pointer page of each tree level, if the MIN_REC_FLAG is set,
      the rest of the record contents (except the child page number)
      is basically garbage. From this garbage you could deduce at which point
      the child was originally split.)
      
      page_scan_method_t: Replace with bool, as there are only 2 values.
      
      dict_stats_scan_page(): Replace the parameter scan_method with is_leaf.
      Ignore the bogus (garbage) delete-mark flag if !is_leaf.
      e7bf8bca
    • Igor Babaev's avatar
  3. 23 Aug, 2017 8 commits
    • Sergei Petrunia's avatar
      MDEV-13602: rocksdb.index_merge_rocksdb2 failed in buildbot · ae0759ad
      Sergei Petrunia authored
      - Add include/index_merge*. Upstream has different files than MariaDB,
        use copies theirs, not ours.
      - There was a prblem with running "DDL-like" commands with binlog=ON:
        MariaDB sets binlog_format=STATEMENT for the duration of such command
        to prevent RBR replication from catching (and replicating) updates to
        system tables.
        However, MyRocks tries to prevent any writes to MyRocks tables with
        binlog_format!=ROW.
      - Added exceptions for DDL-type commands (ANALYZE TABLE, OPTIMIZE TABLE)
      - Added special handling for "LOCK TABLE(s) myrocks_table WRITE".
      ae0759ad
    • Marko Mäkelä's avatar
      The test failed once on Buildbot with the result difference: · 06b4b99f
      Marko Mäkelä authored
       # ib_logfile0 expecting FOUND
      -FOUND 3 /public|gossip/ in ib_logfile0
      +FOUND 2 /public|gossip/ in ib_logfile0
      
      The most plausible explanation for this difference
      should be that the redo log payload grew was so big that
      one of the strings (for writing the undo log record,
      clustered index record, and secondary index record)
      was written to ib_logfile1 instead of ib_logfile0.
      
      Let us run the test with --innodb-log-files-in-group=1 so that
      only a single log file will be used.
      06b4b99f
    • Marko Mäkelä's avatar
      Adjust InnoDB debug assertions for Oracle Bug#25551311 aka Bug#23517560 · 81bd81fb
      Marko Mäkelä authored
      The MySQL 5.6.36 merge (commit 0af98182
      in MariaDB Server 10.0.31, 10.1.24, 10.2.7) introduced a change from
      Oracle:
      Bug#25551311 BACKPORT BUG #23517560 REMOVE SPACE_ID RESTRICTION
      FOR UNDO TABLESPACES
      
      Some debug assertions in MariaDB 10.2 were still assuming that the
      InnoDB undo tablespace IDs start from 1. With the above mentioned
      change, the undo tablespace IDs must be contiguous and nonzero.
      81bd81fb
    • Marko Mäkelä's avatar
      MDEV-13167 InnoDB key rotation is not skipping unused pages · 36a97172
      Marko Mäkelä authored
      In key rotation, we must initialize unallocated but previously
      initialized pages, so that if encryption is enabled on a table,
      all clear-text data for the page will eventually be overwritten.
      But we should not rotate keys on pages that were never allocated
      after the data file was created.
      
      According to the latching order rules, after acquiring the
      tablespace latch, no page latches of previously allocated user pages
      may be acquired. So, key rotation should check the page allocation
      status after acquiring the page latch, not before. But, the latching
      order rules also prohibit accessing pages that were not allocated first,
      and then acquiring the tablespace latch. Such behaviour would indeed
      result in a deadlock when running the following tests:
      encryption.innodb_encryption-page-compression
      encryption.innodb-checksum-algorithm
      
      Because the key rotation is accessing potentially unallocated pages, it
      cannot reliably check if these pages were allocated. It can only check
      the page header. If the page number is zero, we can assume that the
      page is unallocated.
      
      fil_crypt_rotate_pages(): Skip pages that are known to be uninitialized.
      
      fil_crypt_rotate_page(): Detect uninitialized pages by FIL_PAGE_OFFSET.
      Page 0 is never encrypted, and on other pages that are initialized,
      FIL_PAGE_OFFSET must contain the page number.
      
      fil_crypt_is_page_uninitialized(): Remove. It suffices to check the
      page number field in fil_crypt_rotate_page().
      36a97172
    • Marko Mäkelä's avatar
      Code clean-up related to MDEV-13167 · e52dd13c
      Marko Mäkelä authored
      xdes_get_descriptor_const(): New function, to get read-only access to
      the allocation descriptor.
      
      fseg_page_is_free(): Only acquire a shared latch on the tablespace,
      not an exclusive latch. Calculate the descriptor page address before
      acquiring the tablespace latch. If the page number is out of bounds,
      return without fetching any page. Access only one descriptor page.
      
      fsp_page_is_free(), fsp_page_is_free_func(): Remove.
      Use fseg_page_is_free() instead.
      
      fsp_init_file_page(): Move the debug parameter into a separate function.
      
      btr_validate_level(): Remove the unused variable "seg".
      e52dd13c
    • Marko Mäkelä's avatar
      MDEV-13485 MTR tests fail massively with --innodb-sync-debug · 59caf2c3
      Marko Mäkelä authored
      The parameter --innodb-sync-debug, which is disabled by default,
      aims to find potential deadlocks in InnoDB.
      
      When the parameter is enabled, lots of tests failed. Most of these
      failures were due to bogus diagnostics. But, as part of this fix,
      we are also fixing a bug in error handling code and removing dead
      code, and fixing cases where an uninitialized mutex was being
      locked and unlocked.
      
      dict_create_foreign_constraints_low(): Remove an extraneous
      mutex_exit() call that could cause corruption in an error handling
      path. Also, do not unnecessarily acquire dict_foreign_err_mutex.
      Its only purpose is to control concurrent access to
      dict_foreign_err_file.
      
      row_ins_foreign_trx_print(): Replace a redundant condition with a
      debug assertion.
      
      srv_dict_tmpfile, srv_dict_tmpfile_mutex: Remove. The
      temporary file is never being written to or read from.
      
      log_free_check(): Allow SYNC_FTS_CACHE (fts_cache_t::lock)
      to be held.
      
      ha_innobase::inplace_alter_table(), row_merge_insert_index_tuples():
      Assert that no unexpected latches are being held.
      
      sync_latch_meta_init(): Properly initialize dict_operation_lock_key
      at SYNC_DICT_OPERATION. dict_sys->mutex is SYNC_DICT, and
      the now-removed SRV_DICT_TMPFILE was wrongly registered at
      SYNC_DICT_OPERATION.
      
      buf_block_init(): Correctly register buf_block_t::debug_latch.
      It was previously misleadingly reported as LATCH_ID_DICT_FOREIGN_ERR.
      
      latch_level_t: Correct the relative latching order of
      SYNC_IBUF_PESS_INSERT_MUTEX,SYNC_INDEX_TREE and
      SYNC_FILE_FORMAT_TAG,SYNC_DICT_OPERATION to avoid bogus failures.
      
      row_drop_table_for_mysql(): Avoid accessing btr_defragment_mutex
      if the defragmentation thread has not been started. This is the
      case during fts_drop_orphaned_tables() in recv_recovery_rollback_active().
      
      fil_space_destroy_crypt_data(): Avoid acquiring fil_crypt_threads_mutex
      when it is uninitialized. We may have created crypt_data before the
      mutex was created, and the mutex creation would be skipped if
      InnoDB startup failed or --innodb-read-only was specified.
      59caf2c3
    • Marko Mäkelä's avatar
      Remove the unused redo log record type MLOG_INIT_FILE_PAGE · 1621d32e
      Marko Mäkelä authored
      InnoDB stopped generating the MLOG_INIT_FILE_PAGE record in
      MySQL 5.7.5. Starting with MySQL 5.7.9 (which was imported to
      MariaDB Server 10.2.2), the InnoDB redo log format tag prevents
      crash recovery from old-format redo logs.
      
      Remove the dead code for dealing with MLOG_INIT_FILE_PAGE.
      1621d32e
    • Marko Mäkelä's avatar
      MDEV-13452 Assertion `!recv_no_log_write' failed at startup · 825b6a35
      Marko Mäkelä authored
      The previous fix (commit dcdc1c6d)
      should have removed the assertion from log_close(), because every
      caller that requires this assertion is already asserting that log
      writes are allowed. When fil_names_clear() is called, it must be
      able to write the MLOG_CHECKPOINT records. The purpose of the debug
      variable recv_no_log_write is to prevent the creation of page-level
      redo log records, or modifications to persistent data.
      825b6a35
  4. 22 Aug, 2017 1 commit
  5. 21 Aug, 2017 2 commits
  6. 20 Aug, 2017 1 commit
  7. 19 Aug, 2017 1 commit
    • Igor Babaev's avatar
      This patch complements the patch for mdev-10855. · d7b45e01
      Igor Babaev authored
      It allows to push conditions into derived with window functions not
      only in the cases when the window specifications of these window
      functions use the same partition, but also in the cases when the window
      functions use partitions that share only some fields. In these
      cases only the conditions over the common fields are pushed.
      d7b45e01
  8. 18 Aug, 2017 8 commits
    • Marko Mäkelä's avatar
      MDEV-13559 encryption.innodb-redo-badkey failed in buildbot · 86fc5ece
      Marko Mäkelä authored
      Add suppressions for the read and decompression errors.
      This may be 10.3 specific and related to MDEV-13536 which increases
      purge activity. But it does not hurt to suppress rarely occurring
      and plausible error messages for this fault-injection test already in 10.2.
      86fc5ece
    • Marko Mäkelä's avatar
      MDEV-13570 Assertion failure !srv_read_only_mode in --innodb-read-only... · 8a9e9d89
      Marko Mäkelä authored
      MDEV-13570 Assertion failure !srv_read_only_mode in --innodb-read-only shutdown when buf_resize_thread is active
      
      logs_empty_and_mark_files_at_shutdown(): Skip the debug assertion
      when the buf_resize_thread is active.
      8a9e9d89
    • Marko Mäkelä's avatar
      MDEV-13575 On failure, Mariabackup --backup --safe-slave-backup may forget to... · 8a3e2970
      Marko Mäkelä authored
      MDEV-13575 On failure, Mariabackup --backup --safe-slave-backup may forget to START SLAVE SQL_THREAD
      
      backup_release(): New function, refactored from backup_finish().
      Release some resources that may have been acquired by backup_startup()
      and should be released even after a failed operation.
      
      xtrabackup_backup_low(): Refactored from xtrabackup_backup_func().
      
      xtrabackup_backup_func(): Always call backup_release() after calling
      backup_start().
      8a3e2970
    • Daniel Bartholomew's avatar
      bump the VERSION · 72ac85cd
      Daniel Bartholomew authored
      72ac85cd
    • Alexander Barkov's avatar
    • Marko Mäkelä's avatar
      MDEV-13754 Memory leak in mariabackup.incremental_backup · 605b8352
      Marko Mäkelä authored
      The test mariabackup.incremental_backup revealed a memory leak
      in have_queries_to_wait_for(). The problem is that
      xb_mysql_query() is being invoked with bool use_result=true
      but the result is not being freed by mysql_store_result().
      There are similar leaks in other functions.
      
      have_queries_to_wait_for(): Invoke mysql_free_result() to
      clean up after the mysql_store_result() that was invoked
      by xb_mysql_query().
      
      select_incremental_lsn_from_history(): Plug the leak on failure.
      
      kill_long_queries(): Plug the memory leak.
      (This function always leaked memory when it was called.)
      605b8352
    • Marko Mäkelä's avatar
      MDEV-13574 related Mariabackup code cleanup (non-functional change) · 74ce0cf1
      Marko Mäkelä authored
      have_queries_to_wait_for(), kill_long_queries(): Declare and initialize
      variables in one go.
      74ce0cf1
    • Marko Mäkelä's avatar
      Follow-up fix to MDEV-12988 backup fails if innodb_undo_tablespaces>0 · e9e051d2
      Marko Mäkelä authored
      The fix broke mariabackup --prepare --incremental.
      
      The restore of an incremental backup starts up (parts of) InnoDB twice.
      First, all data files are discovered for applying .delta files. Then,
      after the .delta files have been applied, InnoDB will be restarted
      more completely, so that the redo log records will be applied via the
      buffer pool.
      
      During the first startup, the buffer pool is not initialized, and thus
      trx_rseg_get_n_undo_tablespaces() must not be invoked. The apply of
      the .delta files will currently assume that the --innodb-undo-tablespaces
      option correctly specifies the number of undo tablespace files, just
      like --backup does.
      
      The second InnoDB startup of --prepare for applying the redo log will
      properly invoke trx_rseg_get_n_undo_tablespaces().
      
      enum srv_operation_mode: Add SRV_OPERATION_RESTORE_DELTA for
      distinguishing the apply of .delta files from SRV_OPERATION_RESTORE.
      
      srv_undo_tablespaces_init(): In mariabackup --prepare --incremental,
      in the initial SRV_OPERATION_RESTORE_DELTA phase, do not invoke
      trx_rseg_get_n_undo_tablespaces() because the buffer pool or the
      redo logs are not available. Instead, blindly rely on the parameter
      --innodb-undo-tablespaces.
      e9e051d2
  9. 17 Aug, 2017 7 commits
  10. 16 Aug, 2017 6 commits