1. 20 Jul, 2020 10 commits
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-22970 Possible corruption of page_compressed tables, or · c8936686
      Thirunarayanan Balathandayuthapani authored
                 when scrubbing is enabled
      
      buf_read_recv_pages(): Ignore the page to read if it is already
      present in the freed ranges.
      
      store_freed_or_init_rec(): Store the ranges only if scrubbing
      is enabled or page compressed tablespace.
      
      recv_init_crash_recovery_space(): Add the freed range only when
      scrubbing or page compressed tablespace.
      
      range_set::contains(): Search the value is present in ranges.
      
      range_set::remove_if_exists(): Remove the value if exist in ranges.
      
      mtr_t::init(): Handles the scenario that mini-transaction may allocate
      a page that had just been freed.
      
      recv_sys_t::parse(): Note down the FREE and INIT redo log irrespective
      of STORE value.
      
      Removed innodb_tablespaces_scrubbing from test case
      c8936686
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · 4d4865de
      Marko Mäkelä authored
      4d4865de
    • Marko Mäkelä's avatar
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · 4b959bd8
      Marko Mäkelä authored
      4b959bd8
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · acc58fd8
      Marko Mäkelä authored
      acc58fd8
    • Marko Mäkelä's avatar
      Merge 10.1 into 10.2 · ca9276e3
      Marko Mäkelä authored
      ca9276e3
    • Marko Mäkelä's avatar
      MDEV-23190 InnoDB data file extension is not crash-safe · 57ec42bc
      Marko Mäkelä authored
      When InnoDB is extending a data file, it is updating the FSP_SIZE
      field in the first page of the data file.
      
      In commit 8451e090 (MDEV-11556)
      we removed a work-around for this bug and made recovery stricter,
      by making it track changes to FSP_SIZE via redo log records, and
      extend the data files before any changes are being applied to them.
      
      It turns out that the function fsp_fill_free_list() is not crash-safe
      with respect to this when it is initializing the change buffer bitmap
      page (page 1, or generally, N*innodb_page_size+1). It uses a separate
      mini-transaction that is committed (and will be written to the redo
      log file) before the mini-transaction that actually extended the data
      file. Hence, recovery can observe a reference to a page that is
      beyond the current end of the data file.
      
      fsp_fill_free_list(): Initialize the change buffer bitmap page in
      the same mini-transaction.
      
      The rest of the changes are fixing a bug that the use of the separate
      mini-transaction was attempting to work around. Namely, we must ensure
      that no other thread will access the change buffer bitmap page before
      our mini-transaction has been committed and all page latches have been
      released.
      
      That is, for read-ahead as well as neighbour flushing, we must avoid
      accessing pages that might not yet be durably part of the tablespace.
      
      fil_space_t::committed_size: The size of the tablespace
      as persisted by mtr_commit().
      
      fil_space_t::max_page_number_for_io(): Limit the highest page
      number for I/O batches to committed_size.
      
      MTR_MEMO_SPACE_X_LOCK: Replaces MTR_MEMO_X_LOCK for fil_space_t::latch.
      
      mtr_x_space_lock(): Replaces mtr_x_lock() for fil_space_t::latch.
      
      mtr_memo_slot_release_func(): When releasing MTR_MEMO_SPACE_X_LOCK,
      copy space->size to space->committed_size. In this way, read-ahead
      or flushing will never be invoked on pages that do not yet exist
      according to FSP_SIZE.
      57ec42bc
    • Marko Mäkelä's avatar
      98e2c17e
    • Marko Mäkelä's avatar
      14543afd
    • Marko Mäkelä's avatar
      MDEV-22771 Instant extension of CHAR column is wrongly allowed · 0a7faed7
      Marko Mäkelä authored
      commit 854c219a (MDEV-17301)
      broke a constraint: Fixed-length columns cannot be extended in InnoDB
      without rebuilding the table.
      
      ha_innobase::can_convert_string(): Correct the condition. We must
      not allow any instantaneous change to the length of CHAR columns
      measured in characters. For any format other than ROW_FORMAT=REDUNDANT,
      we can allow the length in bytes to be extended if mbminlen<mbmaxlen held
      before the change of the character set.
      0a7faed7
  2. 18 Jul, 2020 1 commit
  3. 17 Jul, 2020 1 commit
  4. 16 Jul, 2020 9 commits
    • Julius Goryavsky's avatar
      MDEV-20401: revert unnecessary change · a1e52e7f
      Julius Goryavsky authored
      a1e52e7f
    • Julius Goryavsky's avatar
      MDEV-20401: revert unnecessary change · 1ba8df4c
      Julius Goryavsky authored
      1ba8df4c
    • Julius Goryavsky's avatar
    • Julius Goryavsky's avatar
      MDEV-20401: Server incorrectly auto-sets lower_case_file_system value · b3cae9db
      Julius Goryavsky authored
      Server auto-sets lower_case_file_system value based on default
      datadir's behavior instead of instead of using the directory specified
      by the user through the configuration file or command line options.
      
      This patch fixes this problem.
      b3cae9db
    • Julius Goryavsky's avatar
      MDEV-20401: Server incorrectly auto-sets lower_case_file_system value · 4412a461
      Julius Goryavsky authored
      Server auto-sets lower_case_file_system value based on default
      datadir's behavior instead of instead of using the directory specified
      by the user through the configuration file or command line options.
      
      This patch fixes this problem.
      4412a461
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · 054f1036
      Marko Mäkelä authored
      054f1036
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · 3280edda
      Marko Mäkelä authored
      3280edda
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 73aa31fb
      Marko Mäkelä authored
      73aa31fb
    • Marko Mäkelä's avatar
      MDEV-21347 innodb_log_optimize_ddl=OFF is not crash safe · 147d4b1e
      Marko Mäkelä authored
      In commit 0f90728b (MDEV-16809)
      we introduced the configuration option innodb_log_optimize_ddl
      for controlling whether native index creation or table-rebuild
      in InnoDB should avoid writing full redo log.
      
      Fungo Wang reported that this option is causing occasional failures.
      The reason is that pages may be written to data files in an
      inconsistent state. Applying log records to such inconsistent pages
      may fail.
      
      The solution is to always invoke PageBulk::finish() before page latches
      may be released, to ensure that the page contents is in a consistent
      state.
      
      Something similar was implemented in MySQL 8.0.13:
      mysql/mysql-server@d1254b947354e0f5b7223b09c521bd85f22e1e31
      
      buf_block_t::skip_flush_check: Remove. Suppressing consistency checks
      is a bad idea.
      
      PageBulk::needs_finish(): New predicate: Determine whether
      PageBulk::finish() must fix up the page.
      
      PageBulk::init(): Clear PAGE_DIRECTION to ensure that needs_finish()
      will hold. We change the field from PAGE_NO_DIRECTION to 0
      and back without writing redo log. This trick avoids the need
      to introduce any new data member to PageBulk.
      
      PageBulk::insert(): Replace some high-level accessors to bypass
      debug assertions related to PAGE_HEAP_TOP that we will be violating
      until finish() has been executed.
      
      PageBulk::finish(): Tolerate m_rec_no==0. We must invoke this also
      on an empty page, to ensure that PAGE_HEAP_TOP is initialized.
      
      PageBulk::commit(): Always invoke finish().
      
      PageBulk::release(), BtrBulk::pageSplit(), BtrBulk::storeExt(),
      BtrBulk::finish(): Invoke PageBulk::finish().
      147d4b1e
  5. 15 Jul, 2020 11 commits
    • Marko Mäkelä's avatar
      Make page validation stricter · fee11c77
      Marko Mäkelä authored
      page_simple_validate_old(), page_simple_validate_new():
      Require PAGE_N_DIR_SLOTS to be at least 2.
      fee11c77
    • Marko Mäkelä's avatar
      MDEV-23183 Infinite loop on page_validate() on corrupted page · 38b4c078
      Marko Mäkelä authored
      MDEV-22721 (commit eba2d10a)
      inadvertently introduced an infinite loop.
      
      page_validate(): Remove the infinite loop.
      38b4c078
    • Daniel Black's avatar
      MDEV-23175: my_timer_milliseconds ftime deprecated - clock_gettime replacement · 20512a68
      Daniel Black authored
      Linux glibc has deprecated ftime resutlting in a compile error on Fedora-32.
      
      Per manual clock_gettime is the suggested replacement. Because my_timer_milliseconds
      is a relative time used by largely the perfomrance schema, CLOCK_MONOTONIC_COARSE
      is used. This has been available since Linux-2.6.32.
      
      The low overhead is shows in the unittest:
      
          $ unittest/mysys/my_rdtsc-t
          1..11
          # ----- Routine ---------------
          # myt.cycles.routine          :             5
          # myt.nanoseconds.routine     :            11
          # myt.microseconds.routine    :            13
          # myt.milliseconds.routine    :            18
          # myt.ticks.routine           :            17
          # ----- Frequency -------------
          # myt.cycles.frequency        :    3596597014
          # myt.nanoseconds.frequency   :    1000000000
          # myt.microseconds.frequency  :       1000000
          # myt.milliseconds.frequency  :          1039
          # myt.ticks.frequency         :           103
          # ----- Resolution ------------
          # myt.cycles.resolution       :             1
          # myt.nanoseconds.resolution  :             1
          # myt.microseconds.resolution :             1
          # myt.milliseconds.resolution :             1
          # myt.ticks.resolution        :             1
          # ----- Overhead --------------
          # myt.cycles.overhead         :           118
          # myt.nanoseconds.overhead    :           234
          # myt.microseconds.overhead   :           222
          # myt.milliseconds.overhead   :            30
          # myt.ticks.overhead          :          4946
          ok 1 - my_timer_init() did not crash
          ok 2 - The cycle timer is strictly increasing
          ok 3 - The cycle timer is implemented
          ok 4 - The nanosecond timer is increasing
          ok 5 - The nanosecond timer is implemented
          ok 6 - The microsecond timer is increasing
          ok 7 - The microsecond timer is implemented
          ok 8 - The millisecond timer is increasing
          ok 9 - The millisecond timer is implemented
          ok 10 - The tick timer is increasing
          ok 11 - The tick timer is implemented
      20512a68
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · e67daa56
      Marko Mäkelä authored
      e67daa56
    • Vladislav Vaintroub's avatar
      Fix compile warning · 9c8420fe
      Vladislav Vaintroub authored
      9c8420fe
    • Marko Mäkelä's avatar
      Revert MDEV-20453 (string_view) · ced3ec4c
      Marko Mäkelä authored
      In fsp_path_to_space_name(), we would access a byte right before
      the start of the string, tripping AddressSanitizer.
      
      This reverts commit d87006a1
      and commit a7634281.
      ced3ec4c
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · 9936cfd5
      Marko Mäkelä authored
      9936cfd5
    • Alexander Barkov's avatar
      MDEV-23162 Improve Protocol performance for numeric data · eb2eaba7
      Alexander Barkov authored
      Avoid character set conversion for numeric data in this scenario:
      
      SET NAMES utf8;
      CREATE OR REPLACE TABLE t1 (a INT);
      INSERT INTO t1 VALUES (1);
      SELECT a FROM t1;
      eb2eaba7
    • Alexander Barkov's avatar
      Preparatory changes for MDEV-23162 Improve Protocol performance for numeric data · be98036f
      Alexander Barkov authored
      - Renaming this virtual method store() to store_str():
          store(const char *str, size_t length, CHARSET_INFO *src_cs, CHARSET_INFO *dst_cs)
        We'll be adding more variants of store*() soon. This change will help to avoid
        ambiguities during overloading.
      
      - Adding a helper method store_ident().
      
      - Renaming store_str(const LEX_CSTRING &s...) to store_lex_cstring(),
        to avoid ambiguties during overloading.
      
      - Adding a helper method store() for backward compatibility, to avoid a lot of
        changes in the code now. But eventually we should replace store() to
        non-ambiguius methods store_str() or store_ident().
      
      - Adding a helper method Protocol::needs_conversion() and reusing it
        in two places.
      be98036f
    • Daniel Black's avatar
      check_linker_flag: use for linker flags · 7473e184
      Daniel Black authored
      -Wl,-z,relro,-z,now are linker flags and should
      be checked as such.
      
      TODO: perform module, exe shared checks separately
      rather than a pure linker check.
      7473e184
    • Varun Gupta's avatar
      MDEV-22851: Engine independent index statistics are incorrect for large tables on Windows · dfdfeecb
      Varun Gupta authored
      An oveflow was happening on windows because on Windows sizeof(ulong) is 4 bytes
      while it is 8 bytes on Linux.
      Switched avg_frequency and avg length for column statistics to ulonglong.
      Switched avg_frequency for index statistics to ulonglong.
      dfdfeecb
  6. 14 Jul, 2020 8 commits