1. 02 Dec, 2020 5 commits
  2. 01 Dec, 2020 7 commits
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · 589cf8db
      Marko Mäkelä authored
      589cf8db
    • Vlad Lesin's avatar
      MDEV-22929 MariaBackup option to report and/or continue when corruption is encountered · e30a05f4
      Vlad Lesin authored
      Post-push Windows compilation errors fix.
      e30a05f4
    • Monty's avatar
      After merge fixes · 7edfed63
      Monty authored
      Change thd->mdl_context.release_transactional_locks() to
      thd->mdl_release_transactional_locks()
      7edfed63
    • Marko Mäkelä's avatar
      MDEV-24323 Crash on recovery after kill during instant ADD COLUMN · 73f34336
      Marko Mäkelä authored
      row_undo_ins_parse_undo_rec(): Do not try to read non-existing
      virtual column information for the metadata record.
      73f34336
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 81ab9ea6
      Marko Mäkelä authored
      81ab9ea6
    • Marko Mäkelä's avatar
      MDEV-21962 fixup: Remove buf_pool_contains_zip() · e76e1288
      Marko Mäkelä authored
      The replacement is buf_pool.contains_zip().
      e76e1288
    • Vlad Lesin's avatar
      MDEV-22929 MariaBackup option to report and/or continue when corruption is encountered · e6b3e38d
      Vlad Lesin authored
      The new option --log-innodb-page-corruption is introduced.
      
      When this option is set, backup is not interrupted if innodb corrupted
      page is detected. Instead it logs all found corrupted pages in
      innodb_corrupted_pages file in backup directory and finishes with error.
      
      For incremental backup corrupted pages are also copied to .delta file,
      because we can't do LSN check for such pages during backup,
      innodb_corrupted_pages will also be created in incremental backup
      directory.
      
      During --prepare, corrupted pages list is read from the file just after
      redo log is applied, and each page from the list is checked if it is allocated
      in it's tablespace or not. If it is not allocated, then it is zeroed out,
      flushed to the tablespace and removed from the list. If all pages are removed
      from the list, then --prepare is finished successfully and
      innodb_corrupted_pages file is removed from backup directory. Otherwise
      --prepare is finished with error message and innodb_corrupted_pages contains
      the list of the pages, which are detected as corrupted during backup, and are
      allocated in their tablespaces, what means backup directory contains corrupted
      innodb pages, and backup can not be considered as consistent.
      
      For incremental --prepare corrupted pages from .delta files are applied
      to the base backup, innodb_corrupted_pages is read from both base in
      incremental directories, and the same action is proceded for corrupted
      pages list as for full --prepare. innodb_corrupted_pages file is
      modified or removed only in base directory.
      
      If DDL happens during backup, it is also processed at the end of backup
      to have correct tablespace names in innodb_corrupted_pages.
      e6b3e38d
  3. 30 Nov, 2020 11 commits
    • Monty's avatar
      MDEV 15532 Assertion `!log->same_pk' failed in row_log_table_apply_delete · 828471cb
      Monty authored
      The reason for the failure is that
      thd->mdl_context.release_transactional_locks()
      was called after commit & rollback even in cases where the current
      transaction is still active.
      
      For 10.2, 10.3 and 10.4 the fix is simple:
      - Replace all calls to thd->mdl_context.release_transactional_locks() with
        thd->release_transactional_locks(). The thd function will only call
        the mdl_context function if there are no active transactional locks.
        In 10.6 we will better fix where we will change the return value for
        some trans_xxx() functions to indicate if transaction did close the
        transaction or not. This will avoid the need of the indirect call.
      
      Other things:
      - trans_xa_commit() and trans_xa_rollback() will automatically
        call release_transactional_locks() if the transaction is closed.
      - We can't do that for the other functions as the caller of many of these
        are doing additional work (like close_thread_tables) before calling
        release_transactional_locks().
      - Added missing abort_result_set() and missing DBUG_RETURN in
        select_create::send_eof()
      - Fixed wrong indentation in injector::transaction::commit()
      828471cb
    • Monty's avatar
      Fixed maria.create test · c5375764
      Monty authored
      c5375764
    • Monty's avatar
      MDEV-15532 Assertion `!log->same_pk' failed in row_log_table_apply_delete · a3531775
      Monty authored
      The real fix for MDEV-15532 will be pushed into 10.2 and 10.6
      This is an additional fix for 10.4.
      
      In 10.4 trans_xa_detach was introduced.  However THD::cleanup() assumes
      that after trans_xa_detach() is done, there is no registered transactions
      anymore. In the 10.2 patch there will be an assert to ensure this, which
      will cause 10.4 to fail.
      
      The fix used is to reset the transaction flags in trans_xa_detach().
      a3531775
    • Monty's avatar
      Fixed maria.create test · 6261b1f4
      Monty authored
      6261b1f4
    • Vladislav Vaintroub's avatar
      Clarify some comments. · 1435f35b
      Vladislav Vaintroub authored
      - the intention for my_getevents syscall is now better explained,
      why are we using it (to be able to interrupt io_getevents syscall via
      io_destroy()).
      
      - Fix comment for MAX_EVENTS in getevent_thread_routine.
      MAX_EVENTS is more of less arbitrary constant, chosen such that events array
      is big enough to get multiple simultaneous io completions, but small
      enough so it does not blow the thread's stack.
      1435f35b
    • Vladislav Vaintroub's avatar
      MDEV-24295 Reduce wakeups by tpool maintenance timer, when server is idle · 5bb5d4ad
      Vladislav Vaintroub authored
      If maintenance timer does not do much for prolonged time, it will
      wake up less frequently, once every 4 seconds instead of once every 0.4
      second.
      
      It will wakeup more often if thread creation is throttled, to avoid stalls.
      5bb5d4ad
    • Monty's avatar
    • Sergei Petrunia's avatar
      11196347
    • Marko Mäkelä's avatar
      MDEV-24308: Revert for Windows · e34e53b5
      Marko Mäkelä authored
      For some reason, InnoDB debug tests on Windows fail due to rw_lock_t
      if the function call overhead for some os_thread_ code is removed.
      
      This change worked fine on Windows in combination with MDEV-24142.
      e34e53b5
    • Varun Gupta's avatar
      MDEV-21265: IN predicate conversion to IN subquery should be allowed for a... · b4379df5
      Varun Gupta authored
      MDEV-21265: IN predicate conversion to IN subquery should be allowed for a broader set of datatype comparison
      
      Allow materialization strategy when collations on the
      inner and outer sides of an IN subquery are the same and the
      character set of the inner side is a proper subset of the character
      set on the outer side.
      This allows conversion from utf8mb3 to utf8mb4
      as the former is a subset of the later.
      This is only allowed when IN predicate is converted to an IN subquery
      
      Backported part of the patch (d6a00d9b) of MDEV-17905.
      b4379df5
    • Marko Mäkelä's avatar
      MDEV-24308: Remove some os_thread_ functions · 8fa6e363
      Marko Mäkelä authored
      os_thread_pf(): Remove.
      
      os_thread_eq(), os_thread_yield(), os_thread_get_curr_id():
      Define as macros.
      
      ut_print_timestamp(), ut_sprintf_timestamp(): Simplify.
      8fa6e363
  4. 27 Nov, 2020 1 commit
    • Igor Babaev's avatar
      MDEV-24242 Query returns wrong result while using big_tables=1 · b92391d5
      Igor Babaev authored
      When executing set operations in a pipeline using only one temporary table
      additional scans of intermediate results may be needed. The scans are
      performed with usage of the rnd_next() handler function that might
      leave record buffers used for the temporary table not in a state that
      is good for following writes into the table. For example it happens for
      aria engine when the last call of rnd_next() encounters only deleted
      records. Thus a cleanup of record buffers is needed after each such scan
      of the temporary table.
      
      Approved by Oleksandr Byelkin <sanja@mariadb.com>
      b92391d5
  5. 26 Nov, 2020 7 commits
  6. 25 Nov, 2020 9 commits
    • Sergei Golubchik's avatar
      MDEV-24230 subquery on information_schema fails with error message · f3b10354
      Sergei Golubchik authored
      disable thd->count_cuted_fields when populating internal temporary
      tables for I_S, because this is how SELECT works standalone.
      And if the SELECT is a part of INSERT or UPDATE or RETURN or SET or
      anything else that enables thd->count_cuted_fields, this counting should
      only apply when storing the result of the SELECT in a field or a
      variable, not when populating internal temporary tables for I_S.
      f3b10354
    • Sergei Golubchik's avatar
    • Eugene Kosov's avatar
      MDEV-24275 InnoDB persistent stats analyze forces full scan forcing lock crash · 5991bd62
      Eugene Kosov authored
      This is a fixup patch for MDEV-23991 afc9d00c
      
      We really should read result.n_leaf_pages, which was set previously.
      
      Analysis and fix was provided by Jukka Santala. Thanks!
      
      Reviewed by: Marko Mäkelä
      5991bd62
    • Marko Mäkelä's avatar
      MDEV-24280 InnoDB triggers too many independent periodic tasks · 657fcdf4
      Marko Mäkelä authored
      A side effect of MDEV-16264 is that a large number of threads will
      be created at server startup, to be destroyed after a minute or two.
      
      One source of such thread creation is srv_start_periodic_timer().
      InnoDB is creating 3 periodic tasks: srv_master_callback (1Hz)
      srv_error_monitor_task (1Hz), and srv_monitor_task (0.2Hz).
      
      It appears that we can merge srv_error_monitor_task and srv_monitor_task
      and have them invoked 4 times per minute (every 15 seconds). This will
      affect our ability to enforce innodb_fatal_semaphore_wait_threshold and
      some computations around BUF_LRU_STAT_N_INTERVAL.
      
      We could remove srv_master_callback along with the DROP TABLE queue
      at some point of time in the future. We must keep it independent
      of the innodb_fatal_semaphore_wait_threshold detection, because
      the background DROP TABLE queue could get stuck due to dict_sys
      being locked by another thread. For now, srv_master_callback
      must be invoked once per second, so that
      innodb_flush_log_at_timeout=1 can work.
      
      BUF_LRU_STAT_N_INTERVAL: Reduce the precision and extend the time
      from 50*1 second to 4*15 seconds.
      
      srv_error_monitor_timer: Remove.
      
      MAX_MUTEX_NOWAIT: Increase from 20*1 second to 2*15 seconds.
      
      srv_refresh_innodb_monitor_stats(): Avoid a repeated call to time(NULL).
      Change the interval to less than 60 seconds.
      
      srv_monitor(): Renamed from srv_monitor_task.
      
      srv_monitor_task(): Renamed from srv_error_monitor_task().
      Invoked only once in 15 seconds. Invoke also srv_monitor().
      Increase the fatal_cnt threshold from 10*1 second to 1*15 seconds.
      
      sync_array_print_long_waits_low(): Invoke time(NULL) only once.
      Remove a bogus message about printouts for 30 seconds. Those
      printouts were effectively already disabled in MDEV-16264
      (commit 5e62b6a5).
      657fcdf4
    • Marko Mäkelä's avatar
      MDEV-24278 InnoDB page cleaner keeps waking up on idle server · 7b1252c0
      Marko Mäkelä authored
      The purpose of the InnoDB page cleaner subsystem is to write out
      modified pages from the buffer pool to data files. When the
      innodb_max_dirty_pages_pct_lwm is not exceeded or
      innodb_adaptive_flushing=ON decides not to write out anything,
      the page cleaner should keep sleeping indefinitely until the state
      of the system changes: a dirty page is added to the buffer pool such
      that the page cleaner would no longer be idle.
      
      buf_flush_page_cleaner(): Explicitly note when the page cleaner is idle.
      When that happens, use mysql_cond_wait() instead of mysql_cond_timedwait().
      
      buf_flush_insert_into_flush_list(): Wake up the page cleaner if needed.
      
      innodb_max_dirty_pages_pct_update(),
      innodb_max_dirty_pages_pct_lwm_update():
      Wake up the page cleaner just in case.
      
      Note: buf_flush_ahead(), buf_flush_wait_flushed() and shutdown are
      already waking up the page cleaner thread.
      7b1252c0
    • Marko Mäkelä's avatar
      MDEV-24270: Clarify some comments · f693b725
      Marko Mäkelä authored
      f693b725
    • Vladislav Vaintroub's avatar
      Fix misspelling. · 2de95f7a
      Vladislav Vaintroub authored
      Kudos to Marko for finding.
      2de95f7a
    • Vladislav Vaintroub's avatar
      Cleanup. Remove obsolete comment · af98fddc
      Vladislav Vaintroub authored
      af98fddc
    • Vladislav Vaintroub's avatar