1. 11 Dec, 2017 1 commit
    • Vladislav Vaintroub's avatar
      MDEV-14113 Use abortive TCP close, in case server closes the connection · 70d6c944
      Vladislav Vaintroub authored
      first, and we do not not care whether client has received all data.
      
      This is a TCP optimization to avoid TIME_WAIT in TCP connection teardown.
      
      This patch would abort connection on timeout, which usually happens when
      client reads a large result set, at slower pace then the server can
      write.
      
      The patch also cleans up socket timeout handling, so that Windows
      is consistent with another platforms (using nonblocking socket IO
       + waiting in poll/select on single socket, rather than setsockopt).
      This makes identifying timeouts easier.
      
      
      Also removed the superficial shutdown() before closesocket() in a few
      places where it was used, because it  was never needed , and
      reportedly breaks SO_LINGER on Windows.
      70d6c944
  2. 10 Dec, 2017 1 commit
  3. 09 Dec, 2017 1 commit
    • Vesa Pentti's avatar
      MDEV-12501 -- set --maturity-level by default · 99bcec29
      Vesa Pentti authored
        * Note: breaking change; since this commit, a plugin that has
          worked so far might get rejected due to plugin maturity
        * mariabackup is not affected (allows all plugins)
        * VERSION file defines SERVER_MATURITY, which defines the
          corresponding numeric value as SERVER_MATURITY_LEVEL in
          include/mysql_version.h
        * The default value for 'plugin_maturity' is SERVER_MATURITY_LEVEL - 1
        * Logs a warning if a plugin has maturity lower than
          SERVER_MATURITY_LEVEL
        * Tests suppress the plugin maturity warning
        * Tests use --plugin-maturity=unknown by default so as not to fail
          due to the stricter plugin maturity handling
      99bcec29
  4. 08 Dec, 2017 10 commits
    • Sergey Vojtovich's avatar
      c60095a8
    • Sergey Vojtovich's avatar
      MDEV-14529 - InnoDB rw-locks: optimize memory barriers · b04f2a0f
      Sergey Vojtovich authored
      Relax memory barrier for lock_word.
      
      rw_lock_lock_word_decr() - used to acquire rw-lock, thus we only need to issue
      ACQUIRE when we succeed locking.
      
      rw_lock_x_lock_func_nowait() - same as above, but used to attempt to acquire
      X-lock.
      
      rw_lock_s_unlock_func() - used to release S-lock, RELEASE is what we need here.
      
      rw_lock_x_unlock_func() - used to release X-lock. Ideally we'd need only RELEASE
      here, but due to mess with waiters (they must be loaded after lock_word is
      stored) we have to issue both ACQUIRE and RELEASE.
      
      rw_lock_sx_unlock_func() - same as above, but used to release SX-lock.
      
      rw_lock_s_lock_spin(), rw_lock_x_lock_func(), rw_lock_sx_lock_func() -
      fetch-and-store to waiters has to issue only ACQUIRE memory barrier, so that
      waiters are stored before lock_word is loaded.
      
      Note that there is violation of RELEASE-ACQUIRE protocol here, because we do
      on lock:
      
        my_atomic_fas32_explicit((int32*) &lock->waiters, 1, MY_MEMORY_ORDER_ACQUIRE);
        my_atomic_load32_explicit(&lock->lock_word, MY_MEMORY_ORDER_RELAXED);
      
      on unlock
      
        my_atomic_add32_explicit(&lock->lock_word, X_LOCK_DECR, MY_MEMORY_ORDER_ACQ_REL);
        my_atomic_load32_explicit((int32*) &lock->waiters, MY_MEMORY_ORDER_RELAXED);
      
      That is we kind of synchronize ACQUIRE on lock_word with ACQUIRE on waiters.
      It was there before this patch. Simple fix may have negative performance impact.
      Proper fix requires refactoring of lock_word.
      b04f2a0f
    • Sergey Vojtovich's avatar
      MDEV-14529 - InnoDB rw-locks: optimize memory barriers · 51bb18f9
      Sergey Vojtovich authored
      Relax memory barrier for waiters: these 2 stores must be completed before
      os_event_set() finishes. This is guaranteed by RELEASE barrier issued by
      mutex.exit() of os_event_set().
      51bb18f9
    • Sergey Vojtovich's avatar
      MDEV-14529 - InnoDB rw-locks: optimize memory barriers · 5b624f00
      Sergey Vojtovich authored
      Remove volatile modifier from waiters: it's not supposed for inter-thread
      communication, use appropriate atomic operations instead.
      
      Changed waiters to int32_t, my_atomic friendly type.
      5b624f00
    • Sergey Vojtovich's avatar
      MDEV-14529 - InnoDB rw-locks: optimize memory barriers · 57d20f11
      Sergey Vojtovich authored
      Remove volatile modifier from lock_word: it's not supposed for inter-thread
      communication, use appropriate atomic operations instead.
      57d20f11
    • Sergey Vojtovich's avatar
      MDEV-14529 - InnoDB rw-locks: optimize memory barriers · c73e77da
      Sergey Vojtovich authored
      Change lock_word from lint to int32_t: the latter is my_atomic_* friendly type.
      c73e77da
    • Monty's avatar
      db715ff3
    • Monty's avatar
      Search for galera libraries also in /usr/lib64/galera-3 · 68cd5435
      Monty authored
      This is where Codership's offical rpm's puts them
      68cd5435
    • Monty's avatar
      Cleanups · c4581735
      Monty authored
      - Remove not used thd_rpl_is_parallel()
      - Remove not used mysql_notify_thread_having_shared_lock()
      - Remove not needed LOCK_thread_count from MYSQL_BIN_LOG::reset_logs()
        - LOCK_thread_count is not protecting against rollback, so this
          code and comment is not needed
      - Remove mutex_locks in slave.cc that are not needed.
        Added THD::assert_not_linked() to ensure that it was safe to remove
      - Fixed not repeatable test load_data_stmt_view
      - Updated binlog_killed to test removal of mutex
        (thanks to Andrei Elkin for test)
      - More code comments
      c4581735
    • Varun Gupta's avatar
      6d63a034
  5. 07 Dec, 2017 5 commits
  6. 06 Dec, 2017 11 commits
    • Marko Mäkelä's avatar
      Merge bb-10.2-ext into 10.3 · 976f6fb1
      Marko Mäkelä authored
      976f6fb1
    • Marko Mäkelä's avatar
      Merge 10.2 into bb-10.2-ext · ce076765
      Marko Mäkelä authored
      ce076765
    • Marko Mäkelä's avatar
      Follow-up fix to MDEV-13201 Assertion `srv_undo_sources || ...` failed on... · 77fb7ccb
      Marko Mäkelä authored
      Follow-up fix to MDEV-13201 Assertion `srv_undo_sources || ...` failed on shutdown during DDL operation
      
      Introduce the debug flag trx_t::persistent_stats to suppress the
      assertion for the updates of persistent statistics during fast
      shutdown.
      
      dict_stats_exec_sql(): Do execute the statement even though shutdown
      has been initiated.
      77fb7ccb
    • Marko Mäkelä's avatar
      MDEV-14511 Use fewer transactions for updating InnoDB persistent statistics · 7dc6066d
      Marko Mäkelä authored
      dict_stats_exec_sql(): Expect the caller to always provide a transaction.
      Remove some redundant assertions. The caller must hold dict_sys->mutex,
      but holding dict_operation_lock is only necessary for accessing
      data dictionary tables, which we are not accessing.
      
      dict_stats_save_index_stat(): Acquire dict_sys->mutex
      for invoking dict_stats_exec_sql().
      
      dict_stats_save(), dict_stats_update_for_index(), dict_stats_update(),
      dict_stats_drop_index(), dict_stats_delete_from_table_stats(),
      dict_stats_delete_from_index_stats(), dict_stats_drop_table(),
      dict_stats_rename_in_table_stats(), dict_stats_rename_in_index_stats(),
      dict_stats_rename_table(): Use a single caller-provided
      transaction that is started and committed or rolled back by the caller.
      
      dict_stats_process_entry_from_recalc_pool(): Let the caller provide
      a transaction object.
      
      ha_innobase::open(): Pass a transaction to dict_stats_init().
      
      ha_innobase::create(), ha_innobase::discard_or_import_tablespace():
      Pass a transaction to dict_stats_update().
      
      ha_innobase::rename_table(): Pass a transaction to
      dict_stats_rename_table(). We do not use the same transaction
      as the one that updated the data dictionary tables, because
      we already released the dict_operation_lock. (FIXME: there is
      a race condition; a lock wait on SYS_* tables could occur
      in another DDL transaction until the data dictionary transaction
      is committed.)
      
      ha_innobase::info_low(): Pass a transaction to dict_stats_update()
      when calculating persistent statistics.
      
      alter_stats_norebuild(), alter_stats_rebuild(): Update the
      persistent statistics as well. In this way, a single transaction
      will be used for updating the statistics of a whole table, even
      for partitioned tables.
      
      ha_innobase::commit_inplace_alter_table(): Drop statistics for
      all partitions when adding or dropping virtual columns, so that
      the statistics will be recalculated on the next handler::open().
      This is a refactored version of Oracle Bug#22469660 fix.
      
      RecLock::add_to_waitq(), lock_table_enqueue_waiting():
      Do not allow a lock wait to occur for updating statistics
      in a data dictionary transaction, such as DROP TABLE. Instead,
      return the previously unused error code DB_QUE_THR_SUSPENDED.
      
      row_merge_lock_table(), row_mysql_lock_table(): Remove dead code
      for handling DB_QUE_THR_SUSPENDED.
      
      row_drop_table_for_mysql(), row_truncate_table_for_mysql():
      Drop the statistics as part of the data dictionary transaction.
      After TRUNCATE TABLE, the statistics will be recalculated on
      subsequent ha_innobase::open(), similar to how the logic after
      the above-mentioned Oracle Bug#22469660 fix in
      ha_innobase::commit_inplace_alter_table() works.
      
      btr_defragment_thread(): Use a single transaction object for
      updating defragmentation statistics.
      
      dict_stats_save_defrag_stats(), dict_stats_save_defrag_stats(),
      dict_stats_process_entry_from_defrag_pool(),
      dict_defrag_process_entries_from_defrag_pool(),
      dict_stats_save_defrag_summary(), dict_stats_save_defrag_stats():
      Add a parameter for the transaction.
      
      dict_stats_empty_table(): Make public. This will be called by
      row_truncate_table_for_mysql() after dropping persistent statistics,
      to clear the memory-based statistics as well.
      7dc6066d
    • Sergei Petrunia's avatar
      MDEV-14563: Wrong query plan for query with no PK · 2c1e4d4d
      Sergei Petrunia authored
      Part #2: Don't use the new code for the clustered PK, it is handled
      in the special way right above.
      2c1e4d4d
    • Sergei Petrunia's avatar
      MDEV-14563: Wrong query plan for query with no PK · a6254e5e
      Sergei Petrunia authored
      TABLE_SHARE::init_from_binary_frm_image() calls handler_file->index_flags()
      before it has set TABLE_SHARE::primary_key (it is 0 while it should be
      MAX_KEY in my example).
      This causes MyRocks to report wrong index flags (it thinks it's a PK while
      it is not), which causes invalid query plans later on.
      
      Do the only thing that seems feasible: adjust field->part_of key to have
      correct value in ha_rocksdb::open.
      a6254e5e
    • Sergei Petrunia's avatar
      MDEV-14433: RocksDB may show empty or incorrect output with rocksdb_strict_collation_check=off · c3803914
      Sergei Petrunia authored
      Part#1: Set field->part_of_key correctly for PK fields.
      c3803914
    • Marko Mäkelä's avatar
    • Marko Mäkelä's avatar
    • Marko Mäkelä's avatar
      b1cd5ca2
    • Marko Mäkelä's avatar
      e9bc0f75
  7. 05 Dec, 2017 5 commits
    • Marko Mäkelä's avatar
      Merge 10.1 into 10.2 · 1d526f31
      Marko Mäkelä authored
      1d526f31
    • Marko Mäkelä's avatar
      MDEV-14587 dict_stats_process_entry_from_defrag_pool() fails to call... · 63cbb982
      Marko Mäkelä authored
      MDEV-14587 dict_stats_process_entry_from_defrag_pool() fails to call dict_table_close() when index==NULL
      
      dict_stats_process_entry_from_defrag_pool(): Simplify the logic,
      and always call dict_table_close() when dict_table_open() returned
      a non-NULL handle.
      63cbb982
    • Heinz Wiesinger's avatar
      Add "leaves" algorithm to oqgraph. · a34b976d
      Heinz Wiesinger authored
      This algorithm returns all reachable leaf nodes from a given origin,
      or all root nodes that can reach a given destination.
      a34b976d
    • Marko Mäkelä's avatar
      MDEV-13670/MDEV-14550 Error log flood : "InnoDB: page_cleaner: 1000ms... · d1ab8903
      Marko Mäkelä authored
      MDEV-13670/MDEV-14550 Error log flood :  "InnoDB: page_cleaner: 1000ms intended loop took N ms. The settings might not be optimal."
      
      Silence the error log output that was introduced in MySQL 5.7
      (MariaDB 10.2.2) if log_warnings=2 or less.
      
      We should still figure out what these messages really indicate
      and how to solve the problems.
      
      pc_sleep_if_needed(): Add a parameter for the current time,
      so that there will be fewer successive calls to ut_time_ms()
      with no I/O between them.
      
      buf_flush_page_cleaner_coordinator(): Exit the first loop
      whenever shutdown has been requested. At the start of the loop,
      call ut_time_ms() only once. Do not display the message if
      log_warnings=2 or less.
      d1ab8903
    • Vesa Pentti's avatar
      Revert "MDEV-12501 -- set --maturity-level by default" · 5868a184
      Vesa Pentti authored
      This reverts commit 1af2d7ba.
      5868a184
  8. 04 Dec, 2017 6 commits
    • Vesa Pentti's avatar
      MDEV-12501 -- set --maturity-level by default · 1af2d7ba
      Vesa Pentti authored
        * Note: breaking change; since this commit, a plugin that has
          worked so far might get rejected due to plugin maturity
        * mariabackup is not affected (allows all plugins)
        * VERSION file defines SERVER_MATURITY, which defines the
          corresponding numeric value as SERVER_MATURITY_LEVEL in
          include/mysql_version.h
        * The default value for 'plugin_maturity' is SERVER_MATURITY_LEVEL - 1
        * Logs a warning if a plugin has maturity lower than
          SERVER_MATURITY_LEVEL
        * Tests suppress the plugin maturity warning
        * Tests use --plugin-maturity=unknown by default so as not to fail
          due to the stricter plugin maturity handling
      1af2d7ba
    • Marko Mäkelä's avatar
      Follow-up to MDEV-12698: Adjust some comments · 8be75480
      Marko Mäkelä authored
      The function dict_stats_update_if_needed() replaced
      row_update_statistics_if_needed(). Adjust the comments accordingly.
      8be75480
    • Varun Gupta's avatar
      MDEV-7773: Aggregate stored functions · 60c44658
      Varun Gupta authored
      This commit implements aggregate stored functions. The basic idea behind
      the feature is:
      
      * Implement a special instruction FETCH GROUP NEXT ROW that will pause
      the execution of the stored function. When the instruction is reached,
      execution of the initial query resumes "as if" the function returned.
      This gives the server the opportunity to advance to the next row in the
      result set.
      
      * Stored aggregates behave like regular aggregate functions. The
      implementation of thus resides in the class Item_sum_sp. Because it is
      an aggregate function, for each new row in the group, the
      Item_sum_sp::add() method will be called. This is when execution resumes
      and the function does another iteration to "add" one extra element to
      the final result.
      
      * When the end of group is reached, val_xxx() method will be called for
      the item. This case is handled by another execute step for the stored
      function, only with a special flag to force a call to the return
      handler. See Item_sum_sp::execute() for details.
      
      To allow this pause and resume semantic, we must preserve the function
      context across executions. This is stored in Item_sp::sp_query_arena only for
      aggregate stored functions, but has no impact for regular functions.
      
      We also enforce aggregate functions to include the "FETCH GROUP NEXT ROW"
      instruction.
      Signed-off-by: default avatarVicențiu Ciorbaru <vicentiu@mariadb.org>
      60c44658
    • Vicențiu Ciorbaru's avatar
      Remove the side effect of setting m_sp from Item_sp::init_result_field · 7448b01b
      Vicențiu Ciorbaru authored
      Item_sp::init_result_field no longer takes sp_head* parameter. It
      expects the m_sp member to be already set to something valid.
      7448b01b
    • Varun Gupta's avatar
      Refactor parts of Item_func_sp into Item_sp · c12d1ed4
      Varun Gupta authored
      In preparation for implementing custom aggregate functions, refactor
      the common code between regular stored functions and aggregate stored
      functions. This includes:
      
      * initialising SP result field
      * executing a SP
      * access checks
      
      In addition, refactor sp_head::execute_function to take two extra
      parameters, a function rcontext and a Query_arena. These two paremeters
      were initially initialised and destroyed within
      sp_head::execute_function, but for aggregate stored functions we will
      require control over their lifetime. The owner of these objects now
      becomes Item_sp.
      Signed-off-by: default avatarVicențiu Ciorbaru <vicentiu@mariadb.org>
      c12d1ed4
    • Marko Mäkelä's avatar