row_prebuilt_t::m_no_prefetch: Remove (it was always false).
row_prebuilt_t::m_read_virtual_key: Remove (it was always false).

Only ha_innopart ever set these fields.

innobase_rename_table(): Invoke dict_stats_wait_bg_to_stop_using_table()
to ensure that dict_stats_update() cannot be accessing the table name
that we will be modifying. If we are executing RENAME rather than TRUNCATE,
reset the flag at the end so that persistent statistics can be calculated
again.

The race condition was encountered with ASAN and rr.
Sorry, there is no test case, like there is for nothing related to
dict_stats_wait_bg_to_stop_using_table(). The entire code is an ugly
work-around for the failure of dict_stats_process_entry_from_recalc_pool()
to acquire MDL.

Note: It appears that an ALTER TABLE that is not rebuilding the table
will fail to reset the flag that blocks the processing of statistics.

row_number() over () window function can be used without any column in the OVER
clause. Additionally, the item doesn't reference any tables, as it's not
effectively referencing any table. Rather it is specifically built based
on the end temporary table used for window function computation.

This caused remove_const function to wrongly drop it from the ORDER
list. Effectively, we shouldn't be dropping any window function from the
ORDER clause, so adjust remove_const to account for that.

Reviewed by: Sergei Petrunia sergey@mariadb.com

In btr_index_rec_validate(), externally stored column
check is missing while matching the length of the field
with the length of the field data stored in record.
Fetch the length of the externally stored part and compare it
with the fixed field length.

MDEV-20648 InnoDB: Failing assertion: !(*node)->being_extended, innodb.log_data_file_size failed in buildbot, assertion `!space->is_stopping()'

InnoDB should check whether the tablespace is being deleted
while extending the tablespace.

When doing a truncate on an Innodb under lock tables, InnoDB would rename
the old table to #sql-... and recreate a new 't1' table. The table lock
would still be on the #sql-table.

When doing ALTER TABLE, Innodb would do the changes on the #sql table
(which would disappear on close).
When the SQL layer, as part of inline alter table, would close the
original t1 table (#sql in InnoDB) and then reopen the t1 table, Innodb
would notice that this does not match it's own (old) t1 table and
generate an error.

Fixed by adding code in truncate table that if we are under lock tables
and truncating an InnoDB table, we would close, reopen and lock the
table after truncate. This will remove the #sql table and ensure that
lock tables is using the new empty table.

Reviewer: Marko Mäkelä

Attempt to execute EXPLAIN statement on multi-table DELETE statement
leads to firing firing of the assertion
  DBUG_ASSERT(! is_set());
in the method Diagnostics_area::set_eof_status.

For example, above mentioned assertion failure happens
in case any of the following statements
  EXPLAIN DELETE FROM t1.* USING t1
  EXPLAIN DELETE b FROM t1 AS a JOIN t1 AS b
are executed in prepared statement mode provided the table t1
does exist.

This assertion is hit by the reason that a status of
Diagnostics_area is set twice. The first time it is set from
the function do_select() when the method multi_delete::send_eof()
called. The second time it is set when the method
Explain_query::send_explain() calls the method select_send::send_eof
(this method invokes the method Diagnostics_area::set_eof_status that
finally hits assertion)

The second invocation for a setter method of the class Diagnostics_area
is correct and run to send a response containing explain data.

But first invocation of a setter method of the class Diagnostics_area
is wrong since the function do_select() shouldn't be called at all
for handling of the EXPLAIN statement.

The reason by that the function do_select() is called during handling of
the EXPLAIN statement is that the flag SELECT_DESCRIBE not set in the
data member JOIN::select_options. The flag SELECT_DESCRIBE
if is copied from values select_lex->options.

During parsing of EXPLAIN statement this flag is set but latter reset
from the function reinit_stmt_before_use() that is called on
execution of prepared statement.
  void reinit_stmt_before_use(THD *thd, LEX *lex)
  {
    ...
    for (; sl; sl= sl->next_select_in_list())
    {
      if (sl->changed_elements & TOUCHED_SEL_COND)
      {
        /* remove option which was put by mysql_explain_union() */
        sl->options&= ~SELECT_DESCRIBE;
      ...
      }
   ...
  }

So, to fix the issue the flag SELECT_DESCRIBE is set forcibly at the
mysql_select() function in case thd->lex->describe set,
that is in case EXPLAIN being executed.

use _RR_TRACE_DIR=dir instead of -o dir, as the former can store
multiple traces in dir (if, e.g., the test restarts mysqld)

suppress uninitialized warning when $exe is undefined (--manual-XXX)

Running statements with SET STATEMENT FOR clause is handled incorrectly in
case the whole statement is executed in prepared statement mode.
For example, running of the following statement
  SET STATEMENT sql_mode = 'NO_ENGINE_SUBSTITUTION' FOR CREATE TABLE t1 AS SELECT CONCAT('abc') AS c1;
results in different definition of the table t1 depending on whether
the statement is executed as a prepared or as a regular statement.

In first case the column c1 is defined as
  `c1` varchar(3) DEFAULT NULL
in the last case the column c1 is defined as
  `c1` varchar(3) NOT NULL

Different definition for the column c1 arise due to the fact that
a value of the data memeber Item_func_concat::maybe_null depends on
whether strict mode is on or off. Below is definition of the method
fix_fields() of the class Item_str_func that is base class for the
class Item_func_concat that is created on parsing the
SET STATEMENT FOR clause.

bool Item_str_func::fix_fields(THD *thd, Item **ref)
{
  bool res= Item_func::fix_fields(thd, ref);
  /*
    In Item_str_func::check_well_formed_result() we may set null_value
    flag on the same condition as in test() below.
  */
  maybe_null= maybe_null || thd->is_strict_mode();
  return res;
}

Although the clause SET STATEMENT sql_mode = 'NO_ENGINE_SUBSTITUTION' FOR
is parsed on PREPARE phase during processing of the prepared statement,
real setting of the sql_mode system variable is done on EXECUTION phase.
On the other hand, the method Item_str_func::fix_fields is called on PREPARE
phase. In result, thd->is_strict_mode() returns true during calling the method
Item_str_func::fix_fields(), the data member maybe_null is assigned the value
true and column c1 is defined as DEFAULT NULL.

To fix the issue the system variables listed in the SET STATEMENT FOR clause
are set at the beginning of handling the PREPARE phase just right before
calling  the function check_prepared_statement() and their original values
restored immediate after return from this function.

Additionally, to avoid code duplication the source code used in the function
mysql_execute_command for setting variables, specified by SET STATEMENT
clause, were extracted to the standalone functions
run_set_statement_if_requested(). This new function is called from
the function  mysql_execute_command() and the method
Prepared_statement::prepare().

Backport of 4bc31a90

Include client libraries for auth caching_sha2_password and
sha256_password in the libmariadb3 client library package.

volatile != atomic.

volatile has no memory barrier schemantics, its for mmaped IO
so lets allow some optimizer gains and stop pretending it helps
with memory atomicity.

The MDEV lists a SEGV an assumption is made that an address was
partially read. As C packs structs strictly in order and on arm64 the
cache line size is 128 bits. A pointer (link - 64 bits), followed
by a hashnr (uint32 - 32 bits), leaves the following key (uchar *
64 bits), neither naturally aligned to any pointer and worse, split
across a cache line which is the processors view of an atomic
reservation of memory.

lf_dynarray_lvalue is assumed to return a 64 bit aligned address.

As a solution move the 32bit hashnr to the end so we don't get the
*key pointer split across two cache lines.

Tested by: Krunal Bauskar
Reviewer: Marko Mäkelä

Introduced by 85828b8f

This is running 2 git processes in parallel, which, if unlucky can cause
either of them to fail with "File already exists" error.

- The commit 5fd3c747(MDEV-24709)
resets the recv_no_ibuf_operations in
recv_recovery_from_checkpoint_start(), but InnoDB fails to reset
the variable recv_no_log_write() during that time and that leads
to the assert failure.

1. wait for the binlog thread to reach the certain state, don't use
   a debug_sync that's incorrectly placed to detect the state
2. no need to do a (non-deterministic) `show binlog events` to verify
   what is guaranteed by the directly preceding line

if it's the whole content of a test anyway.

add a new "debugger" to mtr, that runs the executable
under valgrind in gdb. valgrind pid is auto-detected,
but the delay (sleep) and vgdb path are hard-coded for now

"debugger" is anything that wraps execution of a target
binary (mysqld or mysqltest). Currently the list includes:
gdb, ddd, dbx, lldb, valgrind, strace, ktrace, rr,
devenv, windbg, vsjitdebugger.

for every debugger xxx, mtr will recognize four options:
--xxx, --boot-xxx, --manual-xxx, --client-xxx.
They all support an optional "=string" argument. String
being a semicolon-separated list of commands (e.g. for gdb)
or one (not semicolon-separated) command line of options
(e.g. for valgrind). Or both (e.g. --gdb='-quiet -nh;info files'

In embedded both --xxx and --client-xxx work.

Functionality changed/removed:
* --rr-args is gone
* --rr-dir is gone
* --manual-debug is gone
* --debugger={devenv|vc|windbg|vc_express|vsjitdebugger} is gone
* --strace-option is gone
* --stracer={strace|ktrace} is gone
* --valgrind only enables it for the server, not for everything
* --valgrind-all is gone
* --valgrind-mysqltest is gone
* --valgrind-mysqld is gone
* --valgrind-options is gone
* --valgrind-option is gone
* --valgrind-path is gone
* --callgrind is gone
* one cannot combine --valgrind --gdb anymore
* valgrind report doesn't add a fake test line to the output
* vc and vcexpress on windows are no longer supported

don't allocate Column_statistics_collected objects that won't
be used.

minor style fixes (StringBuffer<>, etc)

only collect persistent stats for columns explicitly listed
by the user in the  ANALYZE TABLE PERSISTENT FOR COLUMNS (...)
clause. The engine can extend table->read_set as much as
it wants, it should not affect the collected statistics.

Test case from the 3b94309a applies - it used to crash,
because ha_partition extended table->read_set after the loop that
initialized some objects based on bits in the read_set but before the
loop that used these objects based on bits in the read_set.

This reverts the commit 3b94309a but keeps the test

Because the fix is a hack that isn't supposed to do anything,
and relies on a side-effect of rnd_init inside ha_partition.

A different fix is coming up.

- This is caused by commit ad6171b9
(MDEV-22456). InnoDB reloads the evicted table again from dictionary.
In that case, AHI entries and current index object mismatches
happens. When index object mismatches then InnoDB should drop
the page hash AHI entries for the block. In
btr_search_drop_page_hash_index(), InnoDB should take exclusive
lock on the AHI latch if index is already freed to avoid the
freed memory access during buf_pool_resize()

this should simplify run-time cluster management

MDEV-24873 : galera.galera_as_slave_ctas MTR failed: Assertion `(&(&LOCK_thd_data)->m_mutex)->count > 0 && pthread_equal(pthread_self(), (&(&LOCK_thd_data)->m_mutex)->thread)' failed in sql_class.cc on THD::awake(killed_state)

Problem was that thd::awake assumes now that you hold THD::LOCK_thd_data
so we need to keep it when we call wsrep_thd_awake from
wsrep_abort_transaction.

Problem was that we tried to lock THD::LOCK_thd_data after we have
acquired lock_sys mutex. This is against mutex ordering rules.

Stabilize test case.

Let us avoid the excessive allocation of explicit record locks
(a work-around of MDEV-24813) so that the test will execute
much faster under AddressSanitizer, MemorySanitizer, Valgrind.

The issue here was the read_set bitmap was not set for a field which
was used as a reference in an inner select.
We need to make sure that if we are in an inner select and we have
references from outer select then we update the table bitmaps for
such references.

Introduced a function in the class Item_subselect that would
update bitmaps of table for the references within a
subquery that are defined in outer selects.

Follow-up fix to commit 26f50335(MDEV-23449)
The GROUP BY clause inside IN/ALL/ANY subquery is removed
when there is no aggregate function or HAVING clause in the subquery.

When the GROUP BY clause is removed, a subquery can also be removed
if it part of the GROUP BY clause. This is done inside the function
remove_redundant_subquery_clauses. Here we walk over the GROUP BY list
and remove a subselect from its unit via the callback function
eliminate_subselect_processor.

The issue here was that when the query was being re-executed it was trying
to reinitialize the select that was removed as stated above.
This is not required, so the fix would be to remove select_lex
both from tree lex structure and the global list of nodes so that
we don't do the reinitialization again.

The issue here was histogram statistics were being used even when
the level of optimizer_use_condition_selectivity doesn't allow
usage of statistics from histogram.

The histogram statistics are read for a table only when
optimizer_use_condition_selectivity > 3. But the TABLE structure can be
stored in the internal table cache and be reused for the next query.
So in this case the histogram statistics will be available for the next query.

The fix would be to make sure to use the histogram statistics only when
optimizer_use_condition_selectivity > 3.

The test innodb.innodb_bug60049 used to check that the record
(ID,NAME)=(12,'SYS_FOREIGN_COLS') is the last record in the
secondary index of the system table SYS_TABLES.
But, ever since commit 23365584
or mysql/mysql-server@082d59670f2616f68af37666fac0f23dbeb43099
that record no longer is the last one in the table!

The more recent test innodb.purge_secondary covers the purge
functionality much better.

innobase_rename_column_try(): When renaming SYS_FIELDS records
for secondary indexes, try to use both formats of SYS_FIELDS.POS
as keys, in case the PRIMARY KEY includes a column prefix.

Without this fix, an ALTER TABLE that renames a column followed
by a server restart (or LRU eviction of the table definition
from dict_sys) would make the table inaccessible.