It was possibile for a user to create an interlocked state which may go on
for a significant period of time. There is a tight loop in the FTWRL code
path that tries to repeatedly acquire a read lock. As the weight of FTWRL
lock is the smallest among others, it's always selected by the deadlock
detector, but can never be killed.

Imaging the following sequence:

    connection_0                 connection_1

 GET_LOCK("l1", 0);
                             LOCK TABLES t WRITE;
 FLUSH TABLES WITH READ LOCK;
                             GET_LOCK("l1", 1000);

The GET_LOCK statement in connection_1 triggers the deadlock detector,
which tries to select the lock in FTWRL, since its weight is 0. However,
since a loop in Global_read_lock::lock_global_read_lock() tries to always
win, it tries to acquire lock again. Which invokes the deadlock detector,
and that cycle continues until GET_LOCK in connection_1 times out.

This patch resolves the live-locking by introducing a dynamic bonus to the
deadlock weight associated with every lock. Each lock gets a bonus weight
each time it's selected by the deadlock detector. In case of a live-lock
situation, those locks that cannot be killed, get additional weight each
iteration. Eventually their weight becomes so high that the deadlock
detector shifts its attention to other lock, until it find the one that
can be killed.

		failed in dtuple_convert_big_rec

In dtuple_convert_big_rec(), InnoDB fails to consider the
instant metadata blob while choosing the variable length
field.

implements in a native way get_exepath which gives reliably the full
path.

not be dropped if the DEFINER is custom. Revert changes
to MDEV-23102 tests as they were designed to catch
this corner case.
The explanation for this corner case is that users
historically used to tweak the mysql.user table and
probably still do even though mysql.user is now a view.
Thus, if the DEFINER of the view is not default, i.e.
root@localhost or mariadb.sys@localhost, we should avoid
dropping the view during upgrade process to not discard
potential custom changes.

if a query used no fields from an I_S table, we were creating a temp
table with one, first, field (as a table cannot have zero fields),
with its length truncated to 1.

Now - force also this dummy field to be a normal field, not a BLOB

don't use `result` both for an error status and to remember
if the mutex was locked

if it's the whole content of a test anyway.

view definition in tests as it wastes lots of time when they fail
for minor fixes like adding/changing a new column in the view, also
it's less code

The mysql.user view password_expired column should display the right
result, in sync with whether an account has its password expired or not

For mariadb 10.4+ upgrades before this commit, the mysql.user view needs
to be dropped and recreated to actually make the view display the
correct value for the password_expired column.

This is fixup for commit f06a0b53.

Since we do not stop at corrupted page error, there is no reason to log
a backup error.

row_prebuilt_t::m_no_prefetch: Remove (it was always false).
row_prebuilt_t::m_read_virtual_key: Remove (it was always false).

Only ha_innopart ever set these fields.

innobase_rename_table(): Invoke dict_stats_wait_bg_to_stop_using_table()
to ensure that dict_stats_update() cannot be accessing the table name
that we will be modifying. If we are executing RENAME rather than TRUNCATE,
reset the flag at the end so that persistent statistics can be calculated
again.

The race condition was encountered with ASAN and rr.
Sorry, there is no test case, like there is for nothing related to
dict_stats_wait_bg_to_stop_using_table(). The entire code is an ugly
work-around for the failure of dict_stats_process_entry_from_recalc_pool()
to acquire MDL.

Note: It appears that an ALTER TABLE that is not rebuilding the table
will fail to reset the flag that blocks the processing of statistics.

row_number() over () window function can be used without any column in the OVER
clause. Additionally, the item doesn't reference any tables, as it's not
effectively referencing any table. Rather it is specifically built based
on the end temporary table used for window function computation.

This caused remove_const function to wrongly drop it from the ORDER
list. Effectively, we shouldn't be dropping any window function from the
ORDER clause, so adjust remove_const to account for that.

Reviewed by: Sergei Petrunia sergey@mariadb.com

The bug caused crashes of the server when processing queries with nested
table value constructors (TVC) . It happened because the grammar rules to
parse TVC used the same global lists for both nested TVC and nesting TVC.
As a result invalid select trees were constructed for queries with nested
TVC and this led to crashes at the prepare stage.
This patch provides its own lists structures for each TVC nest level.

Besides the patch fixes a bug in the function wrap_tvc() that missed
inheritance of the SELECT_LEX::exclude_from_table_unique_test for
selects that wrapped TVCs. This inheritance is critical for specifications
of derived tables that employ nested TVCs.

Approved by dmitry.shulga@mariadb.com

In btr_index_rec_validate(), externally stored column
check is missing while matching the length of the field
with the length of the field data stored in record.
Fetch the length of the externally stored part and compare it
with the fixed field length.

MDEV-20648 InnoDB: Failing assertion: !(*node)->being_extended, innodb.log_data_file_size failed in buildbot, assertion `!space->is_stopping()'

InnoDB should check whether the tablespace is being deleted
while extending the tablespace.

This is a backport of commit 18535a40
from 10.6.

lock_release(): Implement innodb_evict_tables_on_commit_debug.
Before releasing any locks, collect the identifiers of tables to
be evicted. After releasing all locks, look up for the tables and
evict them if it is safe to do so.

trx_commit_in_memory(): Invoke trx_update_mod_tables_timestamp()
before lock_release(), so that our locks will protect the tables
from being evicted.

When doing a truncate on an Innodb under lock tables, InnoDB would rename
the old table to #sql-... and recreate a new 't1' table. The table lock
would still be on the #sql-table.

When doing ALTER TABLE, Innodb would do the changes on the #sql table
(which would disappear on close).
When the SQL layer, as part of inline alter table, would close the
original t1 table (#sql in InnoDB) and then reopen the t1 table, Innodb
would notice that this does not match it's own (old) t1 table and
generate an error.

Fixed by adding code in truncate table that if we are under lock tables
and truncating an InnoDB table, we would close, reopen and lock the
table after truncate. This will remove the #sql table and ensure that
lock tables is using the new empty table.

Reviewer: Marko Mäkelä

Fixed binary logging in ANALYZE TABLE to work as optimize table

Attempt to execute EXPLAIN statement on multi-table DELETE statement
leads to firing firing of the assertion
  DBUG_ASSERT(! is_set());
in the method Diagnostics_area::set_eof_status.

For example, above mentioned assertion failure happens
in case any of the following statements
  EXPLAIN DELETE FROM t1.* USING t1
  EXPLAIN DELETE b FROM t1 AS a JOIN t1 AS b
are executed in prepared statement mode provided the table t1
does exist.

This assertion is hit by the reason that a status of
Diagnostics_area is set twice. The first time it is set from
the function do_select() when the method multi_delete::send_eof()
called. The second time it is set when the method
Explain_query::send_explain() calls the method select_send::send_eof
(this method invokes the method Diagnostics_area::set_eof_status that
finally hits assertion)

The second invocation for a setter method of the class Diagnostics_area
is correct and run to send a response containing explain data.

But first invocation of a setter method of the class Diagnostics_area
is wrong since the function do_select() shouldn't be called at all
for handling of the EXPLAIN statement.

The reason by that the function do_select() is called during handling of
the EXPLAIN statement is that the flag SELECT_DESCRIBE not set in the
data member JOIN::select_options. The flag SELECT_DESCRIBE
if is copied from values select_lex->options.

During parsing of EXPLAIN statement this flag is set but latter reset
from the function reinit_stmt_before_use() that is called on
execution of prepared statement.
  void reinit_stmt_before_use(THD *thd, LEX *lex)
  {
    ...
    for (; sl; sl= sl->next_select_in_list())
    {
      if (sl->changed_elements & TOUCHED_SEL_COND)
      {
        /* remove option which was put by mysql_explain_union() */
        sl->options&= ~SELECT_DESCRIBE;
      ...
      }
   ...
  }

So, to fix the issue the flag SELECT_DESCRIBE is set forcibly at the
mysql_select() function in case thd->lex->describe set,
that is in case EXPLAIN being executed.

           used in set function

If a subselect is formed by a table value constructor (TVC) then the
following transformation is applied at the prepare stage:
  VALUES (v1), ... (vn) => SELECT * FROM (VALUES (v1), ... (vn)) tvc_x.
The transformation is performed by the function wrap_tvc() that resets
THD::LEX::current select to the top level select of the result of the
transformation. After the call of wrap_tvc() in the function
Item_subselect::wrap_tvc_into_select() the field THD::LEX::current must be
reset to the same select as before the call. It was not done. As a result
if the subselect formed by a TVC was an argument of a set function then
an assertion was hit in the function Item_sum::check_sum_func().

Approved by Oleksandr Byelkin <sanja@mariadb.com>

Before the changes two things could happen:
- "path required name explain_filename path" error
- unit test never finishead (as it tried to execute just /bin/sh as
  a test case)

Fixes also:
MDEV-24942 Server crashes in _ma_rec_pack... with DEFAULT() on BLOB

This was caused by two different bugs, both related to that the default
value for the blob was not calculated before it was used:
- There where now Item_default_value::..result() wrappers, which is
  needed as item in HAVING uses these.  This causes crashes when
  using a reference to a DEFAULT(blob_field) in HAVING. It also
  caused wrong results when used with other fields with default value
  expressions that are not constants.
- create_tmp_field() did not take into account that blob fields with
  default expressions are not yet initialized. Fixed by treating
  Item_default_value(blob) like a normal item expression.

The failure happened for group by queries when all tables where marked as
'const tables' (tables with 0-1 matching rows) and no row matched the
where clause and there was in addition a direct reference to a field.

In this case the field would not be properly reset and the query would
return 'random data' that happended to be in table->record[0].

Fixed by marking all const tables as null tables in this particular case.

Sergei also provided an extra test case for the code.

@reviewer Sergei Petrunia <psergey@askmonty.org>

eprintf() was missing a va_start(), which caused wrong filename to be
printed when printing recovery trace.

Added also missing new line when printing "Table is crashed" to trace file

MDEV-24965 With ALTER USER ...IDENTIFIED BY command, password doesn't replaced by asterisks in audit log.

Test result fixed.

use _RR_TRACE_DIR=dir instead of -o dir, as the former can store
multiple traces in dir (if, e.g., the test restarts mysqld)

suppress uninitialized warning when $exe is undefined (--manual-XXX)

MDEV-24965 With ALTER USER ...IDENTIFIED BY command, password doesn't replaced by asterisks in audit log.

Check for the ALTER USER command added.

The reason for the crash was that there was not a write lock to
protect against file rotations in the server_audit plugin after an
audit plugin patch to changed audit mutexes to read & write locks.

The fixes are:
* Moving server_audit.c to use read & write locks (which improves
  performance).
* Added functionality in file_logger.c to not do file rotations until
  it is allowed by the caller (done without any interface changes for
  the logging service).
* Move checking of file size limit to server_audit.c and if it is time to
  do a rotation change the read lock to a write lock and tell file_logger
  that it is now allowed to rotate the log files.