When the query using a recursive CTE whose definition contained wildcard
symbols in the recursive part was processed at the prepare stage an
assertion was hit if the query was executed without any default database
set. The failure happened when the function insert_fields() tried to check
column privileges for the temporary table created for a recursive
reference to the CTE. No acl checks are needed for any CTE. That's why this
check should be blocked as well. The patch formulates a stricter condition
at which this check is to be blocked that covers the case when a query
using recursive CTEs is executed with no default database set.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

For table references to CTEs the field TABLE_LIST::db must be set to
an empty string as it's done for table references to derived tables in
order CTEs to be processed similar to how derived tables are processed.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

MDEV-24139: CHECK_CLAUSE field in INFORMATION_SCHEMA.CHECK_CONSTRAINTS truncate check constraints expressions

- Reviewed by: daniel@mariadb.org

Introduced with 6b7918d5 in `10.2` just check for helper and handle
it if exist.

Reviewed by: cvicentiu@mariadb.org

- MDEV-24177: main.sp2 test fails: Result length mismatch
- MDEV-24178: main.upgrade_MDEV-19650 test fails: Result length mismatch

Reviewed by: serg@mariadb.com

           mergeable derived table

Do not check privileges for derived tables/CTEs and their fields.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

innobase_space_shutdown(): Remove. We want this step to be executed
before the message "InnoDB: Shutdown completed; log sequence number "
is output by innodb_shutdown(). It used to be executed after that step.

innodb_shutdown(): Duplicate the code that used to live in
innobase_space_shutdown().

innobase_init_abort(): Merge with innobase_space_shutdown().

events.

The log line should be added behind the filters.

The new option --log-innodb-page-corruption is introduced.

When this option is set, backup is not interrupted if innodb corrupted
page is detected. Instead it logs all found corrupted pages in
innodb_corrupted_pages file in backup directory and finishes with error.

For incremental backup corrupted pages are also copied to .delta file,
because we can't do LSN check for such pages during backup,
innodb_corrupted_pages will also be created in incremental backup
directory.

During --prepare, corrupted pages list is read from the file just after
redo log is applied, and each page from the list is checked if it is allocated
in it's tablespace or not. If it is not allocated, then it is zeroed out,
flushed to the tablespace and removed from the list. If all pages are removed
from the list, then --prepare is finished successfully and
innodb_corrupted_pages file is removed from backup directory. Otherwise
--prepare is finished with error message and innodb_corrupted_pages contains
the list of the pages, which are detected as corrupted during backup, and are
allocated in their tablespaces, what means backup directory contains corrupted
innodb pages, and backup can not be considered as consistent.

For incremental --prepare corrupted pages from .delta files are applied
to the base backup, innodb_corrupted_pages is read from both base in
incremental directories, and the same action is proceded for corrupted
pages list as for full --prepare. innodb_corrupted_pages file is
modified or removed only in base directory.

If DDL happens during backup, it is also processed at the end of backup
to have correct tablespace names in innodb_corrupted_pages.

The reason for the failure is that
thd->mdl_context.release_transactional_locks()
was called after commit & rollback even in cases where the current
transaction is still active.

For 10.2, 10.3 and 10.4 the fix is simple:
- Replace all calls to thd->mdl_context.release_transactional_locks() with
  thd->release_transactional_locks(). The thd function will only call
  the mdl_context function if there are no active transactional locks.
  In 10.6 we will better fix where we will change the return value for
  some trans_xxx() functions to indicate if transaction did close the
  transaction or not. This will avoid the need of the indirect call.

Other things:
- trans_xa_commit() and trans_xa_rollback() will automatically
  call release_transactional_locks() if the transaction is closed.
- We can't do that for the other functions as the caller of many of these
  are doing additional work (like close_thread_tables) before calling
  release_transactional_locks().
- Added missing abort_result_set() and missing DBUG_RETURN in
  select_create::send_eof()
- Fixed wrong indentation in injector::transaction::commit()

Reviewed by:serg@mariadb.com

This is a fixup patch for MDEV-23991 afc9d00c

We really should read result.n_leaf_pages, which was set previously.

Analysis and fix was provided by Jukka Santala. Thanks!

Reviewed by: Marko Mäkelä

The nonnull attribute is not applicable to parameters that are
passed by reference, at least not in the Intel compiler.
Let us remove the reference indirection, which was only there
so that the pointer could be assigned to NULL, and let the
callers perform that task.

row_log_allocate(): Fix a bug in out-of-memory error handling
that would leave a pointer to freed memory.

In galera_3nodes.galera_safe_to_bootstrap node restart can happen too soon, when earlier SST joiner process is still active in the node.
Similar issue may hurt other mtr tests as well.

This is second variant of fix for this issue. Here we only change rsync SST script to wait a little bit if lingering SST rsync is observed to be in execution.
We assume that the previous mysqld and SST processes have been already signaled to abort during earlier stataup attempt.

If other SST methods (than rsync) suffer from similar overlapping SST execution, they should be sorted out separately within each SST method handler scripts.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Add test case

A bogus error message was issued when a condition was pushed into a
materialized derived table or view specified as union of selects with
aggregation when the corresponding columns of the selects had different
names. This happened because the expression pushed into having clauses of
the selects was adjusted for the names of the first select of the union.
The easiest solution was to rename the columns of the other selects to be
name compatible with the columns of the first select.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

Explicitly setting encoding to UTF-8 when writing to file and
replacing wide characters from MTR_RES_FAILED when writing to
XML file. The wide characters are not allowed in XML.

FindBlockX::operator(): Return false if an x-latched block is found.
Previously, we were incorrectly returning false if the block was in
the log, only if not x-latched.

It is unknown if this mistake had any visible impact. Often,
we would register both MTR_MEMO_BUF_FIX and MTR_MEMO_PAGE_X_FIX
for the same block.

           in TABLE_LIST::is_recursive_with_tables

After the patch for MDEV-23619 the code of st_select_lex::cleanup started
using the list st_select_lex::leaf_tables. This list is built for any
query with FROM clause in the function setup_tables(). If such query is
used in a stored procedure it must be ensured that the list is empty
before each new call of the procedure. Otherwise if the first call of
the procedure is successful while the second call reports an error before
the setup_tables() is invoked then list st_select_lex::leaf_tables would
point to a piece of memory that has been already freed.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

Add wait_condition.

Add primary key and wait condition.

Add wait_conditions.

Reviewer: Brad Smith

The fix of MDEV-23456 (commit b1009ae5)
introduced a livelock between page flushing and a thread that is
executing buf_page_create().

buf_page_create(): If the current mini-transaction is holding
an exclusive latch on the page, do not attempt to acquire another
one, and do not care about any I/O fix.

mtr_t::have_x_latch(): Replaces mtr_t::get_fix_count().

dyn_buf_t::for_each_block(const Functor&) const: A new variant.

rw_lock_own(): Add a const qualifier.

Reviewed by: Thirunarayanan Balathandayuthapani

Due to a premature cleanup of the unit that specified a recursive CTE
used in the second operand of union the server fell into an infinite
loop in the reported test case. In other cases this premature cleanup
could cause other problems.
The bug is the result of a not quite correct fix for MDEV-17024. The
unit that specifies a recursive CTE has to be cleaned only after the
cleanup of the last external reference to this CTE. It means that
cleanups of the unit triggered not by the cleanup of a external
reference to the CTE must be blocked.
Usage of local table chains in selects to get external references to
recursive CTEs was not correct either because of possible merges of
some selects.

Also fixed a minor bug in st_select_lex::set_explain_type() that caused
typing 'RECURSIVE UNION' instead of 'UNION' in EXPLAIN output for external
references to a recursive CTE.

Analysis:
========
Writes to 'rli->log_space_total' needs to be synchronized, otherwise both
SQL_THREAD and IO_THREAD can try to modify the variable simultaneously
resulting in incorrect rli->log_space_total.  In the current test scenario
SQL_THREAD is trying to decrement 'rli->log_space_total' in 'purge_first_log'
and IO_THREAD is trying to increment the 'rli->log_space_total' in
'queue_event' simultaneously. Hence test occasionally fails with  result
mismatch.

Fix:
===
Convert 'rli->log_space_total' variable to atomic type.

As noted in commit 0b66d3f7,
MariaDB does not support CREATE TABLESPACE for InnoDB.
Hence, some code that was added in
commit fec844ac
and originally in
mysql/mysql-server@c71dd213bd434c0579e454ab8880e6d3756b0fb0
is unused in MariaDB and should be removed.

trx_undo_left(): Return 0 in case of an overflow, instead of
returning a negative number interpreted as a large positive number.
Also, add debug assertions to check that the pointer is within
the page area. This should allow us to catch bugs like
MDEV-24096 easier in the future.

The function ibuf_merge_or_delete_for_page() was always being
invoked with update_ibuf_bitmap=true ever since
commit cd623508
fixed up something after MDEV-9566.

Furthermore, the parameter page_size is never being passed as a
null pointer, and therefore it should better be a reference to
a constant object.

- Patch is solving generating report on warning
        To repeat the error run single worker:
        ```
        ./mtr --mysqld=--lock-wait-timeout=-xx 1st 1st --force --parallel 1
        ```
        or `N` workers with `N+1` tests with failures and `force`
        ```
        ./mtr --mysqld=--lock-wait-timeout=-xx 1st 1st grant5 --force --parallel 2
        ```
- Patch is doing cosmetic fix of `current_test` log file which holds the old log value of test `CURRENT TEST:..` in `mark_log()` in case of `unknown option` and as such
  the logic which is using it's content doesn't output valid log content and doesn't generate valid `$test->{'comment'}` message.asdf

- Closing the socket/handler after the removing the handler from IO for
consistency

Reviewed by: serg@mariadb.com

The row_log_t::mutex is a mutex, yet it was instrumented as
rw-lock in PERFORMANCE_SCHEMA.

This bug could manifest itself for a query with WHERE condition containing
top level OR formula such that each conjunct contained a single-range
condition supported by the same index. One of these range conditions must
be fully covered by another range condition that is used later in the OR
formula. Additionally at least one of these condition should be ANDed with
a sargable range condition supported by a different index.

There were several attempts to fix related problems for OR conditions after
the backport of range optimizer code from MySQL (commit
0e19f3e3). Unfortunately the first of these
fixes contained typo remained unnoticed until recently. This typo bug led
to rejection of valid range accesses. This patch fixed this typo bug.
The fix revealed another two bugs: one in a constructor for SEL_ARG,
the other in the function tree_or(). Both are fixed in this patch.

Add a testcase.

Part#1: Revert the patch that caused it:

commit 291be494
Author: Igor Babaev <igor@askmonty.org>
Date:   Thu Sep 24 22:02:00 2020 -0700

    MDEV-23811: With large number of indexes optimizer chooses an inefficient plan