If the backup finished in the middle of a Aria bulk load insert,
which could happen with LOAD DATA INFILE, CREATE ... SELECT etc)
there was a chance that Aria recovery would fail on the backup.

Fixed by ensuring that bulk load operations for Aria are not allowed
under BACKUP LOCK.
I also changed so that the table TRN is updated just before truncate
which ensures that old redo's for the table are ignored.
I also enabled Aria redo for DDL's to be able to repeat REPAIR commands.
Without this change recovery would not work on repaired tables.

Notes:
- We take the backup lock protection at the end of bulk insert (as we
  don't want to keep the lock over a very long running insert).
  If mariadb-backup keeps the backup lock too long,  this may fail with
  a lock timeout. In this case the batch insert will fail and the table
  will be truncated (set to it's original state).

btr_cur_t::open_random_leaf(): Replaces btr_cur_open_at_rnd_pos().
Acquire a shared latch on each page, and finally release all
latches except the one on the leaf page.

This fixes a race condition between the purge of history and
btr_estimate_number_of_different_key_vals(), which turned out
to only hold a buffer-fix on the randomly chosen leaf page.
Typically, an assertion would fail in page_rec_is_supremum().

ibuf_contract(): Start from the beginning of the change buffer,
to simplify the logic. Starting with
commit b42294bc
it does not matter much where the change buffer merge is being initiated.

The race condition may have been introduced as early as
mysql/mysql-server@ac74632293bea967b352d1b472abedeeaa921b98
from where it was copied to
commit 2e814d47.

Reviewed by: Vladislav Lesin
Tested by: Matthias Leich

The version was fixed to be Fedora 36 due to previous issues on
Gitlab-CI, but those seem to be solved now.

Use 'mariadb' name in scripts and server binary as Fedora switched name in
https://src.fedoraproject.org/rpms/mariadb/c/df76620f9e8a9b3f14da8a615050feeac2c62e26

Switch to using the `default:` section supported by newer Gitlab-CI,
see https://docs.gitlab.com/ee/ci/yaml/#default.

Also define an explicit timeout of 3 hours to ensure builds don't time
out if the default timeout is too short.

NOTE TO MERGERS: These changes are version independent and should be
merged up on all MariaDB branches 10.6 -> 10.11.

The cause of regression was handling for ROWNUM() function.
For queries like

  SELECT ROWNUM() FROM ... ORDER BY ...

ROWNUM() should be computed before the ORDER BY.
The computation was moved to be before the ORDER BY for any entries in
the select list that had RAND_TABLE_BIT set.

This had a negative impact on queries in form:

  SELECT sp_func() FROM t1 ORDER BY ... LIMIT n

where sp_func() is NOT declared as DETERMINISTIC (and so has
RAND_TABLE_BIT set).

The fix is to require evaluation for sorting only for the ROWNUM()
function. Functions that just have RAND_TABLE_BIT() can be computed
after ORDER BY ... LIMIT is applied.

(think about a possible index that satisfies the ORDER BY clause. In
that case, the the rows would be read in the needed order and we would
stop after reading LIMIT rows, achieving the same effect).

- Mariabackup fails to open the undo tablespaces while applying delta
files to the corresponding data file. Mariabackup opens the
undo tablespaces first time in srv_undo_tablespaces_init() and does
tries to open the undo tablespaces in xtrabackup_apply_deltas() with
conflicting mode and leads to the failure.

- Mariabackup should close the undo tablespaces before applying
the incremental delta files.

thd->lex->in_sum_func->max_arg_level cannot be set to a
bigger value of select->nest_level if select is null.

This fixes up 4eb8e51c
which missed a second parent.

recv_sys_t::recover_deferred(): If the file has been determined
to be large enough, skip the call to os_file_set_size(), which
would use the current value of FSP_SIZE, which during a multi-batch
recovery can be smaller than the actual file size.

os_file_io(): Also display the file offset in the warning message about
partial I/O.

os_file_read(): Merged with os_file_read_no_error_handling().
Crashing on a partial page read is as unhelpful as crashing on a
corrupted page read (commit 0b47c126).
Report the file name if it is available via IORequest.

The public data member can be checked directly by the only caller.

fil_space_t::prepare_acquired(): Do not attempt to extend (or shrink)
files that will be processed by recv_sys_t::recover_deferred().

Some builders in our CI, most notably FreeBSD and IBM AIX, do not support
sparse files. Also, Microsoft Windows requires special means for creating
sparse files. Since these platforms do not run ./mtr --big-test, we will
for now simply move the test to a separate file that requires that option.

- Information_schema.innodb_tablespaces_encryption should print
undo tablespace name as innodb_undo001, innodb_undo002 and soon.

- Encryption test should include undo tablespaces count when
the tests are waiting for the condition to check whether all
tables are encrypted or decrypted.

Add a build and test job for each of ASAN, MSAN, TSAN, and UBSAN to the
GitLab pipeline such that current vulnerabilities will be more easily
visible and on each new commit, we can ensure that there are no
additional errors introduced. Furthermore, sanitizer test runs are run
separate from the existing mysql-test-run to isolate sanitizer error
from functional errors.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.

This commit contains only a mtr test for reproducing the issue in MDEV-29512
The actual fix will be pushed in wsrep-lib repository

The hanging in MDEV-29512 happens when binlog purging is attempted, and there is
one local BF aborted transaction waiting for commit monitor.

The test will launch two node cluster and enable binlogging with expire log days,
to force binlog purging to happen.
A local transaction is executed so that will become BF abort victim, and has advanced
to replication stage waiting for commit monitor for final cleanup (to mark position in innodb)
after that, applier is released to complete the BF abort and due to binlog configuration,
starting the binlog purging. This is where the hanging would occur, if code is buggy
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Test MDEV-26575 fails when it runs after MDEV-25389. This is because
the latter simulates a failure while an applier thread is
created in `start_wsrep_THD()`. The failure was not handled correctly
and would not cleanup the created THD from the global
`server_threads`. A subsequent shutdown would hang and eventually fail
trying to close this THD.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Fix `wsrep_table_accessible_when_detached()` so that commands that
access no tables are rejected while a node is disconnected from a
cluster.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Prevent wsrep files from being installed if WITH_WSREP=OFF.

Reviewed by Daniel Black
Additionally excluded #include wsrep files and galera* files
along with galera/wsrep tests.

mysql-test/include/have_wsrep.inc remainds as its used by
a few isolated tests.
Co-authored-by: Chris Ross <cross2@cisco.com>

recv_log_recover_10_4(): Widen the operand of bitwise and to 64 bits,
so that the upgrade check will work when the redo log record is located
more than 4 gigabytes from the start of the first file.

ibuf_init_at_db_start(): Validate the change buffer root page.
A later version may stop creating a change buffer, and this
validation check will prevent a downgrade from such later versions.

ibuf_max_size_update(): If the change buffer was not loaded, do nothing.

dict_boot(): Merge the local variable "error" to "err". Ignore
failures of ibuf_init_at_db_start() if innodb_force_recovery>=4.

In commit a03dd94b as well as
mysql/mysql-server@6ef8c343445a26aaf9ebd76d72cf57db44b481f5
the iterations were changed so that the variable "scanned"
would remain 0 when the first list item qualifies for eviction.

buf_LRU_free_from_unzip_LRU_list(), buf_LRU_free_from_common_LRU_list():
Increment "scanned" when a block can be freed.

buf_LRU_free_from_common_LRU_list(): Remove a redundant condition.
Whenever this function is invoked, buf_pool.LRU should be nonempty,
hence something should always be scanned.

Thanks to Jean-François Gagné for reporting this.

The very lightest of load would decimate any buffer pool loading
to ~1 page per second. As seen in MDEV-29343 this resulting in
a load taking over an hour on a high end system.

Since MDEV-26547 the fetching is asynchronous, however the loading
has equal access to the IO as the SQL queries.

Signed-off-by: Weijun-H <huangweijun1001@gmail.com>

Signed-off-by: Weijun-H <huangweijun1001@gmail.com>

Problem:
========
There is a race condition in binlog.binlog_checkpoint between the
binlog background thread creating a binlog checkpoint event, and the
connection thread binlogging a query event for creating a table.
Because the test outputs the events for validation, the order
between these two events can be different, resulting in a failed
test.

Solution:
========
Instead of outputting the binlog events, use assert_grep to validate
the content of the binlog is correct.

Reviewed By:
============
Andrei Elkin <andrei.elkin@mariadb.com>

A few query cache functions don't use THD pointer
so its removed from their interface.

Fix the regression introduced in
dfb41fdd.

In the restructure of mysql_rm_table_no_locks the early condition
of !frm_error that enabled non_tmp_table_deleted, and hence the
query cache invalidation, was removed.

The query_cache_invalidate1(thd, dbnorm) called after
mysql_rm_table_no_locks depends on the query cache removal
(for unexamined reasons).

Under DROP DATABASE, in mysql_rm_table_no_locks, dont_log_query
is true preventing the late setting of non_tmp_table_deleted
(which retained one of its purposes as a replication deletion
of temporary tables, but not query cache invalidation).

The non_temp_tables_count however can still be used to invalidate
the query cache.