innodb_preshutdown(): Terminate the encryption threads before
the page cleaner thread can be shut down.

innodb_shutdown(): Always wait for the encryption threads and
page cleaner to shut down.

srv_shutdown_all_bg_threads(): Wait for the encryption threads and
the page cleaner to shut down. (After an aborted startup,
innodb_shutdown() would not be called.)

row_get_background_drop_list_len_low(): Remove.

os_thread_count: Remove. Alternatively, at the end of
srv_shutdown_all_bg_threads() we could try to wait longer
for the count to reach 0. On some platforms, an assertion
os_thread_count==0 could fail even after a small delay,
even though in the core dump all threads would have exited.

srv_shutdown_threads(): Renamed from srv_shutdown_all_bg_threads().
Do not wait for the page cleaner to shut down, because the later
innodb_shutdown(), which may invoke
logs_empty_and_mark_files_at_shutdown(), assumes that it exists.

storage engines are generally initialized in some random order
(by iterating the hash of plugin names).

S3 fails if it's initialized before Aria.

But it looks that while S3 needs Aria, it does not need
Aria to be initialized before S3. S3 copies maria_hton and then
overwrites every single member of it, so it can handle Aria being
initialized after S3.

fix TokuDB to compile with the perfschema

view_pseudo_hton cannot be dereferenced

first try to resolve the function name as a native function,
and only then look within pluggable data types

Fix the Item_func_json_objectagg::fix_fields to save the orig_args.

Ubuntu Xenial

Fixes 7f613ebd (MDEV-7284 INDEX: CREATE OR REPLACE).

We must make the test depend on a debug version of Galera.

In commit 7eda5561 we removed a
valid debug assertion.

AddressSanitizer would trip when writing undo log for an INSERT
operation that follows the problematic ALTER TABLE because the
v_indexes would refer to an index object that was freed during
the execution of ALTER TABLE.

The operation to remove NOT NULL attribute from a column should
lead into any affected indexes to be dropped and re-created.
But, the SQL layer set the ha_alter_info->handler_flags to
HA_INPLACE_ADD_UNIQUE_INDEX_NO_WRITE|ALTER_COLUMN_NULLABLE
(that is, InnoDB was asked to add an index and change the column,
but not to drop the index that depended on the column).

Let us restore the debug assertion that catches this problem
outside AddressSanitizer, and 'defuse' the offending ALTER TABLE
statement in the test until something has been fixed in the SQL layer.

MDEV-23935: Fix warnings generated during compilation of plugin/auth_pam/testing/pam_mariadb_mtr.c on MacOS

During build server 10.4 on MacOS warnings like the following one
are generated on compiling the file plugin/auth_pam/testing/pam_mariadb_mtr.c

server-10.4/plugin/auth_pam/testing/pam_mariadb_mtr.c:25:22: error:
initializing 'char *' with an expression of type 'const char [23]'
discards qualifiers [-Werror,-Wincompatible-pointer-types-discards-qualifiers]
{ PAM_TEXT_INFO, "Challenge input first." }

The reason of the warnings is that the data member pam_message::msg is declared
as 'char *' on MacOS but initializer of the data member is 'const char *'.

To eliminate warnings the compiler option
  -Wno-incompatible-pointer-types-discards-qualifiers
has been added to the compiler flags used for compiling the file
pam_mariadb_mtr.c.

MDEV-20945: BACKUP UNLOCK + FTWRL assertion failure | SIGSEGV in I_P_List
from MDL_context::release_lock on INSERT w/ BACKUP LOCK (on optimized
builds) | Assertion `ticket->m_duration == MDL_EXPLICIT' failed

BACKUP LOCK behavior is modified so it won't be used wrong:
- BACKUP LOCK should commit any active transactions.
- BACKUP LOCK should not be allowed in stored procedures.
- When BACKUP LOCK is active, don't allow any DDL's for that connection.
- FTWRL is forbidden on the same connection while BACKUP LOCK is active.

Reviewed-by: monty@mariadb.com

Some functions on ha_partition call functions on all partitions, but handler->reset() is only called that pruned by m_partitions_to_reset. So Spider didn't clear pointer on unpruned partitions, if the unpruned partitions are used by next query, Spider reference the pointer that is already freed.

Some functions on ha_partition call functions on all partitions, but handler->reset() is only called that pruned by m_partitions_to_reset. So Spider didn't clear pointer on unpruned partitions, if the unpruned partitions are used by next query, Spider reference the pointer that is already freed.

Change xarecover_handlerton so that transaction with WSREP prefixed
xids are rolled back when Galera is disabled.
Reviewd-by: Jan Lindström <jan.lindstrom@mariadb.com>

Compiler warnings like one listed below are generated during server build on MacOS:
In file included from server-10.2-MDEV-23564/mysys_ssl/openssl.c:33:
In file included from /usr/local/include/openssl/evp.h:16:
In file included from /usr/local/include/openssl/bio.h:20:
/usr/local/include/openssl/crypto.h:206:10: warning: 'CRYPTO_cleanup_all_ex_data' macro redefined [-Wmacro-redefined]
           ^
  /mariadb/server-10.2-MDEV-23564/include/ssl_compat.h:46:9: note: previous definition is here
          ^

In case MariaDB serer is build with -DCMAKE_BUILD_TYPE=Debug it results in
build error.

The reason of compiler warnings is that header file <ssl_compat.h>
included before the openssl system header files. File ssl_compat.h
contains some macros with the same names as SSL API functions declared
in the openssl system header files. It resulted in duplicate
symbols that produces compiler warnings.

To fix the issue the header file ssl_compat.h should be included
after a line where openssl system header is included.

It was only from CMake-3.14.0 that CMAKE_REQUIRED_LINK_OPTIONS
was used in CHECK_CXX_SOURCE_COMPILES. Without this, it could be
the case (as was on OSX) that a flag was never checked in
CHECK_CXX_SOURCE_COMPILES, the CHECK successfully passed, but
failed at link time.

As such we use CMAKE_REQUIRED_LIBRARIES to include the flags to check
as its compatible enough with the cmake versions for non-Windows
compilers/linkers.

Tested on x86_64 with:
* 3.11.4
* 3.17.4

Corrects: 7473e184

In the future:
* cmake >=3.14.0 can use CMAKE_REQUIRED_LINK_OPTIONS
* cmake >=3.18.0 can use CHECK_LINKER_FLAG (with policy CMP0057 NEW)
(e.g: commit c7ac2deff9a2c965887dcc67cbf2a3a7c3e0123d)

CMAKE_REQUIRED_LIBRARIES suggested by serg@mariadb.com

Reviewed-by: anel@mariadb.org

MDEV-22871 tried to optimize the buf_page_t initialization in
buf_page_init_for_read() by initializing everything while the block
is in freed state, and only afterwards attaching the block to
buf_pool.page_hash.

In an rr replay trace, we have multiple threads executing in
buf_page_optimistic_get() on the same buf_block_t while the block is
being freed and reallocated several times in buf_page_init_for_read().
Because also the buf_page_t::id() is changing, the buf_pool.page_hash
is being protected by a different rw-lock than the one that
buf_page_optimistic_get() are successfully read-locking.

buf_page_optimistic_get(): Validate also buf_page_t::id() after
acquiring the buf_pool.page_hash latch.

Reviewed by: Thirunarayanan Balathandayuthapani

In MDEV-12353, we changed the redo log format so that whenever
any data field is changed, the unchanged prefix of the bytes
will by default be omitted from the redo log record.

However, in order for the MDEV-23199 logic to work, changes of
FSP_SPACE_FLAGS must always be written in full (mtr_t::FORCED),
so that crash recovery can parse them. We must know the flags
in advance, because we might be applying log to other pages than
page 0 first.

recv_scan_log_recs(): Correct a misleading message about starting
crash recovery. Display both the checkpoint LSN and the start
of the records that end in the FILE_CHECKPOINT marker.

Reviewed by: Thirunarayanan Balathandayuthapani

The only applicable InnoDB change to MariaDB that was made
between MySQL 5.6.49 and MySQL 5.6.50 is MDEV-23999.

fts_query_t::nested_sub_exp: Keep track of nested
fts_ast_visit_sub_exp() calls.

fts_ast_visit_sub_exp(): Return DB_OUT_OF_MEMORY if the
maximum recursion depth is exceeded.

This is motivated by a change in MySQL 5.6.50:
mysql/mysql-server@e2a46b4834a0030e5380d45789772cb6beab6e92
Bug #29929684 USING MANY NESTED ARGUMENTS WITH BOOLEAN FTS CAN LEAD
TO TERMINATE SERVER

Updated submodule libmarias3 which fixes the problem
Enabled disable S3 tests.

- Indentation changes
- Fixed wrong name for used in DBUG_ENTER
- Added some code comments

This failure was caused because of several bugs:
- Someone had removed s3-slave-ignore-updates=1 from slave.cnf, which
  caused the slave to remove files that the master was working on.
- Bug in ha_partition::change_partitions() that didn't reset m_new_file
  in case of errors. This caused crashes in ha_maria::extra() as the
  maria handler was called on files that was already closed.
- In ma_pagecache there was a bug that when one got a read error one a
  big block (s3 block), it left the flag PCBLOCK_BIG_READ on for the page
  which cased an assert when the page where flushed.
- Flush all cached tables in case of ignored ALTER TABLE

Note that when merging code from 10.3, that fixes the partition bug, use
the code from this patch instead.

Changes to ma_pagecache.cc written or reviewed by Sanja

This commit fixed the problems with S3 after the "DROP TABLE FORCE" changes.
It also fixes all failing replication S3 tests.

A slave is delayed if it is trying to execute replicated queries on a
table that is already converted to S3 by the master later in the binlog.

Fixes for replication events on S3 tables for delayed slaves:
- INSERT and INSERT ... SELECT and CREATE TABLE are ignored but written
  to the binary log.   UPDATE & DELETE will be fixed in a future commit.

Other things:
- On slaves with --s3-slave-ignore-updates set, allow S3 tables to be
  opened in read-write mode. This was done to be able to
  ignore-but-replicate queries like insert.  Without this change any
  open of an S3 table failed with 'Table is read only' which is too
  early to be able to replicate the original query.
- Errors are now printed if handler::extra() call fails in
  wait_while_tables_are_used().
- Error message for row changes are changed from HA_ERR_WRONG_COMMAND
  to HA_ERR_TABLE_READONLY.
- Disable some maria_extra() calls for S3 tables. This could cause
  S3 tables to fail in some cases.
- Added missing thr_lock_delete() to ma_open() in case of failure.
- Removed from mysql_prepare_insert() the not needed argument 'table'.

Other things:
- Updated help text for --gdb

In row_undo_ins_remove_clust_rec() and similar places,
an assertion !node->trx->dict_operation_lock_mode could fail,
because an online ALTER is not allowed to run at the same time
while DDL operations on the table are being rolled back.

This race condition would be fixed by always acquiring an InnoDB
table lock in ha_innobase::prepare_inplace_alter_table() or
prepare_inplace_alter_table_dict(), or by ensuring that recovered
transactions are protected by MDL that would block concurrent DDL
until the rollback has been completed.

This reverts commit 15093639
and commit 22c4a751.

The function innodb_show_mutex_status() is the only ultimate caller of
LatchCounter::iterate() via MutexMonitor::iterate(). Because the call
is not protected by LatchCounter::m_mutex, any mutex_create() or
mutex_free() that is invoked concurrently during the execution, bad
things such as a crash could happen.

The most likely way for this to happen is buffer pool resizing,
which could cause buf_block_t::mutex (which existed before MDEV-15053)
to be created or freed. We could also register InnoDB mutexes in
TrxFactory::init() if trx_pools needs to grow.

The view INFORMATION_SCHEMA.INNODB_MUTEXES is not affected, because it
only displays information about rw-locks, not mutexes.

This commit intentionally touches also MutexMonitor::iterate()
and the only code that interfaces with LatchCounter::iterate()
to make it clearer for future readers that the scattered code
that is obfuscated by templates belongs together.

This is based on
mysql/mysql-server@273a93396f49c7e0a8b07b260128d9a990c2b154