In commit 2647fd10 (MDEV-19445)
we fixed a race condition around the INFORMATION_SCHEMA tables
that access the table identified by the global variable
innodb_ft_aux_table. Thanks to MDEV-16678 we could fix it
even better by using MDL instead of the InnoDB dict_sys latches.

Before entering DML or DDL execution in the storage engine, the SQL layer
will have acquired metadata lock (MDL) on the current table name as well
as the names of FOREIGN KEY (grand)child tables (that is,
tables whose REFERENCES clauses point to the current table).
The MDL prevents any metadata changes to these tables, such as
RENAME, TRUNCATE, DROP, ALTER.

While the MDL on the current table prevents dict_table_t::foreign_set
from being modified, it does not prevent the table metadata that the
stored pointers are pointing to from being modified.

The MDL on the child tables will prevent both dict_table_t::referenced_set
as well as the pointed child table metadata from being modified.

wsrep_row_upd_index_is_foreign(): Do not unnecessarily acquire the
data dictionary latch if Galera replication is not enabled.

ha_innobase::can_switch_engines(): Rely on MDL. We are not dereferencing
any pointers stored in the sets.

row_mysql_freeze_data_dictionary(), row_mysql_unfreeze_data_dictionary():
Remove.

row_update_for_mysql(): Call init_fts_doc_id_for_ref() only once.

In ALTER TABLE...IMPORT TABLESPACE and FLUSH TABLES...FOR EXPORT
the SQL layer is protecting the current table with MDL. We do not
need InnoDB latches.

row_undo(): Remove the unnecessary acquisition and release of
dict_sys.latch. This was supposed to prevent the table from being
dropped while the undo log record is being rolled back. But, thanks to
trx_resurrect_table_locks() that was introduced in
mysql/mysql-server@935ba09d52c1908bde273ad1940b5ab919d9763d
and commit c291ddfd
as well as commit 1bd681c8
(MDEV-25506 part 3) tables will be protected against dropping
due to table locks.

This reverts commit 0049d5b5
(which had reverted a previous attempt of fixing this) and addresses
an earlier race condition with the following:

prepare_inplace_alter_table_dict(): If recovered transactions hold
locks on the table while we are executing online ADD INDEX, acquire
a table lock so that the rollback of those recovered transactions will
not interfere with the ADD INDEX.

The InnoDB internal SQL parser never supported this syntax.

Since commit 1bd681c8 tablespaces are
dropped after the table has been dropped. No dict_sys latch protection
is relevant during the execution of fil_delete_tablespace().

Since commit 1bd681c8
dict_stats_exec_sql() is no longer playing dirty tricks.
We can remove the workaround in lock_release().

fts_lock_table(): Fix a regression that was introduced in
commit da65cb4d (MDEV-25936).

Changes:
changed mysqld -> mariadbd for some more error messages that were left.

Merge commit 2b6f8044 introduced
the debug injection point dict_sys_mutex_avoid to make the test
innodb.instant_alter_crash work even after the MDEV-23991 fix
(commit afc9d00c).

Thanks to DDL being atomic and crash-safe in MariaDB 10.6
(mainly thanks to commit 7762ee5d)
we do not actually need this hack anymore.

trx_flush_log_if_needed_low(): Do flush for both
innodb_flush_log_at_trx_commit=1 and
innodb_flush_log_at_trx_commit=3.

This is the 10.6 version of
10.2 commit 5f8651ac
which fixed a regression that had been introduced
in 2e814d47
(breaking 288eeb3a).

AIX error message didn't include the filename like:

2021-07-22  9:43:04 0 [ERROR] mariadbd: Can't create/write to file '' (Errcode: 20 "Not a directory")
2021-07-22  9:43:04 0 [ERROR] mariadbd: Can't create/write to file '' (Errcode: 20 "Not a directory")

Reviewer: Marko

row_ins_index_entry_set_vals(): Remove an assertion that trivially
holds because the 16-bit dict_col_t::len cannot represent the value
UNIV_PAGE_SIZE_MAX.

MDEV-26193 exposed a race condition in the test
mariabackup.log_page_corruption by no longer waking up purge tasks
on every transaction commit. (Note that there never was a guarantee
that the purge tasks would actually run as quickly as the test expected
it to happen; that would depend on the CPU load.)

Starting with commit 6e12ebd4
(MDEV-25062), srv_wake_purge_thread_if_not_active() became
more expensive operation, especially on NUMA systems, because
instead of reading an atomic global variable trx_sys.rseg_history_len
we are traversing up to 128 cache lines in trx_sys.history_exists().

trx_t::commit_cleanup(): Do not wake up purge at all.
We will wake up purge about once per second in srv_master_callback().

srv_master_do_active_tasks(), srv_master_do_idle_tasks():
Move some duplicated code to srv_master_callback().

srv_master_callback(): Invoke purge_coordinator_timer_callback()
to ensure that purge will be periodically woken up, even if the
latest execution of trx_t::commit_cleanup() allowed the purge view
to advance but did not wake up purge.
Do not call log_free_check(), because every thread that is going
to generate redo log is supposed to call that function anyway,
before acquiring any page latches. Additional calls to the function
every few seconds should not make any difference.

srv_shutdown_threads(): Ensure that srv_shutdown_state can be at most
SRV_SHUTDOWN_INITIATED in srv_master_callback(), by first invoking
srv_master_timer.reset() before changing srv_shutdown_state.
(Note: We first terminate the srv_master_callback and only then
terminate the purge tasks. Thus, the purge subsystem should exist
when srv_master_callback() invokes purge_coordinator_timer_callback()
if it was initiated in the first place.

It is implementation-defined whether alignment requirements
that are larger than std::max_align_t (typically 8 or 16 bytes)
will be honored by the compiler and linker.

It turns out that on IBM AIX, both alignas() and MY_ALIGNED()
only guarantees alignment up to 16 bytes.

For some data structures, specifying alignment to the CPU
cache line size (typically 64 or 128 bytes) is a mere performance
optimization, and we do not really care whether the requested
alignment is guaranteed.

But, for the correct operation of direct I/O, we do require that
the buffers be aligned at a block size boundary.

field_ref_zero: Define as a pointer, not an array.
For innochecksum, we can make this point to unaligned memory;
for anything else, we will allocate an aligned buffer from the heap.
This buffer will be used for overwriting freed data pages when
innodb_immediate_scrub_data_uncompressed=ON. And exactly that code
hit an assertion failure on AIX, in the test innodb.innodb_scrub.

log_sys.checkpoint_buf: Define as a pointer to aligned memory
that is allocated from heap.

log_t::file::write_header_durable(): Reuse log_sys.checkpoint_buf
instead of trying to allocate an aligned buffer from the stack.

The lazy deletion of clean blocks from buf_pool.flush_list that was
introduced in commit 6441bc61 (MDEV-25113)
introduced a race condition around the function
buf_flush_relocate_on_flush_list().

The test innodb_zip.wl5522_debug_zip as well as the buffer pool
resizing tests would occasionally fail in debug builds due to
buf_pool.flush_list.count disagreeing with the actual length of the
doubly-linked list.

The safe procedure for relocating a block in buf_pool.flush_list should be
as follows, now that we implement lazy deletion from buf_pool.flush_list:

1. Acquire buf_pool.mutex.
2. Acquire the exclusive buf_pool.page_hash.latch.
3. Acquire buf_pool.flush_list_mutex.
4. Copy the block descriptor.
5. Invoke buf_flush_relocate_on_flush_list().
6. Release buf_pool.flush_list_mutex.

buf_flush_relocate_on_flush_list(): Assert that
buf_pool.flush_list_mutex is being held. Invoke
buf_page_t::oldest_modification() only once, using
std::memory_order_relaxed, now that the mutex protects us.

buf_LRU_free_page(), buf_LRU_block_remove_hashed(): Avoid
an unlock-lock cycle on hash_lock. (We must not acquire hash_lock
while already holding buf_pool.flush_list_mutex, because that
could lead to a deadlock due to latching order violation.)

pmull_supported is not necessarily defined before crc32c_aarch64
Signed-off-by: Nia Alarie <nia@NetBSD.org>

SHOW PLUGINS has a more complete view of the installed
plugins into the server.

The mysql.user is a compatibility view that doesn't
show the complete authentication picture. Use global_priv.

Add `show create user` for default users to more clearly
represent its contents.

In MDEV-26029 the intention was that page_compressed tables would
be written as regular (non-sparse) files if the file is stored on
a thinly provisioned block device.

We were incorrectly requesting os_file_set_size() to create
sparse files even on thinly provisioned storage.

fil_space_extend_must_retry(): Extend the file in the correct fashion.

fil_ibd_create(), recv_sys_t::recover_deferred(): Only create a
sparse file for page_compressed tables if thin provisioning is not
detected.

fil_ibd_create(): Remove code that should have been removed in
commit 86dc7b4d already.
We no longer wrote an initialized page to the file, but we would
still allocate a page image in memory and write it.

xb_space_create_file(): Remove an unnecessary page write.
(This is a functional change for Mariabackup.)

We only support the instantaneous removal of the NOT NULL attribute for
ROW_FORMAT=REDUNDANT tables.

There were several places where a statement delimiter missed so
such statements were interpreted as multi-statements and expectedly failed
in PS mode. An appropriate statement delimiters have been added
to fix the issues. Addinitinally, the operators
 --enable_prepare_warnings/--disable_prepare_warnings have been added
around statements that use depricated syntax SELECT INTO to don't
miss warnings.

The performance_schema data related to instrument "wait/synch/mutex/threadpool/group_mutex" was incorrect. The root cause is the group_mutex was initialized in thread_group_init before registerd in PSI_register(mutex).
The fix is to put PSI_register before thread_group_init, which ensures the right order.

The root cause of test failure is that on optimization
of the statement clause 'order by (select 1)' in outer select
is done incorrect in case the statement run in PS mode.

COM_STMT_BULK_EXECUTE, just like COM_STMT_EXECUTE can also skip result set
metadata, if bulk is used with statement that returns result set, i.e
INSERT/DELETE RETURNING.

commit the forgotten result file

followup for b81803f0

still cannot be enabled permanently, but at least they
could be run manually, if needed

Also, remove comparison lsn > flush/write lsn, prior to calling
log_write_up_to. The checks and early returns are part of this function.

In case the test main.opt_trace is run with the option --ps-protocol
it fails since querying from the table INFORMATION_SCHEMA.OPTIMIZER_TRACE
produces an output that differed from the expected one in the following way:
  @@ -2829,14 +2829,6 @@
                     }
                   },
                   {
  -                  "transformation": {
  -                    "select_id": 2,
  -                    "from": "IN (SELECT)",
  -                    "to": "semijoin",
  -                    "chosen": true
  -                  }
  -                },
  -                {
                     "expanded_query": "/* select#2 */ select t10.pk from t10"
                   }

The table INFORMATION_SCHEMA.OPTIMIZER_TRACE is filled when optimizer_trace is on.
The reason of missing above mentioned pieces in query result set is that
the C++ macros
  OPT_TRACE_TRANSFORM(thd, trace_wrapper, trace_transform,
                      select_lex->select_number,
                      "IN (SELECT)", "semijoin");
located in the standalone function check_and_do_in_subquery_rewrites()
is executed twice in case the statement
  explain extended select * from t1 where a in (select pk from t10);
is run in PS mode. The first time it is executed on PREPARE phase and
the second time on EXECUTE phase. The output produced by this macros on
EXECUTE phase rewrites the output produced on PREPARE phase.
In result test failed in case it was run in PS mode.

To make test output uniform regardless the test is run in PS or normal
mode the operator '--source include/protocol.inc' has been added to
the file opt_trace.test and extra opt_trace,ps.rdiff file has been added.

Additionally, added operators
  --enable_prepared_warnings/--disable_prepared_warnings
in order to store warnings in result file that received on PREPARE phase
during running the statemement 'SELECT INTO'.