Also update the SEL_ARG graph weight in:
- sel_add()
- SEL_ARG::clone()

Make key_{and,or}_with_limit() to also verify weight for the arguments
(There is no single point to verify SEL_ARG graphs constructed from
conditions that are not AND-OR formulas, so we hope that those are
connected with AND/OR and do it here).

The purpose of the test was to ensure that the SX (update) mode of
index tree and buffer page latches are being used.

The test has become unstable, possibly due to changes related to
buf_pool.mutex and buf_pool.page_hash, or to the use of MDL in the
purge of transaction history.

In 10.6, the test depends on instrumentation that was refactored
or removed in MDEV-24142.

The use of different latching modes can better be indirectly observed
through high-concurrency benchmarks. For MDEV-14637, a performance test
was conducted where the finer-grained latching and
BTR_CUR_FINE_HISTORY_LENGTH were removed. It caused a 20% performance
regression for UPDATE and somewhat smaller for INSERT.

Any new problem with latching granularity should be easily caught by
performance testing, or by stress tests with Random Query Generator.

There are no IO threads anymore.

(Variant #5, full patch, for 10.5)

Do not produce SEL_ARG graphs that would yield huge numbers of ranges.
Introduce a concept of SEL_ARG graph's "weight". If we are about to
produce a graph whose "weight" exceeds the limit, remove the parts
of SEL_ARG graph that represent the biggest key parts. Do so until
the graph's is within the limit.

Includes
- debug code to verify SEL_ARG graph weight
- A user-visible @@optimizer_max_sel_arg_weight to control the optimization
- Logging the optimization into the optimizer trace.

The implementation for MDEV-17048 apperas to be direct copy from mysql version.
The group commit works differently in mariadb and the assert in wsrep_unregister_from_group_commit() is too strict.

The reason is that in: Wsrep_high_priority_service::log_dummy_write_set(), the transaction will undergo full rollback:
    {
      cs.before_rollback();
      cs.after_rollback();
    }

After that, the client's transaction state is set to be:  wsrep::transaction::s_aborted.
The execution then continues execution by:

...
 wsrep_register_for_group_commit(m_thd);
...
 wsrep_unregister_from_group_commit(m_thd);

The bogus assert in wsrep_unregister_from_group_commit() allows only transactions states of :s_ordered_commit or s_aborting.

As the fix, I brought back the same assert as is present in MariaDB 10.4 version.

a. The change makes `mariadb-upgrade` detect if `MYSQL_JSON` data type is needed.
b. Install the data type if it's not installed.
c. Uninstalls the data type once finished.
d. Create `.opt` and `.inc` files `have_type_mysql_json` and adapt the
tests

Reviewed by: vicentiu@mariadb.org

Ever since commit 007f68c3,
ALTER TABLE no longer invokes handler::open() after
handler::commit_inplace_alter_table().

ha_innobase::reload_statistics(): Reload or recompute statistics
after ALTER TABLE.

innodb_notify_tabledef_changed(): A new function to invoke
ha_innobase::reload_statistics().

handlerton::notify_tabledef_changed(): Add the parameter handler*
so that ha_innobase::reload_statistics() can be invoked.

ha_partition::notify_tabledef_changed(),
partition_notify_tabledef_changed(): Pass through the call
to any partitions or subpartitions.

This is based on code that was supplied by Monty.

The check is added.

During recovery, InnoDB fails if it tries to apply a FREE_PAGE
and WRITE record to the page. InnoDB encryption thread accesses
the freed page and writes redo log for it.

This is similar to commit deadec4e (MDEV-24569)
InnoDB is missing buf_page_free() while freeing the segment.
To avoid accessing of freed page in buffer pool, InnoDB should
mark the pages as FREED while freeing the segment. Also to
avoid reading of freed page, InnoDB should check the
allocation bitmap page.

fseg_free_step(): Mark the page in buffer pool as FREED

fseg_free_step_not_header(): Mark the page in buffer pool as FREED

buf_dump(): Ignore the freed pages while dumping the buffer pool content

fil_crypt_get_page_throttle_func(): Skip the rotation for FREED page
to avoid the assert failure during recovery

fil_crypt_rotate_page(): Skip the rotation for the freed page

Reviewed-by: Marko Mäkelä

MDEV-24693 LeakSanitizer: detected memory leaks in mem_heap_create_block_func / fts_optimize_create_msg

- This issue is caused by the commit bf1f9b59
(MDEV-24638). Delay the creation of SYNC message in
fts_optimize_request_sync_table. So that InnoDB can avoid creating
the message if the table already has SYNC message in fts_optimize_wq queue

* galera
* galera_sr
* galera_3nodes

Correct test.

the top level

We may end up with an empty leaf page (containing only an ADD COLUMN
metadata record) that is not the root page.

innobase_add_instant_try(): Disable an optimization for a non-canonical
empty table that contains a metadata record somewhere else than in
the root page.

btr_pcur_store_position(): Tolerate a non-canonical empty table.

MDEV-24659 Assertion !fsp_is_system_temporary(bpage->id().space()) failed in buf_flush_relocate_on_flush_list()

When commit 5eb53955 (MDEV-12227)
removed the pages of temporary tables from the buf_pool.flush_list,
an adjustment to the buffer pool resizing was forgotten.

buf_pool_t::realloc(): Do not invoke buf_flush_relocate_on_flush_list()
for pages that belong to the temporary tablespace. Also, deduplicate
some code at the end.

buf_page_t::set_corrupt_id(): Tolerate oldest_modification()==1
(the dummy value) for temporary tablespace pages. The revised
buf_pool_t::realloc() may invoke this on dirty temporary tablespace pages.

Ever since commit 947efe17
InnoDB no longer writes binlog position in one place.
It will not at all be written to the TRX_SYS page, and
instead it will be written to the undo log header page that
changes the transaction state.

trx_rseg_mem_restore(): Recover the information from the latest
written page.

fts_optimize_request_sync_table() can avoid the repetitive
FTS SYNC request of the table if the table already has FTS_SYNC
message in fts_optimize_wq queue.

Reviewed-by: Marko Mäkelä

MDEV-24463 : galera.galera_sst_mysqldump_with_key MTR failed: 'INSERT failed: 1213: Deadlock found when trying to get lock

We need to complete SST if both new and old start positions are
not same as initial positions. If they are initial positions
just set local uuid and seqno.

While reusing the cached undo log block, mtr expects the page
write to change while writing the trx id. cached undo log block
could contain bytes which were originally written for some other
transaction. So InnoDB should make mtr to do MAYBE_NOP while reusing
cached undo log block.

Reviewed-by: Marko Mäkelä

I_S tables were materialized too late, an attempt to use table
statistics before the table was created caused a crash.

Let's move table creation up. it only needs read_set to
be calculated properly, this happens in JOIN::optimize_inner(),
after semijoin transformation.

Note that tables are not populated at that point, so most of the
statistics would make no sense anyway. But at least field sizes
will be correct. And it won't crash.

The problem was that update_timing_fields_for_event() didn't release all
MDL locks it took.

MDEV-24596 : Assertion `state_ == s_exec || state_ == s_quitting' failed in wsrep::client_state::disable_streaming

There were multiple problems here
* wsrep_trx_fragment_size should not be set when wsrep is disabled or provider is not loaded
* wsrep_trx_fragment_unit should not be set when wsrep is disabled or provider is not loaded
* wsrep_debug has no effect if wsrep is disabled or provider is not loaded
* wsrep_start_position should not be set when wsrep is disabled or provider is not loaded any other value than default
* wsrep_start_position should be changed only when we are joiner or initialized
* wsrep_start_position should be allowed to set only a value that exits, thus
we need to add error handling to wsrep_sst_complete

Fix for MDEV-23033 fixes a problem in replication applying of transactions, which contain cascading foreign key delete for a table, which has indexed virtual column.
This fix adds slave_fk_event_map flag for table, to mark when the prelocking is needed for applying of a transaction.
See commit 608b0ee5 for more details.
However, this fix is targeted for async replication only, Rows_log_event::do_apply_event() has condition to rule out galera replication from the fix domain, and use cases suffering from MDEV-23033 and related MDEV-21153 will fail in galera cluster.

The fix in this commit removes the condition to rule out the setting of slave_fk_event_map flag from galera replication, and makes the fix in MDEV-23033 effective for galera replication as well.

However, the above fix has caused regressions for some galera_sr suite tests, which run tests for streaming replication.
This regression can be observed e.g. by: /mtr galera_sr.galera_sr_multirow_rollback  --mysqld=--slave_run_triggers_for_rbr=yes
These galera_sr suite tests were failing in last phase of replication applying, where actual transaction is already applied, and streaming replication related meta data needs to be updated in wsrep system tables.
Opening the wsrep system tables failed for corrupt data in THD::lex:query_tables_list. The fix in this commit uses back query table list for the duration of fragment update operation.

Finally, a mtr test for virtual column support has been added. galera.galera_virtual_column.test has as first test a scenario from MDEV-21153

new fix
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Fix for MDEV-23033 fixes a problem in replication applying of transactions, which contain cascading foreign key delete for a table, which has indexed virtual column.
This fix adds slave_fk_event_map flag for table, to mark when the prelocking is needed for applying of a transaction.
See commit 608b0ee5 for more details.
However, this fix is targeted for async replication only, Rows_log_event::do_apply_event() has condition to rule out galera replication from the fix domain, and use cases suffering from MDEV-23033 and related MDEV-21153 will fail in galera cluster.

The fix in this commit removes the condition to rule out the setting of slave_fk_event_map flag from galera replication, and makes the fix in MDEV-23033 effective for galera replication as well.

Finally, a mtr test for virtual column support has been added. galera.galera_virtual_column.test has as first test a scenario from MDEV-21153
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

MDEV-24577: Fix warnings generated during compilation of plugin/auth_pam/testing/pam_mariadb_mtr.c on FreeBSD

Compiler warnings generated on building MariaDB server for BSD has the same
reason as in case building is performed on MacOS. Both platforms do use
clang as a C/C++ compiler. So, fix the compiler warnings in case the compiler
is clang doesn't matter what kind of building platform do we use for building.

This is a follow-up patch for the following bug reports:
  MDEV-23564: CMAKE failing due to deprecated Apple GSS method
  MDEV-23935: Fix warnings generated during compilation of
              plugin/auth_pam/testing/pam_mariadb_mtr.c on MacOS

Do not init encryption threads if shutdown is in progress.