This is a complete rewrite of DROP TABLE, also as part of other DDL,
such as ALTER TABLE, CREATE TABLE...SELECT, TRUNCATE TABLE.

The background DROP TABLE queue hack is removed.
If a transaction needs to drop and create a table by the same name
(like TRUNCATE TABLE does), it must first rename the table to an
internal #sql-ib name. No committed version of the data dictionary
will include any #sql-ib tables, because whenever a transaction
renames a table to a #sql-ib name, it will also drop that table.
Either the rename will be rolled back, or the drop will be committed.

Data files will be unlinked after the transaction has been committed
and a FILE_RENAME record has been durably written. The file will
actually be deleted when the detached file handle returned by
fil_delete_tablespace() will be closed, after the latches have been
released. It is possible that a purge of the delete of the SYS_INDEXES
record for the clustered index will execute fil_delete_tablespace()
concurrently with the DDL transaction. In that case, the thread that
arrives later will wait for the other thread to finish.

HTON_TRUNCATE_REQUIRES_EXCLUSIVE_USE: A new handler flag.
ha_innobase::truncate() now requires that all other references to
the table be released in advance. This was implemented by Monty.

ha_innobase::delete_table(): If CREATE TABLE..SELECT is detected,
we will "hijack" the current transaction, drop the table in
the current transaction and commit the current transaction.
This essentially fixes MDEV-21602. There is a FIXME comment about
making the check less failure-prone.

ha_innobase::truncate(), ha_innobase::delete_table():
Implement a fast path for temporary tables. We will no longer allow
temporary tables to use the adaptive hash index.

dict_table_t::mdl_name: The original table name for the purpose of
acquiring MDL in purge, to prevent a race condition between a
DDL transaction that is dropping a table, and purge processing
undo log records of DML that had executed before the DDL operation.
For #sql-backup- tables during ALTER TABLE...ALGORITHM=COPY, the
dict_table_t::mdl_name will differ from dict_table_t::name.

dict_table_t::parse_name(): Use mdl_name instead of name.

dict_table_rename_in_cache(): Update mdl_name.

For the internal FTS_ tables of FULLTEXT INDEX, purge would
acquire MDL on the FTS_ table name, but not on the main table,
and therefore it would be able to run concurrently with a
DDL transaction that is dropping the table. Previously, the
DROP TABLE queue hack prevented a race between purge and DDL.
For now, we introduce purge_sys.stop_FTS() to prevent purge from
opening any table, while a DDL transaction that may drop FTS_
tables is in progress. The function fts_lock_table(), which will
be invoked before the dictionary is locked, will wait for
purge to release any table handles.

trx_t::drop_table_statistics(): Drop statistics for the table.
This replaces dict_stats_drop_index(). We will drop or rename
persistent statistics atomically as part of DDL transactions.
On lock conflict for dropping statistics, we will fail instantly
with DB_LOCK_WAIT_TIMEOUT, because we will be holding the
exclusive data dictionary latch.

trx_t::commit_cleanup(): Separated from trx_t::commit_in_memory().
Relax an assertion around fts_commit() and allow DB_LOCK_WAIT_TIMEOUT
in addition to DB_DUPLICATE_KEY. The call to fts_commit() is
entirely misplaced here and may obviously break the consistency
of transactions that affect FULLTEXT INDEX. It needs to be fixed
separately.

dict_table_t::n_foreign_key_checks_running: Remove (MDEV-21175).
The counter was a work-around for missing meta-data locking (MDL)
on the SQL layer, and not really needed in MariaDB.

ER_TABLE_IN_FK_CHECK: Replaced with ER_UNUSED_28.

HA_ERR_TABLE_IN_FK_CHECK: Remove.

row_ins_check_foreign_constraints(): Do not acquire
dict_sys.latch either. The SQL-layer MDL will protect us.

This was reviewed by Thirunarayanan Balathandayuthapani
and tested by Matthias Leich.

We must invoke online_retry_drop_indexes() before
ha_innobase_inplace_ctx::trx->id is assigned.

The implementation of MDEV-24626 was not entirely correct.
We could occasionally fail to remove some *.ibd files on recovery.

deferred_spaces: Keep track of FILE_DELETE records.

deferred_spaces.add(): Do not allow duplicate file names.

recv_rename_files(): Preserve some of renamed_spaces entries for
deferred_spaces.reinit_all().

Thanks to Thirunarayanan Balathandayuthapani for noticing that
deferred_spaces.add() must filter out duplicate file names,
as well as some debugging help.

Occasionally, the test innodb.alter_copy would fail in MariaDB 10.6.1,
reporting DB_MISSING_HISTORY during CHECK TABLE. It started to occur during
the development of MDEV-25180, which introduced purge_sys.stop_SYS().
If we delay purge more during DDL operations, then the test would
almost always fail. The reason is that during startup we will restore
a purge view, and CHECK TABLE would still use REPEATABLE READ
even though innodb_read_only is set and other isolation levels
than READ UNCOMMITTED are not guaranteed to work.

ha_innobase::check(): Use READ UNCOMMITTED isolation level if
innodb_read_only is set or innodb_force_recovery exceeds 3.

dict_set_corrupted(): Do not update the persistent data dictionary
if innodb_force_recovery exceeds 3.

MDEV-25857: MTR should report at least last test that was executed in case of shutdown and not-completed

- Report the test name in case not all tests are completed and server
closed the connection
- Rerport the failure of the last test with the server log in case of
server shutdown.
- Ignore stackdump files (obtained on Windows).

Reviewed by: wlad@mariadb.com

If temporary internal table is in use `hton` will not be set. Skip check
if DDL should be replicated in this case.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

buf_read_ibuf_merge_pages(): If space->size is 0, invoke
fil_space_get_size() to determine the size of the tablespace
by reading the header page. Only after that proceed to delete
any entries that are beyond the end of the tablespace.
Otherwise, we could be deleting valid entries that actually
need to be applied.

This fixes a regression that had been introduced in
commit b80df9eb (MDEV-21069),
which aimed to avoid crashes during DROP TABLE of corrupted tables.

No fix for this bug is needed starting from version 10.4.

If a select query contained an ORDER BY clause that followed a LIMIT clause
or an ORDER BY clause or ORDER BY with LIMIT the EXPLAIN output for the
query showed an execution plan different from that was actually executed.

Approved by Roman Nozdrin <roman.nozdrin@mariadb.com>

- Support building ARM64 packages
- require InstallerVersion 5.0 -required for arm64, also
it is there already on Windows 7

Hard to do a test case, but tested by hand and verified that mysql_upgrade
will update the encrypted MariaDB tables.

This happended when an aria table was already used by the system before
running zerofill, which could happen with Aria system tables.

Fixed by using a proper page type when reading pages in zerofill

ibuf_read_merge_pages(): Disable some code that was added in MDEV-20394
in order to avoid a server hang if the change buffer is corrupted,
presumably due to the race condition in recovery that was later fixed in
MDEV-24449. The code will still be available in debug builds when
the command line option --debug=d,ibuf_merge_corruption is specified.

Due to MDEV-19514, the impact of this code is much worse starting
with the 10.5 series. In older versions, the code was only enabled
during a shutdown with innodb_fast_shutdown=0, but in 10.5 it was
active during the normal operation of the server.

Problem was that Field_newdate() didn't allocate a string big enough for
the result.

Problem was that str->alloc(length) needed a buffer of length+1 as
decimals2string() will add an end null.

This is because on different compilation and server configurations the
memory usage is different and the query can get killed in different places
with different error messages as a result.

Reviewer: None (trival change)

This fixed the MySQL bug# 20338 about misuse of double underscore
prefix __WIN__, which was old MySQL's idea of identifying Windows
Replace it by _WIN32 standard symbol for targeting Windows OS
(both 32 and 64 bit)

Not that connect storage engine is not fixed in this patch (must be
fixed in "upstream" branch)

This partially reverts commit d7321893.

The *.jar files are not being built and all Debian builds are failing
as dh_install stops on missing files. To build them we would need to also
add new Java build dependencies.

In a stable release (10.2->10.5) we shouldn't add new files and certainly
not any new build dependencies, so reverting commit.

Also, the files are located in a different path, and already included
in the mariadb-test-data package:

  /usr/share/mysql/mysql-test/plugin/connect/connect/std_data/JavaWrappers.jar
  /usr/share/mysql/mysql-test/plugin/connect/connect/std_data/JdbcMariaDB.jar
  /usr/share/mysql/mysql-test/plugin/connect/connect/std_data/Mongo2.jar
  /usr/share/mysql/mysql-test/plugin/connect/connect/std_data/Mongo3.jar

This change needs to be redesigned and applies only on 10.6 or newer.

The fix is to quote service name parameter, when it is passed to
mysql_upgrade_service subprocess.

* Clean up autobake-deb.sh

  - No need to define any TokuDB rules, there is no such package
  - No need to define RocksDB arch, it already has "Architecture:" line
  - No need to dh-systemd backwards compat stanza, neither Debian Jessie
    nor Ubuntu Xenial has any new MariaDB 10.5 releases anymore
  - Minor spelling fixes

* Ensure dch runs non-interactively so builds pass with new dch version

  A recent version of dch (available in Ubuntu Hirsute and Debian Bullseye)
  had a change in behaviour that it started prompting if the DEBEMAIL or
  EMAIL variable as unset, asking for confirmation. We can't have anything
  interactive in our build scripts, so prevent this prompt by giving
  --controlmaint to the command, so it always uses the name and email from
  the debian/control file and does not prompt anything.

  The command-line argument has been around for a long time, so it is safe
  to use on all Debian/Ubuntu builds we have.

  See https://manpages.debian.org/jessie/devscripts/dch.1.en.html

  Since MariaDB 10.5 is the oldest release we still release for Ubuntu Hisute
  and Debian Bullseye, merge this on 10.5 and from there merge up to latest.
  No need to consider 10.2, 10.3 and 10.4 as those will not be released for
  Ubuntu Bullseye or Ubuntu Hirsute.

* Minor Salsa-CI cleanup

  - Fix spelling (synced from downstream Debian)

* Many minor spelling fixes (synced from downstream Debian)

ha_heap::external_lock contains some consistency checks for the table,#
in a debug compilation.

This code suffers from lack of synchronization, in a rare case
where mysql_lock_tables() fail, and unlock is forced, even if lock was
not previously taken.

To workaround, require EXTRA_DEBUG compile definition in order to activate
the consistency checks.The code still might be useful in some cases - but
the audience are developers looking for errors in single-threaded scenarios,
rather than multiuser stress-tests.

Commit b5615eff introduced comment in result file during shutdown.
In case of Windows for the tests involving `file_key_managment.so` as plugin-load-add the tests will be overwritten with .dll extension.
The same happens with environment variable `$FILE_KEY_MANAGMENT_SO`.
So the patch is removing the extension to be extension agnostic.

Reviewed by: wlad@mariadb.com

MDEV-25777: JAVA_INCLUDE_PATH and JAVA_INCLUDE_PATH2 not found with out of source configuration and Ninja generator

- As solution `PLUGIN_CONNECT=NO` use early check to disable plugin:
  Solution suggested by wlad@mariadb.com
- `JNI_FOUND` is a internal result variable and should be set with
cached library and header variables (like `JAVA_INCLUDE_PATH`) defined.
  * Note: wrapper cmake/FindJNI.cmake runs first time and cmake native Find<module> returns only cached variable, like `JAVA_INCLUDE_PATH`, results variable are not cached).

Reviewed by: serg@mariadb.com

If a join query uses a derived table (view / CTE) with GROUP BY clause then
the execution plan for such join may employ split optimization. When this
optimization is employed the derived table is not materialized. Rather only
some partitions of the derived table are subject to grouping. Split
optimization can be applied only if:
- there are some indexes over the tables used in the join specifying the
  derived table whose prefixes partially cover the field items used in the
  GROUP BY list (such indexes are called splitting indexes)
- the WHERE condition of the join query contains conjunctive equalities
  between columns of the derived table that comprise major parts of
  splitting indexes and columns of the other join tables.

When the optimizer evaluates extending of a partial join by the rows of the
derived table it always considers a possibility of using split optimization.
Different splitting indexes can be used depending on the extended partial
join. At some rare conditions, for example, when there is a non-splitting
covering index for a table joined in the join specifying the derived table
usage of a splitting index to produce rows needed for grouping may be still
less beneficial than usage of such covering index without any splitting
technique. The function JOIN_TAB::choose_best_splitting() must take this
into account.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

If a join query uses a derived table (view / CTE) with GROUP BY clause then
the execution plan for such join may employ split optimization. When this
optimization is employed the derived table is not materialized. Rather only
some partitions of the derived table are subject to grouping. Split
optimization can be applied only if:
- there are some indexes over the tables used in the join specifying the
  derived table whose prefixes partially cover the field items used in the
  GROUP BY list (such indexes are called splitting indexes)
- the WHERE condition of the join query contains conjunctive equalities
  between columns of the derived table that comprise major parts of
  splitting indexes and columns of the other join tables.

When the optimizer evaluates extending of a partial join by the rows of the
derived table it always considers a possibility of using split optimization.
Different splitting indexes can be used depending on the extended partial
join. At some rare conditions, for example, when there is a non-splitting
covering index for a table joined in the join specifying the derived table
usage of a splitting index to produce rows needed for grouping may be still
less beneficial than usage of such covering index without any splitting
technique. The function JOIN_TAB::choose_best_splitting() must take this
into account.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

only perform the "correct table name" check for *new* generated columns,
but not for already existing ones - they're guaranteed to be valid