trx_undo_report_row_operation(): Fix a race condition that was introduced
in commit f74023b9 (MDEV-15090).
We must not access undo_block after the page latch has been released
in mtr_t::commit(), because the block could be evicted or replaced.

Let us simply refuse an upgrade from earlier versions if the
upgrade procedure was not followed. This simplifies the purge,
commit, and rollback of transactions.

Before upgrading to MariaDB 10.3 or later, a clean shutdown
of the server (with innodb_fast_shutdown=1 or 0) is necessary,
to ensure that any incomplete transactions are rolled back.
The undo log format was changed in MDEV-12288. There is only
one persistent undo log for each transaction.

Builds on travis-ci.org ceased on 2021-06-15.

spider tests use

--let $var= many;sql;statements
--eval $var

and this doesn't work in ps

and *never* disable tests in suite.pm based on $::opt_big_test,
this will make the test skipped both as too big (for ./mtr)
and as too small (for ./mtr --big --big).

- Make innodb_fts.misc_debug test case more stable.

In commit 1c35a3f6 a useless
computation that used the variable was removed.

Additional improvements aimed at improving operational
reliability of the SST scripts:

1) Script need to give rsync and stunnel a short time to
   terminate after "kill -9" before the first PID check
   using ps utility;
2) The temporary file used to create the binlog index could
   sometimes remain in the data directory if tar failed and
   then may be reused without being cleaned up (the next
   time when SST was run) - now it's fixed;
3) The temporary file used to build the binlog index is now
   created using mktemp and, if this variable is present in
   the configuration file, in tmpdir;
4) Checking the secret tag in SST via rsync is made faster
   and does not require creating a temporary file, which
   could remain in the data directory in case of failure;
5) Added "-F" option to grep to check the tag when using
   mariabackup/xtrabackup-v2 - to avoid possible collisions
   in case of special characters in the tag value (unlikely
   scenario, but the new check is more reliable).

This commit fixes a bug was originally discovered during the
galera_nbo_sst_slave mtr test for 10.6 branch. However it is
relevant for all versions and can lead to intermittent SST
crashes via rsync on very fast server restarts - when a new
SST process (for example, after starting a new server instance)
overlaps the old SST process started by the previous, already
terminated server. This overlap can result in the new rsync
being killed instead of the old rsync, or the pid file from
the new rsync being killed, which then lead to problems.

a new warning -Wunused-but-set-variable was introduced recently to clang

- During online alter conversion from compact to redundant,
virtual column field length already set during
innobase_get_computed_value(). Skip the char(n) check for
virtual column in row_merge_buf_add()

MDEV-25872 InnoDB: Assertion failure in row_merge_read_clustered_index upon ALTER on table with indexed virtual columns

- InnoDB fails to check DB_COMPUTE_VALUE_FAILED error in
row_merge_read_clustered_index() and wrongly asserts that
the buffer shouldn't be ran out of memory. Alter table
should give warning when the column value is being
truncated.

Problem:
=======
- InnoDB iterates the fil_system space list to encrypt the
tablespace in case of key rotation. But it is not
necessary for any encryption plugin which doesn't do
key version rotation.

Solution:
=========
- Introduce a new variable called srv_encrypt_rotate to
indicate whether encryption plugin does key rotation

fil_space_crypt_t::key_get_latest_version(): Enable the
srv_encrypt_rotate only once if current key version is
higher than innodb_encyrption_rotate_key_age

fil_crypt_must_default_encrypt(): Default encryption tables
should be added to default_encryp_tables list if
innodb_encyrption_rotate_key_age is zero and encryption
plugin doesn't do key version rotation

fil_space_create(): Add the newly created space to
default_encrypt_tables list if
fil_crypt_must_default_encrypt() returns true

Removed the nondeterministic select from
innodb-key-rotation-disable test. By default,
InnoDB adds the tablespace to the rotation list and
background crypt thread does encryption of tablespace.
So these select doesn't give reliable results.

- Rename the rotation list to default_encrypt_tables in
fil_system_t. Because rotation list naming could be
misleading when it comes to key version rotation

- Rename is_in_rotation_list to is_in_default_encrypt
in fil_space_t

- Rename keyrotate_next function to default_encrypt_next

fil_system_t::default_encrypt_next(): Find the next
suitable default encrypt table if beginning of
default_encrypt_tables list has been scheduled
to be deleted

Brad Smith made this OpenBSD file based of the
FreeBSD cmake directives in commit ab589043
by the Monty Program Ab.

As such the Oracle Copyright header isn't really applicable.

Fix malloc/delete mismatch. This causes a double free in the cleanup.

closes #1845

* read_command_buf is a pointer now, sizeof() no longer reflects its
  length, read_command_buflen is.
* my_safe_print_str() prints multiple screens of '\0' bytes after the
  query end and up to read_command_buflen. Use fprintf() instead.
* when setting connection->name to "-closed_connection-" update
  connection->name_len to match.

Not all environments have 'diff' installed. Most notably CentOS 8
does not have diff out-of-the-box. Thus users running 'cmake .' and
'make' would fail to build MariaDB, and they would think the error
was in ABI incompatibilities due to the error message emitted by CMake
when in reality simply 'diff' was missing.

This fixes it and makes the developer experience better by simply
skipping the diffing if 'diff' is not found.

Closes #1846

This was missed in commit 1d0f70c2.

* Make Item_in_optimizer::fix_fields inherit the with_window_func
  attribute of the subquery's left expression (the subquery itself
  cannot have window functions that are aggregated in this select)

* Make Item_cache_wrapper::Item_cache_wrapper() inherit
  with_window_func attribute of the item it is caching.

Occasionally, the test innodb.alter_copy would fail in MariaDB 10.6.1,
reporting DB_MISSING_HISTORY during CHECK TABLE. It started to occur during
the development of MDEV-25180, which introduced purge_sys.stop_SYS().
If we delay purge more during DDL operations, then the test would
almost always fail. The reason is that during startup we will restore
a purge view, and CHECK TABLE would still use REPEATABLE READ
even though innodb_read_only is set and other isolation levels
than READ UNCOMMITTED are not guaranteed to work.

ha_innobase::check(): Use READ UNCOMMITTED isolation level if
innodb_read_only is set or innodb_force_recovery exceeds 3.

dict_set_corrupted(): Do not update the persistent data dictionary
if innodb_force_recovery exceeds 3.

buf_read_ibuf_merge_pages(): If space->size is 0, invoke
fil_space_get_size() to determine the size of the tablespace
by reading the header page. Only after that proceed to delete
any entries that are beyond the end of the tablespace.
Otherwise, we could be deleting valid entries that actually
need to be applied.

This fixes a regression that had been introduced in
commit b80df9eb (MDEV-21069),
which aimed to avoid crashes during DROP TABLE of corrupted tables.

If a select query contained an ORDER BY clause that followed a LIMIT clause
or an ORDER BY clause or ORDER BY with LIMIT the EXPLAIN output for the
query showed an execution plan different from that was actually executed.

Approved by Roman Nozdrin <roman.nozdrin@mariadb.com>

This partially reverts commit d7321893.

The *.jar files are not being built and all Debian builds are failing
as dh_install stops on missing files. To build them we would need to also
add new Java build dependencies.

In a stable release (10.2->10.5) we shouldn't add new files and certainly
not any new build dependencies, so reverting commit.

Also, the files are located in a different path, and already included
in the mariadb-test-data package:

  /usr/share/mysql/mysql-test/plugin/connect/connect/std_data/JavaWrappers.jar
  /usr/share/mysql/mysql-test/plugin/connect/connect/std_data/JdbcMariaDB.jar
  /usr/share/mysql/mysql-test/plugin/connect/connect/std_data/Mongo2.jar
  /usr/share/mysql/mysql-test/plugin/connect/connect/std_data/Mongo3.jar

This change needs to be redesigned and applies only on 10.6 or newer.

The fix is to quote service name parameter, when it is passed to
mysql_upgrade_service subprocess.

ha_heap::external_lock contains some consistency checks for the table,#
in a debug compilation.

This code suffers from lack of synchronization, in a rare case
where mysql_lock_tables() fail, and unlock is forced, even if lock was
not previously taken.

To workaround, require EXTRA_DEBUG compile definition in order to activate
the consistency checks.The code still might be useful in some cases - but
the audience are developers looking for errors in single-threaded scenarios,
rather than multiuser stress-tests.

MDEV-25777: JAVA_INCLUDE_PATH and JAVA_INCLUDE_PATH2 not found with out of source configuration and Ninja generator

- As solution `PLUGIN_CONNECT=NO` use early check to disable plugin:
  Solution suggested by wlad@mariadb.com
- `JNI_FOUND` is a internal result variable and should be set with
cached library and header variables (like `JAVA_INCLUDE_PATH`) defined.
  * Note: wrapper cmake/FindJNI.cmake runs first time and cmake native Find<module> returns only cached variable, like `JAVA_INCLUDE_PATH`, results variable are not cached).

Reviewed by: serg@mariadb.com

If a join query uses a derived table (view / CTE) with GROUP BY clause then
the execution plan for such join may employ split optimization. When this
optimization is employed the derived table is not materialized. Rather only
some partitions of the derived table are subject to grouping. Split
optimization can be applied only if:
- there are some indexes over the tables used in the join specifying the
  derived table whose prefixes partially cover the field items used in the
  GROUP BY list (such indexes are called splitting indexes)
- the WHERE condition of the join query contains conjunctive equalities
  between columns of the derived table that comprise major parts of
  splitting indexes and columns of the other join tables.

When the optimizer evaluates extending of a partial join by the rows of the
derived table it always considers a possibility of using split optimization.
Different splitting indexes can be used depending on the extended partial
join. At some rare conditions, for example, when there is a non-splitting
covering index for a table joined in the join specifying the derived table
usage of a splitting index to produce rows needed for grouping may be still
less beneficial than usage of such covering index without any splitting
technique. The function JOIN_TAB::choose_best_splitting() must take this
into account.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

only perform the "correct table name" check for *new* generated columns,
but not for already existing ones - they're guaranteed to be valid

This bug could manifest itself after pushing a where condition over a
mergeable derived table / view / CTE DT into a grouping view / derived
table / CTE V whose item list contained set functions with constant
arguments such as MIN(2), SUM(1) etc. In such cases the field references
used in the condition pushed into the view V that correspond set functions
are wrapped into Item_direct_view_ref wrappers. Due to a wrong implementation
of the virtual method const_item() for the class Item_direct_view_ref the
wrapped set functions with constant arguments could be erroneously taken
for constant items. This could lead to a wrong result set returned by the
main select query in 10.2. In 10.4 where a possibility of pushing condition
from HAVING into WHERE had been added this could cause a crash.

Approved by Sergey Petrunya <sergey.petrunya@mariadb.com>