The original intention of the setting innodb_force_recovery=3 was to
disable background activity that could create trouble, most notably,
the rollback of incomplete transactions, and the purge of transaction
history.

MySQL 5.6 introduced more background threads, it is creating
dict_stats_thread and fts_optimize_thread even though these threads
are at least as non-essential as the rollback and purge. These
threads are in fact worse, because they can create new transactions
on their own.

innobase_start_or_create_for_mysql(): Do not create any internal
undo log sources unless innodb_force_recovery<3.

Parameters can be MYSQL_TYPE_VARCHAR for long data load.

These errors should only be possible when bootstrap is aborted
and restarted.

buf_flush_init_for_writing(): Reset the FIL_PAGE_TYPE
of the TRX_SYS page to the canonical value
FIL_PAGE_TYPE_TRX_SYS instead of FIL_PAGE_TYPE_UNKNOWN.

When running setup fields during the final step of insert using select
the final setup_fields does not have any sum functions. Our current
condition for calling split_sum_func however would attempt to use an empty
NULL sum_func_list, if the item contained a window function.

The solution is to not perform another split_sum_func for the item
containing a window function if we do not actually have a sum_func_list.

This will also change the minimum and maximum value of
innodb_log_file_size to 1MiB and 512GiB, respectively.

MDEV-12975 InnoDB redo log minimum size check uses detected file size instead of requested innodb_log_file_size

log_calc_max_ages(): Use the requested size in the check, instead of
the detected redo log size. The redo log will be resized at startup
if it differs from what has been requested.

The option was basically duplicating InnoDB functionality.
Persistent statistics can be accessed via the tables
mysql.innodb_table_stats and mysql.innodb_index_stats.

These functions cannot possibly work in MariaDB 10.2, and it is
uncertain if they really work in 10.1 either.

This is another attempt to fix the bug mdev-12992.
This patch introduces st_select_lex::context_analysis_place for
the place in SELECT where context analysis is currently performed.
It's similar to st_select_lex::parsing_place, but it is used at
the preparation stage.

             lock not released after timeout

Release GRL if FLUSH TABLES phase failed.

When the btr_search_latch was split into an array of latches
in MySQL 5.7.8 as part of the Oracle Bug#20985298 fix, the "caching"
of the latch across storage engine API calls was removed, and
the field trx->has_search_latch would only be set during a short
time frame in the execution of row_search_mvcc(), which was
formerly called row_search_for_mysql().

This means that the column
INFORMATION_SCHEMA.INNODB_TRX.TRX_ADAPTIVE_HASH_LATCHED will always
report 0. That column cannot be removed in MariaDB 10.2, but it
can be removed in future releases.

trx_t::has_search_latch: Remove.

trx_assert_no_search_latch(): Remove.

row_sel_try_search_shortcut_for_mysql(): Remove a redundant condition
on trx->has_search_latch (it was always true).

sync_check_iterate(): Make the parameter const.

sync_check_functor_t: Make the operator() const, and remove result()
and the virtual destructor. There is no need to have mutable state
in the functors.

sync_checker<bool>: Replaces dict_sync_check and btrsea_sync_check.

sync_check: Replaces btrsea_sync_check.

dict_sync_check: Instantiated from sync_checker.

sync_allowed_latches: Use std::find() directly on the array.
Remove the std::vector.

TrxInInnoDB::enter(), TrxInInnoDB::exit(): Remove obviously redundant
debug assertions on trx->in_depth, and use equality comparison against 0
because it could be more efficient on some architectures.

The sole purpose of handlerton::release_temporary_latches and its wrapper
function was to release the InnoDB adaptive hash index latch
(btr_search_latch).

When the btr_search_latch was split into an array of latches
in MySQL 5.7.8 as part of the Oracle Bug#20985298 fix, the "caching"
of the latch across storage engine API calls was removed. As part of that,
the function trx_search_latch_release_if_reserved() was changed to an
assertion and the function trx_reserve_search_latch_if_not_reserved()
was removed, and handlerton::release_temporary_latches() practically
became a no-op.

Note: MDEV-12121 replaced the function
trx_search_latch_release_if_reserved()
with the more appropriately named macro trx_assert_no_search_latch().

dict_sys_tables_type_to_tf(): Change the parameter n_cols to not_redundant.

dict_tf_is_valid_not_redundant(): Refactored from dict_tf_is_valid().

dict_sys_tables_type_valid(): Replaces dict_sys_tables_type_validate().
Use the common function dict_tf_is_valid_not_redundant(), which validates
PAGE_COMPRESSION_LEVEL more strictly.

DICT_TF_GET_UNUSED(flags): Remove.

innodb.table_flags: Adjust the test case. Due to the MDEV-12873 fix
in 10.2, the corrupted flags for table test.td would be converted,
and a tablespace flag mismatch will occur when trying to open the file.

MDEV-12873 InnoDB SYS_TABLES.TYPE incompatibility for PAGE_COMPRESSED=YES in MariaDB 10.2.2 to 10.2.6

Remove the SHARED_SPACE flag that was erroneously introduced in
MariaDB 10.2.2, and shift the SYS_TABLES.TYPE flags back to where
they were before MariaDB 10.2.2. While doing this, ensure that
tables created with affected MariaDB versions can be loaded,
and also ensure that tables created with MySQL 5.7 using the
TABLESPACE attribute cannot be loaded.

MariaDB 10.2.2 picked the SHARED_SPACE flag from MySQL 5.7,
shifting the MariaDB 10.1 flags PAGE_COMPRESSION, PAGE_COMPRESSION_LEVEL,
ATOMIC_WRITES by one bit. The SHARED_SPACE flag would always
be written as 0 by MariaDB, because MariaDB does not support
CREATE TABLESPACE or CREATE TABLE...TABLESPACE for InnoDB.

So, instead of the bits AALLLLCxxxxxxx we would have
AALLLLC0xxxxxxx if the table was created with MariaDB 10.2.2
to 10.2.6. (AA=ATOMIC_WRITES, LLLL=PAGE_COMPRESSION_LEVEL,
C=PAGE_COMPRESSED, xxxxxxx=7 bits that were not moved.)

PAGE_COMPRESSED=NO implies LLLLC=00000. That is not a problem.

If someone created a table in MariaDB 10.2.2 or 10.2.3 with
the attribute ATOMIC_WRITES=OFF (value 2; AA=10) and without
PAGE_COMPRESSED=YES or PAGE_COMPRESSION_LEVEL, the table should be
rejected. We ignore this problem, because it should be unlikely
for anyone to specify ATOMIC_WRITES=OFF, and because 10.2.2 and
10.2.2 were not mature releases. The value ATOMIC_WRITES=ON (1)
would be interpreted as ATOMIC_WRITES=OFF, but starting with
MariaDB 10.2.4 the ATOMIC_WRITES attribute is ignored.

PAGE_COMPRESSED=YES implies that PAGE_COMPRESSION_LEVEL be between
1 and 9 and that ROW_FORMAT be COMPACT or DYNAMIC. Thus, the affected
wrong bit pattern in SYS_TABLES.TYPE is of the form AALLLL10DB00001
where D signals the presence of a DATA DIRECTORY attribute and B is 1
for ROW_FORMAT=DYNAMIC and 0 for ROW_FORMAT=COMPACT. We must interpret
this bit pattern as AALLLL1DB00001 (discarding the extraneous 0 bit).

dict_sys_tables_rec_read(): Adjust the affected bit pattern when
reading the SYS_TABLES.TYPE column. In case of invalid flags,
report both SYS_TABLES.TYPE (after possible adjustment) and
SYS_TABLES.MIX_LEN.

dict_load_table_one(): Replace an unreachable condition on
!dict_tf2_is_valid() with a debug assertion. The flags will already
have been validated by dict_sys_tables_rec_read(); if that validation
fails, dict_load_table_low() will have failed.

fil_ibd_create(): Shorten an error message about a file pre-existing.

Datafile::validate_to_dd(): Clarify an error message about tablespace
flags mismatch.

ha_innobase::open(): Remove an unnecessary warning message.

dict_tf_is_valid(): Simplify and stricten the logic. Validate the
values of PAGE_COMPRESSION. Remove error log output; let the callers
handle that.

DICT_TF_BITS: Remove ATOMIC_WRITES, PAGE_ENCRYPTION, PAGE_ENCRYPTION_KEY.
The ATOMIC_WRITES is ignored once the SYS_TABLES.TYPE has been validated;
there is no need to store it in dict_table_t::flags. The PAGE_ENCRYPTION
and PAGE_ENCRYPTION_KEY are unused since MariaDB 10.1.4 (the GA release
was 10.1.8).

DICT_TF_BIT_MASK: Remove (unused).

FSP_FLAGS_MEM_ATOMIC_WRITES: Remove (the flags are never read).

row_import_read_v1(): Display an error if dict_tf_is_valid() fails.

dict_table_t::thd: Remove. This was only used by btr_root_block_get()
for reporting decryption failures, and it was only assigned by
ha_innobase::open(), and never cleared. This could mean that if a
connection is closed, the pointer would become stale, and the server
could crash while trying to report the error. It could also mean
that an error is being reported to the wrong client. It is better
to use current_thd in this case, even though it could mean that if
the code is invoked from an InnoDB background operation, there would
be no connection to which to send the error message.

Remove dict_table_t::crypt_data and dict_table_t::page_0_read.
These fields were never read.

fil_open_single_table_tablespace(): Remove the parameter "table".

BULK execution moved to a new command.

innodb.row_format_redundant: Really corrupt the SYS_TABLES.MIX_LEN,
and do not use any debug instrumentation. For tables created in the
system tablespace, the contents of the column will be ignored.
Only the table t1 will refuse to load.

dict_load_table_one(): Remove the DBUG_EXECUTE_IF instrumentation.
Omit a redundant error message "incorrect flags in SYS_TABLES".

dict_sys_tables_rec_read(): Partially revert the Oracle Bug#21644827
fix, and always report errors by the return value.

fts_create_in_mem_aux_table(): Do not rely on dict_table_t::flags2,
but instead evaluate the tablespace ID.

DICT_TF2_BITS: Reduce to the correct value of 7. The two extra
high-order bits were specific to MySQL 5.7.

Cover innodb.table_flags with the new innodb_page_size.combinations
32k and 64k.

dict_sys_tables_type_validate(): Remove an assertion that made a
check in the function redundant. Remove the excessive output to
the error log, as the invalid SYS_TABLES.TYPE value is already being
output.

Add a missing #include "sync0types.h" that was removed in MDEV-12674.

Add a test case for corrupting SYS_TABLES.TYPE,
and for ROW_FORMAT=REDUNDANT, the unused field SYS_TABLES.MIX_LEN
that must be ignored (InnoDB before MySQL 5.5 wrote uninitialized
garbage to this column).

MariaDB 10.0 appears to validate the SYS_TABLES.TYPE properly.
This is a test-only change.

in innodb_read_only mode.

The reason for the hang is that there was no notification received about
completed read io. File handles are bound to completion_port, and there
were no background "write" threads that would be waiting on completion_port,
only 2 "read" threads waiting on read_completion_port were active.

The fix is to use a single IO completion port for all IOs, if
innodb_read_only is set.

log_crypt(): Remove the useless error log output that was
accidentally introduced in MDEV-11782. These messages could be emitted
to the server error log during crash recovery.

10.2 follow-up to MDEV-13039 innodb_fast_shutdown=0 crash due premature purge shutdown before fts_optimize_shutdown()

srv_start_state_t: Document the flags. Replace SRV_START_STATE_STAT
with SRV_START_STATE_REDO. The srv_bg_undo_sources replaces the
original use of SRV_START_STATE_STAT.

dict_stats_thread_started, buf_dump_thread_started,
buf_flush_page_cleaner_thread_started: Remove (unused).

srv_shutdown_all_bg_threads(): Always wait for the I/O threads
to exit, also in read-only mode.

os_thread_free(): Remove.

fil_crypt_read_crypt_data(): Remove an unnecessary
acquisition of fil_system->mutex. Remove a duplicated condition
from the callers.

When the server is started in innodb_read_only mode, there cannot be
any writes to persistent InnoDB/XtraDB files. Just like the creation
of buf_flush_page_cleaner_thread is skipped in this case, also
the creation of the XtraDB-specific buf_flush_lru_manager_thread
should be skipped.

When the SELECT query from a trigger that used a subquery
in its SELECT list was prepared the counter select_n_having_items
was incremented in the constructor Item::Item(THD *thd).
As a result each invocation of the trigger required more and more
memory for the ref_pointer_array for this SELECT.
Made sure that the counter st_select_lex::select_n_having_items
would be incremented only at the first execution of such trigger.