Allow combination of non-instant, non-rebuilding operations with
changes of table options that do not require a rebuild.

For example, DROP INDEX or ADD INDEX can be performed with
ALGORITHM=NOCOPY together with changing such table options.
Changing the table options alone would be allowed with ALGORITHM=INSTANT.

INNOBASE_ALTER_NOCREATE: A new set of flags, for operations that
are refused for ALGORITHM=INSTANT and do not involve creating
index trees.

Move ALTER_RENAME_INDEX to the proper place (INNOBASE_ALTER_INSTANT).

innobase_need_rebuild(): Do not require a rebuild if
INNOBASE_ALTER_NOREBUILD operations are combined with ALTER_OPTIONS.

ha_innobase::prepare_inplace_alter_table(),
ha_innobase::inplace_alter_table(): Use the fast path if
ALTER_OPTIONS is combined with INNOBASE_ALTER_NOCREATE.
In this case, the actual changes would be deferred to
ha_innobase::commit_inplace_alter_table().

This follows up to commit 755187c8.

TRX_UNDO_INSERT_METADATA: Renamed from TRX_UNDO_INSERT_DEFAULT

trx_undo_metadata: Renamed from trx_undo_default_rec

For instant ALTER TABLE, we store a hidden metadata record at the
start of the clustered index, to indicate how the format of the
records differs from the latest table definition.

The term 'default row' is too specific, because it applies to
instant ADD COLUMN only, and we will be supporting more classes
of instant ALTER TABLE later on. For instant ADD COLUMN, we
store the initial default values in the metadata record.

Remove some code duplication and dead code. If no 'default row'
record exists, the root page must be in the conventional format.
Should the page type already be FIL_PAGE_TYPE_INSTANT, we would
necessarily hit a debug assertion failure in page_set_instant().

The crash occurs when the Spider node server attempts to create an error
message stating that the temporary table is not found.  The function to
create the error message is called with incorrect parameters.

I fixed the crash by correcting the incorrect parameter values.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Cherry-Picked:
  Commit e3396161 on branch bb-10.3-mdev-17144

The crash occurs when the Spider node server attempts to create an error
message stating that the temporary table is not found.  The function to
create the error message is called with incorrect parameters.

I fixed the crash by correcting the incorrect parameter values.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Merged:
  Commit e3396161 branch bb-10.3-mdev-17144

The function JOIN_TAB::choose_best_splitting() did not take into account
that for some tables whose fields were used in the GROUP BY list of
the specification of a splittable materialized derived there might exist
no elements in the array ext_keyuses_for_splitting.

The crash occurs when the Spider node server attempts to create an error
message stating that the temporary table is not found.  The function to
create the error message is called with incorrect parameters.

I fixed the crash by correcting the incorrect parameter values.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Now that ha_innobase::prepare_inplace_alter_table() is accessing
ha_alter_info->create_info->option_struct, we must initialize it in
the Mroonga wrapper for ALTER TABLE based on the parsed table options
for the wrap_altered_table.

The table option page_compression_level is something that only
affects future writes, not actually the data format. Therefore,
we can allow instant changes of this option.

Similarly, the table option page_compressed can be set on a
previously uncompressed table without rebuilding the table,
because an uncompressed page would be considered valid when
reading a page_compressed table.

Removing the page_compressed option will continue to require
the table to be rebuilt.

ha_innobase_inplace_ctx::page_compression_level: The requested
page_compression_level at the start of ALTER TABLE, or 0 if
page_compressed=OFF.

alter_options_need_rebuild(): Renamed from
create_option_need_rebuild(). Allow page_compression_level and
page_compressed to be changed as above, without rebuilding the table.

ha_innobase::check_if_supported_inplace_alter(): Allow ALGORITHM=INSTANT
for ALTER_OPTIONS if the table is not to be rebuilt. If rebuild is
needed, set ha_alter_info->unsupported_reason.

innobase_page_compression_try(): Update SYS_TABLES.TYPE according
to the table flags, for an instant change of page_compression_level
or page_compressed.

commit_cache_norebuild(): Adjust dict_table_t::flags, fil_space_t::flags
and (if needed) FSP_SPACE_FLAGS if page_compression_level was specified.

MDEV-17203 Move fractional second truncation from Item_xxx_typecast::get_date() to Time and Datetime constructors

The optimizer erroneously allowed to use join cache when joining a
splittable materialized table together with splitting optimization.
As a consequence in some rare cases the server returned wrong result
sets for queries with materialized derived.

This patch allows to use either join cache without usage of splitting
technique for materialization of a splittable derived table or splitting
without usage of join cache when joining such table. The costs the these
alternatives are compared and the best variant is chosen.

A debug assertion would fail if an instant ADD COLUMN operation
involves splitting the leftmost leaf page and storing a default
value off-page. Another debug assertion could fail if the
default value does not fit in an undo log page.

btr_cur_pessimistic_update(): Invoke rec_offs_make_valid()
in order to prevent rec_offs_validate() assertion failure.

innobase_add_instant_try(): Invoke btr_cur_pessimistic_update()
with the BTR_KEEP_POS_FLAG, which is the correct course of action
when BLOBs may need to be written. Whenever returning true,
ensure that my_error() will have been called.

The problem occurs in 10.2 and earlier releases of MariaDB Server because the
Partition Engine was not pushing the engine conditions to the underlying
storage engine of each partition.  This caused Spider to return the first 5
rows in the table with the data provided by the customer.  2 of the 5 rows
did not qualify the WHERE clause, so they were removed from the result set by
the server.

To fix the problem, I have back-ported support for engine condition pushdown
in the Partition Engine from MariaDB Server 10.3 to 10.2 and 10.1.  In 10.3
and 10.4 I have merged the comments and the test case.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Cherry-Picked:
  Commit ed49f9aa on branch 10.3

The problem occurs in 10.2 and earlier releases of MariaDB Server because the
Partition Engine was not pushing the engine conditions to the underlying
storage engine of each partition.  This caused Spider to return the first 5
rows in the table with the data provided by the customer.  2 of the 5 rows
did not qualify the WHERE clause, so they were removed from the result set by
the server.

To fix the problem, I have back-ported support for engine condition pushdown
in the Partition Engine from MariaDB Server 10.3 to 10.2 and 10.1.  In 10.3
and 10.4 I have merged the comments and the test case.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Merged:
  Commit eb2ca3d4 on branch bb-10.2-MDEV-16912

The blob key length could be shorter than the length of the entire blob,
for example,

CREATE TABLE t1 (b BLOB, i INT, KEY(b(8)));
INSERT INTO t1 VALUES (REPEAT('a',9),1);

The key length is 8, while the blob length is 9.
So we need to set the correct key length in Field_blob::sort_string().

Implement a 10.4 redo log format, which extends the 10.3 format
by introducing the MLOG_MEMSET record.

MLOG_MEMSET: A new redo log record type for filling an area with a byte.

mlog_memset(): Write the MLOG_MEMSET record.

mlog_parse_nbytes(): Handle MLOG_MEMSET as well.

trx_rseg_header_create(): Reduce the redo log volume by making use of
mlog_memset() and the zero-initialization that happens inside page
allocation.

fil_addr_null: Remove.

flst_init(): Create a variant that takes a zero-initialized
buf_block_t* as a parameter, and only writes the FIL_NULL using
mlog_memset().

flst_zero_addr(): A variant of flst_write_addr() that writes
a null address using mlog_memset() for the FIL_NULL.

The following fixes are replacing some use of MLOG_WRITE_STRING
with the more compact MLOG_MEMSET record, or eliminating
redundant redo log writes:

btr_store_big_rec_extern_fields(): Invoke mlog_memset() for
zero-initializing the tail of the ROW_FORMAT=COMPRESSED BLOB page.

trx_sysf_create(), trx_rseg_format_upgrade(): Invoke mlog_memset()
for zero-initializing the page trailer.

fsp_header_init(), trx_rseg_header_create():
Remove redundant zero-initializations.

Stop supporting the additional *trunc.log files that were
introduced via MySQL 5.7 to MariaDB Server 10.2 and 10.3.

DB_TABLESPACE_TRUNCATED: Remove.

purge_sys.truncate: A new structure to track undo tablespace
file truncation.

srv_start(): Remove the call to buf_pool_invalidate(). It is
no longer necessary, given that we no longer access things in
ways that violate the ARIES protocol. This call was originally
added for innodb_file_format, and it may later have been necessary
for the proper function of the MySQL 5.7 TRUNCATE recovery, which
we are now removing.

trx_purge_cleanse_purge_queue(): Take the undo tablespace as a
parameter.

trx_purge_truncate_history(): Rewrite everything mostly in a
single function, replacing references to undo::Truncate.

recv_apply_hashed_log_recs(): If any redo log is to be applied,
and if the log_sys.log.subformat indicates that separately
logged truncate may have been used, refuse to proceed except if
innodb_force_recovery is set. We will still refuse crash-upgrade
if TRUNCATE TABLE was logged. Undo tablespace truncation would
only be logged in undo*trunc.log files, which we are no longer
checking for.

Return code from starting MariaDB.

MDEV-16052 galera mtr galera_certification_double_failure fails with deadlock

MDEV-15845 Test failure on galera.galera_concurrent_ctas

With the TRUNCATE by rename, create, drop (MDEV-13564),
old tables with invalid ROW_FORMAT attribute could not be
truncated. Introduce a sloppy mode for allowing the TRUNCATE.

create_table_info_t::prepare_create_table(): Add the parameter
strict=true.

ha_innobase::create(): Pass strict=false if trx!=NULL
(the create is part of TRUNCATE).

It turned out that ha_innobase::truncate() would prematurely
commit the transaction already before the completion of the
ha_innobase::create(). All of this must be atomic.

innodb.truncate_crash: Use the correct DEBUG_SYNC point, and
tolerate non-truncation of the table, because the redo log
for the TRUNCATE transaction commit might be flushed due to
some InnoDB background activity.

dict_build_tablespace_for_table(): Merge to the function
dict_build_table_def_step().

dict_build_table_def_step(): If a table is being created during
an already started data dictionary transaction (such as TRUNCATE),
persistently write the table_id to the undo log header before
creating any file. In this way, the recovery of TRUNCATE will be
able to delete the new file before rolling back the rename of
the original table.

dict_table_rename_in_cache(): Add the parameter replace_new_file,
used as part of rolling back a TRUNCATE operation.

fil_rename_tablespace_check(): Add the parameter replace_new.
If the parameter is set and a file identified by new_path exists,
remove a possible tablespace and also the file.

create_table_info_t::create_table_def(): Remove some debug assertions
that no longer hold. During TRUNCATE, the transaction will already
have been started (and performed a rename operation) before the
table is created. Also, remove a call to dict_build_tablespace_for_table().

create_table_info_t::create_table(): Add the parameter create_fk=true.
During TRUNCATE TABLE, do not add FOREIGN KEY constraints to the
InnoDB data dictionary, because they will also not be removed.

row_table_add_foreign_constraints(): If trx=NULL, do not modify
the InnoDB data dictionary, but only load the FOREIGN KEY constraints
from the data dictionary.

ha_innobase::create(): Lock the InnoDB data dictionary cache only
if no transaction was passed by the caller. Unlock it in any case.

innobase_rename_table(): Add the parameter commit = true.
If !commit, do not lock or unlock the data dictionary cache.

ha_innobase::truncate(): Lock the data dictionary before invoking
rename or create, and let ha_innobase::create() unlock it and
also commit or roll back the transaction.

trx_undo_mark_as_dict(): Renamed from trx_undo_mark_as_dict_operation()
and declared global instead of static.

row_undo_ins_parse_undo_rec(): If table_id is set, this must
be rolling back the rename operation in TRUNCATE TABLE, and
therefore replace_new_file=true.

sql/table.cc:8561:42: error: non-constant-expression cannot be narrowed
                             from type 'uint' (aka 'unsigned int') to
                             '__darwin_suseconds_t' (aka 'int') in
                             initializer list [-Wc++11-narrowing]
  timeval end_time= {thd->query_start(), uint(thd->query_start_sec_part())};
                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
sql/table.cc:8561:42: note: insert an explicit cast to silence this issue
  timeval end_time= {thd->query_start(), uint(thd->query_start_sec_part())};
                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                         static_cast<__darwin_suseconds_t>( )