MDEV-13310 Preparing an incremental backup twice can corrupt data

The last parameter to this function is now,"bool is_sparse", like in 10.1
rather than the  unused/useless "bool is_readonly", merged from MySQL 5.7

Like in 10.1, this function now supports sparse files, and efficient
platform specific mechanisms for file extension

os_file_set_size() is now consistenly used in all places where
innodb files are extended.

galera_events test shows a regression with the original fix for MW-416
Reason was that Events::drop_event() can be called also from inside event
execution, and there we have a speacial treatment for event, which executes
"DROP EVENT" statement, and runs TOI replication inside the event processing body.
This resulted in executing WSREP_TO_ISOLATION two times for such DROP EVENT statement.
Fix is to call WSREP_TO_ISOLATION_BEGIN only in Events::drop_event()

Changed return code for replicatio error to TRUE.
This is aligned with native mysql convention to return TRUE (defined to 1) or FALSE (defined to 0) from a bool function.
This is wrong, but follows the mysql conventiosn, at least...

Moved TOI replication to happen after ACL checking for commands:
SQLCOM_CREATE_EVENT
SQLCOM_ALTER_EVENT
SQLCOM_DROP_EVENT
SQLCOM_CREATE_VIEW
SQLCOM_CREATE_TRIGGER
SQLCOM_DROP_TRIGGER
SQLCOM_INSTALL_PLUGIN
SQLCOM_UNINSTALL_PLUGIN

MariaDB 10.1 introduced non-indexed virtual columns for InnoDB tables.
When MySQL 5.7 introduced virtual columns in InnoDB tables, it also
introduced the table SYS_VIRTUAL that stores metadata on virtual
columns. This table does not initially exist in data files that were
imported from 10.1. So, we do not always have virtual column metadata
inside InnoDB.

dict_index_contains_col_or_prefix(): In the clustered index records,
all non-virtual columns are present and no virtual columns are present.

ha_innobase::build_template(): In the clustered index, do not
include virtual columns in the query template. The SQL layer is
supposed to compute the virtual column values when needed.

When btr_create() invokes btr_free_root() after running out of space,
fseg_create() would have acquired an SX-latch on the root page, not
an X-latch. Relax the debug assertion in btr_free_root() accordingly.

(In this case, SX-latch and X-latch are equivalent. During the CREATE
operation there should be MDL_EXCLUSIVE and dict_operation_lock X-latch
preventing concurrent access to the index. Normally the purpose of the
SX-latch is to allow concurrent reads of the root page while certain
fields in the root page are updated in place.)

Some innobase/xtrabackup changes around  from 10.1 are null merged
, in partucular using os_set_file_size to extend tablespaces in server
or mariabackup.
They require non-trivial amount of additional  work in 10.2, due to
innobase differences between 10.1 and 10.2

        Space for the next operation wasn't reserved in
        Item_func_spatial_relate::val_str()

Remove .delta file after it was successfully applied

This incorrect alignment can later lead to memcpy over buffer boundaries,
and to a crash.

Fix incremental prepare to change file size while applying
delta file, if delta file contains page 0 with the new size.

Change fil_space_extend_must_retry() to use this function.

In MariaDB Server 10.1, this problem manifests itself only as
a debug assertion failure in page_zip_decompress() when an insert
requires a page to be decompressed.

In MariaDB 10.1, the encryption of InnoDB data files repurposes the
previously unused field FILE_FLUSH_LSN for an encryption key version.
This field was only used in the first page of each file of the system
tablespace. For ROW_FORMAT=COMPRESSED tables, the field was always
written as 0 until encryption was implemented.

There is no bug in the encryption, because the buffer pool blocks will
not be written to files. Instead, copies of the blocks will be encrypted.
In these encrypted copies, the key version field will be updated before
the buffer is written to the file. The field in the buffer pool is
basically garbage that does not really matter.

Already in MariaDB 10.0, the memset() calls to reset this unused field
in buf_flush_update_zip_checksum() and buf_flush_write_block_low()
are unnecessary, because fsp_init_file_page_low() would guarantee that
the field is always 0 in the buffer pool (unless 10.1 encryption is
used).

Removing the unnecessary memset() calls makes page_zip_decompress()
happy and will prevent a SPATIAL INDEX corruption bug in
MariaDB Server 10.2. In MySQL 5.7.5, as part of WL#6968, the same
field was repurposed for an R-tree split sequence number (SSN) and
these memset() were removed. (Because of the repurposing, MariaDB
encryption is not available for tables that contain SPATIAL INDEX.)

Fixing the asymmetry in the array field_types_merge_rules[][]
which caused data loss when mixing FLOAT + BIGINT in UNIONs
or hybrid functions:

1. FLOAT  + INT    = DOUBLE
2. FLOAT  + BIGINT = FLOAT
3. INT    + FLOAT  = DOUBLE
4. BIGINT + FLOAT  = DOUBLE

Now FLOAT + BIGINT (as in #2) also produces DOUBLE, like the cases #1,#3,#4 do.

TRUE.

        Let's make empty brackets to be the syntax error in the JSON
        paths.

using JSON_UNQUOTE as an argument.

        The coercibility and repertoire should be set properly for the
        JSON_UNQUOTE.

        Check validity to the end of the JSON in the json_length
        function.

        The 'value' state should be saved to be compared against the
        next array item.

not just for creating new files.

Use os_file_set_size() in fil_space_extend_must_retry()

MDEV-13953 mariadb-10.2.9/storage/innobase/row/row0import.cc:3602: possible cut'n'paste error ?
MDEV-13955 mariadb-10.2.9/sql/log_event.cc:7671: possible wrong operator ?

functions.

There is no reason to export IO this functionality from server, if it can
be easily implemented with CRT.

Note: for the sake of compatibility, we do not use _open() on Windows,
but open file with CreateFile, with FILE_SHARE_DELETE flag,
so it can be renamed or deleted when open. Also atomic append flag is used.

Fixes some warnings.

fil_space_extend_must_retry(): If the table is being truncated,
do not call fil_flush_low(). The operation is covered by the
truncate log. File extension during TRUNCATE only occurs
if there are many indexes on the table. With smaller innodb_page_size,
the file extension occurs already with fewer indexes on the table.

This is error-prone and causes warnings on Windows

wrep_sst_common and wsrep_sst_xtrabackup-v2 bugfixes to read sst parameters from configs

The function dict_disable_redo_if_temporary() was supposed to
disable redo logging for temporary tables. It was invoked
unnecessarily for two read-only operations:
row_undo_search_clust_to_pcur() and
dict_stats_update_transient_for_index().

When a table is not temporary and not in the system tablespace,
the tablespace should be flagged for MLOG_FILE_NAME logging.
We do not need this overhead for temporary tables. Therefore,
either mtr_t::set_log_mode() or mtr_t::set_named_space() should
be invoked.

dict_table_t::is_temporary(): Determine if a table is temporary.

dict_table_is_temporary(): Redefined as a macro wrapper for
dict_table_t::is_temporary().

dict_disable_redo_if_temporary(): Remove.

The method handlerton::replace_native_transaction_in_thd was added
into MySQL 5.7 in WL#6860, Binlogging XA-prepared transaction.
In MariaDB we do not have that change yet.

The field dict_table_t::big_rows was only used for determining if
the adaptive hash index should be used when the internal InnoDB SQL
parser is used. That parser is only used for modifying the InnoDB
data dictionary, updating persistent tables, and for fulltext indexes.

This is regression caused by patch of mdev-10767.

1st problem :- mktmp is invoked without '-t' or specifing tmp directory.
2nd problem :- Since eval_log_error redirect stderr to stdout  '2>' will
return nothing. and hence $wr_logfile will be empty.

Patch Credit:- Andrii Nikitin