Introduce the configuration option innodb_log_optimize_ddl
for controlling whether native index creation or table-rebuild
in InnoDB should keep optimizing the redo log
(and writing MLOG_INDEX_LOAD records to ensure that
concurrent backup would fail).

By default, we have innodb_log_optimize_ddl=ON, that is,
the default behaviour that was introduced in MariaDB 10.2.2
(with the merge of InnoDB from MySQL 5.7) will be unchanged.

BtrBulk::m_trx: Replaces m_trx_id. We must be able to check for
KILL QUERY even if !m_flush_observer (innodb_log_optimize_ddl=OFF).

page_cur_insert_rec_write_log(): Declare globally, so that this
can be called from PageBulk::insert().

row_merge_insert_index_tuples(): Remove the unused parameter trx_id.

row_merge_build_indexes(): Enable or disable redo logging based on
the innodb_log_optimize_ddl parameter.

PageBulk::init(), PageBulk::insert(), PageBulk::finish(): Write
redo log records if needed. For ROW_FORMAT=COMPRESSED, redo log
will be written in PageBulk::compress() unless we called
m_mtr.set_log_mode(MTR_LOG_NO_REDO).

Native ALTER TABLE is never invoked on temporary tables.

This patch fixes another problem introduced by the patch for mdev-4817.
The latter changed Item_cond::fix_fields() in such a way that it could
call the virtual method is_expensive(). With the first its call
the method saves the result in Item::is_expensive_cache. For all next
calls the method returns the result from this cache. So if the item
once was determined as expensive the method always returns true.
For subqueries it's not good, because non-optimized subqueries always
is considered as expensive.
It means that the cache should be invalidated after the call of
optimize_constant_subqueries().

	At most one transaction can be active at a time for temporary
tables. There is no need to check previous version of record for the
temporary tables.

Actually if we use "set password for " command this changes the checksum
of mysql.user table
-localhost      root            Y       Y       Y       Y       Y       Y       Y       Y     YY       Y       Y       Y       Y       Y       Y       Y       Y       Y       Y       Y     $
Y       Y       Y       Y       Y       Y       Y                                       0     00       0                       N       N               0.000000
+localhost      root            Y       Y       Y       Y       Y       Y       Y       Y     YY       Y       Y       Y       Y       Y       Y       Y       Y       Y       Y       Y     Y
Y       Y       Y       Y       Y       Y       Y                                       0     00       0       mysql_native_password           N       N               0.000000
In short we replace '' with mysql_native_password
which make checksum to be different, and hence check test case fails.

So we use UPDATE mysql.user command.

The problem occurs on Ubuntu where a Spider package is installed on the system
separately from the MariaDB package.  MariaDB and Spider upgrades leave the
Spider plugin improperly installed.  Spider is present in the mysql.plugin
table but is not present in information_schema.

The problem has been corrected in Spider's installation script.  Logic has
been added to check for Spider entries in both information_schema and
mysql.plugin.  If Spider is present in mysql.plugin but is not present in
information_schema, then Spider is first removed from mysql.plugin.  The
subsequent plugin install of Spider will insert entries in both mysql.plugin
and information_schema.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Cherry-Picked:
  Commit 0897d81 on branch bb-10.3-MDEV-15786

This helped fix MDEV-16779.

Declare all fields of purge_vcol_info_t private, and add
accessor functions.

During a table-rebuilding operation, the function table_name_parse()
can encounter a table name that starts with #sql. Here is an example
of a failure:

CURRENT_TEST: gcol.innodb_virtual_basic
mysqltest: At line 1204: query 'alter table t drop column d ' failed:
2013: Lost connection to MySQL server during query

Let us just remove these bogus debug assertions.

If the final renaming phase during ALTER TABLE never fails, it
should not do any harm to skip the purge. If it does fail, then
we might end up 'leaking' some delete-marked records in the
indexes on virtual columns of the original table, and these
garbage records would keep consuming space until the indexes are
dropped or the table is successfully rebuilt.

This is a regression caused by the fix of MDEV-15855.

purge_vcol_info_t::set_used(): Add a missing condition.

row_purge_poss_sec(): Invoke set_used() in order to
have !is_first_fetch() when retrying.

This should affect at least rec_printer() output.

Replace pairs of rw_lock_own() calls with
calls to rw_lock_own_flagged().

These calls are only present in debug builds.

as a separate source for data

Actually MDEV-15867 and MDEV-16192 are same, Slave adds "or replace" to create
table stmt. So create table t1 is create or replace on slave. So this bug
is not because of replication, We can get this bug on general server if we
manually add or replace to create query.

Problem:- So if we try to create table t1 (same name as of temp table t1 ) via
   CREATE or replace TABLE t AS SELECT * FROM t;
Since in this query we are creating table from select * from t1 , we call
unique_table function to see whether if source and destination table are same.
But there is one issue unique_table does not account if source table is tmp table
in this case source and destination table can be same.

Solution:- We will change find_dup_table to not to look for temp table if
CHECK_DUP_SKIP_TEMP_TABLE flag is on.

The parameter innodb_lock_schedule_algorithm was introduced in
MariaDB Server 10.1.19, 10.2.13, 10.3.4 as part of MDEV-11039.
In MariaDB 10.1, the default value of the parameter is 'fcfs',
that is, the existing algorithm is used by default. But in
later versions of MariaDB Server, the parameter was 'vats',
enabling the new algorithm.

Because the new algorithm is triggering a debug assertion failure
that suggests corruption of the transactional lock data structures,
we will revert to the old algorithm by default until we have
resolved the problem.

Problem:
========
Truncate operation holds MDL on the table (t1) and tries to
acquire InnoDB dict_operation_lock. Purge holds dict_operation_lock
and tries to acquire MDL on the table (t1) to evaluate virtual
column expressions for indexed virtual columns.
It leads to deadlock of purge and truncate table (DDL).

Solution:
=========
If purge tries to acquire MDL on the table then it should do the following:

i) Purge should release all innodb latches (including dict_operation_lock)
before acquiring metadata lock on the table.

ii) After acquiring metadata lock on the table, it should check whether the
table was dropped or renamed. If the table is dropped then purge should
ignore the undo log record. If the table is renamed then it should
release the old MDL and acquire MDL on the new name.

iii) Once purge acquires MDL, it should use the SQL table handle for all
the remaining virtual index for the purge record.

purge_node_t: Introduce new virtual column information to know whether
the MDL was acquired successfully.

This is joint work with Marko Mäkelä.

Add an explicit redo log flush. In this test
innodb_flush_log_at_trx_commit was 2 by default.
It is also possible that this failure occurs because of MDEV-15740.

1. The changed variant did not fail without the patch for MDEV-16629
while the original test case did fail.
2. In any case the test case should go to cte_recursive_not_embedded.test
that was not created yet.

At the end of a test, 'connection default' should be in a usable state.
This was not the case, because there was a preceding 'send' without a
'reap'. If 'reap' was added, an error would be reported because the
server was restarted after the 'send'. It is easiest to 'send' from a
separate connection and do the restart from 'connection default'.

Make dict_table_t::n_ref_count private, and protect it with
a combination of dict_sys->mutex and atomics. We want to be
able to invoke dict_table_t::release() without holding
dict_sys->mutex.

When processing a query containing with clauses a call of the function
check_dependencies_in_with_clauses() before opening tables used in the
query is necessary if with clauses include specifications of recursive
CTEs.
This call was missing if such a query belonged to a stored function.
This caused misbehavior of the server: it could report a fake error
as in the test case for MDEV-16629 or the executed query could hang
as in the test cases for MDEV-16661 and MDEV-15151.

Marko mentions, it could be caused by  MDEV-15740 where InnoDB does not
flush redo log as often as it should, with innodb_flush_log_at_trx_commit=1

The workaround is to use innodb_flush_log_at_trx_commit=2, which,
according to MDEV-15740 is more durable.

One can create table with the same name for `field` and `table` `check` constraint.
For example:
`create table t(a int check(a>0), constraint a check(a>10));`
But when inserting new rows same error is always raised.
For example with
```insert into t values (-1);```
and
```insert into t values (10);```
same error `ER_CONSTRAINT_FAILED` is obtained and it is not clear which constraint is violated.
This patch solve this error so that in case if field constraint is violated the first parameter
in the error message is `table.field_name` and if table constraint is violated the first parameter
in error message is `constraint_name`.

use OPEN_ALWAYS instead, since we know file already exist.

Disks with native 4K sectors need 4K alignment and size for  unbuffered IO
(i.e for files opened with FILE_FLAG_NO_BUFFERING)

Innodb opens redo log with FILE_FLAG_NO_BUFFERING, however it always does
512byte IOs. Thus, the IO on 4K native sectors will fail, rendering
Innodb non-functional.

The fix is to check whether OS_FILE_LOG_BLOCK_SIZE is multiple of logical
sector size, and if it is not, reopen the redo log without
FILE_FLAG_NO_BUFFERING flag.

Use GetLastError() instead.

table->in_use is not always set and a KILL signal can arrive anytime.

MDEV-7257 made a dump thread to read from binlog concurrently with
writers as long as the read bytes are below a water-mark
(MYSQL_BIN_LOG::binlog_end_pos). However it appeared to be possible a
dump thread reader reach out for bytes past the water mark through a
feature of IO_CACHE that fills in the internal buffer and while doing
so it could read what the reader is not supposed to see (the bytes
above MYSQL_BIN_LOG::binlog_end_pos).

The issue is fixed with constraining the IO_CACHE buffer fill to respect
the watermark.

An added unit test proves reading from file is bound to an external
parameter
passed to {IO_CACHE::end_of_file} cache member.

rnd_pos_by_record calls ha_rnd_pos, which does the counting