splittable derived

If one of joined tables of the processed query is a materialized derived
table (or view or CTE) with GROUP BY clause then under some conditions it
can be subject to split optimization. With this optimization new equalities
are injected into the WHERE condition of the SELECT that specifies this
derived table. The injected equalities are generated for all join orders
with which the split optimization can employed. After the best join order
has been chosen only certain of this equalities are really needed. The
others can be safely removed. If it's not done and some of injected
equalities involve expressions over semi-joins with look-up access then
the query may return a wrong result set.
This patch effectively removes equalities injected for split optimization
that are needed only at the optimization stage and not needed for execution.

Approved by serg@mariadb.com

In commit eaeb8ec4 (MDEV-24653)
an incorrect debug assertion was introduced.

btr_pcur_store_position(): If the only record in the page is the
instant ALTER TABLE metadata record, we cannot expect there to be
a successor page. The situation could be improved by MDEV-24673 later.

This partially reverts 66106130

AIX detects tell in the configure however
it really isn't there. Use the my_seek aka lseek
implementation.

If a query with implicit grouping contains in MIN/MAX set function in the
select list over a column that is a part of an index then the query
might be subject to MIN/MAX optimization. With this optimization the
server performs a look-up into an index, fetches a value of the column C
used in the MIN/MAX function and substitute the MIN/MAX expression for this
value. This allows to eliminate the table containing C from further join
processing. In order the optimization to be applied the WHERE condition
must be a conjunction of simple equality/inequality predicates or/and
BETWEEN predicates.
The bug fixed in the patch resulted in fetching a wrong value from the
index used for MIN/MAX optimization. It may happened when a BETWEEN
predicate containing the MIN/MAX value followed a strict inequality.

Approved by dmitry.shulga@mariadb.com

Before MDEV-14638, there was no race condition between the
execution of fetch_data_into_cache() and transaction commit.

fetch_data_into_cache(): Acquire trx_t::mutex before checking
trx_t::state, to prevent a concurrent transition from
TRX_STATE_COMMITTED_IN_MEMORY to TRX_STATE_NOT_STARTED
in trx_commit_in_memory().

ha_innobase::info_low(): While collecting statistics for
ANALYZE TABLE, ensure that dict_stats_process_entry_from_recalc_pool()
is not executing on the same table.

We observed result differences for the test innodb.innodb_stats because
dict_stats_empty_index() was being invoked by the background statistics
calculation while ha_innobase::analyze() was executing
dict_stats_analyze_index_level().

Tests with 4096-byte sector size confirm that it is
safe to use O_DIRECT with page_compressed tables.
That had been disabled on Linux, in an attempt to fix MDEV-21584
which had been filed for the O_DIRECT problems earlier.

The fil_node_t::block_size was being set mostly correctly until
commit 10dd290b (MDEV-17380)
introduced a regression in MariaDB Server 10.4.4.

fil_node_t::read_page0(): Initialize fil_node_t::block_size.
This will probably make similar code in fil_space_extend_must_retry()
redundant, but we play it safe and will not remove that code.

Thanks to Vladislav Vaintroub for testing this on Microsoft Windows
using an old-fashioned rotational hard disk with 4KiB sector size.

Reviewed by: Vladislav Vaintroub

This patch implements OS_DATA_FILE_NO_O_DIRECT on Windows.

This had been originally added in
mysql/mysql-server@192bb153b675fe09037a53e456a79eee7211e3a7
with the motivation to disable O_DIRECT for the dedicated tablespace
for temporary tables. In MariaDB Server,
commit 5eb53955 (MDEV-12227)
should be a better solution.

The code became orphaned later in
mysql/mysql-server@c61244c0e6c58727cffebfb312ac415a463fa0fe
and it had been applied to MariaDB Server 10.2.2 in
commit 2e814d47 and
commit fec844ac.

Thanks to Vladislav Vaintroub for spotting this.

MDEV-24916 : Assertion `current_stmt_binlog_format == BINLOG_FORMAT_STMT || current_stmt_binlog_format == BINLOG_FORMAT_ROW' failed in THD::is_current_stmt_binlog_format_row

Store old value of binlog format before wsrep code so that
if we bail out because wsrep is not ready for connections
we can restore binlog format correctly.

MDEV-18874 : Galera test MW-286 causes Mutex = TTASEventMutex<GenericPolicy>]: Assertion `!is_owned()' failed. assertion

MDEV-24649 galera.galera_bf_lock_wait MTR failed with sigabrt: Assertion `!is_ow
ned()' failed in sync0policy.ic on MutexDebug with Mutex = TTASEventMutex<GenericPolicy>

Bug was fixed as part of MDEV-23328, this just adds test cases to
regression set.

As evidenced by converstation on Zulip,
knowing where to install a plugin is important too.

Adding any unknown option to the "[mysqld_safe]" section makes
mysqld impossible to start with mysqld_multi. For example, after
adding the unknown option "numa_interleave" to the "[mysqld_safe]"
section, mysqld_multi exits with the following diagnostics:

[ERROR] /usr/local/mysql/bin/mysqld: unknown option '--numa_interleave'

To get rid of this behavior, this patch adds the passing of the default
group suffix from mysqld_multi to the mysqld_safe side.

Change the default timestamping URL, again
http://timestamp.globalsign.com/?signature=sha2 seems to work fine atm

connection.

Ignore harmless X509_R_CERT_ALREADY_IN_HASH_TABLE, similar to how Curl or
other projects treat it.

Keyvalue can be longer than REC_VERSION_56_MAX_INDEX_COL_LEN
and this leads out-of-array reference. Use dynamic memory
allocation using actual max length of key value.

Online log for insert operation of redundant table fails with
index->is_instant() assert. Purge can reset the n_core_fields when
alter is waiting to upgrade MDL for commit phase of DDL. In the
meantime, any insert DML tries to log the operation fails with
index is not being instant.

row_log_get_n_core_fields(): Get the n_core_fields of online log
for the given index.

rec_get_converted_size_comp_prefix_low(): Use n_core_fields of online
log when InnoDB calculates the size of data tuple during redundant
row format table rebuild.

rec_convert_dtuple_to_rec_comp(): Use n_core_fields of online log
when InnoDB does the conversion of data tuple to record during
redudant row format table rebuild.

- Adding the test case which has more than 129 instant columns.

The query causing the issue here has implicit grouping for we
have to produce one row with special values for the aggregates
(depending on each aggregate function), and NULL values for all
non-aggregate fields.

The subselect item where implicit grouping was being done,
null_value for the subselect item was not being set for
the case when the implicit grouping produces NULL values
for the items in the select list of the subquery.
This which was leading to the crash.

The fix would be to set the null_value when all the values
for the row column have NULL values.

Further changes are

1) etting null_value for Item_singlerow_subselect only
   after val_* functions have been called.
2) Introduced a parameter null_value_inside to Item_cache that
   would store be set to TRUE if any of the arguments of the
   Item_cache are null.

Reviewed And co-authored by Monty

MDEV-25105 (commit 7a4fbb55)
in MariaDB 10.6 will refuse the innodb_checksum_algorithm
values none, innodb, strict_none, strict_innodb.

We will issue a deprecation warning if innodb_checksum_algorithm
is set to any of these non-default unsafe values.

innodb_checksum_algorithm=crc32 was made the default in
MySQL 5.7 and MariaDB Server 10.2, and given that older versions
of the server have reached their end of life, there is no valid
reason to use anything else than innodb_checksum_algorithm=crc32
or innodb_checksum_algorithm=strict_crc32 in MariaDB 10.3.

Reviewed by: Sergei Golubchik

This bug manifested itself when executing queries with multiple reference
to a CTE specified by a query expression with union and having its
column names explicitly declared. In this case the server returned a bogus
error message about unknown column name. It happened because while for the
first reference to the CTE the names of the columns returned by the CTE
specification were properly changed to match the CTE definition for the
other references it was not done. This was a consequence  of not quite
complete code of the function With_element::clone_parsed_spec() that forgot
to set the reference to the CTE definition for unit structures representing
non-first CTE references.

Approved by dmitry.shulga@mariadb.com

This bug could affect multi-way join queries with embedded outer joins that
contained a conjunctive IS NULL predicate over a non-nullable column from
inner table of an outer join. The predicate could occur in WHERE condition
or in ON condition. Due to this bug a wrong result set could be returned by
the query. The bug manifested itself only when join buffers were employed
for join operations.

The problem appeared because
- a bug in the function JOIN_CACHE::get_match_flag_by_pos that not always
  returned proper match flags for embedding outer joins stored together
  with table rows put a join buffer.
- bug in the function JOIN_CACHE::join_matching_records that not always
  correctly determined that a row from the buffer could be skipped due
  to applied 'not_exists' optimization.
Example:
  SELECT * FROM t1 LEFT JOIN ((t2 LEFT JOIN t3 ON c = d) JOIN t4) ON b = e
    WHERE e IS NULL;

The patch introduces a new function that finds the match flag for a record
from join buffer specifying the buffer where this flag has to be found.
The function is called JOIN_CACHE::get_match_flag_by_pos_from_join_buffer().
Now this function rather than JOIN_CACHE::get_match_flag_by_pos() is used
in JOIN_CACHE::skip_if_matched() to check whether a record from the join
buffer must be ignored when extending the record by null complements.
Also the code of the function JOIN_CACHE::skip_if_not_needed_match() has
been changed. The function checks whether a record from the join buffer
still may produce some useful extensions.
Also some clarifying comments has been added.

Approved by monty@mariadb.com.

InnoDB set the space in dict_table_t as NULL when table
is discarded. So InnoDB shouldn't use the space present
in table to detect whether the given tablespace is
temporary tablespace.

btr_node_ptr_max_size(): Let us remove the debug assertion that was
added in MDEV-14637. The assertion assumed that no additional
indexes exist in mysql.innodb_index_stats or mysql.innodb_table_stats.
The code path is working around an incorrect definition of a table,
interpreting VARCHAR(64) as the more correct VARCHAR(199).

No test case will be added, because MDEV-24579 proves that executing
DDL on the statistics tables involves a race condition. The test
case included the following:

	ALTER TABLE mysql.innodb_index_stats ADD KEY (stat_name);
	CREATE TABLE t (a INT) ENGINE=InnoDB STATS_PERSISTENT=1;

A bogus error message was issued for any outer references occurred in
ON expressions used in subqueries. This prevented execution of queries
containing subqueries as soon as they used outer references in their ON
clauses. This happened because the Name_resolution_context structure
created for any ON expression erroneously had the field outer_context set
to NULL. The fields select_lex of this structure was not set correctly
either.

The idea of the fix was taken from mysql code of the function
push_new_name_resolution_context().

Approved by dmitry.shulga@mariadb.com

MDEV-24868 Server crashes in optimize_schema_tables_memory_usage after select from information_schema.innodb_sys_columns

optimize_schema_tables_memory_usage() crashed when its argument included
TABLE struct that was not fully initialized.

To prevent such a crash, we check if a table is an information schema table at
the beginning of each iteration.

Closes #1768

As a special hack global memory_used isn't SHOW_LONG_STATUS
but still relies on calc_sum_of_all_status() being called.

followup for 63f91927

Incorrect processing of an auto-incrementing field in the
WSREP-related code during applying transactions results in
a duplicate key being created. This is due to the fact that
at the beginning of the write_row() and update_row() functions,
the values of the auto-increment parameters are used, which
are read from the parameters of the current thread, but further
along the code other values are used, which are read from global
variables (when applying a transaction). This can happen when
the cluster configuration has changed while applying a transaction
(for example in the high_priority_service mode for Galera 4).
Further during IST processing duplicating key is detected, and
processing of the DB_DUPLICATE_KEY return code (inside innodb,
in the write_row() handler) results in a call to the
wsrep_thd_self_abort() function.