by compound index

This typo bug may lead to wrong result sets for equi-join queries where
the join operation is supported by a compound index such that the order of
its components differs from the order of the corresponding columns in
the table the index belongs to. The bug manifests itself only when usage
of the BNLH algorithm is forced.

The fix for the bug was provided by Chu Huaxing.

As suggested by Vladislav Vaintroub, let us remove misleading
and malformatted startup messages.

Even if the global variable srv_use_atomic_writes were set, we would
still invoke my_test_if_atomic_write() to check if writes are atomic
with a particular page size.

When using the default innodb_page_size=16k, page writes should be
atomic on NTFS when using ROW_FORMAT=COMPRESSED and KEY_BLOCK_SIZE<=4.

Disabling srv_use_atomic_writes when innodb_file_per_table=OFF does
not make sense, because that is a dynamic parameter.

We also correct the documentation string of innodb_use_atomic_writes
and remove the duplicate variable innobase_use_atomic_writes.

The debug parameter innodb_simulate_comp_failures injected compression
failures for ROW_FORMAT=COMPRESSED tables, breaking the pre-existing
logic that I had implemented in the InnoDB Plugin for MySQL 5.1 to prevent
compressed page overflows. A much better check is already achieved by
defining UNIV_ZIP_COPY at the compilation time.
(Only UNIV_ZIP_DEBUG is part of cmake -DWITH_INNODB_EXTRA_DEBUG=ON.)

AIX doesn't have getgrouplist so ensure function is checked.

The HAVE_POSIX_GETGROUPLIST check was insufficient.

Mariabackup SST fails if "--log-bin" option is added with no value
to command line parameters at server startup. This is because the SST
scripts do not correctly interpret the "--- log-bin" option without a
value. This patch adds correct handling of the "--log-bin" parameter
without value to the general part of the parameter parsing (for SST
scripts) and fixes the problem. Also added a test that checks the
correct operation of the server after the fix.

This partially reverts 66106130

AIX detects tell in the configure however
it really isn't there. Use the my_seek aka lseek
implementation.

If a query with implicit grouping contains in MIN/MAX set function in the
select list over a column that is a part of an index then the query
might be subject to MIN/MAX optimization. With this optimization the
server performs a look-up into an index, fetches a value of the column C
used in the MIN/MAX function and substitute the MIN/MAX expression for this
value. This allows to eliminate the table containing C from further join
processing. In order the optimization to be applied the WHERE condition
must be a conjunction of simple equality/inequality predicates or/and
BETWEEN predicates.
The bug fixed in the patch resulted in fetching a wrong value from the
index used for MIN/MAX optimization. It may happened when a BETWEEN
predicate containing the MIN/MAX value followed a strict inequality.

Approved by dmitry.shulga@mariadb.com

ha_innobase::info_low(): While collecting statistics for
ANALYZE TABLE, ensure that dict_stats_process_entry_from_recalc_pool()
is not executing on the same table.

We observed result differences for the test innodb.innodb_stats because
dict_stats_empty_index() was being invoked by the background statistics
calculation while ha_innobase::analyze() was executing
dict_stats_analyze_index_level().

Tests with 4096-byte sector size confirm that it is
safe to use O_DIRECT with page_compressed tables.
That had been disabled on Linux, in an attempt to fix MDEV-21584
which had been filed for the O_DIRECT problems earlier.

The fil_node_t::block_size was being set mostly correctly until
commit 10dd290b (MDEV-17380)
introduced a regression in MariaDB Server 10.4.4.

fil_node_t::read_page0(): Initialize fil_node_t::block_size.
This will probably make similar code in fil_space_extend_must_retry()
redundant, but we play it safe and will not remove that code.

Thanks to Vladislav Vaintroub for testing this on Microsoft Windows
using an old-fashioned rotational hard disk with 4KiB sector size.

Reviewed by: Vladislav Vaintroub

This patch implements OS_DATA_FILE_NO_O_DIRECT on Windows.

This had been originally added in
mysql/mysql-server@192bb153b675fe09037a53e456a79eee7211e3a7
with the motivation to disable O_DIRECT for the dedicated tablespace
for temporary tables. In MariaDB Server,
commit 5eb53955 (MDEV-12227)
should be a better solution.

The code became orphaned later in
mysql/mysql-server@c61244c0e6c58727cffebfb312ac415a463fa0fe
and it had been applied to MariaDB Server 10.2.2 in
commit 2e814d47 and
commit fec844ac.

Thanks to Vladislav Vaintroub for spotting this.

MDEV-24916 : Assertion `current_stmt_binlog_format == BINLOG_FORMAT_STMT || current_stmt_binlog_format == BINLOG_FORMAT_ROW' failed in THD::is_current_stmt_binlog_format_row

Store old value of binlog format before wsrep code so that
if we bail out because wsrep is not ready for connections
we can restore binlog format correctly.

MDEV-18874 : Galera test MW-286 causes Mutex = TTASEventMutex<GenericPolicy>]: Assertion `!is_owned()' failed. assertion

MDEV-24649 galera.galera_bf_lock_wait MTR failed with sigabrt: Assertion `!is_ow
ned()' failed in sync0policy.ic on MutexDebug with Mutex = TTASEventMutex<GenericPolicy>

Bug was fixed as part of MDEV-23328, this just adds test cases to
regression set.

As evidenced by converstation on Zulip,
knowing where to install a plugin is important too.

Adding any unknown option to the "[mysqld_safe]" section makes
mysqld impossible to start with mysqld_multi. For example, after
adding the unknown option "numa_interleave" to the "[mysqld_safe]"
section, mysqld_multi exits with the following diagnostics:

[ERROR] /usr/local/mysql/bin/mysqld: unknown option '--numa_interleave'

To get rid of this behavior, this patch adds the passing of the default
group suffix from mysqld_multi to the mysqld_safe side.

Change the default timestamping URL, again
http://timestamp.globalsign.com/?signature=sha2 seems to work fine atm

connection.

Ignore harmless X509_R_CERT_ALREADY_IN_HASH_TABLE, similar to how Curl or
other projects treat it.

Keyvalue can be longer than REC_VERSION_56_MAX_INDEX_COL_LEN
and this leads out-of-array reference. Use dynamic memory
allocation using actual max length of key value.

The query causing the issue here has implicit grouping for we
have to produce one row with special values for the aggregates
(depending on each aggregate function), and NULL values for all
non-aggregate fields.

The subselect item where implicit grouping was being done,
null_value for the subselect item was not being set for
the case when the implicit grouping produces NULL values
for the items in the select list of the subquery.
This which was leading to the crash.

The fix would be to set the null_value when all the values
for the row column have NULL values.

Further changes are

1) etting null_value for Item_singlerow_subselect only
   after val_* functions have been called.
2) Introduced a parameter null_value_inside to Item_cache that
   would store be set to TRUE if any of the arguments of the
   Item_cache are null.

Reviewed And co-authored by Monty

This bug manifested itself when executing queries with multiple reference
to a CTE specified by a query expression with union and having its
column names explicitly declared. In this case the server returned a bogus
error message about unknown column name. It happened because while for the
first reference to the CTE the names of the columns returned by the CTE
specification were properly changed to match the CTE definition for the
other references it was not done. This was a consequence  of not quite
complete code of the function With_element::clone_parsed_spec() that forgot
to set the reference to the CTE definition for unit structures representing
non-first CTE references.

Approved by dmitry.shulga@mariadb.com

This bug could affect multi-way join queries with embedded outer joins that
contained a conjunctive IS NULL predicate over a non-nullable column from
inner table of an outer join. The predicate could occur in WHERE condition
or in ON condition. Due to this bug a wrong result set could be returned by
the query. The bug manifested itself only when join buffers were employed
for join operations.

The problem appeared because
- a bug in the function JOIN_CACHE::get_match_flag_by_pos that not always
  returned proper match flags for embedding outer joins stored together
  with table rows put a join buffer.
- bug in the function JOIN_CACHE::join_matching_records that not always
  correctly determined that a row from the buffer could be skipped due
  to applied 'not_exists' optimization.
Example:
  SELECT * FROM t1 LEFT JOIN ((t2 LEFT JOIN t3 ON c = d) JOIN t4) ON b = e
    WHERE e IS NULL;

The patch introduces a new function that finds the match flag for a record
from join buffer specifying the buffer where this flag has to be found.
The function is called JOIN_CACHE::get_match_flag_by_pos_from_join_buffer().
Now this function rather than JOIN_CACHE::get_match_flag_by_pos() is used
in JOIN_CACHE::skip_if_matched() to check whether a record from the join
buffer must be ignored when extending the record by null complements.
Also the code of the function JOIN_CACHE::skip_if_not_needed_match() has
been changed. The function checks whether a record from the join buffer
still may produce some useful extensions.
Also some clarifying comments has been added.

Approved by monty@mariadb.com.

btr_node_ptr_max_size(): Let us remove the debug assertion that was
added in MDEV-14637. The assertion assumed that no additional
indexes exist in mysql.innodb_index_stats or mysql.innodb_table_stats.
The code path is working around an incorrect definition of a table,
interpreting VARCHAR(64) as the more correct VARCHAR(199).

No test case will be added, because MDEV-24579 proves that executing
DDL on the statistics tables involves a race condition. The test
case included the following:

	ALTER TABLE mysql.innodb_index_stats ADD KEY (stat_name);
	CREATE TABLE t (a INT) ENGINE=InnoDB STATS_PERSISTENT=1;

A bogus error message was issued for any outer references occurred in
ON expressions used in subqueries. This prevented execution of queries
containing subqueries as soon as they used outer references in their ON
clauses. This happened because the Name_resolution_context structure
created for any ON expression erroneously had the field outer_context set
to NULL. The fields select_lex of this structure was not set correctly
either.

The idea of the fix was taken from mysql code of the function
push_new_name_resolution_context().

Approved by dmitry.shulga@mariadb.com

As a special hack global memory_used isn't SHOW_LONG_STATUS
but still relies on calc_sum_of_all_status() being called.

followup for 63f91927

Incorrect processing of an auto-incrementing field in the
WSREP-related code during applying transactions results in
a duplicate key being created. This is due to the fact that
at the beginning of the write_row() and update_row() functions,
the values of the auto-increment parameters are used, which
are read from the parameters of the current thread, but further
along the code other values are used, which are read from global
variables (when applying a transaction). This can happen when
the cluster configuration has changed while applying a transaction
(for example in the high_priority_service mode for Galera 4).
Further during IST processing duplicating key is detected, and
processing of the DB_DUPLICATE_KEY return code (inside innodb,
in the write_row() handler) results in a call to the
wsrep_thd_self_abort() function.

Since we do not stop at corrupted page error, there is no reason to log
a backup error.

row_prebuilt_t::m_no_prefetch: Remove (it was always false).
row_prebuilt_t::m_read_virtual_key: Remove (it was always false).

Only ha_innopart ever set these fields.

innobase_rename_table(): Invoke dict_stats_wait_bg_to_stop_using_table()
to ensure that dict_stats_update() cannot be accessing the table name
that we will be modifying. If we are executing RENAME rather than TRUNCATE,
reset the flag at the end so that persistent statistics can be calculated
again.

The race condition was encountered with ASAN and rr.
Sorry, there is no test case, like there is for nothing related to
dict_stats_wait_bg_to_stop_using_table(). The entire code is an ugly
work-around for the failure of dict_stats_process_entry_from_recalc_pool()
to acquire MDL.

Note: It appears that an ALTER TABLE that is not rebuilding the table
will fail to reset the flag that blocks the processing of statistics.

row_number() over () window function can be used without any column in the OVER
clause. Additionally, the item doesn't reference any tables, as it's not
effectively referencing any table. Rather it is specifically built based
on the end temporary table used for window function computation.

This caused remove_const function to wrongly drop it from the ORDER
list. Effectively, we shouldn't be dropping any window function from the
ORDER clause, so adjust remove_const to account for that.

Reviewed by: Sergei Petrunia sergey@mariadb.com

In btr_index_rec_validate(), externally stored column
check is missing while matching the length of the field
with the length of the field data stored in record.
Fetch the length of the externally stored part and compare it
with the fixed field length.