MDEV-20923:UBSAN: member access within address … which does not point to an object of type 'xid_count_per_binlog'

Problem:
-------
Accessing a member within 'xid_count_per_binlog' structure results in
following error when 'UBSAN' is enabled.

member access within address 0xXXX which does not point to an object of type
'xid_count_per_binlog'

Analysis:
---------
The problem appears to be that no constructor for 'xid_count_per_binlog' is
being called, and thus the vtable will not be initialized.

Fix:
---
Defined a parameterized constructor for 'xid_count_per_binlog' class.

Original patch from Daniel Black <daniel@linux.ibm.com>, backported to 10.1.

..as they have their own tools that parses those files, such as
opensysusers[1] that handles sysusers file and opentmpfiles[2] that
handles tmpfiles.d settings

Because of this. Move both sysusers and tmpfiles 'if' function
outside systemd function, allowing independent install
Signed-off-by: Rafli Akmal <thefallenrat@artixlinux.org>

[1] - https://github.com/artix-linux/opensysusers
[2] - https://github.com/OpenRC/opentmpfiles

Changes done by vicentiu@mariadb.org, from original author patch:

Installing sysusers and tmpfiles without checking for systemd existence
means that by default, cmake will ALWAYS install these files. Our
general policy is we do not install things which are not needed.
However, there is a valid use case when these files are useful, as is
described above.

To allow this, provide an extra switch that can be enabled during
configuring by doing -DINSTALL_SYSTEMD_{SYSUSERS|TMPFILES}=True

This will use the default path INSTALL_SYSTEMD_{SYSUSERS|TMPFILES}DIR
fetched from install_layout.cmake for rpm & deb based layouts
respectively, or they must be overriden if the install_layout is
standalone.

Example:

cmake . -DINSTALL_SYSTEMD_SYSUSERS=True -DINSTALL_SYSTEMD_SYSUSERSDIR=/etc/sysusers.d

These files were installed to:

${INSTALL_SYSTEMD_SYSUSERSDIR}/sysusers.conf
${INSTALL_SYSTEMD_TMPFILESDIR}/tmpfiles.conf

Instead rename the files to more descriptive file names 'mariadb.conf'.

MDEV-21490: binlog tests fail with valgrind: Conditional jump or move depends on uninitialised value in sql_ex_info::init

Problem:
=======
P1) Conditional jump or move depends on uninitialised value(s)
    sql_ex_info::init(char const*, char const*, bool) (log_event.cc:3083)

code: All the following variables are not initialized.
----
  return ((cached_new_format != -1) ? cached_new_format :
    (cached_new_format=(field_term_len > 1 || enclosed_len > 1 ||
    line_term_len > 1 || line_start_len > 1 || escaped_len > 1)));

P2) Conditional jump or move depends on uninitialised value(s)
    Rows_log_event::Rows_log_event(char const*, unsigned
      int, Format_description_log_event const*) (log_event.cc:9571)

Code: Uninitialized values is reported for 'var_header_len' variable.
----
  if (var_header_len < 2 || event_len < static_cast<unsigned
      int>(var_header_len + (post_start - buf)))

P3) Conditional jump or move depends on uninitialised value(s)
    Table_map_log_event::pack_info(Protocol*) (log_event.cc:11553)

code:'m_table_id' is uninitialized.
----
  void Table_map_log_event::pack_info(Protocol *protocol)
  ...
  size_t bytes= my_snprintf(buf, sizeof(buf), "table_id: %lu (%s.%s)",
                              m_table_id, m_dbnam, m_tblnam);

Fix:
===
P1 - Fix)
Initialize cached_new_format,field_term_len, enclosed_len, line_term_len,
line_start_len, escaped_len members in default constructor.

P2 - Fix)
"var_header_len" is initialized by reading the event buffer. In case of an
invalid event the buffer will contain invalid data. Hence added a check to
validate the event data. If event_len is smaller than valid header length
return immediately.

P3 - Fix)
'm_table_id' within Table_map_log_event is initialized by reading data from
the event buffer. Use 'VALIDATE_BYTES_READ' macro to validate the current
state of the buffer. If it is invalid return immediately.

Setting "streamfmt=mbstream" in the "[sst]" section causes SST to fail
because the format automatically switches to 'tar' by default (insead
of mbstream).

To fix this, we need to add mbstream to the list of valid values for
the format, making it synonymous with xbstream. This must be done both
in the SST script and when parsing the options of the corresponding
utilities.

If the initialization of the wsrep provider failed, in some
cases the internal variable wrep_inited indicating that the
initialization has already been completed is still set to
"1", which then leads to confusion in the initialization
status. To solve the problem, we should set this variable
to "1" only if the wsrep provider initialization really
completed successfully.

An earlier issue has already been fixed for branch 10.4,
and this patch contains a fix for earlier versions (where
Galera 3.x is used).

test case only

The only change is a change of the version number.
In MySQL 5.6.46, the copyright comments in a number of files were changed
in mysql/mysql-server@f1a006ece7521cb02f9b961e6fad04d12ddfbab3
but there was no functional change to InnoDB code.
This was also reflected by XtraDB. We are not changing the copyright
comments in MariaDB Server for now.

Between MySQL 5.6.46 and 5.6.47, InnoDB was not changed at all.

Actually, we had forgotten to update the InnoDB version number to
5.6.46. With this change, we are updating InnoDB
from 5.6.45 to 5.6.47 and XtraDB from 5.6.45-86.1 to 5.6.46-86.2.

Remove the offending test case. This sort of error is hard to test in
all possible corner cases and thus makes the test less valuable. The
overflow error will be covered by warnings generated by the compiler,
which is much more reliable in the general case.

* size represents the size of an element in the Unique class
* full_size is used when the Unique class counts the number of
duplicates stored per element. This requires additional space per Unique
element.

The write-heavy test innodb_zip.wl6501_scale_1 timed out on
10.2 60d7011c for me.
Out of os_aio_n_segments=6, 5 are waiting for an event in
os_aio_simulated_handler(). One thread is waiting for a
write to complete in buf_dblwr_add_to_batch(), but that
would never happen, because nothing is waking up the simulated AIO
handler threads.

This hang appears to have been introduced in MySQL 5.6.12
in mysql/mysql-server@26cfde776cdf5ce61bd5cc494dfc1df28c76977f.

Dont assign Item_field variables to point to Item_string objects (even if we
don't make any dangerous calls for them).

Item_cond inherits from Item_args but doesn't store its arguments
as function arguments, which means it has zero arguments.
Don't call memcpy in this case.

(Variant #2 of the patch, which keeps the sp_head object inside the
MEM_ROOT that sp_head object owns)
(10.3 requires extra work due to sp_package, will commit a separate
patch for it)

sp_head::operator new() and operator delete() were dereferencing sp_head*
pointers to memory that didn't hold a valid sp_head object (it was
not created/already destroyed).
This caused UBSan to crash when looking up type information.

Fixed by providing static sp_head::create() and sp_head::destroy() methods.

The string doesn't appear to be null-terminated when binlog checksums are
enabled. This causes a corrupt binlog name in the error message when a
slave is ahead of the master.

  modified:   storage/connect/mysql-test/connect/disabled.def
  modified:   storage/connect/mysql-test/connect/r/grant.result
  modified:   storage/connect/mysql-test/connect/r/ini_grant.result
  modified:   storage/connect/mysql-test/connect/r/mysql_grant.result
  modified:   storage/connect/mysql-test/connect/r/xml2_grant.result
  modified:   storage/connect/mysql-test/connect/r/xml_grant.result
  modified:   storage/connect/mysql-test/connect/t/grant.inc
  modified:   storage/connect/mysql-test/connect/t/grant.test
  modified:   storage/connect/mysql-test/connect/t/grant2.test
  modified:   storage/connect/mysql-test/connect/t/ini_grant.test
  modified:   storage/connect/mysql-test/connect/t/mysql_grant.test

  Failed compile when XML table type is not supported.
  Was because XMLDEF was unconditionally called from REST table.
  modified:   storage/connect/tabrest.cpp

Problem:
========
SHOW BINLOG EVENTS FROM <pos> reports following ASAN error.

AddressSanitizer: heap-buffer-overflow on address
READ of size 1 at 0x60e00009cf71 thread T28
#0 0x55e37e034ae2 in net_field_length

Fix:
===
**Part10: Avoid reading out of buffer**

Problem:
========
SHOW BINLOG EVENTS FROM <pos> reports following assert when ASAN is enabled.

Query_log_event::Query_log_event(const char*, uint,
    const Format_description_log_event*, Log_event_type):
  Assertion `(pos) + (6) <= (end)' failed

Fix:
===
**Part9: Removed additional DBUG_ASSERT**

Problem:
========
SHOW BINLOG EVENTS FROM <pos> reports following ASAN error

AddressSanitizer: SEGV on unknown address
The signal is caused by a READ memory access.
User_var_log_event::User_var_log_event(char const*, unsigned int,
    Format_description_log_event const*)

Implemented part of upstream patch.
commit: mysql/mysql-server@a3a497ccf7ecacc900551fb1e47ea4078b45c351

Fix:
===
**Part8: added checks to avoid reading out of buffer limits**

Problem:
========
SHOW BINLOG EVENTS FROM <pos> reports following ASAN error
"heap-buffer-overflow on address" and some times it asserts.

Table_map_log_event::Table_map_log_event(const char*, uint,
    const Format_description_log_event*)
Assertion `m_field_metadata_size <= (m_colcnt * 2)' failed.

Fix:
===
**Part7: Avoid reading out of buffer**


Converted debug assert to error handler code.

Problem:
========
SHOW BINLOG EVENTS FROM <pos> reports following ASAN error

AddressSanitizer: heap-buffer-overflow on address 0x60400002acb8
Load_log_event::copy_log_event(char const*, unsigned long, int,
    Format_description_log_event const*)

Fix:
===
**Part6: Moved the event_len validation to the begin of copy_log_event function**

Problem:
========
SHOW BINLOG EVENTS FROM <pos> reports following ASAN error

AddressSanitizer: heap-buffer-overflow on address
String::append(char const*, unsigned int)
Query_log_event::pack_info(Protocol*)

Fix:
===
**Part5: Added check to catch buffer overflow**

Problem:
========
SHOW BINLOG EVENTS FROM <pos> reports following ASAN error

heap-buffer-overflow within "my_strndup" in Rotate_log_event

my_strndup /mysys/my_malloc.c:254
Rotate_log_event::Rotate_log_event(char const*, unsigned int,
    Format_description_log_event const*)

Fix:
===
**Part4: Improved the check for event_len validation**

Problem:
========
SHOW BINLOG EVENTS FROM <pos> reports following crash when ASAN is enabled.

SEGV on unknown address
in inline_mysql_mutex_destroy
in my_bitmap_free
in Update_rows_log_event::~Update_rows_log_event()

Fix:
===
**Part3: Initialize m_cols_ai.bitmap to NULL**

Problem:
========
SHOW BINLOG EVENTS FROM <pos> reports following assert when ASAN is enabled.

Rows_log_event::Rows_log_event(const char*, uint,
    const Format_description_log_event*):
Assertion `var_header_len >= 2'

Implemented part of upstream patch.
commit: mysql/mysql-server@a3a497ccf7ecacc900551fb1e47ea4078b45c351

Fix:
===
**Part2: Avoid reading out of buffer limits**