Assertions failed due to incorrect handling of the --tc-heuristic-recover
option when InnoDB is in read-only mode either due to innodb_read_only=1
or innodb_force_recovery>3. InnoDB failed to refuse a XA COMMIT or
XA ROLLBACK operation, and there were errors in the error handling in
the upper layer.

This was fixed by making InnoDB XA operations respect the
high_level_read_only flag. The InnoDB part of the fix and
parts of the test main.tc_heuristic_recover were provided
by Marko Mäkelä.

LOCK_log mutex lock/unlock had to be added to fix MDEV-13438.
The measure is confirmed by mysql sources as well.

For testing of the conflicting option combination, mysql-test-run is
made to export a new $MYSQLD_LAST_CMD. It holds the very last value
generated by mtr.mysqld_start().  Even though the options have been
also always stored in $mysqld->{'started_opts'} there were no access
to them beyond the automatic server restart by mtr through the expect
file interface.

Effectively therefore $MYSQLD_LAST_CMD represents a more general
interface to $mysqld->{'started_opts'} which can be used in wider
scopes including server launch with incompatible options.

Notice another existing method to restart the server with incompatible
options relying on $MYSQLD_CMD is is aware of $mysqld->{'started_opts'}
(the actual options that the server is launched by mtr). In order to use
this method they would have to be provided manually.

NOTE: When merging to 10.2, the file search_pattern_in_file++.inc
should be replaced with the pre-existing search_pattern_in_file.inc.

- make re-bootstrap run with all extra options, not only InnoDB ones
- re-use previously created bootstrap.sql
- add --console
- fix debian patch to keep it applicable

Signed-off-by: Daniel Black <daniel.black@au.ibm.com>

A better way of finding mysqld

The motivation for this is that Perl is moving towards not having
current directory ./ in @INC by default. This is causing
mysql-test-run.pl to fail in latest Debian Unstable:

  https://lists.debian.org/debian-devel-announce/2016/08/msg00013.html

However, we have `use "lib"`, there is no need for current directory
in @INC, except for a gross hack. In mtr_cases.pm, there is a
`require "mtr_misc.pl"`, which hides mtr_misc.pl away in mtr_cases
namespace. And things only work because mysql-test-run.pl loads it
with a different name, `require "lib/mtr_misc.pl"`! (Perl will
`require` only once for each unique filename).

Fix this by only using `require` in main program, and referencing
functions with :: scope from other namespaces. For multi-use in
different namespaces, proper `use` modules should be used.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

               FROM THE CURRENT DIRECTORY

DESCRIPTION
===========
When 'mysqlaccess' tool is run, it reads (and executes) the
content of its configuration file 'mysqlaccess.conf' from
the current directory. This is not a recommended behaviour
as someone with ill intentions can insert malicious
instructions into this file which could be executed
whenever this tool is run.

ANALYSIS
========
The configuration file is presently looked for, in the
following folders (in given order):
1. Current directory
2. SYSCONFDIR       //This gets expanded
3. /etc/

Owing to the reasons mentioned above, we should not permit
the file to be in the current directory. Since the other
two folders are assumed to be accessible only to authorized
people, the config file is safe to be read from there.

FIX
===
Modified the script so that it looks for the config file
now in the following two folders (in the given order):
1. SYSCONFDIR
2. /etc/

If it's absent from above locations but present in current
directory, an error is thrown asking the user to move the
file to one of the above locations and retry.

NOTE
====
The location paths and their precedence are not documented
for this tool. It needs to be noted as part of the
associated documentation.

This removes functionality of where ./mtr --mem /tmp/dir could be a directory.

Now MTR_MEM=/tmp/dir ./mtr is needed.

The case where MTR_MEM=/tmp/dir ./mtr --mem has the equivalent effect.
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>

--mem works better as a pure flag, because it can be followed by command-line arguments (test names). If the option is allowed to have a value, the test name which directly follows it will be treated as the option value instead. It is possible to implement workarounds to avoid this, but they would not be completely reliable, and there is no practical purpose of such extension of functionality to justify them.

Problem:- When MariaDB is compiled with jemalloc support, And we run mtr valgrind
test, valgrind interferes with libjemalloc and returns false errors.

Solution:- Run valgrind with --soname-synonyms=somalloc=libjemalloc* or
--soname-synonyms=somalloc=NONE depending on whether we are dynamically
linking or statically linking.
Signed-off-by: Sachin Setiya <sachin.setiya@mariadb.com>

The standalone warning is not a sign of a problem, just of slowness,
so it should be added to global suppressions. If a real problem
happens, there will be other errors

* Revert "Make --mem a pure flag. If there is need to specifically set the location"

This reverts commit 716621db.

* MDEV-11619: mtr: when --mem is pure flag, conflicts with $MTR_MEM

Conflicts occurs when MTR_MEM=/xxx/yy ./mtr --mem is invoked. Here
the --mem option overrides opt_mem leaving the default path to be chosen.

This change makes when MTR_MEM set, opt_mem, the flag, is also
set. Both the environment and flag can no be set without conflicting.
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>

* MDEV-11619: if opt_mem is a path include it first

* MDEV-11619: MTR_MEM locations - don't follow symlinks

From Bjorn Munch it seems symlinks can confuse some
tests. Lets just avoid those.

(ref: https://github.com/mysql/mysql-server/pull/116#issuecomment-268479774)
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>

Description :
=============
When a MTR test run is started, it initializes the server and creates
the datadir under '$MYSQL_TEST_DIR/var'('/tmp/var' or '/dev/shm/var'
if --mem option is used) location and then copies it to the datadir
location of server(s).

If $parallel == 1, datadir location of the server is
'$MYSQL_TEST_DIR/var/data'. If $parallel > 1, datadir location of any
server is '$MYSQL_TEST_DIR/var/<thread_num>/data'.

This is the reason MTR searches for the initialized datadir in 2
locations('$opt_vardir' and '$opt_vardir/..') from the current vardir
location..

But this can cause few problems. If a directory with the name 'data'
already exists under '$MYSQL_TEST_DIR' and if the MTR run is started
with parallel value 1, then

1. copytree($install_db, '$opt_vardir/..') command will fail if the
user doesn't have the access permission to '$MYSQL_TEST_DIR/data'
directory.
2. Unnecessary contents from '$MYSQL_TEST_DIR/data' directory will be
copied to server datadir location and this might affect the server
startup.

Fix :
=====
Depending on the $parallel value decide whether the path for the
initialize datadir is "$opt_vardir"(i.e $parallel = 1) or
"$opt_vardir/.."(i.e $parallel > 1).
Reviewed-by: Deepa Dixit <deepa.dixit@oracle.com>
Reviewed-by: Srikanth B R <srikanth.b.r@oracle.com>
RB: 14773

              RESTRICTED IN ALL GA RELEASES

Back port of WL#6782 to 5.5 and 5.6. This also includes
back port of Bug#20771331, Bug#20741572 and Bug#20770671.
Bug#24695274 and Bug#24679907 are also handled along with
this.

              RESTRICTED IN ALL GA RELEASES

Back port of WL#6782 to 5.5 and 5.6. This also includes
back port of Bug#20771331, Bug#20741572 and Bug#20770671.
Bug#24695274 and Bug#24679907 are also handled along with
this.

The motivation for this is that Perl is moving towards not having
current directory ./ in @INC by default. This is causing
mysql-test-run.pl to fail in latest Debian Unstable:

  https://lists.debian.org/debian-devel-announce/2016/08/msg00013.html

However, we have `use "lib"`, there is no need for current directory
in @INC, except for a gross hack. In mtr_cases.pm, there is a
`require "mtr_misc.pl"`, which hides mtr_misc.pl away in mtr_cases
namespace. And things only work because mysql-test-run.pl loads it
with a different name, `require "lib/mtr_misc.pl"`! (Perl will
`require` only once for each unique filename).

Fix this by only using `require` in main program, and referencing
functions with :: scope from other namespaces. For multi-use in
different namespaces, proper `use` modules should be used.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

The issue was that when running with valgrind the wait for master_pos_Wait()
was not long enough.

This patch also fixes two other failures that could affect rpl_mdev6020:
- check_if_conflicting_replication_locks() didn't properly check domains
- 'did_mark_start_commit' was after signals to other threads was sent which could
  get the variable read too early.

- Fixed typos
- Added --core-on-failure to mysql-test-run
- More DBUG_PRINT in viosocket.c
- Don't forget CLIENT_REMEMBER_OPTIONS for compressed slave protocol
- Removed not used stage variables

removed

When running mysql-test-run without a parallel parameter, it might
be impossible to find a free port.

Signed-off-by: Daniel Black <daniel.black@au.ibm.com>

This fixes:
MDEV-7693 MariaDB - mysql-test - SUITE:funcs_1 - funcs_1.is_tables_mysql is failing on TABLE_COLLATION /DATABASE_COLLATION
MDEV-8487 mysql-test - main.information_schema 'xtradb' fails
MDEV-8486 mysql-test - main.bootstrap fails

https://github.com/MariaDB/server/pull/56

it was never doing anything anyway

(cherry picked from commit 3eb933e0eb55f962404a04741767177e12a9885f)

Conflicts:
	mysql-test/mysql-test-run.pl

Do not attempt to run unix-specific code on Windows

Analysis: At check_trx_exists function InnoDB allocates
a new trx if no trx is found from thd but this newly
allocated trx is not registered to thd. This is unsafe,
because nothing prevents InnoDB plugin from being uninstalled
while there's active transaction. This can cause crashes, hang
and any other odd behavior. It may also corrupt stack, as
functions pointers are not available after dlclose.

Fix: The fix is to use thd_set_ha_data() when
manipulating per-connection handler data. It does appropriate
plugin locking.

One can not see in PB2 test logs which unit tests have been run
and passed.

This patchs adds an option --unit-tests-report to mtr which
include the ctest report in mtr output.  It will also turn on unit
testing if not explicitly turned off with --no-unit-tests or
equivalent.

In manual runs one can always look in the ctest.log file in mtr
vardir.

--unit-tests are replaced with --unit-tests-report in files under
mysql-test/collections/ to activate report in PB2.

Add some suppressions that were missing. They are for if a STOP SLAVE is
executed early during IO thread startup, when it is negotiating with the
master. The master connection may be killed in the middle of a
mysql_real_query(), which is not a test failure if it is a network error.

This also caught one real code error, fixed with this commit: The I/O thread
would fail to automatically reconnect if a network error happened while
fetching the value of @@GLOBAL.gtid_domain_id.

1) fix 5.5.42 vs 5.5.41 problem, nested links;
2) fix older MariaDB vs MySQL problem, var_auto_** dirs were not
   removed from the tmpfs location.
Both problems were caused by vardir being expanded to real path
too early, in two different places in the script code

(Without this, it happened for me that realpath() failed returning
undef for the default vardir. This in turn caused mysql-test-run.pl to
delete the source mysql-test/ directory.)

Backport from 10.1, it's not nice to get one's source directory nuked
by a rouge mysql-test-run.

mtr internally does the following:
1. $default_vardir=....
2. $opt_vardir=$default_vardir unless $opt_vardir;
3. $opt_vardir=realpath $opt_vardir unless IS_WINDOWS
4. if ($opt_vardir eq $default_vardir) { .... use /dev/shm ... }

thus we have to realpath $default_datadir too, otherwise
the comparison logic might be broken

This patch ports the work that facebook has performed
to make innochecksum handle compressed tables.
the basic idea is to use actual innodb-code to perform
checksum verification rather than duplicating in innochecksum.cc.
to make this work, innodb code has been annotated with
lots of #ifndef UNIV_INNOCHECKSUM so that it can be
compiled outside of storage/innobase.

A new testcase is also added that verifies that innochecksum
works on compressed/non-compressed tables.

Merged from commit fabc79d2ea976c4ff5b79bfe913e6bc03ef69d42 
from https://code.google.com/p/google-mysql/

The actual steps to produce this patch are:

    take innochecksum from 5.6.14
    apply changes in innodb from facebook patches needed to make innochecksum compile
    apply changes in innochecksum from facebook patches
    add handcrafted testcase

The referenced facebook patches used are:

    https://github.com/facebook/mysql-5.6/commit/91e25120e75272db4cdbc07d0e45877d9dea5715
    https://github.com/facebook/mysql-5.6/commit/847fe76ea5239b09fa361b023c56e6be76d32046
    https://github.com/facebook/mysql-5.6/commit/1135628a5a9b3412621b93233478f3804bcef51a
    https://github.com/facebook/mysql-5.6/commit/4dbf7c240ce2f08b7d6572d9452c9779ce90641c

Let mtr handle SIGHUP the same way as SIGINT. This solves stale mysqld processes
left after broken/closed ssh connection.

The test case tried to trigger a DEBUG_SYNC point at the end of a SELECT
SLEEP(5) statement. It did this by using EXECUTE 2, intending to trigger first
at the end of SET DEBUG_SYNC, and second at the end of the SELECT SLEEP(5).

However, in --ps-protocol mode, this does not work, because the SELECT is
executed in two steps (Prepare followed by Execute). Thus, the DEBUG_SYNC got
triggered too early, during the Prepare stage rather than Execute, and the
test case could race and information_schema.processlist see the thread in the
wrong state.

This patch fixes by changing the way the DEBUG_SYNC point is triggered. Now we
add a DBUG injection inside the code for SLEEP(5). This ensures that the
DEBUG_SYNC point is not activated until the SLEEP(5) is running, ensuring
that the following wait for completion will be effective.

change SSL methods to be SSLv23 (according to openssl manpage:
"A TLS/SSL connection established with these methods may understand
the SSLv2, SSLv3, TLSv1, TLSv1.1 and TLSv1.2 protocols") from
TLSv1 methods, that go back to the initial SSL implementation
in MySQL in 2001.

OpenSSL default ciphers are different if TLSv1.2 is enabled,
so tests need to take this into account.

Description:- Replace, the string replacement utility fails
on the windows platform.

Analysis:- The utility takes files and multiple string
pairs as input. In the windows platform, the utility maps
the standard input incorrectly (Considers 0 as the fd for
standard input instead of 2048 on windows). Hence when the
input file is supplied to the utility due to the incorrect
mapping, a bad file descriptor error is generated causing
the utility to exit with an error.

Fix:- "my_fileno()" is used for getting the correct file
despscriptor for standard input.

Description:- Replace, the string replacement utility fails
on the windows platform.

Analysis:- The utility takes files and multiple string
pairs as input. In the windows platform, the utility maps
the standard input incorrectly (Considers 0 as the fd for
standard input instead of 2048 on windows). Hence when the
input file is supplied to the utility due to the incorrect
mapping, a bad file descriptor error is generated causing
the utility to exit with an error.

Fix:- "my_fileno()" is used for getting the correct file
despscriptor for standard input.