MDEV-4485: Master did not allow slave to connect from the very start (empty GTID pos) if GTIDs from other multi_source master was present

The idea in the code was to protect the user that tries to connect a slave
to a master with completely different domains than what was intended. If
none of the domains in the start position are present at all in the master
binlog, we gave an error.

However, this is a stupid idea. Because when a slave connects to a master
to start replication from the very start of binlogs - such as when setting
up new master->slave servers from scratch - there will be just this
situation, the requested slave position is empty for all the domains in the
master's binlog.

So the code that gives this error is wrong, and the solution is simply to
remove it.

When @@GLOBAL.gtid_strict_mode=1, then certain operations result
in error that would otherwise result in out-of-order binlog files
between servers.

GTID sequence numbers are now allocated independently per domain;
this results in less/no holes in GTID sequences, increasing the
likelyhood that diverging binlogs will be caught by the slave when
GTID strict mode is enabled.

The problem was the Gtid_list event which is logged to the binlog in
10.0 and is not understood by the 5.5 server.

This event is supposed to be replaced with a dummy event for 5.5
servers. But the very first event logged in the very first binlog
has an empty list of GTID, which makes the event too short to be
replacable with an empty event.

The fix is to pad the empty Gtid_list event to be big enough to
be replacable by a dummy event.

Change of user interface to be more logical and more in line with expectations
to work similar to old-style replication.

User can now explicitly choose in CHANGE MASTER whether binlog position is
taken into account (master_gtid_pos=current_pos) or not (master_gtid_pos=
slave_pos) when slave connects to master.

@@gtid_pos is replaced by three separate variables @@gtid_slave_pos (can
be set by user, replicated GTIDs only), @@gtid_binlog_pos (read only), and
@@gtid_current_pos (a combination of the two, most recent GTID within each
domain). mysql.rpl_slave_state is renamed to mysql.gtid_slave_pos to match.

This fixes MDEV-4474.

There was missing a check for THD::killed after THD::enter_cond(). This could
cause the binlog dump thread to miss the kill signal during server shutdown
and hang until it was force-closed.

Also fix a race in a test case that occasionally fails in Buildbot.

Implement START SLAVE UNTIL master_gtid_pos = "<GTID position>".

Add test cases, including a test showing how to use this to promote
a new master among a set of slaves.

A bugfix of HandlerSocket is not applied to mariadb yet

- If one specifies --force twice to myisamchk and aria_chk, then we will try to finnish the repair even if sort_buffer would be too small.
  This was done by dynamically allocate buffer handler objects as long as memory lasts.
- New option for myisamchk and aria_chk: create-missing-keys
- Changed default size of myisam_sort_buffer_size from 8M to 128M.
- Changed default size of sort_buffer_size in aria_chk from 128M to 256M.
- Increased information in error message about 'sort_buffer_size' beeing to small.
- Print also to 'show warnings' if repair was retried.
- Increased size of internal sort-buffer-readers from 16K to 128K
- Changed printing of 'number of records' to use %ll instead of casting to long
- Changed buffer sizes for myisam and aria to use MY_ALIGN_DOWN() to get same number of bytes allocated on different machines.





include/my_global.h:
  Added MY_ALIGN_DOWN() to get previous alignment (for big memory areas)
include/myisam.h:
  Increased size of types to be able to handle more records
include/myisamchk.h:
  Increased size of types to be able to handle more records
  Added T_FORCE_SORT_MEMORY to force repair to work even if sort_buffer would not be big enough
mysql-test/r/myisam.result:
  Updated result
mysql-test/r/mysqld--help.result:
  Updated result
mysql-test/r/repair.result:
  Updated result
mysql-test/suite/maria/maria.result:
  Added test cases for some fixed bugs in MyISAM to verify that Aria doesn't have them too.
mysql-test/suite/maria/maria.test:
  Added test cases for some fixed bugs in MyISAM to verify that Aria doesn't have them too.
mysql-test/suite/maria/maria3.result:
  Updated result after sort buffer size increase
mysql-test/suite/sys_vars/r/aria_sort_buffer_size_basic.result:
  Updated result after sort buffer size increase
mysql-test/suite/sys_vars/r/myisam_sort_buffer_size_basic.result:
  Updated result after sort buffer size increase
mysql-test/suite/sys_vars/t/myisam_sort_buffer_size_basic.test:
  Updated result after sort buffer size increase
mysql-test/t/myisam.test:
  Fixed error messages to not print system specific data
mysql-test/t/repair.test:
  Fixed error messages to not print system specific data
storage/maria/ha_maria.cc:
  Print also to 'show warnings' if repair was retried
  Changed default size of sort_buffer_size from 128M to 256M (same as in mysqld)
storage/maria/ma_check.c:
  Renamed USE_BUFFER_INIT -> PAGE_BUFFER_INIT
storage/maria/ma_sort.c:
  Increased size of internal sort-buffer-readers from 16K to 128K
  Increased size of types to be able to handle more records
  Added support for T_FORCE_SORT_MEMORY
  Don't allocate too many extra BUFFPEK at a time (they are probably not needed)
  Improved error message for "sort_buffer_size is too small"
  Changed printing of 'number of records' to use %ll instead of casting to long
  Fixed bug where maria_update_key_parts() was called too early.
  Fixed bug in detecting result from read_to_buffer().
  Added 'out of memory' checking when calling 'alloc_dynamic()'.
storage/maria/maria_chk.c:
  Added --create-missing-keys
  If one specifies --force twice then we will try to finnish the repair even if sort_buffer would be too small.
  check_param.sort_buffer_length varialble was used with wrong type.
storage/maria/maria_def.h:
  Increased size of types to be able to handle more records
  Use MY_ALIGN_DOWN() to get same number of bytes allocated on different machines
  Renamed USE_BUFFER_INIT -> PAGE_BUFFER_INIT
storage/maria/maria_ftdump.c:
  Renamed USE_BUFFER_INIT -> PAGE_BUFFER_INIT
storage/maria/maria_read_log.c:
  Use PAGE_BUFFER_INIT for page cache
storage/myisam/ha_myisam.cc:
  Changed default size of myisam_sort_buffer_size from 8M to 128M
storage/myisam/mi_check.c:
  Renamed USE_BUFFER_INIT -> KEY_BUFFER_INIT
storage/myisam/myisam_ftdump.c:
  Renamed USE_BUFFER_INIT -> KEY_BUFFER_INIT
storage/myisam/myisamchk.c:
  Added --create-missing-keys
  If one specifies --force twice then we will try to finnish the repair even if sort_buffer would be too small.
  check_param.sort_buffer_length varialble was used with wrong type.
  Renamed USE_BUFFER_INIT -> KEY_BUFFER_INIT
storage/myisam/myisamdef.h:
  Increased SORT_BUFFER_INIT to 64M
  (speeds up repair a lot and most machines have nowadays a lot of memory)
  Use MY_ALIGN_DOWN() to get same number of bytes allocated on different machines
  Renamed USE_BUFFER_INIT -> KEY_BUFFER_INIT
storage/myisam/sort.c:
  Increased size of internal sort-buffer-readers from 16K to 128K
  Increased size of types to be able to handle more records
  Added support for T_FORCE_SORT_MEMORY
  Don't allocate too many extra BUFFPEK at a time (they are probably not needed)
  Improved error message for "sort_buffer_size is too small"
  Changed printing of 'number of records' to use %ll instead of casting to long
  Fixed bug in detecting result from read_to_buffer().
  Added 'out of memory' checking when calling 'alloc_dynamic()'.

Solaris fixes:
- Fixed that wait_timeout_func and wait_timeout tests works on solaris
- We have to compile without NO_ALARM on Solaris as Solaris doesn't support timeouts on sockets with setsockopt(.. SO_RCVTIMEO).
- Fixed that compile-solaris-amd64-debug works (before that we got a wrong ELF class: ELFCLASS64 on linkage)
- Added missing sync_with_master
Other bug fixes:
- Free memory for rpl_global_gtid_binlog_state before exit() to avoid 'accessing uninitalized mutex' error.



BUILD/FINISH.sh:
  Fixed issues on Solaris with ksh
BUILD/compile-solaris-amd64-debug:
  Added missing -m64 flag
configure.cmake:
  We have to compile without NO_ALARM on Solaris as Solaris doesn't support timeouts on sockets with setsockopt(.. SO_RCVTIMEO)
mysql-test/suite/rpl/t/rpl_gtid_mdev4473.test:
  - Added missing sync_with_master (fix by knielsen)
sql-common/client.c:
  Added () to get rid of compiler warning
sql/item_strfunc.cc:
  Fixed compiler warning
sql/log.cc:
  Free memory for static variable rpl_global_gtid_binlog_state before exit()
  - If we are compiling with safemalloc, we would try to call sf_free() for some members after sf_terminate() was called, which would result of trying to access the uninitalized mutex 'sf_mutex'
sql/multi_range_read.cc:
  Fixed compiler warnings of converting double to ulong.
sql/opt_range.cc:
  Fixed compiler warnings of converting double to ulong or uint
  - Better to have all variables that can be number of rows as 'ha_rows'
sql/rpl_gtid.cc:
  Added rpl_binlog_state::free() to be able to free memory for static objects before exit()
sql/rpl_gtid.h:
  Added rpl_binlog_state::free() to be able to free memory for static objects before exit()
sql/set_var.cc:
  Fixed compiler warning
sql/sql_join_cache.cc:
  Fixed compiler warnings of converting double to uint
sql/sql_show.cc:
  Added cast to get rid of compiler warning
sql/sql_statistics.cc:
  Remove code that didn't do anything.
  (store_record() with record[0] is a no-op)
storage/xtradb/os/os0file.c:
  Added  __attribute__ ((unused))
support-files/compiler_warnings.supp:
  Ignore warnings from atomic_add_64_nv
  (was not able to fix this with a cast as the macro is a bit different between systems)
vio/viosocket.c:
  Added more DBUG_PRINT

MDEV-4473: mysql_binlog_send() starts sending events from wrong GTID position in some master failover scenarios

Suppose binlog file X has in its Gtid_list_event: 0-1-3,0-2-5, and suppose the
slave requests to start replicating after 0-1-3.

In this case the bug was that master would start sending events from the start
of X. This is wrong, because 0-2-4 and 0-2-5 are contained in X-1, and are
needed by the slave. So these events were lost.

On the other hand, if the slave requested 0-2-5, then it _is_ correct to start
sending from the beginning of binlog file X, because 0-2-5 is the last GTID
logged in earlier binlogs. The difference is that 0-2-5 is the last of the
GTIDs in the Gtid_list_event. The problem was that the code did not check that
the matched GTID was the last one in the list.

Fixed by checking if the gtid requested by slave that matches a gtid in the
Gtid_list_event is the last event for that domain in the list. If not, go back
to a prior binlog to ensure all needed events are sent to slave.

mysql-test/include/show_events.inc:
  Backport --let $binlog_file=LAST, used by MDEV-4473 test case.

This makes it clear that the error code has nothing to do with errno.


mysql-test/include/mtr_warnings.sql:
  Fixed suppression for new slave error messages
mysql-test/lib/My/Test.pm:
  Use 'send' instead of 'print' to avoid errors about "wrong class ... back attempt"
mysql-test/lib/v1/mtr_report.pl:
  Fixed suppression for new slave error messages
mysql-test/mysql-test-run.pl:
  Fixed suppression for new slave error messages
  Removed warning from perl 5.16.2 about arrays
mysql-test/r/flush_read_lock.result:
  Fixed suppression for new slave error messages
sql/rpl_reporting.cc:
  Instead of writing "Errcode" to the log for Slave errors, use "Internal MariaDB error code"

- Calls to cleanup_load_tmpdir() could delete temporary files for another master connection
- Concurrent LOAD DATA commands from two master connections could use the same file name

Other bug fixes:
- Enlarge buffer for connection names with 'special characters' one can't store in filenames

Optimization:
- Don't do 'lower case' of connection names. We can use cmp_connection_name, where we already have the connection name in lower case.


mysql-test/suite/multi_source/load_data.result:
  Test case for MDEV-4352
mysql-test/suite/multi_source/load_data.test:
  Test case for MDEV-4352
sql/log_event.cc:
  Fixed: MDEV-4352
  - Calls to cleanup_load_tmpdir() could delete temporary files for another master connection
  - Concurrent LOAD DATA commands from two master connections could use the same file name
  
  The fix was to add the connection name (if one exists) to all slave temporary files used by LOAD DATA
sql/rpl_mi.cc:
  Enlarge buffer for connection names with 'special characters' one can't store in filenames
  Use mi->cmp_connection_name for connection file names.
sql/rpl_rli.cc:
  Use mi->cmp_connection_name for connection file names.
sql/slave.cc:
  Removed not needed empty line
sql/sql_const.h:
  Added MAX_FILENAME_MBWIDTH to be able to calculate buffer length for connection_names stored in file names
sql/sql_repl.cc:
  Use mi->cmp_connection_name for connection file names.

When the slave connects, the master skips binlog event groups
until it reaches the position requested by the slave. To
identify event groups, it needs to detect COMMIT events. But
this detection did not correctly handle binlog checksums, so
could incorrectly skip extra groups due to not detecting the
end of an event group.

Server shutdown timeout of 10 seconds in test cases is too little for heavily
loaded test servers.

Fix innodb_bug12902967 to not fail with wrong error log output if we have
warnings about too few AIO handles for InnoDB.

Fix typo which could lead to unnecessarily replacing GTID event with dummy
event.

The slave dump thread running on the master only checked thd->killed whenever
it reached the end of a binlog file, not between events. This could
unnecessarily delay server shutdown.

This was found by code inspection while tracking down some occasional "forcing
close of thread..." errors in Buildbot. Hopefully this will fix the failures,
but the fix is correct in any case.

Also increase the wait during server shutdown, 2 seconds is a bit tight in
case of heavy I/O stall, and it seems better to delay shutdown a bit than
force-kill threads unnecessarily.

Also fix some races in test cases that restart the mysqld server. The .expect
file should be changed with --append_file, --remove_file + --write_file
creates a short window where mysqld can error out due to .expect file missing.

Fixed a wrong result from mysqld--help.test.

This bug in the code of get_column_range_cardinality() could lead to
wrong estimates of number of records in ranges for non-nullable columns.  

MDEV-4398 - remove incorrect fix, replace with correct one  -  change default to  OFF for innodb_use_fallocate

Sampling of selectivity of LIKE predicate.

tables retrieved by range scans.

Added comments forgotten in the previous patch. 

mysql_options() with MYSQL_OPT_USE_THREAD_SPECIFIC_MEMORY

include/mysql.h:
  Added MYSQL_OPT_USE_THREAD_SPECIFIC_MEMORY
include/mysql.h.pp:
  Updated file
sql-common/client.c:
  Marked client memory as THREAD_SPECIFIC
sql/event_db_repository.cc:
  Fixed compiler warning
sql/slave.cc:
  Marked client memory as THREAD_SPECIFIC
storage/federatedx/federatedx_io_mysql.cc:
  Marked client memory as THREAD_SPECIFIC
storage/federatedx/ha_federatedx.cc:
  Marked client memory as THREAD_SPECIFIC
storage/sphinx/ha_sphinx.cc:
  Marked client memory as THREAD_SPECIFIC

sql/keycaches.cc:
  Added free_all_rpl_filters() to be able to free all filters at cleanup
sql/keycaches.h:
  Added prototype
sql/rpl_rli.cc:
  Fixed compiler warning
sql/slave.cc:
  Free all rpl_filters at cleanup
sql/sp.cc:
  Fixed compiler warning when not all struct elements was initialized
sql/sql_acl.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_events_waits.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_events_waits_summary.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_ews_global_by_event_name.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_file_instances.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_file_summary.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_performance_timers.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_setup_consumers.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_setup_instruments.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_setup_timers.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_sync_instances.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_threads.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/xtradb/os/os0file.c:
  Fixed compiler warning when not all struct elements was initialized