"load data" statements were written to the binlog as a mix of the original statement
and bits recreated from parse-info. This relied on implementation details and broke
with IGNORE_SPACES and versioned comments.

We now completely resynthesize the query for LOAD DATA for binlog (which among other
things normalizes them somewhat with regard to case, spaces, etc.).
We have already parsed the query properly, so we make use of that rather
than mix-and-match string literals and parsed items.
This should make us safe with regard to versioned comments, even those
spanning multiple tokens. Also no longer affected by IGNORE_SPACES.

mysql-test/r/mysqlbinlog.result:
  LOAD DATA INFILE normalized
mysql-test/suite/binlog/r/binlog_killed_simulate.result:
  LOAD DATA INFILE normalized
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
  LOAD DATA INFILE normalized
mysql-test/suite/binlog/r/binlog_stm_blackhole.result:
  LOAD DATA INFILE normalized
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
  LOAD DATA INFILE normalized
mysql-test/suite/rpl/r/rpl_innodb_mixed_dml.result:
  LOAD DATA INFILE normalized
mysql-test/suite/rpl/r/rpl_loaddata.result:
  LOAD DATA INFILE normalized
mysql-test/suite/rpl/r/rpl_loaddata_fatal.result:
  LOAD DATA INFILE normalized; offsets adjusted to reflect that
mysql-test/suite/rpl/r/rpl_loaddata_map.result:
  LOAD DATA INFILE normalized
mysql-test/suite/rpl/r/rpl_loaddatalocal.result:
  test for #43746 - trying to break LOAD DATA part of parser
mysql-test/suite/rpl/r/rpl_stm_log.result:
  LOAD DATA INFILE normalized
mysql-test/suite/rpl/t/rpl_loaddatalocal.test:
  try to break the LOAD DATA part of the parser (test for #43746)
mysql-test/t/mysqlbinlog.test:
  LOAD DATA INFILE normalized; adjust offsets to reflect that
sql/log_event.cc:
  clean up Load_log_event::print_query and friends so they don't print
  excess spaces. add support for printing charset names to print_query.
sql/log_event.h:
  We already have three places where we synthesize LOAD DATA queries.
  Better use one of those!
sql/sql_lex.h:
  When binlogging LOAD DATA statements, we make up the statement to
  be logged (from the parse-info, rather than substrings of the
  original query) now. Consequently, we no longer need (string-)
  pointers into the original query.
sql/sql_load.cc:
  Completely rewrote write_execute_load_query_log_event() to synthesize the
  LOAD DATA statement wholesale, rather than piece it together from
  synthesized bits and literal excerpts from the original query. This
  will not only give us a nice, normalized statement (all uppercase,
  no excess spaces, etc.), it will also handle comments, including
  versioned comments right, which is certainly more than we can say
  about the previous incarnation.
sql/sql_yacc.yy:
  We're no longer assembling LOAD DATA statements from bodyparts of the
  original query, so some bookkeeping in the parser can go.

      
The test case rpl_do_grant fails sporadically on PB2 with "Access
denied for user 'create_rout_db'@'localhost' ...". Inspecting the
test case, one may find that if issues a GRANT on the master
connection and immediately after it creates two new connections
(one to the master and one to the slave) using the credentials
set with the GRANT.
      
Unfortunately, there is no synchronization between master and
slave after the grant and before the connections are
established. This can result in slave not having executed the
GRANT by the time the connection is attempted.
      
This patch fixes this by deploying a sync_slave_with_master
between the grant and the connections attempt.

      
The test case creates two temporary tables, then closes the
connection, waits for it to disconnect, then syncs the slave with
the master, checks for remaining opened temporary tables on
slave (which should be 0) and finally drops the used
database (mysqltest).
      
Unfortunately, sometimes, the test fails with one open table on
the slave. This seems to be caused by the fact that waiting for
the connection to be closed is not sufficient. The test needs to
wait for the DROP event to be logged and only then synchronize
the slave with the master and proceed with the check. This is
caused by the asynchronous nature of the disconnect wrt
binlogging of the DROP temporary table statement.
      
We fix this by deploying a call to wait_for_binlog_event.inc
on the test case, which makes execution to wait for the DROP
temp tables event before synchronizing master and slave.

Problem: LOGGER::general_log_write() relied on valid "thd" parameter passed
but had inconsistent "if (thd)" check.

Fix: as we always pass a valid "thd" parameter to the method, 
redundant check removed.


sql/log.cc:
  Fix for bug#47130: misplaced or redundant check for null pointer?
    - code clean-up, as we rely on the "thd" parameter in the
  LOGGER::general_log_write(), redundant "if (thd)" check removed, 
  added assert(thd) instead.

The problem is that argument buffer can be used as result buffer
and it leads to argument value change.
The fix is to use 'old buffer' as result buffer only
if first argument is not constant item.


mysql-test/r/func_str.result:
  test result
mysql-test/t/func_str.test:
  test case
sql/item_strfunc.cc:
  The problem is that argument buffer can be used as result buffer
  and it leads to argument value change.
  The fix is to use 'old buffer' as result buffer only
  if first argument is not constant item.

In RBR, There is an inconsistency between slaves and master.
When INSERT statement which includes an auto_increment field is executed,
Store engine of master will check the value of the auto_increment field. 
It will generate a sequence number and then replace the value, if its value is NULL or empty.
if the field's value is 0, the store engine will do like encountering the NULL values 
unless NO_AUTO_VALUE_ON_ZERO is set into SQL_MODE.
In contrast, if the field's value is 0, Store engine of slave always generates a new sequence number 
whether or not NO_AUTO_VALUE_ON_ZERO is set into SQL_MODE.

SQL MODE of slave sql thread is always consistency with master's.
Another variable is related to this bug.
If generateing a sequence number is decided by the values of
table->auto_increment_field_not_null and SQL_MODE(if includes MODE_NO_AUTO_VALUE_ON_ZERO)
The table->auto_increment_is_not_null is FALSE, which causes this bug to appear. ..

partial backport of bug43138 fix


mysql-test/r/warnings.result:
  test result
mysql-test/t/warnings.test:
  test case
sql/sql_class.cc:
  partial backport of bug43138 fix
sql/sql_class.h:
  partial backport of bug43138 fix
sql/sql_table.cc:
  partial backport of bug43138 fix

on Windows in dbug.c) -- part 2: a patch for the DBUG subsystem
to detect misuse of DBUG_ENTER / DBUG_RETURN macros.
5.1 version.

Archive engine returns wrong values for average record length
and max data length.

With this fix they're calculated as following:
- max data length is 2 ^ 63 where large files are supported
  and INT_MAX32 where this is not supported;
- average record length is data length / records in data file.

mysql-test/r/archive.result:
  A test case for BUG#29203.
mysql-test/t/archive.test:
  A test case for BUG#29203.
storage/archive/ha_archive.cc:
  Better estimation for average row length and maximal data
  file length.

Create temporary InnoDB table fails on case insensitive
filesystems, when lower_case_table_names is 2 (e.g. OS X)
and temporary directory path contains upper case letters.

The problem was that tmpdir prefix was converted to lower
case when table was created, but was passed as is when
table was opened.

Fixed by leaving tmpdir prefix part intact.

mysql-test/r/lowercase_mixed_tmpdir_innodb.result:
  A test case for BUG#45638.
mysql-test/t/lowercase_mixed_tmpdir_innodb-master.opt:
  A test case for BUG#45638.
mysql-test/t/lowercase_mixed_tmpdir_innodb-master.sh:
  A test case for BUG#45638.
mysql-test/t/lowercase_mixed_tmpdir_innodb.test:
  A test case for BUG#45638.
sql/handler.cc:
  Fixed get_canonical_filename() to not lowercase filesystem
  path prefix for temporary tables.

Updates the results of all the out-dated test suites and adds 
the special mysqltest command to enable innodb for the tests that need it.

      
The parser rule for expressions in a udf parameter list contains 
two hacks: 
First, the parser input stream is read verbatim, bypassing 
the lexer.
Second, the Item::name field is overwritten. If the argument to a
udf was a field, the field's name as seen by name resolution was
overwritten this way.
If the field name was quoted or escaped, it would appear as e.g. "`field`".
Fixed by not overwriting field names.

mysql-test/r/udf.result:
  Bug#46259: Test result.
mysql-test/t/udf.test:
  Bug#46259: Test case.
sql/sql_yacc.yy:
  Bug#46259: Fix.

This test case uses mysqlbinlog to dump the content of master-bin.000001,
but the content of master-bin.000001 is not that this test needs.

MTR runs a lot of test cases on one server, so when this test starts, the current binlog file
might not be master-bin.000001, or there are other events are written by tests before.
'RESET MASTER' command must be called at the begin, it ensures that binlog of this test
is wrote to master-bin.000001 correctly.  

Three other tests have the same problem, They were fixed together.
mysqlbinlog-cp932
binlog_incident
binlog_tmp_table

Postfix.
extra/rpl_tests/rpl_row_sp006.test had changed to fix this bug.
extra/rpl_tests/rpl_row_sp006.test is also referenced by rpl_ndb_sp006,
So rpl_row_sp006.result must be changed too.

 
The external 'for' loop in remove_dup_with_compare() handled 
HA_ERR_RECORD_DELETED by just starting over without advancing 
to the next record which caused an infinite loop. 
 
This condition could be triggered on certain data by a SELECT 
query containing DISTINCT, GROUP BY and HAVING clauses. 

Fixed remove_dup_with_compare() so that we always advance to 
the next record when receiving HA_ERR_RECORD_DELETED from 
rnd_next(). 

mysql-test/r/distinct.result:
  Added a test case for bug #46159.
mysql-test/t/distinct.test:
  Added a test case for bug #46159.
sql/sql_select.cc:
  Fixed remove_dup_with_compare() so that we always advance to 
  the next record when receiving HA_ERR_RECORD_DELETED from 
  rnd_next().

           The plugin feature is disabled, you need HAVE_DLOPEN

Selectively skip tests that require dynamic loading (ie: static builds).

mysql-test/include/have_dynamic_loading.inc:
  Add require file.
mysql-test/t/ps_not_windows.test:
  Test requires dynamic loading support.

(Backport)

Problem is that when insert (ha_start_bulk_insert) in i partitioned table,
it will call ha_start_bulk_insert for every partition, used or not.

Solution is to delay the call to the partitions ha_start_bulk_insert until
the first row is to be inserted into that partition


sql/ha_partition.cc:
  Bug#35845: unneccesary call to ha_start_bulk_insert for not used partitions
  
  Using a bitmap for keeping record of which partitions for which
  ha_start_bulk_insert has been called, and check against that if one
  should call it before continue with the insert/update, or if it has already
  been called.
  
  This way it will only call ha_start_bulk_insert for the used partitions.
  There is also a little prediction on how many rows that will be inserted into
  the current partition, it will guess on equal distribution of the records
  across all partitions, accept for the first used partition, which will guess
  at 50% of the given estimate, if it is a monotonic partitioning function.
sql/ha_partition.h:
  Bug#35845: unneccesary call to ha_start_bulk_insert for not used partitions
  
  Added help variables and function for delaying ha_bulk_insert until it has
  to be called.
  
  Fixed a comment.

on subquery inside a SP 

Problem: repeated call of a SP containing an incorrect query with a 
subselect may lead to failed ASSERT().

Fix: set proper sublelect's state in case of error occured during 
subquery transformation.


mysql-test/r/sp.result:
  Fix for bug#46629: Item_in_subselect::val_int(): Assertion `0' 
  on subquery inside a SP 
    - test result.
mysql-test/t/sp.test:
  Fix for bug#46629: Item_in_subselect::val_int(): Assertion `0' 
  on subquery inside a SP 
    - test case.
sql/item_subselect.cc:
  Fix for bug#46629: Item_in_subselect::val_int(): Assertion `0' 
  on subquery inside a SP 
    - don't set Item_subselect::changed in the Item_subselect::fix_fields()
  if an error occured during subquery transformation.
  That prevents us of further processing incorrect subqueries after 
  Item_in_subselect::select_in_like_transformer().

SELECT with join (not only self-join) from archive table may
return incomplete result set, when result set size exceeds
join buffer size.

The problem was that archive row counter was initialzed too
early, when ha_archive::info() method was called. Later,
when optimizer exceeds join buffer, it attempts to reuse
handler without calling ha_archive::info() again (which is
correct).

Fixed by moving row counter initialization from
ha_archive::info() to ha_archive::rnd_init().

mysql-test/r/archive.result:
  A test case for BUG#46961.
mysql-test/t/archive.test:
  A test case for BUG#46961.
storage/archive/ha_archive.cc:
  Since a cursor may get reused without a call to ::info(),
  move assignment of scan_rows to a proper place, that is
  ::rnd_init().

Memory allocated in TMP_TABLE_PARAM::copy_field is not cleaned up.
The fix is to clean up TMP_TABLE_PARAM::copy_field array in JOIN::destroy.


mysql-test/r/explain.result:
  test result
mysql-test/t/explain.test:
  test case
sql/sql_select.cc:
  Memory allocated in TMP_TABLE_PARAM::copy_field is not cleaned up.
  The fix is to clean up TMP_TABLE_PARAM::copy_field array in JOIN::destroy.

                    name as existing view

When trying to create a table with the same name as existing view with
join, mysql server crashes.

The problem is when create table is issued with the same name as view, while
verifying with the existing tables, we assume that base table object is 
created always.

In this case, since it is a view over multiple tables, we don't have the 
mysql derived table object.

Fixed the logic which checks if there is an existing table to not to assume
that table object is created when the base table is view over multiple 
tables.

mysql-test/r/create.result:
  BUG#46384 - mysqld segfault when trying to create table with same 
              name as existing view
  
  Testcase for the bug
mysql-test/t/create.test:
  BUG#46384 - mysqld segfault when trying to create table with same 
              name as existing view
  
  Testcase for the bug
sql/sql_insert.cc:
  BUG#46384 - mysqld segfault when trying to create table with same 
                  name as existing view
      
  Fixed create_table_from_items() method to properly check, if the base table 
  is a view over multiple tables.

                                for CREATE TABLE...LIKE...

Add my_sync.c to mysqltest sources list in CMakeLists.txt

client/CMakeLists.txt:
  BUG#46591 - .frm file isn't sync'd with sync_frm enabled
              for CREATE TABLE...LIKE...
  
  Add my_sync.c to mysqltest sources list