when used with --tab

1) New syntax: added CHARACTER SET clause to the
  SELECT ... INTO OUTFILE (to complement the same clause in
  LOAD DATA INFILE).
  mysqldump is updated to use this in --tab mode.

2) ESCAPED BY/ENCLOSED BY field parameters are documented as
   accepting CHAR argument, however SELECT .. INTO OUTFILE
   silently ignored rests of multisymbol arguments.
   For the symmetrical behavior with LOAD DATA INFILE the
   server has been modified to fail with the same error:

     ERROR 42000: Field separator argument is not what is
                  expected; check the manual

3) Current LOAD DATA INFILE recognizes field/line separators
   "as is" without converting from client charset to data
   file charset. So, it is supposed, that input file of
   LOAD DATA INFILE consists of data in one charset and
   separators in other charset. For the compatibility with
   that [buggy] behaviour SELECT INTO OUTFILE implementation
   has been saved "as is" too, but the new warning message
   has been added:

     Non-ASCII separator arguments are not fully supported

   This message warns on field/line separators that contain
   non-ASCII symbols.


client/mysqldump.c:
  mysqldump has been updated to call SELECT ... INTO OUTFILE
  statement with a charset from the --default-charset command
  line parameter.
mysql-test/r/mysqldump.result:
  Added test case for bug #30946.
mysql-test/r/outfile_loaddata.result:
  Added test case for bug #30946.
mysql-test/t/mysqldump.test:
  Added test case for bug #30946.
mysql-test/t/outfile_loaddata.test:
  Added test case for bug #30946.
sql/field.cc:
  String conversion code has been moved from check_string_copy_error()
  to convert_to_printable() for reuse.
sql/share/errmsg.txt:
  New WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED message has been added.
sql/sql_class.cc:
  The select_export::prepare() method has been modified to:
  
    1) raise the ER_WRONG_FIELD_TERMINATORS error on multisymbol
       ENCLOSED BY/ESCAPED BY field arguments like LOAD DATA INFILE;
  
    2) warn with a new WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED
       message on non-ASCII field or line separators.
  
  The select_export::send_data() merhod has been modified to
  convert item data to output charset (see new SELECT INTO OUTFILE
  syntax). By default the BINARY charset is used for backward
  compatibility.
sql/sql_class.h:
  The select_export::write_cs field added to keep output
  charset.
sql/sql_load.cc:
  mysql_load has been modified to warn on non-ASCII field or
  line separators with a new WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED
  message.
sql/sql_string.cc:
  New global function has been added: convert_to_printable()
  (common code has been moved from check_string_copy_error()).
sql/sql_string.h:
  New String::is_ascii() method and new global convert_to_printable()
  function have been added.
sql/sql_yacc.yy:
  New syntax: added CHARACTER SET clause to the
  SELECT ... INTO OUTFILE (to complement the same clause in
  LOAD DATA INFILE). By default the BINARY charset is used for
  backward compatibility.

If using statement based replication (SBR), repeatedly calling
statements which are unsafe for SBR will cause a warning message
to be written to the error for each statement. This might lead
to filling up the error log and there is no way to disable this
behavior.

The solution is to only log these message (about statements unsafe
for statement based replication) if the log_warnings option is set.

For example:

SET GLOBAL LOG_WARNINGS = 0;
INSERT INTO t1 VALUES(UUID());
SET GLOBAL LOG_WARNINGS = 1;
INSERT INTO t1 VALUES(UUID());

In this case the message will be printed only once:

[Warning] Statement may not be safe to log in statement format.
          Statement: INSERT INTO t1 VALUES(UUID())

mysql-test/suite/binlog/r/binlog_stm_unsafe_warning.result:
  Add test case result for Bug#46265
mysql-test/suite/binlog/t/binlog_stm_unsafe_warning-master.opt:
  Make log_error value available.
mysql-test/suite/binlog/t/binlog_stm_unsafe_warning.test:
  Add test case for Bug#46265
sql/sql_class.cc:
  Print warning only if the log_warnings is enabled.

Initialize LOCK_open as a adapative mutex on platforms where the
PTHREAD_ADAPTIVE_MUTEX_INITIALIZER_NP macro is available. The flag
indicates that a thread should spin (busy wait) for some time on a
locked adaptive mutex before blocking (sleeping). It's intended to
to alleviate performance problems due to LOCK_open being a highly
contended mutex.

sql/mysqld.cc:
  Initialize LOCK_open as a adapative mutex.

      
A REPLACE in the MERGE engine is actually a REPLACE
into one (FIRST or LAST) of the underlying MyISAM
tables. So in effect the server works on the meta
data of the MERGE table, while the real insert happens
in the MyISAM table.

The MERGE table has no index, while MyISAM has a
unique index. When a REPLACE into a MERGE table (
and the REPLACE conflicts with a duplicate in a
child table) is done, we try to access the duplicate
key information for the MERGE table. This information
actually does not exist, hence this results in a crash.

The problem can be resolved by modifying the MERGE
engine to provide us the duplicate key information
directly, instead of just returning the MyISAM index
number as the error key. Then the SQL layer (or "the
server") does not try to access the key_info of the
MERGE table, which does not exist.

The current patch modifies the MERGE engine to provide
the position for a record where a unique key violation
occurs.

include/myisammrg.h:
  Bug#45800 crash when replacing into a merge table and there is a duplicate
  
  Add a member to the st_mymerge_info structure that will
  store the duplicate key offset in the MERGE table. This
  offset will be the sum of the record offset of the MyISAM
  table within the MERGE table and the offset of the record
  within the MyISAM table.
mysql-test/r/merge.result:
  Bug#45800 crash when replacing into a merge table and there is a duplicate
  
  Result file for the test case.
mysql-test/t/merge.test:
  Bug#45800 crash when replacing into a merge table and there is a duplicate
  
  Added test case for both REPLACE and INSERT...ON DUPLICATE UPDATE.
storage/myisammrg/ha_myisammrg.cc:
  Bug#45800 crash when replacing into a merge table and there is a duplicate
  
  The info method now will process the HA_STATUS_ERRKEY flag
  and will return the index and the offset of the duplicate
  key.
storage/myisammrg/ha_myisammrg.h:
  Bug#45800 crash when replacing into a merge table and there is a duplicate
  
  Set the HA_DUPLICATE_POS flag to indicate that the duplicate
  key information is now available in the MERGE storage engine.
storage/myisammrg/myrg_info.c:
  Bug#45800 crash when replacing into a merge table and there is a duplicate
  
  We modify the myrg_status function to return the position of the
  duplicate key. The duplicate key position in the MERGE table will
  be the MyISAM file_offset and the offset within the MyISAM table
  of the start position of the records.

Fixed wrong prototype declaration which cased build failure on solaris.

          an assertion in a debug build.

The reason is that the C API doesn't support multiple result sets for prepared
statements and attempting to execute a stored routine which returns multiple result
sets sometimes lead to a network error. The network error sets the diagnostic area
prematurely which later leads to the assert when an attempt is made to set a second
server state.

This patch fixes the issue by changing the scope of the error code returned by
sp_instr_stmt::execute() to include any error which happened during the execution.
To assure that Diagnostic_area::is_sent really mean that the message was sent all
network related functions are checked for return status.

libmysqld/lib_sql.cc:
  * Changed prototype to return success/failure status on net_send_error_packet(),
    net_send_ok(), net_send_eof(), write_eof_packet().
mysql-test/r/sp_notembedded.result:
  * Added test case for bug 44521
mysql-test/t/sp_notembedded.test:
  * Added test case for bug 44521
sql/protocol.cc:
  * Changed prototype to return success/failure status on net_send_error_packet(),
    net_send_ok(), net_send_eof(), write_eof_packet().
sql/protocol.h:
  * Changed prototype to return success/failure status on net_send_error_packet(),
    net_send_ok(), net_send_eof(), write_eof_packet().
sql/sp_head.cc:
  * Changed prototype to return success/failure status on net_send_error_packet(),
    net_send_ok(), net_send_eof(), write_eof_packet().

Bug#46354, when defining partitions without subpartition definition after defining it with the first partition and using list partition caused crash, fixed by more error checks in parser

those keywords do nothing in 5.1 (they are meant for future versions, for example featuring the Maria engine)
so they are here removed from the syntax. Adding those keywords to future versions when needed is:
- WL#5034 "Add TRANSACTIONA=0|1 and PAGE_CHECKSUM=0|1 clauses to CREATE TABLE"
- WL#5037 "New ROW_FORMAT value for CREATE TABLE: PAGE"

mysql-test/r/create.result:
  test that syntax is not accepted
mysql-test/t/create.test:
  test that syntax is not accepted
sql/handler.cc:
  remove ROW_FORMAT=PAGE
sql/handler.h:
  Mark unused objects, but I don't remove them by fear of breaking any plugin which includes this file
  (see also table.h)
sql/lex.h:
  removing syntax
sql/sql_show.cc:
  removing output of noise keywords in SHOW CREATE TABLE and INFORMATION_SCHEMA.TABLES
sql/sql_table.cc:
  removing TRANSACTIONAL
sql/sql_yacc.yy:
  removing syntax
sql/table.cc:
  removing TRANSACTIONAL, PAGE_CHECKSUM. Their place in the frm file is not reclaimed,
  for compatibility with older 5.1.
sql/table.h:
  Mark unused objects, but I don't remove them by fear of breaking any plugin which includes this file
  (and there are several engines which use the content TABLE_SHARE and thus rely on a certain binary
  layout of this structure).

consumption (CPU) for upgrading a large log table can be intense.
Therefore, truncate the general_log table beforehand if running
mysql_upgrade test with Valgrind.

mysql-test/t/mysql_upgrade.test:
  Truncate log table if running with Valgrind.

            compression 
 
Since uint3korr() may read 4 bytes depending on build flags and 
platform, allocate 1 extra "safety" byte in the network buffer 
for cases when uint3korr() in my_real_read() is called to read
last 3 bytes in the buffer. 
 
It is practically hard to construct a reliable and reasonably 
small test case for this bug as that would require constructing 
input stream such that a certain sequence of bytes in a 
compressed packet happens to be the last 3 bytes of the network 
buffer. 


sql/net_serv.cc:
  Allocate 1 extra "safety" byte in the network buffer for cases 
  when uint3korr() is used to read last 3 bytes in the buffer.

If the log_bin_trust_function_creators option is not defined, creating a stored
function requires either one of the modifiers DETERMINISTIC, NO SQL, or READS
SQL DATA. Executing a stored function should also follows the same rules if in
STATEMENT mode. However, this was not happening and a wrong error was being
printed out: ER_BINLOG_ROW_RBR_TO_SBR.

The patch makes the creation and execution compatible and prints out the correct
error ER_BINLOG_UNSAFE_ROUTINE when a stored function without one of the modifiers
above is executed in STATEMENT mode.

The maximum value of the max_join_size variable is set by converting
a signed type (long int) with negative value (-1) to a wider unsigned
type (unsigned long long), which yields the largest possible value of
the wider unsigned type -- as per the language conversion rules. But,
depending on build options, the type of the max_join_size might be a
shorter type (ha_rows - unsigned long) which causes the warning to be
thrown once the large value is truncated to fit.

The solution is to ensure that the maximum value of the variable is
always set to the maximum value of integer type of max_join_size.

Furthermore, it would be interesting to always have a fixed type for
this variable, but this would incur in a change of behavior which is
not acceptable for a GA version. See Bug#35346.

sql/mysqld.cc:
  Set max value for type.

during bootstrap on a embedded server.

libmysqld/lib_sql.cc:
  Handle a failure during bootstrap.

Post-merge fix: test case could fail due to a conversion of the
max_join_size value to a integer. Fixed by preserving the value
as a string for comparison purposes.

tests/mysql_client_test.c:
  Preserve max_join_size value as a string instead of converting
  it to a integer -- value can be larger then the type used.

One of the tests introduced for this bug was failing 
because of path size restriction in windows.
Moved the test case to a new test which is disabled under windows.

mysql-test/r/partition_not_embedded.result:
  updated test results after removing a test case.
mysql-test/r/partition_rename_longfilename.result:
  Test result for partition_rename_longfilename
mysql-test/t/partition_not_embedded.test:
  Removed the test case which tests renaming partition table such that
  the file name is 255 char long.
mysql-test/t/partition_rename_longfilename.test:
  Test case to test renaming partition table such that
  the file name is 255 char long.
  Moved from partition_no_embedded.

           to wrong result
      
When using MIXED mode and issuing 'CREATE TEMPORARY TABLE t_tmp',
the statement is logged if the current binlogging mode is
STATEMENT. This causes the slave to replay the instruction and
create the temporary table as well. If there is no switch to ROW
mode, and later on a 'DROP TEMPORARY TABLE t_tmp' is issued, then
this statement will also be logged and the slave will
remove/close the temporary table.
      
However, if there is a switch to ROW mode between the CREATE and
DROP TEMPORARY table, the DROP statement will not be logged,
leaving the slave with a dangling temporary table.
      
This patch addresses this, by always logging a DROP TEMPORARY
TABLE IF EXISTS when in mixed mode and a drop statement is issued
for temporary table(s).

mysql-test/suite/rpl/r/rpl_temp_table_mix_row.result:
  Updated result file.
mysql-test/suite/rpl/t/rpl_temp_table_mix_row.test:
  Added test case.
sql/sql_table.cc:
  When dropping table(s) in mixed mode and current statement 
  logging is ROW, builds an extra DROP TEMPORARY TABLE IF 
  EXISTS for temporary tables that are being dropped. Later, 
  it logs the extra drop statement.

mysqld

The problem was that enabling the event scheduler inside a init
file caused the server to crash upon start-up. The crash occurred
because the event scheduler wasn't being initialized before the
commands in the init-file are processed.

The solution is to initialize the event scheduler before the init
file is read. The patch also disables the event scheduler during
bootstrap and makes the bootstrap operation robust in the
presence of background threads.

mysql-test/std_data/init_file.dat:
  Add test case for Bug#43587
sql/event_scheduler.cc:
  Signal that the thread_count has been decremented.
sql/events.cc:
  Disable the event scheduler during bootstrap.
sql/mysql_priv.h:
  Export variable.
sql/mysqld.cc:
  Initialize the event scheduler before commands are executed.
sql/sql_parse.cc:
  Signal that the bootstrap thread is done.

mysqld

******
manual merge 5.0-bugteam --> 5.1-bugteam (bug 38816)

            procedures causes crashes!

The problem of that bugreport was mostly fixed by the
patch for bug 38691.
However, attached test case focused on another crash or
valgrind warning problem: SHOW PROCESSLIST query accesses
freed memory of SP instruction that run in a parallel
connection.

Changes of thd->query/thd->query_length in dangerous
places have been guarded with the per-thread
LOCK_thd_data mutex (the THD::LOCK_delete mutex has been
renamed to THD::LOCK_thd_data).


sql/ha_myisam.cc:
  Bug #38816: kill + flush tables with read lock + stored
              procedures causes crashes!
  
  Modification of THD::query/query_length has been guarded
  with the a THD::set_query() method call/LOCK_thd_data
  mutex.
  Unnecessary locking with the global LOCK_thread_count
  mutex has been removed.
sql/log_event.cc:
  Bug #38816: kill + flush tables with read lock + stored
              procedures causes crashes!
  
  Modification of THD::query/query_length has been guarded
  with the THD::set_query()) method call/LOCK_thd_data
  mutex.
sql/slave.cc:
  Bug #38816: kill + flush tables with read lock + stored
              procedures causes crashes!
  
  Modification of THD::query/query_length has been guarded
  with the THD::set_query() method call/LOCK_thd_data mutex.
  
  The THD::LOCK_delete mutex has been renamed to
  THD::LOCK_thd_data.
sql/sp_head.cc:
  Bug #38816: kill + flush tables with read lock + stored
              procedures causes crashes!
  
  Modification of THD::query/query_length has been guarded
  with the a THD::set_query() method call/LOCK_thd_data
  mutex.
sql/sql_class.cc:
  Bug #38816: kill + flush tables with read lock + stored
              procedures causes crashes!
  
  The new THD::LOCK_thd_data mutex and THD::set_query()
  method has been added to guard modifications of THD::query/
  THD::query_length fields, also the Statement::set_statement()
  method has been overloaded in the THD class.
  
  The THD::LOCK_delete mutex has been renamed to
  THD::LOCK_thd_data.
sql/sql_class.h:
  Bug #38816: kill + flush tables with read lock + stored
              procedures causes crashes!
  
  The new THD::LOCK_thd_data mutex and THD::set_query()
  method has been added to guard modifications of THD::query/
  THD::query_length fields, also the Statement::set_statement()
  method has been overloaded in the THD class.
  
  The THD::LOCK_delete mutex has been renamed to
  THD::LOCK_thd_data.
sql/sql_insert.cc:
  Bug #38816: kill + flush tables with read lock + stored
              procedures causes crashes!
  
  Modification of THD::query/query_length has been guarded
  with the a THD::set_query() method call/LOCK_thd_data
  mutex.
sql/sql_parse.cc:
  Bug #38816: kill + flush tables with read lock + stored
              procedures causes crashes!
  
  Modification of THD::query/query_length has been guarded
  with the a THD::set_query() method call/LOCK_thd_data mutex.
sql/sql_repl.cc:
  Bug #38816: kill + flush tables with read lock + stored
              procedures causes crashes!
  
  The THD::LOCK_delete mutex has been renamed to
  THD::LOCK_thd_data.
sql/sql_show.cc:
  Bug #38816: kill + flush tables with read lock + stored
              procedures causes crashes!
  
  Inter-thread read of THD::query/query_length field has
  been protected with a new per-thread LOCK_thd_data
  mutex in the mysqld_list_processes function.