GitLab

Problem: mysqlbinlog exits without any error code in case of
file write error. It is because of the fact that the calls
to Log_event::print() method does not return a value and the
thus any error were being ignored.

Resolution: We resolve this problem by checking for the 
IO_CACHE::error == -1 after every call to Log_event:: print()
and terminating the further execution.

client/mysqlbinlog.cc:
  - handled error conditions during event->print() calls
  - added check for error in end_io_cache()
mysys/my_write.c:
  Added debug code to simulate file write error.
  error returned will be ENOSPC=> error no space on the disk
sql/log_event.cc:
  Added debug code to simulate file write error, by reducing the size of io cache.

      
Problem: After the fix for Bug#12589870, a new field that
stores the length of db name was added in the buffer that
stores the query to be executed. Unlike for the plain user
session, the replication execution did not allocate the
necessary chunk in Query-event constructor. This caused an
invalid read while accessing this field.
      
Solution: We fix this problem by allocating a necessary chunk
in the buffer created in the Query_log_event::Query_log_event()
and store the length of database name.

sql/log_event.cc:
  Added a new field in the buffer created in the
  Query_log_event's constructor and store the length
  of database name.

BUG#11761686 insert_id event is not filtered.
  
Two issues are covered.
  
INSERT into autoincrement field which is not the first part in the composed primary key 
is unsafe by autoincrement logging design. The case is specific to MyISAM engine
because Innodb does not allow such table definition.
  
However no warnings and row-format logging in the MIXED mode was done, and
that is fixed.
  
Int-, Rand-, User-var log-events were not filtered along with their parent
query that made possible them to screw up execution context of the following
query.
  
Fixed with deferring their execution until the parent query.

******
Bug#11754117 

Post review fixes.

mysql-test/suite/rpl/r/rpl_auto_increment_bug45679.result:
  a new result file is added.
mysql-test/suite/rpl/r/rpl_filter_tables_not_exist.result:
  results updated.
mysql-test/suite/rpl/t/rpl_auto_increment_bug45679.test:
  regression test for BUG#11754117-45670 is added.
mysql-test/suite/rpl/t/rpl_filter_tables_not_exist.test:
  regression test for filtering issue of BUG#11754117 - 45670 is added.
sql/log_event.cc:
  Logics are added for deferring and executing events associated 
  with the Query event.
sql/log_event.h:
  Interface to deferred events batch execution is added.
sql/rpl_rli.cc:
  initialization for new RLI members is added.
sql/rpl_rli.h:
  New members to RLI are added to facilitate deferred events gathering
  and execution control;
  two general character RLI cleanup methods are constructed.
sql/rpl_utility.cc:
  Deferred_log_events methods are difined.
sql/rpl_utility.h:
  A new class Deferred_log_events is defined to implement
  IRU events gathering, execution and cleanup.
sql/slave.cc:
  Necessary changes to initialize `rli->deferred_events' and prevent
  deferred event deletion in the main read-exec branch.
sql/sql_base.cc:
  A new safe-check function for multi-part pk with auto-increment is defined
  and deployed in lock_tables().
sql/sql_class.cc:
  Initialization for a new member and replication cleanups are added
  to THD class.
sql/sql_class.h:
  THD class receives a new member to hold a specific execution
  context for slave applier.
sql/sql_parse.cc:
  Execution of the deferred event in started prior to its parent query.

Analysis:
========================
sql_mode "NO_BACKSLASH_ESCAPES": When user want to use backslash as character input,
instead of escape character in a string literal then sql_mode can be set to 
"NO_BACKSLASH_ESCAPES". With this mode enabled, backslash becomes an ordinary 
character like any other. 

SQL_MODE set applies to the current client session. And while creating the stored 
procedure, MySQL stores the current sql_mode and always executes the stored 
procedure in sql_mode stored with the Procedure, regardless of the server SQL 
mode in effect when the routine is invoked.  

In the scenario (for which bug is reported), the routine is created with 
sql_mode=NO_BACKSLASH_ESCAPES. And routine is executed with the invoker sql_mode
is "" (NOT SET) by executing statement "call testp('Axel\'s')".
Since invoker sql_mode is "" (NOT_SET), the '\' in 'Axel\'s'(argument to function)
is considered as escape character and column "a" (of table "t1") values are 
updated with "Axel's". The binary log generated for above update operation is as below,

  set sql_mode=XXXXXX (for no_backslash_escapes)
  update test.t1 set a= NAME_CONST('var',_latin1'Axel\'s' COLLATE 'latin1_swedish_ci');

While logging stored procedure statements, the local variables (params) used in
statements are replaced with the NAME_CONST(var_name, var_value) (Internal function) 
(http://dev.mysql.com/doc/refman/5.6/en/miscellaneous-functions.html#function_name-const)

On slave, these logs are applied. NAME_CONST is parsed to get the variable and its
value. Since, stored procedure is created with sql_mode="NO_BACKSLASH_ESCAPES", the sql_mode
is also logged in. So that at slave this sql_mode is set before executing the statements
of routine.  So at slave, sql_mode is set to "NO_BACKSLASH_ESCAPES" and then while
parsing NAME_CONST of string variable, '\' is considered as NON ESCAPE character
and parsing reported error for "'" (as we have only one "'" no backslash). 

At slave, parsing was proper with sql_mode "NO_BACKSLASH_ESCAPES".
But above error reported while writing bin log, "'" (of Axel's) is escaped with
"\" character. Actually, all special characters (n, r, ', ", \, 0...) are escaped
while writing NAME_CONST for string variable(param, local variable) in bin log 
Airrespective of "NO_BACKSLASH_ESCAPES" sql_mode. So, basically, the problem is 
that logging string parameter does not take into account sql_mode value.

Fix:
========================
So when sql_mode is set to "NO_BACKSLASH_ESCAPES", escaping  characters as 
(n, r, ', ", \, 0...) should be avoided. To do so, added a check to not to
escape such characters while writing NAME_CONST for string variables in bin 
log. 
And when sql_mode is set to NO_BACKSLASH_ESCAPES, quote character "'" is
represented as ''.
http://dev.mysql.com/doc/refman/5.6/en/string-literals.html (There are several 
ways to include quote characters within a string: )



mysql-test/r/sql_mode.result:
  Added test case for Bug#12601974.
mysql-test/suite/binlog/r/binlog_sql_mode.result:
  Appended result of test cases added for Bug#12601974.
mysql-test/suite/binlog/t/binlog_sql_mode.test:
  Added test case for Bug#12601974.
mysql-test/t/sql_mode.test:
  Appended result of test cases added for Bug#12601974.

HA_ERR was returning 0 (null string) when no error happened 
(error=0). Since HA_ERR is used in DBUG_PRINT, regardless there 
was an error or not, the server could crash in solaris debug
builds.

We fix this by:

  - deploying an assertion that ensures that the function 
    is not called when no error has happened;
  - making sure that HA_ERR is only called when an error 
    happened;
  - making HA_ERR return "No Error", instead of 0, for 
    non-debug builds if it is called when no error happened.

This will make HA_ERR return values to work with DBUG_PRINT on
solaris debug builds.

                  
The server crashes if it processes table map events that are
corrupted, especially if they map different tables to the same
identifier. This could happen, for instance, due to BUG 56226.
                  
We fix this by checking whether the table map has already been
mapped before actually applying the event. If it has been mapped
with different settings an error is raised and the slave SQL
thread stops. If it has been mapped with same settings the event
is skipped. If the table is set to be ignored by the filtering
rules, there is no change in behavior: the event is skipped and
ids are not checked.


mysql-test/suite/rpl/t/rpl_row_corruption.test:
  Added a simple test case that checks both cases:
  - multiple table maps with the same identifier
  - multiple table maps with the same identifier, but only one
    is processed (the others are filtered out)

  
The slave was not able to find the correct row in the innodb
table, because the row fetched from the innodb table would not
match the before image. This happened because the (don't care)
bytes in the NULLed fields would change once the row was stored
in the storage engine (from zero to the default value). This
would make bulk memory comparison (using memcmp) to fail.
  
We fix this by taking a preventing measure and avoiding memcmp
for tables that contain nullable fields. Therefore, we protect
the slave search routine from engines that return arbitrary
values for don't care bytes (in the nulled fields). Instead, the
slave thread will only check null_bits and those fields that are
not set to NULL when comparing the before image against the
storage engine row.

mysql-test/extra/rpl_tests/rpl_record_compare.test:
  Added test case to the include file so that this is tested 
  with more than one engine.
mysql-test/suite/rpl/r/rpl_row_rec_comp_innodb.result:
  Result update.
mysql-test/suite/rpl/r/rpl_row_rec_comp_myisam.result:
  Result update.
mysql-test/suite/rpl/t/rpl_row_rec_comp_myisam.test:
  Moved the include file last, so that the result from
  BUG#11766865 is not intermixed with the result for
  BUG#11760454.
sql/log_event.cc:
  Skips memory comparison if the table has nullable 
  columns and compares only non-nulled fields in the
  field comparison loop.

mysqlbinlog only prints "use $database" statements to its output stream
when the active default database changes between events. This will cause
"No Database Selected" error when dropping and recreating that database.

To fix the problem, we clear print_event_info->db when printing an event
of CREATE/DROP/ALTER database statements, so that the Query_log_event
after such statements will be printed with the use 'db' anyway except
transaction keywords.

mysql-test/r/mysqlbinlog.result:
  Test result for Bug#50914.
mysql-test/t/mysqlbinlog.test:
  Added test to verify if the approach of the mysqlbinlog prints
  "use $database" statements to its output stream will cause
  "No Database Selected" error when dropping and recreating
  that database.
sql/log_event.cc:
  Updated code to clear print_event_info->db when printing an event
  of CREATE/DROP/ALTER database statements, so that the Query_log_event
  after such statements will be printed with the use 'db' anyway except
  transaction keywords.

Normally, auto_increment value is generated for the column by
inserting either NULL or 0 into it. NO_AUTO_VALUE_ON_ZERO
suppresses this behavior for 0 so that only NULL generates
the auto_increment value. This behavior is also followed by
a slave, specifically by the SQL Thread, when applying events
in the statement format from a master. However, when applying
events in the row format, the flag was ignored thus causing
an assertion failure:
"Assertion failed: next_insert_id == 0, file .\handler.cc"

In fact, we never need to generate a auto_increment value for
the column when applying events in row format on slave. So we
don't allow it to happen by using 'MODE_NO_AUTO_VALUE_ON_ZERO'.

Refactoring: Get rid of all the sql_mode checks to rows_log_event
when applying it for avoiding problems caused by the inconsistency
of the sql_mode on slave and master as the sql_mode is not set for
Rows_log_event.

mysql-test/extra/rpl_tests/rpl_auto_increment.test:
  Added test to verify if the assertion of "next_insert_id == 0"
  will fail in ha_external_lock() function.
mysql-test/suite/rpl/r/rpl_auto_increment.result:
  Test result for bug#56662.
sql/log_event.cc:
  Added code to not allow generation of auto_increment value when
  processing rows event by adding 'MODE_NO_AUTO_VALUE_ON_ZERO' to
  sql_mode.
sql/rpl_record.cc:
  Added code to get rid of the 'MODE_STRICT_TRANS_TABLES'  and
  MODE_STRICT_ALL_TABLES check to the table when appling the
  rows event and treat it as no strict.

Addressing review comments.

When a query fails with a different error on the slave,
the sql thread outputs a message (M) containing:

  1. the error message format for the master error code
  2. the master error code
  3. the error message for the slave's error code
  4. the slave error code

Given that the slave has no information on the error message
itself that the master outputs, it can only print its own
version of the message format (but stripped from the 
additional data if the message format requires). This may
confuse users.

To fix this we augment the slave's message (M) to explicitly
state that the master's message is actually an error message 
format, the one associated with the given master error code 
and that the slave server knows about.

"Grantor" columns' data is lost when replicating mysql.tables_priv.
Slave SQL thread used its default user ''@'' as the grantor of GRANT|REVOKE
statements executing on it.

In this patch, current user is put in query log event for all GRANT and REVOKE
statement, SQL thread uses the user in query log event as grantor.


mysql-test/suite/rpl/r/rpl_do_grant.result:
  Add test for this bug.
mysql-test/suite/rpl/t/rpl_do_grant.test:
  Add test for this bug.
sql/log_event.cc:
  Refactoring THD::current_user_used and related functions.
  current_user_used is used to judge if current user should be
  binlogged in query log event. So it is better to call it m_binlog_invoker.
  The related functions are renamed too.
sql/sql_class.cc:
  Refactoring THD::current_user_used and related functions.
  current_user_used is used to judge if current user should be
  binlogged in query log event. So it is better to call it m_binlog_invoker.
  The related functions are renamed too.
sql/sql_class.h:
  Refactoring THD::current_user_used and related functions.
  current_user_used is used to judge if current user should be
  binlogged in query log event. So it is better to call it m_binlog_invoker.
  The related functions are renamed too.
sql/sql_parse.cc:
  Call binlog_invoker() for GRANT and REVOKE statements.

Rows events were applied wrongly on the temporary table with the same name.
But rows events are generated only for base tables. As temporary
table's data never be binlogged on row mode. Normally, base table of the
same name cannot be updated if a temporary table has the same name.
But there are two cases which can generate rows events on 
the base table of same name.
      
Case1: 'CREATE TABLE ... SELECT' statement.
In mixed format, it will generate rows events if it is unsafe.
      
Case2: Drop a transactional temporary table in a transaction
       (happens only on 5.5+).
BEGIN;
DROP TEMPORARY TABLE t1;       # t1 is a InnoDB table
INSERT INTO t1 VALUES(rand()); # t1 is a MyISAM table
COMMIT;
'DROP TEMPORARY TABLE' will be put in the transaction cache and
binlogged after the rows events generated by the 'INSERT' statement.
      
After this patch, slave opens only base table when applying a rows event.

Tag or remove unused arguments and variables.

regex/main.c:
  Use the real prototype.
sql/ha_ndbcluster.cc:
  Make conditions less ambiguous.

When slave executes a transaction bigger than slave's max_binlog_cache_size,
slave will crash. It is caused by the assert that server should only roll back
the statement but not the whole transaction if the error ER_TRANS_CACHE_FULL 
happens. But slave sql thread always rollbacks the whole transaction when
an error happens.
            
Ather this patch, we always clear any error set in sql thread(it is different
from the error in 'SHOW SLAVE STATUS') and it is cleared before rolling back
the transaction.


mysql-test/suite/rpl/r/rpl_binlog_max_cache_size.result:
  SET binlog_cache_size and max_binlog_cache_size for all test cases.
  Add test case for bug#55375.
mysql-test/suite/rpl/t/rpl_binlog_max_cache_size-master.opt:
  binlog_cache_size and max_binlog_cache_size can be set in the client connection.
  so remove this option file.
mysql-test/suite/rpl/t/rpl_binlog_max_cache_size.test:
  SET binlog_cache_size and max_binlog_cache_size for all test cases.
  Add test case for bug#55375.
sql/log_event.cc:
  Some functions don't return the error code, so it is a wrong error code.
  The error should always be set into thd->main_da. So we use 
  slave_rows_error_report to report the right error.
sql/slave.cc:
  exec_relay_log_event() need call cleanup_context() to clear context. 
  clearup_context() will call end_trans().
          
  Clear thd's error before cleanup_context. It avoid to trigger the assert
  which cause this bug.

      
Suprisingly, a Slave_log_event would show up in the binary
log. This event is never used and should not appear in the
logs. As such, when the slave (or the mysqlbinlog tool) reads the
event, it will hit an invalid pointer (reference to the
descriptor event when deserializing the Slave_log_event was
purposodely set to NULL).
      
The presence of the Slave_log_event denotes a corrupted log, but
we cannot tell how the log got corrupted in the first
place. However, we can make the server cope with such events when
it reads them - in case of log corruption - and fail gracefully.
     
This patch makes the server/mysqlbinlog to report that it has
found an invalid log event when Slave_log_event is read.

Although the C standard mandates that sprintf return the number
of bytes written, some very ancient systems (i.e. SunOS 4)
returned a pointer to the buffer instead. Since these systems
are not supported anymore and are hopefully long dead by now,
simply remove the portability wrapper that dealt with this
discrepancy. The autoconf check was causing trouble with GCC.

Fix the memory leak

DROP USER 
RENAME USER CURRENT_USER() ...
GRANT ... TO CURRENT_USER()
REVOKE ... FROM CURRENT_USER()
ALTER DEFINER = CURRENT_USER() EVENTbut, When these statements are binlogged, CURRENT_USER() just is binlogged
as 'CURRENT_USER()', it is not expanded to the real user name. When slave 
executes the log event, 'CURRENT_USER()' is expand to the user of slave 
SQL thread, but SQL thread's user name always NULL. This breaks the replication.

After this patch, session's user will be written into query log events 
if these statements call CURREN_USER() or 'ALTER EVENT' does not assign a definer.


mysql-test/include/diff_tables.inc:
  Expend its abilities.
  Now it can diff not only in sessions of 'master' and 'slave', but 
  other sessions as well.
mysql-test/include/rpl_diff_tables.inc:
  Diff the same table between master and slaves.
sql/log_event.cc:
  session's user will be written into Query_log_event, if is_current_user_used() is TRUE.
  On slave SQL thread, Only thd->variables.current_user is written into Query_log_event,
  if it exists.
sql/sql_acl.cc:
  On slave SQL thread, grantor should copy from thd->variables.current_user, if it exists
sql/sql_class.h:
  On slave SQL thread, thd->variables.current_user is used to store the applying event's
  invoker.

DROP USER 
RENAME USER CURRENT_USER() ...
GRANT ... TO CURRENT_USER()
REVOKE ... FROM CURRENT_USER()
ALTER DEFINER = CURRENT_USER() EVENTbut, When these statements are binlogged, CURRENT_USER() just is binlogged
as 'CURRENT_USER()', it is not expanded to the real user name. When slave 
executes the log event, 'CURRENT_USER()' is expand to the user of slave 
SQL thread, but SQL thread's user name always NULL. This breaks the replication.

After this patch, session's user will be written into query log events 
if these statements call CURREN_USER() or 'ALTER EVENT' does not assign a definer.


mysql-test/include/diff_tables.inc:
  Expend its abilities.
  Now it can diff not only in sessions of 'master' and 'slave', but 
  other sessions as well.
sql/log_event.cc:
  session's user will be written into Query_log_event, if is_current_user_used() is TRUE.
  On slave SQL thread, Only thd->invoker is written into Query_log_event,
  if it exists.
sql/sql_acl.cc:
  On slave SQL thread, grantor should copy from thd->invoker, if it exists
sql/sql_class.h:
  On slave SQL thread, thd->invoker is used to store the applying event's
  invoker.

Apart strict-aliasing warnings, fix the remaining warnings
generated by GCC 4.4.4 -Wall and -Wextra flags.

One major source of warnings was the in-house function my_bcmp
which (unconventionally) took pointers to unsigned characters
as the byte sequences to be compared. Since my_bcmp and bcmp
are deprecated functions whose only difference with memcmp is
the return value, every use of the function is replaced with
memcmp as the special return value wasn't actually being used
by any caller.

There were also various other warnings, mostly due to type
mismatches, missing return values, missing prototypes, dead
code (unreachable) and ignored return values.

BUILD/SETUP.sh:
  Remove flags that are implied by -Wall and -Wextra.
  Do not warn about unused parameters in C++.
BUILD/check-cpu:
  Print only the compiler version instead of verbose banner.
  Although the option is gcc specific, the check was only
  being used for GCC specific checks anyway.
client/mysql.cc:
  bcmp is no longer defined.
client/mysqltest.cc:
  Pass a string to function expecting a format string.
  Replace use of bcmp with memcmp.
cmd-line-utils/readline/Makefile.am:
  Always define _GNU_SOURCE when compiling GNU readline.
  Required to make certain prototypes visible.
cmd-line-utils/readline/input.c:
  Condition for the code to be meaningful.
configure.in:
  Remove check for bcmp.
extra/comp_err.c:
  Use appropriate type.
extra/replace.c:
  Replace use of bcmp with memcmp.
extra/yassl/src/crypto_wrapper.cpp:
  Do not ignore the return value of fgets. Retrieve the file
  position if fgets succeed -- if it fails, the function will
  bail out and return a error.
extra/yassl/taocrypt/include/blowfish.hpp:
  Use a single array instead of accessing positions of the sbox_
  through a subscript to pbox_.
extra/yassl/taocrypt/include/runtime.hpp:
  One definition of such functions is enough.
extra/yassl/taocrypt/src/aes.cpp:
  Avoid potentially ambiguous conditions.
extra/yassl/taocrypt/src/algebra.cpp:
  Rename arguments to avoid shadowing related warnings.
extra/yassl/taocrypt/src/blowfish.cpp:
  Avoid potentially ambiguous conditions.
extra/yassl/taocrypt/src/integer.cpp:
  Do not define type within a anonymous union.
  Use a variable to return a value instead of
  leaving the result in a register -- compiler
  does not know the logic inside the asm.
extra/yassl/taocrypt/src/misc.cpp:
  Define handler for pure virtual functions.
  Remove unused code.
extra/yassl/taocrypt/src/twofish.cpp:
  Avoid potentially ambiguous conditions.
extra/yassl/testsuite/test.hpp:
  Function must have C language linkage.
include/m_string.h:
  Remove check which relied on bcmp being defined -- they weren't
  being used as bcmp is only visible when _BSD_SOURCE is defined.
include/my_bitmap.h:
  Remove bogus helpers which were used only in a few files and
  were causing warnings about dead code.
include/my_global.h:
  Due to G++ bug, always silence false-positive uninitialized
  variables warnings when compiling C++ code with G++.
  Remove bogus helper.
libmysql/Makefile.shared:
  Remove built-in implementation of bcmp.
mysql-test/lib/My/SafeProcess/safe_process.cc:
  Cast pid to largest possible type for a process identifier.
mysys/mf_loadpath.c:
  Leave space of the ending nul.
mysys/mf_pack.c:
  Replace bcmp with memcmp.
mysys/my_bitmap.c:
  Dead code removal.
mysys/my_gethwaddr.c:
  Remove unused variable.
mysys/my_getopt.c:
  Silence bogus uninitialized variable warning.
  Do not cast away the constant qualifier.
mysys/safemalloc.c:
  Cast to expected type.
mysys/thr_lock.c:
  Silence bogus uninitialized variable warning.
sql/field.cc:
  Replace bogus helper with a more appropriate logic which is
  used throughout the code.
sql/item.cc:
  Remove bogus logical condition which always evaluates to TRUE.
sql/item_create.cc:
  Simplify code to avoid signedness related warnings.
sql/log_event.cc:
  Replace use of bcmp with memcmp.
  No need to use helpers for simple bit operations.
sql/log_event_old.cc:
  Replace bmove_align with memcpy.
sql/mysqld.cc:
  Move use declaration of variable to the ifdef block where it
  is used. Remove now-unnecessary casts and arguments.
sql/set_var.cc:
  Replace bogus helpers with simple and classic bit operations.
sql/slave.cc:
  Cast to expected type and silence bogus warning.
sql/sql_class.h:
  Don't use enum values as bit flags, the supposed type safety is
  bogus as the combined bit flags are not a value in the enumeration.
sql/udf_example.c:
  Only declare variable when necessary.
sql/unireg.h:
  Replace use of bmove_align with memcpy.
storage/innobase/os/os0file.c:
  Silence bogus warning.
storage/myisam/mi_open.c:
  Remove bogus cast, DBUG_DUMP expects a pointer to unsigned
  char.
storage/myisam/mi_page.c:
  Remove bogus cast, DBUG_DUMP expects a pointer to unsigned
  char.
strings/bcmp.c:
  Remove built-in bcmp.
strings/ctype-ucs2.c:
  Silence bogus warning.
tests/mysql_client_test.c:
  Use a appropriate type as expected by simple_command().

  
Post-push fix.
  
There was a valgrind issue on the loop that checks whether there
are NULL fields in the UNIQUE KEY or not. In detail, for the last 
iteration the server may read out of the key_part array boundaries,
making valgrind to output warnings.

We fix this by correcting the loop, ie, moving the part that reads
from the key_part to be inside the loop statement block. This way
the assignment is protected by the loop condition.

When using Unique Keys with nullable parts in RBR, the slave can
choose the wrong row to update. This happens because a table with
an unique key containing nullable parts cannot strictly guarantee 
uniqueness. As stated in the manual, for all engines, a UNIQUE 
index allows multiple NULL values for columns that can contain 
NULL.

We fix this at the slave by extending the checks before assuming
that the row found through an unique index is is the correct
one. This means that when a record (R) is fetched from the storage
engine and a key that is not primary (K) is used, the server does 
the following: 

 - If K is unique and has no nullable parts, it returns R;
 - Otherwise, if any field in the before image that is part of K
   is null do an index scan;
 - If there is no NULL field in the BI part of K, then return R.

A side change: renamed the existing test case file and added a
test case covering the changes in this patch.

This patch fixes two problems described as follows:

1 - If there is an on-going transaction and a temporary table is created or
dropped, any failed statement that follows the "create" or "drop commands"
triggers a rollback and by consequence the slave will go out sync because
the binary log will have a wrong sequence of events.

To fix the problem, we changed the expression that evaluates when the
cache should be flushed after either the rollback of a statment or
transaction.

2 - When a "CREATE TEMPORARY TABLE SELECT * FROM" was executed the
OPTION_KEEP_LOG was not set into the thd->options. For that reason, if
the transaction had updated only transactional engines and was rolled
back at the end (.e.g due to a deadlock) the changes were not written
to the binary log, including the creation of the temporary table.
      
To fix the problem, we have set the OPTION_KEEP_LOG into the thd->options
when a "CREATE TEMPORARY TABLE SELECT * FROM" is executed.

sql/log.cc:
  Reorganized the code based on the following functions:
  
  - bool ending_trans(const THD* thd, const bool all);
  - bool trans_has_updated_non_trans_table(const THD* thd);
  - bool trans_has_no_stmt_committed(const THD* thd, const bool all);
  - bool stmt_has_updated_non_trans_table(const THD* thd);
sql/log.h:
  Added functions to organize the code in log.cc.
sql/log_event.cc:
  Removed the OPTION_KEEP_LOG since it must be used only when
  creating and dropping temporary tables.
sql/log_event_old.cc:
  Removed the OPTION_KEEP_LOG since it must be used only when
  creating and dropping temporary tables.
sql/sql_parse.cc:
  When a "CREATE TEMPORARY TABLE SELECT * FROM" was executed the
  OPTION_KEEP_LOG was not set into the thd->options.
        
  To fix the problem, we have set the OPTION_KEEP_LOG into the
  thd->options when a "CREATE TEMPORARY TABLE SELECT * FROM"
  is executed.

           of sync

In RBR, sometimes the table->s->last_null_bit_pos can be zero. This
has impact at the slave when it compares records fetched from the
storage engine against records in the binary log event. If
last_null_bit_pos is zero the slave, while comparing in
log_event.cc:record_compare function, would set all bits in the last
null_byte to 1 (assumed all 8 were unused) . Thence it would loose the
ability to distinguish records that were similar in contents except
for the fact that some field was null in one record, but not in the
other. Ultimately this would cause wrong matches, and in the specific
case depicted in the bug report the same record would be updated
twice, resulting in a lost update.

Additionally, in the record_compare function the slave was setting the
X bit unconditionally. There are cases that the X bit does not exist
in the record header. This could also lead to wrong matches between
records.

We fix both by conditionally resetting the bits: (i) unused null_bits
are set if last_null_bit_pos > 0; (ii) X bit is set if
HA_OPTION_PACK_RECORD is in use.

mysql-test/extra/rpl_tests/rpl_record_compare.test:
  Shared part of the test case for MyISAM and InnoDB.
mysql-test/suite/rpl/t/rpl_row_rec_comp_innodb.test:
  InnoDB test case.
mysql-test/suite/rpl/t/rpl_row_rec_comp_myisam.test:
  MyISAM test case. Added also coverage for Field_bits case.
sql/log_event.cc:
  Deployed conditional setting of unused bits at record_compare.
sql/log_event_old.cc:
  Same change as in log_event.cc.

When mysqlbinlog was given the --database=X flag, it always printed
'ROLLBACK TO', but the corresponding 'SAVEPOINT' statement was not
printed. The replicated filter(replicated-do/ignore-db) and binlog
filter (binlog-do/ignore-db) has the same problem. They are solved
in this patch together.

After this patch, We always check whether the query is 'SAVEPOINT'
statement or not. Because this is a literal check, 'SAVEPOINT' and
'ROLLBACK TO' statements are also binlogged in uppercase with no
any comments.

The binlog before this patch can be handled correctly except one case
that any comments are in front of the keywords. for example:
 /* bla bla */ SAVEPOINT a;
 /* bla bla */ ROLLBACK TO a;

           for InnoDB
            
The class Field_bit_as_char stores the metadata for the
field incorrecly because bytes_in_rec and bit_len are set
to (field_length + 7 ) / 8 and 0 respectively, while
Field_bit has the correct values field_length / 8 and
field_length % 8.
            
Solved the problem by re-computing the values for the
metadata based on the field_length instead of using the
bytes_in_rec and bit_len variables.
            
To handle compatibility with old server, a table map
flag was added to indicate that the bit computation is
exact. If the flag is clear, the slave computes the
number of bytes required to store the bit field and
compares that instead, effectively allowing replication
*without conversion* from any field length that require
the same number of bytes to store.


mysql-test/suite/rpl/t/rpl_typeconv_innodb.test:
  Adding test to check compatibility for bit field
  replication when using InnoDB
sql/field.cc:
  Extending compatible_field_size() with flags from
  table map to allow fields to check master info.
sql/field.h:
  Extending compatible_field_size() with flags from
  table map to allow fields to check master info.
sql/log.cc:
  Removing table map flags since they are not used
  outside table map class.
sql/log_event.cc:
  Removing flags parameter from table map constructor
  since it is not used and does not have to be exposed.
sql/log_event.h:
  Adding flag to denote that bit length for bit field type
  is exact and not potentially rounded to even bytes.
sql/rpl_utility.cc:
  Adding fields to table_def to store table map flags.
sql/rpl_utility.h:
  Removing obsolete comment and adding flags to store
  table map flags from master.

MySQL with gcc 4.3.2
      
This is the final patch in the context of this bug. 

cmd-line-utils/readline/rlmbutil.h:
  Changed in a previous patch, reverted by a backport.
cmd-line-utils/readline/text.c:
  Static var initialization.
extra/yassl/include/yassl_error.hpp:
  SetErrorString handles errors outside of the YasslError
  enum.
extra/yassl/src/ssl.cpp:
  SetErrorString handles errors outside of the YasslError
  enum.
extra/yassl/src/yassl_error.cpp:
  SetErrorString handles errors outside of the YasslError
  enum.

I found three issues during the analysis:
 1. Memory leak caused by temp_buf not being freed;
 2. Memory leak caused when handling argv;
 3. Conditional jump that depended on unitialized values.

Issue #1
--------

  DESCRIPTION: when mysqlbinlog is reading from a remote location
  the event temp_buf references the incoming stream (in NET
  object), which is not freed by mysqlbinlog explicitly. On the
  other hand, when it is reading local binary log, it points to a
  temporary buffer that needs to be explicitly freed. For both
  cases, the temp_buf was not freed by mysqlbinlog, instead was
  set to 0.  This clearly disregards the free required in the
  second case, thence creating a memory leak.

  FIX: we make temp_buf to be conditionally freed depending on
  the value of remote_opt. Found out that similar fix is already
  in most recent codebases.

Issue #2 
--------

  DESCRIPTION: load_defaults is called by parse_args, and it
  reads default options from configuration files and put them
  BEFORE the arguments that are already in argc and argv. This is
  done resorting to MEM_ROOT. However, parse_args calls
  handle_options immediately after which changes argv. Later when
  freeing the defaults, pointers to MEM_ROOT won't match, causing
  the memory not to be freed:

  void free_defaults(char **argv)
  {
    MEM_ROOT ptr
    memcpy_fixed((char*) &ptr,(char *) argv - sizeof(ptr), sizeof(ptr));
    free_root(&ptr,MYF(0));
  }

  FIX: we remove load_defaults from parse_args and call it
  before. Then we save argv with defaults in defaults_argv BEFORE
  calling parse_args (which inside can then call handle_options
  at will). Actually, found out that this is in fact kind of a
  backport for BUG#38468 into 5.1, so I merged in the test case
  as well and added error check for load_defaults call.

  Fix based on:
  revid:zhenxing.he@sun.com-20091002081840-uv26f0flw4uvo33y


Issue #3 
--------

  DESCRIPTION: the structure st_print_event_info constructor
  would not initialize the sql_mode member, although it did for
  sql_mode_inited (set to false). This would later raise the
  warning in valgrind when printing the sql_mode in the event
  header, as this print out is protected by a check against
  sql_mode_inited and sql_mode variables. Given that sql_mode was
  not initialized valgrind would output the warning.

  FIX: we add initialization of sql_mode to the
  st_print_event_info constructor.
 

client/mysqlbinlog.cc:
  - Conditionally free ev->temp_buf.
  - save defaults_argv before handle_options is called.
mysql-test/t/mysqlbinlog.test:
  Added test case from BUG#38468.
sql/log_event.cc:
  Added initialization of sql_mode for st_print_event_info.

into slow log
      
While processing a statement, down the mysql_parse execution
stack, the thd->enable_slow_log can be assigned to
opt_log_slow_admin_statements, depending whether one is executing
administrative statements, such as ALTER TABLE, OPTIMIZE,
ANALYZE, etc, or not. This can have an impact on slow logging for
statements that are executed after an administrative statement
execution is completed.
      
When executing statements directly from the user this is fine
because, the thd->enable_slow_log is reset right at the beginning
of the dispatch_command function, ie, everytime a new statement
is set is set to execute.
      
On the other hand, for slave SQL thread (sql_thd) the story is a
bit different. When in SBR the sql_thd applies statements by
calling mysql_parse. Right after, it calls log_slow_statement
function to log them if they take too long. Calling mysql_parse
directly is fine, but also means that dispatch_command function
is bypassed. As a consequence, thd->enable_slow_log does not get
a chance to be reset before the next statement to be executed by
the sql_thd. If the statement just executed by the sql_thd was an
administrative statement and logging of admin statements was
disabled, this means that sql_thd->enable_slow_log will be set to
0 (disabled) from that moment on. End result: sql_thd stops
logging slow statements.
      
We fix this by resetting the value of sql_thd->enable_slow_log to
the value of opt_log_slow_slave_statements right after
log_slow_stement is called by the sql_thd.

Rename method as to not hide a base.
Reorder attributes initialization.
Remove unused variable.
Rework code to silence a warning due to assignment used as truth value.


sql/item_strfunc.cc:
  Rename method as to not hide a base.
sql/item_strfunc.h:
  Rename method as to not hide a base.
sql/log_event.cc:
  Reorder attributes initialization.
sql/rpl_injector.cc:
  Rework code to silence a warning due to assignment used as truth value.
sql/rpl_record.cc:
  Remove unused variable.
sql/sql_db.cc:
  Rework code to silence a warning due to assignment used as truth value.
sql/sql_parse.cc:
  Rework code to silence a warning due to assignment used as truth value.
sql/sql_table.cc:
  Rework code to silence a warning due to assignment used as truth value.

When replicating from 4.1 master to 5.0 slave START SLAVE UNTIL can stop too late.
The necessary in calculating of the beginning of an event the event's length
did not correspond to the master's genuine information at the event's execution time.
That piece of info was changed at the event's relay-logging due to binlog_version<4 event
conversion by IO thread.

Fixed with storing the master genuine Query_log_event size into a new status
variable at relay-logging of the event. The stored info is extacted at the event
execution and participate further to caclulate the correct start position of the event
in the until-pos stopping routine.
The new status variable's algorithm will be only active when the event comes
from the master of version < 5.0 (binlog_version < 4).


mysql-test/r/rpl_until.result:
  results changed.
mysql-test/std_data/bug47142_master-bin.000001:
  a binlog from 4.1 master to replace one of the running 5.x master is added as 
  part of Bug #47142 regression test.
mysql-test/t/rpl_until.test:
  Regression test for Bug #47142 is added.
sql/log_event.cc:
  Storing and extracting the master's genuine size of the event from the status
  var of the event packet header.
  The binlog_version<4 query-log-event is 
  a. converted into the modern binlog_version==4 to store the original size of the event
     into a new status var; the converted representation goes into the relay log.
  b. the converted event is read out and the stored size is engaged in the start pos calculation.
  
  The new status is active only for events that IO thread instantiates for the sake of the conversion.
sql/log_event.h:
  Incrementing the max szie of MAX_SIZE_LOG_EVENT_STATUS because of the new status var;
  Defining the new status variable to hold the master's genuine event size;
  Augmenting the Query_log_event with a new member to hold a value to store/extact from the status
  var of the event packet header.

PB2 run uncovered issue that needs further analysis.

BUG#49481: RBR: MyISAM and bit fields may cause slave to stop on delete: 
cant find record
      
BUG#49482: RBR: Replication may break on deletes when MyISAM tables + 
char field are used

When using MyISAM tables, despite the fact that the null bit is
set for some fields, their old value is still in the row. This
can cause the comparison of records to fail when the slave is
doing an index or range scan.

We fix this by avoiding memcmp for MyISAM tables when comparing
records. Additionally, when comparing field by field, we first
check if both fields are not null and if so, then we compare
them. If just one field is null we return failure immediately. If
both fields are null, we move on to the next field.

Fixed valgrind failure on PB2.

sql/log_event.cc:
  Added code to fix valgrind failure on PB2.

Fixed valgrind failure on PB2.

sql/log_event.cc:
  Added code to fix valgrind failure on PB2.