In order to improve the performance when replicating to partitioned
myisam tables with row-based format, the number of rows of current 
rows log event is estimated and used to setup storage engine for bulk
inserts.

      
The replication filtering rules were inappropiately applied when
executing BINLOG pseudo-query.  The rules are supposed to be active
only at times when the slave's sql thread executes an event.
            
Fixed with correcting a condition to call replication rules only if
the slave sql thread executes the event.

mysql-test/suite/rpl/r/rpl_binlog_query_filter_rules.result:
  new result file
mysql-test/suite/rpl/t/rpl_binlog_query_filter_rules-master.opt:
  a filtering option that would refuse to replicate a row event of the main test on
  slave
mysql-test/suite/rpl/t/rpl_binlog_query_filter_rules.test:
  a regression test for the bug
sql/log_event.cc:
  avoiding to call the filtering rules if the execution thread is not a slave.

- fixing double problem on big endian machines
- modifying regex_replace to replace negative numbers
  Previously only positive numbers where replaced.

Exchanging "m_cols" and "m_cols_ai".

Very confusing variable naming :(

- Implementing --base64-format=decode-rows, to display
  SQL-alike decoded row events without their BINLOG statements.
- Adding --base64-format=decode-rows into tests when
  calling mysqlbinlog to avoid non-deterministic results
- Removing resetting of last_table_id in "RESET MASTER",
  which appeared to be dangerous.

Implementing -v command line parameter to mysqlbinlog
to decode and print row events.

mysql-test/include/mysqlbinlog_row_engine.inc
mysql-test/r/mysqlbinlog_row.result
mysql-test/r/mysqlbinlog_row_big.result
mysql-test/r/mysqlbinlog_row_innodb.result
mysql-test/r/mysqlbinlog_row_myisam.result
mysql-test/r/mysqlbinlog_row_trans.result
mysql-test/t/mysqlbinlog_row.test
mysql-test/t/mysqlbinlog_row_big.test
mysql-test/t/mysqlbinlog_row_innodb.test
mysql-test/t/mysqlbinlog_row_myisam.test
mysql-test/t/mysqlbinlog_row_trans.test
  Adding tests 

client/Makefile.am
  Adding new files to symlink
  
client/mysqlbinlog.cc
  Adding -v option

sql/log_event.cc
  Impelentations of the new methods

sql/log_event.h
  Declaration of the new methods and member

sql/mysql_priv.h
  Adding new function prototype

sql/rpl_tblmap.cc
  Adding pre-processor conditions 

sql/rpl_tblmap.h
  Adding pre-processor conditions 

sql/rpl_utility.h
  Adding pre-processor conditions 

sql/sql_base.cc
  Adding reset_table_id_sequence() function.

sql/sql_repl.cc
  Resetting table_id on "RESET MASTER"
  
.bzrignore
  Ignoring new symlinked files

On certain kinds of errors (e.g., out of stack), a call to Item_func_
set_user_var::fix_fields() might fail.  Since the return value of this
call was not checked inside User_var_log_event::exec_event(), continuing
execution after this will cause a crash inside Item_func_set_user_var::
update_hash().

The bug is fixed by aborting execution of the event with an error if
fix_fields() fails, since it is not possible to continue execution anyway.


sql/log_event.cc:
  Aborting execution of event if fix_fields() fails since execution
  of update_hash() might cause a crash.

When flushing tables, there were a slight chance that the flush was occuring
between processing of two table map events. Since the tables are opened
one by one, it might result in that the tables were not valid and that sub-
sequent locking of tables would cause the slave to crash.

The problem is solved by opening and locking all tables at once using
simple_open_n_lock_tables(). Also, the patch contain a change to open_tables()
so that pre-locking only takes place when the trg_event_map is not zero, which
was not the case before (this caused the lock to be placed in thd->locked_tables
instead of thd->lock since the assumption was that triggers would be called
later and therefore the tables should be pre-locked).


mysql-test/suite/rpl/r/rpl_found_rows.result:
  Result change
mysql-test/suite/rpl/r/rpl_row_inexist_tbl.result:
  Result change
mysql-test/suite/rpl/t/rpl_found_rows.test:
  Adding drop of table that was created in test.
mysql-test/suite/rpl/t/rpl_slave_status.test:
  Adding waits for slave start and stop to ensure that test works.
sql/log_event.cc:
  Replacing table-by-table open and lock with a single call
  to simple_open_n_lock_tables(), which in turn required some
  changes to other code.
sql/log_event_old.cc:
  Replacing table-by-table open and lock with a single call
  to simple_open_n_lock_tables(), which in turn required some
  changes to other code.
sql/sql_base.cc:
  Extending check inside open_tables() so that pre-locking in only done if
  tables->trg_egent_map is non-zero.
mysql-test/include/wait_for_slave_sql_to_start.inc:
  New BitKeeper file ``mysql-test/include/wait_for_slave_sql_to_start.inc''

The bug allow multiple executing transactions working with non-transactional
to interfere with each others by interleaving the events of different trans-
actions.

Bug is fixed by writing non-transactional events to the transaction cache and
flushing the cache to the binary log at statement commit. To mimic the behavior
of normal statement-based replication, we flush the transaction cache in row-
based mode when there is no committed statements in the transaction cache,
which means we are committing the first one. This means that it will be written
to the binary log as a "mini-transaction" with just the rows for the statement.

Note that the changes here does not take effect when building the server with
HAVE_TRANSACTIONS set to false, but it is not clear if this was possible before
this patch either.

For row-based logging, we also have that when AUTOCOMMIT=1, the code now always
generates a BEGIN/COMMIT pair for single statements, or BEGIN/ROLLBACK pair in the
case of non-transactional changes in a statement that was rolled back. Note that
for the case where changes to a non-transactional table causes a rollback due
to error, the statement will now be logged with a BEGIN/ROLLBACK pair, even
though some changes has been committed to the non-transactional table.


mysql-test/extra/rpl_tests/rpl_row_delayed_ins.test:
  Removing SHOW BINLOG EVENTS causing test to be non-deterministic.
mysql-test/r/ctype_cp932_binlog_row.result:
  Result change.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
  Result change.
mysql-test/suite/binlog/r/binlog_multi_engine.result:
  Result file change.
mysql-test/suite/binlog/r/binlog_row_binlog.result:
  Result file change.
mysql-test/suite/binlog/r/binlog_row_ctype_ucs.result:
  Result file change.
mysql-test/suite/binlog/r/binlog_row_insert_select.result:
  Result file change.
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
  Result file change.
mysql-test/suite/binlog/r/binlog_stm_binlog.result:
  Result file change.
mysql-test/suite/binlog/t/binlog_base64_flag.test:
  Removing table that will be used in test to prevent failing if preceeding
  tests forgot to drop the table.
mysql-test/suite/rpl/r/rpl_rbr_to_sbr.result:
  Result file change.
mysql-test/suite/rpl/r/rpl_row_basic_11bugs.result:
  Result file change.
mysql-test/suite/rpl/r/rpl_row_create_table.result:
  Result file change.
mysql-test/suite/rpl/r/rpl_row_delayed_ins.result:
  Result file change.
mysql-test/suite/rpl/r/rpl_row_flsh_tbls.result:
  Result file change.
mysql-test/suite/rpl/r/rpl_row_inexist_tbl.result:
  Result file change.
mysql-test/suite/rpl/r/rpl_row_log.result:
  Result file change.
mysql-test/suite/rpl/r/rpl_row_log_innodb.result:
  Result file change.
mysql-test/suite/rpl/r/rpl_row_until.result:
  Result file change.
mysql-test/suite/rpl/r/rpl_slave_skip.result:
  Result file change.
mysql-test/suite/rpl/r/rpl_switch_stm_row_mixed.result:
  Result file change.
mysql-test/suite/rpl/r/rpl_truncate_2myisam.result:
  Result file change.
mysql-test/suite/rpl/t/rpl_row_create_table.test:
  Binlog position change.
mysql-test/suite/rpl/t/rpl_row_flsh_tbls.test:
  Binlog position change.
mysql-test/suite/rpl/t/rpl_row_mysqlbinlog.test:
  Binlog position change. Added stop position to mysqlbinlog argments to prevent
  extreneous output.
mysql-test/suite/rpl/t/rpl_row_until.test:
  Binlog position change.
mysql-test/suite/rpl/t/rpl_slave_skip.test:
  Binlog position change.
mysql-test/suite/rpl/t/rpl_switch_stm_row_mixed.test:
  Removing extreneous SHOW BINLOG EVENTS causing test to be non-deterministic.
mysql-test/suite/rpl_ndb/r/rpl_ndb_log.result:
  Result change.
sql/log.cc:
  Adding variable at_least_one_stmt to denote that there is at least one
  statement committed to the transaction cache (but there might be more).
  
  Removing duplicate checks from binlog_end_trans(). The transaction cache
  should always be committed or rolled back when this function is called.
  
  Correcting conditions for binlog_rollback() and binlog_commit() and removing
  the previous "invisible commit" in favor of always using explicit commits
  in the binary log.
sql/log_event.cc:
  Marking table map event to be cached. Removing Muted_query_log_event from code.
sql/log_event.h:
  Removing unused class Muted_query_log_event.
sql/sql_insert.cc:
  Adding missing call to ha_autocommit_or_rollback() for delayed thread. Marking
  CREATE-SELECT statements as transactional, since they don't need to be logged.

Problem: in mixed and statement mode, a query that refers to a
system variable will use the slave's value when replayed on
slave. So if the value of a system variable is inserted into a
table, the slave will differ from the master.
Fix: mark statements that refer to a system variable as "unsafe",
meaning they will be replicated by row in mixed mode and produce a warning
in statement mode. There are some exceptions: some variables are actually
replicated. Those should *not* be marked as unsafe.
BUG#34732: mysqlbinlog does not print default values for auto_increment variables
Problem: mysqlbinlog does not print default values for some variables,
including auto_increment_increment and others. So if a client executing
the output of mysqlbinlog has different default values, replication will
be wrong.
Fix: Always print default values for all variables that are replicated.
I need to fix the two bugs at the same time, because the test cases would
fail if I only fixed one of them.


include/m_ctype.h:
  Added definition of ILLEGAL_CHARSET_INFO_NUMBER. We just need a symbol
  for a number that will never be used by any charset. ~0U should be safe
  since charset numbers are sequential, starting from 0.
mysql-test/include/commit.inc:
  Upated test to avoid making statements unsafe.
mysql-test/r/commit_1innodb.result:
  Updated test needs updated result file.
mysql-test/r/mysqlbinlog.result:
  Updated result file.
mysql-test/r/mysqlbinlog2.result:
  Updated result file.
mysql-test/r/user_var-binlog.result:
  Updated result file.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
  Updated result file.
mysql-test/suite/binlog/r/binlog_stm_ctype_ucs.result:
  Updated result file.
mysql-test/suite/binlog/r/binlog_unsafe.result:
  Modified test file needs modified result file.
mysql-test/suite/binlog/t/binlog_base64_flag.test:
  Need to filter out pseudo_thread_id from result since it is
  nondeterministic.
mysql-test/suite/binlog/t/binlog_unsafe.test:
  Add tests that using variables is unsafe. The 'CREATE VIEW' tests didn't
  make sense, so I removed them. SHOW WARNINGS is not necessary either,
  because we get warnings for each statement in the result file.
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
  Updated result file.
mysql-test/suite/rpl/r/rpl_skip_error.result:
  Updated result file.
mysql-test/suite/rpl/t/rpl_skip_error.test:
  The test used @@server_id, which is not safe to replicate, so it would
  have given a warning. The way it used @@server_id was hackish (issue a
  query on master that removes rows only on master), so I replaced it by a
  more robust way to do the same thing (connect to slave and insert the
  rows only there).
  Also clarified what the test case does.
mysql-test/t/mysqlbinlog2.test:
  Use --short-form instead of manually filtering out nondeterministic stuff
  from mysqlbinlog (because we added the nondeterministic @@pseudo_thread_id
  to the output).
sql/item_func.cc:
  Added method of Item_func_get_system_var that indicates whether the given
  system variable will be written to the binlog or not.
sql/item_func.h:
  Added method of Item_func_get_system_var that indicates whether the given
  system variable will be written to the binlog or not.
sql/log_event.cc:
   - auto_increment_offset was not written to the binlog if
  auto_increment_increment=1
   - mysqlbinlog did not output default values for some variables
  (BUG#34732). In st_print_event_info, we remember for each variable whether
  it has been printed or not. This is achieved in different ways for
  different variables:
      - For auto_increment_*, lc_time_names, charset_database_number,
        we set the default values in st_print_event_info to something
        illegal, so that it will look like they have changed the first time
        they are seen.
      - For charset, sql_mode, pseudo_thread_id, we add a flag to
        st_print_event_info which indicates whether the variable has been
        printed.
      - Since pseudo_thread_id is now printed more often, and its value is
        not guaranteed to be constant across different runs of the same
        test script, I replaced it by a constant if --short-form is used.
   - Moved st_print_event_info constructor from log_event.h to log_event.cc,
  since it now depends on ILLEGAL_CHARSET_NUMBER, which is defined in
  m_ctype.h, which is better to include from a .cc file than from a header
  file.
sql/log_event.h:
  Added fields to st_print_event_info that indicate whether some of the
  variables have been written or not. Since the initialization of
  charset_database_number now depends on ILLEGAL_CHARSET_INFO_NUMBER, which
  is defined in a header file, which we'd better not include from this
  header file -- I moved the constructor from here to log_event.cc.
sql/set_var.cc:
  System variables now have a flag binlog_status, which indicates if they
  are written to the binlog. If nothing is specified, all variables are
  marked as not written to the binlog (NOT_IN_BINLOG) when created. In this
  file, the variables that are written to the binlog are marked with
  SESSION_VARIABLE_IN_BINLOG.
sql/set_var.h:
  Added flag binlog_status to class sys_var. Added a getter and a
  constructor parameter that sets it.
  Since I had to change sys_var_thd_enum constructor anyways, I simplified
  it to use default values of arguments instead of three copies of the
  constructor.
sql/sql_yacc.yy:
  Mark statements that refer to a system variable as "unsafe",
  meaning they will be replicated by row in mixed mode. Added comment to
  explain strange piece of code just above.
mysql-test/include/diff_tables.inc:
  New auxiliary test file that tests whether two tables (possibly one on
  master and one on slave) differ.
mysql-test/suite/rpl/r/rpl_variables.result:
  New test case needs new result file.
mysql-test/suite/rpl/r/rpl_variables_stm.result:
  New test file needs new result file.
mysql-test/suite/rpl/t/rpl_variables.test:
  Test that INSERT of @@variables is replicated correctly (by switching to
  row-based mode).
mysql-test/suite/rpl/t/rpl_variables_stm.test:
  Test that replication of @@variables which are replicated explicitly works
  as expected in statement mode (without giving warnings).

There was a failure in that show slave status displayed a wrong message
when slave stopped at processing a row event inserting to a default-less
column.

The problem seem to have ceased after recent fixes in rbr code.
However, the test was not updated to carry testing of the case commented-out.

Uncommenting and editing the test.
Notice, Bug#23907 is most probably a duplicate of this one.


mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test:
  uncommenting the test that previously failed.
mysql-test/suite/rpl/r/rpl_extraCol_innodb.result:
  results changed due to the correct expected error message of an added test
  is displayed.
mysql-test/suite/rpl/r/rpl_extraCol_myisam.result:
  results changed to reflect the test changes
mysql-test/suite/rpl_ndb/r/rpl_ndb_extraCol.result:
  results changed to reflect the uncommented test
sql/log_event.cc:
  correcting of a merge with bug#12713 to use the correct object holding
  the error code.

Rename client_last_error to last_error and client_last_errno to last_errno
to not break connectors which use the internal net structure for error handling.


include/mysql_com.h:
  Rename client_last_error to last_error, client_last_errno to last_errno.
include/mysql_h.ic:
  Rename client_last_error to last_error, client_last_errno to last_errno.
libmysql/libmysql.c:
  Rename client_last_error to last_error, client_last_errno to last_errno.
libmysql/manager.c:
  Rename client_last_error to last_error, client_last_errno to last_errno.
libmysqld/lib_sql.cc:
  Rename client_last_error to last_error, client_last_errno to last_errno.
libmysqld/libmysqld.c:
  Rename client_last_error to last_error, client_last_errno to last_errno.
server-tools/instance-manager/mysql_connection.cc:
  Rename client_last_error to last_error, client_last_errno to last_errno.
sql/log_event.cc:
  Rename client_last_error to last_error, client_last_errno to last_errno.
sql-common/client.c:
  Rename client_last_error to last_error, client_last_errno to last_errno.
sql/log_event_old.cc:
  Rename client_last_error to last_error, client_last_errno to last_errno.
sql/net_serv.cc:
  Rename client_last_error to last_error, client_last_errno to last_errno.
sql/repl_failsafe.cc:
  Rename client_last_error to last_error, client_last_errno to last_errno.

            log-slave-updates and circul repl

Slave SQL thread may execute one extra event when there are events
skipped by slave I/O thread (e.g. originated by the same server).
Whereas it was requested not to do so by the UNTIL condition.

This happens because we compare with the end position of previously
executed event. This is fine when there are no skipped by slave I/O
thread events, as end position of previous event equals to start
position of to be executed event. Otherwise this position equals to
start position of skipped event.

This is fixed by:
- reading the event to be executed before checking if the until condition
  is satisfied.
- comparing the start position of the event to be executed. Since we do
  not have the start position available, we compute it by subtracting
  event length from end position (which is available).
- if there are no events on the event queue at the slave sql starting
  time, that meet until condition, we stop immediately, as in this
  case we do not want to wait for next event.


mysql-test/r/rpl_dual_pos_advance.result:
  A test case for BUG#13861.
mysql-test/t/rpl_dual_pos_advance.test:
  A test case for BUG#13861.
sql/log_event.cc:
  Store length of event. This is needed for further calculation of
  the beginning of event.
sql/slave.cc:
  Slave SQL thread may execute one extra event when there are events
  skipped by slave I/O thread (e.g. originated by the same server).
  Whereas it was requested not to do so by the UNTIL condition.
  
  This happens because we compare with the end position of previously
  executed event. This is fine when there are no skipped by slave I/O
  thread events, as end position of previous event equals to start
  position of to be executed event. Otherwise this position equals to
  start position of skipped event.
  
  This is fixed by:
  - reading the event to be executed before checking if the until condition
    is satisfied.
  - comparing the start position of the event to be executed. Since we do
    not have the start position available, we compute it by subtracting
    event length from end position (which is available).
  - if there are no events on the event queue at the slave sql starting
    time, that meet until condition, we stop immediately, as in this
    case we do not want to wait for next event.
sql/slave.h:
  Added master_log_pos parametr to is_until_satisfied().
mysql-test/t/rpl_dual_pos_advance-slave.opt:
  New BitKeeper file ``mysql-test/t/rpl_dual_pos_advance-slave.opt''

a SELECT doesn't cause ROLLBACK of statem".

The idea of the fix is to ensure that we always commit the current
statement at the end of dispatch_command(). In order to not issue
redundant disc syncs, an optimization of the two-phase commit
protocol is implemented to bypass the two phase commit if
the transaction is read-only.


mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result:
  Update test results.
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
  Update test results.
mysql-test/suite/rpl_ndb/t/disabled.def:
  Disable the tests, for which this changeset reveals a bug:
  the injector thread does not always add 'statement commit' to the
  rows injected in circular replication set up.
  To be investigated separately.
sql/ha_ndbcluster_binlog.cc:
  Add close_thread_tables() to run_query: this ensures
  that all tables are closed and there is no pending statement transaction.
sql/handler.cc:
  Implement optimisation of read-only transactions.
  If a transaction consists only of DML statements that do not change
  data, we do not perform a two-phase commit for it 
  (run one phase commit only).
sql/handler.h:
  Implement optimisation of read-only transactions.
  If a transaction consists only of DML statements that do not change
  data, we do not perform a two-phase commit for it 
  (run one phase commit only).
sql/log.cc:
  Mark the binlog transaction read-write whenever it's started.
  We never read from binlog, so it's safe and least intrusive to add
  this mark up here.
sql/log_event.cc:
  Update to the new layout of thd->transaction.
sql/rpl_injector.cc:
  Always commit statement transaction before committing the global one.
sql/sp.cc:
  Ad comments.
sql/sp_head.cc:
  Add comments.
sql/sql_base.cc:
  Commit transaction at the end of the statement. Always.
sql/sql_class.cc:
  Update thd_ha_data to return the right pointer in the new layout.
  
  Fix select_dumpvar::send_data to properly return operation status.
  A test case from commit.inc would lead to an assertion failure in the 
  diagnostics area (double assignment). Not test otherwise by the test suite.
sql/sql_class.h:
  Implement a new layout of storage engine transaction info in which 
  it is easy to access all members related to the handlerton only
  based on ht->slot.
sql/sql_cursor.cc:
  Update to the new layout of thd->transaction.
sql/sql_delete.cc:
  Remove wrong and now redundant calls to ha_autocommit_or_rollback.
  The transaction is committed in one place, at the end of the statement.
  Remove calls to mysql_unlock_tables, since some engines count locks
  and commit statement transaction in unlock_tables(), which essentially
  equates mysql_unlock_tables to ha_autocommit_or_rollback.
  Previously it was necessary to unlock tables soon because we wanted
  to avoid sending of 'ok' packet to the client under locked tables.
  This is no longer necessary, since OK packet is also sent from one place
  at the end of transaction.
sql/sql_do.cc:
  Add DO always clears the error, we must rollback the current
  statement before this happens. Otherwise the statement will be committed,
  and not rolled back in the end.
sql/sql_insert.cc:
  Remove wrong and now redundant calls to ha_autocommit_or_rollback.
  The transaction is committed in one place, at the end of the statement.
  Remove calls to mysql_unlock_tables, since some engines count locks
  and commit statement transaction in unlock_tables(), which essentially
  equates mysql_unlock_tables to ha_autocommit_or_rollback.
  Previously it was necessary to unlock tables soon because we wanted
  to avoid sending of 'ok' packet to the client under locked tables.
  This is no longer necessary, since OK packet is also sent from one place
  at the end of transaction.
sql/sql_load.cc:
  Remove wrong and now redundant calls to ha_autocommit_or_rollback.
  The transaction is committed in one place, at the end of the statement.
  Remove calls to mysql_unlock_tables, since some engines count locks
  and commit statement transaction in unlock_tables(), which essentially
  equates mysql_unlock_tables to ha_autocommit_or_rollback.
  Previously it was necessary to unlock tables soon because we wanted
  to avoid sending of 'ok' packet to the client under locked tables.
  This is no longer necessary, since OK packet is also sent from one place
  at the end of transaction.
sql/sql_parse.cc:
  Implement optimisation of read-only transactions: bypass 2-phase
  commit for them.
  Always commit statement transaction before commiting the global one.
  Fix an unrelated crash in check_table_access, when called from 
  information_schema.
sql/sql_partition.cc:
  Partitions commit at the end of a DDL operation.
  Make sure that send_ok() is done only if the commit has succeeded.
sql/sql_table.cc:
  Use ha_autocommit_or_rollback and end_active_trans everywhere.
  Add end_trans to mysql_admin_table, so that it leaves no pending
  transaction.
sql/sql_udf.cc:
  Remvove a redundant call to close_thread_tables()
sql/sql_update.cc:
  Remove wrong and now redundant calls to ha_autocommit_or_rollback.
  The transaction is committed in one place, at the end of the statement.
  Remove calls to mysql_unlock_tables, since some engines count locks
  and commit statement transaction in unlock_tables(), which essentially
  equates mysql_unlock_tables to ha_autocommit_or_rollback.
  Previously it was necessary to unlock tables soon because we wanted
  to avoid sending of 'ok' packet to the client under locked tables.
  This is no longer necessary, since OK packet is also sent from one place
  at the end of transaction.
mysql-test/include/commit.inc:
  New BitKeeper file ``mysql-test/include/commit.inc''
mysql-test/r/commit_1innodb.result:
  New BitKeeper file ``mysql-test/r/commit_1innodb.result''
mysql-test/t/commit_1innodb.test:
  New BitKeeper file ``mysql-test/t/commit_1innodb.test''

The error message due to lack of the default value for an extra field
was not as informative as it should be.

Fixed with improving the scheme of gathering, propagating and reporting
errors in applying rows events. 
The scheme is in the following.
Any kind of error of processing of a row event incidents are to be 
registered with my_error().
In the end Rows_log_event::do_apply_event() invokes rli->report() with the 
message to display consisting of all the errors.
This mimics `show warnings' displaying.
A simple test checks three errors in processing an event.
Two hunks - a user level error and pushing it into the list - 
have been devoted to already fixed Bug@31702.

Some open issues relating to this artifact listed on BUG@21842 page and
on WL@3679.
Todo: to synchronize the statement in the tests comments on Update and Delete
events may not stop when an extra field does not have a default with wl@3228 spec.


include/my_base.h:
  A new handler level error code that is supposed to be mapped to a set of more
  specific ER_ user level errors.
mysql-test/extra/rpl_tests/rpl_row_tabledefs.test:
  Adding yet another extra fields to see more than one error in show
  slave status' report.
mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result:
  results changed (the error message etc)
mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result:
  results changed
sql/log_event.cc:
  Refining slave_rows_error_report to iterate on the list of gathered errors;
  Simplifying signature of prepare_record as the function does not call
  rli->report to leave that duty to the event's top level code.
sql/log_event.h:
  adding a corrupt event error pushing. The error will be seen with
  show slave status.
sql/log_event_old.cc:
  similar to log_event.cc changes
sql/rpl_record.cc:
  prepare_record only pushes an error to the list
sql/rpl_record.h:
  signature changed
sql/share/errmsg.txt:
  The user level error code that corresponds to HA_ERR_CORRUPT_EVENT.
  The error will be reported in show slave status if such a failure happens.

BitKeeper/deleted/.del-show_binlog_events2.inc:
  Delete: mysql-test/include/show_binlog_events2.inc
client/mysqlbinlog.cc:
  char -> uchar for raw memory.
sql/item_cmpfunc.cc:
  Adding cast to remove warning when converting negative integer
  to unsigned type.
sql/log_event.cc:
  char -> uchar for raw memory.
sql/log_event.h:
  char -> uchar for raw memory.
sql/rpl_utility.cc:
  Adding cast to remove warning when converting negative integer
  to unsigned type.
sql/slave.cc:
  char -> uchar for raw memory.
sql/sql_repl.cc:
  char -> uchar for raw memory.
sql-common/client.c:
  char -> uchar for raw memory.

Main problem: mysql 5.1 cannot read binlogs from 4.1.
Subproblem 1: There is a mistake in sql_ex_info::init. The read_str()
function updates its first argument to point to the next character to
read. However, it is applied only to a copy of the buffer pointer, so the
real buffer pointer is not updated.
Fix 1: do not take a copy of the buffer pointer. The copy was needed
because sql_ex_info::init does not use the const attribute on some of its
arguments. So we add the const attribute, too.
Subproblem 2: The first BINLOG statement is asserted to be a
FORMAT_DESCRIPTION_LOG_EVENT, but 4.1 binlogs begin with START_EVENT_V3.
Fix 2: allow START_EVENT_V3 too.


mysql-test/suite/binlog/std_data/binlog_old_version_4_1.000001:
  New BitKeeper file ``mysql-test/suite/binlog/std_data/binlog_old_version_4_1.000001''
mysql-test/suite/binlog/r/binlog_old_versions.result:
  Updated result file.
mysql-test/suite/binlog/t/binlog_old_versions.test:
  Added a test reading an old 4.1 binlog.
sql/log_event.cc:
  1. Added const keyword at the following places:
   - input buffer for pretty_print_str
   - input buffer for write_str
   - input buffer, end pointer, and return value from sql_ex_info::init
  2. Fixed the bug by not taking a copy of buf before calling read_str in
  sql_ex_info::init().
sql/log_event.h:
  Added const keyword to fields of the sql_ex_info struct.
  Added const keyword to arguments and return value of sql_ex_info::init
sql/sql_binlog.cc:
  The first BINLOG statement must describe the format for future BINLOG
  statements. Otherwise, we do not know how to read the BINLOG statement.
  Problem: only FORMAT_DESCRIPTION_EVENT is currently allowed as the first
  event. Binlogs from 4.1 begin with a START_EVENT_V3, which serves the
  same purpose.
  Fix: We now allow the first BINLOG statement to be a START_EVENT_V3, as
  well as a FORMAT_DESCRIPTION_EVENT.

Problem: When slave reads format_description_log_event, it checks if the
master is a version that uses an old binlog format. See also BUG#27779.
Not all possible server_versions were listed.
Fix: Check for all server_versions which use the old binlog_format.


sql/log_event.cc:
  In the place where we check if server_version indicates that master is
  the alcatel branch, we now check all currently possible alcatel versions,
  not just a subset. Added comment to explain which clones are affected.

 - make sure to reset the read and write sets


handler.cc, log_event.cc:
  - make sure to reset the read and write sets


sql/handler.cc:
  - make sure to reset the read and write sets
sql/log_event.cc:
  - make sure to reset the read and write sets

Problem: Replication fails when master is mysql-5.1-wl2325-5.0-drop6 and
slave is mysql-5.1-new-rpl. The reason is that, in
mysql-5.1-wl2325-5.0-drop6, the event type id's were different than in
mysql-5.1-new-rpl.
Fix (in mysql-5.1-new-rpl):
 (1) detect that the server that generated the events uses the old
format, by checking the server version of the format_description_log_event
This patch recognizes mysql-5.1-wl2325-5.0-drop6p13-alpha,
mysql-5.1-wl2325-5.0-drop6, mysql-5.1-wl2325-5.0, mysql-5.1-wl2325-no-dd.
 (2) if the generating server is old, map old event types to new event
types using a permutation array.

I've also added a test case which reads binlogs for four different
versions.


mysql-test/suite/binlog/std_data/binlog_old_version_5_1-telco.000001:
  BitKeeper file /home/sven/bk/b27779-old_row_events/5.1-new-rpl/mysql-test/suite/binlog/std_data/binlog_old_version_5_1-telco.000001
mysql-test/suite/binlog/std_data/binlog_old_version_5_1-wl2325_row.000001:
  BitKeeper file /home/sven/bk/b27779-old_row_events/5.1-new-rpl/mysql-test/suite/binlog/std_data/binlog_old_version_5_1-wl2325_row.000001
mysql-test/suite/binlog/std_data/binlog_old_version_5_1-wl2325_stm.000001:
  BitKeeper file /home/sven/bk/b27779-old_row_events/5.1-new-rpl/mysql-test/suite/binlog/std_data/binlog_old_version_5_1-wl2325_stm.000001
mysql-test/suite/binlog/std_data/binlog_old_version_5_1_17.000001:
  BitKeeper file /home/sven/bk/b27779-old_row_events/5.1-new-rpl/mysql-test/suite/binlog/std_data/binlog_old_version_5_1_17.000001
mysql-test/suite/binlog/std_data/binlog_old_version_5_1_23.000001:
  BitKeeper file /home/sven/bk/b27779-old_row_events/5.1-new-rpl/mysql-test/suite/binlog/std_data/binlog_old_version_5_1_23.000001
sql/log_event.cc:
  Added code to read events generated by
  mysql-5.1-wl2325-5.0-drop6p13-alpha, mysql-5.1-wl2325-5.0-drop6,
  mysql-5.1-wl2325-5.0, mysql-5.1-wl2325-no-dd.
  More precisely, the event type id's had different numbers in
  those versions. To fix, we add a permutation array which maps old_id to
  new_id when the format_description_log_event indicates that the
  originating server is of the old type. We also need to permute the
  post_header_len array accordingly.
sql/log_event.h:
  sql/log_event.h@1.169, 2008-01-09 11:34:37+01:00, sven@riska.(none) +5 -1
  Added declaration needed in log_event.cc. Also, the destructor of
  Format_description_log_event is sometimes called when post_header_len is
  null, so we must pass the MY_ALLOW_ZERO_PTR flag to my_free.
mysql-test/suite/binlog/r/binlog_old_versions.result:
  Result file for new test.
mysql-test/suite/binlog/t/binlog_old_versions.test:
  New test case that loads binlogs from several old versions.

mysql-test/r/binlog_start_comment.result:
  Result change.
mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result:
  Result change.
mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result:
  Result change.
mysql-test/suite/rpl/r/rpl_timezone.result:
  Result change.
mysql-test/suite/rpl_ndb/r/rpl_ndb_transaction.result:
  Result change.
mysql-test/suite/rpl_ndb/t/rpl_ndb_transaction.test:
  Masking out columns with binlog positions from slave status.
mysql-test/t/binlog_start_comment.test:
  Added missing drop table causing subsequent tests to fail.
sql/log_event.cc:
  Changing last_err* fields to client_last_err*
sql/log_event_old.cc:
  Changing last_err* fields to client_last_err*

Now, every transaction (including autocommit transactions) start with
a BEGIN and end with a COMMIT/ROLLBACK in the binlog.
Added a test case, and updated lots of test case result files.


mysql-test/t/rpl_transaction-master.opt:
  BitKeeper file /home/sven/bk/b26395-autocommit-xa/5.0-rpl/mysql-test/t/rpl_transaction-master.opt
mysql-test/t/rpl_transaction-slave.opt:
  BitKeeper file /home/sven/bk/b26395-autocommit-xa/5.0-rpl/mysql-test/t/rpl_transaction-slave.opt
mysql-test/r/mix_innodb_myisam_binlog.result:
  Updated result file
mysql-test/r/multi_update.result:
  Updated result file
mysql-test/r/rpl_transaction.result:
  New result file for new test case.
mysql-test/r/sp_trans_log.result:
  Updated result file
mysql-test/r/variables-big.result:
  Updated result file
mysql-test/t/rpl_transaction.test:
  New test case.
sql/log.cc:
   - Always write BEGIN and COMMIT around statements, even in autocommit
     mode.
   - Added comments for binlog_commit and binlog_rollback.
sql/log_event.cc:
  Added debug trigger to avoid writing xid events to the binlog.

- make sure to reset the read and write sets

fine

The reason of this bug is that when mysqlbinlog dumps a query, the query is written to
output with a delimeter appended right after it, if the query string ends with a '--'
comment, then the delimeter would be considered as part of the comment, if there are any
statements after this query, then it will cause a syntax error.

Start a newline before appending delimiter after a query string


mysql-test/r/mysqlbinlog.result:
  Update result for BUG#32205
mysql-test/r/mysqlbinlog2.result:
  Update result for BUG#32205
mysql-test/r/user_var-binlog.result:
  Update result for BUG#32205
mysql-test/suite/binlog/r/binlog_stm_ctype_ucs.result:
  Update result for BUG#32205
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
  Update result for BUG#32205
mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam.test:
  fix test for BUG#32205
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
  Update result for BUG#32205
mysql-test/suite/rpl/r/rpl_stm_charset.result:
  Update result for BUG#32205
sql/log_event.cc:
  Start a newline before appending delimiter after a query string
mysql-test/suite/binlog/r/binlog_start_comment.result:
  Add test for BUG#32205
mysql-test/suite/binlog/t/binlog_start_comment.test:
  Add test for BUG#32205

The reason of this bug is that when mysqlbinlog dumps a query, the query is written to
output with a delimeter appended right after it, if the query string ends with a '--'
comment, then the delimeter would be considered as part of the comment, if there are any
statements after this query, then it will cause a syntax error.

Start a newline before appending delimiter after a query string


mysql-test/r/ctype_ucs_binlog.result:
  Update test result for BUG#32205
mysql-test/r/mix_innodb_myisam_binlog.result:
  Update test result for BUG#32205
mysql-test/r/mysqlbinlog.result:
  Update test result for BUG#32205
mysql-test/r/rpl_charset.result:
  Update test result for BUG#32205
mysql-test/r/rpl_timezone.result:
  Update test result for BUG#32205
mysql-test/r/user_var-binlog.result:
  Update test result for BUG#32205
mysql-test/t/mix_innodb_myisam_binlog.test:
  Fixed for BUG#32205
sql/log_event.cc:
  Start a newline before appending delimiter after a query string
mysql-test/r/binlog_start_comment.result:
  Add test for BUG#32205
mysql-test/t/binlog_start_comment.test:
  Add test for BUG#32205

Problem: it is unsafe to read base64-printed events without first
reading the Format_description_log_event (FD).  Currently, mysqlbinlog
cannot print the FD.

As a side effect, another bug has also been fixed: When mysqlbinlog
--start-position=X was specified, no ROLLBACK was printed. I changed
this, so that ROLLBACK is always printed.

This patch does several things:

 - Format_description_log_event (FD) now print themselves in base64
   format.

 - mysqlbinlog is now able to print FD events.  It has three modes:
    --base64-output=auto    Print row events in base64 output, and print
                            FD event.  The FD event is printed even if
                            it is outside the range specified with
                            --start-position, because it would not be
                            safe to read row events otherwise. This is
                            the default.

    --base64-output=always  Like --base64-output=auto, but also print
                            base64 output for query events.  This is
                            like the old --base64-output flag, which
                            is also a shorthand for
                            --base64-output=always

    --base64-output=never   Never print base64 output, generate error if
                            row events occur in binlog.  This is
                            useful to suppress the FD event in binlogs
                            known not to contain row events (e.g.,
                            because BINLOG statement is unsafe,
                            requires root privileges, is not SQL, etc)

 - the BINLOG statement now handles FD events correctly, by setting
   the thread's rli's relay log's description_event_for_exec to the
   loaded event.

   In fact, executing a BINLOG statement is almost the same as reading
   an event from a relay log.  Before my patch, the code for this was
   separated (exec_relay_log_event in slave.cc executes events from
   the relay log, mysql_client_binlog_statement in sql_binlog.cc
   executes BINLOG statements).  I needed to augment
   mysql_client_binlog_statement to do parts of what
   exec_relay_log_event does.  Hence, I did a small refactoring and
   moved parts of exec_relay_log_event to a new function, which I
   named apply_event_and_update_pos.  apply_event_and_update_pos is
   called both from exec_relay_log_event and from
   mysql_client_binlog_statement.

 - When a non-FD event is executed in a BINLOG statement, without
   previously executing a FD event in a BINLOG statement, it generates
   an error, because that's unsafe.  I took a new error code for that:
   ER_NO_FORMAT_DESCRIPTION_EVENT_BEFORE_BINLOG_STATEMENTS.

   In order to get a decent error message containing the name of the
   event, I added the class method char*
   Log_event::get_type_str(Log_event_type type), which returns a
   string name for the given Log_event_type.  This is just like the
   existing char* Log_event::get_type_str(), except it is a class
   method that takes the log event type as parameter.

   I also added PRE_GA_*_ROWS_LOG_EVENT to Log_event::get_type_str(),
   so that names of old rows event are properly printed.

 - When reading an event, I added a check that the event type is known
   by the current Format_description_log_event. Without this, it may
   crash on bad input (and I was struck by this several times).

 - I patched the following test cases, which all contain BINLOG
   statements for row events which must be preceded by BINLOG
   statements for FD events:
    - rpl_bug31076

While I was here, I fixed some small things in log_event.cc:

 - replaced hard-coded 4 by EVENT_TYPE_OFFSET in 3 places

 - replaced return by DBUG_VOID_RETURN in one place

 - The name of the logfile can be '-' to indicate stdin.  Before my
   patch, the code just checked if the first character is '-'; now it
   does a full strcmp().  Probably, all arguments that begin with a -
   are already handled somewhere else as flags, but I still think it
   is better that the code reflects what it is supposed to do, with as
   little dependencies as possible on other parts of the code.  If we
   one day implement that all command line arguments after -- are
   files (as most unix tools do), then we need this.

I also fixed the following in slave.cc:

 - next_event() was declared twice, and queue_event was not static but
   should be static (not used outside the file).


client/client_priv.h:
  Declared the new option for base64 output.
client/mysqlbinlog.cc:
   - Change from using the two-state command line option
    "default/--base64-output" to the three-state
    "--base64-output=[never|auto|always]"
   - Print the FD event even if it is outside the --start-position range.
   - Stop if a row event is about to be printed without a preceding FD
     event.
   - Minor fixes:
      * changed 4 to EVENT_TYPE_OFFSET in some places
      * Added comments
      * before, "mysqlbinlog -xyz" read from stdin; now it does not
        (only "mysqlbinlog -" reads stdin).
mysql-test/r/mysqlbinlog2.result:
  Updated result file: mysqlbinlog now prints ROLLBACK always.
mysql-test/suite/binlog/t/disabled.def:
  The test must be disabled since it reveals another bug: see BUG#33247.
mysql-test/suite/rpl/r/rpl_bug31076.result:
  Updated result file
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
  Updated result file
mysql-test/suite/rpl/t/rpl_bug31076.test:
  Had to add explicit Format_description_log_event before other BINLOG
  statements
mysql-test/t/mysqlbinlog2.test:
  we must suppress base64 output in result file because it contains a
  timestamp
sql/log_event.cc:
   - Made FD events able to print themselves
   - Added check that the current FD event knows about the event type, when
     an event is about to be read. (Hint to reviewers: I had to re-indent
     a big block because of this; use diff -b)
      * To get a decent error message, I also added a class method
        const char* Log_event::get_type_str(Log_event_type)
        which converts number to event type string without having a
        Log_event object.
      * Made Log_event::get_type_str aware of PRE_GA_*_ROWS_LOG_EVENT.
   - Minor fixes:
      * Changed return to DBUG_VOID_RETURN
sql/log_event.h:
   - Declared enum to describe the three base64_output modes
   - Use the enum instead of a flag
   - Declare the new class method get_type_str (see log_event.cc)
sql/share/errmsg.txt:
  Added error msg.
sql/slave.cc:
   - Factored out part of exec_relay_log_event to the new function
     apply_event_and_update_pos, because that code is needed when executing
     BINLOG statements. (this is be functionally equivalent to the
     previous code, except: (1) skipping events is now optional, controlled
     by a parameter to the new function (2) the return value of
     exec_relay_log_event has changed; see next item).
   - Changed returned error value to always be 1. Before, it would return
     the error value from apply_log_event, which was unnecessary. This
     change is safe because the exact return value of exec_relay_log_event
     is never examined; it is only tested to be ==0 or !=0.
   - Added comments describing exec_relay_log_event and
     apply_event_and_update_pos.
   - Minor fixes:
      * Removed duplicate declaration of next_event, made queue_event
        static.
      * Added doxygen code to include this file.
sql/slave.h:
  Declared the new apply_event_and_update_pos
sql/sql_binlog.cc:
   - Made mysql_binlog_statement set the current FD event when the given
     event is an FD event. This entails using the new function
     apply_event_and_update_pos from slave.cc instead of just calling the
     ev->apply method.
   - Made mysql_binlog_statement fail if the first BINLOG statement is not
     an FD event.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
  New test file needs new result file
mysql-test/suite/binlog/t/binlog_base64_flag.test:
  Added test case to verify that:
   - my patch fixes the bug
   - the new --base64-output flag works as expected
   - base64 events not preceded by an FD event give an error
   - an event of a type not known by the current FD event fails cleanly.
mysql-test/suite/binlog/std_data/binlog-bug32407.000001:
  BitKeeper file /home/sven/bk/b32407-5.1-new-rpl-mysqlbinlog_base64/mysql-test/suite/binlog/std_data/binlog-bug32407.000001

Now, every transaction (including autocommit transactions) starts with
a BEGIN and ends with a COMMIT/ROLLBACK in the binlog.
Added a test case, and updated lots of test case result files.


mysql-test/r/multi_update.result:
  Updated result file
mysql-test/r/sp_trans_log.result:
  Updated result file
mysql-test/suite/binlog/r/binlog_innodb.result:
  Updated result file
mysql-test/suite/binlog/r/binlog_multi_engine.result:
  Updated result file
mysql-test/suite/binlog/r/binlog_stm_blackhole.result:
  Updated result file
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
  Updated result file
mysql-test/suite/ndb/r/ndb_binlog_format.result:
  Updated result file
mysql-test/suite/rpl/r/rpl_innodb_mixed_dml.result:
  Updated result file
mysql-test/suite/rpl/r/rpl_row_charset_innodb.result:
  Updated result file
mysql-test/suite/rpl/r/rpl_row_create_table.result:
  Updated result file
mysql-test/suite/rpl/r/rpl_row_log_innodb.result:
  Updated result file
mysql-test/suite/rpl/r/rpl_switch_stm_row_mixed.result:
  Updated result file
mysql-test/suite/rpl/r/rpl_truncate_3innodb.result:
  Updated result file
mysql-test/suite/rpl/t/rpl_row_create_table.test:
  Updated result file
mysql-test/suite/rpl_ndb/r/rpl_ndb_stm_innodb.result:
  Updated result file
sql/log.cc:
   - Always write BEGIN and COMMIT around statements, even in autocommit
     mode.
   - Added comments for binlog_commit and binlog_rollback.
sql/log_event.cc:
  Added debug trigger to avoid writing xid events to the binlog.
mysql-test/suite/rpl_ndb/r/rpl_ndb_transaction.result:
  Results for new test case
mysql-test/suite/rpl_ndb/t/rpl_ndb_transaction-master.opt:
  Options for new test case
mysql-test/suite/rpl_ndb/t/rpl_ndb_transaction-slave.opt:
  Options for new test case
mysql-test/suite/rpl_ndb/t/rpl_ndb_transaction.test:
  Added new test case.

sql/log_event.cc:
  relaxing compilation condition as HA_ERR is started to be used for error
  reporting.

mysql-test/extra/rpl_tests/rpl_row_basic.test:
  changes due to the strict mode gets default. This hunk is being added after merging
  with changes from another bug fixes.
mysql-test/suite/rpl/r/rpl_ignore_table.result:
  results changed
mysql-test/suite/rpl/r/rpl_row_basic_2myisam.result:
  results changed
mysql-test/suite/rpl/r/rpl_row_basic_3innodb.result:
  results changed
mysql-test/suite/rpl/t/rpl_ignore_table.test:
  two queries on mysql db tables are replicated according to binlog_format, ie
  in row-based when that is requested. Due to cancelling the idempotent default
  row based events that previously exectuted successfully now stop the slave sql
  thread.
  We have to explicitly request the idempotent slave execution mode.
mysql-test/suite/rpl_ndb/r/rpl_ndb_dd_advance.result:
  results changed. They are being committed only now since there was another
  bug which fixes my working clone did not have.
mysql-test/suite/rpl_ndb/r/rpl_row_basic_7ndb.result:
  results changed
sql/log_event.cc:
  typo in manual merge earlier

cause ROLLBACK of statement", part 1. Review fixes.

Do not send OK/EOF packets to the client until we reached the end of 
the current statement.
This is a consolidation, to keep the functionality that is shared by all 
SQL statements in one place in the server.
Currently this functionality includes:
- close_thread_tables()
- log_slow_statement().

After this patch and the subsequent patch for Bug#12713, it shall also include:
- ha_autocommit_or_rollback()
- net_end_statement()
- query_cache_end_of_result().

In future it may also include:
- mysql_reset_thd_for_next_command().


include/mysql_com.h:
  Rename now unused members of NET: no_send_ok, no_send_error, report_error.
  These were server-specific variables related to the client/server
  protocol. They have been made obsolete by this patch.
  
  Previously the same members of NET were used to store the error message
  both on the client and on the server. 
  The error message was stored in net.last_error (client: mysql->net.last_error,
  server: thd->net.last_error).
  The error code was stored in net.last_errno (client: mysql->net.last_errno,
  server: thd->net.last_errno).
  The server error code and message are now stored elsewhere 
  (in the Diagnostics_area), thus NET members are no longer used by the
  server.
  Rename last_error to client_last_error, last_errno to client_last_errno
  to avoid potential bugs introduced by merges.
include/mysql_h.ic:
  Update the ABI file to reflect a rename. 
  Renames do not break the binary compatibility.
libmysql/libmysql.c:
  Rename last_error to client_last_error, last_errno to client_last_errno.
  This is necessary to ensure no unnoticed bugs introduced by merged
  changesets.
  
  Remove net.report_error, net.no_send_ok, net.no_send_error.
libmysql/manager.c:
  Rename net.last_errno to net.client_last_errno.
libmysqld/lib_sql.cc:
  Rename net.last_errno to net.client_last_errno.
  
  Update the embedded implementation of the client-server protocol to
  reflect the refactoring of protocol.cc.
libmysqld/libmysqld.c:
  Rename net.last_errno to net.client_last_errno.
mysql-test/r/events.result:
  Update to reflect the change in mysql_rm_db(). Now we drop stored
  routines and events for a given database name only if there
  is a directory for this database name. ha_drop_database() and
  query_cache_invalidate() are called likewise. 
  Previously we would attempt to drop routines/events even if database
  directory was not found (it worked, since routines and events are stored
  in tables). This fixes Bug 29958 "Weird message on DROP DATABASE if mysql.proc
  does not exist".
  The change was done because the previous code used to call send_ok()
  twice, which led to an assertion failure when asserts against it were
  added by this patch.
mysql-test/r/grant.result:
  Fix the patch for Bug 16470, now FLUSH PRIVILEGES produces an error 
  if mysql.procs_priv is missing.
  This fixes the assert that send_ok() must not called after send_error()
  (the original patch for Bug 16470 was prone to this).
mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result:
  Produce a more detailed error message.
mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result:
  Produce a more detailed error message.
mysql-test/t/grant.test:
  Update the test, now FLUSH PRIVILEGES returns an error if mysql.procs_priv
  is missing.
server-tools/instance-manager/mysql_connection.cc:
  Rename net.last_errno to net.client_last_errno.
sql/ha_ndbcluster_binlog.cc:
  Add asserts. 
  
  Use getters to access statement status information.
  
  Add a comment why run_query() is broken. Reset the diagnostics area
  in the end of run_query() to fulfill the invariant that the diagnostics_area
  is never assigned twice per statement (see the comment in the code
  when this can happen). We still do not clear thd->is_fatal_error and
  thd->is_slave_error, which may lead to bugs, I consider the whole affair
  as something to be dealt with separately.
sql/ha_partition.cc:
  fatal_error() doesn't set an error by itself. Perhaps we should
  remove this method altogether and instead add a flag to my_error 
  to set thd->is_fatal_error property.
  
  Meanwhile, this change is a part of inspection made to the entire source
  code with the goal to ensure that fatal_error()
  is always accompanied by my_error().
sql/item_func.cc:
  There is no net.last_error anymore. Remove the obsolete assignment.
sql/log_event.cc:
  Use getters to access statement error status information.
sql/log_event_old.cc:
  Use getters to access statement error status information.
sql/mysqld.cc:
  Previously, if a continue handler for an error was found, my_message_sql() 
  would not set an error in THD. Since the current statement
  must be aborted in any case, find_handler() had a hack to assign 
  thd->net.report_error to 1.
  
  Remove this hack. Set an error in my_message_sql() even if the continue
  handler is found. The error will be cleared anyway when the handler
  is executed. This is one action among many in this patch to ensure the 
  invariant that whenever thd->is_error() is TRUE, we have a message in 
  thd->main_da.message().
sql/net_serv.cc:
  Use a full-blown my_error() in net_serv.cc to report an error,
  instead of just setting net->last_errno. This ensures the invariant that
  whenever thd->is_error() returns TRUE, we have a message in 
  thd->main_da.message().
  
  Remove initialization of removed NET members.
sql/opt_range.cc:
  Use my_error() instead of just raising thd->net.report_error. 
  This ensures the invariant that whenever thd->is_error() returns TRUE, 
  there is a message in thd->main_da.message().
sql/opt_sum.cc:
  Move invocation of fatal_error() right next to the place where
  we set the error message. That makes it easier to track that whenever
  fatal_error() is called, there is a message in THD.
sql/protocol.cc:
  Rename send_ok() and send_eof() to net_send_ok() and net_send_eof() 
  respectively. These functions write directly to the network and are not 
  for use anywhere outside the client/server protocol code. 
  
  Remove the code that was responsible for cases when either there is 
  no error code, or no error message, or both.
  Instead the calling code ensures that they are always present. Asserts
  are added to enforce the invariant.
  
  Instead of a direct access to thd->server_status and thd->total_warn_count
  use function parameters, since these from now on don't always come directly
  from THD.
  
  Introduce net_end_statement(), the single-entry-point replacement API for 
  send_ok(), send_eof() and net_send_error().
  
  Implement Protocol::end_partial_result_set to use in select_send::abort()
  when there is a continue handler.
sql/protocol.h:
  Update declarations.
sql/repl_failsafe.cc:
  Use getters to access statement status information in THD.
  Rename net.last_error to net.client_last_error.
sql/rpl_record.cc:
  Set an error message in prepare_record() if there is no default
  value for the field -- later we do print this message to the client.
sql/rpl_rli.cc:
  Use getters to access statement status information in THD.
sql/slave.cc:
  In create_table_from_dump() (a common function that is used in 
  LOAD MASTER TABLE SQL statement and COM_LOAD_MASTER_DATA), instead of hacks
  with no_send_ok, clear the diagnostics area when mysql_rm_table() succeeded.
  
  Update has_temporary_error() to work correctly when no error is set.
  This is the case when Incident_log_event is executed: it always returns
  an error but does not set an error message.
  
  Use getters to access error status information.
sql/sp_head.cc:
  Instead of hacks with no_send_error, work through the diagnostics area 
  interface to suppress sending of OK/ERROR packets to the client.
  
  Move query_cache_end_of_result before log_slow_statement(), similarly
  to how it's done in dispatch_command().
sql/sp_rcontext.cc:
  Remove hacks with assignment of thd->net.report_error, they are not
  necessary any more (see the changes in mysqld.cc).
sql/sql_acl.cc:
  Use getters to access error status information in THD.
sql/sql_base.cc:
  Access thd->main_da.sql_errno() only if there is an error. This fixes
  a bug when auto-discovery, that was effectively disabled under pre-locking.
sql/sql_binlog.cc:
  Remove hacks with no_send_ok/no_send_error, they are not necessary 
  anymore: the caller is responsible for network communication.
sql/sql_cache.cc:
  Disable sending of OK/ERROR/EOF packet in the end of dispatch_command
  if the response has been served from the query cache. This raises the 
  question whether we should store EOF packet in the query cache at all,
  or generate it anew for each statement (we should generate it anew), but
  this is to be addressed separately.
sql/sql_class.cc:
  Implement class Diagnostics_area. Please see comments in sql_class.h
  for details.
  
  Fix a subtle coding mistake in select_send::send_data: when on slave, 
  an error in Item::send() was ignored.
  The problem became visible due to asserts that the diagnostics area is
  never double assigned.
  
  Remove initialization of removed NET members.
  
  In select_send::abort() do not call select_send::send_eof(). This is
  not inheritance-safe. Even if a stored procedure continue handler is
  found, the current statement is aborted, not succeeded.
  Instead introduce a Protocol API to send the required response, 
  Protocol::end_partial_result_set().
  
  This simplifies implementation of select_send::send_eof(). No need
  to add more asserts that there is no error, there is an assert inside
  Diagnostics_area::set_ok_status() already.
  
  Leave no trace of no_send_* in the code.
sql/sql_class.h:
  Declare class Diagnostics_area. 
  
  Remove the hack with no_send_ok from
  Substatement_state.
  
  Provide inline implementations of send_ok/send_eof.
  
  Add commetns.
sql/sql_connect.cc:
  Remove hacks with no_send_error. 
  
  Since now an error in THD is always set if net->error, it's not necessary
  to check both net->error and thd->is_error() in the do_command loop.
  
  Use thd->main_da.message() instead of net->last_errno.
  
  Remove the hack with is_slave_error in sys_init_connect. Since now we do not
  reset the diagnostics area in net_send_error (it's reset at the beginning
  of the next statement), we can access it safely even after 
  execute_init_command.
sql/sql_db.cc:
  Update the code to satisfy the invariant that the diagnostics area is never
  assigned twice.
  Incidentally, this fixes Bug 29958 "Weird message on DROP DATABASE if 
  mysql.proc does not exist".
sql/sql_delete.cc:
  Change multi-delete to abort in abort(), as per select_send protocol.
  Fixes the merge error with the test for Bug 29136
sql/sql_derived.cc:
  Use getters to access error information.
sql/sql_insert.cc:
  Use getters to access error information.
sql-common/client.c:
  Rename last_error to client_last_error, last_errno to client_last_errno.
sql/sql_parse.cc:
  Remove hacks with no_send_error. Deploy net_end_statement().
  
  The story of COM_SHUTDOWN is interesting. Long story short, the server 
  would become on its death's door, and only no_send_ok/no_send_error assigned
  by send_ok()/net_send_error() would hide its babbling from the client.
  
  First of all, COM_QUIT does not require a response. So, the comment saying
  "Let's send a response to possible COM_QUIT" is not only groundless 
  (even mysqladmin shutdown/mysql_shutdown() doesn't send COM_QUIT after 
  COM_SHUTDOWN), it's plainly incorrect.
  
  Secondly, besides this additional 'OK' packet to respond to a hypothetical
  COM_QUIT, there was the following code in dispatch_command():
  
  if (thd->killed)
    thd->send_kill_message();
  if (thd->is_error()
    net_send_error(thd);
  
  This worked out really funny for the thread through which COM_SHUTDOWN
  was delivered: we would get COM_SHUTDOWN, say okay, say okay again, 
  kill everybody, get the kill signal ourselves, and then attempt to say 
  "Server shutdown in progress" to the client that is very likely long gone.
  
  This all became visible when asserts were added that the Diagnostics_area
  is not assigned twice.
  
  Move query_cache_end_of_result() to the end of dispatch_command(), since
  net_send_eof() has been moved there. This is safe, query_cache_end_of_result()
  is a no-op if there is no started query in the cache.
  
  Consistently use select_send interface to call abort() or send_eof()
  depending on the operation result.
  
  Remove thd->fatal_error() from reset_master(), it was a no-op. 
  in hacks with no_send_error woudl save us
  from complete breakage of the client/server protocol.
  
  Consistently use select_send::abort() whenever there is an error, 
  and select_send::send_eof() in case of success.
  The issue became visible due to added asserts.
sql/sql_partition.cc:
  Always set an error in THD whenever there is a call to fatal_error().
sql/sql_prepare.cc:
  Deploy class Diagnostics_area.
  Remove the unnecessary juggling with the protocol in 
  Select_fetch_protocol_binary::send_eof(). EOF packet format is 
  protocol-independent.
sql/sql_select.cc:
  Call fatal_error() directly in opt_sum_query.
  Call my_error() whenever we call thd->fatal_error().
sql/sql_servers.cc:
  Use getters to access error information in THD.
sql/sql_show.cc:
  Use getters to access error information in THD.
  
  Add comments.
  
  Call my_error() whenever we call fatal_error().
sql/sql_table.cc:
  Replace hacks with no_send_ok with the interface of the diagnostics area.
  
  Clear the error if ENOENT error in ha_delete_table().
sql/sql_update.cc:
  Introduce multi_update::abort(), which is the proper way to abort a
  multi-update. This fixes the merge conflict between this patch and
  the patch for Bug 29136.
sql/table.cc:
  Use a getter to access error information in THD.
sql/tztime.cc:
  Use a getter to access error information in THD.

    without PK
Bug#31609 Not all RBR slave errors reported as errors
bug#32468 delete rows event on a table with foreign key constraint fails

The first two bugs comprise idempotency issues.
First, there was no error code reported under conditions of the bug
description although the slave sql thread halted.
Second, executions were different with and without presence of prim key in
the table.
Third, there was no way to instruct the slave whether to ignore an error
and skip to the following event or to halt.
Fourth, there are handler errors which might happen due to idempotent
applying of binlog but those were not listed among the "idempotent" error
list.

All the named issues are addressed.
Wrt to the 3rd, there is the new global system variable, changeble at run
time, which controls the slave sql thread behaviour.
The new variable allows further extensions to mimic the sql_mode
session/global variable.
To address the 4th, the new bug#32468 had to be fixed as it was staying
in the way.


include/my_bitmap.h:
  basic operations with bits of an integer type are added.
mysql-test/extra/rpl_tests/rpl_foreign_key.test:
  regression test for bug#32468
mysql-test/extra/rpl_tests/rpl_row_basic.test:
  changes due to bug#31552/31609 idempotency is not default any longer
mysql-test/extra/rpl_tests/rpl_row_tabledefs.test:
  changes due to bug#31552/31609 idempotency is not default any longer
mysql-test/suite/rpl/r/rpl_foreign_key_innodb.result:
  results changed
mysql-test/suite/rpl/r/rpl_idempotency.result:
  results changed
mysql-test/suite/rpl/r/rpl_row_basic_11bugs.result:
  results changed
mysql-test/suite/rpl/r/rpl_row_basic_2myisam.result:
  results changed
mysql-test/suite/rpl/r/rpl_row_basic_3innodb.result:
  results changed
mysql-test/suite/rpl/r/rpl_row_mystery22.result:
  results changed
mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result:
  results changed
mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result:
  results changed
mysql-test/suite/rpl/r/rpl_temporary_errors.result:
  results changed
mysql-test/suite/rpl/t/rpl_idempotency.test:
  extenstions to the test providing testing of complements to the
  idempotent error set and checking how slave halts when it faces an error
  from the list when the mode is STRICT.
mysql-test/suite/rpl/t/rpl_row_basic_11bugs.test:
  changes due to bug#31552/31609 idempotency is not default any longer.
mysql-test/suite/rpl/t/rpl_row_mystery22.test:
  changes due to bug#31552/31609 idempotency is not default any longer
mysql-test/suite/rpl/t/rpl_temporary_errors.test:
  changes due to bug#31552/31609 idempotency is not default any longer
mysql-test/suite/rpl_ndb/r/rpl_ndb_dd_advance.result:
  results changed
mysql-test/suite/rpl_ndb/r/rpl_row_basic_7ndb.result:
  results changed
sql/log_event.cc:
  the fix for bug#32468 delete rows event on a table with foreign key constraint fails
  ensures the flags are set at proper time so that their values will be caught
  by innodb.
  reseting the flags is done along the common error and errorless execution
  path.
  The list of idempotent error is extended with foreign keys related items.
  NDB engine write events are designed with the replace sematics in mind.
  Therefore the corrsponding ndb handler's flag are (re)set regardless of
  the slave's execution mode.
  Rows_log_event::write_row() starts using the bool replace argument as its
  caller sets it depending on the event's execution mode.
sql/log_event.h:
  adding a new member to hold the slave's mode during execution of the event.
sql/mysql_priv.h:
  changes to link the command line option with the new global sys var.
sql/mysqld.cc:
  introduction of the new command line option.
  providing its initialization to a default.
  changes to link the command line option with the new global sys var.
sql/rpl_rli.cc:
  rli post-event-execution cleanup restores the default bits.
sql/set_var.cc:
  The new "standard" sys_var_set class' and the new global system var related
  declarations and definitions.
  fix_slave_exec_mode() is used as with the update method of a new class so
  as at time of the command line arguments parsing.
sql/set_var.h:
  new declarations. The class for the new global sys var is based on
  yet another new "standard" one.
sql/share/errmsg.txt:
  slave_exec_mode setting error;
  slave inconsistency error which may be not an error when the intention
  is "idempotent". I.e consisting of row-based events binlog is being
  applied for the 2nd (more) time.
sql/sql_class.h:
  The names for the bits of the new sever slave_exec_mode_options.
mysql-test/suite/rpl/t/rpl_idempotency-master.opt:
  innodb is necessary
mysql-test/suite/rpl/t/rpl_idempotency-slave.opt:
  innodb is necessary, as well as the tests start with non-default
  IDEMPOTENT slave execution mode.

In the patch for BUG#21842, the code for handling old rows events were
refactored.  There were a bug in the refactored code (possibly introduced
after the patch for BUG#21842) that caused caused the refactored old events
to read a columns bitmap after image even though there is no such bitmap
for old events. As a result, the reading got out of sync, and started reading
invalid data.

This patch removes all trace of the after image column bitmap from the refactored
old events and removes functions that are no longer needed because they are empty. 


sql/log_event.cc:
  Adding debug printouts and adding old rows events names to output
  so that they are not printed as unknown events.
sql/log_event_old.cc:
  Adding debug printouts to see how the old events are partitioned
  when being decoded.  Removing all traces of the column bitmap for
  the after image in the old events since there is none.
  
  Removing the following functions since they are no longer needed:
  - Update_rows_log_event_old::init()
  - Update_rows_log_event_old::~Update_rows_log_event_old()
  
  Removing unused local variable.
sql/log_event_old.h:
  Removing all traces of the column bitmap for the after image in
  the old events since there is none.
  
  Removing the following functions since they are no longer needed:
  - Update_rows_log_event_old::init()
  - Update_rows_log_event_old::is_valid()
  - Update_rows_log_event_old::~Update_rows_log_event_old()
  
  Removing unused local variable.
mysql-test/suite/bugs/r/rpl_bug31583.result:
  New BitKeeper file ``mysql-test/suite/bugs/r/rpl_bug31583.result''
mysql-test/suite/bugs/t/rpl_bug31583.test:
  New BitKeeper file ``mysql-test/suite/bugs/t/rpl_bug31583.test''

Problem: some pieces of code relied on the default character
set settings, which didn't work in case of default character set
ucs2.
Fix: Specifying character set explicitly, not to depend on
the default settings.


mysql-test/suite/rpl_ndb/r/rpl_ndb_ctype_ucs2_def.result:
  Recording correct test result
mysql-test/suite/rpl_ndb/t/disabled.def:
  Enabling test
sql/ha_ndbcluster_binlog.cc:
  Character set of the NDB helper tables should
  not rely of the default settings, to avoid creating
  tables in different character sets on master and slave.
  Adding explicit character set. UTF8 should be fine.
  
  character_set_client should not rely on the default
  character set settings, which can be ucs2. Helper SQL queries
  sent by NDB are all in pure ASCII. Setting client_character_set
  to latin1.
sql/log_event.cc:
  Adding assert to make sure that binary log does not have 
  queries in character set which parser does not understand
  (e.g. ucs2).

The client program 'mysqlbinlog' crashed when trying to print a User_var_log_event holding
a floating-point value since the format specifier for my_b_printf() does not support
floating-point format specifiers.

This patch prints the floating-point number to an internal buffer, and then writes
that buffer to the output instead.


mysql-test/r/mysqlbinlog.result:
  Result file change.
mysql-test/t/mysqlbinlog.test:
  Adding test that mysqlbinlog can write and read back User_var_log_event
  for real, decimal, integer, and string. These are the only types supported
  for user variables.
sql/log_event.cc:
  Using my_sprintf() to print floating-point value of User_var_log_event value to a
  character buffer and then to the real output, since my_b_printf() does not
  support floating-point format. Also adding macro to give buffer size needed 
  for printing floating-point numbers in %g format.

Corrections to get_str_len_and_pointer().


sql/log_event.cc:
  Adding missing return at end of get_str_len_and_pointer()
  and correcting computation of missing bytes.

When running mysqlbinlog on a 64-bit machine with a corrupt relay log,
it causes mysqlbinlog to crash. In this case, the crash is caused
because a request for 18446744073709534806U bytes is issued, which
apparantly can be served on a 64-bit machine (speculatively, I assume)
but this causes the memcpy() issued later to copy the data to segfault.

The request for the number of bytes is caused by a computation
of data_len - server_vars_len where server_vars_len is corrupt in such
a sense that it is > data_len. This causes a wrap-around, with the
the data_len given above.

This patch adds a check that if server_vars_len is greater than
data_len before the substraction, and aborts reading the event in
that case marking the event as invalid. It also adds checks to see
that reading the server variables does not go outside the bounds
of the available space, giving a limited amount of integrity check.


mysql-test/r/mysqlbinlog.result:
  Result change.
mysql-test/t/mysqlbinlog.test:
  Adding test that it fails gracefully for a corrupt relay log.
sql/log_event.cc:
  Adding check that status var length does not cause wrap-around
  when performing subtraction. Extending get_str_len_and_pointer() to
  check that the string can actually be read without reading outside
  bounds. Adding checks when reading server variables from the Query-
  log_event so that the variable can really be read. Abort reading
  and mark the event as invalid otherwise.
mysql-test/std_data/corrupt-relay-bin.000624:
  BitKeeper file /home/mats/devel/b31793-mysql-5.0-rpl/mysql-test/std_data/corrupt-relay-bin.000624