without PK
Bug#31609 Not all RBR slave errors reported as errors
bug#32468 delete rows event on a table with foreign key constraint fails

The first two bugs comprise idempotency issues.
First, there was no error code reported under conditions of the bug
description although the slave sql thread halted.
Second, executions were different with and without presence of prim key in
the table.
Third, there was no way to instruct the slave whether to ignore an error
and skip to the following event or to halt.
Fourth, there are handler errors which might happen due to idempotent
applying of binlog but those were not listed among the "idempotent" error
list.

All the named issues are addressed.
Wrt to the 3rd, there is the new global system variable, changeble at run
time, which controls the slave sql thread behaviour.
The new variable allows further extensions to mimic the sql_mode
session/global variable.
To address the 4th, the new bug#32468 had to be fixed as it was staying
in the way.


include/my_bitmap.h:
  basic operations with bits of an integer type are added.
mysql-test/extra/rpl_tests/rpl_foreign_key.test:
  regression test for bug#32468
mysql-test/extra/rpl_tests/rpl_row_basic.test:
  changes due to bug#31552/31609 idempotency is not default any longer
mysql-test/extra/rpl_tests/rpl_row_tabledefs.test:
  changes due to bug#31552/31609 idempotency is not default any longer
mysql-test/suite/rpl/r/rpl_foreign_key_innodb.result:
  results changed
mysql-test/suite/rpl/r/rpl_idempotency.result:
  results changed
mysql-test/suite/rpl/r/rpl_row_basic_11bugs.result:
  results changed
mysql-test/suite/rpl/r/rpl_row_basic_2myisam.result:
  results changed
mysql-test/suite/rpl/r/rpl_row_basic_3innodb.result:
  results changed
mysql-test/suite/rpl/r/rpl_row_mystery22.result:
  results changed
mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result:
  results changed
mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result:
  results changed
mysql-test/suite/rpl/r/rpl_temporary_errors.result:
  results changed
mysql-test/suite/rpl/t/rpl_idempotency.test:
  extenstions to the test providing testing of complements to the
  idempotent error set and checking how slave halts when it faces an error
  from the list when the mode is STRICT.
mysql-test/suite/rpl/t/rpl_row_basic_11bugs.test:
  changes due to bug#31552/31609 idempotency is not default any longer.
mysql-test/suite/rpl/t/rpl_row_mystery22.test:
  changes due to bug#31552/31609 idempotency is not default any longer
mysql-test/suite/rpl/t/rpl_temporary_errors.test:
  changes due to bug#31552/31609 idempotency is not default any longer
mysql-test/suite/rpl_ndb/r/rpl_ndb_dd_advance.result:
  results changed
mysql-test/suite/rpl_ndb/r/rpl_row_basic_7ndb.result:
  results changed
sql/log_event.cc:
  the fix for bug#32468 delete rows event on a table with foreign key constraint fails
  ensures the flags are set at proper time so that their values will be caught
  by innodb.
  reseting the flags is done along the common error and errorless execution
  path.
  The list of idempotent error is extended with foreign keys related items.
  NDB engine write events are designed with the replace sematics in mind.
  Therefore the corrsponding ndb handler's flag are (re)set regardless of
  the slave's execution mode.
  Rows_log_event::write_row() starts using the bool replace argument as its
  caller sets it depending on the event's execution mode.
sql/log_event.h:
  adding a new member to hold the slave's mode during execution of the event.
sql/mysql_priv.h:
  changes to link the command line option with the new global sys var.
sql/mysqld.cc:
  introduction of the new command line option.
  providing its initialization to a default.
  changes to link the command line option with the new global sys var.
sql/rpl_rli.cc:
  rli post-event-execution cleanup restores the default bits.
sql/set_var.cc:
  The new "standard" sys_var_set class' and the new global system var related
  declarations and definitions.
  fix_slave_exec_mode() is used as with the update method of a new class so
  as at time of the command line arguments parsing.
sql/set_var.h:
  new declarations. The class for the new global sys var is based on
  yet another new "standard" one.
sql/share/errmsg.txt:
  slave_exec_mode setting error;
  slave inconsistency error which may be not an error when the intention
  is "idempotent". I.e consisting of row-based events binlog is being
  applied for the 2nd (more) time.
sql/sql_class.h:
  The names for the bits of the new sever slave_exec_mode_options.
mysql-test/suite/rpl/t/rpl_idempotency-master.opt:
  innodb is necessary
mysql-test/suite/rpl/t/rpl_idempotency-slave.opt:
  innodb is necessary, as well as the tests start with non-default
  IDEMPOTENT slave execution mode.

into  kindahl-laptop.dnsalias.net:/home/bkroot/mysql-5.1-rpl

into  koti.dsl.inet.fi:/home/elkin/MySQL/merge-5.1


mysql-test/suite/binlog/r/binlog_killed.result:
  Auto merged
mysql-test/suite/binlog/t/binlog_killed.test:
  Auto merged
mysql-test/suite/rpl/t/rpl_dual_pos_advance.test:
  Auto merged
mysql-test/suite/rpl/t/rpl_temporary.test:
  manual merge
sql/log.cc:
  manual merge

into  koti.dsl.inet.fi:/home/elkin/MySQL/TEAM/FIXES/5.0/bug27571_asyn_killed_flags

refining non-deterministic tests.
The new Bug@32148 is in the way. Adjuting the tests to be somehow useful.
 


mysql-test/r/binlog_killed.result:
  results changed
mysql-test/t/binlog_killed.test:
  refining the tests as killing is inherently non-deterministic;
  leaving todos.

tests fixing: refreshing the results file.


mysql-test/suite/binlog/r/binlog_killed.result:
  results changed

into  koti.dsl.inet.fi:/home/elkin/MySQL/TEAM/FIXES/5.0/bug28597-log_name_upgrade

BitKeeper/deleted/.del-binlog_killed_simulate.result:
  Delete: mysql-test/r/binlog_killed_simulate.result

into  koti.dsl.inet.fi:/home/elkin/MySQL/merge-5.1


BitKeeper/deleted/.del-binlog_killed_bug27571-master.opt:
  Auto merged
BitKeeper/deleted/.del-binlog_killed_bug27571.test:
  Auto merged

fixing tests


mysql-test/r/binlog_killed_simulate.result:
  the new tests' result

removing extra tests (on 5.1 that's been already done)


BitKeeper/deleted/.del-binlog_killed_bug27571-master.opt:
  Delete: mysql-test/t/binlog_killed_bug27571-master.opt
BitKeeper/deleted/.del-binlog_killed_bug27571.test:
  Delete: mysql-test/t/binlog_killed_bug27571.test

into  koti.dsl.inet.fi:/home/elkin/MySQL/merge-5.1


mysql-test/suite/binlog/r/binlog_killed.result:
  manual merge ul
mysql-test/suite/binlog/t/binlog_killed.test:
  manual merge with 5.0 specific tests; use local
mysql-test/suite/binlog/t/binlog_killed_simulate.test:
   manual merge with 5.0 specific tests; use local
mysql-test/t/ctype_ucs.test:
  Auto merged
mysql-test/t/func_regexp.test:
  Auto merged
mysql-test/r/ctype_ucs.result:
  automerge
mysql-test/r/ctype_utf8.result:
  automerge
mysql-test/r/func_regexp.result:
  automerge
mysql-test/t/ctype_utf8.test:
  automerge
sql/item_cmpfunc.cc:
  automerge
sql/item_cmpfunc.h:
  automerge
sql/sql_class.h:
  automerge
sql/sql_insert.cc:
  automerge
sql/sql_parse.cc:
  automerge
sql/sql_update.cc:
  automerge

commit is specific for 5.0 to eliminated non-deterministic tests.
Those tests run only in 5.1 env where there is a necessary devices such
as processlist table of info_schema.


mysql-test/r/binlog_killed.result:
  results changed
mysql-test/t/binlog_killed.test:
  removing non-deterministic part of the test
mysql-test/t/binlog_killed_simulate.test:
  adding the guard same as for 5.1 version

into  koti.dsl.inet.fi:/home/elkin/MySQL/TEAM/FIXES/5.0/bug27571_asyn_killed_flags


sql/sql_insert.cc:
  Auto merged
sql/sql_update.cc:
  Auto merged

non-deterministic tests refining.
This particular patch tested on two archs.


mysql-test/suite/binlog/t/binlog_killed.test:
  deploying synchronization of killing with reaching the explected state
  of the prey's connection.

non-deterministic tests execution on some platforms.


mysql-test/suite/binlog/r/binlog_killed.result:
  results changed
mysql-test/suite/binlog/t/binlog_killed.test:
  correcting the prev test's cleanup;
  increasing time to wait by to be killed query;
  adding the masking out error due to Bug@32140 wrong error code caught when an SF() call is interruped with KILL query

into  kindahl-laptop.dnsalias.net:/home/bkroot/mysql-5.1-rpl

refining tests as they appear to be non-deterministic.


mysql-test/suite/binlog/r/binlog_killed.result:
  results changed
mysql-test/suite/binlog/t/binlog_killed.test:
  restructuring the test to deliver more deterministic outcome.
  order-by for update and delete did not guaratee the order of scanning.
  --error are masked with zero in order to catch more info if killing
  really will happen to be non-deterministic on some platform (witnessed
  something like that on dl145j)
mysql-test/suite/binlog/t/binlog_killed_simulate.test:
  important guard added

into  koti.dsl.inet.fi:/home/elkin/MySQL/TEAM/FIXES/5.0/bug28597-log_name_upgrade

Since bug@20166, which replaced the binlog file name generating to base
on pidfile_name instead of the previous glob_hostname, the binlog file
name suddenly started to be stored solely in the absolute path format,
including a case when --log-bin option meant a relative path.
What's more serious, the path for binlog file can lead unrequestedly 
to pid-file directory so that after any proper fix for this bug
there might be similar to the bug report consequences for one who
upgrades from post-fix-bug@20166-pre-fix-bug@28597 to post-fix-bug@28597.

Fixed with preserving`pidfile_name' (intr.by bug@20166) but stripping
off its directory part. This restores the original logics of storing
the names in compatible with --log-bin option format and with the
requirement for --log-bin ralative path to corresond to the data directory.
Side effects for this fix:

effective fixing bug@27070, refining its test;
ensuring no overrun for buff can happen anymore (Bug#31836 
insufficient space reserved for the suffix of relay log file name);
bug#31837  --remove_file $MYSQLTEST_VARDIR/tmp/bug14157.sql missed
in rpl_temporary.test;
fixes Bug@28603  Invalid log-bin default location;


mysql-test/t/rpl_dual_pos_advance.test:
  After correcting the logics of log file name composing workaround for
  
    Bug #27070 server logs are created unrequested and in wrong directory
  
  is removed.
mysql-test/t/rpl_temporary.test:
  remaining temp file of the test removed
sql/log.cc:
  stripping off the directory part of `pidfile_name' for binlog name
  generating (which fixes two more bugs on wrong binlog file location);
  ensuring no overrun for buff can happen anymore (Bug #31836 
  insufficient space reserved for the suffix of relay log file name);

into  mysql.com:/home/ksm/commits/mysql-5.1-new-rpl

mysql-test/lib/mtr_cases.pl:
  updated mtr_cases.pl
mysql-test/mysql-test-run.pl:
  updated mtr

into  koti.dsl.inet.fi:/home/elkin/MySQL/TEAM/FIXES/5.1/bug31554-rpl_truncate

Actually, the failure happened with 3innodb as well. Most probably
the reason is in failing to delete a binlog file on __NT__ so that
that master increments the index of the binlog file.
The test results hide valueable warning that windows could generate
about that.

The scope of this fix is to make sure we have such warning and
to lessen chances for binlog file being held at time of closing.
The dump thread is getting a good chance to leave and 
release the file for its successful deletion.

We shall watch over the two tests as regression is not excluded.
In that case we would have an extra info possibly explaining why
__NT__ env can not close/delete the file.
However, regardless of that reason, there is alwasy workaround to mask out
non-deterministic binlog index number.



mysql-test/extra/rpl_tests/rpl_truncate_helper.test:
  enable warnings for other than DROP queries;
  wait for slave's full stop which ensures io thread left and that
  will be regarded by dump thread to leave and close the binlog file;
  relocating reset master to the end of the test so that the caller of
  this helper should start with the binlog name which has not yet been
  affected/reset since its creation.
mysql-test/suite/rpl/r/rpl_truncate_2myisam.result:
  results changed
mysql-test/suite/rpl/r/rpl_truncate_3innodb.result:
  results changed

into  pilot.mysql.com:/data/msvensson/mysql/mysql-5.1-rpl

  Corrected issues found in preparing to push code


mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test:
  Corrected issues found in preparing to push code

into  mysql.com:/usr/clones/rpl-push

into  kindahl-laptop.dnsalias.net:/home/bkroot/mysql-5.1-rpl

destination. Removing wrong (local temp) tests.


mysql-test/suite/binlog/t/binlog_killed_simulate-master.opt:
  Rename: mysql-test/t/binlog_killed_simulate-master.opt -> mysql-test/suite/binlog/t/binlog_killed_simulate-master.opt
BitKeeper/deleted/.del-binlog_killed_bug27571.test:
  Delete: mysql-test/t/binlog_killed_bug27571.test
BitKeeper/deleted/.del-binlog_killed_bug27571-master.opt:
  Delete: mysql-test/t/binlog_killed_bug27571-master.opt
mysql-test/suite/binlog/r/binlog_killed.result:
  results changed
mysql-test/suite/binlog/t/binlog_killed.test:
  binlog offset made 5.1 specific
mysql-test/suite/binlog/t/binlog_killed_simulate.test:
  binlog offset + guard to run the test with stmt/mixed format
mysql-test/suite/binlog/r/binlog_killed_simulate.result:
  new results file

mysql-test/suite/rpl/r/rpl_innodb_bug28430.result:
  Result file change.
mysql-test/suite/rpl/t/rpl_innodb_bug28430.test:
  Include file show_slave_status.inc should be used for portable SHOW SLAVE STATUS.

into  kindahl-laptop.dnsalias.net:/home/bk/b19958-mysql-5.1-rpl


sql/log_event.cc:
  Auto merged

The rpl_trigger test case indicated a problem with idempotency support when run
under row-based replication, which this patch fixes.

However, despite this, the test is not designed for execution under row-based
replication and hence rpl_trigger.test is not executed under row-based
replication.

The problem is that the test expects triggers to be executed when the slave
updates rows on the slave, and this is (deliberately) not done with row-based
replication.


sql/log_event.cc:
  Adding function to print symbolic name of handler errors for debug purposes.
  Ignoring some more error messages to provide full idempotency support for
  update and delete operations.
mysql-test/suite/rpl/r/rpl_idempotency.result:
  New BitKeeper file ``mysql-test/suite/rpl/r/rpl_idempotency.result''
mysql-test/suite/rpl/t/rpl_idempotency.test:
  New BitKeeper file ``mysql-test/suite/rpl/t/rpl_idempotency.test''

into  mysql.com:/usr/clones/rpl-push


mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test:
  Auto merged
mysql-test/suite/rpl/r/rpl_extraCol_innodb.result:
  Auto merged
mysql-test/suite/rpl/r/rpl_extraCol_myisam.result:
  Auto merged
mysql-test/suite/rpl_ndb/r/rpl_ndb_extraCol.result:
  Auto merged

BitKeeper/deleted/.del-rpl_slave_skip-slave.opt:
  Delete: mysql-test/t/rpl_slave_skip-slave.opt
BitKeeper/deleted/.del-rpl_slave_skip.test:
  Delete: mysql-test/t/rpl_slave_skip.test
BitKeeper/deleted/.del-rpl_slave_skip.result:
  Delete: mysql-test/r/rpl_slave_skip.result

involved bug#12691, bug#27571


mysql-test/suite/rpl/r/rpl_sp_effects.result:
  results changed
sql/slave.cc:
  pended manual merge done (mats)
sql/sql_delete.cc:
  manual merge
sql/sql_insert.cc:
  manual merge
sql/sql_load.cc:
  manual merge
sql/sql_update.cc:
  manual merge.

into  koti.dsl.inet.fi:/home/elkin/MySQL/5.1-merge-bug27571


client/mysql.cc:
  Auto merged
mysql-test/r/ctype_euckr.result:
  Auto merged
mysql-test/r/ctype_uca.result:
  Auto merged
mysql-test/suite/binlog/r/binlog_killed.result:
  Auto merged
mysql-test/suite/binlog/t/binlog_killed.test:
  Auto merged
sql/item_cmpfunc.cc:
  Auto merged
sql/log_event.cc:
  Auto merged
sql/log_event.h:
  Auto merged
sql/mysqld.cc:
  Auto merged
sql/sp_head.cc:
  Auto merged
sql/sql_parse.cc:
  Auto merged
strings/ctype-euc_kr.c:
  Auto merged
mysql-test/suite/rpl/r/rpl_sp_effects.result:
  manual merge ul
mysql-test/suite/rpl/t/rpl_sp_effects.test:
  manual merge
sql/slave.cc:
  leaving for manual merge
sql/sql_delete.cc:
  leaving for manual merge
sql/sql_insert.cc:
  leaving for manual merge
sql/sql_load.cc:
  leaving for manual merge
sql/sql_update.cc:
  leaving for manual merge

into  koti.dsl.inet.fi:/home/elkin/MySQL/TEAM/FIXES/5.0/bug27571_asyn_killed_flags

into  mysql.com:/home/bar/mysql-work/mysql-5.0-rpl-merge


mysql-test/r/ctype_utf8.result:
  Auto merged
mysql-test/r/func_regexp.result:
  Auto merged
mysql-test/t/ctype_utf8.test:
  Auto merged
mysql-test/t/func_regexp.test:
  Auto merged
sql/item_cmpfunc.cc:
  Auto merged
sql/item_cmpfunc.h:
  Auto merged
sql/sql_class.h:
  Auto merged
sql/sql_insert.cc:
  Auto merged
sql/sql_parse.cc:
  Auto merged
sql/sql_update.cc:
  Auto merged
mysql-test/r/ctype_ucs.result:
  After merge fix
mysql-test/t/ctype_ucs.test:
  After merge fix

into  mysql.com:/home/bar/mysql-work/mysql-5.1-new-rpl-merge


client/mysql.cc:
  Auto merged
mysql-test/r/ctype_euckr.result:
  Auto merged
mysql-test/r/ctype_uca.result:
  Auto merged
mysql-test/r/ctype_utf8.result:
  Auto merged
mysql-test/r/func_regexp.result:
  Auto merged
mysql-test/suite/rpl/r/rpl_bug31076.result:
  Auto merged
mysql-test/t/ctype_utf8.test:
  Auto merged
mysql-test/t/func_regexp.test:
  Auto merged
mysql-test/t/partition.test:
  Auto merged
sql/field.cc:
  Auto merged
sql/field.h:
  Auto merged
sql/handler.cc:
  Auto merged
sql/item_cmpfunc.cc:
  Auto merged
sql/item_cmpfunc.h:
  Auto merged
sql/item_xmlfunc.cc:
  Auto merged
sql/log.cc:
  Auto merged
sql/log.h:
  Auto merged
sql/log_event.cc:
  Auto merged
sql/mysqld.cc:
  Auto merged
sql/sp_head.cc:
  Auto merged
sql/sql_class.cc:
  Auto merged
sql/sql_class.h:
  Auto merged
sql/sql_delete.cc:
  Auto merged
sql/sql_insert.cc:
  Auto merged
sql/sql_parse.cc:
  Auto merged
sql/sql_repl.cc:
  Auto merged
sql/sql_select.cc:
  Auto merged
sql/sql_show.cc:
  Auto merged
sql/sql_table.cc:
  Auto merged
sql/sql_update.cc:
  Auto merged
strings/ctype-euc_kr.c:
  Auto merged
sql/sql_yacc.yy:
  Reverting Rafal's changes: TRANSACTIONAL_SYM was removed in a mistake.
mysql-test/r/ctype_ucs.result:
  After merge fix
mysql-test/suite/rpl/t/rpl_bug31076.test:
  After merge fix.
mysql-test/t/ctype_ucs.test:
  After megre fix

           Query_log_event::error_code

A query can perform completely having the local var error of mysql_$query
zero, where $query in insert, update, delete, load,
and be  binlogged with error_code e.g KILLED_QUERY while there is no
reason do to so.
That can happen because Query_log_event consults thd->killed flag to
evaluate error_code.

Fixed with implementing a scheme suggested and partly implemented at
time of bug@22725 work-on. error_status is cached immediatly after the
control leaves the main rows-loop and that instance always corresponds
to `error' the local of mysql_$query functions. The cached value
is passed to Query_log_event constructor, not the default thd->killed
which can be changed in between of the caching and the constructing.


mysql-test/r/binlog_killed.result:
  results changed
mysql-test/t/binlog_killed.test:
  Demonstrating that effective killing during rows-loop execution leads to the speficied actions:
  binlogging with the error for a query modified a not-transactional table or
  rolling back effects for transactional table;
  
  fixing possible non-determinism with ID when query_log_enabled;
  
  leave commented out tests for multi-update,delete due to another bug;
  
  removing an obsolete tests template;
  
  changing system rm to --remove_file.
sql/log_event.cc:
  adding killed status arg
sql/log_event.h:
  added killed status arg
sql/sql_delete.cc:
  deploying the update part patch for delete, multi-delete
sql/sql_insert.cc:
  deploying the update-part patch for insert..select
sql/sql_load.cc:
  deploying the update-part patch for load data.
  simulation added.
sql/sql_update.cc:
  Impementing the fix as described in the comments left by bug@22725.
  Also simulation of killing after the loop that would affect binlogging in the old code.
mysql-test/t/binlog_killed_bug27571-master.opt:
  post rows-loop killing simulation's options
mysql-test/t/binlog_killed_bug27571.test:
  Checking that if killing happens inbetween of the end of rows loop and
  recording into binlog that will not lead to recording any error incl
  the killed error.
mysql-test/t/binlog_killed_simulate-master.opt:
  simulation options
mysql-test/t/binlog_killed_simulate.test:
  tests for 
  a query (update is choosen) being killed after the row-loop;
  load data killed within the loop - effective killed error in the event is gained.