GitLab

The cause of the bug was an incomplete fix for bug 18080.
The problem was that setup_tables() unconditionally reset the
name resolution context to its 'tables' argument, which pointed
to the first table of an SQL statement.

The bug fix limits resetting of the name resolution context in
setup_tables() only in the cases when the context was not set
by earlier parser/optimizer phases.


mysql-test/r/insert_select.result:
  Test for BUG#21774.
mysql-test/t/insert_select.test:
  Test for BUG#21774.
sql/sql_base.cc:
  Do not reset the name resolution contect unconditionally.
  Instead set the context to 'tables' only if it was not
  set before calling setup_tables().
sql/sql_insert.cc:
  Added asserts to make sure that in the case of INSERT ... VALUES ...
  statements it is not necessary to reset the name resolution context
  to the first table, because there is only one table in the list of
  tables anyway. The actual code is not removed in order not to
  confuse it with the actual bug fix.
sql/sql_parse.cc:
  Removed unnecessary reset of the name resolution context.
  The context is anyway unconditionally reset in mysql_insert()
  and mysql_prepare_insert().

This bug report was two problems:
1) LAST_INSERT_ID() returns a value which does not exist in the table
2) the reporter would want it to return the autoinc id of the updated
row.
1) is a real bug, 2) is a feature request.
In July I implemented 2) in 5.1 (which automatically fixes 1).
This has not yet been documented or released, so is changeable.
Precisely, recently Paul and a user found an easy workaround to give
2), which works in 4.1-5.0-5.1. So I can revert my code for 2),
because it's not needed, that's what I do here;
we forget about 2) (we will document the workaround).
But when I revert my code for 2), 1) comes back. We solve 1) by saying
that if INSERT ON DUPLICATE KEY UPDATE updates a row, it's like a
regular UPDATE: LAST_INSERT_ID() should not be affected (instead of
returning a non-existent value).
So note: no behaviour change compared to the last released 5.1; just
a bugfix for 1).


mysql-test/r/innodb_mysql.result:
  result update
mysql-test/t/innodb_mysql.test:
      test for the new way to fix BUG#19243: that if INSERT ON DUPLICATE
      KEY UPDATE updates a row, SELECT LAST_INSERT_ID() is not affected.
      Test of the workaround for people who want SELECT LAST_INSERT_ID()
      to return the autoinc id of the updated row.
sql/sql_insert.cc:
  No need to change LAST_INSERT_ID() if INSERT ON DUPLICATE KEY UPDATE
  updates a row, there is a workaround to achieve this without changing
  code: just add "autoinc_col=LAST_INSERT_ID(autoinc_col)" to your
  ON DUPLICATE KEY UPDATE clause.
  Prevent LAST_INSERT_ID() to contain an inexistent value in this case:
  if the row is updated it should be like a regular UPDATE: don't
  affect LAST_INSERT_ID() (achieved by marking that we didn't generate
  an id for this row: insert_id_for_cur_row=0).

SELECT right instead of INSERT right was required for an insert into to a view.
This wrong behaviour appeared after the fix for bug #20989. Its intention was
to ask only SELECT right for all tables except the very first for a complex
INSERT query. But that patch has done it in a wrong way and lead to asking 
a wrong access right for an insert into a view.

The setup_tables_and_check_access() function now accepts two want_access
parameters. One will be used for the first table and the second for other
tables.


mysql-test/t/view.test:
  Added a test case for bug#21261: Wrong access rights was required for an insert into a view
mysql-test/r/view.result:
  Added a test case for bug#21261: Wrong access rights was required for an insert into a view
sql/sql_update.cc:
  Fixed bug#21261: Wrong access rights was required for an insert into a view
  Modified to use updated setup_tables_and_check_access() function.
sql/sql_select.cc:
  Fixed bug#21261: Wrong access rights was required for an insert into a view
  Modified to use updated setup_tables_and_check_access() function.
sql/sql_load.cc:
  Fixed bug#21261: Wrong access rights was required for an insert into a view
  Modified to use updated setup_tables_and_check_access() function.
sql/sql_insert.cc:
  Fixed bug#21261: Wrong access rights was required for an insert into a view
  Modified to use updated setup_tables_and_check_access() function.
sql/sql_delete.cc:
  Fixed bug#21261: Wrong access rights was required for an insert into a view
  Modified to use updated setup_tables_and_check_access() function.
sql/sql_base.cc:
  Fixed bug#21261: Wrong access rights was required for an insert into a view
  The setup_tables_and_check_access() function now accepts two want_access
  parameters. One will be used for the first table and the second for other
  tables.
sql/mysql_priv.h:
  Fixed bug#21261: Wrong access rights was required for an insert into a view
  The setup_tables_and_check_access() function now accepts two want_access
  parameters.

Continued implementation of WL#1324 (table name to filename encoding)

The intermediate (not temporary) files of the new table
during ALTER TABLE was visible for SHOW TABLES. These
intermediate files are copies of the original table with
the changes done by ALTER TABLE. After all the data is
copied over from the original table, these files are renamed 
to the original tables file names. So they are not temporary 
files. They persist after ALTER TABLE, but just with another 
name.

In 5.0 the intermediate files are invisible for SHOW TABLES
because all file names beginning with "#sql" were suppressed.

This failed since 5.1.6 because even temporary table names were
converted when making file names from them. The prefix became
converted to "@0023sql". Converting the prefix during SHOW TABLES
would suppress the listing of user tables that start with "#sql".

The solution of the problem is to continue the implementation of
the table name to ...

REPLACE ... SELECT would require INSERT privileges on certain tables
when SELECT really suffices. Require INSERT only on target table.


mysql-test/r/insert_select.result:
  Bug#20989: View '(null).(null)' references invalid table(s)... on SQL SECURITY INVOKER
  
  Show that REPLACE ... SELECT requires INSERT privileges only on target table.
  (revised test with more view-fu)
mysql-test/t/insert_select.test:
  Bug#20989: View '(null).(null)' references invalid table(s)... on SQL SECURITY INVOKER
  
  Show that REPLACE ... SELECT requires INSERT privileges only on target table.
  (revised test with more view-fu)
sql/sql_insert.cc:
  Bug#20989: View '(null).(null)' references invalid table(s)... on SQL SECURITY INVOKER
  
  require SELECT rather than INSERT privs on tables that constitute the views we'll read

We now reset the THD members related to auto_increment+binlog in
MYSQL_LOG::write(). This is better than in THD::cleanup_after_query(),
which was not able to distinguish between SELECT myfunc1(),myfunc2()
and INSERT INTO t SELECT myfunc1(),myfunc2() from a binlogging point
of view.
Rows_log_event::exec_event() now calls lex_start() instead of
mysql_init_query() because the latter now does too much (it resets
the binlog format).


mysql-test/extra/rpl_tests/rpl_insert_id.test:
  fix after merge
mysql-test/mysql-test-run.pl:
  -v does not bring useful information when running valgrind; I remove it;
  if you think it's useful add it back.
mysql-test/r/binlog_stm_mix_innodb_myisam.result:
  Position columns of SHOW BINLOG EVENTS are replaced by # (more robust
  if the size of an event changes).
mysql-test/r/rpl_insert_id.result:
  fix after merge
mysql-test/r/rpl_loaddata.result:
  The binlog positions change, because one event disappeared; indeed there
  was this ...

mysql-test/r/rpl_switch_stm_row_mixed.result:
  result update.
  I noticed the appearance of an additional table_map+rows in the binlog
  when INSERT DELAYED, since I pulled the latest fixes to RBR of 
  INSERT DELAYED. This is ok, I put the reason why into sql_insert.cc.
sql/sql_insert.cc:
  A comment about the new row-based binlogging of INSERT DELAYED.
  Removing duplicate MY_HOOKS (from a wrong merge of the past??).

this is a cleanup patch for our current auto_increment handling:
new names for auto_increment variables in THD, new methods to manipulate them
(see sql_class.h), some move into handler::, causing less backup/restore
work when executing substatements. 
This makes the logic hopefully clearer, less work is is needed in
mysql_insert().
By cleaning up, using different variables for different purposes (instead
of one for 3 things...), we fix those bugs, which someone may want to fix
in 5.0 too:
BUG#20339 "stored procedure using LAST_INSERT_ID() does not replicate
statement-based"
BUG#20341 "stored function inserting into one auto_increment puts bad
data in slave"
BUG#19243 "wrong LAST_INSERT_ID() after ON DUPLICATE KEY UPDATE"
(now if a row is updated, LAST_INSERT_ID() will return its id)
and re-fixes:
BUG#6880 "LAST_INSERT_ID() value changes during multi-row INSERT"
(already fixed differently by Ramil in 4.1)
Test of documented behaviour of mysql_insert_id() (there was no test).
The behaviour changes introduced are:
- LAST_INSERT_ID() now returns "the first autogenerated auto_increment value
successfully inserted", instead of "the first autogenerated auto_increment
value if any row was successfully inserted", see auto_increment.test.
Same for mysql_insert_id(), see mysql_client_test.c.
- LAST_INSERT_ID() returns the id of the updated row if ON DUPLICATE KEY
UPDATE, see auto_increment.test. Same for mysql_insert_id(), see
mysql_client_test.c.
- LAST_INSERT_ID() does not change if no autogenerated value was successfully 
inserted (it used to then be 0), see auto_increment.test.
- if in INSERT SELECT no autogenerated value was successfully inserted,
mysql_insert_id() now returns the id of the last inserted row (it already
did this for INSERT VALUES), see mysql_client_test.c.
- if INSERT SELECT uses LAST_INSERT_ID(X), mysql_insert_id() now returns X
(it already did this for INSERT VALUES), see mysql_client_test.c.
- NDB now behaves like other engines wrt SET INSERT_ID: with INSERT IGNORE,
the id passed in SET INSERT_ID is re-used until a row succeeds; SET INSERT_ID
influences not only the first row now.

Additionally, when unlocking a table we check that the thread is not keeping
a next_insert_id (as the table is unlocked that id is potentially out-of-date);
forgetting about this next_insert_id is done in a new
handler::ha_release_auto_increment().

Finally we prepare for engines capable of reserving finite-length intervals
of auto_increment values: we store such intervals in THD. The next step
(to be done by the replication team in 5.1) is to read those intervals from
THD and actually store them in the statement-based binary log. NDB
will be a good engine to test that.


mysql-test/extra/binlog_tests/binlog.test:
  Testing that if INSERT_ID is set to a value too big for the
  column's type, the binlogged INSERT_ID is the truncated value
  (important if slave has a column of a "wider" numeric type).
  Testing binlogging of INSERT_ID with INSERT DELAYED, to be sure that 
  we binlog an INSERT_ID event only for the delayed rows which use one.
mysql-test/extra/rpl_tests/rpl_insert_id.test:
  Testcase for BUG#20339 "stored procedure using
  LAST_INSERT_ID() does not replicate statement-based".
  Testcase for BUG#20341 "stored function inserting into one
  auto_increment puts bad data in slave".
mysql-test/extra/rpl_tests/rpl_loaddata.test:
  Test that LOAD DATA INFILE sets a value for a future LAST_INSERT_ID().
mysql-test/r/auto_increment.result:
  behaviour change: when INSERT totally fails (not even succeeds
  partially and then rolls back), don't change last_insert_id().
  Behaviour change: LAST_INSERT_ID() is now the first successfully inserted,
  autogenerated, id.
  Behaviour change: if INSERT ON DUPLICATE KEY UPDATE, if the table has auto_increment
  and a row is updated, then LAST_INSERT_ID() returns the id of this row.
mysql-test/r/binlog_row_binlog.result:
  result update
mysql-test/r/binlog_stm_binlog.result:
  result update
mysql-test/r/insert.result:
  result update
mysql-test/r/rpl_insert_id.result:
  result update
mysql-test/r/rpl_loaddata.result:
  result update
mysql-test/r/rpl_ndb_auto_inc.result:
  ndb's behaviour is now like other engines wrt SET INSERT_ID
  in a multi-row INSERT:
  - with INSERT IGNORE: the id passed in SET INSERT_ID is re-used until
  a row succeeds.
  - generally, SET INSERT_ID sets the first value and other values are
  simply computed from this first value, instead of previously where
  the 2nd and subsequent values where not influenced by SET INSERT_ID;
  this good change is due to the removal of "thd->next_insert_id=0"
  from ha_ndbcluster.
mysql-test/t/auto_increment.test:
  A testcase of BUG#19243: if ON DUPLICATE KEY UPDATE updates a row,
  LAST_INSERT_ID() now returns the id of the row.
  Test of new behaviour of last_insert_id() when no autogenerated value was
  inserted, or when only some autogenerated value (not the first of them) was
  inserted.
mysql-test/t/insert.test:
  testing INSERT IGNORE re-using generated values
sql/ha_federated.cc:
  update for new variables.
sql/ha_ndbcluster.cc:
  handler::auto_increment_column_changed not needed, equivalent to
  (insert_id_for_cur_row > 0).
  thd->next_insert_id=0 not needed anymore; it was used to force
  handler::update_auto_increment() to call ha_ndbcluster::get_auto_increment()
  for each row of a multi-row INSERT, now this happens naturally
  because NDB says "I have reserved you *one* value" in get_auto_increment(),
  so handler::update_auto_increment() calls again for next row.
sql/handler.cc:
  More comments, use of new methods and variables. Hopes to be clearer
  than current code.
  thd->prev_insert_id not in THD anymore: it is managed locally by inserters
  (like mysql_insert()).
  THD::clear_next_insert_id is now equivalent to
  handler::next_insert_id > 0.
  get_auto_increment() reserves an interval of values from the engine,
  uses this interval for next rows of the statement, until interval
  is exhausted then it asks for another interval (of a bigger size
  than the first one; size doubles until reaching 65535 then it stays constant).
  If doing statement-based binlogging, intervals are remembered in a list
  for storage in the binlog.
  For "forced" insert_id values (SET INSERT_ID or replication slave),
  forced_auto_inc_intervals is non-empty and the handler takes its intervals
  from there, without calling get_auto_increment().
  ha_release_auto_increment() resets the handler's auto_increment variables;
  it calls release_auto_increment() which is handler-dependent and
  serves to return to the engine any unused tail of the last used
  interval.
  If ending a statement, next_insert_id>0 means that autoinc values have been
  generated or taken from the master's binlog (in a replication slave) so
  we clear those values read from binlog, so that next top- or sub-
  statement does not use them.
sql/handler.h:
  handler::auto_increment_changed can be replaced by
  (handler::insert_id_for_cur_row > 0).
  THD::next_insert_id moves into handler (more natural, and prepares
  for the day when we'll support a single statement inserting into
  two tables - "multi-table INSERT" like we have UPDATE - will this
  happen?).
  This move makes the backup/restore of THD::next_insert_id when entering
  a substatement unneeded, as each substatement has its own handler
  objects.
sql/item_func.cc:
  new names for variables.
  For the setting of what mysql_insert_id() will return to the client,
  LAST_INSERT_ID(X) used to simply pretend that the generated autoinc
  value for the current row was X, but this led to having no reliable
  way to know the really generated value, so we now have a bool:
  thd->arg_of_last_insert_id_function which enables us to know that
  LAST_INSERT_ID(X) was called (and then X can be found in
  thd->first_successful_insert_id_in_prev_stmt).
sql/log.cc:
  new variable names for insert_ids. Removing some unused variables in the slow
  log.
sql/log_event.cc:
  new variable names, comments. Preparing for when master's won't binlog
  LAST_INSERT_ID if it was 0.
sql/set_var.cc:
  new variable names.
  The last change repeats how Bar fixed BUG#20392
  "INSERT_ID session variable has weird value" in 5.0.
sql/sql_class.cc:
  new variables for insert_id. In THD::cleanup_after_query() we fix
  BUG#20339 "stored procedure using LAST_INSERT_ID() does not replicate
  statement-based" (will one want to fix it in 5.0?). Many comments
  about what stored functions do to auto_increment.
  In reset|restore_sub_statement_state(), we need to backup less
  auto_inc variables as some of them have moved to the handler;
  we backup/restore those which are about the current top- or sub-
  statement, *not* those about the statement-based binlog
  (which evolve as the top- and sub-statement execute).
  Because we split THD::last_insert_id into 
  THD::first_successful_insert_id_in_prev_stmt and
  THD::auto_inc_intervals_for_binlog (among others), we fix
  BUG#20341 "stored function inserting into one auto_increment
  puts bad data in slave": indeed we can afford to not backup/restore
  THD::auto_inc_intervals_for_binlog (which fixes the bug) while still
  backing up / restoring THD::first_successful_insert_id_in_prev_stmt
  (ensuring that the top-level LAST_INSERT_ID() is not affected by INSERTs
  done by sub-statements, as is desirable and tested in rpl_insert_id.test).
sql/sql_class.h:
  new variables and methods for auto_increment.
  Some THD members move into handler (those which are really about
  the table being inserted), some stay in THD (those which are
  about what a future LAST_INSERT_ID() should return, or about
  what should be stored into the statement-based binlog).
  THD::next_insert_id moves to handler::.
  THD::clear_next_insert_id removed (had become equivalent
  to next_insert_id > 0).
  THD::last_insert_id becomes four:
  THD::first_successful_insert_id_in_cur_stmt,
  THD::auto_inc_intervals_for_binlog,
  handler::insert_id_for_cur_row,
  THD::first_successful_insert_id_in_prev_stmt.
  THD::current_insert_id becomes:
  THD::first_successful_insert_id_in_prev_stmt_for_binlog
  THD::prev_insert_id is removed, handler can just use
  handler::insert_id_for_cur_row instead (which is more accurate:
  for the first row, prev_insert_id was set before get_auto_increment
  was called, so was 0, causing a call to
  get_auto_increment() for the 2nd row if the 1st row fails;
  here we don't need the call as insert_id_for_cur_row has
  the value of the first row).
  THD::last_insert_id_used becomes: stmt_depends_on_first_row_in_prev_stmt
  THD::insert_id_used is removed (equivalent to
  auto_inc_intervals_for_binlog non empty).
  The interval returned by get_auto_increment() and currently being
  consumed is handler::auto_inc_interval_for_cur_row.
  Comments to explain each of them.
  select_insert::last_insert_id becomes autoinc_value_of_last_inserted_row.
sql/sql_insert.cc:
  the "id" variable is not changed for each row now; it used to compensate for
  this contradiction:
  - thd->last_insert_id supposed job was to keep the id of the first row
  - but it was updated for every row
  - so mysql_insert() made sure to catch its first value and restore it at the end of stmt.
  Now THD keeps the first value in first_successful_insert_id_in_cur_stmt,
  and value of the row in insert_id_for_cur_row. So "id" only serves to fill
  mysql_insert_id(), as depending on some conditions, "id" must be different
  values.
  Prev_insert_id moves from THD to write_record().
  We now set LAST_INSERT_ID() in ON DUPLICATE KEY UPDATE too (BUG#19243).
  In an INSERT DELAYED, we still "reset auto-increment caching" but differently
  (by calling ha_release_auto_increment()).
sql/sql_load.cc:
  no need to fiddle with "id", THD maintains
  THD::first_successful_insert_id_in_cur_stmt by itself and correctly now.
  ha_release_auto_increment() is now (logically) called before we unlock
  the table.
sql/sql_parse.cc:
  update to new variable names.
  Assertion that reset_thd_for_next_command() is not called for every
  substatement of a routine (I'm not against it, but if we do this change,
  statement-based binlogging needs some adjustments).
sql/sql_select.cc:
  update for new variable names
sql/sql_table.cc:
  next_insert_id not needed in mysql_alter_table(), THD manages.
sql/sql_update.cc:
  update for new variable names.
  Even though this is UPDATE, an insert id can be generated (by
  LAST_INSERT_ID(X)) and should be recorded because mysql_insert_id() wants
  to know about it.
sql/structs.h:
  A class for "discrete" intervals (intervals of integer numbers with a certain
  increment between them): Discrete_interval, and a class for a list of such
  intervals: Discrete_intervals_list
tests/mysql_client_test.c:
  tests of behaviour of mysql_insert_id(): there were no such tests, while in
  our manual we document its behaviour. In comments you'll notice the behaviour
  changes introduced (there are 5).

Fixing typo and potential memory problem.
Reducing number of concurrent mysqlslap threads since tests fail
in pushbuild due to too many threads.


mysql-test/r/rpl_insert.result:
  Result change.
mysql-test/t/rpl_insert.test:
  Reducing number of threads since it doesn't pass pushbuild.
sql/sql_insert.cc:
  Fixing typo and potential memory problem.

statement-based" (bugfix was committed today):
we verify that now it works in mixed mode. And a comment.


mysql-test/r/rpl_switch_stm_row_mixed.result:
  result update
mysql-test/t/rpl_switch_stm_row_mixed.test:
  testcase for BUG#20633
sql/sql_insert.cc:
  the #ifdef was not necessary; a comment.

The bug was that if the server was running in mixed binlogging mode,
and an INSERT DELAYED used some needing-row-based components like UUID(),
the server didn't binlog this row-based but statement-based, which
thus failed to insert correct data on the slave.
This changeset implements that when a delayed_insert thread is created,
if the server's global binlog mode is "mixed", that thread will use row-based.
This also fixes BUG#20633 "INSERT DELAYED RAND() or @user_var does not
replicate statement-based": we don't fix it in statement-based mode (would
require bookeeping of rand seeds and user variables used by each row),
but at least it will now work in mixed mode (as row-based will be used).
We re-enable rpl_switch_stm_row_mixed.test (so BUG#18590
which was about re-enabling this test, will be closed) to test the fixes.
Between when it was disabled and now, some good changes to row-based
binlogging (no generation of table map events for non-changed tables)
induce changes in the test's result file.


mysql-test/r/rpl_switch_stm_row_mixed.result:
  result update.
  Note that some pieces of binlog are gone, not due to my test but to changes
  to the row-based binlogging code (non-changed tables don't generate
  table map binlog events now) done while the test was disabled.
mysql-test/t/disabled.def:
  this test works now
mysql-test/t/rpl_switch_stm_row_mixed.test:
  testing fix to make INSERT DELAYED work in mixed mode
sql/sql_insert.cc:
  In mixed binlogging mode, the delayed_insert system thread now always
  uses row-based binlogging.
  This makes replication of INSERT DELAYED VALUES(RAND()) or VALUES(@A)
  work in mixed mode (it does not in statement-based).

before update trigger on NDB table".

Two main changes:
- We use TABLE::read_set/write_set bitmaps for marking fields used by
  statement instead of Field::query_id in 5.1.
- Now when we mark columns used by statement we take into account columns 
  used by table's triggers instead of marking all columns as used if table
  has triggers.


mysql-test/r/federated.result:
  Changed test in order to make it work with RBR.
  RBR changes the way in which we execute "DELETE FROM t1" statement - we don't
  use handler::delete_all_rows() method if RBR is enabled (see bug#19066).
  As result federated engine produces different sequences of statements for
  remote server in non-RBR and in RBR cases. And this changes order of the
  rows inserted by following INSERT statements.
mysql-test/t/federated.test:
  Changed test in order to make it work with RBR.
  RBR changes the way in which we execute "DELETE FROM t1" statement - we don't
  use handler::delete_all_rows() method if RBR is enabled (see bug#19066).
  As result federated engine produces different sequences of statements for
  remote server in non-RBR and in RBR cases. And this changes order of the
  rows inserted by following INSERT statements.
sql/ha_partition.cc:
  Added handling of HA_EXTRA_WRITE_CAN_REPLACE/HA_EXTRA_WRITE_CANNOT_REPLACE
  to ha_partition::extra().
sql/item.cc:
  Adjusted comment after merge. In 5.1 we use TABLE::read_set/write_set
  bitmaps instead of Field::query_id for marking columns used.
sql/log_event.cc:
  Write_rows_log_event::do_before_row_operations():
    Now we explicitly inform handler that we want to replace rows so it can
    promote operation done by write_row() to replace.
sql/mysql_priv.h:
  Removed declaration of mark_fields_used_by_triggers_for_insert_stmt() which
  is no longer used (we have TABLE::mark_columns_needed_for_insert() instead).
sql/sql_insert.cc:
  Adjusted code after merge. Get rid of mark_fields_used_by_triggers_for_insert_stmt()
  as now we use TABLE::mark_columns_needed_for_insert() for the same purprose.
  Aligned places where we call this method with places where we call
  mark_fields_used_by_triggers_for_insert() in 5.0.
  Finally we no longer need to call handler::extra(HA_EXTRA_WRITE_CAN_REPLACE)
  in case of REPLACE statement since in 5.1 write_record() marks all columns
  as used before doing actual row replacement.
sql/sql_load.cc:
  Adjusted code after merge. In 5.1 we use TABLE::mark_columns_needed_for_insert() instead of
  mark_fields_used_by_triggers_for_insert_stmt() routine. We also no longer
  need to call handler::extra(HA_EXTRA_RETRIEVE_ALL_COLS) if we execute LOAD
  DATA REPLACE since in 5.1 write_record() will mark all columns as used before
  doing actual row replacement.
sql/sql_trigger.cc:
  Table_triggers_list::mark_fields_used():
    We use TABLE::read_set/write_set bitmaps for marking fields used instead
    of Field::query_id in 5.1.
sql/sql_trigger.h:
  TABLE::mark_columns_needed_for_* methods no longer need to be friends of
  Table_triggers_list class as intead of dirrectly accessing its private
  members they can use public Table_triggers_list::mark_fields_used() method.
  Also Table_triggers)list::mark_fields_used() no longer needs THD argument.
sql/table.cc:
  TABLE::mark_columns_needed_for_*():
    Now we mark columns which are really used by table's triggers instead of
    marking all columns as used if table has triggers.

Reverting to old behaviour of writing the query before all rows
have been written.


mysql-test/r/rpl_row_delayed_ins.result:
  Result change
sql/sql_class.cc:
  Adding debug message to binlog_query()
sql/sql_insert.cc:
  - Changing write_delayed() to use a LEX_STRING for the query.
  - Adding query string to class delayed_row.
  - Removing query string from class delayed_insert.
  - Adding code to copy query string and delete it when the row
    is executed.
  - Logging query at first row instead of after all rows are 
    inserted (reverting to old behaviour).
  - Flushing the pending row event after all rows have been inserted.
    This is necessary since binlog_query() is called before all rows
    instead of after.
mysql-test/r/rpl_insert.result:
  New BitKeeper file ``mysql-test/r/rpl_insert.result''
mysql-test/t/rpl_insert.test:
  New BitKeeper file ``mysql-test/t/rpl_insert.test''

auto_increment breaks binlog":
if slave's table had a higher auto_increment counter than master's (even
though all rows of the two tables were identical), then in some cases,
REPLACE and INSERT ON DUPLICATE KEY UPDATE failed to replicate
statement-based (it inserted different values on slave from on master).
write_record() contained a "thd->next_insert_id=0" to force an adjustment
of thd->next_insert_id after the update or replacement. But it is this
assigment introduced indeterminism of the statement on the slave, thus
the bug. For ON DUPLICATE, we replace that assignment by a call to
handler::adjust_next_insert_id_after_explicit_value() which is deterministic
(does not depend on slave table's autoinc counter). For REPLACE, this
assignment can simply be removed (as REPLACE can't insert a number larger
than thd->next_insert_id).
We also move a too early restore_auto_increment() down to when we really know
that we can restore the value.


mysql-test/r/rpl_insert_id.result:
  result update, without the bugfix, slave's "3 350" were "4 350".
mysql-test/t/rpl_insert_id.test:
  test for BUG#20188 "REPLACE or ON DUPLICATE KEY UPDATE in
  auto_increment breaks binlog".
  There is, in this order:
  - a test of the bug for the case of REPLACE
  - a test of basic ON DUPLICATE KEY UPDATE functionality which was not
  tested before
  - a test of the bug for the case of ON DUPLICATE KEY UPDATE
sql/handler.cc:
  the adjustment of next_insert_id if inserting a big explicit value, is
  moved to a separate method to be used elsewhere.
sql/handler.h:
  see handler.cc
sql/sql_insert.cc:
  restore_auto_increment() means "I know I won't use this autogenerated
  autoincrement value, you are free to reuse it for next row". But we were
  calling restore_auto_increment() in the case of REPLACE: if write_row() fails
  inserting the row, we don't know that we won't use the value, as we are going to
  try again by doing internally an UPDATE of the existing row, or a DELETE
  of the existing row and then an INSERT. So I move restore_auto_increment()
  further down, when we know for sure we failed all possibilities for the row.
  Additionally, in case of REPLACE, we don't need to reset THD::next_insert_id:
  the value of thd->next_insert_id will be suitable for the next row.
  In case of ON DUPLICATE KEY UPDATE, resetting thd->next_insert_id is also
  wrong (breaks statement-based binlog), but cannot simply be removed, as
  thd->next_insert_id must be adjusted if the explicit value exceeds it.
  We now do the adjustment by calling
  handler::adjust_next_insert_id_after_explicit_value() (which, contrary to
  thd->next_insert_id=0, does not depend on the slave table's autoinc counter,
  and so is deterministic).

When compiling INSERT statements the check whether columns are provided values
depends on the flag whether a field is used in that query (Field::query_id).
However the check for updatability of VIEW columns (check_view_insertability())
was calling fix_fields() and thus setting the Field::query_id even for the 
view fields that are not referenced in the current INSERT statement.
So the correct check for columns without default values 
( check_that_all_fields_are_given_values() ) is assuming that all the VIEW
columns were mentioned in the INSERT field list and was issuing no 
warnings or errors.
Fixed check_view_insertability() to turn off the flag whether or not to set
Field::query_id (THREAD::set_query_id) before calling fix fields and restore
it when it's done.


mysql-test/r/view.result:
  Bug #16110: insert permitted into view col w/o default value
    * test case
mysql-test/t/view.test:
  Bug #16110: insert permitted into view col w/o default value
    * test case
sql/sql_insert.cc:
  Bug #16110: insert permitted into view col w/o default value
    * avoid setting the "field used" flag for fields when checking view columns 
      for updatability.
    * a missing DBUG_RETURN added.

NDB table".

SQL-layer was not marking fields which were used in triggers as such. As
result these fields were not always properly retrieved/stored by handler
layer. So one might got wrong values or lost changes in triggers for NDB,
Federated and possibly InnoDB tables.
This fix solves the problem by marking fields used in triggers
appropriately.

Also this patch contains the following cleanup of ha_ndbcluster code:

We no longer rely on reading LEX::sql_command value in handler in order
to determine if we can enable optimization which allows us to handle REPLACE
statement in more efficient way by doing replaces directly in write_row()
method without reporting error to SQL-layer.
Instead we rely on SQL-layer informing us whether this optimization
applicable by calling handler::extra() method with
HA_EXTRA_WRITE_CAN_REPLACE flag.
As result we no longer apply this optimzation in cases when it should not
be used (e.g. if we have on delete triggers on table) and use in some
additional cases when it is applicable (e.g. for LOAD DATA REPLACE).

Finally this patch includes fix for bug#20728 "REPLACE does not work
correctly for NDB table with PK and unique index".
  
This was yet another problem which was caused by improper field mark-up.
During row replacement fields which weren't explicity used in REPLACE
statement were not marked as fields to be saved (updated) so they have
retained values from old row version. The fix is to mark all table
fields as set for REPLACE statement. Note that in 5.1 we already solve
this problem by notifying handler that it should save values from all
fields only in case when real replacement happens.


include/my_base.h:
  Added HA_EXTRA_WRITE_CAN_REPLACE, HA_EXTRA_WRITE_CANNOT_REPLACE - new
  parameters for ha_extra() method. We use them to inform handler that
  write_row() which tries to insert new row into the table and encounters
  some already existing row with same primary/unique key can replace old
  row with new row instead of reporting error.
mysql-test/r/federated.result:
  Additional test for bug#18437 "Wrong values inserted with a before update
  trigger on NDB table".
mysql-test/r/ndb_replace.result:
  Added test for bug #20728 "REPLACE does not work correctly for NDB table
  with PK and unique index". Updated wrong results from older test.
mysql-test/t/federated.test:
  Additional test for bug#18437 "Wrong values inserted with a before update
  trigger on NDB table".
mysql-test/t/ndb_replace.test:
  Added test for bug #20728 "REPLACE does not work correctly for NDB table
  with PK and unique index".
sql/ha_ndbcluster.cc:
  We no longer rely on reading LEX::sql_command value in handler in order
  to determine if we can enable optimization which allows us to handle REPLACE
  statement in more efficient way by doing replaces directly in write_row()
  method without reporting error to SQL-layer.
  Instead we rely on SQL-layer informing us whether this optimization
  applicable by calling handler::extra() method with
  HA_EXTRA_WRITE_CAN_REPLACE flag.
  As result we no longer apply this optimization in cases when it should not
  be used (e.g. if we have on delete triggers on table) and use in some
  additional cases when it is applicable (e.g. for LOAD DATA REPLACE).
sql/item.cc:
  Item_trigger_field::setup_field():
    Added comment explaining why we don't set Field::query_id in this method.
sql/mysql_priv.h:
  mysql_alter_table() function no longer takes handle_duplicates argument.
  Added declaration of mark_fields_used_by_triggers_for_insert_stmt() function.
sql/sql_delete.cc:
  Mark fields which are used by ON DELETE triggers so handler will retrieve
  values for these fields.
sql/sql_insert.cc:
  Explicitly inform handler that we are doing REPLACE (using ha_extra() method)
  in cases when it can promote insert operation done by write_row() to replace.
  Also when we do REPLACE we want to store values for all columns so we should
  inform handler about it.
  Finally we should mark fields used by ON UPDATE/ON DELETE triggers as such
  so handler can properly retrieve/restore values in these fields during
  execution of REPLACE and INSERT ... ON DUPLICATE KEY UPDATE statements.
sql/sql_load.cc:
  Explicitly inform handler that we are doing LOAD DATA REPLACE (using
  ha_extra() method) in cases when it can promote insert operation done by
  write_row() to replace.
  Also when we do replace we want to save (replace) values for all columns
  so we should inform handler about it.
  Finally to properly execute LOAD DATA for table with triggers we should
  mark fields used by ON INSERT triggers as such so handler can properly
  store values for these fields.
sql/sql_parse.cc:
  mysql_alter_table() function no longer takes handle_duplicates argument.
sql/sql_table.cc:
  Got rid of handle_duplicates argument in mysql_alter_table() and
  copy_data_between_tables() functions. These functions were always
  called with handle_duplicates == DUP_ERROR and thus contained dead
  (and probably incorrect) code.
sql/sql_trigger.cc:
  Added Table_triggers_list::mark_fields_used() method which is used to mark
  fields read/set by triggers as such so handlers will be able properly
  retrieve/store values in these fields.
sql/sql_trigger.h:
  Table_triggers_list:
    Added mark_fields_used() method which is used to mark fields read/set by
    triggers as such so handlers will be able properly retrieve/store values
    in these fields. To implement this method added 'trigger_fields' member
    which is array of lists linking items for all fields used in triggers
    grouped by event and action time.
sql/sql_update.cc:
  Mark fields which are used by ON UPDATE triggers so handler will retrieve
  and save values for these fields.
mysql-test/r/ndb_trigger.result:
  Added test for bug#18437 "Wrong values inserted with a before update trigger
  on NDB table".
mysql-test/t/ndb_trigger.test:
  Added test for bug#18437 "Wrong values inserted with a before update trigger
  on NDB table".

Last round of review fixes


BUILD/compile-pentium-gcov:
  No change
sql/ha_ndbcluster.h:
  Last round of review changes
sql/ha_partition.h:
  Last round of review changes
sql/handler.h:
  Last round of review changes
sql/item_sum.cc:
  Last round of review changes
sql/sql_acl.cc:
  Last round of review changes
sql/sql_insert.cc:
  Last round of review changes
sql/sql_select.cc:
  Last round of review changes
sql/sql_table.cc:
  Last round of review changes
sql/sql_union.cc:
  Last round of review changes
sql/sql_update.cc:
  Last round of review changes

 Bug#19022 "Memory bug when switching db during trigger execution"
 Bug#17199 "Problem when view calls function from another database."
 Bug#18444 "Fully qualified stored function names don't work correctly in
            SELECT statements"

 Documentation note: this patch introduces a change in behaviour of prepared
 statements.

 This patch adds a few new invariants with regard to how THD::db should
 be used. These invariants should be preserved in future:

  - one should never refer to THD::db by pointer and always make a deep copy
    (strmake, strdup)
  - one should never compare two databases by pointer, but use strncmp or
    my_strncasecmp
  - TABLE_LIST object table->db should be always initialized in the parser or
    by creator of the object.

    For prepared statements it means that if the current database is changed
    after a statement is prepared, the database that was current at prepare
    remains active. This also means that you can not prepare a statement that
    implicitly refers to the current database if the latter is not set.
    This is not documented, and therefore needs documentation. This is NOT a
    change in behavior for almost all SQL statements except:
     - ALTER TABLE t1 RENAME t2 
     - OPTIMIZE TABLE t1
     - ANALYZE TABLE t1
     - TRUNCATE TABLE t1 --
     until this patch t1 or t2 could be evaluated at the first execution of
     prepared statement. 

     CURRENT_DATABASE() still works OK and is evaluated at every execution
     of prepared statement.

     Note, that in stored routines this is not an issue as the default
     database is the database of the stored procedure and "use" statement
     is prohibited in stored routines.

  This patch makes obsolete the use of check_db_used (it was never used in the
  old code too) and all other places that check for table->db and assign it
  from THD::db if it's NULL, except the parser.

 How this patch was created: THD::{db,db_length} were replaced with a
 LEX_STRING, THD::db. All the places that refer to THD::{db,db_length} were
 manually checked and:
  - if the place uses thd->db by pointer, it was fixed to make a deep copy
  - if a place compared two db pointers, it was fixed to compare them by value
    (via strcmp/my_strcasecmp, whatever was approproate)
 Then this intermediate patch was used to write a smaller patch that does the
 same thing but without a rename.

 TODO in 5.1:
   - remove check_db_used
   - deploy THD::set_db in mysql_change_db

 See also comments to individual files.


mysql-test/r/create.result:
  Modify the result file: a database can never be NULL.
mysql-test/r/ps.result:
  Update test results (Bug#17199 et al)
mysql-test/r/sp.result:
  Update test results (Bug#17199 et al)
mysql-test/t/create.test:
  Update the id of the returned error.
mysql-test/t/ps.test:
  Add test coverage for prepared statements and current database. In scope of
  work on Bug#17199 "Problem when view calls function from another database."
mysql-test/t/sp.test:
  Add a test case for Bug#17199 "Problem when view calls function from another
  database." and Bug#18444 "Fully qualified stored function names don't work
  correctly in SELECT statements". Test a complementary problem.
sql/item_strfunc.cc:
  Touch the code that reads thd->db (cleanup).
sql/log_event.cc:
  While we are at it, replace direct access to thd->db with a method.
  Should simplify future conversion of THD::db to LEX_STRING.
sql/slave.cc:
  While we are at it, replace direct access to thd->db with a method.
  Should simplify future conversion of THD::db to LEX_STRING.
sql/slave.h:
  Remove a declaration for a method that is used only in one module.
sql/sp.cc:
  Rewrite sp_use_new_db: this is a cleanup that I needed in order to understand
  this function and ensure that it has no bugs.
sql/sp.h:
  Add a new declaration for sp_use_new_db (uses LEX_STRINGs) and a comment.
sql/sp_head.cc:
  - drop sp_name_current_db_new - a creator of sp_name class that was used
  when sp_name was created for an identifier without an explicitly initialized
  database. Now we pass thd->db to constructor of sp_name right in the 
  parser.
  - rewrite sp_head::init_strings: name->m_db is always set now
  - use the new variant of sp_use_new_db
  - we don't need to update thd->db with SP MEM_ROOT pointer anymore when
  parsing a stored procedure, as noone will refer to it (yes!)
sql/sp_head.h:
  - remove unneded methods and members
sql/sql_class.h:
  - introduce 3 THD  methods to work with THD::db:
    .set_db to assign the current database
    .reset_db to reset the current database (temporarily) or set it to NULL
    .opt_copy_db_to - to deep-copy thd->db to a pointer if it's not NULL
sql/sql_db.cc:
  While we are at it, replace direct access to thd->db with a method.
  Should simplify future conversion of THD::db to LEX_STRING.
sql/sql_insert.cc:
  - replace checks with asserts: table_list->db must be always set in the parser.
sql/sql_lex.h:
  - add a comment
sql/sql_parse.cc:
  - implement the invariant described in the changeset comment.
  - remove juggling with lex->sphead in SQLCOM_CREATE_PROCEDURE:
    now db_load_routine uses its own LEX object and doesn't damage the main
    LEX.
  - add DBUG_ASSERT(0) to unused "check_db_used"
sql/sql_table.cc:
  - replace a check with an assert (table_ident->db)
sql/sql_trigger.cc:
  While we are at it, replace direct access to thd->db with a method.
  Should simplify future conversion of THD::db to LEX_STRING.
sql/sql_udf.cc:
  - use thd->set_db instead of direct modification of to thd->db
sql/sql_view.cc:
  - replace a check with an assert (view->db)
sql/sql_yacc.yy:
  - make sure that we always copy table->db or name->db or ident->db or
    select_lex->db from thd->db if the former is not set. If thd->db
    is not set but is accessed, return an error.
sql/tztime.cc:
  - be nice, never copy thd->db by pointer.

Bug#17294 - INSERT DELAYED puting an \n before data
Bug#16611 - INSERT DELAYED corrupts data
Bug#13707 - Server crash with INSERT DELAYED on MyISAM table
Combined as Bug#16218.

INSERT DELAYED crashed in 5.0 on a table with a varchar that 
could be NULL and was created pre-5.0 (Bugs 16218 and 13707).
INSERT DELAYED corrupted data in 5.0 on a table with varchar 
fields that was created pre-5.0 (Bugs 17294 and 16611).

In case of INSERT DELAYED the open table is copied from the
delayed insert thread to be able to create a record for the 
queue. When copying the fields, a method was used that did 
convert old varchar to new varchar fields and did not set up 
some pointers into the record buffer of the table.

The field conversion was guilty for the misinterpretation of 
the record contents by the delayed insert thread. The wrong
pointer setup was guilty for the crashes.

For Bug 13707 (Server crash with INSERT DELAYED on MyISAM table)
I fixed the above mentioned method to set up one of the pointers.
For Bug 16218 I set up the other pointers too.

But when looking at the corruptions I got aware that converting
the field type was totally wrong for INSERT DELAYED. The copied
table is used to create a record that is to be sent to the
delayed insert thread. Of course it can interpret the record
correctly only if all field types are the same in both table
objects.

So I revoked the fix for Bug 13707 and changed the new_field() 
method so that it can suppress conversions.

No test case as this is a migration problem. One needs to
create a table with 4.x and use it with 5.x. I added two
test scripts to the bug report.


sql/field.cc:
  Bug#16218 - Crash on insert delayed
  Bug#17294 - INSERT DELAYED puting an \n before data
  Bug#16611 - INSERT DELAYED corrupts data
  Bug#13707 - Server crash with INSERT DELAYED on MyISAM table
  Combined as Bug#16218.
  
  Added parameter 'keep_type' to Field::new_field().
  
  Undid the change from Bug 13707 (Server crash with INSERT 
  DELAYED on MyISAM table).
  I solved all four bugs in sql/sql_insert.cc by making exact
  duplicates of the fields. The new_field() method converts
  certain field types, which is wrong for INSERT DELAYED.
sql/field.h:
  Bug#13707 - Server crash with INSERT DELAYED on MyISAM table
  Combined as Bug#16218.
  
  Added parameter 'keep_type' to Field::new_field().
sql/sql_insert.cc:
  Bug#16218 - Crash on insert delayed
  Bug#17294 - INSERT DELAYED puting an \n before data
  Bug#16611 - INSERT DELAYED corrupts data
  Bug#13707 - Server crash with INSERT DELAYED on MyISAM table
  Combined as Bug#16218.
  
  Added comments. Made small style fixes.
  Used the new parameter 'keep_type' of Field::new_field()
  to avoid field type conversion. The table copy must have
  exactly the same types of fields as the original table.
  Otherwise the record contents created by the foreground 
  thread could be misinterpreted by the delayed insert thread.
sql/sql_select.cc:
  Bug#16218 - Crash on insert delayed
  Bug#17294 - INSERT DELAYED puting an \n before data
  Bug#16611 - INSERT DELAYED corrupts data
  Bug#13707 - Server crash with INSERT DELAYED on MyISAM table
  Combined as Bug#16218.
  
  Added parameter 'keep_type' to Field::new_field().
  
  Undid the change from Bug 13707 (Server crash with INSERT 
  DELAYED on MyISAM table).
  I solved all four bugs in sql/sql_insert.cc by making exact
  duplicates of the fields. The new_field() method converts
  certain field types, which is wrong for INSERT DELAYED.
sql/sql_trigger.cc:
  Bug#16218 - Crash on insert delayed
  Bug#17294 - INSERT DELAYED puting an \n before data
  Bug#16611 - INSERT DELAYED corrupts data
  Bug#13707 - Server crash with INSERT DELAYED on MyISAM table
  Combined as Bug#16218.
  
  Added parameter 'keep_type' to Field::new_field().
  
  Undid the change from Bug 13707 (Server crash with INSERT 
  DELAYED on MyISAM table).
  I solved all four bugs in sql/sql_insert.cc by making exact
  duplicates of the fields. The new_field() method converts
  certain field types, which is wrong for INSERT DELAYED.
sql/table.cc:
  Bug#16218 - Crash on insert delayed
  Bug#17294 - INSERT DELAYED puting an \n before data
  Bug#16611 - INSERT DELAYED corrupts data
  Bug#13707 - Server crash with INSERT DELAYED on MyISAM table
  Combined as Bug#16218.
  
  Added parameter 'keep_type' to Field::new_field().
  
  Undid the change from Bug 13707 (Server crash with INSERT 
  DELAYED on MyISAM table).
  I solved all four bugs in sql/sql_insert.cc by making exact
  duplicates of the fields. The new_field() method converts
  certain field types, which is wrong for INSERT DELAYED.

Addendum fixes after changing the condition variable
for the global read lock.

The stress test suite revealed some deadlocks. Some were
related to the new condition variable (COND_global_read_lock)
and some were general problems with the global read lock.

It is now necessary to signal COND_global_read_lock whenever 
COND_refresh is signalled.

We need to wait for the release of a global read lock if one 
is set before every operation that requires a write lock.
But we must not wait if we have locked tables by LOCK TABLES.
After setting a global read lock a thread waits until all
write locks are released.


mysql-test/r/lock_multi.result:
  Bug#16986 - Deadlock condition with MyISAM tables
  Addendum fixes after changing the condition variable
  for the global read lock.
  Added test results.
mysql-test/t/lock_multi.test:
  Bug#16986 - Deadlock condition with MyISAM tables
  Addendum fixes after changing the condition variable
  for the global read lock.
  Added tests for possible deadlocks that did not occur
  with the stress test suite.
mysys/thr_lock.c:
  Bug#16986 - Deadlock condition with MyISAM tables
  Addendum fixes after changing the condition variable
  for the global read lock.
  Added a protection against an infinite loop that occurs
  with the test case for Bug #20662.
sql/lock.cc:
  Bug#16986 - Deadlock condition with MyISAM tables
  Addendum fixes after changing the condition variable
  for the global read lock.
  Signal COND_global_read_lock whenever COND_refresh
  is signalled by using the new function broadcast_refresh().
  Added the definition of a new function that signals 
  COND_global_read_lock whenever COND_refresh is signalled.
sql/mysql_priv.h:
  Bug#16986 - Deadlock condition with MyISAM tables
  Addendum fixes after changing the condition variable
  for the global read lock.
  Added a declaration for a new function that signals 
  COND_global_read_lock whenever COND_refresh is signalled.
sql/sql_base.cc:
  Bug#16986 - Deadlock condition with MyISAM tables
  Addendum fixes after changing the condition variable
  for the global read lock.
  Signal COND_global_read_lock whenever COND_refresh
  is signalled by using the new function broadcast_refresh().
sql/sql_handler.cc:
  Bug#16986 - Deadlock condition with MyISAM tables
  Addendum fixes after changing the condition variable
  for the global read lock.
  Signal COND_global_read_lock whenever COND_refresh
  is signalled by using the new function broadcast_refresh().
sql/sql_insert.cc:
  Bug#16986 - Deadlock condition with MyISAM tables
  Addendum fixes after changing the condition variable
  for the global read lock.
  Removed global read lock handling from inside of 
  INSERT DELAYED. It is handled on a higher level now.
sql/sql_parse.cc:
  Bug#16986 - Deadlock condition with MyISAM tables
  Addendum fixes after changing the condition variable
  for the global read lock.
  Wait for the release of a global read lock if one is set
  before every operation that requires a write lock.
  But don't wait if locked tables exist already.
sql/sql_table.cc:
  Bug#16986 - Deadlock condition with MyISAM tables
  Addendum fixes after changing the condition variable
  for the global read lock.
  Removed global read lock handling from inside of 
  CREATE TABLE. It is handled on a higher level now.
  Signal COND_global_read_lock whenever COND_refresh
  is signalled by using the new function broadcast_refresh().

Review fixes


mysql-test/r/partition.result:
  Review fixes
mysql-test/t/partition.test:
  Review fixes
sql/sql_insert.cc:
  Fixed a bug

sql/handler.cc:
  Generating table maps from all locks that can be available: THD::extra_lock,
  THD::lock, and THD::locked_tables.
sql/sql_class.h:
  Adding member Open_tables:state::extra_lock to hold the extra lock used by select_create.
  Removing select_insert::lock.
sql/sql_insert.cc:
  Adding member Open_tables:state::extra_lock to hold the extra lock used by select_create.
  Removing select_insert::lock.

Review comments


mysql-test/t/partition.test:
  Changed procedure names ensured procedures were dropped
sql/ha_ndbcluster.h:
  Improved name of method
sql/ha_partition.h:
  Improved name of method
sql/handler.h:
  Improved name of method
  Removed deprecated constants
sql/item_sum.cc:
  Improved name of method
sql/sql_acl.cc:
  Improved name of method
sql/sql_insert.cc:
  Removed use of HA_WRITE_SKIP and introduced is_fatal_error instead
sql/sql_select.cc:
  Improved name of method
sql/sql_table.cc:
  Improved name of method
  Reintroduced dead code for future possible use
sql/sql_union.cc:
  Improved name of method
sql/sql_update.cc:
  Improved name of method

Need to flag when a copy is needed to not overwrite a create_info
object connected to the lex structure


sql/mysql_priv.h:
  Need to flag when a copy is needed to not overwrite a create_info
  object connected to the lex structure
sql/sql_insert.cc:
  Need to flag when a copy is needed to not overwrite a create_info
  object connected to the lex structure
sql/sql_parse.cc:
  Need to flag when a copy is needed to not overwrite a create_info
  object connected to the lex structure
sql/sql_table.cc:
  Need to flag when a copy is needed to not overwrite a create_info
  object connected to the lex structure

though unneeded". It's indeed unneeded, as slave is only interested in
permanent tables, and permanent tables don't depend on temporary tables
when in row-based binlogging mode. And other CREATE TEMPORARY TABLE
(referring no table or with LIKE) already don't write the CREATE to
binlog in row-based mode. 


mysql-test/r/rpl_row_create_table.result:
  result update (note that no lines appear in SHOW BINLOG EVENTS further below, which is how we
  see the bug is fixed)
mysql-test/t/rpl_row_create_table.test:
  testing if a CREATE TEMPORARY TABLE SELECT goes to binlog (it should not)
  when in row-based binlogging mode. A few lines after, there is a SHOW
  BINLOG EVENTS; before the bugfix it showed the CREATE TEMPORARY TABLE.
sql/sql_class.h:
  a method to access select_create::create_info from outside of select_create
  ("read-only" access). Making get_thd() "read-only" too.
sql/sql_insert.cc:
  The function (hook) which writes CREATE TABLE to binlog when in row-based
  binlogging mode, for CREATE TABLE SELECT, now does nothing if the table
  is temporary (as in row-based mode, temp tables are not replicated).
  This is consistent with CREATE TEMPORARY TABLE LIKE and
  CREATE TEMPORARY TABLE, which don't write any CREATE to binlog in row-based
  mode.

a fatal error. New handling of ignore error in place.


mysql-test/t/partition.test:
  New test case
sql/ha_ndbcluster.h:
  New handler method to check if error can be ignored
sql/ha_partition.h:
  New handler method to check if error can be ignored
sql/handler.h:
  New handler method to check if error can be ignored
sql/sql_acl.cc:
  Use new handler method
sql/sql_insert.cc:
  Use new handler method
sql/sql_table.cc:
  Use new handler method
sql/sql_union.cc:
  Use new handler method
sql/sql_update.cc:
  Use new handler method

INSERT triggers".

In cases when REPLACE was internally executed via update and table had
on update (on delete) triggers defined we exposed the fact that such
optimization used by callng on update (not calling on delete) triggers.
Such behavior contradicts our documentation which describes REPLACE as
INSERT with optional DELETE.

This fix just disables this optimization for tables with on delete triggers.
The optimization is still applied for tables which have on update but have
no on delete triggers, we just don't invoke on update triggers in this case
and thus don't expose information about optimization to user.

Also added test coverage for values returned by ROW_COUNT() function (and
thus for values returned by mysql_affected_rows()) for various forms of
INSERT.


mysql-test/r/insert.result:
  Added test for values returned by ROW_COUNT() function (and thus for values
  returned by mysql_affected_rows()) for various forms of INSERT. We didn't
  have coverage for this before and since this fix touches related code it is
  better to add it now.
mysql-test/r/trigger.result:
  Adjusted test after fixing bug#13479 "REPLACE activates UPDATE trigger, not
  the DELETE and INSERT triggers".
mysql-test/t/insert.test:
  Added test for values returned by ROW_COUNT() function (and thus for values
  returned by mysql_affected_rows()) for various forms of INSERT. We didn't
  have coverage for this before and since this fix touches related code it is
  better to add it now.
mysql-test/t/trigger.test:
  Adjusted test after fixing bug#13479 "REPLACE activates UPDATE trigger, not
  the DELETE and INSERT triggers".
sql/sql_insert.cc:
  write_record():
    We should not expose that internally we sometimes execute REPLACE
    via UPDATE instead of documented INSERT + DELETE pair. So we should not
    use this optimization for tables with on delete triggers. OTOH it is ok
    to use it for tables which have on update but have no on delete triggers,
    we just should not invoke on update triggers in this case.

The INSERT DELAYED should not maintain its own private auto-increment
counter, because this is assuming that other threads cannot insert
into the table while the INSERT DELAYED thread is inserting, which is
a wrong assumption.

So the start of processing of a batch of INSERT rows in the 
INSERT DELAYED thread must be treated as a start of a new statement
and cached next_insert_id must be cleared.


mysql-test/r/delayed.result:
  test suite for the bug
mysql-test/t/delayed.test:
  test suite for the bug
sql/sql_insert.cc:
  Reset auto-increment cacheing before processing
  the next batch of inserts in the handler thread

This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes

Changes that requires code changes in other code of other storage engines.
(Note that all changes are very straightforward and one should find all issues
by compiling a --debug build and fixing all compiler errors and all
asserts in field.cc while running the test suite),

- New optional handler function introduced: reset()
  This is called after every DML statement to make it easy for a handler to
  statement specific cleanups.
  (The only case it's not called is if force the file to be closed)

- handler::extra(HA_EXTRA_RESET) is removed. Code that was there before
  should be moved to handler::reset()

- table->read_set contains a bitmap over all columns that are needed
  in the query.  read_row() and similar functions only needs to read these
  columns
- table->write_set contains a bitmap over all columns that will be updated
  in the query. write_row() and update_row() only needs to update these
  columns.
  The above bitmaps should now be up to date in all context
  (including ALTER TABLE, filesort()).

  The handler is informed of any changes to the bitmap after
  fix_fields() by calling the virtual function
  handler::column_bitmaps_signal(). If the handler does caching of
  these bitmaps (instead of using table->read_set, table->write_set),
  it should redo the caching in this code. as the signal() may be sent
  several times, it's probably best to set a variable in the signal
  and redo the caching on read_row() / write_row() if the variable was
  set.

- Removed the read_set and write_set bitmap objects from the handler class

- Removed all column bit handling functions from the handler class.
  (Now one instead uses the normal bitmap functions in my_bitmap.c instead
  of handler dedicated bitmap functions)

- field->query_id is removed. One should instead instead check
  table->read_set and table->write_set if a field is used in the query.

- handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and
  handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now
  instead use table->read_set to check for which columns to retrieve.

- If a handler needs to call Field->val() or Field->store() on columns
  that are not used in the query, one should install a temporary
  all-columns-used map while doing so. For this, we provide the following
  functions:

  my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set);
  field->val();
  dbug_tmp_restore_column_map(table->read_set, old_map);

  and similar for the write map:

  my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set);
  field->val();
  dbug_tmp_restore_column_map(table->write_set, old_map);

  If this is not done, you will sooner or later hit a DBUG_ASSERT
  in the field store() / val() functions.
  (For not DBUG binaries, the dbug_tmp_restore_column_map() and
  dbug_tmp_restore_column_map() are inline dummy functions and should
  be optimized away be the compiler).

- If one needs to temporary set the column map for all binaries (and not
  just to avoid the DBUG_ASSERT() in the Field::store() / Field::val()
  methods) one should use the functions tmp_use_all_columns() and
  tmp_restore_column_map() instead of the above dbug_ variants.

- All 'status' fields in the handler base class (like records,
  data_file_length etc) are now stored in a 'stats' struct. This makes
  it easier to know what status variables are provided by the base
  handler.  This requires some trivial variable names in the extra()
  function.

- New virtual function handler::records().  This is called to optimize
  COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true.
  (stats.records is not supposed to be an exact value. It's only has to
  be 'reasonable enough' for the optimizer to be able to choose a good
  optimization path).

- Non virtual handler::init() function added for caching of virtual
  constants from engine.

- Removed has_transactions() virtual method. Now one should instead return
  HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support
  transactions.

- The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument
  that is to be used with 'new handler_name()' to allocate the handler
  in the right area.  The xxxx_create_handler() function is also
  responsible for any initialization of the object before returning.

  For example, one should change:

  static handler *myisam_create_handler(TABLE_SHARE *table)
  {
    return new ha_myisam(table);
  }

  ->

  static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root)
  {
    return new (mem_root) ha_myisam(table);
  }

- New optional virtual function: use_hidden_primary_key().
  This is called in case of an update/delete when
  (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined
  but we don't have a primary key. This allows the handler to take precisions
  in remembering any hidden primary key to able to update/delete any
  found row. The default handler marks all columns to be read.

- handler::table_flags() now returns a ulonglong (to allow for more flags).

- New/changed table_flags()
  - HA_HAS_RECORDS	    Set if ::records() is supported
  - HA_NO_TRANSACTIONS	    Set if engine doesn't support transactions
  - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE
                            Set if we should mark all primary key columns for
			    read when reading rows as part of a DELETE
			    statement. If there is no primary key,
			    all columns are marked for read.
  - HA_PARTIAL_COLUMN_READ  Set if engine will not read all columns in some
			    cases (based on table->read_set)
 - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS
   			    Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION.
 - HA_DUPP_POS              Renamed to HA_DUPLICATE_POS
 - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE
			    Set this if we should mark ALL key columns for
			    read when when reading rows as part of a DELETE
			    statement. In case of an update we will mark
			    all keys for read for which key part changed
			    value.
  - HA_STATS_RECORDS_IS_EXACT
			     Set this if stats.records is exact.
			     (This saves us some extra records() calls
			     when optimizing COUNT(*))
			    

- Removed table_flags()
  - HA_NOT_EXACT_COUNT     Now one should instead use HA_HAS_RECORDS if
			   handler::records() gives an exact count() and
			   HA_STATS_RECORDS_IS_EXACT if stats.records is exact.
  - HA_READ_RND_SAME	   Removed (no one supported this one)

- Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk()

- Renamed handler::dupp_pos to handler::dup_pos

- Removed not used variable handler::sortkey


Upper level handler changes:

- ha_reset() now does some overall checks and calls ::reset()
- ha_table_flags() added. This is a cached version of table_flags(). The
  cache is updated on engine creation time and updated on open.


MySQL level changes (not obvious from the above):

- DBUG_ASSERT() added to check that column usage matches what is set
  in the column usage bit maps. (This found a LOT of bugs in current
  column marking code).

- In 5.1 before, all used columns was marked in read_set and only updated
  columns was marked in write_set. Now we only mark columns for which we
  need a value in read_set.

- Column bitmaps are created in open_binary_frm() and open_table_from_share().
  (Before this was in table.cc)

- handler::table_flags() calls are replaced with handler::ha_table_flags()

- For calling field->val() you must have the corresponding bit set in
  table->read_set. For calling field->store() you must have the
  corresponding bit set in table->write_set. (There are asserts in
  all store()/val() functions to catch wrong usage)

- thd->set_query_id is renamed to thd->mark_used_columns and instead
  of setting this to an integer value, this has now the values:
  MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE
  Changed also all variables named 'set_query_id' to mark_used_columns.

- In filesort() we now inform the handler of exactly which columns are needed
  doing the sort and choosing the rows.

- The TABLE_SHARE object has a 'all_set' column bitmap one can use
  when one needs a column bitmap with all columns set.
  (This is used for table->use_all_columns() and other places)

- The TABLE object has 3 column bitmaps:
  - def_read_set     Default bitmap for columns to be read
  - def_write_set    Default bitmap for columns to be written
  - tmp_set          Can be used as a temporary bitmap when needed.
  The table object has also two pointer to bitmaps read_set and write_set
  that the handler should use to find out which columns are used in which way.

- count() optimization now calls handler::records() instead of using
  handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true).

- Added extra argument to Item::walk() to indicate if we should also
  traverse sub queries.

- Added TABLE parameter to cp_buffer_from_ref()

- Don't close tables created with CREATE ... SELECT but keep them in
  the table cache. (Faster usage of newly created tables).


New interfaces:

- table->clear_column_bitmaps() to initialize the bitmaps for tables
  at start of new statements.

- table->column_bitmaps_set() to set up new column bitmaps and signal
  the handler about this.

- table->column_bitmaps_set_no_signal() for some few cases where we need
  to setup new column bitmaps but don't signal the handler (as the handler
  has already been signaled about these before). Used for the momement
  only in opt_range.cc when doing ROR scans.

- table->use_all_columns() to install a bitmap where all columns are marked
  as use in the read and the write set.

- table->default_column_bitmaps() to install the normal read and write
  column bitmaps, but not signaling the handler about this.
  This is mainly used when creating TABLE instances.

- table->mark_columns_needed_for_delete(),
  table->mark_columns_needed_for_delete() and
  table->mark_columns_needed_for_insert() to allow us to put additional
  columns in column usage maps if handler so requires.
  (The handler indicates what it neads in handler->table_flags())

- table->prepare_for_position() to allow us to tell handler that it
  needs to read primary key parts to be able to store them in
  future table->position() calls.
  (This replaces the table->file->ha_retrieve_all_pk function)

- table->mark_auto_increment_column() to tell handler are going to update
  columns part of any auto_increment key.

- table->mark_columns_used_by_index() to mark all columns that is part of
  an index.  It will also send extra(HA_EXTRA_KEYREAD) to handler to allow
  it to quickly know that it only needs to read colums that are part
  of the key.  (The handler can also use the column map for detecting this,
  but simpler/faster handler can just monitor the extra() call).

- table->mark_columns_used_by_index_no_reset() to in addition to other columns,
  also mark all columns that is used by the given key.

- table->restore_column_maps_after_mark_index() to restore to default
  column maps after a call to table->mark_columns_used_by_index().

- New item function register_field_in_read_map(), for marking used columns
  in table->read_map. Used by filesort() to mark all used columns

- Maintain in TABLE->merge_keys set of all keys that are used in query.
  (Simplices some optimization loops)

- Maintain Field->part_of_key_not_clustered which is like Field->part_of_key
  but the field in the clustered key is not assumed to be part of all index.
  (used in opt_range.cc for faster loops)

-  dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map()
   tmp_use_all_columns() and tmp_restore_column_map() functions to temporally
   mark all columns as usable.  The 'dbug_' version is primarily intended
   inside a handler when it wants to just call Field:store() & Field::val()
   functions, but don't need the column maps set for any other usage.
   (ie:: bitmap_is_set() is never called)

- We can't use compare_records() to skip updates for handlers that returns
  a partial column set and the read_set doesn't cover all columns in the
  write set. The reason for this is that if we have a column marked only for
  write we can't in the MySQL level know if the value changed or not.
  The reason this worked before was that MySQL marked all to be written
  columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden
  bug'.

- open_table_from_share() does not anymore setup temporary MEM_ROOT
  object as a thread specific variable for the handler. Instead we
  send the to-be-used MEMROOT to get_new_handler().
  (Simpler, faster code)



Bugs fixed:

- Column marking was not done correctly in a lot of cases.
  (ALTER TABLE, when using triggers, auto_increment fields etc)
  (Could potentially result in wrong values inserted in table handlers
  relying on that the old column maps or field->set_query_id was correct)
  Especially when it comes to triggers, there may be cases where the
  old code would cause lost/wrong values for NDB and/or InnoDB tables.

- Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags:
  OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG.
  This allowed me to remove some wrong warnings about:
  "Some non-transactional changed tables couldn't be rolled back"

- Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset
  (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose
  some warnings about
  "Some non-transactional changed tables couldn't be rolled back")

- Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table()
  which could cause delete_table to report random failures.

- Fixed core dumps for some tests when running with --debug

- Added missing FN_LIBCHAR in mysql_rm_tmp_tables()
  (This has probably caused us to not properly remove temporary files after
  crash)

- slow_logs was not properly initialized, which could maybe cause
  extra/lost entries in slow log.

- If we get an duplicate row on insert, change column map to read and
  write all columns while retrying the operation. This is required by
  the definition of REPLACE and also ensures that fields that are only
  part of UPDATE are properly handled.  This fixed a bug in NDB and
  REPLACE where REPLACE wrongly copied some column values from the replaced
  row.

- For table handler that doesn't support NULL in keys, we would give an error
  when creating a primary key with NULL fields, even after the fields has been
  automaticly converted to NOT NULL.

- Creating a primary key on a SPATIAL key, would fail if field was not
  declared as NOT NULL.


Cleanups:

- Removed not used condition argument to setup_tables

- Removed not needed item function reset_query_id_processor().

- Field->add_index is removed. Now this is instead maintained in
  (field->flags & FIELD_IN_ADD_INDEX)

- Field->fieldnr is removed (use field->field_index instead)

- New argument to filesort() to indicate that it should return a set of
  row pointers (not used columns). This allowed me to remove some references
  to sql_command in filesort and should also enable us to return column
  results in some cases where we couldn't before.

- Changed column bitmap handling in opt_range.cc to be aligned with TABLE
  bitmap, which allowed me to use bitmap functions instead of looping over
  all fields to create some needed bitmaps. (Faster and smaller code)

- Broke up found too long lines

- Moved some variable declaration at start of function for better code
  readability.

- Removed some not used arguments from functions.
  (setup_fields(), mysql_prepare_insert_check_table())

- setup_fields() now takes an enum instead of an int for marking columns
   usage.

- For internal temporary tables, use handler::write_row(),
  handler::delete_row() and handler::update_row() instead of
  handler::ha_xxxx() for faster execution.

- Changed some constants to enum's and define's.

- Using separate column read and write sets allows for easier checking
  of timestamp field was set by statement.

- Remove calls to free_io_cache() as this is now done automaticly in ha_reset()

- Don't build table->normalized_path as this is now identical to table->path
  (after bar's fixes to convert filenames)

- Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to
  do comparision with the 'convert-dbug-for-diff' tool.


Things left to do in 5.1:

- We wrongly log failed CREATE TABLE ... SELECT in some cases when using
  row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result)
  Mats has promised to look into this.

- Test that my fix for CREATE TABLE ... SELECT is indeed correct.
  (I added several test cases for this, but in this case it's better that
  someone else also tests this throughly).
  Lars has promosed to do this.


BitKeeper/etc/ignore:
  added mysys/test_bitmap
include/base64.h:
  Removed my_global.h, as this must be included first in any program
include/heap.h:
  Added heap_reset() (Required by new handler interface)
include/my_base.h:
  Removed HA_EXTRA_RESET. MySQL will now call ::reset() instead of ::extra(HA_EXTRA_RESET).
  HA_EXTRA_RETRIVE_ALL_COLS and HA_EXTRA_RETRIVE_PRIMARY key are deleted as the column bitmaps makes these unnecessary
include/my_bitmap.h:
  Remove my_pthread.h (should be included at upper level)
  Introduced my_bitmap_map typedef to make it the bitmap handling more like a black box
  Added bitmap_is_overlapping(), bitmap_test_and_clear(), bitmap_copy() and bitmap_cmp()
  Made bitmap_set_bit(), bitmap_flip_bit(), bitmap_clear_bit() return void
include/myisam.h:
  Added mi_reset() (Required by new handler interface)
include/myisammrg.h:
  Added myrg_reset() (Required by new handler interface)
include/mysql_com.h:
  Added flag FIELD_IN_ADD_INDEX to be able to remove Field->add_index
mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test:
  Added testing of CREATE ... SELECT in a mixed environment
  (This found some bugs that Mats is going to fix shortly)
mysql-test/install_test_db.sh:
  Simplify ldata usage
  Added --tmpdir=. option to mysqld bootstrap (Removed some warnings when TMPDIR was wrongly set)
mysql-test/mysql-test-run.pl:
  Added --tmpdir=. to bootstrap
mysql-test/mysql-test-run.sh:
  Use copy instead of INSTALL_DB for master and slave databases.
  (Speeds up startup time a lot!)
  Remove snapshot directories at startup (removes some strange warnings)
mysql-test/r/binlog_row_mix_innodb_myisam.result:
  Added testing of CREATE ... SELECT in a mixed environment
  (This found some bugs that Mats is going to fix shortly)
mysql-test/r/binlog_stm_mix_innodb_myisam.result:
  Added testing of CREATE ... SELECT in a mixed environment
mysql-test/r/create.result:
  Some extra tests of warnings and number of tables opened by CREATE ... SELECT
mysql-test/r/federated.result:
  Drop some left over tables
  Added testing of multiple table update and multiple table delete (with and without keys)
mysql-test/r/func_gconcat.result:
  Enable some disabled tests (converted them slightly to be predictable)
mysql-test/r/func_time.result:
  Added drop of test function
mysql-test/r/innodb_mysql.result:
  Added tests for CREATE ... SELECT
mysql-test/r/insert.result:
  More tests
  Added testing of duplicate columns in insert
mysql-test/r/loaddata.result:
  Added testing LOAD DATA ... SET ...
mysql-test/r/multi_update.result:
  Test multi updates and deletes using primary key and without
mysql-test/r/ndb_index_unique.result:
  Better error message
mysql-test/r/ndb_replace.result:
  New correct result after fixing REPLACE handling with NDB
mysql-test/r/rpl_ddl.result:
  Now we don't get these (wrong) warnings anymore
mysql-test/r/view_grant.result:
  Drop used views
mysql-test/t/create.test:
  Some extra tests of warnings and number of tables opened by CREATE ... SELECT
mysql-test/t/federated.test:
  Drop some left over tables
  Added testing of multiple table update and multiple table delete (with and without keys)
mysql-test/t/func_gconcat.test:
  Enable some disabled tests (converted them slightly to be predictable)
mysql-test/t/func_time.test:
  Added drop of test function
mysql-test/t/innodb_mysql.test:
  Added tests for CREATE ... SELECT
mysql-test/t/insert.test:
  More tests
  Added testing of duplicate columns in insert
mysql-test/t/loaddata.test:
  Added testing LOAD DATA ... SET ...
mysql-test/t/multi_update.test:
  Test multi updates and deletes using primary key and without
mysql-test/t/view_grant.test:
  Drop used views
mysql-test/valgrind.supp:
  Added supression of not needed warnings when printing stack trace
mysys/base64.c:
  Include my_global.h first
mysys/my_bitmap.c:
  Added bitmap_is_overlapping(), bitmap_test_and_clear() and bitmap_copy()
  Changed logic of bitmap handling to be a bit more efficent (Did this together with Mikael Ronström)
  Now the 'extra, not used bits' in the bitmap are assumed to have a 'random value' and the bitmap functions are free to change them whenever needed.
  Changed how mutex is allocated to make 'bitmap_free()' function simpler.
mysys/thr_lock.c:
  Added 0x before thread pointers (for easier comparison of DBUG traces)
sql/event.cc:
  Ensure 'use_all_columns()' is used for event tables
  Don't print warning that event table is damaged if it doesn't exists.
sql/field.cc:
  Added ASSERT_COLUMN_MARKED_FOR_WRITE in all store() methods and ASSERT_COLUMN_MARKED_FOR_READ in all val() methods to catch wrong setting if table->read_set and table->write_set
  (Rest of changes are only indentation cleanups)
sql/field.h:
  Removed Field->query_id (replaced by table->read_set and table->write_set)
  Removed Field->fieldnr (use Field->field_index instead)
  Removed Field->add_index (Use Field->flags instead)
  Add Field->part_of_key_not_clustered (for usage in opt_range.cc)
sql/filesort.cc:
  Added paramater sort_postion to filesort() to force sorting by position instead of storing all fields in the result set.
  This allowed me to remove checking of sql_command.
  Create a temporary column bitmap for fields that are used by the sorting process.
  Use column bitmaps instead of query_id
sql/ha_berkeley.cc:
  Update to 'newer' table handler interface
sql/ha_berkeley.h:
  Update to 'newer' table handler interface
sql/ha_federated.cc:
  Update to 'newer' table handler interface
  Only read columns that are needed from remote server.
  In case of eq ranges, don't generate two conditions in the WHERE clause
  (this can still be optimized, but would require a bigger code change)
  Use 'simpler to use' XXXX_LEN' macros
  A bit simpler logic in ::write_row() when creating statements.
  In update, only include test of fields actually read.
  (This greatly simplifies the queries sent by the federated engine)
  Similar changes done for delete_row()
sql/ha_federated.h:
  Update to 'newer' table handler interface
  Changed XXX_LEN macros to use sizeof(...)-1, to simplify usage in ha_federated.cc
  Added HA_PRIMARY_KEY_REQUIRED_FOR_DELETE to tell MySQL to read all primary key columns in case of DELETE
sql/ha_heap.cc:
  Update to 'newer' table handler interface
sql/ha_heap.h:
  Update to 'newer' table handler interface
sql/ha_innodb.cc:
  Update to 'newer' table handler interface
  - Update innobase_create_handler() to new interface
  - Removed HA_NOT_EXACT_COUNT (not needed)
  - Renamed HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION.
  - Prefixed base status variables with 'stats'
  - Use table column bitmaps instead of ha_get_bit_in_read_set()
  - Added ::reset(), with code from ::extra(HA_EXTRA_RESET)
  - Removed HA_EXTRA_RETRIVE_ALL_COLS and HA_EXTRA_RETRIEVE_PRIMARY_KEY as
    the table->read_set and table->write_set bitmaps now are accurate
sql/ha_innodb.h:
  Update to 'newer' table handler interface
  - table_flags are now ulonglong
  - Added reset() method
  - Removed not needed ha_retrieve_all_cols() and ha_retrieve_all_pk() columns.
  - Made build_template() a class function to be able to easier access class variables
sql/ha_myisam.cc:
  Update to 'newer' table handler interface
sql/ha_myisam.h:
  Update to 'newer' table handler interface
sql/ha_myisammrg.cc:
  Update to 'newer' table handler interface
sql/ha_myisammrg.h:
  Update to 'newer' table handler interface
sql/ha_ndbcluster.cc:
  Update to 'newer' table handler interface
  Fixed use_blob_value() to be accurate
  In ::complemented_read() we have to check both the read and write bitmap as the old code did mark all changed columns also in the read map
  Correct dumping of field data with DBUG_DUMP
  Prefix addresses in DBUG_PRINT with 0x
  Fixed usage of not initialized memory
  Update to use field->flags & FIELD_IN_ADD_INDEX instead of field->add_index.
sql/ha_ndbcluster.h:
  Update to 'newer' table handler interface
sql/ha_ndbcluster_binlog.cc:
  Mark usage of all columns in ndbcluster binlog tables
  false -> FALSE, true -> TRUE
  Use table->s->all_set instead of creating a temporary bitmap.
sql/ha_partition.cc:
  Update to 'newer' table handler interface
  Added memroot to initialise_partitions() and related functions to get faster memory allocation.
  partition_create_handler() is now responsible for initialisation of the partition object
  Some trivial optimizations and indentation fixes
  Ensure that table_flags() are up to date
  Removed documentation for removed HA_EXTRA flags
  Fixed 'strange' usage of m_file[i] in new_handlers_from_part_info()that worked in current code 'by chance'
sql/ha_partition.h:
  Update to 'newer' table handler interface
sql/handler.cc:
  create_xxx handler now takes MEMROOT as an argument to simplify memory allocation.
  Much simpler get_new_handler()
  (Initialization of the object is now handled by the create method for the engine)
  Moved all allocation of bitmap handling to the TABLE object (in table.cc)
  Added column_bitmaps_signal() to signal column usage changes.
  Changed binlog_log_row() to use the exiusting all_set bitmap in the table object.
  Added ha_reset() function to test that the file object is ok at end of statement and call handler::reset()
  Added use_hidden_primary_key() to signal handler that we we are going to read and update + delete the row and the handler should thus remember the position for the row
sql/handler.h:
  Added HA_NO_TRANSACTIONS, HA_PARTIAL_COLUMN_READ, HA_REQUIRES_KEY_COLUMNS_FOR_DELETE,HA_PRIMARY_KEY_REQUIRED_FOR_DELETE and HA_HAS_RECORDS
  Removed HA_NOT_EXACT_COUNT, HA_READ_RND_SAME
  HA_DUPP_POS -> HA_DUPLICATE_POS
  HA_NOT_EXACT_COUNT replaced by HA_STATS_RECORDS_IS_EXACT, HA_HAS_RECORDS and records()
  HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION
  Added future row type 'ROW_TYPE_PAGES'
  Added MEM_ROOT to handlerton 'create' function
  Added ha_statistics, a structure for all status variable in the base handler class.
  Moved all status variables in the handler class into a stats structs to improve readability.
  ha_table_flags() is now a cached (not virtual) version of table_flags()
  reset() doesn't anymore call extra(HA_EXTRA_RESET) but is a function of it's own.
  Renamed dupp_ref to dup_ref
  Renamed not used handler::sortkey
  Moved read_set and write_set to TABLE structure
  handler::init() function added for cacheing of virtual constants from engine.
sql/item.cc:
  Added register_field_in_read_map() for marking used columns in expression.
  This is used by filesort() for creating an optimal column bitmap while retrieving columns for sorting.
  Initalize value.cs_info.character_set_client to fix core dump bug with --debug
  set_query_id -> mark_used_columns
  Mark used columns in read_set OR write_set.
sql/item.h:
  Removed reset_query_id_processor() as it's not needed anymore.
  Added register_field_in_read_map()
  Added extra argument to Item::walk() to indicate if we should also
  traverse sub queries.
sql/item_cmpfunc.cc:
  Temporary mark used columns to be read/writable
  Update Item::walk to new interface
sql/item_cmpfunc.h:
  Added extra argument to Item::walk() to indicate if we should also traverse sub queries.
sql/item_func.cc:
  Update Item::walk() to new interface
  table_flags() -> ha_table_flags()
sql/item_func.h:
  Update Item::walk() to new interface
sql/item_row.cc:
  Update Item::walk() to new interface
sql/item_row.h:
  Update Item::walk() to new interface
sql/item_strfunc.h:
  Update Item::walk() to new interface
sql/item_subselect.cc:
  Added Item_subselect::walk()
  (It was a bug it was missing before. Not sure what kind of bugs this could have caused)
sql/item_subselect.h:
  Update Item::walk() to new interface
sql/item_sum.cc:
  Update Item::walk() to new interface
  Updates for new handler interace
sql/item_sum.h:
  Update Item::walk() to new interface
sql/key.cc:
  Updates for new handler interace
sql/log.cc:
  Mark all columns used for log tables
  Split options flag
  Ensured that second argument to trans_register_ha is a bool
sql/log_event.cc:
  Fixed comments to be withing 79 characters
  Use OPTION_KEEP_LOG instead of OPTION_STATUS_NO_TRANS_UPDATE to remove wrong warnings
  Updates for new handler interface
  Use 0x%lx instead of %p (portability problem)
sql/mysql_priv.h:
  Added OPTION_KEEP_LOG to indicate that we should replicate the binlog even on rollback
  Removed not used 'conds' argument to setup_tables
sql/mysqld.cc:
  Indentation fixes and removed old comment
sql/opt_range.cc:
  Update to new handler and bitmap interface.
  Fixed calls to cp_buffer_from_ref() and walk() (new argument).
  Create new temporary bitmaps for ror scans.
  (Needed because of handler changes and to get more accurate column bitmaps than before)
  Remove not needed file->ha_reset() call before file->close().
  Some trivial optimization and indentation fixes.
  Use Field->part_of_key_not_clustered() to check if field is part of a key, instead of looping over all key parts.
  
  Added flag 'in_ror_merged_scan' to allow ::get_next() to know that we need a special column bitmap to only fetch pointer to record.
  This is needed because ror scan uses the same TABLE object but different file objects, which creates problem for the column bitmap handling.
  (This is a temporary solution. A better one would be to allocate an own TABLE object for ROR scans)
  
  Optimized bitmap handling in ror scans:
  - Start bitmap at position 0, not 1
  - Use same bitmap size as in TABLE
  - Use table->read_set and table->write_set to create column bitmaps instead of looping over all fields in table
sql/opt_range.h:
  Added 'in_ror_merged_scan' to indicate if we are doing a ROR scan
  Added temporary column bitmaps used in ROR scans
sql/opt_sum.cc:
  Added get_ext_record_count() which is used in COUNT() optimization if handler has HA_HAS_RECORDS
  Note that we don't call this if handler has HA_STATS_RECORDS_IS_EXACT set.
sql/protocol.cc:
  We need to mark columns as readable in ::store() as we sometimes return default value for fields to the user
sql/records.cc:
  Updates for new handler interface
sql/set_var.cc:
  Handle splitting OPTION_STATUS_NO_TRANS_UPDATE to two flags
sql/share/errmsg.txt:
  Fixed wrong
sql/sp.cc:
  Mark that we are using all columns for the proc table
  Update call to setup_tables() to use new prototype
sql/sp_head.cc:
  Removed QQ comment
sql/spatial.cc:
  Removed wrong QQ comment
sql/sql_acl.cc:
  Mark that we need all columns for acl tables
  Supply memroot to some 'new' calls.
  Indentation fixes
sql/sql_base.cc:
  set_query_id removed
  Ensure we call ha_reset() at end of each statement
  Mark read columns in read_set and changed columns in write_set (Before all columns was marked in read set)
  Fixed marking of some columns that was not proplerly marked before
  Maintain in TABLE->merge_keys set of all keys that are used in some way
  Removed not used 'conds' argument from setup_tables()
  Remove not used setting of 'dupp_field' in insert_fields()
  Added missing FN_LIBCHAR in mysql_rm_tmp_tables()
  (This has probably caused us to not properly remove temporary files after crash)
sql/sql_bitmap.h:
  Added is_overlapping()
sql/sql_class.cc:
  Slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log.
  set_query_id -> mark_used_columns
  Simpler variable usage in pack_row()  (cleanup)
  Moved some variable declartion at start of function for better code readability
sql/sql_class.h:
  Added enum_mark_columns
  Updated comments
  Renamed dupp_field -> dup_field
  Added virtual function 'can_rollback_data()' to select_insert() to be used in CREATE ... SELECT to optimize use of OPTION_STATUS_NO_TRANS_UPDATE.
  (This fixes a bug in CREATE ... SELECT where we did give wrong warnings when using non transacational tables)
sql/sql_delete.cc:
  Updates to new handler interface
  Call table->mark_columns_needed_for_delete() to allow us to put additional columns in column usage maps if handler so requires.
  Call table->prepare_for_position() to tell handler that we are going to call ha_position().
  Removed call to free_io_cache(). (io_cache is now removed in ha_reset()).
  Fixed calls to setup_tables()
sql/sql_do.cc:
  Update call to setup_fields()
sql/sql_handler.cc:
  Tell handler tables to always read all columns.
  Use temporary column map when storing value in field for later index usage
sql/sql_help.cc:
  Makr all used fields to be read
  Update call to setup_fields()
sql/sql_insert.cc:
  Tell handler we are going to update the auto_increment column
  dupp_field -> dup_field
  Set column usage bits for timestamp field.
  Call table->mark_columns_needed_for_insert() and table->mark_auto_increment_column()
  Removed not used argument from mysql_prepare_insert_check_table().
  
  If we get an duplicate row on insert, change column map to read and write all columns while retrying the operatation.
  This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled.
  This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row.
  
  Setup new bitmaps for delayed insert rows
  Remove reseting of next_number_fields as it will be reset on next call to handler_insert()
  
  Fixed usage of thd->options and OPTION_STATUS_NO_TRANS_UPDATE.
  The issue was that one should not to reset this flag as it may be set by a previous statement.
  The way it was now used caused us to loose some warnings and get other wrong warnings when using non transactional tables mixed with transactional.
  I fixed it by introducing 'select_insert::can_rollback_data' to inform send_error() that the given statement can be rolled back (which in case of CREATE TABLE can always be done)
  Don't close tables created with CREATE ... SELECT but keep them in the table cache.
  
  Moved out MY_HOOKS from inside function (better readability)
sql/sql_load.cc:
  Update to use new handler and column marking interface
  Update using setup_tables()
sql/sql_olap.cc:
  Update calls to setup_tables
  Use enums instead of constants to setup_fields()
sql/sql_parse.cc:
  Handle OPTION_KEEP_LOG:
  - Set it on CREATE TEMPORARY TABLE / DROP TABLE
  - Reset it when OPTION_STATUS_NO_TRANS_UPDATE is reset
  - Don't set it for CREATE ... SELECT (this is handled in select_create class)
  Remove reseting of OPTION_STATUS_NO_TRANS_UPDATE in begin_trans() as this should already be reset.
  If in autocommit mode, reset OPTION_KEEP_LOG and OPTION_STATUS_NO_TRANS_UPDATE to not give warnings in future commands
sql/sql_partition.cc:
  Update walk() usage
  Trivial indentation fixes
sql/sql_plugin.cc:
  Mark all columns as used for plugins
sql/sql_prepare.cc:
  Added assert to find out hidden bugs in character_set_client (got an error in debug binary when this not set correctly)
  Updates for new handler interface
  Update calls to setup_fields()
sql/sql_repl.cc:
  Indentation fixes
sql/sql_select.cc:
  Update call to setup_tables() and setup_fields()
  Remove some old disabled code
  Update to new hadler interface
  Indentation cleanups
  Added column bitmaps for temporary tables.
  Remove updating of the removed slots in the Field class
  Added TABLE argument to cp_buffer_from_ref() (To be able to install temporary column maps)
  For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution.
sql/sql_select.h:
  Indentaition fixes.
  Install temporary column usage maps when needed
  Added TABLE element to cp_buffer_from_ref()
sql/sql_show.cc:
  Update to new handler interface
  Mark all columns used for internal tables.
  Style fixes.
  Added support for 'future' ROW_TYPE_PAGES.
  Don't allocate TMP_TABLE_PARAM with calloc. The 'init()' function will initialize the structure properly.
sql/sql_table.cc:
  Update to new handler interface
  Simple my_snprintf -> strmake()
  Changed some constants to defines
  Don't test for NULL in primary key (as we a couple of line above force the PRIMARY KEY to be NOT NULL)
  Change field->add_index to use field->flags & FIELD_IN_ADD_INDEX
  Mark all columns as used for ALTER TABLE
  Style fixes
  Update call to filesort()
sql/sql_trigger.h:
  Added friend functions to be able to test if triggers exists for table we are going to insert/update or delete in.
sql/sql_udf.cc:
  Mark all columns as used for udf system table.
sql/sql_union.cc:
  Update call to walk()
  Update to new handler interface
sql/sql_update.cc:
  Remove query_id argument from compare_record()
  Use column bitmaps instead of query_id.
  We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set, because compare_record() can't in this case know if a not read column changed value.
  Update call to setup_fields()
  Using separate column read and write sets allows for easier checking of timestamp field was set by statement.
  Removed call to free_io_cache() as this is now done in ha_reset()
  Call table->mark_columns_needed_for_update() and table->prepare_for_position()
  Style fixes
sql/sql_view.cc:
  Style fixes
sql/table.cc:
  Remove implicitely include 'errno.h'
  Remove code for building normalized path, as this is now identical to 'path'
  Remove field->fieldnr
  Added update of field->part_of_key_not_clustered()
  Create column bitmaps in TABLE and TABLE_SHARE
  Don't setup a temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler()
  Update to new handler interface
  Update call to walk()
  Added new functions:
  - st_table::clear_column_bitmaps()
  - st_table::prepare_for_position()
  - st_table::mark_columns_used_by_index()
  - st_table::restore_column_maps_after_mark_index()
  - st_table::mark_columns_used_by_index_no_reset()
  - st_table::mark_auto_increment_column()
  - st_table::mark_columns_needed_for_delete()
  - st_table::mark_columns_needed_for_update()
  - st_table::mark_columns_needed_for_insert()
sql/table.h:
  Moved column usage bitmaps from handler to TABLE
  Added to TABLE_SHARE all_set and column_bitmap_size
  Added to TABLE merge_keys, bitmap_init_values, def_read_set, def_write_set, tmp_set, read_set and write_set.
  Declared all new table column bitmap functions
  Added TABLE functions column_bitmaps_set(), column_bitmaps_set_no_signal(), use_all_columns() and default_column_bitmaps()
  Added functions: tmp_use_all_columns() and tmp_restore_column_map() to temporarly switch column bitmaps
  Added functions: dbug_tmp_use_all_columns() and dbug_tmp_restore_column_map() to temporarly switch column bitmaps to avoid asserts in Field::store() and Field::val().
sql/tztime.cc:
  Mark all columns as used for timezone tables
storage/archive/ha_archive.cc:
  Update to new handler interface
storage/archive/ha_archive.h:
  Update to new handler interface
storage/blackhole/ha_blackhole.cc:
  Update to new handler interface
storage/blackhole/ha_blackhole.h:
  Update to new handler interface
  removed not needed flag HA_DUPP_POS
storage/csv/ha_tina.cc:
  Update to new handler interface
storage/csv/ha_tina.h:
  Update to new handler interface
storage/example/ha_example.cc:
  Update to new handler interface
storage/example/ha_example.h:
  Update to new handler interface
storage/heap/hp_extra.c:
  Added heap_reset() (Required by new handler interface)
storage/heap/hp_test2.c:
  Use heap_reset()
storage/myisam/ft_boolean_search.c:
  Fixed compiler warning
storage/myisam/mi_extra.c:
  Added mi_reset() (Required by new handler interface)
storage/myisam/mi_search.c:
  Fixed DBUG_PRINT messages to use 0x%lx instead of %lx
storage/myisam/mi_test2.c:
  Use mi_reset()
storage/myisam/myisampack.c:
  Use mi_reset()
storage/myisammrg/myrg_extra.c:
  Added myrg_reset() (Required by new handler interface)
unittest/mysys/base64.t.c:
  Include my_global.h
  Don't include implictely include file 'stdlib.h'

New prototype for get_auto_increment() (but new arguments not yet used), to be able
to reserve a finite interval of auto_increment values from cooperating engines.
A hint on how many values to reserve is found in handler::estimation_rows_to_insert,
filled by ha_start_bulk_insert(), new wrapper around start_bulk_insert().
NOTE: this patch changes nothing, for all engines. But it makes the API ready for those
engines which will want to do reservation.
More csets will come to complete WL#3146.


sql/ha_berkeley.cc:
  update to new prototype of get_auto_increment
sql/ha_berkeley.h:
  update to new prototype of get_auto_increment
sql/ha_heap.cc:
  update to new prototype of get_auto_increment
sql/ha_heap.h:
  update to new prototype of get_auto_increment
sql/ha_innodb.cc:
  update to new prototype of get_auto_increment
sql/ha_innodb.h:
  update to new prototype of get_auto_increment
sql/ha_myisam.cc:
  update to new prototype of get_auto_increment
sql/ha_myisam.h:
  update to new prototype of get_auto_increment
sql/ha_ndbcluster.cc:
  update to new prototype of get_auto_increment
sql/ha_ndbcluster.h:
  update to new prototype of get_auto_increment
sql/ha_partition.cc:
  update to new prototype of get_auto_increment
sql/ha_partition.h:
  update to new prototype of get_auto_increment
sql/handler.cc:
  new prototype of get_auto_increment, comments, preparation for when the MySQL layer is capable
  of getting finite auto_increment intervals from cooperating engines.
sql/handler.h:
  a wrapper around start_bulk_insert(): ha_bulk_insert(), which stores the argument (number
  of rows expected for insertion) into a member of handler: estimation_rows_to_insert.
  This member will soon be used to decide how much auto_increment values we want to reserve
  from cooperating engines.
  New prototype for get_auto_increment, preparing for cooperating engines to reserve finite auto_increment
  intervals.
  release_auto_increment() will be used by the MySQL layer to inform the engine that it has
  not used all of the interval (engine can reclaim it).
sql/log.cc:
  note for the future
sql/log_event.cc:
  name of wrapper
sql/sql_insert.cc:
  name of wrapper. When done with inserting, return unused auto_inc values to engine.
sql/sql_load.cc:
  name of wrapper. When done with inserting, return unused auto_inc values to engine.
sql/sql_select.cc:
  name of wrapper
sql/sql_table.cc:
  name of wrapper
storage/archive/ha_archive.cc:
  update to new prototype of get_auto_increment.
  Archive reserves only one value (Archive's god - Brian - told me that's the truth).
storage/archive/ha_archive.h:
  update to new prototype of get_auto_increment()

Switched to writing out table maps for tables that are locked when
the first row in a statement is seen.


mysql-test/include/master-slave.inc:
  Moved code to reset master and slave into separate file.
mysql-test/r/binlog_row_blackhole.result:
  Result change
mysql-test/r/binlog_row_mix_innodb_myisam.result:
  Result change
mysql-test/r/ndb_binlog_ignore_db.result:
  Result change
mysql-test/r/rpl_ndb_charset.result:
  Result change
mysql-test/r/rpl_row_basic_11bugs.result:
  Result change
mysql-test/r/rpl_row_charset.result:
  Result change
mysql-test/r/rpl_row_create_table.result:
  Result change
mysql-test/t/rpl_row_basic_11bugs.test:
  Added test to check that no events are generated when no rows are changed.
mysql-test/t/rpl_row_create_table.test:
  Master log position changed
sql/handler.cc:
  Adding function write_locked_table_maps() that will write table maps for all
  tables locked for write.
  Using "table->in_use" instead of "current_thd" since tables are now locked
  when the function is called.
  Removing old code to write table map.
sql/log_event.cc:
  Added assertion
sql/sql_class.cc:
  Removing code to write "dummy termination event".
sql/sql_class.h:
  Adding getter for binlog_table_maps.
sql/sql_insert.cc:
  Setting thd->lock before calling write_record for the execution of
  CREATE-SELECT and INSERT-SELECT since they keep multiple locks in the
  air at the same time.
mysql-test/include/master-slave-reset.inc:
  New BitKeeper file ``mysql-test/include/master-slave-reset.inc''

The check for view security was lacking several points :
1. Check with the right set of permissions : for each table ref that
participates in a view there were the right credentials to use in it's
security_ctx member, but these weren't used for checking the credentials.
This makes hard enforcing the SQL SECURITY DEFINER|INVOKER property
consistently.
2. Because of the above the security checking for views was just ruled out
in explicit ways in several places.
3. The security was checked only for the columns of the tables that are
brought into the query from a view. So if there is no column reference
outside of the view definition it was not detecting the lack of access to
the tables in the view in SQL SECURITY INVOKER mode.

The fix below tries to fix the above 3 points.


mysql-test/r/grant.result:
  removed nondeterminism (unspecified order) in some test output
mysql-test/r/view_grant.result:
  Somewhat extended test case for the bug and similar queries.
mysql-test/t/grant.test:
  removed nondeterminism (unspecified order) in some test output
mysql-test/t/view_grant.test:
  Somewhat extended test case for the bug and similar queries.
sql/mysql_priv.h:
  A wrapper for setup_tables that also checks access to the tables
sql/sql_acl.cc:
  removed artificial security check stop and used the table ref's credentials.
sql/sql_base.cc:
  a wrapper for setup_tables to check access to the tables
sql/sql_delete.cc:
  wrapper called.
sql/sql_insert.cc:
  wrapper called
sql/sql_load.cc:
  wrapper called
sql/sql_parse.cc:
  wrapper called and artificial check stop removed
sql/sql_select.cc:
  wrapper called
sql/sql_update.cc:
  wrapper called
sql/table.cc:
  Mask table access to the view error as well.

Remove dflt_field from field structure as this was only needed when createing temporary table and I found another soultion that doesn't increase the size of the field structure for all table instances. (Better fix for bug #19089)
Fixed compiler warnings
Fixed valgrind warning in Item_date_add_intervall::eq. (Recoding of bugfix #19490)


sql/field.cc:
  remove dflt_field from field structure (not needed)
  Simple cleanup of code that been copied elsewhere
sql/field.h:
  remove dflt_field from field structure (not needed)
sql/item.h:
  Removed compiler warnings
sql/item_timefunc.cc:
  Fixed Item_date_add_intervall::eq
  The problem was that when we call 'eq' 'this' is not fixed, which means we can't call const_item() or a value function.
  I fixed this so that we check eq for all arguments and that the sign and type are identical.
  (The original code gave a 'accessing uninitialized data' in valgrind.
sql/mysql_priv.h:
  Added default fields to create_tmp_field
sql/sql_insert.cc:
  New default_field parameter to create_tmp_field()
sql/sql_select.cc:
  New default_field parameter to create_tmp_field()
  Use this in create_tmp_table() to set right default value for a field

mysql-test/extra/binlog_tests/innodb_stat.test:
  test doesn't work with the embedded server
sql/log.cc:
  code shouldn't work without row based replication
sql/sql_insert.cc:
  code shouldn't work without row based replication

 - Move class MY_HOOKS out of function


sql/sql_insert.cc:
  Move class MY_HOOKS out of function 'select_create::prepare'

mysql-test/r/sp.result:
  After-merge fixes
mysql-test/r/trigger-grant.result:
  After-merge fixes
mysql-test/t/trigger-grant.test:
  After-merge fixes
sql/sql_insert.cc:
  After-merge fixes

Added #ifdef's to make code work even when the system is built without
row-based replication.


mysql-test/r/create.result:
  Result change
mysql-test/r/innodb_mysql.result:
  Result change
mysql-test/t/create.test:
  Moving InnoDB specific test to innodb_mysql.test
mysql-test/t/innodb_mysql.test:
  Moving InnoDB-specific test from create.test
mysql-test/t/rpl_rbr_to_sbr.test:
  Test only sensible if we have row-based replication compiled in.
mysql-test/t/rpl_row_basic_8partition.test:
  Test only sensible if we have row-based replication compiled in.
sql/log.cc:
  Adding HAVE_ROW_BASED_REPLICATION guards on code to work without
  row-based replication.
sql/set_var.cc:
  Adding HAVE_ROW_BASED_REPLICATION guards on code to work without
  row-based replication.
sql/set_var.h:
  Adding HAVE_ROW_BASED_REPLICATION guards on code to work without
  row-based replication.
sql/share/errmsg.txt:
  Adding HAVE_ROW_BASED_REPLICATION guards on code to work without
  row-based replication.
sql/sql_base.cc:
  Adding HAVE_ROW_BASED_REPLICATION guards on code to work without
  row-based replication.
sql/sql_class.cc:
  Adding HAVE_ROW_BASED_REPLICATION guards on code to work without
  row-based replication.
sql/sql_class.h:
  Adding HAVE_ROW_BASED_REPLICATION guards on code to work without
  row-based replication.
sql/sql_insert.cc:
  Adding HAVE_ROW_BASED_REPLICATION guards on code to work without
  row-based replication.
sql/sql_parse.cc:
  Adding HAVE_ROW_BASED_REPLICATION guards on code to work without
  row-based replication.
sql/sql_table.cc:
  Not running hooks

or implicitly uses stored function gives "Table not locked" error'

CREATE TABLE ... SELECT ... statement which was explicitly or implicitly
(through view) using stored function gave "Table not locked" error.

The actual bug resides in the current locking scheme of CREATE TABLE SELECT
code, which first opens and locks tables of the SELECT statement itself,
and then, having SELECT tables locked, creates the .FRM, opens the .FRM and
acquires lock on it. This scheme opens a possibility for a deadlock, which
was present and ignored since version 3.23 or earlier. This scheme also
conflicts with the invariant of the prelocking algorithm -- no table can
be open and locked while there are tables locked in prelocked mode.

The patch makes an exception for this invariant when doing CREATE TABLE ...
SELECT, thus extending the possibility of a deadlock to the prelocked mode.
We can't supply a better fix in 5.0.


mysql-test/r/sp.result:
  Added tests for bugs#12472/#15137 'CREATE TABLE ... SELECT ... which
  explicitly or implicitly uses stored function gives "Table not locked" error'
mysql-test/t/sp.test:
  Added tests for bugs#12472/#15137 'CREATE TABLE ... SELECT ... which
  explicitly or implicitly uses stored function gives "Table not locked" error'
sql/mysql_priv.h:
  Added flag which can be passed to open_table() routine in order to ignore
  set of locked tables and prelocked mode.
  We don't need declaration of create_table_from_items() any longer as it was
  moved into sql_insert.cc and made static.
sql/sql_base.cc:
  open_table():
    Added flag which allows open table ignoring set of locked tables and
    prelocked mode.
sql/sql_insert.cc:
  Moved create_table_from_items() from sql_table.cc to sql_insert.cc as it was
  not used outside of sql_insert.cc and contains code which is specific for
  CREATE TABLE ... SELECT.
  Also now when we are executing CREATE TABLE ... SELECT ... statement which
  SELECT part requires execution in prelocked mode we ignore set of locked
  tables in order to get access to the table we just have created.
  We probably don't want to do this if we are under real LOCK TABLES since
  it will widen window for deadlock too much.
sql/sql_table.cc:
  Moved create_table_from_items() routine into sql_insert.cc, since it was not
  used anywhere outside of this file and contains logic which is specific for
  CREATE TABLE ... SELECT statement.