New server (as of 5.1.21) provides new features:
  - SHOW CREATE TRIGGER;
  - character set information for SHOW TRIGGERS and SHOW CREATE
    EVENT | FUNCTION | PROCEDURE statements.
Mysqldump uses these features to generate proper dump.

The bug happened when new mysqldump was used to dump older servers.
The problem was that 5.1.21 new features are not available, so
mysqldump exited with error code or just crashed.

The fix is to detect if mysqldump has ben run against older server
and don't use new 5.1.21 functionality in this case. Certainly,
the dump generated for the older server suffers from the character
set problems fixed by BUG#16291 and the like.


client/mysqldump.c:
  Don't use new server features if they are not available.

It has been possible to lock a view in 5.1 for some time.


mysql-test/r/lock.result:
  Update results (Bug#5719)
mysql-test/t/lock.test:
  Add a test case for Bug#5719 "impossible to lock VIEW"

sql/sql_class.cc:
  Fix a valgrind warning (row_count_func is used before it was initialized
  in SQLCOM_CALL)

mysql-test/r/log_tables.result:
  Update results.
mysql-test/t/log_tables.test:
  Silence a race condition: TRUNCATE code issues mysql_frm_type without
  a metadata lock, and finds no table if hits the moment when
  ALTER is swapping two tables.

into  bodhi.(none):/opt/local/work/mysql-5.1-runtime


client/mysqldump.c:
  Auto merged
mysql-test/r/func_time.result:
  Auto merged
mysql-test/r/mysqldump.result:
  Auto merged
mysql-test/t/func_time.test:
  Auto merged
sql/sql_base.cc:
  Auto merged
sql/sql_table.cc:
  Auto merged
sql/table.cc:
  Auto merged
mysql-test/include/mix1.inc:
  Manual merge.
mysql-test/r/innodb_mysql.result:
  Manual merge.

mysql-test/r/func_time.result:
  Update results (use fixed datetime values instead of NOW()).
mysql-test/t/func_time.test:
  Use fixed datetime values instead of NOW(): the test would have a sporadic
  failure when current day changed between two consequtive calls to
  NOW(). The test actually tests FROM_DAYS/TO_DAYS functions, 
  so use of NOW() is not necessary.

into  bodhi.(none):/opt/local/work/mysql-5.1-runtime


client/mysqldump.c:
  Auto merged
mysql-test/r/mysqldump.result:
  Auto merged
mysql-test/t/disabled.def:
  Auto merged
mysql-test/t/mysqldump.test:
  Auto merged
sql/handler.cc:
  Auto merged
sql/lock.cc:
  Auto merged
sql/sql_base.cc:
  Auto merged
sql/sql_lex.cc:
  Auto merged
sql/sql_parse.cc:
  Auto merged
sql/sql_table.cc:
  Auto merged
sql/table.cc:
  Auto merged
sql/table.h:
  Auto merged
mysql-test/include/mix1.inc:
  Manual merge.
mysql-test/r/innodb_mysql.result:
  Manual merge.

sql/sql_base.cc:
  Fix an incorrect manual merge.

into  bodhi.(none):/opt/local/work/mysql-5.0-runtime


client/mysqldump.c:
  Auto merged
mysql-test/r/mysqldump.result:
  Auto merged
sql/sql_base.cc:
  Auto merged
sql/table.cc:
  Auto merged
mysql-test/r/innodb_mysql.result:
  Manual merge.
mysql-test/t/innodb_mysql.test:
  Manual merge.
sql/sql_table.cc:
  Manual merge.

into  bodhi.(none):/opt/local/work/mysql-5.1-runtime


sql/mysql_priv.h:
  Auto merged
sql/sql_class.h:
  Auto merged
sql/sql_lex.cc:
  Auto merged
mysql-test/include/mix1.inc:
  Manual merge.
mysql-test/r/innodb_mysql.result:
  Manual merge.
sql/handler.h:
  Manual merge.
sql/lock.cc:
  Manual merge.
sql/opt_range.cc:
  Manual merge.
sql/sql_base.cc:
  Manual merge.
sql/sql_table.cc:
  Manual merge.

into  weblab.(none):/home/marcsql/TREE/mysql-5.1-cleanup


sql/log.cc:
  Auto merged

into  janus.mylan:/usr/home/serg/Abk/mysql-5.1

existing table->no_replicate code is used instead

sql/handler.h:
  Fix a warning.

mysql-test/r/ddl_i18n_koi8r.result:
  Update result file.
mysql-test/r/ddl_i18n_utf8.result:
  Update result file.

- Removed unused variable.


sql/sql_lex.cc:
  - Removed unused variable

- Removed unused variable.


sql/sql_lex.cc:
  - Removed unused variable

into  adventure.(none):/home/thek/Development/cpp/mysql-5.1-runtime


sql/table.cc:
  Auto merged
sql/table.h:
  Auto merged

into  bk-internal.mysql.com:/data0/bk/mysql-5.1-arch

into  weblab.(none):/home/marcsql/TREE/mysql-5.1-25422-d


client/mysqldump.c:
  Auto merged
mysql-test/r/log_state.result:
  Auto merged
mysql-test/r/show_check.result:
  Auto merged
mysql-test/t/show_check.test:
  Auto merged
sql/mysql_priv.h:
  Auto merged
sql/share/errmsg.txt:
  Auto merged
sql/sp.cc:
  Auto merged

into  ibm.:/home/alik/Documents/MySQL/devel/5.1-rt-merge


mysql-test/r/mysqldump-max.result:
  Auto merged
mysql-test/r/openssl_1.result:
  Auto merged
mysql-test/t/show_check.test:
  Auto merged
sql/sp.cc:
  Auto merged
sql/sql_yacc.yy:
  Auto merged
client/mysqldump.c:
  Manual merge.
mysql-test/r/mysqldump.result:
  Manual merge + use local.

mysql-test/r/mysqldump.result:
  Update result file.

into  janus.mylan:/usr/home/serg/Abk/mysql-5.1

into  adventure.(none):/home/thek/Development/cpp/mysql-5.0-runtime

into  adventure.(none):/home/thek/Development/cpp/bug29929/my51-bug29929


mysql-test/r/sp-prelocking.result:
  Auto merged
mysql-test/t/sp-prelocking.test:
  Auto merged
sql/sql_lex.cc:
  Auto merged
sql/table.cc:
  Auto merged
sql/table.h:
  SCCS merged

When a table was explicitly locked with LOCK TABLES no associated
tables from any related trigger on the subject table were locked.
As a result of this the user could experience unexpected locking
behavior and statement failures similar to "failed: 1100: Table'xx'
was not locked with LOCK TABLES".

This patch fixes this problem by making sure triggers are
pre-loaded on any statement if the subject table was explicitly
locked with LOCK TABLES.


mysql-test/r/sp-prelocking.result:
  Added test case
mysql-test/t/sp-prelocking.test:
  Added test case
sql/sql_lex.cc:
  - Moved some conditional logic out of the table iteration.
  - Added event map values for LOCK TABLE command.
sql/table.cc:
  - Refactored set_trg_event_tpye into the two simpler functions set_trg_event_map
    and set_trg_event_map as methods for manipulating the table event map.
    The original function was only called from st_lex::set_trg_event_type_for_tables
    so it was possible to move the event map creation logic to this function as
    a loop optimization.
sql/table.h:
  - Refactored set_trg_event_tpye into the two simpler functions set_trg_event_map
    and set_trg_event_map as methods for manipulating the table event map.
    The original function was only called from st_lex::set_trg_event_type_for_tables
    so it was possible to move the event map creation logic to this function as
    a loop optimization.

mysqldump generates view defitions in two stages:

  - dump CREATE TABLE statements for the temporary tables.  For each view a
    temporary table, that has the same structure as the view is created.

  - dump DROP TABLE statements for the temporary tables and CREATE VIEW
    statements for the view.

This approach is required because views can have dependencies on each other
(a view can use other views). So, they should be created in the particular
order. mysqldump however is not smart enough, so in order to resolve
dependencies it creates temporary tables first of all.

The problem was that mysqldump might have generated incorrect dump for the
temporary table when a view has non-ASCII column name. That happened when
default-character-set is not utf8.

The fix is to:

  1. Switch character_set_client for the mysqldump's connection to binary
     before issuing SHOW FIELDS statement in order to avoid conversion.
    
  2. Dump switch character_set_client statements to UTF8 and back for
     CREATE TABLE statement that is issued to create temporary table.


client/mysqldump.c:
  1. Switch character_set_results for mysqldump's connection to
  binary before SHOW FIELDS in order to avoid conversion to client
  character set.
  
  2. Dump switch character_set_client statements to UTF8 and back
  for CREATE TABLE statement.
mysql-test/r/mysqldump.result:
  Update result file.
mysql-test/t/mysqldump.test:
  Test case for BUG#30027.

The problem was a race condition on shutdown -- when IM got shutdown
request while a guarded mysqld is starting. In this case the Guardian
thread tried to stop the mysqld, but might fail if the mysqld hadn't
created pid-file so far. When this happened, the mysqld-monitor thread
didn't stop, so the assert in Thread_registry happened.

The fix is to make several attempts to stop mysqld if it is active.


server-tools/instance-manager/guardian.cc:
  Try to stop mysqld several times if it is still active.
server-tools/instance-manager/instance.cc:
  Make Instance::kill_mysqld() to return operation status.
server-tools/instance-manager/instance.h:
  Make Instance::kill_mysqld() to return operation status.
server-tools/instance-manager/thread_registry.cc:
  Log unregistered thread ids.

updated to keypart_map api


storage/myisam/mi_test2.c:
  Bug #30094 mi_test_all: assertion failure
  updated to keypart_map api
  prefix char keys are not supported anymore

between perm and temp tables. Review fixes.

The original bug report complains that if we locked a temporary table
with LOCK TABLES statement, we would not leave LOCK TABLES mode
when this temporary table is dropped.

Additionally, the bug was escalated when it was discovered than
when a temporary transactional table that was previously
locked with LOCK TABLES statement was dropped, futher actions with
this table, such as UNLOCK TABLES, would lead to a crash.

The problem originates from incomplete support of transactional temporary
tables. When we added calls to handler::store_lock()/handler::external_lock()
to operations that work with such tables, we only covered the normal
server code flow and did not cover LOCK TABLES mode. 
In LOCK TABLES mode, ::external_lock(LOCK) would sometimes be called without
matching ::external_lock(UNLOCK), e.g. when a transactional temporary table
was dropped. Additionally, this table would be left in the list of LOCKed 
TABLES.

The patch aims to address this inadequacy. Now, whenever an instance
of 'handler' is destroyed, we assert that it was priorly
external_lock(UNLOCK)-ed. All the places that violate this assert
were fixed.

This patch introduces no changes in behavior -- the discrepancy in
behavior will be fixed when we start calling ::store_lock()/::external_lock()
for all tables, regardless whether they are transactional or not, 
temporary or not.


mysql-test/r/innodb_mysql.result:
  Update test results (Bug#24918)
mysql-test/t/innodb_mysql.test:
  Add a test case for Bug#24918
sql/handler.h:
  Make handler::external_lock() a protected method. Backport from 5.1 its
  public wrapper handler::ha_external_lock().
  Assert that the handler is not closed if it is still locked.
sql/lock.cc:
  In mysql_lock_tables only call lock_external() for the list of tables that
  we called store_lock() for. 
  E.g. get_lock_data() does not add non-transactional temporary tables to the
  lock list, so lock_external() should not be called for them.
  
  Use handler::ha_external_lock() instead of handler::external_lock().
  
  Add comments for mysql_lock_remove(), parameterize one strange
  side effect that it has. At least in one place where mysql_lock_remove
  is used, this side effect is not desired (DROP TABLE). The parameter
  will be dropped in 5.1, along with the side effect.
sql/mysql_priv.h:
  Update declaration of mysql_lock_remove().
sql/opt_range.cc:
  Deploy handler::ha_external_lock() instead of handler::external_lock()
sql/sql_base.cc:
  When closing a temporary table, remove the table from the list of LOCKed 
  TABLES of this thread, in case it's there. 
  It's there if it is a transactional temporary table.
  Use a new declaration of mysql_lock_remove().
sql/sql_class.h:
  Extend the comment for THD::temporary_tables.
sql/sql_table.cc:
  Deploy handler::ha_external_lock() instead of handler::external_lock()

into  mysql.com:/home/svoj/devel/mysql/BUG29957/mysql-5.1-engines


storage/myisam/mi_extra.c:
  Auto merged
mysql-test/r/alter_table.result:
  Manual merge
mysql-test/t/alter_table.test:
  Manual merge

into  mysql.com:/home/svoj/devel/mysql/BUG29957/mysql-5.0-engines

INSERT/DELETE/UPDATE followed by ALTER TABLE within LOCK TABLES
may cause table corruption on Windows.

That happens because ALTER TABLE writes outdated shared state
info into index file.

Fixed by removing obsolete workaround.

Affects MyISAM tables on Windows only.


myisam/mi_extra.c:
  On windows when mi_extra(HA_EXTRA_PREPARE_FOR_DELETE) is called,
  we release external lock and close index file. If we're in LOCK
  TABLES, MyISAM state info doesn't get updated until UNLOCK TABLES.
  
  That means when we release external lock and we're in LOCK TABLES,
  we may write outdated state info.
  
  As SQL layer closes all table instances, we do not need this
  workaround anymore.
mysql-test/r/alter_table.result:
  A test case for BUG#29957.
mysql-test/t/alter_table.test:
  A test case for BUG#29957.

Bug#25422 (Hang with log tables)
Bug 17876 (Truncating mysql.slow_log in a SP after using cursor locks the
          thread)
Bug 23044 (Warnings on flush of a log table)
Bug 29129 (Resetting general_log while the GLOBAL READ LOCK is set causes
           a deadlock)

Prior to this fix, the server would hang when performing concurrent
ALTER TABLE or TRUNCATE TABLE statements against the LOG TABLES,
which are mysql.general_log and mysql.slow_log.

The root cause traces to the following code:
in sql_base.cc, open_table()
  if (table->in_use != thd)
  {
    /* wait_for_condition will unlock LOCK_open for us */
    wait_for_condition(thd, &LOCK_open, &COND_refresh);
  }
The problem with this code is that the current implementation of the
LOGGER creates 'fake' THD objects, like
- Log_to_csv_event_handler::general_log_thd
- Log_to_csv_event_handler::slow_log_thd
which are not associated to a real thread running in the server,
so that waiting for these non-existing threads to release table locks
cause the dead lock.

In general, the design of Log_to_csv_event_handler does not fit into the
general architecture of the server, so that the concept of general_log_thd
and slow_log_thd has to be abandoned:
- this implementation does not work with table locking
- it will not work with commands like SHOW PROCESSLIST
- having the log tables always opened does not integrate well with DDL
operations / FLUSH TABLES / SET GLOBAL READ_ONLY

With this patch, the fundamental design of the LOGGER has been changed to:
- always open and close a log table when writing a log
- remove totally the usage of fake THD objects
- clarify how locking of log tables is implemented in general.

See WL#3984 for details related to the new locking design.

Additional changes (misc bugs exposed and fixed):

1)

mysqldump which would ignore some tables in dump_all_tables_in_db(),
 but forget to ignore the same in dump_all_views_in_db().

2)

mysqldump would also issue an empty "LOCK TABLE" command when all the tables
to lock are to be ignored (numrows == 0), instead of not issuing the query.

3)

Internal errors handlers could intercept errors but not warnings
(see sql_error.cc).

4)

Implementing a nested call to open tables, for the performance schema tables,
exposed an existing bug in remove_table_from_cache(), which would perform:
  in_use->some_tables_deleted=1;
against another thread, without any consideration about thread locking.
This call inside remove_table_from_cache() was not required anyway,
since calling mysql_lock_abort() takes care of aborting -- cleanly -- threads
that might hold a lock on a table.
This line (in_use->some_tables_deleted=1) has been removed.


sql/handler.cc:
  Moved logic for system / log tables in the SQL layer.
sql/handler.h:
  Moved logic for system / log tables in the SQL layer.
sql/lock.cc:
  Revised locking of log tables
sql/log.cc:
  Major cleanup: changed how log tables are locked / written to.
sql/log.h:
  Major cleanup: changed how log tables are locked / written to.
sql/mysql_priv.h:
  performance schema helpers
sql/slave.cc:
  open_ltable() lock flags
sql/sp.cc:
  open_ltable() lock flags
sql/sql_acl.cc:
  open_ltable() lock flags
sql/sql_class.h:
  performance schema helpers
sql/sql_delete.cc:
  log tables cleanup in TRUNCATE
sql/sql_error.cc:
  Internal handlers can also intercept warnings
sql/sql_insert.cc:
  open_ltable() lock flags
sql/sql_parse.cc:
  performance schema helpers
sql/sql_plugin.cc:
  open_ltable() lock flags
sql/sql_rename.cc:
  log tables cleanup in RENAME
sql/sql_servers.cc:
  open_ltable() lock flags
sql/sql_show.cc:
  Move INFORMATION_SCHEMA_NAME to table.cc
sql/sql_table.cc:
  log tables cleanup (admin operations, ALTER TABLE)
sql/sql_udf.cc:
  open_ltable() lock flags
sql/table.cc:
  Implemented TABLE_CATEGORY.
sql/share/errmsg.txt:
  Changed the wording and name of ER_CANT_READ_LOCK_LOG_TABLE
sql/table.h:
  Implemented TABLE_CATEGORY.
storage/csv/ha_tina.cc:
  Moved logic for system / log tables in the SQL layer.
storage/csv/ha_tina.h:
  Moved logic for system / log tables in the SQL layer.
storage/myisam/ha_myisam.cc:
  Moved logic for system / log tables in the SQL layer.
storage/myisam/ha_myisam.h:
  Moved logic for system / log tables in the SQL layer.
client/mysqldump.c:
  Don't lock tables in the ignore list.
  Don't issue empty LOCK TABLES queries.
sql/sql_base.cc:
  log tables cleanup
  performance schema helpers
mysql-test/r/ps.result:
  Adjust test results
mysql-test/r/show_check.result:
  Adjust test results
mysql-test/r/status.result:
  Adjust test results
mysql-test/t/log_state.test:
  Added tests for Bug#29129
mysql-test/t/ps.test:
  Make the test output deterministic
mysql-test/t/show_check.test:
  Make the test output deterministic
mysql-test/r/log_state.result:
  Changed the default location of the log output to LOG_FILE,
  for backward compatibility with MySQL 5.0
  ---
  Adjust test results
mysql-test/r/log_tables.result:
  cleanup for -ps-protocol
mysql-test/t/log_tables.test:
  cleanup for -ps-protocol
sql/set_var.cc:
  Changed the default location of the log output to LOG_FILE,
  for backward compatibility with MySQL 5.0
  ---
  log tables cleanup

into  bk-internal.mysql.com:/data0/bk/mysql-5.1-arch