of order ::open() call by MySQL

Forward port of r2629

Under some conditions MySQL calls ::open with search_latch leading
to a deadlock as we try to acquire dict_sys->mutex inside ::open
breaking the latching order. The fix is to release search_latch.

Reviewed by: Heikki

Merge a change from MySQL (fix the failing innodb test):

  ------------------------------------------------------------
  revno: 2646.12.1
  committer: Mattias Jonsson <mattiasj@mysql.com>
  branch nick: wl4176_2-51-bugteam
  timestamp: Mon 2008-08-11 20:02:03 +0200
  message:
    Bug#20129: ALTER TABLE ... REPAIR PARTITION ... complains that
    partition is corrupt
    
    The main problem was that ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR
    PARTITION took another code path (over mysql_alter_table instead of
    mysql_admin_table) which differs in two ways:
    1) alter table opens the tables in a different way than admin tables do
       resulting in returning with error before it tried the command
    2) alter table does not start to send any diagnostic rows to the client
       which the lower admin functions continue to use -> resulting in
       assertion crash
    
    The fix:
    Remapped ALTER TABLE t ANALYZE/CHECK/OPTIMIZE/REPAIR PARTITION to use
    the same code path as ANALYZE/CHECK/OPTIMIZE/REPAIR TABLE t.
    Adding check in mysql_admin_table to setup the partition list for
    which partitions that should be used.
    
    
    Partitioned tables will still not work with
    REPAIR TABLE/PARTITION USE_FRM, since that requires moving partitions
    to tables, REPAIR TABLE t USE_FRM, and check that the data still
    fulfills the partitioning function and then move the table back to
    being a partition.
    
    NOTE: I have removed the following functions from the handler
    interface:
    analyze_partitions, check_partitions, optimize_partitions,
    repair_partitions
    Since they are not longer needed.
    THIS ALTERS THE STORAGE ENGINE API

I have verified that OPTIMIZE TABLE actually rebuilds the table
and calls ANALYZE.

Approved by:	Heikki

value field in prebuilt. This field tracks the last value in an autoincrement
interval. We use this value to check whether we need to update a table's
AUTOINC counter, if the value written to a table is less than this value
then we avoid updating the table's AUTOINC value in order to reduce
mutex contention. If it's not reset (e.g., after a DELETE statement) then
there is the possibility of missing updates to the table's AUTOINC counter
resulting in a subsequent duplicate row error message under certain 
conditions (see the test case for details).

Bug #38839 - auto increment does not work properly with InnoDB after update

instead of parsing the query string.  This fixes MySQL Bug #37885 without
us having to implement lexical analysis of SQL comments in yet another place.

thd_is_select(): A new predicate.

row_search_for_mysql(): Use thd_is_select().

Approved by Heikki.

Fix Bug#38185 ha_innobase::info can hold locks even when called with HA_STATUS_NO_LOCK

The fix is to call fsp_get_available_space_in_free_extents() from
ha_innobase::info() only if HA_STATUS_NO_LOCK is not present in the flag
*AND*
change get_schema_tables_record() in MySQL's sql/sql_show.cc to call
::info() *without* HA_STATUS_NO_LOCK whenever a user issues SELECT FROM
information_schema.tables;

Without the change to sql/sql_show.cc this patch would lead to Bug#32440
resurfacing. I.e. delete_length would never be updated in ::info() and
will remain 0 forever, resulting in the free space not being shown
anywhere.

This is the change to sql/sql_show.cc for reference, it needs to be
committed to the MySQL repo before or at the same time with this change
to ha_innodb.cc:

 --- patch begins here ---
 --- sql/sql_show.cc.orig	2008-07-23 09:32:14.000000000 +0300
 +++ sql/sql_show.cc	2008-07-23 09:32:19.000000000 +0300
 @@ -3549,8 +3549,7 @@ static int get_schema_tables_record(THD 
  
      if(file)
      {
 -      file->info(HA_STATUS_VARIABLE | HA_STATUS_TIME | HA_STATUS_AUTO |
 -                 HA_STATUS_NO_LOCK);
 +      file->info(HA_STATUS_VARIABLE | HA_STATUS_TIME | HA_STATUS_AUTO);
        enum row_type row_type = file->get_row_type();
        switch (row_type) {
        case ROW_TYPE_NOT_USED:
 --- patch ends here ---

Approved by:	Heikki

Removed UNIV_INLINE qualifier from btr_search_info_get_ref_count().
Otherwise compilation failed on non-debug builds.

Pointed by: Vasil

Fixed a timing hole where a thread dropping an index can free the
in-memory index struct while another thread is still using
that structure to remove entries from adaptive hash index belonging
to one of the pages that belongs to the index being dropped.

The fix is to have a reference counter in the index struct and to
wait for this counter to drop to zero beforing freeing the struct.

Reviewed by: Heikki

Merge a change from MySQL (this fixes the failing innodb-replace test):

  revno: 2659
  committer: Mattias Jonsson <mattiasj@mysql.com>
  branch nick: b31210-51-bugteam
  timestamp: Tue 2008-06-03 13:25:41 +0200
  message:
    Bug#31210: INSERT DELAYED crashes server when used on partitioned tables
      
    Problem was an unclear error message since it could suggest that
    MyISAM did not support INSERT DELAYED.
    Changed the error message to say that DELAYED is not supported by the
    table, instead of the table's storage engine.
    The confusion is that a partitioned table is in somewhat sense using
    the partitioning storage engine, which in turn uses the ordinary
    storage engine. By saying that the table does not support DELAYED we
    do not give any extra informantion about the storage engine or if it
    is partitioned.
  modified:
    mysql-test/r/innodb-replace.result
    mysql-test/t/innodb-replace.test
    mysql-test/t/merge.test
    mysql-test/t/partition_hash.test
    sql/share/errmsg.txt
    sql/sql_insert.cc

 
Fix Bug#36942 Performance problem in lock_get_n_rec_locks (SHOW INNODB STATUS)
by not calling lock_get_n_rec_locks() from lock_print_info_summary() on
production builds.
 
Approved by:	Heikki (via IM)

 
Fix Bug#36941 Performance problem in ha_print_info (SHOW INNODB STATUS)
by disabling some of the code in ha_print_info() in production builds.
 
Approved by:	Heikki (via IM)

was not reset to 0 when the table was truncated.

Bug #37531 : After truncate, auto_increment behaves incorrectly for InnoDB

Merge change from MySQL (this fixes the failing innodb test):

  ChangeSet@1.2628  2008-05-13 17:01:02+05:00  gshchepa@host.loc
    Merge host.loc:/work/bugs/5.0-bugteam-30059
    into  host.loc:/work/bk/5.1-bugteam

NOTE: the diff for mysql-test/r/innodb.result in this ChangeSet looks
empty but the line:

Data truncated for column 'c' at row 1

is present in annotated mysql-test/r/innodb.result@1.215 [1]
and not present in annotated mysql-test/r/innodb.result@1.216 [2] so it
must have been deleted in the above ChangeSet.

[1] http://mysql.bkbits.net:8080/mysql-5.1/mysql-test/r/innodb.result?PAGE=anno&REV=4825e033JNz_463Ha9pwInSjqLvIQg
[2] http://mysql.bkbits.net:8080/mysql-5.1/mysql-test/r/innodb.result?PAGE=anno&REV=482982fcUoskWjhU2xMH2BZ4Gyukfg

Merge change from MySQL (this almost fixes the failing innodb test):

  ChangeSet@1.2613, 2008-05-07 09:58:21+04:00, sergefp@mysql.com +5 -0
    BUG#35850 "Performance regression in 5.1.23/5.1.24"
    - Disable the "prefer full scan on clustered primary key over full scan
      of any secondary key" rule introduced by BUG#35850.
    - Update test results accordingly
    (bk trigger: file this for BUG#35850)
  
    mysql-test/r/innodb.result@1.214, 2008-05-07 09:57:31+04:00, sergefp@mysql.com +15 -15
      BUG#35850 "Performance regression in 5.1.23/5.1.24"
      - Update test results

Fix Bug#36149 Read buffer overflow in srv0start.c found during "make test"

Use strncmp(3) instead of memcmp(3) to avoid reading past end of the string
if it is empty (*str == '\0'). This bug is _not_ a buffer overflow.

Discussed with:	Sunny (via IM)

storage engine". The test for REPLACE was an error of ommission since it's
classified as a simple INSERT. For REPLACE statements we don't acquire
the special AUTOINC lock for AUTOINC_NEW_STYLE_LOCKING with this fix.

Fix Bug#36819 ut_usectime does not handle errors from gettimeofday

by retrying gettimeofday() several times if it fails in ut_usectime().
If it fails on all calls then return error to the caller to be handled
at higher level.

Update the variable innodb_row_lock_time_max in SHOW STATUS output only
if ut_usectime() was successful.

code has been tested on a big-endian machine too.

Fix Bug#11894 innodb_file_per_table crashes w/ Windows .sym symbolic
link hack

The crash was due to un-handled error 3 (path not found). In the case
of file per table, change the call to os_file_handle_error_no_exit()
from os_file_handle_error(). Also, checks for full path pattern during
table create (Windows only), which is used in symbolic link and temp
table creation.

Approved by:	Heikki

Fix Bug#36600 SHOW STATUS takes a lot of CPU in buf_get_latched_pages_number

by removing the Innodb_buffer_pool_pages_latched variable from
SHOW STATUS output in non-UNIV_DEBUG compilation.
 
Approved by:	Heikki

Fix Bug#36434 ha_innodb.so is installed in the wrong directory

Change pkglib_LTLIBRARIES with pkgplugin_LTLIBRARIES which has been
forgotten in this commit: http://lists.mysql.com/commits/40206

Non-functional white space change in Makefile.am:
Use tabs for indentation and be consistent about spaces around the
equal sign.

Merge change from MySQL:

  ChangeSet@1.2563, 2008-03-18 19:42:04+04:00, gluh@mysql.com +1 -0
    Bug#35406 5.1-opt crashes on select from I_S.REFERENTIAL_CONSTRAINTS
    added intialization of f_key_info.referenced_key_name for the case when
    referenced table is dropped

Merge changes from MySQL:

  ChangeSet@1.2645, 2007-12-19 13:24:43+00:00, jperkin@chorlton.adsl.perkin.org.uk +14 -0
    Add new pkgplugindir handling to seperate plugins from libraries,
    and allow override for binary distributions.  Extend mysql_config
    to print compiled-in plugin location for third-party plugins to
    use.  Resolves bug#31736.

and

  ChangeSet@1.2646, 2008-01-02 13:00:46+00:00, jperkin@chorlton.adsl.perkin.org.uk +11 -0
    Use pkglibdir to simplify pkgplugindir, and fix the path in a
    couple of Makefiles.  Continuation of the fix for bug#31736.

Add vim modeline to hint it that plug.in is a config file
so it can be colorized.

Non-functional change: use tabs for indentation in plug.in.

Fix r2430, it should be CFLAGS="$CFLAGS ..." not CFLAGS="..."

Use CFLAGS instead of INNODB_CFLAGS to avoid having incompatible flags
inserted in CFLAGS and INNODB_CFLAGS and to avoid compiling mysql and
innodb with different flags.

Discussed with:	Sunny

* Use INNODB_CFLAGS insead of cluttering CFLAGS with InnoDB specific
  flags. CFLAGS are used to compile every file in the MySQL source tree.

* Add INNODB_DYNAMIC_CFLAGS to the flags of the dynamic plugin and use
  -prefer-non-pic to make the dynamic plugin faster on i386.

Approved by:	Sunny

Fix Bug#36169 create innodb compressed table with too large row size crashed

Sometimes it is possible that
row_drop_table_for_mysql(index->table_name, trx, FALSE); is invoked in
row_create_index_for_mysql() when the index object is freed so copy the
table name to a safe place beforehand and use the copy.

Approved by:	Sunny

and handler_delete

Add the calls to ha_statistic_increment() in ha_innobase::delete_row()
and ha_innobase::update_row().

In InnoDB, the row count is only a rough estimate used by SQL
optimization. InnoDB is now return row count 0 for TRUNCATE operation.

Change the fix for Bug#32440 to show bytes instead of kilobytes in
INFORMATION_SCHEMA.TABLES.DATA_FREE.

Suggested by:	Domas Mituzas <domas@mysql.com>
Approved by:	Heikki

Swap the order in which mysql_thd, mysql_query_str and *mysql_query_str
are checked for non-NULL.

Suggested by:	Marko

Check whether *trx->mysql_query_str is != NULL in addition to
trx->mysql_query_str. This adds more safety.

This may or may not fix Bug#35226 RBR event crashes slave.

Merge change from MySQL (this fixes the failing innodb test):

ChangeSet@1.1810.3601.4, 2008-02-07 02:33:21+04:00, gshchepa@host.loc +9 -0
  Fixed bug#30059.
  Server handles truncation for assignment of too-long values
  into CHAR/VARCHAR/TEXT columns in a different ways when the
  truncated characters are spaces:
  1. CHAR(N) columns silently ignore end-space truncation;
  2. TEXT columns post a truncation warning/error in the
     non-strict/strict mode.
  3. VARCHAR columns always post a truncation note in
     any mode.

  Space truncation processing has been synchronised over
  CHAR/VARCHAR/TEXT columns: current behavior of VARCHAR
  columns has been propagated as standard.

  Binary-encoded string/BLOB columns are not affected.

Fix Bug#34300 Tinyblob & tinytext fields currupted after export/import and alter in 5.1

Copy the BLOB fields, that are stored internally, to a safe place
(prebuilt->blob_heap) when converting a row from InnoDB format to
MySQL format in row_sel_store_mysql_rec().

The bug was introduced in:

 ------------------------------------------------------------------------
 r587 | osku | 2006-05-23 15:35:58 +0300 (Tue, 23 May 2006) | 3 lines
 
 Optimize BLOB selects by using prebuilt->blob_heap directly instead of first
 reading BLOB data to a temporary heap and then copying it to
 prebuilt->blob_heap.
 ------------------------------------------------------------------------

Approved by:	Heikki

Fix typo in comment.

the size of the UNDO slots cache lists (insert and upate). If either of
cached lists has more than 500 entries then we add any UNDO slots that are
freed, to the common free list instead of the cache list, this is to avoid
the case where all the free slots end up in only one of the lists on startup
after a crash.

Tested with test case for 26590 and passes all mysql-test(s).

 
Fix Bug#34823:
fsync() occasionally returns ENOLCK and causes InnoDB to restart mysqld

Create a wrapper to fsync(2) that retries the operation if the error is
ENOLCK. Use that wrapper instead of fsync(2).

Approved by:	Heikki

 
Fix Bug#35220 ALTER TABLE too picky on reserved word "foreign".
 
In ALTER TABLE, change the internal parser to search for
``FOREIGN[[:space:]]'' instead of only ``FOREIGN'' when parsing
ALTER TABLE ... DROP FOREIGN KEY ...; otherwise it could be mistaken
with ALTER TABLE ... DROP foreign_col;
 
Approved by:	Heikki