variable innodb_file_format.  Implement file format version stamping of
*.ibd files and SYS_TABLES.TYPE.

This change breaks introduces an incompatible change for for
compressed tables.  We can do this, as we have not released yet.

innodb-zip.test: Add tests for stricter KEY_BLOCK_SIZE and ROW_FORMAT
checks.

DICT_TF_COMPRESSED_MASK, DICT_TF_COMPRESSED_SHIFT: Replace with
DICT_TF_ZSSIZE_MASK, DICT_TF_ZSSIZE_SHIFT.

DICT_TF_FORMAT_MASK, DICT_TF_FORMAT_SHIFT, DICT_TF_FORMAT_51,
DICT_TF_FORMAT_ZIP: File format version, stored in table->flags,
in the .ibd file header, and in SYS_TABLES.TYPE.

dict_create_sys_tables_tuple(): Write the table flags to SYS_TABLES.TYPE
if the format is at least DICT_TF_FORMAT_ZIP.  For old formats
(DICT_TF_FORMAT_51), write DICT_TABLE_ORDINARY as the table type.

DB_TABLE_ZIP_NO_IBD: Remove the error code.  The error handling is done
in ha_innodb.cc; as a failsafe measure, dict_build_table_def_step() will
silently clear the compression and format flags instead of returning this
error.

dict_mem_table_create(): Assert that no extra bits are set in the flags.

dict_sys_tables_get_zip_size(): Rename to dict_sys_tables_get_flags().
Check all flag bits, and return ULINT_UNDEFINED if the combination is
unsupported.

dict_boot(): Document the SYS_TABLES columns N_COLS and TYPE.

dict_table_get_format(), dict_table_set_format(),
dict_table_flags_to_zip_size(): New accessors to table->flags.

dtuple_convert_big_rec(): Introduce the auxiliary variables
local_len, local_prefix_len.  Store a 768-byte prefix locally
if the file format is less than DICT_TF_FORMAT_ZIP.

dtuple_convert_back_big_rec(): Restore the columns.

srv_file_format: New variable: innodb_file_format.

fil_create_new_single_table_tablespace(): Replace the parameter zip_size
with table->flags.

fil_open_single_table_tablespace(): Replace the parameter zip_size_in_k
with table->flags.  Check the flags.

fil_space_struct, fil_space_create(), fil_op_write_log():
Replace zip_size with flags.

fil_node_open_file(): Note a TODO item for InnoDB Hot Backup.
Check that the tablespace flags match.

fil_space_get_zip_size(): Rename to fil_space_get_flags().  Add a
wrapper for fil_space_get_zip_size().

fsp_header_get_flags(): New function.

fsp_header_init_fields(): Replace zip_size with flags.

FSP_SPACE_FLAGS: New name for the tablespace flags.  This field used
to be called FSP_PAGE_ZIP_SIZE, or FSP_LOWEST_NO_WRITE.  It has always
been written as 0 in MySQL/InnoDB versions 4.1 to 5.1.

MLOG_ZIP_FILE_CREATE: Rename to MLOG_FILE_CREATE2.  Add a 32-bit
parameter for the tablespace flags.

ha_innobase::create(): Check the table attributes ROW_FORMAT and
KEY_BLOCK_SIZE.  Issue errors if they are inappropriate, or warnings
if the inherited attributes (in ALTER TABLE) will be ignored.

PAGE_ZIP_MIN_SIZE_SHIFT: New constant: the 2-logarithm of PAGE_ZIP_MIN_SIZE.

There is one consideration: fil_init() chooses the tablespace hash size
based on the initial value of srv_file_per_table.  However, this is nothing
new: InnoDB could be started with innodb_file_per_table=0 even though
*.ibd files exist.

srv_file_per_table: Declare as my_bool instead of ibool, because
MYSQL_SYSVAR_BOOL() expects a pointer to my_bool.  Document the
variable also in srv0srv.h.

innobase_start_or_create_for_mysql(): Note why it is OK to temporarily
clear srv_file_per_table.

innobase_file_per_table: Remove.

innobase_raw_format(), move the definition from row0row.c to 
ha_innodb.cc. After this change, row0row.c no longer references 
system_charset_info (Mantis issue #17). Patch prepared by Vasil, 
tested by Calvin, and reviewed by Marko.

Use innobase_strcasecmp() insteaed of strcasecmp() in i_s.cc and get rid
of strings.h (that file is not present on Windows).

Move the prototype of innobase_strcasecmp() from ha_innodb.cc and
dict0dict.c to ha_prototypes.h.

Approved by:	Heikki

symbols.  Use it for all definitions of non-static variables and functions.

lexyy.c, make_flex.sh: Declare yylex as UNIV_INTERN, not static.  It is
referenced from pars0grm.c.

Actually, according to
	nm .libs/ha_innodb.so|grep -w '[ABCE-TVXYZ]'
the following symbols are still global:

* The vtable for class ha_innodb
* pars0grm.c: The function yyparse() and the variables yychar, yylval, yynerrs

The required changes to the Bison-generated file pars0grm.c will be addressed
in a separate commit, which will add a script similar to make_flex.sh.

The class ha_innodb is renamed from class ha_innobase by a #define.  Thus,
there will be no clash with the builtin InnoDB.  However, there will be some
overhead for invoking virtual methods of class ha_innodb.  Ideas for making
the vtable hidden are welcome.  -fvisibility=hidden is not available in GCC 3.

to the undo log, also store the original length of the column, so that the
changes will be correctly undone in transaction rollback or when fetching
previous versions of the row.

innodb-zip.test: New file, for tests of the compression.

upd_field_t: Add orig_len, the original length of new_val.

btr_push_update_extern_fields(): Restore the original prefix of the column.
Add the parameter heap where memory will be allocated if necessary.

trx_undo_rec_get_col_val(): Add the output parameter orig_len.

trx_undo_page_report_modify_ext(): New function: Write an externally
stored column to the undo log.  This is only called from
trx_undo_page_report_modify(), and this is the only caller of
trx_undo_page_fetch_ext().

trx_undo_update_rec_get_update(): Read the original length of the column
prefix to upd_field->orig_len.

Forward port of r2236
Introduce retry/sleep logic as a workaround for a transient bug
where ::open fails for partitioned tables randomly if we are using
one file per table.  (Bug #33349)


Reviewed by: Heikki

(Revisions 2146:2155 were already merged.)

kernel_mutex must be released before calling this function.

innobase_mysql_end_print_arbitrary_thd(),
innobase_mysql_prepare_print_arbitrary_thd(): Assert that the
kernel_mutex is not being held by the current thread.

 
Non-functional change:
 
Move the prototypes of
innobase_mysql_prepare_print_arbitrary_thd() and
innobase_mysql_end_print_arbitrary_thd() from lock0lock.c to
ha_prototypes.h

Suggested by:	Marko
Approved by:	Marko

Add C test for innobase_convert_name() in addition to the mysql-test one.

innodb_information_schema.test. Add tests that display most columns from
INFORMATION_SCHEMA.INNODB_LOCKS.  Test that quoting of table names works
and respects SQL_MODE='ANSI_QUOTES'.

innobase_print_identifier(): Remove.

innobase_convert_identifier(): New function,
based on innobase_print_identifier().

innobase_convert_name(): New function, similar to ut_print_namel(), but
using a memory buffer.

ut_print_namel(): Use innobase_convert_name().

fill_innodb_locks_from_cache(): Convert lock_table and lock_index by
calling innobase_convert_name().

a compressed table in the system tablespace.

db0err.h: Introduce the error code DB_TABLE_ZIP_NO_IBD.  Replace the
#define directives with an enum, to ease future code merges.  These
error codes are never written out to files or displayed to the user.
Thus they need not remain constant.

dict_build_table_def_step(): Return DB_TABLE_ZIP_NO_IBD instead of DB_ERROR.

create_table_def(): Report ER_ILLEGAL_HA_CREATE_OPTION "KEY_BLOCK_SIZE"
when the table creation fails with DB_TABLE_ZIP_NO_IBD.

redefined so that the dynamic plugin can replace the builtin InnoDB
in MySQL 5.1.

ha_innodb.cc, handler0alter.cc: #include "univ.i" before any other InnoDB
header files or before defining any symbols

innodb_redefine.h: New file, to contain a mapping of symbols.  The idea
is that this file will be replaced in the build process; because this
is a large file that can be generated automatically, it does not make sense
to keep it under version control.

univ.i: #include "innodb_redefine.h" and #define ha_innobase ha_innodb

Makefile.am (ha_innodb_la_CXXFLAGS): Remove -Dha_innobase=ha_innodb

NOTE: there are still some issues in the source code.  One known issue is
the #undef mutex_free in sync0sync.h, which will cause the plugin to call the
function mutex_free in the builtin InnoDB.  The preprocessor symbols defined
in innodb_redefine.h must not be undefined or redefined anywhere in the code.

plugin "InnoDB", not "InnoDBzip".

We can disable the builtin InnoDB by mysqld --skip-innodb.  If the
builtin InnoDB is not disabled, installing the InnoDB plugin by the same
name will not work.

innodb_plugin_init(): Ignore differences in the PLUGIN_VAR_READONLY flag.

ha_innobase::write_row(): The printf format %p expects const void*.

STRUCT_FLD: Do not use the GCC extension when __STRICT_ANSI__ is defined.

row_merge_read_clustered_index(): Compound initializers must not contain
variables.  Assign to struct fields instead.

dfield_dup(): New function for duplicating the data pointed to by dfield_t.

dfield_set_len(), dfield_set_data(): Add Valgrind instrumentation.

to replace a built-in instance of InnoDB in mysqld.  This is work in
progress, with several limitations:

* Other plugins defined in the builtin InnoDB are not disabled.
  However, InnoDB in MySQL 5.1 only defines the storage engine plugin,
  no INFORMATION_SCHEMA plugins.

* The global C symbols in ha_innodb.so except innodb_hton_ptr and
  builtin_innobase_plugin will have to be redefined, e.g., by objcopy.

* The storage engine cannot be called "InnoDB" to avoid a conflict with
  the builtin name.  Here we call it InnoDBzip.

innobase_hton_name[]: Rename to "InnoDBzip" when building a dynamic plugin.

innodb_plugin_init(): New function for the dynamic plugin, to copy
and redirect configuration parameters from the builtin InnoDB.

innodb_dynamic: New configuration parameter.  This has to be added
to the builtin InnoDB in MySQL 5.1.  Also, innodb_hton_ptr must be made
global there.

innobase_init(): Interpret the parameter innodb_dynamic.

Makefile.am: Redefine class ha_innobase to ha_innodb by a preprocessor
define.  Apparently, C++ classes cannot be easily renamed by objcopy.

INNODB_ZIP and INNODB_ZIP_RESET from ha_innodb.cc to i_s.cc.

plugin_author, END_OF_ST_FIELD_INFO, i_s_info: New common constants.

trx_i_s_common_deinit(): Renamed to i_s_common_deinit().

in ST_FIELD_INFO.

ha_innobase::update_thd(void): New function, to call the inline function
ha_innobase::update_thd(THD*).

check_trx_exists(): Make static.  handler0alter.cc does not need to call
this function.

dtuple_create_for_mysql(), dtuple_free_for_mysql(): Remove.

ha_innobase::records_in_range(): Use mem_heap_create(), mem_heap_free(),
and dtuple_create() instead of the removed functions above.  Since r1587,
InnoDB C++ functions can invoke inlined C functions.

innodb_lock_waits. See
https://svn.innodb.com/innobase/InformationSchema/TransactionsAndLocks
for design notes.

Things that need to be resolved before this goes live:
* MySQL must add thd_get_thread_id() function to their code
  http://bugs.mysql.com/30930
* Allocate memory from mem_heap instead of using mem_alloc()
* Copy table name and index name into the cache because they may be
  freed later which will result in referencing freed memory

Approved by:	Marko

(smart ALTER TABLE) from ha_innodb.cc to a separate module, handler0alter.cc.

existed before the fast-index-creation code was merged, in r1414.

Do not call row_prebuilt_table_obsolete(innodb_table) until after
the table has been renamed to a temporary name.  Otherwise, the
following could happen when creating a clustered index on table T:

(a1) copy T to T#1
(a2) mark T obsolete
(b1) add the name of T to the background drop list
(a3) rename T to T#2 and T#1 to T
(b2) drop the table on the background

These steps are executed by two threads, a and b.

If b1 is executed after a3, all is fine: the name T#2 will be added
to the background drop list.  If b1 is executed before a3, the name
T will be added, and T#2 will survive.

These should have gone to branches/fts.

Spotted by: Marko and Ken

Reviewed by: Heikki

fast index creation.

innobase_convert_tablename(): New function, to remove the filename
encoding of a database and table name.

ha_innobase::add_index(): Invoke innobase_convert_tablename() before
calling my_error().

innodb-index.result: Update the error messages.

the dictionary operations.

TODO: ha_innobase::final_drop_index() should do the same, and set
trx->dict_operation as well.

temporary table cannot be created or renamed.

innodb-index.test,result: Add test cases where a temporary table already
exists.