Fix Bug#36169 create innodb compressed table with too large row size crashed

Sometimes it is possible that
row_drop_table_for_mysql(index->table_name, trx, FALSE); is invoked in
row_create_index_for_mysql() when the index object is freed, so copy the
table name to a safe place beforehand and use the copy.

Approved by:	Sunny

that in the fix but the earlier check fails because it tries to be too
strict and assumes that if the block is on the clean list then it can
only be in state io fix NONE.

described in Mantis#30. Specifically:

- Allow innodb_file_format to take string arguments
- Make innodb_file_format system variable a string instead of a number
- Implement the callback functions
- Update warning messages

Three new functions are implemented:

file_format_name_lookup(): Validate the file format name and return
its corresponding id.

innodb_file_format_check(): Check if it is a valid file format. This
function is registered as a callback with MySQL.

innodb_file_format_update(): Update the global variable using the
"saved" value. This functions is registered as a callback with MySQL.

has a config variable that has since been renamed, we use a mapping table
to check for matches.

make_binary_release.sh which is passed to dynconfig. If the target
architecture is i[3456]86 then we disable the --with-pic flag.

Introduced a new session level config param innodb_strict_mode.
If it is set we do extra checking at time of table creation
and fail if any invalid combination of KEY_BLOCK_SIZE
or ROW_FORMAT is specified.

Reviewed by: Sunny

Keep it simple.

Reflect the rename of build-plugin.sh.

Rename build-plugin.sh to make_binary_release.sh.

Add an aux script for preparing the source archives of the plugin that we
are going to ship.

Add a README file containing simple instructions how to build
the plugin from source.

statistics, but only the cumulated count and duration of relocation
operations.

in INFORMATION_SCHEMA.INNODB_CMP*.

i_s_cmp_fields_info, i_s_cmpmem_fields_info: Define the field lengths
as MY_INT32_NUM_DECIMAL_DIGITS or MY_INT64_NUM_DECIMAL_DIGITS.

i_s_cmpmem_fields_info: Define "relocation_ops" as MYSQL_TYPE_LONGLONG.

and columns as suggested by Ken.

INNODB_COMPRESSION, INNODB_COMPRESSION_RESET:
Rename to INNODB_CMP, INNODB_CMP_RESET, with the following columns:
page_size
compress_ops
compress_ops_ok
compress_time
uncompress_ops
uncompress_time

INNODB_COMPRESSION_BUDDY, INNODB_COMPRESSION_BUDDY_RESET:
Rename to INNODB_CMPMEM, INNODB_CMPMEM_RESET, with the following columns:
page_size
pages_used
pages_free
relocation_ops
relocation_time

INNODB_COMPRESSION_BUDDY and INNODB_COMPRESSION_BUDDY_RESET.

buf_buddy_stat_struct, buf_buddy_stat_t, buf_buddy_stat[]:
Statistics of the buddy system grouped by block size.

i_s_innodb_compression_buddy, i_s_innodb_compression_buddy_reset:
New INFORMATION_SCHEMA plugins.

i_s_compression_buddy_fields_info[]: Define the fields:
size, used, free, relocated, relocated_sec.

i_s_compression_buddy_fill_low(), i_s_compression_buddy_fill(),
i_s_compression_buddy_reset_fill(): Fill the fields.

i_s_compression_buddy_init(), i_s_compression_buddy_reset_init():
Initialize the tables.

i_s_compression_fill_low(): Do not acquire or release the buffer pool
mutex.  The page_zip_stat[] is not protected by any mutex.

i_s_innodb_compression, i_s_innodb_compression_reset: Change the
description to say "compression" instead of "compressed buffer pool".

INNODB_ZIP and INNODB_ZIP_RESET to
INNODB_COMPRESSION and INNODB_COMPRESSION_RESET,
and remove the statistics of the buddy system.

This change was discussed with Ken.  It makes the tables shorter
and easier to understand.  The removed data will be represented in
the tables INNODB_COMPRESSION_BUDDY and INNODB_COMPRESSION_BUDDY_RESET
that will be added later.

i_s_innodb_zip, i_s_innodb_zip_reset, i_s_zip_fields_info[],
i_s_zip_fill_low(), i_s_zip_fill(), i_s_zip_reset_fill(),
i_s_zip_init(), i_s_zip_reset_init(): Replace "zip" with "compression".

i_s_compression_fields_info[]: Remove "used", "free",
"relocated", "relocated_usec".  In "compressed_usec" and "decompressed_usec",
replace microseconds with seconds ("usec" with "sec").

page_zip_decompress(): Correct a typo in the function comment.

PAGE_ZIP_SSIZE_BITS, PAGE_ZIP_NUM_SSIZE: New constants.

page_zip_stat_t, page_zip_stat: Statistics of the compression, grouped
by page size.

page_zip_simple_validate(): Assert that page_zip->ssize is reasonable.

Add the prototype of check_global_access() if MySQL version is less than
5.1.24 to make zip compile with 5.1.23. This was removed when MySQL added
the prototype in their code in 5.1.24 but we still need zip to compile with
older versions.

This patch tries to enable resizeable buffer pool by polling the configuration
parameter for the buffer pool size, which is not a good solution.  The
right way could be to have an update function callback of a settable
MySQL variable to send an event to the master thread.

It could also make sense to expose the buffer pool chunks to the user.
The first chunk would never be freed.  Other chunks than the first one
would only be used for allocating page frames (uncompressed or compressed)
and block descriptors of of compressed pages (buf_page_t).  That is, other
users of the buffer pool, such as mem_heap_create_block() and the lock
table, would be restricted to the first chunk.  This would allow other
chunks to be freed by simply flushing any dirty blocks that they contain.

It might also be worthwhile to create multiple chunks initially, based on
the initial buffer pool size and the HugeTLB page size.  In that way, the
buffer pool could be reduced from the initial configuration at runtime.

single-table tablespaces.  This bug was reported by Sunny as Mantis issue #26.

fil_space_create(), fil_create_new_single_table_tablespace(),
fil_open_single_table_tablespace(), fsp_header_init_fields():
Add ut_a(flags != DICT_TF_COMPACT).

dict_build_table_def_step(), row_import_tablespace_for_mysql(),
row_truncate_table_for_mysql(): Pass correct flags to
fil_create_new_single_table_tablespace() or fil_open_single_table_tablespace().

The function dict_sys_tables_get_flags() already ensures that the
table flags are 0 unless ROW_FORMAT=COMPACT.

does not fit by itself on the compressed page.

my_error(ER_TOO_BIG_ROWSIZE, ...).  Otherwise, MySQL can report
"Got error 139 from storage engine" instead of the appropriate
error message.

ha_innobase::index_read(), ha_innobase::general_fetch():
Replace if-else if-else with switch-case.
Pass table->flags to convert_error_code_to_mysql().

innodb_check_for_record_too_big_error().  Remove.  This code belongs to
convert_error_code_to_mysql().

convert_error_code_to_mysql(): Add the parameter "flags", for table flags.
Translate DB_TOO_BIG_RECORD into ER_TOO_BIG_ROWSIZE.

create_index(): Add the parameter "flags".

create_clustered_index_when_no_primary(): Replace the parameter "comp"
with "flags".

innobase_drop_database(): Remove the #ifdef'd-out call to 
convert_error_code_to_mysql().

innodb_row_format>0.

ROW_FORMAT=COMPRESSED when neither innodb_file_per_table nor
innodb_file_format are set.

Throw warnings, not errors for wrong ROW_FORMAT or KEY_BLOCK_SIZE,
so that any table dump can be loaded.

As of this change, InnoDB supports the following table formats:

ROW_FORMAT=REDUNDANT
	the only format before MySQL/InnoDB 5.0.3
ROW_FORMAT=COMPACT
	the new default format of MySQL/InnoDB 5.0.3
ROW_FORMAT=DYNAMIC
	uncompressed, no prefix in the clustered index record for BLOBs
ROW_FORMAT=COMPRESSED
	like ROW_FORMAT=DYNAMIC, but zlib compressed B-trees and BLOBs;
	the compressed page size is specified by KEY_BLOCK_SIZE in
	kilobytes (1, 2, 4, 8, or 16; default 8)

KEY_BLOCK_SIZE=1, 2, 4, 8, or 16: implies ROW_FORMAT=COMPRESSED;
ignored if ROW_FORMAT is not COMPRESSED

KEY_BLOCK_SIZE=anything else: ignored

The InnoDB row format is displayed in the 4th column (Row_format) of
the output of SHOW TABLE STATUS.  The Create_options column may show
ROW_FORMAT= and KEY_BLOCK_SIZE=, but they do not necessarily have
anything to do with InnoDB.

The table format can also be queried like this:

SELECT table_schema, table_name, row_format
FROM information_schema.tables
WHERE engine='innodb' and row_format in ('Compressed','Dynamic');

When Row_format='Compressed', KEY_BLOCK_SIZE should usually correspond
to the compressed page size.  But the .frm file could be manipulated
to show any KEY_BLOCK_SIZE.

For some reason, INFORMATION_SCHEMA.TABLES.CREATE_OPTIONS does not
include KEY_BLOCK_SIZE.  It does include row_format (spelled in
lowercase).  This looks like a MySQL bug, because the table
INFORMATION_SCHEMA.TABLES probably tries to replace SHOW TABLE STATUS.
I reported this as Bug #35275 <http://bugs.mysql.com/35275>.

ha_innobase::get_row_type(): Add ROW_TYPE_COMPRESSED, ROW_TYPE_DYNAMIC.

ha_innobase::create(): Implement ROW_FORMAT=COMPRESSED and
ROW_FORMAT=DYNAMIC.  Do not throw errors for wrong ROW_FORMAT or
KEY_BLOCK_SIZE, but issue warnings instead.

ha_innobase::check_if_incompatible_data(): Return COMPATIBLE_DATA_NO
if KEY_BLOCK_SIZE has been specified.

innodb.result: Adjust the result for the warning issued for ROW_FORMAT=FIXED.

innodb-zip.test: Add tests.  Query INFORMATION_SCHEMA.TABLES for ROW_FORMAT.

parameters that were somehow wrong in r2363.

externally stored columns.

innodb-zip.test: Correct the test case.  Without the fixes, the test
would fail, because the BLOB would be prepended with a 768-byte prefix
of the data.

row_upd_index_replace_new_col_vals_index_pos(),
row_upd_index_replace_new_col_vals(): Use only one "heap"
parameter that must be non-NULL.  When fetching externally
stored columns, use upd_field_t::orig_len.

upd_get_field_by_field_no(): New accessor function, for retrieving
an field from an update vector by field_no.

row_upd_index_replace_new_col_val(): New function, for replacing the
value from an update vector.  This used to be duplicated code in
row_upd_index_replace_new_col_vals_index_pos() and
row_upd_index_replace_new_col_vals().

btr_index_rec_validate_report(), btr_index_rec_validate(): Add const
qualifiers to page, rec, index.

row_vers_old_has_index_entry(), row_vers_build_for_consistent_read():
Add const qualifier to rec.

row_vers_build_for_semi_consistent_read(): Add const qualifier to rec
and old_vers.

row_sel_store_row_id_to_prebuilt(): Add const qualifiers to
index_rec, index, offsets.

row_sel_build_prev_vers_for_mysql(): Add const qualifier to rec.

row_sel_get_clust_rec_for_mysql(): Add const qualifier to rec and out_rec.

row_sel_try_search_shortcut_for_mysql(): Add const qualifier to out_rec.

can create the table innodb_table_monitor and see in the error log which
tables are incompatible with older versions of InnoDB.