Skip to content

M
MariaDB
Commit ab1e6fef authored by Marko Mäkelä's avatar Marko Mäkelä
Browse files
Options

MDEV-11623 MariaDB 10.1 fails to start datadir created with 

MariaDB 10.0/MySQL 5.6 using innodb-page-size!=16K

The storage format of FSP_SPACE_FLAGS was accidentally broken
already in MariaDB 10.1.0. This fix is bringing the format in
line with other MySQL and MariaDB release series.

Please refer to the comments that were added to fsp0fsp.h
for details.

This is an INCOMPATIBLE CHANGE that affects users of
page_compression and non-default innodb_page_size. Upgrading
to this release will correct the flags in the data files.
If you want to downgrade to earlier MariaDB 10.1.x, please refer
to the test innodb.101_compatibility how to reset the
FSP_SPACE_FLAGS in the files.

NOTE: MariaDB 10.1.0 to 10.1.20 can misinterpret
uncompressed data files with innodb_page_size=4k or 64k as
compressed innodb_page_size=16k files, and then probably fail
when trying to access the pages. See the comments in the
function fsp_flags_convert_from_101() for detailed analysis.

Move PAGE_COMPRESSION to FSP_SPACE_FLAGS bit position 16.
In this way, compressed innodb_page_size=16k tablespaces will not
be mistaken for uncompressed ones by MariaDB 10.1.0 to 10.1.20.

Derive PAGE_COMPRESSION_LEVEL, ATOMIC_WRITES and DATA_DIR from the
dict_table_t::flags when the table is available, in
fil_space_for_table_exists_in_mem() or fil_open_single_table_tablespace().
During crash recovery, fil_load_single_table_tablespace() will use
innodb_compression_level for the PAGE_COMPRESSION_LEVEL.

FSP_FLAGS_MEM_MASK: A bitmap of the memory-only fil_space_t::flags
that are not to be written to FSP_SPACE_FLAGS. Currently, these will
include PAGE_COMPRESSION_LEVEL, ATOMIC_WRITES and DATA_DIR.

Introduce the macro FSP_FLAGS_PAGE_SSIZE(). We only support
one innodb_page_size for the whole instance.

When creating a dummy tablespace for the redo log, use
fil_space_t::flags=0. The flags are never written to the redo log files.

Remove many FSP_FLAGS_SET_ macros.

dict_tf_verify_flags(): Remove. This is basically only duplicating
the logic of dict_tf_to_fsp_flags(), used in a debug assertion.

fil_space_t::mark: Remove. This flag was not used for anything.

fil_space_for_table_exists_in_mem(): Remove the unnecessary parameter
mark_space, and add a parameter for table flags. Check that
fil_space_t::flags match the table flags, and adjust the (memory-only)
flags based on the table flags.

fil_node_open_file(): Remove some redundant or unreachable conditions,
do not use stderr for output, and avoid unnecessary server aborts.

fil_user_tablespace_restore_page(): Convert the flags, so that the
correct page_size will be used when restoring a page from the
doublewrite buffer.

fil_space_get_page_compressed(), fsp_flags_is_page_compressed(): Remove.
It suffices to have fil_space_is_page_compressed().

FSP_FLAGS_WIDTH_DATA_DIR, FSP_FLAGS_WIDTH_PAGE_COMPRESSION_LEVEL,
FSP_FLAGS_WIDTH_ATOMIC_WRITES: Remove, because these flags do not
exist in the FSP_SPACE_FLAGS but only in memory.

fsp_flags_try_adjust(): New function, to adjust the FSP_SPACE_FLAGS
in page 0. Called by fil_open_single_table_tablespace(),
fil_space_for_table_exists_in_mem(), innobase_start_or_create_for_mysql()
except if --innodb-read-only is active.

fsp_flags_is_valid(ulint): Reimplement from the scratch, with
accurate comments. Do not display any details of detected
inconsistencies, because the output could be confusing when
dealing with MariaDB 10.1.x data files.

fsp_flags_convert_from_101(ulint): Convert flags from buggy
MariaDB 10.1.x format, or return ULINT_UNDEFINED if the flags
cannot be in MariaDB 10.1.x format.

fsp_flags_match(): Check the flags when probing files.
Implemented based on fsp_flags_is_valid()
and fsp_flags_convert_from_101().

dict_check_tablespaces_and_store_max_id(): Do not access the
page after committing the mini-transaction.

IMPORT TABLESPACE fixes:

AbstractCallback::init(): Convert the flags.

FetchIndexRootPages::operator(): Check that the tablespace flags match the
table flags. Do not attempt to convert tablespace flags to table flags,
because the conversion would necessarily be lossy.

PageConverter::update_header(): Write back the correct flags.
This takes care of the flags in IMPORT TABLESPACE.
parent a9d00db1
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 1483 additions and 1497 deletions
mysql-test/suite/innodb/include/ibd_convert.pl 0 → 100644
View file @ ab1e6fef
#!/usr/bin/perl
# Convert tablespace flags to the format understood by MariaDB 10.1.0..10.1.20,
# with the assumption that the flags were correct.
sub convert_to_mariadb_101
{
my ($file, $page_size) = @_;
open(FILE, "+<", $file) or die "Unable to open $file\n";
sysread(FILE, $_, $page_size)==$page_size||die "Unable to read $file\n";
sysseek(FILE, 0, 0)||die "Unable to seek $file\n";
# FIL_PAGE_DATA + FSP_SPACE_FLAGS = 38 + 16 = 54 bytes from the start
my($flags) = unpack "x[54]N";
my $badflags = ($flags & 0x3f);
my $compression_level=6;
$badflags |= 1<<6|$compression_level<<7 if ($flags & 1 << 16);
$badflags |= ($flags & 15 << 6) << 7; # PAGE_SSIZE
substr ($_, 54, 4) = pack("N", $badflags);
# Replace the innodb_checksum_algorithm=none checksum
substr ($_, 0, 4) = pack("N", 0xdeadbeef);
substr ($_, $page_size - 8, 4) = pack("N", 0xdeadbeef);
syswrite(FILE, $_, $page_size)==$page_size||die "Unable to write $file\n";
close(FILE);
}
mysql-test/suite/innodb/r/101_compatibility.result 0 → 100644
View file @ ab1e6fef
#
# MDEV-11623 MariaDB 10.1 fails to start datadir created with
# MariaDB 10.0/MySQL 5.6 using innodb-page-size!=16K
#
call mtr.add_suppression("InnoDB: adjusting FSP_SPACE_FLAGS of tablespace");
SET GLOBAL innodb_file_per_table=1;
SET GLOBAL innodb_file_format=Barracuda;
CREATE TABLE tr(a INT)ENGINE=InnoDB ROW_FORMAT=REDUNDANT;
CREATE TABLE tc(a INT)ENGINE=InnoDB ROW_FORMAT=COMPACT;
CREATE TABLE td(a INT)ENGINE=InnoDB ROW_FORMAT=DYNAMIC;
CREATE TABLE tz(a INT)ENGINE=InnoDB ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=1;
CREATE TABLE tdd(a INT) ENGINE=InnoDB, DATA DIRECTORY='MYSQL_TMP_DIR';
CREATE TABLE tp(a INT) ENGINE=InnoDB page_compressed=1;
CREATE TABLE ti(a INT) ENGINE=InnoDB;
FLUSH TABLES ti FOR EXPORT;
backup: ti
UNLOCK TABLES;
ALTER TABLE ti DISCARD TABLESPACE;
restore: ti .ibd and .cfg files
ALTER TABLE ti IMPORT TABLESPACE;
BEGIN;
INSERT INTO tr VALUES(1);
INSERT INTO tc VALUES(1);
INSERT INTO td VALUES(1);
INSERT INTO tz VALUES(1);
INSERT INTO tdd VALUES(1);
INSERT INTO tp VALUES(1);
INSERT INTO ti VALUES(1);
# Kill the server
CHECK TABLE tr,tc,td,tz,tdd,tp,ti;
Table Op Msg_type Msg_text
test.tr check status OK
test.tc check status OK
test.td check status OK
test.tz check status OK
test.tdd check status OK
test.tp check status OK
test.ti check status OK
CHECK TABLE tr,tc,td,tz,tdd,tp,ti;
Table Op Msg_type Msg_text
test.tr check status OK
test.tc check status OK
test.td check status OK
test.tz check status OK
test.tdd check status OK
test.tp check status OK
test.ti check status OK
DROP TABLE tr,tc,td,tz,tdd,tp,ti;
mysql-test/suite/innodb/r/doublewrite.result
View file @ ab1e6fef
......@@ -39,6 +39,8 @@ set global innodb_buf_flush_list_now = 1;
# Kill the server
# Make the first page (page_no=0) of the user tablespace
# full of zeroes.
#
# MDEV-11623: Use old FSP_SPACE_FLAGS in the doublewrite buffer.
check table t1;
Table Op Msg_type Msg_text
test.t1 check status OK
......
mysql-test/suite/innodb/r/innodb-wl5522-debug-zip.result
View file @ ab1e6fef
call mtr.add_suppression("InnoDB: Page for tablespace .* ");
call mtr.add_suppression("InnoDB: Page for tablespace ");
call mtr.add_suppression("InnoDB: Invalid FSP_SPACE_FLAGS=0x");
FLUSH TABLES;
SET GLOBAL innodb_file_per_table = 1;
SELECT @@innodb_file_per_table;
......@@ -565,7 +566,7 @@ ERROR HY000: Tablespace has been discarded for table 't1'
restore: t1 .ibd and .cfg files
SET SESSION debug_dbug="+d,fsp_flags_is_valid_failure";
ALTER TABLE test_wl5522.t1 IMPORT TABLESPACE;
ERROR HY000: Internal error: Cannot reset LSNs in table '"test_wl5522"."t1"' : Unsupported
ERROR HY000: Internal error: Cannot reset LSNs in table '"test_wl5522"."t1"' : Data structure corruption
SET SESSION debug_dbug="-d,fsp_flags_is_valid_failure";
DROP TABLE test_wl5522.t1;
unlink: t1.ibd
......
mysql-test/suite/innodb/t/101_compatibility.test 0 → 100644
View file @ ab1e6fef
--source include/have_innodb.inc
--source include/not_embedded.inc
-- echo #
-- echo # MDEV-11623 MariaDB 10.1 fails to start datadir created with
-- echo # MariaDB 10.0/MySQL 5.6 using innodb-page-size!=16K
-- echo #
# This is actually testing the opposite: starting the fixed 10.1 with
# buggy 10.1 files (by manually converting the flags in the files).
call mtr.add_suppression("InnoDB: adjusting FSP_SPACE_FLAGS of tablespace");
SET GLOBAL innodb_file_per_table=1;
SET GLOBAL innodb_file_format=Barracuda;
let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
let MYSQLD_DATADIR=`select @@datadir`;
CREATE TABLE tr(a INT)ENGINE=InnoDB ROW_FORMAT=REDUNDANT;
CREATE TABLE tc(a INT)ENGINE=InnoDB ROW_FORMAT=COMPACT;
CREATE TABLE td(a INT)ENGINE=InnoDB ROW_FORMAT=DYNAMIC;
--disable_warnings
CREATE TABLE tz(a INT)ENGINE=InnoDB ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=1;
--enable_warnings
--replace_result $MYSQL_TMP_DIR MYSQL_TMP_DIR
EVAL CREATE TABLE tdd(a INT) ENGINE=InnoDB, DATA DIRECTORY='$MYSQL_TMP_DIR';
CREATE TABLE tp(a INT) ENGINE=InnoDB page_compressed=1;
CREATE TABLE ti(a INT) ENGINE=InnoDB;
FLUSH TABLES ti FOR EXPORT;
perl;
do "$ENV{MTR_SUITE_DIR}/include/innodb-util.pl";
ib_backup_tablespaces("test", "ti");
EOF
UNLOCK TABLES;
ALTER TABLE ti DISCARD TABLESPACE;
perl;
do "$ENV{MTR_SUITE_DIR}/include/innodb-util.pl";
ib_discard_tablespaces("test", "ti");
ib_restore_tablespaces("test", "ti");
do "$ENV{MTR_SUITE_DIR}/include/ibd_convert.pl";
my $ps = $ENV{INNODB_PAGE_SIZE};
my $dd = $ENV{MYSQLD_DATADIR};
convert_to_mariadb_101("$dd/test/ti.ibd", $ps);
EOF
ALTER TABLE ti IMPORT TABLESPACE;
BEGIN;
INSERT INTO tr VALUES(1);
INSERT INTO tc VALUES(1);
INSERT INTO td VALUES(1);
INSERT INTO tz VALUES(1);
INSERT INTO tdd VALUES(1);
INSERT INTO tp VALUES(1);
INSERT INTO ti VALUES(1);
--source include/kill_mysqld.inc
perl;
do "$ENV{MTR_SUITE_DIR}/include/ibd_convert.pl";
my $ps = $ENV{INNODB_PAGE_SIZE};
my $dd = $ENV{MYSQLD_DATADIR};
convert_to_mariadb_101("$dd/ibdata1", $ps);
convert_to_mariadb_101("$dd/test/tr.ibd", $ps);
convert_to_mariadb_101("$dd/test/tc.ibd", $ps);
convert_to_mariadb_101("$dd/test/td.ibd", $ps);
convert_to_mariadb_101("$dd/test/tz.ibd", 1024) if $ps<32768;
convert_to_mariadb_101("$dd/test/tp.ibd", $ps);
convert_to_mariadb_101("$dd/test/ti.ibd", $ps);
convert_to_mariadb_101("$ENV{MYSQL_TMP_DIR}/test/tdd.ibd", $ps);
EOF
--source include/start_mysqld.inc
CHECK TABLE tr,tc,td,tz,tdd,tp,ti;
--source include/shutdown_mysqld.inc
perl;
do "$ENV{MTR_SUITE_DIR}/include/ibd_convert.pl";
my $ps = $ENV{INNODB_PAGE_SIZE};
my $dd = $ENV{MYSQLD_DATADIR};
convert_to_mariadb_101("$dd/ibdata1", $ps);
convert_to_mariadb_101("$dd/test/tr.ibd", $ps);
convert_to_mariadb_101("$dd/test/tc.ibd", $ps);
convert_to_mariadb_101("$dd/test/td.ibd", $ps);
convert_to_mariadb_101("$dd/test/tz.ibd", 1024) if $ps<32768;
convert_to_mariadb_101("$dd/test/tp.ibd", $ps);
convert_to_mariadb_101("$dd/test/ti.ibd", $ps);
convert_to_mariadb_101("$ENV{MYSQL_TMP_DIR}/test/tdd.ibd", $ps);
EOF
--let $restart_parameters=--innodb-read-only
--source include/start_mysqld.inc
CHECK TABLE tr,tc,td,tz,tdd,tp,ti;
--source include/shutdown_mysqld.inc
--let $restart_parameters=
--source include/start_mysqld.inc
DROP TABLE tr,tc,td,tz,tdd,tp,ti;
mysql-test/suite/innodb/t/doublewrite.test
View file @ ab1e6fef
......@@ -17,7 +17,7 @@ call mtr.add_suppression("space header page consists of zero bytes.*test.t1");
call mtr.add_suppression("checksum mismatch in tablespace.*test.t1");
call mtr.add_suppression("Current page size .* != page size on page");
call mtr.add_suppression("innodb-page-size mismatch in tablespace.*test.t1");
call mtr.add_suppression("Database page corruption");
call mtr.add_suppression("Trying to recover page.*from the doublewrite buffer");
--enable_query_log
let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
......@@ -65,14 +65,48 @@ set global innodb_buf_flush_list_now = 1;
--echo # Make the first page (page_no=0) of the user tablespace
--echo # full of zeroes.
--echo #
--echo # MDEV-11623: Use old FSP_SPACE_FLAGS in the doublewrite buffer.
perl;
use IO::Handle;
my $fname= "$ENV{'MYSQLD_DATADIR'}test/t1.ibd";
my $page_size = $ENV{INNODB_PAGE_SIZE};
my $page;
open(FILE, "+<", $fname) or die;
FILE->autoflush(1);
binmode FILE;
print FILE chr(0) x ($ENV{'INNODB_PAGE_SIZE'});
sysread(FILE, $page, $page_size)==$page_size||die "Unable to read $file\n";
sysseek(FILE, 0, 0)||die "Unable to seek $file\n";
die unless syswrite(FILE, chr(0) x $page_size, $page_size) == $page_size;
close FILE;
open(FILE, "+<", "$ENV{MYSQLD_DATADIR}ibdata1")||die "cannot open ibdata1\n";
sysseek(FILE, 6 * $page_size - 190, 0)||die "Unable to seek ibdata1\n";
sysread(FILE, $_, 12) == 12||die "Unable to read TRX_SYS\n";
my($magic,$d1,$d2)=unpack "NNN";
die "magic=$magic, $d1, $d2\n" unless $magic == 536853855 && $d2 >= $d1 + 64;
sysseek(FILE, $d1 * $page_size, 0)||die "Unable to seek ibdata1\n";
# Find the page in the doublewrite buffer
for (my $d = $d1; $d < $d2 + 64; $d++)
{
sysread(FILE, $_, $page_size)==$page_size||die "Cannot read doublewrite\n";
next unless $_ eq $page;
sysseek(FILE, $d * $page_size, 0)||die "Unable to seek ibdata1\n";
# Write buggy MariaDB 10.1.x FSP_SPACE_FLAGS to the doublewrite buffer
my($flags) = unpack "x[54]N";
my $badflags = ($flags & 0x3f);
my $compression_level=6;
$badflags |= 1<<6|$compression_level<<7 if ($flags & 1 << 16);
$badflags |= ($flags & 15 << 6) << 7; # PAGE_SSIZE
substr ($_, 54, 4) = pack("N", $badflags);
# Replace the innodb_checksum_algorithm=none checksum
substr ($_, 0, 4) = pack("N", 0xdeadbeef);
substr ($_, $page_size - 8, 4) = pack("N", 0xdeadbeef);
syswrite(FILE, $_, $page_size)==$page_size||die "Unable to write $file\n";
close(FILE);
exit 0;
}
die "Did not find the page in the doublewrite buffer ($d1,$d2)\n";
EOF
--source include/start_mysqld.inc
......
mysql-test/suite/innodb/t/innodb-wl5522-debug-zip.test
View file @ ab1e6fef
......@@ -17,7 +17,8 @@
# allow test to run only when innodb-page-size=16
--source include/have_innodb_16k.inc
call mtr.add_suppression("InnoDB: Page for tablespace .* ");
call mtr.add_suppression("InnoDB: Page for tablespace ");
call mtr.add_suppression("InnoDB: Invalid FSP_SPACE_FLAGS=0x");
FLUSH TABLES;
let MYSQLD_DATADIR =`SELECT @@datadir`;
......
storage/innobase/buf/buf0dblwr.cc
View file @ ab1e6fef
......@@ -589,6 +589,21 @@ buf_dblwr_process()
continue;
}
if (page_no == 0) {
/* Check the FSP_SPACE_FLAGS. */
ulint flags = fsp_header_get_flags(page);
if (!fsp_flags_is_valid(flags)
&& fsp_flags_convert_from_101(flags)
== ULINT_UNDEFINED) {
ib_logf(IB_LOG_LEVEL_WARN,
"Ignoring a doublewrite copy of page "
ULINTPF ":0 due to invalid flags 0x%x",
space_id, int(flags));
continue;
}
/* The flags on the page should be converted later. */
}
/* Write the good page from the doublewrite buffer to
the intended position. */
......
storage/innobase/dict/dict0load.cc
View file @ ab1e6fef
......@@ -1052,8 +1052,6 @@ dict_check_tablespaces_and_store_max_id(
btr_pcur_store_position(&pcur, &mtr);
mtr_commit(&mtr);
/* For tables created with old versions of InnoDB,
SYS_TABLES.MIX_LEN may contain garbage. Such tables
would always be in ROW_FORMAT=REDUNDANT. Pretend that
......@@ -1087,16 +1085,19 @@ dict_check_tablespaces_and_store_max_id(
if (space_id == 0) {
/* The system tablespace always exists. */
ut_ad(!discarded);
goto next_tablespace;
mem_free(name);
goto loop;
}
mtr_commit(&mtr);
switch (dict_check) {
case DICT_CHECK_ALL_LOADED:
/* All tablespaces should have been found in
fil_load_single_table_tablespaces(). */
if (fil_space_for_table_exists_in_mem(
space_id, name, TRUE, !(is_temp || discarded),
false, NULL, 0)
space_id, name, !(is_temp || discarded),
false, NULL, 0, flags)
&& !(is_temp || discarded)) {
/* If user changes the path of .ibd files in
*.isl files before doing crash recovery ,
......@@ -1128,8 +1129,8 @@ dict_check_tablespaces_and_store_max_id(
/* Some tablespaces may have been opened in
trx_resurrect_table_locks(). */
if (fil_space_for_table_exists_in_mem(
space_id, name, FALSE, FALSE,
false, NULL, 0)) {
space_id, name, false,
false, NULL, 0, flags)) {
break;
}
/* fall through */
......@@ -1191,7 +1192,6 @@ dict_check_tablespaces_and_store_max_id(
max_space_id = space_id;
}
next_tablespace:
mem_free(name);
mtr_start(&mtr);
......@@ -2382,8 +2382,8 @@ dict_load_table(
table->ibd_file_missing = TRUE;
} else if (!fil_space_for_table_exists_in_mem(
table->space, name, FALSE, FALSE, true, heap,
table->id)) {
table->space, name, false, true, heap,
table->id, table->flags)) {
if (DICT_TF2_FLAG_IS_SET(table, DICT_TF2_TEMPORARY)) {
/* Do not bother to retry opening temporary tables. */
......
storage/innobase/fil/fil0fil.cc
View file @ ab1e6fef
This diff is collapsed.
storage/innobase/fsp/fsp0fsp.cc
View file @ ab1e6fef
......@@ -660,6 +660,7 @@ fsp_header_init_fields(
ulint space_id, /*!< in: space id */
ulint flags) /*!< in: tablespace flags (FSP_SPACE_FLAGS) */
{
flags &= ~FSP_FLAGS_MEM_MASK;
ut_a(fsp_flags_is_valid(flags));
mach_write_to_4(FSP_HEADER_OFFSET + FSP_SPACE_ID + page,
......@@ -710,7 +711,7 @@ fsp_header_init(
mlog_write_ulint(header + FSP_SIZE, size, MLOG_4BYTES, mtr);
mlog_write_ulint(header + FSP_FREE_LIMIT, 0, MLOG_4BYTES, mtr);
mlog_write_ulint(header + FSP_SPACE_FLAGS, flags,
mlog_write_ulint(header + FSP_SPACE_FLAGS, flags & ~FSP_FLAGS_MEM_MASK,
MLOG_4BYTES, mtr);
mlog_write_ulint(header + FSP_FRAG_N_USED, 0, MLOG_4BYTES, mtr);
......
storage/innobase/include/dict0dict.ic
View file @ ab1e6fef
/*****************************************************************************
Copyright (c) 1996, 2016, Oracle and/or its affiliates
Copyright (c) 2013, 2016, MariaDB Corporation
Copyright (c) 2013, 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -914,10 +914,13 @@ dict_tf_to_fsp_flags(
ulint table_flags) /*!< in: dict_table_t::flags */
{
ulint fsp_flags;
ulint page_compression = DICT_TF_GET_PAGE_COMPRESSION(table_flags);
ulint page_compression_level = DICT_TF_GET_PAGE_COMPRESSION_LEVEL(table_flags);
ulint page_compression_level = DICT_TF_GET_PAGE_COMPRESSION_LEVEL(
table_flags);
ulint atomic_writes = DICT_TF_GET_ATOMIC_WRITES(table_flags);
ut_ad((DICT_TF_GET_PAGE_COMPRESSION(table_flags) == 0)
== (page_compression_level == 0));
DBUG_EXECUTE_IF("dict_tf_to_fsp_flags_failure",
return(ULINT_UNDEFINED););
......@@ -925,30 +928,23 @@ dict_tf_to_fsp_flags(
fsp_flags = DICT_TF_HAS_ATOMIC_BLOBS(table_flags) ? 1 : 0;
/* ZIP_SSIZE and ATOMIC_BLOBS are at the same position. */
fsp_flags |= table_flags & DICT_TF_MASK_ZIP_SSIZE;
fsp_flags |= table_flags & DICT_TF_MASK_ATOMIC_BLOBS;
/* In addition, tablespace flags also contain the page size. */
fsp_flags |= fsp_flags_set_page_size(fsp_flags, UNIV_PAGE_SIZE);
fsp_flags |= table_flags
& (DICT_TF_MASK_ZIP_SSIZE | DICT_TF_MASK_ATOMIC_BLOBS);
/* The DATA_DIR flag is in a different position in fsp_flag */
fsp_flags |= DICT_TF_HAS_DATA_DIR(table_flags)
? FSP_FLAGS_MASK_DATA_DIR : 0;
fsp_flags |= FSP_FLAGS_PAGE_SSIZE();
/* In addition, tablespace flags also contain if the page
compression is used for this table. */
fsp_flags |= FSP_FLAGS_SET_PAGE_COMPRESSION(fsp_flags, page_compression);
if (page_compression_level) {
fsp_flags |= FSP_FLAGS_MASK_PAGE_COMPRESSION;
}
/* In addition, tablespace flags also contain page compression level
if page compression is used for this table. */
fsp_flags |= FSP_FLAGS_SET_PAGE_COMPRESSION_LEVEL(fsp_flags, page_compression_level);
ut_a(fsp_flags_is_valid(fsp_flags));
/* In addition, tablespace flags also contain flag if atomic writes
is used for this table */
fsp_flags |= FSP_FLAGS_SET_ATOMIC_WRITES(fsp_flags, atomic_writes);
if (DICT_TF_HAS_DATA_DIR(table_flags)) {
fsp_flags |= 1U << FSP_FLAGS_MEM_DATA_DIR;
}
ut_a(fsp_flags_is_valid(fsp_flags));
ut_a(dict_tf_verify_flags(table_flags, fsp_flags));
fsp_flags |= atomic_writes << FSP_FLAGS_MEM_ATOMIC_WRITES;
fsp_flags |= page_compression_level << FSP_FLAGS_MEM_COMPRESSION_LEVEL;
return(fsp_flags);
}
......
storage/innobase/include/dict0pagecompress.h
View file @ ab1e6fef
/*****************************************************************************
Copyright (C) 2013 SkySQL Ab. All Rights Reserved.
Copyright (C) 2013, 2017, MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -56,17 +56,6 @@ dict_table_page_compression_level(
const dict_table_t* table) /*!< in: table */
__attribute__((const));
/********************************************************************//**
Verify that dictionary flags match tablespace flags
@return true if flags match, false if not */
UNIV_INLINE
ibool
dict_tf_verify_flags(
/*=================*/
ulint table_flags, /*!< in: dict_table_t::flags */
ulint fsp_flags) /*!< in: fil_space_t::flags */
__attribute__((const));
/********************************************************************//**
Extract the atomic writes flag from table flags.
@return true if atomic writes are used, false if not used */
......
storage/innobase/include/dict0pagecompress.ic
View file @ ab1e6fef
/*****************************************************************************
Copyright (C) 2013 SkySQL Ab. All Rights Reserved.
Copyright (C) 2013, 2017, MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -24,92 +24,6 @@ page compression and atomic writes information to dictionary.
Created 11/12/2013 Jan Lindström jan.lindstrom@skysql.com
***********************************************************************/
/********************************************************************//**
Verify that dictionary flags match tablespace flags
@return true if flags match, false if not */
UNIV_INLINE
ibool
dict_tf_verify_flags(
/*=================*/
ulint table_flags, /*!< in: dict_table_t::flags */
ulint fsp_flags) /*!< in: fil_space_t::flags */
{
ulint table_unused = DICT_TF_GET_UNUSED(table_flags);
ulint compact = DICT_TF_GET_COMPACT(table_flags);
ulint ssize = DICT_TF_GET_ZIP_SSIZE(table_flags);
ulint atomic_blobs = DICT_TF_HAS_ATOMIC_BLOBS(table_flags);
ulint data_dir = DICT_TF_HAS_DATA_DIR(table_flags);
ulint page_compression = DICT_TF_GET_PAGE_COMPRESSION(table_flags);
ulint page_compression_level = DICT_TF_GET_PAGE_COMPRESSION_LEVEL(table_flags);
ulint atomic_writes = DICT_TF_GET_ATOMIC_WRITES(table_flags);
ulint post_antelope = FSP_FLAGS_GET_POST_ANTELOPE(fsp_flags);
ulint zip_ssize = FSP_FLAGS_GET_ZIP_SSIZE(fsp_flags);
ulint fsp_atomic_blobs = FSP_FLAGS_HAS_ATOMIC_BLOBS(fsp_flags);
ulint page_ssize = FSP_FLAGS_GET_PAGE_SSIZE(fsp_flags);
ulint fsp_unused = FSP_FLAGS_GET_UNUSED(fsp_flags);
ulint fsp_page_compression = FSP_FLAGS_GET_PAGE_COMPRESSION(fsp_flags);
ulint fsp_page_compression_level = FSP_FLAGS_GET_PAGE_COMPRESSION_LEVEL(fsp_flags);
ulint fsp_atomic_writes = FSP_FLAGS_GET_ATOMIC_WRITES(fsp_flags);
DBUG_EXECUTE_IF("dict_tf_verify_flags_failure",
return(ULINT_UNDEFINED););
ut_a(!table_unused);
ut_a(!fsp_unused);
ut_a(page_ssize == 0 || page_ssize != 0); /* silence compiler */
ut_a(compact == 0 || compact == 1); /* silence compiler */
ut_a(data_dir == 0 || data_dir == 1); /* silence compiler */
ut_a(post_antelope == 0 || post_antelope == 1); /* silence compiler */
if (ssize != zip_ssize) {
fprintf(stderr,
"InnoDB: Error: table flags has zip_ssize %ld"
" in the data dictionary\n"
"InnoDB: but the flags in file has zip_ssize %ld\n",
ssize, zip_ssize);
return (FALSE);
}
if (atomic_blobs != fsp_atomic_blobs) {
fprintf(stderr,
"InnoDB: Error: table flags has atomic_blobs %ld"
" in the data dictionary\n"
"InnoDB: but the flags in file has atomic_blobs %ld\n",
atomic_blobs, fsp_atomic_blobs);
return (FALSE);
}
if (page_compression != fsp_page_compression) {
fprintf(stderr,
"InnoDB: Error: table flags has page_compression %ld"
" in the data dictionary\n"
"InnoDB: but the flags in file ahas page_compression %ld\n",
page_compression, fsp_page_compression);
return (FALSE);
}
if (page_compression_level != fsp_page_compression_level) {
fprintf(stderr,
"InnoDB: Error: table flags has page_compression_level %ld"
" in the data dictionary\n"
"InnoDB: but the flags in file has page_compression_level %ld\n",
page_compression_level, fsp_page_compression_level);
return (FALSE);
}
if (atomic_writes != fsp_atomic_writes) {
fprintf(stderr,
"InnoDB: Error: table flags has atomic writes %ld"
" in the data dictionary\n"
"InnoDB: but the flags in file has atomic_writes %ld\n",
atomic_writes, fsp_atomic_writes);
return (FALSE);
}
return(TRUE);
}
/********************************************************************//**
Extract the page compression level from dict_table_t::flags.
These flags are in memory, so assert that they are valid.
......
storage/innobase/include/fil0fil.h
View file @ ab1e6fef
/*****************************************************************************
Copyright (c) 1995, 2016, Oracle and/or its affiliates.
Copyright (c) 2013, 2016, MariaDB Corporation.
Copyright (c) 2013, 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -44,7 +44,6 @@ Created 10/25/1995 Heikki Tuuri
// Forward declaration
struct trx_t;
struct fil_space_t;
typedef std::list<const char*> space_name_list_t;
......@@ -271,10 +270,6 @@ struct fil_space_t {
an insert buffer merge request for a
page because it actually was for the
previous incarnation of the space */
ibool mark; /*!< this is set to TRUE at database startup if
the space corresponds to a table in the InnoDB
data dictionary; so we can print a warning of
orphaned tablespaces */
ibool stop_ios;/*!< TRUE if we want to rename the
.ibd file of tablespace and want to
stop temporarily posting of new i/o
......@@ -303,7 +298,8 @@ struct fil_space_t {
/*!< recovered tablespace size in pages;
0 if no size change was read from the redo log,
or if the size change was implemented */
ulint flags; /*!< tablespace flags; see
ulint flags; /*!< FSP_SPACE_FLAGS and FSP_FLAGS_MEM_ flags;
see fsp0fsp.h,
fsp_flags_is_valid(),
fsp_flags_get_zip_size() */
ulint n_reserved_extents;
......@@ -455,6 +451,7 @@ fil_node_create(
ibool is_raw) /*!< in: TRUE if a raw device or
a raw disk partition */
MY_ATTRIBUTE((nonnull, warn_unused_result));
#ifdef UNIV_LOG_ARCHIVE
/****************************************************************//**
Drops files from the start of a file space, so that its size is cut by
......@@ -618,7 +615,7 @@ fil_read_first_page(
ibool one_read_already, /*!< in: TRUE if min and max
parameters below already
contain sensible data */
ulint* flags, /*!< out: tablespace flags */
ulint* flags, /*!< out: FSP_SPACE_FLAGS */
ulint* space_id, /*!< out: tablespace ID */
#ifdef UNIV_LOG_ARCHIVE
ulint* min_arch_log_no, /*!< out: min of archived
......@@ -836,6 +833,14 @@ fil_create_new_single_table_tablespace(
ulint key_id) /*!< in: encryption key_id */
__attribute__((nonnull, warn_unused_result));
#ifndef UNIV_HOTBACKUP
/** Try to adjust FSP_SPACE_FLAGS if they differ from the expectations.
(Typically when upgrading from MariaDB 10.1.0..10.1.20.)
@param[in] space_id tablespace ID
@param[in] flags desired tablespace flags */
UNIV_INTERN
void
fsp_flags_try_adjust(ulint space_id, ulint flags);
/********************************************************************//**
Tries to open a single-table tablespace and optionally checks the space id is
right in it. If does not succeed, prints an error message to the .err log. This
......@@ -864,7 +869,7 @@ fil_open_single_table_tablespace(
bool validate, /*!< in: Do we validate tablespace? */
bool fix_dict, /*!< in: Can we fix the dictionary? */
ulint id, /*!< in: space id */
ulint flags, /*!< in: tablespace flags */
ulint flags, /*!< in: expected FSP_SPACE_FLAGS */
const char* tablename, /*!< in: table name in the
databasename/tablename format */
const char* filepath, /*!< in: tablespace filepath */
......@@ -905,25 +910,18 @@ fil_tablespace_exists_in_mem(
/*=========================*/
ulint id); /*!< in: space id */
#ifndef UNIV_HOTBACKUP
/*******************************************************************//**
Returns TRUE if a matching tablespace exists in the InnoDB tablespace memory
/** Check if a matching tablespace exists in the InnoDB tablespace memory
cache. Note that if we have not done a crash recovery at the database startup,
there may be many tablespaces which are not yet in the memory cache.
@return TRUE if a matching tablespace exists in the memory cache */
@return whether a matching tablespace exists in the memory cache */
UNIV_INTERN
ibool
bool
fil_space_for_table_exists_in_mem(
/*==============================*/
ulint id, /*!< in: space id */
const char* name, /*!< in: table name in the standard
'databasename/tablename' format */
ibool mark_space, /*!< in: in crash recovery, at database
startup we mark all spaces which have
an associated table in the InnoDB
data dictionary, so that
we can print a warning about orphaned
tablespaces */
ibool print_error_if_does_not_exist,
bool print_error_if_does_not_exist,
/*!< in: print detailed error
information to the .err log if a
matching tablespace is not found from
......@@ -931,7 +929,8 @@ fil_space_for_table_exists_in_mem(
bool adjust_space, /*!< in: whether to adjust space id
when find table space mismatch */
mem_heap_t* heap, /*!< in: heap memory */
table_id_t table_id); /*!< in: table id */
table_id_t table_id, /*!< in: table id */
ulint table_flags); /*!< in: table flags */
#else /* !UNIV_HOTBACKUP */
/********************************************************************//**
Extends all tablespaces to the size stored in the space header. During the
......
storage/innobase/include/fil0pagecompress.h
View file @ ab1e6fef
/*****************************************************************************
Copyright (C) 2013, 2016 MariaDB Corporation. All Rights Reserved.
Copyright (C) 2013, 2017 MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -44,20 +44,11 @@ Returns the page compression flag of the space, or false if the space
is not compressed. The tablespace must be cached in the memory cache.
@return true if page compressed, false if not or space not found */
UNIV_INLINE
ibool
bool
fil_space_is_page_compressed(
/*=========================*/
ulint id); /*!< in: space id */
/*******************************************************************//**
Returns the page compression flag of the space, or false if the space
is not compressed. The tablespace must be cached in the memory cache.
@return true if page compressed, false if not or space not found */
UNIV_INTERN
ibool
fil_space_get_page_compressed(
/*=========================*/
fil_space_t* space); /*!< in: space id */
/*******************************************************************//**
Returns the atomic writes flag of the space, or false if the space
is not using atomic writes. The tablespace must be cached in the memory cache.
@return atomic write table option value */
......
storage/innobase/include/fsp0fsp.h
View file @ ab1e6fef
This diff is collapsed.
storage/innobase/include/fsp0fsp.ic
View file @ ab1e6fef
/*****************************************************************************
Copyright (c) 1995, 2012, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2013, SkySQL Ab. All Rights Reserved.
Copyright (c) 2013, 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -46,107 +46,6 @@ fsp_descr_page(
return((page_no & (zip_size - 1)) == FSP_XDES_OFFSET);
}
/********************************************************************//**
Validate and return the tablespace flags, which are stored in the
tablespace header at offset FSP_SPACE_FLAGS. They should be 0 for
ROW_FORMAT=COMPACT and ROW_FORMAT=REDUNDANT. The newer row formats,
COMPRESSED and DYNAMIC, use a file format > Antelope so they should
have a file format number plus the DICT_TF_COMPACT bit set.
@return true if check ok */
UNIV_INLINE
bool
fsp_flags_is_valid(
/*===============*/
ulint flags) /*!< in: tablespace flags */
{
ulint post_antelope = FSP_FLAGS_GET_POST_ANTELOPE(flags);
ulint zip_ssize = FSP_FLAGS_GET_ZIP_SSIZE(flags);
ulint atomic_blobs = FSP_FLAGS_HAS_ATOMIC_BLOBS(flags);
ulint page_ssize = FSP_FLAGS_GET_PAGE_SSIZE(flags);
ulint unused = FSP_FLAGS_GET_UNUSED(flags);
ulint page_compression = FSP_FLAGS_GET_PAGE_COMPRESSION(flags);
ulint page_compression_level = FSP_FLAGS_GET_PAGE_COMPRESSION_LEVEL(flags);
ulint atomic_writes = FSP_FLAGS_GET_ATOMIC_WRITES(flags);
DBUG_EXECUTE_IF("fsp_flags_is_valid_failure", return(false););
/* fsp_flags is zero unless atomic_blobs is set. */
/* Make sure there are no bits that we do not know about. */
if (unused != 0 || flags == 1) {
fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted unused %lu\n",
flags, unused);
return(false);
} else if (post_antelope) {
/* The Antelope row formats REDUNDANT and COMPACT did
not use tablespace flags, so this flag and the entire
4-byte field is zero for Antelope row formats. */
if (!atomic_blobs) {
fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted atomic_blobs %lu\n",
flags, atomic_blobs);
return(false);
}
}
if (!atomic_blobs) {
/* Barracuda row formats COMPRESSED and DYNAMIC build on
the page structure introduced for the COMPACT row format
by allowing long fields to be broken into prefix and
externally stored parts. */
if (post_antelope || zip_ssize != 0) {
fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted zip_ssize %lu atomic_blobs %lu\n",
flags, zip_ssize, atomic_blobs);
return(false);
}
} else if (!post_antelope || zip_ssize > PAGE_ZIP_SSIZE_MAX) {
fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted zip_ssize %lu max %d\n",
flags, zip_ssize, PAGE_ZIP_SSIZE_MAX);
return(false);
} else if (page_ssize > UNIV_PAGE_SSIZE_MAX) {
/* The page size field can be used for any row type, or it may
be zero for an original 16k page size.
Validate the page shift size is within allowed range. */
fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted page_ssize %lu max %lu\n",
flags, page_ssize, UNIV_PAGE_SSIZE_MAX);
return(false);
} else if (UNIV_PAGE_SIZE != UNIV_PAGE_SIZE_ORIG && !page_ssize) {
fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted page_ssize %lu max %lu:%d\n",
flags, page_ssize, UNIV_PAGE_SIZE, UNIV_PAGE_SIZE_ORIG);
return(false);
}
/* Page compression level requires page compression and atomic blobs
to be set */
if (page_compression_level || page_compression) {
if (!page_compression || !atomic_blobs) {
fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted page_compression %lu\n"
"InnoDB: Error: page_compression_level %lu atomic_blobs %lu\n",
flags, page_compression, page_compression_level, atomic_blobs);
return(false);
}
}
if (atomic_writes > ATOMIC_WRITES_OFF) {
fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted atomic_writes %lu\n",
flags, atomic_writes);
return (false);
}
#if UNIV_FORMAT_MAX != UNIV_FORMAT_B
# error "UNIV_FORMAT_MAX != UNIV_FORMAT_B, Add more validations."
#endif
/* The DATA_DIR field can be used for any row type so there is
nothing here to validate. */
return(true);
}
/********************************************************************//**
Determine if the tablespace is compressed from dict_table_t::flags.
@return TRUE if compressed, FALSE if not compressed */
......@@ -191,10 +90,10 @@ UNIV_INLINE
ulint
fsp_flags_get_page_size(
/*====================*/
ulint flags) /*!< in: tablespace flags */
ulint flags) /*!< in: tablespace flags */
{
ulint page_size = 0;
ulint ssize = FSP_FLAGS_GET_PAGE_SSIZE(flags);
ulint page_size = 0;
ulint ssize = FSP_FLAGS_GET_PAGE_SSIZE(flags);
/* Convert from a 'log2 minus 9' to a page size in bytes. */
if (UNIV_UNLIKELY(ssize)) {
......@@ -211,50 +110,6 @@ fsp_flags_get_page_size(
}
#ifndef UNIV_INNOCHECKSUM
/********************************************************************//**
Add the page size to the tablespace flags.
@return tablespace flags after page size is added */
UNIV_INLINE
ulint
fsp_flags_set_page_size(
/*====================*/
ulint flags, /*!< in: tablespace flags */
ulint page_size) /*!< in: page size in bytes */
{
ulint ssize = 0;
ulint shift;
/* Page size should be > UNIV_PAGE_SIZE_MIN */
ut_ad(page_size >= UNIV_PAGE_SIZE_MIN);
ut_ad(page_size <= UNIV_PAGE_SIZE_MAX);
if (page_size == UNIV_PAGE_SIZE_ORIG) {
ut_ad(0 == FSP_FLAGS_GET_PAGE_SSIZE(flags));
return(flags);
}
for (shift = UNIV_PAGE_SIZE_SHIFT_MAX;
shift >= UNIV_PAGE_SIZE_SHIFT_MIN;
shift--) {
ulint mask = (1 << shift);
if (page_size & mask) {
ut_ad(!(page_size & ~mask));
ssize = shift - UNIV_ZIP_SIZE_SHIFT_MIN + 1;
break;
}
}
ut_ad(ssize);
ut_ad(ssize <= UNIV_PAGE_SSIZE_MAX);
flags = FSP_FLAGS_SET_PAGE_SSIZE(flags, ssize);
ut_ad(fsp_flags_is_valid(flags));
return(flags);
}
/********************************************************************//**
Calculates the descriptor index within a descriptor page.
@return descriptor index */
......
storage/innobase/include/fsp0pagecompress.h
View file @ ab1e6fef
/*****************************************************************************
Copyright (C) 2013, 2015, MariaDB Corporation. All Rights Reserved.
Copyright (C) 2013, 2017, MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -47,15 +47,6 @@ fsp_header_get_compression_level(
/*=============================*/
const page_t* page); /*!< in: first page of a tablespace */
/********************************************************************//**
Determine if the tablespace is page compressed from dict_table_t::flags.
@return TRUE if page compressed, FALSE if not compressed */
UNIV_INLINE
ibool
fsp_flags_is_page_compressed(
/*=========================*/
ulint flags); /*!< in: tablespace flags */
/********************************************************************//**
Extract the page compression level from tablespace flags.
A tablespace has only one physical page compression level
......
storage/innobase/include/fsp0pagecompress.ic
View file @ ab1e6fef
/*****************************************************************************
Copyright (C) 2013, 2015, MariaDB Corporation. All Rights Reserved.
Copyright (C) 2013, 2017, MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -25,18 +25,6 @@ Created 11/12/2013 Jan Lindström jan.lindstrom@mariadb.com
***********************************************************************/
/********************************************************************//**
Determine if the tablespace is page compressed from dict_table_t::flags.
@return TRUE if page compressed, FALSE if not page compressed */
UNIV_INLINE
ibool
fsp_flags_is_page_compressed(
/*=========================*/
ulint flags) /*!< in: tablespace flags */
{
return(FSP_FLAGS_GET_PAGE_COMPRESSION(flags));
}
/********************************************************************//**
Determine the tablespace is page compression level from dict_table_t::flags.
@return page compression level or 0 if not compressed*/
......@@ -125,21 +113,15 @@ Extract the page compression from space.
@return true if space is page compressed, false if space is not found
or space is not page compressed. */
UNIV_INLINE
ibool
bool
fil_space_is_page_compressed(
/*=========================*/
ulint id) /*!< in: space id */
{
ulint flags;
flags = fil_space_get_flags(id);
if (flags && flags != ULINT_UNDEFINED) {
return(fsp_flags_is_page_compressed(flags));
}
ulint flags = fil_space_get_flags(id);
return(0);
return(flags != ULINT_UNDEFINED
&& FSP_FLAGS_HAS_PAGE_COMPRESSION(flags));
}
#endif /* UNIV_INNOCHECKSUM */
......
storage/innobase/row/row0import.cc
View file @ ab1e6fef
/*****************************************************************************
Copyright (c) 2012, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2015, 2016, MariaDB Corporation.
Copyright (c) 2015, 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -371,8 +371,7 @@ class AbstractCallback : public PageCallback {
m_space(ULINT_UNDEFINED),
m_xdes(),
m_xdes_page_no(ULINT_UNDEFINED),
m_space_flags(ULINT_UNDEFINED),
m_table_flags(ULINT_UNDEFINED) UNIV_NOTHROW { }
m_space_flags(ULINT_UNDEFINED) UNIV_NOTHROW { }
/**
Free any extent descriptor instance */
......@@ -535,10 +534,6 @@ class AbstractCallback : public PageCallback {
/** Flags value read from the header page */
ulint m_space_flags;
/** Derived from m_space_flags and row format type, the row format
type is determined from the page header. */
ulint m_table_flags;
};
/** Determine the page size to use for traversing the tablespace
......@@ -553,6 +548,19 @@ AbstractCallback::init(
const page_t* page = block->frame;
m_space_flags = fsp_header_get_flags(page);
if (!fsp_flags_is_valid(m_space_flags)) {
ulint cflags = fsp_flags_convert_from_101(m_space_flags);
if (cflags == ULINT_UNDEFINED) {
ib_logf(IB_LOG_LEVEL_ERROR,
"Invalid FSP_SPACE_FLAGS=0x%x",
int(m_space_flags));
return(DB_CORRUPTION);
}
m_space_flags = cflags;
}
/* Clear the DATA_DIR flag, which is basically garbage. */
m_space_flags &= ~(1U << FSP_FLAGS_POS_RESERVED);
/* Since we don't know whether it is a compressed table
or not, the data is always read into the block->frame. */
......@@ -640,46 +648,6 @@ struct FetchIndexRootPages : public AbstractCallback {
return(m_space);
}
/**
Check if the .ibd file row format is the same as the table's.
@param ibd_table_flags - determined from space and page.
@return DB_SUCCESS or error code. */
dberr_t check_row_format(ulint ibd_table_flags) UNIV_NOTHROW
{
dberr_t err;
rec_format_t ibd_rec_format;
rec_format_t table_rec_format;
if (!dict_tf_is_valid(ibd_table_flags)) {
ib_errf(m_trx->mysql_thd, IB_LOG_LEVEL_ERROR,
ER_TABLE_SCHEMA_MISMATCH,
".ibd file has invlad table flags: %lx",
ibd_table_flags);
return(DB_CORRUPTION);
}
ibd_rec_format = dict_tf_get_rec_format(ibd_table_flags);
table_rec_format = dict_tf_get_rec_format(m_table->flags);
if (table_rec_format != ibd_rec_format) {
ib_errf(m_trx->mysql_thd, IB_LOG_LEVEL_ERROR,
ER_TABLE_SCHEMA_MISMATCH,
"Table has %s row format, .ibd "
"file has %s row format.",
dict_tf_to_row_format_string(m_table->flags),
dict_tf_to_row_format_string(ibd_table_flags));
err = DB_CORRUPTION;
} else {
err = DB_SUCCESS;
}
return(err);
}
/**
Called for each block as it is read from the file.
@param offset - physical offset in the file
......@@ -743,12 +711,17 @@ FetchIndexRootPages::operator() (
m_indexes.push_back(Index(id, page_no));
if (m_indexes.size() == 1) {
m_table_flags = dict_sys_tables_type_to_tf(
m_space_flags,
page_is_comp(page) ? DICT_N_COLS_COMPACT : 0);
err = check_row_format(m_table_flags);
/* Check that the tablespace flags match the table flags. */
ulint expected = dict_tf_to_fsp_flags(m_table->flags);
if (!fsp_flags_match(expected, m_space_flags)) {
ib_errf(m_trx->mysql_thd, IB_LOG_LEVEL_ERROR,
ER_TABLE_SCHEMA_MISMATCH,
"Expected FSP_SPACE_FLAGS=0x%x, .ibd "
"file contains 0x%x.",
unsigned(expected),
unsigned(m_space_flags));
return(DB_CORRUPTION);
}
}
}
......@@ -1965,21 +1938,14 @@ PageConverter::update_header(
"- ignored");
}
ulint space_flags = fsp_header_get_flags(get_frame(block));
if (!fsp_flags_is_valid(space_flags)) {
ib_logf(IB_LOG_LEVEL_ERROR,
"Unsupported tablespace format %lu",
(ulong) space_flags);
return(DB_UNSUPPORTED);
}
mach_write_to_8(
get_frame(block) + FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION,
m_current_lsn);
/* Write back the adjusted flags. */
mach_write_to_4(FSP_HEADER_OFFSET + FSP_SPACE_FLAGS
+ get_frame(block), m_space_flags);
/* Write space_id to the tablespace header, page 0. */
mach_write_to_4(
get_frame(block) + FSP_HEADER_OFFSET + FSP_SPACE_ID,
......
storage/innobase/row/row0mysql.cc
View file @ ab1e6fef
......@@ -4335,6 +4335,7 @@ row_drop_table_for_mysql(
switch (err) {
ibool is_temp;
ulint table_flags;
case DB_SUCCESS:
/* Clone the name, in case it has been allocated
......@@ -4343,6 +4344,7 @@ row_drop_table_for_mysql(
space_id = table->space;
ibd_file_missing = table->ibd_file_missing;
table_flags = table->flags;
is_temp = DICT_TF2_FLAG_IS_SET(table, DICT_TF2_TEMPORARY);
/* If there is a temp path then the temp flag is set.
......@@ -4358,9 +4360,9 @@ row_drop_table_for_mysql(
}
/* We do not allow temporary tables with a remote path. */
ut_a(!(is_temp && DICT_TF_HAS_DATA_DIR(table->flags)));
ut_a(!(is_temp && DICT_TF_HAS_DATA_DIR(table_flags)));
if (space_id && DICT_TF_HAS_DATA_DIR(table->flags)) {
if (space_id && DICT_TF_HAS_DATA_DIR(table_flags)) {
dict_get_and_save_data_dir_path(table, true);
ut_a(table->data_dir_path);
......@@ -4426,8 +4428,9 @@ row_drop_table_for_mysql(
if (err == DB_SUCCESS && space_id > TRX_SYS_SPACE) {
if (!is_temp
&& !fil_space_for_table_exists_in_mem(
space_id, tablename, FALSE,
print_msg, false, NULL, 0)) {
space_id, tablename,
print_msg, false, NULL, 0,
table_flags)) {
/* This might happen if we are dropping a
discarded tablespace */
err = DB_SUCCESS;
......
storage/innobase/srv/srv0start.cc
View file @ ab1e6fef
......@@ -673,8 +673,7 @@ create_log_files(
sprintf(logfilename + dirnamelen, "ib_logfile%u", INIT_LOG_FILE0);
fil_space_create(
logfilename, SRV_LOG_SPACE_FIRST_ID,
fsp_flags_set_page_size(0, UNIV_PAGE_SIZE),
logfilename, SRV_LOG_SPACE_FIRST_ID, 0,
FIL_LOG,
NULL /* no encryption yet */,
true /* this is create */);
......@@ -1159,7 +1158,7 @@ open_or_create_data_files(
crypt_data = fil_space_create_crypt_data(FIL_SPACE_ENCRYPTION_DEFAULT, FIL_DEFAULT_ENCRYPTION_KEY);
}
flags = fsp_flags_set_page_size(0, UNIV_PAGE_SIZE);
flags = FSP_FLAGS_PAGE_SSIZE();
fil_space_create(name, 0, flags, FIL_TABLESPACE,
crypt_data, (*create_new_db) == true);
......@@ -1307,7 +1306,7 @@ srv_undo_tablespace_open(
fil_set_max_space_id_if_bigger(space);
/* Set the compressed page size to 0 (non-compressed) */
flags = fsp_flags_set_page_size(0, UNIV_PAGE_SIZE);
flags = FSP_FLAGS_PAGE_SSIZE();
fil_space_create(name, space, flags, FIL_TABLESPACE,
NULL /* no encryption */,
true /* create */);
......@@ -2296,9 +2295,7 @@ innobase_start_or_create_for_mysql(void)
sprintf(logfilename + dirnamelen, "ib_logfile%u", 0);
fil_space_create(logfilename,
SRV_LOG_SPACE_FIRST_ID,
fsp_flags_set_page_size(0, UNIV_PAGE_SIZE),
FIL_LOG,
SRV_LOG_SPACE_FIRST_ID, 0, FIL_LOG,
NULL /* no encryption yet */,
true /* create */);
......@@ -2714,6 +2711,13 @@ innobase_start_or_create_for_mysql(void)
}
if (!srv_read_only_mode) {
const ulint flags = FSP_FLAGS_PAGE_SSIZE();
for (ulint id = 0; id <= srv_undo_tablespaces; id++) {
if (fil_space_get(id)) {
fsp_flags_try_adjust(id, flags);
}
}
/* Create the thread which watches the timeouts
for lock waits */
thread_handles[2 + SRV_MAX_N_IO_THREADS] = os_thread_create(
......
storage/xtradb/buf/buf0dblwr.cc
View file @ ab1e6fef
......@@ -589,6 +589,21 @@ buf_dblwr_process()
continue;
}
if (page_no == 0) {
/* Check the FSP_SPACE_FLAGS. */
ulint flags = fsp_header_get_flags(page);
if (!fsp_flags_is_valid(flags)
&& fsp_flags_convert_from_101(flags)
== ULINT_UNDEFINED) {
ib_logf(IB_LOG_LEVEL_WARN,
"Ignoring a doublewrite copy of page "
ULINTPF ":0 due to invalid flags 0x%x",
space_id, int(flags));
continue;
}
/* The flags on the page should be converted later. */
}
/* Write the good page from the doublewrite buffer to
the intended position. */
......
storage/xtradb/dict/dict0load.cc
View file @ ab1e6fef
......@@ -1052,8 +1052,6 @@ dict_check_tablespaces_and_store_max_id(
btr_pcur_store_position(&pcur, &mtr);
mtr_commit(&mtr);
/* For tables created with old versions of InnoDB,
SYS_TABLES.MIX_LEN may contain garbage. Such tables
would always be in ROW_FORMAT=REDUNDANT. Pretend that
......@@ -1087,16 +1085,19 @@ dict_check_tablespaces_and_store_max_id(
if (space_id == 0) {
/* The system tablespace always exists. */
ut_ad(!discarded);
goto next_tablespace;
mem_free(name);
goto loop;
}
mtr_commit(&mtr);
switch (dict_check) {
case DICT_CHECK_ALL_LOADED:
/* All tablespaces should have been found in
fil_load_single_table_tablespaces(). */
if (fil_space_for_table_exists_in_mem(
space_id, name, TRUE, !(is_temp || discarded),
false, NULL, 0)
space_id, name, !(is_temp || discarded),
false, NULL, 0, flags)
&& !(is_temp || discarded)) {
/* If user changes the path of .ibd files in
*.isl files before doing crash recovery ,
......@@ -1128,8 +1129,8 @@ dict_check_tablespaces_and_store_max_id(
/* Some tablespaces may have been opened in
trx_resurrect_table_locks(). */
if (fil_space_for_table_exists_in_mem(
space_id, name, FALSE, FALSE,
false, NULL, 0)) {
space_id, name, false,
false, NULL, 0, flags)) {
break;
}
/* fall through */
......@@ -1191,7 +1192,6 @@ dict_check_tablespaces_and_store_max_id(
max_space_id = space_id;
}
next_tablespace:
mem_free(name);
mtr_start(&mtr);
......@@ -2383,8 +2383,8 @@ dict_load_table(
table->ibd_file_missing = TRUE;
} else if (!fil_space_for_table_exists_in_mem(
table->space, name, FALSE, FALSE, true, heap,
table->id)) {
table->space, name, false, true, heap,
table->id, table->flags)) {
if (DICT_TF2_FLAG_IS_SET(table, DICT_TF2_TEMPORARY)) {
/* Do not bother to retry opening temporary tables. */
......
storage/xtradb/fil/fil0fil.cc
View file @ ab1e6fef
This diff is collapsed.
storage/xtradb/fsp/fsp0fsp.cc
View file @ ab1e6fef
......@@ -663,6 +663,7 @@ fsp_header_init_fields(
ulint space_id, /*!< in: space id */
ulint flags) /*!< in: tablespace flags (FSP_SPACE_FLAGS) */
{
flags &= ~FSP_FLAGS_MEM_MASK;
ut_a(fsp_flags_is_valid(flags));
mach_write_to_4(FSP_HEADER_OFFSET + FSP_SPACE_ID + page,
......@@ -713,7 +714,7 @@ fsp_header_init(
mlog_write_ulint(header + FSP_SIZE, size, MLOG_4BYTES, mtr);
mlog_write_ulint(header + FSP_FREE_LIMIT, 0, MLOG_4BYTES, mtr);
mlog_write_ulint(header + FSP_SPACE_FLAGS, flags,
mlog_write_ulint(header + FSP_SPACE_FLAGS, flags & ~FSP_FLAGS_MEM_MASK,
MLOG_4BYTES, mtr);
mlog_write_ulint(header + FSP_FRAG_N_USED, 0, MLOG_4BYTES, mtr);
......
storage/xtradb/include/dict0dict.ic
View file @ ab1e6fef
/*****************************************************************************
Copyright (c) 1996, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2013, 2016, MariaDB
Copyright (c) 2013, 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -918,10 +918,13 @@ dict_tf_to_fsp_flags(
ulint table_flags) /*!< in: dict_table_t::flags */
{
ulint fsp_flags;
ulint page_compression = DICT_TF_GET_PAGE_COMPRESSION(table_flags);
ulint page_compression_level = DICT_TF_GET_PAGE_COMPRESSION_LEVEL(table_flags);
ulint page_compression_level = DICT_TF_GET_PAGE_COMPRESSION_LEVEL(
table_flags);
ulint atomic_writes = DICT_TF_GET_ATOMIC_WRITES(table_flags);
ut_ad((DICT_TF_GET_PAGE_COMPRESSION(table_flags) == 0)
== (page_compression_level == 0));
DBUG_EXECUTE_IF("dict_tf_to_fsp_flags_failure",
return(ULINT_UNDEFINED););
......@@ -929,30 +932,23 @@ dict_tf_to_fsp_flags(
fsp_flags = DICT_TF_HAS_ATOMIC_BLOBS(table_flags) ? 1 : 0;
/* ZIP_SSIZE and ATOMIC_BLOBS are at the same position. */
fsp_flags |= table_flags & DICT_TF_MASK_ZIP_SSIZE;
fsp_flags |= table_flags & DICT_TF_MASK_ATOMIC_BLOBS;
/* In addition, tablespace flags also contain the page size. */
fsp_flags |= fsp_flags_set_page_size(fsp_flags, UNIV_PAGE_SIZE);
fsp_flags |= table_flags
& (DICT_TF_MASK_ZIP_SSIZE | DICT_TF_MASK_ATOMIC_BLOBS);
/* The DATA_DIR flag is in a different position in fsp_flag */
fsp_flags |= DICT_TF_HAS_DATA_DIR(table_flags)
? FSP_FLAGS_MASK_DATA_DIR : 0;
fsp_flags |= FSP_FLAGS_PAGE_SSIZE();
/* In addition, tablespace flags also contain if the page
compression is used for this table. */
fsp_flags |= FSP_FLAGS_SET_PAGE_COMPRESSION(fsp_flags, page_compression);
if (page_compression_level) {
fsp_flags |= FSP_FLAGS_MASK_PAGE_COMPRESSION;
}
/* In addition, tablespace flags also contain page compression level
if page compression is used for this table. */
fsp_flags |= FSP_FLAGS_SET_PAGE_COMPRESSION_LEVEL(fsp_flags, page_compression_level);
ut_a(fsp_flags_is_valid(fsp_flags));
/* In addition, tablespace flags also contain flag if atomic writes
is used for this table */
fsp_flags |= FSP_FLAGS_SET_ATOMIC_WRITES(fsp_flags, atomic_writes);
if (DICT_TF_HAS_DATA_DIR(table_flags)) {
fsp_flags |= 1U << FSP_FLAGS_MEM_DATA_DIR;
}
ut_a(fsp_flags_is_valid(fsp_flags));
ut_a(dict_tf_verify_flags(table_flags, fsp_flags));
fsp_flags |= atomic_writes << FSP_FLAGS_MEM_ATOMIC_WRITES;
fsp_flags |= page_compression_level << FSP_FLAGS_MEM_COMPRESSION_LEVEL;
return(fsp_flags);
}
......
storage/xtradb/include/dict0pagecompress.h
View file @ ab1e6fef
/*****************************************************************************
Copyright (C) 2013 SkySQL Ab. All Rights Reserved.
Copyright (C) 2013, 2017, MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -56,17 +56,6 @@ dict_table_page_compression_level(
const dict_table_t* table) /*!< in: table */
__attribute__((const));
/********************************************************************//**
Verify that dictionary flags match tablespace flags
@return true if flags match, false if not */
UNIV_INLINE
ibool
dict_tf_verify_flags(
/*=================*/
ulint table_flags, /*!< in: dict_table_t::flags */
ulint fsp_flags) /*!< in: fil_space_t::flags */
__attribute__((const));
/********************************************************************//**
Extract the atomic writes flag from table flags.
@return true if atomic writes are used, false if not used */
......
storage/xtradb/include/dict0pagecompress.ic
View file @ ab1e6fef
/*****************************************************************************
Copyright (C) 2013 SkySQL Ab. All Rights Reserved.
Copyright (C) 2013, 2017, MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -24,92 +24,6 @@ page compression and atomic writes information to dictionary.
Created 11/12/2013 Jan Lindström jan.lindstrom@skysql.com
***********************************************************************/
/********************************************************************//**
Verify that dictionary flags match tablespace flags
@return true if flags match, false if not */
UNIV_INLINE
ibool
dict_tf_verify_flags(
/*=================*/
ulint table_flags, /*!< in: dict_table_t::flags */
ulint fsp_flags) /*!< in: fil_space_t::flags */
{
ulint table_unused = DICT_TF_GET_UNUSED(table_flags);
ulint compact = DICT_TF_GET_COMPACT(table_flags);
ulint ssize = DICT_TF_GET_ZIP_SSIZE(table_flags);
ulint atomic_blobs = DICT_TF_HAS_ATOMIC_BLOBS(table_flags);
ulint data_dir = DICT_TF_HAS_DATA_DIR(table_flags);
ulint page_compression = DICT_TF_GET_PAGE_COMPRESSION(table_flags);
ulint page_compression_level = DICT_TF_GET_PAGE_COMPRESSION_LEVEL(table_flags);
ulint atomic_writes = DICT_TF_GET_ATOMIC_WRITES(table_flags);
ulint post_antelope = FSP_FLAGS_GET_POST_ANTELOPE(fsp_flags);
ulint zip_ssize = FSP_FLAGS_GET_ZIP_SSIZE(fsp_flags);
ulint fsp_atomic_blobs = FSP_FLAGS_HAS_ATOMIC_BLOBS(fsp_flags);
ulint page_ssize = FSP_FLAGS_GET_PAGE_SSIZE(fsp_flags);
ulint fsp_unused = FSP_FLAGS_GET_UNUSED(fsp_flags);
ulint fsp_page_compression = FSP_FLAGS_GET_PAGE_COMPRESSION(fsp_flags);
ulint fsp_page_compression_level = FSP_FLAGS_GET_PAGE_COMPRESSION_LEVEL(fsp_flags);
ulint fsp_atomic_writes = FSP_FLAGS_GET_ATOMIC_WRITES(fsp_flags);
DBUG_EXECUTE_IF("dict_tf_verify_flags_failure",
return(ULINT_UNDEFINED););
ut_a(!table_unused);
ut_a(!fsp_unused);
ut_a(page_ssize == 0 || page_ssize != 0); /* silence compiler */
ut_a(compact == 0 || compact == 1); /* silence compiler */
ut_a(data_dir == 0 || data_dir == 1); /* silence compiler */
ut_a(post_antelope == 0 || post_antelope == 1); /* silence compiler */
if (ssize != zip_ssize) {
fprintf(stderr,
"InnoDB: Error: table flags has zip_ssize %ld"
" in the data dictionary\n"
"InnoDB: but the flags in file has zip_ssize %ld\n",
ssize, zip_ssize);
return (FALSE);
}
if (atomic_blobs != fsp_atomic_blobs) {
fprintf(stderr,
"InnoDB: Error: table flags has atomic_blobs %ld"
" in the data dictionary\n"
"InnoDB: but the flags in file has atomic_blobs %ld\n",
atomic_blobs, fsp_atomic_blobs);
return (FALSE);
}
if (page_compression != fsp_page_compression) {
fprintf(stderr,
"InnoDB: Error: table flags has page_compression %ld"
" in the data dictionary\n"
"InnoDB: but the flags in file ahas page_compression %ld\n",
page_compression, fsp_page_compression);
return (FALSE);
}
if (page_compression_level != fsp_page_compression_level) {
fprintf(stderr,
"InnoDB: Error: table flags has page_compression_level %ld"
" in the data dictionary\n"
"InnoDB: but the flags in file has page_compression_level %ld\n",
page_compression_level, fsp_page_compression_level);
return (FALSE);
}
if (atomic_writes != fsp_atomic_writes) {
fprintf(stderr,
"InnoDB: Error: table flags has atomic writes %ld"
" in the data dictionary\n"
"InnoDB: but the flags in file has atomic_writes %ld\n",
atomic_writes, fsp_atomic_writes);
return (FALSE);
}
return(TRUE);
}
/********************************************************************//**
Extract the page compression level from dict_table_t::flags.
These flags are in memory, so assert that they are valid.
......
storage/xtradb/include/fil0fil.h
View file @ ab1e6fef
/*****************************************************************************
Copyright (c) 1995, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2013, 2016, MariaDB Corporation.
Copyright (c) 2013, 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -45,7 +45,6 @@ Created 10/25/1995 Heikki Tuuri
// Forward declaration
struct trx_t;
struct fil_space_t;
typedef std::list<const char*> space_name_list_t;
......@@ -264,10 +263,6 @@ struct fil_space_t {
an insert buffer merge request for a
page because it actually was for the
previous incarnation of the space */
ibool mark; /*!< this is set to TRUE at database startup if
the space corresponds to a table in the InnoDB
data dictionary; so we can print a warning of
orphaned tablespaces */
ibool stop_ios;/*!< TRUE if we want to rename the
.ibd file of tablespace and want to
stop temporarily posting of new i/o
......@@ -296,7 +291,8 @@ struct fil_space_t {
/*!< recovered tablespace size in pages;
0 if no size change was read from the redo log,
or if the size change was implemented */
ulint flags; /*!< tablespace flags; see
ulint flags; /*!< FSP_SPACE_FLAGS and FSP_FLAGS_MEM_ flags;
see fsp0fsp.h,
fsp_flags_is_valid(),
fsp_flags_get_zip_size() */
ulint n_reserved_extents;
......@@ -449,6 +445,7 @@ fil_node_create(
ibool is_raw) /*!< in: TRUE if a raw device or
a raw disk partition */
MY_ATTRIBUTE((nonnull, warn_unused_result));
#ifdef UNIV_LOG_ARCHIVE
/****************************************************************//**
Drops files from the start of a file space, so that its size is cut by
......@@ -620,7 +617,7 @@ fil_read_first_page(
ibool one_read_already, /*!< in: TRUE if min and max
parameters below already
contain sensible data */
ulint* flags, /*!< out: tablespace flags */
ulint* flags, /*!< out: FSP_SPACE_FLAGS */
ulint* space_id, /*!< out: tablespace ID */
lsn_t* min_flushed_lsn, /*!< out: min of flushed
lsn values in data files */
......@@ -832,6 +829,14 @@ fil_create_new_single_table_tablespace(
ulint key_id) /*!< in: encryption key_id */
__attribute__((nonnull, warn_unused_result));
#ifndef UNIV_HOTBACKUP
/** Try to adjust FSP_SPACE_FLAGS if they differ from the expectations.
(Typically when upgrading from MariaDB 10.1.0..10.1.20.)
@param[in] space_id tablespace ID
@param[in] flags desired tablespace flags */
UNIV_INTERN
void
fsp_flags_try_adjust(ulint space_id, ulint flags);
/********************************************************************//**
Tries to open a single-table tablespace and optionally checks the space id is
right in it. If does not succeed, prints an error message to the .err log. This
......@@ -860,7 +865,7 @@ fil_open_single_table_tablespace(
bool validate, /*!< in: Do we validate tablespace? */
bool fix_dict, /*!< in: Can we fix the dictionary? */
ulint id, /*!< in: space id */
ulint flags, /*!< in: tablespace flags */
ulint flags, /*!< in: expected FSP_SPACE_FLAGS */
const char* tablename, /*!< in: table name in the
databasename/tablename format */
const char* filepath, /*!< in: tablespace filepath */
......@@ -901,25 +906,18 @@ fil_tablespace_exists_in_mem(
/*=========================*/
ulint id); /*!< in: space id */
#ifndef UNIV_HOTBACKUP
/*******************************************************************//**
Returns TRUE if a matching tablespace exists in the InnoDB tablespace memory
/** Check if a matching tablespace exists in the InnoDB tablespace memory
cache. Note that if we have not done a crash recovery at the database startup,
there may be many tablespaces which are not yet in the memory cache.
@return TRUE if a matching tablespace exists in the memory cache */
@return whether a matching tablespace exists in the memory cache */
UNIV_INTERN
ibool
bool
fil_space_for_table_exists_in_mem(
/*==============================*/
ulint id, /*!< in: space id */
const char* name, /*!< in: table name in the standard
'databasename/tablename' format */
ibool mark_space, /*!< in: in crash recovery, at database
startup we mark all spaces which have
an associated table in the InnoDB
data dictionary, so that
we can print a warning about orphaned
tablespaces */
ibool print_error_if_does_not_exist,
bool print_error_if_does_not_exist,
/*!< in: print detailed error
information to the .err log if a
matching tablespace is not found from
......@@ -927,7 +925,8 @@ fil_space_for_table_exists_in_mem(
bool adjust_space, /*!< in: whether to adjust space id
when find table space mismatch */
mem_heap_t* heap, /*!< in: heap memory */
table_id_t table_id); /*!< in: table id */
table_id_t table_id, /*!< in: table id */
ulint table_flags); /*!< in: table flags */
#else /* !UNIV_HOTBACKUP */
/********************************************************************//**
Extends all tablespaces to the size stored in the space header. During the
......
storage/xtradb/include/fil0pagecompress.h
View file @ ab1e6fef
/*****************************************************************************
Copyright (C) 2013, 2016 MariaDB Corporation. All Rights Reserved.
Copyright (C) 2013, 2017 MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -44,20 +44,11 @@ Returns the page compression flag of the space, or false if the space
is not compressed. The tablespace must be cached in the memory cache.
@return true if page compressed, false if not or space not found */
UNIV_INLINE
ibool
bool
fil_space_is_page_compressed(
/*=========================*/
ulint id); /*!< in: space id */
/*******************************************************************//**
Returns the page compression flag of the space, or false if the space
is not compressed. The tablespace must be cached in the memory cache.
@return true if page compressed, false if not or space not found */
UNIV_INTERN
ibool
fil_space_get_page_compressed(
/*=========================*/
fil_space_t* space); /*!< in: space id */
/*******************************************************************//**
Returns the atomic writes flag of the space, or false if the space
is not using atomic writes. The tablespace must be cached in the memory cache.
@return atomic write table option value */
......
storage/xtradb/include/fsp0fsp.h
View file @ ab1e6fef
This diff is collapsed.
storage/xtradb/include/fsp0fsp.ic
View file @ ab1e6fef
/*****************************************************************************
Copyright (c) 1995, 2012, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2013, SkySQL Ab. All Rights Reserved.
Copyright (c) 2013, 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -46,107 +46,6 @@ fsp_descr_page(
return((page_no & (zip_size - 1)) == FSP_XDES_OFFSET);
}
/********************************************************************//**
Validate and return the tablespace flags, which are stored in the
tablespace header at offset FSP_SPACE_FLAGS. They should be 0 for
ROW_FORMAT=COMPACT and ROW_FORMAT=REDUNDANT. The newer row formats,
COMPRESSED and DYNAMIC, use a file format > Antelope so they should
have a file format number plus the DICT_TF_COMPACT bit set.
@return true if check ok */
UNIV_INLINE
bool
fsp_flags_is_valid(
/*===============*/
ulint flags) /*!< in: tablespace flags */
{
ulint post_antelope = FSP_FLAGS_GET_POST_ANTELOPE(flags);
ulint zip_ssize = FSP_FLAGS_GET_ZIP_SSIZE(flags);
ulint atomic_blobs = FSP_FLAGS_HAS_ATOMIC_BLOBS(flags);
ulint page_ssize = FSP_FLAGS_GET_PAGE_SSIZE(flags);
ulint unused = FSP_FLAGS_GET_UNUSED(flags);
ulint page_compression = FSP_FLAGS_GET_PAGE_COMPRESSION(flags);
ulint page_compression_level = FSP_FLAGS_GET_PAGE_COMPRESSION_LEVEL(flags);
ulint atomic_writes = FSP_FLAGS_GET_ATOMIC_WRITES(flags);
DBUG_EXECUTE_IF("fsp_flags_is_valid_failure", return(false););
/* fsp_flags is zero unless atomic_blobs is set. */
/* Make sure there are no bits that we do not know about. */
if (unused != 0 || flags == 1) {
fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted unused %lu\n",
flags, unused);
return(false);
} else if (post_antelope) {
/* The Antelope row formats REDUNDANT and COMPACT did
not use tablespace flags, so this flag and the entire
4-byte field is zero for Antelope row formats. */
if (!atomic_blobs) {
fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted atomic_blobs %lu\n",
flags, atomic_blobs);
return(false);
}
}
if (!atomic_blobs) {
/* Barracuda row formats COMPRESSED and DYNAMIC build on
the page structure introduced for the COMPACT row format
by allowing long fields to be broken into prefix and
externally stored parts. */
if (post_antelope || zip_ssize != 0) {
fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted zip_ssize %lu atomic_blobs %lu\n",
flags, zip_ssize, atomic_blobs);
return(false);
}
} else if (!post_antelope || zip_ssize > PAGE_ZIP_SSIZE_MAX) {
fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted zip_ssize %lu max %d\n",
flags, zip_ssize, PAGE_ZIP_SSIZE_MAX);
return(false);
} else if (page_ssize > UNIV_PAGE_SSIZE_MAX) {
/* The page size field can be used for any row type, or it may
be zero for an original 16k page size.
Validate the page shift size is within allowed range. */
fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted page_ssize %lu max %lu\n",
flags, page_ssize, UNIV_PAGE_SSIZE_MAX);
return(false);
} else if (UNIV_PAGE_SIZE != UNIV_PAGE_SIZE_ORIG && !page_ssize) {
fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted page_ssize %lu max %lu:%d\n",
flags, page_ssize, UNIV_PAGE_SIZE, UNIV_PAGE_SIZE_ORIG);
return(false);
}
#if UNIV_FORMAT_MAX != UNIV_FORMAT_B
# error "UNIV_FORMAT_MAX != UNIV_FORMAT_B, Add more validations."
#endif
/* Page compression level requires page compression and atomic blobs
to be set */
if (page_compression_level || page_compression) {
if (!page_compression || !atomic_blobs) {
fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted page_compression %lu\n"
"InnoDB: Error: page_compression_level %lu atomic_blobs %lu\n",
flags, page_compression, page_compression_level, atomic_blobs);
return(false);
}
}
if (atomic_writes > ATOMIC_WRITES_OFF) {
fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted atomic_writes %lu\n",
flags, atomic_writes);
return (false);
}
/* The DATA_DIR field can be used for any row type so there is
nothing here to validate. */
return(true);
}
/********************************************************************//**
Determine if the tablespace is compressed from dict_table_t::flags.
@return TRUE if compressed, FALSE if not compressed */
......@@ -191,10 +90,10 @@ UNIV_INLINE
ulint
fsp_flags_get_page_size(
/*====================*/
ulint flags) /*!< in: tablespace flags */
ulint flags) /*!< in: tablespace flags */
{
ulint page_size = 0;
ulint ssize = FSP_FLAGS_GET_PAGE_SSIZE(flags);
ulint page_size = 0;
ulint ssize = FSP_FLAGS_GET_PAGE_SSIZE(flags);
/* Convert from a 'log2 minus 9' to a page size in bytes. */
if (UNIV_UNLIKELY(ssize)) {
......@@ -211,50 +110,6 @@ fsp_flags_get_page_size(
}
#ifndef UNIV_INNOCHECKSUM
/********************************************************************//**
Add the page size to the tablespace flags.
@return tablespace flags after page size is added */
UNIV_INLINE
ulint
fsp_flags_set_page_size(
/*====================*/
ulint flags, /*!< in: tablespace flags */
ulint page_size) /*!< in: page size in bytes */
{
ulint ssize = 0;
ulint shift;
/* Page size should be > UNIV_PAGE_SIZE_MIN */
ut_ad(page_size >= UNIV_PAGE_SIZE_MIN);
ut_ad(page_size <= UNIV_PAGE_SIZE_MAX);
if (page_size == UNIV_PAGE_SIZE_ORIG) {
ut_ad(0 == FSP_FLAGS_GET_PAGE_SSIZE(flags));
return(flags);
}
for (shift = UNIV_PAGE_SIZE_SHIFT_MAX;
shift >= UNIV_PAGE_SIZE_SHIFT_MIN;
shift--) {
ulint mask = (1 << shift);
if (page_size & mask) {
ut_ad(!(page_size & ~mask));
ssize = shift - UNIV_ZIP_SIZE_SHIFT_MIN + 1;
break;
}
}
ut_ad(ssize);
ut_ad(ssize <= UNIV_PAGE_SSIZE_MAX);
flags = FSP_FLAGS_SET_PAGE_SSIZE(flags, ssize);
ut_ad(fsp_flags_is_valid(flags));
return(flags);
}
/********************************************************************//**
Calculates the descriptor index within a descriptor page.
@return descriptor index */
......
storage/xtradb/include/fsp0pagecompress.h
View file @ ab1e6fef
/*****************************************************************************
Copyright (C) 2013, 2015, MariaDB Corporation. All Rights Reserved.
Copyright (C) 2013, 2017, MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -47,15 +47,6 @@ fsp_header_get_compression_level(
/*=============================*/
const page_t* page); /*!< in: first page of a tablespace */
/********************************************************************//**
Determine if the tablespace is page compressed from dict_table_t::flags.
@return TRUE if page compressed, FALSE if not compressed */
UNIV_INLINE
ibool
fsp_flags_is_page_compressed(
/*=========================*/
ulint flags); /*!< in: tablespace flags */
/********************************************************************//**
Extract the page compression level from tablespace flags.
A tablespace has only one physical page compression level
......
storage/xtradb/include/fsp0pagecompress.ic
View file @ ab1e6fef
/*****************************************************************************
Copyright (C) 2013, 2015, MariaDB Corporation. All Rights Reserved.
Copyright (C) 2013, 2017, MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -25,18 +25,6 @@ Created 11/12/2013 Jan Lindström jan.lindstrom@mariadb.com
***********************************************************************/
/********************************************************************//**
Determine if the tablespace is page compressed from dict_table_t::flags.
@return TRUE if page compressed, FALSE if not page compressed */
UNIV_INLINE
ibool
fsp_flags_is_page_compressed(
/*=========================*/
ulint flags) /*!< in: tablespace flags */
{
return(FSP_FLAGS_GET_PAGE_COMPRESSION(flags));
}
/********************************************************************//**
Determine the tablespace is page compression level from dict_table_t::flags.
@return page compression level or 0 if not compressed*/
......@@ -125,21 +113,15 @@ Extract the page compression from space.
@return true if space is page compressed, false if space is not found
or space is not page compressed. */
UNIV_INLINE
ibool
bool
fil_space_is_page_compressed(
/*=========================*/
ulint id) /*!< in: space id */
{
ulint flags;
flags = fil_space_get_flags(id);
if (flags && flags != ULINT_UNDEFINED) {
return(fsp_flags_is_page_compressed(flags));
}
ulint flags = fil_space_get_flags(id);
return(0);
return(flags != ULINT_UNDEFINED
&& FSP_FLAGS_HAS_PAGE_COMPRESSION(flags));
}
#endif /* UNIV_INNOCHECKSUM */
......
storage/xtradb/row/row0import.cc
View file @ ab1e6fef
/*****************************************************************************
Copyright (c) 2012, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2016, MariaDB Corporation.
Copyright (c) 2015, 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -371,8 +371,7 @@ class AbstractCallback : public PageCallback {
m_space(ULINT_UNDEFINED),
m_xdes(),
m_xdes_page_no(ULINT_UNDEFINED),
m_space_flags(ULINT_UNDEFINED),
m_table_flags(ULINT_UNDEFINED) UNIV_NOTHROW { }
m_space_flags(ULINT_UNDEFINED) UNIV_NOTHROW { }
/**
Free any extent descriptor instance */
......@@ -535,10 +534,6 @@ class AbstractCallback : public PageCallback {
/** Flags value read from the header page */
ulint m_space_flags;
/** Derived from m_space_flags and row format type, the row format
type is determined from the page header. */
ulint m_table_flags;
};
/** Determine the page size to use for traversing the tablespace
......@@ -553,6 +548,19 @@ AbstractCallback::init(
const page_t* page = block->frame;
m_space_flags = fsp_header_get_flags(page);
if (!fsp_flags_is_valid(m_space_flags)) {
ulint cflags = fsp_flags_convert_from_101(m_space_flags);
if (cflags == ULINT_UNDEFINED) {
ib_logf(IB_LOG_LEVEL_ERROR,
"Invalid FSP_SPACE_FLAGS=0x%x",
int(m_space_flags));
return(DB_CORRUPTION);
}
m_space_flags = cflags;
}
/* Clear the DATA_DIR flag, which is basically garbage. */
m_space_flags &= ~(1U << FSP_FLAGS_POS_RESERVED);
/* Since we don't know whether it is a compressed table
or not, the data is always read into the block->frame. */
......@@ -640,46 +648,6 @@ struct FetchIndexRootPages : public AbstractCallback {
return(m_space);
}
/**
Check if the .ibd file row format is the same as the table's.
@param ibd_table_flags - determined from space and page.
@return DB_SUCCESS or error code. */
dberr_t check_row_format(ulint ibd_table_flags) UNIV_NOTHROW
{
dberr_t err;
rec_format_t ibd_rec_format;
rec_format_t table_rec_format;
if (!dict_tf_is_valid(ibd_table_flags)) {
ib_errf(m_trx->mysql_thd, IB_LOG_LEVEL_ERROR,
ER_TABLE_SCHEMA_MISMATCH,
".ibd file has invlad table flags: %lx",
ibd_table_flags);
return(DB_CORRUPTION);
}
ibd_rec_format = dict_tf_get_rec_format(ibd_table_flags);
table_rec_format = dict_tf_get_rec_format(m_table->flags);
if (table_rec_format != ibd_rec_format) {
ib_errf(m_trx->mysql_thd, IB_LOG_LEVEL_ERROR,
ER_TABLE_SCHEMA_MISMATCH,
"Table has %s row format, .ibd "
"file has %s row format.",
dict_tf_to_row_format_string(m_table->flags),
dict_tf_to_row_format_string(ibd_table_flags));
err = DB_CORRUPTION;
} else {
err = DB_SUCCESS;
}
return(err);
}
/**
Called for each block as it is read from the file.
@param offset - physical offset in the file
......@@ -743,12 +711,17 @@ FetchIndexRootPages::operator() (
m_indexes.push_back(Index(id, page_no));
if (m_indexes.size() == 1) {
m_table_flags = dict_sys_tables_type_to_tf(
m_space_flags,
page_is_comp(page) ? DICT_N_COLS_COMPACT : 0);
err = check_row_format(m_table_flags);
/* Check that the tablespace flags match the table flags. */
ulint expected = dict_tf_to_fsp_flags(m_table->flags);
if (!fsp_flags_match(expected, m_space_flags)) {
ib_errf(m_trx->mysql_thd, IB_LOG_LEVEL_ERROR,
ER_TABLE_SCHEMA_MISMATCH,
"Expected FSP_SPACE_FLAGS=0x%x, .ibd "
"file contains 0x%x.",
unsigned(expected),
unsigned(m_space_flags));
return(DB_CORRUPTION);
}
}
}
......@@ -1968,21 +1941,14 @@ PageConverter::update_header(
"- ignored");
}
ulint space_flags = fsp_header_get_flags(get_frame(block));
if (!fsp_flags_is_valid(space_flags)) {
ib_logf(IB_LOG_LEVEL_ERROR,
"Unsupported tablespace format %lu",
(ulong) space_flags);
return(DB_UNSUPPORTED);
}
mach_write_to_8(
get_frame(block) + FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION,
m_current_lsn);
/* Write back the adjusted flags. */
mach_write_to_4(FSP_HEADER_OFFSET + FSP_SPACE_FLAGS
+ get_frame(block), m_space_flags);
/* Write space_id to the tablespace header, page 0. */
mach_write_to_4(
get_frame(block) + FSP_HEADER_OFFSET + FSP_SPACE_ID,
......
storage/xtradb/row/row0mysql.cc
View file @ ab1e6fef
......@@ -4362,6 +4362,7 @@ row_drop_table_for_mysql(
switch (err) {
ibool is_temp;
ulint table_flags;
case DB_SUCCESS:
/* Clone the name, in case it has been allocated
......@@ -4370,6 +4371,7 @@ row_drop_table_for_mysql(
space_id = table->space;
ibd_file_missing = table->ibd_file_missing;
table_flags = table->flags;
is_temp = DICT_TF2_FLAG_IS_SET(table, DICT_TF2_TEMPORARY);
/* If there is a temp path then the temp flag is set.
......@@ -4385,9 +4387,9 @@ row_drop_table_for_mysql(
}
/* We do not allow temporary tables with a remote path. */
ut_a(!(is_temp && DICT_TF_HAS_DATA_DIR(table->flags)));
ut_a(!(is_temp && DICT_TF_HAS_DATA_DIR(table_flags)));
if (space_id && DICT_TF_HAS_DATA_DIR(table->flags)) {
if (space_id && DICT_TF_HAS_DATA_DIR(table_flags)) {
dict_get_and_save_data_dir_path(table, true);
ut_a(table->data_dir_path);
......@@ -4453,8 +4455,9 @@ row_drop_table_for_mysql(
if (err == DB_SUCCESS && space_id > TRX_SYS_SPACE) {
if (!is_temp
&& !fil_space_for_table_exists_in_mem(
space_id, tablename, FALSE,
print_msg, false, NULL, 0)) {
space_id, tablename,
print_msg, false, NULL, 0,
table_flags)) {
/* This might happen if we are dropping a
discarded tablespace */
err = DB_SUCCESS;
......
storage/xtradb/srv/srv0start.cc
View file @ ab1e6fef
......@@ -703,8 +703,7 @@ create_log_files(
sprintf(logfilename + dirnamelen, "ib_logfile%u", INIT_LOG_FILE0);
fil_space_create(
logfilename, SRV_LOG_SPACE_FIRST_ID,
fsp_flags_set_page_size(0, UNIV_PAGE_SIZE),
logfilename, SRV_LOG_SPACE_FIRST_ID, 0,
FIL_LOG,
NULL /* no encryption yet */,
true /* this is create */);
......@@ -1193,7 +1192,7 @@ open_or_create_data_files(
crypt_data = fil_space_create_crypt_data(FIL_SPACE_ENCRYPTION_DEFAULT, FIL_DEFAULT_ENCRYPTION_KEY);
}
flags = fsp_flags_set_page_size(0, UNIV_PAGE_SIZE);
flags = FSP_FLAGS_PAGE_SSIZE();
fil_space_create(name, 0, flags, FIL_TABLESPACE,
crypt_data, (*create_new_db) == true);
......@@ -1341,7 +1340,7 @@ srv_undo_tablespace_open(
fil_set_max_space_id_if_bigger(space);
/* Set the compressed page size to 0 (non-compressed) */
flags = fsp_flags_set_page_size(0, UNIV_PAGE_SIZE);
flags = FSP_FLAGS_PAGE_SSIZE();
fil_space_create(name, space, flags, FIL_TABLESPACE,
NULL /* no encryption */,
true /* create */);
......@@ -2374,9 +2373,7 @@ innobase_start_or_create_for_mysql(void)
sprintf(logfilename + dirnamelen, "ib_logfile%u", 0);
fil_space_create(logfilename,
SRV_LOG_SPACE_FIRST_ID,
fsp_flags_set_page_size(0, UNIV_PAGE_SIZE),
FIL_LOG,
SRV_LOG_SPACE_FIRST_ID, 0, FIL_LOG,
NULL /* no encryption yet */,
true /* create */);
......@@ -2771,6 +2768,13 @@ innobase_start_or_create_for_mysql(void)
}
if (!srv_read_only_mode) {
const ulint flags = FSP_FLAGS_PAGE_SSIZE();
for (ulint id = 0; id <= srv_undo_tablespaces; id++) {
if (fil_space_get(id)) {
fsp_flags_try_adjust(id, flags);
}
}
/* Create the thread which watches the timeouts
for lock waits */
thread_handles[2 + SRV_MAX_N_IO_THREADS] = os_thread_create(
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7