This occured in gcc-6.2.1.

The variable wasn't used so was no need to be volatile either.
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>

MDEV-10515: Correct stat_tables_par test results

If page_compression (introduced in MariaDB Server 10.1) is enabled,
the logical action is to not preallocate space to the data files,
but to only logically extend the files with zeroes.

fil_create_new_single_table_tablespace(): Create smaller files for
ROW_FORMAT=COMPRESSED tables, but adhere to the minimum file size of
4*innodb_page_size.

fil_space_extend_must_retry(), os_file_set_size(): On Windows,
use SetFileInformationByHandle() and FILE_END_OF_FILE_INFO,
which depends on bumping _WIN32_WINNT to 0x0600.
FIXME: The files are not yet set up as sparse, so
this will currently end up physically extending (preallocating)
the files, wasting storage for unused pages.

os_file_set_size(): Add the parameter "bool sparse=false" to declare
that the file is to be extended logically, instead of being preallocated.
The only caller with sparse=true is
fil_create_new_single_table_tablespace().
(The system tablespace cannot be created with page_compression.)

fil_space_extend_must_retry(), os_file_set_size(): Outside Windows,
use ftruncate() to extend files that are supposed to be sparse.
On systems where ftruncate() is limited to files less than 4GiB
(if there are any), fil_space_extend_must_retry() retains the
old logic of physically extending the file.

fil_extend_space_to_desired_size(): Use a proper type cast when
computing start_offset for the posix_fallocate() call on 32-bit systems
(where sizeof(ulint) < sizeof(os_offset_t)). This could affect 32-bit
systems when extending files that are at least 4 MiB long.

This bug existed in MariaDB 10.0 before MDEV-11520. In MariaDB 10.1
it had been fixed in MDEV-11556.

a large memory buffer on Windows

fil_extend_space_to_desired_size(), os_file_set_size(): Use calloc()
for memory allocation, and handle failures. Properly check the return
status of posix_fallocate(), and pass the correct arguments to
posix_fallocate().

On Windows, instead of extending the file by at most 1 megabyte at a time,
write a zero-filled page at the end of the file.
According to the Microsoft blog post
https://blogs.msdn.microsoft.com/oldnewthing/20110922-00/?p=9573
this will physically extend the file by writing zero bytes.
(InnoDB never uses DeviceIoControl() to set the file sparse.)

I tested that the file extension works properly with a multi-file
system tablespace, both with --innodb-use-fallocate and
--skip-innodb-use-fallocate (the default):

./mtr \
--mysqld=--innodb-use-fallocate \
--mysqld=--innodb-autoextend-increment=1 \
--mysqld=--innodb-data-file-path='ibdata1:5M;ibdata2:5M:autoextend' \
--parallel=auto --force --retry=0 --suite=innodb &

ls -lsh mysql-test/var/*/mysqld.1/data/ibdata2
(several samples while running the test)

Increase tokudb_lock_timeout for the test

The function trx_purge_stop() was calling os_event_reset(purge_sys->event)
before calling rw_lock_x_lock(&purge_sys->latch). The os_event_set()
call in srv_purge_coordinator_suspend() is protected by that X-latch.

It would seem a good idea to consistently protect both os_event_set()
and os_event_reset() calls with a common mutex or rw-lock in those
cases where os_event_set() and os_event_reset() are used
like condition variables, tied to changes of shared state.

For each os_event_t, we try to document the mutex or rw-lock that is
being used. For some events, frequent calls to os_event_set() seem to
try to avoid hangs. Some events are never waited for infinitely, only
timed waits, and os_event_set() is used for early termination of these
waits.

os_aio_simulated_put_read_threads_to_sleep(): Define as a null macro
on other systems than Windows. TODO: remove this altogether and disable
innodb_use_native_aio on Windows.

os_aio_segment_wait_events[]: Initialize only if innodb_use_native_aio=0.

MDEV-11802 aims to fix the root cause instead.

The failure happens due to a race condition between processing
a row event (INSERT) and an automatically generated event
DROP TEMPORARY TABLE. Even though DROP has a higher GTID, it can
become visible in @@gtid_slave_pos before the row event with
a lower GTID has been committed. Since the test makes the slave
to synchronize with the master using GTID, the waiting stops
as soon as GTID of the DROP TEMPORARY TABLE becomes visible,
and if changes from the previous event haven't been applied yet,
the error occurs.

According to Kristian (see the comment to MDEV-10631), the real
problem is that DROP TEMPORARY TABLE is logged in the row mode
at all. For this particular test, since DROP does not do anything,
nothing prevents it from competing with the prior transaction.

The workaround for the test is to add a meaningful event
after DROP TEMPORARY TABLE, so that the slave would wait on its
GTID instead of the one from DROP.

Additionally (unrelated to this problem) removed FLUSH TABLES,
which, as the comment stated, should have been removed after
MDEV-6403 was fixed.

The standalone warning is not a sign of a problem, just of slowness,
so it should be added to global suppressions. If a real problem
happens, there will be other errors

* Revert "Make --mem a pure flag. If there is need to specifically set the location"

This reverts commit 716621db.

* MDEV-11619: mtr: when --mem is pure flag, conflicts with $MTR_MEM

Conflicts occurs when MTR_MEM=/xxx/yy ./mtr --mem is invoked. Here
the --mem option overrides opt_mem leaving the default path to be chosen.

This change makes when MTR_MEM set, opt_mem, the flag, is also
set. Both the environment and flag can no be set without conflicting.
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>

* MDEV-11619: if opt_mem is a path include it first

* MDEV-11619: MTR_MEM locations - don't follow symlinks

From Bjorn Munch it seems symlinks can confuse some
tests. Lets just avoid those.

(ref: https://github.com/mysql/mysql-server/pull/116#issuecomment-268479774)
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>

fil_space_extend_must_retry(): When innodb_use_fallocate=ON,
initialize pages_added = size - space->size so that posix_fallocate()
will actually attempt to extend the file, instead of keeping the same size.

This is a regression from MDEV-11556 which refactored
the InnoDB data file extension.

Test results are distorted by a small rounding error
during an intermediate stage of calculating the result.
By using the SQL ROUND function we stablise tests.
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>

Add the warning to the global_suppressions list.

NOT NULL predicate was not added to tables in case of an update query having a subquery.

buf_page_is_checksum_valid_crc32()
buf_page_is_checksum_valid_innodb()
buf_page_is_checksum_valid_none():
	Use ULINTPF instead of %lu and %u for ib_uint32_t

fil_space_verify_crypt_checksum():
	Check that page is really empty if checksum and
	LSN are zero.

fil_space_verify_crypt_checksum():
	Correct the comment to be more agurate.

buf0buf.h:
	Remove unnecessary is_corrupt variable from
	buf_page_t structure.

Use the ib_int64_t type alias instead of the standard type int64_t,
so that the code will compile on Microsoft Visual Studio 2013.

recv_writer_thread(): Do not assign recv_writer_thread_active=true
in order to avoid a race condition with
recv_recovery_from_checkpoint_finish().

recv_init_crash_recovery(): Assign recv_writer_thread_active=true
before creating recv_writer_thread.

Use include/kill_and_restart_mysqld.inc instead of triggering
DBUG_SUICIDE() with debug instrumentation. In this way, more tests
can be run with --valgrind or on non-debug servers.

InnoDB can wrongly ignore the end of data files when using
innodb_page_size=32k or innodb_page_size=64k. These page sizes
use an allocation extent size of 2 or 4 megabytes, not 1 megabyte.

This issue does not affect MariaDB Server 10.2, which is using
the correct WL#5757 code from MySQL 5.7.

That said, it does not make sense to ignore the tail of data files.
The next time the data file needs to be extended, it would be extended
to a multiple of the extent size, once the size exceeds one extent.

Do not effectively set DEBUG_DBUG='d' by setting DEBUG_DBUG='-d,...'.
Instead, restore the saved value of DEBUG_DBUG.

Also, split the test innodb_fts.innodb_fts_misc_debug into
innodb_fts.crash_recovery and innodb_fts.misc_debug, and enable
these tests for --valgrind, the latter test for --embedded,
and the former tests for the non-debug server.

Thanks to Sergei Golubchik for this suggestion.

In order to include the fractional part, convert lock_wait_timeout
(in secs) to nanoseconds before calculating the absolute time for
timeout.

dynamic class GenericPolicy<TTASEventMutex<GenericPolicy> >'; vtable

Instead using mem_heap_alloc and memset, use mem_heap_zalloc
directly.

compatibility problems

Pages that are encrypted contain post encryption checksum on
different location that normal checksum fields. Therefore,
we should before decryption check this checksum to avoid
unencrypting corrupted pages. After decryption we can use
traditional checksum check to detect if page is corrupted
or unencryption was done using incorrect key.

Pages that are page compressed do not contain any checksum,
here we need to fist unencrypt, decompress and finally
use tradional checksum check to detect page corruption
or that we used incorrect key in unencryption.

buf0buf.cc: buf_page_is_corrupted() mofified so that
compressed pages are skipped.

buf0buf.h, buf_block_init(), buf_page_init_low():
removed unnecessary page_encrypted, page_compressed,
stored_checksum, valculated_checksum fields from
buf_page_t

buf_page_get_gen(): use new buf_page_check_corrupt() function
to detect corrupted pages.

buf_page_check_corrupt(): If page was not yet decrypted
check if post encryption checksum still matches.
If page is not anymore encrypted, use buf_page_is_corrupted()
traditional checksum method.

If page is detected as corrupted and it is not encrypted
we print corruption message to error log.
If page is still encrypted or it was encrypted and now
corrupted, we will print message that page is
encrypted to error log.

buf_page_io_complete(): use new buf_page_check_corrupt()
function to detect corrupted pages.

buf_page_decrypt_after_read(): Verify post encryption
checksum before tring to decrypt.

fil0crypt.cc: fil_encrypt_buf() verify post encryption
checksum and ind fil_space_decrypt() return true
if we really decrypted the page.

fil_space_verify_crypt_checksum(): rewrite to use
the method used when calculating post encryption
checksum. We also check if post encryption checksum
matches that traditional checksum check does not
match.

fil0fil.ic: Add missed page type encrypted and page
compressed to fil_get_page_type_name()

Note that this change does not yet fix innochecksum tool,
that will be done in separate MDEV.

Fix test failures caused by buf page corruption injection.

Remove the debug parameter innodb_force_recovery_crash that was
introduced into MySQL 5.6 by me in WL#6494 which allowed InnoDB
to resize the redo log on startup.

Let innodb.log_file_size actually start up the server, but ensure
that the InnoDB storage engine refuses to start up in each of the
scenarios.