- 17 May, 2021 1 commit
-
-
Julius Goryavsky authored
This commit contains a large set of further bug fixes and improvements to SST scripts for Galera, continuing the work that was started in MDEV-24962 to make SST scripts work smoothly in different network configurations (especially using ipv6) and with different environment settings: 1) The ipv6 addresses were incorrectly handled in the SST script for rsync (incorrect address substitution for establishing a connection, incorrect address substitution for bind, and so on); 2) Checking the locality of the ip-address in SST scripts did not support ipv6 addresses (such as "[::1]"), which were falsely identified as non-local ip, which further did not allow running two SSTs on different local addresses on the same machine. On the other hand, this bug masked some other errors (related to handling ipv6 addresses); 3) The code for checking the locality of the ip address was different in the SST scripts for rsync and for mysqldump, with individual flaws. This code is now made common and moved to wsrep_sst_common; 4) Waiting for the start of the transport channel (socat, nc, rsync, stunnel) in the wait_for_listen() and check_pid_and_port() functions did not process ipv6 addresses correctly in all cases (not for all branches); 5) Waiting for the start of the transport channel (socat, nc, rsync, stunnel) in the wait_for_listen() and check_pid_and_port() functions for some code branches could give a false positive result due to the textual match of prefixes in the port number and/or PID of the process; 6) Waiting for the start of the transport channel (socat, nc, rsync, stunnel) was supported through different utilities in SST scripts for mariabackup and for rsync, and with various minor flaws in the code. Now the code is still different in these scripts, but it supports a common set of utilities (lsof, ss, sockstat) and is synchronized across patterns that used to check the output of these utilities; 7) In SST via mariabackup, the signal about readiness to receive data is sometimes sent too early - immediately after listen(), and not after accept() (which are called by socat or netcat utility). 8) Checking availability of the some options of some utilities was done using the grep pattern, which easily gives false positives; 9) Common name (CN) for local addresses, if not explicitly specified, is now always replaced to "localhost" to avoid the need to generate many separate certificates for local addresses of one machine and not to depend on which the local address is currently used in test (ipv4 or ipv6, etc.); 10) In tests galera_sst_mariabackup_encrypt_with_key_server and galera_sst_rsync_encrypt_with_key_server the correct certificate is selected to avoid commonname (CN) mismatch problems; 11) Further refactoring to protect against spaces in file names. 12) Further general refactoring to eliminate bash-specific constructs or to improve code readability; 13) The code for setting options for the nc (netcat) utility was different in different scripts for SST - now it is made identical. 14) Fixed long-time broken encryption via xbcrypt in combination with mariabackup and added support for key-based encryption via openssl utility, which is now enabled by default for encrypt=1 mode (this default mode can be changed using a new configuration file option "encypt-format=openssl|xbcrypt", which can be placed in the [mysqld], [sst] or in the [xtrabackup] section) - this change will allow us to use and to test the encypt=1 encryption without installing non-standard third-party utilities.
-
- 10 May, 2021 3 commits
-
-
Daniel Bartholomew authored
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
- 09 May, 2021 1 commit
-
-
Vladislav Vaintroub authored
This is incosistent with other single config generators, where the default type has always been RelWithDebInfo. Fixed by moving setting of CMAKE_BUILD_TYPE right before PROJECT.
-
- 08 May, 2021 5 commits
-
-
Sergei Golubchik authored
unless explicitly overruled with -DWITH_JEMALLOC=no
-
Sergei Golubchik authored
XA transaction only allows to access data in specific states, in ACTIVE, but not in IDLE or PREPARE. But even then one should be able to run SHOW STATUS.
-
Sergei Golubchik authored
if mysqld is restarted during a test, the debugger script should be reused or overwritten, but not appended to the existing file
-
Sergei Golubchik authored
mysqltest has no --gdb option
-
Sergei Golubchik authored
This reverts commit 72fa9dab but doesn't recover deleted jars - they still exist in mysql-test/connect/std_data, no need to have them twice. Also it removes a redundant copy on JavaWrappers.jar
-
- 07 May, 2021 5 commits
-
-
Vladislav Vaintroub authored
-
Marko Mäkelä authored
-
Oleksandr Byelkin authored
-
Oleksandr Byelkin authored
-
Oleksandr Byelkin authored
-
- 06 May, 2021 1 commit
-
-
Alexey Yurchenko authored
and configuration. 1. Pass joiner's authentication information to donor together with address in State Transfer Request. This allows joiner to authenticate donor on connection. Previously joiner would accept data from anywhere. 2. Deprecate custom SSL configuration variables tca, tcert and tkey in favor of more familiar ssl-ca, ssl-cert and ssl-key. For backward compatibility tca, tcert and tkey are still supported. 3. Allow falling back to server-wide SSL configuration in [mysqld] if no SSL configuration is found in [sst] section of the config file. 4. Introduce ssl-mode variable in [sst] section that takes standard values and has following effects: - old-style SSL configuration present in [sst]: no effect otherwise: - ssl-mode=DISABLED or absent: retains old, backward compatible behavior and ignores any other SSL configuration - ssl-mode=VERIFY*: verify joiner's certificate and CN on donor, verify donor's secret on joiner (passed to donor via State Transfer Request) BACKWARD INCOMPATIBLE BEHAVIOR - anything else enables new SSL configuration convetions but does not require verification ssl-mode should be set to VERIFY only in a fully upgraded cluster. Examples: [mysqld] ssl-cert=/path/to/cert ssl-key=/path/to/key ssl-ca=/path/to/ca [sst] -- server-wide SSL configuration is ignored, SST does not use SSL [mysqld] ssl-cert=/path/to/cert ssl-key=/path/to/key ssl-ca=/path/to/ca [sst] ssl-mode=REQUIRED -- use server-wide SSL configuration for SST but don't attempt to verify the peer identity [sst] ssl-cert=/path/to/cert ssl-key=/path/to/key ssl-ca=/path/to/ca ssl-mode=VERIFY_CA -- use SST-specific SSL configuration for SST and require verification on both sides Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
-
- 05 May, 2021 9 commits
-
-
Alexey Yurchenko authored
1. Fix eval command line to correctly pass stunnel option to rsync on donor. 2. Deprecate `tkey`, `tcert` and `tca` options in [sst] section in favor of conventional `ssl-key`, `ssl-cert` and `ssl-ca`, but keep their precedence for backward compatibility. 3. Default to require SSL encryption if at least SSL key and cert files are specified in configuration, either in [sst] or [mysqld] sections. 4. Enable `verify*` option for stunnel on donor only if a. CA file is specified somewhere in the configuration b. it is explicitly requested in [sst] section by either specifying ssl-mode or CA file there. In this case if ssl-mode is not explicitly given, it defaults to VERIFY_CA. ssl-mode maps to stunnel options as follows: VERIFY_CA -> verifyChain = yes VERIFY_IDENTITY -> verifyPeer = yes Example to require donor to verify joiner identity: ``` [mysqld] ssl-cert=/path/to/cert ssl-key=/path/to/key ssl-ca=/path/to/ca [sst] ssl-mode=VERIFY_IDENTITY ``` 5. If SSL verification is requested, joiner verifies donor by checking the secret passed to donor via SST request. Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
-
Julius Goryavsky authored
-
Julius Goryavsky authored
-
Nikita Malyavin authored
after b9d64989 the test for MDEV-16962 is not suitable anymore (and probably the bug is not reproducible).
-
Oleksandr Byelkin authored
1) Checked presence of mysys_var. 2) Removed unneeded thd_clear_errors, its content added to THD::change_user.
-
Nikita Malyavin authored
-
Nikita Malyavin authored
-
Oleksandr Byelkin authored
-
Sujatha authored
-
- 04 May, 2021 5 commits
-
-
Oleksandr Byelkin authored
-
Sergei Golubchik authored
When you only need view structure, don't call handle_derived with DT_CREATE and rely on its internal hackish check to skip DT_CREATE. Because handle_derived is called from many different places, and this internal hackish check is indiscriminative. Instead, just don't ask handle_derived to do DT_CREATE if you don't want it to do DT_CREATE.
-
Nikita Malyavin authored
-
Jan Lindström authored
galera_var_wsrep_on_off : Add wait conditions to make sure DDL is replicated before continuing. wsrep.[variables|variables_debug] : Remove unnecessary parts and add check to correct number of variables or skip galera_ssl_reload: Add version check and SSL checks.
-
Sergei Golubchik authored
When you only need view structure, don't call handle_derived with DT_CREATE and rely on its internal hackish check to skip DT_CREATE. Because handle_derived is called from many different places, and this internal hackish check is indiscriminative. Instead, just don't ask handle_derived to do DT_CREATE if you don't want it to do DT_CREATE.
-
- 03 May, 2021 6 commits
-
-
Julius Goryavsky authored
After switching to the new mariabackup interface (instead of the outdated innobackupex interface, which is supported for compatibility), we need to explicitly pass a path to the datadir directory as a parameter, since in the new interface the value of this option is not automatically set in such a way that it always matches the SST/IST logic. This commit adds passing this option as an explicit parameter to mariabackup. This commit also removed unnecessary options that are not used and not supported by mariabackup. Also, numerous flaws in the common wsrep_sst_common script have been fixed: 1) There are many bash-specific constructs in the script that may not be supported by other interpreters, which can lead to the most unexpected errors during SST, because failures in the interpretation of bash-specific constructs lead to incorrect parsing of arguments; 2) There is parse_cnf() function which is often called by other scripts for the "mysqld" or "--mysqld" group, but it does not take into account the default group suffix, which leads to reading values only from the default group, which then leads to errors due to reading the default values instead of the values for a specific group; 3) Some options such as --user, --innodb-data-home-dir or --datadir are not removed from the --mysqld-args list, although they are processed inside scripts (and passing of these options funther may cause problems for mariabackup); 4) If an argument that the script understands is present in the --mysqld-args list twice, then this causes SST to fail, instead of reading the most recent value; 5) The "--host" parameter is technically still supported among the arguments of the SST scripts, but in reality scripts do not work with it as expected, especially if it has an IPv6 address; 6) If the port number is absent in the --address parameter value, but the port number is explicitly passed through the --port argument, then the scripts for mariabackup and xtrabackup-v2 fail; 7) If a new address interface is used (with the --address parameter), then automatic default port substitution is not performed, although it is supported for the legacy --host/--port interface. 8) If there are spaces in the parameter values after --mysqld_args, then their further transfer does not occur correctly, which causes mariabackup to fail during SST - the space splits the argument in such a way that it breaks the parsing of the following parameters; 9) If most of the parameters that are names or paths to the files or directories contain spaces, then SST scripts fail in an unpredictable way due to incorrect variable substitutions; 10) If the --log-bin option is passed among the arguments of myqlds (--mysqld-args) without a parameter, and the --binlog option is not specified, then the script cannot substitute the default name for binlog and cannot construct binlog name using the --log-basename argument (which is against server specifications); 11) Tail slashes are not removed from the directory names, which, upon further substitution, leads to the appearance of a double slash in the file paths; 12) The explicit --binlog parameter (which is now always transmitted from the server side) and the "hidden" --log-bin parameter in the list of arguments after --mysqld-args are perceived as two different parameters in different parts of the scripts, and if they are do not match for some reason, this will lead to failures during SST; Also, all new changes from the 10.6 branch have been migrated here, including the latest pull requests for authentication (only the part that concerns SST scripts). It also fixes dozens of other bugs in all SST scripts.
-
Julius Goryavsky authored
Removed numerous extra blank lines and spaces that interfere with reading and understanding program code, making it more difficult to find errors in scripts. I also removed all extra trailing spaces at the ends of lines, which lead to marking extra lines as changes (in subsequent changes). The amount of indentation in some parts of the code has also been normalized.
-
Nikita Malyavin authored
-
Oleksandr Byelkin authored
-
Sergei Petrunia authored
Fix a race condition in the testcase. The testcase assumed that State='Sending data' means that the thread is already in an InnoDB lock wait. This is not case, there is a gap between the state changing to Sending data and execution reaching the point where it is waiting for a lock. Use a more precise check instead, through I_S.INNODB_TRX.
-
Vladislav Vaintroub authored
-
- 30 Apr, 2021 4 commits
-
-
Sergei Petrunia authored
The problem was caused by the following scenario: Subquery's table has two indexes, KEY a(a), KEY a_b(a,b) - LATERAL DERIVED optimization decides to use index a. = The subquery uses ref access over key a. - test_if_skip_sort_order() sees that KEY a_b satisfies the subquery's GROUP BY clause, and attempts to switch to it. = It fails to do so, because KEYUSE objects for index a_b are switched off. Fixed by disallowing to change the ref access key if it uses KEYUSE objects injected by LATERAL DERIVED optimization.
-
Sujatha authored
Problem: ======== 180511 11:07:58 [ERROR] Slave I/O: Unexpected master's heartbeat data: heartbeat is not compatible with local info;the event's data: log_file_name mysql-bin.000009 log_pos 1054262041, Error_code: 1623 Analysis: ========= In replication setup when master server doesn't have any events to send to slave server it sends an 'Heartbeat_log_event'. This event carries the current binary log filename and offset details. The offset values is stored within 4 bytes of event header. When the size of binary log is higher than UINT32_MAX the log_pos values will not fit in 4 bytes memory. It overflows and hence slave stops with an error. Fix: === Since we cannot extend the common_header of Log_event class, a greater than 4GB value of Log_event::log_pos is made to be transported with a HeartBeat event's sub-header. Log_event::log_pos in such case is set to zero to indicate that the 8 byte sub-header is allocated in the event. In case of cross version replication following behaviour is expected OLD - Server without fix NEW - Server with fix OLD<->NEW : works bidirectionally as long as the binlog offset is (normally) within 4GB. When log_pos > UINT32_MAX OLD->NEW : The 'log_pos' is bound to overflow and NEW slave may report an invalid event/incompatible heart beat event error. NEW->OLD : Since patched server sets log_pos=0 on overflow, OLD slave will report invalid event error.
-
Thirunarayanan Balathandayuthapani authored
- Fixing post-push failure of innodb_fts_misc_1 test case.
-
Thirunarayanan Balathandayuthapani authored
InnoDB tries to fetch the deleted doc ids for discarded tablespace. In i_s_fts_deleted_generic_fill(), InnoDB needs to check whether the table is discarded or not before fetching deleted doc ids.
-