An error occurred fetching the project authors.
- 19 May, 2021 3 commits
-
-
Monty authored
This change is to get rid of randomly failing tests, especially those that reads random position of the binary log. From looking at the logs it's clear that some failures is because of a read char (with value >= 128) is converted to a big long value. Using uchar everywhere makes this much less likely to happen. Another benefit is that a lot of cast of char to uchar could be removed. Other things: - Removed some extra space before '=' and '+=' in assignments - Fixed indentations and lines > 80 characters - Replace '16' with 'element_size' (from class definition) in Gtid_list_log_event()
-
Monty authored
Changes: - To detect automatic strlen() I removed the methods in String that uses 'const char *' without a length: - String::append(const char*) - Binary_string(const char *str) - String(const char *str, CHARSET_INFO *cs) - append_for_single_quote(const char *) All usage of append(const char*) is changed to either use String::append(char), String::append(const char*, size_t length) or String::append(LEX_CSTRING) - Added STRING_WITH_LEN() around constant string arguments to String::append() - Added overflow argument to escape_string_for_mysql() and escape_quotes_for_mysql() instead of returning (size_t) -1 on overflow. This was needed as most usage of the above functions never tested the result for -1 and would have given wrong results or crashes in case of overflows. - Added Item_func_or_sum::func_name_cstring(), which returns LEX_CSTRING. Changed all Item_func::func_name()'s to func_name_cstring()'s. The old Item_func_or_sum::func_name() is now an inline function that returns func_name_cstring().str. - Changed Item::mode_name() and Item::func_name_ext() to return LEX_CSTRING. - Changed for some functions the name argument from const char * to to const LEX_CSTRING &: - Item::Item_func_fix_attributes() - Item::check_type_...() - Type_std_attributes::agg_item_collations() - Type_std_attributes::agg_item_set_converter() - Type_std_attributes::agg_arg_charsets...() - Type_handler_hybrid_field_type::aggregate_for_result() - Type_handler_geometry::check_type_geom_or_binary() - Type_handler::Item_func_or_sum_illegal_param() - Predicant_to_list_comparator::add_value_skip_null() - Predicant_to_list_comparator::add_value() - cmp_item_row::prepare_comparators() - cmp_item_row::aggregate_row_elements_for_comparison() - Cursor_ref::print_func() - Removes String_space() as it was only used in one cases and that could be simplified to not use String_space(), thanks to the fixed my_vsnprintf(). - Added some const LEX_CSTRING's for common strings: - NULL_clex_str, DATA_clex_str, INDEX_clex_str. - Changed primary_key_name to a LEX_CSTRING - Renamed String::set_quick() to String::set_buffer_if_not_allocated() to clarify what the function really does. - Rename of protocol function: bool store(const char *from, CHARSET_INFO *cs) to bool store_string_or_null(const char *from, CHARSET_INFO *cs). This was done to both clarify the difference between this 'store' function and also to make it easier to find unoptimal usage of store() calls. - Added Protocol::store(const LEX_CSTRING*, CHARSET_INFO*) - Changed some 'const char*' arrays to instead be of type LEX_CSTRING. - class Item_func_units now used LEX_CSTRING for name. Other things: - Fixed a bug in mysql.cc:construct_prompt() where a wrong escape character in the prompt would cause some part of the prompt to be duplicated. - Fixed a lot of instances where the length of the argument to append is known or easily obtain but was not used. - Removed some not needed 'virtual' definition for functions that was inherited from the parent. I added override to these. - Fixed Ordered_key::print() to preallocate needed buffer. Old code could case memory overruns. - Simplified some loops when adding char * to a String with delimiters.
-
Monty authored
The problem was that when one used String::alloc() to allocate a string, the String ensures that there is space for an extra NULL byte in the buffer and if not, reallocates the string. This is a problem with the String::set_int() that calls alloc(21), which forces extra malloc/free calls to happen. - We do not anymore re-allocate String if alloc() is called with the Allocated_length. This reduces number of malloc() allocations, especially one big re-allocation in Protocol::send_result_Set_metadata() for almost every query that produced a result to the connnected client. - Avoid extra mallocs when using LONGLONG_BUFFER_SIZE This can now be done as alloc() doesn't increase buffers if new length is not bigger than old one. - c_ptr() is redesigned to be safer (but a bit longer) than before. - Remove wrong usage of c_ptr_quick() c_ptr_quick() was used in many cases to get the pointer to the used buffer, even when it didn't need to be \0 terminated. In this case ptr() is a better substitute. Another problem with c_ptr_quick() is that it did not guarantee that the string would be \0 terminated. - item_val_str(), an API function not used currently by the server, now always returns a null terminated string (before it didn't always do that). - Ensure that all String allocations uses STRING_PSI_MEMORY_KEY. The old mixed usage of performance keys caused assert's when String buffers where shrunk. - Binary_string::shrink() is simplifed - Fixed bug in String(const char *str, size_t len, CHARSET_INFO *cs) that used Binary_string((char *) str, len) instead of Binary_string(str,len). - Changed argument to String() creations and String.set() functions to use 'const char*' instead of 'char*'. This ensures that Alloced_length is not set, which gives safety against someone trying to change the original string. This also would allow us to use !Alloced_length in c_ptr() if needed. - Changed string_ptr_cmp() to use memcmp() instead of c_ptr() to avoid a possible malloc during string comparision.
-
- 30 Apr, 2021 1 commit
-
-
Sujatha authored
Problem: ======== 180511 11:07:58 [ERROR] Slave I/O: Unexpected master's heartbeat data: heartbeat is not compatible with local info;the event's data: log_file_name mysql-bin.000009 log_pos 1054262041, Error_code: 1623 Analysis: ========= In replication setup when master server doesn't have any events to send to slave server it sends an 'Heartbeat_log_event'. This event carries the current binary log filename and offset details. The offset values is stored within 4 bytes of event header. When the size of binary log is higher than UINT32_MAX the log_pos values will not fit in 4 bytes memory. It overflows and hence slave stops with an error. Fix: === Since we cannot extend the common_header of Log_event class, a greater than 4GB value of Log_event::log_pos is made to be transported with a HeartBeat event's sub-header. Log_event::log_pos in such case is set to zero to indicate that the 8 byte sub-header is allocated in the event. In case of cross version replication following behaviour is expected OLD - Server without fix NEW - Server with fix OLD<->NEW : works bidirectionally as long as the binlog offset is (normally) within 4GB. When log_pos > UINT32_MAX OLD->NEW : The 'log_pos' is bound to overflow and NEW slave may report an invalid event/incompatible heart beat event error. NEW->OLD : Since patched server sets log_pos=0 on overflow, OLD slave will report invalid event error.
-
- 31 Mar, 2021 1 commit
-
-
Sergei Petrunia authored
rocksdb_checkpoint_request() should call FlushWAL(sync=true) (which does write-out and sync), not just SyncWAL() (which just syncs without writing out) Followup: the test requires debug sync facility (This is a backport to 10.5)
-
- 30 Mar, 2021 1 commit
-
-
Sergei Petrunia authored
rocksdb_checkpoint_request() should call FlushWAL(sync=true) (which does write-out and sync), not just SyncWAL() (which just syncs without writing out)
-
- 22 Mar, 2021 1 commit
-
-
Otto Kekäläinen authored
Reseting -> Resetting Unknow -> Unknown capabilites -> capabilities choosen -> chosen direcory -> directory informations -> information openned -> opened refered -> referred to access -> one to access missmatch -> mismatch succesfully -> successfully dont -> don't
-
- 12 Feb, 2021 2 commits
-
-
Sergei Golubchik authored
adaptation of 29bbcac0 for 10.4
-
Sergei Golubchik authored
-
- 21 Jan, 2021 1 commit
-
-
Sujatha authored
Problem: ======== Auto purge of relaylogs stops when relay-log-file is 'slave-relay-log.999999' and slave_parallel_threads is enabled. Analysis: ========= The problem is that in Relay_log_info::inc_group_relay_log_pos() function, when two log names are compared via strcmp() function, it gives correct result, when log name sequence numbers are of same digits(6 digits), But when the number goes to 7 digits, a 999999 compares greater than 1000000, which is wrong, hence the bug. Fix: ==== Extract the numeric extension part of the file name, convert it into unsigned long and compare. Thanks to David Zhao for the contribution.
-
- 16 Sep, 2020 1 commit
-
-
Sujatha authored
Problem: ======= SHOW BINLOG EVENTS FROM <"random"-pos> caused a variety of failures as reported in MDEV-18046. They are fixed but that approach is not future-proof as well as is not optimal to create extra check for being constructed event parameters. Analysis: ========= "show binlog events from <pos>" code considers the user given position as a valid event start position. The code starts reading data from this event start position onwards and tries to map it to a set of known events. Each event has a specific event structure and asserts have been added to ensure that, read event data, satisfies the event specific requirements. When a random position is supplied to "show binlog events command" the event structure specific checks will fail and they result in assert. For example: https://jira.mariadb.org/browse/MDEV-18046 In the bug description user executes CREATE TABLE/INSERT and ALTER SQL commands. When a crazy offset like "SHOW BINLOG EVENTS FROM 365" is provided code assumes offset 365 as valid event begin and proceeds to EVENT_LEN_OFFSET reads some random length and comes up with a crazy event which didn't exits in the binary log. In this quoted example scenario, event read at offset 365 is considered as "Update_rows_log_event", which is not present in binary log. Since this is a random event its validation fails and code results in assert/segmentation fault, as shown below. mysqld: /data/src/10.4/sql/log_event.cc:10863: Rows_log_event::Rows_log_event( const char*, uint, const Format_description_log_event*): Assertion `var_header_len >= 2' failed. 181220 15:27:02 [ERROR] mysqld got signal 6 ; #7 0x00007fa0d96abee2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6 #8 0x000055e744ef82de in Rows_log_event::Rows_log_event (this=0x7fa05800d390, buf=0x7fa05800d080 "", event_len=254, description_event=0x7fa058006d60) at /data/src/10.4/sql/log_event.cc:10863 #9 0x000055e744f00cf8 in Update_rows_log_event::Update_rows_log_event Since we are reading random data repeating the same command SHOW BINLOG EVENTS FROM 365 produces different types of crashes with different events. MDEV-18046 reported 10 such crashes. In order to avoid such scenarios user provided starting offset needs to be validated for its correctness. Best way of doing this is to make use of checksums if they are available. MDEV-18046 fix introduced the checksum based validation. The issue still remains in cases where binlog checksums are disabled. Please find the following bug reports. MDEV-22473: binlog.binlog_show_binlog_event_random_pos failed in buildbot, server crashed in read_log_event MDEV-22455: Server crashes in Table_map_log_event, binlog.binlog_invalid_read_in_rotate failed in buildbot Fix: ==== When binlog checksum is disabled, perform scan(via reading event by event), to validate the requested FROM <pos> offset. Starting from offset 4 read the event_length of next_event in the binary log. Using the next_event length advance current offset to point to next event. Repeat this process till the current offset is less than or equal to crazy offset. If current offset is higher than crazy offset provide appropriate invalid input offset error.
-
- 13 Aug, 2020 1 commit
-
-
Marko Mäkelä authored
Fix some more cases after merging commit 31aef3ae. Some warnings look possibly genuine, others are clearly bogus.
-
- 16 Jun, 2020 1 commit
-
-
Sujatha authored
Problem: ======= The "Start binlog_dump" message hasn't been updated to include the slave's requested GTID position: 20:05:05 139836760311552 [Note] Start binlog_dump to slave_server(2), pos(, 4) For diagnostic purposes, it would be helpful if the GTID position were included. Fix: === Imporve "Start binlog_dump" print message to include "using_gtid" and "GTID position" requested by slave. Ex: [Note] Start binlog_dump to slave_server(2), pos(, 4), using_gtid(1), gtid('1-1-201,2-2-100') [Note] Start binlog_dump to slave_server(3), pos('mariadb-bin.004142', 507988273), using_gtid(0), gtid('')
-
- 15 May, 2020 1 commit
-
-
Monty authored
-
- 24 Apr, 2020 1 commit
-
-
Jan Lindström authored
Replaced WSREP_ON macro by single global variable WSREP_ON that is then updated at server statup and on wsrep_on and wsrep_provider update functions.
-
- 14 Mar, 2020 1 commit
-
-
Andrei Elkin authored
Lifted long standing limitation to the XA of rolling it back at the transaction's connection close even if the XA is prepared. Prepared XA-transaction is made to sustain connection close or server restart. The patch consists of - binary logging extension to write prepared XA part of transaction signified with its XID in a new XA_prepare_log_event. The concusion part - with Commit or Rollback decision - is logged separately as Query_log_event. That is in the binlog the XA consists of two separate group of events. That makes the whole XA possibly interweaving in binlog with other XA:s or regular transaction but with no harm to replication and data consistency. Gtid_log_event receives two more flags to identify which of the two XA phases of the transaction it represents. With either flag set also XID info is added to the event. When binlog is ON on the server XID::formatID is constrained to 4 bytes. - engines are made aware of the server policy to keep up user prepared XA:s so they (Innodb, rocksdb) don't roll them back anymore at their disconnect methods. - slave applier is refined to cope with two phase logged XA:s including parallel modes of execution. This patch does not address crash-safe logging of the new events which is being addressed by MDEV-21469. CORNER CASES: read-only, pure myisam, binlog-*, @@skip_log_bin, etc Are addressed along the following policies. 1. The read-only at reconnect marks XID to fail for future completion with ER_XA_RBROLLBACK. 2. binlog-* filtered XA when it changes engine data is regarded as loggable even when nothing got cached for binlog. An empty XA-prepare group is recorded. Consequent Commit-or-Rollback succeeds in the Engine(s) as well as recorded into binlog. 3. The same applies to the non-transactional engine XA. 4. @@skip_log_bin=OFF does not record anything at XA-prepare (obviously), but the completion event is recorded into binlog to admit inconsistency with slave. The following actions are taken by the patch. At XA-prepare: when empty binlog cache - don't do anything to binlog if RO, otherwise write empty XA_prepare (assert(binlog-filter case)). At Disconnect: when Prepared && RO (=> no binlogging was done) set Xid_cache_element::error := ER_XA_RBROLLBACK *keep* XID in the cache, and rollback the transaction. At XA-"complete": Discover the error, if any don't binlog the "complete", return the error to the user. Kudos ----- Alexey Botchkov took to drive this work initially. Sergei Golubchik, Sergei Petrunja, Marko Mäkelä provided a number of good recommendations. Sergei Voitovich made a magnificent review and improvements to the code. They all deserve a bunch of thanks for making this work done!
-
- 10 Mar, 2020 2 commits
-
-
Alexander Barkov authored
-
Sergei Golubchik authored
-
- 28 Jan, 2020 1 commit
-
-
Alexander Barkov authored
-
- 08 Jan, 2020 1 commit
-
- 07 Jan, 2020 1 commit
-
-
Sujatha authored
Problem: ======== SHOW BINLOG EVENTS FROM <pos> causes a variety of failures, some of which are listed below. It is not a race condition issue, but there is some non-determinism in it. Analysis: ======== "show binlog events from <pos>" code considers the user given position as a valid event start position. The code starts reading data from this event start position onwards and tries to map it to a set of known events. Each event has a specific event structure and asserts have been added to ensure that read event data satisfies the event specific requirements. When a random position is supplied to "show binlog events command" the event structure specific checks will fail and they result in assert. Fix: ==== The fix is split into different parts. Each part addresses either an ASAN issue or an assert/crash. **Part1: Checksum based position validation when checksum is enabled** Using checksum validate the very first event read at the user specified position. If there is a checksum mismatch report an appropriate error for the invalid event.
-
- 02 Dec, 2019 1 commit
-
-
Faustin Lammler authored
The lintian check complains on spelling error: https://salsa.debian.org/mariadb-team/mariadb-10.3/-/jobs/95739
-
- 10 Nov, 2019 1 commit
-
-
Andrei Elkin authored
MDEV-19376 Repl_semi_sync_master::commit_trx assertion failure: ... || !m_active_tranxs->is_tranx_end_pos(trx_wait_binlog_name, trx_wait_binlog_pos) The assert indicates that the current transaction got caught uncleaned from the semisync master's cache when it is signaled to proceed upon its ack receive. The reason of missed cleanup turns out to be a flaw in the gtid connect mode. A submitted by connecting slave value of its last received event's binlog file *name* was adopted into {{Repl_semi_sync_master::m_reply_file_name}} as a part of semisync initialization. Notice that the initialization still refines the position part of the submitted last received event's binlog coordinates. The master side binlog filename:pos refinement is specific to the gtid connect mode for purpose of computing the latest binlog file to resume slave feeding from. Effectively in the gtid connect mode the computed resumption filename:pos may appear smaller in which case a new post-connect time committing transaction may be logged with its filename:pos also less than the submitted coordinates and that triggers the assert. Fixed with making the semisync initialization to use the refined filename:pos. It is guaranteed to be less than any new generated transaction's binlog:pos.
-
- 13 Oct, 2019 2 commits
-
-
Oleksandr Byelkin authored
Now both offset and limit are stored and do not chenged during execution (offset is decreased during processing in versions before 10.5). (Big part of this changes made by Monty)
-
Oleksandr Byelkin authored
-
- 08 Oct, 2019 1 commit
-
-
Sachin Setiya authored
MDEV-20574 Position of events reported by mysqlbinlog is wrong with encrypted binlogs, SHOW BINLOG EVENTS reports the correct one. Analysis Mysqlbinlog output for encrypted binary log #Q> insert into tab1 values (3,'row 003') #190912 17:36:35 server id 10221 end_log_pos 980 CRC32 0x53bcb3d3 Table_map: `test`.`tab1` mapped to number 19 # at 940 #190912 17:36:35 server id 10221 end_log_pos 1026 CRC32 0xf2ae5136 Write_rows: table id 19 flags: STMT_END_F Here we can see Table_map_log_event ends at 980 but Next event starts at 940. And the reason for that is we do not send START_ENCRYPTION_EVENT to the slave Solution:- Send Start_encryption_log_event as Ignorable_log_event to slave(mysqlbinlog), So that mysqlbinlog can update its log_pos. Since Slave can request multiple FORMAT_DESCRIPTION_EVENT while master does not have so We only update slave master pos when master actually have the FORMAT_DESCRIPTION_EVENT. Similar logic should be applied for START_ENCRYPTION_EVENT. Also added the test case when new server reads the data from old server which does not send START_ENCRYPTION_EVENT to slave. Master Slave Upgrade Scenario. When Slave is updated first, Slave will have extra logic of handling START_ENCRYPTION_EVENT But master willnot be sending START_ENCRYPTION_EVENT. So there will be no issue. When Master is updated first, It will send START_ENCRYPTION_EVENT to slave , But slave will ignore this event in queue_event.
-
- 11 May, 2019 1 commit
-
-
Vicențiu Ciorbaru authored
* Update wrong zip-code
-
- 01 Apr, 2019 2 commits
-
-
Monty authored
On some systems with 10,000+ binlogs, show binary logs could block log rotation for more than 10 seconds. This patch fixes this by first caching all binary log names and releases all mutexes while calculating the sizes of the binary logs. Other things: - Ensure that reinit_io_cache() sets end_of_file when moving to read_cache. This ensures that external changes of the underlying file is known to the cache. - get_binlog_list() is made more efficent and show_binlogs() is changed to call get_binlog_list() Reviewed by Andrei Elkin
-
Monty authored
-
- 12 Mar, 2019 1 commit
-
-
Sergey Vojtovich authored
The patches features an optional shutdown behavior to hold on until after all connected slaves have been sent the last binlogged event. The connected slave is one whose START SLAVE has been acknowledged and that was not stopped since that though it could be technically reconnecting in background. The solution therefore disallows killing the dump thread until is has found EOF of the latest binlog file. It is up to the shutdown requester (DBA) to set up a sufficiently large shutdown timeout value for shudown to wait patiently until lagging behind slaves have been synchronized. On the other hand if a specific slave needs exclusion from synchronization the DBA would have to stop it manually which would terminate its dump thread. `mysqladmin shutdown' is extended with a `--wait_for_all_slaves' option which translates to `SHUTDOW WAIT FOR ALL SLAVES' sql query to enable the feature on the client side. The patch also performs a small refactoring of the server shutdown around close_connections() to introduce kill thread phases which are two as of current.
-
- 19 Feb, 2019 1 commit
-
-
mkaruza authored
Temporary disable WSREP while executing RESET MASTER. In situation when 2 nodes are both master/slave first stop slave on both and than reset master. Enforce stricter causality check with wsrep_sync_wait.
-
- 28 Jan, 2019 2 commits
-
-
Sergey Vojtovich authored
Implemented and integrated THD_list as a replacement for the global thread list. It uses own mutex instead of LOCK_thread_count for THD list protection. Removed unused first_global_thread() and next_global_thread(). delayed_insert_threads is now protected by LOCK_delayed_insert. Although this patch doesn't fix very wrong synchronization of this variable. After this patch there are only 2 legitimate uses of LOCK_thread_count left, both in mysqld.cc: thread_count and ready_to_exit. Aim is to reduce usage of LOCK_thread_count and COND_thread_count. Part of MDEV-15135.
-
Sergey Vojtovich authored
LOG_INFO::lock was useless. It could've only protect against concurrent iterators execution, which was already protected by LOCK_thread_count. Use LOCK_thd_data instead of LOCK_thread_count as a protection against THD::current_linfo reset. Aim is to reduce usage of LOCK_thread_count and COND_thread_count. Part of MDEV-15135.
-
- 23 Jan, 2019 2 commits
-
-
Aditya A authored
PROBLEM ------- Memory sanitizer reports uninitialized comparisons in log_in_use(), because strings are compared with memcmp() instead of strncmp. FIX --- Use strncmp() to compare strings
-
Brave Galera Crew authored
-
- 28 Jun, 2018 2 commits
-
-
Andrei Elkin authored
MDEV-7257 made a dump thread to read from binlog concurrently with writers as long as the read bytes are below a water-mark (MYSQL_BIN_LOG::binlog_end_pos). However it appeared to be possible a dump thread reader reach out for bytes past the water mark through a feature of IO_CACHE that fills in the internal buffer and while doing so it could read what the reader is not supposed to see (the bytes above MYSQL_BIN_LOG::binlog_end_pos). The issue is fixed with constraining the IO_CACHE buffer fill to respect the watermark. An added unit test proves reading from file is bound to an external parameter passed to {IO_CACHE::end_of_file} cache member.
-
Andrei Elkin authored
MDEV-7257 made a dump thread to read from binlog concurrently with writers as long as the read bytes are below a water-mark (MYSQL_BIN_LOG::binlog_end_pos). However it appeared to be possible a dump thread reader reach out for bytes past the water mark through a feature of IO_CACHE that fills in the internal buffer and while doing so it could read what the reader is not supposed to see (the bytes above MYSQL_BIN_LOG::binlog_end_pos). The issue is fixed with constraining the IO_CACHE buffer fill to respect the watermark. An added unit test proves reading from file is bound to an external parameter passed to {IO_CACHE::end_of_file} cache member.
-
- 04 Jun, 2018 1 commit
-
-
Sergei Golubchik authored
-
- 06 May, 2018 1 commit
-
-
Monty authored
Added to: - if (error) - Lex - sql_yacc.yy and sql_yacc_ora.yy - In header files to alloc() calls - Added thd argument to thd_net_is_killed()
-
- 12 Mar, 2018 1 commit
-
-
Andrei Elkin authored
replicate_events_marked_for_skip=FILTER_ON_MASTER [Note this is a cherry-pick from 10.2 branch.] When events of a big transaction are binlogged offsetting over 2GB from the beginning of the log the semisync master's dump thread lost such events. The events were skipped by the Dump thread that found their skipping status erroneously. The current fixes make sure the skipping status is computed correctly. The test verifies them simulating the 2GB offset.
-