GitLab

Row-based replication requires the types of columns on the
master and slave to be approximately the same (some safe
conversions between strings are allowed), but does not
allow safe conversions between fields of similar types such
as TINYINT and INT.

This patch implement type conversions between similar fields
on the master and slave.

The conversions are controlled using a new variable
SLAVE_TYPE_CONVERSIONS of type SET('ALL_LOSSY','ALL_NON_LOSSY').

Non-lossy conversions are any conversions that do not run the
risk of losing any information, while lossy conversions can
potentially truncate the value. The column definitions are
checked to decide if the conversion is acceptable.

If neither conversion is enabled, it is required that the
definitions of the columns are identical on master and slave.

Conversion is done by creating an internal conversion table,
unpacking the master data into it, and then copy the data to
the real table on the slave.

.bzrignore:
  New files added
client/Makefile.am:
  New files added
client/mysqlbinlog.cc:
  Functions in rpl_utility.cc is now needed by mysqlbinlog.cc.
libmysqld/Makefile.am:
  New files added
mysql-test/extra/rpl_tests/check_type.inc:
  Test include file to check a single type conversion.
mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test:
  Switching to use INT instead of TEXT for column that should not have matching types.
mysql-test/extra/rpl_tests/rpl_row_basic.test:
  Adding code to enable type conversions for BIT tests since InnoDB
  cannot handle them properly due to incorrect information stored as
  metadata.
mysql-test/extra/rpl_tests/type_conversions.test:
  Test file to check a set of type conversions
  with current settings of slave_type_conversions.
mysql-test/suite/rpl/t/rpl_typeconv.test:
  Test file to test conversions from master to slave with
  all possible values for slave_type_conversions.
  
  The test also checks that the slave_type_conversions
  variable works as expected.
sql/field.cc:
  Changing definition of compatible_field_size to both check if 
  two field with identical base types are compatible and give an
  order between them if they are compatible.
  
  This only implement checking on the slave, so it will not affect
  replication from an old master to a new slave.
sql/field.h:
  Changing prototypes for functions:
  - compatible_field_size()
  - init_for_tmp_table()
  - row_pack_length()
sql/log_event.cc:
  Changing compability checks to build a conversion table if the fields
  are compatible, but does not have the same base type.
sql/log_event_old.cc:
  Changing compability checks to build a conversion table if the fields
  are compatible, but does not have the same base type.
sql/mysql_priv.h:
  Adding global option variable for SLAVE_TYPE_CONVERSIONS
sql/mysqld.cc:
  Adding SLAVE_TYPE_CONVERSIONS global server variable.
sql/rpl_record.cc:
  Changing unpack_row to use the conversion table if present.
sql/rpl_rli.h:
  Removing function get_tabledef and replacing it with get_table_data().
  This function retrieve data for table opened for replication, not just
  table definition.
sql/rpl_utility.cc:
  Function table_def::compatible_with is changed to compare table on master
  and slave for compatibility and generate a conversions table if they are
  compatible.
  
  Computing real type of fields from metadata for ENUM and SET types.
  Computing pack_length correctly for ENUM, SET, and BLOB types.
  
  Adding optimization to not check compatibility if no
  slave type conversions are enabled.
sql/rpl_utility.h:
  Changing prototypes since implementation has changed.
  
  Modifying table_def::type() to return real type instead of stored type.
sql/set_var.cc:
  Adding SLAVE_TYPE_CONVERSIONS variable.
sql/set_var.h:
  Adding SLAVE_TYPE_CONVERSIONS variable.
sql/share/errmsg.txt:
  Adding error messages for slave type conversions.
sql/sql_class.h:
  Adding SLAVE_TYPE_CONVERSIONS variable.
sql/sql_select.cc:
  Correcting create_virtual_tmp_table() to compute null bit positions
  correctly in the presence of bit fields.

The mentioned on the bug report set of bugs fixes have not be pushed to the main trees.

Fixed with extracting commits done to 6.0-rpl tree and applying them to the main 5.1.
Notes.
1. part of changes - the mtr's specific - were packported to the main 5.0 tree for mtr v1
   as http://lists.mysql.com/commits/46562
   However, there is no that fix anymore in the mtr v2. (This fact was mailed to mtr maintaining
   people).

2. Bug@36929  crash in kill_zombie_dump_threads-> THD::awake() with replication tests
   is not backported because the base code of the patch is libevent and that was removed
   from the main trees due to its instability.

client/mysqlbinlog.cc:
  fixes for BUG#35546
mysql-test/suite/rpl/r/rpl_bug41902.result:
  the new tests result file is added.
mysql-test/suite/rpl/t/rpl_bug41902-slave.opt:
  conf file for bug41902 testing is added.
mysql-test/suite/rpl/t/rpl_bug41902.test:
  regression tests for Bug #41902 is added.
sql/log.cc:
  collection of changes due to Bug #48463.
sql/log.h:
  collection of changes due to Bug #48463.
sql/rpl_rli.h:
  collection of changes due to Bug #48463.
sql/slave.cc:
  collection of changes due to Bug #48463.
sql/sql_repl.cc:
  collection of changes due to Bug #48463.

The reason for the bug is that mysqtest as well as other client tools
running in test suite (mysqlbinlog, mysqldump) will first try to connect 
whatever database has created shared memory with default base name 
"MySQL" and use this. (Same effect could be seen on Unix if mtr would
not care to calculate "port" and "socket" parameter).
      
The fix ensures that all client tools and  running in mtr use unique  
per-database shared memory base parameters, so there is no possibility
to clash with already installed one. We use socket name for shared memory 
base (it's known to be unique). This shared-memory-base is written to the
MTR config file to the [client] and [mysqld] sections. Fix made also made 
sure all client tools understand and correctly handle --shared-memory-base.
Prior to this patch  it was not the case for  mysqltest, mysqlbinlog and 
mysql_client_test.
      
All new connections done from mtr scripts via connect() will by default 
set shared-memory-base. And finally, there is a possibility to force 
shared memory or pipe connection and overwrite shared memory/pipe base name
from within mtr scripts via optional PIPE or SHM modifier. This functionality
was manually backported from 6.0
(original patch  http://lists.mysql.com/commits/74749)

Append the description of the 'decode-rows' value for --base64-output argument.

There are some problems about help text:
- It is stated that "auto" is the default twice. It need be stated only once.
- It is stated that --base64-output is short for --base64-output=always. But that sounds
like the default is "always", not "auto".

Make the help text clear as following:
Determine when the output statements should be base64-encoded BINLOG 
statements: 'never' disables it and works only for binlogs without 
row-based events; 'auto' prints base64 only when necessary (i.e., 
for row-based events and format description events); 'always' prints 
base64 whenever possible. 'always' is for debugging only and should 
not be used in a production system. If this argument is not given, 
the default is 'auto'; if it is given with no argument, 'always' is used.

    added code to exit a tool if the forced config file wasn't found

per-file comments:
  client/mysql.cc
Bug#47216      programs should quit if the file specified by --defaults-file option isn't found
    added code to exit a tool if the forced config file wasn't found
  client/mysql_upgrade.c
Bug#47216      programs should quit if the file specified by --defaults-file option isn't found
    added code to exit a tool if the forced config file wasn't found
  client/mysqladmin.cc
Bug#47216      programs should quit if the file specified by --defaults-file option isn't found
    added code to exit a tool if the forced config file wasn't found
  client/mysqlcheck.c
Bug#47216      programs should quit if the file specified by --defaults-file option isn't found
    added code to exit a tool if the forced config file wasn't found
  client/mysqldump.c
Bug#47216      programs should quit if the file specified by --defaults-file option isn't found
    added code to exit a tool if the forced config file wasn't found
  client/mysqlimport.c
Bug#47216      programs should quit if the file specified by --defaults-file option isn't found
    added code to exit a tool if the forced config file wasn't found
  client/mysqlshow.c
Bug#47216      programs should quit if the file specified by --defaults-file option isn't found
    added code to exit a tool if the forced config file wasn't found
  client/mysqlslap.c
Bug#47216      programs should quit if the file specified by --defaults-file option isn't found
    added code to exit a tool if the forced config file wasn't found
  mysql-test/t/mysql.test
Bug#47216      programs should quit if the file specified by --defaults-file option isn't found
   test added
  sql/mysqld.cc
Bug#47216      programs should quit if the file specified by --defaults-file option isn't found
    added code to exit a tool if the forced config file wasn't found
  storage/myisam/myisamchk.c
Bug#47216      programs should quit if the file specified by --defaults-file option isn't found
    added code to exit a tool if the forced config file wasn't found
  storage/myisam/myisampack.c
Bug#47216      programs should quit if the file specified by --defaults-file option isn't found
    added code to exit a tool if the forced config file wasn't found

There were two memory leaks in mysqlbinlog command, one was already
fixed by previous patches, another one was that defaults_argv was
set to the value of argv after parse_args, in which called
handle_options after calling load_defaults and changed the value
of argv, and caused the memory allocated for defaults arguments
not freed.

Fixed the problem by setting defaults_argv right after calling
load_defaults.

The 'BEGIN/COMMIT/ROLLBACK' log event could be filtered out if the
database is not selected by --database option of mysqlbinlog command.
This can result in problem if there are some statements in the
transaction are not filtered out.

To fix the problem, mysqlbinlog will output 'BEGIN/ROLLBACK/COMMIT' 
in regardless of the database filtering rules.

client/mysqlbinlog.cc:
  Skip the database check for BEGIN/COMMIT/ROLLBACK log events.
mysql-test/r/mysqlbinlog.result:
  Test result for bug#46998
mysql-test/t/mysqlbinlog.test:
  Added test to verify if the 'BEGIN', 'COMMIT' and 'ROLLBACK' are output
  in regardless of database filtering

Bug#24509 - 2048 file descriptor limit on windows needs increasing, also 
WL#3049 - improved Windows I/O
                        
The patch replaces the use of the POSIX I/O interfaces in mysys on Windows with 
the Win32 API calls (CreateFile, WriteFile, etc). The Windows HANDLE for the open 
file is stored in the my_file_info struct, along with a flag for append mode 
because the Windows API does not support opening files in append mode in all cases)
The default max open files has been increased to 16384 and can be increased further
by setting --max-open-files=<value> during the server start.
                              
Another major change in this patch that almost all Windows specific file IO code
has been moved to a new file my_winfile.c, greatly reducing the amount of code 
in #ifdef blocks within mysys, thus improving readability.
                               
                                    
Minor enhancements:
- my_(f)stat() is changed to use __stati64 structure wi...

            
mysqlbinlog --database parameter was being ignored when processing
row events. As such no event filtering would take place.
            
This patch addresses this by deploying a call to shall_skip_database
when table_map_events are handled (as these contain also the name of
the database). All other rows events referencing the table id for the
filtered map event, will also be skipped.

client/mysqlbinlog.cc:
  Added shall_skip_database call to the part of the code that handles 
  Table_map_log_events. It inspects the database name and decides whether
  to filter the event or not. Furthermore, if table map event is filtered
  next events referencing the table id in the table map event, will also
  be filtered.
mysql-test/suite/binlog/t/binlog_row_mysqlbinlog_db_filter.test:
  Test case that checks if row events are actually filtered out.
sql/log_event.h:
  Added a map for holding the currently ignored table map events.
  Table map events are inserted when they shall be skipped and removed
  once the last row event in the statement is processed.

intentionally unmentioned (it is just a placeholder). (Bug #41403)

are supported. (Bug #42021)

handle the --skip-password option correctly. (Bug #28479)

  
The --hexdump option crashed mysqlbinlog when used together  
with the --read-from-remote-server option due to use of  
uninitialized memory.  
  
Since Log_event::print_header() relies on temp_buf to be  
initialized when the --hexdump option is present,  
dump_remote_log_entries() was fixed to setup temp_buf to point  
to the start of a binlog event as done in  
dump_local_log_entries().  
 
The root cause of this bug is identical to the one for 
bug #17654. The latter was fixed in 5.1 and up, so this 
patch is backport of the patches for bug #17654 to 5.0. 
 
Only 5.0 needs a changelog entry. 

client/mysqlbinlog.cc:
  Fixed dump_remote_log_entries() so that temp_buf is initialized 
  as it may be used later by Log_event::print_header() if the 
  --hexdump option is present.
mysql-test/r/mysqlbinlog.result:
  Added a test case for bug #41943.
mysql-test/t/mysqlbinlog.test:
  Added a test case for bug #41943.

- Remove bothersome warning messages.  This change focuses on the warnings 
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length

- Implementing --base64-format=decode-rows, to display
  SQL-alike decoded row events without their BINLOG statements.
- Adding --base64-format=decode-rows into tests when
  calling mysqlbinlog to avoid non-deterministic results
- Removing resetting of last_table_id in "RESET MASTER",
  which appeared to be dangerous.

Implementing -v command line parameter to mysqlbinlog
to decode and print row events.

mysql-test/include/mysqlbinlog_row_engine.inc
mysql-test/r/mysqlbinlog_row.result
mysql-test/r/mysqlbinlog_row_big.result
mysql-test/r/mysqlbinlog_row_innodb.result
mysql-test/r/mysqlbinlog_row_myisam.result
mysql-test/r/mysqlbinlog_row_trans.result
mysql-test/t/mysqlbinlog_row.test
mysql-test/t/mysqlbinlog_row_big.test
mysql-test/t/mysqlbinlog_row_innodb.test
mysql-test/t/mysqlbinlog_row_myisam.test
mysql-test/t/mysqlbinlog_row_trans.test
  Adding tests 

client/Makefile.am
  Adding new files to symlink
  
client/mysqlbinlog.cc
  Adding -v option

sql/log_event.cc
  Impelentations of the new methods

sql/log_event.h
  Declaration of the new methods and member

sql/mysql_priv.h
  Adding new function prototype

sql/rpl_tblmap.cc
  Adding pre-processor conditions 

sql/rpl_tblmap.h
  Adding pre-processor conditions 

sql/rpl_utility.h
  Adding pre-processor conditions 

sql/sql_base.cc
  Adding reset_table_id_sequence() function.

sql/sql_repl.cc
  Resetting table_id on "RESET MASTER"
  
.bzrignore
  Ignoring new symlinked files

When linking with some external programs, "multiple definition 
of `init_time'"

Rename init_time() to my_init_time() to avoid collision with other
libraries (particularly libmng).


client/mysqlbinlog.cc:
  Rename init_time() to my_init_time().
include/my_time.h:
  Rename init_time() to my_init_time().
sql-common/my_time.c:
  Rename init_time() to my_init_time().
sql/init.cc:
  Rename init_time() to my_init_time().
sql/tztime.cc:
  Rename init_time() to my_init_time().

Problem: mysqlbinlog does not free memory if an error happens.
Fix: binlog-processing functions do not call exit() anymore. Instead, they
print an error and return an error code. Error codes are propagated all
the way back to main, and all allocated memory is freed on the way.


client/mysqlbinlog.cc:
  - New error handling policy: functions processing binlogs don't just
    exit() anymore. Instead, they print a message and return an error
    status.
  - New policy for the global `mysql' and `glob_description_event': these
    are not passed as parameters anymore. The global pointer is used
    instead.
  - More error situations are detected and reported.
  - Better error messages: the program never terminates with exit status 1
    without explanation any more. Fixed spelling errors. Use consistent
    format of messages (a single line beginning with "ERROR: " or
    "WARNING: " and ending with "." is printed to stderr.)
  - New memory handling: memory is always freed on program termination.
  - Better comments: more functions are explained, doxygen is used, and
    more precise formulations in some existing comments.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
  Result file updated since output format of mysqlbinlog changed while the
  test was disabled.
mysql-test/suite/binlog/t/binlog_killed.test:
  Mysqlbinlog now works as described when the binlog is open. Hence, the
  --force-if-open flag must be passed
mysql-test/suite/binlog/t/binlog_killed_simulate.test:
  Mysqlbinlog now works as described when the binlog is open. Hence, the
  --force-if-open flag must be passed
mysql-test/suite/binlog/t/disabled.def:
  Now that mysqlbinlog cleans up after itself on abnormal termination, we
  can enable this test again.

slash in filenames also for Create_file_log_event.


client/mysqlbinlog.cc:
  BUG#34355: mysqlbinlog outputs backslash as path separator for 4.1 binlogs
  Problem: When the windows version of mysqlbinlog reads 4.1 binlogs
  containing LOAD DATA INFILE, it outputs backslashes as path separators in
  filenames. However, the output is typically piped to a client, and client
  expects forward slashes.
  Fix: Replace '\\' by '/' in filenames.

This patch adds code to convert the path for LOAD DATA INFILE 
events to a Unix path which is needed for running mysql client
on Windows. 


client/mysqlbinlog.cc:
  BUG#32205 : Replaying statements from mysqlbinlog fails with a syntax error, replicates fine
  
  This patch adds code to convert the path for LOAD DATA INFILE 
  events to a Unix path which is needed for running mysql client
  on Windows. 
  
  The 'd:\x\y\z' path becomes 'd:xyz' when run in the client.

BitKeeper/deleted/.del-show_binlog_events2.inc:
  Delete: mysql-test/include/show_binlog_events2.inc
client/mysqlbinlog.cc:
  char -> uchar for raw memory.
sql/item_cmpfunc.cc:
  Adding cast to remove warning when converting negative integer
  to unsigned type.
sql/log_event.cc:
  char -> uchar for raw memory.
sql/log_event.h:
  char -> uchar for raw memory.
sql/rpl_utility.cc:
  Adding cast to remove warning when converting negative integer
  to unsigned type.
sql/slave.cc:
  char -> uchar for raw memory.
sql/sql_repl.cc:
  char -> uchar for raw memory.
sql-common/client.c:
  char -> uchar for raw memory.

Problem: it is unsafe to read base64-printed events without first
reading the Format_description_log_event (FD).  Currently, mysqlbinlog
cannot print the FD.

As a side effect, another bug has also been fixed: When mysqlbinlog
--start-position=X was specified, no ROLLBACK was printed. I changed
this, so that ROLLBACK is always printed.

This patch does several things:

 - Format_description_log_event (FD) now print themselves in base64
   format.

 - mysqlbinlog is now able to print FD events.  It has three modes:
    --base64-output=auto    Print row events in base64 output, and print
                            FD event.  The FD event is printed even if
                            it is outside the range specified with
                            --start-position, because it would not be
                            safe to read row events otherwise. This is
                            the default.

    --base64-output=always  Like --base64-output=auto, but also print
                            base64 output for query events.  This is
                            like the old --base64-output flag, which
                            is also a shorthand for
                            --base64-output=always

    --base64-output=never   Never print base64 output, generate error if
                            row events occur in binlog.  This is
                            useful to suppress the FD event in binlogs
                            known not to contain row events (e.g.,
                            because BINLOG statement is unsafe,
                            requires root privileges, is not SQL, etc)

 - the BINLOG statement now handles FD events correctly, by setting
   the thread's rli's relay log's description_event_for_exec to the
   loaded event.

   In fact, executing a BINLOG statement is almost the same as reading
   an event from a relay log.  Before my patch, the code for this was
   separated (exec_relay_log_event in slave.cc executes events from
   the relay log, mysql_client_binlog_statement in sql_binlog.cc
   executes BINLOG statements).  I needed to augment
   mysql_client_binlog_statement to do parts of what
   exec_relay_log_event does.  Hence, I did a small refactoring and
   moved parts of exec_relay_log_event to a new function, which I
   named apply_event_and_update_pos.  apply_event_and_update_pos is
   called both from exec_relay_log_event and from
   mysql_client_binlog_statement.

 - When a non-FD event is executed in a BINLOG statement, without
   previously executing a FD event in a BINLOG statement, it generates
   an error, because that's unsafe.  I took a new error code for that:
   ER_NO_FORMAT_DESCRIPTION_EVENT_BEFORE_BINLOG_STATEMENTS.

   In order to get a decent error message containing the name of the
   event, I added the class method char*
   Log_event::get_type_str(Log_event_type type), which returns a
   string name for the given Log_event_type.  This is just like the
   existing char* Log_event::get_type_str(), except it is a class
   method that takes the log event type as parameter.

   I also added PRE_GA_*_ROWS_LOG_EVENT to Log_event::get_type_str(),
   so that names of old rows event are properly printed.

 - When reading an event, I added a check that the event type is known
   by the current Format_description_log_event. Without this, it may
   crash on bad input (and I was struck by this several times).

 - I patched the following test cases, which all contain BINLOG
   statements for row events which must be preceded by BINLOG
   statements for FD events:
    - rpl_bug31076

While I was here, I fixed some small things in log_event.cc:

 - replaced hard-coded 4 by EVENT_TYPE_OFFSET in 3 places

 - replaced return by DBUG_VOID_RETURN in one place

 - The name of the logfile can be '-' to indicate stdin.  Before my
   patch, the code just checked if the first character is '-'; now it
   does a full strcmp().  Probably, all arguments that begin with a -
   are already handled somewhere else as flags, but I still think it
   is better that the code reflects what it is supposed to do, with as
   little dependencies as possible on other parts of the code.  If we
   one day implement that all command line arguments after -- are
   files (as most unix tools do), then we need this.

I also fixed the following in slave.cc:

 - next_event() was declared twice, and queue_event was not static but
   should be static (not used outside the file).


client/client_priv.h:
  Declared the new option for base64 output.
client/mysqlbinlog.cc:
   - Change from using the two-state command line option
    "default/--base64-output" to the three-state
    "--base64-output=[never|auto|always]"
   - Print the FD event even if it is outside the --start-position range.
   - Stop if a row event is about to be printed without a preceding FD
     event.
   - Minor fixes:
      * changed 4 to EVENT_TYPE_OFFSET in some places
      * Added comments
      * before, "mysqlbinlog -xyz" read from stdin; now it does not
        (only "mysqlbinlog -" reads stdin).
mysql-test/r/mysqlbinlog2.result:
  Updated result file: mysqlbinlog now prints ROLLBACK always.
mysql-test/suite/binlog/t/disabled.def:
  The test must be disabled since it reveals another bug: see BUG#33247.
mysql-test/suite/rpl/r/rpl_bug31076.result:
  Updated result file
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
  Updated result file
mysql-test/suite/rpl/t/rpl_bug31076.test:
  Had to add explicit Format_description_log_event before other BINLOG
  statements
mysql-test/t/mysqlbinlog2.test:
  we must suppress base64 output in result file because it contains a
  timestamp
sql/log_event.cc:
   - Made FD events able to print themselves
   - Added check that the current FD event knows about the event type, when
     an event is about to be read. (Hint to reviewers: I had to re-indent
     a big block because of this; use diff -b)
      * To get a decent error message, I also added a class method
        const char* Log_event::get_type_str(Log_event_type)
        which converts number to event type string without having a
        Log_event object.
      * Made Log_event::get_type_str aware of PRE_GA_*_ROWS_LOG_EVENT.
   - Minor fixes:
      * Changed return to DBUG_VOID_RETURN
sql/log_event.h:
   - Declared enum to describe the three base64_output modes
   - Use the enum instead of a flag
   - Declare the new class method get_type_str (see log_event.cc)
sql/share/errmsg.txt:
  Added error msg.
sql/slave.cc:
   - Factored out part of exec_relay_log_event to the new function
     apply_event_and_update_pos, because that code is needed when executing
     BINLOG statements. (this is be functionally equivalent to the
     previous code, except: (1) skipping events is now optional, controlled
     by a parameter to the new function (2) the return value of
     exec_relay_log_event has changed; see next item).
   - Changed returned error value to always be 1. Before, it would return
     the error value from apply_log_event, which was unnecessary. This
     change is safe because the exact return value of exec_relay_log_event
     is never examined; it is only tested to be ==0 or !=0.
   - Added comments describing exec_relay_log_event and
     apply_event_and_update_pos.
   - Minor fixes:
      * Removed duplicate declaration of next_event, made queue_event
        static.
      * Added doxygen code to include this file.
sql/slave.h:
  Declared the new apply_event_and_update_pos
sql/sql_binlog.cc:
   - Made mysql_binlog_statement set the current FD event when the given
     event is an FD event. This entails using the new function
     apply_event_and_update_pos from slave.cc instead of just calling the
     ev->apply method.
   - Made mysql_binlog_statement fail if the first BINLOG statement is not
     an FD event.
mysql-test/suite/binlog/r/binlog_base64_flag.result:
  New test file needs new result file
mysql-test/suite/binlog/t/binlog_base64_flag.test:
  Added test case to verify that:
   - my patch fixes the bug
   - the new --base64-output flag works as expected
   - base64 events not preceded by an FD event give an error
   - an event of a type not known by the current FD event fails cleanly.
mysql-test/suite/binlog/std_data/binlog-bug32407.000001:
  BitKeeper file /home/sven/bk/b32407-5.1-new-rpl-mysqlbinlog_base64/mysql-test/suite/binlog/std_data/binlog-bug32407.000001

after merge fix :-/

make sure that if builder configured with a non-standard (!= 3306)
default TCP port that value actually gets used throughout. if they
didn't configure a value, assume "use a sensible default", which
will be read from /etc/services or, failing that, from the factory
default. That makes the order of preference
- command-line option
- my.cnf, where applicable
- $MYSQL_TCP_PORT environment variable
- /etc/services (unless configured --with-tcp-port)
- default port (--with-tcp-port=... or factory default)


client/mysql.cc:
  Bug #15327: configure: --with-tcp-port option being partially ignored
  
  make help on --port a little more clear
client/mysql_upgrade.c:
  Bug #15327: configure: --with-tcp-port option being partially ignored
  
  make help on --port a little more clear
client/mysqladmin.cc:
  Bug #15327: configure: --with-tcp-port option being partially ignored
  
  make help on --port a little more clear
client/mysqlbinlog.cc:
  Bug...

(This is becasue 'when' is not anymore set in constructor)


client/mysqlbinlog.cc:
  strcpy -> strmov
sql/log.cc:
  Added flag dont_set_created
sql/log_event.cc:
  Moved time handling to inline function.
  Moved setting of 'created' to ::write() function.
  Added flag dont_set_created to define if 'created' should be set or not.
  This was needed as 'when' is not set in Log_event() anymore.
  Fixed some missed thd -> thd_arg
sql/log_event.h:
  Indentation fixed
  Added inline get_time() function.
  Added dont_set_created flag to Start_log_event_v3

Moved duplicated code to inline function store_timestamp()
Save thd->time_zone_used when logging to table as CSV internally cases it to be changed
Added MYSQL_LOCK_IGNORE_FLUSH to log tables to avoid deadlock in case of flush tables.
Mark log tables with TIMESTAMP_NO_AUTO_SET to avoid automatic timestamping
Set TABLE->no_replicate on open



client/mysqlbinlog.cc:
  Fixed several memory leaks (most in case of error situations)
mysql-test/r/events_logs_tests.result:
  Made long_query_timeout smaller to ensure next query comes into log
mysql-test/r/variables.result:
  Make it safe to run test with --log
mysql-test/t/events_logs_tests.test:
  Made long_query_timeout smaller to ensure next query comes into log
mysql-test/t/variables.test:
  Make it safe to run test with --log
sql/field.cc:
  Moved duplicated code to inline function store_timestamp()
sql/field.h:
  Moved duplicated code to inline function store_timestamp()
sql/handler.cc:
  Reorder checks in likely order
  Simplify checks if we should do binary logging
  (no_replicate is set once and for all when table is opened)
sql/log.cc:
  Save thd->time_zone_used as CVS internally cases it to be changed
  Use Field_timestamp->store_timestamp instead of automatic timestamps.
  This gives us correct timestamp even if thd->set_time() is not called (in case
  of connect) and we don't have to store thd->query_start_used anymore.
sql/sql_base.cc:
  Removed not needed comment
  Moved LINT_INIT() to after declaration
  Renamed temporary variable to avoid compiler warning
  Added MYSQL_LOCK_IGNORE_FLUSH to log tables to avoid deadlock in case of flush tables.
  Mark log tables with TIMESTAMP_NO_AUTO_SET to avoid automatic timestamping
sql/table.cc:
  Set TABLE->no_replicate on open

  Add CMakeLists.txt to source distribution
CMakeLists.txt:
  Added missing '${MYSQLD_EXE_SUFFIX}' to "mysqld" targets new in 5.1
  Manual merge from 5.0 (bug#30118)
CMakeLists.txt, mysqlbinlog.cc, lib_sql.cc:
  No need to test on USING_CMAKE, it is the only Windows build


client/mysqlbinlog.cc:
  No need to test on USING_CMAKE, it is the only Windows build
libmysqld/CMakeLists.txt:
  No need to test on USING_CMAKE, it is the only Windows build
libmysqld/lib_sql.cc:
  No need to test on USING_CMAKE, it is the only Windows build
libmysqld/Makefile.am:
  Add CMakeLists.txt to source distribution
libmysqld/examples/Makefile.am:
  Add CMakeLists.txt to source distribution
sql/CMakeLists.txt:
  Added missing '${MYSQLD_EXE_SUFFIX}' to "mysqld" targets new in 5.1
client/CMakeLists.txt:
  Manual merge from 5.0 (bug#30118)
libmysql/CMakeLists.txt:
  Manual merge from 5.0 (bug#30118)
mysys/CMakeLists.txt:
  Manual merge from 5.0 (bug#30118)
zlib/CMakeLists.txt:
  Manual merge from 5.0 (bug#30118)

Fixed compiler warnings, errors and link errors
Fixed new bug on Solaris with gethrtime()
Added --debug-check option to all mysql clients to print errors and memory leaks
Added --debug-info to all clients. This now works as --debug-check but also prints memory and cpu usage


BUILD/compile-solaris-sparc-debug:
  Remove old cpu options
client/client_priv.h:
  Added OPT_DBUG_CHECK
client/mysql.cc:
  --debug-info now prints memory usage 
  Added --debug-check
client/mysql_upgrade.c:
  --debug-info now prints memory usage 
  Added --debug-check
client/mysqladmin.cc:
  --debug-info now prints memory usage 
  Added --debug-check
client/mysqlbinlog.cc:
  --debug-info now prints memory usage 
  Added --debug-check
client/mysqlcheck.c:
  --debug-info now prints memory usage 
  Added --debug-check
client/mysqldump.c:
  --debug-info now prints memory usage 
  Added --debug-check
client/mysqlimport.c:
  --debug-info now prints memory usage 
  Added --debug-check
client/mysqlshow.c:
  --debug-info now prints memory usage 
  Added --debug-check
client/mysqlslap.c:
  --debug-info now prints memory usage 
  Added --debug-check
client/mysqltest.c:
  --debug-info now prints memory usage 
  Added --debug-check
include/my_sys.h:
  Added extra option to TERMINATE to not print statistics
libmysql/libmysql.c:
  Fixed compiler warning
mysql-test/mysql-test-run.pl:
  --debug-info -> --debug-check to not print memory usage
mysys/my_getsystime.c:
  Moved fast time calculation to my_micro_time_and_time()
  Fixed bug in previous push related to HAVE_GETHRTIME
mysys/my_init.c:
  Print not freed memory in my_end() if MY_CHECK_ERROR is given
mysys/my_static.c:
  Cleanup
mysys/safemalloc.c:
  Added extra option to TERMINATE to not print statistics
sql/item_xmlfunc.cc:
  Fixed compiler warning
sql/sql_test.cc:
  Fixed TERMINATE() call
unittest/mysys/base64-t.c:
  Fixed link error
unittest/mysys/bitmap-t.c:
  Fixed link error
unittest/mysys/my_atomic-t.c:
  Fixed link error

fixing FD event issue that showed up on pb.


client/mysqlbinlog.cc:
  resetting temp_buf specially for FD event to avoid double-freeing

This patch corrects a problem encountered when reading the binlog from a remote
host. The application was crashing because the buffer variable (temp_buf) in 
log_event was not pointing to the incoming data. For a normal file read, this 
buffer is allocated by a previous call of read_log_event. However, when reading
from a remote host, the first call to read_log_event is not executed therefore
no buffer is allocated. Furthermore, there is no need to allocate a new buffer 
because the incoming stream is what needs to be read. 

This patch adds the call to initialize the temp_buf variable if reading from a 
remote host. It also adds a check at destroy time to ensure the temp_buf is not
freed if reading from a remote host.


client/mysqlbinlog.cc:
  BUG#17654 : --read-from-remote-server causes core
  
  This patch corrects a problem when reading from a remote host. The temp_buf 
  variable of the log_event class is undefined. This patch assigns the temp_buf
  variable to the address of the incoming stream. This allows the print functions
  to print the binlog events correctly.
mysql-test/r/rpl_row_mysqlbinlog.result:
  BUG#17654 : --read-from-remote-server causes core
  
  This patch adds the results for the test that were disabled when the bug report
  was investigated. The patch also adds an additional test was added to ensure 
  the output of reading from a remote host is the same as reading from a local file.
mysql-test/t/rpl_row_mysqlbinlog.test:
  BUG#17654 : --read-from-remote-server causes core
  
  This patch enables the portions of the test that were disabled when the bug report
  was investigated. The patch also adds an additional test was added to ensure 
  the output of reading from a remote host is the same as reading from a local file.

The following type conversions was done:

- Changed byte to uchar
- Changed gptr to uchar*
- Change my_string to char *
- Change my_size_t to size_t
- Change size_s to size_t

Removed declaration of byte, gptr, my_string, my_size_t and size_s. 

Following function parameter changes was done:
- All string functions in mysys/strings was changed to use size_t
  instead of uint for string lengths.
- All read()/write() functions changed to use size_t (including vio).
- All protocoll functions changed to use size_t instead of uint
- Functions that used a pointer to a string length was changed to use size_t*
- Changed malloc(), free() and related functions from using gptr to use void *
  as this requires fewer casts in the code and is more in line with how the
  standard functions work.
- Added extra length argument to dirname_part() to return the length of the
  created string.
- Changed (at least) following functions to t...

Taking code from before BUG#22583 and incorporating as events to be able
to read old events. Also incorporating old pack and unpack functions
into patch.


client/Makefile.am:
  Adding files log_event_old.{h,cc} and rpl_record_old.{h,cc}
client/mysqlbinlog.cc:
  Adding log_event_old.cc.
libmysqld/Makefile.am:
  Adding files log_event_old.{h,cc} and rpl_record_old.{h,cc}
sql/CMakeLists.txt:
  Adding files log_event_old.{h,cc} and rpl_record_old.{h,cc}
sql/Makefile.am:
  Adding files log_event_old.{h,cc} and rpl_record_old.{h,cc}
sql/log_event.cc:
  Adding code to read pre-GA rows events.
sql/log_event.h:
  Refactoring to support inheritance and including "old" events definitions.
sql/log_event_old.cc:
  New BitKeeper file ``sql/log_event_old.cc''
sql/log_event_old.h:
  New BitKeeper file ``sql/log_event_old.h''
sql/rpl_record_old.cc:
  New BitKeeper file ``sql/rpl_record_old.cc''
sql/rpl_record_old.h:
  New BitKeeper file ``sql/rpl_record_old.h''

a better fix, that works with 4.1 servers which don't send a fake
Format_description_log_event.

Server starts any binlog dump from Format_description_log_event,
this shifted all offset calculations in mysqlbinlog and made it
to stop the dump earlier than --stop-position. Now mysqlbinlog
takes Format_description_log_event into account


mysql-test/r/mysqlbinlog2.result:
  Bug#27171 mysqlbinlog produces different output depends from option -R
mysql-test/t/mysqlbinlog2.test:
  Bug#27171 mysqlbinlog produces different output depends from option -R

mysys/default.c:
  Fixed bug.
ndb/src/mgmclient/CommandInterpreter.cpp:
  Added parenthesis around the expression.
sql/mysqld.cc:
  Fixed compiler warnings.
  
  Added a missing component in options struct (bug).
sql-common/my_time.c:
  Removed garbage.
sql/sql_table.cc:
  A possible use of a variable uninitialized.
support-files/compiler_warnings.supp:
  BitKeeper file /home/my/bk/mysql-4.1-main/support-files/compiler_warnings.supp

when it was printing a Query event, it produced invalid SQL (missing
the BINLOG keyword, so the SQL started with the base64 string, which
is incorrect).
Note: no testcase; I have a .test which shows that the bugfix works,
but it triggers BUG#26361 and so gives Valgrind warnings. I'm sending
this test to the fixer of BUG#26361 for her/him to push when she/he
fixes BUG#26361.


client/mysqlbinlog.cc:
  writing the header (a line started with "#", i.e. a comment) and the
  body (the real operation) of an event to the same IO_CACHE
  (result_cache) confused the logic of Log_event::print_base64()
  (which is that if the cache is not empty then the BINLOG keyword
  should not be printed); it caused the BINLOG keyword to miss hence
  a syntactically wrong output of "mysqlbinlog --base64-output"
  for Query events.
  So we just use the two IO_CACHE already available in "print_event_info".
sql/log_event.cc:
  using the new small inline function.
  Note that the replication code should one day be fixed to trap all
  errors (like disk write errors).
sql/log_event.h:
  small inline function to group two operations: copying an IO_CACHE
  to a FILE, and reinitializing this IO_CACHE for being filled again.
sql/records.cc:
  fix after merge

- Mis-matched SAFEMALLOC defines caused misleading error message.


client/mysqlbinlog.cc:
  Bug#23736 Pointer free error in mysqlbinlog
  - Re-worked the Load_log_processor so that it frees it's resources before 
  my_end is called.  This is necessary because SAFEMALLOC's _my_free calls
  pthread_mutex_lock() using THR_LOCK_malloc which is cleaned up in my_end().
include/my_sys.h:
  Bug#23736 Pointer free error in mysqlbinlog
  - Define DYNAMIC_ARRAY beofore MY_TMPDIR
  - Add DYNAMIC_ARRAY to MY_TMP_DIR
mysys/array.c:
  Bug#23736 Pointer free error in mysqlbinlog
  - SAFEMALLOC should not be unconditionally undef'd.
mysys/mf_tempdir.c:
  Bug#23736 Pointer free error in mysqlbinlog
  - Use struct's DYNAMIC_ARRAY.
  - Use DYNAMIC_ARRAY:delete_dynamic function instead of my_free

around the original functions. These will ensure that error
message is always in unique form, reduce code and print the
right alternatives automatically in an error case.


client/mysql.cc:
  Changed find_type to find_type_or_exit
client/mysqladmin.cc:
  Changed find_type to find_type_or_exit
client/mysqlbinlog.cc:
  Changed find_type to find_type_or_exit
client/mysqlcheck.c:
  Changed find_type to find_type_or_exit
client/mysqldump.c:
  Changed find_type to find_type_or_exit
client/mysqlimport.c:
  Changed find_type to find_type_or_exit
client/mysqlshow.c:
  Changed find_type to find_type_or_exit
client/mysqlslap.c:
  Changed find_type to find_type_or_exit
include/typelib.h:
  Added find_type_or_exit
mysql-test/r/mysql_protocols.result:
  Fixed result.
mysys/typelib.c:
  Added find_type_or_exit
sql/mysqld.cc:
  Added use of find_type_or_exit and find_bit_type_or_exit
  Fixed a missing break; from an option handling. (Bug
  in --tc-heuristic-recover)