• unknown's avatar
    WL#3228 (NDB) : RBR using different table defs on slave/master · d4671354
    unknown authored
    This patch adds the ability to store extra field metadata in the table
    map event. This data can include pack_length() or field_lenght() for
    fields such as CHAR or VARCHAR enabling developers to add code that
    can check for compatibilty between master and slave columns. More 
    importantly, the extra field metadata can be used to store data from the
    master correctly should a VARCHAR field on the master be <= 255 bytes 
    while the same field on the slave is > 255 bytes. 
    
    The patch also includes the needed changes to unpack to ensure that data
    which is smaller on the master can be unpacked correctly on the slave.
    
    WL#3915 : (NDB) master's cols > slave
    
    Slave starts accepting and handling rows of master's tables which have more columns.
    The most important part of implementation is how to caclulate the amount of bytes to
    skip for unknown by slave column.
    
    
    mysql-test/suite/binlog/t/binlog_row_mix_innodb_myisam.test:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch changes the test to coincide with changes to binlog
      size of table map event.
    mysql-test/suite/rpl/r/rpl_row_basic_11bugs.result:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch contains a new result file as a result of the change to the
      size of the tablemap log event.
    mysql-test/suite/rpl/r/rpl_row_create_table.result:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch contains a new result file as a result of the change to the
      size of the tablemap log event.
    mysql-test/suite/rpl/r/rpl_row_flsh_tbls.result:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch contains a new result file as a result of the change to the
      size of the tablemap log event.
    mysql-test/suite/rpl/r/rpl_row_inexist_tbl.result:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch contains a new result file as a result of the change to the
      size of the tablemap log event.
    mysql-test/suite/rpl/r/rpl_row_log.result:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch contains a new result file as a result of the change to the
      size of the tablemap log event.
    mysql-test/suite/rpl/r/rpl_row_log_innodb.result:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch contains a new result file as a result of the change to the
      size of the tablemap log event.
    mysql-test/suite/rpl/r/rpl_row_max_relay_size.result:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch contains a new result file as a result of the change to the
      size of the tablemap log event.
    mysql-test/suite/rpl/r/rpl_row_tabledefs_2myisam.result:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch contains a new result file as a result of the change to the
      size of the tablemap log event.
    mysql-test/suite/rpl/r/rpl_row_tabledefs_3innodb.result:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch contains a new result file as a result of the change to the
      size of the tablemap log event.
    mysql-test/suite/rpl/r/rpl_row_until.result:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch contains a new result file as a result of the change to the
      size of the tablemap log event.
    mysql-test/suite/rpl/r/rpl_skip_error.result:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch contains a new result file as a result of the change to the
      size of the tablemap log event.
    mysql-test/suite/rpl/t/disabled.def:
      WL#3915  master's cols > slave
      
      Disabled the rpl_stm_extraColmaster_ndb test because statement-based
      replication is not supported in NDB at this time. It can be enabled
      when statement-based replication for NDB is released.
    mysql-test/suite/rpl/t/rpl_row_create_table.test:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch corrects binlog positions a result of the change to the
      size of the tablemap log event.
    mysql-test/suite/rpl/t/rpl_row_flsh_tbls.test:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch corrects binlog positions a result of the change to the
      size of the tablemap log event.
    mysql-test/suite/rpl_ndb/r/rpl_ndb_log.result:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch contains a new result file as a result of the change to the
      size of the tablemap log event.
    mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb.result:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch contains a new result file as a result of the change to the
      size of the tablemap log event.
    sql/field.cc:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch includes updates to the unpack() methods for the variable
      length fields. A new parameter was added (from_length) that is the
      value stored in the field_metadata of the table map from the table_def
      class. If the value is non-zero and less than what the field on the 
      slave is then use the from_length else use the original value from the
      field on the slave.
    sql/field.h:
      L#3228 : RBR using different table defs on slave/master
      
      This patch includes updates to the unpack() methods for the variable
      length fields. A new parameter was added (from_length) that is the
      value stored in the field_metadata of the table map from the table_def
      class.
    sql/log_event.cc:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch adds methods to calculate the field metadata size, prepare
      the field metadata for writing to the binlog, and additions to the
      Table_map_log_event::write_body method to include the field metadata 
      in the table map that is written to the binlog.
      
      WL#3915  master's cols > slave
      
      copying extra (slave's) fields returns early if master's table version is wider;
      removing assert in the way of master > slave cols.
    sql/log_event.h:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch adds method declarations and variables needed to support
      storing field metadata in the table map that is written to the binlog.
    sql/rpl_record.cc:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch modifies the unpack_row() method to unpack fields passing in
      the value from the table_def class. This value is the extra field
      metadata stored there from the master.
      
      WL#3915  master's cols > slave
      
      adding a snippet that shift exectution curson donw the row skipping unknown by slave
      fields' data.
    sql/rpl_rli.h:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch adds a helper function to retrieve the table_def for a given
      table in the RPL_TABLE_LIST structure.
    sql/rpl_utility.cc:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch adds a helper method that retrieves the correct size 
      parameter for the field. This method is used to compare the size as
      sent by the master with that on the slave for all types of fields that
      can vary in size and storage requirements. 
      
      WL#3915  master's cols > slave
      
      Remove warning message for master's cols > slave.
    sql/rpl_utility.h:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch changes the table_def class constructor to pass in the raw
      data read from the table map and extract it into an array of dimension
      size (number of fields). It also adds a method to return the field 
      metadata for any field. The method returns the data stored in the table
      map or 0 if no data was stored for that field. Lastly, a method to return
      the results of field->maybe_null() is included so that the slave can
      determine if a field that is not on the slave is null.
    mysql-test/suite/rpl/t/rpl_colSize.test:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch contains a new test designed to test the feature of having
      columns on the master that are smaller than what is on the slave.
    mysql-test/suite/rpl/t/rpl_extraColmaster_innodb-master.opt:
      WL#3915  master's cols > slave
      
      option for innodb
    mysql-test/suite/rpl/t/rpl_extraColmaster_innodb-slave.opt:
      WL#3915  master's cols > slave
      
      option for innodb
    mysql-test/suite/rpl/t/rpl_extraColmaster_innodb.test:
      WL#3915  master's cols > slave
      
      Test of innodb. Test runs in both statement- and row-based replication.
    mysql-test/suite/rpl/t/rpl_extraColmaster_myisam.test:
      WL#3915  master's cols > slave
      
      Test of myisam. Test runs in both statement- and row-based replication.
    mysql-test/suite/rpl/r/rpl_colSize.result:
      WL#3228 : RBR using different table defs on slave/master
      
      This patch contains a result file for the new test designed to test the 
      feature of having columns on the master that are smaller than what is 
      on the slave.
    mysql-test/suite/rpl/t/rpl_row_extraColmaster_ndb.test:
      WL#3915  master's cols > slave
      
      Test of ndb. Test runs in row-based replication.
    mysql-test/suite/rpl/t/rpl_stm_extraColmaster_ndb.test:
      WL#3915  master's cols > slave
      
      Test of ndb. Test runs in statement-based replication.
    mysql-test/suite/rpl/r/rpl_extraColmaster_innodb.result:
      WL#3915  master's cols > slave
      
      new results
    mysql-test/suite/rpl/r/rpl_extraColmaster_myisam.result:
      WL#3915  master's cols > slave
      
      new results
    mysql-test/extra/rpl_tests/rpl_extraMaster_Col.test:
      WL#3915  master's cols > slave
      
      basic tests checking altering and skipping extra fields by slave.
      The fields can be of any possible types.
    mysql-test/suite/rpl/r/rpl_row_extraColmaster_ndb.result:
      WL#3915  master's cols > slave
      
      new results
    d4671354
field.cc 268 KB