Skip to content

M
MariaDB
Commit 4b2092eb authored by Olivier Bertrand's avatar Olivier Bertrand
Browse files
Options

This is a new version of the CONNECT storage engine. It was developed in 

a sub-branch of this one and merged by pushing all the changes from it.
This version adds the following to CONNECT:

- MRR support (similar to the MyISAM one)
- Block, Remote and dynamic indexing
- Partitioning support (using the PARTITION engine)

Here is a list of the commited changes made in the sub-branch:
========================================================================

------------------------------------------------------------
revno: 4009
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Thu 2014-07-17 18:13:51 +0200
message:
  This commit brings many changes, in particular two important ones:
  1) Support of partitioning by connect. A table can be partitioned
     by files, this is an enhanced MULTIPLE table. It can be also
     partitioned by sub-tables like TBL and this enables table sharding.
  2) Handling a CONNECT bug that causes in some cases extraneous rows
     to remain in the table after an UPDATE or DELETE when the command
     uses indexing (for not fixed file tables). Until a real fix is
     done, CONNECT tries to ignore indexing and if it cannot do it
     abort the command with an error message.

  - Add tests on partitioning
  added:
    storage/connect/mysql-test/connect/r/part_file.result
    storage/connect/mysql-test/connect/r/part_table.result
    storage/connect/mysql-test/connect/t/part_file.test
    storage/connect/mysql-test/connect/t/part_table.test

  - Temporary fix
  modified:
    sql/sql_partition.cc

  - Add partition support
  modified:
    storage/connect/ha_connect.cc
    storage/connect/ha_connect.h
    storage/connect/reldef.cpp
    storage/connect/reldef.h
    storage/connect/tabdos.cpp

  - Add functions ha_connect::IsUnique and ha_connect::CheckColumnList
  modified:
    storage/connect/ha_connect.cc
    storage/connect/ha_connect.h

  - Prevent updating a partition table column that is part of
    the partition function (outward tables only)
  modified:
    storage/connect/ha_connect.cc

  - Support INSERT/UPDATE/DELETE for PROXY tables
  modified:
    storage/connect/tabutil.cpp

  - Handle the bug on updating rows via indexing. Waiting for a real fix,
    Don't use indexing when possible else raise an error and abort.
  modified:
    storage/connect/ha_connect.cc

  - dbuserp->UseTemp set to TMP_AUTO
  modified:
    storage/connect/connect.cc

  - Add members nox, abort and only
  modified:
    storage/connect/ha_connect.cc
    storage/connect/ha_connect.h

  - Add arguments nox and abort to CntCloseTable
  modified:
    storage/connect/connect.cc
    storage/connect/connect.h
    storage/connect/filamap.cpp
    storage/connect/filamap.h
    storage/connect/filamdbf.cpp
    storage/connect/filamdbf.h
    storage/connect/filamfix.cpp
    storage/connect/filamfix.h
    storage/connect/filamtxt.cpp
    storage/connect/filamtxt.h
    storage/connect/filamvct.cpp
    storage/connect/filamvct.h
    storage/connect/filamzip.cpp
    storage/connect/filamzip.h
    storage/connect/ha_connect.cc

  - Add arguments abort to CloseTableFile and RenameTempFile
  modified:
    storage/connect/filamap.cpp
    storage/connect/filamap.h
    storage/connect/filamdbf.cpp
    storage/connect/filamdbf.h
    storage/connect/filamfix.cpp
    storage/connect/filamfix.h
    storage/connect/filamtxt.cpp
    storage/connect/filamtxt.h
    storage/connect/filamvct.cpp
    storage/connect/filamvct.h
    storage/connect/filamzip.cpp
    storage/connect/filamzip.h
    storage/connect/tabdos.cpp
    storage/connect/tabdos.h
    storage/connect/tabvct.cpp
    storage/connect/xtable.h

  - Fix info->records when file does not exists
  modified:
    storage/connect/connect.cc

  - Close XML table when opened for info
  modified:
    storage/connect/connect.cc

  - Add function VCTFAM::GetFileLength
  modified:
    storage/connect/filamvct.cpp
    storage/connect/filamvct.h

  - Column option DISTRIB -> ENUM
  modified:
    storage/connect/ha_connect.cc

  - Options connect, query_string and partname allways available
  modified:
    storage/connect/ha_connect.cc

  - Add function MYSQLC::GetTableSize
  modified:
    storage/connect/myconn.cpp
    storage/connect/myconn.h

  - Add new special columns (PARTNAME, FNAME, FPATH, FTYPE and FDISK)
  modified:
    storage/connect/colblk.cpp
    storage/connect/colblk.h
    storage/connect/plgdbsem.h
    storage/connect/table.cpp

  - Add function ExtractFromPath
  modified:
    storage/connect/colblk.cpp
    storage/connect/plgdbsem.h
    storage/connect/plgdbutl.cpp

  - Enhance Cardinality for some table types
  modified:
    storage/connect/tabdos.cpp
    storage/connect/tabmysql.cpp
    storage/connect/tabmysql.h
    storage/connect/tabodbc.cpp
    storage/connect/tabodbc.h
    storage/connect/tabsys.cpp
    storage/connect/tabsys.h
    storage/connect/xindex.cpp
    storage/connect/xindex.h
    storage/connect/xtable.h

  - Add test on special column
  modified:
    storage/connect/tabfmt.cpp

  - Add new files (added for block indexing)
  modified:
    storage/connect/CMakeLists.txt
------------------------------------------------------------
revno: 4007 [merge]
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Sat 2014-05-31 12:31:26 +0200
message:
  - Begin adding support of partition tables
  modified:
    storage/connect/ha_connect.cc
    storage/connect/ha_connect.h
    storage/connect/reldef.cpp

  - Add INSERT/UPDATE support to PROXY tables
  modified:
    storage/connect/tabutil.cpp
    storage/connect/tabutil.h

  - Take care of SPECIAL columns
  modified:
    storage/connect/filamdbf.cpp
    storage/connect/reldef.h
    storage/connect/tabfmt.cpp

  -Typo and misc
  modified:
    storage/connect/odbconn.cpp
    storage/connect/tabfix.cpp
    storage/connect/xindex.cpp
------------------------------------------------------------
revno: 4006
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Sat 2014-05-10 12:21:08 +0200
message:
  - FIX some MAP and XMAP errors (such as mapped indexes not closed)
    Do not put version in XML files header
    Remove HTON_NO_PARTITION for testing
    Fix a wrong return (instead of DBUG_RETURN) in index_init
    Plus a few typos
  modified:
    storage/connect/connect.cc
    storage/connect/filter.cpp
    storage/connect/ha_connect.cc
    storage/connect/maputil.cpp
    storage/connect/mysql-test/connect/r/alter_xml.result
    storage/connect/mysql-test/connect/r/xml.result
    storage/connect/table.cpp
    storage/connect/tabxml.cpp
    storage/connect/xindex.cpp
    storage/connect/xindex.h
    storage/connect/xtable.h
------------------------------------------------------------
revno: 4005
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Fri 2014-05-02 15:55:45 +0200
message:
  - Adding fetched columns to Dynamic index key (unique only)
    Fix two bugs concerning added KXYCOL's:
    1 - Not set during reading
    2 - Val_K not set in FastFind
  modified:
    storage/connect/connect.cc
    storage/connect/filamtxt.h
    storage/connect/tabdos.cpp
    storage/connect/tabfix.cpp
    storage/connect/table.cpp
    storage/connect/valblk.h
    storage/connect/xindex.cpp
    storage/connect/xindex.h
    storage/connect/xtable.h
------------------------------------------------------------
revno: 4003
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Wed 2014-04-30 10:48:29 +0200
message:
  - Implementation of adding selected columns to dynamic indexes.
  modified:
    storage/connect/connect.cc
    storage/connect/ha_connect.cc
    storage/connect/ha_connect.h
    storage/connect/tabdos.cpp
    storage/connect/tabdos.h
    storage/connect/tabvct.cpp
    storage/connect/tabvct.h
    storage/connect/xindex.cpp
    storage/connect/xindex.h
------------------------------------------------------------
revno: 4001
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Sat 2014-04-26 00:17:26 +0200
message:
  - Implement dynamic indexing
  modified:
    storage/connect/connect.cc
    storage/connect/filter.cpp
    storage/connect/filter.h
    storage/connect/ha_connect.cc
    storage/connect/ha_connect.h
    storage/connect/tabdos.cpp
    storage/connect/tabdos.h
    storage/connect/table.cpp
    storage/connect/xindex.cpp
    storage/connect/xindex.h
    storage/connect/xtable.h
------------------------------------------------------------
revno: 3995
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Sun 2014-03-23 18:49:19 +0100
message:
  - Work in progress
  modified:
    storage/connect/filter.h
    storage/connect/ha_connect.cc
    storage/connect/ha_connect.h
    storage/connect/mysql-test/connect/r/alter.result
    storage/connect/mysql-test/connect/r/xml.result
------------------------------------------------------------
revno: 3991
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Mon 2014-03-10 18:59:36 +0100
message:
  - Adding files needed for block indexing
  added:
    storage/connect/array.cpp
    storage/connect/array.h
    storage/connect/blkfil.cpp
    storage/connect/blkfil.h
    storage/connect/filter.cpp
    storage/connect/filter.h
========================================================================
This commit of the main branch adds:

- A change needed to have the engine function check_if_supported_inplace_alter
  called for partition tables (was done manually in the sub-branch) by adding
  the preparser define: PARTITION_SUPPORTS_INPLACE_ALTER
modified:
  sql/CMakeLists.txt

- A fix concerning the FileExists function. It was needed to force the function
  table_flags to return the same flags for all partitions. This is tested by
  the partition engine and raises an error if flags are not equal.
  The way file name, table name and connection string are retrieved has been
  modified to cope with it.
modified:
  storage/connect/ha_connect.cc
  storage/connect/ha_connect.h
  storage/connect/reldef.cpp

- A few typos, such as the version string.
modified:
  storage/connect/ha_connect.cc

- Updating some test result files because some warnings are no more raised.
modified:
  storage/connect/mysql-test/connect/r/occur.result
  storage/connect/mysql-test/connect/r/part_file.result
  storage/connect/mysql-test/connect/r/pivot.result
parent 6c8fbe61
Hide whitespace changes
Inline Side-by-side
Showing with 41 additions and 30 deletions
sql/CMakeLists.txt
View file @ 4b2092eb
......@@ -30,7 +30,7 @@ ${CMAKE_CURRENT_BINARY_DIR}/lex_hash.h
SET_SOURCE_FILES_PROPERTIES(${GEN_SOURCES} PROPERTIES GENERATED 1)
ADD_DEFINITIONS(-DMYSQL_SERVER -DHAVE_EVENT_SCHEDULER -DHAVE_POOL_OF_THREADS)
ADD_DEFINITIONS(-DMYSQL_SERVER -DHAVE_EVENT_SCHEDULER -DHAVE_POOL_OF_THREADS -DPARTITION_SUPPORTS_INPLACE_ALTER)
IF(SSL_DEFINES)
ADD_DEFINITIONS(${SSL_DEFINES})
ENDIF()
......
storage/connect/ha_connect.cc
View file @ 4b2092eb
......@@ -170,7 +170,7 @@
#define SZWMIN 4194304 // Minimum work area size 4M
extern "C" {
char version[]= "Version 1.03.0002 May 03, 2014";
char version[]= "Version 1.03.0002 July 17, 2014";
char compver[]= "Version 1.03.0002 " __DATE__ " " __TIME__;
#if defined(WIN32)
......@@ -745,7 +745,7 @@ ulonglong ha_connect::table_flags() const
flags|= (HA_HAS_RECORDS | HA_STATS_RECORDS_IS_EXACT);
// No data change on ALTER for outward tables
if (!IsFileType(type) || hp->FileExists(pos->filename))
if (!IsFileType(type) || hp->FileExists(pos->filename, true))
flags|= HA_NO_COPY_ON_ALTER;
} // endif pos
......@@ -814,6 +814,22 @@ PTOS ha_connect::GetTableOptionStruct(TABLE_SHARE *s)
return (tsp) ? tsp->option_struct : NULL;
} // end of GetTableOptionStruct
/****************************************************************************/
/* Return the string eventually formatted with partition name. */
/****************************************************************************/
char *ha_connect::GetRealString(const char *s)
{
char *sv;
if (IsPartitioned() && s) {
sv= (char*)PlugSubAlloc(xp->g, NULL, strlen(s) + strlen(partname));
sprintf(sv, s, partname);
} else
sv= (char*)s;
return sv;
} // end of GetRealString
/****************************************************************************/
/* Return the value of a string option or NULL if not specified. */
/****************************************************************************/
......@@ -826,7 +842,7 @@ char *ha_connect::GetStringOption(char *opname, char *sdef)
LEX_STRING cnc= (tshp) ? tshp->connect_string : table->s->connect_string;
if (cnc.length)
opval= cnc.str;
opval= GetRealString(cnc.str);
} else if (!stricmp(opname, "Query_String"))
opval= thd_query_string(table->in_use)->str;
......@@ -837,11 +853,11 @@ char *ha_connect::GetStringOption(char *opname, char *sdef)
else if (!stricmp(opname, "Type"))
opval= (char*)options->type;
else if (!stricmp(opname, "Filename"))
opval= (char*)options->filename;
opval= GetRealString(options->filename);
else if (!stricmp(opname, "Optname"))
opval= (char*)options->optname;
else if (!stricmp(opname, "Tabname"))
opval= (char*)options->tabname;
opval= GetRealString(options->tabname);
else if (!stricmp(opname, "Tablist"))
opval= (char*)options->tablist;
else if (!stricmp(opname, "Database") ||
......@@ -5665,7 +5681,7 @@ int ha_connect::create(const char *name, TABLE *table_arg,
#endif // WITH_PARTITION_STORAGE_ENGINE
if (g->Alchecked == 0 &&
(!IsFileType(type) || FileExists(options->filename))) {
(!IsFileType(type) || FileExists(options->filename, false))) {
if (part_info) {
sprintf(g->Message, "Data repartition in %s is unchecked", partname);
push_warning(thd, Sql_condition::WARN_LEVEL_WARN, 0, g->Message);
......@@ -5699,7 +5715,8 @@ int ha_connect::create(const char *name, TABLE *table_arg,
#if defined(WITH_PARTITION_STORAGE_ENGINE)
if (part_info)
strcpy(partname, strrchr(name, (inward ? slash : '#')) + 1);
strcpy(partname,
decode(g, strrchr(name, (inward ? slash : '#')) + 1));
#endif // WITH_PARTITION_STORAGE_ENGINE
if ((rc= optimize(table->in_use, NULL))) {
......@@ -5736,10 +5753,12 @@ int ha_connect::create(const char *name, TABLE *table_arg,
- file does not exist or is void
- user has file privilege
*/
bool ha_connect::FileExists(const char *fn)
bool ha_connect::FileExists(const char *fn, bool bf)
{
if (!fn || !*fn)
return false;
else if (IsPartitioned() && bf)
return true;
if (table) {
char *s, tfn[_MAX_PATH], filename[_MAX_PATH], path[128];
......@@ -5904,7 +5923,7 @@ bool ha_connect::NoFieldOptionChange(TABLE *tab)
*/
enum_alter_inplace_result
ha_connect::check_if_supported_inplace_alter(TABLE *altered_table,
Alter_inplace_info *ha_alter_info)
Alter_inplace_info *ha_alter_info)
{
DBUG_ENTER("check_if_supported_alter");
......@@ -6007,7 +6026,7 @@ ha_connect::check_if_supported_inplace_alter(TABLE *altered_table,
char *fn= GetStringOption("filename");
tshp= NULL;
if (FileExists(fn)) {
if (FileExists(fn, false)) {
strcpy(g->Message, "Operation denied. Table data would be lost.");
my_message(ER_UNKNOWN_ERROR, g->Message, MYF(0));
DBUG_RETURN(HA_ALTER_ERROR);
......@@ -6067,8 +6086,10 @@ ha_connect::check_if_supported_inplace_alter(TABLE *altered_table,
"Alter operations not supported together by CONNECT", MYF(0));
DBUG_RETURN(HA_ALTER_ERROR);
} else if (outward) {
push_warning(thd, Sql_condition::WARN_LEVEL_WARN, 0,
"This is an outward table, table data were not modified.");
if (IsFileType(type))
push_warning(thd, Sql_condition::WARN_LEVEL_WARN, 0,
"This is an outward table, table data were not modified.");
DBUG_RETURN(HA_ALTER_INPLACE_EXCLUSIVE_LOCK);
} else
DBUG_RETURN(HA_ALTER_INPLACE_NOT_SUPPORTED);
......
storage/connect/ha_connect.h
View file @ 4b2092eb
......@@ -182,6 +182,7 @@ class ha_connect: public handler
static bool connect_init(void);
static bool connect_end(void);
TABTYPE GetRealType(PTOS pos= NULL);
char *GetRealString(const char *s);
char *GetStringOption(char *opname, char *sdef= NULL);
PTOS GetTableOptionStruct(TABLE_SHARE *s= NULL);
bool GetBooleanOption(char *opname, bool bdef);
......@@ -193,7 +194,7 @@ class ha_connect: public handler
bool SetIntegerOption(char *opname, int n);
bool SameInt(TABLE *tab, char *opn);
bool SameBool(TABLE *tab, char *opn);
bool FileExists(const char *fn);
bool FileExists(const char *fn, bool bf);
bool NoFieldOptionChange(TABLE *tab);
PFOS GetFieldOptionStruct(Field *fp);
void *GetColumnOption(PGLOBAL g, void *field, PCOLINFO pcf);
......
storage/connect/mysql-test/connect/r/occur.result
View file @ 4b2092eb
......@@ -193,8 +193,6 @@ Kevin 8
Lisbeth 2
Mary 2
ALTER TABLE xpet MODIFY number INT NOT NULL;
Warnings:
Warning 1105 This is an outward table, table data were not modified.
SELECT * FROM xpet;
name race number
John dog 2
......
storage/connect/mysql-test/connect/r/part_file.result
View file @ 4b2092eb
......@@ -88,8 +88,6 @@ FNAME FTYPE
# Testing partitioning on a void outward table
#
ALTER TABLE dr1 file_name='part*.*';
Warnings:
Warning 1105 This is an outward table, table data were not modified.
CREATE TABLE t1 (
rwid INT(6) DEFAULT 0 SPECIAL=ROWID,
rnum INT(6) DEFAULT 0 SPECIAL=ROWNUM,
......@@ -104,8 +102,6 @@ PARTITION by range columns(id) (
PARTITION `1` VALUES LESS THAN(10),
PARTITION `2` VALUES LESS THAN(50),
PARTITION `3` VALUES LESS THAN(MAXVALUE));
Warnings:
Warning 1105 Open(rb) error 2 on DATADIR/test/part%s.txt: No such file or directory
SHOW INDEX FROM t1;
Table Non_unique Key_name Seq_in_index Column_name Collation Cardinality Sub_part Packed Null Index_type Comment Index_comment
t1 0 PRIMARY 1 id A NULL NULL NULL XINDEX
......
storage/connect/mysql-test/connect/r/pivot.result
View file @ 4b2092eb
......@@ -59,8 +59,6 @@ Joe 5 14.00 0.00 12.00
# Restricting the columns in a Pivot Table
#
ALTER TABLE pivex DROP COLUMN week;
Warnings:
Warning 1105 This is an outward table, table data were not modified.
SELECT * FROM pivex;
Who Beer Car Food
Beth 51.00 0.00 29.00
......
storage/connect/reldef.cpp
View file @ 4b2092eb
......@@ -143,16 +143,13 @@ char *RELDEF::GetStringCatInfo(PGLOBAL g, PSZ what, PSZ sdef)
char *name, *sval= NULL, *s= Hc->GetStringOption(what, sdef);
if (s) {
if (Hc->IsPartitioned() &&
(!stricmp(what, "filename") || !stricmp(what, "tabname")
|| !stricmp(what, "connect"))) {
name= Hc->GetPartName();
sval= (char*)PlugSubAlloc(g, NULL, strlen(s) + strlen(name));
sprintf(sval, s, name);
} else {
if (!Hc->IsPartitioned() ||
(stricmp(what, "filename") && stricmp(what, "tabname")
&& stricmp(what, "connect"))) {
sval= (char*)PlugSubAlloc(g, NULL, strlen(s) + 1);
strcpy(sval, s);
} // endif partitioned
} else
sval= s;
} else if (!stricmp(what, "filename")) {
// Return default file name
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7