manual.texi full-text manual added

Docs/manual.texi

@@ -11670,6 +11670,7 @@ to restart @code{mysqld} with @code{--skip-grant-tables} to be able to run
 * GRANT::                       @code{GRANT} and @code{REVOKE} syntax
 * CREATE INDEX::                @code{CREATE INDEX} syntax
 * DROP INDEX::                  @code{DROP INDEX} syntax
+* CREATE COLLECTION::           @code{CREATE COLLECTION} syntax
 * Comments::                    Comment syntax
 * CREATE FUNCTION::             @code{CREATE FUNCTION} syntax
 * Reserved words::              Is @strong{MySQL} picky about reserved words?
@@ -13435,6 +13436,13 @@ mysql> CREATE TABLE test (
 For @code{BLOB} and @code{TEXT} columns, you must index a prefix of the
 column, you cannot index the entire thing.
 
+In @strong{MySQL} 3.23.23 or later, you can also create special indexes
+called @strong{collections}. They are used for full-text search. Only
+@code{MyISAM} table type supports collections. Collection can be created
+only from @code{VARCHAR}, @code{BLOB}, and @code{TEXT} columns.
+Indexing always happens over the entire column, partial indexing is not
+supported. See @ref{MySQL full-text search} for details of operation.
+
 @node Multiple-column indexes, Other-vendor column types, Indexes, Column types
 @subsection Multiple-column indexes
 
@@ -14150,6 +14158,17 @@ mysql> select STRCMP('text2', 'text');
 mysql> select STRCMP('text', 'text');
         -> 0
 @end example
+
+@findex MATCH ... AGAINST()
+@item MATCH (col1,col2,...) AGAINST (expr)
+@code{MATCH ... AGAINST()} is used for full-text search and returns
+relevance - similarity measure between the text in columns
+@code{(col1,col2,...)} and the query @code{expr}. Relevance is a
+positive floating point number. Zero relevance means no similarity.
+For @code{MATCH ... AGAINST()} to work, a @code{COLLECTION}
+must be created first. @xref{CREATE TABLE, , @code{CREATE TABLE}}.
+@code{MATCH ... AGAINST()} is available in @code{MySQL} 3.23.23 or later.
+For details and usage examples see @xref{MySQL full-text search}.
 @end table
 
 @findex Casts
@@ -16159,6 +16178,7 @@ create_definition:
   or    KEY [index_name] (index_col_name,...)
   or    INDEX [index_name] (index_col_name,...)
   or    UNIQUE [INDEX] [index_name] (index_col_name,...)
+  or    COLLECTION [collection_name] (collection_col_name,...)
   or    [CONSTRAINT symbol] FOREIGN KEY index_name (index_col_name,...)
             [reference_definition]
   or    CHECK (expr)
@@ -16401,6 +16421,14 @@ When you use @code{ORDER BY} or @code{GROUP BY} with a @code{TEXT} or
 @code{BLOB} column, only the first @code{max_sort_length} bytes are used.
 @xref{BLOB,  , @code{BLOB}}.
 
+@item
+In @strong{MySQL} 3.23.23 or later, you can also create special indexes
+called @strong{collections}. They are used for full-text search. Only
+@code{MyISAM} table type supports collections. Collection can be created
+from any mix of @code{VARCHAR}, @code{BLOB}, and @code{TEXT} columns.
+Indexing always happens over the entire column, partial indexing is not
+supported. See @ref{MySQL full-text search} for details of operation.
+
 @item
 The @code{FOREIGN KEY}, @code{CHECK} and @code{REFERENCES} clauses don't
 actually do anything.  The syntax for them is provided only for compatibility,
@@ -16570,6 +16598,7 @@ alter_specification:
   or    ADD INDEX [index_name] (index_col_name,...)
   or    ADD PRIMARY KEY (index_col_name,...)
   or    ADD UNIQUE [index_name] (index_col_name,...)
+  or    ADD COLLECTION [collection_name] (collection_col_name,...)
   or    ALTER [COLUMN] col_name @{SET DEFAULT literal | DROP DEFAULT@}
   or    CHANGE [COLUMN] old_col_name create_definition
   or    MODIFY [COLUMN] create_definition
@@ -19765,7 +19794,7 @@ For more information about how @strong{MySQL} uses indexes, see
 @ref{MySQL indexes, , @strong{MySQL} indexes}.
 
 @findex DROP INDEX
-@node DROP INDEX, Comments, CREATE INDEX, Reference
+@node DROP INDEX, CREATE COLLECTION, CREATE INDEX, Reference
 @section @code{DROP INDEX} syntax
 
 @example
@@ -19778,8 +19807,30 @@ prior to version 3.22.  In 3.22 or later, @code{DROP INDEX} is mapped to an
 @code{ALTER TABLE} statement to drop the index.
 @xref{ALTER TABLE, , @code{ALTER TABLE}}.
 
+@findex CREATE COLLECTION
+@node CREATE COLLECTION, Comments, DROP INDEX, Reference
+@section @code{CREATE COLLECTION} syntax
+
+@example
+CREATE COLLECTION collection_name ON tbl_name (col_name,... )
+@end example
+
+@code{CREATE COLLECTION} statement is mapped to an
+@code{ALTER TABLE} statement to create collections.
+@xref{ALTER TABLE, , @code{ALTER TABLE}}.
+
+A column list of the form @code{(col1,col2,...)} creates a
+multiple-column collection. Search in such a collection means a search
+over the concatenated columns that comprise the collection.
+
+There is no special @code{DROP COLLECTION} statement.
+@code{DROP INDEX} should be used to drop collections instead.
+
+Only @code{VARCHAR}, @code{BLOB}, and @code{TEXT} columns can be part
+of the collection. See @ref{MySQL full-text search} for details of operation.
+
 @findex Comment syntax
-@node Comments, CREATE FUNCTION, DROP INDEX, Reference
+@node Comments, CREATE FUNCTION, CREATE COLLECTION, Reference
 @section Comment syntax
 
 The @strong{MySQL} server supports the @code{# to end of line}, @code{--
@@ -33953,9 +34004,10 @@ working on the @strong{MySQL} code.
 
 @menu
 * MySQL threads::               MySQL threads
+* MySQL full-text search::      MySQL full-text search
 @end menu
 
-@node MySQL threads,  , MySQL internals, MySQL internals
+@node MySQL threads, MySQL full-text search, MySQL internals, MySQL internals
 @section MySQL threads
 
 The @strong{MySQL} server creates the the following threads:
@@ -33994,6 +34046,76 @@ started to read and apply updates from the master.
 @code{mysqladmin processlist} only shows the connection and @code{INSERT
 DELAYED} threads.
 
+@node MySQL full-text search, , MySQL threads, MySQL internals
+@section MySQL full-text search
+
+Since version 3.23.23, @strong{MySQL} has support for full-text indexing
+and searching. Full-text index in @strong{MySQL} is a special type of index
+called @strong{collection}. Collections can be created from @code{VARCHAR},
+@code{TEXT}, and @code{BLOB} columns at @code{CREATE TABLE}
+time or added later with @code{ALTER TABLE} or @code{CREATE COLLECTION}.
+Collection is queried with @code{MATCH} function.
+
+@example
+mysql> CREATE TABLE t (a VARCHAR(200), b TEXT, COLLECTION (a,b));
+Query OK, 0 rows affected (0.00 sec)
+mysql> INSERT INTO t VALUES
+    ->   ('MySQL has now support', 'for full-text search'),
+    ->   ('Full-text indexes', 'are called collections'),
+    ->   ('Only MyISAM tables','support collections'),
+    ->   ('Function MATCH ... AGAINST()','is used to do a search'),
+    ->   ('Full-text search in MySQL', 'implements vector space model');
+Query OK, 5 rows affected (0.00 sec)
+Records: 5  Duplicates: 0  Warnings: 0
+mysql> SELECT * FROM t WHERE MATCH (a,b) AGAINST ('MySQL');
++---------------------------+-------------------------------+
+| a                         | b                             |
++---------------------------+-------------------------------+
+| MySQL has now support     | for full-text search          |
+| Full-text search in MySQL | implements vector-space-model |
++---------------------------+-------------------------------+
+2 rows in set (0.00 sec)
+mysql> SELECT *,MATCH a,b AGAINST ('collections support') as x FROM t;
++------------------------------+-------------------------------+--------+
+| a                            | b                             | x      |
++------------------------------+-------------------------------+--------+
+| MySQL has now support        | for full-text search          | 0.3834 |
+| Full-text indexes            | are called collections        | 0.3834 |
+| Only MyISAM tables           | support collections           | 0.7668 |
+| Function MATCH ... AGAINST() | is used to do a search        |      0 |
+| Full-text search in MySQL    | implements vector space model |      0 |
++------------------------------+-------------------------------+--------+
+5 rows in set (0.00 sec)
+@end example
+
+Function @code{MATCH} matches a natural language query @code{AGAINST} a
+text collection. For every row in a table it returns relevance -
+similarity measure between the text in that row (in the columns, that
+are part of the collection) and the query. When it used in a @code{WHERE}
+clause (see example above) the rows returned are automatically sorted
+with relevance decreasing. Relevance is a non-negative floating point
+number. Zero relevance means no similarity. Relevance is computed based
+on number of words in the row and number of unique words in that row,
+total number of words in the collection, number of documents (rows),
+that contain a particular word, etc.
+
+MySQL uses very simple parser to split text into words. "Word" is
+any sequence of letters, numbers, @code{'}, and @code{_}. Any "word"
+that is present in stopword list or just too short (3 characters or less)
+is ignored.
+
+Every correct word in the collection and in the query is weighted,
+according to their significance in the query or collection. This way, a
+word that is present in many documents will have lower weight (and may
+even have a zero weight), because it has lower semantic value in this
+particular collection. Otherwise, if the word is rare, it will receive a
+higher weight. Weights of the words are then combined to compute the
+relevance.
+
+Such a technique works best with big collections (in fact, it was
+carefully tuned up this way). For very small tables word distribution
+does not reflect adequately their semantical value, and this model
+may sometimes produce bizarre results.
 
 @page
 @node Environment variables, Users, MySQL internals, Top