add emoji search

[strong-xiaoge]

Hope to increase support for emoji search

[sanikolaev]

Here's an example:

mysql> drop table if exists t; create table t(f text) charset_table='non_cjk,U+1F601'; call keywords('abc😁', 't'); insert into t(f) values('abc'),('abc😁'); select * from t where match('abc'); select * from t where match('abc😁');
--------------
drop table if exists t
--------------

Query OK, 0 rows affected (0.01 sec)

--------------
create table t(f text) charset_table='non_cjk,U+1F601'
--------------

Query OK, 0 rows affected (0.00 sec)

--------------
call keywords('abc😁', 't')
--------------

+------+-----------+------------+
| qpos | tokenized | normalized |
+------+-----------+------------+
| 1    | abc😁       | abc😁        |
+------+-----------+------------+
1 row in set (0.00 sec)

--------------
insert into t(f) values('abc'),('abc😁')
--------------

Query OK, 2 rows affected (0.00 sec)

--------------
select * from t where match('abc')
--------------

+---------------------+------+
| id                  | f    |
+---------------------+------+
| 1515816049293918213 | abc  |
+---------------------+------+
1 row in set (0.00 sec)
--- 1 out of 1 results in 0ms ---

--------------
select * from t where match('abc😁')
--------------

+---------------------+---------+
| id                  | f       |
+---------------------+---------+
| 1515816049293918214 | abc😁     |
+---------------------+---------+
1 row in set (0.00 sec)
--- 1 out of 1 results in 0ms ---

[strong-xiaoge]

At the same time, it also needs to support the use of Chinese search, how to set it?

[sanikolaev]

- charset_table='non_cjk,U+1F601'
+ charset_table='chinese,non_cjk,U+1F601'
+ morphology='icu_chinese'

Please take this course for more details - https://play.manticoresearch.com/icu-chinese/

[strong-xiaoge]

I understand. Thank you for your answer.