Spellcheck + dictionaries

[FuriusBaco]

Proposal:

Please consider checking words through SUGGEST first using the standard dictionary of Russian or any other language and first suggesting substitutions from there, and only then looking for them in the index. Optional, of course.

Yes, for academic texts where there are no errors, the current implementation is quite suitable. But for the forum, it doesn't always work well. For example, the word “pamagite” is replaced by “pomagite”, “pamagiti”, “pamagitya”, “poamagite”, etc.

This will also be good for the words in the query that are not on the site, but there may still be useful information in the query.

Checklist:

^{To be completed by the assignee. Check off tasks that have been completed or are not applicable.}

• Task estimated
• Specification created, reviewed and approved
• Implementation completed
• Tests developed
• Documentation updated
• Documentation proofread
• Changelog updated
• OpenAPI YAML updated and issue created to rebuild clients

[tomatolog]

what kind of dictionary you suggesting? Could you provide reproducible case - config, data to populate with the forum index and data to populate with the standard dictionary of Russian and query to both indexes with the result sets there it is clear why idx_right should be used but not the idx_forum.

Do you mean that you create and full another index with right content when use this form of CALL SUGGEST(word, 'idx_right,idx_forum') and if the 1st search in the idx_right term levinstein distance is closer to N (some new param) then not to search in the idx_forum ?

As if search into both idx_right and idx_forum what is the condition why to use idx_right but not idx_forum if levinstein distance is closer for term from idx_forum ?

[FuriusBaco]

I can't fill in the indexes. Any forum will do.

A dictionary is a reliable source, so it should be used first. By the way, such a "CALL SUGGEST(words, 'idx_right,idx_forum')" format would also be very good! For example, I have a database with 1.7 million words. There is a similar one on Github (https://github.com/danakt/russian-words ). Manticore has its own dictionary (ru.pak, for example).

Now I'm making a Speller using just such a system - first a dictionary, then a forum. After some time, I will be able to tell you about the results.

You can make two modes - the priority of dictionaries as they are enumerated and as the Levenshtein distance increases, or a SORT BY operator or something similar.