Thanks for the note, we'll take a look!

The suggestion for the lemmatizer is included in #12554.

For the poor tagging, etc. with statistical models for the tokens with diacritics, I think the best option would be to configure custom NORM, PREFIX, and SUFFIX features for ru and uk that strip diacritics. If you wanted to try this out with the current spacy release (v3.5), you could use a custom language to customize these methods, called lex_attr_getters in the defaults similar to this:

https://spacy.io/usage/linguistic-features#language-subclass

The defaults would be extended similar to this:

Wonderful! Thank you for the quick PR and suggestions.

I'm a noob when it comes to spaCy. I'm using it to generate tags on anki flashcards to study Russian. But, if I understand you correctly, the model I use should be trained with diacritics. Is that correct (e.g. ru_core_news_lg will not work)?

I ask because I tried making a custom language and the results were still unsatisfactory (even with a patch similar to #12554).

DIACRITICS_RE = re.compile(f'[{COMBINING_DIACRITICS}]')
def norm(s: str) -> str:
    return DIACRITICS_RE.sub('', s.lower())
def prefix(s: str) -> str:
    return DIACRITICS_RE.sub('', s.lower())[0]
def suffix(s: str) -> str:
    return DIACRITICS_RE.sub('', s.lower())[-3:]
ATTR_GETTERS = spacy.lang.ru.LEX_ATTRS
ATTR_GETTERS.update({
    attrs.NORM: norm,
    attrs.PREFIX: prefix,
    attrs.SUFFIX: suffix,
})

class CustomRussianDefaults(Russian.Defaults):
    lex_attr_getters = ATTR_GETTERS

@spacy.registry.languages("custom_ru")
class CustomRussian(Russian):
    lang = "custom_ru"
    Defaults = CustomRussianDefaults

nlp = ru_core_news_lg.load()
# omitted the patching of _pymorphy_lemmatize
nlp.lang = 'custom_ru'

Test

>>> nlp('Я ви́жу му́жа и жену́')[-1].morph
Animacy=Inan|Case=Acc|Gender=Fem|Number=Sing
>>> nlp('Я вижу мужа и жену')[-1].morph
Animacy=Anim|Case=Acc|Gender=Fem|Number=Sing

The Animacy for жену́ is inanimate with diacritics, which is incorrect.

The language and language defaults really needs to be set before the pipeline is loaded at all, but you can test this a bit by modifying the pipeline on-the-fly instead. (A few things may already be cached so it might not work 100%.)

nlp = spacy.load("ru_core_news_lg")
nlp.vocab.lex_attr_getters.update(...)

A cleaner version would basically make a copy of ru_core_news_lg where [nlp.lang] is edited to custom_ru. But with the above you should be able to test most things out. And keep in mind that the statistical models will still make mistakes, especially for ambiguous cases.

I had the same problem and discovered at least a workaround:
One can create two docs, one with the original stressed text, and one with the text with diacritics removed.
That way you can iterate through the docs in parallel, getting the correct (stressed) text from doc 1 while getting the grammatical information from doc 2.

It's half as fast, but it does work.