Fix keys name for Transformer

[Adel-Moumen]

What does this PR do?

This PR solves an issue due to #2489. Indeed, this PR modify the name of a key in decoder transformer self.mutihead_attn to self.multihead_attn. Doing so breaks the loading of state dict since it does not recognize the previous key. In order to solve this problem, I introduce a new function called map_old_state_dict_weights which is directly applied within torch_recovery, average_checkpoints and _load_from_state_dict. The first is when you are loading a checkpoint, the second when you are trying to avg multiple checkpoints (the issue here is that you need to make sure that before doing the avg, every ckpts has the same keys), and the latter when you are loading the state_dict directly from the object which can be the case in our codebase (i.e. bypassing checkpointer).

Before submitting

• Did you read the contributor guideline?
• Did you make sure your PR does only one thing, instead of bundling different changes together?
• Did you make sure to update the documentation with your changes? (if necessary)
• Did you write any new necessary tests? (not for typos and docs)
• Did you verify new and existing tests pass locally with your changes?
• Did you list all the breaking changes introduced by this pull request?
• Does your code adhere to project-specific code style and conventions?

PR review

Reviewer checklist

• Is this pull request ready for review? (if not, please submit in draft mode)
• Check that all items from Before submitting are resolved
• Make sure the title is self-explanatory and the description concisely explains the PR
• Add labels and milestones (and optionally projects) to the PR so it can be classified
• Confirm that the changes adhere to compatibility requirements (e.g., Python version, platform)
• Review the self-review checklist to ensure the code is ready for review

[asumagic]

Shouldn't we have a more generic state_dict fixup function that would call into map_old_state_dict_weights, even if the general function only calls that one for now?

[Adel-Moumen]

I don't understand your question/remark sorry haha. Could you please explain a bit more what you meant by general function?

[asumagic]

Having a function where we could add other state_dict fixes in the future.

[Adel-Moumen]

so you mean something like:

def foo(state_dict):
	state_dict(state_dict)

so that someone can do :

def foo(state_dict):
	bar(state_dict)
	state_dict(state_dict)

easily?

[asumagic]

It is unclear the extent of what this is, in particular, to my understanding:

• This can affect any part of the key, which could look like somemodule.somesubmodule.somefield rather than just somefield
• This will do partial matches on keys (e.g. "a": "b" will turn _a into _b)

The behavior should be documented to avoid accidentally including renames with too many side effects.

[asumagic]

Should we have a warning here?

[Adel-Moumen]

Yep. Either a warning or a log in debug mode.