I want to get the“hidden_states” data in the Bert model

[dogdogpp]

Describe the feature request

I'm running a TTS project where the Bert model needs to get the“hidden_states” component of the middle layer output for later processing. I'm using C + + and can't seem to find a way to get the middle layer output, any suggestions?

Describe scenario use case

This method is handy for using the Bert Model, because hidden_states are also useful features, which can be implemented in the Python version of Transformer, but is difficult to implement in C + + .

[tianleiwu]

You can add "hidden_states" to model output, and export the model to onnx so that onnx model also has the output.

For example, in huggingface transformers, you can pass pass output_hidden_states=True so that the hidden states is output.
https://github.com/huggingface/transformers/blob/37bba2a32d2742a10216ffd925bb8f145a732ce1/src/transformers/models/megatron_bert/modeling_megatron_bert.py#L527

[dogdogpp]

自动回复你好，我已收到来信，将会尽快回复，谢谢。

[dogdogpp]

Yes. However, the transformer only has a python version, and what I need to achieve is to get this data in the C++ version, and I can't use the transformer. Is there a way to make it happen, thanks!

您可以在模型输出中添加“hidden_states”，并将模型导出到 onnx，以便 onnx 模型也具有输出。

例如，在 huggingface transformers 中，您可以传递 pass，以便输出隐藏状态。 https://github.com/huggingface/transformers/blob/37bba2a32d2742a10216ffd925bb8f145a732ce1/src/transformers/models/megatron_bert/modeling_megatron_bert.py#L527`output_hidden_states=True`