Unexpected and missing key's in state-dict when constructing DeepLabv3+ model

[jackfoster97]

Sorry for the large copy and paste dump. I get the below when trying to run the preprocess_dataset.sh file.

I am not quite sure what the issue is and cannot find any help online, any chance you might know what is going wrong?

Constructing DeepLabv3+ model...
Number of classes: 20
Output stride: 16
Number of Input Channels: 3
unexpected key "source_graph_2_fea.node_fea_for_res" in state_dict
unexpected key "source_graph_2_fea.node_fea_for_hidden" in state_dict
unexpected key "source_graph_2_fea.weight" in state_dict
unexpected key "source_skip_conv.0.weight" in state_dict
unexpected key "source_skip_conv.0.bias" in state_dict
unexpected key "source_semantic.weight" in state_dict
unexpected key "source_semantic.bias" in state_dict
unexpected key "middle_semantic.weight" in state_dict
unexpected key "middle_semantic.bias" in state_dict
unexpected key "middle_source_featuremap_2_graph.pre_fea" in state_dict
unexpected key "middle_source_featuremap_2_graph.weight" in state_dict
unexpected key "middle_source_graph_conv1.weight" in state_dict
unexpected key "middle_source_graph_conv2.weight" in state_dict
unexpected key "middle_source_graph_conv3.weight" in state_dict
unexpected key "middle_source_graph_2_fea.node_fea_for_res" in state_dict
unexpected key "middle_source_graph_2_fea.node_fea_for_hidden" in state_dict
unexpected key "middle_source_graph_2_fea.weight" in state_dict
unexpected key "middle_skip_conv.0.weight" in state_dict
unexpected key "middle_skip_conv.0.bias" in state_dict
unexpected key "transpose_graph_source2target.weight" in state_dict
unexpected key "transpose_graph_source2target.adj" in state_dict
unexpected key "transpose_graph_target2source.weight" in state_dict
unexpected key "transpose_graph_target2source.adj" in state_dict
unexpected key "transpose_graph_middle2source.weight" in state_dict
unexpected key "transpose_graph_middle2source.adj" in state_dict
unexpected key "transpose_graph_middle2target.weight" in state_dict
unexpected key "transpose_graph_middle2target.adj" in state_dict
unexpected key "transpose_graph_source2middle.weight" in state_dict
unexpected key "transpose_graph_source2middle.adj" in state_dict
unexpected key "transpose_graph_target2middle.weight" in state_dict
unexpected key "transpose_graph_target2middle.adj" in state_dict
unexpected key "fc_graph_source.weight" in state_dict
unexpected key "fc_graph_target.weight" in state_dict
unexpected key "fc_graph_middle.weight" in state_dict
missing keys in state_dict: "{'xception_features.block10.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block3.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block19.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block15.rep.5.pointwise_bn.num_batches_tracked', 'xception_features.block10.rep.3.depthwise_bn.num_batches_tracked', 'xception_features.block16.rep.3.depthwise_bn.num_batches_tracked', 'xception_features.block19.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.block3.rep.3.depthwise_bn.num_batches_tracked', 'xception_features.conv5.depthwise_bn.num_batches_tracked', 'xception_features.block11.rep.5.pointwise_bn.num_batches_tracked', 'xception_features.block20.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block13.rep.3.pointwise_bn.num_batches_tracked', 'transpose_graph.weight', 'xception_features.block10.rep.3.pointwise_bn.num_batches_tracked', 'aspp2.atrous_convolution.pointwise_bn.num_batches_tracked', 'xception_features.block8.rep.3.pointwise_bn.num_batches_tracked', 'xception_features.block11.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.block6.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block1.rep.2.pointwise_bn.num_batches_tracked', 'xception_features.block6.rep.3.depthwise_bn.num_batches_tracked', 'xception_features.block16.rep.1.pointwise_bn.num_batches_tracked', 'fc_graph.weight', 'xception_features.block2.block2_lastconv.1.depthwise_bn.num_batches_tracked', 'xception_features.block8.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block3.skipbn.num_batches_tracked', 'xception_features.conv4.pointwise_bn.num_batches_tracked', 'xception_features.block16.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.block4.rep.5.pointwise_bn.num_batches_tracked', 'xception_features.block2.rep.1.pointwise_bn.num_batches_tracked', 'decoder.1.atrous_convolution.pointwise_bn.num_batches_tracked', 'xception_features.block13.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block2.rep.3.depthwise_bn.num_batches_tracked', 'xception_features.block13.rep.3.depthwise_bn.num_batches_tracked', 'xception_features.block15.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block17.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block5.rep.3.pointwise_bn.num_batches_tracked', 'xception_features.block9.rep.3.depthwise_bn.num_batches_tracked', 'xception_features.block16.rep.5.pointwise_bn.num_batches_tracked', 'xception_features.block14.rep.5.pointwise_bn.num_batches_tracked', 'xception_features.block20.rep.3.depthwise_bn.num_batches_tracked', 'xception_features.block1.skipbn.num_batches_tracked', 'xception_features.block1.rep.0.depthwise_bn.num_batches_tracked', 'xception_features.block16.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block19.rep.5.pointwise_bn.num_batches_tracked', 'xception_features.block7.rep.3.depthwise_bn.num_batches_tracked', 'decoder.0.atrous_convolution.depthwise_bn.num_batches_tracked', 'xception_features.block17.rep.3.pointwise_bn.num_batches_tracked', 'xception_features.block8.rep.5.pointwise_bn.num_batches_tracked', 'xception_features.block6.rep.3.pointwise_bn.num_batches_tracked', 'xception_features.block2.block2_lastconv.1.pointwise_bn.num_batches_tracked', 'global_avg_pool.2.num_batches_tracked', 'aspp3.atrous_convolution.pointwise_bn.num_batches_tracked', 'xception_features.block6.rep.5.pointwise_bn.num_batches_tracked', 'xception_features.block6.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block12.rep.3.depthwise_bn.num_batches_tracked', 'decoder.0.atrous_convolution.pointwise_bn.num_batches_tracked', 'xception_features.block16.rep.3.pointwise_bn.num_batches_tracked', 'xception_features.block3.rep.3.pointwise_bn.num_batches_tracked', 'xception_features.bn1.num_batches_tracked', 'xception_features.block11.rep.3.depthwise_bn.num_batches_tracked', 'xception_features.block11.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block12.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block13.rep.5.pointwise_bn.num_batches_tracked', 'xception_features.block8.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block14.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.block11.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block7.rep.3.pointwise_bn.num_batches_tracked', 'xception_features.block13.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.block6.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.conv5.pointwise_bn.num_batches_tracked', 'xception_features.block1.rep.2.depthwise_bn.num_batches_tracked', 'aspp4.atrous_convolution.pointwise_bn.num_batches_tracked', 'xception_features.block4.rep.3.depthwise_bn.num_batches_tracked', 'xception_features.block19.rep.3.depthwise_bn.num_batches_tracked', 'xception_features.block3.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.block14.rep.3.pointwise_bn.num_batches_tracked', 'xception_features.block1.rep.4.pointwise_bn.num_batches_tracked', 'xception_features.block12.rep.3.pointwise_bn.num_batches_tracked', 'xception_features.block1.rep.0.pointwise_bn.num_batches_tracked', 'xception_features.block18.rep.3.depthwise_bn.num_batches_tracked', 'xception_features.conv4.depthwise_bn.num_batches_tracked', 'xception_features.conv3.pointwise_bn.num_batches_tracked', 'xception_features.block7.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.block17.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.block19.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block20.skipbn.num_batches_tracked', 'transpose_graph.adj', 'xception_features.block5.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block14.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block12.rep.5.pointwise_bn.num_batches_tracked', 'xception_features.block12.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block1.rep.4.depthwise_bn.num_batches_tracked', 'xception_features.block15.rep.1.depthwise_bn.num_batches_tracked', 'aspp3.atrous_convolution.depthwise_bn.num_batches_tracked', 'xception_features.block9.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.block20.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block9.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block18.rep.5.pointwise_bn.num_batches_tracked', 'xception_features.block20.rep.5.pointwise_bn.num_batches_tracked', 'aspp2.atrous_convolution.depthwise_bn.num_batches_tracked', 'xception_features.block15.rep.3.pointwise_bn.num_batches_tracked', 'xception_features.block14.rep.3.depthwise_bn.num_batches_tracked', 'xception_features.block18.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block19.rep.3.pointwise_bn.num_batches_tracked', 'xception_features.block5.rep.5.pointwise_bn.num_batches_tracked', 'xception_features.block17.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block15.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block8.rep.3.depthwise_bn.num_batches_tracked', 'aspp1.bn.num_batches_tracked', 'xception_features.block2.rep.3.pointwise_bn.num_batches_tracked', 'xception_features.conv3.depthwise_bn.num_batches_tracked', 'xception_features.block14.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block18.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.block4.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block18.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block8.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.block5.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.block9.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block18.rep.3.pointwise_bn.num_batches_tracked', 'xception_features.block17.rep.5.pointwise_bn.num_batches_tracked', 'xception_features.block13.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block11.rep.3.pointwise_bn.num_batches_tracked', 'xception_features.block4.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.block2.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.block7.rep.5.pointwise_bn.num_batches_tracked', 'xception_features.block10.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block5.rep.3.depthwise_bn.num_batches_tracked', 'xception_features.block20.rep.3.pointwise_bn.num_batches_tracked', 'decoder.1.atrous_convolution.depthwise_bn.num_batches_tracked', 'xception_features.block4.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block5.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block9.rep.3.pointwise_bn.num_batches_tracked', 'xception_features.block17.rep.3.depthwise_bn.num_batches_tracked', 'xception_features.block2.skipbn.num_batches_tracked', 'xception_features.bn2.num_batches_tracked', 'xception_features.block7.rep.5.depthwise_bn.num_batches_tracked', 'xception_features.block20.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.block9.rep.5.pointwise_bn.num_batches_tracked', 'xception_features.block4.rep.3.pointwise_bn.num_batches_tracked', 'feature_projection_bn1.num_batches_tracked', 'concat_projection_bn1.num_batches_tracked', 'xception_features.block3.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block10.rep.1.depthwise_bn.num_batches_tracked', 'xception_features.block7.rep.1.pointwise_bn.num_batches_tracked', 'xception_features.block3.rep.5.pointwise_bn.num_batches_tracked', 'xception_features.block12.rep.1.depthwise_bn.num_batches_tracked', 'aspp4.atrous_convolution.depthwise_bn.num_batches_tracked', 'xception_features.block15.rep.3.depthwise_bn.num_batches_tracked', 'xception_features.block10.rep.5.pointwise_bn.num_batches_tracked'}"
--images_path (/tmp/tmp.93UyFgH4hX) is a file, reading it for a list of files...
Found 1 images
Will output files in /home/ubuntu/data/dev/projects/latent-pose-reenactment/segmentation-cropped with names relative to /home/ubuntu/data/dev/projects/latent-pose-reenactment/images-cropped.
Example:
The segmentation for: /home/ubuntu/data/dev/projects/latent-pose-reenactment/images-cropped/jack/00000.jpg
Will be put in: /home/ubuntu/data/dev/projects/latent-pose-reenactment/segmentation-cropped/jack
2022-08-11 12:47:48: 0 / 1
/home/ubuntu/anaconda3/envs/lpr/lib/python3.7/site-packages/torch/nn/functional.py:2941: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
Average inference time: 0.46386814300001333
/home/ubuntu/data/dev/projects/latent-pose-reenactment/utils

[shrubb]

Hi, check #28. I think it's not an error.

[jackfoster97]

Hi Shrubb, thank you that was useful.

Sorry just a follow-up question as I am progressing through the process. In the final stage of running drive.py I am receiving this error message:

Traceback (most recent call last):
File "drive.py", line 68, in
generator.load_state_dict(running_averages['generator'])
File "/home/ubuntu/anaconda3/envs/lpr/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1045, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for Generator:
Missing key(s) in state_dict: "identity_embedding".

Just wondering if you know what I am doing wrong to be getting this message?

Thanks in advance

[shrubb]

I don't know... Just make sure you're exactly following the instructions. We tested that multiple times, it should work.

[brianw0924]

same problem

RuntimeError: Error(s) in loading state_dict for Generator:
Missing key(s) in state_dict: "identity_embedding".

[jackfoster97]

hi brainw0924, I am pretty sure I managed to get past this issue by restarting the process again for both the target and source video/images