DeepFlux for Skeletons in the Wild

Introduction

The code and trained models of:

DeepFlux for Skeletons in the Wild, CVPR 2019 [Paper]

Citation

Please cite the related works in your publications if it helps your research:

@inproceedings{wang2019deepflux,
  title={DeepFlux for Skeletons in the Wild},
  author={Wang, Yukang and Xu, Yongchao and Tsogkas, Stavros and Bai, Xiang and Dickinson, Sven and Siddiqi, Kaleem},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={5287--5296},
  year={2019}
}

Prerequisite

• Caffe and VGG-16 pretrained model [VGG_ILSVRC_16_layers.caffemodel]

• Datasets: [SK-LARGE], [SYM-PASCAL]

• OpenCV 3.4.3 (C++ or Python, optional)

• MATLAB

Usage

1. Install Caffe

cp Makefile.config.example Makefile.config
# adjust Makefile.config (for example, enable python layer)
make all -j16
# make sure to include $CAFFE_ROOT/python to your PYTHONPATH.
make pycaffe

Please refer to Caffe Installation to ensure other dependencies.

2. Data and model preparation

# download datasets and pretrained model then
mkdir data && mv [your_dataset_folder] data/
mkdir models && mv [your_pretrained_model] models/
# data augmentation
cd data/[your_dataset_folder]
matlab -nodisplay -r "run augmentation.m; exit"

3. Training scripts

# an example on SK-LARGE dataset
cd examples/DeepFlux/
python train.py --gpu [your_gpu_id] --dataset sklarge --initmodel ../../models/VGG_ILSVRC_16_layers.caffemodel

For training an end-to-end version of DeepFlux, adding the --e2e option.

4. Evaluation scripts

# an example on SK-LARGE dataset
cd evaluation/
# inference with C++
./eval_cpp.sh ../../data/SK-LARGE/images/test ../../data/SK-LARGE/groundTruth/test ../../models/sklarge_iter_40000.caffemodel
# inference with Python
./eval_py.sh ../../data/SK-LARGE/images/test ../../data/SK-LARGE/groundTruth/test ../../models/sklarge_iter_40000.caffemodel
# inference with Python (end-to-end version)
./eval_py_e2e.sh ../../data/SK-LARGE/images/test ../../data/SK-LARGE/groundTruth/test ../../models/sklarge_iter_40000.caffemodel

Results and Trained Models

SK-LARGE

Backbone	F-measure	Comment & Link
VGG-16	0.732	CVPR submission [Google drive]
VGG-16	0.735	different_lr [Google drive]
VGG-16	0.737	end-to-end [Google drive]

SYM-PASCAL

Backbone	F-measure	Comment & Link
VGG-16	0.502	CVPR submission [Google drive]
VGG-16	0.558	different_lr [Google drive]
VGG-16	0.569	end-to-end [Google drive]

*different_lr means different learning rates for backbone and additional layers

*lambda=0.4, k1=3, k2=4 for all models with post-processing