Skip to content

Official Implementation of Official Implementation of ArtNeRF: A Stylized Neural Field for 3D-Aware Cartoonized Face Synthesis

Notifications You must be signed in to change notification settings

silence-tang/ArtNeRF

Repository files navigation

Environment

  • GPU: 1 NVIDIA GeForce RTX 2080 Ti with 11GB memory is enough.
  • OS: Linux Ubuntu 18.04 LTS
  • IDE: Visual Studio Code 2022.09
  • Others: Python3.7 + PyTorch1.8.1 + CUDA10.1

Preparation

  • To prepare data and pretraind models, please check all the file folders in this project and follow the guidance in readme.md.
  • To accelerate the training process, we precompute the 512-dim style code for every artistic human face, you can download style_codes.csv and place it under ArtNerf/.

Training

  • The model is trained by conducting a two-stage training strategy: pretraining on CelebA and fine-tuning on both AAHQ and CelebA.
  • The whole model is composed of 1 generator and 3 dicriminators. disc_real guides the gen to generate natural human faces and disc_style guides the gen to generate stylized human faces. disc_latent helps ensure the style-consistency between generated faces and ref faces.
  • We use a style blending module to help stabilize the cross-domain transfer learning process and allow users to change the extent to which the generated images is stylized(level can be changed from 0 to 11).

Examples

Main results

Style Image fake_a fake_b (i = 0) fake_b (i = 3) fake_b (i = 11)
style_img_1.png fake_a_128_yaw_only_1.gif fake_b1_128_yaw_only_1.gif fake_b2_128_yaw_only_1.gif fake_a_128_yaw_only_1.gif
style_img_3.png fake_a_128_yaw_only_3.gif fake_b1_128_yaw_only_3.gif fake_b2_128_yaw_only_3.gif fake_a_128_yaw_only_3.gif
style_img_5.png fake_a_128_yaw_only_5.gif fake_b1_128_yaw_only_5.gif fake_b2_128_yaw_only_5.gif fake_a_128_yaw_only_5.gif
style_img_6.png fake_a_128_yaw_only_6.gif fake_b1_128_yaw_only_6.gif fake_b2_128_yaw_only_6.gif fake_a_128_yaw_only_6.gif
style_img_7.png fake_a_128_yaw_only_7.gif fake_b1_128_yaw_only_7.gif fake_b2_128_yaw_only_7.gif fake_a_128_yaw_only_7.gif

Here shows some other stylized face avatars with different resolutions.

64×64

64_yaw_only_0.gif 64_yaw_only_1.gif 64_yaw_only_2.gif 64_yaw_only_3.gif 64_yaw_only_4.gif
64_yaw_only_5.gif 64_yaw_only_6.gif 64_yaw_only_7.gif 64_yaw_only_8.gif 64_yaw_only_9.gif

128x128

128_yaw_only_0.gif 128_yaw_only_1.gif 128_yaw_only_2.gif 128_yaw_only_5.gif
128_yaw_only_9.gif 128_yaw_only_6.gif 128_yaw_only_8.gif 128_yaw_only_3.gif

Latent Space Interpolation


Following the traditional face synthesis models like StyleGAN, we can perform interpolation between any two latent codes.

fake_a_128_front_1.gif fake_a_128_front_3.gif fake_a_128_front_4.gif fake_a_128_yaw_only_4.gif fake_b1_128_front_3.gif
fake_b1_128_front_1.gif interp_z_b_yaw_only_6.gif interp_z_b_yaw_only_7.gif interp_z_b_yaw_only_8.gif fake_b1_128_front_2.gif
interp_z_b_yaw_only_1.gif interp_z_b_yaw_only_4.gif interp_z_b_yaw_only_10.gif fake_a_128_yaw_only_0.gif fake_a_128_yaw_only_2.gif 

| fake_b1_128_front_1.gif | interp_z_b_yaw_only_6.gif | interp_z_b_yaw_only_7.gif | interp_z_b_yaw_only_8.gif | fake_b1_128_front_2.gif | | interp_z_b_yaw_only_1.gif | interp_z_b_yaw_only_4.gif | interp_z_b_yaw_only_10.gif | fake_a_128_yaw_only_0.gif | fake_a_128_yaw_only_2.gif  |

About

Official Implementation of Official Implementation of ArtNeRF: A Stylized Neural Field for 3D-Aware Cartoonized Face Synthesis

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages