Skip to content

Sisyphus: A Cautionary Tale of Using Polynomial Activations in Privacy-Preserving Deep Learning

License

Notifications You must be signed in to change notification settings

kvgarimella/sisyphus-ppml

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Sisyphus: A Cautionary Tale of Using Low-Degree Polynomial Activations in Privacy-Preserving Deep Learning

This repository contains the code for the Sisyphus framework, a set of methods for wholesale ReLU replacement using polynomial activation functions in Private Inference. The repo is structured as followed:

  1. models: PyTorch implementation of various network architectures
  2. data: Instructions for downloading MNIST, CIFAR, and TinyImageNet
  3. experiments
    • baselines: pipeline to train baseline networks with ReLU
    • tayloy_approx: Taylor series approximation of ReLU
    • poly_regression: Polynomial regression fit of ReLU
    • quail: Quadratic Imitation Learning training pipeline
    • approxminmax_quail: ApproxMinMaxNorm implementation
    • test_networks: simply test loss and accuracy evaluation script

Installation

Clone this repo:

git clone https://github.com/sisyphus-project/sisyphus-ppml.git
cd sisyphus-ppml

Install the required Python packages:

pip install -r requirements.txt

Setup two environment variables (for the datasets and models). You may want to add these environment variables to your bashrc file.

export PYTHONPATH="$PYTHONPATH:$(pwd)/models"
export DATASET_DIR=$(pwd)/data

Follow the instructions in the data directory to download the datasets. We use wandb to log our experiments.

Citation

If you find our work useful, kindly cite us with:

@inproceedings{garimella2021sisyphus,
      author={Garimella, Karthik  and Jha, Nandan Kumar and Reagen, Brandon},
      title={Sisyphus: A Cautionary Tale of Using Low-Degree
      Polynomial Activations in Privacy-Preserving Deep Learning}, 
      booktitle	= {ACM CCS Workshop on Private-preserving Machine Learning},
      year={2021},
      doi={10.48550/ARXIV.2107.12342}
}

Example

To run a baseline model, move to the baselines directory and run:

python train_mnist.py --project=sisyphus-baseline --name=mnist-mlp --model=mlp_bn

For more detailed instructions on running experiments, please refer to the READMEs in each subdirectory.

About

Sisyphus: A Cautionary Tale of Using Polynomial Activations in Privacy-Preserving Deep Learning

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published