Same, Same But Different

An Tensorflow.keras implementation of ICML2019 paper "Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorization"(https://arxiv.org/pdf/1902.01917.pdf)

Quick Start

This code has been tested on Ubuntu 18.04, Python 3.7, Tensorflow 2.0

• Clone this repository

    git clone https://github.com/Adamdad/Samesame.git 
  

• Network equalization

    python equalization.py
  

  In this code, I default equalize the Inception_v3 implemented by keras.application

• Network visualization(in visual_weights_per_channel.py)

  • Per-channel convolution kernel weight visualization

      def visual_weight(model)
    

  • Per-channel activation feature map visualization

      def visual_activation(model,x)
    

  • Network architecture visualization

      def visual_graph(model,fig_name='model.png')
    

Quantization

The post-training quantization is implemented with TF-Lite, with two mode of "weight quantization" and "full quantization". See details in def qevaluate

Performance Compare

Imagenet Validation result

Network	32bit		32bit-Equalized		8bit-weight		8bit-Equalized-weight
	top1	top5	top1	top5	top1	top5	top1	top5
Inception-v3	76.276	93.032	75.61	92.58

Citation

@article{meller2019same,
  title={Same, same but different-recovering neural network quantization error through weight factorization},
  author={Meller, Eldad and Finkelstein, Alexander and Almog, Uri and Grobman, Mark},
  journal={arXiv preprint arXiv:1902.01917},
  year={2019}
}