README

Currently the code is structured under Tensorflow 1.14.0. If GPUs are not available, please directly remove the GPU setting when initialize the session.

FLAGS

The code implements variational inference for Gaussian Process-Gated Hierarchical Mixture of Experts (GPHME) approximated using random Fourier features. The code accepts the following options:

• --fold             Fold of dataset
• --seed             Seed for Tensorflow and Numpy
• --n_rff             Number of random features
• --h_tree           Height of trees
• --dataset           Name of dataset
• --mc_test           Number of Monte Carlo samples for predictions
• --mc_train         Number of Monte Carlo samples for training
• --ard_type         How to treat Omega: it can be 0 for 'ISO-N', 1 for 'ISO-L', and 2 for 'NIS-N'
• --duration           Duration in minutes
• --optimizer         Optimizer: adam, adagrad, adadelta, or sgd
• --batch_size         Batch size
• --kernel_type       Kernel: RBF, arccosine, or identity
• --less_prints         Disables evaluations during the training steps
• --theta_fixed         Number of iterations to keep theta fixed at the beginning
• --n_iterations       Number of iterations (batches) to train the DGP model
• --display_step       Display progress every display_step iterations
• --learning_rate         Learning rate for optimizers
• --local_reparam       Use the local reparameterization trick
• --q_Omega_fixed     Number of iterations to keep posterior of Omega fixed at the beginning

EXAMPLES

Here are two examples to run the GPHME model on regression and classification tasks:

REGRESSION

python experiments/sdt_rff_regression_gpu.py --seed=12345 --dataset=ABA --fold=1 --q_Omega_fixed=1000 \
--theta_fixed=4000 --ard_type=0 --optimizer=adam --h_tree=2 --learning_rate=0.001 --n_rff=50 \
--batch_size=200 --mc_train=100 --mc_test=100 --n_iterations=100000 --display_step=250 --duration=60 \
--kernel_type=RBF

CLASSIFICATION

python experiments/sdt_rff_classification_gpu.py --seed=12345 --dataset=OPT --fold=1 --q_Omega_fixed=1000 \
--theta_fixed=4000 --ard_type=0 --optimizer=adam --h_tree=2 --learning_rate=0.001 --n_rff=100 \
--batch_size=200 --mc_train=100 --mc_test=100 --n_iterations=100000 --display_step=250 --duration=60 \
--kernel_type=arccosine