Skip to content

Implementation of Inverse Reinforcement Learning (IRL) algorithms in Python/Tensorflow. Deep MaxEnt, MaxEnt, LPIRL

Notifications You must be signed in to change notification settings

yrlu/irl-imitation

Repository files navigation

irl-imitation

DOI

Implementation of selected Inverse Reinforcement Learning (IRL) algorithms in Python/Tensorflow.

$ python demo.py

Implemented Algorithms
  • Linear inverse reinforcement learning (Ng & Russell, 2000)
  • Maximum entropy inverse reinforcement learning (Ziebart et al., 2008)
  • Maximum entropy deep inverse reinforcement learning (Wulfmeier et al., 2015)
Implemented MDPs & Solver
  • 2D gridworld
  • 1D gridworld
  • Value iteration

If you use this software in your publications, please cite it using the following BibTeX entry:

@misc{lu2017irl-imitation,
  author = {Lu, Yiren},
  doi = {10.5281/zenodo.6796157},
  month = {7},
  title = {{Implementations of inverse reinforcement learning algorithms in Python/Tensorflow}},
  url = {https://github.com/yrlu/irl-imitation},
  year = {2017}
}

Dependencies

  • python 2.7
  • cvxopt
  • Tensorflow 0.12.1
  • matplotlib

Linear Inverse Reinforcement Learning

$ python linear_irl_gridworld.py --act_random=0.3 --gamma=0.5 --l1=10 --r_max=10

Maximum Entropy Inverse Reinforcement Learning

(This implementation is largely influenced by Matthew Alger's maxent implementation)

$ python maxent_irl_gridworld.py --height=10 --width=10 --gamma=0.8 --n_trajs=100 --l_traj=50 --no-rand_start --learning_rate=0.01 --n_iters=20

$ python maxent_irl_gridworld.py --gamma=0.8 --n_trajs=400 --l_traj=50 --rand_start --learning_rate=0.01 --n_iters=20

Maximum Entropy Deep Inverse Reinforcement Learning

  • Following Wulfmeier et al. 2015 paper: Maximum Entropy Deep Inverse Reinforcement Learning. FC version implemented. The implementation does not follow exactly the model proposed in the paper. Some tweaks applied including elu activations, clipping gradients, l2 regularization etc.
  • $ python deep_maxent_irl_gridworld.py --help for options descriptions
$ python deep_maxent_irl_gridworld.py --learning_rate=0.02 --n_trajs=200 --n_iters=20

MIT License