Inter-services communicaton using AQMP Brokers

Exploring inter-services communication using AQMP brokers like RabbitMQ among python and java based microservices.

Setup

cd into each of the front-end, backend services directory and initiate them:

vagrant up --provider virtualbox

java-api-backend$ gradle run
java-api-backend$ cd ../knockout-frontend

knockout-frontend$ python -m SimpleHTTPServer 8090
knockout-frontend$ cd ../python-scraping-service

python-scraping-service$ pip install -r requirements.txt
python-scraping-service$ python worker.py

Prerequisites

Note: You must have wget and apt-get installed on Mac OS X for the vagrant scripts to execute successfully.

To install wget, simply run: brew install wget

Vagrant Setup

I've used virtualbox with ubuntu/trusty64 environment for running this. If you don't have it setup, the easiest to do this in Mac OS X is using Homebrew Cask:

Install

	$ brew cask install virtualbox
	$ brew cask install vagrant

Vagrant Manager helps you manage all your VMs in one place from menubar

	$ brew cask install vagrant-manager

Every Vagrant development environment requires a box. You can search for boxes at link

The virtual box used for this demo is ubuntu/trusty64 link

	$ vagrant init ubuntu/trusty64; 

You should get a CL message: A Vagrantfile has been placed in this directory. You are now ready to vagrant up your first virtual environment! Please read the comments in the Vagrantfile as well as documentation on vagrantup.com for more information on using Vagrant.

	$ vagrant up --provider virtualbox

To get rid of the vagrant VM, just run vagrant destroy

Also, you may find it easier to manage the virtual boxes using vagrant-manager, which can be installed using brew install vagrant-manager

Note: to make things easier, in application.properties the line spring.jpa.hibernate.ddl-auto=create allows us to automatically create the tables when starting up. However, this means all tables get erased with each new start. Change the line to spring.jpa.hibernate.ddl-auto=validate (after you have started up the backend at least once) to avoid data loss.

Python Modules and Requirements

Pika link: Pika is a pure-Python implementation of the AMQP 0-9-1 protocol that tries to stay fairly independent of the underlying network support library.

Python Scraper

For the actual scraping of a website’s summary, we use https://github.com/raeidsaqur/sumy library. Please see their docs (especially as it requires to have nltk installed on your system).

The scraper is not doing much except using the sumy library and returning the result.

To run the python scraper, install requirements.txt (preferably in a virtualenv) and then simply run worker.py:

(venv)python-scraping-service$ python worker.py 

Spring-boot Java Requirements

Spring has a boot starter module available for RabbitMQ link org.springframework.boot:spring-boot-starter-amqp, which we add to our dependencies.

Knockout JS front-end Requirements

Reference:

1. https://pika.readthedocs.io/en/0.10.0/index.html
2. https://atlas.hashicorp.com/boxes/search
3. https://atlas.hashicorp.com/ubuntu/boxes/trusty64
4. Spring Boot RabbitMQ