Skip to content

Deploy A/B testing infrastructure in a containerized microservice architecture for Machine Learning applications.

License

Notifications You must be signed in to change notification settings

data-max-hq/ab-testing-in-ml

Repository files navigation

A/B Testing for ML applications

Deploy A/B testing infrastructure in a containerized microservice architecture for Machine Learning applications.

Requirements

This repository uses Kubernetes, helm, ambassador, seldon-core, and seldon-core-analytics.

Getting started

Install prerequisites

  1. Install helm

    brew install helm
    
  2. Create a Kubernetes cluster: AKS, EKS, GKE or local cluster. For local clusters, one can use minikube, kind, or k3s. For instance:

    1. GKE
     gcloud container clusters create demo-cluster-ab-test \
       --zone=europe-west3-a \
       --disk-size=30GB \
       --cluster-version=1.24.12-gke.1000 \
       --machine-type=e2-highcpu-4

Train model

Prepare the model artifacts:

make train

Build container images

One can build the images locally, or use Cloud Submit:

Locally

docker build -t ab-test:a -f Dockerfile.a .
docker build -t ab-test:b -f Dockerfile.b .
docker build -t streamlit-app:v1.0.0 -f Dockerfile.streamlit .

Cloud Submit

sh build-modela.sh
sh build-modelb.sh
sh build-streamlit.sh

Deploy required components

  • emissary-ingress
make emissary
  • Prometheus
make prometheus
  • Grafana
make grafana
  • seldon-core
make seldon-core
  • podmonitor
make podmonitor

Deployment

make abtest

Start Streamlit App

make streamlit

Port-forward Grafana

make port-grafana

Contact

Sadik Bakiu (sadik [at] data-max.io)

Developed with ❤ at Data Max

References

About

Deploy A/B testing infrastructure in a containerized microservice architecture for Machine Learning applications.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published