versions/2022/tools/python/gradio_demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "521a8e94",
   "metadata": {},
   "source": [
    "### Gradio and HuggingFace\n",
    "\n",
    "In this demo, we show how to build ready to deploy or use deep learning models. \n",
    "\n",
    "Hugging Face hosts thousands of pre-trained models in [Model Hub](https://huggingface.co/models). They also built high-level APIs so we can easily use and deploy pre-trained models using [Pipeline](https://huggingface.co/docs/transformers/main_classes/pipelines). \n",
    "\n",
    "`gradio` provides APIs so we can easily build web applications that use our pre-trained models from Hugging Face. `gradio` also provides APIs so we can easily incorporate input and output web UIs.\n",
    "\n",
    "After building the `gradio`, we can have permanent hosting using [Hugging Face Spaces](https://huggingface.co/blog/gradio-spaces). \n",
    "\n",
    "Let us first install Hugging Face `transformers` and `gradio`.\n",
    "\n",
    "**Note:** For some examples, it is best to launch the app in another tab to enable access to the required inputs such as microphone or webcam. Running the app may also lock the `python` kernel and the notebook becomes unresponsive. In that case, please restart the kernel and clear the output. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "54da9029",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install transformers\n",
    "!pip install gradio"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a21369ff",
   "metadata": {},
   "source": [
    "#### Hello world in gradio\n",
    "\n",
    "As a tradition, let us build the simplest `gradio` app. It accepts a `text` input and calls the `greet()` function to process this input and convert into another text. The output of `greet()` becomes the output of the `gradio` app.\n",
    "\n",
    "To see our application, we call `launch()` after constructing our `gradio` `Interface`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "5c2c37a3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running on local URL:  http://127.0.0.1:7860\n",
      "\n",
      "To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7860/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import gradio as gr\n",
    "\n",
    "def greet(name):\n",
    "    return \"Hello \" + name + \"!!\"\n",
    "\n",
    "gr.Interface(fn=greet, inputs=\"text\", outputs=\"text\").launch()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74361507",
   "metadata": {},
   "source": [
    "#### Object Recognition using ResNet18\n",
    "\n",
    "In our discussion about PyTorch, we used a pre-trained ResNet18 model from `torchvision`. We use `jupyter` notebook to show the results. The `jupyter` notebook is not an application that we can deploy and other people use with ease. The same with Google's colab. \n",
    "\n",
    "In this example, we use `gradio` to build a simple app that an end user can easily interact with. We reuse the code from our previous [example](https://github.com/roatienza/Deep-Learning-Experiments/blob/master/versions/2022/tools/python/pytorch_demo.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "d73061fd",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/rowel/anaconda3/envs/llm/lib/python3.10/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.\n",
      "  warnings.warn(\n",
      "/Users/rowel/anaconda3/envs/llm/lib/python3.10/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.\n",
      "  warnings.warn(msg)\n",
      "/Users/rowel/anaconda3/envs/llm/lib/python3.10/site-packages/gradio/inputs.py:257: UserWarning: Usage of gradio.inputs is deprecated, and will not be supported in the future, please import your component from gradio.components\n",
      "  warnings.warn(\n",
      "/Users/rowel/anaconda3/envs/llm/lib/python3.10/site-packages/gradio/deprecation.py:40: UserWarning: `optional` parameter is deprecated, and it has no effect\n",
      "  warnings.warn(value)\n",
      "/Users/rowel/anaconda3/envs/llm/lib/python3.10/site-packages/gradio/outputs.py:197: UserWarning: Usage of gradio.outputs is deprecated, and will not be supported in the future, please import your components from gradio.components\n",
      "  warnings.warn(\n",
      "/Users/rowel/anaconda3/envs/llm/lib/python3.10/site-packages/gradio/deprecation.py:40: UserWarning: The 'type' parameter has been deprecated. Use the Number component instead.\n",
      "  warnings.warn(value)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running on local URL:  http://127.0.0.1:7868\n",
      "\n",
      "To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7868/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import gradio as gr\n",
    "import torch \n",
    "import torchvision\n",
    "import torchvision.transforms as transforms\n",
    "import requests\n",
    "from einops import rearrange\n",
    "\n",
    "normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],\n",
    "                                 std=[0.229, 0.224, 0.225])\n",
    "\n",
    "resnet = torchvision.models.resnet18(pretrained=True)\n",
    "resnet.eval()\n",
    "\n",
    "# Download human-readable labels for ImageNet.\n",
    "response = requests.get(\"https://git.io/JJkYN\")\n",
    "labels = response.text.split(\"\\n\")\n",
    "\n",
    "def classify(img):\n",
    "    # By default, gradio image is numpy\n",
    "    img = torch.from_numpy(img)\n",
    "    # Numpy image is channel last. PyTorch is channel 1st.\n",
    "    img = rearrange(img, 'h w c -> c h w')\n",
    "    \n",
    "    # The transforms before prediction\n",
    "    img = torchvision.transforms.Resize(256)(img)\n",
    "    img = torchvision.transforms.CenterCrop(224)(img).float()/255.\n",
    "    img = normalize(img)\n",
    "    \n",
    "    # We insert batch size of 1\n",
    "    img = rearrange(img, 'c h w  -> 1 c h w')\n",
    "    \n",
    "    # The actual prediction\n",
    "    with torch.no_grad():\n",
    "        pred = resnet(img)\n",
    "    \n",
    "    # Convert the prediction to probabilities\n",
    "    pred = torch.nn.functional.softmax(pred, dim=1)\n",
    "    # Remove the batch dim. torch.squeeze() can also be used.\n",
    "    pred = rearrange(pred, \"1 j->j\")\n",
    "    \n",
    "    # torch to numpy space\n",
    "    pred = pred.cpu().numpy()\n",
    "    \n",
    "    return {labels[i]: float(pred[i]) for i in range(1000)}\n",
    "    \n",
    "\n",
    "gr.Interface(fn=classify, \n",
    "             inputs=gr.inputs.Image(shape=(224, 224)),\n",
    "             outputs=gr.outputs.Label(num_top_classes=5),\n",
    "             title=\"1k Object Recognition\",\n",
    "             examples=['wonder_cat.jpg', 'aki_dog.jpg',],\n",
    "             description=\"Demonstrates a pre-trained model from torchvision for image classification.\",\n",
    "             allow_flagging=\"never\").launch(inbrowser=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea09cf26",
   "metadata": {},
   "source": [
    "#### Using HuggingFace and Gradio\n",
    "\n",
    "Loading a pre-trained model from torchvision, pre-processing the input, and post processing the output are all messy. Sometimes, we just want to load and use a machine learning model. Hugging Face provides a shortcut for all these steps through the use of `pipeline`. In `pipeline`, we supply the task name and the pre-trained model that is stored in Hugging Face Model Hub.\n",
    "\n",
    "In this example, we use a much better model compared to ResNet18. It is called [BEIT](https://arxiv.org/abs/2106.08254) and can classify objects up to about 22k categories. We construct the `gradio` app by calling `from_pipeline()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "510928fc",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/rowel/anaconda3/envs/llm/lib/python3.10/site-packages/transformers/models/beit/feature_extraction_beit.py:28: FutureWarning: The class BeitFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use BeitImageProcessor instead.\n",
      "  warnings.warn(\n",
      "/Users/rowel/anaconda3/envs/llm/lib/python3.10/site-packages/gradio/deprecation.py:40: UserWarning: The 'type' parameter has been deprecated. Use the Number component instead.\n",
      "  warnings.warn(value)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running on local URL:  http://127.0.0.1:7869\n",
      "\n",
      "To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7869/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import gradio as gr\n",
    "from transformers import pipeline\n",
    "\n",
    "pipe = pipeline(task=\"image-classification\", \n",
    "                 # model that can do 22k-category classification\n",
    "                 model=\"microsoft/beit-base-patch16-224-pt22k-ft22k\")\n",
    "gr.Interface.from_pipeline(pipe, \n",
    "                           title=\"22k Image Classification\",\n",
    "                           description=\"Object Recognition using Microsoft BEIT\",\n",
    "                           examples = ['wonder_cat.jpg', 'aki_dog.jpg',],\n",
    "                           allow_flagging=\"never\").launch(inbrowser=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb86c1b6",
   "metadata": {},
   "source": [
    "#### Automatic Speech Recognition (ASR)\n",
    "\n",
    "Let us shift to audio or speech domain. In this example, we demonstrate an Automatic Speech Recognition (ASR) system. We will use our microphone to record audio which is then converted to text using ASR. In this example, best to open the application in another browser tab by setting `inbrowser=True`.\n",
    "\n",
    "Before running the `gradio` app, this ASR requires `sentencepice` module. Let us install it first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "d73f6123",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: sentencepiece in /Users/rowel/anaconda3/envs/llm/lib/python3.10/site-packages (0.1.97)\r\n"
     ]
    }
   ],
   "source": [
    "!pip install sentencepiece"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6ad04d0",
   "metadata": {},
   "source": [
    "In this ASR, we use Facebook [S2T](https://arxiv.org/abs/2010.05171), a transformer-based speech to text model that is trained on librispeech dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "66d394c8",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running on local URL:  http://127.0.0.1:7870\n",
      "\n",
      "To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7870/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/rowel/anaconda3/envs/llm/lib/python3.10/site-packages/gradio/processing_utils.py:242: UserWarning: Trying to convert audio automatically from int32 to 16-bit int format.\n",
      "  warnings.warn(warning.format(data.dtype))\n",
      "/Users/rowel/anaconda3/envs/llm/lib/python3.10/site-packages/transformers/generation/utils.py:1273: UserWarning: Neither `max_length` nor `max_new_tokens` has been set, `max_length` will default to 200 (`generation_config.max_length`). Controlling `max_length` via the config is deprecated and `max_length` will be removed from the config in v5 of Transformers -- we recommend using `max_new_tokens` to control the maximum length of the generation.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "import gradio as gr\n",
    "from transformers import pipeline\n",
    "\n",
    "pipe = pipeline(task=\"automatic-speech-recognition\", \n",
    "                model=\"facebook/s2t-medium-librispeech-asr\")\n",
    "gr.Interface.from_pipeline(pipe,\n",
    "                           title=\"Automatic Speech Recognition (ASR)\",\n",
    "                           description=\"Using pipeline with Facebook S2T for ASR.\",\n",
    "                           examples=['data/ljspeech.wav',],\n",
    "                           ).launch(inbrowser=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6cfb3b5",
   "metadata": {},
   "source": [
    "#### Text to Speech (TTS)\n",
    "\n",
    "Let us do the reverse of ASR or Text to Speech (TTS). In this example, we supply text and this text is converted to speech using the voice of Linda Johnson. We use a pre-trained model of [FastSpeech2](https://arxiv.org/abs/2006.04558) that is provided by Facebook in Model Hub.\n",
    "\n",
    "In this example, we use [`load()`](https://gradio.app/docs/#load) method to load the pre-trained model. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "276b27ea",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fetching model from: https://huggingface.co/facebook/fastspeech2-en-ljspeech\n",
      "Running on local URL:  http://127.0.0.1:7865\n",
      "\n",
      "To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7865/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import gradio as gr\n",
    "gr.Interface.load(\"huggingface/facebook/fastspeech2-en-ljspeech\", \n",
    "                  description=\"TTS using FastSpeech2\",\n",
    "                  title=\"Text to Speech (TTS)\",\n",
    "                  examples=[[\"The quick brown fox jumps over the lazy dog.\"]]\n",
    "                  ).launch()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "744178e3",
   "metadata": {},
   "source": [
    "#### Text Generation using GPT2\n",
    "\n",
    "In this example, we use a large language model (LLM) by OpenAI to generate text. It is called [GPT2](https://github.com/openai/gpt-2). Text generation is one of the tasks where a language model can help us. Basically, we provide the initial text made of few words. Then, the model will continue it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "25086159",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fetching model from: https://huggingface.co/gpt2\n",
      "Running on local URL:  http://127.0.0.1:7866\n",
      "\n",
      "To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7866/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import gradio as gr\n",
    "gr.Interface.load(\"huggingface/gpt2\",\n",
    "                  title=\"Text Generation\",\n",
    "                  description=\"Using GPT2.\",\n",
    "                  allow_flagging=\"never\").launch(inbrowser=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9061beba",
   "metadata": {},
   "source": [
    "#### Using `gr.Series()` to automatically chain the I/O of multiple models\n",
    "\n",
    "In this example, we use the previous text generator as input to our TTS. We can for example use to generate a podcast on a certain topic. In this case, we replace GPT2 with a much better model called GPT Neo. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "303ac84b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fetching model from: https://huggingface.co/EleutherAI/gpt-neo-2.7B\n",
      "Fetching model from: https://huggingface.co/facebook/fastspeech2-en-ljspeech\n",
      "Running on local URL:  http://127.0.0.1:7867\n",
      "\n",
      "To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7867/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import gradio as gr\n",
    "\n",
    "\n",
    "textgen = gr.Interface.load(\"huggingface/EleutherAI/gpt-neo-2.7B\")\n",
    "\n",
    "tts = gr.Interface.load(\"huggingface/facebook/fastspeech2-en-ljspeech\")\n",
    "\n",
    "iface = gr.Series(textgen, tts)\n",
    "\n",
    "iface.title = \"Generated Text to Speech\"\n",
    "iface.allow_flagging = \"never\"\n",
    "iface.launch(inbrowser=True)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "afcd2461",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}