colab/ggmlv3_q4_0.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyM9WbudQYrVFksXUrt4Opt3",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/liltom-eth/llama2-webui/blob/main/colab/ggmlv3_q4_0.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7O5JSosg5-rx"
      },
      "outputs": [],
      "source": [
        "%cd /content\n",
        "!pip install llama2-wrapper\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from llama2_wrapper import LLAMA2_WRAPPER, get_prompt\n",
        "\n",
        "llama2_wrapper = LLAMA2_WRAPPER()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8rgb1ckl72wC",
        "outputId": "d9ca2e20-26a5-490b-86f2-1a182e533b20"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Running on backend llama.cpp.\n",
            "Use default model path: ./models/llama-2-7b-chat.ggmlv3.q4_0.bin\n",
            "Start downloading model to: ./models/llama-2-7b-chat.ggmlv3.q4_0.bin\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "prompt = get_prompt(\"Hi do you know Pytorch?\")\n",
        "print(llama2_wrapper(prompt))"
      ],
      "metadata": {
        "id": "Qz2xAqozTIf6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1380fa52-3d4a-4ac5-ed02-7faefe7ec2f6"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  Yes, I'm familiar with PyTorch! PyTorch is an open-source deep learning framework that is widely used for building and training neural networks. It was originally developed by Facebook and is now maintained by the PyTorch Foundation.\n",
            "\n",
            "Here are some key features and capabilities of PyTorch:\n",
            "\n",
            "1. **Tensor Computation**: PyTorch provides a powerful tensor computation engine that allows for complex mathematical operations on large datasets.\n",
            "2. **Autograd**: PyTorch's autograd system automatically computes gradients, which can save a lot of time and effort during training.\n",
            "3. **Dynamic Compute**: PyTorch's dynamic compute system allows for more efficient computation by only computing the necessary computations at runtime.\n",
            "4. **Memory-efficient**: PyTorch is designed to be memory-efficient, which is important for training large models that require a lot of memory.\n",
            "5. **Accelerators**: PyTorch supports a wide range of accelerators, including GPUs, TPUs, and FPGAs, which can significantly speed up training times.\n",
            "6. **Modules**: PyTorch provides a wide range of pre-built modules for common tasks, such as convolutional layers, recurrent neural networks, and more.\n",
            "7. **Extensive Community**: PyTorch has a large and active community of developers and users, which can be helpful for getting support and staying up-to-date with the latest developments.\n",
            "8. **Easy Integration**: PyTorch can be easily integrated with other popular deep learning frameworks, such as TensorFlow and Keras.\n",
            "9. **Pythonic**: PyTorch is written in Python, which is a popular and easy-to-learn programming language.\n",
            "10. **Flexible**: PyTorch allows for a wide range of customization options, which can be useful for building and training unique models.\n",
            "\n",
            "Overall, PyTorch is a powerful and flexible deep learning framework that can be used for a wide range of applications, including computer vision, natural language processing, and more.\n"
          ]
        }
      ]
    }
  ]
}