examples/web-retrieval.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Internet Retrieval with Claude\n",
    "\n",
    "This notebook provides a step-by-step guide for using the Brave search tool with Claude. We will:\n",
    "\n",
    "1. Set up the environment and imports\n",
    "2. Build a search tool to query the Brave search engine\n",
    "3. Test the search tool  \n",
    "4. Create a Claude client with access to the tool \n",
    "5. Compare Claude's responses with and without access to the tool\n",
    "\n",
    "## Imports and Configuration \n",
    "\n",
    "First we'll import libraries and load environment variables. This includes setting up logging so we can monitor the process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import sys\n",
    "import dotenv\n",
    "import anthropic\n",
    "import nest_asyncio\n",
    "nest_asyncio.apply()  # This is a workaround for asyncio + jupyter notebooks\n",
    "\n",
    "sys.path.append(os.path.abspath(os.path.join(os.getcwd(), os.pardir)))\n",
    "\n",
    "import claude_retriever\n",
    "\n",
    "# Load environment variables\n",
    "dotenv.load_dotenv()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [],
   "source": [
    "# Import and configure logging \n",
    "import logging\n",
    "logger = logging.getLogger()\n",
    "logger.setLevel(logging.INFO)\n",
    "\n",
    "# Create a handler to log to stdout\n",
    "handler = logging.StreamHandler(sys.stdout)\n",
    "handler.setFormatter(logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s'))\n",
    "logger.addHandler(handler)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Make a Brave Search Tool"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [],
   "source": [
    "After importing the BraveSearchTool class, we can easily initalize a new BraveSearchTool object.\n",
    "\n",
    "Let's test the search tool to make sure it works:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [],
   "source": [
    "from claude_retriever.searcher.searchtools.websearch import BraveSearchTool\n",
    "\n",
    "# Create a searcher\n",
    "brave_search_tool = BraveSearchTool(brave_api_key=os.environ[\"BRAVE_API_KEY\"], summarize_with_claude=True, anthropic_api_key=os.environ[\"ANTHROPIC_API_KEY\"])\n",
    "\n",
    "# Run a test query\n",
    "query = \"2023 World Cup\"\n",
    "\n",
    "results = brave_search_tool.search(query=query, n_search_results_to_use=3)\n",
    "print(results)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use Claude with Retrieval"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we simply pass the search tool to a Claude client, allowing Claude access to use it when generating responses. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [],
   "source": [
    "ANTHROPIC_SEARCH_MODEL = \"claude-2\"\n",
    "\n",
    "client = claude_retriever.ClientWithRetrieval(api_key=os.environ['ANTHROPIC_API_KEY'], search_tool = brave_search_tool)\n",
    "\n",
    "query = \"Who scored the most goals in the 2023 Women's World Cup?\"\n",
    "prompt = f'{anthropic.HUMAN_PROMPT} {query}{anthropic.AI_PROMPT}'"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is the basic response to the query (no access to the tool)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [],
   "source": [
    "basic_response = client.completions.create(\n",
    "    prompt=prompt,\n",
    "    stop_sequences=[anthropic.HUMAN_PROMPT],\n",
    "    model=ANTHROPIC_SEARCH_MODEL,\n",
    "    max_tokens_to_sample=1000,\n",
    ")\n",
    "print('-'*50)\n",
    "print('Basic response:')\n",
    "print(prompt + basic_response.completion)\n",
    "print('-'*50)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's try getting a completion from Claude equipped with the Brave search tool."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [],
   "source": [
    "augmented_response = client.completion_with_retrieval(\n",
    "    query=query,\n",
    "    model=ANTHROPIC_SEARCH_MODEL,\n",
    "    n_search_results_to_use=1,\n",
    "    max_searches_to_try=3,\n",
    "    max_tokens_to_sample=1000)\n",
    "\n",
    "print('-'*50)\n",
    "print('Augmented response:')\n",
    "print(prompt + augmented_response)\n",
    "print('-'*50)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Coconut",
   "language": "coconut",
   "name": "coconut"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "python",
    "version": 3
   },
   "file_extension": ".coco",
   "mimetype": "text/x-python3",
   "name": "coconut",
   "pygments_lexer": "coconut",
   "version": "2.2.0"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}