00_Prerequisites/bedrock_basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "88a5ab2f-d044-4956-b75b-7408d9c3e323",
   "metadata": {},
   "source": [
    "# Amazon Bedrock boto3 Prerequisites\n",
    "\n",
    "> *This notebook should work well with the **`Data Science 3.0`** kernel in SageMaker Studio*\n",
    "\n",
    "---\n",
    "\n",
    "In this demo notebook, we demonstrate how to use the [`boto3` Python SDK](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) to work with [Amazon Bedrock](https://aws.amazon.com/bedrock/) Foundation Models.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6aeedd9f-f0a3-4f8e-934d-22f6f7a89de5",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "\n",
    "Run the cells in this section to install the packages needed by the notebooks in this workshop. ⚠️ You will see pip dependency errors, you can safely ignore these errors. ⚠️\n",
    "\n",
    "IGNORE ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "108c611c-7246-45c4-9f1e-76888b5076eb",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "%pip install --no-build-isolation --force-reinstall \\\n",
    "    \"boto3>=1.28.57\" \\\n",
    "    \"awscli>=1.29.57\" \\\n",
    "    \"botocore>=1.31.57\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27610c0f-7de6-4440-8f76-decf30e3c5ca",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Create the boto3 client\n",
    "\n",
    "Interaction with the Bedrock API is done via the AWS SDK for Python: [boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html).\n",
    "\n",
    "#### Use different clients\n",
    "The boto3 provides different clients for Amazon Bedrock to perform different actions. The actions for [`InvokeModel`](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) and [`InvokeModelWithResponseStream`](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html) are supported by Amazon Bedrock Runtime where as other operations, such as [ListFoundationModels](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_ListFoundationModels.html), are handled via [Amazon Bedrock client](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_Operations_Amazon_Bedrock.html).\n",
    "\n",
    "\n",
    "#### Use the default credential chain\n",
    "\n",
    "If you are running this notebook from [Amazon Sagemaker Studio](https://aws.amazon.com/sagemaker/studio/) and your Sagemaker Studio [execution role](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html) has permissions to access Bedrock you can just run the cells below as-is. This is also the case if you are running these notebooks from a computer whose default AWS credentials have access to Bedrock.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ae2b2a05-78a9-40ca-9b5e-121030f9ede1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import os\n",
    "import sys\n",
    "\n",
    "import boto3\n",
    "\n",
    "boto3_bedrock = boto3.client('bedrock')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e9174c4-326a-463e-92e1-8c7e47111269",
   "metadata": {},
   "source": [
    "#### Validate the connection\n",
    "\n",
    "We can check the client works by trying out the `list_foundation_models()` method, which will tell us all the models available for us to use "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f67b4466-12ff-4975-9811-7a19c6206604",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "[models['modelId'] for models in boto3_bedrock.list_foundation_models()['modelSummaries']]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f690043-df45-448f-8fa6-1ea8b06f1087",
   "metadata": {
    "tags": []
   },
   "source": [
    "---\n",
    "\n",
    "## `InvokeModel` body and output\n",
    "\n",
    "The `invoke_model()` method of the Amazon Bedrock runtime client (`InvokeModel` API) will be the primary method we use for most of our Text Generation and Processing tasks - whichever model we're using.\n",
    "\n",
    "Although the method is shared, the format of input and output varies depending on the foundation model used - as described below:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4650fa3-a831-4039-9fd6-749926d35979",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "### Amazon Titan Large and Premier\n",
    "\n",
    "#### Input\n",
    "```json\n",
    "{   \n",
    "    \"inputText\": \"<prompt>\",\n",
    "    \"textGenerationConfig\" : { \n",
    "        \"maxTokenCount\": 512,\n",
    "        \"stopSequences\": [],\n",
    "        \"temperature\": 0.1,  \n",
    "        \"topP\": 0.9\n",
    "    }\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"inputTextTokenCount\": 613,\n",
    "    \"results\": [{\n",
    "        \"tokenCount\": 219,\n",
    "        \"outputText\": \"<output>\"\n",
    "    }]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7adb6bee-7654-4269-9127-9afa4e823454",
   "metadata": {},
   "source": [
    "### Anthropic Claude\n",
    "\n",
    "#### Input\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"prompt\": \"\\n\\nHuman:<prompt>\\n\\nAnswer:\",\n",
    "    \"max_tokens_to_sample\": 300,\n",
    "    \"temperature\": 0.5,\n",
    "    \"top_k\": 250,\n",
    "    \"top_p\": 1,\n",
    "    \"stop_sequences\": [\"\\n\\nHuman:\"]\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"completion\": \"<output>\",\n",
    "    \"stop_reason\": \"stop_sequence\"\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb94aa2f-30da-499b-b3f5-02f102dbb1ea",
   "metadata": {},
   "source": [
    "### Stability AI Stable Diffusion XL\n",
    "\n",
    "#### Input\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"text_prompts\": [\n",
    "        {\"text\": \"this is where you place your input text\"}\n",
    "    ],\n",
    "    \"cfg_scale\": 10,\n",
    "    \"seed\": 0,\n",
    "    \"steps\": 50\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{ \n",
    "    \"result\": \"success\", \n",
    "    \"artifacts\": [\n",
    "        {\n",
    "            \"seed\": 123, \n",
    "            \"base64\": \"<image in base64>\",\n",
    "            \"finishReason\": \"SUCCESS\"\n",
    "        },\n",
    "        //...\n",
    "    ]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80f4adca-cfc4-439b-84b7-e528398684e3",
   "metadata": {
    "tags": []
   },
   "source": [
    "---\n",
    "\n",
    "## Common inference parameter definitions\n",
    "\n",
    "### Randomness and Diversity\n",
    "\n",
    "Foundation models generally support the following parameters to control randomness and diversity in the \n",
    "response.\n",
    "\n",
    "**Temperature** – Large language models use probability to construct the words in a sequence. For any \n",
    "given next word, there is a probability distribution of options for the next word in the sequence. When \n",
    "you set the temperature closer to zero, the model tends to select the higher-probability words. When \n",
    "you set the temperature further away from zero, the model may select a lower-probability word.\n",
    "\n",
    "In technical terms, the temperature modulates the probability density function for the next tokens, \n",
    "implementing the temperature sampling technique. This parameter can deepen or flatten the density \n",
    "function curve. A lower value results in a steeper curve with more deterministic responses, and a higher \n",
    "value results in a flatter curve with more random responses.\n",
    "\n",
    "**Top K** – Temperature defines the probability distribution of potential words, and Top K defines the cut \n",
    "off where the model no longer selects the words. For example, if K=50, the model selects from 50 of the \n",
    "most probable words that could be next in a given sequence. This reduces the probability that an unusual \n",
    "word gets selected next in a sequence.\n",
    "In technical terms, Top K is the number of the highest-probability vocabulary tokens to keep for Top-\n",
    "K-filtering - This limits the distribution of probable tokens, so the model chooses one of the highest-\n",
    "probability tokens.\n",
    "\n",
    "**Top P** – Top P defines a cut off based on the sum of probabilities of the potential choices. If you set Top \n",
    "P below 1.0, the model considers the most probable options and ignores less probable ones. Top P is \n",
    "similar to Top K, but instead of capping the number of choices, it caps choices based on the sum of their \n",
    "probabilities.\n",
    "For the example prompt \"I hear the hoof beats of ,\" you may want the model to provide \"horses,\" \n",
    "\"zebras\" or \"unicorns\" as the next word. If you set the temperature to its maximum, without capping \n",
    "Top K or Top P, you increase the probability of getting unusual results such as \"unicorns.\" If you set the \n",
    "temperature to 0, you increase the probability of \"horses.\" If you set a high temperature and set Top K or \n",
    "Top P to the maximum, you increase the probability of \"horses\" or \"zebras,\" and decrease the probability \n",
    "of \"unicorns.\"\n",
    "\n",
    "### Length\n",
    "\n",
    "The following parameters control the length of the generated response.\n",
    "\n",
    "**Response length** – Configures the minimum and maximum number of tokens to use in the generated \n",
    "response.\n",
    "\n",
    "**Length penalty** – Length penalty optimizes the model to be more concise in its output by penalizing \n",
    "longer responses. Length penalty differs from response length as the response length is a hard cut off for \n",
    "the minimum or maximum response length.\n",
    "\n",
    "In technical terms, the length penalty penalizes the model exponentially for lengthy responses. 0.0 \n",
    "means no penalty. Set a value less than 0.0 for the model to generate longer sequences, or set a value \n",
    "greater than 0.0 for the model to produce shorter sequences.\n",
    "\n",
    "### Repetitions\n",
    "\n",
    "The following parameters help control repetition in the generated response.\n",
    "\n",
    "**Repetition penalty (presence penalty)** – Prevents repetitions of the same words (tokens) in responses. \n",
    "1.0 means no penalty. Greater than 1.0 decreases repetition."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce22c308-ebbf-4ef5-a823-832b7c236e31",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Try out the models\n",
    "\n",
    "With some theory out of the way, let's see the models in action! Run the cells below to see basic, synchronous example invocations for each model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "6a0a79b9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import boto3\n",
    "import botocore\n",
    "import json \n",
    "\n",
    "bedrock_runtime = boto3.client('bedrock-runtime')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "893872fe-04fa-4f09-9736-6c6173ec1fc2",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Amazon Titan Text Premier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7df55eed-a3cf-426c-95ea-ec60dade6477",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# If you'd like to try your own prompt, edit this parameter!\n",
    "prompt_data = \"\"\"Command: Write me a blog about making strong business decisions as a leader.\n",
    "\n",
    "Blog:\n",
    "\"\"\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "174a8eb1-f9a4-4946-bfe2-550d21487f48",
   "metadata": {},
   "source": [
    "Next, we will construct the body with the `prompt_data` above, and add a optional parameters like `topP` and `temperature`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "dd2bb671-6b10-4948-9e5e-95d6ced3b86f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "As a leader, making strong business decisions is crucial for the success of your organization. Here are some tips to help you make informed and effective decisions:\n",
      "\n",
      "1. Gather all relevant information: Before making a decision, gather as much information as possible. This includes data, feedback from stakeholders, and expert opinions. Make sure you have a clear understanding of the problem you are trying to solve and the potential outcomes of your decision.\n",
      "\n",
      "2. Consider multiple perspectives: It's important to consider multiple perspectives when making a decision. This means taking into account the needs and concerns of all stakeholders, including employees, customers, and shareholders. By considering multiple perspectives, you can make a decision that is more likely to be accepted and supported by all parties involved.\n",
      "\n",
      "3. Set clear goals: Before making a decision, set clear goals for what you want to achieve. This will help you focus on the most important factors and avoid getting sidetracked by less important issues. Make sure your goals are specific, measurable, achievable, relevant, and time-bound (SMART).\n",
      "\n",
      "4. Weigh the pros and cons: Once you have gathered all the relevant information and considered multiple perspectives, it's time to weigh the pros and cons of each option. This means evaluating the potential benefits and risks of each decision and determining which option is most likely to achieve your goals.\n",
      "\n",
      "5. Seek feedback: It's important to seek feedback from others before making a decision. This can help you identify potential blind spots or biases and ensure that you are making the best possible decision. Seek feedback from trusted advisors, colleagues, or mentors who have experience in the area you are making a decision about.\n",
      "\n",
      "6. Make a decision: Once you have gathered all the relevant information, considered multiple perspectives, set clear goals, weighed the pros and cons, and sought feedback, it's time to make a decision. Be confident in your decision and communicate it clearly to all stakeholders.\n",
      "\n",
      "7. Implement the decision: Once you have made a decision, it's important to implement it effectively. This means developing a plan for how you will achieve your goals and ensuring that all stakeholders are on board with the plan. It's also important to monitor progress and make adjustments as needed to ensure that you are achieving your goals.\n",
      "\n",
      "8. Learn from your mistakes: Finally, it's important to learn from your mistakes and use them as an opportunity to improve your decision-making skills. This means reflecting on your decisions and identifying areas where you can improve. It also means seeking feedback from others and being open to new ideas and approaches.\n",
      "\n",
      "By following these tips, you can make strong business decisions that are informed, effective, and supported by all stakeholders. Remember, making decisions is an ongoing process, and it's important to continually improve your decision-making skills over time.\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "\n",
    "    body = json.dumps({\"inputText\": prompt_data, \"textGenerationConfig\" : {\"topP\":0.95, \"temperature\":0.2}})\n",
    "    modelId = \"amazon.titan-text-premier-v1:0\" # \n",
    "    accept = \"application/json\"\n",
    "    contentType = \"application/json\"\n",
    "\n",
    "    response = bedrock_runtime.invoke_model(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "    print(response_body.get(\"results\")[0].get(\"outputText\"))\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d7c0fe6-576a-4380-89aa-726bab5d65ff",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Anthropic Claude"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a725de2-bdea-4d86-b12d-d1d7cdda010b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# If you'd like to try your own prompt, edit this parameter!\n",
    "prompt_data = \"\"\"Human: Write me a blog about making strong business decisions as a leader.\n",
    "\n",
    "Assistant:\n",
    "\"\"\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0ba33ac0-fa16-4c4f-b882-e838d0cb5830",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "body = json.dumps({\"prompt\": prompt_data, \"max_tokens_to_sample\": 500})\n",
    "modelId = \"anthropic.claude-instant-v1\"  # change this to use a different version from the model provider\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "try:\n",
    "\n",
    "    response = bedrock_runtime.invoke_model(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "    print(response_body.get(\"completion\"))\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc498bea",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Stability Stable Diffusion XL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "173e51a2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "prompt_data = \"a landscape with trees\"\n",
    "body = json.dumps({\n",
    "    \"text_prompts\": [{\"text\": prompt_data}],\n",
    "    \"cfg_scale\": 10,\n",
    "    \"seed\": 20,\n",
    "    \"steps\": 50\n",
    "})\n",
    "modelId = \"stability.stable-diffusion-xl-v1\"\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "try:\n",
    "\n",
    "    response = bedrock_runtime.invoke_model(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "    print(response_body[\"result\"])\n",
    "    print(f'{response_body.get(\"artifacts\")[0].get(\"base64\")[0:80]}...')\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a271fa6-13fd-480a-87a5-3702d29a5c43",
   "metadata": {},
   "source": [
    "**Note:** The output is a [base64 encoded](https://docs.python.org/3/library/base64.html) string of the image data. You can use any image processing library (such as [Pillow](https://pillow.readthedocs.io/en/stable/)) to decode the image as in the example below:\n",
    "\n",
    "```python\n",
    "import base64\n",
    "import io\n",
    "from PIL import Image\n",
    "\n",
    "base_64_img_str = response_body.get(\"artifacts\")[0].get(\"base64\")\n",
    "image = Image.open(io.BytesIO(base64.decodebytes(bytes(base_64_img_str, \"utf-8\"))))\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "45072848-000a-4c22-8f08-2647e5c2230e",
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'response_body' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[2], line 5\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mio\u001b[39;00m\n\u001b[1;32m      3\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mPIL\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m Image\n\u001b[0;32m----> 5\u001b[0m base_64_img_str \u001b[38;5;241m=\u001b[39m \u001b[43mresponse_body\u001b[49m\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124martifacts\u001b[39m\u001b[38;5;124m\"\u001b[39m)[\u001b[38;5;241m0\u001b[39m]\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mbase64\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m      6\u001b[0m image \u001b[38;5;241m=\u001b[39m Image\u001b[38;5;241m.\u001b[39mopen(io\u001b[38;5;241m.\u001b[39mBytesIO(base64\u001b[38;5;241m.\u001b[39mdecodebytes(\u001b[38;5;28mbytes\u001b[39m(base_64_img_str, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mutf-8\u001b[39m\u001b[38;5;124m\"\u001b[39m))))\n\u001b[1;32m      7\u001b[0m image\n",
      "\u001b[0;31mNameError\u001b[0m: name 'response_body' is not defined"
     ]
    }
   ],
   "source": [
    "import base64\n",
    "import io\n",
    "from PIL import Image\n",
    "\n",
    "base_64_img_str = response_body.get(\"artifacts\")[0].get(\"base64\")\n",
    "image = Image.open(io.BytesIO(base64.decodebytes(bytes(base_64_img_str, \"utf-8\"))))\n",
    "image"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4621a301-53e4-4182-9fce-8ee422813e9d",
   "metadata": {},
   "source": [
    "## Generate streaming output\n",
    "\n",
    "For large language models, it can take noticeable time to generate long output sequences. Rather than waiting for the entire response to be available, latency-sensitive applications may like to **stream** the response to users.\n",
    "\n",
    "Run the code below to see how you can achieve this with Bedrock's `invoke_model_with_response_stream()` method - returning the response body in separate chunks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "c69627e3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "As a leader, making strong business decisions is crucial for the success of your organization. Here are some key principles to guide you in making informed decisions:\n",
      "\n",
      "Define your goals: Clearly define your organization's goals and objectives. This will help you prioritize your decisions and ensure that they align with your long-term vision.\n",
      "\n",
      "Research and gather information: Conduct thorough research and gather relevant information about the decision you need to make. This includes analyzing data, seeking input from stakeholders, and consulting experts.\n",
      "\n",
      "Consider multiple perspectives: Seek input and perspectives from your team, stakeholders, and experts. Different perspectives can provide valuable insights and help"
     ]
    }
   ],
   "source": [
    "from IPython.display import clear_output, display, display_markdown, Markdown\n",
    "prompt_data = \"\"\"Command: Write me a blog about making strong business decisions as a leader.\n",
    "\n",
    "Blog:\n",
    "\"\"\"\n",
    "\n",
    "body = json.dumps({\"inputText\": prompt_data})\n",
    "modelId = \"amazon.titan-tg1-large\"  # (Change this, and the request body, to try different models)\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "try:\n",
    "\n",
    "    response = bedrock_runtime.invoke_model_with_response_stream(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    stream = response.get('body')\n",
    "    output = []\n",
    "\n",
    "    if stream:\n",
    "        for event in stream:\n",
    "            chunk = event.get('chunk')\n",
    "            if chunk:\n",
    "                chunk_obj = json.loads(chunk.get('bytes').decode())\n",
    "                text = chunk_obj['outputText']\n",
    "                display_markdown(Markdown(print(text, end='')))\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adf097eb-1bfa-41f1-8135-cdd4ad6e9983",
   "metadata": {},
   "source": [
    "### Anthropic Claude (messages API)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bd800ef5-78f7-43d4-b73e-c37c7609cb40",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# If you'd like to try your own prompt, edit this parameter!\n",
    "prompt_data = \"\"\"Human: Write me 500 word paragraph about making strong business decisions as a leader.\n",
    "\n",
    "Assistant:\n",
    "\"\"\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ef24e0c-dda8-4f5d-bbcb-db389e67e713",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "messages_API_body = {\n",
    "    \"anthropic_version\": \"bedrock-2023-05-31\", \n",
    "    \"max_tokens\": int(500/0.75),\n",
    "    \"messages\": [\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": [\n",
    "                {\n",
    "                    \"type\": \"text\",\n",
    "                    \"text\": prompt_data\n",
    "                }\n",
    "            ]\n",
    "        }\n",
    "    ]\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "be7fbbe8-78b8-4071-b80c-3f954185fad8",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from IPython.display import clear_output, display, display_markdown, Markdown\n",
    "\n",
    "body = json.dumps(messages_API_body)\n",
    "modelId = \"anthropic.claude-3-haiku-20240307-v1:0\"  # (Change this to try different model versions)\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "try:\n",
    "\n",
    "    response = bedrock_runtime.invoke_model_with_response_stream(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    \n",
    "    stream = response.get('body')\n",
    "    \n",
    "    \n",
    "    output = []\n",
    "\n",
    "    if stream:\n",
    "        for event in stream:\n",
    "            chunk = event.get('chunk')\n",
    "            if chunk:\n",
    "                chunk_obj = json.loads(chunk.get('bytes').decode())\n",
    "                if 'delta' in chunk_obj:\n",
    "                    delta_obj = chunk_obj.get('delta', None)\n",
    "                    if delta_obj:\n",
    "                        text = delta_obj.get('text', None)\n",
    "                        print(text,end='')\n",
    "                        if not text :\n",
    "                            break\n",
    "                    # output.append(text[0]) if type(text) is list and len(text)>0 else output.append('')\n",
    "                    # display_markdown(Markdown(text))\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ef3451d-b66a-4b11-a1ed-734bf9e7bbec",
   "metadata": {},
   "source": [
    "## Generate embeddings\n",
    "\n",
    "Use text embeddings to convert text into meaningful vector representations. You input a body of text \n",
    "and the output is a (1 x n) vector. You can use embedding vectors for a wide variety of applications. \n",
    "Bedrock currently offers Titan Embeddings for text embedding that supports text similarity (finding the \n",
    "semantic similarity between bodies of text) and text retrieval (such as search).\n",
    "\n",
    "At the time of writing you can use `amazon.titan-embed-text-v1` as embedding model via the API. The input text size is 8192 tokens and the output vector length is 1536.\n",
    "\n",
    "To use a text embeddings model, use the InvokeModel API operation or the Python SDK.\n",
    "Use InvokeModel to retrieve the vector representation of the input text from the specified model.\n",
    "\n",
    "\n",
    "\n",
    "#### Input\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"inputText\": \"<text>\"\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"embedding\": []\n",
    "}\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9645dbd8",
   "metadata": {},
   "source": [
    "Let's see how to generate embeddings of some text:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1085cc56",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "prompt_data = \"Amazon Bedrock supports foundation models from industry-leading providers such as \\\n",
    "AI21 Labs, Anthropic, Stability AI, and Amazon. Choose the model that is best suited to achieving \\\n",
    "your unique goals.\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5c54b424",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "body = json.dumps({\"inputText\": prompt_data})\n",
    "modelId = \"amazon.titan-embed-g1-text-02\"  # (Change this to try different embedding models)\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "try:\n",
    "\n",
    "    response = bedrock_runtime.invoke_model(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "    embedding = response_body.get(\"embedding\")\n",
    "    print(f\"The embedding vector has {len(embedding)} values\\n{embedding[0:3]+['...']+embedding[-3:]}\")\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a48a0e8-147d-4525-a6b2-68a09af1b2c4",
   "metadata": {},
   "source": [
    "## Next steps\n",
    "\n",
    "In this notebook we showed some basic examples of invoking Amazon Bedrock models using the AWS Python SDK. You're now ready to explore the other labs to dive deeper on different use-cases and patterns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f8bb76df-4e99-4ebe-a954-53992ad317dc",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 57,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.trn1.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 58,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1.32xlarge",
    "vcpuNum": 128
   },
   {
    "_defaultOrder": 59,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1n.32xlarge",
    "vcpuNum": 128
   }
  ],
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science 3.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/sagemaker-data-science-310-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}