docs/gen-qa-openai.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SMFUgtvbFeLQ"
      },
      "source": [
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pinecone-io/examples/blob/master/docs/gen-qa-openai.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/pinecone-io/examples/blob/master/docs/gen-qa-openai.ipynb)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v0to-QXCQjsm"
      },
      "source": [
        "# Retrieval Enhanced Generative Question Answering with OpenAI\n",
        "\n",
        "#### Fixing LLMs that Hallucinate\n",
        "\n",
        "In this notebook we will learn how to query relevant contexts to our queries from Pinecone, and pass these to a generative OpenAI model to generate an answer backed by real data sources. Required installs for this notebook are:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VpMvHAYRQf9N",
        "outputId": "42705370-c5bf-417b-fe8a-3f6d0fe28fbb"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m72.0/72.0 kB\u001b[0m \u001b[31m1.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m199.9/199.9 kB\u001b[0m \u001b[31m4.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m12.3/12.3 MB\u001b[0m \u001b[31m37.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m34.9/34.9 MB\u001b[0m \u001b[31m22.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m346.6/346.6 kB\u001b[0m \u001b[31m9.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m76.0/76.0 kB\u001b[0m \u001b[31m3.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m73.5/73.5 kB\u001b[0m \u001b[31m3.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m73.4/73.4 kB\u001b[0m \u001b[31m3.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m11.1/11.1 MB\u001b[0m \u001b[31m28.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m143.8/143.8 kB\u001b[0m \u001b[31m7.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "lida 0.0.10 requires fastapi, which is not installed.\n",
            "lida 0.0.10 requires kaleido, which is not installed.\n",
            "lida 0.0.10 requires python-multipart, which is not installed.\n",
            "lida 0.0.10 requires uvicorn, which is not installed.\n",
            "llmx 0.0.15a0 requires cohere, which is not installed.\n",
            "llmx 0.0.15a0 requires tiktoken, which is not installed.\n",
            "bigframes 0.19.0 requires pandas<2.1.4,>=1.5.0, but you have pandas 2.1.4 which is incompatible.\n",
            "google-colab 1.0.0 requires pandas==1.5.3, but you have pandas 2.1.4 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0m"
          ]
        }
      ],
      "source": [
        "!pip install -qU \\\n",
        "    openai==0.27.7 \\\n",
        "    pinecone-client==3.1.0 \\\n",
        "    pinecone-datasets==0.7.0 \\\n",
        "    tqdm \\\n",
        "    pinecone-notebooks==0.1.1"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NhWnLkHqmeWI"
      },
      "source": [
        "---\n",
        "\n",
        "## Building a Knowledge Base\n",
        "\n",
        "Building more reliable LLMs tools requires an external _\"Knowledge Base\"_, a place where we can store and use to efficiently retrieve information. We can think of this as the external _long-term memory_ of our LLM.\n",
        "\n",
        "We will need to retrieve information that is semantically related to our queries, to do this we need to use _\"dense vector embeddings\"_. These can be thought of as numerical representations of the *meaning* behind our sentences.\n",
        "\n",
        "There are many options for creating these dense vectors, like open source [sentence transformers](https://pinecone.io/learn/nlp/) or OpenAI's [ada-002 model](https://youtu.be/ocxq84ocYi0). We will use OpenAI's offering in this example.\n",
        "\n",
        "We have already precomputed the embeddings here to speed things up. If you'd like to work through the full process however, check out [this notebook](https://github.com/pinecone-io/examples/blob/master/learn/generation/openai/gen-qa-openai.ipynb).\n",
        "\n",
        "To download our precomputed embeddings we use Pinecone datasets:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "EI2iYxq16or9",
        "outputId": "55a8dd92-34e1-4237-a5c4-76016346999d"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-79bd9601-b35b-43df-b5db-1fe6e0f9ef91\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "      <th>values</th>\n",
              "      <th>sparse_values</th>\n",
              "      <th>metadata</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>35Pdoyi6ZoQ-t0.0</td>\n",
              "      <td>[-0.010402066633105278, -0.018359748646616936,...</td>\n",
              "      <td>None</td>\n",
              "      <td>{'channel_id': 'UCv83tO5cePwHMt1952IVVHw', 'en...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>35Pdoyi6ZoQ-t18.48</td>\n",
              "      <td>[-0.011849376372992992, 0.0007984379190020263,...</td>\n",
              "      <td>None</td>\n",
              "      <td>{'channel_id': 'UCv83tO5cePwHMt1952IVVHw', 'en...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>35Pdoyi6ZoQ-t32.36</td>\n",
              "      <td>[-0.014534404501318932, -0.0003158661129418760...</td>\n",
              "      <td>None</td>\n",
              "      <td>{'channel_id': 'UCv83tO5cePwHMt1952IVVHw', 'en...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>35Pdoyi6ZoQ-t51.519999999999996</td>\n",
              "      <td>[-0.011597747914493084, -0.007550035137683153,...</td>\n",
              "      <td>None</td>\n",
              "      <td>{'channel_id': 'UCv83tO5cePwHMt1952IVVHw', 'en...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>35Pdoyi6ZoQ-t67.28</td>\n",
              "      <td>[-0.015879768878221512, 0.0030445053707808256,...</td>\n",
              "      <td>None</td>\n",
              "      <td>{'channel_id': 'UCv83tO5cePwHMt1952IVVHw', 'en...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-79bd9601-b35b-43df-b5db-1fe6e0f9ef91')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-79bd9601-b35b-43df-b5db-1fe6e0f9ef91 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-79bd9601-b35b-43df-b5db-1fe6e0f9ef91');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-eaa0e335-bae7-4637-b5cf-06cb7a175ef2\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-eaa0e335-bae7-4637-b5cf-06cb7a175ef2')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-eaa0e335-bae7-4637-b5cf-06cb7a175ef2 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "                                id  \\\n",
              "0                 35Pdoyi6ZoQ-t0.0   \n",
              "1               35Pdoyi6ZoQ-t18.48   \n",
              "2               35Pdoyi6ZoQ-t32.36   \n",
              "3  35Pdoyi6ZoQ-t51.519999999999996   \n",
              "4               35Pdoyi6ZoQ-t67.28   \n",
              "\n",
              "                                              values sparse_values  \\\n",
              "0  [-0.010402066633105278, -0.018359748646616936,...          None   \n",
              "1  [-0.011849376372992992, 0.0007984379190020263,...          None   \n",
              "2  [-0.014534404501318932, -0.0003158661129418760...          None   \n",
              "3  [-0.011597747914493084, -0.007550035137683153,...          None   \n",
              "4  [-0.015879768878221512, 0.0030445053707808256,...          None   \n",
              "\n",
              "                                            metadata  \n",
              "0  {'channel_id': 'UCv83tO5cePwHMt1952IVVHw', 'en...  \n",
              "1  {'channel_id': 'UCv83tO5cePwHMt1952IVVHw', 'en...  \n",
              "2  {'channel_id': 'UCv83tO5cePwHMt1952IVVHw', 'en...  \n",
              "3  {'channel_id': 'UCv83tO5cePwHMt1952IVVHw', 'en...  \n",
              "4  {'channel_id': 'UCv83tO5cePwHMt1952IVVHw', 'en...  "
            ]
          },
          "execution_count": 2,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from pinecone_datasets import load_dataset\n",
        "\n",
        "dataset = load_dataset('youtube-transcripts-text-embedding-ada-002')\n",
        "# we drop sparse_values as they are not needed for this example\n",
        "dataset.documents.drop(['metadata'], axis=1, inplace=True)\n",
        "dataset.documents.rename(columns={'blob': 'metadata'}, inplace=True)\n",
        "dataset.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VMyJjt1cnwcH"
      },
      "source": [
        "Now we need a place to store these embeddings and enable a efficient _vector search_ through them all. To do that we use Pinecone."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Lfd9lu6JbwoF"
      },
      "source": [
        "## Creating an Index\n",
        "\n",
        "Now the data is ready, we can set up our index to store it.\n",
        "\n",
        "We begin by initializing our connection to Pinecone. To do this we need a [free API key](https://app.pinecone.io)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "if not os.environ.get(\"PINECONE_API_KEY\"):\n",
        "    from pinecone_notebooks.colab import Authenticate\n",
        "    Authenticate()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "KPXw1sEubwoF"
      },
      "outputs": [],
      "source": [
        "from pinecone import Pinecone\n",
        "\n",
        "api_key = os.environ.get(\"PINECONE_API_KEY\")\n",
        "\n",
        "# configure client\n",
        "pc = Pinecone(api_key=api_key)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eoxXhUtUbwoG"
      },
      "source": [
        "Now we setup our index specification, this allows us to define the cloud provider and region where we want to deploy our index. You can find a list of all [available providers and regions here](https://docs.pinecone.io/docs/projects)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "AWFjG54mbwoG"
      },
      "outputs": [],
      "source": [
        "from pinecone import ServerlessSpec\n",
        "\n",
        "cloud = os.environ.get('PINECONE_CLOUD') or 'aws'\n",
        "region = os.environ.get('PINECONE_REGION') or 'us-east-1'\n",
        "\n",
        "spec = ServerlessSpec(cloud=cloud, region=region)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "L4C_TQWcFeLY",
        "tags": [
          "parameters"
        ]
      },
      "outputs": [],
      "source": [
        "index_name = 'gen-qa-openai-fast'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UPNwQTH0RNcl",
        "outputId": "954a84ba-cc9c-4796-fd12-28920024470c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'dimension': 1536,\n",
              " 'index_fullness': 0.0,\n",
              " 'namespaces': {},\n",
              " 'total_vector_count': 0}"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# check if index already exists (it shouldn't if this is first time)\n",
        "if index_name not in pc.list_indexes().names():\n",
        "    # if does not exist, create index\n",
        "    pc.create_index(\n",
        "        index_name,\n",
        "        dimension=1536,  # dimensionality of text-embedding-ada-002\n",
        "        metric='cosine',\n",
        "        spec=spec\n",
        "    )\n",
        "# connect to index\n",
        "index = pc.Index(index_name)\n",
        "# view index stats\n",
        "index.describe_index_stats()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nELBmqxxzeqL"
      },
      "source": [
        "We can see the index is currently empty with a `total_vector_count` of `0`. We can begin populating it with OpenAI `text-embedding-ada-002` built embeddings like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "vPb9liovzrc8"
      },
      "outputs": [],
      "source": [
        "for batch in dataset.iter_documents(batch_size=100):\n",
        "    index.upsert(batch)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2yiF91IbyGYo"
      },
      "source": [
        "Now we've added all of our langchain docs to the index. With that we can move on to retrieval and then answer generation."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VfP0TQVeG1hO"
      },
      "source": [
        "## Retrieval"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kXXYoANbG13a"
      },
      "source": [
        "To search through our documents we first need to create a query vector `xq`. Using `xq` we will retrieve the most relevant chunks from the LangChain docs. To create that query vector we must initialize a `text-embedding-ada-002` embedding model with OpenAI. For this, you need an [OpenAI API key](https://platform.openai.com/)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "2O0kRYO8G7Qi"
      },
      "outputs": [],
      "source": [
        "import openai\n",
        "\n",
        "# get api key from platform.openai.com\n",
        "openai.api_key = os.getenv('OPENAI_API_KEY') or 'sk-...'\n",
        "\n",
        "embed_model = \"text-embedding-ada-002\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "LF1U_yZGojRJ"
      },
      "outputs": [],
      "source": [
        "query = (\n",
        "    \"Which training method should I use for sentence transformers when \" +\n",
        "    \"I only have pairs of related sentences?\"\n",
        ")\n",
        "\n",
        "res = openai.Embedding.create(\n",
        "    input=[query],\n",
        "    engine=embed_model\n",
        ")\n",
        "\n",
        "# retrieve from Pinecone\n",
        "xq = res['data'][0]['embedding']\n",
        "\n",
        "# get relevant contexts (including the questions)\n",
        "res = index.query(vector=xq, top_k=2, include_metadata=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GH_DkmsNomww",
        "outputId": "8cf6f57e-7d20-4d01-eb33-e2c126280c63"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'matches': [{'id': 'pNvujJ1XyeQ-t418.88',\n",
              "              'metadata': {'channel_id': 'UCv83tO5cePwHMt1952IVVHw',\n",
              "                           'end': 568.0,\n",
              "                           'published': '2021-11-24 16:24:24 UTC',\n",
              "                           'start': 418.0,\n",
              "                           'text': 'pairs of related sentences you can go '\n",
              "                                   'ahead and actually try training or '\n",
              "                                   'fine-tuning using NLI with multiple '\n",
              "                                   \"negative ranking loss. If you don't have \"\n",
              "                                   'that fine. Another option is that you have '\n",
              "                                   'a semantic textual similarity data set or '\n",
              "                                   'STS and what this is is you have so you '\n",
              "                                   'have sentence A here, sentence B here and '\n",
              "                                   'then you have a score from from 0 to 1 '\n",
              "                                   'that tells you the similarity between '\n",
              "                                   'those two scores and you would train this '\n",
              "                                   'using something like cosine similarity '\n",
              "                                   \"loss. Now if that's not an option and your \"\n",
              "                                   'focus or use case is on building a '\n",
              "                                   'sentence transformer for another language '\n",
              "                                   'where there is no current sentence '\n",
              "                                   'transformer you can use multilingual '\n",
              "                                   'parallel data. So what I mean by that is '\n",
              "                                   'so parallel data just means translation '\n",
              "                                   'pairs so if you have for example a English '\n",
              "                                   'sentence and then you have another '\n",
              "                                   'language here so it can it can be anything '\n",
              "                                   \"I'm just going to put XX and that XX is \"\n",
              "                                   'your target language you can fine-tune a '\n",
              "                                   'model using something called multilingual '\n",
              "                                   'knowledge distillation and what that does '\n",
              "                                   'is takes a monolingual model for example '\n",
              "                                   'in English and using those translation '\n",
              "                                   'pairs it distills the knowledge the '\n",
              "                                   'semantic similarity knowledge from that '\n",
              "                                   'monolingual English model into a '\n",
              "                                   'multilingual model which can handle both '\n",
              "                                   'English and your target language. So '\n",
              "                                   \"they're three options quite popular very \"\n",
              "                                   'common that you can go for and as a '\n",
              "                                   'supervised methods the chances are that '\n",
              "                                   'probably going to outperform anything you '\n",
              "                                   'do with unsupervised training at least for '\n",
              "                                   'now. So if none of those sound like '\n",
              "                                   'something',\n",
              "                           'title': 'Today Unsupervised Sentence Transformers, '\n",
              "                                    'Tomorrow Skynet (how TSDAE works)',\n",
              "                           'url': 'https://youtu.be/pNvujJ1XyeQ'},\n",
              "              'score': 0.865291,\n",
              "              'values': []},\n",
              "             {'id': 'WS1uVMGhlWQ-t747.92',\n",
              "              'metadata': {'channel_id': 'UCv83tO5cePwHMt1952IVVHw',\n",
              "                           'end': 906.0,\n",
              "                           'published': '2021-10-20 17:06:20 UTC',\n",
              "                           'start': 747.0,\n",
              "                           'text': \"pooling approach. Or we can't use it in \"\n",
              "                                   'its current form. Now the solution to this '\n",
              "                                   'problem was introduced by two people in '\n",
              "                                   '2019 Nils Reimers and Irenia Gurevich. '\n",
              "                                   'They introduced what is the first sentence '\n",
              "                                   'transformer or sentence BERT. And it was '\n",
              "                                   'found that sentence BERT or S BERT '\n",
              "                                   'outformed all of the previous Save the Art '\n",
              "                                   'models on pretty much all benchmarks. Not '\n",
              "                                   'all of them but most of them. And it did '\n",
              "                                   'it in a very quick time. So if we compare '\n",
              "                                   'it to BERT, if we wanted to find the most '\n",
              "                                   'similar sentence pair from 10,000 '\n",
              "                                   'sentences in that 2019 paper they found '\n",
              "                                   'that with BERT that took 65 hours. With S '\n",
              "                                   'BERT embeddings they could create all the '\n",
              "                                   'embeddings in just around five seconds. '\n",
              "                                   'And then they could compare all those with '\n",
              "                                   \"cosine similarity in 0.01 seconds. So it's \"\n",
              "                                   'a lot faster. We go from 65 hours to just '\n",
              "                                   'over five seconds which is I think pretty '\n",
              "                                   \"incredible. Now I think that's pretty much \"\n",
              "                                   'all the context we need behind sentence '\n",
              "                                   'transformers. And what we do now is dive '\n",
              "                                   'into a little bit of how they actually '\n",
              "                                   'work. Now we said before we have the core '\n",
              "                                   'transform models and what S BERT does is '\n",
              "                                   'fine tunes on sentence pairs using what is '\n",
              "                                   'called a Siamese architecture or Siamese '\n",
              "                                   'network. What we mean by a Siamese network '\n",
              "                                   'is that we have what we can see, what can '\n",
              "                                   'view as two BERT models that are identical '\n",
              "                                   'and the weights between those two models '\n",
              "                                   'are tied. Now in reality when implementing '\n",
              "                                   'this we just use a single BERT model. And '\n",
              "                                   'what we do is we process one sentence, a '\n",
              "                                   'sentence A through the model and then we '\n",
              "                                   'process another sentence, sentence B '\n",
              "                                   \"through the model. And that's the sentence \"\n",
              "                                   'pair. So with our cross-linked we were '\n",
              "                                   'processing the sentence pair together. We '\n",
              "                                   'were putting them both together, '\n",
              "                                   'processing them all at once. This time we '\n",
              "                                   'process them separately. And during '\n",
              "                                   'training what happens is the weights '\n",
              "                                   'within BERT are optimized to reduce the '\n",
              "                                   'difference between two vector embeddings '\n",
              "                                   'or two sentence',\n",
              "                           'title': 'Intro to Sentence Embeddings with '\n",
              "                                    'Transformers',\n",
              "                           'url': 'https://youtu.be/WS1uVMGhlWQ'},\n",
              "              'score': 0.86448735,\n",
              "              'values': []}],\n",
              " 'namespace': '',\n",
              " 'usage': {'read_units': 6}}"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "res"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A5dDWPGoIrd9"
      },
      "source": [
        "We write some functions to handle the retrieval and completion steps:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "92NmGGJ1TKQp"
      },
      "outputs": [],
      "source": [
        "limit = 3750\n",
        "\n",
        "import time\n",
        "\n",
        "def retrieve(query):\n",
        "    res = openai.Embedding.create(\n",
        "        input=[query],\n",
        "        engine=embed_model\n",
        "    )\n",
        "\n",
        "    # retrieve from Pinecone\n",
        "    xq = res['data'][0]['embedding']\n",
        "\n",
        "    # get relevant contexts\n",
        "    contexts = []\n",
        "    time_waited = 0\n",
        "    while (len(contexts) < 3 and time_waited < 60 * 12):\n",
        "        res = index.query(vector=xq, top_k=3, include_metadata=True)\n",
        "        contexts = contexts + [\n",
        "            x['metadata']['text'] for x in res['matches']\n",
        "        ]\n",
        "        print(f\"Retrieved {len(contexts)} contexts, sleeping for 15 seconds...\")\n",
        "        time.sleep(15)\n",
        "        time_waited += 15\n",
        "\n",
        "    if time_waited >= 60 * 12:\n",
        "        print(\"Timed out waiting for contexts to be retrieved.\")\n",
        "        contexts = [\"No contexts retrieved. Try to answer the question yourself!\"]\n",
        "\n",
        "\n",
        "    # build our prompt with the retrieved contexts included\n",
        "    prompt_start = (\n",
        "        \"Answer the question based on the context below.\\n\\n\"+\n",
        "        \"Context:\\n\"\n",
        "    )\n",
        "    prompt_end = (\n",
        "        f\"\\n\\nQuestion: {query}\\nAnswer:\"\n",
        "    )\n",
        "    # append contexts until hitting limit\n",
        "    for i in range(1, len(contexts)):\n",
        "        if len(\"\\n\\n---\\n\\n\".join(contexts[:i])) >= limit:\n",
        "            prompt = (\n",
        "                prompt_start +\n",
        "                \"\\n\\n---\\n\\n\".join(contexts[:i-1]) +\n",
        "                prompt_end\n",
        "            )\n",
        "            break\n",
        "        elif i == len(contexts)-1:\n",
        "            prompt = (\n",
        "                prompt_start +\n",
        "                \"\\n\\n---\\n\\n\".join(contexts) +\n",
        "                prompt_end\n",
        "            )\n",
        "    return prompt\n",
        "\n",
        "\n",
        "def complete(prompt):\n",
        "    # instructions\n",
        "    sys_prompt = \"You are a helpful assistant that always answers questions.\"\n",
        "    # query text-davinci-003\n",
        "    res = openai.ChatCompletion.create(\n",
        "        model='gpt-4o-mini-2024-07-18',\n",
        "        messages=[\n",
        "            {\"role\": \"system\", \"content\": sys_prompt},\n",
        "            {\"role\": \"user\", \"content\": prompt}\n",
        "        ],\n",
        "        temperature=0\n",
        "    )\n",
        "    return res['choices'][0]['message']['content'].strip()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 178
        },
        "id": "LwsZuxiTvU2d",
        "outputId": "33b465e7-360b-442c-8dbc-3359fd3ea50b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Retrieved 3 contexts, sleeping for 15 seconds...\n"
          ]
        },
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "\"Answer the question based on the context below.\\n\\nContext:\\npooling approach. Or we can't use it in its current form. Now the solution to this problem was introduced by two people in 2019 Nils Reimers and Irenia Gurevich. They introduced what is the first sentence transformer or sentence BERT. And it was found that sentence BERT or S BERT outformed all of the previous Save the Art models on pretty much all benchmarks. Not all of them but most of them. And it did it in a very quick time. So if we compare it to BERT, if we wanted to find the most similar sentence pair from 10,000 sentences in that 2019 paper they found that with BERT that took 65 hours. With S BERT embeddings they could create all the embeddings in just around five seconds. And then they could compare all those with cosine similarity in 0.01 seconds. So it's a lot faster. We go from 65 hours to just over five seconds which is I think pretty incredible. Now I think that's pretty much all the context we need behind sentence transformers. And what we do now is dive into a little bit of how they actually work. Now we said before we have the core transform models and what S BERT does is fine tunes on sentence pairs using what is called a Siamese architecture or Siamese network. What we mean by a Siamese network is that we have what we can see, what can view as two BERT models that are identical and the weights between those two models are tied. Now in reality when implementing this we just use a single BERT model. And what we do is we process one sentence, a sentence A through the model and then we process another sentence, sentence B through the model. And that's the sentence pair. So with our cross-linked we were processing the sentence pair together. We were putting them both together, processing them all at once. This time we process them separately. And during training what happens is the weights within BERT are optimized to reduce the difference between two vector embeddings or two sentence\\n\\n---\\n\\npairs of related sentences you can go ahead and actually try training or fine-tuning using NLI with multiple negative ranking loss. If you don't have that fine. Another option is that you have a semantic textual similarity data set or STS and what this is is you have so you have sentence A here, sentence B here and then you have a score from from 0 to 1 that tells you the similarity between those two scores and you would train this using something like cosine similarity loss. Now if that's not an option and your focus or use case is on building a sentence transformer for another language where there is no current sentence transformer you can use multilingual parallel data. So what I mean by that is so parallel data just means translation pairs so if you have for example a English sentence and then you have another language here so it can it can be anything I'm just going to put XX and that XX is your target language you can fine-tune a model using something called multilingual knowledge distillation and what that does is takes a monolingual model for example in English and using those translation pairs it distills the knowledge the semantic similarity knowledge from that monolingual English model into a multilingual model which can handle both English and your target language. So they're three options quite popular very common that you can go for and as a supervised methods the chances are that probably going to outperform anything you do with unsupervised training at least for now. So if none of those sound like something\\n\\n---\\n\\nWe're going to be focusing on one specific training method which I think is quite useful because all it really needs is a reasonably small data set of parallel data which is simply translation pairs from a source language like English to whichever other language you're using. So obviously if you are wanting to train a sentence transformer in a language that doesn't really have that much data, it's particularly sentence similarity data, this can be really useful for actually taking a high performing, for example, English sentence transformer and transferring that knowledge or distilling that knowledge into a sentence transformer for your own language. So I think this will be pretty useful for a lot of you. And let's jump straight into it. Before we really get into the whole multilingual sentence transformer part of the video, I just want to sort of give an impression of what these multilingual sentence transformers are actually doing. So on here we can see a single English sentence or brief phrase down at the bottom, I love plants, and the rest of these are all in Italian. So what we have here are a vector representations of dense vector representations of these phrases. And a monolingual sentence transformer, which is most of the sentence transformers, will only cope with one language. So we would hope that phrases that have a similar meaning end up within the same sort of vector space. So like we have for amo lippiante here, and I love plants, these are kind of in the same space. A monolingual sentence transformer would do that for similar sentences. So in English, we might have I love plants and I like plants, which is actually what we have up here. So this here is Italian for I like plants. And we would hope that they're in a similar area, whereas irrelevant or\\n\\nQuestion: Which training method should I use for sentence transformers when I only have pairs of related sentences?\\nAnswer:\""
            ]
          },
          "execution_count": 19,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# first we retrieve relevant items from Pinecone\n",
        "query_with_contexts = retrieve(query)\n",
        "query_with_contexts"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "ioDVGF7lkDQL",
        "outputId": "304ece44-0205-41b5-8c98-e5e5d47ee21f"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'The recommended training method for sentence transformers when you only have pairs of related sentences is to use NLI (Natural Language Inference) with multiple negative ranking loss.'"
            ]
          },
          "execution_count": 20,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# then we complete the context-infused query\n",
        "complete(query_with_contexts)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OPO36aN8QoPZ"
      },
      "source": [
        "And we get a pretty great answer straight away, specifying to use _multiple-rankings loss_ (also called _multiple negatives ranking loss_)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sjOBIQ5rFeLZ"
      },
      "source": [
        "Once we're done with the index we delete it to save resources:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "PpJp-xExFeLa"
      },
      "outputs": [],
      "source": [
        "pc.delete_index(index_name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2kNh44bEFeLe"
      },
      "source": [
        "---"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "ml",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.6"
    },
    "vscode": {
      "interpreter": {
        "hash": "57376684f67c5d7b1589c855d7d0f1a1bdf8944ab1b903e711fdbf39434567bb"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}