03-Quering-AOpenAI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d59d527f-1100-45ff-b051-5f7c9029d94d",
   "metadata": {},
   "source": [
    "# Queries with and without Azure OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb9a9444-dc90-4fc3-aea7-8ee918301aba",
   "metadata": {},
   "source": [
    "So far, you have your Search Engine loaded **from two different data sources in two diferent indexes**, on this notebook we are going to try some example queries and then use Azure OpenAI service to see if we can get a good answer for the user query.\n",
    "\n",
    "The idea is that a user can ask a question about the dialogues of the TV Show FRIENDS (first datasource index) or about Covid (second datasource/index), and the engine will respond accordingly.\n",
    "\n",
    "This **Multi-Index** demo, mimics the scenario where a company loads multiple type of documents of different types and about completly different topics and the search engine must respond with the most relevant results."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71f6c7e3-9037-4b1e-ae17-1deaa27b9c08",
   "metadata": {},
   "source": [
    "## Set up variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8e50b404-a061-49e7-a3c7-c6eabc98ff0f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "import urllib\n",
    "import requests\n",
    "import random\n",
    "import json\n",
    "from collections import OrderedDict\n",
    "from IPython.display import display, HTML, Markdown\n",
    "from typing import List\n",
    "from operator import itemgetter\n",
    "\n",
    "# LangChain Imports needed\n",
    "from langchain_openai import AzureChatOpenAI\n",
    "from langchain_openai import AzureOpenAIEmbeddings\n",
    "from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder\n",
    "\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_core.retrievers import BaseRetriever\n",
    "from langchain_core.callbacks import CallbackManagerForRetrieverRun\n",
    "from langchain_core.runnables import ConfigurableField\n",
    "\n",
    "\n",
    "# Our own libraries needed\n",
    "from common.prompts import DOCSEARCH_PROMPT_TEXT\n",
    "from common.utils import get_search_results\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv(\"credentials.env\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2f2c22f8-79ab-405c-95e8-77a1978e53bc",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Setup the Payloads header\n",
    "headers = {'Content-Type': 'application/json','api-key': os.environ['AZURE_SEARCH_KEY']}\n",
    "params = {'api-version': os.environ['AZURE_SEARCH_API_VERSION']}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9297d29b-1f61-4dce-858e-bf4272172dba",
   "metadata": {},
   "source": [
    "## Multi-Index Search queries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5a46e2d3-298a-4708-83de-9e108b1a117a",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Text-based Indexes that we are going to query (from Notebook 01 and 02)\n",
    "index1_name = \"srch-index-files\"\n",
    "index2_name = \"srch-index-csv\"\n",
    "indexes = [index2_name, index1_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c62ebb2-d7be-4bfb-b1ba-4db86c11839a",
   "metadata": {},
   "source": [
    "Try questions that you think might be answered or addressed in the dialogues of Friends, or that can be addressed by medical publications about COVID in 2020-2021. Try comparing the results with the open version of ChatGPT.<br>\n",
    "\n",
    "**Example Questions you can ask**:\n",
    "- Is Chandler ever jealous of Richard?\n",
    "- Who is Mindy?\n",
    "- What happened between Ross and Rachel in Vegas?\n",
    "- What are some examples of reinforcement learning in virus spread?\n",
    "- What are the main risk factors for Covid-19?\n",
    "- What medicine reduces inflamation in the lungs?\n",
    "- Why Covid doesn't affect kids that much compared to adults?\n",
    "- Does chloroquine really works against covid?\n",
    "- Who won the 1994 soccer world cup? # This question should yield no answer if the system is correctly grounded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b9b53c14-19bd-451f-aa43-7ad27ccfeead",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "QUESTION = \"Is Chandler ever jealous of Richard?\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6d925eb-7f9c-429e-a62a-4c37d7702caf",
   "metadata": {},
   "source": [
    "### Search on both indexes individually and aggragate results\n",
    "\n",
    "#### **Note**: \n",
    "In order to standarize the indexes, **there must be 6 mandatory fields present on each index**: `id, title, name, location, chunk, chunkVector`. This is so that each document can be treated the same along the code. Also, **all indexes must have a semantic configuration**.\n",
    "\n",
    "We are going to use Semantic Hybrid Queries: vector search and keyword search, with semantic ranking over the merged result set for optimal results!. Per documentation:\n",
    "> Hybrid search combines text (keyword) and vector queries in a single search request. All subqueries in the request execute in parallel. The results are merged and reordered by new search scores, using Reciprocal Rank Fusion (RRF) to return a unified result set. In many cases, per benchmark tests, hybrid queries with semantic ranking return the most relevant results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "faf2e30f-e71f-4533-ab52-27d048b80a89",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n",
      "Index: srch-index-csv Results Found: 2801, Results Returned: 50\n",
      "200\n",
      "Index: srch-index-files Results Found: 2896, Results Returned: 50\n"
     ]
    }
   ],
   "source": [
    "agg_search_results = dict()\n",
    "\n",
    "# Whenever you use semantic ranking with vectors, make sure k is set to 50. \n",
    "# Semantic ranker uses up to 50 matches as input. Specifying less than 50 deprives the semantic ranking models of necessary inputs.\n",
    "k = 50 \n",
    "\n",
    "for index in indexes:\n",
    "    search_payload = {\n",
    "        \"search\": QUESTION, # Text query\n",
    "        \"select\": \"id, title, name, location, chunk\",\n",
    "        \"count\":\"true\",\n",
    "        \"top\": k,\n",
    "        \"queryType\": \"semantic\",\n",
    "        \"semanticConfiguration\": \"my-semantic-config\",\n",
    "        \"captions\": \"extractive\",\n",
    "        \"answers\": \"extractive\",\n",
    "        \"vectorQueries\": [  # Vector query\n",
    "            {\n",
    "                \"text\": QUESTION, \n",
    "                \"fields\": \"chunkVector\", \n",
    "                \"kind\": \"text\", \n",
    "                \"k\": k\n",
    "            }\n",
    "        ],\n",
    "        \"debug\": \"all\",\n",
    "    }\n",
    "\n",
    "    r = requests.post(os.environ['AZURE_SEARCH_ENDPOINT'] + \"/indexes/\" + index + \"/docs/search\",\n",
    "                     data=json.dumps(search_payload), headers=headers, params=params)\n",
    "    print(r.status_code)\n",
    "\n",
    "    search_results = r.json()\n",
    "    agg_search_results[index]=search_results\n",
    "    print(\"Index:\", index, \"Results Found: {}, Results Returned: {}\".format(search_results['@odata.count'], len(search_results['value'])))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f33018be-350d-4c54-b491-a86bc1cfffb6",
   "metadata": {},
   "source": [
    "#### **Important Note**: \n",
    "You may encounter errors (502) when attempting to search for results IF the indexer is still processing documents. This occurs because the embedding model is heavily utilized by the indexer, hitting its TPM quota. If you experience search errors, please try again or wait until the indexing is complete, which may take several hours."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "255c40f5-d836-480c-8c68-06a2282c8146",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# agg_search_results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7fd0fe5-4ee0-42e2-a920-72b93a407389",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Display the top results (from both searches) based on the score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "9e938337-602d-4b61-8141-b8c92a5d91da",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<h4>Top Answers</h4>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5>Answer - score: 0.92</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Monica Geller: So you stole that tape from Richard's apartment. Chandler Bing: Whoho ho. Listen to the judgement from the porn star. Chandler Bing: Because that's who I am, okay. I... You shouldn't be jealous. You should feel bad for him. Chandler Bing: Oh, yeah, well, poor Richard. Y'. Monica Geller: Chandler, this is not our problem."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<h4>Top Results</h4>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s09/e07/c11.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s09/e07/c11.txt</a> - score: 2.7</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Monica Geller: So you stole that tape from Richard's apartment. Chandler Bing: Whoho ho. Listen to the judgement from the porn star. Monica Geller: Why in the world would you take this tape and and why ... Chandler Bing: Because that's who I am, okay. I'm sure a mature man like Richard could see a tape like that and it wouldn't bother."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e23/c02.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e23/c02.txt</a> - score: 2.42</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "I... Monica Geller: Phoebe. Oh, Phoebe, I'm so sorry. Have you been here long. Phoebe Buffay: It's okay. What the hell took you so long. Monica Geller: Okay, you can not tell Chandler. Okay. That I ran into Richard. Phoebe Buffay: Which Richard. Monica Geller: The Richard. Phoebe Buffay: Richard Simmons. Oh my God. Monica Geller: Noo. My."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e23/c03.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e23/c03.txt</a> - score: 2.38</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Phoebe Buffay: So, so far is this trip to Vegas better or worse than the trip to London. Chandler Bing: Ohh that's the worse thing that can happen on an anniversary ever. All right, so you decided to tell him about the Richard thing. Monica Geller: Okay, I umm, I ran into Richard yesterday and he asked me if I wanted to go for a bite."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s06/e25/c01.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s06/e25/c01.txt</a> - score: 2.37</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "...: nan Monica G What uh-What did you-WhatRichard Burke: I still love you. And I know I probably shouldn't even be here telling you this, I mean you're with Chandler a guy I really like, and if you say he's straight I'll believe you. Richard Burke: Well yeah, I'm sorry. Richard Burke: Well I know I was an idiot. Richard Burke: Working with blind."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e20/c09.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e20/c09.txt</a> - score: 2.34</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Chandler Bing: Kick save and... denied. Richard Burke: But... he gets it back, pass to the middle, lines it up and.  ... Man you are inRichard Burke: Well, we had a table in college. Chandler Bing: Oh really, I didn't know they had foosball in the 1800's. Chandler Bing: Oh hey listen, don't be mad at him, it's our fault. Richard."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s06/e25/c05.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s06/e25/c05.txt</a> - score: 2.31</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Joey Tribbiani: Chandler giving you a hard time huh. Joey Tribbiani: Well, you know Chandler. Monica Geller: What. Joey Tribbiani: Chandler is a complex fellow, one who is unlikely to take a wife. Monica Geller: Is that some kind of boat talk. Joey Tribbiani: Monica face it, Chandler is against marriage. Joey Tribbiani... Monica Geller: Yeah!."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s06/e24/c07.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s06/e24/c07.txt</a> - score: 2.3</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Chandler Bing: Okay umm, before I meant you I had really little life and I couldn't imagine growing old with... unkn... Chandler Bing: I know, but just let me say it. Monica Geller: Oh my God, Richard. Chandler Bing: What?! I'm Chandler. Oh, that's Richard. Monica Geller: Oh God, maybe he won't see us. Richard Burke: Well, my nose got lonely.."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s06/e25/c08.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s06/e25/c08.txt</a> - score: 2.28</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Monica Geller: I don't know. Richard Burke: Oh, okay. Well, I'll just leave the door open and go sit on the couch. Monica Geller: Chandler is such an idiot. Richard Burke: Drink. Monica Geller: Yeah, I'll have a scotch. Richard Burke: ...on the rocks with a twist. Richard Burke: Oh really. Monica Geller: Yeah but it was because I-I had an eye exam."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e20/c02.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e20/c02.txt</a> - score: 2.25</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "... go so it's between my friend Eric Prower who has breath issuesHey, why don't you ask Richard. Joey Tribbiani: Ok, uh, hey Richard, if you had an extra ticket to the Knicks game and you had to choose between a friend who smells and one who bruises you who would you pick. Richard Burke: Wow. Chandler Bing: I don't know, Richard's really nice and."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e24/c05.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e24/c05.txt</a> - score: 2.24</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Joey Tribbiani: Come on, Chandler, I want this part soo much. Just one kiss, I won't tell anyone. Chandler Bing: Joey, no means no. Joey Tribbiani: Aww, Rach, I think you look cute And you, uh, you, you I could eat with a spoon. Ross Geller: Get away from me I said no. Monica Geller: Richard buzzed. He's waiting downstairs. Joey."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e23/c09.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e23/c09.txt</a> - score: 2.2</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Monica Geller: Thank you. Phoebe Buffay: Thanks. Monica Geller: I can't believe this. This is like the worst night ever. Phoebe Buffay: Y'know Monica you had a minor setback in your relationship with Chandler. Big deal! It's only Chandler. I am so sorry. Monica Geller: This is crazy. I mean, it's such a stupid argument. I don't even."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s06/e25/c11.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s06/e25/c11.txt</a> - score: 2.2</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Richard Burke: Chandler. Chandler Bing: Where is she. I'm not scared of you. Richard Burke: She's not here and please come in. Chandler Bing: Scotch on the rocks, with a twist, on a coaster. Ha-ha, Monica! Monica. Richard Burke: Okay, she was here, but she left. Chandler Bing: Well where did she go. Rich... And can I give you a piece."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e20/c04.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e20/c04.txt</a> - score: 2.16</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Phoebe Buffay: Yeah, I talked to my grandma about the Old Yeller incident, and she told me that my mom used to not show us the ends of sad movies to shield us from the pain and sadness. Chandler Bing: Hey.  ... HeyWhere is he, where's Richard. Chandler Bing: Your boyfriend is so cool. Chandler Bing: Yeah, he let us drive his Jaguar. Hey Chandler,."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s01/e06/c03.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s01/e06/c03.txt</a> - score: 2.14</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Chandler Bing: Hey, kids. #ALL#: Hey. Phoebe Buffay: No, 'cause this line is passion, and this is... just a line. Chandler Bing: Well, I can't believe I've been here almost seven seconds and you haven't asked me how my date went. Monica Geller: Oh, right, right. I mean, anthropologically speaking- unknown: nan Ross G... Fine, alright, now you'll."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e15/c04.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e15/c04.txt</a> - score: 2.13</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Chandler Bing: Sounds like a fun party. Rachel Green: And that crazy party animal will be ... Chandler Bing: Very, very funny, but don't say things like that in front of Monica. I don't want you putting any ideas in her head. Chandler Bing: Yeah, so, what's that supposed to mean. Rachel Green: Hey, Chandler, don't freak out. I'm telling you."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e22/c01.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e22/c01.txt</a> - score: 2.05</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Ross Geller: What's going on. Rac... I'm not gonna go see your ex-boyfriend. Chandler Bing: Oh, Richard. That's all I ever hear, Richard, Richard, Richard. Monica Geller: Since we've been going out, I think I've mentioned his name twice. Chandler Bing: Okay, so Richard, Richard. Monica Geller: It's not Richard. Okay. Chandler Bing:."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e24/c03.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e24/c03.txt</a> - score: 2.02</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Ross Geller: What's she look like. Chandler Bing: Well, we haven't exactly met, we just stayed up all night talking on the internet. Monica Geller: Woo-hoo, geeek. Chandler Bing: I li... Rachel Green: Wow. What's that like. Chandler Bing: It's like this, me, no jokes. Phoebe Buffay: All right, stop it, you're freaking me out. Richard Burke: Oh,."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s03/e13/c03.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s03/e13/c03.txt</a> - score: 2.02</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Chandler Bing: Yeah, baby. Ross Geller: What are you doing. Chandler Bing: Making chocolate milk. Do you ... Rachel Green: Somebody got in late last night. Monica Geller: Yeah well, I ran into Richard. unknown: nan Rachel Green: When did this happen. Monica Geller: Oh, um, around 8:02. Rachel Green: Monica, what are you doing.."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s09/e20/c11.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s09/e20/c11.txt</a> - score: 1.9</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Friends hooking up is a bad idea. Rachel Green: Please, what about you and Chandler. Monica Geller: That's different. Rachel Green: No. Ross: Hey R... Ross: I'm smarter than him! unknown: nan Phoebe Buffay: Hey, thank you so much for these tickets, Chandler. Chandler Bing: Oh well, this was a really important experience for me, and I wanted to."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e05/c02.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e05/c02.txt</a> - score: 1.86</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Please let me be happy. Ross Geller: Go over there and tell that woman the truth. Chandler Bing: All right. Ross Geller: Go. Chandler Bing: Hi. Jade: Hi. Chandler Bing: Listen, I have to, uh, um, I have to, I have to confess something. Jade: Yes. Chandler Bing: Whoever stood you up is a jerk. Jade: How did you--. Chandler Bing: I don't know. I."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s03/e13/c02.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s03/e13/c02.txt</a> - score: 1.85</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Monica Geller: You see that guy. He's in classics now, but y'know as soon as we leave he's going straight to the porn. Richard Burke: He's gonna go up to the counter with Citizen Kane, Vertigo, and Clockwork Orgy. Monica Geller: Yeah. Richard Burke: I missed this. Monica Geller: Me too. Richard Burke: So, you wanna get a hamburger or."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e16/c08.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e16/c08.txt</a> - score: 1.84</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Richard Burke: How ya doin'. Monica Geller: I'm a twinkie. Richard Burke: Really. Monica Geller: Oh, this is so hard. Richard Burke: Yeah, I know. I hate it too. Monica Geller: Maybe we should just tell your parents first. Richard Burke: My parents are dead. Monica Geller: God, you are so lucky. I mean, I mean. . . you... It's like a."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e16/c11.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e16/c11.txt</a> - score: 1.81</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Monica Geller: Hey there. Richard Burke: What. Monica Geller: Nothing, I just heard something nice about you. Richard Burke: Humm, really. Judy Geller: Richard. Richard. Your son isn't seeing anyone is he. Richard Burke: Uhh, not that I know of. Judy Geller: Well, I was thinking, why does... Monica Geller: Yes, a relationship. For."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s09/e20/c08.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s09/e20/c08.txt</a> - score: 1.8</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Monica Geller: Hey. Joey said no autographs. But if she's gettting one, then I want one too: To Monica. Monica Geller: Look at you with all... Rachel Green: Yeah. Monica Geller: I guess you have forgotten all about Joey. Rachel Green: Yeah, well, I guess I have forgotten about Joey and clearly you've forgotten about Chandler. Monica."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e15/c11.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e15/c11.txt</a> - score: 1.79</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Monica Geller: Wow, is that Michelle. Richard Burke: Yep. Richard Burke: Ya know, she's having another baby. Richard Burke: No no. Henry's almost two and he's talking and everyting. Richard Burke: Yeah. I mean I'm dating a man who's pool I once peed in. Richard Burke: I didn't need to know that. Richard Burke: Yeah, yeah, maybe.."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e04/c03.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e04/c03.txt</a> - score: 1.75</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Rachel Green: So Chandler, have you heard about Monica's secret boyfriend. Chandler Bing: Uhh, yeah. She uh, she uh, she uh might've mentioned him. Rachel Green: So Mon, when are we gonna meet this new secret waiter man. Monica Geller: Ohh, he's really shy. I-I don't think he's up to meeting everyone yet. You said that. Monica Geller:."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s03/e13/c09.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s03/e13/c09.txt</a> - score: 1.73</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Monica Geller: So we can be friends who sleep together. Richard Burke: Absolutely, this will just be something we do, like racquetball. Monica Geller: Sounds smart and healthy to me.  ...her racquetballRichard Burke: Just your dad. Although that's actually racquetball. Monica Geller: Oh. Richard Burke: You want me to cancel it. Monica."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e01/c06.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e01/c06.txt</a> - score: 1.73</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Monica Geller: Y'know, maybe it's best that we never got to do it again. Chandler Bing: Yeah, it kinda makes that-that one night special. Chandler Bing: 'Kay! unknown: nan Joey Tribbiani: Can I ask you something. Chandler Bing: Uhh, no. Joey Tribbiani: Felicity and I, we're watching My Giant, and I was thinking, \"I'm never gonna be as."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s04/e24/c23.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s04/e24/c23.txt</a> - score: 1.73</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Monica Geller: Y'know, maybe it's best that we never got to do it again. Chandler Bing: Yeah, it kinda makes that-that one night special. Chandler Bing: 'Kay! unknown: nan Joey Tribbiani: Can I ask you something. Chandler Bing: Uhh, no. Joey Tribbiani: Felicity and I, we're watching My Giant, and I was thinking, \"I'm never gonna be as."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s01/e13/c04.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s01/e13/c04.txt</a> - score: 1.71</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Joey Tribbiani: It's like if you woke up one day and found out your dad was leading this double life. He's like actually some spy, working for the C.I.A. Why can't you stop staring at my breasts. Chandler Bing: What? What. Rachel Green: Did you not get a good enough look the other day. Chandler Bing: Y'know, I don't see that happening. Rachel."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s06/e25/c02.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s06/e25/c02.txt</a> - score: 1.7</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Rachel Green: Isn't it incredible. Monica and Chandler, gettin' married. Phoebe Buffay: I know, they're gonna be so happy together. Rachel Green: I'm so happy and not at all jealous. No God, definitely not jealous! unknown: nan Rachel Green: I mean I'm probably 98% happy, maybe 2% jealous. And I mean w... I'm like 90/10. Rachel Green:."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s07/e05/c10.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s07/e05/c10.txt</a> - score: 1.69</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Joey Tribbiani: See. That's a great smile! Easy. Now, pretend I have a camera. You're changing it. Chandler Bing: I can't help it. Joey Tribbiani: All right, all right, all right, all right, you wanna know what I do when I take resume shots. Chandler Bing: Borrow money from me. Joey Tribbiani: Okay, first-first of all, you want to."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e10/c07.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e10/c07.txt</a> - score: 1.68</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Chandler Bing: Hey, we're having some fun now, huh, Ross. Wanna do another one, huh Russ. Chandler Bing: And weenie number two has it. Unless, of course, nine-down, Knight... You could try... but you would not be successful. Chandler Bing: OK, I'm gonna get some more coffee before the pinching and eye-poking begins. Russ: I know what."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e12/c04.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e12/c04.txt</a> - score: 1.68</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Chandler Bing: Hey. Everybody at work loved you last night. Monica Geller: Really. Chandler Bing: And. They like me more just because I was with ya. Monica Geller: Hey, I thought you alre... Chandler Bing: Oh I used too, but then Joey thought it would be fun to go to Central Park and hit rocks at...bigger rocks. Hey Rach, do you have."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s09/e06/c12.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s09/e06/c12.txt</a> - score: 1.68</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "What's... Monica Geller: Yes. What is the end of that sentence. Joey Tribbiani: Uhm. A penis model. Anyway, hey. Did you tell Chandler that some guy from work is the funniest guy you've ever met. Monica Geller: Yeah, so. Ross Geller: Wow. Joey Tribbiani: Really. Do you not know Chandler. Monica Geller: Is that why he's acting so weird. He's."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s03/e12/c15.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s03/e12/c15.txt</a> - score: 1.67</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Rachel Green: So ah, did you have fun at the bachelor party last night. Chandler Bing: Oh yeah, yeah. Look what I got, look what I got. Let it be me. Rachel Green: Honey, that's very sweet, it just seems to me though, that if two people love each other and trust each other, like we do, there's no reason to be jealous. Ross Geller: I."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e23/c09.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e23/c09.txt</a> - score: 1.67</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Richard Burke: Ooh, duct tape. Was I supposed to bring something too. Monica Geller: This is for the scratchy twins out there. I taped oven mits to their hands. Richard Burke: You're strict. Monica Geller: It's for their own good. Richard Burke: You kn... Monica Geller: You would not. I can't believe this. I hate this, you're too."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s04/e10/c11.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s04/e10/c11.txt</a> - score: 1.64</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Rachel Green: Chandler. Patrick just uh, ended things with me. Did you or did you not tell him that I was looking for a serious relationship. Chandler Bing: I did. I absolutely did. Sure. Chandler Bing: Well, actually it's a hockey team, so it's angry Canadians with no teeth. Rachel Green: Well that sounds fun too. unknown: nan."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s08/e21/c05.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s08/e21/c05.txt</a> - score: 1.63</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Rachel Green: Hi Pheebs. Phoebe Buffay: Hey. Oh, how did baby shopping go. Rachel Green: You don't understand. You didn't see how brazen she was. Phoebe Buffay: Sounds like you're a little jealous. Rachel Green: No! I'm not. I-I-I just think it's wrong. It's-it's that I'm-Here I am about to pop and he's out picking up some sh... Are."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s03/e05/c03.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s03/e05/c03.txt</a> - score: 1.61</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Chandler Bing: Does anyone else think David Copperfield is cute. Monica Geller: No, but he told me, he thinks your a fox. Chandler Bing: All right, Janice, likes him. In fact she likes him so much she put him on her freebie list. Joey Tribbiani: Her what. Chandler Bing: Well, we have a deal, where we each get to pick five celebrities."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e03/c08.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e03/c08.txt</a> - score: 1.59</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Joey Tribbiani: Check it out, check it out. Heckles' high school yearbook. Chandler Bing: Wow, he looks so normal. Phoebe Buffay: He's even kind of cute. Joey Tribbiani: \"Heckles, you crack me up in science class. You're the funniest kid in school. Chandler Bing: Funniest? Heckles. Joey Tribbiani: That's what it says. Chandler Bing: Wow, Heckles."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e15/c08.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e15/c08.txt</a> - score: 1.55</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Richard Burke: I'm going to look into your eyes now. Monica Geller: Really. Richard Burke: Yeah, that's my job. Alright, look up. . . look down, now open your eyes, now look down. Monica Geller: Good, they feel good, in my head. Richard Burke: So, it's great to see ya. Monica Geller: You too. Richard Burke: You too. Monica Geller: OK,."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s03/e13/c13.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s03/e13/c13.txt</a> - score: 1.52</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Monica Geller: Ow. Richard Burke: Really. Well, it's just like everyone else's apartment. It's got rooms, walls, and ceilings. Richard's Date:... Monica Geller: Y'know what, I've got to walk out of here right now, 'cause getting over you is the hardest thing that I have ever had to do. I don't think I could do it again. Richard Burke:."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e18/c11.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e18/c11.txt</a> - score: 1.46</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Richard Burke: That's it. That's the giant number you were afraid to tell me. Monica Geller: Well yeah. Richard Burke: Well, that's not bad at all. I mean, you had me thinkin it was like a fleet. Monica Geller: You really ok with it. Richard Burke: Oh honey, I'm fine. Monica Geller: Oh, yay. Ok about that two. Richard Burke: What.."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e18/c04.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e18/c04.txt</a> - score: 1.43</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Phoebe Buffay: And a crusty old man said I'll do what I can and the rest of the rats played moroccas. That's it, thanks, good night. Richard Burke: Phoebe's got another job, right. Rachel Green: Great set tonight Phoebs. Richard Burke: Um, we should go too, I got patients at 8 in the moring. Well, I ... Richard Burke: I don't know, I."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e15/c03.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e15/c03.txt</a> - score: 1.42</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Monica Geller: Sorry we're late. Richard Burke: Ah, that's OK, come on in. Um, I'm sorry, is Monica Geller coming. I was told she was. Richard Burke: Oh, well obviously you know Barbara... Monica Geller: The head tilt. Richard Burke: Yeah, since the divorce, when anybody asks me how I am, it's always with a sympathetic head tilt. 'How ya doin'?."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s07/e23/c10.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s07/e23/c10.txt</a> - score: 1.42</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Richard Burke: Hey Joey, could you uh, go through these lines with me.  ...ey TribbianiRichard Burke: Just the last two pages. Joey Tribbiani: All right. unknown: nan Richard Burke: I found the picture! unknown: nan Joey Tribbiani: Picture. Richard Burke: Could you uh, could you lower your script. Joey Tribbiani: Okay uh, look I know."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e24/c04.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e24/c04.txt</a> - score: 1.38</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Monica Geller: Where's Benny. There he is. Where's Benny, there he is. Richard Burke: Awww. You know that's probably why babies learn to talk, so they can tell grown ups to cut it out. Monica Geller: H... Just a little thing, no pressure. Richard Burke: Okay. Monica Geller: Did you ever, uh, like, think about the future. Richard."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e18/c09.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s02/e18/c09.txt</a> - score: 1.36</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Monica Geller: Well it wasn't that many guys. I mean, if you consider how many guys there actually are, it's a very small percentage. Rachel Green: Hey, it's not that big a deal, I was just curious. Ross Geller: G'night. Richard Burke: Night Richard. Good luck Mon. Monica Geller: Alright, before I tell you, uh, why don't you tell me."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s03/e13/c06.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D\">https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s03/e13/c06.txt</a> - score: 1.35</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "unknown: nan Richard Burke: So when people complement me on my cooking should I, what do I say. Monica Geller: You say 'Thank you very much,' and then you buy me something pretty. Monica Geller: You touch people's eyeballs every day and this feels weird. Richard Burke: Yeah, well, sure I touch them, but I spent years learning not to squish them. ."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(HTML('<h4>Top Answers</h4>'))\n",
    "\n",
    "for index,search_results in agg_search_results.items():\n",
    "\n",
    "    for result in search_results['@search.answers']:\n",
    "        if result['score'] > 0.5: # Show answers that are at least 50% of the max possible score=1\n",
    "            display(HTML('<h5>' + 'Answer - score: ' + str(round(result['score'],2)) + '</h5>'))\n",
    "            display(HTML(result['text']))\n",
    "\n",
    "            \n",
    "print(\"\\n\\n\")\n",
    "display(HTML('<h4>Top Results</h4>'))\n",
    "\n",
    "content = dict()\n",
    "ordered_content = OrderedDict()\n",
    "\n",
    "\n",
    "for index,search_results in agg_search_results.items():\n",
    "    for result in search_results['value']:\n",
    "        if result['@search.rerankerScore'] > 1:# Show answers that are at least 25% of the max possible score=4\n",
    "            content[result['id']]={\n",
    "                                    \"title\": result['title'],\n",
    "                                    \"chunk\": result['chunk'], \n",
    "                                    \"name\": result['name'], \n",
    "                                    \"location\": result['location'] ,\n",
    "                                    \"caption\": result['@search.captions'][0]['text'],\n",
    "                                    \"score\": result['@search.rerankerScore'],\n",
    "                                    \"index\": index\n",
    "                                    }\n",
    "    \n",
    "#After results have been filtered we will Sort and add them as an Ordered list\\n\",\n",
    "for id in sorted(content, key= lambda x: content[x][\"score\"], reverse=True):\n",
    "    ordered_content[id] = content[id]\n",
    "    url = str(ordered_content[id]['location']) + os.environ['BLOB_SAS_TOKEN']\n",
    "    title = str(ordered_content[id]['title']) if (ordered_content[id]['title']) else ordered_content[id]['name']\n",
    "    score = str(round(ordered_content[id]['score'],2))\n",
    "    display(HTML('<h5><a href=\"'+ url + '\">' + ordered_content[id]['location'] + '</a> - score: '+ score + '</h5>'))\n",
    "    display(HTML(ordered_content[id]['caption']))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52a6d3e6-afb2-4fa7-96d3-69bc2373ded5",
   "metadata": {},
   "source": [
    "### Comments on Query results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84e02227-6a92-4944-86f8-6c1e38d90fe4",
   "metadata": {},
   "source": [
    "As seen above the semantic re-ranking feature of Azure AI Search service is decent. It gives answers (sometimes) and also the top results with the corresponding file and the paragraph where the answers is possible located.\n",
    "\n",
    "Let's see if we can make this better with Azure OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8df3e6d4-9a09-4b0f-b328-238738ccfaec",
   "metadata": {},
   "source": [
    "# Using Azure OpenAI\n",
    "\n",
    "To use OpenAI to get a better answer to our question, the thought process is simple: let's **give the answer and the content of the documents from the search result to the GPT model as context and let it provide a better response**. This is what RAG (Retreival Augmented Generation) is about.\n",
    "\n",
    "Now, before we do this, we need to understand a few things first:\n",
    "\n",
    "1) Chainning and Prompt Engineering\n",
    "2) Embeddings\n",
    "\n",
    "We will use a library call **LangChain** that wraps a lot of boiler plate code.\n",
    "Langchain is one library that does a lot of the prompt engineering for us under the hood, for more information see [here](https://python.langchain.com/docs/introduction/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "eea62a7d-7e0e-4a93-a89c-20c96560c665",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Set the ENV variables that Langchain needs to connect to Azure OpenAI\n",
    "os.environ[\"OPENAI_API_VERSION\"] = os.environ[\"AZURE_OPENAI_API_VERSION\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "325d9138-2250-4f6b-bc88-50d7957f8d33",
   "metadata": {},
   "source": [
    "**Important Note**: Starting now, we will utilize Azure OpenAI models. Please ensure that you have deployed the following models within the Azure OpenAI portal:\n",
    "\n",
    "- text-embedding-3-large\n",
    "- gpt-4o\n",
    "- gpt-4o-mini\n",
    "\n",
    "Reference for Azure OpenAI models (regions, limits, dimensions, etc): [HERE](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e7c720e-ece1-45ad-9d01-2dfd15c182bb",
   "metadata": {},
   "source": [
    "## A gentle intro to chaining LLMs and prompt engineering"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2bcd7028-5a6c-4296-8c85-4f420d408d69",
   "metadata": {},
   "source": [
    "Chains refer to sequences of calls - whether to an LLM, a tool, or a data preprocessing step.\n",
    "\n",
    "Azure OpenAI is a type of LLM (provider) that you can use but there are others like Cohere, Huggingface, etc.\n",
    "\n",
    "Chains can be simple (i.e. Generic) or specialized (i.e. Utility).\n",
    "\n",
    "* Generic — A single LLM is the simplest chain. It takes an input prompt and the name of the LLM and then uses the LLM for text generation (i.e. output for the prompt).\n",
    "\n",
    "Here’s an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "13df9247-e784-4e04-9475-55e672efea47",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "COMPLETION_TOKENS = 2000\n",
    "llm = AzureChatOpenAI(deployment_name=os.environ[\"GPT4oMINI_DEPLOYMENT_NAME\"], \n",
    "                      temperature=0, \n",
    "                      max_tokens=COMPLETION_TOKENS)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "a3b55adb-6f98-4f15-b67a-9fbba5820560",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "output_parser = StrOutputParser()\n",
    "prompt = ChatPromptTemplate.from_messages([\n",
    "    (\"system\", \"You are an assistant that give thorough responses to users.\"),\n",
    "    (\"user\", \"{input}. Give your response in {language}\")\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6417d052-0035-4635-93e8-2bd3ec50d796",
   "metadata": {},
   "source": [
    "The | symbol is similar to a unix pipe operator, which chains together the different components feeds the output from one component as input into the next component."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "77a37e60-a1ef-4750-a1ec-9e4fe5ba07fa",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "chain = prompt | llm | output_parser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "6be6b4df-ee2c-4a0c-8ad3-a672d70f4f8d",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Yes, Chandler does experience feelings of jealousy towards Richard in the TV show \"Friends.\" This jealousy primarily arises during the earlier seasons when Monica starts dating Richard, who is significantly older than her. Chandler, being one of Monica's close friends, feels protective of her and is concerned about the age difference and the potential for Richard to hurt her.\n",
       "\n",
       "In particular, Chandler's jealousy is highlighted in Season 2, Episode 24 (\"The One with Barry and Mindy\"), when he expresses discomfort with Monica's relationship with Richard. He worries about how serious their relationship is and whether Richard is the right person for her. Chandler's jealousy is more about his concern for Monica's well-being and his protective instincts as a friend rather than a romantic rivalry.\n",
       "\n",
       "Overall, while Chandler's jealousy is not a central theme, it does surface in moments throughout the series, reflecting his loyalty to Monica and his feelings about her relationship with Richard."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 30.5 ms, sys: 242 μs, total: 30.7 ms\n",
      "Wall time: 1.93 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "display(Markdown(chain.invoke({\"input\": QUESTION, \"language\": \"English\"})))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd8539d0-a538-4368-82c3-5f91d8370f1e",
   "metadata": {},
   "source": [
    "**Note**: this is the first time you use OpenAI in this Accelerator, so if you get a Resource not found error, is most likely because the name of your OpenAI model deployment is different than the environmental variable set above `os.environ[\"GPT4oMINI_DEPLOYMENT_NAME\"]`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50ed014c-0c6b-448c-b995-fe7970b92ad5",
   "metadata": {},
   "source": [
    "Great!!, now you know how to create a simple prompt and use a chain in order to answer a general question using ChatGPT knowledge!. \n",
    "\n",
    "It is important to note that we rarely use generic chains as standalone chains. More often they are used as building blocks for Utility chains (as we will see next). Also important to notice is that we are NOT using our documents or the result of the Azure Search yet, just the knowledge of ChatGPT on the data it was trained on."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12c48038-b1af-4228-8ffb-720e554fd3b2",
   "metadata": {
    "tags": []
   },
   "source": [
    "**The second type of Chains are Utility:**\n",
    "\n",
    "* Utility — These are specialized chains, comprised of many building blocks to help solve a specific task. For example, LangChain supports some end-to-end chains (such as `create_retrieval_chain` for QnA Doc retrieval, Summarization, etc).\n",
    "\n",
    "We will build our own specific chain in this workshop for digging deeper and solve our use case of enhancing the results of Azure AI Search."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0454ddb-44d8-4fa9-929a-5e5563dd28f8",
   "metadata": {},
   "source": [
    "\n",
    "But before dealing with the utility chain needed, let's first review the concept of Embeddings and Vector Search and RAG. \n",
    "\n",
    "## Embeddings and Vector Search\n",
    "\n",
    "From the Azure OpenAI documentation ([HERE](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/embeddings?tabs=python)), An embedding is a special format of data representation that can be easily utilized by machine learning models and algorithms. The embedding is an information dense representation of the semantic meaning of a piece of text. Each embedding is a vector of floating point numbers, such that the distance between two embeddings in the vector space is correlated with semantic similarity between two inputs in the original format. For example, if two texts are similar, then their vector representations should also be similar. \n",
    "\n",
    "### Why Do We Need Vectors?\n",
    "\n",
    "Vectors are essential for several reasons:\n",
    "\n",
    "- **Semantic Richness**: They convert the semantic meaning of text into mathematical vectors, capturing nuances that simple keyword searches miss. This makes them incredibly powerful for understanding and processing language.\n",
    "- **Human-like Searching**: Searching using vector distances mimics the human approach to finding information based on context and meaning, rather than relying solely on exact word matches.\n",
    "- **Efficiency in Scale**: Vector representations allow for efficient handling and searching of large datasets. By reducing complex text to numerical vectors, algorithms can quickly sift through vast amounts of information.\n",
    "\n",
    "### Understanding LLM Tokens' Context Limitation\n",
    "\n",
    "Large Language Models (LLMs) like GPT come with a token limit for each input, which poses a challenge when dealing with lengthy documents or extensive data sets. This limitation restricts the model's ability to understand and generate responses based on the full context of the information provided. It becomes crucial, therefore, to devise strategies that can effectively manage and circumvent this limitation to leverage the full power of LLMs.\n",
    "\n",
    "To address this challenge, the solution incorporates several key steps:\n",
    "\n",
    "1. **Segmenting Documents**: Breaking down large documents into smaller, manageable segments.\n",
    "2. **Vectorization of Chunks**: Converting these segments into vectors, making them compatible with vector-based search techniques.\n",
    "3. **Hybrid Search**: Employing both vector and text search methods to pinpoint the most relevant segments in relation to the query.\n",
    "4. **Optimal Context Provision**: Presenting the LLM with the most pertinent segments, ensuring a balance between detail and brevity to stay within token limits.\n",
    "\n",
    "\n",
    "Our ultimate goal is to rely solely on vector indexes and hybrid searchs (vector + text). While it is possible to manually code parsers with OCR for various file types and develop a scheduler to synchronize data with the index, there is a more efficient alternative: **Azure AI Search has automated chunking strategies and vectorization**.\n",
    "\n",
    "It's important to note that **document segmentation and vectorization have already been completed in AI Azure Search**, as seen in the `ordered_content` dictionary. This pre-processing step simplifies subsequent operations, ensuring rapid response times and adherence to the token limits of the chosen OpenAI model.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80e79235-3d8b-4713-9336-5004cc4a1556",
   "metadata": {},
   "source": [
    "So really, our only job now is to make sure that the results from the Azure AI Search queries fit on the LLM context size, and then let it do its magic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "12682a1b-df92-49ce-a638-7277103f6cb3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "index_name = \"srch-index-files\"\n",
    "index2_name = \"srch-index-csv\"\n",
    "indexes = [index1_name, index2_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78a6d6a7-18ef-45b2-a216-3c1f50006593",
   "metadata": {},
   "source": [
    "In order to not duplicate code, we have put many of the code used above into functions. These functions are in the `common/utils.py` and `common/prompts.py` files. This way we can use these functios in the app that we will build later.\n",
    "\n",
    "`get_search_results()` do the multi-index search and returns the combined ordered list of documents/chunks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "3bccca45-d1dd-476f-b109-a528b857b6b3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of results: 50\n"
     ]
    }
   ],
   "source": [
    "k = 50  # play with this parameter and see the quality of the final answer\n",
    "ordered_results = get_search_results(QUESTION, indexes, k=k, reranker_threshold=1)\n",
    "print(\"Number of results:\",len(ordered_results))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "7714f38a-daaa-4fc5-a95a-dd025d153216",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Uncomment the below line if you want to inspect the ordered results\n",
    "# ordered_results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "235d4238-df6e-40eb-ab38-4fe6db614acd",
   "metadata": {},
   "source": [
    "Now let's create a Prompt Template that will ground the response only in the chunks retrieve by our hybrid AI Search."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "f86ed786-aca0-4e25-947b-d9cf3a82665c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "template = \"\"\"Answer the question thoroughly, based **ONLY** on the following context:\n",
    "{context}\n",
    "\n",
    "Important: Assume you know nothing about the subject, only based your answer on the context above.\n",
    "\n",
    "Question: {question}\n",
    "\"\"\"\n",
    "prompt = ChatPromptTemplate.from_template(template)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "25cba3d1-b5ab-4e28-96b3-ef923d99dc9f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Yes, Chandler Bing does exhibit jealousy towards Richard Burke in the context provided. In one exchange, Chandler expresses his discomfort and jealousy when he imagines Monica being with Richard, particularly when he pictures her in a compromising situation. He makes comments that indicate he is bothered by Richard's lingering feelings for Monica and the fact that Richard keeps a tape of Monica, suggesting that he feels threatened by Richard's past relationship with her. Additionally, Chandler's reactions and comments throughout the dialogues indicate that he is concerned about Richard's presence in Monica's life and how it affects their relationship."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 14.7 ms, sys: 3.91 ms, total: 18.6 ms\n",
      "Wall time: 4.13 s\n"
     ]
    }
   ],
   "source": [
    "%%time \n",
    "# Creation of our custom chain\n",
    "chain = prompt | llm | output_parser\n",
    "\n",
    "try:\n",
    "    display(Markdown(chain.invoke({\"question\": QUESTION, \"context\": ordered_results})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "417925af-486a-40bc-a290-28c35968c581",
   "metadata": {},
   "source": [
    "# Improving the Prompt and adding citations\n",
    "\n",
    "We could see above that in the answer given by GPT4o-mini, there is no citations or references. **How do we know if the answer is grounded on the context or not?**\n",
    "\n",
    "Let's see if this can be improved by Prompt Engineering.<br>\n",
    "On `common/prompts.py` we created a prompt called `DOCSEARCH_PROMPT_TEXT` check it out!\n",
    "\n",
    "**Let's also create a custom Retriever class** so we can plug it in easily within the chain building. \n",
    "Note: we can also use the Azure AI Search retriever class [HERE](https://python.langchain.com/docs/integrations/vectorstores/azuresearch), however we want to create a custom Retriever for the following reasons:\n",
    "\n",
    "1) We want to do multi-index searches in one call\n",
    "2) Easier to teach complex concepts of LangChain in this notebook\n",
    "3) We want to use the REST API vs the Python Azure Search SDK"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "bdf31f99-0dfb-423a-81f5-03018e61d9a9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "class CustomRetriever(BaseRetriever):\n",
    "    \n",
    "    topK : int\n",
    "    reranker_threshold : int\n",
    "    indexes: List\n",
    "    sas_token: str = None\n",
    "    search_filter: str = None\n",
    "    \n",
    "    def _get_relevant_documents(self, query: str) -> List[dict]:\n",
    "        \n",
    "        ordered_results = get_search_results(query, self.indexes, k=self.topK, \n",
    "                                             reranker_threshold=self.reranker_threshold, \n",
    "                                             sas_token=self.sas_token, search_filter=self.search_filter)\n",
    "        top_docs = []\n",
    "        for key,value in ordered_results.items():\n",
    "            location = value[\"location\"] if value[\"location\"] is not None else \"\"\n",
    "            document = {\"source\": location,\n",
    "                        \"score\": value[\"score\"],\n",
    "                        \"page_content\": value[\"chunk\"]}\n",
    "            top_docs.append(document)\n",
    "\n",
    "        return top_docs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "19b39c79-c827-4437-b58b-6a6fae53b968",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create the retriever\n",
    "retriever = CustomRetriever(indexes=indexes, topK=k, reranker_threshold=1, sas_token=os.environ['BLOB_SAS_TOKEN'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "c7aa4f58-4791-40a0-80c5-6582e0574579",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "50"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Test retreiver\n",
    "results = retriever.invoke(QUESTION)\n",
    "len(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "11b6546f-b5c5-4168-97fc-2636c50e41c2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# We can create now a dynamically configurable llm object that can change the model at runtime\n",
    "dynamic_llm = AzureChatOpenAI(deployment_name=os.environ[\"GPT4oMINI_DEPLOYMENT_NAME\"], \n",
    "                              temperature=0.5, max_tokens=COMPLETION_TOKENS).configurable_alternatives(\n",
    "    # This gives this field an id\n",
    "    # When configuring the end runnable, we can then use this id to configure this field\n",
    "    ConfigurableField(id=\"model\"),\n",
    "    # This sets a default_key.\n",
    "    # If we specify this key, the default LLM  (initialized above) will be used\n",
    "    default_key=\"gpt4omini\",\n",
    "    # This adds a new option, with name `gpt4o`\n",
    "    gpt4o=AzureChatOpenAI(deployment_name=os.environ[\"GPT4o_DEPLOYMENT_NAME\"], \n",
    "                         temperature=0.5, max_tokens=COMPLETION_TOKENS),\n",
    "    # You can add more configuration options here\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "0144dd4d-b5ff-4585-816a-fd1d0a93e544",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Define prompt template\n",
    "DOCSEARCH_PROMPT = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\"system\", DOCSEARCH_PROMPT_TEXT + \"\\n\\nCONTEXT:\\n{context}\\n\\n\"),\n",
    "        (\"human\", \"{question}\"),\n",
    "    ]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "d7da2f31-cf5d-4f3a-aad5-67b50b56968e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Declaration of the chain with the dynamic llm and the new prompt\n",
    "configurable_chain = (\n",
    "    {\n",
    "        \"context\": itemgetter(\"question\") | retriever, # Passes the question to the retriever and the results are assign to context\n",
    "        \"question\": itemgetter(\"question\")\n",
    "    }\n",
    "    | DOCSEARCH_PROMPT  # Passes the input variables above to the prompt template\n",
    "    | dynamic_llm   # Passes the finished prompt to the LLM\n",
    "    | StrOutputParser()  # converts the output (Runnable object) to the desired output (string)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "b67200e5-d3ae-4c86-9f69-bc7b964ab532",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Yes, Chandler Bing does express feelings of jealousy towards Richard Burke in various interactions. For instance, in one conversation, Chandler confronts Monica about Richard keeping a tape of her, suggesting that Richard is not over her and implying that he should feel bad for Richard instead of being jealous [[1]](https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s09/e07/c11.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D). \n",
       "\n",
       "Additionally, when Monica runs into Richard and has lunch with him, she chooses not to tell Chandler because she believes it would freak him out, especially since it is close to their anniversary [[2]](https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e23/c02.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D). \n",
       "\n",
       "In another instance, Chandler directly expresses his jealousy when he learns that Monica had lunch with Richard, as he feels insecure about his relationship with her and how Richard could potentially still have feelings for her [[3]](https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s06/e25/c11.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D). \n",
       "\n",
       "Overall, Chandler's jealousy is a recurring theme in his interactions with Monica regarding Richard."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 37.7 ms, sys: 8.83 ms, total: 46.5 ms\n",
      "Wall time: 8.16 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "try:\n",
    "    display(Markdown(configurable_chain.with_config(configurable={\"model\": \"gpt4omini\"}).invoke({\"question\": QUESTION})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8661b48d-1e57-4a70-9b0a-cc59f9093267",
   "metadata": {},
   "source": [
    "As seen above, we were able to improve the quality and breath of the answer and add citations with only prompt engineering!\n",
    "\n",
    "#### Let's try again, but with GPT-4o"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "efcfac6b-bac2-40c6-9ded-e4ee38e3093f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Yes, Chandler does experience jealousy towards Richard. In one instance, Chandler expresses his jealousy when he discovers that Monica had lunch with Richard. Monica tries to assure Chandler that it was nothing, but Chandler is clearly bothered by the situation, even though he tries to hide it by saying he's not mad [[3]](https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e23/c03.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D).\n",
       "\n",
       "Additionally, Chandler's jealousy is evident when he confronts Richard after Monica goes to see him. Chandler accuses Richard of making Monica \"think\" about their relationship, which clearly upsets him. Chandler even reveals to Richard that he was planning to propose to Monica, showing how serious he is about their relationship and how threatened he feels by Richard's presence [[11]](https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s06/e25/c11.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D)."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 31.7 ms, sys: 10.3 ms, total: 42 ms\n",
      "Wall time: 21 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    display(Markdown(configurable_chain.with_config(configurable={\"model\": \"gpt4o\"}).invoke({\"question\": QUESTION})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c791651-2c56-4de7-a232-8ff94937b938",
   "metadata": {},
   "source": [
    "**Answers from GPT-4o-mini and GPT-4o can vary ever time you run it!, and they are all correct most of the time**\n",
    "\n",
    "However if you try many times, you will see that GPT-4o provide better answers and is better at following instructions and citations and it is less prune to hallucinate. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6690453b-a9b1-4907-bd43-8c6b3ecba26e",
   "metadata": {},
   "source": [
    "## Adding Streaming to improve user experience and performance\n",
    "\n",
    "One way to make the response look faster is to stream the answer, so the user can see the response as it is typed. To do this, we just simply need to call the method `stream` instead of `invoke`. More on Streaming and Callbacks in later notebooks, but for now, this is one simple way to do it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "6d250c88-5984-438f-8390-1d93756048ab",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Yes, Chandler does show signs of jealousy towards Richard in several instances. For example, Chandler expresses insecurity about Monica's past with Richard when he mentions that Richard is not over Monica and keeps a tape of her, which makes Chandler feel inadequate and jealous [[1]](https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s09/e07/c11.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D). Additionally, Chandler becomes concerned when Monica runs into Richard and has lunch with him, even though she assures Phoebe that she felt nothing for Richard anymore [[2]](https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s05/e23/c02.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D). Lastly, Chandler confronts Richard when Monica is contemplating their relationship, showing his jealousy and concern that Richard might win Monica back [[3]](https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s06/e25/c11.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D)."
     ]
    }
   ],
   "source": [
    "for chunk in configurable_chain.with_config(configurable={\"model\": \"gpt4o\"}).stream({\"question\": QUESTION}):\n",
    "    print(chunk, end=\"\", flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "487bf690-3398-4824-96bc-b28e4da7e524",
   "metadata": {},
   "source": [
    "## Testing Groundness"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bcb4f9e-4fec-4e23-8d89-be0cf900e34d",
   "metadata": {},
   "source": [
    "Let’s ask the same question again, but this time for Season 1, where we know Richard doesn’t appear."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "ad7644c3-e92e-4e6c-9a3e-a64f6f036be8",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "search_filter = \"substringof('/s01/', location)\"\n",
    "retriever = CustomRetriever(indexes=indexes, topK=k, reranker_threshold=1, \n",
    "                            sas_token=os.environ['BLOB_SAS_TOKEN'],\n",
    "                            search_filter=search_filter)\n",
    "configurable_chain = (\n",
    "    {\n",
    "        \"context\": itemgetter(\"question\") | retriever, # Passes the question to the retriever and the results are assign to context\n",
    "        \"question\": itemgetter(\"question\")\n",
    "    }\n",
    "    | DOCSEARCH_PROMPT  # Passes the input variables above to the prompt template\n",
    "    | dynamic_llm   # Passes the finished prompt to the LLM\n",
    "    | StrOutputParser()  # converts the output (Runnable object) to the desired output (string)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "69e78ed8-e03e-4b9b-a9d6-d4fbd9563b66",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Empty Search Response\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "The tools did not provide relevant information. I cannot answer this from prior knowledge."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 32.2 ms, sys: 4.02 ms, total: 36.2 ms\n",
      "Wall time: 1.27 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "try:\n",
    "    display(Markdown(configurable_chain.with_config(configurable={\"model\": \"gpt4o\"}).invoke({\"question\": QUESTION})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "778c84a9-d4bc-44f0-9270-2896e756e14c",
   "metadata": {},
   "source": [
    "**Perfect!**, even know the model knows the answer based on its training data, it is grounding the answer only on the results from the context retrieved from azure AI search"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f347373a-a5be-473d-b64e-0f6b6dbcd0e0",
   "metadata": {},
   "source": [
    "# Summary\n",
    "##### By using OpenAI, the answers to user questions are way better than taking just the results from Azure AI Search. So the summary is:\n",
    "- Utilizing Azure AI Search, we conduct a multi-index hybrid search that identifies the top chunks of documents from each index.\n",
    "- Subsequently, Azure OpenAI utilizes these extracted chunks as context, comprehends the content, and employs it to deliver optimal answers.\n",
    "- Best of two worlds!\n",
    "\n",
    "##### Important observations on this notebook:\n",
    "\n",
    "1) Answers with GPT-4o-mini and GPT-4o are both correct, but GPT-4o seems have more breath and depth on its answers.\n",
    "2) Both models provide good and diverse citations in the right format.\n",
    "3) Streaming the answers improves the user experience big time!\n",
    "4) We achieved a good level of groundness using prompt engineering"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdc6e2fe-1c34-4952-99ad-14940f022379",
   "metadata": {},
   "source": [
    "# NEXT\n",
    "In the next notebook, we are going to see how we can treat complex and large documents separately, also using Vector Search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2233a40e-bc1f-497a-984a-3b77f0a9fff0",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "GPTSearch3 (Python 3.12)",
   "language": "python",
   "name": "gptsearch3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}