15-FastAPI-API.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3dd1de00-d697-4b2f-a656-60177046d24d",
   "metadata": {},
   "source": [
    "# Building the FastAPI Backend"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f15266ef-9986-4e04-9cc8-a0195979af99",
   "metadata": {},
   "source": [
    "## **1. Introduction**\n",
    "\n",
    "In our journey of deploying a Smart Bot, we’ve already covered:\n",
    "\n",
    "1. **Notebook 13:** How to deploy a backend API using **Azure Bot Service**.  \n",
    "2. **Notebook 14:** How to interact with the Bot Service API programmatically using **POST requests**.  \n",
    "\n",
    "### **Azure Bot Service: Pros and Cons**\n",
    "\n",
    "#### ✅ **Pros:**\n",
    "- Seamless integration with multiple communication channels (O365 emails, MS Teams, web chat, etc.).\n",
    "- Comprehensive **Python SDKs** with utilities like typing indicators, proactive messages, and file uploads.\n",
    "- Built-in **authentication** and **logging mechanisms**.\n",
    "- SDK support for **Python**, **JavaScript**, and **.NET**.\n",
    "- Integration with **Application Insights Service** for monitoring.\n",
    "- Backed by **Microsoft's support and product teams**.\n",
    "\n",
    "#### ❌ **Cons:**\n",
    "- No support for **streaming**.\n",
    "- Lack of **private endpoint** support.\n",
    "- Cannot be containerized or run on **Kubernetes** or **container apps**.\n",
    "- Steeper **learning curve** for mastering its capabilities.\n",
    "\n",
    "### **Why FastAPI?**\n",
    "To address these limitations, we’ll build a **self-contained Backend API using FastAPI**. This backend will:\n",
    "- Be containerizable (e.g., using Docker).\n",
    "- Deployable on any platform supporting containers.\n",
    "- Allow flexibility and easier integration.\n",
    "\n",
    "\n",
    "In this notebook, we will zip the code and upload it to a new slot in the same Azure Web App service where the BotService API resides."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e185ba88-2a67-42e2-9f46-4aeaf3ba1e5f",
   "metadata": {},
   "source": [
    "## **2. Understanding the FastAPI Server**\n",
    "\n",
    "Our primary server code resides in:\n",
    "\n",
    "📁 `apps/backend/fastapi/app/server.py`\n",
    "\n",
    "### **Key Endpoints in `server.py`:**\n",
    "1. **`/docs/`**  \n",
    "   - Displays the **OpenAPI (Swagger)** documentation for the API.\n",
    "\n",
    "2. **`/stream/`**  \n",
    "   - Streams live events and tokens.\n",
    "\n",
    "3. **`/invoke/`**  \n",
    "   - Accepts single queries and returns responses in **JSON format**.\n",
    "\n",
    "4. **`/batch/`**  \n",
    "   - Processes multiple queries simultaneously and returns a list of answers.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc98f528-707b-4a39-9a0b-3bc76ecd74a1",
   "metadata": {},
   "source": [
    "## **3. Deployment to Azure App Service**\n",
    "\n",
    "Deployment instructions are detailed in:\n",
    "\n",
    "📄 `apps/backend/fastapi/README.md`\n",
    "\n",
    "**➡️ Follow the README guide to complete the deployment!**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b185280-9f2c-4961-8b09-f8a8b12ce025",
   "metadata": {
    "tags": []
   },
   "source": [
    "## **4. Interacting with the API**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b3f7cd5e-0ce0-41cc-995a-ffd1fff83c38",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import requests\n",
    "import json\n",
    "import random\n",
    "import uuid\n",
    "from sseclient import SSEClient"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6adc3d3-2a27-47de-8984-926db605e259",
   "metadata": {},
   "source": [
    "### Base URL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "53845890-5010-4b92-bdb3-2f4635d92290",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "BASE_URL = \"https://<YOUR_BACKEND_WEBAPP_NAME>-staging.azurewebsites.net\"  # Note that \"-staging\" is the Azure App Service slot where the LangServe API was deployed\n",
    "BASE_URL = \"http://localhost:8000\"  # or wherever your FastAPI app runs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f869c5e5-4e74-4cc6-b326-079a6d0d65de",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Define client-side python functions to consume the endpoints `/invoke`, `/stream` and `/batch`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "0f40e622-8811-491d-894e-22a021a7b3b0",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def call_invoke(question: str, thread_id: str = \"\"):\n",
    "    \"\"\"\n",
    "    Test the /invoke endpoint by sending a single question in JSON.\n",
    "    Expects a JSON response with { \"final_answer\": ... }.\n",
    "    \"\"\"\n",
    "    url = f\"{BASE_URL}/invoke\"\n",
    "    payload = {\n",
    "        \"user_input\": question,\n",
    "        \"thread_id\": thread_id or str(uuid.uuid4())\n",
    "    }\n",
    "\n",
    "    print(f\"\\n--- Calling /invoke with question: {question}\")\n",
    "    try:\n",
    "        response = requests.post(url, json=payload)\n",
    "        response.raise_for_status()\n",
    "        data = response.json()\n",
    "        print(f\"[/invoke response] final_answer: {data.get('final_answer')}\")\n",
    "    except requests.HTTPError as err:\n",
    "        print(f\"[/invoke] HTTP Error: {err}\")\n",
    "    except Exception as e:\n",
    "        print(f\"[/invoke] An error occurred: {e}\")\n",
    "        \n",
    "\n",
    "\n",
    "def call_batch(questions: list[str], thread_id: str = \"\"):\n",
    "    \"\"\"\n",
    "    Test the /batch endpoint by sending multiple questions in a single request.\n",
    "    The endpoint returns { \"answers\": [...] }.\n",
    "    \"\"\"\n",
    "    url = f\"{BASE_URL}/batch\"\n",
    "    payload = {\n",
    "        \"questions\": questions,\n",
    "        \"thread_id\": thread_id or str(uuid.uuid4())\n",
    "    }\n",
    "\n",
    "    print(f\"\\n--- Calling /batch with questions: {questions}\")\n",
    "    try:\n",
    "        resp = requests.post(url, json=payload)\n",
    "        resp.raise_for_status()\n",
    "        data = resp.json()\n",
    "        answers = data.get(\"answers\", [])\n",
    "        for i, ans in enumerate(answers):\n",
    "            print(f\"Q: {questions[i]}\")\n",
    "            print(f\"A: {ans}\")\n",
    "            print(\"-----\")\n",
    "    except requests.HTTPError as err:\n",
    "        print(f\"[/batch] HTTP Error: {err}\")\n",
    "    except Exception as e:\n",
    "        print(f\"[/batch] An error occurred: {e}\")\n",
    "\n",
    "\n",
    "def process_partial_text(text: str):\n",
    "    \"\"\"\n",
    "    Example of how to handle partial text from SSE.\n",
    "    In real usage, you'd update your UI, buffer it, etc.\n",
    "    \"\"\"\n",
    "    print(text, end=\"\", flush=True)\n",
    "\n",
    "\n",
    "def call_stream(question: str, thread_id: str = \"\"):\n",
    "    \"\"\"\n",
    "    Demonstrates how to call your FastAPI /stream endpoint with sseclient-py in sync mode.\n",
    "    \"\"\"\n",
    "    url = f\"{BASE_URL}/stream\"\n",
    "    payload = {\n",
    "        \"user_input\": question,\n",
    "        \"thread_id\": thread_id\n",
    "    }\n",
    "\n",
    "    with requests.post(url, json=payload, stream=True) as resp:\n",
    "        resp.raise_for_status()\n",
    "        client = SSEClient(resp)\n",
    "        for event in client.events():\n",
    "            evt_type = event.event\n",
    "            evt_data = event.data\n",
    "\n",
    "            if evt_type == 'metadata':\n",
    "                info = json.loads(evt_data)\n",
    "                print(f\"\\n[Metadata] run_id={info.get('run_id', '')}\")\n",
    "\n",
    "            elif evt_type == 'data':\n",
    "                # The server is sending partial chunk(s) as a \"data\" event\n",
    "                # so we treat it as partial text to display:\n",
    "                process_partial_text(evt_data)\n",
    "\n",
    "            elif evt_type == 'on_tool_start':\n",
    "                print(f\"\\n[Tool Start] {evt_data}\")\n",
    "\n",
    "            elif evt_type == 'on_tool_end':\n",
    "                print(f\"\\n[Tool End] {evt_data}\")\n",
    "\n",
    "            elif evt_type == 'end':\n",
    "                # This signals no more data will follow.\n",
    "                print(f\"\\n[Done Streaming] Final text: {evt_data}\")\n",
    "\n",
    "            elif evt_type == 'error':\n",
    "                # The server encountered an exception or error\n",
    "                print(f\"\\n[SSE Error] {evt_data}\")\n",
    "\n",
    "            else:\n",
    "                # Some unexpected event name\n",
    "                print(f\"\\n[Unrecognized event={evt_type}] {evt_data}\")\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48adee48-1734-48e6-b92c-42bec7891339",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Generate a random session ID for the conversation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e1a537d2-dca5-43bc-a8ff-a91d4f058e3f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using thread_id/session_id: session-5e426240-c770-4997-889d-519ab6153b80\n"
     ]
    }
   ],
   "source": [
    "random_session_id = \"session-\" + str(uuid.uuid4())\n",
    "print(f\"Using thread_id/session_id: {random_session_id}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a24e89e-abaf-4a78-8749-80d9337b512e",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Test the `/invoke` endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ce42c7f8-fafd-47b6-b2f2-92ee2222f02d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "--- Calling /invoke with question: @apisearch, what is the price of ethereum today?\n",
      "[/invoke response] final_answer: The current price of Ethereum (ETH) against the US Dollar (USD) is $3484.00.\n"
     ]
    }
   ],
   "source": [
    "call_invoke(\"@apisearch, what is the price of ethereum today?\", thread_id=random_session_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "879ef229-5eac-4999-bc0a-abb9e68f9e21",
   "metadata": {},
   "source": [
    "### Test the `/batch` endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8e035f4c-5a18-4d75-824a-b3e701daeecd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "--- Calling /batch with questions: ['What is the capital of France?', 'Who wrote Pride and Prejudice?', 'Whats the distance to the Moon?']\n",
      "Q: What is the capital of France?\n",
      "A: The capital of France is Paris. It is the largest city in France and has been a major center for finance, diplomacy, commerce, culture, fashion, and gastronomy since the 17th century[[1]](https://en.wikipedia.org/wiki/Paris).\n",
      "-----\n",
      "Q: Who wrote Pride and Prejudice?\n",
      "A: \"Pride and Prejudice\" was written by Jane Austen. It was published in 1813 and is one of her most famous works[[1]](https://en.wikipedia.org/wiki/Pride_and_Prejudice).\n",
      "-----\n",
      "Q: Whats the distance to the Moon?\n",
      "A: The current distance from Earth to the Moon is approximately 378,576 kilometers (235,287 miles) as of today[[1]](https://starlust.org/how-far-away-is-the-moon-now/).\n",
      "-----\n"
     ]
    }
   ],
   "source": [
    "questions_batch = [\n",
    "    \"What is the capital of France?\",\n",
    "    \"Who wrote Pride and Prejudice?\",\n",
    "    \"Whats the distance to the Moon?\",\n",
    "]\n",
    "call_batch(questions_batch, thread_id=random_session_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24ef120b-b751-4a09-bb4a-73f602a009c3",
   "metadata": {},
   "source": [
    "### Test the `/stream` endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "21f0382f-3960-46ed-8f68-77ac735c90c2",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "[Tool Start] Starting documents_retrieval\n",
      "\n",
      "[Tool End] Done documents_retrieval\n",
      "The scene where Joey wears all of Chandler's clothes is a humorous moment from the show \"Friends.\" Joey, in retaliation for Chandler hiding his clothes, decides to wear everything Chandler owns. He walks into the room wearing multiple layers of Chandler's clothing and exclaims, \"Look at me! I'm Chandler! Could I be wearing any more clothes?\" He even jokes about going commando, saying, \"Maybe if I wasn't going commando...\" The scene is made even funnier by Joey's exaggerated movements and the sheer volume of clothes he's wearing, which makes it difficult for him to move comfortably [[source]](https://blobstorageixqo5iaqmpzwc.blob.core.windows.net/friends/s06/e20/c09.txt?sv=2022-11-02&ss=b&srt=sco&sp=rltfx&se=2026-01-02T09:04:19Z&st=2025-01-02T01:04:19Z&spr=https&sig=q%2FjY9R25rdc%2BIH1iiq1uPIBm82xECsN9d%2B2ftdM1SJI%3D)."
     ]
    }
   ],
   "source": [
    "stream_question = \"@docsearch, describe the scene where Joey wears all of Chandler's clothes\"\n",
    "call_stream(stream_question, thread_id=random_session_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a26c78e1-8dca-4a28-a20c-6cfe3641c17f",
   "metadata": {
    "tags": []
   },
   "source": [
    "## **5. Understanding SSE (Server-Sent Events)**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2932652-6c1c-4e66-bb8f-4dd52a528388",
   "metadata": {},
   "source": [
    "In designing the FastAPI backend `/stream` endpoint, we decided to adhere to the **standard SSE (Server-Sent Events) format** for streaming data from the server to clients. This decision ensures:\n",
    "\n",
    "1. **Compatibility Across Platforms:** Any standard SSE client library, regardless of programming language or platform, can consume our server's streaming responses without requiring custom adaptations.  \n",
    "2. **Simplified Integration:** Developers can use widely available SSE libraries to handle real-time data streaming with minimal effort.  \n",
    "3. **Scalability and Maintainability:** A standard protocol simplifies debugging, future upgrades, and integration with different frontend or backend technologies.  \n",
    "\n",
    "By following the standard SSE format, we maximize **portability** and **ease of use**, allowing clients to interact with our streaming API seamlessly."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "681cd266-351a-4cdf-94fe-b3ac204c0749",
   "metadata": {},
   "source": [
    "### **5.1. Client Libraries for SSE**\n",
    "\n",
    "Below are commonly used libraries across different environments that are compatible with our server's SSE implementation for the `/stream` events endpoint:\n",
    "\n",
    "#### **Browser-Side Libraries**\n",
    "1. **Native EventSource API:** Modern browsers natively support SSE via the `EventSource` API.  \n",
    "2. **eventsource-polyfill:** Ensures compatibility with older browsers.  \n",
    "\n",
    "```bash\n",
    "npm install eventsource-polyfill\n",
    "```\n",
    "3. **reconnecting-eventsource:** Adds auto-reconnect capabilities to handle connection drops gracefully.\n",
    "```bash\n",
    "npm install reconnecting-eventsource\n",
    "```\n",
    "#### **Node.js Libraries:**\n",
    "\n",
    "1. **eventsource:** A Node.js implementation of the browser's EventSource API.\n",
    "```bash\n",
    "npm install eventsource\n",
    "```\n",
    "2. **sse-client:** Provides additional features for managing SSE connections in Node.js.\n",
    "```bash\n",
    "npm install sse-client\n",
    "```\n",
    "\n",
    "#### **Python Libraries:**\n",
    "\n",
    "1. **sseclient-py:** Simplifies consuming SSE streams in Python applications.\n",
    "```bash\n",
    "pip install sseclient-py\n",
    "```\n",
    "\n",
    "#### **Framework-Specific Libraries:**\n",
    "\n",
    "1. **React:** react-sse for React hooks integration.\n",
    "```bash\n",
    "npm install react-sse\n",
    "```\n",
    "2. **Vue.js:** vue-sse for Vue-specific SSE handling.\n",
    "```bash\n",
    "npm install vue-sse\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5b6e18b6-c107-4197-b823-3dc8003b2c88",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "GPTSearch3 (Python 3.12)",
   "language": "python",
   "name": "gptsearch3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}