L3_linear_II_per_group.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "z9ln8ylCKHsx"
   },
   "source": [
    "# L3-D - Linear Quantization II: Per Group Quantization\n",
    "\n",
    "In this lesson, you will continue to learn about different granularities of performing linear quantization. You will cover `per group` in this notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the next cell to import all of the functions you have used before in the previous lesson(s) of `Linear Quantization II` to follow along with the video.\n",
    "\n",
    "- To access the `helper.py` file, you can click `File --> Open...`, on the top left."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "height": 100
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "\n",
    "from helper import linear_q_symmetric_per_channel, get_q_scale_symmetric, linear_dequantization\n",
    "from helper import plot_quantization_errors, quantization_error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JA1-rcLz4t4D"
   },
   "source": [
    "## Different Granularities for Quantization\n",
    "- For simplicity, you'll perform these using Symmetric mode."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yDin7Rm6Dzqu"
   },
   "source": [
    "### Per Group\n",
    "- For simplicity, you'll quantize a 2D tensor along the rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "executionInfo": {
     "elapsed": 143,
     "status": "ok",
     "timestamp": 1705361591253,
     "user": {
      "displayName": "Marc Sun",
      "userId": "00829270524676809963"
     },
     "user_tz": 300
    },
    "height": 268,
    "id": "0ZwexEliDik5"
   },
   "outputs": [],
   "source": [
    "def linear_q_symmetric_per_group(tensor, group_size,\n",
    "                                 dtype=torch.int8):\n",
    "    \n",
    "    t_shape = tensor.shape\n",
    "    assert t_shape[1] % group_size == 0\n",
    "    assert tensor.dim() == 2\n",
    "    \n",
    "    tensor = tensor.view(-1, group_size)\n",
    "    \n",
    "    quantized_tensor, scale = linear_q_symmetric_per_channel(\n",
    "                                tensor, dim=0, dtype=dtype)\n",
    "    \n",
    "    quantized_tensor = quantized_tensor.view(t_shape)\n",
    "    \n",
    "    return quantized_tensor, scale"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 425
    },
    "executionInfo": {
     "elapsed": 1955,
     "status": "ok",
     "timestamp": 1705361594322,
     "user": {
      "displayName": "Marc Sun",
      "userId": "00829270524676809963"
     },
     "user_tz": 300
    },
    "height": 217,
    "id": "J2mM-wv1VwyI",
    "outputId": "4c9904ce-e288-44b4-db53-e9f7fadebdf3"
   },
   "outputs": [],
   "source": [
    "def linear_dequantization_per_group(quantized_tensor, scale, \n",
    "                                    group_size):\n",
    "    \n",
    "    q_shape = quantized_tensor.shape\n",
    "    quantized_tensor = quantized_tensor.view(-1, group_size)\n",
    "    \n",
    "    dequantized_tensor = linear_dequantization(quantized_tensor, \n",
    "                                               scale, 0)\n",
    "    \n",
    "    dequantized_tensor = dequantized_tensor.view(q_shape)\n",
    "    \n",
    "    return dequantized_tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": [
    "test_tensor = torch.rand((6, 6))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note:** Since the values are random, what you see in the video might be different than what you will get."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": [
    "group_size = 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 149
   },
   "outputs": [],
   "source": [
    "quantized_tensor, scale = linear_q_symmetric_per_group(\n",
    "    test_tensor, group_size=group_size)\n",
    "\n",
    "dequantized_tensor = linear_dequantization_per_group(\n",
    "    quantized_tensor, scale, group_size=group_size)\n",
    "\n",
    "plot_quantization_errors(\n",
    "    test_tensor, quantized_tensor, dequantized_tensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 47
   },
   "outputs": [],
   "source": [
    "print(f\"\"\"Quantization Error : \\\n",
    "{quantization_error(test_tensor, dequantized_tensor)}\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [
    "NgmjMISuIyzF",
    "3dMVgqcNJCqE",
    "MFx2m7RmzRd5",
    "l6XbkmOzYMrC",
    "8NS1TnQt6E6v"
   ],
   "provenance": [
    {
     "file_id": "12_pQW6LB80u98m72_YKwM4ph7eb5Uf3g",
     "timestamp": 1705360205453
    },
    {
     "file_id": "1U9pm4j_uAD8EO7OrEPvdpwFjQKO3suuH",
     "timestamp": 1700237940920
    }
   ]
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}