P4_License_Plate_Recognition_BASE.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Práctica 4: Reconocimiento de Matrículas\n",
    "\n",
    "Este notebook implementa un prototipo de reconocimiento de matrículas de vehículos en video. Los objetivos de esta práctica incluyen la detección y seguimiento de personas y vehículos, el reconocimiento de matrículas visibles en los vehículos, y la exportación de los resultados en un video y un archivo CSV."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Objetivos\n",
    "\n",
    "La práctica se enfoca en desarrollar un sistema de detección y reconocimiento de objetos que cumpla con los siguientes requisitos:\n",
    "\n",
    "- Detección y seguimiento: Identificación y rastreo de personas y vehículos presentes en el video.\n",
    "- Reconocimiento de matrículas: Detección de matrículas en los vehículos y reconocimiento del texto usando OCR.\n",
    "- Conteo total de clases: Recuento acumulativo de cada tipo de objeto detectado.\n",
    "- Exportación de resultados: Generación de un video que visualice los resultados y exportación de un archivo CSV con el detalle de las detecciones."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparación del entorno"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cv2\n",
    "import time\n",
    "import math\n",
    "import csv\n",
    "from collections import defaultdict, Counter\n",
    "from ultralytics import YOLO\n",
    "import easyocr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def initialize_model(model_path):\n",
    "    \"\"\"Initialize the YOLO model for detection.\"\"\"\n",
    "    return YOLO(model_path)\n",
    "\n",
    "def initialize_reader():\n",
    "    \"\"\"Initialize the EasyOCR reader.\"\"\"\n",
    "    return easyocr.Reader(['en'])  \n",
    "\n",
    "def initialize_video_writer(cap, output_video_path):\n",
    "    \"\"\"Set up the video writer for the processed video.\"\"\"\n",
    "    fourcc = cv2.VideoWriter_fourcc(*'mp4v')  # Codec\n",
    "    fps = cap.get(cv2.CAP_PROP_FPS)\n",
    "    frame_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))\n",
    "    frame_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))\n",
    "    return cv2.VideoWriter(output_video_path, fourcc, fps, (frame_width, frame_height))\n",
    "\n",
    "def write_csv_header(csv_file_path):\n",
    "    \"\"\"Prepare CSV file for logging.\"\"\"\n",
    "    with open(csv_file_path, mode='w', newline='') as file:\n",
    "        writer = csv.writer(file)\n",
    "        writer.writerow(['frame', 'object_type', 'confidence', 'tracking_id', 'x1', 'y1', 'x2', 'y2',\n",
    "                         'license_plate_confidence', 'mx1', 'my1', 'mx2', 'my2', 'license_plate_text'])\n",
    "\n",
    "def put_text(frame, text, position, color=(0, 255, 0), font_scale=0.6, thickness=2, bg_color=(0, 0, 0)):\n",
    "    \"\"\"Helper function to put text with background on the frame.\"\"\"\n",
    "    text_size = cv2.getTextSize(text, cv2.FONT_HERSHEY_SIMPLEX, font_scale, thickness)[0]\n",
    "    text_x, text_y = position\n",
    "    box_coords = ((text_x, text_y - text_size[1] - 5), (text_x + text_size[0] + 5, text_y + 5))\n",
    "    cv2.rectangle(frame, box_coords[0], box_coords[1], bg_color, cv2.FILLED)\n",
    "    cv2.putText(frame, text, position, cv2.FONT_HERSHEY_SIMPLEX, font_scale, color, thickness)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Parameters \n",
    "video_path = 'C0142.mp4'  # Path to input video\n",
    "model_path = 'yolo11n.pt'  # Path to YOLO model\n",
    "license_plate_detector_model_path = 'runs/detect/license_plate_detector/weights/best.pt'  # Path to license plate detector model\n",
    "\n",
    "output_video_path = 'output_video.mp4'  # Path to save the annotated output video\n",
    "csv_file_path = 'detection_tracking_log.csv'  # Path to save the CSV log file\n",
    "show_video = True  # Set to True to display the video while processing\n",
    "classes_to_detect = [0, 1, 2, 3, 5]  # Class IDs to detect (e.g., [0, 2] for person and car)\n",
    "\n",
    "model = initialize_model(model_path)\n",
    "license_plate_detector = YOLO(license_plate_detector_model_path)\n",
    "reader = initialize_reader()\n",
    "\n",
    "# Define class names and colors for display\n",
    "class_names = {\n",
    "    0: \"person\",\n",
    "    1: \"bicycle\",\n",
    "    2: \"car\",\n",
    "    3: \"motorbike\",\n",
    "    5: \"bus\"\n",
    "}\n",
    "class_colors = {\n",
    "    0: (255, 255, 255),\n",
    "    1: (0, 255, 0),\n",
    "    2: (0, 0, 255),\n",
    "    3: (255, 255, 0),\n",
    "    5: (0, 255, 255)\n",
    "}\n",
    "\n",
    "# Dictionary to store the best plate and its confidence for each track_id\n",
    "vehicle_plates = {}\n",
    "\n",
    "# Persistent total count of each class across all frames\n",
    "total_class_count = Counter()\n",
    "# Track unique IDs for each class to count only once\n",
    "seen_ids = defaultdict(set)\n",
    "frame_number = 0  # Initialize frame counter\n",
    "\n",
    "blur_enabled = True # Set to True to blur faces\n",
    "paused = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Open the video file and set up output for processed video\n",
    "cap = cv2.VideoCapture(video_path)\n",
    "fourcc = cv2.VideoWriter_fourcc(*'mp4v')  # Codec\n",
    "fps = cap.get(cv2.CAP_PROP_FPS)\n",
    "frame_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))\n",
    "frame_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))\n",
    "out = initialize_video_writer(cap, output_video_path)\n",
    "write_csv_header(csv_file_path)\n",
    "\n",
    "# Loop through each frame\n",
    "while cap.isOpened():\n",
    "    if not paused:\n",
    "        ret, frame = cap.read()\n",
    "        if not ret:\n",
    "            break\n",
    "\n",
    "        start_time = time.time()\n",
    "        frame_number += 1\n",
    "\n",
    "        # Run YOLO detection and tracking\n",
    "        results = model.track(frame, persist=True, classes=classes_to_detect)\n",
    "        current_frame_count = Counter()\n",
    "\n",
    "        # Process detections\n",
    "        for result in results:\n",
    "            boxes = result.boxes\n",
    "\n",
    "            for box in boxes:\n",
    "                x1, y1, x2, y2 = map(int, box.xyxy[0])\n",
    "                cls = int(box.cls[0])\n",
    "                confidence = round(float(box.conf[0]), 2)\n",
    "\n",
    "                if box.id is not None:\n",
    "                    track_id = int(box.id[0].tolist())\n",
    "                    if track_id not in seen_ids[cls]:\n",
    "                        seen_ids[cls].add(track_id)\n",
    "                        total_class_count[class_names[cls]] += 1\n",
    "\n",
    "                    # License plate recognition for cars\n",
    "                    license_plate_text = \"\"\n",
    "                    plate_confidence = None\n",
    "                    mx1, my1, mx2, my2 = None, None, None, None\n",
    "\n",
    "                    # Check if the detected object is a car, then detect license plate within its bounding box\n",
    "                    if class_names[cls] in [\"car\", \"motorbike\", \"bus\"]:\n",
    "                        vehicle_img = frame[y1:y2, x1:x2]  # Crop the vehicle area to search for license plate\n",
    "                        \n",
    "                        # Check if the cropped image is large enough for license plate detection\n",
    "                        min_plate_size = 80\n",
    "                        if vehicle_img.shape[0] < min_plate_size or vehicle_img.shape[1] < min_plate_size:\n",
    "                            continue\n",
    "                        \n",
    "                        # Check if the confidence is high enough for license plate detection\n",
    "                        if confidence < 0.7:\n",
    "                            continue\n",
    "                        \n",
    "                        # Run license plate detector model on the cropped vehicle image\n",
    "                        plate_results = license_plate_detector.predict(vehicle_img)\n",
    "\n",
    "                        # Process license plate detection results\n",
    "                        if plate_results and len(plate_results[0].boxes) > 0:\n",
    "                            for plate_box in plate_results[0].boxes:\n",
    "                                # Get bounding box coordinates for the license plate, adjusted to the frame's coordinates\n",
    "                                px1, py1, px2, py2 = map(int, plate_box.xyxy[0])\n",
    "                                px1, py1, px2, py2 = px1 + x1, py1 + y1, px2 + x1, py2 + y1  # Adjust to the car's bounding box position\n",
    "                                                            \n",
    "                                # Draw bounding box for license plate\n",
    "                                background_color = (255, 255, 255)  # White background for contrast\n",
    "                                cv2.rectangle(frame, (px1, py1), (px2, py2), background_color, 2)\n",
    "                                    \n",
    "                                # Extract the license plate text using OCR\n",
    "                                license_plate_roi = frame[py1:py2, px1:px2]\n",
    "                                \n",
    "                                # Resize based on the plate size\n",
    "                                plate_height, plate_width = license_plate_roi.shape[:2]\n",
    "                                scale_factor = 100.0 / plate_height\n",
    "                                resized_plate = cv2.resize(\n",
    "                                    license_plate_roi, None, fx=scale_factor, fy=scale_factor,\n",
    "                                    interpolation=cv2.INTER_CUBIC)\n",
    "\n",
    "                                # Convert to grayscale\n",
    "                                gray_plate = cv2.cvtColor(resized_plate, cv2.COLOR_BGR2GRAY)\n",
    "\n",
    "                                # Apply CLAHE\n",
    "                                clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))\n",
    "                                equalized_plate = clahe.apply(gray_plate)\n",
    "\n",
    "                                # Denoise the image\n",
    "                                denoised_plate = cv2.fastNlMeansDenoising(equalized_plate, None, 10, 7, 21)\n",
    "\n",
    "                                # Adaptive thresholding with adjusted parameters\n",
    "                                thresh_plate = cv2.adaptiveThreshold(\n",
    "                                    denoised_plate, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C,\n",
    "                                    cv2.THRESH_BINARY_INV, 11, 2)\n",
    "\n",
    "                                # Morphological operations\n",
    "                                kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 3))\n",
    "                                morph_plate = cv2.morphologyEx(thresh_plate, cv2.MORPH_CLOSE, kernel)\n",
    "                                morph_plate = cv2.morphologyEx(morph_plate, cv2.MORPH_OPEN, kernel)\n",
    "                                morph_plate = cv2.bitwise_not(morph_plate)\n",
    "\n",
    "                                plate_ocr_results = reader.readtext(morph_plate, allowlist='0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ')\n",
    "                                \n",
    "                                if plate_ocr_results:\n",
    "                                    license_plate_text = plate_ocr_results[0][-2]\n",
    "                                    plate_confidence = round(plate_ocr_results[0][-1], 2)\n",
    "                                    \n",
    "                                    # Check if confidence is above threshold\n",
    "                                    if plate_confidence >= 0.2:\n",
    "                                        # Update the vehicle_plates dictionary\n",
    "                                        if (track_id not in vehicle_plates) or (plate_confidence > vehicle_plates[track_id]['confidence']):\n",
    "                                            vehicle_plates[track_id] = {\n",
    "                                                'plate': license_plate_text,\n",
    "                                                'confidence': plate_confidence\n",
    "                                            }\n",
    "                                            # Save the processed license plate image in /plates folder\n",
    "                                            cv2.imwrite(f'plates/{frame_number}_{track_id}_{license_plate_text}.png', morph_plate)\n",
    "\n",
    "                                        # Save coordinates for CSV logging\n",
    "                                        mx1, my1, mx2, my2 = px1, py1, px2, py2\n",
    "                                    \n",
    "                                assigned_plate = vehicle_plates.get(track_id, None)\n",
    "                                if assigned_plate:\n",
    "                                    # Draw the assigned plate on the frame\n",
    "                                    background_color = (255, 255, 255)  # White background for contrast\n",
    "                                    high_contrast_color = (0, 0, 0)  # Black text\n",
    "                                    put_text(frame, f\"Plate: {assigned_plate['plate']}\", (x1, y2 + 40), color=high_contrast_color, bg_color=background_color)\n",
    "\n",
    "                                    # Update license_plate_text and plate_confidence for CSV logging\n",
    "                                    license_plate_text = assigned_plate['plate']\n",
    "                                    plate_confidence = assigned_plate['confidence']\n",
    "                                else:\n",
    "                                    # If no plate assigned yet, set to empty\n",
    "                                    license_plate_text = \"\"\n",
    "                                    plate_confidence = None\n",
    "                    \n",
    "                    # Draw bounding box and label for the detected object\n",
    "                    color = class_colors.get(cls, (0, 0, 0))\n",
    "                    cv2.rectangle(frame, (x1, y1), (x2, y2), color, 3)\n",
    "                    put_text(frame, f\"{class_names[cls]} {confidence}\", (x1, y1 - 10), color=color)\n",
    "                    put_text(frame, f\"ID: {track_id}\", (x1, y2 + 20), color=color)\n",
    "                                    \n",
    "                    # Anonimización condicional de personas\n",
    "                    if class_names[cls] == \"person\" and blur_enabled:\n",
    "                        person_roi = frame[y1:y2, x1:x2]\n",
    "                        blurred_person = cv2.GaussianBlur(person_roi, (51, 51), 30)\n",
    "                        frame[y1:y2, x1:x2] = blurred_person\n",
    "                        \n",
    "                    # Write to CSV\n",
    "                    with open(csv_file_path, mode='a', newline='') as file:\n",
    "                        writer = csv.writer(file)\n",
    "                        writer.writerow([frame_number, class_names[cls], confidence, track_id, x1, y1, x2, y2,\n",
    "                                        plate_confidence, mx1, my1, mx2, my2, license_plate_text])\n",
    "\n",
    "                    current_frame_count[class_names[cls]] += 1\n",
    "\n",
    "        # Display counts and FPS\n",
    "        y_offset = 30\n",
    "        for cls, count in total_class_count.items():\n",
    "            put_text(frame, f\"Total {cls}: {count}\", (10, y_offset))\n",
    "            y_offset += 20\n",
    "\n",
    "        for cls, count in current_frame_count.items():\n",
    "            put_text(frame, f\"Frame {cls}: {count}\", (10, y_offset), color=(255, 255, 255))\n",
    "            y_offset += 20\n",
    "\n",
    "        fps_calc = 1.0 / (time.time() - start_time)\n",
    "        put_text(frame, f\"FPS: {fps_calc:.2f}\", (10, y_offset), color=(255, 255, 255))\n",
    "\n",
    "        # Write frame to output video\n",
    "        out.write(frame)\n",
    "\n",
    "    # Optionally display the frame\n",
    "    if show_video:\n",
    "        cv2.imshow('Detection and Tracking', frame)\n",
    "        key = cv2.waitKey(1 if not paused else 0) & 0xFF\n",
    "        if key == 27: # Tecla Esc\n",
    "            break\n",
    "        elif key == ord(' '):  # Tecla Espacio\n",
    "            paused = not paused\n",
    "        elif key == ord('b'):  # Tecla para alternar desenfoque\n",
    "            blur_enabled = not blur_enabled  # Cambia el estado de desenfoque\n",
    "            print(f\"Desenfoque {'habilitado' if blur_enabled else 'deshabilitado'}\")\n",
    "        \n",
    "# Release resources\n",
    "cap.release()\n",
    "out.release()\n",
    "cv2.destroyAllWindows()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Resultados\n",
    "\n",
    "Esta sección se presentan los resultados obtenidos. Cargaremos el archivo CSV para revisar el recuento total de cada tipo de objeto detectado, así como los detalles de las detecciones de matrículas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Cargar el archivo CSV de resultados\n",
    "import pandas as pd\n",
    "\n",
    "results_df = pd.read_csv('detection_tracking_log.csv')\n",
    "print(\"Resumen de detecciones por clase:\")\n",
    "print(results_df['object_type'].value_counts())\n",
    "\n",
    "print(\"\\nEjemplo de datos de detección de matrículas:\")\n",
    "display(results_df[results_df['object_type'] == 'car'].head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusión\n",
    "\n",
    "En esta práctica se ha desarrollado un prototipo funcional que permite:\n",
    "\n",
    "- Detectar y seguir personas y vehículos en video.\n",
    "- Detectar y leer matrículas en vehículos mediante un modelo YOLO y OCR.\n",
    "- Exportar los resultados visuales en un video y los datos de detección en un archivo CSV.\n",
    "\n",
    "Este prototipo constituye una herramienta útil para el análisis automatizado de video en aplicaciones de monitoreo y seguridad, con posibilidad de mejoras futuras en el rendimiento y precisión del OCR de matrículas."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "VC_P1",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}