Skip to content

Latest commit

 

History

History
87 lines (64 loc) · 4.46 KB

README.md

File metadata and controls

87 lines (64 loc) · 4.46 KB

SARC Round-2 FAQ Portal : Team Mars

Project Overview

This project implements a question-answering system using Natural Language Processing (NLP) techniques, specifically the TF-IDF (Term Frequency-Inverse Document Frequency) method, combined with cosine similarity. The system answers user queries by finding the most similar pre-defined question and providing the corresponding answer. The project has the following components:

  • A backend built with Flask that processes and handles API requests.
  • A front-end website hosted on GitHub Pages to interact with the user.
  • The API is deployed on Render for production usage.

How to use

  • Live Version: You can access the live version of the project through this link:
    FAQ Portal - Team Mars

  • To use this code locally:

    1. Clone the Repository:
      Run the following command in your terminal:

      git clone https://github.com/madhav64op/FAQ_SARC_Team_Mass.git

    2. Navigate to the Project Directory:

      cd FAQ_SARC_Team_Mass

    3. Open the Project in Visual Studio Code:
      Open Visual Studio Code and select File > Open Folder, then choose the FAQ_SARC_Team_Mass folder.

    4. Run the Live Server:

      • Install the Live Server extension from the VS Code marketplace if you haven't already.
      • Once installed, right-click on the index.html file in the Explorer panel and select Open with Live Server.

    5. Access the Application:
      The application will open in your default web browser at http://127.0.0.1:5500/ (or another port as specified by Live Server).

Now, you can interact with the FAQ Portal locally!

What we did

  1. Text Preprocessing and Vectorization:

    • The project utilizes TfidfVectorizer from sklearn to convert input questions into vector representations.

  2. Cosine Similarity:

    • To determine the similarity between user-provided questions and the dataset, cosine_similarity is applied to find the most relevant answer.

  3. Flask-based API:

    • The Flask framework processes API requests and returns the most similar question-answer pair.

  4. Cross-Origin Resource Sharing (CORS):

    • Flask-CORS enables the front-end on GitHub Pages to interact with the backend API hosted on Render. We encountered some issues with CORS licenses while integrating it with our React front-end.

  5. React Integration:

    • We tried the front-end using React to create a user-friendly interface for submitting questions and receiving answers but faced issues with the API.

  6. RNN-Bi-directional LSTM Model:

    • Initially, we also tried to implement an RNN-Bi-directional LSTM model for more advanced question-answering capabilities. However, we faced several challenges, including model training issues and performance constraints, which led us to adopt the current approach using TF-IDF and cosine similarity.

System Workflow

  1. Input: The user submits a question through the front-end interface.
  2. Backend Processing:
    • The submitted question is vectorized using TfidfVectorizer.
    • The cosine similarity between the input question and predefined questions is computed.
    • The most similar question and its corresponding answer are retrieved.
  3. Output: The backend sends the most similar question and its answer as a JSON response to the front-end, which displays it to the user.

Code Structure

Backend (Flask-based API)

  • File: api.py
    • Libraries Used:
      • pandas: For handling the dataset in a tabular format.
      • TfidfVectorizer: To vectorize text questions.
      • cosine_similarity: To calculate the similarity between user input and the dataset questions.
      • Flask, CORS: To handle API requests and manage cross-origin resource sharing.
    • Key Functions:
      • json_to_df: Converts a JSON object to a Pandas DataFrame.
      • find_most_similar_question_with_answer: Computes cosine similarity and returns the most similar question-answer pair.
      • /predict: Flask route to process POST requests containing a question and return the predicted answer.
    • Deployment: Hosted on Render platform.

Front-End

  • Website: The front-end interface is a simple website that allows users to input their questions. It sends the question to the Flask API and displays the answer on the page.
  • Deployment: The website is hosted on GitHub Pages.