Drive Link For all Detailed Demonstration Videos\
Detailed Drive link for videos, sample dataset and custom labelled images
Colab File Link for model inference and named entity recognition
Table of Contents
We have developed a web application that takes doctor's prescription image as input and provides important information such as medicines prescribed, dosage, frequency, diagnostic test and doctor's details. During, the process the user is asked to validate the predictions. The validated data is stored in the database and our prediction model is trained using the stored data in order to fine tune for doctor's prescriptions.
- We considered employing two machine learning models to solve this issue.
- The first model, which extracts text from the prescription image, is an optical character recognition model.
- The required entities in the text that will be shown in the user interface of our web application are then located and classified by a named entity recognition model, which receives the extracted text as its input.
- The client will have the option to validate our model's predictions if our models are confident about the predictions, or alternatively the same prescription image will be given to a network of clients who will annotate it.
- Our models will be retrained after a predetermined time using the annotated data that will be saved in the database.
To test the web application, you need to create a virtual environment and install the dependencies.
To test the web application, follow the instructions below and install the prerequisites.
Install Anaconda Distribution
Open Anaconda Prompt and Update conda environment
conda update conda
Set up a virtual environment
conda create -n venv python=3.8
Install dependencies in the virtual environment
pip install -r requirements.txt
Migrate Database and Run Server
python manage.py makemigrations
python manage.py migrate
python manage.py runserver
- Handwritten prescription Digitizer - All data points as follows will be extracted from from handwritten prescriptions available in variety of formats and compiled into a digital prescription in a common format . Printed data such as doctor's details will be identified from the prescription pad.
- Prescription Review Network - When a user uploads a photo of a prescription, our model will predict the contents with a certain confidence. If the confidence falls below a threshold the prescription should be sent to a network of pharmacists.
- Prescription Annotator - We intend to offer an interactive UI tool that will allow users to quickly and conveniently correct model predictions and prescription labels.
- Pharmacist Dashboard - A pharmacist profile containing statistics like number of patients served and successful contributions made for the "PRESCRIPTION REVIEWER". Along with that they can also view their overall performance.
Users can upload a prescription
Can view and manage all the uploaded prescriptions
Can edit the labels generated by the models and verify the accuracy
Can use the integrated labelling tool for labelling the data or correct the output of the model.
Extraction of
- Medicine
- Dosage
- Frequency
A interactive cross platform application where a user can manage, edit, annotate the prescriptions, custom train the model to improvise the accuracy
Output
Text Detection (ROI-Region of Interest)
Text Extraction
OUTPUT FORMAT -> Text, Bouding Box Coordinates, Confidence of each detection
\
\
Medcoder.mp4
Drive Link For all Detailed Demonstration Videos\
Detailed Drive link for videos, sample dataset and custom labelled images
Colab File Link for model inference and named entity recognition