Smart Gas Meter Reader

Table of Contents

• Overview
• Programming the ESP32-CAM
• Setting up the server
• Creating training datasets for the models
• Training the Object Detector
• Training the Image Classifier Model
• Steps of Prediction
  • Balancing
  • Dial-plate Detection
  • Sharpening and Resizing
  • Applying Adaptive Threshold Algorithm
  • Contour Searching on Threshold Image
  • Classify the Image pieces
• Results in flask web app

Overview

• The ESP-CAM captures images of an analog gas meter.
• The camera communicates with a Linux server via MQTT at regular intervals.
  • The server instructs the camera to take a photo.
  • The camera sends the photo to the server.
  • The server processes the image.
  • the server sends a sleep command to the camera.
• The server processes the image
  • OpenCV is used to preprocess the image.
  • TensorFlow Lite model detects the dial plate's location.
  • The dial plate is cut into 8 pieces.
  • A simple CNN model classifies each piece.

Programming the ESP32-CAM

Esp32-cam setup readme

Setting up the server

Server setup readme

Creating training datasets for the models

1. Object Detection (Tensorflow Lite - EfficientDet)

   • The dataset is labeled using the labelImg tool.

   

2. Image Classification (Tensorflow - CNN)

   • The images ( ) have to be sorted into folders, where each folder represents a class. The dataset may be combined with the MNIST dataset to increase the number of training samples.
   • The folder structure:

     ├── 0
     │   ├── 0_1.jpg
     │   ├── 0_2.jpg
     │   ├── ...
     ├── ...
     ├── 9
     │   ├── 9_1.jpg
     │   ├── 9_2.jpg
     │   ├── ...
     

Training the Object Detector

1. Set Up Google Colab:

   • Open the Model Maker Object Detection for Android Figurine notebook in Google Colab.
   • Some changes are needed to train the model with the custom dataset.

2. Prepare the Dataset:

   • Upload the labeled dataset to Google Colab.

3. Train the Model:

   • Follow the steps in the Colab notebook to train the object detection model.
   • Download the trained TensorFlow Lite model.

Training the Image Classifier Model

Use the train/number_classification/train_and_eval.ipynb notebook to train the model.

Steps of prediction:

1. Balancing

   If the image is tilted, the object detector marks a bigger area. Therefor it is important to balance the images. For this task, Hugh lines algorithm is used.

2. Dial-plate detection

3. Sharpening and resizing

   The detected images are resized to 140x1000 pixels.

4. Applying Adaptive threshold algorithm

5. Contour searching on threshold image

   To find the coordinates of the numbers we utilize the contours of the threshold image. On the basis of the found contours' coordinates, the 140x1000 px images are cut into 8 pieces.

6. Classify the image pieces

   A tensorflow CNN model classifies the images into 10 classes.

7. Results

   The sharpened image of the dial plate is saved in the /results folder with the filename constructed from a sequential number, the predictions and the timestamp.
   Pattern: <seq_num>_<first_5_nums>_<last_3_nums>_<timestamp>.jpg
   Example: 8_13678_902_1668970881.jpg

Results in flask web app

The images from the /results folder are copied to the flask app's static/images folder. The images are displayed on the web page.