SmartShield Backend

The SmartShield Backend manages the storage and distribution of results generated by our models.

Architecture

The backend is built using Nx, pnpm, NestJS, and Docker, each chosen for their strengths in managing dependencies, ensuring scalability, and supporting robust development.

• NestJS is a trusted framework for high-traffic, high-demand systems, known for its focus on scalability, security, and robustness—making it ideal for developing resilient applications.
• Nx simplifies dependency management, enhances build performance, and provides tools for scaling, testing, and organizing code, which is especially beneficial as the project grows.
• pnpm improves dependency management by reducing disk usage, accelerating installation speeds, and ensuring consistent dependency versions, creating a more efficient development experience.
• Docker enables consistent, reproducible deployments by packaging the application and its dependencies in isolated containers, making scaling and deployment across environments straightforward.

Services Managed by Docker

The Docker environment includes Redis and PostgreSQL services, each selected for specific purposes within the backend system.

• Redis is used for fast data retrieval:
  • Unregistered servers register with the backend to receive a unique ID, linked with their MAC address for easy identification.
  • Redis stores these server IDs, allowing for rapid access to relevant data associated with each server.
• PostgreSQL serves as a durable storage solution for archiving logs and annotations. This data supports continuous model training and improvements, while also storing essential data across the system.

Usage Documentation

Prerequisites

• Ensure Docker and Docker Compose are installed.
• Install pnpm globally if not already installed ( for Dev Only ).

Production Setup

1. Copy and Configure Environment Variables

   cp .env.prod.example .env.prod

   Fill in any necessary values in .env.prod.

2. Build and Run Containers

   docker-compose -f docker-compose.prod.yml up --build -d

3. Access the Application

   • The backend API is available at http://localhost:3000/api.

Development Setup

1. Copy and Configure Environment Variables

   cp .env.dev.example .env.dev

   Fill in any necessary values in .env.dev.

2. Build and Run Containers: Postgres DB and Redis

   docker-compose -f docker-compose.dev.yml up -d

3. Install Dependencies and Run NestJS Backend

   pnpm install
   pnpm nx serve backend

4. Access the Application

   • The backend API is available at http://localhost:3000/api.

Populating the Database

In order to test the backend and also the React Dashboard, we can use synthetic data.

This data should be random and should simulate real-time data stream.

First, you should create a user:

• Send a POST request to http://{{HOST_NAME}}/api/user with this format:

{
   "name": "Abdou",
   "email": "abdou@cs.ieee.insat.tn"
}"

• Use the UUID from the response from the server and copy it to ./populating db/generate_servers.py in the user_uuid variable.
• Run the command: cd populating\ db/ to get to the folder.
• Run the command: python3 generate_servers.py this will create 20 server instances simulating the servers that are within the organization and should have subscribed to the server in order to get an id.
• The previous script will collect the ids of the created servers returned from the servers. Copy this array into ./populating db/generate_data.py in the servers variable.
• Run the command: python3 generate_data.py this will generate data in real time while sleeping for some time in between to simulate real-time data generation.