Skip to content

Latest commit

 

History

History
50 lines (44 loc) · 4.31 KB

README.md

File metadata and controls

50 lines (44 loc) · 4.31 KB

XQA - XML Quality Assurance CircleCI

1. Introduction

  • XQA is a suite of Docker based microservices that improves the scalability of BaseX - an XML database engine.
  • Instead of loading each XML file into a single BaseX engine, XQA distributes them - via an AMQP message broker - across multiple BaseX engines.
  • XQA ships with a basic web frontend - as well as a REST API - which lets you run XQuery, and analytics, against the distributed data.

2. High Level Design

High Level Design

3. Key Features

  • Easy to deploy / extend:
    • each microservice runs in its own container.
    • add extra BaseX engines easily - they self register against the AMQP message broker.
    • can run in either a docker-compose or k8s environment.
  • Proven scalability & performance improvements - graphs in xqa-perf show:
    • ingest timing statistics.
    • XML file distribution.
  • Quality Radiatior for CI; static analysis and coverage metrics.
  • Transparency:
    • container console logging.
    • JSON instrumentation sent to a central PstgreSQL instance.

4. Open Source Technologies

  • ActiveMQ
  • Angular
  • Docker - containers can be built from GitHub or pull'd from hub.docker.com.
  • Java 10/11.
  • PostgreSQL 11
  • Python 3.6
  • Ubuntu 18.04

5. Microservices

microservices

CI GitHub
Build Status xqa-commons-qpid-jms a Maven Central shared library.
Build Status xqa-db PostgreSQL.
Build Status xqa-db-amqp AMQP interface to PostgresSQL.
Build Status xqa-ingest XML file loader.
Build Status xqa-ingest-balancer distributes XML across XQA BaseX engine(s).
Build Status xqa-message-broker ActiveMQ.
Build Status xqa-perf end to end integration tests, with Matplotlib graphs.
Build Status xqa-query-balancer XQA REST API.
Build Status xqa-query-ui UI for querying XQA.
Build Status xqa-shard BaseX engine with AMQP interface.
xqa-test-data a collection of XML files used by XQA.

6. Limitatons

XQA is a proof of concept project. It scratched an itch and achieved what it set out to prove. Refer to the GitHub issue board for outstanding issues.