Data Platform Overview

The Angelis data strategy is built around two clearly separated universes: Transactional and Analytics. They are independent by design — each optimized for its own workload — and connected through a minimal, well-defined interface.

┌─────────────────────────────────┐     ┌──────────────────────────────────────┐
│       TRANSACTIONAL UNIVERSE    │     │          ANALYTICS UNIVERSE          │
│                                 │     │                                      │
│  Microservices + Databases      │────▶│  Data Lake  →  Consumption  →  Viz   │
│  (source of truth)              │     │  (analytical copies, never write)    │
└─────────────────────────────────┘     └──────────────────────────────────────┘

Transactional systems own the data. The analytics universe holds read-only copies, structured for querying and visualization. No analytics tool ever writes back to a transactional database.

Architecture

Five-layer pipeline from transactional DBs through ADLS, Dremio, and Superset. Read more →

Integration Guide

Step-by-step guide for teams adding new data to the analytics universe. Read more →

---

Transactional Persistence

The source of all data. Each microservice owns its database and is the sole writer to it.

Workforce Management  ──▶  PostgreSQL
Identity & Access     ──▶  PostgreSQL (CIAM)
Safety & Compliance   ──▶  MongoDB
SaferApp (external)   ──▶  MSSQL       (read-only integration, not owned by Angelis)
Fit2000 (external)    ──▶  MySQL        (read-only integration, not owned by Angelis)

Nothing outside a service’s own application code writes to its database. Analytics tools query from the Data Lake onwards, never directly from transactional stores.

PostgreSQL (Main Application Database)

Purpose: Primary business data storage
Contains:
    - Companies, Projects, Assignments
    - Profiles (Workers), ProfileCompany relationships
    - Internal Users (Admins, Managers)
    - Licenses, Subscriptions, Offerings
    - Notifications
    - Audit Logs
    - Roles and Permissions

Connection Management:
    - SQLAlchemy ORM with connection pooling
    - Environment: ANGELIS_POSTGRES_DB_HOST, ANGELIS_POSTGRES_DB_NAME
    - Pool size: 10 (configurable)
    - Max overflow: 20
    - Pool pre-ping: Enabled for health checks

MSSQL (SaferApp Test Results)

Purpose: External test results database
Contains:
    - SaferApp test results (bi_saferapp table)
    - Test dates, results, worker RUTs
    - Faena (project) associations

Connection Management:
    - pyodbc with ODBC Driver 18 for SQL Server
    - Connection timeout: 30 seconds (configurable)
    - Environment: ANGELIS_SQLSERVER_DB_HOST, ANGELIS_SQLSERVER_DB_NAME
    - TrustServerCertificate: Yes
    - Pool pre-ping: Enabled

MySQL (Fit2000 Test Results)

Purpose: External Fit2000 test results
Contains:
    - Fit2000 test results (ktest table)
    - User-faena relationships
    - Test dates and results

Connection Management:
    - SQLAlchemy with MySQL driver
    - Environment: ANGELIS_FIT2000_DB_HOST, ANGELIS_FIT2000_DB_NAME
    - Connection pooling enabled

Redis (Caching & Sessions)

Purpose: Caching and session management
Usage:
    - Session storage
    - Frequently accessed data caching
    - Notification connection management
    - Rate limiting (future)

Configuration:
    - Environment: ANGELIS_REDIS_URL
    - Connection pooling

MongoDB (Checkingmate / Safety & Compliance)

The Safety & Compliance domain (Checkingmate Backend) uses MongoDB as its primary data store for safety forms, templates, anomaly records, and compliance data.

---

Data Consistency Strategy

ACID Compliance Areas

Critical Operations Requiring ACID:
    - User authentication and authorization
    - Company and project creation
    - Assignment management
    - License and subscription management
    - Notification persistence
    - Audit log entries

Eventual Consistency Areas

Operations Accepting Eventual Consistency:
    - Test result aggregation from external databases
    - Report generation and Azure upload
    - Notification delivery (SSE)
    - Dashboard analytics
    - Background task processing

---

Database Relationships

erDiagram
    Company ||--o{ InternalUser : has
    Company ||--o{ Project : has
    Company ||--o{ Subscription : has
    Company ||--o{ ProfileCompany : employs

    Project ||--o{ Assignment : contains
    Assignment ||--o{ ProfileAssignment : links

    Profile ||--o{ ProfileCompany : belongs_to
    Profile ||--o{ License : has
    Profile ||--o{ ProfileAssignment : assigned_to

    ProfileCompany }o--|| Profile : "reporting_head (supervisor)"

    InternalUser ||--o{ AuditLog : creates
    InternalUser ||--o{ Profile : creates

    Subscription }o--|| Offering : subscribes_to

    ProfileCompany }o--|| Company : works_for
    ProfileCompany }o--|| Profile : is

---

Technology Stack Summary

Database	Technology	Purpose
PostgreSQL	SQLAlchemy 2.0 / Alembic	Main application data
MSSQL	pyodbc (ODBC Driver 18)	SaferApp test results
MySQL	SQLAlchemy + MySQL driver	Fit2000 test results
Redis	Redis client	OTP, caching, sessions
MongoDB	(Checkingmate Backend)	Safety forms, templates, anomaly data
Azure Blob Storage	SAS URLs	Report files (PDF/CSV), profile images