🏗️ nptmpl Architecture Guide

This document provides a technical overview of the nptmpl architecture, including component diagrams and sequence diagrams for key operations. It is intended for developers contributing to or maintaining the project.

System Overview

nptmpl is composed of three major subsystems:

CLI — The user-facing command-line interface, implemented using the Command Pattern.
Core Engine — Template compression, decompression, Jinja2 rendering, and metadata management.
Server — A FastAPI-based HTTP registry with a SQLite backend, Jinja2-templated Web UI, and WebSocket live updates.

Component Diagram

graph TB
    subgraph CLI["CLI Layer (nptmpl.cli)"]
        APP["CLIApp (Argument Parser)"]
        GEN["GeneratorCommands"]
        REG["RegistryCommands"]
        SYS["SystemCommands"]
        SRV["ServerCommands"]
    end

    subgraph Core["Core Engine (nptmpl.core)"]
        META["MetadataManager"]
        FS["FileSystemEngine"]
        STORE["TemplateStore"]
        CFG["ConfigManager"]
        VER["Version (SemVer)"]
        REMOTE["RemoteTransport (SSH/HTTP)"]
    end

    subgraph Server["Server Layer (nptmpl.server)"]
        MAIN["FastAPI App (main.py)"]
        API["API Router (api.py)"]
        PUB["Public Router (public.py)"]
        ADMIN["Admin Router (admin.py)"]
        DB["DatabaseManager (db.py)"]
        INSP["TarballInspector"]
        WS["WebSocketManager"]
        TMPL["Jinja2 Templates"]
    end

    subgraph Storage["Persistent Storage"]
        SQLITE[("registry.db (SQLite/WAL)")]
        TARBALL["data.tar.gz Archives"]
    end

    APP --> GEN
    APP --> REG
    APP --> SYS
    APP --> SRV

    GEN --> FS
    GEN --> META
    REG --> STORE
    REG --> REMOTE
    SYS --> STORE
    SYS --> CFG
    SRV --> MAIN

    MAIN --> API
    MAIN --> PUB
    MAIN --> ADMIN
    MAIN --> WS

    API --> DB
    API --> INSP
    PUB --> DB
    PUB --> INSP
    ADMIN --> DB

    DB --> SQLITE
    API --> TARBALL
    INSP --> TARBALL
    PUB --> TMPL

    STORE --> FS
    STORE --> META

Sequence Diagram: `nptmpl clone`

Shows how a clone operation works from CLI invocation to rendered output.

sequenceDiagram
    participant User
    participant CLI as CLIApp
    participant REG as RegistryCommands
    participant STORE as TemplateStore
    participant FS as FileSystemEngine
    participant META as MetadataManager

    User->>CLI: nptmpl clone group/name ./dest
    CLI->>REG: clone(args)
    REG->>STORE: clone_template(target, dest, vars)

    alt Remote URL provided
        STORE->>STORE: _fetch_remote(url)
        Note over STORE: Downloads .tar.gz via HTTP/SSH
    else Local registry
        STORE->>STORE: _resolve_local(target)
    end

    STORE->>META: load(archive_path)
    META-->>STORE: TemplateMetadata

    STORE->>FS: decompress(archive, dest)
    Note over FS: Extracts tar.gz with path traversal checks

    STORE->>FS: render_tree(dest, variables)
    Note over FS: Jinja2 renders file contents and filenames

    opt Post-clone hooks defined
        STORE->>FS: run_hooks(dest, hooks)
    end

    STORE-->>REG: Success
    REG-->>CLI: Print result
    CLI-->>User: ✅ Template cloned

Sequence Diagram: `nptmpl push`

Shows the server-side flow when a template is pushed to the HTTP registry.

sequenceDiagram
    participant Client as CLI / curl
    participant API as API Router
    participant Lock as push_lock (Threading)
    participant FS as FileSystemEngine
    participant INSP as TarballInspector
    participant DB as DatabaseManager
    participant WS as WebSocketManager

    Client->>API: POST /api/v1/templates/push
    Note over API: Bearer Token validation

    API->>Lock: acquire()
    Note over Lock: Thread-safe asyncio lock

    API->>API: run_in_threadpool(stream_to_disk)
    Note over API: Streams upload to temp file

    API->>INSP: run_in_threadpool(verify_integrity)
    INSP-->>API: Valid ✓

    API->>INSP: run_in_threadpool(extract_readme)
    INSP-->>API: README content

    API->>FS: run_in_threadpool(move to storage)

    API->>DB: run_in_threadpool(add_template_version)
    DB-->>API: Committed ✓

    API->>Lock: release()

    API->>WS: broadcast(registry_update)
    Note over WS: Notifies all connected browsers

    API-->>Client: 200 OK {"message": "Published"}

Concurrency Model

The server uses a hybrid concurrency architecture:

AsyncIO Event Loop: Handles HTTP request routing, WebSocket connections, and response serialization without blocking.
Thread Pool Offloading: All synchronous operations (SQLite queries, filesystem I/O, tarball inspection) are dispatched via run_in_threadpool to prevent event loop starvation.
SQLite WAL Mode: Write-Ahead Logging enables concurrent reads alongside a single writer, maximizing throughput.
Push Lock: A threading lock (asyncio.Lock bridged to threadpool) serializes write operations to prevent race conditions during template uploads.

Database Schema

CREATE TABLE templates (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    group_name TEXT NOT NULL,
    name TEXT NOT NULL,
    author TEXT NOT NULL,
    email TEXT,
    description TEXT,
    languages TEXT,     -- JSON array
    license TEXT,
    url TEXT,
    demo TEXT,          -- Relative path to demo asset (image/PDF)
    added_date TEXT,
    tags TEXT,           -- JSON array
    download_count INTEGER DEFAULT 0,
    UNIQUE(group_name, name)
);

CREATE TABLE versions (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    template_id INTEGER NOT NULL,
    version TEXT NOT NULL,
    added_date TEXT NOT NULL,
    readme_content TEXT,
    FOREIGN KEY (template_id) REFERENCES templates(id) ON DELETE CASCADE,
    UNIQUE(template_id, version)
);

CLI Command Pattern

The CLI uses a Command Pattern architecture for modularity:

Command Module	Responsibilities
`GeneratorCommands`	`init`, `add` — Template creation and local registry insertion
`RegistryCommands`	`clone`, `push`, `search`, `remote` — Local and remote template operations
`SystemCommands`	`list`, `detail`, `remove`, `update`, `doctor`, `config`, `path`, `reindex` — System introspection and maintenance
`ServerCommands`	`server`, `ui` — HTTP registry server lifecycle

All command modules extend BaseCommand, which provides shared access to the TemplateStore, ConfigManager, and authentication utilities.