Files
pfm/Architecture.md
T
2026-10-28 09:28:06 +02:00

26 KiB

PostFundManagement (PFM) Platform — Architectural Specification

Document Version: 2.0
Target Runtimes: .NET 10.0 (C# 13) | React 19 / TypeScript 5.8 / Vite
Scope: System Architecture, Security Topology, RBAC Enforcement, and Maker-Checker Pattern


1. System Topology & Architectural Layers

The PFM platform is constructed using Clean Architecture and Domain-Driven Design (DDD) principles to guarantee strict decoupling of business rules, persistence concerns, and user interfaces.

┌────────────────────────────────────────────────────────────────────────┐
│                          PRESENTATION TIER                             │
│  PostFundManagement.ConsoleUI      │  PostFundManagement.PartnerUI     │
│  (Internal Staff & Governance)     │  (External Recipient Portal)      │
│  React 19 + TypeScript + Vite      │  React 19 + WebGL (OGL) + Vite    │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │ HTTPS / REST (v1/api/*) + OIDC PKCE
┌───────────────────────────────────▼────────────────────────────────────┐
│                             API GATEWAY                                │
│  PostFundManagement.Api (ASP.NET Core Minimal APIs / Carter)           │
│  Authentication: JWT Bearer + OpenTelemetry Observability              │
│  Quartz Client: Scheduler-Only (ZeroSizeThreadPool / Enqueues Jobs)    │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │
┌───────────────────────────────────▼────────────────────────────────────┐
│                          APPLICATION LAYER                             │
│  PostFundManagement.Application                                        │
│  CQRS (Mediator), Business Validators, Domain Services, Event Handlers │
└──────────────────┬─────────────────────────────────┬───────────────────┘
                   │                                 │
┌──────────────────▼──────────────────┐   ┌──────────▼───────────────────┐
│            DOMAIN LAYER             │   │    INFRASTRUCTURE LAYER      │
│  PostFundManagement.Domain          │   │  PostFundManagement.Infrastr.│
│  Entities, Value Objects, Enums,    │   │  PostgreSQL (EF Core 10),    │
│  Domain Invariants, Audit Models    │   │  Skoruba STS, S3 Garage,     │
└─────────────────────────────────────┘   │  Redis, Quartz (pfm-scheduler)│
                                          └──────────────┬───────────────┘
                                                         │
┌────────────────────────────────────────────────────────▼───────────────┐
│                     BACKGROUND JOBS WORKER HOST                        │
│  PostFundManagement.Jobs (.NET 10 Background Host / Quartz Runner)     │
│  Recurring Job Registration (JobsWorker) & Mediated Execution Engine   │
│  Quartz Runner: Full Execution (DefaultThreadPool MaxConcurrency = 5)  │
└────────────────────────────────────────────────────────────────────────┘

2. Security & Identity Architecture

Authentication and authorization are separated cleanly across system boundaries:

  • Security Token Service (STS): Duende IdentityServer / Skoruba Administration (https://sts.security.khongisa.co.za) issues cryptographically signed JWT access tokens using OpenID Connect (OIDC) with Proof Key for Code Exchange (PKCE).
  • Audience & Scopes: Web clients request standard OIDC scopes (openid, profile, roles, email, address) along with the target API resource scope (pfm-api-dev).
  • Tenant Isolation: Every authenticated principal carries a tenant_id claim. Multitenant query filters in EF Core enforce cross-tenant partition boundaries at the data access layer.
  • Partner Isolation: Grantees and external recipients possess the partner role. The ConsoleUI enforces immediate lockout for partner roles, restricting external users exclusively to PartnerUI.

3. Role-Based Access Control (RBAC) Architecture

PFM implements a fine-grained, claim-based RBAC model where system permissions are decomposed into 15 operational domains and 5 progressive action capabilities.

3.1 Role Claims Inventory

Claim Key Operational Domain Functional Scope
dashboard Executive Dashboard Macro portfolio KPIs, disbursed capital, and ISO 31000 risk heatmap
governance Governance & Structure Holding entities, funder portfolios, programmes, rules, and instruments
projects_milestones Projects & Milestones Work breakdown structure (WBS), contractual milestones, and activities
grant_financials Grants & Financials Statutory funding awards, budget lines, contracts, and disbursement tranches
evidence_risks M&E, Evidence & Risks Deliverable evidence verification queue, indicators, and risk registers
beneficiaries Beneficiaries & Impact Demographic targets, POPIA compliance items, and outcome deliverables
change_management Change Management Project issues, contractual change requests, and budget variations
site_visits Site Visits Physical field inspections, checklist observations, and inspector findings
settings_invites Settings Invites Onboarding invitations and candidate notifications dispatch
settings_users Settings Users User provisioning, role assignments, and account lockouts
settings_auditlog Settings Audit Log Immutable audit ledger and system telemetry records
settings_notifications Settings Notifications SMTP engines, notification channels, and dispatch telemetry
tenant_self_service Tenant Self-Service Tenant profile configuration and delegated sub-user management
tenant_management Tenant Management Multi-tenant directory, partition allocation, and solution ownership
unrestricted System Administration Root administrative bypass across all system partitions

3.2 Claim Actions & Capability Hierarchy

Action capabilities within a claim domain follow a strict subset hierarchy:

\text{view} \subset \text{add} \subset \text{update} \quad \text{and} \quad \text{view} \subset \text{export}
  • view: Read-only access to entity lists, detailed records, metrics, and progress scores.
  • add: Can create new records, edit draft records, submit milestone evidence, and change workflow statuses other than approval (includes view).
  • update: Authoritative governance capability. Can execute approval status transitions (Approved, Active contracts) on records (includes view and add).
  • export: Can generate and download structured spreadsheets (.xlsx) and formal PDF reports (includes view).
  • all: Comprehensive authority (view, add, update, export) over the claim domain.

3.3 Authoritative Role-to-Claims Matrix

Role Assigned Claim Domains Standard Capabilities Record Approval Authority (update)
Admin unrestricted all Yes (System-wide)
Executive dashboard, governance, projects_milestones, grant_financials, evidence_risks, tenant_self_service all Yes
Manager dashboard, projects_milestones, grant_financials, evidence_risks, beneficiaries, change_management, site_visits, settings_invites, settings_users, settings_auditlog, settings_notifications, tenant_self_service all Yes
Supervisor dashboard, projects_milestones, grant_financials, evidence_risks, beneficiaries, change_management, site_visits, settings_invites, settings_users view, add, update, export
(Dashboard: view, export)
Yes
Contributor projects_milestones, grant_financials, evidence_risks, beneficiaries, change_management, site_visits, settings_invites view, add ❌ Strictly Prohibited

4. Maker-Checker Pattern Architecture

To eliminate fraud risk, self-dealing, and conflicts of interest, PFM enforces the Maker-Checker Pattern (Segregation of Duties / Dual Authorization).

4.1 Core Principle

The user who creates or drafts a record (the "Maker") is strictly prohibited from verifying or approving that record (the "Checker"), regardless of their administrative role or RBAC claim privileges.

Even if an Executive, Manager, or Supervisor possesses the update claim action on a domain, they cannot approve a record if they are identified as the record's creator.

flowchart LR
    A["Maker (User A)<br/>Creates Record"] --> B["Record Persisted<br/>with createdBy = User A"]
    B --> C{"Checker Evaluation<br/>(Approver Login)"}
    C -->|"Checker is User A"| D["Approval BLOCKED: Maker-Checker Violation"]
    C -->|"Checker is User B with update claim"| E["Approval GRANTED: Status Approved"]

4.2 Record Attribution Model

Every operational domain entity immutably captures the creator identity:

  • createdBy: Cryptographic user identifier (Guid / Subject ID) populated from the verified token claim context.GetUserId() upon creation.
  • approvedBy: User identifier of the independent verifier populated upon successful transition to Approved.

4.3 Client-Side Enforcement Hooks & Helpers

In PostFundManagement.ConsoleUI, the Maker-Checker rule is evaluated dynamically via useRbac():

  • isMaker(createdBy): Compares createdBy against current user user.id, sub, and user.email.
  • canApproveRecord(claim, createdBy): Evaluates: \text{canApproveRecord} = \text{canApprove}(\text{claim}) \land \neg\text{isMaker}(\text{createdBy})
  • getApprovalRestrictionReason(claim, createdBy): Surfaces explanatory feedback:
    • "Requires update/approval permission" if the claim lacks update.
    • "Maker-Checker policy: Record creators cannot approve their own records" if isMaker is true.

4.4 UI Guard Implementations

  1. Status Dropdowns: The Approved option in status <select> controls is disabled with contextual labels:
    <option value={ApprovalStatus.Approved} disabled={!canApproveRecord('projects_milestones', record.createdBy)}>
      Approved {isMaker(record.createdBy) ? '(Maker-Checker: Cannot approve own record)' : ''}
    </option>
    
  2. Action Buttons: Fast-track "Approve" buttons (e.g. Budget Approval, Disbursement Authorisation, Evidence Review) are hidden or accompanied by a Maker-Checker indicator badge when viewed by the creator.
  3. Export Controls: <ExportButton> enforces domain claim checks explicitly (claim="grant_financials"), rejecting unauthenticated exports.

5. Caching Strategy, Mutation Versioning & Performance Optimization

To achieve sub-50ms page transitions while guaranteeing data freshness across mutations in a distributed architecture, PFM employs a multi-tiered caching topology.

┌────────────────────────────────────────────────────────────────────────┐
│ REACT QUERY CACHE (In-Memory Browser Tier)                             │
│ staleTime: 2m (dynamic) | 5m (taxonomy) | gcTime: 10m                  │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │ Cache Miss / Background Refetch
┌───────────────────────────────────▼────────────────────────────────────┐
│ AXIOS CLIENT INTERCEPTOR & MUTATION VERSIONING (_v)                     │
│ Reads/Browsing: attaches active `_v` epoch token                       │
│ Mutations (POST/PUT/PATCH/DELETE): auto-bumps epoch -> new `_v` token  │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │ HTTP GET /v1/api/*?_v=<epoch>
┌───────────────────────────────────▼────────────────────────────────────┐
│ ASP.NET CORE OUTPUTCACHE (Redis / Memory Tier)                         │
│ Base Policy: 2-minute expiration partitioned by URI + QueryString       │
│ Hits same `_v` -> 2ms response | New `_v` -> DB query & cache refresh  │
└────────────────────────────────────────────────────────────────────────┘

5.1 The OutputCache Stale Data Challenge

ASP.NET Core Minimal APIs in PostFundManagement.Api attach .CacheOutput() to GET endpoints, using a 2-minute base sliding expiration backed by Redis / Memory cache.

Because OutputCache keys are generated from the HTTP Method, Host, Path, and Query String, a naive client refetch after a mutation (e.g. queryClient.invalidateQueries({ queryKey: ['projects'] })) calls GET /v1/api/projects. OutputCache intercepts this identical URI, returning the pre-mutation response cached earlier. The updated project status or approved milestone does not reflect in the UI until the 2-minute cache expires.

5.2 Client-Side Mutation Epoch Versioning

PFM solves this without modifying backend endpoints via Client-Side Mutation Versioning:

  1. Epoch Tracking (activeMutationEpoch): httpClient.ts tracks an active mutation timestamp initialized from sessionStorage or Date.now().
  2. Automatic Mutation Interceptor: Every successful mutating HTTP response (POST, PUT, PATCH, DELETE with HTTP 2xx) triggers bumpMutationEpoch().
  3. Query Parameter Injection: Outgoing GET requests to internal APIs automatically append _v: getMutationEpoch().
  4. Instant Invalidation: When a mutation succeeds, the epoch is bumped. Subsequent React Query refetches send the new _v, producing an immediate cache miss in ASP.NET Core OutputCache. The backend queries PostgreSQL, returns fresh data, and caches the new state under the new _v key.
  5. Cache Hit Snappiness: Navigations between pages while no mutations have occurred reuse the stable _v, yielding sub-5ms responses from Redis/Memory OutputCache and 0ms from React Query in-memory cache.
  6. Explicit Freshness (_t): When Cache-Control: no-cache is specified or during explicit refresh operations, a unique timestamp _t is attached to force a guaranteed fresh fetch.

5.3 RBAC Startup Pipeline Acceleration

Following the introduction of fine-grained RBAC and Maker-Checker checks, startup performance was streamlined to eliminate sequential network waterfalls:

  • Cache-First Role Claims: fetchAndCacheSystemRoles and fetchAndCacheRoleClaims check in-memory and localStorage caches before dispatching network calls.
  • Parallel User Resolution: getUserRolesFromApi and getUserClaimsFromApi are resolved concurrently using Promise.allSettled.
  • Parallel Claim Resolution: Claims across all assigned roles are retrieved concurrently rather than in serial loops.
  • Immediate Non-Blocking Session Render: If valid session tokens and user profiles exist in localStorage, AuthProvider immediately unblocks UI rendering (isLoading: false) and runs tenant lockout checks, account synchronization, and claim revalidation asynchronously in the background.
  • In-Memory User Sync & Lockout Caching: isUserSyncedInBrowser and verifyTenantLockout maintain in-memory references to eliminate synchronous JSON.parse operations from the request pipeline.

6. Distributed Background Job Processing Architecture (PostFundManagement.Jobs)

The PFM platform decouples synchronous API request handling from asynchronous, long-running, and scheduled workloads using a distributed Quartz.NET (v4.6+) clustering topology backed by PostgreSQL (pfm-scheduler).

6.1 Split-Node Topology: Scheduler Client vs. Execution Host

To maintain high API throughput, low memory footprint, and complete process isolation, PFM implements a split-node architecture between PostFundManagement.Api and PostFundManagement.Jobs:

┌──────────────────────────────────────────────┐
│          PostFundManagement.Api              │
│       (API Gateway / Scheduler Client)       │
│                                              │
│  * AddQuartzSchedulerClient()                │
│  * ZeroSizeThreadPool (0 execution threads)  │
│  * SchedulerStatus.Created (Standby mode)    │
│  * Strictly schedules / enqueues events      │
│  * Never acquires triggers or executes jobs  │
└──────────────────────┬───────────────────────┘
                       │ Schedules IEvent jobs via IJobOrchestrator
                       ▼
┌──────────────────────────────────────────────┐
│           PostgreSQL JobStore                │
│             (pfm-scheduler)                  │
│                                              │
│  * Clustered tables (QRTZ_TRIGGERS, etc.)    │
│  * System.Text.Json serialized IEvent payload│
│  * Distributed locking & check-in heartbeat  │
└──────────────────────▲───────────────────────┘
                       │
                       │ Acquires triggers & executes (Polling loop)
┌──────────────────────┴───────────────────────┐
│         PostFundManagement.Jobs              │
│        (Dedicated Worker Service)            │
│                                              │
│  * AddQuartzScheduler()                      │
│  * DefaultThreadPool (MaxConcurrency = 5)    │
│  * SchedulerStatus.Running (Started loop)    │
│  * JobsWorker background service:            │
│    - Registers recurring cron jobs on start  │
│    - Enqueues one-time jobs when needed      │
│  * Executes MediatorJob<TNotification>       │
│  * Deserializes and publishes via IMediator  │
└──────────────────────────────────────────────┘
Characteristic PostFundManagement.Api PostFundManagement.Jobs
Architectural Role API Gateway & Event Producer Dedicated Execution Daemon & Consumer
Registration Extension AddQuartzSchedulerClient(...) AddQuartzScheduler(...)
Thread Pool ZeroSizeThreadPool (0 threads) DefaultThreadPool (MaxConcurrency = 5)
Hosted Service None (AddQuartzHostedService omitted) AddQuartzHostedService(WaitForJobsToComplete = true)
Lifecycle State SchedulerStatus.Created (Standby) SchedulerStatus.Running (Active execution loop)
Readiness Check Verifies DB connectivity without requiring Running Verifies active execution engine & DB connectivity
Trigger Acquisition ❌ Disabled (Cannot acquire triggers or run jobs) ✅ Active (Polls DB, acquires triggers, runs jobs)

6.2 Event-Driven Mediated Orchestration (IEvent & MediatorJob)

Background tasks are modeled as strongly-typed domain events implementing IEvent : INotification (PostFundManagement.Domain.Abstractions).

sequenceDiagram
    autonumber
    participant API as PostFundManagement.Api
    participant Orchestrator as IJobOrchestrator
    participant DB as PostgreSQL pfm-scheduler
    participant Host as PostFundManagement.Jobs
    participant QuartzThread as Quartz Execution Engine
    participant Mediator as IMediator MediatorJob
    participant Handler as Domain Event Handlers

    Note over API,Orchestrator: 1. Event Scheduling (API Node)
    API->>Orchestrator: SendAsync(notification) / ScheduleAsync(notification, cron)
    Orchestrator->>DB: Stores JobDetail and Trigger with serialized Payload JSON
    Note over DB,Host: 2. Distributed Acquisition (Jobs Node)
    QuartzThread->>DB: Polls and acquires due trigger with row-level locks
    QuartzThread->>Host: Dispatches to local thread pool
    Host->>Mediator: Instantiates MediatorJob[TNotification]
    Mediator->>Mediator: Deserializes payload from MergedJobDataMap
    Mediator->>Handler: Publish(notification, cancellationToken)
    Handler-->>Host: Execution complete and updates business data in pfm DB
    Host->>DB: Releases trigger and commits execution state
  1. Scheduling via IJobOrchestrator:
    • SendAsync<TNotification>(notification): Schedules a durable, one-time execution in the onetime-jobs group with StartNow() and DisallowConcurrentExecution().
    • ScheduleAsync<TNotification>(notification, cronExpression): Configures a durable, cron-driven recurring job in the scheduled-jobs group with automatic misfire handling (CronTriggerMisfireInstruction.IgnoreMisfires).
    • InterruptAsync(eventName, correlationId): Allows runtime cancellation of active job executions across the cluster.
  2. Transparent Deserialization & Dispatch (MediatorJob<TNotification>): MediatorJob<TNotification> resolves IMediator from DI, deserializes the JSON payload, initiates OpenTelemetry distributed tracing (MediatorTelemetry.Source), and publishes the event to all registered in-process handlers.

6.3 Recurring vs. One-Time Job Patterns

  • Recurring Jobs: Scheduled by JobsWorker (BackgroundService in PostFundManagement.Jobs) during application startup. These jobs govern periodic health sweeps, SLA tracking, notification batching, and risk evaluations on predictable cron intervals.
  • One-Time Jobs: Primarily scheduled by PostFundManagement.Api as immediate or deferred side-effects of HTTP requests and lifecycle mutations (e.g. user invites, award registration triggers). Furthermore, a running recurring job in PostFundManagement.Jobs can dispatch a one-time job (jobOrchestrator.SendAsync) when handling fan-out tasks or invoking decoupled activities that cannot be triggered directly by the originating job.

6.4 Core Background Processing Capabilities

The background job runner powers four essential operational workloads:

  1. Automated Project Monitoring & Reminders:
    • Sweeps active projects, contractual deliverables, and milestone deadlines.
    • Computes planned vs. actual progress variance and escalates overdue milestones (FR-M04-017, FR-M07-009).
    • Dispatches automated reminders to Grantees/Partners prior to milestone due dates and submission cut-offs.
  2. Risk Monitoring & Automated Notifications:
    • Evaluates project risk registers, Key Risk Indicators (KRIs), and issue escalation thresholds (FR-M09-001 - 011).
    • Detects risk score creep (Likelihood \times Impact) and triggers automated alerts when risk tolerance thresholds are breached.
    • Monitors overdue mitigation action plans and escalates unresolved high-severity risks to Programme Managers.
  3. Email Notification Processing:
    • Continuously consumes and batches pending notification records (Platform.Email / NotificationPlatform.Email).
    • Dispatches transactional emails via MailKit / MimeKit SMTP with exponential backoff and retry policies.
    • Updates delivery status (Sent, Failed, retry count) and logs dispatch telemetry for audit compliance (FR-M25-001).
  4. Platform Extensibility:
    • New background jobs and recurring routines can be introduced by creating an IEvent model in PostFundManagement.Application and scheduling it via JobsWorker or IJobOrchestrator, with zero impact on the API surface.