trackgo-fe/packages/core-storage/README.md

Enterprise Storage Engine (@repo/core-storage)

← Back to Root

@repo/core-storage is the Enterprise-grade, multi-tool storage engine for the Eigen Monorepo.

It provides a unified set of strictly-typed, secure, and fault-tolerant storage mechanisms tailored for React applications. It enforces strict Inversion of Control (IoC)—the core engine knows absolutely nothing about your application's business domains; instead, the consuming apps inject their own configurations and types.

This package provides three primary storage solutions:

1. Secure Local Storage (Strict Key-Gatekeeping & AES encryption)
2. Secure IndexedDB (For larger key-value payloads)
3. Offline-First PouchDB (For document-oriented, bi-directional sync data)

---

🔒 Secure Key-Value Storage (LocalStorage & IndexedDB)

Browser storage is notoriously vulnerable to XSS attacks and pollution. The LocalStorageService and IndexedDBService implement a strict Gatekeeper pattern to solve this.

By forcing developers to register every key explicitly into either plainTextKeys or encryptedKeys, the engine guarantees:

1. No unapproved or rogue keys can ever be written or read (throws a Security Exception).
2. Highly sensitive tokens (e.g., JWTs) are automatically routed through the @repo/utils AES Encryption pipeline before touching the disk.

Architecture

graph TD
    %% ─── Styling Definitions (Dark-Mode Friendly Enterprise Palette) ───
    classDef appEntity fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e3a8a
    classDef coreEntity fill:#e2e8f0,stroke:#64748b,stroke-width:2px,color:#0f172a
    classDef gatekeeper fill:#10b981,stroke:#047857,stroke-width:2px,color:#ffffff
    classDef encrypt fill:#f59e0b,stroke:#b45309,stroke-width:2px,color:#ffffff
    classDef error fill:#f43f5e,stroke:#be123c,stroke-width:2px,color:#ffffff
    classDef database fill:#6366f1,stroke:#4338ca,stroke-width:2px,color:#ffffff

    %% ─── Subgraphs ───
    subgraph Apps ["apps/* (App Autonomy)"]
        REG[[AppStorageKey Config]]
        UI([React Components / API])
        INST{{Storage Instances}}
    end

    subgraph Core ["@repo/core-storage"]
        FAC[Factory: createStorage]
        API[IStorageService API]
        VAL{Runtime Gatekeeper}
        ERR>Throws Security Exception]
        ENC{{AES Encryption Pipeline}}
        LOCAL[(LocalStorage Adapter)]
        IDB[(IndexedDB Adapter)]
    end

    %% ─── Flow & Relationships ───
    %% 1. Initialization Flow
    REG -.->|Injects Keys & Config| FAC
    FAC -.->|Returns| INST
    
    %% 2. Runtime Execution Flow
    UI ===>|getItem / setItem| INST
    INST ---> API
    API ---> VAL
    
    %% 3. Gatekeeper Decision Tree
    VAL -.->|Invalid Key| ERR
    VAL ===>|Sensitive Key| ENC
    
    VAL --->|Plain-text Key| LOCAL
    VAL --->|Plain-text Key| IDB
    
    %% 4. Post-Encryption Storage
    ENC ===>|Encrypted Data| LOCAL
    ENC ===>|Encrypted Data| IDB

    %% ─── Apply Styles ───
    class REG,UI,INST appEntity;
    class FAC,API coreEntity;
    class VAL gatekeeper;
    class ENC encrypt;
    class ERR error;
    class LOCAL,IDB database;

    %% ─── Subgraph Backgrounds (Transparent for Native GitHub Support) ───
    style Apps fill:transparent,stroke:#818cf8,stroke-width:2px,stroke-dasharray: 5 5
    style Core fill:transparent,stroke:#94a3b8,stroke-width:2px,stroke-dasharray: 5 5

Usage & Implementation

import { createLocalStorage, createIndexedDB } from '@repo/core-storage';

// 1. Define allowed keys (Strict Type Safety)
export type AppStorageKey = 'THEME' | 'ACCESS_TOKEN' | 'OFFLINE_CACHE';

// 2. Instantiate Local Storage
export const appStorage = createLocalStorage<AppStorageKey>({
  plainTextKeys: new Set(['THEME']),
  encryptedKeys: new Set(['ACCESS_TOKEN']), // Auto AES encrypted
});

// 3. Usage
await appStorage.setItem('ACCESS_TOKEN', 'ey...'); // Encrypted on disk
const theme = await appStorage.getItem('THEME'); // Plaintext on disk

✅ Do's and ❌ Don'ts

• ✅ DO use TypeScript Literal Types for your storage keys (type Keys = 'A' | 'B') to get full IntelliSense.
• ✅ DO place Session/Auth tokens exclusively inside the encryptedKeys Set.
• ❌ DON'T use native window.localStorage directly anywhere in your React components. It bypasses our encryption and gatekeeper logic.
• ❌ DON'T mix domain data. Keep UI preferences (Theme, Sidebar state) in LocalStorage, and large datasets (Offline Caches) in IndexedDB.

---

🔄 Offline-First Document Storage (PouchDB & CouchDB)

For complex, document-oriented data that requires fault-tolerance, offline support, and bi-directional cloud synchronization, we use the PouchDatabaseManager.

Architecture

graph TD
    %% ─── Styling Definitions (Dark-Mode Friendly Enterprise Palette) ───
    classDef appEntity fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e3a8a
    classDef coreEntity fill:#e2e8f0,stroke:#64748b,stroke-width:2px,color:#0f172a
    classDef localDb fill:#6366f1,stroke:#4338ca,stroke-width:2px,color:#ffffff
    classDef remoteDb fill:#f59e0b,stroke:#b45309,stroke-width:2px,color:#ffffff

    %% ─── Subgraphs ───
    subgraph UI ["Consuming App (apps/*)"]
        COMP([React Components / Forms])
    end

    subgraph CoreStorage ["@repo/core-storage Engine"]
        MGR[PouchDatabaseManager Factory]
        L_SALES[(Local PouchDB: Sales)]
        L_INV[(Local PouchDB: Inventory)]
    end

    subgraph RemoteServer ["CouchDB Cluster"]
        R_SALES[(Remote CouchDB: sales_db)]
        R_INV[(Remote CouchDB: inventory_db)]
    end

    %% ─── Flow & Relationships ───
    COMP ===>|Read / Write| L_SALES
    COMP ===>|Read / Write| L_INV
    
    MGR -.->|Instantiates Multi-DB| L_SALES
    MGR -.->|Instantiates Multi-DB| L_INV

    L_SALES <===>|Native Sync Live and Retry| R_SALES
    L_INV <===>|Native Sync Live and Retry| R_INV

    %% ─── Apply Styles ───
    class COMP appEntity;
    class MGR coreEntity;
    class L_SALES,L_INV localDb;
    class R_SALES,R_INV remoteDb;

    %% ─── Subgraph Backgrounds (Transparent for Native GitHub Support) ───
    style UI fill:transparent,stroke:#3b82f6,stroke-width:2px,stroke-dasharray: 5 5
    style CoreStorage fill:transparent,stroke:#94a3b8,stroke-width:2px,stroke-dasharray: 5 5
    style RemoteServer fill:transparent,stroke:#f59e0b,stroke-width:2px,stroke-dasharray: 5 5

1. Initialization (IoC Factory)

The PouchDatabaseManager acts as a central singleton. It registers and manages all database instances. If a remote URL is provided, it automatically handles background synchronization.

import { PouchDatabaseManager } from '@repo/core-storage';
import type { Item } from './types';

export const dbManager = new PouchDatabaseManager();

export const itemDB = dbManager.register<Item>({
  localName: 'items_db',
  remoteUrl: 'http://admin:password@localhost:5984/items_db' 
});

2. CRUD & MongoDB-style Queries

The registered database returns a PouchDatabaseWrapper. This wrapper fully abstracts the raw PouchDB API into clean, Promise-based helpers, auto-handling _rev conflicts.

Method	Description
create(data)	Inserts a new document. Auto-generates _id if omitted.
update(id, data)	Auto-fetches the latest _rev to merge payloads cleanly.
delete(id)	Auto-fetches the latest _rev to safely remove the document.
getAll()	Retrieves all documents (filters out internal _design/ docs).
find(options)	Queries using MongoDB-style selectors (via pouchdb-find).

// Example: Querying data using selectors
const expensiveItems = await itemDB.find({
  selector: { price: { $gt: 100 }, category: 'electronics' }
});

3. Real-Time Reactivity (onChange Pub/Sub)

We implemented a Publisher-Subscriber (Pub/Sub) pattern inside the wrapper to handle real-time data changes efficiently. The wrapper maintains a single background connection to the changes feed and broadcasts events to all React subscribers.

import { useEffect, useCallback, useState } from 'react';
import { itemDB } from '../core/db';

export function InventoryList() {
  const [items, setItems] = useState([]);

  const loadData = useCallback(async () => {
    const data = await itemDB.getAll();
    setItems(data);
  }, []);

  useEffect(() => {
    loadData();

    // Subscribe to background sync mutations
    const unsubscribe = itemDB.onChange(() => {
      loadData();
    });

    // CRITICAL: Prevent memory leaks
    return () => unsubscribe();
  }, [loadData]);
}

✅ Do's and ❌ Don'ts for PouchDB

• ✅ DO use .onChange() to make your UI reactive to background cloud syncs.
• ✅ DO return the unsubscribe function in your useEffect cleanup block to prevent severe memory leaks.
• ❌ DON'T use db.raw.changes() inside your React components. It creates zombie WebSocket connections and tightly couples your UI to PouchDB's specific API.
• ❌ DON'T pass the _rev property manually when updating or deleting. The wrapper's update() and delete() methods handle revision fetching automatically.

---

⚠️ Troubleshooting

CouchDB CORS Infinite Retries

By providing a remoteUrl, the engine runs bi-directional sync in the background (live: true, retry: true). Fault tolerance is guaranteed: if CouchDB crashes, local reads/writes continue uninterrupted.

However, if your browser blocks CouchDB sync with a CORS error, PouchDB will misinterpret this as a network failure and enter an infinite retry loop, flooding your Network tab.

DO NOT try to fix this in the frontend Vite config or proxy! This is strictly a CouchDB server policy issue. You must enable CORS directly on the CouchDB cluster (editing its local.ini or via its dashboard) to allow origins, credentials, and headers.