trackgo-fe/packages/core-storage/README.md

[← Back to Root](../../README.md)

# Storage Engine (`@repo/core-storage`)

`@repo/core-storage` is the **Enterprise-grade, multi-tool storage engine** for the Eigen Monorepo. 

It provides a unified set of strictly-typed, secure, and fault-tolerant storage mechanisms tailored for React applications. It enforces strict **Inversion of Control (IoC)**—the core engine knows absolutely nothing about your application's business domains; instead, the consuming apps inject their own configurations and types.

This package provides three primary storage solutions:
1. **Secure Local Storage** (Strict Key-Gatekeeping & AES encryption)
2. **Secure IndexedDB** (For larger key-value payloads)
3. **Offline-First PouchDB** (For document-oriented, bi-directional sync data)

---

## 🔒 Secure Key-Value Storage (LocalStorage & IndexedDB)

Browser storage is notoriously vulnerable to XSS attacks and pollution. The `LocalStorageService` and `IndexedDBService` implement a strict **Gatekeeper** pattern to solve this.

By forcing developers to register every key explicitly into either `plainTextKeys` or `encryptedKeys`, the engine guarantees:
1. No unapproved or rogue keys can ever be written or read (throws a `Security Exception`).
2. Highly sensitive tokens (e.g., JWTs) are automatically routed through the `@repo/utils` AES Encryption pipeline before touching the disk.

### Architecture

```mermaid
graph TD
    %% ─── Styling Definitions (Dark-Mode Friendly Enterprise Palette) ───
    classDef appEntity fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e3a8a
    classDef coreEntity fill:#e2e8f0,stroke:#64748b,stroke-width:2px,color:#0f172a
    classDef gatekeeper fill:#10b981,stroke:#047857,stroke-width:2px,color:#ffffff
    classDef encrypt fill:#f59e0b,stroke:#b45309,stroke-width:2px,color:#ffffff
    classDef error fill:#f43f5e,stroke:#be123c,stroke-width:2px,color:#ffffff
    classDef database fill:#6366f1,stroke:#4338ca,stroke-width:2px,color:#ffffff

    %% ─── Subgraphs ───
    subgraph Apps ["apps/* (App Autonomy)"]
        REG[[AppStorageKey Config]]
        UI([React Components / API])
        INST{{Storage Instances}}
    end

    subgraph Core ["@repo/core-storage"]
        FAC[Factory: createStorage]
        API[IStorageService API]
        VAL{Runtime Gatekeeper}
        ERR>Throws Security Exception]
        ENC{{AES Encryption Pipeline}}
        LOCAL[(LocalStorage Adapter)]
        IDB[(IndexedDB Adapter)]
    end

    %% ─── Flow & Relationships ───
    %% 1. Initialization Flow
    REG -.->|Injects Keys & Config| FAC
    FAC -.->|Returns| INST
    
    %% 2. Runtime Execution Flow
    UI ===>|getItem / setItem| INST
    INST ---> API
    API ---> VAL
    
    %% 3. Gatekeeper Decision Tree
    VAL -.->|Invalid Key| ERR
    VAL ===>|Sensitive Key| ENC
    
    VAL --->|Plain-text Key| LOCAL
    VAL --->|Plain-text Key| IDB
    
    %% 4. Post-Encryption Storage
    ENC ===>|Encrypted Data| LOCAL
    ENC ===>|Encrypted Data| IDB

    %% ─── Apply Styles ───
    class REG,UI,INST appEntity;
    class FAC,API coreEntity;
    class VAL gatekeeper;
    class ENC encrypt;
    class ERR error;
    class LOCAL,IDB database;

    %% ─── Subgraph Backgrounds (Transparent for Native GitHub Support) ───
    style Apps fill:transparent,stroke:#818cf8,stroke-width:2px,stroke-dasharray: 5 5
    style Core fill:transparent,stroke:#94a3b8,stroke-width:2px,stroke-dasharray: 5 5
```

### Usage & Implementation

```typescript
import { createLocalStorage, createIndexedDB } from '@repo/core-storage';

// 1. Define allowed keys (Strict Type Safety)
export type AppStorageKey = 'THEME' | 'ACCESS_TOKEN' | 'OFFLINE_CACHE';

// 2. Instantiate Local Storage
export const appStorage = createLocalStorage<AppStorageKey>({
  plainTextKeys: new Set(['THEME']),
  encryptedKeys: new Set(['ACCESS_TOKEN']), // Auto AES encrypted
});

// 3. Usage
await appStorage.setItem('ACCESS_TOKEN', 'ey...'); // Encrypted on disk
const theme = await appStorage.getItem('THEME'); // Plaintext on disk
```

### ✅ Do's and ❌ Don'ts

* **✅ DO use TypeScript Literal Types** for your storage keys (`type Keys = 'A' | 'B'`) to get full IntelliSense.
* **✅ DO place Session/Auth tokens** exclusively inside the `encryptedKeys` Set.
* **❌ DON'T use native `window.localStorage` directly** anywhere in your React components. It bypasses our encryption and gatekeeper logic.
* **❌ DON'T mix domain data.** Keep UI preferences (Theme, Sidebar state) in LocalStorage, and large datasets (Offline Caches) in IndexedDB.

---

## 🔄 Offline-First Document Storage (PouchDB & CouchDB)

For complex, document-oriented data that requires fault-tolerance, offline support, and bi-directional cloud synchronization, we use the `PouchDatabaseManager`. 

### Architecture

```mermaid
graph TD
    %% ─── Styling Definitions (Dark-Mode Friendly Enterprise Palette) ───
    classDef appEntity fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e3a8a
    classDef coreEntity fill:#e2e8f0,stroke:#64748b,stroke-width:2px,color:#0f172a
    classDef localDb fill:#6366f1,stroke:#4338ca,stroke-width:2px,color:#ffffff
    classDef remoteDb fill:#f59e0b,stroke:#b45309,stroke-width:2px,color:#ffffff

    %% ─── Subgraphs ───
    subgraph UI ["Consuming App (apps/*)"]
        COMP([React Components / Forms])
    end

    subgraph CoreStorage ["@repo/core-storage Engine"]
        MGR[PouchDatabaseManager Factory]
        L_SALES[(Local PouchDB: Sales)]
        L_INV[(Local PouchDB: Inventory)]
    end

    subgraph RemoteServer ["CouchDB Cluster"]
        R_SALES[(Remote CouchDB: sales_db)]
        R_INV[(Remote CouchDB: inventory_db)]
    end

    %% ─── Flow & Relationships ───
    COMP ===>|Read / Write| L_SALES
    COMP ===>|Read / Write| L_INV
    
    MGR -.->|Instantiates Multi-DB| L_SALES
    MGR -.->|Instantiates Multi-DB| L_INV

    L_SALES <===>|Native Sync Live and Retry| R_SALES
    L_INV <===>|Native Sync Live and Retry| R_INV

    %% ─── Apply Styles ───
    class COMP appEntity;
    class MGR coreEntity;
    class L_SALES,L_INV localDb;
    class R_SALES,R_INV remoteDb;

    %% ─── Subgraph Backgrounds (Transparent for Native GitHub Support) ───
    style UI fill:transparent,stroke:#3b82f6,stroke-width:2px,stroke-dasharray: 5 5
    style CoreStorage fill:transparent,stroke:#94a3b8,stroke-width:2px,stroke-dasharray: 5 5
    style RemoteServer fill:transparent,stroke:#f59e0b,stroke-width:2px,stroke-dasharray: 5 5
```

### 1. Initialization (IoC Factory)

The `PouchDatabaseManager` acts as a central singleton. It registers and manages all database instances. If a remote URL is provided, it automatically handles background synchronization.

```typescript
import { PouchDatabaseManager } from '@repo/core-storage';
import type { Item } from './types';

export const dbManager = new PouchDatabaseManager();

export const itemDB = dbManager.register<Item>({
  localName: 'items_db',
  remoteUrl: 'http://admin:password@localhost:5984/items_db' 
});
```

### 2. CRUD & MongoDB-style Queries

The registered database returns a `PouchDatabaseWrapper`. This wrapper fully abstracts the raw PouchDB API into clean, Promise-based helpers, auto-handling `_rev` conflicts.

| Method | Description |
|---|---|
| `create(data)` | Inserts a new document. Auto-generates `_id` if omitted. |
| `update(id, data)` | Auto-fetches the latest `_rev` to merge payloads cleanly. |
| `delete(id)` | Auto-fetches the latest `_rev` to safely remove the document. |
| `getAll()` | Retrieves all documents (filters out internal `_design/` docs). |
| `find(options)` | Queries using MongoDB-style selectors (via `pouchdb-find`). |

```typescript
// Example: Querying data using selectors
const expensiveItems = await itemDB.find({
  selector: { price: { $gt: 100 }, category: 'electronics' }
});
```

### 3. Real-Time Reactivity (`onChange` Pub/Sub)

We implemented a **Publisher-Subscriber (Pub/Sub)** pattern inside the wrapper to handle real-time data changes efficiently. The wrapper maintains a *single* background connection to the changes feed and broadcasts events to all React subscribers.

```tsx
import { useEffect, useCallback, useState } from 'react';
import { itemDB } from '../core/db';

export function InventoryList() {
  const [items, setItems] = useState([]);

  const loadData = useCallback(async () => {
    const data = await itemDB.getAll();
    setItems(data);
  }, []);

  useEffect(() => {
    loadData();

    // Subscribe to background sync mutations
    const unsubscribe = itemDB.onChange(() => {
      loadData();
    });

    // CRITICAL: Prevent memory leaks
    return () => unsubscribe();
  }, [loadData]);
}
```

### ✅ Do's and ❌ Don'ts for PouchDB

* **✅ DO use `.onChange()`** to make your UI reactive to background cloud syncs.
* **✅ DO return the `unsubscribe` function** in your `useEffect` cleanup block to prevent severe memory leaks.
* **❌ DON'T use `db.raw.changes()`** inside your React components. It creates zombie WebSocket connections and tightly couples your UI to PouchDB's specific API.
* **❌ DON'T pass the `_rev` property** manually when updating or deleting. The wrapper's `update()` and `delete()` methods handle revision fetching automatically.

---

## ⚠️ Troubleshooting

### CouchDB CORS Infinite Retries
By providing a `remoteUrl`, the engine runs bi-directional sync in the background (`live: true, retry: true`). Fault tolerance is guaranteed: if CouchDB crashes, local reads/writes continue uninterrupted.

However, if your browser blocks CouchDB sync with a **CORS error**, PouchDB will misinterpret this as a network failure and enter an infinite retry loop, flooding your Network tab.

> **DO NOT try to fix this in the frontend Vite config or proxy!**
> This is strictly a CouchDB server policy issue. You must enable CORS directly on the CouchDB cluster (editing its `local.ini` or via its dashboard) to allow `origins`, `credentials`, and `headers`.