feat: implement PouchDB storage layer with CRUD operations and add showcase UI component
This commit is contained in:
Firman Ramdhani
+99
-126
@@ -2,167 +2,140 @@
|
||||
|
||||
[← Back to Root](../../README.md)
|
||||
|
||||
The **Enterprise-grade storage engine** for the monorepo.
|
||||
This package provides an **Offline-First Storage Engine** using PouchDB, tailored for Enterprise React applications. It is built to seamlessly sync with remote CouchDB instances, providing full fault tolerance and offline capabilities.
|
||||
|
||||
This package provides a unified, Factory-based interface for interacting with browser storage (`localStorage` and `IndexedDB`). It enforces strict type safety, **Runtime Validation**, App Autonomy (Inversion of Control), and automatically provides **AES encryption at rest** for sensitive payloads (like access tokens) using `@repo/utils`.
|
||||
## High-Level Overview
|
||||
|
||||
---
|
||||
|
||||
## Architecture & Data Flow
|
||||
Our storage architecture enforces strict **Inversion of Control (IoC)**. The core engine (`@repo/core-storage`) is a pure factory—it knows absolutely nothing about your business domains, data models, or specific databases. Consuming applications (like `apps/web`) dictate the rules by injecting their specific configurations and generic types into the storage engine.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph Apps ["apps/* (App Autonomy)"]
|
||||
REG[[AppStorageKey & App Registries]]
|
||||
UI[React Components / API Interceptors]
|
||||
INST{{Storage Instances}}
|
||||
subgraph UI ["Consuming App (apps/*)"]
|
||||
COMP["React Components / Forms"]
|
||||
end
|
||||
|
||||
subgraph Core ["@repo/core-storage (Engine Factories)"]
|
||||
API[IStorageService API]
|
||||
FAC[createLocalStorage / createIndexedDB]
|
||||
VAL{Runtime Gatekeeper}
|
||||
ENC{{AES Encryption Pipeline}}
|
||||
LOCAL[LocalStorage Adapter]
|
||||
IDB[IndexedDB Adapter]
|
||||
subgraph CoreStorage ["@repo/core-storage Engine"]
|
||||
MGR["PouchDatabaseManager Factory"]
|
||||
L_SALES[("Local PouchDB: Sales")]
|
||||
L_INV[("Local PouchDB: Inventory")]
|
||||
end
|
||||
|
||||
subgraph Browser ["Browser APIs (Native)"]
|
||||
B_LOCAL[(localStorage)]
|
||||
B_IDB[(IndexedDB)]
|
||||
subgraph RemoteServer ["CouchDB Cluster (Cloud/On-Prem)"]
|
||||
R_SALES[("Remote CouchDB: sales_db")]
|
||||
R_INV[("Remote CouchDB: inventory_db")]
|
||||
end
|
||||
|
||||
REG -.->|Injects Keys & Config| FAC
|
||||
FAC --> INST
|
||||
UI -->|getItem / setItem| INST
|
||||
INST --> API
|
||||
API --> VAL
|
||||
|
||||
VAL -.->|Valid Key?| ENC
|
||||
VAL -.->|Invalid Key!| ERR[Throws Security Exception]
|
||||
|
||||
ENC -.->|Sensitive Key| LOCAL & IDB
|
||||
VAL -.->|Plain-text Key| LOCAL & IDB
|
||||
COMP -->|Read / Write| L_SALES
|
||||
COMP -->|Read / Write| L_INV
|
||||
MGR -->|Instantiates Multi-DB| L_SALES
|
||||
MGR -->|Instantiates Multi-DB| L_INV
|
||||
|
||||
LOCAL <--> B_LOCAL
|
||||
IDB <--> B_IDB
|
||||
|
||||
%% Styling Subgraphs
|
||||
style Apps fill:#e7f5ff,stroke:#74c0fc,stroke-width:2px,color:#1864ab
|
||||
style Core fill:#f8f9fa,stroke:#ced4da,stroke-width:2px,color:#495057
|
||||
style Browser fill:#f1f3f5,stroke:#ced4da,stroke-width:2px,color:#495057
|
||||
L_SALES <-->|Native Sync Live and Retry| R_SALES
|
||||
L_INV <-->|Native Sync Live and Retry| R_INV
|
||||
|
||||
%% Styling Nodes
|
||||
style UI fill:#339af0,stroke:#1864ab,color:#fff
|
||||
style REG fill:#1864ab,stroke:#1864ab,color:#fff
|
||||
style INST fill:#339af0,stroke:#1864ab,color:#fff
|
||||
style API fill:#845ef7,stroke:#5f3dc4,color:#fff
|
||||
style FAC fill:#845ef7,stroke:#5f3dc4,color:#fff
|
||||
|
||||
%% Gatekeeper is GREEN (Security Checkpoint), Error is RED
|
||||
style VAL fill:#20c997,stroke:#089981,color:#fff
|
||||
style ERR fill:#fa5252,stroke:#c92a2a,color:#fff
|
||||
|
||||
style LOCAL fill:#845ef7,stroke:#5f3dc4,color:#fff
|
||||
style IDB fill:#845ef7,stroke:#5f3dc4,color:#fff
|
||||
style ENC fill:#fab005,stroke:#e67700,color:#fff
|
||||
style B_LOCAL fill:#868e96,stroke:#495057,color:#fff
|
||||
style B_IDB fill:#868e96,stroke:#495057,color:#fff
|
||||
style MGR fill:#339af0,stroke:#1864ab,color:#fff
|
||||
style L_SALES fill:#845ef7,stroke:#5f3dc4,color:#fff
|
||||
style L_INV fill:#845ef7,stroke:#5f3dc4,color:#fff
|
||||
style R_SALES fill:#fab005,stroke:#e67700,color:#fff
|
||||
style R_INV fill:#fab005,stroke:#e67700,color:#fff
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Primary Goals & Architectural Principles
|
||||
## Core Concepts & Usage
|
||||
|
||||
* **App Autonomy (Inversion of Control)**: The core storage engine does not know about your application's keys. Consuming applications define their own keys, their own `encryptedKeys` sets, and their own `plainTextKeys` sets, injecting them into the factory upon instantiation.
|
||||
* **Runtime Gatekeeper (Defensive Programming)**: The engine validates every `setItem`, `getItem`, and `removeItem` operation. If an app attempts to access a key that wasn't explicitly registered in `encryptedKeys` or `plainTextKeys`, the engine will immediately throw a Security Exception to prevent rogue data access/injection.
|
||||
* **Dual Backend Strategy**:
|
||||
* `createLocalStorage`: Ideal for small, synchronous-like data (tokens, user preferences, settings).
|
||||
* `createIndexedDB`: Built for large, asynchronous data (offline drafts, cached API responses, large arrays/blobs) bypassing the 5MB localStorage limit.
|
||||
* **Security by Default**: Developers don't need to manually encrypt/decrypt data. If a key is passed in the `encryptedKeys` configuration, the engine handles AES encryption transparently.
|
||||
* **Corrupt Data Resilience**: If parsing or decryption fails (e.g., tampered data or changed encryption keys), the corrupt entry is safely removed and returns `null`, preventing the app from crashing.
|
||||
### 1. Initialization & Registration (`PouchDatabaseManager`)
|
||||
|
||||
---
|
||||
The `PouchDatabaseManager` acts as the IoC Factory. Apps use it to register and initialize multiple discrete PouchDB databases using a `PouchConfig`.
|
||||
|
||||
## 🚀 App-Level Setup & Usage
|
||||
|
||||
### 1. Define App Keys and Instantiate (Inversion of Control)
|
||||
|
||||
In your consuming application (e.g., `apps/web/src/core/storage/index.ts`), define your keys and use the factories to create your instances.
|
||||
**Why we use this pattern:** Instead of scattering raw database instantiations across the codebase, the manager centralizes connections. If a database is requested twice, the manager efficiently returns the exact same instance.
|
||||
|
||||
```typescript
|
||||
// apps/web/src/core/storage/index.ts
|
||||
import { createLocalStorage, createIndexedDB } from '@repo/core-storage';
|
||||
import { PouchDatabaseManager } from '@repo/core-storage';
|
||||
import type { Item } from './types';
|
||||
|
||||
// 1. Define Keys
|
||||
export const AppStorageKey = {
|
||||
USER_PROFILE: 'user_profile',
|
||||
ACCESS_TOKEN: 'access_token',
|
||||
LOCALE: 'app_locale',
|
||||
} as const;
|
||||
export const dbManager = new PouchDatabaseManager();
|
||||
|
||||
export type AppStorageKeyValue = (typeof AppStorageKey)[keyof typeof AppStorageKey];
|
||||
|
||||
// 2. Classify Keys
|
||||
export const ENCRYPTED_KEYS = new Set<AppStorageKeyValue>([
|
||||
AppStorageKey.USER_PROFILE,
|
||||
AppStorageKey.ACCESS_TOKEN,
|
||||
]);
|
||||
|
||||
export const PLAIN_KEYS = new Set<AppStorageKeyValue>([
|
||||
AppStorageKey.LOCALE,
|
||||
]);
|
||||
|
||||
// 3. Instantiate Factories
|
||||
export const secureStorage = createLocalStorage<AppStorageKeyValue>({
|
||||
encryptedKeys: ENCRYPTED_KEYS,
|
||||
plainTextKeys: PLAIN_KEYS
|
||||
});
|
||||
|
||||
export const secureIndexedDB = createIndexedDB<AppStorageKeyValue>({
|
||||
dbName: 'eigen_erp_db',
|
||||
storeName: 'web_store',
|
||||
encryptedKeys: ENCRYPTED_KEYS,
|
||||
plainTextKeys: PLAIN_KEYS
|
||||
// Register a strictly-typed database with bi-directional sync
|
||||
export const itemDB = dbManager.register<Item>({
|
||||
localName: 'items_db',
|
||||
remoteUrl: 'http://admin:password@localhost:5984/items_db'
|
||||
});
|
||||
```
|
||||
|
||||
### 2. Usage in App Components
|
||||
### 2. CRUD & Queries (`PouchDatabaseWrapper`)
|
||||
|
||||
Now, you can import your locally-created instances anywhere in your app.
|
||||
When you register a database, you receive a strictly typed `PouchDatabaseWrapper`. This wrapper abstracts away the raw PouchDB API, giving developers clean, Promise-based helper methods without ever needing to pass `dbName` or complex identifiers repeatedly.
|
||||
|
||||
| Method | Description |
|
||||
|---|---|
|
||||
| `create(data)` | Inserts a new document. PouchDB will auto-generate an `_id` if omitted. |
|
||||
| `update(id, data)` | Automatically fetches the latest `_rev` to merge the payload, preventing conflict errors. |
|
||||
| `delete(id)` | Automatically fetches the latest `_rev` to safely remove the document. |
|
||||
| `getOne(id)` | Retrieves a single document by its `_id`. |
|
||||
| `getAll()` | Retrieves all documents, automatically filtering out internal `_design/` docs. |
|
||||
| `find(options)` | Queries using MongoDB-style selectors (via `pouchdb-find`). |
|
||||
|
||||
**Example of `find()` with Selectors:**
|
||||
Instead of pulling all documents into memory and filtering them with JavaScript, we leverage native MongoDB-style selectors for performance:
|
||||
|
||||
```typescript
|
||||
import { secureStorage, AppStorageKey } from '@/core/storage';
|
||||
import type { UserProfile } from '@/types';
|
||||
|
||||
// CREATE / UPDATE
|
||||
// Since USER_PROFILE is in ENCRYPTED_KEYS, it is AES-encrypted automatically.
|
||||
await secureStorage.setItem(AppStorageKey.USER_PROFILE, {
|
||||
id: 1,
|
||||
name: 'Firman',
|
||||
role: 'admin'
|
||||
const expensiveItems = await itemDB.find({
|
||||
selector: {
|
||||
price: { $gt: 100 },
|
||||
category: 'electronics'
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
// READ (Returns null if not found or if decryption fails)
|
||||
const profile = await secureStorage.getItem<UserProfile>(AppStorageKey.USER_PROFILE);
|
||||
if (profile) {
|
||||
console.log('Welcome back,', profile.name);
|
||||
### 3. Real-Time Reactivity (The `onChange` Pub/Sub Pattern)
|
||||
|
||||
**CRITICAL CONCEPT:** We do **not** expose the raw `db.changes()` feed directly to React components. Instead, the `PouchDatabaseWrapper` utilizes a clean Pub/Sub abstraction via the `.onChange(callback)` method.
|
||||
|
||||
**Why we use this pattern:**
|
||||
1. **Memory Safety:** Direct bindings to PouchDB's raw changes feed often lead to zombie listeners and memory leaks. The `.onChange()` returns an unsubscribe function natively tailored for React's `useEffect` cleanup block.
|
||||
2. **Connection Efficiency:** It maintains a *single* WebSocket/Polling connection to the database under the hood. Multiple React components can subscribe to the same wrapper without opening dozens of parallel database connections.
|
||||
|
||||
```tsx
|
||||
import { useEffect, useCallback, useState } from 'react';
|
||||
import { itemDB } from '../core/db';
|
||||
|
||||
export function InventoryList() {
|
||||
const [items, setItems] = useState([]);
|
||||
|
||||
const loadData = useCallback(async () => {
|
||||
const data = await itemDB.getAll();
|
||||
setItems(data);
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
// 1. Initial Load
|
||||
loadData();
|
||||
|
||||
// 2. Subscribe to local mutations AND remote CouchDB syncs
|
||||
const unsubscribe = itemDB.onChange(() => {
|
||||
console.log('Database updated locally or remotely. Refreshing...');
|
||||
loadData();
|
||||
});
|
||||
|
||||
// 3. Prevent memory leaks!
|
||||
return () => {
|
||||
unsubscribe();
|
||||
};
|
||||
}, [loadData]);
|
||||
|
||||
// UI rendering...
|
||||
}
|
||||
|
||||
// DELETE
|
||||
await secureStorage.removeItem(AppStorageKey.USER_PROFILE);
|
||||
```
|
||||
|
||||
### 3. The Runtime Gatekeeper
|
||||
### 4. CouchDB Sync & CORS Troubleshooting
|
||||
|
||||
If you try to access an unregistered key, the engine protects the app by throwing an error at runtime:
|
||||
By providing a `remoteUrl` to the manager, the engine automatically handles bi-directional synchronization in the background (`live: true, retry: true`). If the server goes down, the local app will continue working seamlessly and sync automatically when the connection is restored.
|
||||
|
||||
```typescript
|
||||
// Throws Error: "[Storage Engine] Security Exception: Key 'rogue_key' is not registered..."
|
||||
await secureStorage.setItem('rogue_key' as any, 'hacked');
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
> [!WARNING]
|
||||
> **Migration Hazard**: If you move an existing key from `plainTextKeys` to `encryptedKeys` (or vice versa), existing users will experience a parsing failure on their next session (because the library expects encrypted data but finds plain text). The resilient parser will catch this and gracefully clear the key, which may effectively log them out or reset their local preference.
|
||||
> [!WARNING]
|
||||
> **CORS Infinite Retries & Preflight Failures**
|
||||
> If your browser blocks the synchronization with a CORS error, you will see PouchDB enter an infinite retry loop in the network tab.
|
||||
>
|
||||
> **Do NOT try to fix this in the frontend code!**
|
||||
> This is exclusively a CouchDB server configuration issue. You must enable CORS directly on the CouchDB instance by editing its `local.ini` or using its dashboard configuration to allow origins, credentials, and headers.
|
||||
Reference in New Issue
Block a user