gemstone-nexus · Phase 6
Ledger · ProjectDocuments
Table ProjectDocuments · all tables
| id | doc_id | title | category | source_path | summary | content | content_hash |
|---|---|---|---|---|---|---|---|
| 1 | CAPACITOR-ANDROID-INSTALLATION-PLAYBOOK | Capacitor Android — Practitioner Installation & Operation Playbook | playbook | DevResources/docs/CAPACITOR-ANDROID-INSTALLATION-PLAYBOOK.md | This playbook is the end-to-end install and operate guide. It does **not** replace mobility specs or device UAT scripts — those live in the referenced docs below. | # Capacitor Android — Practitioner Installation & Operation Playbook **Audience:** Practitioners and QA standing up GemStone on a Windows 11 workstation **App ID:** `com.nextech.gemstone` · **App name:** GemStone **Date:** 2026-07-22 · **Capacitor:** 6.x · **Android:** minSdk 22 / compileSdk 34 / targetSdk 34 · **JDK:** 17+ (Studio JBR OK) This playbook is the end-to-end install and operate guide. It does **not** replace mobility specs or device UAT scripts — those live in the referenced docs below. ### Fastest path — hosted debug APK (no Studio) 1. On an Android phone, open **[Install Android](https://gemstone.dataright.net/app/install-android)** (also linked from the Voice sidebar). 2. Tap **Download GemStone Android (debug APK)** and wait until the download **finishes** (notification / Downloads list). 3. **Android 13+:** Settings → Apps → Special app access → **Install unknown apps** → enable for **Chrome** and/or **Files** (the browser/file manager that opens the APK). **Do not look for GemStone in that list** — GemStone is not installed yet and will not appear under Apps until step 4 succeeds. 4. Open **Downloads** → tap `gemstone-debug.apk` → **Install**. Only then does GemStone appear in the launcher / Apps. 5. If the UI hangs on **Installing…**: force-stop **Package Installer** (system app), delete the old APK, re-download (`?v=` cache-bust), re-check Chrome/Files unknown-apps, or use USB `adb install -r` below (bypasses the on-device installer UI). 6. Launch **GemStone**. WebView loads `https://gemstone.dataright.net` (Voice console, Mode A). **Direct download (cache-bust):** [`https://gemstone.dataright.net/downloads/gemstone-debug.apk?v=8edd700ae5e1`](https://gemstone.dataright.net/downloads/gemstone-debug.apk?v=8edd700ae5e1) **USB fallback (Windows):** ```powershell # After downloading gemstone-debug.apk to the PC & "$env:LOCALAPPDATA\Android\Sdk\platform-tools\adb.exe" devices & "$env:LOCALAPPDATA\Android\Sdk\platform-tools\adb.exe" install -r .\gemstone-debug.apk ``` This is a **debug-signed** sideload build for practitioners/QA — not a Play Store release. Package `com.nextech.gemstone`, minSdk 22 / targetSdk 34. Rebuild + redeploy when native code or Cap sync changes (`downloads/gemstone-debug.apk` is served by nginx with `Content-Disposition: attachment`). ## Related docs (do not duplicate) | Doc | Use when | | --- | --- | | [`GS-MOB-01.md`](GS-MOB-01.md) | Field Mode / Extended Session / wake-word / interruption contract | | [`DOC-Device-UAT-Checklist.md`](DOC-Device-UAT-Checklist.md) | Outdoor sun, call interruption 10-trial, wake 8/10 sign-off | | [`DOC-Capacitor-Live-EC2.md`](DOC-Capacitor-Live-EC2.md) | Live EC2 `server.url`, debug TLS trust, env overrides | | [`DOC-Auth0-Provisioning.md`](DOC-Auth0-Provisioning.md) | Auth0 tenant (deferred — **not** required to install) | | [`DOC-Store-Submission-Checklist.md`](DOC-Store-Submission-Checklist.md) | Play Console / AAB assembly when accounts exist | | [`frontend/CAPACITOR.md`](../../frontend/CAPACITOR.md) | Plugin surface, atomic concurrency, gate commands | | [`DOC08-Playbook.md`](DOC08-Playbook.md) | Production session / Didymus / deploy runbooks | --- ## 1. Windows 11 workstation prerequisites ### 1.1 Node.js and npm - **Node.js LTS** (20.x or 22.x recommended). Verify: ```powershell node -v npm -v ``` ### 1.2 Android Studio, SDK, platform & build tools 1. Install [Android Studio](https://developer.android.com/studio) (current stable). 2. Open **Settings → Languages & Frameworks → Android SDK** and install: - **Android SDK Platform** matching compile/target **34** (Android 14) - **Android SDK Build-Tools** (latest 34.x) - **Android SDK Platform-Tools** (`adb`) - **Android Emulator** (if you will not use a physical device) 3. Note **Android SDK Location** (typical: `%LOCALAPPDATA%\Android\Sdk`). Set user env vars (System Properties → Environment Variables), then open a **new** PowerShell: ```powershell # Example — adjust to your SDK path [System.Environment]::SetEnvironmentVariable("ANDROID_HOME", "$env:LOCALAPPDATA\Android\Sdk", "User") # Add to User PATH: %ANDROID_HOME%\platform-tools and %ANDROID_HOME%\emulator ``` Verify: ```powershell adb version sdkmanager --list 2>$null | Select-Object -First 5 # if cmdline-tools installed ``` ### 1.3 JDK 17+ and `JAVA_HOME` Capacitor Android / Gradle in this repo compile with **Java 17** bytecode (`sourceCompatibility` / `targetCompatibility` = `VERSION_17` in `frontend/android`). The **Android Studio JBR (JDK 17 or 21)** is the preferred toolchain for `assembleDebug`. - Prefer Studio JBR: `C:\Program Files\Android\Android Studio\jbr`, **or** Temurin/Oracle **JDK 17+**. - Set `JAVA_HOME` to that JDK root (not JRE). Verify: ```powershell $env:JAVA_HOME = "C:\Program Files\Android\Android Studio\jbr" java -version # 17.x or Studio JBR 21.x is fine with AGP 8.2 echo $env:JAVA_HOME ``` **Do not** point `JAVA_HOME` at JDK 8/11 — those cause `Unsupported class file major version` / toolchain errors. ### 1.4 USB debugging / emulator / device **Physical device** 1. Enable **Developer options** → **USB debugging**. 2. Connect USB; accept the RSA fingerprint prompt on the phone. 3. Verify: ```powershell adb devices # expect: <serial> device ``` **Emulator** 1. Android Studio → **Device Manager** → create a virtual device (API 34 recommended). 2. Start the AVD, then `adb devices` should list `emulator-5554` (or similar) as `device`. --- ## 2. Clone / update the repository and install frontend deps ```powershell # First time git clone https://github.com/<org>/GemStone-Nexus-StackOne.git cd GemStone-Nexus-StackOne # Ongoing git pull origin main cd frontend npm ci ``` Use `npm ci` for a clean, lockfile-faithful install. If `package-lock.json` is missing or intentionally being refreshed, `npm install` is acceptable once, then prefer `npm ci` thereafter. **Do not** commit `.env`, keystores, Auth0 secrets, or `DevResources/project.env` with credentials. Templates live in `.env.example` / `DevResources/project.env.example`. --- ## 3. Development modes GemStone Android is a Capacitor shell around the React/Vite SPA. Two modes matter for practitioners. ### Mode A — Live EC2-hosted UI/API (default debug path) **Intent:** Run Android in parallel with the web stack already deployed on EC2. The WebView loads the remote SPA so Vite `base: "/app/"` stays aligned with nginx. | Item | Value | | --- | --- | | Public SPA | `https://gemstone.dataright.net/app/` (secondary; Field/Install) | | Capacitor `server.url` | `https://gemstone.dataright.net` (Voice console — no trailing slash) | | API / WS | Same origin as the loaded SPA (see `frontend/src/services/apiOrigin.js`) | **TLS:** Live host uses **Let's Encrypt** for `gemstone.dataright.net`. System CAs trust that hostname — no custom Android trust needed. Debug-only `network_security_config` + `ec2_debug.crt` remain only for raw-IP / self-signed fallback (`https://18.116.188.14`). Details: [`DOC-Capacitor-Live-EC2.md`](DOC-Capacitor-Live-EC2.md). ### Mode B — Bundled native assets + configurable API base **Intent:** Ship `dist/` inside the APK; WebView origin is typically `https://localhost`. REST/WS then use `VITE_API_BASE` or the live default origin (`https://gemstone.dataright.net`). ```powershell cd frontend npm run build $env:CAPACITOR_USE_BUNDLED = "1" npm run cap:sync:android ``` | Knob | Effect | | --- | --- | | `CAPACITOR_SERVER_URL` | Override live SPA URL before sync (no trailing slash required) | | `CAPACITOR_USE_BUNDLED=1` | Remove `server.url`; load bundled `dist/` | | `VITE_API_BASE` | Force REST/WS origin at **build** time (no trailing slash), e.g. `https://gemstone.dataright.net` | Apply knobs via `frontend/scripts/apply-capacitor-server.cjs` (invoked by `npm run cap:sync` / `cap:sync:android`). > **Note on Vite `base`:** Keep `base: "/app/"` for web/nginx while Mode A (`server.url`) points at live. A future fully-offline bundled path may need `base: "./"` — do not change that casually without a deliberate web + Capacitor change set (atomic concurrency). --- ## 4. Exact build / sync / install commands All commands assume PowerShell on Windows 11 from the repo root unless noted. ### 4.1 Web build + Capacitor sync (Mode A — recommended) ```powershell cd frontend npm ci npm run build npm run cap:sync:android # equivalent: node scripts/apply-capacitor-server.cjs; npx cap sync android ``` ### 4.2 Open Android Studio ```powershell cd frontend npm run cap:open:android ``` Then **Run ▶ app** using the **debug** variant on an emulator or USB device. First paint loads `https://gemstone.dataright.net/app/` (network must reach that host on TCP 443). ### 4.3 Gradle `assembleDebug` without Studio UI ```powershell cd frontend\android .\gradlew.bat assembleDebug ``` **APK location:** ```text frontend\android\app\build\outputs\apk\debug\app-debug.apk ``` ### 4.4 Install with adb ```powershell adb devices adb install -r "frontend\android\app\build\outputs\apk\debug\app-debug.apk" # or from frontend\android after assemble: adb install -r "app\build\outputs\apk\debug\app-debug.apk" ``` Launch from the device launcher (GemStone) or: ```powershell adb shell am start -n com.nextech.gemstone/.MainActivity ``` ### 4.5 Logcat diagnostics ```powershell # Process-filtered (adjust after launch) adb logcat -c adb logcat | Select-String -Pattern "Capacitor|chromium|GemStone|ExtendedAudio|WakeWord|AndroidRuntime" ``` Useful filters: | Tag / pattern | Why | | --- | --- | | `chromium` / `Console` | WebView JS / network errors | | `Capacitor` | Bridge / plugin load | | `AndroidRuntime` | Native crashes | | `ExtendedAudio` / `WakeWord` | FG service / mobility plugin | --- ## 5. Parallel web/API operation and atomic concurrency ### How Android runs in parallel 1. EC2 continues to serve nginx → web (`/app/`) + API (`/api`, `/healthz`, `/mcp`, sessions WS). 2. Desktop browsers hit the same public URL. 3. Debug Cap Android WebView also hits that URL (Mode A), so SPA and API share origin and CORS stays simple. 4. Native mobility plugins (`GemStoneMobility`, Extended FG mic, wake-word scaffold) run **on device** and call the same REST/WS contracts as web. You do **not** need a second API stack on the laptop for Mode A. Mode B still needs a reachable API origin (`VITE_API_BASE` or live default). ### Atomic concurrency — Capacitor fail = feature fail From [`frontend/CAPACITOR.md`](../../frontend/CAPACITOR.md) and GS-MOB-01: - Web build, `npx cap sync android`, and (when applicable) `npx cap sync ios` ship in the **same change set**. - If Capacitor sync or the native project is broken, the mobility **feature** is failed — do not claim Phase 2B/2C complete from web-only green builds. - Docker frontend gate runs `vitest` + `npm run build` + `npx cap sync android`. Auth0 is **explicitly excluded** from this invariant for install: Auth0 deferred must not block Capacitor install or live-stack parallel operation (see §8). --- ## 6. Signing: debug vs release (no invented credentials) ### Debug signing - Android Studio / Gradle use the local **debug keystore** automatically (`~/.android/debug.keystore` or the Studio-managed equivalent). - Fine for emulator, USB sideload, and internal engineering. **Not** for Play upload. ### Release signing / Play Console - Release signing requires an **upload keystore** owned by NEXTECH, stored in a secrets vault (AWS Secrets Manager / secure store) — **never** committed to git. - This repo’s `frontend/android/app/build.gradle` does **not** ship release `signingConfigs` with passwords. That is intentional. - When Google Play Console access and keystore material exist, follow [`DOC-Store-Submission-Checklist.md`](DOC-Store-Submission-Checklist.md): 1. Generate / retrieve upload keystore offline. 2. Configure release signing via local `keystore.properties` (gitignored) or CI secrets — do not paste passwords into docs or chat logs. 3. Build AAB: `.\gradlew.bat bundleRelease`. 4. Upload to an **internal testing** track first. **Keystore safety** - Treat `.jks` / `.keystore` and passwords as production secrets. - Rotate only with a documented Play App Signing recovery plan. - Never invent or check in placeholder keystore passwords “for completeness.” --- ## 7. Permissions and mobility behavior Declared in `frontend/android/app/src/main/AndroidManifest.xml`: - `INTERNET` - `RECORD_AUDIO` - `MODIFY_AUDIO_SETTINGS` - `FOREGROUND_SERVICE` / `FOREGROUND_SERVICE_MICROPHONE` - `POST_NOTIFICATIONS` Runtime mic: `GemStoneMobility.checkMicrophonePermission()` then `ensureMicrophonePermission()` before WebView `getUserMedia` / Gemini Live / Field Mode. Cap warms the prompt once after the first touch so Conversation is ready. **Android persists** the grant across app restarts until the user revokes it in **Settings → Apps → GemStone → Permissions → Microphone**. On deny, toast + `openAppSettings()` deep-link. | Permission / component | Behavior | | --- | --- | | `INTERNET` | SPA + API + WS | | `RECORD_AUDIO` | Field Mode (`getUserMedia` / `MediaRecorder`) and Extended Session mic; OS-persisted after first Allow | | `FOREGROUND_SERVICE` + `FOREGROUND_SERVICE_MICROPHONE` | Extended Session + wake-word FG services | | `POST_NOTIFICATIONS` | Persistent FG notification (Android 13+) — user must allow | | `ExtendedAudioService` | Extended Session background audio bridge (`FOREGROUND_SERVICE_TYPE_MICROPHONE`) | | `WakeWordService` | Wake-word **scaffold** (notification + service); on-device model + 8/10 killed-state trials are device QA | **OS limitation:** GemStone cannot force “always on” mic if the user (or OEM battery policy) revokes `RECORD_AUDIO`. After revoke, the next Conversation / Live / Field start shows the system prompt again (or permanently denied → Settings). ### Modes (summary — see GS-MOB-01) | Mode | Route | Audio path | | --- | --- | --- | | Field Mode | `/app/field` | WebView mic + ascending dual-tone | | Extended Session | `/app/extended` | Native plugin → FG mic service + cues | | Domain Packs | `/app/packs` | REST install/uninstall | | Didymus queue | `/app/didymus` | Approval queue UI | ### Interruption resume (identical for all three causes) Causes: `audio_focus_loss` | `network_drop` | `app_background` ```text ACTIVE → PAUSED (snapshot) → RESUMABLE (session-ready chime) → ACTIVE ``` REST surface: `/api/sessions` (create, list/recents, get, pause, resumable, resume, end). Full contract: GS-MOB-01 §6. Device sign-off scripts: [`DOC-Device-UAT-Checklist.md`](DOC-Device-UAT-Checklist.md). --- ## 8. Auth0 deferred — guest / stub behavior (must not block install) **Status:** Auth0 live tenant is **deferred** and punchlisted. See [`DOC-Auth0-Provisioning.md`](DOC-Auth0-Provisioning.md). | Fact | Implication for install | | --- | --- | | `/healthz` may report `auth0.configured: false` | **Not** an install failure | | `/app/login` is an Auth0-ready **stub** | Shows “tenant not provisioned” until `VITE_AUTH0_*` exist | | HorizonORBIT login markup lives in HorizonCRM (not cloned) | Layout stub only | | Field / Extended / Packs / Didymus UI | Usable for engineering and device UAT without Universal Login | **Current practitioner behavior:** use the app as an unauthenticated / guest-style engineering client against the live stack. Do **not** wait for Auth0 to run `npm ci`, sync, assemble, or Field Mode mic checks. When secrets land in `DATARIGHT_DEV_SECRETS`, follow the Auth0 provisioning doc — that work is separate from this playbook’s install path. --- ## 9. Troubleshooting | Symptom | Likely cause | Fix | | --- | --- | --- | | Blank WebView / missing assets | Wrong Vite `base` vs load URL; stale sync; bundled mode without rebuild | Mode A: confirm `server.url` → `…/app`. Mode B: `npm run build` then `CAPACITOR_USE_BUNDLED=1` + sync. Hard-reload / reinstall APK. | | API calls hit `localhost` / fail on device | Bundled shell using `https://localhost` without `VITE_API_BASE` / live default; or old client | Confirm Mode A live URL, or rebuild with `VITE_API_BASE=https://gemstone.dataright.net`. See `apiOrigin.js`. | | TLS / `SSLHandshakeException` / net::ERR_CERT | Hitting raw EIP with self-signed leaf | Prefer `https://gemstone.dataright.net` (LE / system CAs). Raw IP still needs debug `ec2_debug.crt`. | | CORS / mixed content | `http` API from `https` WebView; `allowMixedContent: false` | Keep SPA + API on `https://gemstone.dataright.net`. Do not point Cap at `http://` API. | | JDK mismatch / Gradle toolchain errors | `JAVA_HOME` is JDK 8/11 or unset | Point `JAVA_HOME` at Studio JBR or JDK 17+; restart Studio/shell. | | SDK / build-tools missing | Platform 34 or build-tools not installed | Android Studio SDK Manager → install API 34 + build-tools + platform-tools. | | `adb devices` empty / `unauthorized` | USB debug off, bad cable, no RSA accept | Re-enable USB debugging; accept prompt; try another cable/port; `adb kill-server; adb start-server`. | | Microphone denied | Runtime permission not granted | App settings → Permissions → Microphone. Android 13+: also allow Notifications for FG service. | | Stale Capacitor sync | Native project out of date vs `dist` / plugins | `npm run build` → `npm run cap:sync:android` → clean rebuild (`.\gradlew.bat clean assembleDebug`). | | Live UI loads but voice/session APIs 502 | EC2 compose unhealthy | Check `https://gemstone.dataright.net/healthz` from PC; fix host stack (DOC-Deploy-Pipeline) — not a Cap config issue. | | Wake-word never fires from killed state | Scaffold only / model not onboarded | Expected until device QA completes GS-MOB-01 wake gates; do not treat as install failure. | --- ## 10. Verification checklist Mark each row after a successful run on your workstation + device. | # | Check | Command / action | Pass? | | --- | --- | --- | --- | | 0 | Hosted APK | Open `/app/install-android` → Download → install APK; or `curl -I https://gemstone.dataright.net/downloads/gemstone-debug.apk` | ☐ | | 1 | Web build | `cd frontend; npm ci; npm run build` | ☐ | | 2 | Cap sync Android | `npm run cap:sync:android` | ☐ | | 3 | Assemble debug | `cd android; .\gradlew.bat assembleDebug` | ☐ | | 4 | Install | `adb install -r app\build\outputs\apk\debug\app-debug.apk` | ☐ | | 5 | Launch | Open GemStone; SPA paints (Mode A: live `/app/`) | ☐ | | 6 | Health | Device or PC: `https://gemstone.dataright.net/healthz` reachable; Auth0 may be `configured:false` | ☐ | | 7 | Voice capture (Field) | `/app/field` → grant mic → tap → dual-tone ≤ 1s | ☐ | | 8 | Reconnect | Toggle airplane mode briefly during session → pause → resumable → resume | ☐ | | 9 | Field Mode | Heads-Up UI usable indoors; outdoor sun → Device UAT Script A | ☐ | | 10 | Extended Session | Start extended; hear mic-open cue; FG notification present | ☐ | | 11 | Log collection | `adb logcat` capture during a failing step; archive for engineering | ☐ | Formal outdoor / call / wake sign-off remains on [`DOC-Device-UAT-Checklist.md`](DOC-Device-UAT-Checklist.md). --- ## 11. Quick-reference command card ```powershell # Fastest: hosted debug APK (no local build) # https://gemstone.dataright.net/app/install-android # https://gemstone.dataright.net/downloads/gemstone-debug.apk # Prereq smoke (local build path) node -v; npm -v; java -version; adb version; adb devices # Install path (Mode A — live EC2) cd <repo>\frontend npm ci npm run build npm run cap:sync:android npm run cap:open:android # — or — cd android $env:JAVA_HOME = "C:\Program Files\Android\Android Studio\jbr" .\gradlew.bat assembleDebug adb install -r app\build\outputs\apk\debug\app-debug.apk # Bundled assets mode cd <repo>\frontend npm run build $env:CAPACITOR_USE_BUNDLED = "1" npm run cap:sync:android # Optional overrides $env:CAPACITOR_SERVER_URL = "https://gemstone.dataright.net/app" $env:VITE_API_BASE = "https://gemstone.dataright.net" # must rebuild after setting ``` --- ## 12. Post-wiring revision notes As of **2026-07-21**, live-stack Android wiring is recorded complete (`DOC-Capacitor-Live-EC2.md`, punchlist row **Capacitor Android ↔ live EC2**). Revise **this playbook** if any of the following change: | Change | Sections to update | | --- | --- | | EC2 public IP or hostname | Mode A URLs, troubleshooting, verification, command card | | TLS leaf rotation | Debug cert path note; PEM rebuild reminder | | Vite `base` / bundled-only product path | Modes A/B; blank WebView troubleshooting | | Auth0 goes live | §8 guest behavior → real Universal Login + callback URLs for Cap | | Release signing checked into CI (via secrets) | §6 — still never put passwords in this doc | | `appId` / versionCode bumps for store | Header metadata + store checklist cross-link | --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | fc699e305a7e6547713ec6d4d3724259cf203254 |
| 2 | DIDYMUS-ARCH-01 | DIDYMUS-ARCH-01 — Didymus Hub Architecture | didymus | DevResources/docs/DIDYMUS-ARCH-01.md | Didymus is the AI-orchestration hub that registers build suggestions from AI tools and desktop workers, stores them under a schema-enforced approval gate, and presents every task through GemStone as the single universal approval surface. This document is the architecture reference named by DOCA1, DOC02 Phase 3C, and DOC06. Operator steps live in DIDYMUS-PLAYBOOK.md. | # DIDYMUS-ARCH-01 — Didymus Hub Architecture *Worker contract, task storage, and MCP tool surface for GemStone integration* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 21, 2026 ## 1. Purpose Didymus is the AI-orchestration hub that registers build suggestions from AI tools and desktop workers, stores them under a schema-enforced approval gate, and presents every task through GemStone as the single universal approval surface. This document is the architecture reference named by DOCA1, DOC02 Phase 3C, and DOC06. Operator steps live in DIDYMUS-PLAYBOOK.md. ## 2. Hybrid Topology ``` Workers (Cursor, Claude, Gemini, future Ollama/desktop) | submit_task / lease / report_status / verify_output v Didymus Hub (task store + lease coordinator + MCP tools) | | PENDING tasks projected identically v GemStone UI (universal approval surface; Auth0-gated) | | human sets approved_by + approved_at v Worker lease -> RUNNING -> COMPLETED/FAILED -> VERIFIED ``` GemStone never executes worker code itself at MVP. GemStone Agent may submit tasks (`requested_by="GemStone"`) but cannot approve them (DOC03 Section 7). ## 3. DidymusWorker Contract Every registered worker implements: | Method | Responsibility | | --- | --- | | `get_capabilities()` | Returns worker_id, tool family, supported task_types, max concurrency, tier affinity | | `execute_command(task)` | Runs only when task status is APPROVED and lease acquired; returns structured result | | `report_status(task_id)` | Heartbeat and progress; lease renewal | | `verify_output(task_id, result)` | Optional worker-side verification; GemStone may also verify | Workers that cannot implement the full contract are not registered. Capability claims are advisory; the hub enforces lease and status transitions. ## 4. Task Storage Model Logical table `didymus_tasks` (Postgres; mirrored summary rows may land in the SQLite Project Ledger for audit/UI): | Field | Rules | | --- | --- | | `id` | UUID PK | | `requested_by` | Human user_id or `"GemStone"` | | `target_worker` | Registered worker_id | | `task_type` | Allow-listed string | | `payload` | JSON; no secrets | | `status` | PENDING \| APPROVED \| RUNNING \| COMPLETED \| FAILED \| VERIFIED | | `approved_by` | NULL until human approval | | `approved_at` | NULL until human approval | | `org_id` | Required; RLS / query scoping | | `created_at` / `updated_at` | Timestamps | Schema constraints (mandatory): - CHECK: `status = 'RUNNING'` implies `approved_by IS NOT NULL AND approved_at IS NOT NULL` - No direct transition PENDING -> RUNNING - Lease claim is an atomic UPDATE that only succeeds when `status = 'APPROVED'` ## 5. State Machine Identical to DOC03 Section 6: ``` PENDING --(human approved_by + approved_at)--> APPROVED APPROVED --(worker lease)--> RUNNING RUNNING --(report)--> COMPLETED | FAILED COMPLETED --(verify)--> VERIFIED ``` Source never changes the path. Blind-source review: GemStone UI must not badge or color tasks by origin in the default queue view. ## 6. MCP Tool Surface Didymus Hub exposes MCP tools consumed by GemStone and registered workers: | Tool | Caller | Behavior | | --- | --- | --- | | `didymus.submit_task` | Workers, GemStone Agent | Creates PENDING task | | `didymus.list_pending` | GemStone UI | Org-scoped PENDING/APPROVED queue | | `didymus.approve_task` | Human roles only | Sets approved_by/approved_at; status APPROVED | | `didymus.lease_task` | Workers | Atomic lease into RUNNING | | `didymus.report_result` | Workers | COMPLETED/FAILED + payload | | `didymus.verify_task` | GemStone or worker | VERIFIED | Secrets and API keys never travel in MCP payloads; workers read credentials from the secrets tier (AWS Secrets Manager / host env) provisioned per DIDYMUS-PLAYBOOK. ## 7. GemStone Integration Points - Phase 3C API: `POST /api/didymus/tasks`, `POST /api/didymus/tasks/{id}/approve` (DOC03 Section 4). - Platform Dashboard Approval Queue view (DOC21). - Incident runbook if RUNNING without approval (DOC08 Section 3). - UAT: Universal Approval Queue script (DOC20). ## 8. MVP Worker Set | Worker | Family | Notes | | --- | --- | --- | | Cursor | IDE agent | PAT / API credential in secrets tier | | Claude | Cloud LLM agent | API key in secrets tier | | Gemini | Cloud LLM agent | API key in secrets tier | Deferred workers (DOC02 Section 3): Claude Code, AntiGravity, GemStone-as-worker runtime, SolidWorks, KiCad. Sovereign Ollama worker may be added under the same contract without changing the approval gate. ## 9. Security - Cross-org isolation on every list/approve/lease path. - Transcripts and session audio never appear in Didymus task payloads. - Western-origin stack constraints (DOC17) apply to any hub dependencies. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 962b59334e02a9730085a3c781af4bace46ddf7b |
| 3 | DIDYMUS-PLAYBOOK | DIDYMUS-PLAYBOOK — Practitioner Provisioning and Queue Ops | playbook | DevResources/docs/DIDYMUS-PLAYBOOK.md | Operator and practitioner guide for provisioning AI development-tool workers, storing credentials in the secrets tier, registering workers with Didymus, and running the human approval queue. Architecture detail is in DIDYMUS-ARCH-01; this playbook is the how-to companion. | # DIDYMUS-PLAYBOOK — Practitioner Provisioning and Queue Ops *Install, authorize, and operate Didymus AI workers through GemStone* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Phase 3C Implemented · **Date:** July 21, 2026 ## 1. Purpose Operator and practitioner guide for provisioning AI development-tool workers, storing credentials in the secrets tier, registering workers with Didymus, and running the human approval queue. Architecture detail is in DIDYMUS-ARCH-01; this playbook is the how-to companion. ## 2. Preconditions - Auth0 roles available: SuperAdmin, Org Admin, Practitioner (approvers); GemStone Agent may submit only (never approve). - Secrets tier ready: AWS Secrets Manager secret `DATARIGHT_DEV_SECRETS` (us-east-1) on GoldenAMI / EC2, **or** host `.env` injected into `gemstone-api` on the clean local FullBase clone. **Never commit secrets to the repo.** - Didymus runs in-process with GemStone API at Phase 3C (`DIDYMUS_HUB_URL` optional for a future external split; leave unset on the T4 host). - GemStone approval queue UI: `/app/didymus` (Capacitor + web). ## 3. Provision Workers (MVP) For each of **Cursor**, **Claude**, and **Gemini** (optional sovereign **Ollama**): 1. Create or identify the vendor account under the practitioner or org billing owner. 2. Generate the API key, PAT, or OAuth client secret required by that worker. 3. Store the secret in AWS Secrets Manager / host env under these exact keys (no repo files): | Worker | Env key(s) | Where to place | | --- | --- | --- | | Cursor | `DIDYMUS_WORKER_CURSOR_TOKEN` | `DATARIGHT_DEV_SECRETS` JSON field or compose `environment:` on `gemstone-api` | | Claude | `DIDYMUS_WORKER_CLAUDE_API_KEY` | same | | Gemini | `DIDYMUS_WORKER_GEMINI_API_KEY` (fallback: existing `GEMINI_API_KEY`) | same | | Ollama (optional) | `DIDYMUS_WORKER_OLLAMA_ENDPOINT` | e.g. `http://gemstone-ollama:11434` — omit to use simulated/local default | 4. Confirm the API container can read the key at process start (`docker exec gemstone-api printenv DIDYMUS_WORKER_CLAUDE_API_KEY` should print a non-empty value when provisioned). Values must never appear in task payloads, logs, or MCP arguments. 5. Install/authorize the worker runtime on the intended machine (Cursor IDE agent, Claude tooling, Gemini tooling, or Ollama service). Phase 3C hub workers implement `get_capabilities` / `execute_command` / `report_status` / `verify_output`; live vendor calls run when a matching **vault** key or host env key is present — otherwise execute is **simulated** so gates pass on sovereign hosts. ### 3.0 Practitioner vault (preferred for live execute) 1. Ensure host `.env` (never git) has `WORKER_SECRETS_KEY` (32-byte value, base64). Without it `GET /api/didymus/workers/credentials` reports `vault_available: false` and Connect in the UI will fail closed. 2. Sign in (local login when enabled) or use SuperAdmin/dev principal, open **`/app/onboarding/workers`**, Connect Cursor / Claude / Gemini keys. Values are AES-GCM sealed in `worker_credentials` (last-4 only returned). 3. Host env fallbacks (`DIDYMUS_WORKER_*` / `GEMINI_API_KEY`) still work for operators; Gemini Live voice key satisfies Gemini worker `credential_configured` when vault empty. 4. Confirm `GET /api/didymus/workers` / credentials show `configured: true`. Live path: approve → lease → `POST .../execute` with `worker_id` → `result.mode` is `live` or `simulated`. ### 3.1 Secrets placement (EC2 T4 / OD) ```text # Preferred: merge keys into DATARIGHT_DEV_SECRETS (us-east-1), then restart api # so app.core.secrets.load_aws_application_secrets picks them up. # Alternate: host override file (not git-tracked), e.g. /home/dr/AppDev/gemstone-nexus/.env.didymus DIDYMUS_WORKER_CURSOR_TOKEN=... DIDYMUS_WORKER_CLAUDE_API_KEY=... DIDYMUS_WORKER_GEMINI_API_KEY=... ``` Wire overrides into compose via an untracked env file or `environment:` block — do not add them to committed `docker-compose.yml` with real values. ### 3.2 Practitioner checklist (accounts) 1. **Cursor** — Settings → API / Account → create PAT with agent scopes; store as `DIDYMUS_WORKER_CURSOR_TOKEN`. 2. **Claude** — Anthropic Console → API Keys → create key; store as `DIDYMUS_WORKER_CLAUDE_API_KEY`. 3. **Gemini** — Google AI Studio / Cloud → API key; store as `DIDYMUS_WORKER_GEMINI_API_KEY` (or reuse stack `GEMINI_API_KEY` if already present for voice). 4. Confirm `GET /api/didymus/workers` shows `credential_configured: true` for each provisioned worker. ## 4. Register a Worker MVP workers (`cursor`, `claude`, `gemini`) auto-seed into `didymus_workers` for the default org on API startup. Manual / additional registration: ```bash curl -sS -X POST "http://127.0.0.1:8000/api/didymus/workers/register" \ -H "Content-Type: application/json" \ -H "X-Org-Id: 00000000-0000-4000-8000-000000000001" \ -d '{ "worker_id": "cursor", "family": "cursor", "display_name": "Cursor", "task_types": ["code_edit", "refactor", "generic"], "max_concurrency": 1, "tier_affinity": "any" }' ``` Or MCP: workers call `get_capabilities()` locally; hub stores the row via register. Map High Security workers so they do not call cloud-only paths (`tier_affinity` advisory at Phase 3C). Verify: ```bash curl -sS "http://127.0.0.1:8000/api/didymus/workers" -H "X-Org-Id: <org>" ``` Record registration in the Project Ledger (deploy note) without secret values. ## 5. Submit and Approve Tasks 1. Worker or GemStone Agent submits: - REST: `POST /api/didymus/tasks` `{requested_by, target_worker, task_type, payload}` → `PENDING` - MCP: `POST /mcp/didymus/submit_task` 2. Approver opens **`/app/didymus`**. Default cards are identical (blind-source). Audit metadata is behind an explicit expand — never used for color/badge styling. 3. Approver confirms intent → Approve → `approved_by` + `approved_at` set, status `APPROVED`. 4. Worker leases: `POST /api/didymus/tasks/{id}/lease` → `RUNNING`. `PENDING → RUNNING` returns **409**. Direct SQL/API force without approval fields hits **`didymus_tasks_approval_gate`** (schema CHECK). 5. Worker reports: `POST .../report` → `COMPLETED` | `FAILED`; verify → `VERIFIED`. 6. If a task ever reaches RUNNING without approval fields, execute DOC08 Section 3 immediately. ### 5.1 Example submit (three MVP sources) ```bash ORG=00000000-0000-4000-8000-000000000001 for pair in "cursor-agent|cursor" "claude-agent|claude" "gemini-agent|gemini"; do IFS='|' read -r BY W <<< "$pair" curl -sS -X POST "http://127.0.0.1:8000/api/didymus/tasks" \ -H "Content-Type: application/json" -H "X-Org-Id: $ORG" \ -d "{\"requested_by\":\"$BY\",\"target_worker\":\"$W\",\"task_type\":\"generic\",\"title\":\"Example from $W\",\"payload\":{\"summary\":\"Example from $W\"},\"org_id\":\"$ORG\"}" done ``` ## 6. Approval Queue Ops Checklist | Check | Pass criteria | | --- | --- | | Queue freshness | PENDING items visible within seconds of submit (`GET /api/didymus/tasks`) | | Org isolation | Approver never sees another org's tasks (RLS + `X-Org-Id`) | | Blind source | No origin badge/color in default view; `presentation.kind` always `didymus_task` | | Audit | Expand shows `requested_by` / `target_worker` without styling | | Schema gate | `POST .../force-running` returns `rejected: true, by: schema_constraint` | | Reject / leave PENDING | Leaving PENDING is safe; no auto-escalate to RUNNING | Gate script on host: `bash scripts/phase3c-gates.sh` (requires `gemstone-api` container). ## 7. Credential Rotation 1. Generate new secret at vendor. 2. Update secrets tier first; restart `gemstone-api` (and any external worker process) to pick up the new value. 3. Revoke the old secret at vendor. 4. Confirm `credential_configured: true` and a no-op lease/heartbeat still succeeds. 5. Log rotation in the Project Ledger without recording secret values. ## 8. Local vs Primary Env Notes - Primary: T4 GoldenAMI EC2 `i-04333dae160ab0daa` / `18.116.188.14` (production-like secrets via AWS Secrets Manager). - Local secondary: clean clone from `DataRightGoldenOVAFullBase` only; use host env / local secrets file, never the retired `10.0.0.210` GemStone VM. - Disk on T4 is tight (~3.5G free typical after Whisper CUDA) — prune carefully; keep Whisper CUDA images/models. - Social login testing (Google, Facebook, LinkedIn) uses the Auth0 tenant shared with the app; do not embed social app secrets in the mobile binary beyond Auth0 client config. ## 9. Auth0 Tenant Provisioning (practitioner checklist) **Do not invent or commit tenant credentials.** Placeholders (`AUTH0_*` / `VITE_AUTH0_*`) keep the stack bootable with `configured:false` on `/healthz` until a real tenant exists. ### 9.1 Create the Auth0 application 1. In the Auth0 dashboard, create (or select) a tenant for GemStone / NEXTECH. 2. Create a **Regular Web Application** for the Vite/Capacitor SPA (and optionally a Native app for store builds later). 3. Note **Domain**, **Client ID**, and (server-only) **Client Secret**. Never commit the secret; store it in AWS Secrets Manager / host `.env`. 4. Set **Allowed Callback URLs** to include: - `https://<host>/app/login` - Capacitor deep links when native builds are ready (`com.nextech.gemstone://...`) 5. Set **Allowed Logout URLs** and **Allowed Web Origins** to the same host origins (`https://<host>`, `https://<host>/app`). 6. Enable **Universal Login**. Connect social IdPs: Google, Facebook, LinkedIn (each requires its own vendor OAuth app; store those secrets in Auth0, not in this repo). ### 9.2 API / audience 1. Create an Auth0 **API** with identifier/audience matching `AUTH0_AUDIENCE` (e.g. `https://gemstone.dataright.net/api`). 2. Map roles per DOC03 Section 7: **SuperAdmin**, **Org Admin**, **Practitioner**, **GemStone Agent** (Auth0 Actions or Role claim). 3. Prefer org tenancy via Auth0 Organizations (`auth0_org_id` on `orgs`) when enabling multi-tenant login. ### 9.3 Wire into GemStone (after values exist) 1. Set host secrets (never git): - `AUTH0_DOMAIN`, `AUTH0_AUDIENCE`, `AUTH0_CLIENT_ID`, `AUTH0_CLIENT_SECRET`, `AUTH0_ISSUER`, `AUTH0_ENABLED=true` - `VITE_AUTH0_DOMAIN`, `VITE_AUTH0_CLIENT_ID` (rebuild `web` image after changing Vite args) 2. Redeploy: `.\scripts\run-remote-docker-deploy.ps1` 3. Confirm `/healthz` → `checks.auth0.configured: true` and login at `/app/login` reaches Universal Login. 4. Record completion in the Project Ledger (decision/deploy note) without secret values. ### 9.4 UX reference Login layout target remains **HorizonORBIT** from the **HorizonCRM** repo. Keep the Auth0 stub shell until HorizonCRM is cloned and the layout is ported under DataRightTailwind. ## 10. Out of Scope - Desktop SolidWorks / KiCad workers (DOC02 Section 3). - Autonomous execution of any task category. - Storing credentials in git, Capacitor bundled assets, or Didymus task JSON. - Claude Code / AntiGravity / GemStone-as-worker runtime (deferred until three-worker MVP stable 30+ days). --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 6ba9fd6e1ea95ec0450198a03a25f83213d6faa3 |
| 4 | DOC-AWS-SMS-RECOVERY | DOC-AWS-SMS-Recovery | reference | DevResources/docs/DOC-AWS-SMS-Recovery.md | Agent “text me” uses **Amazon SNS SMS**. The account is still in the **SNS SMS sandbox**. Destination `+14193063078` is stuck **Pending** (OTP never completed). US carriers often filter SNS sandbox short-code OTPs. **Delete/recreate of that sandbox number is blocked for 24h after create** — do not keep resending doomed OTPs to the same number. | # DOC-AWS-SMS-Recovery **Account:** `209457336992` · **Region:** `us-east-1` · **Updated:** 2026-07-22 **Policy:** Never Twilio. Prefer SES email notify until SMS production path works. Do not take down `gemstone.dataright.net`. ## Root cause (why SMS lingered) Agent “text me” uses **Amazon SNS SMS**. The account is still in the **SNS SMS sandbox**. Destination `+14193063078` is stuck **Pending** (OTP never completed). US carriers often filter SNS sandbox short-code OTPs. **Delete/recreate of that sandbox number is blocked for 24h after create** — do not keep resending doomed OTPs to the same number. Parallel path **AWS End User Messaging (EUM) toll-free** `+18447531062` is **PENDING** because registration `registration-19835bd2d54e4914893a45181f098dd3` (`US_TOLL_FREE_REGISTRATION`) is **REQUIRES_UPDATES** after denied **v3**. **SES email notify works** (verified recipients include `mfuriate@nex-tech.biz`, `mfuriate@gmail.com`). Use email until SMS is fixed. ## EUM denial (exact) | Version | Status | Reason | |---------|--------|--------| | v1 | DENIED | Missing conditional field | | v2 | DENIED | Missing required field | | v3 | DENIED | **Business Registration Document Required** | | v4 | **DRAFT** (opened 2026-07-22) | Not submitted — waiting on Mark | ### v3 denial (carrier) > Your company identification number could not be independently verified by the carrier. To resolve this, upload an official business registration document (such as Articles of Incorporation, Certificate of Formation, or equivalent government-issued document) in the **Business Registration Document** field of the Company Info section, or update your company identification number if it was entered incorrectly. After making your changes, resubmit your registration. Docs: [Registering your toll-free number](https://docs.aws.amazon.com/sms-voice/latest/userguide/registrations-tfn-register.html) ### Field gaps vs what AWS wants | Field path | Status on denied v3 / draft v4 | Action | |------------|--------------------------------|--------| | `companyInfo.businessRegistrationDoc` | **Missing** (required by denial) | Mark uploads PDF/PNG/JPEG Articles / Certificate of Formation | | `companyInfo.taxId` | **Missing** (never successfully set; v1 had empty/denied) | Mark sets correct EIN | | `companyInfo.taxIdAuthority` | **Missing** | Set `EIN` | | `companyInfo.taxIdCountry` | **Missing** | Set `US` | | `companyInfo.businessType` | Was `SOLE_PROPRIETOR` on v3 (wrong for LLC) | **Fixed on draft v4 → `PRIVATE_PROFIT`** | | Company / contact / use-case / samples / opt-in image | Present on draft v4 (restored from v3) | Review; tweak only if inaccurate | **CLI cannot finish resubmit** without Mark’s EIN and business registration document file. Do not invent tax IDs or upload placeholder docs. ### Draft v4 (agent prep done) - Registration ID: `registration-19835bd2d54e4914893a45181f098dd3` - Toll-free phone ID: `phone-c8c26941520647f8ad840e9a54c1c589` (`+18447531062`, PENDING) - Opt-in attachment already uploaded: `attachment-9c98f8e16ee748e4ab8c47bbc76518ea` After Mark supplies EIN + registration PDF: ```powershell # Upload doc (example paths — Mark provides file) aws pinpoint-sms-voice-v2 create-registration-attachment --region us-east-1 --attachment-body fileb://C:\path\to\CertificateOfFormation.pdf # Then put fields (use returned attachment id; never echo taxId in chat logs) aws pinpoint-sms-voice-v2 put-registration-field-value --registration-id registration-19835bd2d54e4914893a45181f098dd3 --field-path companyInfo.taxIdAuthority --select-choices EIN --region us-east-1 aws pinpoint-sms-voice-v2 put-registration-field-value --registration-id registration-19835bd2d54e4914893a45181f098dd3 --field-path companyInfo.taxIdCountry --text-value US --region us-east-1 aws pinpoint-sms-voice-v2 put-registration-field-value --registration-id registration-19835bd2d54e4914893a45181f098dd3 --field-path companyInfo.taxId --text-value "<EIN>" --region us-east-1 aws pinpoint-sms-voice-v2 put-registration-field-value --registration-id registration-19835bd2d54e4914893a45181f098dd3 --field-path companyInfo.businessRegistrationDoc --registration-attachment-id attachment-XXXXXXXX --region us-east-1 aws pinpoint-sms-voice-v2 submit-registration-version --registration-id registration-19835bd2d54e4914893a45181f098dd3 --region us-east-1 ``` Or complete the same fields in console: **AWS End User Messaging → Configurations → Registrations** → open the REQUIRES_UPDATES registration → edit draft → submit. ## SNS sandbox → production - `get-sms-sandbox-account-status` → `IsInSandbox: true` - Sandbox destination `+14193063078` → **Pending**; opt-out **false** - There is **no CLI** to exit the SMS sandbox. Support API requires Premium Support (not enabled). Console creates the case. ### Console path (Mark) 1. https://console.aws.amazon.com/sns/v3/home?region=us-east-1#/mobile/text-messaging 2. Under **Account information** → **Exit SMS sandbox** (opens Support / Service quota increase). 3. Form answers for **DataRight agent alerts** (transactional, low volume): | Field | Recommended answer | |-------|-------------------| | Service | SNS Text Messaging | | Website / app | https://dataright.net (OmniFlowCRM / DataRight agent status alerts) | | Message type | **Transactional** (not promotional) | | Region | **us-east-1** | | Countries | United States | | Opt-in | Internal operator (Mark) explicitly requests SMS notify for agent/job completion; not marketing blast; STOP/HELP supported where applicable | | Message templates | `DataRight: Ready — <short status one-liner>` / `DataRight agent: <blocker one-liner>` | | Use case detail | Low-volume transactional alerts from Cursor/DataRight ops agents to an authenticated operator phone; expected volume tens–low hundreds SMS/month | | New limit | Request production access / raise monthly SMS spend as needed | **Note:** AWS docs also expect at least one **Verified** sandbox destination and a successful test SMS before approval. With `+141******3078` stuck Pending, either wait out delete cooldown and re-verify, or add a **second** sandbox number Mark provides (agent must not invent a number). ### Related quota (submitted 2026-07-22) - Service Quotas request **SMS Message Spending in USD** `L-BCD4AAF3`: **$1 → $10**, status **CASE_OPENED**, case `178473400100660`. - This raises spend ceiling; it is **not** the same as exiting the SMS sandbox. Mark still must run **Exit SMS sandbox** in console. ## Delivery logging (done 2026-07-22) Enabled SNS SMS delivery status → CloudWatch Logs: - IAM role: `arn:aws:iam::209457336992:role/SNSSuccessFeedback` - Attributes: `DeliveryStatusIAMRole` set; `DeliveryStatusSuccessSamplingRate=100`; `MonthlySpendLimit=1` (until quota increase approved) Inspect after next SMS attempt: CloudWatch log groups `sns/us-east-1/.../DirectPublishToPhoneNumber` (success/failure). Carriers can delay log appearance up to ~72h. ## Secrets (names only) `DATARIGHT_DEV_SECRETS`: `AWS_EUM`, `AWS_EUM_SMS_NUMBER`, `AWS_EUM_SMS_PHONE_ID`, `AWS_SNS_REGION` (+ `SES_FROM_EMAIL` for email path). Never print values. ## Mark actions (ordered) 1. **Use SES email** for agent notify until SMS works (`mfuriate@nex-tech.biz` / `mfuriate@gmail.com`). 2. **EUM v4:** Provide EIN + official business registration PDF → put `taxId` / `taxIdAuthority=EIN` / `taxIdCountry=US` / `businessRegistrationDoc` → **submit** draft v4. 3. **SNS Exit SMS sandbox** via console form (answers above); optionally reply on spend case `178473400100660`. 4. **Sandbox number:** Do **not** keep OTP-spamming `+141******3078` while delete cooldown blocks reset. After cooldown, delete + recreate **or** give agent a **second** E.164 to verify. 5. After production / verified destination: send one short test SMS; confirm CloudWatch delivery logs. 6. Only then point agent notify / secrets at a working origination path (SNS production and/or registered EUM toll-free). ## Never - Twilio - Print secret values or full tax IDs in chat/logs - Spam SMS / marketing - Treat EUM PENDING toll-free as working agent SMS path - Take down production GemStone hosts for SMS work | b607e823d17035102c3ad26624dda9951f19470b |
| 5 | DOC-ATLAS-DATA-MODEL | Atlas Data Model — Index B Durable Schema | reference | DevResources/docs/DOC-Atlas-Data-Model.md | vault may still be stubbed. GemStone remains an MCP *client*; these Postgres tables are the durable local Index B mirror (org + practitioner segmented). | # Atlas Data Model — Index B Durable Schema **Status:** Alembic `0008_atlas_vault` (revises `0007_notebook_links`) **Purpose:** Concrete SQL / JSON contracts for Atlas build while the remote MCP vault may still be stubbed. GemStone remains an MCP *client*; these Postgres tables are the durable local Index B mirror (org + practitioner segmented). Related: [`DOC-Atlas-Index-B.md`](./DOC-Atlas-Index-B.md) (HTTP MCP contract). ## Tenancy | Claim | Source | Enforcement | | --- | --- | --- | | `org_id` | Local JWT (`/api/auth/login`) | API `tenancy.resolve_tenant`; RLS `app.current_org_id` | | `practitioner_id` | Local JWT / `practitioners` row on signup | Writers + Recents/Search/Atlas recall filters | | Auth0 | **Deferred** | Same claim shape reserved; no tenant dependency | Guest / unauthenticated callers cannot read another practitioner’s Atlas segment. When `AUTH_ENABLED=true`, Recents / Search / notebooks / Index B proxies require Bearer JWT; org spoofing via `X-Org-Id` is ignored. ## Tables ### `atlas_named_sessions` Named/dated session index for `/mcp/index-b/index` + `/recall`. ```sql CREATE TABLE atlas_named_sessions ( entry_id UUID PRIMARY KEY DEFAULT gen_random_uuid(), session_id TEXT NOT NULL, org_id UUID NOT NULL REFERENCES orgs(id) ON DELETE CASCADE, practitioner_id TEXT NOT NULL, title TEXT NOT NULL DEFAULT '', summary TEXT NOT NULL DEFAULT '', named_label TEXT, occurred_at TIMESTAMPTZ NOT NULL DEFAULT now(), tags JSONB NOT NULL DEFAULT '[]', turns_preview JSONB NOT NULL DEFAULT '[]', backend TEXT NOT NULL DEFAULT 'local', created_at TIMESTAMPTZ NOT NULL DEFAULT now(), updated_at TIMESTAMPTZ NOT NULL DEFAULT now(), UNIQUE (org_id, session_id) ); ``` **JSON (Index B upsert mirror):** ```json { "session_id": "uuid-or-text", "org_id": "uuid", "practitioner_id": "uuid", "title": "first user utterance…", "summary": "concatenated turns…", "named_label": "optional human name", "occurred_at": "2026-07-22T15:00:00Z", "tags": [], "turns_preview": [{"role": "user", "text": "…"}] } ``` ### `atlas_session_turns` Durable transcript turns (Gemini Live + sovereign + WS). Survives Redis TTL so Voice Recents restore and Atlas indexing stay available. ```sql CREATE TABLE atlas_session_turns ( turn_id UUID PRIMARY KEY DEFAULT gen_random_uuid(), session_id TEXT NOT NULL, org_id UUID NOT NULL REFERENCES orgs(id) ON DELETE CASCADE, practitioner_id TEXT NOT NULL, turn_seq INT NOT NULL DEFAULT 0, role TEXT NOT NULL CHECK (role IN ('user', 'assistant', 'system')), text TEXT NOT NULL DEFAULT '', source TEXT NOT NULL DEFAULT 'sovereign', meta JSONB NOT NULL DEFAULT '{}', created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); ``` `source ∈ {sovereign, gemini_live, ws_session, api, restore}` ### `atlas_fine_tuning_signals` ```sql CREATE TABLE atlas_fine_tuning_signals ( signal_id UUID PRIMARY KEY DEFAULT gen_random_uuid(), org_id UUID NOT NULL REFERENCES orgs(id) ON DELETE CASCADE, practitioner_id TEXT NOT NULL, session_id TEXT, signal_type TEXT NOT NULL CHECK (signal_type IN ('correction', 'rating', 'explicit_feedback')), original_text TEXT, corrected_text TEXT, rating INT CHECK (rating IS NULL OR (rating BETWEEN 1 AND 5)), context TEXT, metadata JSONB NOT NULL DEFAULT '{}', backend TEXT NOT NULL DEFAULT 'local', created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); ``` ### Notebooks (existing + additive) - `notebooks` — org-scoped; optional `named_label` (0008) - `notebook_session_links` — additive links; **never** removes Recents (US-07) ## Writers (Python) | Path | Module | Notes | | --- | --- | --- | | Redis `append_turn` | `app.session.store` → `vault_store.append_turn` | Best-effort durable mirror | | WS session turns | `app.ws.session` → Index B + vault | Already indexes Atlas stub | | Gemini Live | `app.ws.gemstone_live` + Voice client | `source=gemini_live` | | Corrections | `app.atlas.signals.write_fine_tuning_signal` | Dual-write PG + MCP | | Named index | `vault_store.upsert_named_session` | On turn / end | ## API surface (practitioner-scoped) | Method | Path | Scope | | --- | --- | --- | | GET | `/api/sessions?limit=40` | Recents for JWT org + practitioner | | GET | `/api/sessions/{id}` | Restore buffer; Atlas turns fallback | | GET | `/api/sessions/{id}/turns` | Durable turns for restore | | POST | `/api/search/inferenced` | Atlas + Recents + notebooks for prac | | CRUD | `/api/notebooks*` | Org (+ practitioner on create) | | GET | `/api/notebooks/{id}` | Detail + `sessions[{session_id,title,linked_at}]` for Agent open | | POST | `/api/notebooks/{id}/link` | Additive session link (Recents untouched) | | * | `/mcp/atlas/*` | Index B proxy; JWT org/prac | ## RLS All three Atlas tables ENABLE ROW LEVEL SECURITY with `org_id::text = current_setting('app.current_org_id')`. Application filters by `practitioner_id` for segment isolation within an org. ## Auth (local JWT — no Auth0 dependency) ``` POST /api/auth/signup → provisions org + practitioner + user, returns JWT POST /api/auth/login → JWT { sub, org_id, practitioner_id, role, email } GET /api/auth/me → current principal ``` Set on live: `AUTH_ENABLED=true`, `AUTH_ALLOW_SIGNUP=true`, strong `JWT_SECRET`. | 822c9c1f0ec7e7264dd88e3ea7bda2c1c60090c0 |
| 6 | DOC-ATLAS-INDEX-B | Atlas Vault — MCP Index B Integration (DOC02 Phase 2A) | reference | DevResources/docs/DOC-Atlas-Index-B.md | GemStone is an **MCP client** to a remote Atlas Vault. This document describes the Index B contract GemStone speaks, how to point at a real vault, and the local stub used when no remote endpoint is configured. | # Atlas Vault — MCP Index B Integration (DOC02 Phase 2A) GemStone is an **MCP client** to a remote Atlas Vault. This document describes the Index B contract GemStone speaks, how to point at a real vault, and the local stub used when no remote endpoint is configured. ## Durable data model See **[`DOC-Atlas-Data-Model.md`](./DOC-Atlas-Data-Model.md)** for Postgres tables (`atlas_named_sessions`, `atlas_session_turns`, `atlas_fine_tuning_signals`), Alembic `0008_atlas_vault`, and practitioner/org segmentation via local JWT. Writers dual-write those tables even when this remote Index B endpoint is still the stub. ## Status - **Real remote Atlas Vault:** **DEFERRED on punchlist** (`DOC07-Punchlist.md`). Re-checked `DATARIGHT_DEV_SECRETS` 2026-07-21: 21 keys, **no** `ATLAS_*` (`ATLAS_MCP_ENDPOINT` / `ATLAS_MCP_API_KEY` absent). Phase 2A gates run against the in-process stub vault; stub does **not** block engineering. - **Exact human ask:** add `ATLAS_MCP_ENDPOINT`, `ATLAS_MCP_API_KEY` (optional Bearer), optionally `ATLAS_MCP_MODE=auto` to `DATARIGHT_DEV_SECRETS` (us-east-1), then tell the agent secrets are ready for EC2 `.env` wire + `docker compose up -d --force-recreate api` + `/mcp/atlas/health` not stub. - **Production path:** set env (or secrets) and restart `gemstone-api`. ## Environment | Variable | Purpose | Default | | --- | --- | --- | | `ATLAS_MCP_ENDPOINT` | Base URL of Atlas Vault (no trailing slash), e.g. `https://atlas.example.com` | empty → stub | | `ATLAS_MCP_API_KEY` | Optional Bearer token | empty | | `ATLAS_MCP_MODE` | `auto` \| `stub` \| `remote` | `auto` | | `ATLAS_STUB_DB` | SQLite path for stub vault | `$GEMSTONE_DATA_DIR/atlas_stub.db` | | `ATLAS_QUERY_TIMEOUT_S` | HTTP timeout for remote Index B | `5` | Store secrets in AWS Secrets Manager (`APP_SECRETS_ARN` or `DATARIGHT_DEV_SECRETS`) — never commit them. ### Pointing at a real vault ```bash # on EC2 / in .env (values from secrets manager) ATLAS_MCP_ENDPOINT=https://<atlas-host> ATLAS_MCP_API_KEY=<token> ATLAS_MCP_MODE=auto docker compose up -d --force-recreate api curl -sf http://127.0.0.1:8000/mcp/atlas/health ``` When the remote is unreachable, the client **falls back to stub** for query / signal / index / recall so voice sessions degrade gracefully. ## Index B HTTP contract (remote vault) Relative to `ATLAS_MCP_ENDPOINT`: | Method | Path | Purpose | | --- | --- | --- | | GET | `/mcp/index-b/health` | Health (fallback: `/mcp/health`) | | POST | `/mcp/index-b/query` | Context retrieval | | POST | `/mcp/index-b/signal` | Fine-tuning signal write | | POST | `/mcp/index-b/index` | Upsert session summary for recall | | POST | `/mcp/index-b/recall` | Named/dated session recall | Compatibility fallbacks (prior StackOne-style ledger MCP): - Query → `POST /mcp/query` `{ "question": "..." }` - Signal / index → `POST /mcp/write` into `ContextIndex` ### Query body ```json { "practitioner_id": "uuid", "org_id": "uuid", "query": "vendor contract terms", "limit": 8, "session_id": "optional" } ``` ### Signal body ```json { "practitioner_id": "uuid", "org_id": "uuid", "session_id": "optional", "signal_type": "correction|rating|explicit_feedback", "original_text": "...", "corrected_text": "...", "rating": 4, "context": "optional", "metadata": {} } ``` ### Recall body ```json { "practitioner_id": "uuid", "org_id": "uuid", "query": "the conversation from last Tuesday about the vendor contract", "limit": 5 } ``` ## GemStone proxy routes GemStone exposes the same surface for UI / Capacitor (and Swagger): - `GET /mcp/atlas/health` (also `/api/v1/atlas/health`) - `POST /mcp/atlas/query` - `POST /mcp/atlas/signal` - `POST /mcp/atlas/recall` - `POST /mcp/atlas/index` - `POST /mcp/gemstone/correction` — practitioner correction → signal writer WebSocket session protocol additions: - Client `{"type":"recall","query":"..."}` → server `recall_result` - Turns / end auto-index the session into Atlas for later recall ## Gates ```bash make phase2a-gates # or bash scripts/phase2a-gates.sh ``` Exit criteria (DOC02 §6.4): 1. Index B context queries p95 ≤ 800ms under 50 concurrent practitioners 2. Fine-tuning signal verifiably written within 2s of correction 3. Named/dated recall correct in ≥ 9/10 cases ## Stub vs real vault The stub is **gate scaffolding**, not an embedded production Atlas Vault (DOC06). It persists to `ATLAS_STUB_DB` so soak/restart still exercise recall and signal verification. Replace it by configuring `ATLAS_MCP_ENDPOINT`. | c9903bba6dc92e40d07693016d88d4ca2e96744d |
| 7 | DOC-AUTH0-PROVISIONING | DOC — Auth0 Provisioning (Phase 0 → production closeout) | reference | DevResources/docs/DOC-Auth0-Provisioning.md | On punchlist (`DOC07-Punchlist.md` § Deferred). No live Auth0 credentials exist in AWS Secrets Manager or on the T4 host. Stack remains bootable with placeholders. **Phases 0–5 and production prep continue without a live tenant**; `/healthz` already allows `auth0.configured: false`. | # DOC — Auth0 Provisioning (Phase 0 → production closeout) **Status (2026-07-21):** **DEFERRED — non-blocking for engineering.** On punchlist (`DOC07-Punchlist.md` § Deferred). No live Auth0 credentials exist in AWS Secrets Manager or on the T4 host. Stack remains bootable with placeholders. **Phases 0–5 and production prep continue without a live tenant**; `/healthz` already allows `auth0.configured: false`. **Audience:** Practitioner standing up Auth0 for GemStone on the T4 EC2 / local Golden clone. **Companion:** `DIDYMUS-PLAYBOOK.md` §9 (same steps), `DOC-Deploy-Pipeline.md`, `.env.example`, punchlist Auth0 row. ## Verification snapshot (automatable — done) | Check | Result | |-------|--------| | `DATARIGHT_DEV_SECRETS` (us-east-1 / us-east-2) | **No** `AUTH0_*` / `VITE_AUTH0_*` keys (21 keys total; `AUTH_TOKEN_SECRET` is unrelated) | | Sibling secrets (`Labs4You*`, `RusticRavinesSecrets`, `SagePeak_Dev_Secrets`) | **No** Auth0 keys | | EC2 `/home/dr/AppDev/gemstone-nexus/.env` | Keys declared; **all credential values empty**; `AUTH0_ENABLED=false` | | `GET https://18.116.188.14/healthz` → `checks.auth0` | `configured: false`, detail `AUTH0_DOMAIN/AUDIENCE placeholders — tenant TBD` | Wiring and Management API automation are **not possible** until Mark creates a tenant and stores secrets. ## Exact human actions (Mark) ### 1. Create Auth0 tenant + app + API 1. Sign up / log in at [https://auth0.com](https://auth0.com) (free or paid tenant is fine for closeout). 2. Create a **Regular Web Application** (name e.g. `GemStone Nexus`). 3. Create an **API** with Identifier/Audience, e.g. `https://gemstone.dataright.net/api` (this string becomes `AUTH0_AUDIENCE`). 4. Note from the Application settings: - Domain → `AUTH0_DOMAIN` / `VITE_AUTH0_DOMAIN` (e.g. `your-tenant.us.auth0.com`) - Client ID → `AUTH0_CLIENT_ID` / `VITE_AUTH0_CLIENT_ID` - Client Secret → `AUTH0_CLIENT_SECRET` 5. Set Application URIs (adjust if public URL differs): - Allowed Callback URLs: `https://18.116.188.14/app/callback`, `https://gemstone.dataright.net/app/callback`, `http://localhost:5173/app/callback` - Allowed Logout URLs: `https://18.116.188.14/`, `https://gemstone.dataright.net/`, `http://localhost:5173/` - Allowed Web Origins: same origins as above (no path) ### 2. Social connections + roles 1. Enable **Universal Login**. 2. Enable social connections: **Google**, **Facebook**, **LinkedIn** (developer consoles + Auth0 connection setup). 3. Define Auth0 roles (DOC03 §7): `SuperAdmin`, `Org Admin`, `Practitioner`, `GemStone Agent`. 4. (Optional) Auth0 Organizations → map to `orgs.auth0_org_id` + Postgres RLS GUC `app.current_org_id`. ### 3. Store secrets (required before agent can wire EC2) Update AWS Secrets Manager secret **`DATARIGHT_DEV_SECRETS`** (region **us-east-1**; mirror to us-east-2 if that copy is kept in sync) with these **exact key names** (string values, no placeholders): ```text AUTH0_DOMAIN AUTH0_AUDIENCE AUTH0_CLIENT_ID AUTH0_CLIENT_SECRET AUTH0_ISSUER # usually https://<AUTH0_DOMAIN>/ AUTH0_ENABLED # true VITE_AUTH0_DOMAIN VITE_AUTH0_CLIENT_ID ``` **Do not** commit these to git. Host `.env` is filled from Secrets Manager / deploy scripts. ### 4. Reply in chat when ready Tell the agent: **“Auth0 secrets are in DATARIGHT_DEV_SECRETS”** (no need to paste values). Next agent turn will: pull keys → write EC2 `.env` → recreate `api` (+ rebuild `web` for Vite args) → verify `/healthz` `auth0.configured: true` → optional token smoke. ## Why placeholders are intentional until then `/healthz` reports `auth0.configured: false` when `AUTH0_DOMAIN` / `AUTH0_AUDIENCE` are empty. That is **not** a Phase 0 failure and **does not block** Atlas, pen-test prep, BABA checklist, store assembly, device-UAT docs, or load-test prep: compose and required deps (Postgres, Redis, ledger) stay green while the tenant is provisioned out-of-band. User decision 2026-07-21: place Auth0 on the punchlist and move on. ## Login UI note HorizonORBIT login layout lives in **HorizonCRM** (not cloned here). Until that repo is available, `/app/login` remains a DataRightTailwind-compatible stub pointing at Auth0 once configured. ## Related env keys See `.env.example` / `DevResources/project.env.example` for `AUTH0_*` and `VITE_AUTH0_*`. Leave blank until the tenant exists. | b98d20f1e7b6a778541bbc7bcb2d2a6473da069b |
| 8 | DOC-BABA-WESTERN-ORIGIN-CHECKLIST | BABA Western-Origin Checklist (Phase 5 engineering pass) | reference | DevResources/docs/DOC-BABA-Western-Origin-Checklist.md | | Component | Origin claim | Evidence in repo | Engineering status | | --- | --- | --- | --- | | Whisper large-v3 | OpenAI (US) | DOC18 §1; whisper service | Confirmed in DOC18 baseline | | Piper TTS | Rhasspy (US/Intl OSS) | DOC18 §1; `voice/` | Confirmed in DOC18 baseline | | Ollama | Ollama Inc. (US) | DOC18 §1; compose | Confirmed in DOC18 baseline | | webrtcvad | Google (US) | DOC18 §1 | Confirmed in DOC18 baseline | | FastAPI / SQLModel | OSS | DOC18 | Confirmed | | Po… | # BABA Western-Origin Checklist (Phase 5 engineering pass) *Companion to DOC17. Principal sign-off remains required — this checklist is the engineering evidence package, not a substitute for NEXTECH principal audit.* **Date:** 2026-07-21 · **Host evidence:** T4 GoldenAMI `i-04333dae160ab0daa` ## Engineering confirmation (automatable / repo-verified) | Component | Origin claim | Evidence in repo | Engineering status | | --- | --- | --- | --- | | Whisper large-v3 | OpenAI (US) | DOC18 §1; whisper service | Confirmed in DOC18 baseline | | Piper TTS | Rhasspy (US/Intl OSS) | DOC18 §1; `voice/` | Confirmed in DOC18 baseline | | Ollama | Ollama Inc. (US) | DOC18 §1; compose | Confirmed in DOC18 baseline | | webrtcvad | Google (US) | DOC18 §1 | Confirmed in DOC18 baseline | | FastAPI / SQLModel | OSS | DOC18 | Confirmed | | PostgreSQL | PGDG | compose / Alembic | Confirmed | | Redis | Redis Ltd. | compose (AOF) | Confirmed | | Capacitor / Ionic | Ionic (US) | `frontend/package.json` | Confirmed | | MQTT | Eclipse Mosquitto only (if/when introduced) | Not in MVP compose | N/A for current MVP | | AWS messaging | AWS-native only (SNS/SQS/EventBridge) | EventBridge start/stop in ops | Confirmed for schedule; no China-region messaging | | HUME AI | Must remain out of scope | DOC03 / DOC17 | Confirmed not integrated | ## Principal sign-off (practitioner — cannot complete in-agent) | Item | Status | | --- | --- | | Cross-check Syft SBOM (`DevResources/security/sbom-latest.json`) against this table | Pending — on punchlist | | Sign DOC17 checklist rows | Pending — on punchlist | | Record sign-off date + name in Project Ledger | Pending — on punchlist | **Ask:** NEXTECH principal reviews this table + SBOM and signs DOC17; reply in chat when signed so ledger/DOC07 can be updated. ## SBOM generation ```bash bash scripts/security/sbom-stub.sh # Install Syft if skipped: https://github.com/anchore/syft (Western-origin tooling) ``` --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | ef59d1c1ed20c9d65caca9d8393dddb5b1d3efd9 |
| 9 | DOC-CAPACITOR-LIVE-EC2 | Capacitor Android / iOS against live EC2 | reference | DevResources/docs/DOC-Capacitor-Live-EC2.md | Auth0 deferred (not required). Device UAT still open (`DOC-Device-UAT-Checklist.md`). | # Capacitor Android / iOS against live EC2 **Status 2026-07-21:** Engineering wiring COMPLETE for parallel-with-live-stack debug. Auth0 deferred (not required). Device UAT still open (`DOC-Device-UAT-Checklist.md`). **TLS (2026-07-21):** Live host serves a **Let's Encrypt** cert for `gemstone.dataright.net` (HTTP-01 via `scripts/aws/provision-letsencrypt.sh`). Prefer that hostname over the raw Elastic IP for Capacitor and browsers. **Full workstation install playbook:** [`CAPACITOR-ANDROID-INSTALLATION-PLAYBOOK.md`](CAPACITOR-ANDROID-INSTALLATION-PLAYBOOK.md). ## How Android runs against live EC2 Default debug path loads the **remote** SPA so asset `base` stays `/app/` for nginx web: 1. `frontend/capacitor.config.json` sets `server.url` → `https://gemstone.dataright.net/app` (override: `$env:CAPACITOR_SERVER_URL=...; npm run cap:sync` / `$env:CAPACITOR_USE_BUNDLED=1; npm run cap:sync` via `scripts/apply-capacitor-server.cjs`). 2. With the public LE cert, **debug `network_security_config` / `ec2_debug.crt` are obsolete for `gemstone.dataright.net`** (system CAs trust the chain). Those debug assets remain only as a fallback if you still hit the raw IP (`https://18.116.188.14`) with a self-signed leaf. Release builds never included the debug trust set. 3. API clients use `src/services/apiOrigin.js` — same-origin when on the live WebView; `VITE_API_BASE` or live default when bundled on `https://localhost`. ### Open in Android Studio ```powershell cd frontend npm install npm run build npx cap sync android npm run cap:open:android ``` Then Run ▶ **app** (debug) on an emulator or USB device. First load hits the live `/app/` SPA at `https://gemstone.dataright.net/app/`. ### assembleDebug without Studio ```powershell cd frontend\android .\gradlew.bat assembleDebug # APK: app\build\outputs\apk\debug\app-debug.apk ``` ### iOS (macOS) ```bash cd frontend && npm run build && npx cap sync ios && npm run cap:open:ios ``` Public CA hostname needs no custom trust store. Raw-IP / self-signed remains an ATS / trust-store problem (see `Info.plist` exception for the EIP only). ### Overrides | Knob | Effect | | --- | --- | | `CAPACITOR_SERVER_URL` | Alternate live SPA URL (no trailing slash required) | | `CAPACITOR_USE_BUNDLED=1` | Drop `server.url`; load `dist/` from the binary | | `VITE_API_BASE` | Force REST/WS origin (no trailing slash), e.g. `https://gemstone.dataright.net` | | Vite `base` | Keep `/app/` for web/nginx; bundled-native path later may use `./` | ### Web `/app/` unchanged Vite `base: "/app/"` and `BrowserRouter basename="/app"` stay for EC2 nginx. Capacitor does **not** rely on a second Vite base while `server.url` points at live. | 12754381529d4f655ec67055f7af74cff171fc19 |
| 10 | DOC-DEPLOY-PIPELINE | GemStone Deploy Procedure (Stage 2) | reference | DevResources/docs/DOC-Deploy-Pipeline.md | | Role | Where | |------|--------| | Author / edit | Windows workstation (this repo) | | Git | Local commit → `git push` with PAT (from `DATARIGHT_DEV_SECRETS`, never stored in repo) | | Build / run / test | T4 GoldenAMI EC2 Ohio (`i-04333dae160ab0daa`) over plink/SSH | | Local Docker | **Not used** on the co-dev PC | | # GemStone Deploy Procedure (Stage 2) **Status:** Operating model wired; host half verified against T4 EC2. **No commit required for Stage 2** — first full Deploy of product code waits until Stage 3 bootstrap content is committed and pushed. ## Operating model | Role | Where | |------|--------| | Author / edit | Windows workstation (this repo) | | Git | Local commit → `git push` with PAT (from `DATARIGHT_DEV_SECRETS`, never stored in repo) | | Build / run / test | T4 GoldenAMI EC2 Ohio (`i-04333dae160ab0daa`) over plink/SSH | | Local Docker | **Not used** on the co-dev PC | **Deploy** = commit → push → pull-on-host → `docker compose up` → restart/health. ## Fixed paths / hosts (Stage 2 verify) | Item | Value | |------|--------| | Instance | `i-04333dae160ab0daa` (us-east-2) | | Public IP (verified) | `18.116.188.14` (re-check after stop/start if no EIP sticky) | | SSH (primary) | `dr` + `DR_SSH_PASSWORD` via plink; host key pinned | | SSH (break-glass) | `ubuntu` + `C:\Users\Mark\.ssh\gemstone-nexus-key.pem` | | Host key (ED25519) | `SHA256:TSExVVo9lcc48zJsQSrpXxEIHe8Wv7+qDTBl1MHCilw` | | `DEVELOPMENT_DIRECTORY` | `/home/dr/AppDev/gemstone-nexus` | | Git remote on host | `https://github.com/mfuriate/GemStone-Nexus-StackOne.git` (clean URL; PAT only at pull time via Basic auth) | | Compose nginx | Container `gemstone-nginx` publishes host `80`/`443` | | Supervisor | Stage 3 — inside API image / `supervisor/` scaffold; not host-level in Stage 2 | ## Scripts | Script | Purpose | |--------|---------| | `scripts/remote-exec.ps1` | Ad-hoc plink/OpenSSH command; `-RefreshHostKey` after instance replace | | `scripts/ensure-ec2-dev-dir.ps1` | Clone/pull `DEVELOPMENT_DIRECTORY`; PAT from Secrets Manager | | `scripts/run-remote-docker-deploy.ps1` | Host half of Deploy (pull → compose up → health) | | `scripts/aws/deploy.sh` | Same contract when already SSH'd onto the host as `dr` | Config template: `DevResources/project.env.example` → copy to `DevResources/project.env` (gitignored). ## Practitioner Deploy steps 1. **Commit** locally (when you intend to ship a change set). Do not commit secrets. 2. **Push** to `origin` (`git push`). PAT stays in credential helper / Secrets Manager — not in `.git/config` on the EC2. 3. **Pull + compose on host** from Windows: ```powershell .\scripts\run-remote-docker-deploy.ps1 ``` Or on the host: ```bash export GITHUB_PAT=... # from Secrets Manager session ./scripts/aws/deploy.sh ``` 4. Confirm `docker compose ps` and `/healthz` (full green expected after Stage 3). Optional host gates: ```bash make phase0-gates # healthz + WS echo <200ms make rollback # restore prior SHA + compose; budget <5 min make soak-start # 10-min healthy soak in background ``` Optional: `-PushLocal` pushes current `HEAD` before the host pull. ## Host key hygiene After EIP reuse or AMI relaunch, fingerprints change: ```powershell .\scripts\remote-exec.ps1 -RefreshHostKey # then update DR_SSH_HOSTKEY in DevResources/project.env ``` OpenSSH: `StrictHostKeyChecking=accept-new` only after removing the stale `known_hosts` line for that IP (`ssh-keygen -R <ip>`). ## Cost schedule (do not disable casually) EventBridge Scheduler (us-east-2), both **ENABLED**: - `gemstone-nexus-stop` — `cron(0 1 * * ? *)` America/New_York - `gemstone-nexus-start` — `cron(0 9 * * ? *)` America/New_York Terraform: `launch_instance = true` is intentional while this host is live — see `infra/terraform/ohio/README.md`. **Spot capacity note (2026-07-21):** After Stage 2 connectivity verify, the instance entered `stopped` with spot request status `marked-for-stop` ("no Spot capacity available"). `StartInstances` returns `IncorrectSpotRequestState` until capacity returns. Worktree `/home/dr/AppDev/gemstone-nexus` was cloned before the stop; re-run `ensure-ec2-dev-dir.ps1` after the next successful start. Do **not** destroy/recreate via Terraform without approval. ## Local secondary `GemStone-Golden` at `D:\Virtual Machines\GemStone-Golden\` — last IP `10.0.0.224`, `dr` contract PASSED (see `DOC-GemStone-Golden-Host-Verify.md`). Offline secondary; not the Deploy target. | 111e509f7bf55afdd91587117a17bce0366d69e7 |
| 11 | DOC-DEVICE-UAT-CHECKLIST | Device UAT Checklist (DOC20 mobility + Phase 2B/2C gates) | reference | DevResources/docs/DOC-Device-UAT-Checklist.md | | Check | Notes | | --- | --- | | Capacitor Android project present | `frontend/android` after `npm run cap:sync` | | Capacitor iOS project present | `frontend/ios` — requires macOS + Xcode + CocoaPods | | Staging `/healthz` green | Auth0 may still be `configured:false` (deferred); not a device-UAT blocker for UI/mobility scripts | | Android debug → live EC2 | See `DOC-Capacitor-Live-EC2.md`: `server.url` → `/app`, debug TLS trust for `18.116.188.14` | | Physical devices | O… | # Device UAT Checklist (DOC20 mobility + Phase 2B/2C gates) *Practitioner / QA only — cannot be completed by the agent. Engineering scaffolds are in place.* **Date:** 2026-07-21 · **App ID:** `com.nextech.gemstone` · **Staging:** `https://18.116.188.14/app` (or current public URL) · **Punchlist:** `DOC07-Punchlist.md` ## Preconditions | Check | Notes | | --- | --- | | Capacitor Android project present | `frontend/android` after `npm run cap:sync` | | Capacitor iOS project present | `frontend/ios` — requires macOS + Xcode + CocoaPods | | Staging `/healthz` green | Auth0 may still be `configured:false` (deferred); not a device-UAT blocker for UI/mobility scripts | | Android debug → live EC2 | See `DOC-Capacitor-Live-EC2.md`: `server.url` → `/app`, debug TLS trust for `18.116.188.14` | | Physical devices | One Android (recent API) + one iOS; outdoor midday Florida sun for Field Mode | ### Quick start — Android Studio against live stack ```powershell cd frontend npm run build npx cap sync android npm run cap:open:android ``` Run the **debug** variant (release does not ship the self-signed trust config). ## Script A — Field Mode sunlight (DOC20 §5.1–5.2) | Step | Expected | Result (initials / date) | | --- | --- | --- | | Outdoor midday sun, open Field Mode | High-contrast Heads-Up UI legible | _ | | Tap center-screen target | Ascending chime + recording start ≤ 1s | _ | ## Script B — Extended Session + call interruption (DOC20 §5.3–5.4 / DOC02) | Step | Expected | Result | | --- | --- | --- | | Start Extended Session; wait one mic-open cue | Cue audible | _ | | Incoming call during session (10 trials) | Pause clean; one-tap resume; context intact | _ / 10 | ## Script C — Android wake-word (DOC02 Phase 2C) | Step | Expected | Result | | --- | --- | --- | | Force-stop app; speak wake phrase (quiet room) | Foreground-service wake succeeds | _ | | Repeat to 10 trials | ≥ **8/10** | _ / 10 | ## Script D — iOS Siri Shortcut (DOC02 Phase 2C) | Step | Expected | Result | | --- | --- | --- | | Install Siri Shortcut from app scaffold | Shortcut fires GemStone session entry | _ | | Repeat to 10 trials | ≥ **9/10** | _ / 10 | ## Script E — Core session resilience (DOC20 §1) on device | Cause | Restore to ACTIVE with last exchange | Result | | --- | --- | --- | | Background 2 min | Yes, no re-auth | _ | | Incoming call | Identical | _ | | Forced network drop | Identical | _ | ## Sign-off | Role | Name | Date | | --- | --- | --- | | QA / practitioner | | | | Engineering witness (optional) | | | ## Explicit non-claim Until the rows above are signed, Phase 2B/2C **device QA gates** remain open even though engineering COMPLETE on EC2. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | d8fdae285b8ffa92735c6b7f2ecb5779e15f38e4 |
| 12 | DOC-GEMSTONE-GOLDEN-HOST-VERIFY | DOC GemStone Golden Host Verify | reference | DevResources/docs/DOC-GemStone-Golden-Host-Verify.md | # GemStone-Golden host contract verify | # GemStone-Golden host contract verify **Date:** 2026-07-21 **VM:** `D:\Virtual Machines\GemStone-Golden\GemStone-Golden.vmx` **IP (bridged, this session):** `10.0.0.224` (DHCP may change) **Auth:** `dr` via Secrets Manager `DATARIGHT_DEV_SECRETS` / `DR_SSH_PASSWORD` — succeeded (no secrets in this file) ## Parity checklist | Check | Result | Notes | |-------|--------|-------| | OS | PASS | Ubuntu 24.04.4 LTS (Noble) | | Hostname | PASS | `dataright-golden` (GoldenOVA identity) | | `dr` user / sudo / docker group | PASS | uid 1000; passwordless sudo | | Docker runnable | PASS | Engine 27.5.1; Compose v5.1.4; 0 images (clean) | | AppDev layout | PASS | `/home/dr/AppDev` exists; also `admin`, `gemstone`, `/opt/nextech` | | Disk free | PASS | Root LV ~62G, ~42G avail (~30% used) | | Memory / CPU | PASS (base) | Guest: 3.8Gi / 2 vCPU (VMX: memsize 4096, numvcpus 2) | | VMware Tools | PASS | open-vm-tools active | | Soft power-off | PASS | Left off for practitioner | ## Gaps / follow-ups - Hostname still `dataright-golden` (not `gemstone-golden`) — cosmetic unless docs require rename. - AppDev is empty placeholder; stack not bootstrapped yet. - No Docker images yet — expected after clean clone. - ~63G unallocated in ubuntu-vg on 128G disk — grow LV before expanding VMDK. - Recommended resize (powered off): 8 vCPU, 16 GB RAM; LV grow on existing disk; VMDK→256G only if models/images need it. ## Non-actions - No commit. Retired VM not deleted. Resize not applied this session. | 750e476a08b906659b5a2f7215a1dc8e7ef014bf |
| 13 | DOC-PENTEST-ENGAGEMENT-BRIEF | Phase 5 — Penetration Test Engagement Brief | reference | DevResources/docs/DOC-PenTest-Engagement-Brief.md | - Independent third-party assessment covering OWASP Top 10 at minimum. - All Critical and High findings remediated and re-tested before production. - Medium findings: documented remediation plan with owner + date (DOC12 / punchlist). | # Phase 5 — Penetration Test Engagement Brief *DOC16 companion. Agent cannot execute a third-party pen test — this brief is the handoff package.* **Initiative:** GemStone v2.0 · **Date:** 2026-07-21 · **Status:** Ready for vendor engagement (Auth0 live tenant optional) ## 1. Objectives - Independent third-party assessment covering OWASP Top 10 at minimum. - All Critical and High findings remediated and re-tested before production. - Medium findings: documented remediation plan with owner + date (DOC12 / punchlist). ## 2. Scope (in) | Surface | Notes | | --- | --- | | REST ` /api/*` | Dashboard, packs, Didymus, mobility, gemstone ops | | WebSocket `/ws/*` | Session manager, reconnect/grace, org isolation | | Capacitor WebView origins | `com.nextech.gemstone` web assets under `/app` | | RLS / `org_id` isolation | Cross-org under concurrent load | | TierGuard | Sole enforcement; immutability of session tier | | Didymus approval gate | RUNNING requires `approved_by` + `approved_at` | ## 3. Scope (out / deferred) | Item | Reason | | --- | --- | | Production Auth0 social IdP misconfig deep-dive | Live tenant not required for Phase 0–4 engineering close; include when Auth0 provisioned | | Physical device jailbreak / rooted Android | Device UAT is practitioner (DOC20) | | Social engineering of NEXTECH staff | Separate engagement | ## 4. Staging access (practitioner fills) | Field | Value | | --- | --- | | Staging URL | `_TBD_` (e.g. `https://GemStone.DataRight.net` or dedicated staging) | | Auth0 test tenant | `_TBD_` — no production secrets in ticket | | Test org IDs | `_TBD_` | | Contact | NEXTECH principal | | Freeze tag | After Phase 5 engineering green: annotate tip SHA | **Engineering prep status (2026-07-21):** Brief ready for vendor send. Staging IP currently `https://18.116.188.14` (self-signed). Auth0 live tenant deferred on punchlist — mark Auth0 deep-dive out-of-scope or schedule follow-on. SBOM path: `DevResources/security/sbom-latest.json`. Rollback path documented in playbook / `make rollback`. ## 5. Standing absolute rules (must not be violated in testing) Per DOC16: no session transcripts or raw audio to external APIs; AES-256 at rest; TLS 1.3 in transit; human approval before any Didymus RUNNING. ## 6. Deliverables expected from vendor 1. Executive summary + detailed findings with CVSS/severity. 2. Reproduction steps for Critical/High. 3. Re-test attestation after remediation. 4. Medium remediation plan template (owner, date, residual risk). ## 7. Engineering pre-flight checklist (complete before kickoff) - [ ] Release candidate tag frozen and recorded in DOC12 - [ ] Staging URL reachable; `/healthz` green - [ ] Auth0 test tenant credentials in secrets vault (not repo) - [ ] Backup / rollback path verified (`make rollback`) - [ ] SBOM artifact present (`DevResources/security/sbom-latest.json`) --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 882de38c41eb0c9c2d5eb54e3db4d535c96adb9d |
| 14 | DOC-PHASE4-GATE | DOC-Phase4-Gate | reference | DevResources/docs/DOC-Phase4-Gate.md | | View | API | UI | | --- | --- | --- | | Approval Queue | `/api/dashboard/approval-queue` | `/app/dashboard` | | Worker Health | `/api/dashboard/worker-health` | same | | Tier Distribution | `/api/dashboard/tier-distribution` | same | | Open Mode Consent Audit | `/api/dashboard/open-mode-consent` | same | | Research Agent Volume | `/api/dashboard/research-volume` | same | | Audio Pipeline Health | `/api/dashboard/audio-pipeline` | same | | Wake-Word Tuning | `/api/dashboard… | # DOC-Phase4-Gate *DOC02 Phase 4 — Platform Dashboard + polish* **Host:** `i-04333dae160ab0daa` / `18.116.188.14` **Tip:** `a2f767d` (2026-07-21) **Capacitor:** `npx cap sync android` PASS in web image build (vitest 20/20) ## DOC21 views shipped | View | API | UI | | --- | --- | --- | | Approval Queue | `/api/dashboard/approval-queue` | `/app/dashboard` | | Worker Health | `/api/dashboard/worker-health` | same | | Tier Distribution | `/api/dashboard/tier-distribution` | same | | Open Mode Consent Audit | `/api/dashboard/open-mode-consent` | same | | Research Agent Volume | `/api/dashboard/research-volume` | same | | Audio Pipeline Health | `/api/dashboard/audio-pipeline` | same | | Wake-Word Tuning | `/api/dashboard/wake-word` | same | | Domain Pack Adoption | `/api/dashboard/pack-adoption` | same | Aggregate: `GET /api/dashboard` → `view_count: 8` ## Gates | Gate | Result | | --- | --- | | `python -m app.dashboard.phase4_bench` | **PASS** (dashboard + automatable DOC20) | | DOC20 §2 Domain Packs | PASS (6/6, memory) | | DOC20 §4 Didymus blind queue | PASS | | DOC21 eight views | PASS | | DOC20 §1 / §3 / §5 | practitioner_signoff (device / sunlight / call) | | WCAG AA | engineering polish (skip link, focus-visible, Field Mode contrast retained) | | DataRightTailwind | tokens + semantic classes in Capacitor web | | Auth0 login | HorizonORBIT-style stub; no fake tenant | | Whisper CUDA image | retained | ## Disk after deploy ~4.0G free on `/` (was ~2.0G critical; build-cache prune + api/web-only rebuild). Whisper CUDA kept. ## Phase 5 Scaffold only — see `DOC-Phase5-Hardening-Scaffold.md`. External pen test / store / 10k load not executed. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 021f032bf7e5c991bce811a85222c8c14ada77e8 |
| 15 | DOC-PHASE5-HARDENING-SCAFFOLD | Phase 5 Hardening — Status | reference | DevResources/docs/DOC-Phase5-Hardening-Scaffold.md | | Work item | Engineering (this pass) | Practitioner / vendor | | --- | --- | --- | | Playwright E2E ×2 clean | `frontend/e2e/` critical paths + `npm run test:e2e:twice` | Device UAT (DOC20) still practitioner | | Third-party pen test | Checklist + engagement brief | Engage vendor — agent cannot execute | | 10k-session load (p95 < 3s) | Scaled proxy + JSON report (`scripts/load/phase5_load_stub.py`) | Full 10k voice soak after disk/RAM headroom | | Syft SBOM + BABA | Gene… | # Phase 5 Hardening — Status *DOC02 Phase 5 — engineering pass vs practitioner/vendor gates* **Status:** Engineering complete (automatable) · Full DOC02 exit **not** claimed · **Date:** 2026-07-21 ## Scope (DOC02 Phase 5) | Work item | Engineering (this pass) | Practitioner / vendor | | --- | --- | --- | | Playwright E2E ×2 clean | `frontend/e2e/` critical paths + `npm run test:e2e:twice` | Device UAT (DOC20) still practitioner | | Third-party pen test | Checklist + engagement brief | Engage vendor — agent cannot execute | | 10k-session load (p95 < 3s) | Scaled proxy + JSON report (`scripts/load/phase5_load_stub.py`) | Full 10k voice soak after disk/RAM headroom | | Syft SBOM + BABA | Generator + Western-origin checklist | Principal sign-off (DOC17) | | App Store / Play submission | Prep checklist only | Apple/Google accounts required | ## Commands / evidence (2026-07-21) | Check | Result | | --- | --- | | Playwright `npm run test:e2e:twice` | **PASS** — 9/9 then 9/9 (critical paths + smoke) | | Scaled load proxy 200× `/healthz` via HTTPS nginx on `18.116.188.14` | **PASS proxy** — ok 200/200, p95 683ms; `pass_doc02_full_gate: false` | | Root disk on EC2 | ~4.0G free on `/` — no Playwright browsers / no 10k voice soak on host | | Syft SBOM | Skip stub written (`syft` not on PATH) — principal still signs DOC17 | | Capacitor `cap sync` after naming purge | **PASS** (`com.nextech.gemstone` / GemStone) | | Legacy name purge gate | **PASS** (DOC12 ≤2 historical footnotes) | ```bash # Playwright (local or CI with browsers installed — avoid on T4 until ≥8G free) cd frontend && npm ci && npx playwright install chromium npm run build && npm run test:e2e:twice # Load proxy (not the DOC02 10k voice gate) python scripts/load/phase5_load_stub.py --sessions 200 --base https://18.116.188.14 --path /healthz --insecure # SBOM powershell -File scripts/security/sbom-stub.ps1 # or: bash scripts/security/sbom-stub.sh # Legacy product-name purge gate powershell -File scripts/ci/forbid-legacy-product-name.ps1 ``` ## Related docs - `DOC-PenTest-Engagement-Brief.md` - `DOC-Store-Submission-Checklist.md` - `DOC-BABA-Western-Origin-Checklist.md` - DOC14 / DOC16 / DOC17 / DOC18 status sections Do not pull Playwright browsers onto the T4 until root disk has ≥8G free. Keep Whisper CUDA; prune carefully. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 47390fbd4825204ee492912ead73371d351ccf4a |
| 16 | DOC-STORE-SUBMISSION-CHECKLIST | Phase 5 — App Store / Play Store Submission Prep | reference | DevResources/docs/DOC-Store-Submission-Checklist.md | | Blocker | Status | | --- | --- | | Apple Developer Program enrollment | **On punchlist** — required for App Store Connect | | Google Play Console developer account | **On punchlist** — required for Play upload | | Auth0 production tenant + Universal Login | Deferred punchlist — `DOC-Auth0-Provisioning.md` | | Privacy policy + support URLs hosted | TBD by NEXTECH | | Signing keys (iOS distribution / Android upload keystore) | Must live in secrets vault — never in repo | | # Phase 5 — App Store / Play Store Submission Prep *Cannot submit without Apple Developer + Google Play Console accounts. This checklist is assembly-only.* **Date:** 2026-07-21 · **App ID:** `com.nextech.gemstone` · **App name:** GemStone ## Blockers (practitioner / accounts) | Blocker | Status | | --- | --- | | Apple Developer Program enrollment | **On punchlist** — required for App Store Connect | | Google Play Console developer account | **On punchlist** — required for Play upload | | Auth0 production tenant + Universal Login | Deferred punchlist — `DOC-Auth0-Provisioning.md` | | Privacy policy + support URLs hosted | TBD by NEXTECH | | Signing keys (iOS distribution / Android upload keystore) | Must live in secrets vault — never in repo | **Ask when ready:** Create Apple + Google developer accounts; store signing material in Secrets Manager (not git); tell agent accounts exist so upload steps can be walked through against `DOC-Store-Submission-Checklist.md`. ## Engineering package (automatable / done when Capacitor sync green) | Item | Location / command | | --- | --- | | Capacitor config | `frontend/capacitor.config.json` (`appId` / `appName` = GemStone) | | Web build | `cd frontend && npm run build` | | Capacitor sync | `cd frontend && npm run cap:sync` | | Android project | `frontend/android` (after `cap add android` on build host) | | iOS project | `frontend/ios` (requires macOS + Xcode / CocoaPods) | ## Store listing draft fields | Field | Draft | | --- | --- | | Name | GemStone | | Short description | Voice-first practitioner platform with Domain Packs and approval orchestration | | Category | Business / Productivity | | Content rating | Complete questionnaires in consoles (no UGC chat rooms; voice sessions practitioner-scoped) | | Data safety / privacy nutrition | Session audio discarded post-transcription; no external transcript APIs (DOC16) | | Screenshots | Capture Field Mode, Dashboard, Approval Queue, Domain Packs on device | ## Submission steps (when accounts exist) ### Apple App Store 1. Archive from Xcode with distribution signing. 2. Upload via Transporter / Xcode Organizer. 3. Complete App Privacy, export compliance (encryption), background audio entitlement justification (Extended Sessions only). 4. Submit for review — timelines outside NEXTECH control (DOC02 §4). ### Google Play 1. Generate signed AAB (`bundleRelease`). 2. Create internal testing track first; promote to production after smoke. 3. Complete Data safety form aligned with DOC16. 4. Declare foreground-service mic notification for wake-word (GS-MOB-01). ## Explicit non-claim This document does **not** claim App Store or Play Store submission. DOC02 Phase 5 exit requires actual submission packages submitted — blocked on accounts listed above. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 8fe58348396d75e4af82fbe82ace175df9050442 |
| 17 | DOC-SUITE-GAP-SUMMARY | DOC Suite Gap Summary | reference | DevResources/docs/DOC-Suite-Gap-Summary.md | | Capability | Doc source | Live code | Status | |---|---|---|---| | Gemini Live voice (`/ws/gemstone/live`) | contract-notes, punchlist | `api/app/ws/gemstone_live.py`, `web/gemstone-live.js` | **Done** | | Sovereign Whisper/Piper wake loop | DOC03 §2, contract-notes | `whisper_stt`, `piper_tts`, `gemstone-voice.js` | **Done** (primary path is Live in development mode) | | Recents ≥20 + notebooks + inferenced search | DOC03/04 US-07/08, punchlist | `/api/sessions`, `/api/no… | ## Missing / Partial / Done matrix (high signal) | Capability | Doc source | Live code | Status | |---|---|---|---| | Gemini Live voice (`/ws/gemstone/live`) | contract-notes, punchlist | `api/app/ws/gemstone_live.py`, `web/gemstone-live.js` | **Done** | | Sovereign Whisper/Piper wake loop | DOC03 §2, contract-notes | `whisper_stt`, `piper_tts`, `gemstone-voice.js` | **Done** (primary path is Live in development mode) | | Recents ≥20 + notebooks + inferenced search | DOC03/04 US-07/08, punchlist | `/api/sessions`, `/api/notebooks*`, `/api/search/inferenced`, Agent sidebar Recents (`/app/recents` redirects) | **Done** | | Domain Packs (6 first-party) install/uninstall | DOC03 §10–11, Phase 3A/3B | `api/app/packs/*`, `/app/packs`, Alembic `0002` | **Done** (self-install only) | | Didymus approval gate + blind queue | DOC03 §6/12, DOC26 | Alembic `0003`, `/api/didymus/*`, `/app/didymus` | **Done** (schema gate) | | Didymus live worker execute | DIDYMUS-PLAYBOOK | Gemini live path; Cursor/Claude keys optional | **Partial** | | Platform Dashboard (DOC21 ×8) | DOC21, Phase 4 | `/api/dashboard*`, `/app/dashboard` | **Done** (metrics scaffold; some sources empty) | | Local JWT auth (signup/login/me) | DOC-Atlas-Data-Model | `/api/auth/*`, Alembic `0005` | **Done** (Auth0 substitute) | | Auth0 + social + HorizonORBIT login | DOC02/03/Auth0 | `auth0.py` scaffold; login stub; `AUTH0_ENABLED=false` | **Missing** (deferred) | | Atlas remote MCP vault | DOC-Atlas-Index-B | stub when no `ATLAS_*` | **Missing** (deferred) | | Atlas durable PG mirror `0008` | DOC-Atlas-Data-Model | migration + `vault_store.py` writers | **Partial** (schema/writers exist; remote still stub; apply/wire verify needed) | | Project Ledger `/project/*` | DOC19 | `api/app/project/router.py` SQLite views | **Partial** (scaffold; not full DOC19 `ledger_entries` distillation) | | Capacitor ↔ live EC2 APK | DOCA2, Cap docs | wired per punchlist | **Done** eng / **Partial** device QA | | Field Mode / Extended Session / wake scaffolds | GS-MOB-01 | frontend pages + native scaffolds | **Partial** (device UAT open) | | Admin Panel (users/roles/packs/vault) | DOC03 §7 + inferred ops | `/app/admin` + `/api/admin/*` | **Done** (Auth0 deferred) | | GemStoneGuardRails (mandatory framing) | Mark agreement 2026-07-23 | `api/app/guardrails/*`, Alembic `0010`, Admin tab | **Done** | | Org-level Domain Pack availability | DOC03 RBAC | only practitioner self-install | **Missing** | | TierGuard + DOC03 tiers (`high_security`/`standard`/`open_mode`) | DOC03 §1/9, DOC06/08/16 | live tiers = `PUBLIC/INTERNAL/RESTRICTED/SOVEREIGN`; **no** `TierGuard` class; session code can overwrite `tier` | **Missing / drift** | | Open Mode consent UX (US-04) | DOC04 | `consent_events` table + dashboard; session create UX not verified | **Partial** | | External Research Agent + NER scrub | DOC03 §8, DOC16 | dashboard counter only; no Brave/Perplexity/spaCy path | **Missing** | | Cloud STT (Azure/Deepgram) abstraction | DOC03 §2 | providers: Whisper/Ollama/Gemini/Piper only | **Missing** (acceptable until Standard/Open Mode cloud path needed) | | Chunked-hold dual-flag (FR-GS-009) | DOC03/05/08/21 | `audio_pipeline_holds` + dashboard; pipeline write path thin | **Partial** | | WCAG AA automated scan | DOC02 Phase 4 | focus/skip-link polish only | **Partial** | | Playwright E2E ×2 | Phase 5 | 9/9×2 claimed | **Done** eng | | True 10k voice soak | DOC02 §6.10 | HTTP `/healthz` only; voice gate deferred | **Missing** | | Pen test / BABA sign-off / store submit / device UAT | DOC07 deferred | docs ready | **Missing** (human gates) | | Self-hosting Gate 2/3 + `GATE2_SUPERVISED_CYCLE_N` admin | DOC26, DOC06 | Gate 1-ish; admin config UI absent | **Partial / Missing** | | HUME AI | DOC03 §3 | none | **Out of scope** (correct) | --- ## Top 15 remaining engineering items (production-safe order) 1. **Keep Auth0 deferred but harden local auth for production** — `AUTH_ENABLED=true`, strong `JWT_SECRET`, bootstrap SuperAdmin, disable open signup in prod (`AUTH_ALLOW_SIGNUP=false`), tenancy on all sensitive routes. 2. **Ship Admin Panel MVP (see next section)** — user create/invite + role assign + org pack availability; no Auth0 dependency. 3. **Apply/verify Alembic `0008` on live DB** and confirm Recents/Search/Live dual-write into `atlas_*` tables with org+practitioner isolation. 4. **Wire real Atlas remote when secrets land** — graceful stub fallback must remain; health not-stub only after `ATLAS_MCP_*` present. 5. **Fix security-tier model drift** — align to DOC03 enums OR formally amend docs; implement single `TierGuard`; **stop mid-session tier mutation** (`session/manager.py` overwrite path). 6. **Open Mode consent flow (US-04)** — itemized disclosure + `consent_events` write before `open_mode` persist; dashboard 1:1 must be real. 7. **RBAC enforcement audit** — Didymus approve/submit, pack org-availability, SuperAdmin support-view audit; GemStone Agent never approves. 8. **Chunked-hold vectorization failure path (FR-GS-009)** — write holds + in-app + system log ≤5 min; feed dashboard. 9. **Complete Didymus worker credentialing** — Cursor/Claude vault keys; keep approval schema invariant untouched. 10. **Device UAT mobility closeout** — sunlight, call interrupt 10-trial, Android wake 8/10, iOS Siri 9/10 (`DOC-Device-UAT-Checklist.md`). 11. **WCAG AA automated scan** on Capacitor web; fix criticals only. 12. **Research Agent (if in MVP cut)** — scrub → Brave/Perplexity → audit token to ledger; else amend DOC scope. 13. **Auth0 cutover (when practitioner provisions secrets)** — wire 8 keys, social roles, replace stub login; keep local JWT as break-glass until stable. 14. **Phase 5 human gates** — pen-test engagement, BABA principal sign-off, store accounts/submit packages. 15. **True 10k voice soak** — only after feature-complete + capacity; do not claim from `/healthz` proxy. **Do later / defer intentionally:** marketplace packs, multi-language, Teams, SolidWorks/KiCad, Android default-assistant, HUME, SMS sandbox (ops; SES email OK). **HorizonIQ (later)** — see `DOC07-Punchlist.md`: org-wide shared practitioner rules; “make that a rule” merging into `GemStoneGuardRails`; passcode on personal practitioner rules. GemStone MVP personal-rules intent (when built): chat/voice keep-original vs use-new; signed-in Atlas segment only; no personal-rule passcode; GuardRails stay separate. --- ## Admin Panel requirements (extracted from docs) Docs never name a page called “Admin Panel,” but DOC03 §7 + DOC06 + DIDYMUS-PLAYBOOK + Auth0 docs require these **admin capabilities**: | Requirement | Source | Exists today? | |---|---|---| | **User create** | Local: `POST /api/auth/signup` (creates Org Admin + org + practitioner); bootstrap SuperAdmin via env | API yes / **Admin UI no**; signup may be too open for prod | | **User invite** | Implied by Auth0 org + role provisioning; training/ops | **No invite API/UI** | | **Role assign** (`SuperAdmin`, `Org Admin`, `Practitioner`, `GemStone Agent`) | DOC03 §7, Auth0 provisioning | Roles in JWT/store; **no set-role endpoint/UI** | | **Create/modify org-level Domain Pack availability** | DOC03 RBAC row | **Missing** (only self install/uninstall) | | **Install/uninstall pack for self** | DOC03 | **Done** (`/app/packs`) | | **Atlas vault provision** | DOC-Atlas-Index-B | Env/secrets only — **no vault-provision UI**; admin should at most show stub vs remote health + runbook link | | **Worker credential vault** | DIDYMUS-PLAYBOOK | **Partial** (`/app/onboarding/workers`) | | **Wake phrases / Gate2 N from admin/dev session** | DOC05 FR-GS-008, DOC06, DOC26 | Env-only; **no admin session UI** | | **Didymus approve (role-gated)** | DOC03 | Queue UI exists; Auth0 roles deferred → local role claims | | **SuperAdmin support view of other practitioners’ sessions (audited)** | DOC03 | **Not found** | | **Cannot change session tier mid-session** | DOC03 | **Must enforce** (current code can overwrite) | **Suggested Admin Panel checklist for implementer** - Routes: `/app/admin` (SuperAdmin/Org Admin only) - Users: list org users; create; invite (email token or temp password); set role; deactivate - Packs: org allowlist / availability toggle; still use existing install API for self - Atlas: read-only health (`stub|remote`), link to `DOC-Atlas-Index-B` (no secrets in UI) - Platform knobs: `WAKE_PHRASES`, `GATE2_SUPERVISED_CYCLE_N` (persist + audit) - Audit log for role changes + SuperAdmin cross-practitioner views --- ## What NOT to break (production invariants) 1. **Didymus:** no task → `RUNNING` without `approved_by` + `approved_at` (schema CHECK); no auto-execute / no source-colored queue. 2. **Recents:** notebook link is additive only — never hides sessions. 3. **Transcripts / audio:** never send session transcripts or raw audio to external APIs in sovereign/High Security paths; no HUME. 4. **Tenancy:** org RLS + practitioner segment isolation; no guest reading another practitioner’s Atlas. 5. **Atlas degrade:** remote down → stub fallback; don’t hard-fail voice. 6. **Inference modes:** sovereign (local) vs development (Gemini Live); mode switch via ops stays controlled. 7. **Capacitor path:** no React Native/Expo reintroduction; Field vs Extended audio paths stay separate. 8. **Whisper CUDA + disk:** don’t pull Playwright browsers onto tight root; keep voice stack healthy. 9. **`/healthz`:** Auth0/Atlas unconfigured must remain non-blocking until secrets exist. 10. **Deploy hygiene:** no secrets in git; don’t take down `gemstone.dataright.net`; atomic doc/ledger updates with schema/API/UI changes. 11. **Tier immutability + single TierGuard** (once aligned) — no secondary bypass checks. --- ## Implementation-agent checklist (copy/paste) ``` [ ] Confirm live AUTH_ENABLED / AUTH_ALLOW_SIGNUP / JWT_SECRET / bootstrap admin on OD [ ] Admin Panel: users create/invite/role/deactivate (RBAC-gated) [ ] Admin Panel: org Domain Pack availability (DOC03 SuperAdmin/Org Admin) [ ] Admin Panel: Atlas health read-only + wake/GATE2 knobs [ ] Verify Alembic head includes 0008 on live Postgres; smoke vault_store writers [ ] Fix tier model: DOC03 enums + TierGuard OR doc amendment; remove mid-session tier overwrite [ ] Open Mode consent UX → consent_events before persist [ ] Chunked-hold failure dual-flag path → audio_pipeline_holds [ ] Cursor/Claude worker vault keys smoke (Gemini already live) [ ] Device UAT scripts (DOC-Device-UAT-Checklist) — practitioner [ ] WCAG AA automated scan; fix criticals [ ] Auth0 wiring only after “Auth0 secrets are in DATARIGHT_DEV_SECRETS” [ ] Atlas remote only after ATLAS_* in secrets; keep stub fallback [ ] Pen-test / BABA / store — human gates (do not fake) [ ] Do NOT: weaken Didymus gate, Recents additive rule, cross-org isolation, HUME, RN/Expo ``` **Net:** Core practitioner product (Live voice, Recents/Search, packs, Didymus, dashboard, Capacitor eng, local auth, ledger `/project`) is largely **Done**. Production-safe remaining work clusters around **Admin/RBAC**, **tier correctness**, **Atlas durability→remote**, **device/UAT**, then **Phase 5 human closeout**. | 424bbb3074e878db33d54e47df4bf48c2befae3b |
| 18 | DOC-WORKER-KEY-ONBOARDING | DOC — Practitioner Vendor-Key Onboarding & Secure Storage | reference | DevResources/docs/DOC-Worker-Key-Onboarding.md | Didymus live worker execution (Cursor, Claude, Gemini, optional Ollama) runs against **your** vendor accounts. This playbook explains how a practitioner connects each worker by providing an API key/token, how GemStone stores those keys securely, and how to rotate or remove them. It complements `DIDYMUS-PLAYBOOK.md` (operator/host provisioning) and `DIDYMUS-ARCH-01.md` (architecture). The permanent human-approval invariant is unchanged: connecting a key never lets a task run … | # DOC — Practitioner Vendor-Key Onboarding & Secure Storage *Guided onboarding for connecting Didymus AI workers with your own vendor keys* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Implemented · **Date:** 2026-07-21 ## 1. Purpose Didymus live worker execution (Cursor, Claude, Gemini, optional Ollama) runs against **your** vendor accounts. This playbook explains how a practitioner connects each worker by providing an API key/token, how GemStone stores those keys securely, and how to rotate or remove them. It complements `DIDYMUS-PLAYBOOK.md` (operator/host provisioning) and `DIDYMUS-ARCH-01.md` (architecture). The permanent human-approval invariant is unchanged: connecting a key never lets a task run without explicit approval. ## 2. What you need per worker | Worker | Credential | Where to generate | | --- | --- | --- | | **Cursor** | API key (Cloud Agents) | Cursor Dashboard → Integrations / API Keys | | **Claude** | Anthropic API key (`sk-ant-...`) | Anthropic Console → API Keys | | **Gemini** | Google AI / Gemini API key | Google AI Studio → API keys | | **Ollama** (optional) | Endpoint URL (not a secret) | e.g. `http://gemstone-ollama:11434` | You only need keys for the workers you intend to use. A worker with no key runs in **simulated** mode (safe no-op result) so gates still pass. ## 3. Guided onboarding (in the app) 1. Sign in (see §6). You must be authenticated so the key is bound to your practitioner + org. 2. Open **Settings → Connect AI Workers** (`/app/onboarding/workers`). 3. Each worker shows a card with a **Connected / Not connected** status and, when connected, a masked hint (last 4 characters only). 4. Paste the key and select **Connect**. The value is sent once over TLS to `POST /api/didymus/workers/{worker_id}/credential` and is **never returned** by the API afterwards. 5. To replace a key, paste a new value and Connect again (overwrites). To remove a key, select **Disconnect** (`DELETE /api/didymus/workers/{worker_id}/credential`). Status endpoint (no secrets): `GET /api/didymus/workers/credentials` returns each worker's `configured` flag, `source` (`vault` / `env` / null), and `last4`. ## 4. How keys are stored securely - **Encrypted at rest.** Each key is sealed with **AES-256-GCM** before it touches the database. Only ciphertext is stored in the `worker_credentials` table. - **Master key from the secrets tier.** Encryption uses `WORKER_SECRETS_KEY` (base64 of 32 bytes) loaded from AWS Secrets Manager (`DATARIGHT_DEV_SECRETS`) via the standard bootstrap (`aws-secrets.md`). The key is **never** in the repo. - **Fail closed.** If `WORKER_SECRETS_KEY` is absent, the vault refuses to store or use any key (HTTP 503) rather than persisting plaintext. - **Identity-bound.** The GCM associated data binds each ciphertext to your `org_id`, `practitioner_id`, and `worker_id`; a blob cannot be reused under a different identity. - **Write-only + scoped.** The API never returns raw keys; rows are org-isolated (RLS) and unique per `(practitioner, worker)`. - **Never in transit to workers as data.** Keys are decrypted in-process only at execution time and are never logged or placed in Didymus task payloads or MCP arguments (`DIDYMUS-ARCH-01` §9). Vendor calls pass keys via request headers, never query strings. ### Resolution order at execution 1. Your **per-practitioner** key from the encrypted vault. 2. Host **environment fallback** (`DIDYMUS_WORKER_*` / `GEMINI_API_KEY`) for sovereign/backward-compat hosts. 3. Neither present → **simulated** execution. ## 5. Rotation & removal 1. Generate a new key at the vendor. 2. In the app, paste the new key and **Connect** (overwrites the stored value). 3. Revoke the old key at the vendor. 4. Confirm the worker still shows **Connected** with the new `last4`. 5. To offboard entirely, **Disconnect** the worker; the ciphertext row is deleted. Host-level keys (env) rotate per `DIDYMUS-PLAYBOOK.md` §7. ## 6. Login prerequisite Per-practitioner keys require an authenticated session. GemStone issues a local JWT (email/password) whose claims (`org_id`, `practitioner_id`, `role`) scope all interactions; this runs in parallel with the deferred Auth0 tenant and shares the same claim shape. Set `AUTH_ENABLED=true` to enforce token-derived tenancy; see `DOC-Auth0-Provisioning.md` for the eventual Auth0 swap-in. ## 7. Operator checklist | Check | Pass criteria | | --- | --- | | Master key present | `WORKER_SECRETS_KEY` in `DATARIGHT_DEV_SECRETS`; `GET /api/didymus/workers/credentials` → `vault_available: true` | | Store round-trip | Connect a test key → `configured: true`, correct `last4`; raw value never returned | | Fail-closed | With `WORKER_SECRETS_KEY` unset, `POST .../credential` returns 503 | | Live vs simulated | With a valid key, execute reports `mode: live`; without, `mode: simulated` | | No leakage | Keys absent from logs, task payloads, and error messages | ## 8. Out of scope - Storing keys in git, Capacitor bundled assets, or Didymus task JSON. - Autonomous execution without human approval. - Returning raw key values from any endpoint. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | ae828a7c4d45371f3a1968ef9c79d1bc6e5693e5 |
| 19 | DOC01 | DOC01 — Abstract | doc_suite | DevResources/docs/DOC01-Abstract.md | GemStone is the voice-first, mobile-first agentic interface layer of the NEXTECH Triptych (GemStone, Horizon Atlas, Horizon IQ), functioning as an executive assistant that works alongside, ahead of, and behind the individual practitioner. It is a full-capability AI partner, not a chatbot: it takes complex requests, executes autonomously within a human-approved envelope, produces deliverables, asks one clarifying question at a time, and reports outcomes. | # DOC01 — Abstract *Executive summary of the GemStone initiative* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. What GemStone Is GemStone is the voice-first, mobile-first agentic interface layer of the NEXTECH Triptych (GemStone, Horizon Atlas, Horizon IQ), functioning as an executive assistant that works alongside, ahead of, and behind the individual practitioner. It is a full-capability AI partner, not a chatbot: it takes complex requests, executes autonomously within a human-approved envelope, produces deliverables, asks one clarifying question at a time, and reports outcomes. GemStone is the primary interface through which a practitioner interacts with their own Atlas Vault and, where permitted, the aggregated Horizon IQ organizational corpus. ## 2. Core Design Commitments - Mobile-first: Capacitor-native for iOS and Android. - Always-on sessions: WebSocket-based persistent connections, no context loss. - Perfect cross-session recall via Atlas Vault (MCP Index B). - Fine-tuning flywheel: practitioner corrections and ratings improve GemStone continuously. - Domain Packs: curated, installable knowledge bundles. - Human-in-the-loop, always: no self-building feature executes without human approval, at any maturity level, permanently. ## 3. What Changed in This Revision This v2.0 rewrite supersedes the original 23-document suite in three respects: mobile stack migrated from React Native/Expo/PWA to Capacitor; GemStone orchestrates Cursor, Claude, and Gemini (with further workers planned) through Didymus, distinct from HorizonNEXUS; and GemStone is confirmed as the single universal presentation and approval surface for every build suggestion regardless of origin. A fourth addition, the GemStone Mobility Layer, was engineered specifically against documented Gemini Live production failures. ## 3A. Locked Build Decisions (Stage 0, July 2026) The following decisions are locked for the v2.0 build in `GemStone-Nexus-StackOne`. Do not re-litigate during implementation without a written SOW amendment. | Decision | Lock | | --- | --- | | Client | Capacitor (web + Android + iOS); React Native/Expo prohibited (DOCA2) | | Auth | Auth0 roles: SuperAdmin, Org Admin, Practitioner, GemStone Agent; social logins Google, Facebook, LinkedIn | | Inference | Sovereign-first: Ollama + Whisper + Piper on T4 GPU; Gemini development mode available | | Data tier | PostgreSQL (app) + Redis (session state, AOF) + SQLite (Project Ledger); GemStone is MCP client to remote Atlas Vault | | Primary env | T4 GoldenAMI EC2 Ohio `i-04333dae160ab0daa` | | Local secondary | NEW clean clone from `DataRightGoldenOVAFullBase` at `D:\Virtual Machines\DataRightGoldenOVAFullBase\` only; retire dirty GemStone VM `10.0.0.210` | | Repo | Build in `GemStone-Nexus-StackOne`; `DataRightAppBundle` is build machinery only | | Capacitor gate | First-class / atomic concurrency: a Capacitor build or test fail is a feature fail | | UI | Strict DataRightTailwind; login modeled on HorizonORBIT (HorizonCRM repo, clone at frontend stage) | | Naming | After first green GoldenAMI build, purge legacy product naming/imports/paths from the prior StackOne extraction | | Didymus ops | Practitioner provisioning of AI tool credentials per DIDYMUS-PLAYBOOK; architecture in DIDYMUS-ARCH-01 | ## 4. How This Document Set Is Organized This is DOC01 of 23 core docs, plus Amendments 1–2 and the executable references DOC26, DIDYMUS-ARCH-01, GS-MOB-01, and DIDYMUS-PLAYBOOK. DOC02 (SOW) and DOC03 (PRD) form the scope and technical backbone from which every other document derives. DOC06 (Master Prompt) is deliberately the last document in the build sequence, since it aggregates and references the full, completed set into Cursor-facing build instructions rather than standing alone. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | d127cefcad35f51cee7294d7f06baf12c38062ec |
| 20 | DOC02 | DOC02 — Statement of Work | sow | DevResources/docs/DOC02-SOW.md | | Term | Definition | | --- | --- | | Practitioner | The licensed end user of GemStone; owns one Atlas Vault; scoped by org_id and practitioner_id. | | Session | A single continuous voice or text interaction, WebSocket-backed, assigned exactly one Security Tier at creation. | | Security Tier | One of High Security, Standard, or Open Mode; immutable for the life of a session; enforced exclusively by TierGuard. | | Domain Pack | An installable bundle of system prompt extension… | # DOC02 — Statement of Work *Scope, definitions, phased delivery plan, gate criteria, RACI, risk register, and change control* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Definitions | Term | Definition | | --- | --- | | Practitioner | The licensed end user of GemStone; owns one Atlas Vault; scoped by org_id and practitioner_id. | | Session | A single continuous voice or text interaction, WebSocket-backed, assigned exactly one Security Tier at creation. | | Security Tier | One of High Security, Standard, or Open Mode; immutable for the life of a session; enforced exclusively by TierGuard. | | Domain Pack | An installable bundle of system prompt extension, terminology, methodology templates, and knowledge documents for a practice area. | | Didymus Task | A unit of build work registered in the Didymus queue; requires human approval (approved_by/approved_at) before execution regardless of source. | | Worker | An AI tool or desktop application implementing the DidymusWorker contract (Cursor, Claude, Gemini at MVP). | | Notebook | A named, practitioner-created grouping construct in Atlas Vault; session association is additive metadata, never a move. | | Gate | A phase-completion checkpoint with numeric, binary pass/fail criteria; a phase cannot be marked complete until every gate criterion is independently verified. | | MVP | The Phase 1 through Phase 5 scope defined in Section 6 of this SOW; excludes everything in Section 3 (Out of Scope). | ## 2. Scope Statement This SOW covers the design, build, test, and production deployment of: (a) the GemStone core platform, comprising the voice pipeline, WebSocket session layer, Atlas Vault integration, three-tier security model, and Domain Pack system; (b) the GemStone Mobility Layer, comprising the Capacitor mobile application, Field Mode, and Extended Sessions; (c) GemStone participation in Didymus as the universal task presentation and approval surface; and (d) the supporting operational infrastructure (Project Ledger, Platform Dashboard, testing framework) required to run (a) through (c) in production. Deliverables under this SOW are considered complete only when they pass the Gate criteria defined in Section 6, verified by an individual other than the engineer who implemented the deliverable. ## 3. Out of Scope, Phase 1 - Third-party Domain Pack marketplace (Phase 3). Trigger for inclusion: at least 3 first-party packs in production for 90+ days with practitioner-requested pack topics logged. - Claude Code, AntiGravity, and GemStone-as-worker as Didymus workers (planned, deferred). Trigger: Phase 1 three-worker MVP stable in production for 30+ days. - SolidWorks and KiCad desktop-application workers. Trigger: DidymusWorker contract proven stable across all three MVP AI workers; dedicated native-integration engineering allocated. - Multi-language voice support (FR-GS-005, Phase 2). Trigger: Whisper/Piper Phase 1 accuracy targets met in English; non-English accuracy benchmarked. - GemStone for Teams / shared session mode (FR-GS-007, Phase 3). - Horizon IQ direct reporting from GemStone (Phase 2, after Horizon IQ ships). - Custom voice model training per practitioner (Phase 3). - Android default-assistant (VoiceInteractionService) status (deferred stretch goal; foreground-service wake-word detection is committed Phase 1 scope instead). - Custom Android launcher for GemStone (FR-GS-015, noted feature request, not committed). ## 4. Assumptions and Constraints - Primary runtime is the T4 GoldenAMI EC2 in Ohio (`i-04333dae160ab0daa`). Local secondary/offline is a NEW clean VM clone from `DataRightGoldenOVAFullBase` at `D:\Virtual Machines\DataRightGoldenOVAFullBase\` only; the prior GemStone VM at `10.0.0.210` is retired and is not a build target. - Backend is sovereign-first: Ollama + Whisper + Piper on the T4; Gemini is available as development inference mode only (see contract-notes.md). Data tier is PostgreSQL + Redis (AOF) + SQLite Project Ledger; GemStone is an MCP client to a remote Atlas Vault. - Auth is Auth0 with roles SuperAdmin, Org Admin, Practitioner, and GemStone Agent (DOC03 Section 7), including Google, Facebook, and LinkedIn social connections. UI is strict DataRightTailwind; login layout is modeled on HorizonORBIT (HorizonCRM repo). - Capacitor is first-class: web + Android + iOS ship with atomic concurrency; a Capacitor build or test failure fails the feature and blocks the phase gate. - Build vehicle is this repo (`GemStone-Nexus-StackOne`); `DataRightAppBundle` provides the bootstrap pipeline. After the first green GoldenAMI build, legacy product naming from the prior StackOne extraction is purged. - AWS G-series migration is assumed to be underway in parallel but is not a dependency of this SOW; wake-word reliability targets in Section 6 Phase 2C are conditioned on CPU-only inference unless G-series is confirmed live before that phase gate. - HUME AI integration is explicitly out of scope for all phases under this SOW pending formal principal review (see DOC03 Section 3); no phase gate depends on HUME availability. - All AI/ML, messaging, and MQTT components must be Western-origin per standing technology stack constraints; no phase gate may be passed using a prohibited component even temporarily. - iOS App Store review timelines (historically 1 to 3 business days per submission, subject to change without notice by Apple) are outside NEXTECH control and are not counted against internal Phase 5 gate deadlines. - Didymus Hub (DIDYMUS-ARCH-01) is assumed to exist as shared infrastructure; this SOW covers GemStone's integration with Didymus, not Didymus's own build, which is governed by its own specification. Practitioner worker credential provisioning follows DIDYMUS-PLAYBOOK. - Bootstrap and Self-Hosting maturity gates are defined in DOC26-Bootstrap-Self-Hosting.md; Mobility Layer detail is in GS-MOB-01. ## 5. Roles and Responsibilities (RACI) | Deliverable Area | Engineering (Cursor-directed build) | NEXTECH Principal | QA / UAT | | --- | --- | --- | --- | | Voice Pipeline (1A) | Responsible | Accountable | Consulted | | Security Tier Infrastructure | Responsible | Accountable, must sign off before Phase 0 gate closes | Consulted | | Domain Pack Authoring (3B) | Responsible for schema/tooling | Accountable for content accuracy per pack | Consulted | | Didymus Integration (3C) | Responsible | Accountable | Responsible for approval-gate verification | | Mobility Layer Native Bridge (2C) | Responsible | Accountable | Responsible for device-level verification | | Penetration Testing (5) | Informed | Accountable, must review findings personally | Responsible | | HUME AI Review | Informed | Responsible and Accountable, cannot be delegated | Informed | ## 6. Phased Delivery Plan Each phase below lists entry criteria (what must be true to start), the task breakdown, and exit/gate criteria (what must be independently verified to close the phase). A phase cannot begin before its entry criteria are met, and cannot close before every exit criterion passes verification. ## 6.1 Phase 0, Infrastructure Entry criteria: NEXTECH AWS account and Docker registry access provisioned; gemstone-db PostgreSQL instance provisioned. ## 6.1 Phase 0, Task Breakdown 1. Docker Compose stack: gemstone-api, gemstone-mobile (build artifact only at this phase), gemstone-tts, atlas-ollama, gemstone-db, gemstone-redis, mcp-server, NGINX (WebSocket-upgrade configured), Supervisor. 2. gemstone-db schema migration scripts (Alembic) for: sessions, security_tiers, domain_packs, practitioners, orgs. 3. Auth0 tenant configuration with SuperAdmin, Org Admin, Practitioner, GemStone Agent roles (see DOC03 Section 7 for the full RBAC permission matrix). 4. GemStone Project Ledger initialization (ledger-api instance, empty schema). 5. /healthz endpoint returning 200 with dependency status (db, redis, mcp-server reachability). 6. make rollback target implemented and dry-run tested against a non-production environment. ## 6.1 Phase 0, Exit Criteria - All containers report healthy via docker compose ps for 10 consecutive minutes under idle load. - A WebSocket connection to gemstone-api successfully upgrades from HTTP and receives an echo response within 200ms. - /healthz returns 200 with all dependencies marked healthy. - make rollback executed against a deliberately broken deployment and restores the prior known-good state within 5 minutes. ## 6.2 Phase 1A, Voice Pipeline Entry criteria: Phase 0 closed. ## 6.2 Phase 1A, Task Breakdown 1. Containerize Whisper large-v3 in atlas-ollama; expose a gRPC or WebSocket streaming interface for partial and final transcript segments. 2. Containerize Piper TTS in gemstone-tts; expose a streaming interface emitting audio chunks as they are synthesized, not only on full-sentence completion. 3. Implement webrtcvad-based voice activity detection with a configurable pause threshold parameter (default 800ms, override to 2 to 3 seconds for "Deep Thinking Mode"). 4. Implement the STTProvider and TTSProvider abstract base classes (see DOC03 Section 2 for the exact interface) and the WhisperSTTProvider / PiperTTSProvider concrete implementations. 5. Build the round-trip test harness: audio in, transcript out, response text in, audio out, measuring end-to-end latency. ## 6.2 Phase 1A, Exit Criteria - STT word error rate at or below 8% (92%+ accuracy) against a 200-utterance benchmark set covering the six Domain Pack vocabularies. - First TTS audio chunk emitted within 800ms of receiving the first sentence of response text. - End-to-end round-trip latency (speech end to first response audio) at or under 2 seconds, measured at p95 across 100 test utterances. - VAD pause threshold is configurable at runtime without a service restart. ## 6.3 Phase 1B, Session Core Entry criteria: Phase 1A closed. ## 6.3 Phase 1B, Task Breakdown 1. WebSocket session manager: connection registry keyed by session_id, heartbeat/ping-pong at 30-second intervals, graceful reconnect handling. 2. Redis session state store: active session metadata (tier, practitioner_id, org_id, last_activity_at, conversation_buffer). 3. Reconnection logic: on reconnect within a 5-minute grace window, restore session state from Redis and replay the last N conversation turns to the client. 4. Cross-org isolation test: verify a session for org_id=A cannot read or write state belonging to org_id=B under any code path. ## 6.3 Phase 1B, Exit Criteria - A deliberately dropped WebSocket connection reconnects and restores full session state within 5 seconds. - Cross-org isolation test suite passes with zero leakage across 50 concurrent simulated sessions spanning at least 5 distinct org_ids. - Session state survives a gemstone-redis container restart when Redis persistence (AOF) is enabled. ## 6.4 Phase 2A, Atlas Integration Entry criteria: Phase 1B closed; Atlas Vault MCP Index B endpoint available. ## 6.4 Phase 2A, Task Breakdown 1. Implement the context-retrieval client querying MCP Index B for relevant Atlas Vault entries given the current conversation context. 2. Implement the fine-tuning signal writer: on practitioner correction or explicit rating, write a structured signal record to Atlas Vault. 3. Implement named/dated session recall: given a practitioner query like "the conversation from last Tuesday about the vendor contract," resolve to the correct session_id and restore it. ## 6.4 Phase 2A, Exit Criteria - MCP Index B context queries return in 800ms or less at p95 under a simulated load of 50 concurrent practitioners. - A fine-tuning signal is verifiably written to Atlas Vault within 2 seconds of a practitioner correction event. - A practitioner can name a prior session in natural language and GemStone restores the correct one in at least 9 of 10 test cases. ## 6.5 Phase 2B, Mobility Layer Field Mode Entry criteria: Phase 2A closed. See GS-MOB-01 for the full Mobility Layer specification. ## 6.5 Phase 2B, Exit Criteria - Voice capture activates within 1 second of the field-mode button tap, measured on a mid-tier Android device and an iPhone 13 or later. - Heads-Up UI passes a direct-sunlight legibility check (manual test, outdoors, midday Florida sun). - Session list (Recents) correctly displays and restores at least 20 prior sessions in a scroll-and-select test. ## 6.6 Phase 2C, Mobility Layer Extended Sessions Entry criteria: Phase 2B closed. Android is the priority platform; iOS follows once Android is proven. ## 6.6 Phase 2C, Exit Criteria - An Extended Session survives OS audio focus loss from an incoming phone call (one of the three interruption causes in DOC03 Section 5), reaches RESUMABLE after the session-ready chime, and resumes on one tap with full prior context restored, tested on both platforms. - Android wake-word activation succeeds from a fully killed app state in at least 8 of 10 trials under quiet-room conditions. - iOS Siri Shortcut trigger successfully launches GemStone and begins listening in at least 9 of 10 trials. - The periodic mic-open audio cue is audible and occurs at the configured interval in a 10-minute continuous Extended Session test. ## 6.7 Phase 3A/3B, Domain Pack System and First-Party Packs Entry criteria: Phase 2A closed (Domain Packs do not depend on Mobility Layer completion). ## 6.7 Phase 3A/3B, Exit Criteria - A Domain Pack installs and activates in under 10 seconds, measured from practitioner tap to system prompt extension active in the next turn. - All six Phase 1 packs (PE Operating Partner, Healthcare Executive, FinTech/Capital Markets, Management Consulting, Construction/Real Estate, Manufacturing Operations) pass their individual Domain Pack UAT scripts (see DOC20 Section 2). - Uninstalling a pack removes its terminology/system-prompt extension from the next session without affecting prior session history or Atlas Vault content. ## 6.8 Phase 3C, Didymus Integration Entry criteria: Didymus Hub deployed and reachable (per DIDYMUS-ARCH-01); Phase 1B closed. ## 6.8 Phase 3C, Exit Criteria - A task submitted by any of the three MVP workers (Cursor, Claude, Gemini) reaches the RUNNING state only after approved_by and approved_at are populated by a human; verified by direct database inspection, not by UI observation alone. - A GemStone-facing approval queue displays tasks from all three workers with no visual or structural distinction based on source, verified by a blind review test. - Attempting to force a task to RUNNING via direct API call without approval fields populated is rejected by the schema constraint, not merely by application logic. ## 6.9 Phase 4, Polish and Dashboard Entry criteria: Phases 2B/2C/3A/3B/3C all closed. ## 6.9 Phase 4, Exit Criteria - Mobile app passes a WCAG AA automated accessibility scan with zero critical violations. - Platform Dashboard (DOC21) loads its default view in under 2 seconds on a standard broadband connection. - All UAT Structured scripts (DOC20) pass on a clean staging environment. ## 6.10 Phase 5, Production Hardening Entry criteria: Phase 4 closed. ## 6.10 Phase 5, Exit Criteria - Full Playwright E2E suite passes with zero failures on two consecutive runs. - Third-party penetration test complete; all Critical and High findings remediated and re-tested; Medium findings have a documented remediation plan with an owner and date. - Load test sustains 10,000 concurrent sessions with p95 voice round-trip latency remaining under 3 seconds. - Final Syft SBOM generated and BABA compliance audit signed off by a NEXTECH principal. - App Store and Play Store submission packages assembled and submitted. ## 7. Risk Register | Risk | Likelihood | Impact | Mitigation | | --- | --- | --- | --- | | Wake-word false-trigger rate too high for field use | Medium | High, undermines Extended Session usability | Field-condition benchmark before Phase 2C gate; tunable sensitivity parameter; re-test after AWS G-series migration | | iOS background-audio entitlement rejected by App Store review | Low | High, blocks Phase 2C on iOS | Follow Apple background-audio guidelines precisely; avoid any pattern resembling always-on ambient listening; Siri Shortcuts path avoids the highest-risk pattern entirely | | Didymus Hub instability blocks Phase 3C | Medium | Medium, delays but does not block other phases | Phase 3C has no downstream dependents among 2B/2C/3A/3B; can proceed in parallel | | HUME AI review stalls indefinitely | Low | Low, feature already excluded from all phase gates | No mitigation needed; explicitly out of scope per Section 4 | | Whisper/Piper accuracy below target on Domain Pack-specific terminology | Medium | Medium | Benchmark set explicitly includes Domain Pack vocabulary; fine-tuning flywheel improves accuracy post-launch | ## 8. Change Control Any change to the phase boundaries, gate criteria, or in/out-of-scope lists in this SOW requires a written amendment (following the Amendment 1/Amendment 2 pattern already established for Didymus and Mobility Layer integration) reviewed and approved by a NEXTECH principal before implementation begins. Engineering may not silently expand or contract scope based on implementation convenience. ## 9. Delivery Format All Cursor-facing deliverables ship in both .docx and .md formats together, per standing NEXTECH documentation standards. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | f6f64b1953f8c2adb0d47ac1e2466196b9184fe3 |
| 21 | DOC03 | DOC03 — Product Requirements Document | prd | DevResources/docs/DOC03-PRD.md | ``` class SecurityTier(str, Enum): HIGH_SECURITY = "high_security" # numeric value 0 (most restrictive) STANDARD = "standard" # numeric value 1 OPEN_MODE = "open_mode" # numeric value 2 (least restrictive) | # DOC03 — Product Requirements Document *Schemas, API contracts, state machines, and full functional/non-functional requirements* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Core Data Model ``` class SecurityTier(str, Enum): HIGH_SECURITY = "high_security" # numeric value 0 (most restrictive) STANDARD = "standard" # numeric value 1 OPEN_MODE = "open_mode" # numeric value 2 (least restrictive) class Session(SQLModel, table=True): __tablename__ = "sessions" id: UUID = Field(default_factory=uuid4, primary_key=True) org_id: UUID = Field(index=True) practitioner_id: UUID = Field(index=True) tier: SecurityTier = Field(index=True) # immutable after insert; enforced by DB trigger + TierGuard mode: str # "field" | "extended" notebook_ids: list[UUID] = Field(default_factory=list, sa_column=Column(JSON)) started_at: datetime last_activity_at: datetime ended_at: Optional[datetime] = None interruption_snapshot: Optional[dict] = Field(default=None, sa_column=Column(JSON)) class DomainPack(SQLModel, table=True): __tablename__ = "domain_packs" id: UUID = Field(default_factory=uuid4, primary_key=True) name: str = Field(unique=True) version: str system_prompt_extension: str terminology: dict = Field(sa_column=Column(JSON)) methodology_templates: list[dict] = Field(sa_column=Column(JSON)) knowledge_doc_refs: list[str] = Field(sa_column=Column(JSON)) # MCP document IDs class PackInstallation(SQLModel, table=True): __tablename__ = "pack_installations" practitioner_id: UUID = Field(foreign_key="practitioners.id", primary_key=True) pack_id: UUID = Field(foreign_key="domain_packs.id", primary_key=True) installed_at: datetime active: bool = Field(default=True) class Notebook(SQLModel, table=True): __tablename__ = "notebooks" id: UUID = Field(default_factory=uuid4, primary_key=True) practitioner_id: UUID = Field(index=True) name: str created_at: datetime # Recents is never a table; it is always the live query below, so Notebook # membership can NEVER cause a session to disappear from it: # SELECT * FROM sessions WHERE practitioner_id = :pid ORDER BY last_activity_at DESC ``` ## 2. Voice Provider Abstraction ``` class STTProvider(ABC): @abstractmethod async def stream_transcribe(self, audio_chunk: bytes) -> TranscriptSegment: ... @abstractmethod def supports_streaming(self) -> bool: ... class TTSProvider(ABC): @abstractmethod async def stream_synthesize(self, text: str) -> AsyncIterator[bytes]: ... class WhisperSTTProvider(STTProvider): # self-hosted default, all tiers ... class PiperTTSProvider(TTSProvider): # self-hosted default, all tiers ... class AzureSpeechSTTProvider(STTProvider): # cloud, Standard/Open Mode only ... class DeepgramSTTProvider(STTProvider): # cloud, Standard/Open Mode only ... class VoiceProviderFactory: def get_stt(self, tier: SecurityTier, preference: Optional[str]) -> STTProvider: if tier == SecurityTier.HIGH_SECURITY: return WhisperSTTProvider() # cloud providers never permitted at High Security return PROVIDER_REGISTRY.get(preference, WhisperSTTProvider()) class StreamingAdapter: """Wraps a non-streaming provider to emit synthetic partial chunks.""" ... ``` ## 3. HUME AI: Formal Review Status | Consideration | Assessment | | --- | --- | | Status | Pending Review, not integrated, no code path exists for it | | Risk | Behavioral profiling risk; HUME infers emotional/behavioral state from voice beyond transcription | | Minimum tier if approved | Open Mode only, never Standard or High Security, regardless of any Data Processing Agreement | | Resolution path | A NEXTECH principal must review and approve HUME data handling terms in writing, with all Critical/High risk items resolved, before any integration work begins | ## 4. REST and WebSocket API Contract | Method | Path | Request | Response | | --- | --- | --- | --- | | WS | /ws/session/{session_id} | Binary audio frames + JSON control messages | Streamed transcript segments, streamed TTS audio, control acks | | POST | /api/sessions | {practitioner_id, tier, mode} | {session_id, tier, started_at} | | GET | /api/sessions/{id} | - | Full session object including interruption_snapshot if paused | | POST | /api/sessions/{id}/resume | {} | {restored: bool, last_turn: object} | | POST | /api/notebooks | {practitioner_id, name} | {notebook_id} | | POST | /api/notebooks/{id}/link | {session_id} | {linked: true} — additive only, never removes from Recents | | POST | /api/search/inferenced | {practitioner_id, query} | {results: [{session_id, snippet, relevance}]} | | POST | /api/domain-packs/{id}/install | {practitioner_id} | {installed_at, active: true} | | DELETE | /api/domain-packs/{id}/install | {practitioner_id} | {active: false} | | POST | /api/didymus/tasks | {requested_by, target_worker, task_type, payload} | {task_id, status: "PENDING"} | | POST | /api/didymus/tasks/{id}/approve | {approved_by} | {status: "APPROVED", approved_at} | ## 5. Session Lifecycle State Machine ``` CREATED --(tier assigned, immutable)--> ACTIVE ACTIVE --(OS audio focus loss incl. incoming call | network drop | app background)--> PAUSED PAUSED --(interruption_snapshot written: token state, audio buffer, context)--> PAUSED PAUSED --(session-ready chime fires)--> RESUMABLE RESUMABLE --(one tap OR app reopen)--> ACTIVE [restores from interruption_snapshot via Atlas Vault] ACTIVE --(explicit end OR extended timeout with confirmation)--> ENDED # Three interruption causes share this path: OS audio focus loss (incl. incoming call), # network drop, and app backgrounding. Extended timeout is ENDED, not RESUMABLE. ``` ## 6. Didymus Task State Machine (GemStone-Facing View) ``` PENDING --(human sets approved_by + approved_at)--> APPROVED # PENDING -> RUNNING is not a legal transition; the schema constraint blocks it. APPROVED --(worker leases task, atomic UPDATE)--> RUNNING RUNNING --(worker reports result)--> COMPLETED | FAILED COMPLETED --(GemStone or worker verifies output)--> VERIFIED # This state machine is IDENTICAL regardless of whether requested_by is # a human user_id or the string "GemStone". Source never changes the path. ``` ## 7. RBAC Permission Matrix Auth0 is the identity provider. Roles SuperAdmin, Org Admin, Practitioner, and GemStone Agent are provisioned in the Auth0 tenant (DOC02 Phase 0). Social connections Google, Facebook, and LinkedIn are enabled for practitioner-facing login; role claims still come from Auth0 authorization, not from the social IdP alone. | Action | SuperAdmin | Org Admin | Practitioner | GemStone Agent | | --- | --- | --- | --- | --- | | Create/modify org-level Domain Pack availability | Yes | Yes (own org) | No | No | | Install/uninstall a Domain Pack for self | Yes | Yes | Yes | No | | View another practitioner's session content | Yes (support only, audited) | No | No | No | | Approve a Didymus task | Yes | Yes (own org scope) | Yes (own tasks) | No, never | | Submit a Didymus task | Yes | Yes | Yes | Yes (as a proposer only) | | Change a session's tier mid-session | No, nobody can, tier is immutable | No | No | No | ## 8. External Research Agent The Research Agent performs Query Isolation and Entity Scrubbing (spaCy en_core_web_lg, self-hosted NER) on any query before it leaves the platform. Approved web search providers are Brave Search and Perplexity; approved synthesis providers are Claude API and Gemini API. Every external query generates an audit token logged to the Project Ledger. An Open Mode scrubbing toggle exists for practitioners who explicitly accept reduced scrubbing in that tier. ## 9. Three-Tier Security Model Tier cascade evaluates as the minimum numeric value across the chain (Horizon IQ, Atlas, GemStone, Practitioner), so no layer can grant more access than the most restrictive layer above it. TierGuard is the single enforcement point; a secondary tier check anywhere else in the codebase is a defect. Session transcripts are never transmitted externally, in any tier, without exception. ``` def resolve_effective_tier(iq_tier: int, atlas_tier: int, gemstone_tier: int, practitioner_tier: int) -> int: return min(iq_tier, atlas_tier, gemstone_tier, practitioner_tier) class TierGuard: """The ONLY place tier-gated behavior is checked. Do not duplicate this logic.""" def enforce(self, session: Session, action: str) -> None: if action in RESTRICTED_ACTIONS[session.tier]: raise TierViolation(session.id, action, session.tier) ``` ## 10. Domain Pack System | Pack Component | Description | | --- | --- | | System Prompt Extension | Domain-specific behavioral instructions extending the base personality | | Terminology Dictionary | Domain vocabulary GemStone uses fluently in conversation and documents | | Methodology Templates | Pre-built methodology node templates seeded into Atlas Vault | | Knowledge Documents | Curated reference material indexed in MCP for retrieval | | Tool Integrations | Optional domain-specific tool connections | ## 11. First-Party Domain Packs, Phase 1 | Pack Name | Target Practitioner | Key Specializations | | --- | --- | --- | | PE Operating Partner | PE operating partners, portfolio company executives | Deal structuring, EBITDA optimization, board reporting, 100-day plans | | Healthcare Executive | CMOs, CNOs, hospital administrators | Clinical operations, HIPAA/CMS compliance, value-based care | | FinTech / Capital Markets | Fintech founders, investment managers | SEC/FINRA landscape, product structuring, risk frameworks | | Management Consulting | Independent and firm-affiliated consultants | Engagement frameworks, change management, org design | | Construction / Real Estate | Developers, owner-builders, construction managers | Project management, permitting, cost control | | Manufacturing Operations | COOs, plant managers | Lean/Six Sigma, supply chain, production optimization | ## 12. Didymus Orchestration GemStone is the single, universal presentation, approval, and orchestration surface for all build task suggestions, regardless of originating tool. Didymus captures full context/rationale for every suggestion and feeds a distilled version into Atlas Vault. No task, from any source, executes without human approval, at any platform maturity level. ## 13. GemStone Mobility Layer Field Mode: foreground tap-to-talk via standard Capacitor WebView audio. Extended Sessions: native background audio (iOS AVAudioSession + UIBackgroundModes: audio; Android foreground service with FOREGROUND_SERVICE_TYPE_MICROPHONE), periodic mic-open cue, longer timeout with confirmation. Wake-word: configurable multi-phrase list; Android foreground-service on-device model; iOS Siri Shortcuts. Full detail in GS-MOB-01. ## 14. Non-Functional Requirements - AES-256 at rest; TLS 1.3 in transit; no audio data to external APIs. - Raw audio discarded immediately after transcription/indexing/vectorization into Atlas Vault. - Failed vectorization holds audio in chunks, flags in-app and system log. - OWASP Top 10 compliance; penetration testing at Phase 5. - Session state survives Redis restart with AOF persistence enabled. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | b1d8320af00bbd5acfe3807ae8422f30e94a40d9 |
| 22 | DOC04 | DOC04 — Granular User Stories | doc_suite | DevResources/docs/DOC04-Granular-User-Stories.md | US-01: As a practitioner, I can speak a request to GemStone and receive a response without losing context if I step away mid-session. | # DOC04 — Granular User Stories *Practitioner-level stories with Given/When/Then acceptance criteria* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Epic: Core Voice Interaction US-01: As a practitioner, I can speak a request to GemStone and receive a response without losing context if I step away mid-session. Given an ACTIVE session, When the practitioner's app is backgrounded for up to the configured Extended Session timeout, Then the session transitions to PAUSED with an interruption_snapshot written; When the session-ready chime fires, Then state becomes RESUMABLE; When the practitioner returns to the app within the timeout window and taps resume (or reopens the app), Then state restores to ACTIVE with full prior context from the interruption_snapshot. US-02: As a practitioner, I can resume a session exactly where I left off regardless of how it was interrupted (OS audio focus loss including incoming call, network drop, or app backgrounding). Given a session that reached RESUMABLE via any of the three interruption causes (OS audio focus loss including incoming call; network drop; app backgrounding), When the practitioner taps resume or reopens the app, Then the restored context is identical across all three causes, verified by comparing the last_turn payload byte-for-byte. (Extended Session timeout with confirmation is a separate path to ENDED, not an interruption-to-PAUSED/RESUMABLE cause.) ## 2. Epic: Security and Tiers Canonical tier display names are High Security, Standard, and Open Mode (DOC02/DOC10). Wire/enum values are `high_security` / `HIGH_SECURITY`, `standard` / `STANDARD`, and `open_mode` / `OPEN_MODE` (DOC03 Section 1). Acceptance criteria below use enum constants; UI copy uses display names. US-03: As a practitioner, I select a security tier at session start and it cannot change mid-session. Given a session created with tier=STANDARD, When any code path attempts to modify the tier field, Then the write is rejected at the database constraint level, not merely by application logic. US-04: As a practitioner entering Open Mode, I see an itemized disclosure before I confirm. Given a practitioner selecting Open Mode, When the session-creation flow begins, Then an itemized list of specifically reduced protections is displayed and a distinct consent action is logged before tier=OPEN_MODE is persisted. ## 3. Epic: Domain Packs US-05: As a PE operating partner, installing the PE Operating Partner Pack makes GemStone fluent in EBITDA optimization immediately. Given the pack is installed, When the practitioner starts the very next session, Then the system prompt extension and terminology dictionary for that pack are active without requiring an app restart. US-06: As a practitioner, installing or removing packs never touches my core session history. Given a practitioner uninstalls a Domain Pack, When their prior sessions are viewed in Recents, Then all prior session content, transcripts, and Atlas Vault entries remain unchanged. ## 4. Epic: Notebooks, Recents, and Search US-07: As a practitioner, assigning a session to a Notebook never removes it from Recents. Given a session linked to a Notebook via POST /api/notebooks/{id}/link, When the practitioner views Recents, Then the session appears in chronological position with a Notebook badge, not hidden or moved. US-08: As a practitioner, I can find something from a past conversation using natural language. Given a practitioner has a session discussing a specific vendor contract, When they later ask GemStone to find it using approximate natural language, Then inferenced search returns that session in the top 3 results in at least 9 of 10 test cases. ## 5. Epic: Orchestration and Approval US-09: As a practitioner, I see every build suggestion in one place regardless of which AI tool proposed it. Given tasks submitted by GemStone, Cursor, Claude, and Gemini, When the practitioner opens the approval queue, Then all four sources appear in a single unified list with no separate view per source, and source is not visually distinguishable without inspecting metadata. US-10: As a practitioner, GemStone proposing its own tasks changes nothing about my approval responsibility. Given GemStone is Phase-2 self-hosting capable, When GemStone submits a task with requested_by="GemStone", Then the task enters PENDING and requires the identical approval action as any human-submitted or other-tool-submitted task. ## 6. Epic: Field and Mobility US-11: As a practitioner driving or cycling, I can tap a single large button without looking at the screen. Given Field Mode is active, When the practitioner taps anywhere within the locked center-screen target region, Then recording begins within 1 second and an ascending dual-tone chime confirms it audibly. US-12: As a practitioner working hands-free, I can say a configured wake phrase to bring GemStone to life. Given the app is fully killed on Android, When the practitioner says a configured wake phrase within range of the device microphone, Then the foreground-service wake-word model detects it and launches GemStone in at least 8 of 10 trials. US-13: As a practitioner in an Extended Session, I hear a periodic chime confirming the mic is open, and get a confirmation step before timeout. Given an Extended Session has been open for the configured cue interval, When that interval elapses, Then an unobtrusive audio cue plays, and When the session approaches its extended timeout, Then an explicit confirmation step is required before the mic closes. US-14: As a practitioner, a failed transcription notifies me rather than silently disappearing. Given a vectorization pipeline failure on a completed session, When the failure occurs, Then an in-app notification appears and a corresponding system-log entry is written, both within 5 minutes of the failure. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | b8410a9f38f0f098995eecac26aca95abcdc672d |
| 23 | DOC05 | DOC05 — Feature Requests Register | doc_suite | DevResources/docs/DOC05-Feature-Requests-Register.md | | ID | Feature | Acceptance Criteria Summary | Dependency | | --- | --- | --- | --- | | FR-GS-001 | Capacitor Field Mode voice interaction | Voice activates <1s on tap; standard WebView audio only | Phase 1A voice pipeline | | FR-GS-002 | Extended Session native background audio | Survives OS audio-focus interruption; Android priority | Phase 2B Field Mode complete | | FR-GS-004 | Didymus integration, universal approval gate | No task reaches RUNNING without approved_by/appr… | # DOC05 — Feature Requests Register *Committed, deferred, and proposed features with acceptance criteria and dependencies* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Committed, Phase 1 | ID | Feature | Acceptance Criteria Summary | Dependency | | --- | --- | --- | --- | | FR-GS-001 | Capacitor Field Mode voice interaction | Voice activates <1s on tap; standard WebView audio only | Phase 1A voice pipeline | | FR-GS-002 | Extended Session native background audio | Survives OS audio-focus interruption; Android priority | Phase 2B Field Mode complete | | FR-GS-004 | Didymus integration, universal approval gate | No task reaches RUNNING without approved_by/approved_at | Didymus Hub deployed | | FR-GS-006 | Notebooks/Recents/inferenced search | Notebook link never removes session from Recents | Atlas Vault MCP Index B | | FR-GS-008 | Configurable wake-phrase list, configurable Gate 2 count | Multiple phrases active simultaneously; n set via admin session | None, config-only | | FR-GS-009 | Chunked-hold failure-flagging | In-app + system log, both, within 5 minutes of failure | Vectorization pipeline live | ## 2. Deferred | ID | Feature | Deferred To | Reactivation Trigger | | --- | --- | --- | --- | | FR-GS-003 | Always-listening beyond Mobility Layer scope | Phase 2 | Phase 1 wake-word stability proven | | FR-GS-005 | Multi-language voice support | Phase 2 | English accuracy targets met | | FR-GS-007 | GemStone for Teams (shared session) | Phase 3 | Individual-practitioner MVP stable 90+ days | | FR-GS-010 | Claude Code, AntiGravity, GemStone-as-worker | Phase 2 | Three-worker MVP stable 30+ days | | FR-GS-011 | SolidWorks / KiCad desktop workers | Phase 2/3 | DidymusWorker contract proven across AI workers | | FR-GS-012 | Android default-assistant status | Stretch goal | Explicit go-to-market decision to pursue Option 1 | | FR-GS-013 | Third-party Domain Pack marketplace | Phase 3 | See DOC02 Section 3 trigger | | FR-GS-014 | Custom voice model training per practitioner | Phase 3 | Fine-tuning flywheel data volume sufficient | ## 3. Feature Request, Not Yet Scoped or Committed - FR-GS-015: A custom Android launcher for GemStone, noted future consideration, not committed scope. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 4bd49e9071920a9e02d2e29684830ff88c2b2009 |
| 24 | DOC06 | DOC06 — Master Prompt | doc_suite | DevResources/docs/DOC06-Master-Prompt.md | This is the last document in the GemStone set, built only after DOC01 through DOC05 and DOC07 through DOC21 (plus Amendments 1 and 2) are complete, because it aggregates and references them rather than standing alone. Cursor should treat this document as the entry point for any GemStone build task, and should follow its section references back to the source document for full detail rather than relying on this summary alone for anything schema-, endpoint-, or test-critical. | # DOC06 — Master Prompt *Aggregated Cursor build instructions, referencing the complete GemStone document set* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 0. How to Use This Document This is the last document in the GemStone set, built only after DOC01 through DOC05 and DOC07 through DOC21 (plus Amendments 1 and 2) are complete, because it aggregates and references them rather than standing alone. Cursor should treat this document as the entry point for any GemStone build task, and should follow its section references back to the source document for full detail rather than relying on this summary alone for anything schema-, endpoint-, or test-critical. ## 1. Non-Negotiable Technology Constraints - AI/ML stack is exclusively Western-origin (PyTorch, Hugging Face, vLLM, Ollama, Whisper, MLflow, OR-Tools). PaddlePaddle, MindSpore, MegEngine, OneFlow, and all Chinese-origin frameworks are prohibited, no exceptions. See DOC17 Section 1. - Messaging is AWS-native only (SES, AWS End User Messaging, SNS for push/pub-sub only, FCM). Twilio and other third-party messaging vendors are prohibited. - MQTT broker is always Eclipse Mosquitto. EMQX is prohibited. - Mobile frontend is Capacitor. Do not scaffold React Native or Expo for GemStone mobile work; see Amendment 2 for what this superseded. - Capacitor is first-class and subject to atomic concurrency: every frontend change ships web + Android + iOS in the same change set; a Capacitor build or test failure is a feature failure and blocks the phase gate. - UI is strict DataRightTailwind (`DataRightAppBundle/DataRightTailwind/` and `DATARIGHT_TAILWIND.md`). Login is modeled on HorizonORBIT (HorizonCRM repo; clone at frontend stage). Auth0 Universal Login with Google, Facebook, and LinkedIn social connections maps to SuperAdmin, Org Admin, Practitioner, and GemStone Agent. - Inference is sovereign-first: Ollama + Whisper + Piper on the T4; Gemini is development mode only. Switch mode via `PUT /api/v1/gemstone/ops/inference-mode` (see contract-notes.md). - Data tier: PostgreSQL (app schema) + Redis session state with AOF + SQLite Project Ledger. GemStone is an MCP client to the remote Atlas Vault (MCP Index B); do not embed a local Atlas Vault. - Primary build/run target is T4 GoldenAMI EC2 `i-04333dae160ab0daa`. Local secondary is a clean clone of `DataRightGoldenOVAFullBase` only. Do not target the retired GemStone VM at `10.0.0.210`. - Build in this repo (`GemStone-Nexus-StackOne`). `DataRightAppBundle` is bootstrap machinery. After the first green GoldenAMI build, purge legacy product naming from the prior StackOne extraction (CI grep gate required). - Every Didymus task, regardless of source, requires human approval before execution. Never scaffold an autonomous-execution code path for any feature category, at any maturity level. See DOC03 Section 6 for the schema-level enforcement pattern. Practitioner credential provisioning: DIDYMUS-PLAYBOOK. ## 2. Repository and Container Structure ``` gemstone/ gemstone-api/ # FastAPI backend: sessions, tiers, domain packs, Didymus client gemstone-mobile/ # Capacitor app: Field Mode, Extended Sessions, Notebooks/Recents UI gemstone-tts/ # Piper TTS streaming service atlas-ollama/ # Whisper large-v3 STT via Ollama gemstone-db/ # PostgreSQL schema and Alembic migrations gemstone-redis/ # Redis session state, AOF persistence enabled mcp-server/ # MCP Index B client for Atlas Vault integration ledger-api/ # GemStone Project Ledger (DOC19) tests/ unit/ # Pytest (backend), Vitest (frontend) integration/ # Pytest + test DB e2e/ # Playwright uat/ # Structured scripts per DOC20 Makefile docker-compose.yml .env.example ``` ## 3. Required Environment Variables | Variable | Purpose | Source of Truth | | --- | --- | --- | | GEMSTONE_DB_URL | PostgreSQL connection string | DOC03 Section 1 | | GEMSTONE_REDIS_URL | Redis connection string, AOF enabled | DOC02 Section 6.3 | | ATLAS_MCP_ENDPOINT | MCP Index B endpoint for Atlas Vault | DOC02 Section 6.4 | | DIDYMUS_HUB_URL | Didymus Hub endpoint (external dependency) | DIDYMUS-ARCH-01 | | VOICE_TIER_CLOUD_ALLOWED | Boolean gate on cloud STT/TTS providers by tier | DOC03 Section 2 | | RESEARCH_AGENT_PROVIDERS | Comma list of approved providers (brave, perplexity, claude, gemini) | DOC03 Section 8 | | WAKE_PHRASES | JSON array, configurable, multiple simultaneous phrases | FR-GS-008, DOC05 | | GATE2_SUPERVISED_CYCLE_N | Configurable count, set from admin/dev session, not hardcoded | FR-GS-008, DOC26 Section 5 | ## 4. Makefile Targets Cursor Must Implement ``` make up # docker compose up, all services make down # docker compose down make migrate # Alembic upgrade head against gemstone-db make test # runs unit, integration, e2e, uat in sequence; fails fast on any tier make lint # ruff (Python), eslint (TypeScript) make rollback # restore prior known-good deployment; see DOC02 Section 6.1 exit criteria make sbom # regenerate Syft SBOM, cross-check against DOC17 ``` ## 5. Security Tier Enforcement Rules - TierGuard (DOC03 Section 9) is the single enforcement point for tier-gated behavior. Do not implement tier checks in more than one place; a second enforcement point is itself a defect per DOC16 Section 2. - Tier is immutable after session creation, enforced at the database constraint level, not only in application code. See DOC03 Section 1 for the schema and DOC14 Section 3 for the required test. - Session transcripts must never be transmitted externally in any tier. This is an absolute; do not build an exception path even for debugging. ## 6. Orchestration Rules When implementing task-dispatch or approval-queue logic, GemStone is always the single presentation surface. Do not build a separate approval UI for tasks originating from Cursor, Claude, or Gemini; route all suggestions through the same GemStone-facing queue regardless of source, per DOC03 Section 12 and the Didymus task state machine in DOC03 Section 6. Every task record requires approved_by and approved_at fields that structurally block the transition into RUNNING; see DOC14 Section 3 for the required test and DOC08 Section 3 for the incident runbook if this is ever violated. ## 7. Mobility Layer Build Rules Field Mode audio uses standard Capacitor WebView APIs (getUserMedia/MediaRecorder), identical to the current web app implementation; no native plugin is required for this mode. Extended Session audio requires native background-audio entitlements: iOS AVAudioSession with UIBackgroundModes: audio declared in native configuration, and Android a foreground service declared with FOREGROUND_SERVICE_TYPE_MICROPHONE carrying a persistent notification. Do not conflate these two audio paths into a single implementation; see DOC02 Sections 6.5-6.6 for phase-separated gate criteria and GS-MOB-01 Section 10 for the full architecture. Raw audio is captured in full the moment the mic opens, no ambiguous pre-buffer state, and is discarded immediately once transcribed, indexed, and vectorized into Atlas Vault. On failure, hold audio in chunks and flag both in-app and at the system log level; see DOC08 Section 4 for the runbook and DOC21 for the required dashboard view. ## 8. Testing Requirements Before Any Merge Every merge must pass the coverage thresholds in DOC14 Section 4 and include, at minimum, the representative test patterns in DOC14 Section 3 for any touched subsystem. A feature is not complete when it works; it is complete when its corresponding DOC02 phase gate criteria and DOC20 UAT script (if applicable) both pass. ## 9. Document Cross-Reference Index | Document | What Cursor Should Consult It For | | --- | --- | | DOC01 Abstract | High-level orientation to what GemStone is and why | | DOC02 SOW | Phase boundaries, entry/exit gate criteria, RACI, risk register | | DOC03 PRD | Schemas, API contracts, state machines, RBAC matrix, tier logic | | DOC04 User Stories | Given/When/Then acceptance criteria per capability | | DOC05 Feature Requests | What is committed vs. deferred vs. unscoped, with dependencies | | DOC07 Punchlist | Per-phase checklist with owner and verification artifact | | DOC08 Playbook | Operational runbooks for defect scenarios | | DOC09 Glossary | Canonical term definitions | | DOC10 Training and Support | Onboarding scripts, support escalation tiers | | DOC11 Promotion and GTM | Public messaging guardrails | | DOC12 Development History | Why prior architectural decisions were made | | DOC13 Diagrams | Sequence and state diagrams for voice, approval, tier cascade, resume | | DOC14 Unit Test Results | Coverage targets and representative test code | | DOC15 DR/BC | RTO/RPO targets, backup schedule, failover requirements | | DOC16 Security and Pen Testing | Threat model, standing absolute rules, test scope | | DOC17 BABA Compliance | Origin compliance checklist | | DOC18 SBOM | Approved dependency and license baseline | | DOC19 Project Ledger | Ledger schema, Atlas distillation feed | | DOC20 UAT Framework | Structured acceptance scripts, step by step | | DOC21 Platform Dashboard | Required metrics and their calculation | | Amendment 1 | Didymus integration reconciliation against the original suite | | Amendment 2 | Mobility Layer reconciliation against the original suite | | DOC26 Bootstrap/Self-Hosting | Bootstrap and Self-Hosting phase-gate criteria (`DOC26-Bootstrap-Self-Hosting.md`) | | DIDYMUS-ARCH-01 | Didymus hub architecture, worker contract, task storage, MCP tools | | DIDYMUS-PLAYBOOK | Practitioner provisioning of AI workers and approval-queue ops | | GS-MOB-01 | Field Mode, Extended Sessions, native audio bridge, wake-word | ## 10. Formatting Standards - No em dashes in any generated content or documentation. - No horizontal rules in Word documents; use spacing and bold headings. - Segoe UI Semibold for primary headings, Segoe UI for subheadings, Segoe UI Light for body text. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 152ad05ad821bf759d078ee044407bde4ab75db8 |
| 25 | DOC07 | DOC07 — Punchlist | punchlist | DevResources/docs/DOC07-Punchlist.md | | Item | Owner | Verification Artifact | | --- | --- | --- | | Docker Compose stack, all services healthy | Engineering | docker compose ps output, 10-min idle stability log | | gemstone-db schema migrations applied | Engineering | Alembic migration history, schema diff | | Auth0 RBAC roles configured | Engineering | RBAC matrix (DOC03 Sec 7) mapped to live Auth0 config export | | make rollback verified | Engineering | Rollback dry-run log, timestamped | | # DOC07 — Punchlist *Phased build checklist with owning role and verification artifact* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## Phase 0: Infrastructure | Item | Owner | Verification Artifact | | --- | --- | --- | | Docker Compose stack, all services healthy | Engineering | docker compose ps output, 10-min idle stability log | | gemstone-db schema migrations applied | Engineering | Alembic migration history, schema diff | | Auth0 RBAC roles configured | Engineering | RBAC matrix (DOC03 Sec 7) mapped to live Auth0 config export | | make rollback verified | Engineering | Rollback dry-run log, timestamped | ### Deferred / non-blocking (production closeout punchlist) | Item | Priority | Owner | Exact ask / keys | Status | | --- | --- | --- | --- | --- | | **Live Auth0 provisioning + EC2 wiring** | **Deferred — non-blocking for engineering** | Practitioner | Create Auth0 tenant; social Google/Facebook/LinkedIn; four roles (`SuperAdmin`, `Org Admin`, `Practitioner`, `GemStone Agent`); store eight keys in `DATARIGHT_DEV_SECRETS` per `DOC-Auth0-Provisioning.md` (`AUTH0_DOMAIN`, `AUTH0_AUDIENCE`, `AUTH0_CLIENT_ID`, `AUTH0_CLIENT_SECRET`, `AUTH0_ISSUER`, `AUTH0_ENABLED`, `VITE_AUTH0_DOMAIN`, `VITE_AUTH0_CLIENT_ID`). Reply “Auth0 secrets are in DATARIGHT_DEV_SECRETS” for agent wiring. | **DEFERRED 2026-07-21** — `/healthz` allows `auth0.configured:false`; phases 0–5 ran without live tenant. Work continues without live Auth0. | | **Real Atlas Vault credentials** | Deferred — non-blocking (stub gates PASS) | Practitioner | Add `ATLAS_MCP_ENDPOINT`, `ATLAS_MCP_API_KEY` (optional), optionally `ATLAS_MCP_MODE=auto` to `DATARIGHT_DEV_SECRETS`; follow `DOC-Atlas-Index-B.md` → recreate `api` → verify `/mcp/atlas/health` not stub. Confirmed absent from secrets (21 keys, no `ATLAS_*`) 2026-07-21. | **DEFERRED 2026-07-21** — leave stub; Phase 2A gates already PASS on stub. | | Third-party penetration test | Vendor gate | NEXTECH + vendor | Engage vendor using `DOC-PenTest-Engagement-Brief.md`; remediate Critical/High; re-test. | Prep complete; engagement pending | | BABA principal sign-off | Human gate | NEXTECH principal | Sign `DOC-BABA-Western-Origin-Checklist.md` / DOC17 against Syft SBOM; record in ledger. | Engineering checklist complete; sign-off pending | | App Store / Play Store accounts + submit | Account gate | Practitioner | Apple Developer + Google Play Console; signing keys in vault; follow `DOC-Store-Submission-Checklist.md`. | Assembly checklist complete; accounts pending | | Device UAT (DOC20 mobility) | Practitioner QA | QA / practitioner | Outdoor sunlight Field Mode; call interruption 10-trial; Android wake 8/10; iOS Siri 9/10 — see `DOC-Device-UAT-Checklist.md`. | Checklist authored; device runs pending | | **Capacitor Android ↔ live EC2** | Engineering | Engineering | Wire `server.url` + `apiOrigin` + debug TLS trust so Android runs in parallel with web (Auth0 deferred). | **DONE 2026-07-21** — see `DOC-Capacitor-Live-EC2.md`; device QA still open | | True 10k voice soak (DOC02 §6.10) | Capacity / engineering | Engineering | Needs voice RTT instrumentation + more RAM than g4dn.xlarge 16Gi for true concurrent voice. Root EBS grown 30→80G (online growfs; Whisper retained; ~53G free). Docker data ~114G free. **2026-07-21 probes:** 1k HTTPS `/healthz` 1000/1000 p95 199ms; 10k HTTPS `/healthz` 10000/10000 p95 403ms — **HTTP proxy only, `pass_doc02_full_gate: false`**. | **DEFERRED 2026-07-21** — per practitioner direction, skip until the rest of the solution is built out (feature-complete, without per-user-level gating). Revisit after core features land. Disk already grown; true 10k voice still punchlisted. | ### Stage 2 — Operating model (pre–Phase 0 gate) | Item | Owner | Status / Verification | | --- | --- | --- | | plink/`dr` + host-key pin to T4 EC2 | Engineering | Connected; fingerprint recorded in `project.env.example` / DOC-Deploy-Pipeline | | `DEVELOPMENT_DIRECTORY` clone + PAT pull | Engineering | `/home/dr/AppDev/gemstone-nexus`; origin URL clean | | Deploy scripts (commit→push→pull→compose) | Engineering | `scripts/run-remote-docker-deploy.ps1`, `scripts/aws/deploy.sh`, `DOC-Deploy-Pipeline.md` | | EventBridge spot cost schedule | Engineering | `gemstone-nexus-start/stop` ENABLED (09:00/01:00 ET) | | Terraform `launch_instance=true` hygiene | Engineering | Commented intentional; README guardrail updated — do not apply destroy | | Atomic-concurrency doc maintenance | Engineering | DOC08/12/13/19 + punchlist notes updated with Stage 2 | ## Phase 1A/1B: Voice Pipeline and Session Core | Item | Owner | Verification Artifact | | --- | --- | --- | | STT accuracy >= 92% | Engineering + QA | 200-utterance benchmark report | | TTS first chunk <= 800ms | Engineering | Latency measurement log | | WebSocket reconnection <= 5s | Engineering + QA | Reconnection test log, 20 trials | | Cross-org isolation, zero leakage | QA | Isolation test suite report, 50 concurrent sessions | ## Phase 2A: Atlas Integration - MCP Index B queries p95 <= 800ms under 50 concurrent practitioners (Engineering + QA, latency report). - Fine-tuning signal write-back verified within 2s of correction event (Engineering, log trace). - Named/dated session recall correct in 9/10 test cases (QA, test report). - **Status 2026-07-21:** COMPLETE tip `8232695` on EC2 `i-04333dae160ab0daa`. Gates: Index B p95 7.2ms @50, signal 10/10 ≤2s, recall 10/10; Phase 1B regression PASS. Real remote Atlas Vault not in `DATARIGHT_DEV_SECRETS` — stub backend; see `DOC-Atlas-Index-B.md`. ## Phase 2B/2C: Mobility Layer - Field Mode voice activation <1s (Engineering, device test log, Android + iOS). - Heads-Up UI direct-sunlight legibility (QA, manual outdoor test, signed off). - Extended Session survives call interruption, one-tap resume (QA, 10-trial test log per platform). - Android wake-word from killed state, 8/10 trials (QA, device test log). - iOS Siri Shortcut trigger, 9/10 trials (QA, device test log). - Chunked-hold failure-flagging, in-app + system log within 5 min (Engineering + QA, failure-injection test). - **Status 2026-07-21:** Engineering COMPLETE tip `a2af898` on EC2 `i-04333dae160ab0daa`. Field Mode UI + WebView audio + Recents API (≥20); Extended Session Android FG mic + wake FG scaffold; iOS Info.plist/Swift/Siri scaffold; interruption lifecycle REST identical across three causes (gate PASS). Device QA gates (sunlight, call 10-trial, wake 8/10 & 9/10) remain open. HorizonORBIT login still stubbed. ## Phase 3A/3B: Domain Packs - Pack install/activate <10s (Engineering, timing log). - All six packs pass individual UAT scripts (QA, six signed UAT reports, DOC20 Sec 2). - **Status 2026-07-21:** Engineering COMPLETE tip `7042354` on EC2 `i-04333dae160ab0daa`. Alembic `0002_phase3ab`; six first-party packs seeded in Postgres; DOC20 §2 UAT 6/6 PASS (`scripts/phase3ab-gates.sh`); max `activate_ms` **7.7ms** (budget 10000ms); HTTP smoke install→active prompt→uninstall history-untouched. Capacitor `/packs` UI build+sync PASS locally. Practitioner device sign-off optional. ## Phase 3C: Didymus Integration - No task reaches RUNNING without approved_by/approved_at (Engineering + QA, direct DB inspection report). - Approval queue shows no source distinction (QA, blind review test report). - Schema-level rejection of unapproved RUNNING transition (Engineering, forced-failure test log). - **Status 2026-07-21:** Engineering COMPLETE tip `eabc2df` on EC2 `i-04333dae160ab0daa`. Alembic `0003_phase3c`; `didymus_tasks_approval_gate` CHECK; three MVP workers (cursor/claude/gemini); REST + MCP tools; Capacitor `/didymus` blind-source queue; `scripts/phase3c-gates.sh` **PASS** — DOC20 §4 UAT ok, schema reject unapproved RUNNING, DB inspection RUNNING rows have approved_by/approved_at, Whisper CUDA retained (disk ~2.0G free after rebuild). - **Status 2026-07-22 (OD `i-0f5f3b32dfa9c6a12`):** Live worker execute + vendor-key vault shipped. `WORKER_SECRETS_KEY` enabled on host (vault_available=true). Smoke PENDING→APPROVED→schema-reject force-RUNNING→lease RUNNING→Gemini **live** execute COMPLETED (`didymus-ok`). Cursor/Claude practitioner keys still optional via `/app/onboarding/workers` or host env; Gemini uses stack `GEMINI_API_KEY`. Auth0 still deferred (local JWT optional, `AUTH_ENABLED=false`). ## Phase 2B/2C follow-on: Recents / Notebooks / Inferenced search - Recents chronological scroll-and-select ≥20 + notebook dual-visibility badges (US-07). - `POST /api/notebooks` + `POST /api/notebooks/{id}/link` additive (never removes from Recents). - `POST /api/search/inferenced` top-3 ≥9/10 (US-08 / FR-GS-006). - Gemini Live hello accepts `context_text` / `recall_query` for Atlas continuity. - **Status 2026-07-22:** LIVE on OD `i-0f5f3b32dfa9c6a12` tip `238eadb`. Alembic `0007_notebook_links`; `/api/notebooks` + `/api/search/inferenced`; Capacitor `/app/recents`; vault_available=true; Gemini Live `/` retained. - **Status 2026-07-23:** Agent page Notebooks are interactive end-to-end: create → open notebook → **Add current chat** / pick from Recents → click linked conversation restores turns (same path as Recents). `GET /api/notebooks/{id}` returns `sessions[{session_id,title,linked_at}]`. Dual-visibility unchanged (link never removes Recents). - WCAG AA automated scan, zero critical violations (QA, scan report). - Platform Dashboard load <2s (Engineering, performance trace). - All UAT Structured scripts passing on staging (QA, full UAT report, DOC20). ## Phase 5: Production Hardening - Playwright E2E suite, zero failures, two consecutive runs (Engineering, CI log). **Status 2026-07-21:** PASS 9/9 ×2. - Penetration test, Critical/High remediated and re-tested (NEXTECH principal sign-off, pen test report). **Prep:** `DOC-PenTest-Engagement-Brief.md` ready; **vendor engagement pending** (punchlist). - 10,000 concurrent session load test, p95 <3s (Engineering, load test report). **2026-07-21:** root EBS `vol-0907e6a4ef3d6ed66` grown **30→80G** + online growfs (Whisper healthy; ~53G root free). Docker data ~114G free. HTTPS `/healthz` proxy: **1k** 1000/1000 p95 **199ms**; **10k** 10000/10000 p95 **403ms** (`phase5-load-10k-http.json`). **True 10k voice RTT NOT claimed** on g4dn.xlarge 16Gi — punchlist capacity gate. - Final SBOM (Syft) and BABA sign-off (NEXTECH principal, signed compliance doc). **Engineering checklist:** `DOC-BABA-Western-Origin-Checklist.md`; **principal sign-off pending**. - App Store / Play Store submission packages assembled and submitted (Engineering, submission confirmation). **Assembly:** `DOC-Store-Submission-Checklist.md`; **accounts pending**. - Device UAT mobility scripts (sunlight / call / wake / Siri): `DOC-Device-UAT-Checklist.md` — **practitioner pending**. ## Production-safe backlog — Top 15 (DOC suite gap 2026-07-22) Source: [`DOC-Suite-Gap-Summary.md`](./DOC-Suite-Gap-Summary.md). Order for additive, zero-downtime delivery: | # | Item | Status | | --- | --- | --- | | 1 | Harden local AUTH (`AUTH_ENABLED`, strong `JWT_SECRET`, bootstrap SuperAdmin, `AUTH_ALLOW_SIGNUP=false` in prod) | **In progress 2026-07-22** — compose default signup closed; Admin invites replace open signup | | 2 | **Admin Panel MVP** `/app/admin` — users invite/role/deactivate, org pack availability, Atlas health RO, wake/GATE2 knobs | **Shipping 2026-07-22** | | 3 | Verify Alembic `0008` (+ `0009` admin) on live; smoke `vault_store` writers | **Verify on deploy** | | 4 | Wire real Atlas remote when `ATLAS_MCP_*` secrets land (keep stub fallback) | Deferred — secrets | | 5 | TierGuard + stop mid-session tier overwrite (DOC03 immutable) | **Done 2026-07-22** — `session/tier_guard.py`; manager preserves tier on reconnect | | 6 | Open Mode consent UX (US-04) → `consent_events` | Open | | 7 | RBAC enforcement audit (Didymus, packs, support view) | Partial — Admin RBAC gated | | 8 | Chunked-hold dual-flag (FR-GS-009) | Open | | 9 | Didymus Cursor/Claude worker vault keys smoke | Open (Gemini live) | | 10 | Device UAT mobility closeout | Practitioner | | 11 | WCAG AA automated scan; fix criticals | Open | | 12 | Research Agent (or DOC amend out of MVP) | Deferred | | 13 | Auth0 cutover after secrets phrase | Deferred | | 14 | Phase 5 human gates (pen-test, BABA, store) | Human | | 15 | True 10k voice soak (not `/healthz` proxy) | Deferred capacity | **Also shipping with this change set:** Project Ledger ↔ conversation (DOC ingest → `ProjectDocuments` + MCP tools + Gemini Live tool calls + sidebar Ledger link). ### GemStoneGuardRails (2026-07-23) | Item | Status | | --- | --- | | Product `GemStoneGuardRails` — match before answers; seed Mark Furiate / DataRight Horizon | Shipping | | Passcode-gated `guardrails_add`; SuperAdmin passcode + contributor flag | Shipping | | Alembic `0010_gemstone_guardrails`; Admin tab; MCP tools; Live + sovereign inject | Shipping | | Doc | [`DOC-GemStoneGuardRails.md`](./DOC-GemStoneGuardRails.md) | ### Push notifications — SNS + FCM + Capacitor (2026-07-23) | Item | Status | | --- | --- | | Scaffold: `device_push_tokens`, `/api/push/register|unregister`, `notify_practitioner` | **Scaffold shipped** | | Didymus hooks (approval needed / task completed) best-effort; gate untouched | **Scaffold shipped** | | Capacitor `@capacitor/push-notifications` + register on login | **Scaffold shipped** | | Env placeholders `SNS_PUSH_PLATFORM_ARN_ANDROID`, `AWS_SNS_REGION=us-east-1` | **In place** (no FCM in secrets yet) | | **Mark: Firebase Android app** (`com.nextech.gemstone`) + `google-services.json` | **Blocked — Mark** | | **Mark: secrets** `FCM_SERVER_KEY`, `FCM_PROJECT_ID`, then SNS platform app → `SNS_PUSH_PLATFORM_ARN_ANDROID` | **Blocked — Mark** | | Real device push delivery | **Blocked on Firebase** | | APNs / iOS | Deferred | | Doc | [`DOC-Push-Notifications-SNS.md`](./DOC-Push-Notifications-SNS.md) | ### Field Mode DevExternal + export / Artifacts / Send Email (2026-07-23) | Item | Status | | --- | --- | | Field Mode → Dev External Live when available; sessions in Recents/Search/Notebooks | Shipping | | Export transcript → email PDF (practitioner) or Artifacts downloadable | Shipping | | **Send Email** command → “What email…?” → SES report (body + PDF) | Shipping | | Artifacts CRUD `/api/artifacts` + `/app/artifacts`; Alembic `0013_artifacts` | Shipping | ### Practitioner personal rules (GemStone MVP — shipped 2026-07-23) *Distinct from product-wide `GemStoneGuardRails`. HorizonIQ deferrals below remain out of scope.* | Item | Status | | --- | --- | | Personal Atlas segment (`org_id` + `practitioner_id`); Alembic `0011_practitioner_rules` | **MVP shipped** | | Phrase **"Make that a rule"** — capture from recent turns; signed-in required | **MVP shipped** | | Overlap → remind + **keep original** / **use new**; superseded retained (never delete) | **MVP shipped** | | Rule history phrase; active rules in chat/Live context after GuardRails | **MVP shipped** | | MCP: match / propose / commit / history / list; Live + sovereign short-circuit | **MVP shipped** | | Doc | [`DOC-Practitioner-Rules.md`](./DOC-Practitioner-Rules.md) | | Intent (unchanged) | Notes | | --- | --- | | Scope | **Personal** Atlas segment only (per signed-in practitioner) | | Approvals | **keep original** vs **use new** via **chat or voice** | | Auth | User **must be signed in** so the correct Atlas segment is referenced/updated | | Passcode | **No** passcode on personal practitioner rules (MVP). Product GuardRails keep SuperAdmin passcode | ### HorizonIQ (later) *Deferred from practitioner-rules design — out of scope for current GemStone MVP. Revisit under HorizonIQ / suite expansion.* | # | Item | Why deferred | | --- | --- | --- | | 1 | **Org-wide shared practitioner rules** (beyond personal Atlas segment) | Org sharing / multi-practitioner rule packs is HorizonIQ territory; GemStone MVP stays personal-segment only | | 2 | **"Make that a rule" affecting or merging with GemStoneGuardRails** | Product-wide guardrails stay separate for now; no auto-promote from personal rules into GuardRails | | 3 | **Passcode gate on personal practitioner rules** | Personal rules currently no passcode; GuardRails keep SuperAdmin passcode — do not unify gates in MVP | --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 13d4e030c99ba88329f0dfabd8ca2f91ad4e055d |
| 26 | DOC08 | DOC08 — Playbook | playbook | DevResources/docs/DOC08-Playbook.md | 1. Query the sessions table for the session_id; confirm state and check interruption_snapshot is non-null. 2. If interruption_snapshot is null and state is PAUSED, this is a defect: the snapshot write failed silently. Escalate immediately; do not attempt manual state repair without engineering. 3. If state is PAUSED but the session-ready chime should already have fired, confirm cue scheduling; stuck-before-chime is distinct from stuck-after-chime (RESUMABLE). 4. If interrupt… | # DOC08 — Playbook *Operational runbooks for common production scenarios* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Runbook: Session Stuck in PAUSED or RESUMABLE 1. Query the sessions table for the session_id; confirm state and check interruption_snapshot is non-null. 2. If interruption_snapshot is null and state is PAUSED, this is a defect: the snapshot write failed silently. Escalate immediately; do not attempt manual state repair without engineering. 3. If state is PAUSED but the session-ready chime should already have fired, confirm cue scheduling; stuck-before-chime is distinct from stuck-after-chime (RESUMABLE). 4. If interruption_snapshot is present and state is RESUMABLE (or PAUSED ready to resume), verify the practitioner's app can reach POST /api/sessions/{id}/resume; a 4xx/5xx here indicates an API-layer issue, not a data issue. 5. Confirm the resume restores last_turn correctly by comparing against the transcript stored in Atlas Vault for that session_id. ## 2. Runbook: Suspected Tier Violation 1. Pull the session record; confirm the tier field value has never changed by checking the audit log (tier is immutable; any change event is itself the defect). 2. Identify which action triggered the suspected violation and check whether it appears in RESTRICTED_ACTIONS for that tier in the TierGuard configuration. 3. If the action was permitted but should not have been, this is a TierGuard configuration defect, not a data defect; do not patch the individual session, fix the enforcement rule. 4. Confirm no secondary tier-check code path exists elsewhere in the codebase; a second enforcement point is itself a defect regardless of whether it agrees with TierGuard. ## 3. Runbook: Didymus Task Reached RUNNING Without Approval 1. This should be structurally impossible; treat as a Severity 1 defect immediately upon discovery. 2. Freeze the worker that executed the task; do not allow it to lease further tasks until the schema constraint is confirmed intact. 3. Inspect the database constraint directly (not application code) to confirm the approved_by/approved_at NOT NULL and CHECK constraints are present and unmodified. 4. If the constraint was bypassed via direct database write rather than the API, this is an access-control issue, not a Didymus logic issue; escalate accordingly. ## 4. Runbook: Vectorization Failure 1. Confirm the chunked-hold state: raw audio should be present and intact, not deleted, for any session in a failed vectorization state. 2. Confirm both flags fired: in-app notification delivered to the practitioner and a system-log entry created, within 5 minutes of the failure per DOC03 exit criteria. 3. If only one flag fired, this is a defect in the dual-flagging mechanism itself, not a one-off failure; escalate to engineering rather than manually notifying the practitioner as a workaround. 4. Once the underlying failure (transcription, indexing, or vectorization stage) is identified, re-run the pipeline against the held audio chunks; do not discard audio until vectorization succeeds. ## 5. Escalation Matrix | Scenario | Severity | Escalate To | | --- | --- | --- | | Task reaches RUNNING without approval | Sev 1, immediate | NEXTECH principal + engineering lead | | Tier violation confirmed | Sev 1, immediate | NEXTECH principal + engineering lead | | Repeated/systemic vectorization failures | Sev 2, same-day | Engineering lead | | Wake-word false-trigger rate above threshold in field data | Sev 3, next sprint | Engineering lead, informs Gate 2 tuning | | HUME AI status inquiry | Informational | NEXTECH principal, no timeline commitment given | ## 6. Co-dev Deploy (Stage 2 operating model) Primary build host is the Ohio T4 EC2. Windows authors code; Docker runs only on the host. 1. Confirm instance up: `i-04333dae160ab0daa` (public IP may change — verify via AWS CLI). 2. Pin SSH host key if relaunched: `.\scripts\remote-exec.ps1 -RefreshHostKey`; update `DR_SSH_HOSTKEY` in `DevResources/project.env`. 3. Ensure worktree: `.\scripts\ensure-ec2-dev-dir.ps1` → `DEVELOPMENT_DIRECTORY=/home/dr/AppDev/gemstone-nexus`. 4. **Deploy** after a deliberate commit+push: `.\scripts\run-remote-docker-deploy.ps1` (pull → `docker compose up` → health). On-host equivalent: `scripts/aws/deploy.sh`. 5. PAT and `DR_SSH_PASSWORD` come from Secrets Manager `DATARIGHT_DEV_SECRETS` — never commit them. Keep the EC2 `origin` URL free of embedded tokens. 6. Full procedure: `DevResources/docs/DOC-Deploy-Pipeline.md`. Atomic concurrency: every Deploy that changes schema/API/UI must update ledger + DOC12/DOC07 in the same change set (Stage 3+). --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 941ae1407da6ade03137695c45d484ab813f54c5 |
| 27 | DOC09 | DOC09 — Glossary of Terms | doc_suite | DevResources/docs/DOC09-Glossary.md | GemStone: the voice-first, mobile-first agentic interface layer of the NEXTECH Triptych, and the single universal presentation/approval surface for all build task suggestions. | # DOC09 — Glossary of Terms *Consolidated terminology* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## Platform and Orchestration Terms GemStone: the voice-first, mobile-first agentic interface layer of the NEXTECH Triptych, and the single universal presentation/approval surface for all build task suggestions. Didymus: the AI-orchestration hub and session-capture pipeline coordinating Cursor, Claude, Gemini, and future workers. Never referred to as HorizonNEXUS. HorizonNEXUS: the market-facing name for the Triptych (GemStone, Horizon Atlas, Horizon IQ). Distinct from Didymus. Atlas Vault: the per-practitioner memory backbone, accessed via MCP Index B. DidymusWorker: the generic contract (get_capabilities, execute_command, report_status, verify_output) implemented by every connected AI tool or desktop application. TierGuard: the single enforcement point for the three-tier security model. Gate: a phase-completion checkpoint with numeric, binary pass/fail criteria; independently verified, not self-certified by the implementing engineer. ## Voice and Mobility Terms STTProvider / TTSProvider: abstract base classes for the voice provider contract. Field Mode: foreground, tap-to-talk voice interaction via standard Capacitor WebView audio. Extended Session: Didymus-directed or research/planning voice interaction requiring native background-audio entitlements. Notebook: a named, practitioner-created grouping construct; association with a session is additive, never a move. Recents: the chronological, unfiltered session timeline; always a live query, never a filtered table. Inferenced Search: GemStone actively querying Atlas Vault and, where permitted, Horizon IQ, on request. ## Governance Terms Bootstrap Phase / Self-Hosting Phase / Phase Gate / Standing Invariant / Experimental Fork: see DOC26-Bootstrap-Self-Hosting.md Section 10. Research Agent: the query-isolation and entity-scrubbing component gating all external research queries. RACI: Responsible, Accountable, Consulted, Informed; used in DOC02 Section 5 to assign delivery ownership. Security Tier display names: High Security, Standard, Open Mode. Wire/enum values: high_security / HIGH_SECURITY, standard / STANDARD, open_mode / OPEN_MODE (DOC03 Section 1). RESUMABLE: session state after PAUSED when the session-ready chime has fired; one tap or app reopen returns ACTIVE (DOC03 Section 5, DOC13 Section 4). DataRightGoldenOVAFullBase: locked local clean-VM clone source for the secondary/offline environment (`D:\Virtual Machines\DataRightGoldenOVAFullBase\`). --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | af5703318ad8d25635a5a267c44707005055d71b |
| 28 | DOC10 | DOC10 — Training and Support | doc_suite | DevResources/docs/DOC10-Training-and-Support.md | 1. Security tier selection: explain High Security, Standard, and Open Mode (canonical display names; wire values high_security / standard / open_mode per DOC03); confirm the practitioner understands tier is immutable for the session once chosen. Script cue: "Once we start, this session stays at the level we pick, we can't loosen or tighten it partway through." 2. Notebooks, Recents, and inferenced search: introduce as three distinct capabilities, since practitioners coming f… | # DOC10 — Training and Support *Onboarding sequence, support escalation, and scripts* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Onboarding Sequence 1. Security tier selection: explain High Security, Standard, and Open Mode (canonical display names; wire values high_security / standard / open_mode per DOC03); confirm the practitioner understands tier is immutable for the session once chosen. Script cue: "Once we start, this session stays at the level we pick, we can't loosen or tighten it partway through." 2. Notebooks, Recents, and inferenced search: introduce as three distinct capabilities, since practitioners coming from Gemini Live may expect the either-or behavior GemStone deliberately does not have. Script cue: "Putting something in a Notebook never hides it from your recent history, they both just work." 3. Domain Pack selection: help the practitioner identify and install the first-party pack matching their practice area. 4. Wake-phrase configuration: set up the configured phrase list; confirm Field Mode vs. Extended Session intent. 5. iOS practitioners: walk through Siri Shortcuts setup for wake-from-killed-state activation. ## 2. Support Escalation Tiers | Tier | Handles | Escalates To | | --- | --- | --- | | L1 | Onboarding questions, general how-to, wake-phrase setup | L2 for anything involving data/session integrity | | L2 | Session recall issues, Domain Pack install problems, tier confusion | L3 (Engineering) for confirmed defects per DOC08 runbooks | | L3 / Engineering | Confirmed defects, Sev 1/2 escalations from DOC08 | NEXTECH principal per DOC08 Escalation Matrix | ## 3. Standing Support Guidance The human-approval gate is a permanent product characteristic, not a limitation to be worked around: every build suggestion, from any source, requires practitioner approval before execution, and this does not change as GemStone matures. HUME AI voice-emotion features should be described as unavailable pending formal review; support staff should not speculate on a timeline. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | dfb5a523855a3c8cf9b420bf0a13b36ff33acad3 |
| 29 | DOC11 | DOC11 — Promotion and Go-to-Market | doc_suite | DevResources/docs/DOC11-Promotion-and-GTM.md | GemStone is positioned as a full-capability AI partner, not a chatbot, and its mobility layer is explicitly positioned against the documented shortcomings of existing voice-assistant products, without naming competitors directly in public-facing materials. | # DOC11 — Promotion and Go-to-Market *Positioning, messaging guardrails, and channel plan* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Positioning GemStone is positioned as a full-capability AI partner, not a chatbot, and its mobility layer is explicitly positioned against the documented shortcomings of existing voice-assistant products, without naming competitors directly in public-facing materials. ## 2. Messaging Guardrails - Public positioning remains "Strategic Business and Technology Partner," not "Fractional CTO." - Do not reference Raspberry Pi or any specific hardware dependency; GemStone and HorizonOS are hardware-agnostic. - Quantified technology ROI is always stated as $750M+. - Do not publicly detail the Research Agent entity-scrubbing mechanics or specific provider list beyond general "privacy-preserving external research" framing. ## 3. Channel Plan | Channel | Content Type | Cadence | | --- | --- | --- | | The Signal newsletter | Thought leadership on AI discipline, execution frameworks | Per standing Issue cadence | | LinkedIn | Positioning content, no competitor naming | Per standing social schedule | | Direct practitioner onboarding | Product walkthrough per DOC10 Section 1 | At time of license activation | --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | ae064c1a20883641935452f84c67072975a69080 |
| 30 | DOC12 | DOC12 — Development History | doc_suite | DevResources/docs/DOC12-Development-History.md | 1. Original 21-document GemStone suite built: React Native/Expo/PWA mobile stack, Whisper/Piper voice pipeline, Atlas Vault via MCP Index B, Domain Pack framework, three-tier security model. 2. Amendment 1 (v1.1): security tier infrastructure, voice provider abstraction, External Research API added to Phase 1 MVP. HUME AI placed in Pending Review status. 3. Circular dependency identified in applying the standard Triptych-governance DOC26 template to GemStone itself; resolved… | # DOC12 — Development History *Chronological record of major architectural decisions* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## Timeline 1. Original 21-document GemStone suite built: React Native/Expo/PWA mobile stack, Whisper/Piper voice pipeline, Atlas Vault via MCP Index B, Domain Pack framework, three-tier security model. 2. Amendment 1 (v1.1): security tier infrastructure, voice provider abstraction, External Research API added to Phase 1 MVP. HUME AI placed in Pending Review status. 3. Circular dependency identified in applying the standard Triptych-governance DOC26 template to GemStone itself; resolved via Bootstrap/Self-Hosting two-phase lifecycle. 4. Didymus named and specified as the AI-orchestration hub (DIDYMUS-ARCH-01), distinct from HorizonNEXUS. 5. Human-in-the-loop established as a permanent, universal invariant: no autonomous execution for any task category, ever, at any maturity level. 6. Mobile stack migrated from React Native/Expo to Capacitor, aligning with DataRight Workforce and the prior StackOne Capacitor seed. 7. GemStone Mobility Layer specified (GS-MOB-01) in direct response to a documented review of Gemini Live production failures. 8. Universal orchestration architecture confirmed: GemStone is the single presentation/approval surface for all AI-tool-originated suggestions. 9. Field Mode / Extended Session audio architecture split finalized. 10. Android prioritized over iOS for native audio work; full default-assistant replacement considered and deferred. 11. v2.0 rewrite: full document set deepened with real schemas, API contracts, state machines, and testable acceptance criteria following an initial pass judged insufficiently detailed for Cursor to build from directly. 12. Stage 0 decision lock (July 2026): Auth0 + social logins; sovereign-first T4 backend; Postgres+Redis+SQLite ledger; Capacitor atomic concurrency; DataRightTailwind + HorizonORBIT login; primary EC2 `i-04333dae160ab0daa`; local clone source locked to `DataRightGoldenOVAFullBase`; DOC26 / DIDYMUS-ARCH-01 / GS-MOB-01 / DIDYMUS-PLAYBOOK authored into `DevResources/docs/`; legacy product-name purge deferred until first green GoldenAMI build. 13. Stage 1 (July 2026): DataRightAppBundle repaired (22-doc alignment, Capacitor path, missing scripts/Dockerfiles, SaaS modules, phase graph). 14. Stage 2 (July 2026): Operating model + Deploy pipeline. `DEVELOPMENT_DIRECTORY=/home/dr/AppDev/gemstone-nexus` on T4 EC2; plink/`dr` + PEM break-glass; host key pinned; PAT pull via Basic auth without storing token in remote URL; scripts `remote-exec.ps1`, `ensure-ec2-dev-dir.ps1`, `run-remote-docker-deploy.ps1`, `scripts/aws/deploy.sh`; EventBridge start/stop confirmed ENABLED; `launch_instance=true` documented as intentional for live host. Full product Deploy deferred until Stage 3 commit. 15. Phase 3A/3B (July 2026): Domain Pack system — Alembic `0002_phase3ab` extends `domain_packs` + practitioner-scoped `pack_installations` (RLS org_id retained); six first-party packs (PE Operating Partner, Healthcare Executive, FinTech/Capital Markets, Management Consulting, Construction/Real Estate, Manufacturing Operations); DOC03 install/uninstall API; Capacitor `/packs` UI; DOC20 §2 UAT harness with <10s activate gate. 16. Phase 3C (July 2026): Didymus universal approval orchestration — Alembic `0003_phase3c` status machine + `didymus_tasks_approval_gate` (RUNNING/COMPLETED/FAILED/VERIFIED require `approved_by`+`approved_at`); `didymus_workers` registry; three MVP workers (Cursor/Claude/Gemini) implementing DidymusWorker contract; REST `/api/didymus/*` + MCP `/mcp/didymus/*`; Capacitor `/didymus` blind-source queue; DIDYMUS-PLAYBOOK updated with secrets placement and registration steps. 17. Phase 4 (July 2026): Platform Dashboard (DOC21) — Alembic `0004_phase4` metric tables (`consent_events`, `audio_pipeline_holds`, `wake_activations`, `ledger_entries`, `sessions.ended_at`); REST `/api/dashboard` with eight views; Capacitor `/dashboard` under DataRightTailwind tokens; Auth0-ready HorizonORBIT-style login without fake tenant; WCAG AA focus/skip-link polish; DOC20 automatable UAT recorder in `phase4_bench`; Phase 5 hardening scaffold only (Playwright/load/SBOM stubs — pen test external). 18. Phase 5 engineering (July 2026): Playwright critical-path E2E, scaled load proxy + report, Syft/SBOM script + BABA checklist, pen-test engagement brief, store-submission prep checklist; full DOC02 Phase 5 exit still blocked on third-party pen test, 10k voice soak, principal BABA sign-off, and App/Play accounts. 19. Naming purge (July 2026): after first green GoldenAMI build, legacy product naming removed from code/config/docs; CI grep gate `scripts/ci/forbid-legacy-product-name.*` fails on reintroduction. 20. Production closeout punchlist (July 2026): Live Auth0 deferred non-blocking (`DOC-Auth0-Provisioning.md` + DOC07); Atlas vault keys absent from `DATARIGHT_DEV_SECRETS` — stub retained on punchlist; pen-test/BABA/store/device-UAT prep docs complete with human gates listed; EC2 root EBS 30→80G + HTTPS 10k `/healthz` proxy (not voice gate). ## Historical footnote (allowlisted) > StackOne frontend/backend seed for this repo was sourced from a prior StackOne > extraction formerly branded **DotOnTheMap**. That brand name is retained only > in this footnote for provenance; CI allows at most two `DotOnTheMap` mentions > in this file. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | e66c234145db45312d53d32e6b10952844c921df |
| 31 | DOC13 | DOC13 — Diagrams | doc_suite | DevResources/docs/DOC13-Diagrams.md | ``` Practitioner gemstone-api atlas-ollama (Whisper) gemstone-tts (Piper) |-- audio chunk -->| | | | |-- stream ---------->| | | |<-- partial transcript| | | |-- final transcript ->| (VAD end-of-turn) | | |-- query MCP Index B (Atlas context) ----------->| | |-- response t… | # DOC13 — Diagrams *Sequence and state diagrams* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Voice Round-Trip Sequence ``` Practitioner gemstone-api atlas-ollama (Whisper) gemstone-tts (Piper) |-- audio chunk -->| | | | |-- stream ---------->| | | |<-- partial transcript| | | |-- final transcript ->| (VAD end-of-turn) | | |-- query MCP Index B (Atlas context) ----------->| | |-- response text ------------------------------->| | | | stream synthesize --->| |<-- audio chunk --|<--------------------------------------- audio --| | (first chunk within 800ms of first response sentence) | ``` ## 2. Didymus Task Approval Flow ``` Any Worker/GemStone Didymus Hub GemStone UI (universal surface) Human |-- submit_task -------->| | | | |-- PENDING, visible ---------->| | | | |-- displayed, no source badge->| | |<-- approve_task --------------|<---------------------------- | | |-- APPROVED, approved_by/at set| | |<-- lease eligible -----| | | |-- execute, report ---->|-- COMPLETED/VERIFIED -------->| | ``` ## 3. Security Tier Cascade ``` effective_tier = min(iq_tier, atlas_tier, gemstone_tier, practitioner_tier) Example: IQ=Standard(1), Atlas=High Security(0), GemStone=Standard(1), Practitioner=Open Mode(2) -> effective_tier = min(1, 0, 1, 2) = 0 = High Security (Atlas being the most restrictive layer forces the whole session down, regardless of what the practitioner personally selected.) Display names: High Security, Standard, Open Mode. Wire values: high_security, standard, open_mode. ``` ## 4. Session Interruption and Resume ``` ACTIVE --[OS audio focus loss (incl. incoming call) / network drop / app backgrounding]--> snapshot written --> PAUSED PAUSED --[session-ready chime fires]--> RESUMABLE RESUMABLE --[one tap or app reopen]--> restore from Atlas Vault --> ACTIVE (Identical path regardless of which of the three interruption causes triggered it. Extended Session timeout with confirmation ends the session; it is not an interruption cause.) ``` ## 5. Co-dev Deploy Pipeline (Stage 2) ``` Windows co-dev GitHub T4 EC2 (dr) |-- edit / commit ------------>| | |-- git push (PAT local) ----->| | |-- plink: run-remote-docker-deploy.ps1 --------------------->| | |<-- git pull (PAT Basic, ephemeral) | | docker compose up -d [--build] | | nginx :80/:443 (compose) | | supervisor: Stage 3 API image |<-- compose ps / health ------| | DEVELOPMENT_DIRECTORY = /home/dr/AppDev/gemstone-nexus Atomic concurrency: schema + API + Swagger + UI + tests + DOC12/DOC07/ledger in one change set. ``` --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 2afc90475b51a3867d9fca96dd0ea717a2cdfcb2 |
| 32 | DOC14 | DOC14 — Unit Test Results | doc_suite | DevResources/docs/DOC14-Unit-Test-Results.md | GemStone implements a three-tier testing architecture plus a UAT framework. All runs are logged to a dated narrative file and the Project Ledger database. make test executes all tiers in sequence. | # DOC14 — Unit Test Results *Testing architecture, coverage targets, and representative test cases. Dated pass/fail narratives are appended to this document and the Project Ledger as phases close; until then this DOC is the coverage contract, not a completed results log.* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Testing Architecture Overview GemStone implements a three-tier testing architecture plus a UAT framework. All runs are logged to a dated narrative file and the Project Ledger database. make test executes all tiers in sequence. ## 2. Test Tiers | Tier | Framework | Runs In | Scope | | --- | --- | --- | --- | | Unit (backend) | Pytest | gemstone-api container | Service functions, 100% function coverage required | | Unit (frontend) | Vitest + RTL | gemstone-mobile container | React components, 80%+ line coverage required (API clients 90%+; see Section 4) | | Integration | Pytest + test DB | gemstone-api + test postgres | API/DB flows, org_id scoping, tier enforcement | | E2E | Playwright + Chromium | playwright container | Full stack via NGINX, critical user flows | | UAT Structured | Custom script runner | gemstone-api | Predefined flow scripts, pass/fail per step | | UAT Behavioral | Event instrumentation | gemstone-mobile (continuous) | All primary user actions, logged to Ledger | ## 3. Representative Test Cases ``` def test_tier_immutable_after_creation(): session = create_session(tier=SecurityTier.STANDARD) with pytest.raises(IntegrityError): db.execute(update(Session).where(Session.id == session.id).values(tier="open_mode")) def test_didymus_task_cannot_reach_running_without_approval(): task = submit_task(requested_by="Cursor", target_worker="cursor") assert task.status == "PENDING" with pytest.raises(IntegrityError): db.execute(update(DidymusTask).where(DidymusTask.id == task.id).values(status="RUNNING")) def test_notebook_link_does_not_remove_from_recents(): session = create_session(...) link_to_notebook(session.id, notebook_id) recents = get_recents(practitioner_id) assert session.id in [s.id for s in recents] def test_cross_org_isolation_under_concurrency(): sessions_a = [create_session(org_id=ORG_A) for _ in range(25)] sessions_b = [create_session(org_id=ORG_B) for _ in range(25)] # assert no query path for org A ever returns org B data async def test_tts_first_chunk_latency(): start = time.monotonic() first_chunk = await anext(tts_provider.stream_synthesize("Hello there.")) assert (time.monotonic() - start) < 0.8 ``` ## 4. Coverage Requirements | Layer | Coverage Target | Enforcement | | --- | --- | --- | | Python API (services) | 100% function coverage | Merge to main blocked if untested | | Python API (routers) | 90%+ line coverage | Happy-path and primary error cases | | TypeScript (components) | 80%+ line coverage | User interactions and state changes | | TypeScript (API clients) | 90%+ line coverage | Mock responses, all error states | | Integration (API + DB) | Key flows only | Voice session, Atlas context, tier enforcement, Didymus gate, org_id scoping | | E2E (Playwright) | Critical flows | Voice round-trip, pack install, Notebook/Recents, task approval, Field Mode | ## 5. Phase 5 status (2026-07-21) | Item | Status | | --- | --- | | Playwright critical-path suite (`frontend/e2e/`) | **PASS** 2026-07-21: `npm run test:e2e:twice` → 9/9 then 9/9 | | Device / sunlight / wake UAT (DOC20) | Practitioner — not claimed by Playwright | | Unit/integration coverage targets (Section 4) | Standing contract; not re-measured this Phase 5 engineering pass | See `DevResources/docs/DOC-Phase5-Hardening-Scaffold.md` for the Phase 5 engineering vs practitioner matrix. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 15aaaee843054f450564c886b53510f6a17c5cb9 |
| 33 | DOC15 | DOC15 — Disaster Recovery and Business Continuity | doc_suite | DevResources/docs/DOC15-DR-BC.md | | System | RTO | RPO | Backup Frequency | | --- | --- | --- | --- | | gemstone-db (sessions, tiers, ledger) | 15 minutes | 5 minutes | Continuous WAL archiving + hourly snapshot | | Redis session state | 5 minutes | 0 (AOF persistence enabled) | AOF every write | | Atlas Vault (external dependency) | Governed by Atlas Vault's own DR plan | Governed by Atlas Vault's own DR plan | N/A, out of GemStone scope | | Didymus raw archive | 30 minutes | 15 minutes | Hourly snapshot | | # DOC15 — Disaster Recovery and Business Continuity *RTO/RPO targets, backup schedule, and failover* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Recovery Targets | System | RTO | RPO | Backup Frequency | | --- | --- | --- | --- | | gemstone-db (sessions, tiers, ledger) | 15 minutes | 5 minutes | Continuous WAL archiving + hourly snapshot | | Redis session state | 5 minutes | 0 (AOF persistence enabled) | AOF every write | | Atlas Vault (external dependency) | Governed by Atlas Vault's own DR plan | Governed by Atlas Vault's own DR plan | N/A, out of GemStone scope | | Didymus raw archive | 30 minutes | 15 minutes | Hourly snapshot | ## 2. Session-Level Resilience GemStone's state-capture-on-interruption behavior is itself a business-continuity mechanism at the session level: no practitioner work is lost to a dropped connection, backgrounded app, or timed-out session, since all are treated identically as clean, resumable pauses. ## 3. Failover Architecture gemstone-api runs behind NGINX with at least 2 replicas in production; a single replica failure must not drop active WebSocket sessions for more than the Redis-backed reconnection window (5 seconds per DOC02 Phase 1B exit criteria). ## 4. Restoration Test Cadence A full restore-from-backup drill for gemstone-db is performed quarterly at minimum, with results logged to the Project Ledger and any RTO/RPO miss escalated per DOC08 Section 5. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 86a638dd14b746886d74b8eabbdd728dbc3c652a |
| 34 | DOC16 | DOC16 — Security and Penetration Testing | doc_suite | DevResources/docs/DOC16-Security-and-Pen-Testing.md | - Session transcripts are never transmitted externally, in any security tier, without exception. - Human approval is required for every build task execution, without exception, regardless of platform maturity. - No audio data is ever sent to external APIs. - AES-256 encryption at rest; TLS 1.3 in transit. | # DOC16 — Security and Penetration Testing *Threat model, standing rules, and test scope* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Standing Absolute Rules - Session transcripts are never transmitted externally, in any security tier, without exception. - Human approval is required for every build task execution, without exception, regardless of platform maturity. - No audio data is ever sent to external APIs. - AES-256 encryption at rest; TLS 1.3 in transit. ## 2. Threat Model, Selected Entries | Threat | Category | Mitigation | | --- | --- | --- | | Tampering with tier field to escalate access mid-session | Tampering | Database-level immutability constraint; TierGuard as sole enforcement point | | Task forced to RUNNING without approval via direct API/DB write | Elevation of Privilege | Schema-level NOT NULL/CHECK constraints on approved_by/approved_at, not application logic alone | | Cross-org data disclosure under concurrent load | Information Disclosure | org_id scoping enforced at query layer, tested under concurrency (DOC14 Section 3) | | Ambient audio captured outside an explicit mic-open session | Information Disclosure / Privacy | No intermediate buffer state exists by design; capture begins only on button/wake-word event | | Raw audio retained indefinitely after a silent pipeline failure | Repudiation / Information Disclosure | Chunked-hold plus mandatory dual flagging (in-app + system log) | ## 3. Test Scope - Verify TierGuard is the sole enforcement point; scan for secondary or bypassable tier-check implementations. - Verify tier immutability; no code path allows mid-session escalation or downgrade. - Verify OrgID scoping; no cross-org leakage under load or concurrent access. - Verify no ambient audio capture outside an explicitly open mic session. - Verify Android foreground-service microphone use carries the required persistent notification. - Verify iOS background-audio entitlement usage is scoped to Extended Sessions only. - Verify entity scrubbing executes on every Research Agent query before reaching any external provider. - Verify audit tokens are generated and logged for every external query. ## 4. Penetration Testing Full penetration test at Phase 5, covering OWASP Top 10 categories at minimum. All Critical and High findings must be remediated and re-tested before production deployment; Medium findings require a documented remediation plan with owner and date. ### Phase 5 status (2026-07-21) | Item | Status | | --- | --- | | Engineering threat-model + scope (this DOC §§1–3) | Current | | Engagement brief for vendor | `DevResources/docs/DOC-PenTest-Engagement-Brief.md` | | Third-party pen test execution | **Not run** — requires vendor engagement (cannot execute in-agent) | | Critical/High remediation + re-test | Blocked on vendor report | --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 7f6c87be0093fb1d83fec9004fa78250e9ebd626 |
| 35 | DOC17 | DOC17 — BABA Compliance | doc_suite | DevResources/docs/DOC17-BABA-Compliance.md | GemStone and its Mobility Layer adhere to the standing Western-origin technology stack constraint across AI/ML frameworks, messaging infrastructure, and MQTT brokering. Core voice components (Whisper/OpenAI/US, Piper/Rhasspy/US, Ollama/US) are confirmed Western-origin. | # DOC17 — BABA Compliance *Compliance statement and sign-off checklist* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Compliance Statement GemStone and its Mobility Layer adhere to the standing Western-origin technology stack constraint across AI/ML frameworks, messaging infrastructure, and MQTT brokering. Core voice components (Whisper/OpenAI/US, Piper/Rhasspy/US, Ollama/US) are confirmed Western-origin. ## 2. Sign-Off Checklist | Item | Status | Sign-Off | | --- | --- | --- | | AI/ML frameworks confirmed Western-origin | Engineering checklist complete (`DOC-BABA-Western-Origin-Checklist.md`); **principal audit Pending** | NEXTECH principal | | Messaging stack confirmed AWS-native only | Engineering: EventBridge schedule confirmed; no non-AWS messaging in MVP; **principal audit Pending** | NEXTECH principal | | MQTT confirmed Eclipse Mosquitto only | N/A for current MVP compose (Mosquitto not introduced); if added later must be Eclipse only; **principal audit Pending** | NEXTECH principal | | HUME AI confirmed not integrated | Confirmed not integrated; formal principal review remains Pending (DOC03 Sec 3) | NEXTECH principal | | Final SBOM cross-checked against this checklist | Syft script ready (`scripts/security/sbom-stub.sh`); artifact under `DevResources/security/`; **principal cross-check Pending** | NEXTECH principal | Engineering evidence package: `DevResources/docs/DOC-BABA-Western-Origin-Checklist.md`. DOC02 Phase 5 exit still requires NEXTECH principal sign-off — not claimed by this engineering pass. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 6b5cb53a613b8d014c911760e7f17cd0af6bcded |
| 36 | DOC18 | DOC18 — Software Bill of Materials | doc_suite | DevResources/docs/DOC18-SBOM.md | | Component | Origin | License | Container | | --- | --- | --- | --- | | FastAPI | OSS (International) | MIT | gemstone-api | | Whisper large-v3 (weights) | OpenAI (US) | MIT | atlas-ollama | | Ollama runtime | Ollama Inc. (US) | MIT | atlas-ollama | | Piper TTS runtime | Rhasspy (US/International, open-source) | MIT | gemstone-tts | | ONNX Runtime | Microsoft (US) | MIT | gemstone-tts | | webrtcvad | Google (US) | BSD-3-Clause | gemstone-api | | PostgreSQL | PostgreSQL Glob… | # DOC18 — Software Bill of Materials *Component, dependency, and license baseline* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Core Backend and Voice Stack | Component | Origin | License | Container | | --- | --- | --- | --- | | FastAPI | OSS (International) | MIT | gemstone-api | | Whisper large-v3 (weights) | OpenAI (US) | MIT | atlas-ollama | | Ollama runtime | Ollama Inc. (US) | MIT | atlas-ollama | | Piper TTS runtime | Rhasspy (US/International, open-source) | MIT | gemstone-tts | | ONNX Runtime | Microsoft (US) | MIT | gemstone-tts | | webrtcvad | Google (US) | BSD-3-Clause | gemstone-api | | PostgreSQL | PostgreSQL Global Dev Group | PostgreSQL License | gemstone-db | | Redis | Redis Ltd. | RSALv2/SSPL (dual) | gemstone-redis | ## 2. Mobile Stack, Capacitor | Package | Origin | License | Purpose | | --- | --- | --- | --- | | Capacitor core | Ionic (US) | MIT | Native WebView wrapper | | capacitor/network | Ionic (US) | MIT | Offline detection | | capacitor-community/sqlite | Community (International) | MIT | Local snapshot storage | | capacitor/haptics | Ionic (US) | MIT | Tactile confirmation | | capacitor/push-notifications | Ionic (US), FCM | MIT | Session-ready alerts | | Custom native audio bridge | NEXTECH-built | Proprietary | Background audio, both platforms | ## 3. Orchestration Stack | Component | Origin | License | Purpose | | --- | --- | --- | --- | | Didymus Hub (FastAPI + FastMCP) | NEXTECH-built on OSS | Proprietary/MIT mix | Task queue, worker registry | | SQLModel | OSS (International) | MIT | ORM/schema for task and session records | ## 4. Maintenance Syft-generated SBOM regenerated on every dependency change and tracked alongside the Project Ledger; cross-checked against DOC17 at Phase 5. ### Phase 5 status (2026-07-21) | Item | Status | | --- | --- | | Generator script | `scripts/security/sbom-stub.sh` (writes `DevResources/security/sbom-latest.json`) | | Syft on CI/dev PATH | Best-effort — writes skip stub if Syft absent | | DOC17 principal cross-check | Pending (engineering baseline above is not sign-off) | --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 5d3058a2442ce465fec855cd00196ac569a3502b |
| 37 | DOC19 | DOC19 — Project Ledger and Intelligence Layer | doc_suite | DevResources/docs/DOC19-Project-Ledger-Intelligence-Layer.md | The GemStone Project Ledger is a standalone ledger-api instance tracking session history, deploy events, UAT results, Domain Pack versions, and Didymus task/approval history, bidirectionally synced with Development History (DOC12). | # DOC19 — Project Ledger and Intelligence Layer *Schema and Atlas distillation feed* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Purpose The GemStone Project Ledger is a standalone ledger-api instance tracking session history, deploy events, UAT results, Domain Pack versions, and Didymus task/approval history, bidirectionally synced with Development History (DOC12). ## 2. Ledger Schema ``` class LedgerEntry(SQLModel, table=True): __tablename__ = "ledger_entries" id: UUID = Field(default_factory=uuid4, primary_key=True) entry_type: str # "deploy" | "uat_result" | "pack_version" | "didymus_task" | "research_query" payload: dict = Field(sa_column=Column(JSON)) created_at: datetime confidence_score: Optional[float] = None # set when this entry is distilled promoted_to_atlas: bool = Field(default=False) ``` ## 3. Intelligence Layer Feed This ledger feeds the same distillation discipline used platform-wide: decisions, rationale, and outcomes are captured and promoted into Atlas Vault; mechanical process detail is not. Every external Research Agent query audit token is recorded here. ## 4. Stage 2 ledger bootstrap (pre–SQLite service) Until Stage 3 stands up the ledger-api + SQLite file on `/mnt/docker-data`, the project ledger is maintained as markdown + this DOC: | Entry type | Stage 2 record | | --- | --- | | `deploy` | Pipeline documented in `DOC-Deploy-Pipeline.md`; host worktree at `/home/dr/AppDev/gemstone-nexus` (origin HEAD at first clone: initial commit only — product tree still local/untracked pending Stage 3 commit) | | `ops` | EC2 `i-04333dae160ab0daa` / `18.116.188.14`; EventBridge start/stop ENABLED; host key rotated 2026-07-21 | | `decision` | No local Docker; Deploy = commit→push→pull→compose; PAT never stored in EC2 remote URL | Atomic concurrency rule (from day one): any change that touches schema, API, Swagger, UI, or tests updates DOC12 + DOC07 + a ledger entry in the same Deploy. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 133f9c27ad0138110cfd1ae260cd5f38ec791c75 |
| 38 | DOC20 | DOC20 — UAT Framework | doc_suite | DevResources/docs/DOC20-UAT-Framework.md | 1. Start a Standard-tier session and begin a voice conversation. 2. Background the app for 2 minutes (within Extended Session grace window). 3. Reopen the app. Expected: session restores to ACTIVE with the last exchange visible, no re-authentication required. 4. Repeat steps 1-3 substituting an incoming phone call for backgrounding. Expected: identical restoration behavior. 5. Repeat steps 1-3 substituting a forced network disconnect. Expected: identical restoration behavior. | # DOC20 — UAT Framework *Structured acceptance scripts* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Script: Core Session Resilience 1. Start a Standard-tier session and begin a voice conversation. 2. Background the app for 2 minutes (within Extended Session grace window). 3. Reopen the app. Expected: session restores to ACTIVE with the last exchange visible, no re-authentication required. 4. Repeat steps 1-3 substituting an incoming phone call for backgrounding. Expected: identical restoration behavior. 5. Repeat steps 1-3 substituting a forced network disconnect. Expected: identical restoration behavior. ## 2. Script: Domain Pack Install (per pack, run six times) 1. Install the target Domain Pack for a test practitioner account. 2. Start a new session immediately. Expected: system prompt extension and terminology active without app restart. 3. Ask a domain-specific question matching the pack's specialization. Expected: response uses pack terminology fluently. 4. Uninstall the pack. Expected: next new session no longer uses pack terminology; all prior session history and Atlas Vault entries remain unchanged. ## 3. Script: Notebook and Recents Dual Visibility 1. Create a session and note its position in Recents. 2. Create a Notebook and link the session to it. 3. Reload Recents. Expected: the session still appears in its chronological position, now with a Notebook badge. 4. Confirm the session is also visible when browsing the Notebook directly. ## 4. Script: Didymus Universal Approval Queue 1. Submit one task each from GemStone, Cursor, Claude, and Gemini (simulated or real, per test environment). 2. Open the approval queue. Expected: all four appear in one unified list. 3. Without checking metadata, attempt to identify each task's source from presentation alone. Expected: not reliably distinguishable. 4. Approve one task. Expected: approved_by and approved_at populate; task becomes lease-eligible; the other three remain PENDING. ## 5. Script: Mobility Layer Field and Extended Modes 1. In direct outdoor sunlight, activate Field Mode and confirm the high-contrast UI is legible. 2. Tap the center-screen target and confirm the ascending chime and recording start within 1 second. 3. Start an Extended Session, wait through one mic-open cue interval, and confirm the cue is audible. 4. Trigger an incoming call during the Extended Session. Expected: session pauses cleanly and resumes on one tap after the call ends, with prior context intact. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 5c701e9502ed44b4b349782533563547f70b9e11 |
| 39 | DOC21 | DOC21 — Platform Dashboard | doc_suite | DevResources/docs/DOC21-Platform-Dashboard.md | | View | Metric | Source / Calculation | | --- | --- | --- | | Approval Queue | Pending task count by age bucket | COUNT from didymus_tasks WHERE status=PENDING, grouped by created_at age | | Worker Health | Lease success rate per worker | COMPLETED+VERIFIED / total leases, per target_worker, trailing 24h | | Tier Distribution | Active sessions by tier | COUNT from sessions WHERE ended_at IS NULL, grouped by tier | | Open Mode Consent Audit | Consent events vs. Open Mode ses… | # DOC21 — Platform Dashboard *Required views and metric definitions* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Required Dashboard Views | View | Metric | Source / Calculation | | --- | --- | --- | | Approval Queue | Pending task count by age bucket | COUNT from didymus_tasks WHERE status=PENDING, grouped by created_at age | | Worker Health | Lease success rate per worker | COMPLETED+VERIFIED / total leases, per target_worker, trailing 24h | | Tier Distribution | Active sessions by tier | COUNT from sessions WHERE ended_at IS NULL, grouped by tier | | Open Mode Consent Audit | Consent events vs. Open Mode session count | Should always be 1:1; any mismatch is a defect | | Research Agent Volume | External queries per day, by provider | COUNT from ledger_entries WHERE entry_type=research_query, grouped by provider | | Audio Pipeline Health | Chunked-hold failures awaiting resolution | COUNT WHERE vectorization_status=FAILED AND resolved_at IS NULL | | Wake-Word Tuning | False-trigger rate, trailing 7 days | False activations / total activations, once field data exists | | Domain Pack Adoption | Install/active count per pack | COUNT from pack_installations WHERE active=true, grouped by pack_id | --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | cea1c35f27e5f25b2f4ec7e46d0d635c9551bbdd |
| 40 | DOC26 | DOC26 — Bootstrap and Self-Hosting | doc_suite | DevResources/docs/DOC26-Bootstrap-Self-Hosting.md | This document defines the maturity gates that move GemStone from Bootstrap (human-directed build with external AI workers proposing tasks) to Self-Hosting (GemStone may propose build tasks against itself) without ever relaxing the human-approval invariant. It is the executable reference named by DOCA1 and DOC06; phase product gates remain in DOC02. | # DOC26 — Bootstrap and Self-Hosting *Phase-gate criteria for GemStone's Bootstrap-to-Self-Hosting lifecycle* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 21, 2026 ## 1. Purpose This document defines the maturity gates that move GemStone from Bootstrap (human-directed build with external AI workers proposing tasks) to Self-Hosting (GemStone may propose build tasks against itself) without ever relaxing the human-approval invariant. It is the executable reference named by DOCA1 and DOC06; phase product gates remain in DOC02. ## 2. Standing Invariant No Didymus task, from any source including GemStone itself, may enter RUNNING without human `approved_by` and `approved_at`. This invariant is permanent across Bootstrap, Self-Hosting, and any later maturity level. Schema enforcement is required (DOC03 Section 6); application-only checks are insufficient. ## 3. Phase Overview | Phase | Name | GemStone role | Exit intent | | --- | --- | --- | --- | | Gate 0 | Foundation | Consumer of stack only | DOC02 Phase 0 closed | | Gate 1 | Bootstrap | Approval surface for Cursor/Claude/Gemini workers | Phase 3C closed; three MVP workers stable | | Gate 2 | Supervised Self-Hosting | GemStone may submit tasks with `requested_by="GemStone"` | N consecutive supervised cycles pass | | Gate 3 | Operational Self-Hosting | Routine self-proposals allowed under the same approval UI | Blind-source review still required | ## 4. Gate 0 — Foundation Entry: Stage 0 docs locked; T4 GoldenAMI reachable; Auth0 tenant provisioned. Exit: - DOC02 Phase 0 exit criteria all pass. - Postgres, Redis (AOF), SQLite ledger, Ollama, Whisper, Piper, nginx, and MCP client health all green via `/healthz`. - Capacitor project scaffolds and `cap sync` succeeds for Android and iOS (even if store binaries are not yet submitted). ## 5. Gate 1 — Bootstrap Entry: Gate 0 closed; Didymus Hub reachable per DIDYMUS-ARCH-01. Exit: - DOC02 Phase 3C exit criteria all pass. - Workers Cursor, Claude, and Gemini are registered, credentialed per DIDYMUS-PLAYBOOK, and lease only APPROVED tasks. - Blind-source review: reviewers cannot distinguish task origin from UI chrome alone. - `GATE2_SUPERVISED_CYCLE_N` is configured from an admin/dev session (not hardcoded); default recommended starting value is 5. ## 6. Gate 2 — Supervised Self-Hosting Entry: Gate 1 closed; principal explicitly enables GemStone as a proposer. Exit: - At least `GATE2_SUPERVISED_CYCLE_N` consecutive cycles where GemStone submits a task, a human approves it, a worker executes it, and verification succeeds. - Failed or rejected cycles reset the consecutive counter. - DOC14 tests for "cannot reach RUNNING without approval" continue to pass with `requested_by="GemStone"`. ## 7. Gate 3 — Operational Self-Hosting Entry: Gate 2 closed; principal sign-off recorded in the Project Ledger. Exit: - Self-proposed tasks appear in the same universal queue with identical approval UX. - No separate "GemStone auto-apply" path exists in API, UI, or schema. - DOC20 Universal Approval Queue UAT script passes with mixed human and GemStone sources. ## 8. Experimental Forks Any experiment that appears to relax the approval invariant must live in a named experimental fork with no path to `main` until the invariant is proven intact. Forks are not phase gates. ## 9. Relationship to DOC02 DOC26 gates are maturity overlays. Closing DOC02 Phase 3C is necessary but not sufficient for Gate 2. Product phases 4–5 may proceed in parallel with Gate 2/3 once Gate 1 is closed, provided they do not introduce autonomous execution. ## 10. Term Definitions | Term | Definition | | --- | --- | | Bootstrap Phase | Gates 0–1; GemStone does not propose self-build tasks | | Self-Hosting Phase | Gates 2–3; GemStone may propose, never auto-execute | | Phase Gate | Numeric/binary checkpoint verified by someone other than the implementer | | Standing Invariant | Human approval before RUNNING, permanent | | Experimental Fork | Isolated branch for risky experiments; no merge until invariant proven | --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | a3955d7b40a677ef85ba7b5f1cc6e02b22095e4d |
| 41 | DOCA1 | DOCA1 — Amendment 1 | amendment | DevResources/docs/DOCA1-Amendment-1-Didymus-Self-Hosting.md | This amendment folds the Didymus orchestration architecture and the Bootstrap-to-Self-Hosting lifecycle into the core GemStone suite by reference. GemStone is the single, universal presentation and approval surface for every build task suggestion, regardless of originating AI tool. No task, from any source or category, ever executes without human approval; this is a permanent, platform-wide invariant, not a maturity-gated exception. | # DOCA1 — Amendment 1 *Didymus Integration and Self-Hosting Transition* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Summary This amendment folds the Didymus orchestration architecture and the Bootstrap-to-Self-Hosting lifecycle into the core GemStone suite by reference. GemStone is the single, universal presentation and approval surface for every build task suggestion, regardless of originating AI tool. No task, from any source or category, ever executes without human approval; this is a permanent, platform-wide invariant, not a maturity-gated exception. ## 2. Reconciliation Table | Original Document | What Changed | Superseding Section | | --- | --- | --- | | DOC02 SOW | Didymus Integration added as Phase 3C with its own gate criteria | DOC02 Section 6.8 | | DOC03 PRD | Didymus task schema, state machine, and RBAC rows added | DOC03 Sections 6, 7, 12 | | DOC06 Master Prompt | Orchestration rules requiring GemStone as universal surface | DOC06 Section 3 | | DOC13 Diagrams | Task approval flow sequence diagram added | DOC13 Section 2 | | DOC20 UAT Framework | Universal Approval Queue script added | DOC20 Section 4 | ## 3. Full Technical Reference See DIDYMUS-ARCH-01 for the complete hybrid architecture, worker contract, storage model, and MCP tool surface. See DOC26-Bootstrap-Self-Hosting.md for the full Bootstrap and Self-Hosting phase-gate criteria and the standing human-in-the-loop invariant. Practitioner-facing worker install and credential steps are in DIDYMUS-PLAYBOOK.md. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 05e52998c7ebe27505b65befdd6e9cea4add50fe |
| 42 | DOCA2 | DOCA2 — Amendment 2 | amendment | DevResources/docs/DOCA2-Amendment-2-Mobility-Layer.md | This amendment folds the Capacitor-based GemStone Mobility Layer into the core GemStone suite by reference, replacing the original React Native/Expo/PWA mobile stack in full. | # DOCA2 — Amendment 2 *GemStone Mobility Layer* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 17, 2026 ## 1. Summary This amendment folds the Capacitor-based GemStone Mobility Layer into the core GemStone suite by reference, replacing the original React Native/Expo/PWA mobile stack in full. ## 2. Reconciliation Table | Original Document | What Changed | Superseding Section | | --- | --- | --- | | DOC02 SOW | React Native/Expo mobile deliverable replaced with Capacitor Field Mode (2B) and Extended Sessions (2C), each with its own gate criteria | DOC02 Sections 6.5, 6.6 | | DOC03 PRD | Mobility Layer requirements section rewritten for Capacitor architecture | DOC03 Section 13 | | DOC06 Master Prompt | Mobility Layer build rules requiring Capacitor, prohibiting React Native/Expo scaffolding | DOC06 Section 4 | | DOC13 Diagrams | Session interruption and resume diagram added | DOC13 Section 4 | | DOC18 SBOM | React Native/Expo dependencies removed, Capacitor plugin table added | DOC18 Section 2 | | DOC20 UAT Framework | Mobility Layer Field and Extended Modes script added | DOC20 Section 5 | ## 3. Full Technical Reference See GS-MOB-01.md for the complete Field Mode and Extended Session specification, acoustic feedback framework, audio retention policy, and native audio bridge requirements for both iOS and Android. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 3d62ffe5fdd1fab992a4e078e4ac96792d2baa64 |
| 43 | GS-MOB-01 | GS-MOB-01 — GemStone Mobility Layer | mobility | DevResources/docs/GS-MOB-01.md | This document is the executable Mobility Layer specification named by DOCA2, DOC02 Phases 2B/2C, DOC03 Section 13, and DOC06 Section 7. It replaces the superseded React Native/Expo/PWA mobile stack with Capacitor as the sole mobile client. | # GS-MOB-01 — GemStone Mobility Layer *Field Mode, Extended Sessions, native audio bridge, and wake-word* **Initiative:** GemStone · **Version:** v2.0 (Rewrite) · **Status:** Approved for Build · **Date:** July 21, 2026 ## 1. Purpose This document is the executable Mobility Layer specification named by DOCA2, DOC02 Phases 2B/2C, DOC03 Section 13, and DOC06 Section 7. It replaces the superseded React Native/Expo/PWA mobile stack with Capacitor as the sole mobile client. ## 2. Client Stack - React + Vite web app wrapped by Capacitor (`@capacitor/*` current major aligned with the prior StackOne Capacitor seed). - Strict DataRightTailwind for all UI. - Capacitor is first-class: web, Android, and iOS ship via atomic concurrency; a Capacitor fail is a feature fail. - Auth0 Universal Login with Google, Facebook, and LinkedIn social connections; roles map to DOC03 Section 7. ## 3. Modes | Mode | Intent | Audio path | | --- | --- | --- | | Field Mode | Foreground tap-to-talk for brief field interactions | Capacitor WebView: `getUserMedia` / `MediaRecorder` | | Extended Session | Hands-free / longer sessions (Didymus, research, planning) | Native background audio bridge | Do not merge these into a single audio implementation. ## 4. Field Mode (Phase 2B) Requirements: - Single large center-screen tap target; voice capture starts within 1 second of tap (DOC02 / DOC04 US-11). - Ascending dual-tone chime confirms capture start. - Heads-Up UI must remain legible in direct sunlight (manual outdoors check). - Recents list restores at least 20 prior sessions via scroll-and-select. - No native microphone plugin required for Field Mode. Gate criteria: DOC02 Section 6.5. ## 5. Extended Sessions (Phase 2C) Requirements: - Native background audio entitlements: - iOS: `AVAudioSession` with `UIBackgroundModes: audio` - Android: foreground service with `FOREGROUND_SERVICE_TYPE_MICROPHONE` and persistent notification - Periodic mic-open cue at configured interval. - Approaching extended timeout requires explicit confirmation before mic close; confirmed timeout transitions session to ENDED (not RESUMABLE). - Android is priority platform; iOS follows once Android is proven. Gate criteria: DOC02 Section 6.6. ## 6. Interruption and Resume Canonical interruption causes (aligned with DOC03 Section 5 and DOC13 Section 4): 1. OS audio focus loss (including incoming call) 2. Network drop 3. App backgrounding Path: ``` ACTIVE -> PAUSED (interruption_snapshot written) -> RESUMABLE (session-ready chime) -> ACTIVE (one tap or app reopen; restore via Atlas Vault) ``` Resume behavior must be identical across the three causes (DOC04 US-02). Snapshot must be non-null before PAUSED is durable (DOC08 Section 1). ## 7. Wake-Word - Configurable multi-phrase list via `WAKE_PHRASES` (DOC06). - Android: on-device model in a foreground service; activation from fully killed app state in at least 8 of 10 quiet-room trials. - iOS: Siri Shortcuts path; launch and listen in at least 9 of 10 trials. - Android default-assistant (`VoiceInteractionService`) is out of scope for Phase 1 (DOC02 Section 3). ## 8. Acoustic Feedback Framework | Event | Feedback | | --- | --- | | Field Mode capture start | Ascending dual-tone | | Extended Session mic still open | Periodic unobtrusive cue | | Session-ready (RESUMABLE) | Session-ready chime | | Transcription / vectorization failure | In-app notification + system log within 5 minutes | ## 9. Audio Retention Policy - Capture begins the moment the mic opens; no ambiguous pre-buffer state. - Raw audio is discarded immediately after transcription, indexing, and vectorization into Atlas Vault. - On pipeline failure, hold audio in chunks; dual-flag in-app and system log; do not discard until vectorization succeeds (DOC03 Section 14, DOC08 Section 4). - No session transcripts or audio to external APIs; AES-256 at rest, TLS 1.3 in transit. ## 10. Native Audio Bridge Architecture ``` Capacitor JS layer |-- Field Mode --> WebView media APIs |-- Extended Session --> Capacitor plugin bridge |-- iOS: AVAudioSession + background audio mode |-- Android: mic foreground service + notification |-- Wake-word --> Android FG service model / iOS Siri Shortcut |-- Session control --> WebSocket /api session resume endpoints ``` Plugin surface must expose: start/stop capture, interruption callbacks (focus loss, network, background), cue scheduling, and wake-word enable/disable. Implementation detail may evolve; the mode split and interruption contract may not. ## 11. Store and Entitlement Notes - Follow Apple background-audio guidelines; avoid patterns resembling always-on ambient listening. - Play/App Store submission packages are Phase 5 deliverables (DOC02 Section 6.10). - WCAG AA for the mobile UI is a Phase 4 gate. ## 12. Platform status (Phase 2B/2C build) | Surface | Status | | --- | --- | | Field Mode (web + Capacitor WebView) | Implemented — `/app/field`, dual-tone, Recents ≥20 API | | Extended Session Android FG mic service | Implemented scaffold — `ExtendedAudioService` + plugin | | Extended Session iOS AVAudioSession | Scaffold — `Info.plist` background audio + Swift plugin (needs macOS/Xcode) | | Wake-word Android FG service | Scaffold armed (notification + service); on-device model + 8/10 device trials pending | | Wake-word iOS Siri Shortcuts | Scaffold only — App Intent wiring needs macOS; 9/10 device trials pending | | Linux / Windows | Field Mode only; Extended/wake use web fallbacks (documented, not device gates) | | HorizonORBIT login | Out of scope for mobility — Auth0 stub retained | Device QA gates in DOC02 §6.5 / §6.6 (sunlight check, call interruption 10-trial, wake-word trials) remain practitioner/QA sign-off on hardware. --- NEXTECH Business Technology Management LLC · nex-tech.biz · (941) 499-5505 | 6bacfd2c410d72785a1de99eedb66e4b55ada5a3 |
| 44 | AWS-RUNBOOK | GemStone-Nexus — AWS Deploy Runbook | reference | DevResources/docs/aws-runbook.md | > **Status (Stage 2):** Ohio T4 host is live (`i-04333dae160ab0daa`, us-east-2). > Day-to-day Deploy is documented in `DOC-Deploy-Pipeline.md` and driven by > `scripts/run-remote-docker-deploy.ps1` / `scripts/aws/deploy.sh`. > The us-east-1 “Phase 1 baseline” section below is historical plan context; > do not re-run those skeletons against Ohio without review. | # GemStone-Nexus — AWS Deploy Runbook > **Status (Stage 2):** Ohio T4 host is live (`i-04333dae160ab0daa`, us-east-2). > Day-to-day Deploy is documented in `DOC-Deploy-Pipeline.md` and driven by > `scripts/run-remote-docker-deploy.ps1` / `scripts/aws/deploy.sh`. > The us-east-1 “Phase 1 baseline” section below is historical plan context; > do not re-run those skeletons against Ohio without review. Source of truth for the Ohio stack: `DevResources/plans/ohio-terraform-plan.md` and `infra/terraform/ohio/`. ## Target architecture ``` GoDaddy DNS (gemstone.dataright.net) │ A record -> Elastic IP (Ohio live: 18.116.188.14) ▼ [ g4dn.xlarge / T4 | us-east-2 live host ] nginx (TLS, Let's Encrypt HTTP-01) -> api (FastAPI) ├─> ollama (GPU, T4 16GB) ├─> whisper (STT) ├─> piper (TTS) └─> ledger.db (standalone SQLite) nvme0n1 30G = OS only nvme1n1 128G = /mnt/docker-data (docker data-root + Ollama models + ledger + staging) │ ▼ S3 gateway endpoint (vpce-0661ccc0d47e88054) — ledger backups + staged files ``` The stack is self-contained in `DataRight-vpc`. There is **no runtime dependency** on an unrelated on-prem host (`10.0.0.228`). ## Fixed references (from the plan) | Item | Value | |------|-------| | Region | `us-east-1` | | VPC | `DataRight-vpc` = `vpc-053baa45dd75c045a` (`10.0.0.0/16`) | | Public subnet (primary) | `us-east-1a` = `subnet-019ffac204b77426b` (`10.0.0.0/20`) | | Public subnet (fallback) | `us-east-1b` = `subnet-0a6e88b619d84a175` | | IGW | `igw-0a828410ab4af913c` | | S3 gateway endpoint | `vpce-0661ccc0d47e88054` | | Golden launch template | `DataRightGoldenAMITemplate` = `lt-0e1491fcfe8f99a13` | | Golden AMI base to bake from | `ami-0f8a61b66d1accaee` | | New GPU launch template | `DataRightGemStoneGPUSpot` (derived; golden stays pristine) | | Instance type | `g4dn.xlarge` (NVIDIA T4 16GB) — spot | | Persistent data EBS | 128G `/dev/sdb`, `DeleteOnTermination=false` | | Hostname | `gemstone.dataright.net` (Let's Encrypt, GoDaddy DNS) | | Deploy path | `/home/dr/AppDev/gemstone-nexus` (`DEVELOPMENT_DIRECTORY`) | | Primary region (live) | `us-east-2` (Ohio) — instance `i-04333dae160ab0daa` | ## Phase 1 — AWS baseline (do this first; not done yet) 1. **Bake `golden-v4-gpu`** from `ami-0f8a61b66d1accaee`: run the v4 Ansible playbook, then add the NVIDIA driver + `nvidia-container-toolkit`. Preserve the two-disk layout and baked images. Register the new AMI. 2. **New security group** in `DataRight-vpc`: inbound `443` (world), `80` (world — Let's Encrypt HTTP-01 + redirect), `22` (your IP only); outbound all. The golden template's `DataRightDev` SG lives in the default VPC and **cannot** be reused (SGs are VPC-scoped). 3. **Create launch template `DataRightGemStoneGPUSpot`** derived from the golden template, setting: GPU AMI, `InstanceType=g4dn.xlarge`, spot market options, subnet + `AssociatePublicIpAddress=true`, key pair, IAM instance profile (S3 + Secrets Manager), and the 128G `/dev/sdb` with `DeleteOnTermination=false`. 4. **Launch** the spot instance (`scripts/aws/bring-up.sh`). 5. **Mount** the persistent 128G volume at `/mnt/docker-data` (`scripts/aws/mount-ebs.sh`). Format only on first provision. 6. **Associate the Elastic IP** and set the GoDaddy A record (`scripts/aws/associate-eip.sh`). 7. **Verify** GPU is visible to Docker (`nvidia-smi` inside a container) and that a bare `/health` responds over HTTPS. ## Phase 2/3 — Deploy the stack (this repo) Run on the host after Phase 1 (`scripts/aws/deploy.sh`): 1. Clone/sync this repo into `/home/dr/AppDev/gemstone-nexus`. 2. Create `.env` from `.env.example`. On AWS set `APP_SECRETS_ARN` + `SKIP_AWS_SECRETS=0` so the API loads Gemini keys from Secrets Manager via its IAM instance profile. Set `TLS_SAN_DNS=gemstone.dataright.net`. 3. Provision Let's Encrypt: `sudo TLS_EMAIL=... ./scripts/aws/provision-letsencrypt.sh --recreate-nginx` (HTTP-01 webroot; recreates **only** the nginx container). Without LE files, nginx falls back to a self-signed cert in `/etc/gemstone/certs`. 4. `docker compose up -d --build`. The one-shot `ollama-init` service pulls `OLLAMA_CHAT_MODEL` onto the GPU on first boot. 5. Verify: `curl -I https://gemstone.dataright.net/healthz` (no `-k`) and `python scripts/verify_gemstone_dev.py https://gemstone.dataright.net`. ## Spot + persistence - The 128G EBS is **persistent** and re-attached on relaunch, so Docker data-root + Ollama models + `ledger.db` survive spot interruption (no multi-GB re-pull). - **Ledger durability:** snapshot `ledger.db` into `/mnt/docker-data/staging` nightly and sync to S3 via the gateway endpoint. `scripts/aws/snapshot.sh` captures a full EBS snapshot for point-in-time backups. - **Elastic IP** re-associated on relaunch; stable hostname means the Let's Encrypt cert survives relaunches. - A 2-minute spot interruption notice handler should flush/upload the ledger before reclaim (to be wired in Phase 4). ## Cost (us-east-1 estimate) - Dev-hours (~176/mo, auto-stop): ~$35 compute + ~$13 EBS + ~$3.65 EIP ≈ **$52/mo** - Always-on (730/mo): ≈ **$162/mo** ## Open items (not blocking scaffolding) 1. AWS credentials/profile for create actions. 2. GoDaddy DNS: manual A record vs. API automation. ## Temporary on-demand fallback (2026-07-22) Ohio Spot `i-04333dae160ab0daa` / `sir-1s4zg8in` hit `capacity-not-available`. Per ops directive, traffic was moved to a **temporary on-demand** host while Spot is watched for cutover. | Role | ID | Notes | |------|----|-------| | **Serving now (on-demand)** | `i-0f5f3b32dfa9c6a12` | `g4dn.xlarge` us-east-2; EIP `18.116.188.14` / `eipalloc-00556d8ca50728abd`; tag `DataRight-GemStone-gpu-TEMP-OD` | | Original Spot (terminated) | `i-04333dae160ab0daa` | Cancelled `sir-1s4zg8in`; **do not** expect StartInstances on this ID | | Persistent data EBS | `vol-0a1fff82ee46e9095` | `/dev/sdb` ? `/mnt/docker-data` (Docker root, models, ledger) | | Current root EBS | `vol-0591c165d1467bd01` | Recreated from `ami-011a95758542dcaac` (prior root was lost on Spot terminate before DeleteOnTermination=false applied) | Both volumes on the OD instance have `DeleteOnTermination=false`. ### Cutover back to Spot (manual / when capacity returns) 1. Confirm Spot capacity (`get-spot-placement-scores` score > 1, or a probe `request-spot-instances` fulfills). 2. `docker compose ps` healthy on OD; optionally snapshot root + data. 3. Stop OD `i-0f5f3b32dfa9c6a12`; detach root + data volumes (`DeleteOnTermination=false`). 4. Launch Spot (`g4dn.xlarge`, same subnet/SG/AMI or LT `lt-05dd1ba1dec324866`) in **us-east-2a**, stop it, replace root with `vol-0591c165d1467bd01`, attach data volume, start. 5. Associate EIP `eipalloc-00556d8ca50728abd` to the Spot instance; verify `curl -I https://gemstone.dataright.net/healthz` and `/` without `-k` (GET may be required if HEAD returns 405). 6. Terminate the temporary OD only after Spot is healthy and EIP moved. SSH on golden/OD recovery image: `ubuntu` (+ key). App user `dr` home exists; deploy path `/home/dr/AppDev/gemstone-nexus`. | 87f63904a35989babd95d47ed0ada56b3f2e58c9 |
| 45 | CONTRACT-NOTES | GemStone-Nexus — Contract Notes | reference | DevResources/docs/contract-notes.md | Behavioral contract for the ported stack. These notes capture what the API and voice loop promise, and what changed during the port from a prior StackOne extraction. | # GemStone-Nexus — Contract Notes Behavioral contract for the ported stack. These notes capture what the API and voice loop promise, and what changed during the port from a prior StackOne extraction. ## Provenance Sourced from a prior StackOne extraction (`backend/app/gemstone/**` and related `mcp/`, `api/v1/gemstone_ops.py`, `static/gemstone-voice.js`, `voice/`) into a standalone package under `api/app/`. The origin tree remains frozen and untouched. The new stack has **no coupling** back to that origin: no cross-repo imports, no shared ledger, no dependency on `10.0.0.228`. ## Inference modes | Mode | Chat | TTS | Grounding | |------|------|-----|-----------| | `sovereign` (default) | Ollama (local GPU) | Piper (local) | Ledger-grounded only | | `development` | Gemini (external) | Gemini TTS | Ledger + general knowledge | - Canonical ops switch: `PUT /api/v1/gemstone/ops/inference-mode` (body `{ "mode": "sovereign" | "development" }`), implemented in `api/app/gemstone/ops.py`. Mode is persisted in the ledger `GemStoneConfig` table so it survives restarts. - Convenience aliases also exist on the MCP gemstone router (`GET|POST /mcp/gemstone/mode`). Prefer `PUT /api/v1/gemstone/ops/inference-mode` for ops automation and health-coupled mode changes. - Default chat model is `llama3.1:8b` (configurable via `OLLAMA_CHAT_MODEL`). This replaces the prior extraction's `tinyllama`, which was a RAM/speed workaround on CPU; the T4 GPU removes that constraint. ## HTTP surface | Route | Purpose | |-------|---------| | `GET /` | Chat + voice UI (inline HTML) | | `GET /health` | Liveness for compose/nginx healthcheck | | `GET /mcp/health` | Ledger + subsystem health | | `GET /mcp/state` | Current phase/signals from the ledger | | `POST /mcp/chat` | Non-streaming chat | | `POST /mcp/chat/stream` | SSE streaming chat | | `POST /mcp/write` | Ledger write (allow-listed tables) | | `PUT /api/v1/gemstone/ops/inference-mode` | **Canonical** switch inference mode (persisted) | | `GET /api/v1/gemstone/ops/status` | Ops status including inference mode + voice health | | `GET|POST /mcp/gemstone/mode` | Alias: get/set inference mode | | `POST /mcp/gemstone/voice/synthesize` | TTS (Piper or Gemini per mode) | | `GET /mcp/atlas/health` | Phase 2A Atlas Index B health (stub or remote) | | `POST /mcp/atlas/query` | Atlas context retrieval (MCP Index B client) | | `POST /mcp/atlas/signal` | Fine-tuning signal write to Atlas Vault | | `POST /mcp/atlas/recall` | Named/dated session recall | | `POST /mcp/atlas/index` | Upsert session summary for recall | | `POST /mcp/gemstone/correction` | Practitioner correction → Atlas signal | > **Auth note:** The prior StackOne ops endpoints were auth-gated. For the > standalone stack, auth was removed at the app layer — access control is > expected at the network edge (nginx / security group). Re-add app auth before > exposing beyond the intended audience. ## Atlas Vault (Phase 2A) - GemStone is an **MCP client** to a remote Atlas Vault (MCP Index B). - Config: `ATLAS_MCP_ENDPOINT`, `ATLAS_MCP_API_KEY`, `ATLAS_MCP_MODE`. - When the endpoint is unset, an in-process **stub vault** satisfies gates; see `DevResources/docs/DOC-Atlas-Index-B.md` for the remote contract and how to point at a real vault. No Atlas credentials are present in `DATARIGHT_DEV_SECRETS` at Phase 2A close. ## Ledger (standalone) - Fresh StackOne-pattern SQLite DB, schema in `api/app/ledger/schema.sql`, initialized idempotently on startup (`init_db`). - Default path `LEDGER_DB`: local dev `./data/ledger.db`; container `/app/data/ledger.db`; AWS `/mnt/docker-data/gemstone/ledger.db` (persistent EBS). All overridable via env. - `gather_context` (RAG) reads `ContextIndex`, `RulesEngine`, `Glossary`, `DecisionLog`, `Abstract`; defensive against empty/missing tables on a fresh DB. - Embeddings run on local Ollama (`nomic-embed-text`); a deterministic **hash-stub fallback** is retained for when Ollama is unavailable. ## Voice loop Two distinct conversational engines, chosen by inference mode: ### Sovereign (wake-word turn loop) - Client: `web/gemstone-voice.js` — Web Speech API for wake-phrase + capture, falls back to Whisper STT. Wake/end phrases from `GEMSTONE_*` env. - Server: Whisper STT container + Piper TTS container. - Turn-based: "Hey Gemstone" → capture → transcribe → answer → speak → idle. Wake-word gating is intentional here (hands-free field mode). - Whisper hardening: uploads under `WHISPER_MIN_AUDIO_BYTES` (default 2000) and ffmpeg-undecodable clips (short taps / silence / false VAD) return `{"text":"","no_speech":true}` with HTTP 200 — not a 502. Only a genuine Whisper outage (connection/timeout) returns 503 (`WhisperUnavailable`). The STT model sent to faster-whisper-server is now `WHISPER_MODEL` (large-v3), not the `whisper-1` alias. - **Latency (2026-07 measured on live):** Whisper CUDA cold start was ~20–24s STT; warm STT ~0.7s. Gemini TTS default was ~1.8s/chunk; Piper `en_US-lessac-medium` stream first-chunk ~0.18s. Turn-based path now defaults to Piper fast voice, keeps Whisper warm via API keepalive (`WHISPER_WARM_INTERVAL_SEC`), uses 450ms utterance silence (was 800), and prefetches sentence TTS while playing. Gemini Live continuous path is unchanged. ### Development / External (Gemini Live full-duplex — PRIMARY) - Client: `web/gemstone-live.js` (`GemStoneLive`) — continuous mic capture (PCM16 16 kHz) and streamed assistant playback (PCM16 24 kHz). No wake word: enabling Conversation in Dev/External starts a continuous session that stays ACTIVE; only a generous idle timeout (15 min) ends it. This replaces the premature "Session ended — say Hey Gemstone" behavior for desktop conversation. - Server bridge: `api/app/ws/gemstone_live.py` at `ws(s)://…/ws/gemstone/live` proxies the browser to Gemini Live (`BidiGenerateContent`). The Gemini API key never reaches the browser. Server-side VAD + barge-in: Gemini's `interrupted` signal is forwarded so the client flushes playback and keeps listening. - Models churn/retire fast. Default `GEMINI_LIVE_MODEL=gemini-3.1-flash-live-preview` with `GEMINI_LIVE_MODEL_FALLBACKS`; the bridge auto-tries candidates on connect (retired `gemini-live-2.5-flash-preview` / `gemini-2.0-flash-live-001` shut down 2025-12-09). Real-time Google Search grounding is added when `GEMSTONE_DEV_REALTIME_DATA` is on. - Requires `GEMINI_API_KEY` in the API container env (host `.env`, sourced from `DATARIGHT_DEV_SECRETS`). Live is advertised in `/mcp/gemstone/voice/config` (`live.available`) only when mode=development AND the key is present. - Streaming chat over SSE requires nginx to disable buffering on `/mcp/chat/stream` (configured in `nginx/nginx.conf`). WebSocket upgrade for `/ws/` paths must be proxied by nginx. ## Known-good settings preserved - `OLLAMA_READ_TIMEOUT=45` - `OLLAMA_MAX_CONTEXT_CHARS=6000` - Wake phrases, silence timeouts, Piper voice — see `.env.example`. - Gemini secrets pattern: `APP_SECRETS_ARN` (AWS) or `.env` (local dev). ## Intentionally not ported - The prior extraction's Atlas ledger, Postgres embeddings, and non-GemStone services (webapp, workers, MQTT, Redis) — replaced by the standalone SQLite ledger. - Cross-app navigation in the original inline HTML — removed for the standalone UI. | 4bf0574ce2faa915a344c5aaf4565d2d74dbe93d |
| 46 | OHIO-TERRAFORM-PLAN | GemStone-Nexus — Ohio (us-east-2) Terraform Bring-up Plan | plan | DevResources/plans/ohio-terraform-plan.md | Recreate the GemStone dev stack in **us-east-2 (Ohio)** — the home datacenter — as | # GemStone-Nexus — Ohio (us-east-2) Terraform Bring-up Plan Recreate the GemStone dev stack in **us-east-2 (Ohio)** — the home datacenter — as **Terraform-managed** infrastructure: a mirrored DataRight network, a duplicated GPU GoldenAMI, fresh Elastic IP(s), and a spot-first launch path, so we can resume work with spot cost savings. > Guardrail: per the cloud-infra approval rule, Terraform is authored here but **nothing is > applied** without explicit per-step approval. The design deliberately separates "stage > infra" (cheap, no GPU) from "launch GPU instance" (billable) so `apply` never starts a > GPU box on its own. ## Why Ohio - Home datacenter; g4dn.xlarge spot ~**$0.163/hr** (vs us-east-1 ~$0.25, on-demand ~$0.526) — cheapest of the regions checked. - us-east-1 g4dn spot capacity is currently depleted (placement score 1/10 across all AZs). ## What exists today (discovery) Source — **us-east-1** (to mirror): - VPC `vpc-053baa45dd75c045a` `DataRight-vpc` (`10.0.0.0/16`) - Public subnets: `10.0.0.0/20` (us-east-1a), `10.0.16.0/20` (us-east-1b); Private: `10.0.128.0/20`, `10.0.144.0/20` - IGW `igw-0a828410ab4af913c`; S3 gateway endpoint `vpce-0661ccc0d47e88054` - SG `gemstone-nexus-sg`: ingress 443 + 80 (0.0.0.0/0), 22 (`71.208.0.159/32`), egress all - GPU AMI `ami-03639d222ae9be0a2` `golden-v4-gpu` (30G `/dev/sda1` + 128G `/dev/sdb` gp3 + ephemeral), snapshots `snap-0b39665b0aa5f2e47`, `snap-0681ec8c786de0622` - EIP `50.19.136.7` (region-bound; stays in us-east-1) Target — **us-east-2** (Ohio): - Only the **default VPC** `vpc-5e9b5335` (`172.31.0.0/16`) — no DataRight network yet - **No** self-owned AMIs, **no** EIPs, no GemStone key pair - Spot g4dn.xlarge: us-east-2a $0.1663, us-east-2b $0.1629, us-east-2c $0.1749 ## Target architecture (Ohio) ```mermaid flowchart TB dns["GoDaddy DNS: GemStone.DataRight.net"] subgraph oh [us-east-2 DataRight-vpc 10.2.0.0/16] igw["Internet Gateway"] subgraph pub [public subnet us-east-2a] eip["New Elastic IP"] inst["g4dn.xlarge (spot-first) from golden-v4-gpu-oh"] data["128G /mnt/docker-data (persist)"] end s3ep["S3 gateway endpoint (free)"] igw --- pub eip --> inst inst --- data inst -. backups .-> s3ep end amiSrc["us-east-1 AMI ami-03639d222ae9be0a2"] -->|"aws_ami_copy (cross-region)"| amiOh["golden-v4-gpu-oh (Ohio)"] amiOh --> inst dns -->|"A -> new EIP"| eip ``` ## Terraform layout New directory in this repo (segregated, IaC-managed): ``` infra/terraform/ohio/ versions.tf # terraform >=1.6, aws provider ~>5.x providers.tf # provider aws (us-east-2) + aliased provider aws.use1 (source for AMI copy) variables.tf # ssh_cidr(default 0.0.0.0/0), vpc_cidr, az_primary, launch_instance(bool,false), use_spot(bool,true), instance_type, public_key_path, source_ami_id network.tf # VPC, 2 public + 2 private subnets, IGW, public RT, S3 gateway endpoint + assoc security.tf # aws_security_group gemstone-nexus-sg (443/80/22 all 0.0.0.0/0), aws_key_pair (imported public key) ami.tf # aws_ami_copy golden-v4-gpu-dr-oh from the corrected us-east-1 GoldenAMI compute.tf # aws_launch_template DataRightGemStoneGPUSpot + optional aws_instance + aws_eip + assoc (count = var.launch_instance ? 1 : 0) outputs.tf # vpc_id, subnet_ids, sg_id, ami_id, launch_template_id, eip_public_ip terraform.tfvars # concrete values (admin_cidr=71.208.0.159/32, etc.) ``` State: **local** for Phase A (`terraform.tfstate`, gitignored). Before Phase B, migrate to an **S3 backend with native lockfile** (`use_lockfile = true`, Terraform >= 1.10; no DynamoDB) via `terraform init -migrate-state`. State bucket: versioned, SSE-S3 encrypted, Block Public Access on. ### Key resource details (mirrors us-east-1 exactly unless noted) - **VPC** `DataRight-vpc` `10.2.0.0/16` in us-east-2 (distinct range chosen to keep future peering/VPN with us-east-1 and on-prem `10.0.0.x` clean). Subnets: public `10.2.0.0/20` (2a), `10.2.16.0/20` (2b); private `10.2.128.0/20` (2a), `10.2.144.0/20` (2b). IGW + public route table `0.0.0.0/0 -> igw`. S3 **gateway** endpoint (free, no NAT). - **Security group** `gemstone-nexus-sg`: 443/80/22 all from `0.0.0.0/0`, egress all. SSH is intentionally open (not IP-restricted) to allow mobile/remote management, matching the existing DataRight pattern; access is controlled by the `dr` credentials. - **Key pair** `gemstone-nexus-key`: import the **public** half of your existing PEM (`ssh-keygen -y -f C:\Users\Mark\.ssh\gemstone-nexus-key.pem`) so the same key works in both regions. This is a secondary/break-glass path (`ubuntu` + key); the primary access follows the DataRight pattern (`dr` + password via plink) — see below. - **AMI** via `aws_ami_copy` from the **corrected** `golden-v4-gpu-dr` (source region us-east-1) → `golden-v4-gpu-dr-oh`. This copies both snapshots into Ohio. GPU drivers + two-disk layout + Docker data-root=/mnt/docker-data + the DataRight host contract (below) are baked in, so a launched instance is immediately GPU-ready, plink-accessible as `dr`, with the data volume mounted. - **Launch template** `DataRightGemStoneGPUSpot`: instance_type `g4dn.xlarge`, AMI = copied Ohio AMI, SG + key + public subnet, block devices 30G `/dev/sda1` + 128G `/dev/sdb` with **DeleteOnTermination=false** (persist models/ledger). Spot options gated by `var.use_spot` (persistent, interruption=stop). On-demand is the documented fallback (flip `use_spot=false`). - **Instance + EIP** (optional, `count = var.launch_instance ? 1 : 0`): allocate a new EIP and associate to the launched instance. Default `launch_instance=false` so staging apply does not create a billable GPU box. ### DataRight host contract (dr user + plink access) — must be baked in Discovery on the current GPU AMI showed it is **stock Ubuntu + NVIDIA + two-disk only** — it is missing the DataRight golden host contract (no `dr` user, `PasswordAuthentication no`, no `/home/dr/AppDev`, no version marker). To follow our pattern we **re-bake a corrected GoldenAMI** (Phase 0) that includes: - **`dr` sudo user** (in `sudo` + `docker` groups), password set from **`DATARIGHT_DEV_SECRETS`** at bake time (not stored in user-data/IMDS). - **`PasswordAuthentication yes`** so plink (`DR_SSH_USER=dr`, `DR_SSH_PASSWORD`) works, matching StackOne/OmniFlow. SSH (22) is open to `0.0.0.0/0` per the standard pattern to permit mobile/remote management; access is gated by the `dr` credentials. - **`/home/dr/AppDev`** (+ `/home/dr/admin`) and the `/etc/dataright-golden-version` marker. - Prefer establishing this via the DataRight Bootstrap `provision-host.yml` roles (base/user/secrets/docker/firewall) during the bake for fidelity with the standard. `DATARIGHT_DEV_SECRETS` is **duplicated to us-east-2 now (not deferred)** and used to retrieve the `dr` password (and other credentials) during the build, exactly matching the pattern in the other repos/initiatives. Because the password is applied during the bake, the running instance does not need Secrets Manager access at boot (IAM instance profile stays deferred for the app runtime). The GemStone-Nexus repo also gains the standard deploy pattern (from the OmniFlow/StackOne convention): `DR_SSH_HOST/USER/PASSWORD/HOSTKEY`, `DEVELOPMENT_DIRECTORY=/home/dr/AppDev`, `DR_REPO_NAME` in `.env.example`, and a plink-based `scripts/run-remote-docker-deploy.ps1` targeting `/home/dr/AppDev/gemstone-nexus`. ### Representative HCL (the two non-obvious bits) ```hcl # ami.tf — cross-region duplicate of the GPU GoldenAMI resource "aws_ami_copy" "golden_gpu" { name = "golden-v4-gpu-dr-oh" source_ami_id = var.source_ami_id # corrected golden-v4-gpu-dr (Phase 0 output) source_ami_region = "us-east-1" tags = { Project = "GemStone-Nexus", Name = "golden-v4-gpu-dr-oh" } } # compute.tf — spot-first, on-demand via var.use_spot=false resource "aws_launch_template" "gpu" { name = "DataRightGemStoneGPUSpot" image_id = aws_ami_copy.golden_gpu.id instance_type = var.instance_type key_name = aws_key_pair.gemstone.key_name dynamic "instance_market_options" { for_each = var.use_spot ? [1] : [] content { market_type = "spot" spot_options { spot_instance_type = "persistent" instance_interruption_behavior = "stop" } } } # network interface (public subnet + SG + public IP), 30G/128G(persist) BDMs, tags... } ``` ## Phased execution (each phase = explicit approval) - **Phase 0 — Re-bake corrected GoldenAMI (us-east-1, separate approval):** provision the DataRight host contract onto a build instance (reuse the running us-east-1 instance `i-0276f2c824906c3e9`, ideally via Bootstrap `provision-host.yml`): create `dr` (sudo + docker, password from `DATARIGHT_DEV_SECRETS`), enable `PasswordAuthentication yes`, create `/home/dr/AppDev` + `/home/dr/admin`, write the golden marker. Create AMI **`golden-v4-gpu-dr`**. Also **replicate `DATARIGHT_DEV_SECRETS` to us-east-2**. Output: `source_ami_id` for Terraform. - **Phase A — Stage infra (cheap, no GPU):** `terraform apply` with `launch_instance=false`. Creates VPC/subnets/IGW/routes/S3 endpoint (free), SG + key (free), and the AMI copy of `golden-v4-gpu-dr` (small snapshot storage). Result: ready to launch, ~$0/hr compute. - **Phase B — Launch GPU (billable, separate approval):** set `launch_instance=true`, confirm `use_spot=true`, `apply`. If spot returns `InsufficientInstanceCapacity`, I STOP and report (no auto on-demand). You then decide: retry, or set `use_spot=false` for on-demand. - **Phase C — DNS + resume work:** you point `GemStone.DataRight.net` at the new Ohio EIP; deploy the GemStone app (later phase), Let's Encrypt, verify chat/voice/GPU. - **Phase D — decommission us-east-1 (confirmed):** once Ohio is live and verified, terminate the us-east-1 on-demand instance and delete the us-east-1 GemStone resources tagged `Project=GemStone-Nexus` (instance, launch template, `golden-v4-gpu` AMI + 2 snapshots, SG, key pair, and the `50.19.136.7` EIP if you don't want to keep paying for it). Executed only after Ohio passes verification. ## Costs (us-east-2) One-time: - Cross-region AMI/snapshot copy data transfer: ~$0.02/GB of actual snapshot data (near-empty volumes) ≈ **<$0.50 one-time**. Recurring — always present once staged: | Item | Cost | |---|---| | VPC, subnets, IGW, route tables, SG, key pair | Free | | S3 gateway endpoint | Free | | Ohio AMI snapshots (OS + near-empty 128G) | ~$0.30-0.50/mo | | New Elastic IP (charged whether or not associated) | ~$3.65/mo | | **Staged subtotal (no GPU running)** | **~$4-4.5/mo** | Recurring — when the GPU instance exists/runs: | Item | Rate | Notes | |---|---|---| | g4dn.xlarge **spot** (Ohio) | ~$0.163/hr | primary | | g4dn.xlarge **on-demand** | ~$0.526/hr | fallback | | EBS 128G gp3 (persistent data) | ~$10.24/mo | persists across relaunch | | EBS 30G gp3 root | ~$2.40/mo | ephemeral (DeleteOnTermination) | Scenario totals (incl. EIP + snapshots + EBS): - **Dev, spot, ~88 hrs/mo (4h×22d):** ~$14 compute + ~$12.6 EBS + ~$4.15 = **~$31/mo** - **Always-on spot (730 hrs):** ~$119 + ~$12.6 + ~$4.15 = **~$136/mo** - **Always-on on-demand (730 hrs):** ~$384 + ~$12.6 + ~$4.15 = **~$401/mo** Cost control: keep `launch_instance=false` between sessions (or `stop` the instance); spot + auto-stop keeps dev around ~$30/mo. The 128G data volume persists so restarts need no re-pull. ## Decisions (locked) 1. **VPC CIDR:** distinct **`10.2.0.0/16`** (avoids overlap with us-east-1 `10.0.0.0/16` and on-prem `10.0.0.x`). 2. **us-east-1 teardown:** **terminate + clean up** us-east-1 GemStone resources once Ohio is live (Phase D). 3. **Terraform state:** **local for Phase A**, then migrate to an **S3 backend with native lockfile** (`use_lockfile = true`, Terraform >= 1.10, no DynamoDB) before Phase B via `terraform init -migrate-state`. Creating the SSE-S3-encrypted, versioned, private state bucket is a separate approved step before the migration. 4. **Host contract:** **re-bake** a corrected `golden-v4-gpu-dr` AMI including the `dr` sudo user (password from secrets), `PasswordAuthentication yes` for plink, `/home/dr/AppDev`, docker group, and marker — then copy that to Ohio (Phase 0). 5. **Secret:** use **`DATARIGHT_DEV_SECRETS`**, **duplicated to us-east-2 now (not deferred)**; used to retrieve credentials during the build; `dr` password applied at image time so the instance needs no Secrets Manager access at boot. 6. **SSH:** open (`22` from `0.0.0.0/0`) to permit mobile/remote management, matching the existing pattern — not IP-restricted. ## Explicitly NOT in this plan (later, separate approval) GemStone app deployment, Let's Encrypt/TLS, GoDaddy DNS record change, IAM instance profile + S3 backup bucket. This plan gets Ohio "functionally ready to resume work" at the infrastructure layer. | dd78f67315683c5d1234a370591c5b643a32fecb |
| 47 | DOC-GEMSTONEGUARDRAILS | DOC — GemStoneGuardRails | reference | DevResources/docs/DOC-GemStoneGuardRails.md | Before GemStone answers (sovereign chat, Gemini Live, or ledger RAG), it checks **GemStoneGuardRails**. If the user question matches a rule (exact or inferred similarity), GemStone returns the **fixed response** — no freelancing. | # DOC — GemStoneGuardRails *Mandatory framing pack · passcode-gated rule adds · SuperAdmin-managed* **Product name:** `GemStoneGuardRails` (exact) ## Purpose Before GemStone answers (sovereign chat, Gemini Live, or ledger RAG), it checks **GemStoneGuardRails**. If the user question matches a rule (exact or inferred similarity), GemStone returns the **fixed response** — no freelancing. **Dev (External) / development mode:** hard short-circuit is **off by default** (`GUARDRAILS_HARD_ENFORCE_IN_DEV=false`). Research and general Q&A are answered normally; framing is soft. Set `GUARDRAILS_HARD_ENFORCE_IN_DEV=true` to force fixed responses in Dev. **Sovereign** always hard-enforces. Matcher rejects short/generic substring hijacks (`this`, `application`, …) so long research prompts are not blocked by accidental trigger overlap. ## Seed rule | Triggers (examples) | Fixed response | | --- | --- | | who created this / who built this / who made GemStone / similar | `GemStone was created by Mark Furiate and is part of the DataRight Horizon family of products.` | ## Add / update rules 1. User says e.g. `add to GemStoneGuardRails when … say …` 2. GemStone asks for the personal passcode 3. Passcode verified (hashed compare) → rule stored 4. Wrong / missing passcode → refuse (no write) MCP / HTTP tools: - `POST /mcp/guardrails_match` — `{ "query": "…" }` - `GET|POST /mcp/guardrails_list` - `POST /mcp/guardrails_add` — `{ "passcode", "triggers", "response" }` (passcode required) ## Who may manage what | Action | Who | | --- | --- | | Add rules (with passcode) | Passcode holder; future: users with `guardrails_contributor` + passcode | | Create/rotate passcode | **SuperAdmin only** (`PUT /api/admin/guardrails/passcode`) | | Grant `guardrails_contributor` | **SuperAdmin only** | | Org Admin | May list rules in Admin UI; **cannot** manage passcode or contributors | Passcode storage: salted hash in Postgres (`gemstone_guardrails_meta`), same PBKDF2-SHA256 scheme as local user passwords. Optional first-boot seed from env `GEMSTONE_GUARDRAILS_PASSCODE` (plaintext never committed; discarded after hash). SuperAdmin can rotate anytime under `/app/admin` → **guardrails**. ## Injection points 1. Sovereign / Gemini chat — short-circuit **before** ledger RAG 2. `gather_context` — framing block prepended ahead of ledger / packs / Atlas 3. Gemini Live — system instruction + `guardrails_*` tools before ledger tools ## Admin UI `/app/admin` → tab **guardrails**: list rules; **Edit** / **Save** / **Delete** (Cancel while editing) on existing rules via SuperAdmin JWT (`PATCH` / `DELETE /api/admin/guardrails/rules/{id}` — no passcode). **Delete** opens an explicit confirm: **Inactivate** (soft `is_active=false`) vs **Remove permanently** (`DELETE ?permanent=true` hard-deletes the row) vs Cancel. Inactive rows can be **Reactivated** (`POST …/reactivate`). **Add** still requires the passcode (`POST /mcp/guardrails_add`). The API rejects normalized duplicates (trim / collapse whitespace / case-insensitive triggers+response): active → 409 `duplicate_active`; inactive → 409 `duplicate_inactive` with `rule_id` so Admin can reactivate instead of creating another row. SuperAdmin set/rotate passcode; grant/revoke contributor flag on users. ## Migration Alembic `0010_gemstone_guardrails` (additive): rules table, meta table, `users.guardrails_contributor`. ## Smoke 1. Ask: `Who created this application?` → Mark Furiate / DataRight Horizon line 2. `add to GemStoneGuardRails …` without passcode → prompt / refuse write 3. Add with correct passcode → rule listed; subsequent match returns fixed text Auth0 remains deferred. ## Boundary vs practitioner personal rules `GemStoneGuardRails` is **product-wide** (passcode + SuperAdmin). Practitioner personal rules are a separate Atlas segment — see [`DOC-Practitioner-Rules.md`](./DOC-Practitioner-Rules.md) (MVP shipped). | Concern | GemStoneGuardRails (now) | Practitioner personal rules (MVP) | HorizonIQ (later) | | --- | --- | --- | --- | | Scope | Product framing for all users | Personal Atlas segment only | Org-wide shared practitioner rules | | Passcode | SuperAdmin passcode required to add | **No** passcode | Passcode gate on personal rules (deferred) | | Merge / promote | N/A | Does not write into GuardRails | “Make that a rule” → GuardRails (deferred) | | Approvals | Passcode gate | **keep original** vs **use new** via chat or voice; user must be signed in | — | | Phrase | `add to GemStoneGuardRails …` | **Make that a rule** | — | See punchlist: `DOC07-Punchlist.md` → Practitioner personal rules + **HorizonIQ (later)**. | d0c4de025d4d1ddb3d03ec6c84fbba1d1b33755e |
| 48 | DOC-PRACTITIONER-RULES | DOC — Practitioner personal Atlas rules | reference | DevResources/docs/DOC-Practitioner-Rules.md | Each signed-in practitioner can save **personal** fixed-response rules in their Atlas segment (`org_id` + `practitioner_id`). These are consulted **after** product-wide `GemStoneGuardRails` and **before/with** ledger context so freelanced answers do not violate personal preferences. | # DOC — Practitioner personal Atlas rules *Personal segment · signed-in only · distinct from GemStoneGuardRails* ## Purpose Each signed-in practitioner can save **personal** fixed-response rules in their Atlas segment (`org_id` + `practitioner_id`). These are consulted **after** product-wide `GemStoneGuardRails` and **before/with** ledger context so freelanced answers do not violate personal preferences. ## Phrase: "Make that a rule" Say (chat or voice): **Make that a rule** 1. Must be **signed in** — otherwise GemStone asks you to sign in and does **not** write. 2. Captures the decision from recent conversation turns (or an inline `when … say …`). 3. Inference-search over **active** personal rules for overlap: - **Match** → remind you, show the existing rule(s), ask **keep original** vs **use new** (chat or voice). - **Keep original** → no write. - **Use new** → insert new **active** rule; mark old **superseded** (retained; never deleted). - **No match** → create new active rule. ## Rule history Say: **show my rule history** (or **rule history** / **my personal rules**) Superseded rules are excluded from normal match; they appear only in history. ## MCP / HTTP | Tool / path | Purpose | | --- | --- | | `POST /mcp/prac_rules/match` | Match active personal rules | | `GET\|POST /mcp/prac_rules/list` | List active | | `GET\|POST /mcp/prac_rules/history` | Active + superseded | | `POST /mcp/prac_rules/propose` | Propose (overlap → conflict) | | `POST /mcp/prac_rules/commit` | `keep_original` \| `use_new` | | `GET /api/v1/practitioner_rules` | List active (API) | Writes require signed-in tenant identity. ## Injection 1. Sovereign / Gemini chat — short-circuit **after** GuardRails, **before** ledger RAG 2. `gather_context` — personal framing after GuardRails 3. Gemini Live — `prac_rules_*` tools + framing when org/prac known from hello ## Migration Alembic `0011_practitioner_rules` — additive `practitioner_rules` table + RLS (org isolation). ## Boundary | | GemStoneGuardRails | Practitioner personal rules | HorizonIQ (later) | | --- | --- | --- | --- | | Scope | Product-wide | Personal Atlas only | Org-wide / merge | | Passcode | Yes (SuperAdmin) | **No** (MVP) | Personal passcode deferred | | Phrase | `add to GemStoneGuardRails …` | **Make that a rule** | Promote into GuardRails deferred | See `DOC07-Punchlist.md` and `DOC-GemStoneGuardRails.md`. | 5f7b7f069a4cb1881227aceaafa88904f50873ee |
| 49 | DOC-PUSH-NOTIFICATIONS-SNS | DOC — GemStone Push Notifications (AWS SNS + FCM + Capacitor) | reference | DevResources/docs/DOC-Push-Notifications-SNS.md | Do these in order. Do **not** paste secret values into chat or git. | # DOC — GemStone Push Notifications (AWS SNS + FCM + Capacitor) **Status:** Scaffold live 2026-07-23 · Real device push **blocked** until Mark creates Firebase + secrets **Stack:** AWS SNS Mobile Push → FCM (Android) · Capacitor `@capacitor/push-notifications` · **not MQTT** **Policy:** Messaging is AWS-native only (DOC06). No Twilio. **App ID (verified):** Capacitor / Android package = **`com.nextech.gemstone`** **In-app guide link (already live):** `/install-android` → “Enable notifications” → links this doc on GitHub. Admin settings also points here. --- ## Follow the bouncing ball (Mark / ops) Do these in order. Do **not** paste secret values into chat or git. ### 1. Firebase — Android app 1. Open [Firebase Console](https://console.firebase.google.com/). 2. Create a project (e.g. `gemstone-dataright`) or select an existing one. 3. **Add app** → **Android**. 4. Package name: **`com.nextech.gemstone`** (must match Capacitor `appId` / `applicationId` — verified in repo). 5. Register the app (nickname optional; SHA-1 optional for push-only). ### 2. Place `google-services.json` 1. Download **`google-services.json`** from the Firebase Android app settings. 2. Place it at: ``` frontend/android/app/google-services.json ``` 3. Do **not** commit it if it contains project keys you treat as sensitive (prefer secure channel / local only for APK rebuild). Capacitor Push Notifications uses this file at build time; no extra Firebase SDK wiring is required in app code. ### 3. Get FCM credentials for SNS 1. Firebase → Project settings → **Cloud Messaging**. 2. Prefer **Cloud Messaging API (V1)** enabled. 3. For AWS SNS Platform Application (`GCM`), you still need a **Server key** (legacy Cloud Messaging API) **or** follow AWS’s current FCM credential guidance for the platform credential. 4. Copy the server key once locally — **never** paste into chat/git. ### 4. Secrets → `DATARIGHT_DEV_SECRETS` (us-east-1) Update AWS Secrets Manager secret **`DATARIGHT_DEV_SECRETS`** (region **us-east-1**) with these **exact** JSON key names (string values): | Key | When | Purpose | | --- | --- | --- | | `FCM_SERVER_KEY` | Now (after step 3) | FCM legacy server key → SNS `PlatformCredential` for Android (GCM) | | `FCM_PROJECT_ID` | Now (optional but recommended) | Firebase project id (ops / docs; not required for SNS publish path) | | `SNS_PUSH_PLATFORM_ARN_ANDROID` | **After** step 5 | SNS platform application ARN | | `AWS_SNS_REGION` | Already present for SMS/SES | Prefer `us-east-1` | | `SES_FROM_EMAIL` | Already used | Email fallback when push not configured | API reads the same names from env (compose / host). Confirmed in `api/app/core/config.py`: ``` SNS_PUSH_PLATFORM_ARN_ANDROID SNS_PUSH_PLATFORM_ARN_IOS FCM_SERVER_KEY FCM_PROJECT_ID AWS_SNS_REGION SES_FROM_EMAIL ``` When keys are in Secrets Manager (no values in chat), reply to the agent: > **FCM credentials are in DATARIGHT_DEV_SECRETS** ### 5. SNS platform application (Android / FCM) **Only after** `FCM_SERVER_KEY` is in secrets/env. Do not create with empty credentials. ```bash # Region: us-east-1 (match AWS_SNS_REGION) aws sns create-platform-application \ --name GemStone-Android-FCM \ --platform GCM \ --attributes PlatformCredential="<FCM_SERVER_KEY>" \ --region us-east-1 ``` 1. Response includes `PlatformApplicationArn`. 2. Store it as `SNS_PUSH_PLATFORM_ARN_ANDROID` in `DATARIGHT_DEV_SECRETS`. 3. Recreate / restart `gemstone-api` so config picks up the new env/secrets. 4. Verify (no secrets printed): ```bash aws sns list-platform-applications --region us-east-1 \ --query "PlatformApplications[?contains(PlatformApplicationArn, 'GemStone')]" ``` Endpoint creation is automatic at register time (`CreatePlatformEndpoint` + store `sns_endpoint_arn`). ### 6. Rebuild / reinstall the Android APK Rebuild is required after adding `google-services.json` (FCM cannot issue device tokens without it). 1. From `frontend/`: sync + assemble per [CAPACITOR-ANDROID-INSTALLATION-PLAYBOOK.md](./CAPACITOR-ANDROID-INSTALLATION-PLAYBOOK.md) (e.g. `npm run cap:sync:android`, then Android Studio Run **or** Gradle assemble). 2. Install on a physical Android device (emulators often lack reliable FCM). 3. Or use the live install path: **https://gemstone.dataright.net/install-android** after a new APK is published there. ### 7. Sign in → enable → register token → verify test push 1. Open the installed GemStone app (`com.nextech.gemstone`). 2. **Sign in** with your local GemStone account (email/password JWT). 3. Allow **notification permission** when prompted. On login, the app calls Capacitor `PushNotifications.register()` and `POST /api/push/register` with the FCM token. 4. Confirm server-side config (signed-in): ```http GET /api/push/status ``` Expect `sns_android_configured: true` once `SNS_PUSH_PLATFORM_ARN_ANDROID` is loaded by the API. 5. Confirm a row exists for your practitioner in `device_push_tokens` (platform `android`, token present; `sns_endpoint_arn` set when SNS ARN is configured). 6. **Trigger a test push** (no dedicated `/api/push/test` route — use real hooks): - Cause a Didymus event that needs approval, **or** complete/fail a Didymus task — both call `notify_practitioner` / org approval notify (best-effort; never breaks the approval gate). - Or from ops: SNS Console → publish to the device’s `sns_endpoint_arn` with a GCM JSON payload. 7. Device should show the notification. If SNS/FCM is not ready, API falls back to **SES email** when `SES_FROM_EMAIL` and practitioner email are known; otherwise it skips quietly. ### 8. What’s already live vs blocked | Capability | Status | | --- | --- | | Alembic `device_push_tokens` + `/api/push/register` · `/unregister` · `/status` | **Live (scaffold)** | | `notify_practitioner` (SNS or SES or skip) | **Live (scaffold)** | | Didymus hooks (approval needed / task completed) | **Live** — best-effort; gate untouched | | Capacitor plugin + register-on-login | **Live (scaffold)** — tokens work once FCM issues them | | `/install-android` + Admin copy linking this doc | **Live** | | Real Android push delivery | **Blocked** — needs Firebase + `FCM_SERVER_KEY` + SNS platform app + `SNS_PUSH_PLATFORM_ARN_ANDROID` + APK with `google-services.json` | | iOS / APNs | **Deferred** (see below) | Rolling deploys: API + migration are additive; missing FCM/SNS env leaves push as no-op + SES fallback. Live site stays up. ### 9. iOS / APNs (deferred — do not block Android) 1. Apple Developer: App ID + Push Notifications for the iOS bundle id (`com.nextech.gemstone`). 2. Create APNs key or certificate; upload to SNS (`APNS` or `APNS_SANDBOX`). 3. Store `SNS_PUSH_PLATFORM_ARN_IOS` (and optional `APNS_PLATFORM_CREDENTIAL` material) in secrets. 4. Capacitor iOS registration already scaffolds via the same plugin; enable when the APNs ARN exists. --- ## Architecture ``` Android / iOS (Capacitor) │ FCM / APNs device token ▼ POST /api/push/register → device_push_tokens (+ optional SNS endpoint ARN) │ Didymus / other events │ notify_practitioner(practitioner_id, title, body, data) ├─ if SNS_PUSH_PLATFORM_ARN_* set → SNS Publish to endpoint ARN(s) └─ else → SES email to practitioner email (if known) / skip gracefully ``` | Layer | Role | | --- | --- | | Capacitor Push Notifications | Requests permission, obtains FCM token on Android, registers with API on login | | `device_push_tokens` | Per-practitioner token store (`platform`, `token`, `sns_endpoint_arn`, `updated_at`) | | AWS SNS Platform Application | Maps FCM credentials → platform ARN; CreatePlatformEndpoint per device token | | FCM | Google delivery to Android devices | | APNs | Deferred — iOS later (same SNS pattern, `SNS_PUSH_PLATFORM_ARN_IOS`) | | SES fallback | When SNS/FCM not configured: email via `SES_FROM_EMAIL` if practitioner email known | ## Deploy safety - Additive migration + optional env; no hard dependency on FCM at boot. - Didymus approve / lease / schema gate unchanged — notify is try/except best-effort. - Alembic: `0012_device_push_tokens` revises `0011_practitioner_rules`. ## Related - Punchlist: `DOC07-Punchlist.md` (Firebase checklist) - Capacitor install: `CAPACITOR-ANDROID-INSTALLATION-PLAYBOOK.md` - Messaging policy: `DOC06-Master-Prompt.md` - Live UI: `/install-android` (Enable notifications), Admin → notifications note | 77f96dbb30a4fac7af18b6afdc34ef5d21808fc2 |
| 50 | DOC-DIDYMUS-BRIDGE-PUNCHLIST | DOC — Didymus orchestration & desktop bridge (practitioner punchlist) | punchlist | DevResources/docs/DOC-Didymus-Bridge-Punchlist.md | GemStone is the **orchestrator**. Practitioners connect AI workers (Cursor, Claude, Gemini, Ollama). Didymus shows approval-gated tasks and buffered conversation events linked to **registered** desktop sessions. Changes in the GemStone Agent window are mirrored into the Didymus relay as far as APIs allow; **desktop IDE chat windows are not claimed as live** until vendor bridge keys exist. | # DOC — Didymus orchestration & desktop bridge (practitioner punchlist) **Date:** 2026-07-23 · **Status:** Scaffold + live execute path ## Product intent GemStone is the **orchestrator**. Practitioners connect AI workers (Cursor, Claude, Gemini, Ollama). Didymus shows approval-gated tasks and buffered conversation events linked to **registered** desktop sessions. Changes in the GemStone Agent window are mirrored into the Didymus relay as far as APIs allow; **desktop IDE chat windows are not claimed as live** until vendor bridge keys exist. ## What works today 1. Per-practitioner vault keys (`/app/onboarding/workers`) + bouncing-ball guide modal. 2. Approval queue + **Execute** (`POST /api/didymus/tasks/{id}/execute`) → `mode: live|simulated`. 3. Bridge scaffold: `GET /api/didymus/bridge/status`, session register, event buffer, `WS /ws/didymus/bridge`. 4. Honest UI: `mirror_ready` stays false; `pushed_to_desktop` stays false; status lists `blocked_on_keys`. ## Secrets Mark must supply (DATARIGHT_DEV_SECRETS / host `.env`) | Key | Purpose | | --- | --- | | `WORKER_SECRETS_KEY` | AES-GCM vault master (required for Connect UI) | | `DIDYMUS_WORKER_CURSOR_TOKEN` | Cursor **Cloud Agents** execute (not IDE UI mirror) | | `DIDYMUS_WORKER_CLAUDE_API_KEY` | Anthropic Messages API execute | | `DIDYMUS_WORKER_GEMINI_API_KEY` / `GEMINI_API_KEY` | Gemini worker (+ voice) | | `DIDYMUS_CURSOR_DEFAULT_REPO` | Optional default GitHub URL for Cursor tasks | | *Future* Cursor/Claude **desktop bridge** tokens | True bidirectional IDE chat sync | ## Verify (bouncing ball) 1. Sign in → `/app/onboarding/workers` → vault available → Connect keys → badges **Connected (last4)**. 2. `/app/didymus` → Approve → Execute → status shows `live` (or `simulated` if no key). 3. Register a desktop session link → appears as **registered (not mirrored)**. 4. Bridge WS hint shows connected; buffered turns appear after Execute. ## Out of scope until keys/adapters exist - Pushing Agent window turns into Cursor IDE / Claude Desktop chat panes in real time. - Claiming “connected” for desktop UI when only Cloud Agents / Messages API keys are present. | e9463b68f802829297e552bc21764286db809b21 |