Compare commits

...

3 Commits

Author SHA1 Message Date
overseer fc1a2f5103 feat: PostgreSQL migration, OAuth write MCP, Streamable HTTP, env-driven config, admin UI, landing page
- Migrate database from SQLite to PostgreSQL 16 (dual-backend with SQLite fallback)
- Add Streamable HTTP MCP transport (replaces SSE): /readonly/mcp, /write/mcp, /mcp
- Add OAuth ctxd.write scope and public write MCP surface
- Add ctxd.write token validation (write-scoped tokens only on /write/mcp)
- Add env-driven configuration (.env file with env var precedence over ctxd.yaml)
- Add PostgreSQL to docker-compose.yml with healthcheck
- Add psycopg dependency, migration script (SQLite → PostgreSQL)
- Add admin UI: projects tab with typed-confirm delete, user management (list/manage subtabs)
- Add OAuth client management: create, list, revoke (UI, CLI, API)
- Add user active/inactive lifecycle (PATCH/DELETE APIs)
- Add public landing page with themed login form (cookie-based session)
- Add get_client_guide MCP tool (locked LLM-CLIENT.MD in ctxd-docs project)
- Add DELETE /projects/<id> endpoint with cascading deletes
- Add project_delete to db.py with FK ON DELETE SET NULL for audit_log
- Add cookie-based session auth (ctxd_session cookie on login)
- Add landing.html (public host) vs ui.html (internal dashboard)
- Add schema_sqlite.sql for SQLite fallback
- Add auth_password.py (PBKDF2-SHA256 password hashing)
- Add .env.example template with all documented env vars
- Add README.md with full setup, config, API, CLI, and troubleshooting docs
- Add SKILL.md (canonical LLM client guide, lives in project root)
- Update Traefik template: route everything except /mcp
- Update OAuth discovery: advertise ctxd.write scope, /readonly/mcp resource
- Update Hermes MCP config: /mcp endpoint with Bearer header
- Remove DB-level audit_log triggers (conflict with FK ON DELETE SET NULL)
- Remove SSE transport code (replaced by Streamable HTTP)
- Untrack __pycache__ and data/ctxd.db from git
2026-06-24 22:50:54 +00:00
overseer a9ccfa2694 Stop tracking ctxd.db-shm and ctxd.db-wal and add to .gitignore 2026-06-24 11:12:33 +00:00
overseer 5a0aa2d4fe Stop tracking ctxd.yaml and add to .gitignore 2026-06-24 10:57:53 +00:00
32 changed files with 4407 additions and 360 deletions
+22
View File
@@ -0,0 +1,22 @@
# Runtime CTXD data/state
/data/oauth_state.json
/data/web_sessions.json
/data/.ctxd.yaml.swp
/data/ctxd.db*
/data/ctxd.db-shm
/data/ctxd.db-wal
/data/ctxd.yaml
/data/snapshots/
/data/projects/
/data/users/
# Environment files (contains secrets)
.env
app/.env
# PostgreSQL data volume
/data/pg/
# Python cache
__pycache__/
*.py[cod]
+597
View File
@@ -0,0 +1,597 @@
# CTXD — Context Dossier
A single source of truth for multi-harness project context. One canonical `AGENTS.md` per project, served to Claude, Hermes, Codex, Cursor, and any OAuth-capable MCP client via Streamable HTTP.
## Overview
CTXD solves context sprawl: when you work across multiple LLM harnesses (Claude Desktop, Claude Code, Codex CLI, Cursor, Copilot, Hermes), each has its own context file convention (`CLAUDE.md`, `.cursorrules`, `CODEX.md`, etc.). Without a canonical source, these drift independently.
CTXD provides:
- **Multi-file context per project** — `CONTEXT.MD`, `DECISIONS.MD`, `RUNBOOKS.MD`, `PROMPTS.MD`, `GLOSSARY.MD`
- **Compiled view** — all files concatenated with metadata header, served as a single document to agents
- **Sync to repos** — writes `CONTEXT.MD` as `AGENTS.md` + symlinks (`CLAUDE.md`, `.cursorrules`, `CODEX.md``AGENTS.md`)
- **Version-checked writes** — optimistic concurrency with `base_version` to prevent silent overwrites
- **OAuth 2.0 authorization server** — DCR, Authorization Code + PKCE, `ctxd.read` and `ctxd.write` scopes
- **Streamable HTTP MCP** — single-endpoint transport for read-only and write surfaces
- **Web UI** — per-user password login, admin panel, project/file management
- **PostgreSQL backend** — with SQLite fallback for local dev
- **Append-only audit log** — every read, write, create, delete, sync, and search is logged
- **Point-in-time snapshots** — automatic version snapshots with rotation (min 5, max 25 per project)
- **Full-text search** — PostgreSQL `tsvector` with GIN index (FTS5 on SQLite fallback)
## Architecture
```
Context Dossier (container: dossier, 0.0.0.0:9091)
├── PostgreSQL 16 (container: ctxd-postgres) # Primary DB
├── /data # Config, OAuth state, web sessions
│ ├── ctxd.yaml # Fallback config (env vars take precedence)
│ ├── oauth_state.json # OAuth clients, codes, tokens
│ ├── web_sessions.json # Per-user web UI sessions
│ └── snapshots/ # Point-in-time context backups
├── Streamable HTTP MCP endpoints:
│ ├── /readonly/mcp (OAuth ctxd.read) # Read-only tools
│ ├── /write/mcp (OAuth ctxd.write) # Write tools
│ └── /mcp (shared API key) # Internal full MCP
├── OAuth Authorization Server:
│ ├── /.well-known/oauth-authorization-server # Discovery
│ ├── /.well-known/oauth-protected-resource # Resource metadata
│ ├── /oauth/register # Dynamic Client Registration
│ ├── /oauth/authorize # Authorization + PKCE
│ └── /oauth/token # Token + refresh
├── Web UI + REST API (/) # Dashboard, admin, projects, files
└── Landing page (public host only) # Themed login form
```
## Quick Start
### Prerequisites
- Docker and Docker Compose
- A reverse proxy with TLS (Traefik, Caddy, nginx) for public exposure
- (Optional) An existing PostgreSQL 14+ instance if not using the bundled one
### 1. Clone and configure
```bash
cd /mnt/ai-storage/projects/ctxd/app
cp .env.example .env
```
Edit `.env` with your values:
```bash
# Database
DATABASE_URL=postgresql://ctxd:your-password@postgres:5432/ctxd
POSTGRES_USER=ctxd
POSTGRES_PASSWORD=your-strong-password
# Server
CTXD_HOST=0.0.0.0
CTXD_PORT=9091
CTXD_HOME=/data
# Auth
CTXD_AUTH_ENABLED=true
CTXD_API_KEY=your-api-key-here # Generate: python3 -c "import secrets; print(secrets.token_urlsafe(32))"
# OAuth
OAUTH_ENABLED=true
OAUTH_ISSUER=https://ctxd.yourdomain.com
OAUTH_APPROVAL_KEY=your-approval-key # Generate: python3 -c "import secrets; print(secrets.token_urlsafe(32))"
```
### 2. Build and start
```bash
docker compose up -d
```
This starts:
- `ctxd-postgres` — PostgreSQL 16 (Alpine)
- `dossier` — CTXD daemon (web UI + MCP + OAuth + REST API)
### 3. Verify
```bash
# Health check
curl http://localhost:9091/status
# → {"status": "ok", "db": "/data/ctxd.db"}
# List projects (requires API key)
curl http://localhost:9091/projects -H "Authorization: Bearer your-api-key"
```
### 4. Set admin password
```bash
docker exec dossier dossier user-set-password admin -p "your-admin-password"
```
> **Shell quoting:** Use double quotes (`"`) if the password contains single quotes (`'`). Use single quotes (`'`) if it contains `$`, backticks, or `!`. The failure mode is shell expansion, not PBKDF2.
### 5. Access the Web UI
- **LAN:** `http://<server-ip>:9091/`
- **Public (via Traefik):** `https://ctxd.yourdomain.com/`
Sign in with `admin` / your password.
## Configuration
### Environment Variables
All config is driven by environment variables. A `ctxd.yaml` file in `/data` can override built-in defaults, but env vars always take precedence.
**Precedence:** env var > `ctxd.yaml` > built-in default
| Variable | Default | Description |
|----------|---------|-------------|
| **Database** | | |
| `DATABASE_URL` | *(empty)* | PostgreSQL connection string. If empty, falls back to SQLite at `$CTXD_HOME/ctxd.db` |
| `POSTGRES_USER` | `ctxd` | PostgreSQL user (for bundled PG container) |
| `POSTGRES_PASSWORD` | `ctxd_local_dev` | PostgreSQL password (for bundled PG container) |
| `POSTGRES_DB` | `ctxd` | PostgreSQL database name (for bundled PG container) |
| **Server** | | |
| `CTXD_HOST` | `0.0.0.0` | Bind address |
| `CTXD_PORT` | `9091` | Listen port |
| `CTXD_HOME` | `~/.ctx` | Data directory (inside container: `/data`) |
| `LOG_LEVEL` | `info` | Uvicorn log level (`debug`, `info`, `warning`, `error`) |
| **Auth** | | |
| `CTXD_AUTH_ENABLED` | `false` | Enable authentication globally |
| `CTXD_API_KEY` | *(empty)* | Shared API key for Hermes/internal MCP + HTTP auth |
| `CTXD_EXTERNAL_READONLY_KEY` | *(empty)* | Legacy `?key=` on read-only MCP (migration only) |
| **OAuth** | | |
| `OAUTH_ENABLED` | `false` | Enable OAuth authorization server |
| `OAUTH_ISSUER` | *(empty)* | Public URL (used in OAuth discovery metadata) |
| `OAUTH_APPROVAL_KEY` | *(empty)* | Fallback approval key for `/oauth/authorize` |
| `OAUTH_APPROVAL_USER_ID` | `admin` | Which user ID to attribute OAuth approvals to |
| `OAUTH_ACCESS_TOKEN_TTL` | `3600` | Access token lifetime in seconds |
| `OAUTH_REFRESH_TOKEN_TTL` | `2592000` | Refresh token lifetime in seconds (30 days) |
| **Web Sessions** | | |
| `WEB_SESSION_TTL` | `604800` | Session cookie lifetime in seconds (7 days) |
| **Snapshots** | | |
| `SNAPSHOT_MIN_KEEP` | `5` | Minimum snapshots retained per project |
| `SNAPSHOT_MAX_KEEP` | `25` | Maximum snapshots before rotation |
### Using an External PostgreSQL
To use an external PostgreSQL instead of the bundled container:
1. Create a database and user on your external PG instance
2. Set `DATABASE_URL` in `.env` to point to it
3. Start only the dossier service: `docker compose up -d dossier`
The bundled `ctxd-postgres` container will start but is unused. To prevent it from starting, run it with zero scale:
```bash
docker compose up -d --scale postgres=0
```
### Fallback to SQLite
If `DATABASE_URL` is empty or not set, CTXD falls back to SQLite at `$CTXD_HOME/ctxd.db`. This is useful for local development or single-user deployments that don't need PostgreSQL features.
## MCP Surfaces
CTXD exposes three MCP endpoints via Streamable HTTP:
| Endpoint | Auth | Scope | Tools |
|----------|------|-------|-------|
| `/readonly/mcp` | OAuth bearer or API key | `ctxd.read` | `list_projects`, `get_project_context`, `search_context`, `get_project_tags`, `list_files`, `get_file`, `get_client_guide` |
| `/write/mcp` | OAuth bearer | `ctxd.write` | `update_file`, `set_project_tags`, `sync_to_project`, `get_client_guide` |
| `/mcp` | Shared API key | *(full)* | All read + write tools + `sync_to_project`, `auto_generate_tags`, `get_user_profile` |
### Connecting an LLM Client
**Claude Desktop / Claude Web:**
```
Connector URL: https://ctxd.yourdomain.com/readonly/mcp
```
Claude auto-discovers OAuth metadata and registers via DCR. Request `scope=ctxd.read ctxd.write` for write access.
**Hermes Agent:**
```yaml
# ~/.hermes/config.yaml
mcp_servers:
dossier:
url: http://<server-ip>:9091/mcp
timeout: 30
headers:
Authorization: "Bearer your-api-key"
```
**Other MCP clients (Codex, Cursor, etc.):**
- Register an OAuth client via `POST /oauth/register` with your redirect URI
- Connect to `/readonly/mcp` (read) or `/write/mcp` (read + write)
- Use the access token as `Authorization: Bearer <token>`
### MCP Tool Reference
#### Read-only tools (`/readonly/mcp` and `/write/mcp`)
| Tool | Args | Returns |
|------|------|---------|
| `get_client_guide` | *(none)* | Locked `LLM-CLIENT.MD` guide — **call this first** |
| `list_projects` | *(none)* | All projects with version numbers |
| `get_project_context` | `project_id` | Compiled markdown of all context files |
| `search_context` | `query`, `limit?` | FTS results across all projects |
| `get_project_tags` | `project_id` | Metadata tags for a project |
| `list_files` | `project_id` | All context files in a project |
| `get_file` | `project_id`, `file_path` | Single file with metadata header |
#### Write tools (`/write/mcp` only)
| Tool | Args | Returns |
|------|------|---------|
| `update_file` | `project_id`, `file_path`, `content`, `base_version` | `{"ok": true, "new_version": N}` or conflict error |
| `set_project_tags` | `project_id`, `tags[]` | `{"ok": true, "tags": [...]}` |
| `sync_to_project` | `project_id` | Writes `CONTEXT.MD` as `AGENTS.md` + symlinks to project root |
### Locked Files
| File | Scope | Protection |
|------|-------|------------|
| `CONTEXT.MD` | All projects | Cannot delete — minimum required file |
| `CONTEXT.MD` | `ctxd-docs` project only | Cannot update or delete |
| `LLM-CLIENT.MD` | `ctxd-docs` project only | Cannot update or delete |
## OAuth
### Scopes
| Scope | Grants |
|-------|--------|
| `ctxd.read` | Read-only MCP tools |
| `ctxd.write` | Write MCP tools (includes read) |
Request both: `scope=ctxd.read ctxd.write`
### Redirect URIs
| Platform | Redirect URI |
|----------|-------------|
| Claude Desktop | `https://claude.ai/api/mcp/auth_callback` |
| Claude Code | `http://localhost:5555/oauth/callback` |
| Codex CLI | `http://localhost:7777/oauth/callback` |
| Custom | Your app's documented OAuth callback |
### Managing OAuth Clients
**Via Admin UI:** `http://<server-ip>:9091/` → sign in as admin → **admin****oauth clients** tab
**Via CLI:**
```bash
# Create
docker exec dossier dossier oauth-client-create -n "Claude Desktop" --redirect-uri https://claude.ai/api/mcp/auth_callback
# List
docker exec dossier dossier oauth-client-list
# Revoke (invalidates all tokens for that client)
docker exec dossier dossier oauth-client-revoke ctxd_xxxxxxxx
```
**Via API (admin session or API key):**
```bash
# Create
curl -X POST http://localhost:9091/oauth/clients \
-H "Authorization: Bearer your-api-key" \
-H "Content-Type: application/json" \
-d '{"client_name": "Claude Desktop", "redirect_uris": ["https://claude.ai/api/mcp/auth_callback"]}'
# List
curl http://localhost:9091/oauth/clients -H "Authorization: Bearer your-api-key"
# Revoke
curl -X DELETE http://localhost:9091/oauth/clients/ctxd_xxxxxxxx -H "Authorization: Bearer your-api-key"
```
## Public Exposure (Traefik)
### Router Rule
Route everything except the internal MCP endpoint:
```yaml
rule: >
Host(`ctxd.yourdomain.com`) &&
!Path(`/mcp`)
```
This exposes:
- Landing page (`GET /`)
- Login (`POST /auth/login`, `GET /auth/me`)
- Full Web UI dashboard (all REST API endpoints)
- OAuth (`/oauth/*`, `/.well-known/*`)
- Read-only MCP (`/readonly/mcp`)
- Write MCP (`/write/mcp`)
Only blocked: `/mcp` (internal full MCP — shared API key only)
### Landing Page Behavior
- **Not signed in** → themed landing page with login form
- **Signed in** (valid session cookie) → full dashboard
- Cookie: `ctxd_session` (SameSite=Lax, 7-day expiry)
- After login: cookie set + redirect to dashboard
A full Traefik template is at `app/templates/traefik-ctxd-readonly-mcp.yaml` in the `project-context-management` Hermes skill.
## Web UI
### Admin Panel
Sign in as admin → click **admin** in the masthead.
**Tabs:**
1. **oauth clients** — client list (revoke per row) + create client form
2. **users** — user list (id, name, role, active/inactive) + manage users (create, edit, activate, inactivate, delete)
3. **projects** — manage projects (list with remove button, typed-name confirmation for delete)
### Project Files
Each project has multiple context files:
| File | Purpose |
|------|---------|
| `CONTEXT.MD` | Canonical project overview (synced as `AGENTS.md` to repos) |
| `DECISIONS.MD` | Architecture decisions, rationale |
| `RUNBOOKS.MD` | Deploy, troubleshoot, operate procedures |
| `PROMPTS.MD` | Project-specific prompts for different harnesses |
| `GLOSSARY.MD` | Project-specific terms, acronyms |
The compiled view (`get_project_context`) concatenates all files with `## FILENAME` headers and a single metadata block at the top.
## CLI
All commands run inside the container:
```bash
docker exec dossier dossier <command>
```
### Commands
```bash
# Initialize (auto-runs on first container start)
dossier init
# Projects
dossier project-create <project_id> [--display-name "Name"] [--description "Desc"]
dossier project-list
dossier read <project_id> # Print context to stdout
dossier edit <project_id> # Open in $EDITOR
# Context files
dossier file-list <project_id>
dossier file-read <project_id> <file_path>
# Sync
dossier sync <project_id> [path] # Set sync path and/or sync AGENTS.md
# Search
dossier search "query" # FTS across all projects
# Audit
dossier audit [--limit N]
# Users
dossier user-list
dossier user-create <user_id> --display-name "Name" [--password "pw"]
dossier user-set-password <user_id> -p "password"
# OAuth
dossier oauth-client-create [-n "Name"] [--redirect-uri URI]
dossier oauth-client-list
dossier oauth-client-revoke <client_id>
# Import
dossier import-vault <path> # Import from Obsidian vault
```
## REST API
All endpoints require `Authorization: Bearer <api_key>` or `Authorization: Bearer <session_token>` unless noted.
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/` | Web UI (LAN) or landing page (public host) |
| `GET` | `/status` | Health check (no auth) |
| `POST` | `/auth/login` | Web UI login → session token + cookie |
| `POST` | `/auth/logout` | Revoke session + clear cookie |
| `GET` | `/auth/me` | Current session identity |
| `GET` | `/users` | List users |
| `POST` | `/users` | Create user |
| `PATCH` | `/users/<id>` | Update user (admin) |
| `DELETE` | `/users/<id>` | Delete user (admin) |
| `POST` | `/users/<id>/password` | Set password (admin) |
| `GET` | `/oauth/clients` | List OAuth clients (admin) |
| `POST` | `/oauth/clients` | Register OAuth client (admin) |
| `DELETE` | `/oauth/clients/<id>` | Revoke OAuth client (admin) |
| `GET` | `/projects` | List all projects |
| `POST` | `/projects` | Create a project |
| `DELETE` | `/projects/<id>` | Delete a project (admin) |
| `GET` | `/projects/<id>/context` | Compiled context (all files) |
| `POST` | `/projects/<id>/context` | Update context (legacy single-file) |
| `GET` | `/projects/<id>/files` | List context files |
| `GET` | `/projects/<id>/files/<name>` | Read a single file |
| `POST` | `/projects/<id>/files` | Create a new file |
| `PUT` | `/projects/<id>/files/<name>` | Update a file (version-checked) |
| `DELETE` | `/projects/<id>/files/<name>` | Delete a file |
| `POST` | `/projects/<id>/migrate-files` | Migrate single-context to multi-file |
| `GET` | `/projects/<id>/snapshots` | List snapshots |
| `GET` | `/projects/<id>/tags` | Get metadata tags |
| `POST` | `/projects/<id>/tags` | Set metadata tags |
| `POST` | `/projects/<id>/sync` | Sync CONTEXT.MD as AGENTS.md |
| `POST` | `/projects/<id>/import` | Import raw text as context |
| `GET` | `/search?q=...` | Full-text search |
| `GET` | `/audit?limit=N` | Audit log |
| `GET` | `/.well-known/oauth-authorization-server` | OAuth discovery |
| `GET` | `/.well-known/oauth-protected-resource` | Resource metadata |
| `POST` | `/oauth/register` | Dynamic Client Registration |
| `GET/POST` | `/oauth/authorize` | Authorization endpoint |
| `POST` | `/oauth/token` | Token endpoint |
| `POST/GET/DELETE` | `/readonly/mcp` | Read-only MCP (Streamable HTTP) |
| `POST/GET/DELETE` | `/write/mcp` | Write MCP (Streamable HTTP) |
| `POST/GET/DELETE` | `/mcp` | Internal full MCP (API key only) |
## Backups
### PostgreSQL
```bash
# Backup
docker exec ctxd-postgres pg_dump -U ctxd ctxd > backup_$(date +%Y%m%d).sql
# Restore
cat backup_YYYYMMDD.sql | docker exec -i ctxd-postgres psql -U ctxd ctxd
```
### Snapshots
CTXD automatically takes point-in-time snapshots before each context update. Snapshots are stored as files in `/data/snapshots/<project_id>/` and rotated (min 5, max 25 per project).
## Migrating from SQLite to PostgreSQL
If you started with SQLite and want to move to PostgreSQL:
```bash
# 1. Start PostgreSQL
docker compose up -d postgres
# 2. Run the migration script (reads from /data/ctxd.db, writes to DATABASE_URL)
docker exec dossier python3 -m ctxd.migrate_sqlite_to_pg
# 3. Set DATABASE_URL in .env and restart
docker compose up -d dossier
```
The migration handles all tables, rebuilds the FTS index, and skips orphaned rows with FK violations.
## Project Structure
```
/mnt/ai-storage/projects/ctxd/
├── .env # Production environment (gitignored)
├── .env.example # Template (committed)
├── .gitignore
├── SKILL.md # LLM client guide (canonical source)
├── README.md # This file
├── data/ # Runtime data (gitignored)
│ ├── ctxd.yaml # Fallback config (env vars take precedence)
│ ├── ctxd.db # SQLite DB (if no DATABASE_URL)
│ ├── pg/ # PostgreSQL data volume
│ ├── oauth_state.json # OAuth clients, codes, tokens
│ ├── web_sessions.json # Web UI sessions
│ └── snapshots/ # Point-in-time backups
└── app/ # Application source
├── docker-compose.yml
├── Dockerfile
├── pyproject.toml
├── .env # Same as root .env (symlinked or copied)
└── src/ctxd/
├── __init__.py
├── __main__.py # CLI/daemon entry point
├── config.py # Env-driven config with yaml fallback
├── db.py # Database layer (PostgreSQL + SQLite)
├── schema.sql # PostgreSQL schema
├── schema_sqlite.sql # SQLite schema (fallback)
├── server.py # ASGI app: HTTP + MCP + OAuth
├── cli.py # CLI commands
├── ui.html # Web UI dashboard
├── landing.html # Public landing page
├── auth_password.py # PBKDF2 password hashing
└── migrate_sqlite_to_pg.py # One-time migration script
```
## Development
### Local Development (SQLite, no Docker)
```bash
cd app
pip install -e ".[mcp]"
export CTXD_HOME=./dev-data
python -m ctxd init
python -m ctxd
# → http://localhost:9091
```
### Rebuilding After Code Changes
```bash
cd app
docker compose build
docker compose up -d --no-build
# Verify:
curl http://localhost:9091/status
docker logs dossier --tail 20
```
### Key Conventions
- **Metadata headers** are dynamically generated on read, never stored in the DB
- **File paths** are normalized to uppercase with `.MD` extension
- **`CONTEXT.MD`** is the minimum required file — cannot be deleted from any project
- **Version checking** uses `base_version` parameter — mismatches return `409 conflict`
- **Audit log** is append-only at the application layer (every operation is logged)
- **OAuth state** (`oauth_state.json`) and **web sessions** (`web_sessions.json`) are file-based, not in PostgreSQL
## Troubleshooting
### Login fails with "invalid credentials"
```bash
# Reset admin password
docker exec dossier dossier user-set-password admin -p "new-password"
```
If the password contains special characters, use the quoting that matches:
- `'` in password → use double quotes (`"`)
- `$`, `` ` ``, `!` in password → use single quotes (`'`)
### Login works on LAN but not on public host
The public host (`https://ctxd.yourdomain.com`) requires Traefik to route `/auth/login` and `/auth/me`. Check your Traefik router rule includes all paths (use `!Path(`/mcp`)` to block only the internal MCP).
### MCP connection fails
1. Check OAuth discovery: `curl https://ctxd.yourdomain.com/.well-known/oauth-authorization-server`
2. Check MCP endpoint: `curl -o /dev/null -w '%{http_code}' https://ctxd.yourdomain.com/readonly/mcp` → should be `401` (not `404`)
3. If `404`: Traefik isn't routing `/readonly/mcp` — update the router rule
4. If `401`: auth is working — check OAuth token scope and expiry
### PostgreSQL connection fails
```bash
# Check PG is running
docker compose ps postgres
# Check connection
docker exec dossier python3 -c "
import os
import psycopg
conn = psycopg.connect(os.environ['DATABASE_URL'])
print('Connected:', conn.info.server_version)
"
# If password mismatch (PG data volume initialized with different password):
docker exec ctxd-postgres psql -U ctxd -c "ALTER USER ctxd PASSWORD 'new-password'"
# Then update .env with the new password
```
### Container keeps restarting
```bash
docker logs dossier --tail 30
```
Common causes:
- `DATABASE_URL` password doesn't match what PG was initialized with
- `OAUTH_ENABLED=true` but `OAUTH_ISSUER` is empty
- Missing `CTXD_API_KEY` when `CTXD_AUTH_ENABLED=true`
## License
MIT
+180
View File
@@ -0,0 +1,180 @@
---
name: ctxd-client
description: Use when an LLM agent needs to read or update project context via the Context Dossier (CTXD) daemon. Covers OAuth authentication, MCP tool discovery, read-only and write endpoints, and the workflow for keeping project context current.
version: 1.0.0
author: Hermes Agent
license: MIT
platforms:
- linux
- macos
metadata:
hermes:
tags:
- context-management
- ctxd
- mcp-client
- oauth
- project-context
related_skills:
- project-context-management
- native-mcp
---
# CTXD Client — LLM Agent Guide
Use when an LLM agent (Claude Desktop, Codex CLI, Hermes, custom harness) needs to read or update project context stored in a Context Dossier (CTXD) daemon.
## Overview
CTXD is a single source of truth for multi-harness project context. It exposes:
- **Read-only MCP** (`/readonly/mcp`) — OAuth bearer, `ctxd.read` scope, Streamable HTTP
- **Write MCP** (`/write/mcp`) — OAuth bearer, `ctxd.write` scope, Streamable HTTP
- **Web UI** (LAN `:9091` or public host) — per-user password login, admin panel
- **Internal full MCP** (`/mcp`) — shared API key (Hermes/automation only), Streamable HTTP
Public host: `https://ctxd.cubecraftcreations.com` (OAuth + MCP + landing page + dashboard).
## Connection URLs
| Surface | URL | Auth |
|---------|-----|------|
| Read-only MCP | `https://ctxd.cubecraftcreations.com/readonly/mcp` | OAuth `ctxd.read` |
| Write MCP | `https://ctxd.cubecraftcreations.com/write/mcp` | OAuth `ctxd.write` |
| OAuth discovery | `https://ctxd.cubecraftcreations.com/.well-known/oauth-authorization-server` | Public |
| DCR registration | `POST https://ctxd.cubecraftcreations.com/oauth/register` | Public |
| Landing page | `https://ctxd.cubecraftcreations.com/` | Public |
## OAuth Flow
1. **Discover** the authorization server metadata at `/.well-known/oauth-authorization-server`
2. **Register** a client via `POST /oauth/register` (DCR) with your redirect URI
3. **Authorize** — open `/oauth/authorize` in a browser; an admin must approve
4. **Exchange** the authorization code for tokens at `POST /oauth/token`
5. **Use** the access token as `Authorization: Bearer *** on MCP Streamable HTTP connections
### Scopes
| Scope | Grants |
|-------|--------|
| `ctxd.read` | `list_projects`, `get_project_context`, `search_context`, `get_project_tags`, `list_files`, `get_file`, `get_client_guide` |
| `ctxd.write` | `update_file`, `set_project_tags`, `sync_to_project`, `get_client_guide` |
Request both scopes for full read+write: `scope=ctxd.read ctxd.write`
### Redirect URIs by Platform
| Platform | Redirect URI |
|----------|-------------|
| Claude Desktop | `https://claude.ai/api/mcp/auth_callback` |
| Claude Code | `http://localhost:5555/oauth/callback` |
| Codex CLI | `http://localhost:7777/oauth/callback` |
| Custom | Your app's documented OAuth callback |
### Token Lifetime
Access tokens expire per server config (default ~1 hour). Refresh tokens are issued alongside access tokens. Use `POST /oauth/token` with `grant_type=refresh_token` to rotate.
## MCP Tools
### First Call — Always
| Tool | Description |
|------|-------------|
| `get_client_guide` | Return the locked LLM-CLIENT.MD guide. **Call this first in every session.** No arguments. Covers OAuth, MCP tools, read/write endpoints, version-checked updates, and error handling. |
The guide lives in the `ctxd-docs` project as `LLM-CLIENT.MD`. It is **locked** — cannot be updated or deleted by any MCP/API client. If you need the guide updated, ask an admin to edit it via the Web UI.
### Read-only (`/readonly/mcp`)
| Tool | Description |
|------|-------------|
| `list_projects` | All projects with version numbers |
| `get_project_context` | Compiled markdown of all context files for a project |
| `search_context` | FTS5 full-text search across all projects |
| `get_project_tags` | Metadata tags for a project |
| `list_files` | All context files in a project (multi-file mode) |
| `get_file` | Single file with metadata header |
### Write (`/write/mcp`)
| Tool | Description |
|------|-------------|
| `update_file` | Update a single context file with optimistic version checking |
| `set_project_tags` | Replace all metadata tags for a project |
| `sync_to_project` | Write CONTEXT.MD as AGENTS.md + symlinks to project root |
## Workflow: Reading Context
```
1. get_client_guide() → read the locked client guide first
2. list_projects() → find the project slug
3. get_project_context(project_id) → read compiled context
or list_files() + get_file() → read individual files
```
The compiled view returns all files concatenated with `## FILENAME` headers and a metadata block at the top.
## Workflow: Updating Context
```
1. get_file(project_id, file_path) → get current content + version
2. update_file(project_id, file_path, content, base_version)
→ base_version must match current version (optimistic locking)
→ returns {"ok": true, "version": N+1} or {"ok": false, "error": "version_conflict"}
3. On conflict: re-read, merge, retry
```
### What to Update
- **CONTEXT.MD** — canonical project overview (synced as AGENTS.md to repos)
- **DECISIONS.MD** — architecture decisions, rationale
- **RUNBOOKS.MD** — deploy, troubleshoot, operate procedures
- **PROMPTS.MD** — project-specific prompts for different harnesses
- **GLOSSARY.MD** — project-specific terms, acronyms
### What NOT to Put in Context
- Session progress logs (use session_search or audit trail)
- Temporary TODO state
- Single-session debugging notes
- Secrets, keys, tokens
## Locked Files
| File | Protection |
|------|-----------|
| `CONTEXT.MD` | Cannot delete — canonical synced file |
| `LLM-CLIENT.MD` | Cannot update or delete — locked client guide |
Both return `403 cannot_update_locked` on PUT and `400 cannot_delete_context` on DELETE.
## Discipline
1. **Call `get_client_guide()` first** in every session
2. **Read context at the start of every session** for the project you're working on
3. **Update context immediately** when a durable fact changes — not "next session"
4. **Use `base_version`** to avoid overwriting concurrent edits
5. **Don't put session progress in project context** — that's what session history is for
## Error Handling
| Error | Meaning | Action |
|-------|---------|--------|
| `version_conflict` | Someone else updated the file | Re-read, merge, retry with new `base_version` |
| `not_found` | Project or file doesn't exist | Check slug spelling; create project via Web UI |
| `invalid_grant` | OAuth token expired or wrong scope | Refresh token or re-authorize with correct scope |
| `forbidden` | Wrong endpoint for the tool | Write tools only on `/write/mcp`; read tools on `/readonly/mcp` |
| `cannot_delete_context` | Tried to delete CONTEXT.MD or LLM-CLIENT.MD | Protected file — update content instead |
| `cannot_update_locked` | Tried to update LLM-CLIENT.MD | Locked guide — ask admin to update via Web UI |
## Verification Checklist
- [ ] OAuth client registered with correct redirect URI
- [ ] Admin approved the authorization (browser session or approval key)
- [ ] Access token obtained with `ctxd.read` (and `ctxd.write` for updates)
- [ ] `get_client_guide` returns the LLM client guide
- [ ] `list_projects` returns expected projects
- [ ] `get_project_context` returns compiled markdown with metadata header
- [ ] `update_file` succeeds with correct `base_version`
- [ ] Audit log shows the write operation
+64
View File
@@ -0,0 +1,64 @@
# ════════════════════════════════════════════════════════════════════
# CTXD — Context Dossier Environment Configuration
# Copy to .env and fill in your values. All variables are optional
# unless running in Docker (DATABASE_URL, CTXD_API_KEY, OAUTH_* required).
#
# Precedence: env var > ctxd.yaml (in /data) > built-in defaults
# ════════════════════════════════════════════════════════════════════
# ── Database ────────────────────────────────────────────────────────
# Full PostgreSQL connection string. If empty, CTXD falls back to SQLite
# at /data/ctxd.db (or CTXD_HOME/ctxd.db).
DATABASE_URL=postgresql://ctxd:ctxd_local_dev@postgres:5432/ctxd
# Postgres container settings (only used when running the bundled PG)
POSTGRES_USER=ctxd
POSTGRES_PASSWORD=ctxd_local_dev
POSTGRES_DB=ctxd
# ── Server ──────────────────────────────────────────────────────────
# Bind address and port inside the container
CTXD_HOST=0.0.0.0
CTXD_PORT=9091
# Data directory (inside container)
CTXD_HOME=/data
# Uvicorn log level: debug, info, warning, error
LOG_LEVEL=info
# ── Auth ────────────────────────────────────────────────────────────
# Enable authentication globally
CTXD_AUTH_ENABLED=true
# Shared API key for Hermes/internal MCP + HTTP auth
CTXD_API_KEY=
# Legacy read-only key for /readonly/sse ?key= migration
CTXD_EXTERNAL_READONLY_KEY=
# ── OAuth ───────────────────────────────────────────────────────────
# Enable the OAuth authorization server
OAUTH_ENABLED=true
# Public URL of the server (used in OAuth discovery metadata)
OAUTH_ISSUER=https://ctxd.example.com
# Fallback approval key for /oauth/authorize
OAUTH_APPROVAL_KEY=
# Which user ID to attribute OAuth approvals to
OAUTH_APPROVAL_USER_ID=admin
# Token lifetimes in seconds
OAUTH_ACCESS_TOKEN_TTL=3600
OAUTH_REFRESH_TOKEN_TTL=2592000
# ── Web Sessions ────────────────────────────────────────────────────
# Session cookie lifetime in seconds (default: 7 days)
WEB_SESSION_TTL=604800
# ── Snapshots ───────────────────────────────────────────────────────
# Min/max snapshots retained per project before rotation
SNAPSHOT_MIN_KEEP=5
SNAPSHOT_MAX_KEEP=25
+25 -1
View File
@@ -1,6 +1,25 @@
name: ctxd name: ctxd
# Docker Compose reads .env automatically for variable substitution.
# See .env.example for all available variables.
services: services:
postgres:
image: postgres:16-alpine
container_name: ctxd-postgres
restart: unless-stopped
environment:
POSTGRES_DB: ${POSTGRES_DB:-ctxd}
POSTGRES_USER: ${POSTGRES_USER:-ctxd}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-ctxd_local_dev}
volumes:
- ${CTXD_PG_DATA:-/mnt/ai-storage/projects/ctxd/data/pg}:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-ctxd}"]
interval: 5s
timeout: 5s
retries: 5
dossier: dossier:
build: build:
context: . context: .
@@ -8,10 +27,15 @@ services:
container_name: dossier container_name: dossier
restart: unless-stopped restart: unless-stopped
ports: ports:
- "9091:9091" - "${CTXD_PORT:-9091}:${CTXD_PORT:-9091}"
volumes: volumes:
- /mnt/ai-storage/projects/ctxd/data:/data:rw - /mnt/ai-storage/projects/ctxd/data:/data:rw
- /mnt/ai-storage/projects:/projects:ro - /mnt/ai-storage/projects:/projects:ro
- /home/overseer:/host:ro - /home/overseer:/host:ro
env_file:
- .env
environment: environment:
- CTXD_HOME=/data - CTXD_HOME=/data
depends_on:
postgres:
condition: service_healthy
+6
View File
@@ -3,6 +3,12 @@ name = "context-dossier"
version = "0.2.0" version = "0.2.0"
description = "Context Dossier — single source of truth for multi-harness project context" description = "Context Dossier — single source of truth for multi-harness project context"
requires-python = ">=3.11" requires-python = ">=3.11"
dependencies = [
"psycopg[binary]>=3.1",
"mcp>=1.28",
"uvicorn>=0.30",
"pyyaml>=6.0",
]
[project.scripts] [project.scripts]
dossier = "ctxd:cli_entry" dossier = "ctxd:cli_entry"
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+40
View File
@@ -0,0 +1,40 @@
"""Password hashing for CTXD user accounts (stdlib only)."""
from __future__ import annotations
import hashlib
import secrets
PBKDF2_ITERATIONS = 600_000
SCHEME = "pbkdf2_sha256"
def hash_password(password: str) -> str:
if not password:
raise ValueError("password required")
salt = secrets.token_hex(16)
digest = hashlib.pbkdf2_hmac(
"sha256",
password.encode("utf-8"),
salt.encode("utf-8"),
PBKDF2_ITERATIONS,
)
return f"{SCHEME}${PBKDF2_ITERATIONS}${salt}${digest.hex()}"
def verify_password(password: str, token_hash: str | None) -> bool:
if not password or not token_hash:
return False
try:
scheme, iters_s, salt, expected_hex = token_hash.split("$", 3)
if scheme != SCHEME:
return False
digest = hashlib.pbkdf2_hmac(
"sha256",
password.encode("utf-8"),
salt.encode("utf-8"),
int(iters_s),
)
return secrets.compare_digest(digest.hex(), expected_hex)
except (ValueError, TypeError):
return False
+94 -2
View File
@@ -309,16 +309,82 @@ def cmd_user_create(args):
"""Create a new user.""" """Create a new user."""
conn = _db.init_db(CtxConfig.from_home(args.home)) conn = _db.init_db(CtxConfig.from_home(args.home))
try: try:
_db.user_create(conn, args.user_id, args.display_name, args.role) _db.user_create(conn, args.user_id, args.display_name, args.role, password=getattr(args, "password", None))
conn.commit() conn.commit()
print(f"✓ User '{args.user_id}' created.") print(f"✓ User '{args.user_id}' created.")
except Exception as e: except Exception as e:
conn.rollback() conn.rollback()
print(f"Error: {e}") print(f" {e}")
sys.exit(1)
finally: finally:
conn.close() conn.close()
def cmd_user_set_password(args):
"""Set or reset a user's Web UI password."""
conn = _db.init_db(CtxConfig.from_home(args.home))
try:
if _db.user_get(conn, args.user_id) is None:
print(f"✗ User '{args.user_id}' not found.")
sys.exit(1)
_db.user_set_password(conn, args.user_id, args.password)
conn.commit()
print(f"✓ Password set for '{args.user_id}'.")
except Exception as e:
conn.rollback()
print(f"{e}")
sys.exit(1)
finally:
conn.close()
def cmd_oauth_client_create(args):
"""Register an OAuth client for Claude (or other MCP connectors)."""
from .server import OAuthStore, CLAUDE_MCP_REDIRECT_URI
cfg = CtxConfig.from_home(args.home)
store = OAuthStore(cfg)
redirects = args.redirect_uri or [CLAUDE_MCP_REDIRECT_URI]
if isinstance(redirects, str):
redirects = [redirects]
client = store.register_client({
"client_name": args.name or "Claude MCP Client",
"redirect_uris": redirects,
})
issuer = (cfg.oauth_issuer or "").rstrip("/") or "https://ctxd.cubecraftcreations.com"
print(json.dumps({
"client_id": client["client_id"],
"client_secret": client["client_secret"],
"client_name": client.get("client_name"),
"redirect_uris": client.get("redirect_uris"),
"connector_url": f"{issuer}/readonly/sse",
"authorization_server": issuer,
"note": "Claude usually registers via POST /oauth/register automatically; save client_secret now — it is not shown again.",
}, indent=2))
def cmd_oauth_client_list(args):
"""List OAuth clients (no secrets)."""
from .server import OAuthStore
cfg = CtxConfig.from_home(args.home)
store = OAuthStore(cfg)
for c in store.list_clients_public():
print(f"{c.get('client_id')} {c.get('client_name', '')} redirects={c.get('redirect_uris')}")
def cmd_oauth_client_revoke(args):
"""Revoke an OAuth client and invalidate its tokens."""
from .server import OAuthStore
cfg = CtxConfig.from_home(args.home)
store = OAuthStore(cfg)
if not store.revoke_client(args.client_id):
print(f"✗ client not found: {args.client_id}")
sys.exit(1)
print(f"✓ Revoked OAuth client {args.client_id}")
def cmd_import_vault(args): def cmd_import_vault(args):
"""Import context from an existing vault (e.g., OpenClawVault).""" """Import context from an existing vault (e.g., OpenClawVault)."""
cfg = CtxConfig.from_home(args.home) cfg = CtxConfig.from_home(args.home)
@@ -477,6 +543,32 @@ def build_parser() -> argparse.ArgumentParser:
sp.add_argument("user_id") sp.add_argument("user_id")
sp.add_argument("--display-name", "-n", required=True) sp.add_argument("--display-name", "-n", required=True)
sp.add_argument("--role", "-r", default="contributor", choices=["admin", "contributor", "service"]) sp.add_argument("--role", "-r", default="contributor", choices=["admin", "contributor", "service"])
sp.add_argument("--password", help="Optional Web UI login password")
sp.add_argument("--home")
# user set-password
sp = sub.add_parser("user-set-password", help="Set a user's Web UI password")
sp.set_defaults(func=cmd_user_set_password)
sp.add_argument("user_id")
sp.add_argument("--password", "-p", required=True)
sp.add_argument("--home")
# oauth-client-create
sp = sub.add_parser("oauth-client-create", help="Register OAuth client (Claude MCP); prints client_id and client_secret")
sp.set_defaults(func=cmd_oauth_client_create)
sp.add_argument("--name", "-n", default="Claude MCP Client", help="Client display name")
sp.add_argument("--redirect-uri", action="append", dest="redirect_uri", help="Redirect URI (default: Claude MCP callback; repeat for multiple)")
sp.add_argument("--home")
# oauth-client-list
sp = sub.add_parser("oauth-client-list", help="List OAuth clients (no secrets)")
sp.set_defaults(func=cmd_oauth_client_list)
sp.add_argument("--home")
# oauth-client-revoke
sp = sub.add_parser("oauth-client-revoke", help="Revoke OAuth client and invalidate its tokens")
sp.set_defaults(func=cmd_oauth_client_revoke)
sp.add_argument("client_id", help="client_id to revoke (ctxd_…)")
sp.add_argument("--home") sp.add_argument("--home")
# import-vault # import-vault
+110 -11
View File
@@ -1,13 +1,39 @@
""" """
Configuration for ctxd — context daemon. Configuration for ctxd — context daemon.
Environment variables take precedence over ctxd.yaml, which takes precedence
over built-in defaults. This allows fully env-driven deployments (Docker,
Compose, Kubernetes) while preserving file-based config for local dev.
""" """
import os import os
from pathlib import Path from pathlib import Path
def _env_bool(key: str, fallback: bool = False) -> bool:
val = os.environ.get(key, "")
if not val:
return fallback
return val.lower() in ("1", "true", "yes", "on")
def _env_int(key: str, fallback: int) -> int:
val = os.environ.get(key, "")
if not val:
return fallback
try:
return int(val)
except ValueError:
return fallback
def _env_str(key: str, fallback: str = "") -> str:
return os.environ.get(key, fallback)
# Default home directory (~/.ctx) — overridable via CTXD_HOME env var # Default home directory (~/.ctx) — overridable via CTXD_HOME env var
DEFAULT_HOME = Path(os.environ.get("CTXD_HOME", Path.home() / ".ctx")) DEFAULT_HOME = Path(os.environ.get("CTXD_HOME", Path.home() / ".ctx"))
# Defaults for ctxd.yaml # Built-in defaults (lowest precedence)
DEFAULT_CONFIG = { DEFAULT_CONFIG = {
"server": { "server": {
"host": "0.0.0.0", "host": "0.0.0.0",
@@ -20,6 +46,18 @@ DEFAULT_CONFIG = {
"auth": { "auth": {
"enabled": False, "enabled": False,
"api_key": "", "api_key": "",
"external_readonly_key": "",
},
"oauth": {
"enabled": False,
"issuer": "",
"approval_key": "",
"approval_user_id": "admin",
"access_token_ttl_seconds": 3600,
"refresh_token_ttl_seconds": 2592000,
},
"web_sessions": {
"ttl_seconds": 604800,
}, },
"seed": { "seed": {
"admin_user": "admin", "admin_user": "admin",
@@ -31,7 +69,10 @@ DEFAULT_CONFIG = {
class CtxConfig: class CtxConfig:
"""Holds resolved paths and config for a ctxd runtime.""" """Holds resolved paths and config for a ctxd runtime.
Precedence: env var > ctxd.yaml value > built-in default.
"""
def __init__(self, home: Path | str | None = None, config: dict | None = None): def __init__(self, home: Path | str | None = None, config: dict | None = None):
resolved = Path(home) if home else DEFAULT_HOME resolved = Path(home) if home else DEFAULT_HOME
@@ -59,31 +100,87 @@ class CtxConfig:
def config_path(self) -> Path: def config_path(self) -> Path:
return self.home / "ctxd.yaml" return self.home / "ctxd.yaml"
# ── Config accessors ────────────────────────────────────────── @property
def oauth_state_path(self) -> Path:
return self.home / "oauth_state.json"
@property
def web_sessions_path(self) -> Path:
return self.home / "web_sessions.json"
# ── Database ──────────────────────────────────────────────────
@property
def database_url(self) -> str:
return _env_str("DATABASE_URL")
@property
def use_postgres(self) -> bool:
return bool(self.database_url)
# ── Server ────────────────────────────────────────────────────
@property @property
def host(self) -> str: def host(self) -> str:
return self._cfg.get("server", {}).get("host", "127.0.0.1") return _env_str("CTXD_HOST", self._cfg.get("server", {}).get("host", "0.0.0.0"))
@property @property
def port(self) -> int: def port(self) -> int:
return self._cfg.get("server", {}).get("port", 9091) return _env_int("CTXD_PORT", self._cfg.get("server", {}).get("port", 9091))
@property
def log_level(self) -> str:
return _env_str("LOG_LEVEL", "info")
# ── Snapshots ─────────────────────────────────────────────────
@property @property
def min_snapshots(self) -> int: def min_snapshots(self) -> int:
return self._cfg.get("snapshots", {}).get("min_keep", 5) return _env_int("SNAPSHOT_MIN_KEEP", self._cfg.get("snapshots", {}).get("min_keep", 5))
@property @property
def max_snapshots(self) -> int: def max_snapshots(self) -> int:
return self._cfg.get("snapshots", {}).get("max_keep", 25) return _env_int("SNAPSHOT_MAX_KEEP", self._cfg.get("snapshots", {}).get("max_keep", 25))
# ── Auth ────────────────────────────────────────────────────── # ── Auth ──────────────────────────────────────────────────────
@property @property
def auth_enabled(self) -> bool: def auth_enabled(self) -> bool:
return self._cfg.get("auth", {}).get("enabled", False) return _env_bool("CTXD_AUTH_ENABLED", self._cfg.get("auth", {}).get("enabled", False))
@property @property
def api_key(self) -> str: def api_key(self) -> str:
return self._cfg.get("auth", {}).get("api_key", "") return _env_str("CTXD_API_KEY", self._cfg.get("auth", {}).get("api_key", ""))
@property
def external_readonly_key(self) -> str:
return _env_str("CTXD_EXTERNAL_READONLY_KEY", self._cfg.get("auth", {}).get("external_readonly_key", ""))
# ── OAuth ─────────────────────────────────────────────────────
@property
def oauth_enabled(self) -> bool:
return _env_bool("OAUTH_ENABLED", self._cfg.get("oauth", {}).get("enabled", False))
@property
def oauth_issuer(self) -> str:
return _env_str("OAUTH_ISSUER", self._cfg.get("oauth", {}).get("issuer", ""))
@property
def oauth_approval_key(self) -> str:
return _env_str("OAUTH_APPROVAL_KEY", self._cfg.get("oauth", {}).get("approval_key", ""))
@property
def oauth_approval_user_id(self) -> str:
return _env_str("OAUTH_APPROVAL_USER_ID", self._cfg.get("oauth", {}).get("approval_user_id", "admin"))
@property
def oauth_access_token_ttl_seconds(self) -> int:
return _env_int("OAUTH_ACCESS_TOKEN_TTL", self._cfg.get("oauth", {}).get("access_token_ttl_seconds", 3600))
@property
def oauth_refresh_token_ttl_seconds(self) -> int:
return _env_int("OAUTH_REFRESH_TOKEN_TTL", self._cfg.get("oauth", {}).get("refresh_token_ttl_seconds", 2592000))
# ── Web Sessions ───────────────────────────────────────────────
@property
def web_session_ttl_seconds(self) -> int:
return _env_int("WEB_SESSION_TTL", self._cfg.get("web_sessions", {}).get("ttl_seconds", 604800))
# ── Bootstrap ───────────────────────────────────────────────── # ── Bootstrap ─────────────────────────────────────────────────
def ensure_dirs(self): def ensure_dirs(self):
@@ -93,7 +190,9 @@ class CtxConfig:
@classmethod @classmethod
def from_home(cls, home: Path | str | None = None) -> "CtxConfig": def from_home(cls, home: Path | str | None = None) -> "CtxConfig":
"""Load from ctxd.yaml if it exists, otherwise use defaults.""" """Load from ctxd.yaml if it exists, otherwise use defaults.
Env vars always take precedence over yaml values at read time.
"""
home = Path(home).resolve() if home else DEFAULT_HOME home = Path(home).resolve() if home else DEFAULT_HOME
cfg_path = home / "ctxd.yaml" cfg_path = home / "ctxd.yaml"
if cfg_path.exists(): if cfg_path.exists():
@@ -104,7 +203,7 @@ class CtxConfig:
return cls(home=str(home)) return cls(home=str(home))
def save(self): def save(self):
"""Write config to ctxd.yaml.""" """Write config to ctxd.yaml. Rarely needed in env-driven deployments."""
import yaml import yaml
self.ensure_dirs() self.ensure_dirs()
with open(self.config_path, "w") as f: with open(self.config_path, "w") as f:
+403 -110
View File
@@ -1,7 +1,10 @@
""" """
Database layer for ctxd — schema init, CRUD, workspace fork/merge, FTS, audit. Database layer for ctxd — schema init, CRUD, workspace fork/merge, FTS, audit.
All public methods take a sqlite3.Connection as the first argument so callers All public methods take a connection as the first argument so callers
control transactions. This module is stateless — all state is in SQLite. control transactions. This module is stateless — all state is in the DB.
Supports both PostgreSQL (via DATABASE_URL) and SQLite (fallback for local dev).
The public API is identical regardless of backend.
""" """
import json import json
import sqlite3 import sqlite3
@@ -11,15 +14,58 @@ from pathlib import Path
from typing import Any from typing import Any
from .config import CtxConfig from .config import CtxConfig
from .auth_password import hash_password, verify_password
# ── Schema ──────────────────────────────────────────────────────────────────── # ── Schema paths ─────────────────────────────────────────────────────────────
SCHEMA_PATH = Path(__file__).parent / "schema.sql" SCHEMA_PG_PATH = Path(__file__).parent / "schema.sql"
SCHEMA_SQLITE_PATH = Path(__file__).parent / "schema_sqlite.sql"
def init_db(cfg: CtxConfig) -> sqlite3.Connection: def init_db(cfg: CtxConfig):
"""Create ~/.ctx/ dirs + initialize the database from schema.sql.""" """Create ~/.ctx/ dirs + initialize the database.
If DATABASE_URL is set in the environment, connects to PostgreSQL.
Otherwise falls back to SQLite for local development.
Returns a connection object (psycopg.Connection or sqlite3.Connection).
"""
cfg.ensure_dirs() cfg.ensure_dirs()
if cfg.use_postgres:
return _init_pg(cfg)
else:
return _init_sqlite(cfg)
def _init_pg(cfg: CtxConfig):
"""Initialize PostgreSQL database from schema.sql."""
import psycopg
from psycopg.rows import dict_row
conn = psycopg.connect(cfg.database_url, row_factory=dict_row)
# Keep autocommit=False (default) — we use explicit commit() calls
# Check if schema is already initialized by looking for the users table
cur = conn.execute(
"SELECT EXISTS (SELECT FROM pg_tables WHERE schemaname = 'public' AND tablename = 'users')"
)
schema_exists = cur.fetchone()["exists"]
if not schema_exists:
with open(SCHEMA_PG_PATH) as f:
schema_sql = f.read()
# PostgreSQL can execute multiple statements in a single execute() call
conn.execute(schema_sql)
conn.commit()
else:
# Run migrations if needed
_migrate_pg(conn)
return conn
def _init_sqlite(cfg: CtxConfig) -> sqlite3.Connection:
"""Initialize SQLite database (fallback for local dev)."""
fresh = not cfg.db_path.exists() fresh = not cfg.db_path.exists()
conn = sqlite3.connect(str(cfg.db_path)) conn = sqlite3.connect(str(cfg.db_path))
conn.row_factory = sqlite3.Row conn.row_factory = sqlite3.Row
@@ -27,7 +73,7 @@ def init_db(cfg: CtxConfig) -> sqlite3.Connection:
conn.execute("PRAGMA foreign_keys = ON") conn.execute("PRAGMA foreign_keys = ON")
if fresh: if fresh:
with open(SCHEMA_PATH) as f: with open(SCHEMA_SQLITE_PATH) as f:
conn.executescript(f.read()) conn.executescript(f.read())
else: else:
# Migration: add metadata_tags column if it doesn't exist # Migration: add metadata_tags column if it doesn't exist
@@ -35,62 +81,211 @@ def init_db(cfg: CtxConfig) -> sqlite3.Connection:
conn.execute("ALTER TABLE projects ADD COLUMN metadata_tags TEXT DEFAULT '[]'") conn.execute("ALTER TABLE projects ADD COLUMN metadata_tags TEXT DEFAULT '[]'")
conn.commit() conn.commit()
except sqlite3.OperationalError: except sqlite3.OperationalError:
pass # column already exists pass
try:
conn.execute("ALTER TABLE users ADD COLUMN active INTEGER NOT NULL DEFAULT 1")
conn.commit()
except sqlite3.OperationalError:
pass
return conn return conn
def _migrate_pg(conn):
"""Run PostgreSQL migrations for existing databases."""
try:
conn.execute("ALTER TABLE projects ADD COLUMN IF NOT EXISTS metadata_tags TEXT DEFAULT '[]'")
conn.commit()
except Exception:
pass
try:
conn.execute("ALTER TABLE users ADD COLUMN IF NOT EXISTS active BOOLEAN NOT NULL DEFAULT TRUE")
conn.commit()
except Exception:
pass
def now() -> str: def now() -> str:
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
def _is_pg(conn) -> bool:
"""Check if connection is a PostgreSQL connection."""
return not isinstance(conn, sqlite3.Connection)
# ── Helpers ─────────────────────────────────────────────────────────────────── # ── Helpers ───────────────────────────────────────────────────────────────────
def _row_to_dict(row: sqlite3.Row | None) -> dict | None: def _row_to_dict(row) -> dict | None:
if row is None: if row is None:
return None return None
if isinstance(row, dict):
return dict(row)
return dict(row) return dict(row)
def _ph(conn, n: int = 1) -> str:
"""Return placeholder string for the current backend.
PostgreSQL uses %s, SQLite uses ?.
"""
if _is_pg(conn):
return ", ".join(["%s"] * n)
else:
return ", ".join(["?"] * n)
# ── Users ───────────────────────────────────────────────────────────────────── # ── Users ─────────────────────────────────────────────────────────────────────
def user_create(conn, user_id: str, display_name: str, role: str = "contributor"):
def user_create(conn, user_id: str, display_name: str, role: str = "contributor", password: str | None = None, active: bool = True):
token_hash = hash_password(password) if password else None
ph = _ph(conn, 5)
if _is_pg(conn):
active_val = active
else:
active_val = 1 if active else 0
conn.execute( conn.execute(
"INSERT INTO users (user_id, display_name, role) VALUES (?, ?, ?)", f"INSERT INTO users (user_id, display_name, role, token_hash, active) VALUES ({ph})",
(user_id, display_name, role), (user_id, display_name, role, token_hash, active_val),
)
def user_update(conn, user_id: str, *, display_name: str | None = None, role: str | None = None, active: bool | None = None) -> bool:
user = user_get(conn, user_id)
if not user:
return False
fields = []
values: list[Any] = []
ph = _ph(conn, 1)
if display_name is not None:
fields.append(f"display_name = {ph}")
values.append(display_name)
if role is not None:
fields.append(f"role = {ph}")
values.append(role)
if active is not None:
if _is_pg(conn):
fields.append(f"active = {ph}")
values.append(active)
else:
fields.append(f"active = {ph}")
values.append(1 if active else 0)
if not fields:
return True
if _is_pg(conn):
fields.append("updated_at = to_char(now() at time zone 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS\"Z\"')")
else:
fields.append("updated_at = strftime('%Y-%m-%dT%H:%M:%SZ', 'now')")
values.append(user_id)
conn.execute(f"UPDATE users SET {', '.join(fields)} WHERE user_id = {ph}", values)
return True
def user_delete(conn, user_id: str) -> dict:
if user_get(conn, user_id) is None:
return {"ok": False, "error": "not_found"}
ph = _ph(conn, 1)
try:
conn.execute(f"DELETE FROM users WHERE user_id = {ph}", (user_id,))
return {"ok": True}
except (sqlite3.IntegrityError, Exception) as e:
# Check if it's a foreign key violation
if _is_pg(conn):
import psycopg
if isinstance(e, psycopg.errors.ForeignKeyViolation):
return {"ok": False, "error": "user_has_references", "hint": "Inactivate the user instead of deleting."}
raise
return {"ok": False, "error": "user_has_references", "hint": "Inactivate the user instead of deleting."}
def user_set_password(conn, user_id: str, password: str):
ph = _ph(conn, 2)
if _is_pg(conn):
ts_expr = "to_char(now() at time zone 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS\"Z\"')"
else:
ts_expr = "strftime('%Y-%m-%dT%H:%M:%SZ', 'now')"
conn.execute(
f"UPDATE users SET token_hash = {ph}, updated_at = {ts_expr} WHERE user_id = {ph}",
(hash_password(password), user_id),
) )
def user_get(conn, user_id: str) -> dict | None: def user_get(conn, user_id: str) -> dict | None:
ph = _ph(conn, 1)
return _row_to_dict(conn.execute( return _row_to_dict(conn.execute(
"SELECT * FROM users WHERE user_id = ?", (user_id,) f"SELECT * FROM users WHERE user_id = {ph}", (user_id,)
).fetchone()) ).fetchone())
def user_get_ci(conn, user_id: str) -> dict | None:
"""Case-insensitive user lookup (matches idx_users_lower)."""
ph = _ph(conn, 1)
return _row_to_dict(conn.execute(
f"SELECT * FROM users WHERE LOWER(user_id) = LOWER({ph})", (user_id.strip(),)
).fetchone())
def user_authenticate(conn, user_id: str, password: str) -> dict | None:
user = user_get_ci(conn, user_id)
if not user:
return None
# Handle both PostgreSQL BOOLEAN and SQLite INTEGER for active
active = user.get("active")
if active is None:
active = True # Default to active
if isinstance(active, int) and active == 0:
return None
if isinstance(active, bool) and not active:
return None
if not user.get("token_hash"):
return None
if not verify_password(password, user.get("token_hash")):
return None
return user
def user_list(conn) -> list[dict]: def user_list(conn) -> list[dict]:
return [dict(r) for r in conn.execute("SELECT * FROM users ORDER BY user_id").fetchall()] if _is_pg(conn):
return [dict(r) for r in conn.execute("SELECT * FROM users ORDER BY user_id").fetchall()]
else:
return [dict(r) for r in conn.execute("SELECT * FROM users ORDER BY user_id").fetchall()]
# ── Projects ────────────────────────────────────────────────────────────────── # ── Projects ──────────────────────────────────────────────────────────────────
def project_create(conn, project_id: str, display_name: str, description: str = ""): def project_create(conn, project_id: str, display_name: str, description: str = ""):
ph3 = _ph(conn, 3)
ph1 = _ph(conn, 1)
conn.execute( conn.execute(
"INSERT INTO projects (project_id, display_name, description) VALUES (?, ?, ?)", f"INSERT INTO projects (project_id, display_name, description) VALUES ({ph3})",
(project_id, display_name, description), (project_id, display_name, description),
) )
# Also create empty shared context # Also create empty shared context
conn.execute( conn.execute(
"INSERT INTO project_context (project_id, content, version) VALUES (?, '', 0)", f"INSERT INTO project_context (project_id, content, version) VALUES ({ph1}, '', 0)",
(project_id,), (project_id,),
) )
def project_get(conn, project_id: str) -> dict | None: def project_get(conn, project_id: str) -> dict | None:
ph = _ph(conn, 1)
return _row_to_dict(conn.execute( return _row_to_dict(conn.execute(
"SELECT * FROM projects WHERE project_id = ?", (project_id,) f"SELECT * FROM projects WHERE project_id = {ph}", (project_id,)
).fetchone()) ).fetchone())
def project_delete(conn, project_id: str) -> dict:
"""Delete a project and all its dependent rows (cascades).
audit_log.project_id is SET NULL via FK ON DELETE SET NULL.
Returns {'ok': True} or {'ok': False, 'error': ...}."""
if project_get(conn, project_id) is None:
return {"ok": False, "error": "not_found"}
ph = _ph(conn, 1)
conn.execute(f"DELETE FROM projects WHERE project_id = {ph}", (project_id,))
return {"ok": True}
def project_list(conn) -> list[dict]: def project_list(conn) -> list[dict]:
return [dict(r) for r in conn.execute( return [dict(r) for r in conn.execute(
"SELECT project_id, display_name, description, shared_version FROM projects ORDER BY project_id" "SELECT project_id, display_name, description, shared_version FROM projects ORDER BY project_id"
@@ -98,23 +293,30 @@ def project_list(conn) -> list[dict]:
def project_set_sync_path(conn, project_id: str, sync_path: str | None): def project_set_sync_path(conn, project_id: str, sync_path: str | None):
conn.execute( ph = _ph(conn, 2)
"UPDATE projects SET sync_path = ?, auto_sync = 1 WHERE project_id = ?", if _is_pg(conn):
(sync_path, project_id), conn.execute(
) f"UPDATE projects SET sync_path = {ph.split(', ')[0]}, auto_sync = TRUE WHERE project_id = {ph.split(', ')[1]}",
(sync_path, project_id),
)
else:
conn.execute(
f"UPDATE projects SET sync_path = ?, auto_sync = 1 WHERE project_id = ?",
(sync_path, project_id),
)
def project_get_tags(conn, project_id: str) -> list[str]: def project_get_tags(conn, project_id: str) -> list[str]:
"""Get project metadata tags as a list of strings.""" """Get project metadata tags as a list of strings."""
ph = _ph(conn, 1)
row = conn.execute( row = conn.execute(
"SELECT metadata_tags FROM projects WHERE project_id = ?", (project_id,) f"SELECT metadata_tags FROM projects WHERE project_id = {ph}", (project_id,)
).fetchone() ).fetchone()
if row is None: if row is None:
return [] return []
tags = row["metadata_tags"] tags = row["metadata_tags"]
if not tags: if not tags:
return [] return []
import json
try: try:
return json.loads(tags) return json.loads(tags)
except (json.JSONDecodeError, TypeError): except (json.JSONDecodeError, TypeError):
@@ -122,10 +324,9 @@ def project_get_tags(conn, project_id: str) -> list[str]:
def project_set_tags(conn, project_id: str, tags: list[str]): def project_set_tags(conn, project_id: str, tags: list[str]):
"""Set project metadata tags from a list of strings.""" ph = _ph(conn, 2)
import json
conn.execute( conn.execute(
"UPDATE projects SET metadata_tags = ? WHERE project_id = ?", f"UPDATE projects SET metadata_tags = {ph.split(', ')[0]} WHERE project_id = {ph.split(', ')[1]}",
(json.dumps(tags), project_id), (json.dumps(tags), project_id),
) )
@@ -139,7 +340,6 @@ def build_metadata_header(project_id: str, display_name: str | None = None,
TYPE: PROJECT CONTEXT, PROJECT, STATUS: ACTIVE, LAST-UPDATED, TAGS. TYPE: PROJECT CONTEXT, PROJECT, STATUS: ACTIVE, LAST-UPDATED, TAGS.
LAST-UPDATED uses the actual updated_at timestamp, falling back to today. LAST-UPDATED uses the actual updated_at timestamp, falling back to today.
TAGS uses the project's metadata_tags if provided, falling back to project name + CONTEXT.""" TAGS uses the project's metadata_tags if provided, falling back to project name + CONTEXT."""
from datetime import datetime, timezone
project_upper = (display_name or project_id).upper() project_upper = (display_name or project_id).upper()
last_updated = (updated_at or datetime.now(timezone.utc).strftime("%Y-%m-%d")) last_updated = (updated_at or datetime.now(timezone.utc).strftime("%Y-%m-%d"))
if "T" in last_updated: if "T" in last_updated:
@@ -186,10 +386,11 @@ def context_read(conn, project_id: str) -> dict | None:
Returns with metadata header prepended dynamically. Returns with metadata header prepended dynamically.
If content already has a header (including YAML frontmatter from vault imports), If content already has a header (including YAML frontmatter from vault imports),
it is replaced with the current dynamic header.""" it is replaced with the current dynamic header."""
ph = _ph(conn, 1)
row = conn.execute( row = conn.execute(
"SELECT pc.*, p.shared_version, p.display_name FROM project_context pc " f"SELECT pc.*, p.shared_version, p.display_name FROM project_context pc "
"JOIN projects p ON p.project_id = pc.project_id " f"JOIN projects p ON p.project_id = pc.project_id "
"WHERE pc.project_id = ?", (project_id,) f"WHERE pc.project_id = {ph}", (project_id,)
).fetchone() ).fetchone()
if row is None: if row is None:
return None return None
@@ -238,8 +439,9 @@ def context_update(conn, project_id: str, new_content: str, updated_by: str,
Returns {'ok': True, 'new_version': N} or {'ok': False, 'error': 'conflict', Returns {'ok': True, 'new_version': N} or {'ok': False, 'error': 'conflict',
'current_version': N}. 'current_version': N}.
""" """
ph = _ph(conn, 1)
cur = conn.execute( cur = conn.execute(
"SELECT shared_version FROM projects WHERE project_id = ?", f"SELECT shared_version FROM projects WHERE project_id = {ph}",
(project_id,) (project_id,)
) )
row = cur.fetchone() row = cur.fetchone()
@@ -262,15 +464,21 @@ def context_update(conn, project_id: str, new_content: str, updated_by: str,
_snapshot_take(conn, project_id, version_from=current_version, version_to=new_version) _snapshot_take(conn, project_id, version_from=current_version, version_to=new_version)
# Update project_context # Update project_context
ph5 = _ph(conn, 5)
if _is_pg(conn):
ts_expr = "to_char(now() at time zone 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS\"Z\"')"
else:
ts_expr = "strftime('%Y-%m-%dT%H:%M:%SZ', 'now')"
conn.execute( conn.execute(
"UPDATE project_context SET content = ?, version = ?, updated_by = ?, updated_at = ? " f"UPDATE project_context SET content = {_ph(conn,1)}, version = {_ph(conn,1)}, "
"WHERE project_id = ?", f"updated_by = {_ph(conn,1)}, updated_at = {ts_expr} "
(clean_content, new_version, updated_by, now(), project_id), f"WHERE project_id = {_ph(conn,1)}",
(clean_content, new_version, updated_by, project_id)
) )
# Bump shared version # Bump shared version
conn.execute( conn.execute(
"UPDATE projects SET shared_version = ? WHERE project_id = ?", f"UPDATE projects SET shared_version = {_ph(conn,1)} WHERE project_id = {_ph(conn,1)}",
(new_version, project_id), (new_version, project_id)
) )
return {"ok": True, "new_version": new_version, "content": clean_content} return {"ok": True, "new_version": new_version, "content": clean_content}
@@ -279,29 +487,38 @@ def context_update(conn, project_id: str, new_content: str, updated_by: str,
# ── User Profile ────────────────────────────────────────────────────────────── # ── User Profile ──────────────────────────────────────────────────────────────
def profile_read(conn, user_id: str) -> dict | None: def profile_read(conn, user_id: str) -> dict | None:
ph = _ph(conn, 1)
return _row_to_dict(conn.execute( return _row_to_dict(conn.execute(
"SELECT * FROM user_profiles WHERE user_id = ?", (user_id,) f"SELECT * FROM user_profiles WHERE user_id = {ph}", (user_id,)
).fetchone()) ).fetchone())
def profile_update(conn, user_id: str, content: str, base_version: int) -> dict: def profile_update(conn, user_id: str, content: str, base_version: int) -> dict:
ph = _ph(conn, 1)
cur = conn.execute( cur = conn.execute(
"SELECT version FROM user_profiles WHERE user_id = ?", (user_id,) f"SELECT version FROM user_profiles WHERE user_id = {ph}", (user_id,)
) )
row = cur.fetchone() row = cur.fetchone()
if row is None: if row is None:
# Create # Create
ph2 = _ph(conn, 2)
conn.execute( conn.execute(
"INSERT INTO user_profiles (user_id, content, version) VALUES (?, ?, 1)", f"INSERT INTO user_profiles (user_id, content, version) VALUES ({ph2}, 1)",
(user_id, content), (user_id, content)
) )
return {"ok": True, "new_version": 1} return {"ok": True, "new_version": 1}
current_version = row["version"] current_version = row["version"]
if base_version != current_version: if base_version != current_version:
return {"ok": False, "error": "conflict", "current_version": current_version} return {"ok": False, "error": "conflict", "current_version": current_version}
if _is_pg(conn):
ts_expr = "to_char(now() at time zone 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS\"Z\"')"
else:
ts_expr = "strftime('%Y-%m-%dT%H:%M:%SZ', 'now')"
ph4 = _ph(conn, 4)
conn.execute( conn.execute(
"UPDATE user_profiles SET content = ?, version = ?, updated_at = ? WHERE user_id = ?", f"UPDATE user_profiles SET content = {_ph(conn,1)}, version = {_ph(conn,1)}, "
(content, current_version + 1, now(), user_id), f"updated_at = {ts_expr} WHERE user_id = {_ph(conn,1)}",
(content, current_version + 1, user_id)
) )
return {"ok": True, "new_version": current_version + 1} return {"ok": True, "new_version": current_version + 1}
@@ -325,42 +542,51 @@ def workspace_fork(conn, user_id: str, project_id: str) -> dict:
shared_content = ctx["content"] if ctx else "" shared_content = ctx["content"] if ctx else ""
ws_id = str(uuid.uuid4()) ws_id = str(uuid.uuid4())
ph4 = _ph(conn, 4)
conn.execute( conn.execute(
"INSERT INTO user_workspaces (workspace_id, user_id, project_id, base_version) " f"INSERT INTO user_workspaces (workspace_id, user_id, project_id, base_version) "
"VALUES (?, ?, ?, ?)", f"VALUES ({ph4})",
(ws_id, user_id, project_id, base_version), (ws_id, user_id, project_id, base_version),
) )
# Seed workspace with current shared content # Seed workspace with current shared content
ph2 = _ph(conn, 2)
conn.execute( conn.execute(
"INSERT INTO workspace_files (workspace_id, file_path, content) VALUES (?, 'context.md', ?)", f"INSERT INTO workspace_files (workspace_id, file_path, content) VALUES ({ph2.split(', ')[0]}, 'context.md', {ph2.split(', ')[1]})",
(ws_id, shared_content), (ws_id, shared_content),
) )
return {"ok": True, "workspace_id": ws_id, "base_version": base_version} return {"ok": True, "workspace_id": ws_id, "base_version": base_version}
def workspace_get(conn, workspace_id: str) -> dict | None: def workspace_get(conn, workspace_id: str) -> dict | None:
ph = _ph(conn, 1)
return _row_to_dict(conn.execute( return _row_to_dict(conn.execute(
"SELECT * FROM user_workspaces WHERE workspace_id = ?", (workspace_id,) f"SELECT * FROM user_workspaces WHERE workspace_id = {ph}", (workspace_id,)
).fetchone()) ).fetchone())
def workspace_list_for_user(conn, user_id: str, project_id: str | None = None) -> list[dict]: def workspace_list_for_user(conn, user_id: str, project_id: str | None = None) -> list[dict]:
if project_id: if project_id:
ph2 = _ph(conn, 2)
placeholders = ph2.split(", ")
rows = conn.execute( rows = conn.execute(
"SELECT * FROM user_workspaces WHERE user_id = ? AND project_id = ? ORDER BY created_at DESC", f"SELECT * FROM user_workspaces WHERE user_id = {placeholders[0]} AND project_id = {placeholders[1]} "
f"ORDER BY created_at DESC",
(user_id, project_id), (user_id, project_id),
).fetchall() ).fetchall()
else: else:
ph = _ph(conn, 1)
rows = conn.execute( rows = conn.execute(
"SELECT * FROM user_workspaces WHERE user_id = ? ORDER BY created_at DESC", f"SELECT * FROM user_workspaces WHERE user_id = {ph} ORDER BY created_at DESC",
(user_id,), (user_id,),
).fetchall() ).fetchall()
return [dict(r) for r in rows] return [dict(r) for r in rows]
def workspace_read_file(conn, workspace_id: str, file_path: str = "context.md") -> str | None: def workspace_read_file(conn, workspace_id: str, file_path: str = "context.md") -> str | None:
ph2 = _ph(conn, 2)
placeholders = ph2.split(", ")
row = conn.execute( row = conn.execute(
"SELECT content FROM workspace_files WHERE workspace_id = ? AND file_path = ?", f"SELECT content FROM workspace_files WHERE workspace_id = {placeholders[0]} AND file_path = {placeholders[1]}",
(workspace_id, file_path), (workspace_id, file_path),
).fetchone() ).fetchone()
return row["content"] if row else None return row["content"] if row else None
@@ -368,19 +594,27 @@ def workspace_read_file(conn, workspace_id: str, file_path: str = "context.md")
def workspace_write_file(conn, workspace_id: str, content: str, def workspace_write_file(conn, workspace_id: str, content: str,
file_path: str = "context.md"): file_path: str = "context.md"):
ph2 = _ph(conn, 2)
placeholders = ph2.split(", ")
existing = conn.execute( existing = conn.execute(
"SELECT 1 FROM workspace_files WHERE workspace_id = ? AND file_path = ?", f"SELECT 1 FROM workspace_files WHERE workspace_id = {placeholders[0]} AND file_path = {placeholders[1]}",
(workspace_id, file_path), (workspace_id, file_path),
).fetchone() ).fetchone()
if existing: if existing:
if _is_pg(conn):
ts_expr = "to_char(now() at time zone 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS\"Z\"')"
else:
ts_expr = "strftime('%Y-%m-%dT%H:%M:%SZ', 'now')"
conn.execute( conn.execute(
"UPDATE workspace_files SET content = ?, version = version + 1, updated_at = ? " f"UPDATE workspace_files SET content = {_ph(conn,1)}, version = version + 1, "
"WHERE workspace_id = ? AND file_path = ?", f"updated_at = {ts_expr} "
(content, now(), workspace_id, file_path), f"WHERE workspace_id = {_ph(conn,1)} AND file_path = {_ph(conn,1)}",
(content, workspace_id, file_path),
) )
else: else:
ph3 = _ph(conn, 3)
conn.execute( conn.execute(
"INSERT INTO workspace_files (workspace_id, file_path, content) VALUES (?, ?, ?)", f"INSERT INTO workspace_files (workspace_id, file_path, content) VALUES ({ph3})",
(workspace_id, file_path, content), (workspace_id, file_path, content),
) )
@@ -424,31 +658,35 @@ def workspace_submit(conn, workspace_id: str, submitted_by: str,
) )
if not result["ok"]: if not result["ok"]:
return result return result
ph = _ph(conn, 1)
conn.execute( conn.execute(
"UPDATE user_workspaces SET status = 'merged' WHERE workspace_id = ?", f"UPDATE user_workspaces SET status = 'merged' WHERE workspace_id = {ph}",
(workspace_id,), (workspace_id,),
) )
return {"ok": True, "action": "merged", **result} return {"ok": True, "action": "merged", **result}
else: else:
# Create pending change request # Create pending change request
req_id = str(uuid.uuid4()) req_id = str(uuid.uuid4())
ph7 = _ph(conn, 7)
conn.execute( conn.execute(
"INSERT INTO change_requests (request_id, workspace_id, project_id, " f"INSERT INTO change_requests (request_id, workspace_id, project_id, "
"submitted_by, target_version, base_version, diff_summary) " f"submitted_by, target_version, base_version, diff_summary) "
"VALUES (?, ?, ?, ?, ?, ?, ?)", f"VALUES ({ph7})",
(req_id, workspace_id, ws["project_id"], submitted_by, (req_id, workspace_id, ws["project_id"], submitted_by,
target_version, base_version, diff_summary), target_version, base_version, diff_summary),
) )
ph = _ph(conn, 1)
conn.execute( conn.execute(
"UPDATE user_workspaces SET status = 'submitted' WHERE workspace_id = ?", f"UPDATE user_workspaces SET status = 'submitted' WHERE workspace_id = {ph}",
(workspace_id,), (workspace_id,),
) )
return {"ok": True, "action": "submitted", "request_id": req_id} return {"ok": True, "action": "submitted", "request_id": req_id}
def workspace_abandon(conn, workspace_id: str): def workspace_abandon(conn, workspace_id: str):
ph = _ph(conn, 1)
conn.execute( conn.execute(
"UPDATE user_workspaces SET status = 'abandoned' WHERE workspace_id = ?", f"UPDATE user_workspaces SET status = 'abandoned' WHERE workspace_id = {ph}",
(workspace_id,), (workspace_id,),
) )
@@ -456,10 +694,11 @@ def workspace_abandon(conn, workspace_id: str):
def change_request_approve(conn, request_id: str, reviewer_id: str, def change_request_approve(conn, request_id: str, reviewer_id: str,
comments: str = "") -> dict: comments: str = "") -> dict:
"""Approve a change request and merge it into shared context.""" """Approve a change request and merge it into shared context."""
ph = _ph(conn, 1)
row = conn.execute( row = conn.execute(
"SELECT cr.*, ws.project_id FROM change_requests cr " f"SELECT cr.*, ws.project_id FROM change_requests cr "
"JOIN user_workspaces ws ON ws.workspace_id = cr.workspace_id " f"JOIN user_workspaces ws ON ws.workspace_id = cr.workspace_id "
"WHERE cr.request_id = ?", (request_id,) f"WHERE cr.request_id = {ph}", (request_id,)
).fetchone() ).fetchone()
if row is None: if row is None:
return {"ok": False, "error": "not_found"} return {"ok": False, "error": "not_found"}
@@ -467,8 +706,10 @@ def change_request_approve(conn, request_id: str, reviewer_id: str,
return {"ok": False, "error": f"status is {row['status']}"} return {"ok": False, "error": f"status is {row['status']}"}
# Record review # Record review
ph3 = _ph(conn, 3)
conn.execute( conn.execute(
"INSERT INTO reviews (request_id, reviewer_id, decision, comments) VALUES (?, ?, 'approved', ?)", f"INSERT INTO reviews (request_id, reviewer_id, decision, comments) "
f"VALUES ({ph3.split(', ')[0]}, {ph3.split(', ')[1]}, 'approved', {ph3.split(', ')[2]})",
(request_id, reviewer_id, comments), (request_id, reviewer_id, comments),
) )
@@ -483,12 +724,13 @@ def change_request_approve(conn, request_id: str, reviewer_id: str,
if not result["ok"]: if not result["ok"]:
return result return result
ph = _ph(conn, 1)
conn.execute( conn.execute(
"UPDATE change_requests SET status = 'merged' WHERE request_id = ?", f"UPDATE change_requests SET status = 'merged' WHERE request_id = {ph}",
(request_id,), (request_id,),
) )
conn.execute( conn.execute(
"UPDATE user_workspaces SET status = 'merged' WHERE workspace_id = ?", f"UPDATE user_workspaces SET status = 'merged' WHERE workspace_id = {ph}",
(row["workspace_id"],), (row["workspace_id"],),
) )
return {"ok": True, "action": "merged", **result} return {"ok": True, "action": "merged", **result}
@@ -510,18 +752,20 @@ def _snapshot_take(conn, project_id: str, version_from: int, version_to: int,
ts = now().replace(":", "-") ts = now().replace(":", "-")
storage_rel = f"{project_id}/{ts}__v{version_from}-{version_to}" storage_rel = f"{project_id}/{ts}__v{version_from}-{version_to}"
ph9 = _ph(conn, 9)
conn.execute( conn.execute(
"INSERT INTO snapshots (snapshot_id, project_id, user_id, workspace_id, " f"INSERT INTO snapshots (snapshot_id, project_id, user_id, workspace_id, "
"version_from, version_to, storage_path, content_hash, size_bytes) " f"version_from, version_to, storage_path, content_hash, size_bytes) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)", f"VALUES ({ph9})",
(snap_id, project_id, user_id, workspace_id, (snap_id, project_id, user_id, workspace_id,
version_from, version_to, storage_rel, content_hash, len(content)), version_from, version_to, storage_rel, content_hash, len(content)),
) )
def snapshot_list(conn, project_id: str) -> list[dict]: def snapshot_list(conn, project_id: str) -> list[dict]:
ph = _ph(conn, 1)
rows = conn.execute( rows = conn.execute(
"SELECT * FROM snapshots WHERE project_id = ? ORDER BY created_at DESC", f"SELECT * FROM snapshots WHERE project_id = {ph} ORDER BY created_at DESC",
(project_id,), (project_id,),
).fetchall() ).fetchall()
return [dict(r) for r in rows] return [dict(r) for r in rows]
@@ -532,16 +776,19 @@ def snapshot_rotate(conn, project_id: str, max_keep: int = 25, min_keep: int = 5
Prune excess snapshots for a project, keeping at least min_keep. Prune excess snapshots for a project, keeping at least min_keep.
Returns count of pruned snapshots. Returns count of pruned snapshots.
""" """
ph = _ph(conn, 1)
rows = conn.execute( rows = conn.execute(
"SELECT snapshot_id FROM snapshots WHERE project_id = ? " f"SELECT snapshot_id FROM snapshots WHERE project_id = {ph} "
"ORDER BY created_at DESC", (project_id,) f"ORDER BY created_at DESC", (project_id,)
).fetchall() ).fetchall()
if len(rows) <= max_keep: if len(rows) <= max_keep:
return 0 return 0
keep = max(min_keep, max_keep) keep = max(min_keep, max_keep)
to_delete = [r["snapshot_id"] for r in rows[keep:]] to_delete = [r["snapshot_id"] for r in rows[keep:]]
for sid in to_delete: for sid in to_delete:
conn.execute("DELETE FROM snapshots WHERE snapshot_id = ?", (sid,)) conn.execute(
f"DELETE FROM snapshots WHERE snapshot_id = {_ph(conn,1)}", (sid,)
)
return len(to_delete) return len(to_delete)
@@ -551,10 +798,11 @@ def audit_log(conn, user_id: str, operation: str, summary: str,
agent_id: str = "ctx", project_id: str | None = None, agent_id: str = "ctx", project_id: str | None = None,
entity_type: str | None = None, entity_id: str | None = None, entity_type: str | None = None, entity_id: str | None = None,
details: dict | None = None): details: dict | None = None):
ph8 = _ph(conn, 8)
conn.execute( conn.execute(
"INSERT INTO audit_log (user_id, agent_id, project_id, operation, " f"INSERT INTO audit_log (user_id, agent_id, project_id, operation, "
"entity_type, entity_id, summary, details_json) " f"entity_type, entity_id, summary, details_json) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?)", f"VALUES ({ph8})",
(user_id, agent_id, project_id, operation, (user_id, agent_id, project_id, operation,
entity_type, entity_id, summary, entity_type, entity_id, summary,
json.dumps(details) if details else None), json.dumps(details) if details else None),
@@ -569,7 +817,8 @@ def audit_query(conn, **filters) -> list[dict]:
for col in ("user_id", "project_id", "operation", "agent_id"): for col in ("user_id", "project_id", "operation", "agent_id"):
val = filters.get(col) val = filters.get(col)
if val: if val:
wheres.append(f"{col} = ?") ph = _ph(conn, 1)
wheres.append(f"{col} = {ph}")
params.append(val) params.append(val)
if wheres: if wheres:
parts.append("WHERE " + " AND ".join(wheres)) parts.append("WHERE " + " AND ".join(wheres))
@@ -584,13 +833,27 @@ def audit_query(conn, **filters) -> list[dict]:
def search(conn, query: str, limit: int = 10) -> list[dict]: def search(conn, query: str, limit: int = 10) -> list[dict]:
"""Full-text search across all indexed context content.""" """Full-text search across all indexed context content."""
rows = conn.execute( if _is_pg(conn):
"SELECT rowid, content, project_id, file_path, source_type, " ph = _ph(conn, 3)
"rank FROM fts_context WHERE fts_context MATCH ? " placeholders = ph.split(", ")
"ORDER BY rank LIMIT ?", rows = conn.execute(
(query, limit), f"SELECT content, project_id, file_path, source_type, "
).fetchall() f"ts_rank(tsv, plainto_tsquery('english', {placeholders[0]})) as rank "
return [dict(r) for r in rows] f"FROM fts_context WHERE tsv @@ plainto_tsquery('english', {placeholders[1]}) "
f"ORDER BY rank DESC LIMIT {placeholders[2]}",
(query, query, limit),
).fetchall()
return [dict(r) for r in rows]
else:
ph = _ph(conn, 2)
placeholders = ph.split(", ")
rows = conn.execute(
f"SELECT rowid, content, project_id, file_path, source_type, "
f"rank FROM fts_context WHERE fts_context MATCH {placeholders[0]} "
f"ORDER BY rank LIMIT {placeholders[1]}",
(query, limit),
).fetchall()
return [dict(r) for r in rows]
# ── Sync to project root ────────────────────────────────────────────────────── # ── Sync to project root ──────────────────────────────────────────────────────
@@ -677,9 +940,10 @@ def normalize_file_path(file_path: str) -> str:
def file_list(conn, project_id: str) -> list[dict]: def file_list(conn, project_id: str) -> list[dict]:
"""List all files for a project. Returns list of {file_id, file_path, version, updated_at, updated_by}.""" """List all files for a project. Returns list of {file_id, file_path, version, updated_at, updated_by}."""
ph = _ph(conn, 1)
rows = conn.execute( rows = conn.execute(
"SELECT file_id, file_path, version, updated_by, updated_at " f"SELECT file_id, file_path, version, updated_by, updated_at "
"FROM context_files WHERE project_id = ? ORDER BY file_path", f"FROM context_files WHERE project_id = {ph} ORDER BY file_path",
(project_id,) (project_id,)
).fetchall() ).fetchall()
return [_row_to_dict(r) for r in rows] return [_row_to_dict(r) for r in rows]
@@ -688,10 +952,12 @@ def file_list(conn, project_id: str) -> list[dict]:
def file_read(conn, project_id: str, file_path: str) -> dict | None: def file_read(conn, project_id: str, file_path: str) -> dict | None:
"""Read a single context file. Returns with dynamic metadata header prepended.""" """Read a single context file. Returns with dynamic metadata header prepended."""
file_path = normalize_file_path(file_path) file_path = normalize_file_path(file_path)
ph2 = _ph(conn, 2)
placeholders = ph2.split(", ")
row = conn.execute( row = conn.execute(
"SELECT cf.*, p.display_name FROM context_files cf " f"SELECT cf.*, p.display_name FROM context_files cf "
"JOIN projects p ON p.project_id = cf.project_id " f"JOIN projects p ON p.project_id = cf.project_id "
"WHERE cf.project_id = ? AND cf.file_path = ?", f"WHERE cf.project_id = {placeholders[0]} AND cf.file_path = {placeholders[1]}",
(project_id, file_path) (project_id, file_path)
).fetchone() ).fetchone()
if row is None: if row is None:
@@ -731,8 +997,10 @@ def file_create(conn, project_id: str, file_path: str, content: str = "",
file_path = normalize_file_path(file_path) file_path = normalize_file_path(file_path)
# Check if file already exists # Check if file already exists
ph2 = _ph(conn, 2)
placeholders = ph2.split(", ")
existing = conn.execute( existing = conn.execute(
"SELECT file_id FROM context_files WHERE project_id = ? AND file_path = ?", f"SELECT file_id FROM context_files WHERE project_id = {placeholders[0]} AND file_path = {placeholders[1]}",
(project_id, file_path) (project_id, file_path)
).fetchone() ).fetchone()
if existing: if existing:
@@ -742,9 +1010,10 @@ def file_create(conn, project_id: str, file_path: str, content: str = "",
clean = strip_metadata_header(content) clean = strip_metadata_header(content)
clean = clean.lstrip("\n\r ").strip() clean = clean.lstrip("\n\r ").strip()
ph5 = _ph(conn, 5)
conn.execute( conn.execute(
"INSERT INTO context_files (project_id, file_path, content, version, updated_by) " f"INSERT INTO context_files (project_id, file_path, content, version, updated_by) "
"VALUES (?, ?, ?, 1, ?)", f"VALUES ({ph5.split(', ')[0]}, {ph5.split(', ')[1]}, {ph5.split(', ')[2]}, 1, {ph5.split(', ')[3]})",
(project_id, file_path, clean, updated_by) (project_id, file_path, clean, updated_by)
) )
audit_log(conn, updated_by, "create", f"Created file {file_path} in {project_id}", audit_log(conn, updated_by, "create", f"Created file {file_path} in {project_id}",
@@ -759,8 +1028,14 @@ def file_update(conn, project_id: str, file_path: str, new_content: str,
# Normalize # Normalize
file_path = normalize_file_path(file_path) file_path = normalize_file_path(file_path)
# Lock only files in the ctxd-docs project (documentation/guide)
if project_id == "ctxd-docs" and file_path in ("CONTEXT.MD", "LLM-CLIENT.MD"):
return {"ok": False, "error": "cannot_update_locked", "hint": f"{file_path} is locked in ctxd-docs — create a new file instead"}
ph2 = _ph(conn, 2)
placeholders = ph2.split(", ")
row = conn.execute( row = conn.execute(
"SELECT version FROM context_files WHERE project_id = ? AND file_path = ?", f"SELECT version FROM context_files WHERE project_id = {placeholders[0]} AND file_path = {placeholders[1]}",
(project_id, file_path) (project_id, file_path)
).fetchone() ).fetchone()
if row is None: if row is None:
@@ -778,10 +1053,14 @@ def file_update(conn, project_id: str, file_path: str, new_content: str,
clean = clean.lstrip().strip() clean = clean.lstrip().strip()
new_version = current_version + 1 new_version = current_version + 1
if _is_pg(conn):
ts_expr = "to_char(now() at time zone 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS\"Z\"')"
else:
ts_expr = "strftime('%Y-%m-%dT%H:%M:%SZ', 'now')"
conn.execute( conn.execute(
"UPDATE context_files SET content = ?, version = ?, updated_by = ?, " f"UPDATE context_files SET content = {_ph(conn,1)}, version = {_ph(conn,1)}, "
"updated_at = strftime('%Y-%m-%dT%H:%M:%SZ', 'now') " f"updated_by = {_ph(conn,1)}, updated_at = {ts_expr} "
"WHERE project_id = ? AND file_path = ?", f"WHERE project_id = {_ph(conn,1)} AND file_path = {_ph(conn,1)}",
(clean, new_version, updated_by, project_id, file_path) (clean, new_version, updated_by, project_id, file_path)
) )
audit_log(conn, updated_by, "update", f"Updated {file_path} in {project_id} to v{new_version}", audit_log(conn, updated_by, "update", f"Updated {file_path} in {project_id} to v{new_version}",
@@ -789,26 +1068,32 @@ def file_update(conn, project_id: str, file_path: str, new_content: str,
return {"ok": True, "new_version": new_version} return {"ok": True, "new_version": new_version}
def file_delete(conn, project_id: str, file_path: str) -> dict: def file_delete(conn, project_id: str, file_path: str, deleted_by: str = "admin") -> dict:
"""Delete a context file. Returns {'ok': True} or {'ok': False, 'error': ...}.""" """Delete a context file. Returns {'ok': True} or {'ok': False, 'error': ...}."""
file_path = normalize_file_path(file_path) file_path = normalize_file_path(file_path)
# Don't allow deleting CONTEXT.md (it's the canonical synced file) # CONTEXT.MD cannot be deleted from any project (it's the minimum required file)
if file_path == "CONTEXT.MD": if file_path == "CONTEXT.MD":
return {"ok": False, "error": "cannot_delete_context"} return {"ok": False, "error": "cannot_delete_context"}
# LLM-CLIENT.MD locked in ctxd-docs only
if project_id == "ctxd-docs" and file_path == "LLM-CLIENT.MD":
return {"ok": False, "error": "cannot_delete_context"}
ph2 = _ph(conn, 2)
placeholders = ph2.split(", ")
row = conn.execute( row = conn.execute(
"SELECT file_id FROM context_files WHERE project_id = ? AND file_path = ?", f"SELECT file_id FROM context_files WHERE project_id = {placeholders[0]} AND file_path = {placeholders[1]}",
(project_id, file_path) (project_id, file_path)
).fetchone() ).fetchone()
if row is None: if row is None:
return {"ok": False, "error": "not_found"} return {"ok": False, "error": "not_found"}
conn.execute( conn.execute(
"DELETE FROM context_files WHERE project_id = ? AND file_path = ?", f"DELETE FROM context_files WHERE project_id = {_ph(conn,1)} AND file_path = {_ph(conn,1)}",
(project_id, file_path) (project_id, file_path)
) )
audit_log(conn, "admin", "delete", f"Deleted {file_path} from {project_id}", audit_log(conn, deleted_by, "delete", f"Deleted {file_path} from {project_id}",
project_id=project_id, entity_type="file", entity_id=file_path) project_id=project_id, entity_type="file", entity_id=file_path)
return {"ok": True} return {"ok": True}
@@ -822,9 +1107,10 @@ def compiled_read(conn, project_id: str) -> dict | None:
return None return None
# Get all files # Get all files
ph = _ph(conn, 1)
files = conn.execute( files = conn.execute(
"SELECT file_path, content, version, updated_at, updated_by " f"SELECT file_path, content, version, updated_at, updated_by "
"FROM context_files WHERE project_id = ? ORDER BY file_path", f"FROM context_files WHERE project_id = {ph} ORDER BY file_path",
(project_id,) (project_id,)
).fetchall() ).fetchall()
@@ -855,8 +1141,9 @@ def compiled_read(conn, project_id: str) -> dict | None:
) )
# Get the latest version from project_context (for version checking) # Get the latest version from project_context (for version checking)
ph = _ph(conn, 1)
ctx_row = conn.execute( ctx_row = conn.execute(
"SELECT version FROM project_context WHERE project_id = ?", f"SELECT version FROM project_context WHERE project_id = {ph}",
(project_id,) (project_id,)
).fetchone() ).fetchone()
version = ctx_row["version"] if ctx_row else 0 version = ctx_row["version"] if ctx_row else 0
@@ -873,25 +1160,29 @@ def ensure_default_files(conn, project_id: str):
"""Create default context files for a project if they don't exist. """Create default context files for a project if they don't exist.
Migrates existing single-context content into CONTEXT.md.""" Migrates existing single-context content into CONTEXT.md."""
# Check if any files already exist # Check if any files already exist
ph = _ph(conn, 1)
existing = conn.execute( existing = conn.execute(
"SELECT COUNT(*) as cnt FROM context_files WHERE project_id = ?", f"SELECT COUNT(*) as cnt FROM context_files WHERE project_id = {ph}",
(project_id,) (project_id,)
).fetchone() ).fetchone()
if existing and existing["cnt"] > 0: if existing and existing["cnt"] > 0:
return # Already has files return # Already has files
# Get existing single-context content to migrate into CONTEXT.md # Get existing single-context content to migrate into CONTEXT.md
ph = _ph(conn, 1)
ctx_row = conn.execute( ctx_row = conn.execute(
"SELECT content FROM project_context WHERE project_id = ?", f"SELECT content FROM project_context WHERE project_id = {ph}",
(project_id,) (project_id,)
).fetchone() ).fetchone()
existing_content = ctx_row["content"] if ctx_row else "" existing_content = ctx_row["content"] if ctx_row else ""
existing_content = strip_metadata_header(existing_content).strip() existing_content = strip_metadata_header(existing_content).strip()
# Create CONTEXT.md with existing content # Create CONTEXT.md with existing content
ph2 = _ph(conn, 2)
p = ph2.split(", ")
conn.execute( conn.execute(
"INSERT INTO context_files (project_id, file_path, content, version, updated_by) " f"INSERT INTO context_files (project_id, file_path, content, version, updated_by) "
"VALUES (?, 'CONTEXT.MD', ?, 1, 'admin')", f"VALUES ({p[0]}, 'CONTEXT.MD', {p[1]}, 1, 'admin')",
(project_id, existing_content) (project_id, existing_content)
) )
@@ -899,8 +1190,10 @@ def ensure_default_files(conn, project_id: str):
for fname in DEFAULT_FILES: for fname in DEFAULT_FILES:
if fname == "CONTEXT.md": if fname == "CONTEXT.md":
continue # Already created above continue # Already created above
ph2 = _ph(conn, 2)
p = ph2.split(", ")
conn.execute( conn.execute(
"INSERT INTO context_files (project_id, file_path, content, version, updated_by) " f"INSERT INTO context_files (project_id, file_path, content, version, updated_by) "
"VALUES (?, ?, '', 1, 'admin')", f"VALUES ({p[0]}, {p[1]}, '', 1, 'admin')",
(project_id, fname.upper()) (project_id, fname.upper())
) )
+309
View File
@@ -0,0 +1,309 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>CTXD — Context Dossier</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:ital,wght@0,400;0,600;0,700;1,400&display=swap" rel="stylesheet">
<style>
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
:root {
--bg: #1a1a1a;
--paper: #1e1e1e;
--ink: #d4d4d4;
--ink-dim: #888;
--accent: #e5c07b;
--accent3: #98c379;
--border: #2a2a2a;
--border-light: #333;
--input-bg: #222;
--hover: #252525;
--danger: #e06c75;
--font: 'JetBrains Mono', 'Fira Code', 'Cascadia Code', 'IBM Plex Mono', monospace;
}
html { height: 100%; font-size: 15px; }
body {
font-family: var(--font);
background: var(--bg);
color: var(--ink);
height: 100vh;
height: 100dvh;
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
padding: 1.5rem;
}
.landing {
max-width: 420px;
width: 100%;
text-align: center;
}
.landing h1 {
font-size: 1.4rem;
font-weight: 700;
text-transform: uppercase;
letter-spacing: 0.15em;
color: var(--accent);
margin-bottom: 0.35rem;
}
.landing .subtitle {
font-size: 0.7rem;
text-transform: uppercase;
letter-spacing: 0.1em;
color: var(--ink-dim);
margin-bottom: 2rem;
}
.landing .description {
font-size: 0.75rem;
color: var(--ink-dim);
line-height: 1.6;
margin-bottom: 2rem;
text-align: left;
}
.landing .description code {
color: var(--accent);
background: var(--input-bg);
padding: 0.1rem 0.3rem;
font-size: 0.7rem;
}
.login-card {
background: var(--paper);
border: 1px solid var(--border-light);
padding: 1.5rem;
text-align: left;
}
.login-card h2 {
font-size: 0.85rem;
text-transform: uppercase;
letter-spacing: 0.1em;
color: var(--accent);
margin-bottom: 1rem;
}
.login-card label {
display: block;
font-size: 0.65rem;
text-transform: uppercase;
letter-spacing: 0.08em;
color: var(--ink-dim);
margin-bottom: 0.3rem;
margin-top: 0.75rem;
}
.login-card input {
width: 100%;
font-family: var(--font);
font-size: 0.8rem;
padding: 0.55rem 0.65rem;
background: var(--input-bg);
border: 1px solid var(--border);
color: var(--ink);
outline: none;
}
.login-card input:focus { border-color: var(--accent); }
.login-card .actions {
display: flex;
gap: 0.5rem;
margin-top: 1rem;
}
.login-card button {
font-family: var(--font);
font-size: 0.7rem;
text-transform: uppercase;
letter-spacing: 0.08em;
padding: 0.5rem 0.85rem;
cursor: pointer;
border: 1px solid var(--border-light);
background: none;
color: var(--ink);
}
.login-card button:hover { background: var(--hover); }
.login-card button.primary {
background: var(--accent);
color: var(--bg);
border-color: var(--accent);
flex: 1;
}
.login-card button.primary:hover { background: #d4ae5c; }
.toast {
position: fixed;
bottom: 1.5rem;
left: 50%;
transform: translateX(-50%);
background: var(--paper);
border: 1px solid var(--border-light);
color: var(--ink);
font-family: var(--font);
font-size: 0.7rem;
padding: 0.6rem 1rem;
opacity: 0;
transition: opacity 0.2s;
pointer-events: none;
z-index: 200;
}
.toast.show { opacity: 1; }
.toast.error { border-color: var(--danger); color: var(--danger); }
.toast.success { border-color: var(--accent3); color: var(--accent3); }
.links {
margin-top: 1.5rem;
font-size: 0.65rem;
color: var(--ink-dim);
line-height: 1.8;
}
.links a {
color: var(--accent);
text-decoration: none;
}
.links a:hover { text-decoration: underline; }
.status {
margin-top: 1rem;
font-size: 0.6rem;
color: var(--ink-dim);
text-transform: uppercase;
letter-spacing: 0.06em;
}
.status .dot {
display: inline-block;
width: 5px; height: 5px;
border-radius: 50%;
background: var(--accent3);
margin-right: 0.3rem;
vertical-align: middle;
}
</style>
</head>
<body>
<div class="landing">
<h1>CTXD</h1>
<div class="subtitle">Context Dossier</div>
<div class="description">
Single source of truth for multi-harness project context.
One canonical <code>AGENTS.md</code> per project, served to
Claude, Hermes, Codex, Cursor, and any OAuth-capable MCP client.
</div>
<div class="login-card" id="login-card">
<h2>sign in</h2>
<label>user id</label>
<input type="text" id="user-id" placeholder="e.g. admin" autocomplete="username" autocorrect="off" autocapitalize="off" spellcheck="false" onkeydown="if(event.key==='Enter')submitLogin()">
<label>password</label>
<input type="password" id="password" placeholder="password" autocomplete="current-password" onkeydown="if(event.key==='Enter')submitLogin()">
<div class="actions">
<button class="primary" onclick="submitLogin()">sign in</button>
</div>
</div>
<div class="links">
<a href="/.well-known/oauth-authorization-server">OAuth discovery</a> &middot;
<a href="/readonly/sse">read-only MCP</a> &middot;
<a href="/write/sse">write MCP</a>
</div>
<div class="status" id="status">
<span class="dot"></span> connected
</div>
</div>
<div class="toast" id="toast"></div>
<script>
const API = '';
function showToast(msg, type) {
const el = document.getElementById('toast');
el.textContent = msg;
el.className = 'toast show' + (type ? ' ' + type : '');
clearTimeout(window._tt);
window._tt = setTimeout(() => el.classList.remove('show'), 3500);
}
async function submitLogin() {
const uid = document.getElementById('user-id').value.trim();
const pw = document.getElementById('password').value;
if (!uid || !pw) {
showToast('user id and password required', 'error');
return;
}
try {
const res = await fetch('/auth/login', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ user_id: uid, password: pw }),
});
if (res.ok) {
const data = await res.json();
localStorage.setItem('ctxd_session_token', data.token);
showToast('signed in — loading dashboard', 'success');
setTimeout(() => { window.location.href = '/'; }, 500);
} else {
let detail = '';
try { detail = (await res.json()).error || ''; } catch (_) {}
if (res.status === 401) {
showToast(detail === 'invalid credentials' ? 'invalid user id or password' : ('login failed: ' + (detail || res.status)), 'error');
} else if (res.status === 403 && detail === 'account inactive') {
showToast('account inactive — contact an admin', 'error');
} else {
showToast('login failed (' + res.status + ')', 'error');
}
}
} catch (e) {
showToast('network error — check connection', 'error');
}
}
// Check if already signed in (cookie-based — server will redirect to dashboard)
// The "signed in" card is only shown if the server served the landing page
// despite a valid cookie, which shouldn't happen. If it does, offer a redirect.
(async () => {
// If the server served the landing page, we're not authenticated via cookie.
// Try localStorage token as fallback (for backward compat with old sessions).
const token = localStorage.getItem('ctxd_session_token');
if (token) {
try {
const res = await fetch('/auth/me', { headers: { Authorization: 'Bearer ' + token } });
if (res.ok) {
// Token works via Bearer but cookie wasn't set — force redirect with token in cookie
// Re-login to get the cookie set, or just redirect (the dashboard JS uses Bearer too)
document.getElementById('login-card').innerHTML = '<h2>signed in</h2><p style="font-size:0.75rem;color:var(--ink-dim);margin-bottom:0.75rem">You are signed in.</p><div class="actions"><button class="primary" onclick="window.location.href=\'/\'">open dashboard</button></div>';
}
} catch (_) {}
}
})();
// Status check
(async () => {
try {
await fetch('/status');
document.getElementById('status').innerHTML = '<span class="dot"></span> connected';
} catch (_) {
document.getElementById('status').innerHTML = 'disconnected';
}
})();
</script>
</body>
</html>
+192
View File
@@ -0,0 +1,192 @@
#!/usr/bin/env python3
"""
Migrate data from SQLite to PostgreSQL.
Usage:
DATABASE_URL=postgresql://ctxd:ctxd_local_dev@localhost:5432/ctxd \
SQLITE_PATH=/data/ctxd.db \
python3 -m ctxd.migrate_sqlite_to_pg
Or inside the Docker container:
docker exec dossier python3 -m ctxd.migrate_sqlite_to_pg
Environment variables:
DATABASE_URL — PostgreSQL connection string (required)
SQLITE_PATH — Path to SQLite DB file (default: /data/ctxd.db)
"""
import os
import sqlite3
import sys
try:
import psycopg
from psycopg.rows import dict_row
except ImportError:
print("ERROR: psycopg is required. Install with: pip install psycopg[binary]")
sys.exit(1)
SQLITE_PATH = os.environ.get("SQLITE_PATH", "/data/ctxd.db")
PG_URL = os.environ.get("DATABASE_URL", "")
# Tables in migration order (parents before children for FK safety)
TABLES = [
"users",
"projects",
"project_permissions",
"project_context",
"context_files",
"user_profiles",
"user_workspaces",
"workspace_files",
"change_requests",
"reviews",
"snapshots",
"audit_log",
]
# Columns that need type conversion from SQLite INTEGER to PostgreSQL BOOLEAN
BOOL_COLUMNS = {
"users": ["active"],
"projects": ["auto_sync"],
}
def migrate():
if not PG_URL:
print("ERROR: DATABASE_URL environment variable not set")
print("Example: DATABASE_URL=postgresql://ctxd:ctxd_local_dev@localhost:5432/ctxd")
sys.exit(1)
if not os.path.exists(SQLITE_PATH):
print(f"ERROR: SQLite database not found at {SQLITE_PATH}")
sys.exit(1)
print(f"Migrating from SQLite ({SQLITE_PATH}) to PostgreSQL ({PG_URL})")
print()
# Connect to SQLite
sconn = sqlite3.connect(SQLITE_PATH)
sconn.row_factory = sqlite3.Row
# Connect to PostgreSQL
pconn = psycopg.connect(PG_URL, row_factory=dict_row)
pconn.autocommit = False
# Step 1: Clear all data from PostgreSQL tables
print("Clearing existing PostgreSQL data...")
pconn.execute(
"TRUNCATE TABLE fts_context, audit_log, reviews, change_requests, "
"workspace_files, user_workspaces, context_files, project_context, "
"snapshots, user_profiles, project_permissions, projects, users CASCADE"
)
pconn.commit()
print(" Done.")
print()
# Step 2: Migrate each table
total_rows = 0
for table in TABLES:
# Get column names from SQLite
try:
cur = sconn.execute(f"PRAGMA table_info({table})")
columns = [row["name"] for row in cur.fetchall()]
except sqlite3.OperationalError:
print(f" {table}: table not found in SQLite, skipping")
continue
if not columns:
print(f" {table}: no columns found, skipping")
continue
# Read all rows from SQLite
col_str = ", ".join(columns)
try:
cur = sconn.execute(f"SELECT {col_str} FROM {table}")
rows = cur.fetchall()
except sqlite3.OperationalError:
print(f" {table}: error reading from SQLite, skipping")
continue
if not rows:
print(f" {table}: 0 rows")
continue
# Insert into PostgreSQL
val_str = ", ".join(["%s"] * len(columns))
insert_sql = f'INSERT INTO {table} ({col_str}) VALUES ({val_str}) ON CONFLICT DO NOTHING'
bool_cols = BOOL_COLUMNS.get(table, [])
count = 0
for row in rows:
values = []
for col in columns:
val = row[col]
# Convert integer 0/1 to boolean for PostgreSQL BOOLEAN columns
if col in bool_cols and val is not None:
val = bool(val)
values.append(val)
try:
pconn.execute(insert_sql, values)
count += 1
except Exception as fk_err:
# Skip rows with FK violations (e.g. orphaned snapshots)
pconn.rollback()
count_skipped = count_skipped + 1 if 'count_skipped' in dir() else 1
continue
pconn.commit()
total_rows += count
print(f" {table}: {count} rows migrated")
# Step 3: Rebuild FTS index
print()
print("Rebuilding FTS index...")
pconn.execute("DELETE FROM fts_context")
pconn.commit()
# Re-populate FTS from source tables
pconn.execute("""
INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
SELECT project_id, content, project_id, 'context.md', 'project_context',
to_tsvector('english', content)
FROM project_context
WHERE content != ''
""")
fts_pc = pconn.cursor().rowcount
pconn.execute("""
INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
SELECT file_id::text, content, project_id, file_path, 'context_file',
to_tsvector('english', content)
FROM context_files
WHERE content != ''
""")
fts_cf = pconn.cursor().rowcount
pconn.execute("""
INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
SELECT user_id, content, '~user', user_id, 'user_profile',
to_tsvector('english', content)
FROM user_profiles
WHERE content != ''
""")
fts_up = pconn.cursor().rowcount
pconn.commit()
print(f" project_context: {fts_pc} entries")
print(f" context_files: {fts_cf} entries")
print(f" user_profiles: {fts_up} entries")
print()
print("=" * 60)
print(f"Migration complete! {total_rows} total rows migrated.")
print("=" * 60)
sconn.close()
pconn.close()
if __name__ == "__main__":
migrate()
+169 -116
View File
@@ -1,23 +1,20 @@
-- ============================================================================ -- ============================================================================
-- ctxd — Context Daemon Schema -- ctxd — Context Daemon Schema
-- SQLite 3.x, WAL mode, FTS5 -- PostgreSQL 16
-- ============================================================================ -- ============================================================================
-- WAL for concurrent reads during writes; foreign keys enforced
PRAGMA journal_mode = WAL;
PRAGMA foreign_keys = ON;
-- ============================================================================ -- ============================================================================
-- USERS -- USERS
-- ============================================================================ -- ============================================================================
CREATE TABLE users ( CREATE TABLE users (
user_id TEXT PRIMARY KEY, -- uuid or "joshua", "polly", "hermes-gateway" user_id TEXT PRIMARY KEY,
display_name TEXT NOT NULL, display_name TEXT NOT NULL,
role TEXT NOT NULL DEFAULT 'contributor' role TEXT NOT NULL DEFAULT 'contributor'
CHECK (role IN ('admin', 'contributor', 'service')), CHECK (role IN ('admin', 'contributor', 'service')),
token_hash TEXT, -- NULL = no auth (localhost/trusted) token_hash TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')), active BOOLEAN NOT NULL DEFAULT TRUE,
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')) created_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
); );
CREATE UNIQUE INDEX idx_users_lower ON users (LOWER(user_id)); CREATE UNIQUE INDEX idx_users_lower ON users (LOWER(user_id));
@@ -26,22 +23,22 @@ CREATE UNIQUE INDEX idx_users_lower ON users (LOWER(user_id));
-- PROJECTS -- PROJECTS
-- ============================================================================ -- ============================================================================
CREATE TABLE projects ( CREATE TABLE projects (
project_id TEXT PRIMARY KEY, -- uuid or slug "remote-rig" project_id TEXT PRIMARY KEY,
display_name TEXT NOT NULL, display_name TEXT NOT NULL,
description TEXT, description TEXT,
metadata_tags TEXT DEFAULT '[]', -- JSON array of tag strings e.g. '["ARCHITECTURE","3D-PRINTING"]' metadata_tags TEXT DEFAULT '[]',
shared_version INTEGER NOT NULL DEFAULT 0, -- monotonically increasing shared_version INTEGER NOT NULL DEFAULT 0,
auto_sync INTEGER NOT NULL DEFAULT 0, -- boolean: auto-write AGENTS.md to sync_path auto_sync BOOLEAN NOT NULL DEFAULT FALSE,
sync_path TEXT, -- absolute path to project root (nullable) sync_path TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')), created_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')) updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
); );
-- ============================================================================ -- ============================================================================
-- PROJECT PERMISSIONS (admin overrides all) -- PROJECT PERMISSIONS (admin overrides all)
-- ============================================================================ -- ============================================================================
CREATE TABLE project_permissions ( CREATE TABLE project_permissions (
id INTEGER PRIMARY KEY AUTOINCREMENT, id SERIAL PRIMARY KEY,
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE, project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
user_id TEXT NOT NULL REFERENCES users(user_id) ON DELETE CASCADE, user_id TEXT NOT NULL REFERENCES users(user_id) ON DELETE CASCADE,
permission TEXT NOT NULL DEFAULT 'editor' permission TEXT NOT NULL DEFAULT 'editor'
@@ -54,9 +51,9 @@ CREATE TABLE project_permissions (
-- ============================================================================ -- ============================================================================
CREATE TABLE user_profiles ( CREATE TABLE user_profiles (
user_id TEXT PRIMARY KEY REFERENCES users(user_id) ON DELETE CASCADE, user_id TEXT PRIMARY KEY REFERENCES users(user_id) ON DELETE CASCADE,
content TEXT NOT NULL DEFAULT '', -- markdown content TEXT NOT NULL DEFAULT '',
version INTEGER NOT NULL DEFAULT 1, version INTEGER NOT NULL DEFAULT 1,
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')) updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
); );
-- ============================================================================ -- ============================================================================
@@ -66,10 +63,10 @@ CREATE TABLE user_profiles (
-- ============================================================================ -- ============================================================================
CREATE TABLE project_context ( CREATE TABLE project_context (
project_id TEXT PRIMARY KEY REFERENCES projects(project_id) ON DELETE CASCADE, project_id TEXT PRIMARY KEY REFERENCES projects(project_id) ON DELETE CASCADE,
content TEXT NOT NULL DEFAULT '', -- compiled markdown content TEXT NOT NULL DEFAULT '',
version INTEGER NOT NULL DEFAULT 0, -- mirrors projects.shared_version version INTEGER NOT NULL DEFAULT 0,
updated_by TEXT REFERENCES users(user_id), updated_by TEXT REFERENCES users(user_id),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')) updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
); );
-- ============================================================================ -- ============================================================================
@@ -78,13 +75,13 @@ CREATE TABLE project_context (
-- version tracks this file's edit count (independent of the shared version). -- version tracks this file's edit count (independent of the shared version).
-- ============================================================================ -- ============================================================================
CREATE TABLE context_files ( CREATE TABLE context_files (
file_id INTEGER PRIMARY KEY AUTOINCREMENT, file_id SERIAL PRIMARY KEY,
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE, project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
file_path TEXT NOT NULL, -- "decisions/001-use-go.md" file_path TEXT NOT NULL,
content TEXT NOT NULL DEFAULT '', content TEXT NOT NULL DEFAULT '',
version INTEGER NOT NULL DEFAULT 1, -- per-file edit counter version INTEGER NOT NULL DEFAULT 1,
updated_by TEXT REFERENCES users(user_id), updated_by TEXT REFERENCES users(user_id),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')), updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
UNIQUE(project_id, file_path) UNIQUE(project_id, file_path)
); );
@@ -94,16 +91,16 @@ CREATE TABLE context_files (
-- shared version they started from. current_version tracks their edits. -- shared version they started from. current_version tracks their edits.
-- ============================================================================ -- ============================================================================
CREATE TABLE user_workspaces ( CREATE TABLE user_workspaces (
workspace_id TEXT PRIMARY KEY, -- uuid workspace_id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(user_id) ON DELETE CASCADE, user_id TEXT NOT NULL REFERENCES users(user_id) ON DELETE CASCADE,
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE, project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
status TEXT NOT NULL DEFAULT 'in_progress' status TEXT NOT NULL DEFAULT 'in_progress'
CHECK (status IN ('in_progress', 'submitted', 'merged', 'abandoned')), CHECK (status IN ('in_progress', 'submitted', 'merged', 'abandoned')),
base_version INTEGER NOT NULL, -- shared version at fork time base_version INTEGER NOT NULL,
current_version INTEGER NOT NULL DEFAULT 1, current_version INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')), created_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')), updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
UNIQUE(user_id, project_id, status) -- one active workspace per user per project UNIQUE(user_id, project_id, status)
); );
-- ============================================================================ -- ============================================================================
@@ -111,12 +108,12 @@ CREATE TABLE user_workspaces (
-- Mirrors the same file_path as context_files but in the user's workspace. -- Mirrors the same file_path as context_files but in the user's workspace.
-- ============================================================================ -- ============================================================================
CREATE TABLE workspace_files ( CREATE TABLE workspace_files (
file_id INTEGER PRIMARY KEY AUTOINCREMENT, file_id SERIAL PRIMARY KEY,
workspace_id TEXT NOT NULL REFERENCES user_workspaces(workspace_id) ON DELETE CASCADE, workspace_id TEXT NOT NULL REFERENCES user_workspaces(workspace_id) ON DELETE CASCADE,
file_path TEXT NOT NULL, file_path TEXT NOT NULL,
content TEXT NOT NULL DEFAULT '', content TEXT NOT NULL DEFAULT '',
version INTEGER NOT NULL DEFAULT 1, version INTEGER NOT NULL DEFAULT 1,
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')), updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
UNIQUE(workspace_id, file_path) UNIQUE(workspace_id, file_path)
); );
@@ -124,30 +121,29 @@ CREATE TABLE workspace_files (
-- CHANGE REQUESTS — submit / review / merge workflow -- CHANGE REQUESTS — submit / review / merge workflow
-- ============================================================================ -- ============================================================================
CREATE TABLE change_requests ( CREATE TABLE change_requests (
request_id TEXT PRIMARY KEY, -- uuid request_id TEXT PRIMARY KEY,
workspace_id TEXT NOT NULL REFERENCES user_workspaces(workspace_id) ON DELETE CASCADE, workspace_id TEXT NOT NULL REFERENCES user_workspaces(workspace_id) ON DELETE CASCADE,
project_id TEXT NOT NULL REFERENCES projects(project_id), project_id TEXT NOT NULL REFERENCES projects(project_id),
submitted_by TEXT NOT NULL REFERENCES users(user_id), submitted_by TEXT NOT NULL REFERENCES users(user_id),
status TEXT NOT NULL DEFAULT 'pending' status TEXT NOT NULL DEFAULT 'pending'
CHECK (status IN ('pending', 'approved', 'rejected', 'merged')), CHECK (status IN ('pending', 'approved', 'rejected', 'merged')),
-- Snapshot of what changed, stored inline so reviews survive workspace mutation diff_summary TEXT,
diff_summary TEXT, -- free-text summary of changes target_version INTEGER NOT NULL,
target_version INTEGER NOT NULL, -- the shared version this would bump to base_version INTEGER NOT NULL,
base_version INTEGER NOT NULL, -- the shared version they forked from created_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')), updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
); );
-- ============================================================================ -- ============================================================================
-- REVIEWS — approvals/rejections on change requests -- REVIEWS — approvals/rejections on change requests
-- ============================================================================ -- ============================================================================
CREATE TABLE reviews ( CREATE TABLE reviews (
review_id INTEGER PRIMARY KEY AUTOINCREMENT, review_id SERIAL PRIMARY KEY,
request_id TEXT NOT NULL REFERENCES change_requests(request_id) ON DELETE CASCADE, request_id TEXT NOT NULL REFERENCES change_requests(request_id) ON DELETE CASCADE,
reviewer_id TEXT NOT NULL REFERENCES users(user_id), reviewer_id TEXT NOT NULL REFERENCES users(user_id),
decision TEXT NOT NULL CHECK (decision IN ('approved', 'rejected')), decision TEXT NOT NULL CHECK (decision IN ('approved', 'rejected')),
comments TEXT, comments TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')), created_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
UNIQUE(request_id, reviewer_id) UNIQUE(request_id, reviewer_id)
); );
@@ -156,31 +152,29 @@ CREATE TABLE reviews (
-- Stored as files on disk at the path in storage_path. -- Stored as files on disk at the path in storage_path.
-- ============================================================================ -- ============================================================================
CREATE TABLE snapshots ( CREATE TABLE snapshots (
snapshot_id TEXT PRIMARY KEY, -- uuid snapshot_id TEXT PRIMARY KEY,
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE, project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
-- NULL user_id = snapshot of the shared copy; non-NULL = snapshot of a user workspace
user_id TEXT REFERENCES users(user_id) ON DELETE CASCADE, user_id TEXT REFERENCES users(user_id) ON DELETE CASCADE,
workspace_id TEXT REFERENCES user_workspaces(workspace_id) ON DELETE SET NULL, workspace_id TEXT REFERENCES user_workspaces(workspace_id) ON DELETE SET NULL,
version_from INTEGER, -- version range this snapshot covers version_from INTEGER,
version_to INTEGER, version_to INTEGER,
storage_path TEXT NOT NULL, -- relative to ~/.ctx/snapshots/ storage_path TEXT NOT NULL,
content_hash TEXT NOT NULL, -- sha256 of the compiled markdown content_hash TEXT NOT NULL,
size_bytes INTEGER NOT NULL DEFAULT 0, size_bytes INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')) created_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
); );
-- Index for snapshot rotation queries
CREATE INDEX idx_snapshots_cleanup ON snapshots (project_id, user_id, created_at); CREATE INDEX idx_snapshots_cleanup ON snapshots (project_id, user_id, created_at);
-- ============================================================================ -- ============================================================================
-- AUDIT LOG — append-only (INSERT only, never UPDATE or DELETE) -- AUDIT LOG — append-only (INSERT only, never UPDATE or DELETE)
-- ============================================================================ -- ============================================================================
CREATE TABLE audit_log ( CREATE TABLE audit_log (
entry_id INTEGER PRIMARY KEY AUTOINCREMENT, entry_id SERIAL PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(user_id), user_id TEXT NOT NULL REFERENCES users(user_id),
agent_id TEXT NOT NULL DEFAULT 'cli', -- "hermes", "claude-code", "ctx" agent_id TEXT NOT NULL DEFAULT 'cli',
session_id TEXT, -- opaque session identifier session_id TEXT,
project_id TEXT REFERENCES projects(project_id), project_id TEXT REFERENCES projects(project_id) ON DELETE SET NULL,
operation TEXT NOT NULL operation TEXT NOT NULL
CHECK (operation IN ( CHECK (operation IN (
'read', 'update', 'create', 'delete', 'read', 'update', 'create', 'delete',
@@ -188,105 +182,164 @@ CREATE TABLE audit_log (
'sync', 'search', 'export', 'restore', 'sync', 'search', 'export', 'restore',
'login', 'logout', 'import' 'login', 'logout', 'import'
)), )),
entity_type TEXT, -- 'project', 'workspace', 'change_request', 'snapshot', 'user_profile' entity_type TEXT,
entity_id TEXT, -- polymorphic reference entity_id TEXT,
summary TEXT NOT NULL, -- human-readable: "Updated camera-node wiring section" summary TEXT NOT NULL,
details_json TEXT, -- structured payload: diff, version numbers, etc. details_json TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')) created_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
); );
-- Audit queries by user, project, or time range
CREATE INDEX idx_audit_user ON audit_log (user_id, created_at); CREATE INDEX idx_audit_user ON audit_log (user_id, created_at);
CREATE INDEX idx_audit_project ON audit_log (project_id, created_at); CREATE INDEX idx_audit_project ON audit_log (project_id, created_at);
CREATE INDEX idx_audit_agent ON audit_log (agent_id, created_at); CREATE INDEX idx_audit_agent ON audit_log (agent_id, created_at);
CREATE INDEX idx_audit_op ON audit_log (operation, created_at); CREATE INDEX idx_audit_op ON audit_log (operation, created_at);
-- Trigger: audit_log is append-only enforce no updates or deletes at the DB level -- Note: audit_log append-only enforcement is handled at the application layer.
CREATE TRIGGER tr_audit_log_no_update -- DB-level BEFORE UPDATE/DELETE triggers conflict with FK ON DELETE SET NULL
BEFORE UPDATE ON audit_log -- cascades from projects, which internally issue UPDATE statements.
BEGIN
SELECT RAISE(ABORT, 'audit_log is append-only — no updates allowed');
END;
CREATE TRIGGER tr_audit_log_no_delete
BEFORE DELETE ON audit_log
BEGIN
SELECT RAISE(ABORT, 'audit_log is append-only — no deletes allowed');
END;
-- ============================================================================ -- ============================================================================
-- FULL-TEXT SEARCH (FTS5) -- FULL-TEXT SEARCH (tsvector with GIN index)
-- Separate FTS table with triggers to keep index in sync with source tables.
-- ============================================================================ -- ============================================================================
CREATE VIRTUAL TABLE fts_context USING fts5( CREATE TABLE fts_context (
content, id SERIAL PRIMARY KEY,
project_id UNINDEXED, source_id TEXT NOT NULL,
file_path UNINDEXED, content TEXT NOT NULL,
source_type UNINDEXED, -- 'project_context', 'context_file', 'user_profile', 'workspace_file' project_id TEXT NOT NULL,
tokenize='porter unicode61' file_path TEXT NOT NULL,
source_type TEXT NOT NULL,
tsv tsvector NOT NULL,
UNIQUE(source_type, source_id)
); );
-- Triggers to keep FTS index in sync with project_context CREATE INDEX idx_fts_context_tsv ON fts_context USING GIN (tsv);
CREATE TRIGGER tr_fts_project_context_insert AFTER INSERT ON project_context CREATE INDEX idx_fts_context_project ON fts_context (project_id);
BEGIN CREATE INDEX idx_fts_context_source ON fts_context (source_type, source_id);
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
VALUES (NEW.rowid, NEW.content, NEW.project_id, 'context.md', 'project_context');
END;
CREATE TRIGGER tr_fts_project_context_update AFTER UPDATE ON project_context -- ── Trigger functions for project_context ───────────────────────────────────
BEGIN
DELETE FROM fts_context WHERE rowid = OLD.rowid;
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
VALUES (NEW.rowid, NEW.content, NEW.project_id, 'context.md', 'project_context');
END;
CREATE TRIGGER tr_fts_project_context_delete AFTER DELETE ON project_context CREATE OR REPLACE FUNCTION fts_pc_insert() RETURNS TRIGGER AS $$
BEGIN BEGIN
DELETE FROM fts_context WHERE rowid = OLD.rowid; INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
VALUES (NEW.project_id, NEW.content, NEW.project_id, 'context.md', 'project_context',
to_tsvector('english', NEW.content));
RETURN NEW;
END; END;
$$ LANGUAGE plpgsql;
-- Triggers for context_files CREATE TRIGGER tr_fts_pc_insert AFTER INSERT ON project_context
CREATE TRIGGER tr_fts_context_files_insert AFTER INSERT ON context_files FOR EACH ROW EXECUTE FUNCTION fts_pc_insert();
BEGIN
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
VALUES (NEW.file_id + 1000000, NEW.content, NEW.project_id, NEW.file_path, 'context_file');
END;
CREATE TRIGGER tr_fts_context_files_update AFTER UPDATE ON context_files CREATE OR REPLACE FUNCTION fts_pc_update() RETURNS TRIGGER AS $$
BEGIN BEGIN
DELETE FROM fts_context WHERE rowid = OLD.file_id + 1000000; DELETE FROM fts_context WHERE source_type = 'project_context' AND source_id = OLD.project_id;
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type) INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
VALUES (NEW.file_id + 1000000, NEW.content, NEW.project_id, NEW.file_path, 'context_file'); VALUES (NEW.project_id, NEW.content, NEW.project_id, 'context.md', 'project_context',
to_tsvector('english', NEW.content));
RETURN NEW;
END; END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER tr_fts_context_files_delete AFTER DELETE ON context_files CREATE TRIGGER tr_fts_pc_update AFTER UPDATE ON project_context
BEGIN FOR EACH ROW EXECUTE FUNCTION fts_pc_update();
DELETE FROM fts_context WHERE rowid = OLD.file_id + 1000000;
END;
-- Triggers for user_profiles CREATE OR REPLACE FUNCTION fts_pc_delete() RETURNS TRIGGER AS $$
CREATE TRIGGER tr_fts_user_profiles_insert AFTER INSERT ON user_profiles
BEGIN BEGIN
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type) DELETE FROM fts_context WHERE source_type = 'project_context' AND source_id = OLD.project_id;
VALUES (NEW.rowid + 2000000, NEW.content, '~user~', NEW.user_id, 'user_profile'); RETURN OLD;
END; END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER tr_fts_user_profiles_update AFTER UPDATE ON user_profiles CREATE TRIGGER tr_fts_pc_delete AFTER DELETE ON project_context
FOR EACH ROW EXECUTE FUNCTION fts_pc_delete();
-- ── Trigger functions for context_files ─────────────────────────────────────
CREATE OR REPLACE FUNCTION fts_cf_insert() RETURNS TRIGGER AS $$
BEGIN BEGIN
DELETE FROM fts_context WHERE rowid = OLD.rowid + 2000000; INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type) VALUES (NEW.file_id::text, NEW.content, NEW.project_id, NEW.file_path, 'context_file',
VALUES (NEW.rowid + 2000000, NEW.content, '~user~', NEW.user_id, 'user_profile'); to_tsvector('english', NEW.content));
RETURN NEW;
END; END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER tr_fts_cf_insert AFTER INSERT ON context_files
FOR EACH ROW EXECUTE FUNCTION fts_cf_insert();
CREATE OR REPLACE FUNCTION fts_cf_update() RETURNS TRIGGER AS $$
BEGIN
DELETE FROM fts_context WHERE source_type = 'context_file' AND source_id = OLD.file_id::text;
INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
VALUES (NEW.file_id::text, NEW.content, NEW.project_id, NEW.file_path, 'context_file',
to_tsvector('english', NEW.content));
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER tr_fts_cf_update AFTER UPDATE ON context_files
FOR EACH ROW EXECUTE FUNCTION fts_cf_update();
CREATE OR REPLACE FUNCTION fts_cf_delete() RETURNS TRIGGER AS $$
BEGIN
DELETE FROM fts_context WHERE source_type = 'context_file' AND source_id = OLD.file_id::text;
RETURN OLD;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER tr_fts_cf_delete AFTER DELETE ON context_files
FOR EACH ROW EXECUTE FUNCTION fts_cf_delete();
-- ── Trigger functions for user_profiles ─────────────────────────────────────
CREATE OR REPLACE FUNCTION fts_up_insert() RETURNS TRIGGER AS $$
BEGIN
INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
VALUES (NEW.user_id, NEW.content, '~user', NEW.user_id, 'user_profile',
to_tsvector('english', NEW.content));
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER tr_fts_up_insert AFTER INSERT ON user_profiles
FOR EACH ROW EXECUTE FUNCTION fts_up_insert();
CREATE OR REPLACE FUNCTION fts_up_update() RETURNS TRIGGER AS $$
BEGIN
DELETE FROM fts_context WHERE source_type = 'user_profile' AND source_id = OLD.user_id;
INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
VALUES (NEW.user_id, NEW.content, '~user', NEW.user_id, 'user_profile',
to_tsvector('english', NEW.content));
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER tr_fts_up_update AFTER UPDATE ON user_profiles
FOR EACH ROW EXECUTE FUNCTION fts_up_update();
CREATE OR REPLACE FUNCTION fts_up_delete() RETURNS TRIGGER AS $$
BEGIN
DELETE FROM fts_context WHERE source_type = 'user_profile' AND source_id = OLD.user_id;
RETURN OLD;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER tr_fts_up_delete AFTER DELETE ON user_profiles
FOR EACH ROW EXECUTE FUNCTION fts_up_delete();
-- ============================================================================ -- ============================================================================
-- SEED DATA (for development / first-run) -- SEED DATA (for development / first-run)
-- ============================================================================ -- ============================================================================
INSERT INTO users (user_id, display_name, role) VALUES INSERT INTO users (user_id, display_name, role) VALUES
('admin', 'Administrator', 'admin'), ('admin', 'Administrator', 'admin'),
('hermes-gateway', 'Hermes Agent', 'service'); ('hermes-gateway', 'Hermes Agent', 'service')
ON CONFLICT DO NOTHING;
INSERT INTO projects (project_id, display_name, description) VALUES INSERT INTO projects (project_id, display_name, description) VALUES
('welcome', 'Welcome', 'Getting started guide and documentation for ctxd'), ('welcome', 'Welcome', 'Getting started guide and documentation for ctxd'),
('remote-rig', 'RemoteRig', 'Multi-camera remote monitoring system'); ('remote-rig', 'RemoteRig', 'Multi-camera remote monitoring system')
ON CONFLICT DO NOTHING;
-- Project context is seeded by the Python init code (cmd_init) -- Project context is seeded by the Python init code (cmd_init)
-- to ensure real newlines, not literal backslash-n from SQL strings. -- to ensure real newlines, not literal backslash-n from SQL strings.
+282
View File
@@ -0,0 +1,282 @@
-- ============================================================================
-- ctxd — Context Daemon Schema
-- SQLite 3.x, WAL mode, FTS5
-- (Legacy fallback for local dev when DATABASE_URL is not set)
-- ============================================================================
-- WAL for concurrent reads during writes; foreign keys enforced
PRAGMA journal_mode = WAL;
PRAGMA foreign_keys = ON;
-- ============================================================================
-- USERS
-- ============================================================================
CREATE TABLE users (
user_id TEXT PRIMARY KEY,
display_name TEXT NOT NULL,
role TEXT NOT NULL DEFAULT 'contributor'
CHECK (role IN ('admin', 'contributor', 'service')),
token_hash TEXT,
active INTEGER NOT NULL DEFAULT 1 CHECK (active IN (0, 1)),
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
);
CREATE UNIQUE INDEX idx_users_lower ON users (LOWER(user_id));
-- ============================================================================
-- PROJECTS
-- ============================================================================
CREATE TABLE projects (
project_id TEXT PRIMARY KEY,
display_name TEXT NOT NULL,
description TEXT,
metadata_tags TEXT DEFAULT '[]',
shared_version INTEGER NOT NULL DEFAULT 0,
auto_sync INTEGER NOT NULL DEFAULT 0,
sync_path TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
);
-- ============================================================================
-- PROJECT PERMISSIONS (admin overrides all)
-- ============================================================================
CREATE TABLE project_permissions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
user_id TEXT NOT NULL REFERENCES users(user_id) ON DELETE CASCADE,
permission TEXT NOT NULL DEFAULT 'editor'
CHECK (permission IN ('owner', 'editor', 'viewer')),
UNIQUE(project_id, user_id)
);
-- ============================================================================
-- USER PROFILES (personal context — timezone, preferences, style)
-- ============================================================================
CREATE TABLE user_profiles (
user_id TEXT PRIMARY KEY REFERENCES users(user_id) ON DELETE CASCADE,
content TEXT NOT NULL DEFAULT '',
version INTEGER NOT NULL DEFAULT 1,
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
);
-- ============================================================================
-- PROJECT CONTEXT — THE AUTHORITATIVE SHARED COPY
-- ============================================================================
CREATE TABLE project_context (
project_id TEXT PRIMARY KEY REFERENCES projects(project_id) ON DELETE CASCADE,
content TEXT NOT NULL DEFAULT '',
version INTEGER NOT NULL DEFAULT 0,
updated_by TEXT REFERENCES users(user_id),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
);
-- ============================================================================
-- CONTEXT FILES — individual files within a project
-- ============================================================================
CREATE TABLE context_files (
file_id INTEGER PRIMARY KEY AUTOINCREMENT,
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
file_path TEXT NOT NULL,
content TEXT NOT NULL DEFAULT '',
version INTEGER NOT NULL DEFAULT 1,
updated_by TEXT REFERENCES users(user_id),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
UNIQUE(project_id, file_path)
);
-- ============================================================================
-- USER WORKSPACES — per-user forks of a project
-- ============================================================================
CREATE TABLE user_workspaces (
workspace_id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(user_id) ON DELETE CASCADE,
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
status TEXT NOT NULL DEFAULT 'in_progress'
CHECK (status IN ('in_progress', 'submitted', 'merged', 'abandoned')),
base_version INTEGER NOT NULL,
current_version INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
UNIQUE(user_id, project_id, status)
);
-- ============================================================================
-- WORKSPACE FILES — per-user fork of context_files
-- ============================================================================
CREATE TABLE workspace_files (
file_id INTEGER PRIMARY KEY AUTOINCREMENT,
workspace_id TEXT NOT NULL REFERENCES user_workspaces(workspace_id) ON DELETE CASCADE,
file_path TEXT NOT NULL,
content TEXT NOT NULL DEFAULT '',
version INTEGER NOT NULL DEFAULT 1,
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
UNIQUE(workspace_id, file_path)
);
-- ============================================================================
-- CHANGE REQUESTS — submit / review / merge workflow
-- ============================================================================
CREATE TABLE change_requests (
request_id TEXT PRIMARY KEY,
workspace_id TEXT NOT NULL REFERENCES user_workspaces(workspace_id) ON DELETE CASCADE,
project_id TEXT NOT NULL REFERENCES projects(project_id),
submitted_by TEXT NOT NULL REFERENCES users(user_id),
status TEXT NOT NULL DEFAULT 'pending'
CHECK (status IN ('pending', 'approved', 'rejected', 'merged')),
diff_summary TEXT,
target_version INTEGER NOT NULL,
base_version INTEGER NOT NULL,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
);
-- ============================================================================
-- REVIEWS — approvals/rejections on change requests
-- ============================================================================
CREATE TABLE reviews (
review_id INTEGER PRIMARY KEY AUTOINCREMENT,
request_id TEXT NOT NULL REFERENCES change_requests(request_id) ON DELETE CASCADE,
reviewer_id TEXT NOT NULL REFERENCES users(user_id),
decision TEXT NOT NULL CHECK (decision IN ('approved', 'rejected')),
comments TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
UNIQUE(request_id, reviewer_id)
);
-- ============================================================================
-- SNAPSHOTS — point-in-time copies of project or workspace content
-- ============================================================================
CREATE TABLE snapshots (
snapshot_id TEXT PRIMARY KEY,
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
user_id TEXT REFERENCES users(user_id) ON DELETE CASCADE,
workspace_id TEXT REFERENCES user_workspaces(workspace_id) ON DELETE SET NULL,
version_from INTEGER,
version_to INTEGER,
storage_path TEXT NOT NULL,
content_hash TEXT NOT NULL,
size_bytes INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
);
CREATE INDEX idx_snapshots_cleanup ON snapshots (project_id, user_id, created_at);
-- ============================================================================
-- AUDIT LOG — append-only (INSERT only, never UPDATE or DELETE)
-- ============================================================================
CREATE TABLE audit_log (
entry_id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id TEXT NOT NULL REFERENCES users(user_id),
agent_id TEXT NOT NULL DEFAULT 'cli',
session_id TEXT,
project_id TEXT REFERENCES projects(project_id),
operation TEXT NOT NULL
CHECK (operation IN (
'read', 'update', 'create', 'delete',
'submit', 'approve', 'reject', 'merge',
'sync', 'search', 'export', 'restore',
'login', 'logout', 'import'
)),
entity_type TEXT,
entity_id TEXT,
summary TEXT NOT NULL,
details_json TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
);
CREATE INDEX idx_audit_user ON audit_log (user_id, created_at);
CREATE INDEX idx_audit_project ON audit_log (project_id, created_at);
CREATE INDEX idx_audit_agent ON audit_log (agent_id, created_at);
CREATE INDEX idx_audit_op ON audit_log (operation, created_at);
-- Trigger: audit_log is append-only
CREATE TRIGGER tr_audit_log_no_update
BEFORE UPDATE ON audit_log
BEGIN
SELECT RAISE(ABORT, 'audit_log is append-only — no updates allowed');
END;
CREATE TRIGGER tr_audit_log_no_delete
BEFORE DELETE ON audit_log
BEGIN
SELECT RAISE(ABORT, 'audit_log is append-only — no deletes allowed');
END;
-- ============================================================================
-- FULL-TEXT SEARCH (FTS5)
-- ============================================================================
CREATE VIRTUAL TABLE fts_context USING fts5(
content,
project_id UNINDEXED,
file_path UNINDEXED,
source_type UNINDEXED,
tokenize='porter unicode61'
);
-- Triggers to keep FTS index in sync with project_context
CREATE TRIGGER tr_fts_project_context_insert AFTER INSERT ON project_context
BEGIN
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
VALUES (NEW.rowid, NEW.content, NEW.project_id, 'context.md', 'project_context');
END;
CREATE TRIGGER tr_fts_project_context_update AFTER UPDATE ON project_context
BEGIN
DELETE FROM fts_context WHERE rowid = OLD.rowid;
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
VALUES (NEW.rowid, NEW.content, NEW.project_id, 'context.md', 'project_context');
END;
CREATE TRIGGER tr_fts_project_context_delete AFTER DELETE ON project_context
BEGIN
DELETE FROM fts_context WHERE rowid = OLD.rowid;
END;
-- Triggers for context_files
CREATE TRIGGER tr_fts_context_files_insert AFTER INSERT ON context_files
BEGIN
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
VALUES (NEW.file_id + 1000000, NEW.content, NEW.project_id, NEW.file_path, 'context_file');
END;
CREATE TRIGGER tr_fts_context_files_update AFTER UPDATE ON context_files
BEGIN
DELETE FROM fts_context WHERE rowid = OLD.file_id + 1000000;
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
VALUES (NEW.file_id + 1000000, NEW.content, NEW.project_id, NEW.file_path, 'context_file');
END;
CREATE TRIGGER tr_fts_context_files_delete AFTER DELETE ON context_files
BEGIN
DELETE FROM fts_context WHERE rowid = OLD.file_id + 1000000;
END;
-- Triggers for user_profiles
CREATE TRIGGER tr_fts_user_profiles_insert AFTER INSERT ON user_profiles
BEGIN
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
VALUES (NEW.rowid + 2000000, NEW.content, '~user', NEW.user_id, 'user_profile');
END;
CREATE TRIGGER tr_fts_user_profiles_update AFTER UPDATE ON user_profiles
BEGIN
DELETE FROM fts_context WHERE rowid = OLD.rowid + 2000000;
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
VALUES (NEW.rowid + 2000000, NEW.content, '~user', NEW.user_id, 'user_profile');
END;
-- ============================================================================
-- SEED DATA (for development / first-run)
-- ============================================================================
INSERT INTO users (user_id, display_name, role) VALUES
('admin', 'Administrator', 'admin'),
('hermes-gateway', 'Hermes Agent', 'service');
INSERT INTO projects (project_id, display_name, description) VALUES
('welcome', 'Welcome', 'Getting started guide and documentation for ctxd'),
('remote-rig', 'RemoteRig', 'Multi-camera remote monitoring system');
-- Project context is seeded by the Python init code (cmd_init)
-- to ensure real newlines, not literal backslash-n from SQL strings.
+1068 -69
View File
File diff suppressed because it is too large Load Diff
+846 -37
View File
File diff suppressed because it is too large Load Diff
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
View File
-14
View File
@@ -1,14 +0,0 @@
server:
host: 0.0.0.0
port: 9091
snapshots:
min_keep: 5
max_keep: 25
auth:
enabled: true
api_key: Fa50-7cg5x6ObyFjJCjBYQkaNhGF5gMBJluVY6C1OwE
seed:
admin_user: admin
admin_display: Administrator
service_user: hermes-gateway
service_display: Hermes Agent