Compare commits
27 Commits
3c9742ff87
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 295693f641 | |||
| 8395df0f80 | |||
| 3ef4f3e707 | |||
| bbb0215c98 | |||
| bdc984e5ff | |||
| 80635ce011 | |||
| 9bb89ee62f | |||
| bc43e9a8d1 | |||
| d2c6906c4f | |||
| 570b7d1dba | |||
| 451732c867 | |||
| 9e85c1b8ec | |||
| 59609f93c4 | |||
| 07cf223d16 | |||
| ce1c0a175f | |||
| 1c9d8f7648 | |||
| 87f02eb4d1 | |||
| 289c6b9300 | |||
| 12b60ee8c7 | |||
| e3567f649f | |||
| fe63ad350e | |||
| 364c7795d4 | |||
| b9f911994d | |||
| b91d03a6cd | |||
| fc1a2f5103 | |||
| a9ccfa2694 | |||
| 5a0aa2d4fe |
+25
@@ -0,0 +1,25 @@
|
|||||||
|
# Runtime CTXD data/state
|
||||||
|
/data/oauth_state.json
|
||||||
|
/data/web_sessions.json
|
||||||
|
/data/.ctxd.yaml.swp
|
||||||
|
/data/ctxd.db*
|
||||||
|
/data/ctxd.db-shm
|
||||||
|
/data/ctxd.db-wal
|
||||||
|
/data/ctxd.yaml
|
||||||
|
/data/snapshots/
|
||||||
|
/data/projects/
|
||||||
|
/data/users/
|
||||||
|
|
||||||
|
# Environment files (contains secrets)
|
||||||
|
.env
|
||||||
|
app/.env
|
||||||
|
|
||||||
|
# PostgreSQL data volume
|
||||||
|
/data/pg/
|
||||||
|
|
||||||
|
# Python cache
|
||||||
|
__pycache__/
|
||||||
|
*.py[cod]
|
||||||
|
|
||||||
|
# Build artifacts (regenerated by `pip install -e`)
|
||||||
|
*.egg-info/
|
||||||
@@ -0,0 +1,108 @@
|
|||||||
|
# CLAUDE.md
|
||||||
|
|
||||||
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||||
|
|
||||||
|
## What CTXD is
|
||||||
|
|
||||||
|
A single-process daemon that stores per-project "context dossiers" (multiple `.MD` files:
|
||||||
|
`CONTEXT.MD`, `DECISIONS.MD`, `RUNBOOKS.MD`, `PROMPTS.MD`, `GLOSSARY.MD`) and serves them to
|
||||||
|
LLM harnesses (Claude, ChatGPT, Hermes, Codex, Cursor) over **MCP via Streamable HTTP**, plus a
|
||||||
|
web UI and REST API. `CONTEXT.MD` can be synced to a repo as `AGENTS.md` with symlinks
|
||||||
|
(`CLAUDE.md`, `.cursorrules`, `CODEX.md`). The full user/operator guide is `README.md`; this file
|
||||||
|
is the orientation for editing the code.
|
||||||
|
|
||||||
|
All source lives in `app/src/ctxd/`. Run commands from `app/`.
|
||||||
|
|
||||||
|
## Architecture (the parts that need multiple files to grasp)
|
||||||
|
|
||||||
|
- **One ASGI app multiplexes three protocols.** `server.py` → `CombinedApp` (~line 1455) dispatches
|
||||||
|
every request to: REST + Web UI, the OAuth 2.0 authorization server, and the MCP endpoints. There
|
||||||
|
is no separate service per surface — "MCP is 502" and "web UI is 502" are the same process being
|
||||||
|
down. `serve_sync(cfg)` boots it under uvicorn.
|
||||||
|
- **Two MCP transports.** Streamable HTTP is the public one (paths in `MCP_STREAMABLE_PATHS` =
|
||||||
|
`/mcp`, `/readonly/mcp`, `/oauth/mcp`, all served by `CombinedApp`). `mcp_stdio.py` is a *separate*
|
||||||
|
stdin/stdout JSON-RPC server Hermes spawns directly — keep tool definitions in sync between the two
|
||||||
|
when adding tools.
|
||||||
|
- **MCP tools are built in two functions, gated by scope.** `make_mcp_server(cfg, readonly, oauth_scoped)`
|
||||||
|
exposes read tools (+ everything when API-key/LAN); `make_write_mcp_server(cfg)` exposes the write
|
||||||
|
set (`update_file`, `set_project_tags`, `sync_to_project`). OAuth `ctxd.read` vs `ctxd.write` scopes
|
||||||
|
decide what a token sees. To add an MCP tool, edit the `list_tools`/`call_tool` handlers inside these
|
||||||
|
builders.
|
||||||
|
- **`db.py` is plain functions over a `conn`, no ORM, dual-backend.** PostgreSQL is primary; SQLite
|
||||||
|
(`$CTXD_HOME/ctxd.db`) is the fallback when `DATABASE_URL` is empty (`cfg.use_postgres`). The same
|
||||||
|
query strings run on both via `_is_pg(conn)` and the placeholder helper `_ph(conn, n)`. **`schema.sql`
|
||||||
|
(PG) and `schema_sqlite.sql` must be kept in lockstep** — a table added to one must be added to the other.
|
||||||
|
- **OAuth state and web sessions are file-based JSON, not in the DB.** `OAuthStore` (`oauth_state.json`)
|
||||||
|
and `WebSessionStore` (`web_sessions.json`) live in `$CTXD_HOME` (`/data` in the container). They
|
||||||
|
survive DB swaps but are *not* covered by `pg_dump` — back them up separately.
|
||||||
|
- **Metadata headers are computed, never stored.** `build_metadata_header()` prepends the header on
|
||||||
|
read; `strip_metadata_header()` removes it before persisting. Don't store headers in the DB.
|
||||||
|
- **File paths are normalized** to uppercase with `.MD` (`normalize_file_path`). `CONTEXT.MD` is the
|
||||||
|
minimum file and cannot be deleted; in the `ctxd-docs` project, `CONTEXT.MD` and `LLM-CLIENT.MD` are
|
||||||
|
also locked against update/delete.
|
||||||
|
- **Writes are version-checked.** `file_update`/`context_update` take `base_version`; a mismatch is a
|
||||||
|
`409 conflict`. Every mutating op also writes an append-only `audit_log` row and may take a rotating
|
||||||
|
snapshot (`SNAPSHOT_MIN_KEEP`/`MAX_KEEP`).
|
||||||
|
|
||||||
|
### Latent / not wired up
|
||||||
|
|
||||||
|
`schema.sql` and `db.py` define a full collaborative-review flow — `user_workspaces` → `workspace_files`
|
||||||
|
→ `change_requests` → `reviews`, with `workspace_fork`, `workspace_submit`, `change_request_approve`,
|
||||||
|
etc. As of now this is **DB-layer only**: it is exposed through neither `server.py` (REST/MCP) nor
|
||||||
|
`cli.py`. Treat it as scaffolding, not a live feature.
|
||||||
|
|
||||||
|
## Entry points
|
||||||
|
|
||||||
|
`pyproject.toml` defines two console scripts (and `python -m ctxd` dispatches the same way via
|
||||||
|
`__main__.py`, choosing CLI vs daemon by the first arg):
|
||||||
|
|
||||||
|
- `dossier <command>` → CLI (`cli.py`, `cli_entry`). In production these run inside the container:
|
||||||
|
`docker exec ctxd dossier <command>`.
|
||||||
|
- `ctxd` → starts the daemon (`daemon_entry` → `serve_sync`).
|
||||||
|
|
||||||
|
## Build / run / deploy
|
||||||
|
|
||||||
|
Everything runs from `app/`. There is **no test runner or linter configured** — `scripts/test_*.py`
|
||||||
|
are standalone MCP smoke scripts, not a suite.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Local dev (SQLite, no Docker)
|
||||||
|
cd app
|
||||||
|
pip install -e ".[mcp]"
|
||||||
|
export CTXD_HOME=./dev-data
|
||||||
|
python -m ctxd init # initialize DB
|
||||||
|
python -m ctxd # serve → http://localhost:9091
|
||||||
|
|
||||||
|
# Production deploy (Docker + bundled PostgreSQL 16)
|
||||||
|
./scripts/deploy.sh # builds ctxd, starts postgres, waits for healthy, recreates ctxd, smoke-tests /status
|
||||||
|
|
||||||
|
# MCP smoke tests against a running server
|
||||||
|
python scripts/test_unified_mcp.py
|
||||||
|
python scripts/test_write_mcp.py
|
||||||
|
|
||||||
|
# Health
|
||||||
|
curl http://localhost:9091/status # → {"status":"ok", ...}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Deploy gotcha (the #1 source of public 502s):** `ctxd` depends on the `postgres` container being up
|
||||||
|
first. Always use `./scripts/deploy.sh` (or `docker compose up -d`, which starts both). **Never**
|
||||||
|
`docker compose up -d --no-deps ctxd` and **never** `docker restart ctxd` after a code change — the
|
||||||
|
former crash-loops without the DB, the latter runs the old image. After editing code you must rebuild
|
||||||
|
(`docker compose build ctxd`) before recreating.
|
||||||
|
|
||||||
|
## Config
|
||||||
|
|
||||||
|
All config is env-driven through `config.py` (`CtxConfig`), precedence **env var > `data/ctxd.yaml` >
|
||||||
|
default**. Key switches: `DATABASE_URL` (empty ⇒ SQLite fallback), `CTXD_HOME` (`/data` in container),
|
||||||
|
`CTXD_AUTH_ENABLED` + `CTXD_API_KEY` (shared key for LAN/Hermes = full MCP tools), `OAUTH_ENABLED` +
|
||||||
|
`OAUTH_ISSUER` (must be set together or the app won't start cleanly). The full table is in `README.md`.
|
||||||
|
|
||||||
|
## Conventions when changing code
|
||||||
|
|
||||||
|
- Adding a context-data operation: write the `conn`-taking function in `db.py` (handle both backends
|
||||||
|
via `_is_pg`/`_ph`), then expose it in the relevant surface(s) — `server.py` for REST/MCP, `cli.py`
|
||||||
|
for CLI, and `mcp_stdio.py` if Hermes needs it.
|
||||||
|
- Adding a column/table: edit **both** `schema.sql` and `schema_sqlite.sql`; for live PG instances add
|
||||||
|
a guarded migration (see `_migrate_pg` in `db.py` and the standalone `migrate_*.py` scripts).
|
||||||
|
- Don't persist metadata headers; don't bypass `base_version` checking on writes; keep every mutation
|
||||||
|
audit-logged.
|
||||||
@@ -0,0 +1,251 @@
|
|||||||
|
# CTXD — Context Dossier
|
||||||
|
# LLM Installation & Deployment Guide
|
||||||
|
# Feed this file to an LLM to assist with setup and deployment.
|
||||||
|
|
||||||
|
## What is CTXD?
|
||||||
|
|
||||||
|
CTXD (Context Dossier) is a single source of truth for multi-harness project context. It serves one canonical AGENTS.md per project to Claude, Hermes, Codex, Cursor, and any OAuth-capable MCP client via Streamable HTTP. It runs as two Docker containers: a PostgreSQL 16 database and a Python ASGI daemon.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- Docker 24+ and Docker Compose v2+
|
||||||
|
- A reverse proxy with TLS (Traefik recommended; Caddy or nginx also work)
|
||||||
|
- Linux host with ~1GB free RAM
|
||||||
|
- (Optional) Existing PostgreSQL 14+ instance if not using the bundled one
|
||||||
|
|
||||||
|
## Quick Deployment
|
||||||
|
|
||||||
|
1. Clone the repository:
|
||||||
|
git clone ssh://git@code.cubecraftlabs.com:2288/CubeCraft-Creations/CTXD.git
|
||||||
|
cd CTXD/app
|
||||||
|
|
||||||
|
2. Copy the environment template:
|
||||||
|
cp .env.example .env
|
||||||
|
|
||||||
|
3. Generate secrets and fill in .env:
|
||||||
|
python3 -c "import secrets; print(secrets.token_urlsafe(32))" # Use for CTXD_API_KEY
|
||||||
|
python3 -c "import secrets; print(secrets.token_urlsafe(32))" # Use for OAUTH_APPROVAL_KEY
|
||||||
|
|
||||||
|
4. Edit .env with your values. Required variables:
|
||||||
|
- DATABASE_URL — PostgreSQL connection string
|
||||||
|
- POSTGRES_PASSWORD — Password for the bundled PG container
|
||||||
|
- CTXD_API_KEY — Shared key for internal MCP and HTTP auth
|
||||||
|
- OAUTH_ENABLED=true
|
||||||
|
- OAUTH_ISSUER=https://your-domain.com
|
||||||
|
- OAUTH_APPROVAL_KEY — Key for approving OAuth authorizations
|
||||||
|
|
||||||
|
5. Build and start:
|
||||||
|
docker compose up -d
|
||||||
|
|
||||||
|
6. Wait for PostgreSQL to be healthy and the daemon to start:
|
||||||
|
docker compose ps
|
||||||
|
# Both ctxd-postgres and dossier should show "healthy" / "running"
|
||||||
|
|
||||||
|
7. Set the admin password:
|
||||||
|
docker exec ctxd dossier user-set-password admin -p "your-admin-password"
|
||||||
|
|
||||||
|
8. Verify:
|
||||||
|
curl http://localhost:9091/status
|
||||||
|
# Expected: {"status": "ok", "db": "/data/ctxd.db"}
|
||||||
|
|
||||||
|
9. Access the Web UI:
|
||||||
|
http://<server-ip>:9091/
|
||||||
|
Sign in with admin / your-admin-password
|
||||||
|
|
||||||
|
## Environment Variables Reference
|
||||||
|
|
||||||
|
All config is driven by environment variables in .env. Precedence: env var > ctxd.yaml > defaults.
|
||||||
|
|
||||||
|
### Database
|
||||||
|
- DATABASE_URL — PostgreSQL connection string. If empty, falls back to SQLite.
|
||||||
|
- POSTGRES_USER — PostgreSQL user for bundled container (default: ctxd)
|
||||||
|
- POSTGRES_PASSWORD — PostgreSQL password for bundled container
|
||||||
|
- POSTGRES_DB — PostgreSQL database name (default: ctxd)
|
||||||
|
|
||||||
|
### Server
|
||||||
|
- CTXD_HOST — Bind address (default: 0.0.0.0)
|
||||||
|
- CTXD_PORT — Listen port (default: 9091)
|
||||||
|
- CTXD_HOME — Data directory inside container (default: /data)
|
||||||
|
- LOG_LEVEL — Uvicorn log level: debug, info, warning, error (default: info)
|
||||||
|
|
||||||
|
### Auth
|
||||||
|
- CTXD_AUTH_ENABLED — Enable authentication globally (default: false)
|
||||||
|
- CTXD_API_KEY — Shared API key for Hermes/internal MCP and HTTP auth
|
||||||
|
- CTXD_EXTERNAL_READONLY_KEY — Legacy ?key= on read-only MCP (migration only)
|
||||||
|
|
||||||
|
### OAuth
|
||||||
|
- OAUTH_ENABLED — Enable OAuth authorization server (default: false)
|
||||||
|
- OAUTH_ISSUER — Public URL used in OAuth discovery metadata
|
||||||
|
- OAUTH_APPROVAL_KEY — Fallback key for /oauth/authorize approval
|
||||||
|
- OAUTH_APPROVAL_USER_ID — User ID for attributing approvals (default: admin)
|
||||||
|
- OAUTH_ACCESS_TOKEN_TTL — Access token lifetime in seconds (default: 3600)
|
||||||
|
- OAUTH_REFRESH_TOKEN_TTL — Refresh token lifetime in seconds (default: 2592000)
|
||||||
|
|
||||||
|
### Web Sessions
|
||||||
|
- WEB_SESSION_TTL — Session cookie lifetime in seconds (default: 604800)
|
||||||
|
|
||||||
|
### Snapshots
|
||||||
|
- SNAPSHOT_MIN_KEEP — Minimum snapshots per project (default: 5)
|
||||||
|
- SNAPSHOT_MAX_KEEP — Maximum snapshots before rotation (default: 25)
|
||||||
|
|
||||||
|
## MCP Endpoints
|
||||||
|
|
||||||
|
CTXD exposes three MCP surfaces via Streamable HTTP:
|
||||||
|
|
||||||
|
1. /readonly/mcp — Read-only tools (OAuth ctxd.read or API key)
|
||||||
|
Tools: list_projects, get_project_context, search_context, get_project_tags, list_files, get_file, get_client_guide
|
||||||
|
|
||||||
|
2. /write/mcp — Write tools (OAuth ctxd.write only)
|
||||||
|
Tools: update_file, set_project_tags, sync_to_project, get_client_guide
|
||||||
|
|
||||||
|
3. /mcp — Internal full MCP (shared API key only, not exposed publicly)
|
||||||
|
All read + write tools plus auto_generate_tags and get_user_profile
|
||||||
|
|
||||||
|
## Connecting LLM Clients
|
||||||
|
|
||||||
|
### Claude Desktop
|
||||||
|
Connector URL: https://your-domain.com/readonly/mcp
|
||||||
|
Claude auto-discovers OAuth metadata and registers via DCR.
|
||||||
|
For write access, use: https://your-domain.com/write/mcp
|
||||||
|
|
||||||
|
### Hermes Agent
|
||||||
|
Edit ~/.hermes/config.yaml:
|
||||||
|
mcp_servers:
|
||||||
|
dossier:
|
||||||
|
url: http://<server-ip>:9091/mcp
|
||||||
|
timeout: 30
|
||||||
|
headers:
|
||||||
|
Authorization: "Bearer your-api-key"
|
||||||
|
Restart Hermes after changing config.
|
||||||
|
|
||||||
|
### Other MCP Clients
|
||||||
|
1. Register an OAuth client: POST /oauth/register with your redirect URI
|
||||||
|
2. Authorize: GET /oauth/authorize (admin must approve)
|
||||||
|
3. Exchange code: POST /oauth/token
|
||||||
|
4. Connect to /readonly/mcp or /write/mcp with Bearer token
|
||||||
|
|
||||||
|
## Traefik Configuration
|
||||||
|
|
||||||
|
Route everything except the internal MCP endpoint:
|
||||||
|
|
||||||
|
rule: Host(`ctxd.yourdomain.com`) && !Path(`/mcp`)
|
||||||
|
|
||||||
|
This exposes: landing page, dashboard, REST API, OAuth, read-only MCP, write MCP.
|
||||||
|
Only /mcp (internal, API key) is blocked.
|
||||||
|
|
||||||
|
## CLI Commands
|
||||||
|
|
||||||
|
All commands run inside the container:
|
||||||
|
docker exec ctxd dossier <command>
|
||||||
|
|
||||||
|
Commands:
|
||||||
|
init Initialize database
|
||||||
|
project-create <id> [--display-name N] Create a project
|
||||||
|
project-list List projects
|
||||||
|
read <project_id> Print context to stdout
|
||||||
|
edit <project_id> Open in $EDITOR
|
||||||
|
file-list <project_id> List context files
|
||||||
|
file-read <project_id> <file_path> Read a single file
|
||||||
|
sync <project_id> [path] Sync AGENTS.md to project root
|
||||||
|
search "query" Full-text search
|
||||||
|
audit [--limit N] Show audit log
|
||||||
|
user-list List users
|
||||||
|
user-create <id> --display-name "Name" [--password "pw"]
|
||||||
|
user-set-password <id> -p "password"
|
||||||
|
oauth-client-create [-n "Name"] [--redirect-uri URI]
|
||||||
|
oauth-client-list List OAuth clients
|
||||||
|
oauth-client-revoke <client_id> Revoke client and invalidate tokens
|
||||||
|
import-vault <path> Import from Obsidian vault
|
||||||
|
|
||||||
|
## Admin UI
|
||||||
|
|
||||||
|
Sign in as admin, click "admin" in the masthead. Three tabs:
|
||||||
|
1. oauth clients — Create and revoke OAuth clients
|
||||||
|
2. users — List, create, edit, activate, inactivate, delete users
|
||||||
|
3. projects — List and delete projects (typed-name confirmation required)
|
||||||
|
|
||||||
|
## Project Files
|
||||||
|
|
||||||
|
Each project has multiple context files:
|
||||||
|
- CONTEXT.MD — Canonical overview (synced as AGENTS.md, cannot be deleted)
|
||||||
|
- DECISIONS.MD — Architecture decisions and rationale
|
||||||
|
- RUNBOOKS.MD — Deploy, troubleshoot, operate procedures
|
||||||
|
- PROMPTS.MD — Project-specific prompts for different harnesses
|
||||||
|
- GLOSSARY.MD — Project-specific terms and acronyms
|
||||||
|
|
||||||
|
The compiled view (get_project_context) concatenates all files with metadata header.
|
||||||
|
|
||||||
|
## Locked Files
|
||||||
|
|
||||||
|
- CONTEXT.MD — Cannot be deleted from any project (minimum required file)
|
||||||
|
- CONTEXT.MD and LLM-CLIENT.MD — Fully locked in the ctxd-docs project (cannot update or delete)
|
||||||
|
|
||||||
|
## Backups
|
||||||
|
|
||||||
|
PostgreSQL backup:
|
||||||
|
docker exec ctxd-postgres pg_dump -U ctxd ctxd > backup.sql
|
||||||
|
|
||||||
|
PostgreSQL restore:
|
||||||
|
cat backup.sql | docker exec -i ctxd-postgres psql -U ctxd ctxd
|
||||||
|
|
||||||
|
Snapshots are automatic (before each context update, rotated min 5 / max 25 per project).
|
||||||
|
|
||||||
|
## Migrating from SQLite to PostgreSQL
|
||||||
|
|
||||||
|
If you started with SQLite:
|
||||||
|
1. Start PostgreSQL: docker compose up -d postgres
|
||||||
|
2. Run migration: docker exec ctxd python3 -m ctxd.migrate_sqlite_to_pg
|
||||||
|
3. Set DATABASE_URL in .env and restart: docker compose up -d dossier
|
||||||
|
|
||||||
|
## Using an External PostgreSQL
|
||||||
|
|
||||||
|
1. Create database and user on your external PG instance
|
||||||
|
2. Set DATABASE_URL in .env to point to it
|
||||||
|
3. Start only the app: docker compose up -d dossier
|
||||||
|
(or scale postgres to 0: docker compose up -d --scale postgres=0)
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
Login fails: Reset password with "docker exec ctxd dossier user-set-password admin -p newpass"
|
||||||
|
Use double quotes if password contains single quotes. Use single quotes for $ ` ! characters.
|
||||||
|
|
||||||
|
MCP 404: Traefik is not routing /readonly/mcp. Check router rule includes it.
|
||||||
|
MCP 401: Auth is working. Check OAuth token scope and expiry.
|
||||||
|
|
||||||
|
PostgreSQL connection fails: Check DATABASE_URL password matches POSTGRES_PASSWORD.
|
||||||
|
If PG data volume was initialized with a different password, reset it:
|
||||||
|
docker exec ctxd-postgres psql -U ctxd -c "ALTER USER ctxd PASSWORD 'newpass'"
|
||||||
|
Then update .env.
|
||||||
|
|
||||||
|
Container keeps restarting: Check "docker logs ctxd --tail 30".
|
||||||
|
Common causes: wrong PG password, OAUTH_ENABLED=true but OAUTH_ISSUER empty, missing CTXD_API_KEY.
|
||||||
|
|
||||||
|
## File Structure
|
||||||
|
|
||||||
|
CTXD/
|
||||||
|
├── .env Production environment (gitignored, contains secrets)
|
||||||
|
├── .env.example Template with all documented variables
|
||||||
|
├── .gitignore
|
||||||
|
├── README.md Full documentation
|
||||||
|
├── SKILL.md LLM client guide (canonical source for LLM-CLIENT.MD)
|
||||||
|
├── data/ Runtime data (gitignored)
|
||||||
|
│ ├── ctxd.yaml Fallback config (env vars take precedence)
|
||||||
|
│ ├── oauth_state.json OAuth clients, codes, tokens
|
||||||
|
│ ├── web_sessions.json Web UI sessions
|
||||||
|
│ ├── snapshots/ Point-in-time context backups
|
||||||
|
│ └── pg/ PostgreSQL data volume
|
||||||
|
└── app/ Application source
|
||||||
|
├── docker-compose.yml
|
||||||
|
├── Dockerfile
|
||||||
|
├── pyproject.toml
|
||||||
|
└── src/ctxd/
|
||||||
|
├── config.py Env-driven config
|
||||||
|
├── db.py Database layer (PostgreSQL + SQLite)
|
||||||
|
├── schema.sql PostgreSQL schema
|
||||||
|
├── schema_sqlite.sql SQLite schema fallback
|
||||||
|
├── server.py ASGI: HTTP + MCP + OAuth
|
||||||
|
├── cli.py CLI commands
|
||||||
|
├── ui.html Web UI dashboard
|
||||||
|
├── landing.html Public landing page
|
||||||
|
├── auth_password.py PBKDF2 password hashing
|
||||||
|
└── migrate_sqlite_to_pg.py Migration script
|
||||||
@@ -0,0 +1,661 @@
|
|||||||
|
# CTXD — Context Dossier
|
||||||
|
|
||||||
|
A single source of truth for multi-harness project context. One canonical `AGENTS.md` per project, served to Claude, Hermes, Codex, Cursor, and any OAuth-capable MCP client via Streamable HTTP.
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
CTXD solves context sprawl: when you work across multiple LLM harnesses (Claude Desktop, Claude Code, Codex CLI, Cursor, Copilot, Hermes), each has its own context file convention (`CLAUDE.md`, `.cursorrules`, `CODEX.md`, etc.). Without a canonical source, these drift independently.
|
||||||
|
|
||||||
|
CTXD provides:
|
||||||
|
- **Multi-file context per project** — `CONTEXT.MD`, `DECISIONS.MD`, `RUNBOOKS.MD`, `PROMPTS.MD`, `GLOSSARY.MD`
|
||||||
|
- **Compiled view** — all files concatenated with metadata header, served as a single document to agents
|
||||||
|
- **Sync to repos** — writes `CONTEXT.MD` as `AGENTS.md` + symlinks (`CLAUDE.md`, `.cursorrules`, `CODEX.md` → `AGENTS.md`)
|
||||||
|
- **Version-checked writes** — optimistic concurrency with `base_version` to prevent silent overwrites
|
||||||
|
- **OAuth 2.0 authorization server** — DCR, Authorization Code + PKCE, `ctxd.read` and `ctxd.write` scopes
|
||||||
|
- **Streamable HTTP MCP** — single-endpoint transport for read-only and write surfaces
|
||||||
|
- **Web UI** — per-user password login, admin panel, project/file management
|
||||||
|
- **PostgreSQL backend** — with SQLite fallback for local dev
|
||||||
|
- **Append-only audit log** — every read, write, create, delete, sync, and search is logged
|
||||||
|
- **Point-in-time snapshots** — automatic version snapshots with rotation (min 5, max 25 per project)
|
||||||
|
- **Full-text search** — PostgreSQL `tsvector` with GIN index (FTS5 on SQLite fallback)
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
Context Dossier (container: ctxd, 0.0.0.0:9091)
|
||||||
|
├── PostgreSQL 16 (container: ctxd-postgres) # Primary DB
|
||||||
|
├── /data # Config, OAuth state, web sessions
|
||||||
|
│ ├── ctxd.yaml # Fallback config (env vars take precedence)
|
||||||
|
│ ├── oauth_state.json # OAuth clients, codes, tokens
|
||||||
|
│ ├── web_sessions.json # Per-user web UI sessions
|
||||||
|
│ └── snapshots/ # Point-in-time context backups
|
||||||
|
├── Streamable HTTP MCP:
|
||||||
|
│ ├── /mcp (OAuth ctxd.read + ctxd.write; API key on LAN = full tools)
|
||||||
|
│ ├── /readonly/mcp (alias → same OAuth behavior)
|
||||||
|
│ └── /oauth/mcp (alias)
|
||||||
|
├── OAuth Authorization Server:
|
||||||
|
│ ├── /.well-known/oauth-authorization-server # Discovery
|
||||||
|
│ ├── /.well-known/oauth-protected-resource # Resource metadata
|
||||||
|
│ ├── /oauth/register # Dynamic Client Registration
|
||||||
|
│ ├── /oauth/authorize # Authorization + PKCE
|
||||||
|
│ └── /oauth/token # Token + refresh
|
||||||
|
├── Web UI + REST API (/) # Dashboard, admin, projects, files
|
||||||
|
└── Landing page (public host only) # Themed login form
|
||||||
|
```
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
|
||||||
|
- Docker and Docker Compose
|
||||||
|
- A reverse proxy with TLS (Traefik, Caddy, nginx) for public exposure
|
||||||
|
- (Optional) An existing PostgreSQL 14+ instance if not using the bundled one
|
||||||
|
|
||||||
|
### 1. Clone and configure
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /mnt/ai-storage/projects/ctxd/app
|
||||||
|
cp .env.example .env
|
||||||
|
```
|
||||||
|
|
||||||
|
Edit `.env` with your values:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Database
|
||||||
|
DATABASE_URL=postgresql://ctxd:your-password@postgres:5432/ctxd
|
||||||
|
POSTGRES_USER=ctxd
|
||||||
|
POSTGRES_PASSWORD=your-strong-password
|
||||||
|
|
||||||
|
# Server
|
||||||
|
CTXD_HOST=0.0.0.0
|
||||||
|
CTXD_PORT=9091
|
||||||
|
CTXD_HOME=/data
|
||||||
|
|
||||||
|
# Auth
|
||||||
|
CTXD_AUTH_ENABLED=true
|
||||||
|
CTXD_API_KEY=your-api-key-here # Generate: python3 -c "import secrets; print(secrets.token_urlsafe(32))"
|
||||||
|
|
||||||
|
# OAuth
|
||||||
|
OAUTH_ENABLED=true
|
||||||
|
OAUTH_ISSUER=https://ctxd.yourdomain.com
|
||||||
|
OAUTH_APPROVAL_KEY=your-approval-key # Generate: python3 -c "import secrets; print(secrets.token_urlsafe(32))"
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Build and start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Recommended: build + postgres + recreate ctxd (avoids 502 when PG was never started)
|
||||||
|
chmod +x scripts/deploy.sh
|
||||||
|
./scripts/deploy.sh
|
||||||
|
|
||||||
|
# Or manual (always include postgres when DATABASE_URL uses host "postgres"):
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
This starts:
|
||||||
|
- `ctxd-postgres` — PostgreSQL 16 (Alpine)
|
||||||
|
- `ctxd` — CTXD daemon (web UI + MCP + OAuth + REST API)
|
||||||
|
|
||||||
|
**After code changes**, use `./scripts/deploy.sh` or `docker compose up -d` — not `docker restart ctxd` alone (old image) and **not** `docker compose up -d --no-deps ctxd` (skips postgres → crash loop → public **502**).
|
||||||
|
|
||||||
|
### 3. Verify
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Health check
|
||||||
|
curl http://localhost:9091/status
|
||||||
|
# → {"status": "ok", "db": "/data/ctxd.db"}
|
||||||
|
|
||||||
|
# List projects (requires API key)
|
||||||
|
curl http://localhost:9091/projects -H "Authorization: Bearer your-api-key"
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Set admin password
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker exec ctxd dossier user-set-password admin -p "your-admin-password"
|
||||||
|
```
|
||||||
|
|
||||||
|
> **Shell quoting:** Use double quotes (`"`) if the password contains single quotes (`'`). Use single quotes (`'`) if it contains `$`, backticks, or `!`. The failure mode is shell expansion, not PBKDF2.
|
||||||
|
|
||||||
|
### 5. Access the Web UI
|
||||||
|
|
||||||
|
- **LAN:** `http://<server-ip>:9091/`
|
||||||
|
- **Public (via Traefik):** `https://ctxd.yourdomain.com/`
|
||||||
|
|
||||||
|
Sign in with `admin` / your password.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
### Environment Variables
|
||||||
|
|
||||||
|
All config is driven by environment variables. A `ctxd.yaml` file in `/data` can override built-in defaults, but env vars always take precedence.
|
||||||
|
|
||||||
|
**Precedence:** env var > `ctxd.yaml` > built-in default
|
||||||
|
|
||||||
|
| Variable | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| **Database** | | |
|
||||||
|
| `DATABASE_URL` | *(empty)* | PostgreSQL connection string. If empty, falls back to SQLite at `$CTXD_HOME/ctxd.db` |
|
||||||
|
| `POSTGRES_USER` | `ctxd` | PostgreSQL user (for bundled PG container) |
|
||||||
|
| `POSTGRES_PASSWORD` | `ctxd_local_dev` | PostgreSQL password (for bundled PG container) |
|
||||||
|
| `POSTGRES_DB` | `ctxd` | PostgreSQL database name (for bundled PG container) |
|
||||||
|
| **Server** | | |
|
||||||
|
| `CTXD_HOST` | `0.0.0.0` | Bind address |
|
||||||
|
| `CTXD_PORT` | `9091` | Listen port |
|
||||||
|
| `CTXD_HOME` | `~/.ctx` | Data directory (inside container: `/data`) |
|
||||||
|
| `LOG_LEVEL` | `info` | Uvicorn log level (`debug`, `info`, `warning`, `error`) |
|
||||||
|
| **Auth** | | |
|
||||||
|
| `CTXD_AUTH_ENABLED` | `false` | Enable authentication globally |
|
||||||
|
| `CTXD_API_KEY` | *(empty)* | Shared API key for Hermes/internal MCP + HTTP auth |
|
||||||
|
| `CTXD_EXTERNAL_READONLY_KEY` | *(empty)* | Legacy `?key=` on read-only MCP (migration only) |
|
||||||
|
| **OAuth** | | |
|
||||||
|
| `OAUTH_ENABLED` | `false` | Enable OAuth authorization server |
|
||||||
|
| `OAUTH_ISSUER` | *(empty)* | Public URL (used in OAuth discovery metadata) |
|
||||||
|
| `OAUTH_APPROVAL_KEY` | *(empty)* | Fallback approval key for `/oauth/authorize` |
|
||||||
|
| `OAUTH_APPROVAL_USER_ID` | `admin` | Which user ID to attribute OAuth approvals to |
|
||||||
|
| `OAUTH_ACCESS_TOKEN_TTL` | `3600` | Access token lifetime in seconds |
|
||||||
|
| `OAUTH_REFRESH_TOKEN_TTL` | `2592000` | Refresh token lifetime in seconds (30 days) |
|
||||||
|
| **PostgreSQL (container)** | | |
|
||||||
|
| `CTXD_PG_WAIT_SECONDS` | `120` | Entrypoint: max wait for DB before exit (when `DATABASE_URL` set) |
|
||||||
|
| `CTXD_PG_WAIT_INTERVAL` | `2` | Seconds between connection attempts |
|
||||||
|
| **Web Sessions** | | |
|
||||||
|
| `WEB_SESSION_TTL` | `604800` | Session cookie lifetime in seconds (7 days) |
|
||||||
|
| **Snapshots** | | |
|
||||||
|
| `SNAPSHOT_MIN_KEEP` | `5` | Minimum snapshots retained per project |
|
||||||
|
| `SNAPSHOT_MAX_KEEP` | `25` | Maximum snapshots before rotation |
|
||||||
|
|
||||||
|
### Using an External PostgreSQL
|
||||||
|
|
||||||
|
To use an external PostgreSQL instead of the bundled container:
|
||||||
|
|
||||||
|
1. Create a database and user on your external PG instance
|
||||||
|
2. Set `DATABASE_URL` in `.env` to point to it
|
||||||
|
3. Start only the app (no bundled postgres): `docker compose up -d --scale postgres=0 ctxd`
|
||||||
|
|
||||||
|
Ensure `DATABASE_URL` points at your external host (not `postgres`). The entrypoint skips the compose-network wait when the URL is reachable.
|
||||||
|
|
||||||
|
### Fallback to SQLite
|
||||||
|
|
||||||
|
If `DATABASE_URL` is empty or not set, CTXD falls back to SQLite at `$CTXD_HOME/ctxd.db`. This is useful for local development or single-user deployments that don't need PostgreSQL features.
|
||||||
|
|
||||||
|
## MCP Surfaces
|
||||||
|
|
||||||
|
CTXD exposes MCP via Streamable HTTP on **`/mcp`** (single public connector):
|
||||||
|
|
||||||
|
| Endpoint | Auth | Scope | Tools |
|
||||||
|
|----------|------|-------|-------|
|
||||||
|
| `/mcp` | OAuth bearer | `ctxd.read` / `ctxd.write` | Scope-gated read + write tools |
|
||||||
|
| `/mcp` | Shared API key (LAN/Hermes) | *(full)* | All tools including `get_user_profile`, `auto_generate_tags` |
|
||||||
|
| `/readonly/mcp`, `/oauth/mcp` | OAuth (aliases) | same as `/mcp` | Backward-compatible URLs |
|
||||||
|
|
||||||
|
### Connecting an LLM Client
|
||||||
|
|
||||||
|
**Claude Desktop / Claude Web:**
|
||||||
|
```
|
||||||
|
Connector URL: https://ctxd.yourdomain.com/mcp
|
||||||
|
```
|
||||||
|
Claude auto-discovers OAuth metadata and registers via DCR. Request `scope=ctxd.read ctxd.write` for write access.
|
||||||
|
|
||||||
|
**ChatGPT (MCP connector — recommended; no Custom GPT required):**
|
||||||
|
|
||||||
|
ChatGPT can attach a **remote MCP server** in **Developer mode** (Plus/Pro and higher tiers for custom connectors). Use the same public URL as Claude — OAuth on `/mcp`, not REST Actions or a shared API key.
|
||||||
|
|
||||||
|
1. **Server** — deploy current CTXD and expose the public host (Traefik must route `/mcp` and `/oauth/*`; do not block `/mcp`):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd app
|
||||||
|
./scripts/deploy.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Smoke: `curl -sS https://ctxd.yourdomain.com/.well-known/oauth-authorization-server | head`
|
||||||
|
|
||||||
|
2. **ChatGPT** — Settings → **Connectors** → enable **Developer mode** → **Add connector** (MCP / custom remote) → **Server URL:**
|
||||||
|
|
||||||
|
```
|
||||||
|
https://ctxd.yourdomain.com/mcp
|
||||||
|
```
|
||||||
|
|
||||||
|
Start OAuth and copy ChatGPT’s **callback / redirect URL** exactly.
|
||||||
|
|
||||||
|
3. **CTXD admin** — Web UI → **admin** → **oauth clients**:
|
||||||
|
|
||||||
|
- After DCR, the client may appear automatically; otherwise **create client** with ChatGPT’s redirect URI.
|
||||||
|
- Set **allowed scopes** to **`ctxd.read`** and **`ctxd.write`** (create form or **scopes** → save on an existing row).
|
||||||
|
|
||||||
|
4. **Authorize** — complete the browser approval (sign in as CTXD admin in that browser, or use the OAuth approval key). Tokens are capped by the client’s allowed scopes.
|
||||||
|
|
||||||
|
5. **Chat** — enable the CTXD connector for the conversation. Tools include `list_projects`, `get_project_context`, `search_context`, `get_file`, `update_file`, etc. Call **`get_client_guide`** first in a new session.
|
||||||
|
|
||||||
|
**Optional CLI pre-register** (if ChatGPT asks for client credentials before DCR):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker exec ctxd dossier oauth-client-create -n "ChatGPT MCP" \
|
||||||
|
--redirect-uri 'PASTE_CHATGPT_CALLBACK_URL' \
|
||||||
|
--scope "ctxd.read ctxd.write"
|
||||||
|
```
|
||||||
|
|
||||||
|
| If connect fails | Check |
|
||||||
|
|------------------|--------|
|
||||||
|
| **“does not implement OAuth”** | Public host must return **200** JSON (not 502/404) for `/.well-known/oauth-protected-resource/mcp` and `/.well-known/oauth-authorization-server`. Traefik must proxy **all** paths to CTXD `:9091`. Set `OAUTH_ENABLED=true` and `OAUTH_ISSUER=https://ctxd.cubecraftlabs.com`. |
|
||||||
|
| 502 on well-known URLs | Backend down or wrong Traefik service — fix routing before OAuth can work |
|
||||||
|
| 404 on `/mcp` | Router not forwarding `/mcp` to CTXD (often a stale `!Path(/mcp)` rule) |
|
||||||
|
| 401 on `/mcp` | Re-authorize; access token may have expired |
|
||||||
|
| Redirect mismatch | Callback URL in admin must match ChatGPT’s string exactly |
|
||||||
|
| `invalid_request` on authorize | CTXD requires PKCE **S256** (`code_challenge` + `code_challenge_method=S256`) |
|
||||||
|
| Stale routes / old code | `./scripts/deploy.sh` or `docker compose up -d --force-recreate ctxd` after postgres is healthy (not `restart` alone; never `--no-deps` unless PG is already up) |
|
||||||
|
| No write tools | Client scopes include `ctxd.write`; re-authorize after changing scopes |
|
||||||
|
|
||||||
|
**Not required for ChatGPT MCP:** Custom GPT, OpenAPI Actions, or `CTXD_API_KEY` in ChatGPT (public path is OAuth). Hermes continues to use LAN `http://<server-ip>:9091/mcp` with the API key.
|
||||||
|
|
||||||
|
**Hermes Agent:**
|
||||||
|
```yaml
|
||||||
|
# ~/.hermes/config.yaml
|
||||||
|
mcp_servers:
|
||||||
|
dossier:
|
||||||
|
url: http://<server-ip>:9091/mcp
|
||||||
|
timeout: 30
|
||||||
|
headers:
|
||||||
|
Authorization: "Bearer your-api-key"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Other MCP clients (Codex, Cursor, etc.):**
|
||||||
|
- Register an OAuth client via `POST /oauth/register` with your redirect URI
|
||||||
|
- Connect to **`/mcp`** with `scope=ctxd.read ctxd.write`
|
||||||
|
- Use the access token as `Authorization: Bearer <token>`
|
||||||
|
|
||||||
|
### MCP Tool Reference
|
||||||
|
|
||||||
|
#### Read tools (require `ctxd.read` on `/mcp`)
|
||||||
|
|
||||||
|
| Tool | Args | Returns |
|
||||||
|
|------|------|---------|
|
||||||
|
| `get_client_guide` | *(none)* | Locked `LLM-CLIENT.MD` guide — **call this first** |
|
||||||
|
| `list_projects` | *(none)* | All projects with version numbers |
|
||||||
|
| `get_project_context` | `project_id` | Compiled markdown of all context files |
|
||||||
|
| `search_context` | `query`, `limit?` | FTS results across all projects |
|
||||||
|
| `get_project_tags` | `project_id` | Metadata tags for a project |
|
||||||
|
| `list_files` | `project_id` | All context files in a project |
|
||||||
|
| `get_file` | `project_id`, `file_path` | Single file with metadata header |
|
||||||
|
|
||||||
|
#### Write tools (require `ctxd.write` on `/mcp`)
|
||||||
|
|
||||||
|
| Tool | Args | Returns |
|
||||||
|
|------|------|---------|
|
||||||
|
| `update_file` | `project_id`, `file_path`, `content`, `base_version` | `{"ok": true, "new_version": N}` or conflict error |
|
||||||
|
| `set_project_tags` | `project_id`, `tags[]` | `{"ok": true, "tags": [...]}` |
|
||||||
|
| `sync_to_project` | `project_id` | Writes `CONTEXT.MD` as `AGENTS.md` + symlinks to project root |
|
||||||
|
|
||||||
|
### Locked Files
|
||||||
|
|
||||||
|
| File | Scope | Protection |
|
||||||
|
|------|-------|------------|
|
||||||
|
| `CONTEXT.MD` | All projects | Cannot delete — minimum required file |
|
||||||
|
| `CONTEXT.MD` | `ctxd-docs` project only | Cannot update or delete |
|
||||||
|
| `LLM-CLIENT.MD` | `ctxd-docs` project only | Cannot update or delete |
|
||||||
|
|
||||||
|
## OAuth
|
||||||
|
|
||||||
|
### Scopes
|
||||||
|
|
||||||
|
| Scope | Grants |
|
||||||
|
|-------|--------|
|
||||||
|
| `ctxd.read` | Read-only MCP tools |
|
||||||
|
| `ctxd.write` | Write MCP tools (includes read) |
|
||||||
|
|
||||||
|
Request both: `scope=ctxd.read ctxd.write`
|
||||||
|
|
||||||
|
### Redirect URIs
|
||||||
|
|
||||||
|
| Platform | Redirect URI |
|
||||||
|
|----------|-------------|
|
||||||
|
| Claude Desktop | `https://claude.ai/api/mcp/auth_callback` |
|
||||||
|
| ChatGPT (MCP) | Paste callback URL from ChatGPT connector OAuth UI (per connector) |
|
||||||
|
| Claude Code | `http://localhost:5555/oauth/callback` |
|
||||||
|
| Codex CLI | `http://localhost:7777/oauth/callback` |
|
||||||
|
| Custom | Your app's documented OAuth callback |
|
||||||
|
|
||||||
|
### Managing OAuth Clients
|
||||||
|
|
||||||
|
**Via Admin UI:** `http://<server-ip>:9091/` → sign in as admin → **admin** → **oauth clients** tab
|
||||||
|
|
||||||
|
**Via CLI:**
|
||||||
|
```bash
|
||||||
|
# Create
|
||||||
|
docker exec ctxd dossier oauth-client-create -n "Claude Desktop" --redirect-uri https://claude.ai/api/mcp/auth_callback
|
||||||
|
|
||||||
|
# List
|
||||||
|
docker exec ctxd dossier oauth-client-list
|
||||||
|
|
||||||
|
# Revoke (invalidates all tokens for that client)
|
||||||
|
docker exec ctxd dossier oauth-client-revoke ctxd_xxxxxxxx
|
||||||
|
```
|
||||||
|
|
||||||
|
**Via API (admin session or API key):**
|
||||||
|
```bash
|
||||||
|
# Create
|
||||||
|
curl -X POST http://localhost:9091/oauth/clients \
|
||||||
|
-H "Authorization: Bearer your-api-key" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"client_name": "Claude Desktop", "redirect_uris": ["https://claude.ai/api/mcp/auth_callback"]}'
|
||||||
|
|
||||||
|
# List
|
||||||
|
curl http://localhost:9091/oauth/clients -H "Authorization: Bearer your-api-key"
|
||||||
|
|
||||||
|
# Revoke
|
||||||
|
curl -X DELETE http://localhost:9091/oauth/clients/ctxd_xxxxxxxx -H "Authorization: Bearer your-api-key"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Public Exposure (Traefik)
|
||||||
|
|
||||||
|
### Router Rule
|
||||||
|
|
||||||
|
Route the public host to the backend (include `/mcp` — OAuth protects it):
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
rule: Host(`ctxd.yourdomain.com`)
|
||||||
|
```
|
||||||
|
|
||||||
|
This exposes:
|
||||||
|
- Landing page (`GET /`)
|
||||||
|
- Login (`POST /auth/login`, `GET /auth/me`)
|
||||||
|
- Full Web UI dashboard (all REST API endpoints)
|
||||||
|
- OAuth (`/oauth/*`, `/.well-known/*`)
|
||||||
|
- Public MCP (`/mcp` — OAuth read + write)
|
||||||
|
|
||||||
|
Hermes uses the same `/mcp` path on LAN with the shared API key (not exposed via public OAuth).
|
||||||
|
|
||||||
|
### Landing Page Behavior
|
||||||
|
|
||||||
|
- **Not signed in** → themed landing page with login form
|
||||||
|
- **Signed in** (valid session cookie) → full dashboard
|
||||||
|
- Cookie: `ctxd_session` (SameSite=Lax, 7-day expiry)
|
||||||
|
- After login: cookie set + redirect to dashboard
|
||||||
|
|
||||||
|
A full Traefik template is at `app/templates/traefik-ctxd-readonly-mcp.yaml` in the `project-context-management` Hermes skill.
|
||||||
|
|
||||||
|
## Web UI
|
||||||
|
|
||||||
|
### Admin Panel
|
||||||
|
|
||||||
|
Sign in as admin → click **admin** in the masthead.
|
||||||
|
|
||||||
|
**Tabs:**
|
||||||
|
1. **oauth clients** — client list (revoke per row) + create client form
|
||||||
|
2. **users** — user list (id, name, role, active/inactive) + manage users (create, edit, activate, inactivate, delete)
|
||||||
|
3. **projects** — manage projects (list with remove button, typed-name confirmation for delete)
|
||||||
|
|
||||||
|
### Project Files
|
||||||
|
|
||||||
|
Each project has multiple context files:
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
|------|---------|
|
||||||
|
| `CONTEXT.MD` | Canonical project overview (synced as `AGENTS.md` to repos) |
|
||||||
|
| `DECISIONS.MD` | Architecture decisions, rationale |
|
||||||
|
| `RUNBOOKS.MD` | Deploy, troubleshoot, operate procedures |
|
||||||
|
| `PROMPTS.MD` | Project-specific prompts for different harnesses |
|
||||||
|
| `GLOSSARY.MD` | Project-specific terms, acronyms |
|
||||||
|
|
||||||
|
The compiled view (`get_project_context`) concatenates all files with `## FILENAME` headers and a single metadata block at the top.
|
||||||
|
|
||||||
|
## CLI
|
||||||
|
|
||||||
|
All commands run inside the container:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker exec ctxd dossier <command>
|
||||||
|
```
|
||||||
|
|
||||||
|
### Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Initialize (auto-runs on first container start)
|
||||||
|
dossier init
|
||||||
|
|
||||||
|
# Projects
|
||||||
|
dossier project-create <project_id> [--display-name "Name"] [--description "Desc"]
|
||||||
|
dossier project-list
|
||||||
|
dossier read <project_id> # Print context to stdout
|
||||||
|
dossier edit <project_id> # Open in $EDITOR
|
||||||
|
|
||||||
|
# Context files
|
||||||
|
dossier file-list <project_id>
|
||||||
|
dossier file-read <project_id> <file_path>
|
||||||
|
|
||||||
|
# Sync
|
||||||
|
dossier sync <project_id> [path] # Set sync path and/or sync AGENTS.md
|
||||||
|
|
||||||
|
# Search
|
||||||
|
dossier search "query" # FTS across all projects
|
||||||
|
|
||||||
|
# Audit
|
||||||
|
dossier audit [--limit N]
|
||||||
|
|
||||||
|
# Users
|
||||||
|
dossier user-list
|
||||||
|
dossier user-create <user_id> --display-name "Name" [--password "pw"]
|
||||||
|
dossier user-set-password <user_id> -p "password"
|
||||||
|
|
||||||
|
# OAuth
|
||||||
|
dossier oauth-client-create [-n "Name"] [--redirect-uri URI]
|
||||||
|
dossier oauth-client-list
|
||||||
|
dossier oauth-client-revoke <client_id>
|
||||||
|
|
||||||
|
# Import
|
||||||
|
dossier import-vault <path> # Import from Obsidian vault
|
||||||
|
```
|
||||||
|
|
||||||
|
## REST API
|
||||||
|
|
||||||
|
All endpoints require `Authorization: Bearer <api_key>` or `Authorization: Bearer <session_token>` unless noted.
|
||||||
|
|
||||||
|
| Method | Path | Description |
|
||||||
|
|--------|------|-------------|
|
||||||
|
| `GET` | `/` | Web UI (LAN) or landing page (public host) |
|
||||||
|
| `GET` | `/status` | Health check (no auth) |
|
||||||
|
| `POST` | `/auth/login` | Web UI login → session token + cookie |
|
||||||
|
| `POST` | `/auth/logout` | Revoke session + clear cookie |
|
||||||
|
| `GET` | `/auth/me` | Current session identity |
|
||||||
|
| `GET` | `/users` | List users |
|
||||||
|
| `POST` | `/users` | Create user |
|
||||||
|
| `PATCH` | `/users/<id>` | Update user (admin) |
|
||||||
|
| `DELETE` | `/users/<id>` | Delete user (admin) |
|
||||||
|
| `POST` | `/users/<id>/password` | Set password (admin) |
|
||||||
|
| `GET` | `/oauth/clients` | List OAuth clients (admin) |
|
||||||
|
| `POST` | `/oauth/clients` | Register OAuth client (admin) |
|
||||||
|
| `DELETE` | `/oauth/clients/<id>` | Revoke OAuth client (admin) |
|
||||||
|
| `GET` | `/projects` | List all projects |
|
||||||
|
| `POST` | `/projects` | Create a project |
|
||||||
|
| `DELETE` | `/projects/<id>` | Delete a project (admin) |
|
||||||
|
| `GET` | `/projects/<id>/context` | Compiled context (all files) |
|
||||||
|
| `POST` | `/projects/<id>/context` | Update context (legacy single-file) |
|
||||||
|
| `GET` | `/projects/<id>/files` | List context files |
|
||||||
|
| `GET` | `/projects/<id>/files/<name>` | Read a single file |
|
||||||
|
| `POST` | `/projects/<id>/files` | Create a new file |
|
||||||
|
| `PUT` | `/projects/<id>/files/<name>` | Update a file (version-checked) |
|
||||||
|
| `DELETE` | `/projects/<id>/files/<name>` | Delete a file |
|
||||||
|
| `POST` | `/projects/<id>/migrate-files` | Migrate single-context to multi-file |
|
||||||
|
| `GET` | `/projects/<id>/snapshots` | List snapshots |
|
||||||
|
| `GET` | `/projects/<id>/tags` | Get metadata tags |
|
||||||
|
| `POST` | `/projects/<id>/tags` | Set metadata tags |
|
||||||
|
| `POST` | `/projects/<id>/sync` | Sync CONTEXT.MD as AGENTS.md |
|
||||||
|
| `POST` | `/projects/<id>/import` | Import raw text as context |
|
||||||
|
| `GET` | `/search?q=...` | Full-text search |
|
||||||
|
| `GET` | `/audit?limit=N` | Audit log |
|
||||||
|
| `GET` | `/.well-known/oauth-authorization-server` | OAuth discovery |
|
||||||
|
| `GET` | `/.well-known/oauth-protected-resource` | Resource metadata |
|
||||||
|
| `POST` | `/oauth/register` | Dynamic Client Registration |
|
||||||
|
| `GET/POST` | `/oauth/authorize` | Authorization endpoint |
|
||||||
|
| `POST` | `/oauth/token` | Token endpoint |
|
||||||
|
| `POST/GET/DELETE` | `/readonly/mcp` | Read-only MCP (Streamable HTTP) |
|
||||||
|
| `POST/GET/DELETE` | `/write/mcp` | Write MCP (Streamable HTTP) |
|
||||||
|
| `POST/GET/DELETE` | `/mcp` | Internal full MCP (API key only) |
|
||||||
|
|
||||||
|
## Backups
|
||||||
|
|
||||||
|
### PostgreSQL
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Backup
|
||||||
|
docker exec ctxd-postgres pg_dump -U ctxd ctxd > backup_$(date +%Y%m%d).sql
|
||||||
|
|
||||||
|
# Restore
|
||||||
|
cat backup_YYYYMMDD.sql | docker exec -i ctxd-postgres psql -U ctxd ctxd
|
||||||
|
```
|
||||||
|
|
||||||
|
### Snapshots
|
||||||
|
|
||||||
|
CTXD automatically takes point-in-time snapshots before each context update. Snapshots are stored as files in `/data/snapshots/<project_id>/` and rotated (min 5, max 25 per project).
|
||||||
|
|
||||||
|
## Migrating from SQLite to PostgreSQL
|
||||||
|
|
||||||
|
If you started with SQLite and want to move to PostgreSQL:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Start PostgreSQL
|
||||||
|
docker compose up -d postgres
|
||||||
|
|
||||||
|
# 2. Run the migration script (reads from /data/ctxd.db, writes to DATABASE_URL)
|
||||||
|
docker exec ctxd python3 -m ctxd.migrate_sqlite_to_pg
|
||||||
|
|
||||||
|
# 3. Set DATABASE_URL in .env and restart
|
||||||
|
docker compose up -d dossier
|
||||||
|
```
|
||||||
|
|
||||||
|
The migration handles all tables, rebuilds the FTS index, and skips orphaned rows with FK violations.
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
/mnt/ai-storage/projects/ctxd/
|
||||||
|
├── .env # Production environment (gitignored)
|
||||||
|
├── .env.example # Template (committed)
|
||||||
|
├── .gitignore
|
||||||
|
├── SKILL.md # LLM client guide (canonical source)
|
||||||
|
├── README.md # This file
|
||||||
|
├── data/ # Runtime data (gitignored)
|
||||||
|
│ ├── ctxd.yaml # Fallback config (env vars take precedence)
|
||||||
|
│ ├── ctxd.db # SQLite DB (if no DATABASE_URL)
|
||||||
|
│ ├── pg/ # PostgreSQL data volume
|
||||||
|
│ ├── oauth_state.json # OAuth clients, codes, tokens
|
||||||
|
│ ├── web_sessions.json # Web UI sessions
|
||||||
|
│ └── snapshots/ # Point-in-time backups
|
||||||
|
└── app/ # Application source
|
||||||
|
├── docker-compose.yml
|
||||||
|
├── Dockerfile
|
||||||
|
├── pyproject.toml
|
||||||
|
├── .env # Same as root .env (symlinked or copied)
|
||||||
|
└── src/ctxd/
|
||||||
|
├── __init__.py
|
||||||
|
├── __main__.py # CLI/daemon entry point
|
||||||
|
├── config.py # Env-driven config with yaml fallback
|
||||||
|
├── db.py # Database layer (PostgreSQL + SQLite)
|
||||||
|
├── schema.sql # PostgreSQL schema
|
||||||
|
├── schema_sqlite.sql # SQLite schema (fallback)
|
||||||
|
├── server.py # ASGI app: HTTP + MCP + OAuth
|
||||||
|
├── cli.py # CLI commands
|
||||||
|
├── ui.html # Web UI dashboard
|
||||||
|
├── landing.html # Public landing page
|
||||||
|
├── auth_password.py # PBKDF2 password hashing
|
||||||
|
└── migrate_sqlite_to_pg.py # One-time migration script
|
||||||
|
```
|
||||||
|
|
||||||
|
## Development
|
||||||
|
|
||||||
|
### Local Development (SQLite, no Docker)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd app
|
||||||
|
pip install -e ".[mcp]"
|
||||||
|
export CTXD_HOME=./dev-data
|
||||||
|
python -m ctxd init
|
||||||
|
python -m ctxd
|
||||||
|
# → http://localhost:9091
|
||||||
|
```
|
||||||
|
|
||||||
|
### Rebuilding After Code Changes
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd app
|
||||||
|
docker compose build
|
||||||
|
docker compose up -d --no-build
|
||||||
|
# Verify:
|
||||||
|
curl http://localhost:9091/status
|
||||||
|
docker logs ctxd --tail 20
|
||||||
|
```
|
||||||
|
|
||||||
|
### Key Conventions
|
||||||
|
|
||||||
|
- **Metadata headers** are dynamically generated on read, never stored in the DB
|
||||||
|
- **File paths** are normalized to uppercase with `.MD` extension
|
||||||
|
- **`CONTEXT.MD`** is the minimum required file — cannot be deleted from any project
|
||||||
|
- **Version checking** uses `base_version` parameter — mismatches return `409 conflict`
|
||||||
|
- **Audit log** is append-only at the application layer (every operation is logged)
|
||||||
|
- **OAuth state** (`oauth_state.json`) and **web sessions** (`web_sessions.json`) are file-based, not in PostgreSQL
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Login fails with "invalid credentials"
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Reset admin password
|
||||||
|
docker exec ctxd dossier user-set-password admin -p "new-password"
|
||||||
|
```
|
||||||
|
|
||||||
|
If the password contains special characters, use the quoting that matches:
|
||||||
|
- `'` in password → use double quotes (`"`)
|
||||||
|
- `$`, `` ` ``, `!` in password → use single quotes (`'`)
|
||||||
|
|
||||||
|
### Login works on LAN but not on public host
|
||||||
|
|
||||||
|
The public host (`https://ctxd.yourdomain.com`) requires Traefik to route `/auth/login` and `/auth/me`. Check your Traefik router rule includes all paths (use `!Path(`/mcp`)` to block only the internal MCP).
|
||||||
|
|
||||||
|
### MCP connection fails
|
||||||
|
|
||||||
|
1. Check OAuth discovery: `curl https://ctxd.yourdomain.com/.well-known/oauth-authorization-server`
|
||||||
|
2. Check MCP endpoint: `curl -o /dev/null -w '%{http_code}' https://ctxd.yourdomain.com/readonly/mcp` → should be `401` (not `404`)
|
||||||
|
3. If `404`: Traefik isn't routing `/readonly/mcp` — update the router rule
|
||||||
|
4. If `401`: auth is working — check OAuth token scope and expiry
|
||||||
|
|
||||||
|
### PostgreSQL connection fails
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check PG is running
|
||||||
|
docker compose ps postgres
|
||||||
|
|
||||||
|
# Check connection
|
||||||
|
docker exec ctxd python3 -c "
|
||||||
|
import os
|
||||||
|
import psycopg
|
||||||
|
conn = psycopg.connect(os.environ['DATABASE_URL'])
|
||||||
|
print('Connected:', conn.info.server_version)
|
||||||
|
"
|
||||||
|
|
||||||
|
# If password mismatch (PG data volume initialized with different password):
|
||||||
|
docker exec ctxd-postgres psql -U ctxd -c "ALTER USER ctxd PASSWORD 'new-password'"
|
||||||
|
# Then update .env with the new password
|
||||||
|
```
|
||||||
|
|
||||||
|
### Container keeps restarting / public site 502
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose ps -a
|
||||||
|
docker logs ctxd --tail 40
|
||||||
|
```
|
||||||
|
|
||||||
|
| Symptom | Cause | Fix |
|
||||||
|
|---------|--------|-----|
|
||||||
|
| `failed to resolve host 'postgres'` | `ctxd-postgres` not running (often after `--no-deps ctxd`) | `cd app && docker compose up -d postgres ctxd` or `./scripts/deploy.sh` |
|
||||||
|
| `Restarting (1)` on `ctxd` only | Same — app up without DB on compose network | Start postgres first; wait for **(healthy)** |
|
||||||
|
| Cloudflare **502** on public URL | Traefik/backend has no healthy upstream on `:9091` | Fix local `curl http://127.0.0.1:9091/status` first |
|
||||||
|
|
||||||
|
The container **entrypoint waits up to 120s** for PostgreSQL when `DATABASE_URL` is set (`CTXD_PG_WAIT_SECONDS` to override). If postgres never appears, logs print an explicit message instead of an immediate opaque crash.
|
||||||
|
|
||||||
|
Common other causes:
|
||||||
|
- `DATABASE_URL` password doesn't match what PG was initialized with
|
||||||
|
- `OAUTH_ENABLED=true` but `OAUTH_ISSUER` is empty
|
||||||
|
- Missing `CTXD_API_KEY` when `CTXD_AUTH_ENABLED=true`
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
MIT
|
||||||
@@ -0,0 +1,180 @@
|
|||||||
|
---
|
||||||
|
name: ctxd-client
|
||||||
|
description: Use when an LLM agent needs to read or update project context via the Context Dossier (CTXD) daemon. Covers OAuth authentication, MCP tool discovery, read-only and write endpoints, and the workflow for keeping project context current.
|
||||||
|
version: 1.0.0
|
||||||
|
author: Hermes Agent
|
||||||
|
license: MIT
|
||||||
|
platforms:
|
||||||
|
- linux
|
||||||
|
- macos
|
||||||
|
metadata:
|
||||||
|
hermes:
|
||||||
|
tags:
|
||||||
|
- context-management
|
||||||
|
- ctxd
|
||||||
|
- mcp-client
|
||||||
|
- oauth
|
||||||
|
- project-context
|
||||||
|
related_skills:
|
||||||
|
- project-context-management
|
||||||
|
- native-mcp
|
||||||
|
---
|
||||||
|
|
||||||
|
# CTXD Client — LLM Agent Guide
|
||||||
|
|
||||||
|
Use when an LLM agent (Claude Desktop, Codex CLI, Hermes, custom harness) needs to read or update project context stored in Context Dossier (CTXD).
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
CTXD is a single source of truth for multi-harness project context. It exposes:
|
||||||
|
|
||||||
|
- **Public MCP** (`/mcp`) — OAuth read + write (scope-gated) on the public host
|
||||||
|
- **Hermes / automation** — same `http://<lan>:9091/mcp` with `CTXD_API_KEY` (full tool surface)
|
||||||
|
|
||||||
|
Public host: `https://ctxd.cubecraftlabs.com` (OAuth + MCP + landing page + dashboard).
|
||||||
|
|
||||||
|
## Connection URLs
|
||||||
|
|
||||||
|
| Surface | URL | Auth |
|
||||||
|
|---------|-----|------|
|
||||||
|
| Public OAuth MCP (read + write) | `https://ctxd.cubecraftlabs.com/mcp` | OAuth `ctxd.read` and/or `ctxd.write` |
|
||||||
|
| Legacy aliases | `/readonly/mcp`, `/oauth/mcp` | Same behavior as `/mcp` for OAuth |
|
||||||
|
| OAuth discovery | `https://ctxd.cubecraftlabs.com/.well-known/oauth-authorization-server` | Public |
|
||||||
|
| DCR registration | `POST https://ctxd.cubecraftlabs.com/oauth/register` | Public |
|
||||||
|
| Landing page | `https://ctxd.cubecraftlabs.com/` | Public |
|
||||||
|
|
||||||
|
## OAuth Flow
|
||||||
|
|
||||||
|
1. **Discover** the authorization server metadata at `/.well-known/oauth-authorization-server`
|
||||||
|
2. **Register** a client via `POST /oauth/register` (DCR) with your redirect URI
|
||||||
|
3. **Authorize** — open `/oauth/authorize` in a browser; an admin must approve
|
||||||
|
4. **Exchange** the authorization code for tokens at `POST /oauth/token`
|
||||||
|
5. **Use** the access token as `Authorization: Bearer <token>` on MCP connections (Streamable HTTP for read; SSE for write — see Connection URLs)
|
||||||
|
|
||||||
|
### Scopes
|
||||||
|
|
||||||
|
| Scope | Grants |
|
||||||
|
|-------|--------|
|
||||||
|
| `ctxd.read` | `list_projects`, `get_project_context`, `search_context`, `get_project_tags`, `list_files`, `get_file`, `get_client_guide` |
|
||||||
|
| `ctxd.write` | `update_file`, `set_project_tags`, `sync_to_project`, `get_client_guide` |
|
||||||
|
|
||||||
|
Request both scopes for full read+write in **one connector**: `scope=ctxd.read ctxd.write`
|
||||||
|
|
||||||
|
**Admin:** In the Web UI → Admin → OAuth clients, set **allowed scopes** per client (create form or **scopes** on an existing row). Tokens cannot exceed what the client is allowed, even if the user requests more at authorize time.
|
||||||
|
|
||||||
|
### Redirect URIs by Platform
|
||||||
|
|
||||||
|
| Platform | Redirect URI |
|
||||||
|
|----------|-------------|
|
||||||
|
| Claude Desktop | `https://claude.ai/api/mcp/auth_callback` |
|
||||||
|
| Claude Code | `http://localhost:5555/oauth/callback` |
|
||||||
|
| Codex CLI | `http://localhost:7777/oauth/callback` |
|
||||||
|
| Custom | Your app's documented OAuth callback |
|
||||||
|
|
||||||
|
### Token Lifetime
|
||||||
|
|
||||||
|
Access tokens expire per server config (default ~1 hour). Refresh tokens are issued alongside access tokens. Use `POST /oauth/token` with `grant_type=refresh_token` to rotate.
|
||||||
|
|
||||||
|
## MCP Tools
|
||||||
|
|
||||||
|
### First Call — Always
|
||||||
|
|
||||||
|
| Tool | Description |
|
||||||
|
|------|-------------|
|
||||||
|
| `get_client_guide` | Return the locked LLM-CLIENT.MD guide. **Call this first in every session.** No arguments. Covers OAuth, MCP tools, read/write endpoints, version-checked updates, and error handling. |
|
||||||
|
|
||||||
|
The guide lives in the `ctxd-docs` project as `LLM-CLIENT.MD`. It is **locked** — cannot be updated or deleted by any MCP/API client. If you need the guide updated, ask an admin to edit it via the Web UI.
|
||||||
|
|
||||||
|
### Read-only tools (require `ctxd.read` on the same connector)
|
||||||
|
|
||||||
|
| Tool | Description |
|
||||||
|
|------|-------------|
|
||||||
|
| `list_projects` | All projects with version numbers |
|
||||||
|
| `get_project_context` | Compiled markdown of all context files for a project |
|
||||||
|
| `search_context` | FTS5 full-text search across all projects |
|
||||||
|
| `get_project_tags` | Metadata tags for a project |
|
||||||
|
| `list_files` | All context files in a project (multi-file mode) |
|
||||||
|
| `get_file` | Single file with metadata header |
|
||||||
|
|
||||||
|
### Write tools (require `ctxd.write` on the same connector)
|
||||||
|
|
||||||
|
| Tool | Description |
|
||||||
|
|------|-------------|
|
||||||
|
| `update_file` | Update a single context file with optimistic version checking |
|
||||||
|
| `set_project_tags` | Replace all metadata tags for a project |
|
||||||
|
| `sync_to_project` | Write CONTEXT.MD as AGENTS.md + symlinks to project root |
|
||||||
|
|
||||||
|
## Workflow: Reading Context
|
||||||
|
|
||||||
|
```
|
||||||
|
1. get_client_guide() → read the locked client guide first
|
||||||
|
2. list_projects() → find the project slug
|
||||||
|
3. get_project_context(project_id) → read compiled context
|
||||||
|
or list_files() + get_file() → read individual files
|
||||||
|
```
|
||||||
|
|
||||||
|
The compiled view returns all files concatenated with `## FILENAME` headers and a metadata block at the top.
|
||||||
|
|
||||||
|
## Workflow: Updating Context
|
||||||
|
|
||||||
|
```
|
||||||
|
1. get_file(project_id, file_path) → get current content + version
|
||||||
|
2. update_file(project_id, file_path, content, base_version)
|
||||||
|
→ base_version must match current version (optimistic locking)
|
||||||
|
→ returns {"ok": true, "version": N+1} or {"ok": false, "error": "version_conflict"}
|
||||||
|
3. On conflict: re-read, merge, retry
|
||||||
|
```
|
||||||
|
|
||||||
|
### What to Update
|
||||||
|
|
||||||
|
- **CONTEXT.MD** — canonical project overview (synced as AGENTS.md to repos)
|
||||||
|
- **DECISIONS.MD** — architecture decisions, rationale
|
||||||
|
- **RUNBOOKS.MD** — deploy, troubleshoot, operate procedures
|
||||||
|
- **PROMPTS.MD** — project-specific prompts for different harnesses
|
||||||
|
- **GLOSSARY.MD** — project-specific terms, acronyms
|
||||||
|
|
||||||
|
### What NOT to Put in Context
|
||||||
|
|
||||||
|
- Session progress logs (use session_search or audit trail)
|
||||||
|
- Temporary TODO state
|
||||||
|
- Single-session debugging notes
|
||||||
|
- Secrets, keys, tokens
|
||||||
|
|
||||||
|
## Locked Files
|
||||||
|
|
||||||
|
| File | Protection |
|
||||||
|
|------|-----------|
|
||||||
|
| `CONTEXT.MD` | Cannot delete — canonical synced file |
|
||||||
|
| `LLM-CLIENT.MD` | Cannot update or delete — locked client guide |
|
||||||
|
|
||||||
|
Both return `403 cannot_update_locked` on PUT and `400 cannot_delete_context` on DELETE.
|
||||||
|
|
||||||
|
## Discipline
|
||||||
|
|
||||||
|
1. **Call `get_client_guide()` first** in every session
|
||||||
|
2. **Read context at the start of every session** for the project you're working on
|
||||||
|
3. **Update context immediately** when a durable fact changes — not "next session"
|
||||||
|
4. **Use `base_version`** to avoid overwriting concurrent edits
|
||||||
|
5. **Don't put session progress in project context** — that's what session history is for
|
||||||
|
|
||||||
|
## Error Handling
|
||||||
|
|
||||||
|
| Error | Meaning | Action |
|
||||||
|
|-------|---------|--------|
|
||||||
|
| `version_conflict` | Someone else updated the file | Re-read, merge, retry with new `base_version` |
|
||||||
|
| `not_found` | Project or file doesn't exist | Check slug spelling; create project via Web UI |
|
||||||
|
| `invalid_grant` | OAuth token expired or wrong scope | Refresh token or re-authorize with correct scope |
|
||||||
|
| `forbidden` | Wrong scope for the tool | Request `ctxd.write` for write tools; `ctxd.read` for read tools (same MCP URL) |
|
||||||
|
| `cannot_delete_context` | Tried to delete CONTEXT.MD or LLM-CLIENT.MD | Protected file — update content instead |
|
||||||
|
| `cannot_update_locked` | Tried to update LLM-CLIENT.MD | Locked guide — ask admin to update via Web UI |
|
||||||
|
|
||||||
|
## Verification Checklist
|
||||||
|
|
||||||
|
- [ ] OAuth client registered with correct redirect URI
|
||||||
|
- [ ] Admin approved the authorization (browser session or approval key)
|
||||||
|
- [ ] Access token obtained with `ctxd.read` (and `ctxd.write` for updates)
|
||||||
|
- [ ] `get_client_guide` returns the LLM client guide
|
||||||
|
- [ ] `list_projects` returns expected projects
|
||||||
|
- [ ] `get_project_context` returns compiled markdown with metadata header
|
||||||
|
- [ ] `update_file` succeeds with correct `base_version`
|
||||||
|
- [ ] Audit log shows the write operation
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
# ════════════════════════════════════════════════════════════════════
|
||||||
|
# CTXD — Context Dossier Environment Configuration
|
||||||
|
# Copy to .env and fill in your values. All variables are optional
|
||||||
|
# unless running in Docker (DATABASE_URL, CTXD_API_KEY, OAUTH_* required).
|
||||||
|
#
|
||||||
|
# Precedence: env var > ctxd.yaml (in /data) > built-in defaults
|
||||||
|
# ════════════════════════════════════════════════════════════════════
|
||||||
|
|
||||||
|
# ── Database ────────────────────────────────────────────────────────
|
||||||
|
# Full PostgreSQL connection string. If empty, CTXD falls back to SQLite
|
||||||
|
# at /data/ctxd.db (or CTXD_HOME/ctxd.db).
|
||||||
|
DATABASE_URL=postgresql://ctxd:ctxd_local_dev@postgres:5432/ctxd
|
||||||
|
|
||||||
|
# Postgres container settings (only used when running the bundled PG)
|
||||||
|
POSTGRES_USER=ctxd
|
||||||
|
POSTGRES_PASSWORD=ctxd_local_dev
|
||||||
|
POSTGRES_DB=ctxd
|
||||||
|
|
||||||
|
# ── Server ──────────────────────────────────────────────────────────
|
||||||
|
# Bind address and port inside the container
|
||||||
|
CTXD_HOST=0.0.0.0
|
||||||
|
CTXD_PORT=9091
|
||||||
|
|
||||||
|
# Data directory (inside container)
|
||||||
|
CTXD_HOME=/data
|
||||||
|
|
||||||
|
# Uvicorn log level: debug, info, warning, error
|
||||||
|
LOG_LEVEL=info
|
||||||
|
|
||||||
|
# ── Auth ────────────────────────────────────────────────────────────
|
||||||
|
# Enable authentication globally
|
||||||
|
CTXD_AUTH_ENABLED=true
|
||||||
|
|
||||||
|
# Shared API key for Hermes/internal MCP + HTTP auth
|
||||||
|
CTXD_API_KEY=
|
||||||
|
|
||||||
|
# Legacy read-only key for /readonly/sse ?key= migration
|
||||||
|
CTXD_EXTERNAL_READONLY_KEY=
|
||||||
|
|
||||||
|
# ── OAuth ───────────────────────────────────────────────────────────
|
||||||
|
# Enable the OAuth authorization server
|
||||||
|
OAUTH_ENABLED=true
|
||||||
|
|
||||||
|
# Public URL of the server (used in OAuth discovery metadata)
|
||||||
|
OAUTH_ISSUER=https://ctxd.example.com
|
||||||
|
|
||||||
|
# Fallback approval key for /oauth/authorize
|
||||||
|
OAUTH_APPROVAL_KEY=
|
||||||
|
|
||||||
|
# Which user ID to attribute OAuth approvals to
|
||||||
|
OAUTH_APPROVAL_USER_ID=admin
|
||||||
|
|
||||||
|
# Token lifetimes in seconds
|
||||||
|
OAUTH_ACCESS_TOKEN_TTL=3600
|
||||||
|
OAUTH_REFRESH_TOKEN_TTL=2592000
|
||||||
|
|
||||||
|
# ── Web Sessions ────────────────────────────────────────────────────
|
||||||
|
# Session cookie lifetime in seconds (default: 7 days)
|
||||||
|
WEB_SESSION_TTL=604800
|
||||||
|
|
||||||
|
# ── Snapshots ───────────────────────────────────────────────────────
|
||||||
|
# Min/max snapshots retained per project before rotation
|
||||||
|
SNAPSHOT_MIN_KEEP=5
|
||||||
|
SNAPSHOT_MAX_KEEP=25
|
||||||
+32
-3
@@ -1,17 +1,46 @@
|
|||||||
name: ctxd
|
name: ctxd
|
||||||
|
|
||||||
|
# Docker Compose reads .env automatically for variable substitution.
|
||||||
|
# See .env.example for all available variables.
|
||||||
|
#
|
||||||
|
# IMPORTANT: Production uses PostgreSQL (DATABASE_URL host "postgres").
|
||||||
|
# Always start BOTH services: docker compose up -d
|
||||||
|
# Or use: ./scripts/deploy.sh
|
||||||
|
# Do NOT use docker compose up -d --no-deps ctxd unless postgres is already running.
|
||||||
|
|
||||||
services:
|
services:
|
||||||
dossier:
|
postgres:
|
||||||
|
image: postgres:16-alpine
|
||||||
|
container_name: ctxd-postgres
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
POSTGRES_DB: ${POSTGRES_DB:-ctxd}
|
||||||
|
POSTGRES_USER: ${POSTGRES_USER:-ctxd}
|
||||||
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-ctxd_local_dev}
|
||||||
|
volumes:
|
||||||
|
- ${CTXD_PG_DATA:-/mnt/ai-storage/projects/ctxd/data/pg}:/var/lib/postgresql/data
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-ctxd}"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 5
|
||||||
|
|
||||||
|
ctxd:
|
||||||
build:
|
build:
|
||||||
context: .
|
context: .
|
||||||
dockerfile: Dockerfile
|
dockerfile: Dockerfile
|
||||||
container_name: dossier
|
container_name: ctxd
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
ports:
|
ports:
|
||||||
- "9091:9091"
|
- "${CTXD_PORT:-9091}:${CTXD_PORT:-9091}"
|
||||||
volumes:
|
volumes:
|
||||||
- /mnt/ai-storage/projects/ctxd/data:/data:rw
|
- /mnt/ai-storage/projects/ctxd/data:/data:rw
|
||||||
- /mnt/ai-storage/projects:/projects:ro
|
- /mnt/ai-storage/projects:/projects:ro
|
||||||
- /home/overseer:/host:ro
|
- /home/overseer:/host:ro
|
||||||
|
env_file:
|
||||||
|
- .env
|
||||||
environment:
|
environment:
|
||||||
- CTXD_HOME=/data
|
- CTXD_HOME=/data
|
||||||
|
depends_on:
|
||||||
|
postgres:
|
||||||
|
condition: service_healthy
|
||||||
+71
-5
@@ -1,11 +1,77 @@
|
|||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
set -e
|
set -e
|
||||||
|
|
||||||
# Initialize if database doesn't exist
|
wait_for_postgres() {
|
||||||
if [ ! -f "$CTXD_HOME/ctxd.db" ]; then
|
if [ -z "$DATABASE_URL" ]; then
|
||||||
echo "ctxd: initializing database at $CTXD_HOME"
|
return 0
|
||||||
python3 -m ctxd init --home "$CTXD_HOME"
|
fi
|
||||||
|
echo "ctxd: waiting for PostgreSQL (DATABASE_URL is set)..."
|
||||||
|
python3 <<'PY'
|
||||||
|
import os, sys, time
|
||||||
|
|
||||||
|
url = os.environ.get("DATABASE_URL", "")
|
||||||
|
max_wait = int(os.environ.get("CTXD_PG_WAIT_SECONDS", "120"))
|
||||||
|
interval = int(os.environ.get("CTXD_PG_WAIT_INTERVAL", "2"))
|
||||||
|
deadline = time.time() + max_wait
|
||||||
|
last_err = ""
|
||||||
|
attempt = 0
|
||||||
|
while time.time() < deadline:
|
||||||
|
attempt += 1
|
||||||
|
try:
|
||||||
|
import psycopg
|
||||||
|
conn = psycopg.connect(url, connect_timeout=5)
|
||||||
|
conn.close()
|
||||||
|
print("ctxd: PostgreSQL is ready")
|
||||||
|
sys.exit(0)
|
||||||
|
except Exception as e:
|
||||||
|
last_err = str(e)
|
||||||
|
if attempt == 1 or attempt % 10 == 0:
|
||||||
|
print(f"ctxd: postgres not ready (attempt {attempt}): {last_err}", file=sys.stderr)
|
||||||
|
time.sleep(interval)
|
||||||
|
|
||||||
|
print("ctxd: FATAL: PostgreSQL not reachable within wait window.", file=sys.stderr)
|
||||||
|
print(f"ctxd: last error: {last_err}", file=sys.stderr)
|
||||||
|
print("ctxd:", file=sys.stderr)
|
||||||
|
print("ctxd: Fix: start the full stack (postgres + ctxd), e.g.", file=sys.stderr)
|
||||||
|
print("ctxd: cd app && docker compose up -d postgres ctxd", file=sys.stderr)
|
||||||
|
print("ctxd:", file=sys.stderr)
|
||||||
|
print("ctxd: Avoid: docker compose up -d --no-deps ctxd (skips postgres)", file=sys.stderr)
|
||||||
|
print("ctxd: If postgres is on another host, fix DATABASE_URL / network.", file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
wait_for_postgres
|
||||||
|
|
||||||
|
# Initialize if needed
|
||||||
|
if [ -n "$DATABASE_URL" ]; then
|
||||||
|
# PostgreSQL mode — check if schema exists
|
||||||
|
NEEDS_INIT=$(python3 -c "
|
||||||
|
import psycopg, os, sys
|
||||||
|
try:
|
||||||
|
conn = psycopg.connect(os.environ['DATABASE_URL'])
|
||||||
|
cur = conn.execute(\"SELECT EXISTS (SELECT FROM pg_tables WHERE schemaname='public' AND tablename='users')\")
|
||||||
|
if not cur.fetchone()[0]:
|
||||||
|
print('yes')
|
||||||
|
else:
|
||||||
|
print('no')
|
||||||
|
except Exception:
|
||||||
|
print('yes')
|
||||||
|
" 2>/dev/null || echo "yes")
|
||||||
|
if [ "$NEEDS_INIT" = "yes" ]; then
|
||||||
|
echo "ctxd: initializing PostgreSQL database"
|
||||||
|
python3 -m ctxd init --home "$CTXD_HOME"
|
||||||
|
else
|
||||||
|
echo "ctxd: running pending migrations"
|
||||||
|
python3 -m ctxd.migrate_user_fk_set_null 2>&1 || echo "ctxd: migration warning (non-fatal)"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
# SQLite mode — check for db file
|
||||||
|
if [ ! -f "$CTXD_HOME/ctxd.db" ]; then
|
||||||
|
echo "ctxd: initializing database at $CTXD_HOME"
|
||||||
|
python3 -m ctxd init --home "$CTXD_HOME"
|
||||||
|
fi
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "ctxd: starting daemon on 0.0.0.0:9091"
|
echo "ctxd: starting daemon on ${CTXD_HOST:-0.0.0.0}:${CTXD_PORT:-9091}"
|
||||||
exec python3 -m ctxd
|
exec python3 -m ctxd
|
||||||
@@ -3,6 +3,12 @@ name = "context-dossier"
|
|||||||
version = "0.2.0"
|
version = "0.2.0"
|
||||||
description = "Context Dossier — single source of truth for multi-harness project context"
|
description = "Context Dossier — single source of truth for multi-harness project context"
|
||||||
requires-python = ">=3.11"
|
requires-python = ">=3.11"
|
||||||
|
dependencies = [
|
||||||
|
"psycopg[binary]>=3.1",
|
||||||
|
"mcp>=1.28",
|
||||||
|
"uvicorn>=0.30",
|
||||||
|
"pyyaml>=6.0",
|
||||||
|
]
|
||||||
|
|
||||||
[project.scripts]
|
[project.scripts]
|
||||||
dossier = "ctxd:cli_entry"
|
dossier = "ctxd:cli_entry"
|
||||||
|
|||||||
Executable
+46
@@ -0,0 +1,46 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Safe CTXD deploy: always ensures postgres is up before recreating ctxd.
|
||||||
|
# Usage: ./scripts/deploy.sh [--build-only | --no-build]
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||||
|
cd "$ROOT"
|
||||||
|
|
||||||
|
BUILD=1
|
||||||
|
if [[ "${1:-}" == "--no-build" ]]; then
|
||||||
|
BUILD=0
|
||||||
|
elif [[ "${1:-}" == "--build-only" ]]; then
|
||||||
|
docker compose build ctxd
|
||||||
|
echo "Built ctxd image only."
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$BUILD" -eq 1 ]]; then
|
||||||
|
echo "==> docker compose build ctxd"
|
||||||
|
docker compose build ctxd
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "==> docker compose up -d postgres"
|
||||||
|
docker compose up -d postgres
|
||||||
|
|
||||||
|
echo "==> waiting for postgres healthy..."
|
||||||
|
for i in $(seq 1 60); do
|
||||||
|
if docker compose ps postgres 2>/dev/null | grep -q '(healthy)'; then
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
sleep 2
|
||||||
|
done
|
||||||
|
if ! docker compose ps postgres 2>/dev/null | grep -q '(healthy)'; then
|
||||||
|
echo "ERROR: ctxd-postgres did not become healthy. Check: docker compose logs postgres" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "==> docker compose up -d --force-recreate ctxd"
|
||||||
|
docker compose up -d --force-recreate ctxd
|
||||||
|
|
||||||
|
PORT="${CTXD_PORT:-9091}"
|
||||||
|
echo "==> smoke: http://127.0.0.1:${PORT}/status"
|
||||||
|
sleep 3
|
||||||
|
curl -sf "http://127.0.0.1:${PORT}/status" | head -c 200
|
||||||
|
echo ""
|
||||||
|
docker compose ps postgres ctxd
|
||||||
@@ -0,0 +1,150 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Smoke test: one OAuth connector lists read+write tools on /mcp."""
|
||||||
|
import json, urllib.request, urllib.parse, hashlib, base64, secrets, http.client
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
BASE = "http://127.0.0.1:9091"
|
||||||
|
env_path = Path("/mnt/ai-storage/projects/ctxd/app/.env")
|
||||||
|
approval_key = ""
|
||||||
|
for line in env_path.read_text().splitlines():
|
||||||
|
if line.startswith("OAUTH_APPROVAL_KEY="):
|
||||||
|
approval_key = line.split("=", 1)[1]
|
||||||
|
break
|
||||||
|
if not approval_key:
|
||||||
|
raise SystemExit("no OAUTH_APPROVAL_KEY")
|
||||||
|
|
||||||
|
|
||||||
|
class NR(urllib.request.HTTPRedirectHandler):
|
||||||
|
def redirect_request(self, req, fp, code, msg, headers, newurl):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
opener = urllib.request.build_opener(NR)
|
||||||
|
r = opener.open(
|
||||||
|
urllib.request.Request(
|
||||||
|
f"{BASE}/oauth/register",
|
||||||
|
data=json.dumps(
|
||||||
|
{
|
||||||
|
"redirect_uris": ["http://localhost:9999/cb"],
|
||||||
|
"client_name": "unified-mcp-test",
|
||||||
|
"scope": "ctxd.read ctxd.write",
|
||||||
|
}
|
||||||
|
).encode(),
|
||||||
|
headers={"Content-Type": "application/json"},
|
||||||
|
method="POST",
|
||||||
|
),
|
||||||
|
timeout=15,
|
||||||
|
)
|
||||||
|
client = json.loads(r.read())
|
||||||
|
cid, secret = client["client_id"], client["client_secret"]
|
||||||
|
|
||||||
|
cv = secrets.token_urlsafe(32)
|
||||||
|
cc = base64.urlsafe_b64encode(hashlib.sha256(cv.encode()).digest()).rstrip(b"=").decode()
|
||||||
|
params = {
|
||||||
|
"response_type": "code",
|
||||||
|
"client_id": cid,
|
||||||
|
"redirect_uri": "http://localhost:9999/cb",
|
||||||
|
"code_challenge": cc,
|
||||||
|
"code_challenge_method": "S256",
|
||||||
|
"scope": "ctxd.read ctxd.write",
|
||||||
|
"state": "t",
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
opener.open(
|
||||||
|
urllib.request.Request(
|
||||||
|
f"{BASE}/oauth/authorize?" + urllib.parse.urlencode(params),
|
||||||
|
data=urllib.parse.urlencode({"approval_key": approval_key}).encode(),
|
||||||
|
headers={"Content-Type": "application/x-www-form-urlencoded"},
|
||||||
|
method="POST",
|
||||||
|
),
|
||||||
|
timeout=15,
|
||||||
|
)
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
loc = e.headers.get("Location", "")
|
||||||
|
code = dict(urllib.parse.parse_qsl(urllib.parse.urlparse(loc).query)).get("code", "")
|
||||||
|
|
||||||
|
r3 = opener.open(
|
||||||
|
urllib.request.Request(
|
||||||
|
f"{BASE}/oauth/token",
|
||||||
|
data=urllib.parse.urlencode(
|
||||||
|
{
|
||||||
|
"grant_type": "authorization_code",
|
||||||
|
"code": code,
|
||||||
|
"redirect_uri": "http://localhost:9999/cb",
|
||||||
|
"client_id": cid,
|
||||||
|
"client_secret": secret,
|
||||||
|
"code_verifier": cv,
|
||||||
|
}
|
||||||
|
).encode(),
|
||||||
|
headers={"Content-Type": "application/x-www-form-urlencoded"},
|
||||||
|
method="POST",
|
||||||
|
),
|
||||||
|
timeout=15,
|
||||||
|
)
|
||||||
|
token = json.loads(r3.read())["access_token"]
|
||||||
|
|
||||||
|
|
||||||
|
def mcp_post(path: str, body: dict, session_id: str | None = None) -> tuple[int, str, str | None]:
|
||||||
|
headers = {
|
||||||
|
"Authorization": f"Bearer {token}",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"Accept": "application/json, text/event-stream",
|
||||||
|
}
|
||||||
|
if session_id:
|
||||||
|
headers["mcp-session-id"] = session_id
|
||||||
|
conn = http.client.HTTPConnection("127.0.0.1", 9091, timeout=15)
|
||||||
|
conn.request("POST", path, body=json.dumps(body), headers=headers)
|
||||||
|
resp = conn.getresponse()
|
||||||
|
sid = resp.getheader("mcp-session-id")
|
||||||
|
data = resp.read(8000).decode(errors="replace")
|
||||||
|
conn.close()
|
||||||
|
return resp.status, data, sid
|
||||||
|
|
||||||
|
|
||||||
|
init_body = {
|
||||||
|
"jsonrpc": "2.0",
|
||||||
|
"id": 1,
|
||||||
|
"method": "initialize",
|
||||||
|
"params": {
|
||||||
|
"protocolVersion": "2025-03-26",
|
||||||
|
"capabilities": {},
|
||||||
|
"clientInfo": {"name": "unified-test", "version": "1"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
st, raw, sid = mcp_post("/mcp", init_body)
|
||||||
|
print("initialize:", st, "session:", sid)
|
||||||
|
if st != 200:
|
||||||
|
raise SystemExit(raw[:500])
|
||||||
|
|
||||||
|
if sid:
|
||||||
|
st2, raw2, _ = mcp_post(
|
||||||
|
"/mcp",
|
||||||
|
{"jsonrpc": "2.0", "id": 2, "method": "tools/list", "params": {}},
|
||||||
|
session_id=sid.split(",")[0].strip(),
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
st2, raw2, _ = mcp_post("/mcp", {"jsonrpc": "2.0", "id": 2, "method": "tools/list", "params": {}})
|
||||||
|
|
||||||
|
print("tools/list:", st2)
|
||||||
|
names = []
|
||||||
|
for line in raw2.splitlines():
|
||||||
|
if line.startswith("data:"):
|
||||||
|
try:
|
||||||
|
payload = json.loads(line[5:].strip())
|
||||||
|
for t in payload.get("result", {}).get("tools", []):
|
||||||
|
names.append(t.get("name"))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
if '"tools"' in raw2 and not names:
|
||||||
|
try:
|
||||||
|
payload = json.loads(raw2)
|
||||||
|
names = [t["name"] for t in payload.get("result", {}).get("tools", [])]
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
print("tools:", sorted(set(names)))
|
||||||
|
need = {"list_projects", "update_file", "sync_to_project"}
|
||||||
|
missing = need - set(names)
|
||||||
|
if missing:
|
||||||
|
raise SystemExit(f"missing tools: {missing}")
|
||||||
|
print("OK unified connector exposes read+write tools")
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""E2E: OAuth write token -> /write/sse and tools/list via SSE."""
|
||||||
|
import json, urllib.request, urllib.parse, hashlib, base64, secrets, http.client, time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
BASE = "http://127.0.0.1:9091"
|
||||||
|
env_path = Path("/mnt/ai-storage/projects/ctxd/app/.env")
|
||||||
|
approval_key = ""
|
||||||
|
for line in env_path.read_text().splitlines():
|
||||||
|
if line.startswith("OAUTH_APPROVAL_KEY="):
|
||||||
|
approval_key = line.split("=", 1)[1]
|
||||||
|
break
|
||||||
|
if not approval_key:
|
||||||
|
raise SystemExit("no OAUTH_APPROVAL_KEY")
|
||||||
|
|
||||||
|
class NR(urllib.request.HTTPRedirectHandler):
|
||||||
|
def redirect_request(self, *a): return None
|
||||||
|
opener = urllib.request.build_opener(NR)
|
||||||
|
|
||||||
|
r = opener.open(urllib.request.Request(
|
||||||
|
f"{BASE}/oauth/register",
|
||||||
|
data=json.dumps({"redirect_uris": ["http://localhost:9999/cb"], "client_name": "write-test", "scope": "ctxd.read ctxd.write"}).encode(),
|
||||||
|
headers={"Content-Type": "application/json"}, method="POST"), timeout=15)
|
||||||
|
client = json.loads(r.read())
|
||||||
|
cid, secret = client["client_id"], client["client_secret"]
|
||||||
|
|
||||||
|
cv = secrets.token_urlsafe(32)
|
||||||
|
cc = base64.urlsafe_b64encode(hashlib.sha256(cv.encode()).digest()).rstrip(b"=").decode()
|
||||||
|
params = {"response_type": "code", "client_id": cid, "redirect_uri": "http://localhost:9999/cb",
|
||||||
|
"code_challenge": cc, "code_challenge_method": "S256", "scope": "ctxd.read ctxd.write", "state": "t"}
|
||||||
|
try:
|
||||||
|
opener.open(urllib.request.Request(f"{BASE}/oauth/authorize?" + urllib.parse.urlencode(params),
|
||||||
|
data=urllib.parse.urlencode({"approval_key": approval_key}).encode(),
|
||||||
|
headers={"Content-Type": "application/x-www-form-urlencoded"}, method="POST"), timeout=15)
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
loc = e.headers.get("Location", "")
|
||||||
|
code = dict(urllib.parse.parse_qsl(urllib.parse.urlparse(loc).query)).get("code", "")
|
||||||
|
|
||||||
|
r3 = opener.open(urllib.request.Request(f"{BASE}/oauth/token",
|
||||||
|
data=urllib.parse.urlencode({"grant_type": "authorization_code", "code": code, "redirect_uri": "http://localhost:9999/cb",
|
||||||
|
"client_id": cid, "client_secret": secret, "code_verifier": cv}).encode(),
|
||||||
|
headers={"Content-Type": "application/x-www-form-urlencoded"}, method="POST"), timeout=15)
|
||||||
|
token = json.loads(r3.read())["access_token"]
|
||||||
|
|
||||||
|
conn = http.client.HTTPConnection("127.0.0.1", 9091, timeout=5)
|
||||||
|
conn.request("GET", "/write/sse", headers={"Authorization": f"Bearer {token}"})
|
||||||
|
resp = conn.getresponse()
|
||||||
|
print("write/sse status:", resp.status, "ctype:", resp.getheader("content-type"))
|
||||||
|
chunk = resp.read(500).decode(errors="replace")
|
||||||
|
print("body[:200]:", chunk[:200])
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
# POST initialize to /write/messages (needs session from SSE - simplified check)
|
||||||
|
conn2 = http.client.HTTPConnection("127.0.0.1", 9091, timeout=10)
|
||||||
|
body = json.dumps({"jsonrpc": "2.0", "id": 1, "method": "tools/list", "params": {}})
|
||||||
|
conn2.request("POST", "/write/messages", body=body,
|
||||||
|
headers={"Authorization": f"Bearer {token}", "Content-Type": "application/json"})
|
||||||
|
r2 = conn2.getresponse()
|
||||||
|
print("write/messages POST:", r2.status, r2.read(300).decode(errors="replace")[:200])
|
||||||
|
conn2.close()
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
Metadata-Version: 2.4
|
|
||||||
Name: ctxd
|
|
||||||
Version: 0.1.0
|
|
||||||
Summary: Context daemon — single source of truth for multi-harness project context
|
|
||||||
Requires-Python: >=3.11
|
|
||||||
@@ -1,13 +0,0 @@
|
|||||||
pyproject.toml
|
|
||||||
src/ctxd/__init__.py
|
|
||||||
src/ctxd/__main__.py
|
|
||||||
src/ctxd/cli.py
|
|
||||||
src/ctxd/config.py
|
|
||||||
src/ctxd/db.py
|
|
||||||
src/ctxd/mcp_stdio.py
|
|
||||||
src/ctxd/server.py
|
|
||||||
src/ctxd.egg-info/PKG-INFO
|
|
||||||
src/ctxd.egg-info/SOURCES.txt
|
|
||||||
src/ctxd.egg-info/dependency_links.txt
|
|
||||||
src/ctxd.egg-info/entry_points.txt
|
|
||||||
src/ctxd.egg-info/top_level.txt
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
|
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
[console_scripts]
|
|
||||||
ctx = ctxd:cli_entry
|
|
||||||
ctxd = ctxd:daemon_entry
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
ctxd
|
|
||||||
@@ -4,7 +4,10 @@ import sys
|
|||||||
|
|
||||||
if len(sys.argv) > 1 and sys.argv[1] in ('init', 'serve', 'project-list', 'project-create',
|
if len(sys.argv) > 1 and sys.argv[1] in ('init', 'serve', 'project-list', 'project-create',
|
||||||
'read', 'cat', 'edit', 'search', 'sync', 'audit',
|
'read', 'cat', 'edit', 'search', 'sync', 'audit',
|
||||||
'user-list', 'user-create', 'import-vault'):
|
'user-list', 'user-create', 'import-vault',
|
||||||
|
'user-set-password', 'oauth-client-create',
|
||||||
|
'oauth-client-list', 'oauth-client-revoke',
|
||||||
|
'file-list', 'file-read'):
|
||||||
from ctxd.cli import cli_entry
|
from ctxd.cli import cli_entry
|
||||||
cli_entry()
|
cli_entry()
|
||||||
else:
|
else:
|
||||||
@@ -12,7 +15,9 @@ else:
|
|||||||
from ctxd.server import serve_sync
|
from ctxd.server import serve_sync
|
||||||
|
|
||||||
cfg = CtxConfig.from_home()
|
cfg = CtxConfig.from_home()
|
||||||
if not cfg.db_path.exists():
|
# When using PostgreSQL, the DB is external — skip the file check.
|
||||||
|
# When using SQLite, verify the db file exists.
|
||||||
|
if not cfg.use_postgres and not cfg.db_path.exists():
|
||||||
print("Not initialized. Run 'ctx init' first.", file=sys.stderr)
|
print("Not initialized. Run 'ctx init' first.", file=sys.stderr)
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
serve_sync(cfg)
|
serve_sync(cfg)
|
||||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,40 @@
|
|||||||
|
"""Password hashing for CTXD user accounts (stdlib only)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import secrets
|
||||||
|
|
||||||
|
PBKDF2_ITERATIONS = 600_000
|
||||||
|
SCHEME = "pbkdf2_sha256"
|
||||||
|
|
||||||
|
|
||||||
|
def hash_password(password: str) -> str:
|
||||||
|
if not password:
|
||||||
|
raise ValueError("password required")
|
||||||
|
salt = secrets.token_hex(16)
|
||||||
|
digest = hashlib.pbkdf2_hmac(
|
||||||
|
"sha256",
|
||||||
|
password.encode("utf-8"),
|
||||||
|
salt.encode("utf-8"),
|
||||||
|
PBKDF2_ITERATIONS,
|
||||||
|
)
|
||||||
|
return f"{SCHEME}${PBKDF2_ITERATIONS}${salt}${digest.hex()}"
|
||||||
|
|
||||||
|
|
||||||
|
def verify_password(password: str, token_hash: str | None) -> bool:
|
||||||
|
if not password or not token_hash:
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
scheme, iters_s, salt, expected_hex = token_hash.split("$", 3)
|
||||||
|
if scheme != SCHEME:
|
||||||
|
return False
|
||||||
|
digest = hashlib.pbkdf2_hmac(
|
||||||
|
"sha256",
|
||||||
|
password.encode("utf-8"),
|
||||||
|
salt.encode("utf-8"),
|
||||||
|
int(iters_s),
|
||||||
|
)
|
||||||
|
return secrets.compare_digest(digest.hex(), expected_hex)
|
||||||
|
except (ValueError, TypeError):
|
||||||
|
return False
|
||||||
+109
-10
@@ -64,14 +64,15 @@ Multi-camera remote monitoring system.
|
|||||||
|
|
||||||
def _seed_context(conn):
|
def _seed_context(conn):
|
||||||
"""Insert seed project context with real newlines."""
|
"""Insert seed project context with real newlines."""
|
||||||
conn.execute(
|
from ctxd.db import _is_pg, _ph
|
||||||
"INSERT OR IGNORE INTO project_context (project_id, content, version, updated_by) VALUES (?, ?, 0, 'system')",
|
ph = _ph(conn, 2)
|
||||||
('welcome', _WELCOME_CONTENT),
|
placeholders = ph.split(", ")
|
||||||
)
|
if _is_pg(conn):
|
||||||
conn.execute(
|
sql = f"INSERT INTO project_context (project_id, content, version, updated_by) VALUES ({placeholders[0]}, {placeholders[1]}, 0, 'admin') ON CONFLICT DO NOTHING"
|
||||||
"INSERT OR IGNORE INTO project_context (project_id, content, version, updated_by) VALUES (?, ?, 0, 'system')",
|
else:
|
||||||
('remote-rig', _REMOTE_RIG_CONTENT),
|
sql = f"INSERT OR IGNORE INTO project_context (project_id, content, version, updated_by) VALUES ({placeholders[0]}, {placeholders[1]}, 0, 'admin')"
|
||||||
)
|
conn.execute(sql, ('welcome', _WELCOME_CONTENT))
|
||||||
|
conn.execute(sql, ('remote-rig', _REMOTE_RIG_CONTENT))
|
||||||
|
|
||||||
|
|
||||||
def cmd_init(args):
|
def cmd_init(args):
|
||||||
@@ -309,16 +310,87 @@ def cmd_user_create(args):
|
|||||||
"""Create a new user."""
|
"""Create a new user."""
|
||||||
conn = _db.init_db(CtxConfig.from_home(args.home))
|
conn = _db.init_db(CtxConfig.from_home(args.home))
|
||||||
try:
|
try:
|
||||||
_db.user_create(conn, args.user_id, args.display_name, args.role)
|
_db.user_create(conn, args.user_id, args.display_name, args.role, password=getattr(args, "password", None))
|
||||||
conn.commit()
|
conn.commit()
|
||||||
print(f"✓ User '{args.user_id}' created.")
|
print(f"✓ User '{args.user_id}' created.")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
conn.rollback()
|
conn.rollback()
|
||||||
print(f"Error: {e}")
|
print(f"✗ {e}")
|
||||||
|
sys.exit(1)
|
||||||
finally:
|
finally:
|
||||||
conn.close()
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_user_set_password(args):
|
||||||
|
"""Set or reset a user's Web UI password."""
|
||||||
|
conn = _db.init_db(CtxConfig.from_home(args.home))
|
||||||
|
try:
|
||||||
|
if _db.user_get(conn, args.user_id) is None:
|
||||||
|
print(f"✗ User '{args.user_id}' not found.")
|
||||||
|
sys.exit(1)
|
||||||
|
_db.user_set_password(conn, args.user_id, args.password)
|
||||||
|
conn.commit()
|
||||||
|
print(f"✓ Password set for '{args.user_id}'.")
|
||||||
|
except Exception as e:
|
||||||
|
conn.rollback()
|
||||||
|
print(f"✗ {e}")
|
||||||
|
sys.exit(1)
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_oauth_client_create(args):
|
||||||
|
"""Register an OAuth client for Claude (or other MCP connectors)."""
|
||||||
|
from .server import OAuthStore, CLAUDE_MCP_REDIRECT_URI
|
||||||
|
|
||||||
|
cfg = CtxConfig.from_home(args.home)
|
||||||
|
store = OAuthStore(cfg)
|
||||||
|
redirects = args.redirect_uri or [CLAUDE_MCP_REDIRECT_URI]
|
||||||
|
if isinstance(redirects, str):
|
||||||
|
redirects = [redirects]
|
||||||
|
client = store.register_client({
|
||||||
|
"client_name": args.name or "Claude MCP Client",
|
||||||
|
"redirect_uris": redirects,
|
||||||
|
"scopes": getattr(args, "scopes", None) or (
|
||||||
|
[s for s in (args.scope or "").split() if s]
|
||||||
|
or ["ctxd.read", "ctxd.write"]
|
||||||
|
),
|
||||||
|
})
|
||||||
|
issuer = (cfg.oauth_issuer or "").rstrip("/") or "https://ctxd.cubecraftcreations.com"
|
||||||
|
print(json.dumps({
|
||||||
|
"client_id": client["client_id"],
|
||||||
|
"client_secret": client["client_secret"],
|
||||||
|
"client_name": client.get("client_name"),
|
||||||
|
"redirect_uris": client.get("redirect_uris"),
|
||||||
|
"scope": client.get("scope"),
|
||||||
|
"connector_url": f"{issuer}/mcp",
|
||||||
|
"authorization_server": issuer,
|
||||||
|
"note": "Claude usually registers via POST /oauth/register automatically; save client_secret now — it is not shown again.",
|
||||||
|
}, indent=2))
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_oauth_client_list(args):
|
||||||
|
"""List OAuth clients (no secrets)."""
|
||||||
|
from .server import OAuthStore
|
||||||
|
|
||||||
|
cfg = CtxConfig.from_home(args.home)
|
||||||
|
store = OAuthStore(cfg)
|
||||||
|
for c in store.list_clients_public():
|
||||||
|
print(f"{c.get('client_id')} {c.get('client_name', '')} scope={c.get('scope', '')} redirects={c.get('redirect_uris')}")
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_oauth_client_revoke(args):
|
||||||
|
"""Revoke an OAuth client and invalidate its tokens."""
|
||||||
|
from .server import OAuthStore
|
||||||
|
|
||||||
|
cfg = CtxConfig.from_home(args.home)
|
||||||
|
store = OAuthStore(cfg)
|
||||||
|
if not store.revoke_client(args.client_id):
|
||||||
|
print(f"✗ client not found: {args.client_id}")
|
||||||
|
sys.exit(1)
|
||||||
|
print(f"✓ Revoked OAuth client {args.client_id}")
|
||||||
|
|
||||||
|
|
||||||
def cmd_import_vault(args):
|
def cmd_import_vault(args):
|
||||||
"""Import context from an existing vault (e.g., OpenClawVault)."""
|
"""Import context from an existing vault (e.g., OpenClawVault)."""
|
||||||
cfg = CtxConfig.from_home(args.home)
|
cfg = CtxConfig.from_home(args.home)
|
||||||
@@ -477,6 +549,33 @@ def build_parser() -> argparse.ArgumentParser:
|
|||||||
sp.add_argument("user_id")
|
sp.add_argument("user_id")
|
||||||
sp.add_argument("--display-name", "-n", required=True)
|
sp.add_argument("--display-name", "-n", required=True)
|
||||||
sp.add_argument("--role", "-r", default="contributor", choices=["admin", "contributor", "service"])
|
sp.add_argument("--role", "-r", default="contributor", choices=["admin", "contributor", "service"])
|
||||||
|
sp.add_argument("--password", help="Optional Web UI login password")
|
||||||
|
sp.add_argument("--home")
|
||||||
|
|
||||||
|
# user set-password
|
||||||
|
sp = sub.add_parser("user-set-password", help="Set a user's Web UI password")
|
||||||
|
sp.set_defaults(func=cmd_user_set_password)
|
||||||
|
sp.add_argument("user_id")
|
||||||
|
sp.add_argument("--password", "-p", required=True)
|
||||||
|
sp.add_argument("--home")
|
||||||
|
|
||||||
|
# oauth-client-create
|
||||||
|
sp = sub.add_parser("oauth-client-create", help="Register OAuth client (Claude MCP); prints client_id and client_secret")
|
||||||
|
sp.set_defaults(func=cmd_oauth_client_create)
|
||||||
|
sp.add_argument("--name", "-n", default="Claude MCP Client", help="Client display name")
|
||||||
|
sp.add_argument("--redirect-uri", action="append", dest="redirect_uri", help="Redirect URI (default: Claude MCP callback; repeat for multiple)")
|
||||||
|
sp.add_argument("--scope", default="ctxd.read ctxd.write", help="Allowed scopes (space-separated: ctxd.read ctxd.write)")
|
||||||
|
sp.add_argument("--home")
|
||||||
|
|
||||||
|
# oauth-client-list
|
||||||
|
sp = sub.add_parser("oauth-client-list", help="List OAuth clients (no secrets)")
|
||||||
|
sp.set_defaults(func=cmd_oauth_client_list)
|
||||||
|
sp.add_argument("--home")
|
||||||
|
|
||||||
|
# oauth-client-revoke
|
||||||
|
sp = sub.add_parser("oauth-client-revoke", help="Revoke OAuth client and invalidate its tokens")
|
||||||
|
sp.set_defaults(func=cmd_oauth_client_revoke)
|
||||||
|
sp.add_argument("client_id", help="client_id to revoke (ctxd_…)")
|
||||||
sp.add_argument("--home")
|
sp.add_argument("--home")
|
||||||
|
|
||||||
# import-vault
|
# import-vault
|
||||||
|
|||||||
+114
-11
@@ -1,13 +1,39 @@
|
|||||||
"""
|
"""
|
||||||
Configuration for ctxd — context daemon.
|
Configuration for ctxd — context daemon.
|
||||||
|
|
||||||
|
Environment variables take precedence over ctxd.yaml, which takes precedence
|
||||||
|
over built-in defaults. This allows fully env-driven deployments (Docker,
|
||||||
|
Compose, Kubernetes) while preserving file-based config for local dev.
|
||||||
"""
|
"""
|
||||||
import os
|
import os
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
def _env_bool(key: str, fallback: bool = False) -> bool:
|
||||||
|
val = os.environ.get(key, "")
|
||||||
|
if not val:
|
||||||
|
return fallback
|
||||||
|
return val.lower() in ("1", "true", "yes", "on")
|
||||||
|
|
||||||
|
|
||||||
|
def _env_int(key: str, fallback: int) -> int:
|
||||||
|
val = os.environ.get(key, "")
|
||||||
|
if not val:
|
||||||
|
return fallback
|
||||||
|
try:
|
||||||
|
return int(val)
|
||||||
|
except ValueError:
|
||||||
|
return fallback
|
||||||
|
|
||||||
|
|
||||||
|
def _env_str(key: str, fallback: str = "") -> str:
|
||||||
|
return os.environ.get(key, fallback)
|
||||||
|
|
||||||
|
|
||||||
# Default home directory (~/.ctx) — overridable via CTXD_HOME env var
|
# Default home directory (~/.ctx) — overridable via CTXD_HOME env var
|
||||||
DEFAULT_HOME = Path(os.environ.get("CTXD_HOME", Path.home() / ".ctx"))
|
DEFAULT_HOME = Path(os.environ.get("CTXD_HOME", Path.home() / ".ctx"))
|
||||||
|
|
||||||
# Defaults for ctxd.yaml
|
# Built-in defaults (lowest precedence)
|
||||||
DEFAULT_CONFIG = {
|
DEFAULT_CONFIG = {
|
||||||
"server": {
|
"server": {
|
||||||
"host": "0.0.0.0",
|
"host": "0.0.0.0",
|
||||||
@@ -20,6 +46,18 @@ DEFAULT_CONFIG = {
|
|||||||
"auth": {
|
"auth": {
|
||||||
"enabled": False,
|
"enabled": False,
|
||||||
"api_key": "",
|
"api_key": "",
|
||||||
|
"external_readonly_key": "",
|
||||||
|
},
|
||||||
|
"oauth": {
|
||||||
|
"enabled": False,
|
||||||
|
"issuer": "",
|
||||||
|
"approval_key": "",
|
||||||
|
"approval_user_id": "admin",
|
||||||
|
"access_token_ttl_seconds": 3600,
|
||||||
|
"refresh_token_ttl_seconds": 2592000,
|
||||||
|
},
|
||||||
|
"web_sessions": {
|
||||||
|
"ttl_seconds": 604800,
|
||||||
},
|
},
|
||||||
"seed": {
|
"seed": {
|
||||||
"admin_user": "admin",
|
"admin_user": "admin",
|
||||||
@@ -31,7 +69,10 @@ DEFAULT_CONFIG = {
|
|||||||
|
|
||||||
|
|
||||||
class CtxConfig:
|
class CtxConfig:
|
||||||
"""Holds resolved paths and config for a ctxd runtime."""
|
"""Holds resolved paths and config for a ctxd runtime.
|
||||||
|
|
||||||
|
Precedence: env var > ctxd.yaml value > built-in default.
|
||||||
|
"""
|
||||||
|
|
||||||
def __init__(self, home: Path | str | None = None, config: dict | None = None):
|
def __init__(self, home: Path | str | None = None, config: dict | None = None):
|
||||||
resolved = Path(home) if home else DEFAULT_HOME
|
resolved = Path(home) if home else DEFAULT_HOME
|
||||||
@@ -59,31 +100,91 @@ class CtxConfig:
|
|||||||
def config_path(self) -> Path:
|
def config_path(self) -> Path:
|
||||||
return self.home / "ctxd.yaml"
|
return self.home / "ctxd.yaml"
|
||||||
|
|
||||||
# ── Config accessors ──────────────────────────────────────────
|
@property
|
||||||
|
def oauth_state_path(self) -> Path:
|
||||||
|
return self.home / "oauth_state.json"
|
||||||
|
|
||||||
|
@property
|
||||||
|
def web_sessions_path(self) -> Path:
|
||||||
|
return self.home / "web_sessions.json"
|
||||||
|
|
||||||
|
# ── Database ──────────────────────────────────────────────────
|
||||||
|
@property
|
||||||
|
def database_url(self) -> str:
|
||||||
|
return _env_str("DATABASE_URL")
|
||||||
|
|
||||||
|
@property
|
||||||
|
def use_postgres(self) -> bool:
|
||||||
|
return bool(self.database_url)
|
||||||
|
|
||||||
|
# ── Server ────────────────────────────────────────────────────
|
||||||
@property
|
@property
|
||||||
def host(self) -> str:
|
def host(self) -> str:
|
||||||
return self._cfg.get("server", {}).get("host", "127.0.0.1")
|
return _env_str("CTXD_HOST", self._cfg.get("server", {}).get("host", "0.0.0.0"))
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def port(self) -> int:
|
def port(self) -> int:
|
||||||
return self._cfg.get("server", {}).get("port", 9091)
|
return _env_int("CTXD_PORT", self._cfg.get("server", {}).get("port", 9091))
|
||||||
|
|
||||||
|
@property
|
||||||
|
def log_level(self) -> str:
|
||||||
|
return _env_str("LOG_LEVEL", "info")
|
||||||
|
|
||||||
|
@property
|
||||||
|
def demo_mode(self) -> bool:
|
||||||
|
return _env_bool("CTXD_DEMO_MODE", False)
|
||||||
|
|
||||||
|
# ── Snapshots ─────────────────────────────────────────────────
|
||||||
@property
|
@property
|
||||||
def min_snapshots(self) -> int:
|
def min_snapshots(self) -> int:
|
||||||
return self._cfg.get("snapshots", {}).get("min_keep", 5)
|
return _env_int("SNAPSHOT_MIN_KEEP", self._cfg.get("snapshots", {}).get("min_keep", 5))
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def max_snapshots(self) -> int:
|
def max_snapshots(self) -> int:
|
||||||
return self._cfg.get("snapshots", {}).get("max_keep", 25)
|
return _env_int("SNAPSHOT_MAX_KEEP", self._cfg.get("snapshots", {}).get("max_keep", 25))
|
||||||
|
|
||||||
# ── Auth ──────────────────────────────────────────────────────
|
# ── Auth ──────────────────────────────────────────────────────
|
||||||
@property
|
@property
|
||||||
def auth_enabled(self) -> bool:
|
def auth_enabled(self) -> bool:
|
||||||
return self._cfg.get("auth", {}).get("enabled", False)
|
return _env_bool("CTXD_AUTH_ENABLED", self._cfg.get("auth", {}).get("enabled", False))
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def api_key(self) -> str:
|
def api_key(self) -> str:
|
||||||
return self._cfg.get("auth", {}).get("api_key", "")
|
return _env_str("CTXD_API_KEY", self._cfg.get("auth", {}).get("api_key", ""))
|
||||||
|
|
||||||
|
@property
|
||||||
|
def external_readonly_key(self) -> str:
|
||||||
|
return _env_str("CTXD_EXTERNAL_READONLY_KEY", self._cfg.get("auth", {}).get("external_readonly_key", ""))
|
||||||
|
|
||||||
|
# ── OAuth ─────────────────────────────────────────────────────
|
||||||
|
@property
|
||||||
|
def oauth_enabled(self) -> bool:
|
||||||
|
return _env_bool("OAUTH_ENABLED", self._cfg.get("oauth", {}).get("enabled", False))
|
||||||
|
|
||||||
|
@property
|
||||||
|
def oauth_issuer(self) -> str:
|
||||||
|
return _env_str("OAUTH_ISSUER", self._cfg.get("oauth", {}).get("issuer", ""))
|
||||||
|
|
||||||
|
@property
|
||||||
|
def oauth_approval_key(self) -> str:
|
||||||
|
return _env_str("OAUTH_APPROVAL_KEY", self._cfg.get("oauth", {}).get("approval_key", ""))
|
||||||
|
|
||||||
|
@property
|
||||||
|
def oauth_approval_user_id(self) -> str:
|
||||||
|
return _env_str("OAUTH_APPROVAL_USER_ID", self._cfg.get("oauth", {}).get("approval_user_id", "admin"))
|
||||||
|
|
||||||
|
@property
|
||||||
|
def oauth_access_token_ttl_seconds(self) -> int:
|
||||||
|
return _env_int("OAUTH_ACCESS_TOKEN_TTL", self._cfg.get("oauth", {}).get("access_token_ttl_seconds", 3600))
|
||||||
|
|
||||||
|
@property
|
||||||
|
def oauth_refresh_token_ttl_seconds(self) -> int:
|
||||||
|
return _env_int("OAUTH_REFRESH_TOKEN_TTL", self._cfg.get("oauth", {}).get("refresh_token_ttl_seconds", 2592000))
|
||||||
|
|
||||||
|
# ── Web Sessions ───────────────────────────────────────────────
|
||||||
|
@property
|
||||||
|
def web_session_ttl_seconds(self) -> int:
|
||||||
|
return _env_int("WEB_SESSION_TTL", self._cfg.get("web_sessions", {}).get("ttl_seconds", 604800))
|
||||||
|
|
||||||
# ── Bootstrap ─────────────────────────────────────────────────
|
# ── Bootstrap ─────────────────────────────────────────────────
|
||||||
def ensure_dirs(self):
|
def ensure_dirs(self):
|
||||||
@@ -93,7 +194,9 @@ class CtxConfig:
|
|||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def from_home(cls, home: Path | str | None = None) -> "CtxConfig":
|
def from_home(cls, home: Path | str | None = None) -> "CtxConfig":
|
||||||
"""Load from ctxd.yaml if it exists, otherwise use defaults."""
|
"""Load from ctxd.yaml if it exists, otherwise use defaults.
|
||||||
|
Env vars always take precedence over yaml values at read time.
|
||||||
|
"""
|
||||||
home = Path(home).resolve() if home else DEFAULT_HOME
|
home = Path(home).resolve() if home else DEFAULT_HOME
|
||||||
cfg_path = home / "ctxd.yaml"
|
cfg_path = home / "ctxd.yaml"
|
||||||
if cfg_path.exists():
|
if cfg_path.exists():
|
||||||
@@ -104,7 +207,7 @@ class CtxConfig:
|
|||||||
return cls(home=str(home))
|
return cls(home=str(home))
|
||||||
|
|
||||||
def save(self):
|
def save(self):
|
||||||
"""Write config to ctxd.yaml."""
|
"""Write config to ctxd.yaml. Rarely needed in env-driven deployments."""
|
||||||
import yaml
|
import yaml
|
||||||
self.ensure_dirs()
|
self.ensure_dirs()
|
||||||
with open(self.config_path, "w") as f:
|
with open(self.config_path, "w") as f:
|
||||||
|
|||||||
+432
-110
@@ -1,7 +1,10 @@
|
|||||||
"""
|
"""
|
||||||
Database layer for ctxd — schema init, CRUD, workspace fork/merge, FTS, audit.
|
Database layer for ctxd — schema init, CRUD, workspace fork/merge, FTS, audit.
|
||||||
All public methods take a sqlite3.Connection as the first argument so callers
|
All public methods take a connection as the first argument so callers
|
||||||
control transactions. This module is stateless — all state is in SQLite.
|
control transactions. This module is stateless — all state is in the DB.
|
||||||
|
|
||||||
|
Supports both PostgreSQL (via DATABASE_URL) and SQLite (fallback for local dev).
|
||||||
|
The public API is identical regardless of backend.
|
||||||
"""
|
"""
|
||||||
import json
|
import json
|
||||||
import sqlite3
|
import sqlite3
|
||||||
@@ -11,15 +14,58 @@ from pathlib import Path
|
|||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from .config import CtxConfig
|
from .config import CtxConfig
|
||||||
|
from .auth_password import hash_password, verify_password
|
||||||
|
|
||||||
# ── Schema ────────────────────────────────────────────────────────────────────
|
# ── Schema paths ─────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
SCHEMA_PATH = Path(__file__).parent / "schema.sql"
|
SCHEMA_PG_PATH = Path(__file__).parent / "schema.sql"
|
||||||
|
SCHEMA_SQLITE_PATH = Path(__file__).parent / "schema_sqlite.sql"
|
||||||
|
|
||||||
|
|
||||||
def init_db(cfg: CtxConfig) -> sqlite3.Connection:
|
def init_db(cfg: CtxConfig):
|
||||||
"""Create ~/.ctx/ dirs + initialize the database from schema.sql."""
|
"""Create ~/.ctx/ dirs + initialize the database.
|
||||||
|
|
||||||
|
If DATABASE_URL is set in the environment, connects to PostgreSQL.
|
||||||
|
Otherwise falls back to SQLite for local development.
|
||||||
|
Returns a connection object (psycopg.Connection or sqlite3.Connection).
|
||||||
|
"""
|
||||||
cfg.ensure_dirs()
|
cfg.ensure_dirs()
|
||||||
|
|
||||||
|
if cfg.use_postgres:
|
||||||
|
return _init_pg(cfg)
|
||||||
|
else:
|
||||||
|
return _init_sqlite(cfg)
|
||||||
|
|
||||||
|
|
||||||
|
def _init_pg(cfg: CtxConfig):
|
||||||
|
"""Initialize PostgreSQL database from schema.sql."""
|
||||||
|
import psycopg
|
||||||
|
from psycopg.rows import dict_row
|
||||||
|
|
||||||
|
conn = psycopg.connect(cfg.database_url, row_factory=dict_row)
|
||||||
|
# Keep autocommit=False (default) — we use explicit commit() calls
|
||||||
|
|
||||||
|
# Check if schema is already initialized by looking for the users table
|
||||||
|
cur = conn.execute(
|
||||||
|
"SELECT EXISTS (SELECT FROM pg_tables WHERE schemaname = 'public' AND tablename = 'users')"
|
||||||
|
)
|
||||||
|
schema_exists = cur.fetchone()["exists"]
|
||||||
|
|
||||||
|
if not schema_exists:
|
||||||
|
with open(SCHEMA_PG_PATH) as f:
|
||||||
|
schema_sql = f.read()
|
||||||
|
# PostgreSQL can execute multiple statements in a single execute() call
|
||||||
|
conn.execute(schema_sql)
|
||||||
|
conn.commit()
|
||||||
|
else:
|
||||||
|
# Run migrations if needed
|
||||||
|
_migrate_pg(conn)
|
||||||
|
|
||||||
|
return conn
|
||||||
|
|
||||||
|
|
||||||
|
def _init_sqlite(cfg: CtxConfig) -> sqlite3.Connection:
|
||||||
|
"""Initialize SQLite database (fallback for local dev)."""
|
||||||
fresh = not cfg.db_path.exists()
|
fresh = not cfg.db_path.exists()
|
||||||
conn = sqlite3.connect(str(cfg.db_path))
|
conn = sqlite3.connect(str(cfg.db_path))
|
||||||
conn.row_factory = sqlite3.Row
|
conn.row_factory = sqlite3.Row
|
||||||
@@ -27,70 +73,248 @@ def init_db(cfg: CtxConfig) -> sqlite3.Connection:
|
|||||||
conn.execute("PRAGMA foreign_keys = ON")
|
conn.execute("PRAGMA foreign_keys = ON")
|
||||||
|
|
||||||
if fresh:
|
if fresh:
|
||||||
with open(SCHEMA_PATH) as f:
|
with open(SCHEMA_SQLITE_PATH) as f:
|
||||||
conn.executescript(f.read())
|
conn.executescript(f.read())
|
||||||
else:
|
else:
|
||||||
|
# NOTE: SQLite cannot ALTER a FK constraint in place. Existing dev
|
||||||
|
# databases will NOT pick up FK changes in schema_sqlite.sql (e.g.
|
||||||
|
# ON DELETE SET NULL on user_id columns) — delete $CTXD_HOME/ctxd.db
|
||||||
|
# to recreate from the current schema. Only additive ADD COLUMN
|
||||||
|
# migrations are applied here.
|
||||||
# Migration: add metadata_tags column if it doesn't exist
|
# Migration: add metadata_tags column if it doesn't exist
|
||||||
try:
|
try:
|
||||||
conn.execute("ALTER TABLE projects ADD COLUMN metadata_tags TEXT DEFAULT '[]'")
|
conn.execute("ALTER TABLE projects ADD COLUMN metadata_tags TEXT DEFAULT '[]'")
|
||||||
conn.commit()
|
conn.commit()
|
||||||
except sqlite3.OperationalError:
|
except sqlite3.OperationalError:
|
||||||
pass # column already exists
|
pass
|
||||||
|
try:
|
||||||
|
conn.execute("ALTER TABLE users ADD COLUMN active INTEGER NOT NULL DEFAULT 1")
|
||||||
|
conn.commit()
|
||||||
|
except sqlite3.OperationalError:
|
||||||
|
pass
|
||||||
|
|
||||||
return conn
|
return conn
|
||||||
|
|
||||||
|
|
||||||
|
def _migrate_pg(conn):
|
||||||
|
"""Run PostgreSQL migrations for existing databases."""
|
||||||
|
try:
|
||||||
|
conn.execute("ALTER TABLE projects ADD COLUMN IF NOT EXISTS metadata_tags TEXT DEFAULT '[]'")
|
||||||
|
conn.commit()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
try:
|
||||||
|
conn.execute("ALTER TABLE users ADD COLUMN IF NOT EXISTS active BOOLEAN NOT NULL DEFAULT TRUE")
|
||||||
|
conn.commit()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
def now() -> str:
|
def now() -> str:
|
||||||
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||||
|
|
||||||
|
|
||||||
|
def _is_pg(conn) -> bool:
|
||||||
|
"""Check if connection is a PostgreSQL connection."""
|
||||||
|
return not isinstance(conn, sqlite3.Connection)
|
||||||
|
|
||||||
|
|
||||||
|
def is_integrity_error(exc: BaseException) -> bool:
|
||||||
|
"""True if exc is a DB constraint violation (unique/primary-key, FK, check)
|
||||||
|
on either backend. Used to map duplicate-key errors to HTTP 409.
|
||||||
|
|
||||||
|
SQLite raises sqlite3.IntegrityError; psycopg raises subclasses of
|
||||||
|
psycopg.errors.IntegrityError (UniqueViolation, ForeignKeyViolation, ...).
|
||||||
|
"""
|
||||||
|
if isinstance(exc, sqlite3.IntegrityError):
|
||||||
|
return True
|
||||||
|
try:
|
||||||
|
import psycopg
|
||||||
|
except ImportError:
|
||||||
|
return False
|
||||||
|
return isinstance(exc, psycopg.errors.IntegrityError)
|
||||||
|
|
||||||
|
|
||||||
# ── Helpers ───────────────────────────────────────────────────────────────────
|
# ── Helpers ───────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
def _row_to_dict(row: sqlite3.Row | None) -> dict | None:
|
def _row_to_dict(row) -> dict | None:
|
||||||
if row is None:
|
if row is None:
|
||||||
return None
|
return None
|
||||||
|
if isinstance(row, dict):
|
||||||
|
return dict(row)
|
||||||
return dict(row)
|
return dict(row)
|
||||||
|
|
||||||
|
|
||||||
|
def _ph(conn, n: int = 1) -> str:
|
||||||
|
"""Return placeholder string for the current backend.
|
||||||
|
|
||||||
|
PostgreSQL uses %s, SQLite uses ?.
|
||||||
|
"""
|
||||||
|
if _is_pg(conn):
|
||||||
|
return ", ".join(["%s"] * n)
|
||||||
|
else:
|
||||||
|
return ", ".join(["?"] * n)
|
||||||
|
|
||||||
|
|
||||||
# ── Users ─────────────────────────────────────────────────────────────────────
|
# ── Users ─────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
def user_create(conn, user_id: str, display_name: str, role: str = "contributor"):
|
|
||||||
|
def user_create(conn, user_id: str, display_name: str, role: str = "contributor", password: str | None = None, active: bool = True):
|
||||||
|
token_hash = hash_password(password) if password else None
|
||||||
|
ph = _ph(conn, 5)
|
||||||
|
if _is_pg(conn):
|
||||||
|
active_val = active
|
||||||
|
else:
|
||||||
|
active_val = 1 if active else 0
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO users (user_id, display_name, role) VALUES (?, ?, ?)",
|
f"INSERT INTO users (user_id, display_name, role, token_hash, active) VALUES ({ph})",
|
||||||
(user_id, display_name, role),
|
(user_id, display_name, role, token_hash, active_val),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def user_update(conn, user_id: str, *, display_name: str | None = None, role: str | None = None, active: bool | None = None) -> bool:
|
||||||
|
user = user_get(conn, user_id)
|
||||||
|
if not user:
|
||||||
|
return False
|
||||||
|
fields = []
|
||||||
|
values: list[Any] = []
|
||||||
|
ph = _ph(conn, 1)
|
||||||
|
if display_name is not None:
|
||||||
|
fields.append(f"display_name = {ph}")
|
||||||
|
values.append(display_name)
|
||||||
|
if role is not None:
|
||||||
|
fields.append(f"role = {ph}")
|
||||||
|
values.append(role)
|
||||||
|
if active is not None:
|
||||||
|
if _is_pg(conn):
|
||||||
|
fields.append(f"active = {ph}")
|
||||||
|
values.append(active)
|
||||||
|
else:
|
||||||
|
fields.append(f"active = {ph}")
|
||||||
|
values.append(1 if active else 0)
|
||||||
|
if not fields:
|
||||||
|
return True
|
||||||
|
if _is_pg(conn):
|
||||||
|
fields.append("updated_at = to_char(now() at time zone 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS\"Z\"')")
|
||||||
|
else:
|
||||||
|
fields.append("updated_at = strftime('%Y-%m-%dT%H:%M:%SZ', 'now')")
|
||||||
|
values.append(user_id)
|
||||||
|
conn.execute(f"UPDATE users SET {', '.join(fields)} WHERE user_id = {ph}", values)
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def user_delete(conn, user_id: str) -> dict:
|
||||||
|
if user_get(conn, user_id) is None:
|
||||||
|
return {"ok": False, "error": "not_found"}
|
||||||
|
ph = _ph(conn, 1)
|
||||||
|
try:
|
||||||
|
conn.execute(f"DELETE FROM users WHERE user_id = {ph}", (user_id,))
|
||||||
|
return {"ok": True}
|
||||||
|
except Exception as e:
|
||||||
|
if _is_pg(conn):
|
||||||
|
import psycopg
|
||||||
|
# Roll back unconditionally so a failed DELETE never leaves the
|
||||||
|
# shared connection in an aborted-transaction state (see the 500
|
||||||
|
# cascade fix); only an FK violation maps to the soft response.
|
||||||
|
conn.rollback()
|
||||||
|
if isinstance(e, psycopg.errors.ForeignKeyViolation):
|
||||||
|
return {"ok": False, "error": "user_has_references", "hint": "Inactivate the user instead of deleting."}
|
||||||
|
raise
|
||||||
|
# SQLite: only an FK violation is the expected "still referenced" case.
|
||||||
|
# Anything else (I/O error, programming bug, corruption) must propagate
|
||||||
|
# to the global handler as a 500 rather than be masked as a soft 409.
|
||||||
|
if isinstance(e, sqlite3.IntegrityError):
|
||||||
|
return {"ok": False, "error": "user_has_references", "hint": "Inactivate the user instead of deleting."}
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
def user_set_password(conn, user_id: str, password: str):
|
||||||
|
ph = _ph(conn, 2)
|
||||||
|
if _is_pg(conn):
|
||||||
|
ts_expr = "to_char(now() at time zone 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS\"Z\"')"
|
||||||
|
else:
|
||||||
|
ts_expr = "strftime('%Y-%m-%dT%H:%M:%SZ', 'now')"
|
||||||
|
conn.execute(
|
||||||
|
f"UPDATE users SET token_hash = {ph}, updated_at = {ts_expr} WHERE user_id = {ph}",
|
||||||
|
(hash_password(password), user_id),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def user_get(conn, user_id: str) -> dict | None:
|
def user_get(conn, user_id: str) -> dict | None:
|
||||||
|
ph = _ph(conn, 1)
|
||||||
return _row_to_dict(conn.execute(
|
return _row_to_dict(conn.execute(
|
||||||
"SELECT * FROM users WHERE user_id = ?", (user_id,)
|
f"SELECT * FROM users WHERE user_id = {ph}", (user_id,)
|
||||||
).fetchone())
|
).fetchone())
|
||||||
|
|
||||||
|
|
||||||
|
def user_get_ci(conn, user_id: str) -> dict | None:
|
||||||
|
"""Case-insensitive user lookup (matches idx_users_lower)."""
|
||||||
|
ph = _ph(conn, 1)
|
||||||
|
return _row_to_dict(conn.execute(
|
||||||
|
f"SELECT * FROM users WHERE LOWER(user_id) = LOWER({ph})", (user_id.strip(),)
|
||||||
|
).fetchone())
|
||||||
|
|
||||||
|
|
||||||
|
def user_authenticate(conn, user_id: str, password: str) -> dict | None:
|
||||||
|
user = user_get_ci(conn, user_id)
|
||||||
|
if not user:
|
||||||
|
return None
|
||||||
|
# Handle both PostgreSQL BOOLEAN and SQLite INTEGER for active
|
||||||
|
active = user.get("active")
|
||||||
|
if active is None:
|
||||||
|
active = True # Default to active
|
||||||
|
if isinstance(active, int) and active == 0:
|
||||||
|
return None
|
||||||
|
if isinstance(active, bool) and not active:
|
||||||
|
return None
|
||||||
|
if not user.get("token_hash"):
|
||||||
|
return None
|
||||||
|
if not verify_password(password, user.get("token_hash")):
|
||||||
|
return None
|
||||||
|
return user
|
||||||
|
|
||||||
|
|
||||||
def user_list(conn) -> list[dict]:
|
def user_list(conn) -> list[dict]:
|
||||||
return [dict(r) for r in conn.execute("SELECT * FROM users ORDER BY user_id").fetchall()]
|
if _is_pg(conn):
|
||||||
|
return [dict(r) for r in conn.execute("SELECT * FROM users ORDER BY user_id").fetchall()]
|
||||||
|
else:
|
||||||
|
return [dict(r) for r in conn.execute("SELECT * FROM users ORDER BY user_id").fetchall()]
|
||||||
|
|
||||||
|
|
||||||
# ── Projects ──────────────────────────────────────────────────────────────────
|
# ── Projects ──────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
def project_create(conn, project_id: str, display_name: str, description: str = ""):
|
def project_create(conn, project_id: str, display_name: str, description: str = ""):
|
||||||
|
ph3 = _ph(conn, 3)
|
||||||
|
ph1 = _ph(conn, 1)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO projects (project_id, display_name, description) VALUES (?, ?, ?)",
|
f"INSERT INTO projects (project_id, display_name, description) VALUES ({ph3})",
|
||||||
(project_id, display_name, description),
|
(project_id, display_name, description),
|
||||||
)
|
)
|
||||||
# Also create empty shared context
|
# Also create empty shared context
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO project_context (project_id, content, version) VALUES (?, '', 0)",
|
f"INSERT INTO project_context (project_id, content, version) VALUES ({ph1}, '', 0)",
|
||||||
(project_id,),
|
(project_id,),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def project_get(conn, project_id: str) -> dict | None:
|
def project_get(conn, project_id: str) -> dict | None:
|
||||||
|
ph = _ph(conn, 1)
|
||||||
return _row_to_dict(conn.execute(
|
return _row_to_dict(conn.execute(
|
||||||
"SELECT * FROM projects WHERE project_id = ?", (project_id,)
|
f"SELECT * FROM projects WHERE project_id = {ph}", (project_id,)
|
||||||
).fetchone())
|
).fetchone())
|
||||||
|
|
||||||
|
|
||||||
|
def project_delete(conn, project_id: str) -> dict:
|
||||||
|
"""Delete a project and all its dependent rows (cascades).
|
||||||
|
audit_log.project_id is SET NULL via FK ON DELETE SET NULL.
|
||||||
|
Returns {'ok': True} or {'ok': False, 'error': ...}."""
|
||||||
|
if project_get(conn, project_id) is None:
|
||||||
|
return {"ok": False, "error": "not_found"}
|
||||||
|
ph = _ph(conn, 1)
|
||||||
|
conn.execute(f"DELETE FROM projects WHERE project_id = {ph}", (project_id,))
|
||||||
|
return {"ok": True}
|
||||||
|
|
||||||
|
|
||||||
def project_list(conn) -> list[dict]:
|
def project_list(conn) -> list[dict]:
|
||||||
return [dict(r) for r in conn.execute(
|
return [dict(r) for r in conn.execute(
|
||||||
"SELECT project_id, display_name, description, shared_version FROM projects ORDER BY project_id"
|
"SELECT project_id, display_name, description, shared_version FROM projects ORDER BY project_id"
|
||||||
@@ -98,23 +322,30 @@ def project_list(conn) -> list[dict]:
|
|||||||
|
|
||||||
|
|
||||||
def project_set_sync_path(conn, project_id: str, sync_path: str | None):
|
def project_set_sync_path(conn, project_id: str, sync_path: str | None):
|
||||||
conn.execute(
|
ph = _ph(conn, 2)
|
||||||
"UPDATE projects SET sync_path = ?, auto_sync = 1 WHERE project_id = ?",
|
if _is_pg(conn):
|
||||||
(sync_path, project_id),
|
conn.execute(
|
||||||
)
|
f"UPDATE projects SET sync_path = {ph.split(', ')[0]}, auto_sync = TRUE WHERE project_id = {ph.split(', ')[1]}",
|
||||||
|
(sync_path, project_id),
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
conn.execute(
|
||||||
|
f"UPDATE projects SET sync_path = ?, auto_sync = 1 WHERE project_id = ?",
|
||||||
|
(sync_path, project_id),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def project_get_tags(conn, project_id: str) -> list[str]:
|
def project_get_tags(conn, project_id: str) -> list[str]:
|
||||||
"""Get project metadata tags as a list of strings."""
|
"""Get project metadata tags as a list of strings."""
|
||||||
|
ph = _ph(conn, 1)
|
||||||
row = conn.execute(
|
row = conn.execute(
|
||||||
"SELECT metadata_tags FROM projects WHERE project_id = ?", (project_id,)
|
f"SELECT metadata_tags FROM projects WHERE project_id = {ph}", (project_id,)
|
||||||
).fetchone()
|
).fetchone()
|
||||||
if row is None:
|
if row is None:
|
||||||
return []
|
return []
|
||||||
tags = row["metadata_tags"]
|
tags = row["metadata_tags"]
|
||||||
if not tags:
|
if not tags:
|
||||||
return []
|
return []
|
||||||
import json
|
|
||||||
try:
|
try:
|
||||||
return json.loads(tags)
|
return json.loads(tags)
|
||||||
except (json.JSONDecodeError, TypeError):
|
except (json.JSONDecodeError, TypeError):
|
||||||
@@ -122,10 +353,9 @@ def project_get_tags(conn, project_id: str) -> list[str]:
|
|||||||
|
|
||||||
|
|
||||||
def project_set_tags(conn, project_id: str, tags: list[str]):
|
def project_set_tags(conn, project_id: str, tags: list[str]):
|
||||||
"""Set project metadata tags from a list of strings."""
|
ph = _ph(conn, 2)
|
||||||
import json
|
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"UPDATE projects SET metadata_tags = ? WHERE project_id = ?",
|
f"UPDATE projects SET metadata_tags = {ph.split(', ')[0]} WHERE project_id = {ph.split(', ')[1]}",
|
||||||
(json.dumps(tags), project_id),
|
(json.dumps(tags), project_id),
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -139,7 +369,6 @@ def build_metadata_header(project_id: str, display_name: str | None = None,
|
|||||||
TYPE: PROJECT CONTEXT, PROJECT, STATUS: ACTIVE, LAST-UPDATED, TAGS.
|
TYPE: PROJECT CONTEXT, PROJECT, STATUS: ACTIVE, LAST-UPDATED, TAGS.
|
||||||
LAST-UPDATED uses the actual updated_at timestamp, falling back to today.
|
LAST-UPDATED uses the actual updated_at timestamp, falling back to today.
|
||||||
TAGS uses the project's metadata_tags if provided, falling back to project name + CONTEXT."""
|
TAGS uses the project's metadata_tags if provided, falling back to project name + CONTEXT."""
|
||||||
from datetime import datetime, timezone
|
|
||||||
project_upper = (display_name or project_id).upper()
|
project_upper = (display_name or project_id).upper()
|
||||||
last_updated = (updated_at or datetime.now(timezone.utc).strftime("%Y-%m-%d"))
|
last_updated = (updated_at or datetime.now(timezone.utc).strftime("%Y-%m-%d"))
|
||||||
if "T" in last_updated:
|
if "T" in last_updated:
|
||||||
@@ -186,10 +415,11 @@ def context_read(conn, project_id: str) -> dict | None:
|
|||||||
Returns with metadata header prepended dynamically.
|
Returns with metadata header prepended dynamically.
|
||||||
If content already has a header (including YAML frontmatter from vault imports),
|
If content already has a header (including YAML frontmatter from vault imports),
|
||||||
it is replaced with the current dynamic header."""
|
it is replaced with the current dynamic header."""
|
||||||
|
ph = _ph(conn, 1)
|
||||||
row = conn.execute(
|
row = conn.execute(
|
||||||
"SELECT pc.*, p.shared_version, p.display_name FROM project_context pc "
|
f"SELECT pc.*, p.shared_version, p.display_name FROM project_context pc "
|
||||||
"JOIN projects p ON p.project_id = pc.project_id "
|
f"JOIN projects p ON p.project_id = pc.project_id "
|
||||||
"WHERE pc.project_id = ?", (project_id,)
|
f"WHERE pc.project_id = {ph}", (project_id,)
|
||||||
).fetchone()
|
).fetchone()
|
||||||
if row is None:
|
if row is None:
|
||||||
return None
|
return None
|
||||||
@@ -238,8 +468,9 @@ def context_update(conn, project_id: str, new_content: str, updated_by: str,
|
|||||||
Returns {'ok': True, 'new_version': N} or {'ok': False, 'error': 'conflict',
|
Returns {'ok': True, 'new_version': N} or {'ok': False, 'error': 'conflict',
|
||||||
'current_version': N}.
|
'current_version': N}.
|
||||||
"""
|
"""
|
||||||
|
ph = _ph(conn, 1)
|
||||||
cur = conn.execute(
|
cur = conn.execute(
|
||||||
"SELECT shared_version FROM projects WHERE project_id = ?",
|
f"SELECT shared_version FROM projects WHERE project_id = {ph}",
|
||||||
(project_id,)
|
(project_id,)
|
||||||
)
|
)
|
||||||
row = cur.fetchone()
|
row = cur.fetchone()
|
||||||
@@ -262,15 +493,21 @@ def context_update(conn, project_id: str, new_content: str, updated_by: str,
|
|||||||
_snapshot_take(conn, project_id, version_from=current_version, version_to=new_version)
|
_snapshot_take(conn, project_id, version_from=current_version, version_to=new_version)
|
||||||
|
|
||||||
# Update project_context
|
# Update project_context
|
||||||
|
ph5 = _ph(conn, 5)
|
||||||
|
if _is_pg(conn):
|
||||||
|
ts_expr = "to_char(now() at time zone 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS\"Z\"')"
|
||||||
|
else:
|
||||||
|
ts_expr = "strftime('%Y-%m-%dT%H:%M:%SZ', 'now')"
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"UPDATE project_context SET content = ?, version = ?, updated_by = ?, updated_at = ? "
|
f"UPDATE project_context SET content = {_ph(conn,1)}, version = {_ph(conn,1)}, "
|
||||||
"WHERE project_id = ?",
|
f"updated_by = {_ph(conn,1)}, updated_at = {ts_expr} "
|
||||||
(clean_content, new_version, updated_by, now(), project_id),
|
f"WHERE project_id = {_ph(conn,1)}",
|
||||||
|
(clean_content, new_version, updated_by, project_id)
|
||||||
)
|
)
|
||||||
# Bump shared version
|
# Bump shared version
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"UPDATE projects SET shared_version = ? WHERE project_id = ?",
|
f"UPDATE projects SET shared_version = {_ph(conn,1)} WHERE project_id = {_ph(conn,1)}",
|
||||||
(new_version, project_id),
|
(new_version, project_id)
|
||||||
)
|
)
|
||||||
|
|
||||||
return {"ok": True, "new_version": new_version, "content": clean_content}
|
return {"ok": True, "new_version": new_version, "content": clean_content}
|
||||||
@@ -279,29 +516,38 @@ def context_update(conn, project_id: str, new_content: str, updated_by: str,
|
|||||||
# ── User Profile ──────────────────────────────────────────────────────────────
|
# ── User Profile ──────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
def profile_read(conn, user_id: str) -> dict | None:
|
def profile_read(conn, user_id: str) -> dict | None:
|
||||||
|
ph = _ph(conn, 1)
|
||||||
return _row_to_dict(conn.execute(
|
return _row_to_dict(conn.execute(
|
||||||
"SELECT * FROM user_profiles WHERE user_id = ?", (user_id,)
|
f"SELECT * FROM user_profiles WHERE user_id = {ph}", (user_id,)
|
||||||
).fetchone())
|
).fetchone())
|
||||||
|
|
||||||
|
|
||||||
def profile_update(conn, user_id: str, content: str, base_version: int) -> dict:
|
def profile_update(conn, user_id: str, content: str, base_version: int) -> dict:
|
||||||
|
ph = _ph(conn, 1)
|
||||||
cur = conn.execute(
|
cur = conn.execute(
|
||||||
"SELECT version FROM user_profiles WHERE user_id = ?", (user_id,)
|
f"SELECT version FROM user_profiles WHERE user_id = {ph}", (user_id,)
|
||||||
)
|
)
|
||||||
row = cur.fetchone()
|
row = cur.fetchone()
|
||||||
if row is None:
|
if row is None:
|
||||||
# Create
|
# Create
|
||||||
|
ph2 = _ph(conn, 2)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO user_profiles (user_id, content, version) VALUES (?, ?, 1)",
|
f"INSERT INTO user_profiles (user_id, content, version) VALUES ({ph2}, 1)",
|
||||||
(user_id, content),
|
(user_id, content)
|
||||||
)
|
)
|
||||||
return {"ok": True, "new_version": 1}
|
return {"ok": True, "new_version": 1}
|
||||||
current_version = row["version"]
|
current_version = row["version"]
|
||||||
if base_version != current_version:
|
if base_version != current_version:
|
||||||
return {"ok": False, "error": "conflict", "current_version": current_version}
|
return {"ok": False, "error": "conflict", "current_version": current_version}
|
||||||
|
if _is_pg(conn):
|
||||||
|
ts_expr = "to_char(now() at time zone 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS\"Z\"')"
|
||||||
|
else:
|
||||||
|
ts_expr = "strftime('%Y-%m-%dT%H:%M:%SZ', 'now')"
|
||||||
|
ph4 = _ph(conn, 4)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"UPDATE user_profiles SET content = ?, version = ?, updated_at = ? WHERE user_id = ?",
|
f"UPDATE user_profiles SET content = {_ph(conn,1)}, version = {_ph(conn,1)}, "
|
||||||
(content, current_version + 1, now(), user_id),
|
f"updated_at = {ts_expr} WHERE user_id = {_ph(conn,1)}",
|
||||||
|
(content, current_version + 1, user_id)
|
||||||
)
|
)
|
||||||
return {"ok": True, "new_version": current_version + 1}
|
return {"ok": True, "new_version": current_version + 1}
|
||||||
|
|
||||||
@@ -325,42 +571,51 @@ def workspace_fork(conn, user_id: str, project_id: str) -> dict:
|
|||||||
shared_content = ctx["content"] if ctx else ""
|
shared_content = ctx["content"] if ctx else ""
|
||||||
|
|
||||||
ws_id = str(uuid.uuid4())
|
ws_id = str(uuid.uuid4())
|
||||||
|
ph4 = _ph(conn, 4)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO user_workspaces (workspace_id, user_id, project_id, base_version) "
|
f"INSERT INTO user_workspaces (workspace_id, user_id, project_id, base_version) "
|
||||||
"VALUES (?, ?, ?, ?)",
|
f"VALUES ({ph4})",
|
||||||
(ws_id, user_id, project_id, base_version),
|
(ws_id, user_id, project_id, base_version),
|
||||||
)
|
)
|
||||||
# Seed workspace with current shared content
|
# Seed workspace with current shared content
|
||||||
|
ph2 = _ph(conn, 2)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO workspace_files (workspace_id, file_path, content) VALUES (?, 'context.md', ?)",
|
f"INSERT INTO workspace_files (workspace_id, file_path, content) VALUES ({ph2.split(', ')[0]}, 'context.md', {ph2.split(', ')[1]})",
|
||||||
(ws_id, shared_content),
|
(ws_id, shared_content),
|
||||||
)
|
)
|
||||||
return {"ok": True, "workspace_id": ws_id, "base_version": base_version}
|
return {"ok": True, "workspace_id": ws_id, "base_version": base_version}
|
||||||
|
|
||||||
|
|
||||||
def workspace_get(conn, workspace_id: str) -> dict | None:
|
def workspace_get(conn, workspace_id: str) -> dict | None:
|
||||||
|
ph = _ph(conn, 1)
|
||||||
return _row_to_dict(conn.execute(
|
return _row_to_dict(conn.execute(
|
||||||
"SELECT * FROM user_workspaces WHERE workspace_id = ?", (workspace_id,)
|
f"SELECT * FROM user_workspaces WHERE workspace_id = {ph}", (workspace_id,)
|
||||||
).fetchone())
|
).fetchone())
|
||||||
|
|
||||||
|
|
||||||
def workspace_list_for_user(conn, user_id: str, project_id: str | None = None) -> list[dict]:
|
def workspace_list_for_user(conn, user_id: str, project_id: str | None = None) -> list[dict]:
|
||||||
if project_id:
|
if project_id:
|
||||||
|
ph2 = _ph(conn, 2)
|
||||||
|
placeholders = ph2.split(", ")
|
||||||
rows = conn.execute(
|
rows = conn.execute(
|
||||||
"SELECT * FROM user_workspaces WHERE user_id = ? AND project_id = ? ORDER BY created_at DESC",
|
f"SELECT * FROM user_workspaces WHERE user_id = {placeholders[0]} AND project_id = {placeholders[1]} "
|
||||||
|
f"ORDER BY created_at DESC",
|
||||||
(user_id, project_id),
|
(user_id, project_id),
|
||||||
).fetchall()
|
).fetchall()
|
||||||
else:
|
else:
|
||||||
|
ph = _ph(conn, 1)
|
||||||
rows = conn.execute(
|
rows = conn.execute(
|
||||||
"SELECT * FROM user_workspaces WHERE user_id = ? ORDER BY created_at DESC",
|
f"SELECT * FROM user_workspaces WHERE user_id = {ph} ORDER BY created_at DESC",
|
||||||
(user_id,),
|
(user_id,),
|
||||||
).fetchall()
|
).fetchall()
|
||||||
return [dict(r) for r in rows]
|
return [dict(r) for r in rows]
|
||||||
|
|
||||||
|
|
||||||
def workspace_read_file(conn, workspace_id: str, file_path: str = "context.md") -> str | None:
|
def workspace_read_file(conn, workspace_id: str, file_path: str = "context.md") -> str | None:
|
||||||
|
ph2 = _ph(conn, 2)
|
||||||
|
placeholders = ph2.split(", ")
|
||||||
row = conn.execute(
|
row = conn.execute(
|
||||||
"SELECT content FROM workspace_files WHERE workspace_id = ? AND file_path = ?",
|
f"SELECT content FROM workspace_files WHERE workspace_id = {placeholders[0]} AND file_path = {placeholders[1]}",
|
||||||
(workspace_id, file_path),
|
(workspace_id, file_path),
|
||||||
).fetchone()
|
).fetchone()
|
||||||
return row["content"] if row else None
|
return row["content"] if row else None
|
||||||
@@ -368,19 +623,27 @@ def workspace_read_file(conn, workspace_id: str, file_path: str = "context.md")
|
|||||||
|
|
||||||
def workspace_write_file(conn, workspace_id: str, content: str,
|
def workspace_write_file(conn, workspace_id: str, content: str,
|
||||||
file_path: str = "context.md"):
|
file_path: str = "context.md"):
|
||||||
|
ph2 = _ph(conn, 2)
|
||||||
|
placeholders = ph2.split(", ")
|
||||||
existing = conn.execute(
|
existing = conn.execute(
|
||||||
"SELECT 1 FROM workspace_files WHERE workspace_id = ? AND file_path = ?",
|
f"SELECT 1 FROM workspace_files WHERE workspace_id = {placeholders[0]} AND file_path = {placeholders[1]}",
|
||||||
(workspace_id, file_path),
|
(workspace_id, file_path),
|
||||||
).fetchone()
|
).fetchone()
|
||||||
if existing:
|
if existing:
|
||||||
|
if _is_pg(conn):
|
||||||
|
ts_expr = "to_char(now() at time zone 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS\"Z\"')"
|
||||||
|
else:
|
||||||
|
ts_expr = "strftime('%Y-%m-%dT%H:%M:%SZ', 'now')"
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"UPDATE workspace_files SET content = ?, version = version + 1, updated_at = ? "
|
f"UPDATE workspace_files SET content = {_ph(conn,1)}, version = version + 1, "
|
||||||
"WHERE workspace_id = ? AND file_path = ?",
|
f"updated_at = {ts_expr} "
|
||||||
(content, now(), workspace_id, file_path),
|
f"WHERE workspace_id = {_ph(conn,1)} AND file_path = {_ph(conn,1)}",
|
||||||
|
(content, workspace_id, file_path),
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
|
ph3 = _ph(conn, 3)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO workspace_files (workspace_id, file_path, content) VALUES (?, ?, ?)",
|
f"INSERT INTO workspace_files (workspace_id, file_path, content) VALUES ({ph3})",
|
||||||
(workspace_id, file_path, content),
|
(workspace_id, file_path, content),
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -424,31 +687,35 @@ def workspace_submit(conn, workspace_id: str, submitted_by: str,
|
|||||||
)
|
)
|
||||||
if not result["ok"]:
|
if not result["ok"]:
|
||||||
return result
|
return result
|
||||||
|
ph = _ph(conn, 1)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"UPDATE user_workspaces SET status = 'merged' WHERE workspace_id = ?",
|
f"UPDATE user_workspaces SET status = 'merged' WHERE workspace_id = {ph}",
|
||||||
(workspace_id,),
|
(workspace_id,),
|
||||||
)
|
)
|
||||||
return {"ok": True, "action": "merged", **result}
|
return {"ok": True, "action": "merged", **result}
|
||||||
else:
|
else:
|
||||||
# Create pending change request
|
# Create pending change request
|
||||||
req_id = str(uuid.uuid4())
|
req_id = str(uuid.uuid4())
|
||||||
|
ph7 = _ph(conn, 7)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO change_requests (request_id, workspace_id, project_id, "
|
f"INSERT INTO change_requests (request_id, workspace_id, project_id, "
|
||||||
"submitted_by, target_version, base_version, diff_summary) "
|
f"submitted_by, target_version, base_version, diff_summary) "
|
||||||
"VALUES (?, ?, ?, ?, ?, ?, ?)",
|
f"VALUES ({ph7})",
|
||||||
(req_id, workspace_id, ws["project_id"], submitted_by,
|
(req_id, workspace_id, ws["project_id"], submitted_by,
|
||||||
target_version, base_version, diff_summary),
|
target_version, base_version, diff_summary),
|
||||||
)
|
)
|
||||||
|
ph = _ph(conn, 1)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"UPDATE user_workspaces SET status = 'submitted' WHERE workspace_id = ?",
|
f"UPDATE user_workspaces SET status = 'submitted' WHERE workspace_id = {ph}",
|
||||||
(workspace_id,),
|
(workspace_id,),
|
||||||
)
|
)
|
||||||
return {"ok": True, "action": "submitted", "request_id": req_id}
|
return {"ok": True, "action": "submitted", "request_id": req_id}
|
||||||
|
|
||||||
|
|
||||||
def workspace_abandon(conn, workspace_id: str):
|
def workspace_abandon(conn, workspace_id: str):
|
||||||
|
ph = _ph(conn, 1)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"UPDATE user_workspaces SET status = 'abandoned' WHERE workspace_id = ?",
|
f"UPDATE user_workspaces SET status = 'abandoned' WHERE workspace_id = {ph}",
|
||||||
(workspace_id,),
|
(workspace_id,),
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -456,10 +723,11 @@ def workspace_abandon(conn, workspace_id: str):
|
|||||||
def change_request_approve(conn, request_id: str, reviewer_id: str,
|
def change_request_approve(conn, request_id: str, reviewer_id: str,
|
||||||
comments: str = "") -> dict:
|
comments: str = "") -> dict:
|
||||||
"""Approve a change request and merge it into shared context."""
|
"""Approve a change request and merge it into shared context."""
|
||||||
|
ph = _ph(conn, 1)
|
||||||
row = conn.execute(
|
row = conn.execute(
|
||||||
"SELECT cr.*, ws.project_id FROM change_requests cr "
|
f"SELECT cr.*, ws.project_id FROM change_requests cr "
|
||||||
"JOIN user_workspaces ws ON ws.workspace_id = cr.workspace_id "
|
f"JOIN user_workspaces ws ON ws.workspace_id = cr.workspace_id "
|
||||||
"WHERE cr.request_id = ?", (request_id,)
|
f"WHERE cr.request_id = {ph}", (request_id,)
|
||||||
).fetchone()
|
).fetchone()
|
||||||
if row is None:
|
if row is None:
|
||||||
return {"ok": False, "error": "not_found"}
|
return {"ok": False, "error": "not_found"}
|
||||||
@@ -467,8 +735,10 @@ def change_request_approve(conn, request_id: str, reviewer_id: str,
|
|||||||
return {"ok": False, "error": f"status is {row['status']}"}
|
return {"ok": False, "error": f"status is {row['status']}"}
|
||||||
|
|
||||||
# Record review
|
# Record review
|
||||||
|
ph3 = _ph(conn, 3)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO reviews (request_id, reviewer_id, decision, comments) VALUES (?, ?, 'approved', ?)",
|
f"INSERT INTO reviews (request_id, reviewer_id, decision, comments) "
|
||||||
|
f"VALUES ({ph3.split(', ')[0]}, {ph3.split(', ')[1]}, 'approved', {ph3.split(', ')[2]})",
|
||||||
(request_id, reviewer_id, comments),
|
(request_id, reviewer_id, comments),
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -483,12 +753,13 @@ def change_request_approve(conn, request_id: str, reviewer_id: str,
|
|||||||
if not result["ok"]:
|
if not result["ok"]:
|
||||||
return result
|
return result
|
||||||
|
|
||||||
|
ph = _ph(conn, 1)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"UPDATE change_requests SET status = 'merged' WHERE request_id = ?",
|
f"UPDATE change_requests SET status = 'merged' WHERE request_id = {ph}",
|
||||||
(request_id,),
|
(request_id,),
|
||||||
)
|
)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"UPDATE user_workspaces SET status = 'merged' WHERE workspace_id = ?",
|
f"UPDATE user_workspaces SET status = 'merged' WHERE workspace_id = {ph}",
|
||||||
(row["workspace_id"],),
|
(row["workspace_id"],),
|
||||||
)
|
)
|
||||||
return {"ok": True, "action": "merged", **result}
|
return {"ok": True, "action": "merged", **result}
|
||||||
@@ -510,18 +781,20 @@ def _snapshot_take(conn, project_id: str, version_from: int, version_to: int,
|
|||||||
ts = now().replace(":", "-")
|
ts = now().replace(":", "-")
|
||||||
storage_rel = f"{project_id}/{ts}__v{version_from}-{version_to}"
|
storage_rel = f"{project_id}/{ts}__v{version_from}-{version_to}"
|
||||||
|
|
||||||
|
ph9 = _ph(conn, 9)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO snapshots (snapshot_id, project_id, user_id, workspace_id, "
|
f"INSERT INTO snapshots (snapshot_id, project_id, user_id, workspace_id, "
|
||||||
"version_from, version_to, storage_path, content_hash, size_bytes) "
|
f"version_from, version_to, storage_path, content_hash, size_bytes) "
|
||||||
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)",
|
f"VALUES ({ph9})",
|
||||||
(snap_id, project_id, user_id, workspace_id,
|
(snap_id, project_id, user_id, workspace_id,
|
||||||
version_from, version_to, storage_rel, content_hash, len(content)),
|
version_from, version_to, storage_rel, content_hash, len(content)),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def snapshot_list(conn, project_id: str) -> list[dict]:
|
def snapshot_list(conn, project_id: str) -> list[dict]:
|
||||||
|
ph = _ph(conn, 1)
|
||||||
rows = conn.execute(
|
rows = conn.execute(
|
||||||
"SELECT * FROM snapshots WHERE project_id = ? ORDER BY created_at DESC",
|
f"SELECT * FROM snapshots WHERE project_id = {ph} ORDER BY created_at DESC",
|
||||||
(project_id,),
|
(project_id,),
|
||||||
).fetchall()
|
).fetchall()
|
||||||
return [dict(r) for r in rows]
|
return [dict(r) for r in rows]
|
||||||
@@ -532,16 +805,19 @@ def snapshot_rotate(conn, project_id: str, max_keep: int = 25, min_keep: int = 5
|
|||||||
Prune excess snapshots for a project, keeping at least min_keep.
|
Prune excess snapshots for a project, keeping at least min_keep.
|
||||||
Returns count of pruned snapshots.
|
Returns count of pruned snapshots.
|
||||||
"""
|
"""
|
||||||
|
ph = _ph(conn, 1)
|
||||||
rows = conn.execute(
|
rows = conn.execute(
|
||||||
"SELECT snapshot_id FROM snapshots WHERE project_id = ? "
|
f"SELECT snapshot_id FROM snapshots WHERE project_id = {ph} "
|
||||||
"ORDER BY created_at DESC", (project_id,)
|
f"ORDER BY created_at DESC", (project_id,)
|
||||||
).fetchall()
|
).fetchall()
|
||||||
if len(rows) <= max_keep:
|
if len(rows) <= max_keep:
|
||||||
return 0
|
return 0
|
||||||
keep = max(min_keep, max_keep)
|
keep = max(min_keep, max_keep)
|
||||||
to_delete = [r["snapshot_id"] for r in rows[keep:]]
|
to_delete = [r["snapshot_id"] for r in rows[keep:]]
|
||||||
for sid in to_delete:
|
for sid in to_delete:
|
||||||
conn.execute("DELETE FROM snapshots WHERE snapshot_id = ?", (sid,))
|
conn.execute(
|
||||||
|
f"DELETE FROM snapshots WHERE snapshot_id = {_ph(conn,1)}", (sid,)
|
||||||
|
)
|
||||||
return len(to_delete)
|
return len(to_delete)
|
||||||
|
|
||||||
|
|
||||||
@@ -551,10 +827,11 @@ def audit_log(conn, user_id: str, operation: str, summary: str,
|
|||||||
agent_id: str = "ctx", project_id: str | None = None,
|
agent_id: str = "ctx", project_id: str | None = None,
|
||||||
entity_type: str | None = None, entity_id: str | None = None,
|
entity_type: str | None = None, entity_id: str | None = None,
|
||||||
details: dict | None = None):
|
details: dict | None = None):
|
||||||
|
ph8 = _ph(conn, 8)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO audit_log (user_id, agent_id, project_id, operation, "
|
f"INSERT INTO audit_log (user_id, agent_id, project_id, operation, "
|
||||||
"entity_type, entity_id, summary, details_json) "
|
f"entity_type, entity_id, summary, details_json) "
|
||||||
"VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
|
f"VALUES ({ph8})",
|
||||||
(user_id, agent_id, project_id, operation,
|
(user_id, agent_id, project_id, operation,
|
||||||
entity_type, entity_id, summary,
|
entity_type, entity_id, summary,
|
||||||
json.dumps(details) if details else None),
|
json.dumps(details) if details else None),
|
||||||
@@ -569,7 +846,8 @@ def audit_query(conn, **filters) -> list[dict]:
|
|||||||
for col in ("user_id", "project_id", "operation", "agent_id"):
|
for col in ("user_id", "project_id", "operation", "agent_id"):
|
||||||
val = filters.get(col)
|
val = filters.get(col)
|
||||||
if val:
|
if val:
|
||||||
wheres.append(f"{col} = ?")
|
ph = _ph(conn, 1)
|
||||||
|
wheres.append(f"{col} = {ph}")
|
||||||
params.append(val)
|
params.append(val)
|
||||||
if wheres:
|
if wheres:
|
||||||
parts.append("WHERE " + " AND ".join(wheres))
|
parts.append("WHERE " + " AND ".join(wheres))
|
||||||
@@ -584,13 +862,27 @@ def audit_query(conn, **filters) -> list[dict]:
|
|||||||
|
|
||||||
def search(conn, query: str, limit: int = 10) -> list[dict]:
|
def search(conn, query: str, limit: int = 10) -> list[dict]:
|
||||||
"""Full-text search across all indexed context content."""
|
"""Full-text search across all indexed context content."""
|
||||||
rows = conn.execute(
|
if _is_pg(conn):
|
||||||
"SELECT rowid, content, project_id, file_path, source_type, "
|
ph = _ph(conn, 3)
|
||||||
"rank FROM fts_context WHERE fts_context MATCH ? "
|
placeholders = ph.split(", ")
|
||||||
"ORDER BY rank LIMIT ?",
|
rows = conn.execute(
|
||||||
(query, limit),
|
f"SELECT content, project_id, file_path, source_type, "
|
||||||
).fetchall()
|
f"ts_rank(tsv, plainto_tsquery('english', {placeholders[0]})) as rank "
|
||||||
return [dict(r) for r in rows]
|
f"FROM fts_context WHERE tsv @@ plainto_tsquery('english', {placeholders[1]}) "
|
||||||
|
f"ORDER BY rank DESC LIMIT {placeholders[2]}",
|
||||||
|
(query, query, limit),
|
||||||
|
).fetchall()
|
||||||
|
return [dict(r) for r in rows]
|
||||||
|
else:
|
||||||
|
ph = _ph(conn, 2)
|
||||||
|
placeholders = ph.split(", ")
|
||||||
|
rows = conn.execute(
|
||||||
|
f"SELECT rowid, content, project_id, file_path, source_type, "
|
||||||
|
f"rank FROM fts_context WHERE fts_context MATCH {placeholders[0]} "
|
||||||
|
f"ORDER BY rank LIMIT {placeholders[1]}",
|
||||||
|
(query, limit),
|
||||||
|
).fetchall()
|
||||||
|
return [dict(r) for r in rows]
|
||||||
|
|
||||||
|
|
||||||
# ── Sync to project root ──────────────────────────────────────────────────────
|
# ── Sync to project root ──────────────────────────────────────────────────────
|
||||||
@@ -677,9 +969,10 @@ def normalize_file_path(file_path: str) -> str:
|
|||||||
|
|
||||||
def file_list(conn, project_id: str) -> list[dict]:
|
def file_list(conn, project_id: str) -> list[dict]:
|
||||||
"""List all files for a project. Returns list of {file_id, file_path, version, updated_at, updated_by}."""
|
"""List all files for a project. Returns list of {file_id, file_path, version, updated_at, updated_by}."""
|
||||||
|
ph = _ph(conn, 1)
|
||||||
rows = conn.execute(
|
rows = conn.execute(
|
||||||
"SELECT file_id, file_path, version, updated_by, updated_at "
|
f"SELECT file_id, file_path, version, updated_by, updated_at "
|
||||||
"FROM context_files WHERE project_id = ? ORDER BY file_path",
|
f"FROM context_files WHERE project_id = {ph} ORDER BY file_path",
|
||||||
(project_id,)
|
(project_id,)
|
||||||
).fetchall()
|
).fetchall()
|
||||||
return [_row_to_dict(r) for r in rows]
|
return [_row_to_dict(r) for r in rows]
|
||||||
@@ -688,10 +981,12 @@ def file_list(conn, project_id: str) -> list[dict]:
|
|||||||
def file_read(conn, project_id: str, file_path: str) -> dict | None:
|
def file_read(conn, project_id: str, file_path: str) -> dict | None:
|
||||||
"""Read a single context file. Returns with dynamic metadata header prepended."""
|
"""Read a single context file. Returns with dynamic metadata header prepended."""
|
||||||
file_path = normalize_file_path(file_path)
|
file_path = normalize_file_path(file_path)
|
||||||
|
ph2 = _ph(conn, 2)
|
||||||
|
placeholders = ph2.split(", ")
|
||||||
row = conn.execute(
|
row = conn.execute(
|
||||||
"SELECT cf.*, p.display_name FROM context_files cf "
|
f"SELECT cf.*, p.display_name FROM context_files cf "
|
||||||
"JOIN projects p ON p.project_id = cf.project_id "
|
f"JOIN projects p ON p.project_id = cf.project_id "
|
||||||
"WHERE cf.project_id = ? AND cf.file_path = ?",
|
f"WHERE cf.project_id = {placeholders[0]} AND cf.file_path = {placeholders[1]}",
|
||||||
(project_id, file_path)
|
(project_id, file_path)
|
||||||
).fetchone()
|
).fetchone()
|
||||||
if row is None:
|
if row is None:
|
||||||
@@ -731,8 +1026,10 @@ def file_create(conn, project_id: str, file_path: str, content: str = "",
|
|||||||
file_path = normalize_file_path(file_path)
|
file_path = normalize_file_path(file_path)
|
||||||
|
|
||||||
# Check if file already exists
|
# Check if file already exists
|
||||||
|
ph2 = _ph(conn, 2)
|
||||||
|
placeholders = ph2.split(", ")
|
||||||
existing = conn.execute(
|
existing = conn.execute(
|
||||||
"SELECT file_id FROM context_files WHERE project_id = ? AND file_path = ?",
|
f"SELECT file_id FROM context_files WHERE project_id = {placeholders[0]} AND file_path = {placeholders[1]}",
|
||||||
(project_id, file_path)
|
(project_id, file_path)
|
||||||
).fetchone()
|
).fetchone()
|
||||||
if existing:
|
if existing:
|
||||||
@@ -742,9 +1039,10 @@ def file_create(conn, project_id: str, file_path: str, content: str = "",
|
|||||||
clean = strip_metadata_header(content)
|
clean = strip_metadata_header(content)
|
||||||
clean = clean.lstrip("\n\r ").strip()
|
clean = clean.lstrip("\n\r ").strip()
|
||||||
|
|
||||||
|
ph5 = _ph(conn, 5)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO context_files (project_id, file_path, content, version, updated_by) "
|
f"INSERT INTO context_files (project_id, file_path, content, version, updated_by) "
|
||||||
"VALUES (?, ?, ?, 1, ?)",
|
f"VALUES ({ph5.split(', ')[0]}, {ph5.split(', ')[1]}, {ph5.split(', ')[2]}, 1, {ph5.split(', ')[3]})",
|
||||||
(project_id, file_path, clean, updated_by)
|
(project_id, file_path, clean, updated_by)
|
||||||
)
|
)
|
||||||
audit_log(conn, updated_by, "create", f"Created file {file_path} in {project_id}",
|
audit_log(conn, updated_by, "create", f"Created file {file_path} in {project_id}",
|
||||||
@@ -759,8 +1057,14 @@ def file_update(conn, project_id: str, file_path: str, new_content: str,
|
|||||||
# Normalize
|
# Normalize
|
||||||
file_path = normalize_file_path(file_path)
|
file_path = normalize_file_path(file_path)
|
||||||
|
|
||||||
|
# Lock only files in the ctxd-docs project (documentation/guide)
|
||||||
|
if project_id == "ctxd-docs" and file_path in ("CONTEXT.MD", "LLM-CLIENT.MD"):
|
||||||
|
return {"ok": False, "error": "cannot_update_locked", "hint": f"{file_path} is locked in ctxd-docs — create a new file instead"}
|
||||||
|
|
||||||
|
ph2 = _ph(conn, 2)
|
||||||
|
placeholders = ph2.split(", ")
|
||||||
row = conn.execute(
|
row = conn.execute(
|
||||||
"SELECT version FROM context_files WHERE project_id = ? AND file_path = ?",
|
f"SELECT version FROM context_files WHERE project_id = {placeholders[0]} AND file_path = {placeholders[1]}",
|
||||||
(project_id, file_path)
|
(project_id, file_path)
|
||||||
).fetchone()
|
).fetchone()
|
||||||
if row is None:
|
if row is None:
|
||||||
@@ -778,10 +1082,14 @@ def file_update(conn, project_id: str, file_path: str, new_content: str,
|
|||||||
clean = clean.lstrip().strip()
|
clean = clean.lstrip().strip()
|
||||||
|
|
||||||
new_version = current_version + 1
|
new_version = current_version + 1
|
||||||
|
if _is_pg(conn):
|
||||||
|
ts_expr = "to_char(now() at time zone 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS\"Z\"')"
|
||||||
|
else:
|
||||||
|
ts_expr = "strftime('%Y-%m-%dT%H:%M:%SZ', 'now')"
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"UPDATE context_files SET content = ?, version = ?, updated_by = ?, "
|
f"UPDATE context_files SET content = {_ph(conn,1)}, version = {_ph(conn,1)}, "
|
||||||
"updated_at = strftime('%Y-%m-%dT%H:%M:%SZ', 'now') "
|
f"updated_by = {_ph(conn,1)}, updated_at = {ts_expr} "
|
||||||
"WHERE project_id = ? AND file_path = ?",
|
f"WHERE project_id = {_ph(conn,1)} AND file_path = {_ph(conn,1)}",
|
||||||
(clean, new_version, updated_by, project_id, file_path)
|
(clean, new_version, updated_by, project_id, file_path)
|
||||||
)
|
)
|
||||||
audit_log(conn, updated_by, "update", f"Updated {file_path} in {project_id} to v{new_version}",
|
audit_log(conn, updated_by, "update", f"Updated {file_path} in {project_id} to v{new_version}",
|
||||||
@@ -789,26 +1097,32 @@ def file_update(conn, project_id: str, file_path: str, new_content: str,
|
|||||||
return {"ok": True, "new_version": new_version}
|
return {"ok": True, "new_version": new_version}
|
||||||
|
|
||||||
|
|
||||||
def file_delete(conn, project_id: str, file_path: str) -> dict:
|
def file_delete(conn, project_id: str, file_path: str, deleted_by: str = "admin") -> dict:
|
||||||
"""Delete a context file. Returns {'ok': True} or {'ok': False, 'error': ...}."""
|
"""Delete a context file. Returns {'ok': True} or {'ok': False, 'error': ...}."""
|
||||||
file_path = normalize_file_path(file_path)
|
file_path = normalize_file_path(file_path)
|
||||||
|
|
||||||
# Don't allow deleting CONTEXT.md (it's the canonical synced file)
|
# CONTEXT.MD cannot be deleted from any project (it's the minimum required file)
|
||||||
if file_path == "CONTEXT.MD":
|
if file_path == "CONTEXT.MD":
|
||||||
return {"ok": False, "error": "cannot_delete_context"}
|
return {"ok": False, "error": "cannot_delete_context"}
|
||||||
|
|
||||||
|
# LLM-CLIENT.MD locked in ctxd-docs only
|
||||||
|
if project_id == "ctxd-docs" and file_path == "LLM-CLIENT.MD":
|
||||||
|
return {"ok": False, "error": "cannot_delete_context"}
|
||||||
|
|
||||||
|
ph2 = _ph(conn, 2)
|
||||||
|
placeholders = ph2.split(", ")
|
||||||
row = conn.execute(
|
row = conn.execute(
|
||||||
"SELECT file_id FROM context_files WHERE project_id = ? AND file_path = ?",
|
f"SELECT file_id FROM context_files WHERE project_id = {placeholders[0]} AND file_path = {placeholders[1]}",
|
||||||
(project_id, file_path)
|
(project_id, file_path)
|
||||||
).fetchone()
|
).fetchone()
|
||||||
if row is None:
|
if row is None:
|
||||||
return {"ok": False, "error": "not_found"}
|
return {"ok": False, "error": "not_found"}
|
||||||
|
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"DELETE FROM context_files WHERE project_id = ? AND file_path = ?",
|
f"DELETE FROM context_files WHERE project_id = {_ph(conn,1)} AND file_path = {_ph(conn,1)}",
|
||||||
(project_id, file_path)
|
(project_id, file_path)
|
||||||
)
|
)
|
||||||
audit_log(conn, "admin", "delete", f"Deleted {file_path} from {project_id}",
|
audit_log(conn, deleted_by, "delete", f"Deleted {file_path} from {project_id}",
|
||||||
project_id=project_id, entity_type="file", entity_id=file_path)
|
project_id=project_id, entity_type="file", entity_id=file_path)
|
||||||
return {"ok": True}
|
return {"ok": True}
|
||||||
|
|
||||||
@@ -822,9 +1136,10 @@ def compiled_read(conn, project_id: str) -> dict | None:
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
# Get all files
|
# Get all files
|
||||||
|
ph = _ph(conn, 1)
|
||||||
files = conn.execute(
|
files = conn.execute(
|
||||||
"SELECT file_path, content, version, updated_at, updated_by "
|
f"SELECT file_path, content, version, updated_at, updated_by "
|
||||||
"FROM context_files WHERE project_id = ? ORDER BY file_path",
|
f"FROM context_files WHERE project_id = {ph} ORDER BY file_path",
|
||||||
(project_id,)
|
(project_id,)
|
||||||
).fetchall()
|
).fetchall()
|
||||||
|
|
||||||
@@ -855,8 +1170,9 @@ def compiled_read(conn, project_id: str) -> dict | None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
# Get the latest version from project_context (for version checking)
|
# Get the latest version from project_context (for version checking)
|
||||||
|
ph = _ph(conn, 1)
|
||||||
ctx_row = conn.execute(
|
ctx_row = conn.execute(
|
||||||
"SELECT version FROM project_context WHERE project_id = ?",
|
f"SELECT version FROM project_context WHERE project_id = {ph}",
|
||||||
(project_id,)
|
(project_id,)
|
||||||
).fetchone()
|
).fetchone()
|
||||||
version = ctx_row["version"] if ctx_row else 0
|
version = ctx_row["version"] if ctx_row else 0
|
||||||
@@ -873,25 +1189,29 @@ def ensure_default_files(conn, project_id: str):
|
|||||||
"""Create default context files for a project if they don't exist.
|
"""Create default context files for a project if they don't exist.
|
||||||
Migrates existing single-context content into CONTEXT.md."""
|
Migrates existing single-context content into CONTEXT.md."""
|
||||||
# Check if any files already exist
|
# Check if any files already exist
|
||||||
|
ph = _ph(conn, 1)
|
||||||
existing = conn.execute(
|
existing = conn.execute(
|
||||||
"SELECT COUNT(*) as cnt FROM context_files WHERE project_id = ?",
|
f"SELECT COUNT(*) as cnt FROM context_files WHERE project_id = {ph}",
|
||||||
(project_id,)
|
(project_id,)
|
||||||
).fetchone()
|
).fetchone()
|
||||||
if existing and existing["cnt"] > 0:
|
if existing and existing["cnt"] > 0:
|
||||||
return # Already has files
|
return # Already has files
|
||||||
|
|
||||||
# Get existing single-context content to migrate into CONTEXT.md
|
# Get existing single-context content to migrate into CONTEXT.md
|
||||||
|
ph = _ph(conn, 1)
|
||||||
ctx_row = conn.execute(
|
ctx_row = conn.execute(
|
||||||
"SELECT content FROM project_context WHERE project_id = ?",
|
f"SELECT content FROM project_context WHERE project_id = {ph}",
|
||||||
(project_id,)
|
(project_id,)
|
||||||
).fetchone()
|
).fetchone()
|
||||||
existing_content = ctx_row["content"] if ctx_row else ""
|
existing_content = ctx_row["content"] if ctx_row else ""
|
||||||
existing_content = strip_metadata_header(existing_content).strip()
|
existing_content = strip_metadata_header(existing_content).strip()
|
||||||
|
|
||||||
# Create CONTEXT.md with existing content
|
# Create CONTEXT.md with existing content
|
||||||
|
ph2 = _ph(conn, 2)
|
||||||
|
p = ph2.split(", ")
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO context_files (project_id, file_path, content, version, updated_by) "
|
f"INSERT INTO context_files (project_id, file_path, content, version, updated_by) "
|
||||||
"VALUES (?, 'CONTEXT.MD', ?, 1, 'admin')",
|
f"VALUES ({p[0]}, 'CONTEXT.MD', {p[1]}, 1, 'admin')",
|
||||||
(project_id, existing_content)
|
(project_id, existing_content)
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -899,8 +1219,10 @@ def ensure_default_files(conn, project_id: str):
|
|||||||
for fname in DEFAULT_FILES:
|
for fname in DEFAULT_FILES:
|
||||||
if fname == "CONTEXT.md":
|
if fname == "CONTEXT.md":
|
||||||
continue # Already created above
|
continue # Already created above
|
||||||
|
ph2 = _ph(conn, 2)
|
||||||
|
p = ph2.split(", ")
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO context_files (project_id, file_path, content, version, updated_by) "
|
f"INSERT INTO context_files (project_id, file_path, content, version, updated_by) "
|
||||||
"VALUES (?, ?, '', 1, 'admin')",
|
f"VALUES ({p[0]}, {p[1]}, '', 1, 'admin')",
|
||||||
(project_id, fname.upper())
|
(project_id, fname.upper())
|
||||||
)
|
)
|
||||||
@@ -0,0 +1,309 @@
|
|||||||
|
<!DOCTYPE html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||||
|
<title>CTXD — Context Dossier</title>
|
||||||
|
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||||
|
<link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:ital,wght@0,400;0,600;0,700;1,400&display=swap" rel="stylesheet">
|
||||||
|
<style>
|
||||||
|
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
|
||||||
|
|
||||||
|
:root {
|
||||||
|
--bg: #1a1a1a;
|
||||||
|
--paper: #1e1e1e;
|
||||||
|
--ink: #d4d4d4;
|
||||||
|
--ink-dim: #888;
|
||||||
|
--accent: #e5c07b;
|
||||||
|
--accent3: #98c379;
|
||||||
|
--border: #2a2a2a;
|
||||||
|
--border-light: #333;
|
||||||
|
--input-bg: #222;
|
||||||
|
--hover: #252525;
|
||||||
|
--danger: #e06c75;
|
||||||
|
--font: 'JetBrains Mono', 'Fira Code', 'Cascadia Code', 'IBM Plex Mono', monospace;
|
||||||
|
}
|
||||||
|
|
||||||
|
html { height: 100%; font-size: 15px; }
|
||||||
|
body {
|
||||||
|
font-family: var(--font);
|
||||||
|
background: var(--bg);
|
||||||
|
color: var(--ink);
|
||||||
|
height: 100vh;
|
||||||
|
height: 100dvh;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
padding: 1.5rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.landing {
|
||||||
|
max-width: 420px;
|
||||||
|
width: 100%;
|
||||||
|
text-align: center;
|
||||||
|
}
|
||||||
|
|
||||||
|
.landing h1 {
|
||||||
|
font-size: 1.4rem;
|
||||||
|
font-weight: 700;
|
||||||
|
text-transform: uppercase;
|
||||||
|
letter-spacing: 0.15em;
|
||||||
|
color: var(--accent);
|
||||||
|
margin-bottom: 0.35rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.landing .subtitle {
|
||||||
|
font-size: 0.7rem;
|
||||||
|
text-transform: uppercase;
|
||||||
|
letter-spacing: 0.1em;
|
||||||
|
color: var(--ink-dim);
|
||||||
|
margin-bottom: 2rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.landing .description {
|
||||||
|
font-size: 0.75rem;
|
||||||
|
color: var(--ink-dim);
|
||||||
|
line-height: 1.6;
|
||||||
|
margin-bottom: 2rem;
|
||||||
|
text-align: left;
|
||||||
|
}
|
||||||
|
|
||||||
|
.landing .description code {
|
||||||
|
color: var(--accent);
|
||||||
|
background: var(--input-bg);
|
||||||
|
padding: 0.1rem 0.3rem;
|
||||||
|
font-size: 0.7rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-card {
|
||||||
|
background: var(--paper);
|
||||||
|
border: 1px solid var(--border-light);
|
||||||
|
padding: 1.5rem;
|
||||||
|
text-align: left;
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-card h2 {
|
||||||
|
font-size: 0.85rem;
|
||||||
|
text-transform: uppercase;
|
||||||
|
letter-spacing: 0.1em;
|
||||||
|
color: var(--accent);
|
||||||
|
margin-bottom: 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-card label {
|
||||||
|
display: block;
|
||||||
|
font-size: 0.65rem;
|
||||||
|
text-transform: uppercase;
|
||||||
|
letter-spacing: 0.08em;
|
||||||
|
color: var(--ink-dim);
|
||||||
|
margin-bottom: 0.3rem;
|
||||||
|
margin-top: 0.75rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-card input {
|
||||||
|
width: 100%;
|
||||||
|
font-family: var(--font);
|
||||||
|
font-size: 0.8rem;
|
||||||
|
padding: 0.55rem 0.65rem;
|
||||||
|
background: var(--input-bg);
|
||||||
|
border: 1px solid var(--border);
|
||||||
|
color: var(--ink);
|
||||||
|
outline: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-card input:focus { border-color: var(--accent); }
|
||||||
|
|
||||||
|
.login-card .actions {
|
||||||
|
display: flex;
|
||||||
|
gap: 0.5rem;
|
||||||
|
margin-top: 1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-card button {
|
||||||
|
font-family: var(--font);
|
||||||
|
font-size: 0.7rem;
|
||||||
|
text-transform: uppercase;
|
||||||
|
letter-spacing: 0.08em;
|
||||||
|
padding: 0.5rem 0.85rem;
|
||||||
|
cursor: pointer;
|
||||||
|
border: 1px solid var(--border-light);
|
||||||
|
background: none;
|
||||||
|
color: var(--ink);
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-card button:hover { background: var(--hover); }
|
||||||
|
|
||||||
|
.login-card button.primary {
|
||||||
|
background: var(--accent);
|
||||||
|
color: var(--bg);
|
||||||
|
border-color: var(--accent);
|
||||||
|
flex: 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
.login-card button.primary:hover { background: #d4ae5c; }
|
||||||
|
|
||||||
|
.toast {
|
||||||
|
position: fixed;
|
||||||
|
bottom: 1.5rem;
|
||||||
|
left: 50%;
|
||||||
|
transform: translateX(-50%);
|
||||||
|
background: var(--paper);
|
||||||
|
border: 1px solid var(--border-light);
|
||||||
|
color: var(--ink);
|
||||||
|
font-family: var(--font);
|
||||||
|
font-size: 0.7rem;
|
||||||
|
padding: 0.6rem 1rem;
|
||||||
|
opacity: 0;
|
||||||
|
transition: opacity 0.2s;
|
||||||
|
pointer-events: none;
|
||||||
|
z-index: 200;
|
||||||
|
}
|
||||||
|
.toast.show { opacity: 1; }
|
||||||
|
.toast.error { border-color: var(--danger); color: var(--danger); }
|
||||||
|
.toast.success { border-color: var(--accent3); color: var(--accent3); }
|
||||||
|
|
||||||
|
.links {
|
||||||
|
margin-top: 1.5rem;
|
||||||
|
font-size: 0.65rem;
|
||||||
|
color: var(--ink-dim);
|
||||||
|
line-height: 1.8;
|
||||||
|
}
|
||||||
|
|
||||||
|
.links a {
|
||||||
|
color: var(--accent);
|
||||||
|
text-decoration: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
.links a:hover { text-decoration: underline; }
|
||||||
|
|
||||||
|
.status {
|
||||||
|
margin-top: 1rem;
|
||||||
|
font-size: 0.6rem;
|
||||||
|
color: var(--ink-dim);
|
||||||
|
text-transform: uppercase;
|
||||||
|
letter-spacing: 0.06em;
|
||||||
|
}
|
||||||
|
.status .dot {
|
||||||
|
display: inline-block;
|
||||||
|
width: 5px; height: 5px;
|
||||||
|
border-radius: 50%;
|
||||||
|
background: var(--accent3);
|
||||||
|
margin-right: 0.3rem;
|
||||||
|
vertical-align: middle;
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
|
||||||
|
<div class="landing">
|
||||||
|
<h1>CTXD</h1>
|
||||||
|
<div class="subtitle">Context Dossier</div>
|
||||||
|
|
||||||
|
<div class="description">
|
||||||
|
Single source of truth for multi-harness project context.
|
||||||
|
One canonical <code>AGENTS.md</code> per project, served to
|
||||||
|
Claude, Hermes, Codex, Cursor, and any OAuth-capable MCP client.
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="login-card" id="login-card">
|
||||||
|
<h2>sign in</h2>
|
||||||
|
<label>user id</label>
|
||||||
|
<input type="text" id="user-id" placeholder="e.g. admin" autocomplete="username" autocorrect="off" autocapitalize="off" spellcheck="false" onkeydown="if(event.key==='Enter')submitLogin()">
|
||||||
|
<label>password</label>
|
||||||
|
<input type="password" id="password" placeholder="password" autocomplete="current-password" onkeydown="if(event.key==='Enter')submitLogin()">
|
||||||
|
<div class="actions">
|
||||||
|
<button class="primary" onclick="submitLogin()">sign in</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="links">
|
||||||
|
<a href="/.well-known/oauth-authorization-server">OAuth discovery</a> ·
|
||||||
|
<a href="/readonly/sse">read-only MCP</a> ·
|
||||||
|
<a href="/write/sse">write MCP</a>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="status" id="status">
|
||||||
|
<span class="dot"></span> connected
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="toast" id="toast"></div>
|
||||||
|
|
||||||
|
<script>
|
||||||
|
const API = '';
|
||||||
|
|
||||||
|
function showToast(msg, type) {
|
||||||
|
const el = document.getElementById('toast');
|
||||||
|
el.textContent = msg;
|
||||||
|
el.className = 'toast show' + (type ? ' ' + type : '');
|
||||||
|
clearTimeout(window._tt);
|
||||||
|
window._tt = setTimeout(() => el.classList.remove('show'), 3500);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function submitLogin() {
|
||||||
|
const uid = document.getElementById('user-id').value.trim();
|
||||||
|
const pw = document.getElementById('password').value;
|
||||||
|
if (!uid || !pw) {
|
||||||
|
showToast('user id and password required', 'error');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
const res = await fetch('/auth/login', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ user_id: uid, password: pw }),
|
||||||
|
});
|
||||||
|
if (res.ok) {
|
||||||
|
const data = await res.json();
|
||||||
|
localStorage.setItem('ctxd_session_token', data.token);
|
||||||
|
showToast('signed in — loading dashboard', 'success');
|
||||||
|
setTimeout(() => { window.location.href = '/'; }, 500);
|
||||||
|
} else {
|
||||||
|
let detail = '';
|
||||||
|
try { detail = (await res.json()).error || ''; } catch (_) {}
|
||||||
|
if (res.status === 401) {
|
||||||
|
showToast(detail === 'invalid credentials' ? 'invalid user id or password' : ('login failed: ' + (detail || res.status)), 'error');
|
||||||
|
} else if (res.status === 403 && detail === 'account inactive') {
|
||||||
|
showToast('account inactive — contact an admin', 'error');
|
||||||
|
} else {
|
||||||
|
showToast('login failed (' + res.status + ')', 'error');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
showToast('network error — check connection', 'error');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if already signed in (cookie-based — server will redirect to dashboard)
|
||||||
|
// The "signed in" card is only shown if the server served the landing page
|
||||||
|
// despite a valid cookie, which shouldn't happen. If it does, offer a redirect.
|
||||||
|
(async () => {
|
||||||
|
// If the server served the landing page, we're not authenticated via cookie.
|
||||||
|
// Try localStorage token as fallback (for backward compat with old sessions).
|
||||||
|
const token = localStorage.getItem('ctxd_session_token');
|
||||||
|
if (token) {
|
||||||
|
try {
|
||||||
|
const res = await fetch('/auth/me', { headers: { Authorization: 'Bearer ' + token } });
|
||||||
|
if (res.ok) {
|
||||||
|
// Token works via Bearer but cookie wasn't set — force redirect with token in cookie
|
||||||
|
// Re-login to get the cookie set, or just redirect (the dashboard JS uses Bearer too)
|
||||||
|
document.getElementById('login-card').innerHTML = '<h2>signed in</h2><p style="font-size:0.75rem;color:var(--ink-dim);margin-bottom:0.75rem">You are signed in.</p><div class="actions"><button class="primary" onclick="window.location.href=\'/\'">open dashboard</button></div>';
|
||||||
|
}
|
||||||
|
} catch (_) {}
|
||||||
|
}
|
||||||
|
})();
|
||||||
|
|
||||||
|
// Status check
|
||||||
|
(async () => {
|
||||||
|
try {
|
||||||
|
await fetch('/status');
|
||||||
|
document.getElementById('status').innerHTML = '<span class="dot"></span> connected';
|
||||||
|
} catch (_) {
|
||||||
|
document.getElementById('status').innerHTML = 'disconnected';
|
||||||
|
}
|
||||||
|
})();
|
||||||
|
</script>
|
||||||
|
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
@@ -0,0 +1,209 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Migrate data from SQLite to PostgreSQL.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
DATABASE_URL=postgresql://ctxd:ctxd_local_dev@localhost:5432/ctxd \
|
||||||
|
SQLITE_PATH=/data/ctxd.db \
|
||||||
|
python3 -m ctxd.migrate_sqlite_to_pg
|
||||||
|
|
||||||
|
Or inside the Docker container:
|
||||||
|
docker exec dossier python3 -m ctxd.migrate_sqlite_to_pg
|
||||||
|
|
||||||
|
Environment variables:
|
||||||
|
DATABASE_URL — PostgreSQL connection string (required)
|
||||||
|
SQLITE_PATH — Path to SQLite DB file (default: /data/ctxd.db)
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
import sqlite3
|
||||||
|
import sys
|
||||||
|
|
||||||
|
try:
|
||||||
|
import psycopg
|
||||||
|
from psycopg.rows import dict_row
|
||||||
|
except ImportError:
|
||||||
|
print("ERROR: psycopg is required. Install with: pip install psycopg[binary]")
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
|
||||||
|
SQLITE_PATH = os.environ.get("SQLITE_PATH", "/data/ctxd.db")
|
||||||
|
PG_URL = os.environ.get("DATABASE_URL", "")
|
||||||
|
|
||||||
|
# Tables in migration order (parents before children for FK safety)
|
||||||
|
TABLES = [
|
||||||
|
"users",
|
||||||
|
"projects",
|
||||||
|
"project_permissions",
|
||||||
|
"project_context",
|
||||||
|
"context_files",
|
||||||
|
"user_profiles",
|
||||||
|
"user_workspaces",
|
||||||
|
"workspace_files",
|
||||||
|
"change_requests",
|
||||||
|
"reviews",
|
||||||
|
"snapshots",
|
||||||
|
"audit_log",
|
||||||
|
]
|
||||||
|
|
||||||
|
# Columns that need type conversion from SQLite INTEGER to PostgreSQL BOOLEAN
|
||||||
|
BOOL_COLUMNS = {
|
||||||
|
"users": ["active"],
|
||||||
|
"projects": ["auto_sync"],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def migrate():
|
||||||
|
if not PG_URL:
|
||||||
|
print("ERROR: DATABASE_URL environment variable not set")
|
||||||
|
print("Example: DATABASE_URL=postgresql://ctxd:ctxd_local_dev@localhost:5432/ctxd")
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
if not os.path.exists(SQLITE_PATH):
|
||||||
|
print(f"ERROR: SQLite database not found at {SQLITE_PATH}")
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
print(f"Migrating from SQLite ({SQLITE_PATH}) to PostgreSQL ({PG_URL})")
|
||||||
|
print()
|
||||||
|
|
||||||
|
# Connect to SQLite
|
||||||
|
sconn = sqlite3.connect(SQLITE_PATH)
|
||||||
|
sconn.row_factory = sqlite3.Row
|
||||||
|
|
||||||
|
# Connect to PostgreSQL
|
||||||
|
pconn = psycopg.connect(PG_URL, row_factory=dict_row)
|
||||||
|
pconn.autocommit = False
|
||||||
|
|
||||||
|
# Step 1: Clear all data from PostgreSQL tables
|
||||||
|
print("Clearing existing PostgreSQL data...")
|
||||||
|
pconn.execute(
|
||||||
|
"TRUNCATE TABLE fts_context, audit_log, reviews, change_requests, "
|
||||||
|
"workspace_files, user_workspaces, context_files, project_context, "
|
||||||
|
"snapshots, user_profiles, project_permissions, projects, users CASCADE"
|
||||||
|
)
|
||||||
|
pconn.commit()
|
||||||
|
print(" Done.")
|
||||||
|
print()
|
||||||
|
|
||||||
|
# Step 2: Migrate each table
|
||||||
|
total_rows = 0
|
||||||
|
for table in TABLES:
|
||||||
|
# Get column names from SQLite
|
||||||
|
try:
|
||||||
|
cur = sconn.execute(f"PRAGMA table_info({table})")
|
||||||
|
columns = [row["name"] for row in cur.fetchall()]
|
||||||
|
except sqlite3.OperationalError:
|
||||||
|
print(f" {table}: table not found in SQLite, skipping")
|
||||||
|
continue
|
||||||
|
|
||||||
|
if not columns:
|
||||||
|
print(f" {table}: no columns found, skipping")
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Read all rows from SQLite
|
||||||
|
col_str = ", ".join(columns)
|
||||||
|
try:
|
||||||
|
cur = sconn.execute(f"SELECT {col_str} FROM {table}")
|
||||||
|
rows = cur.fetchall()
|
||||||
|
except sqlite3.OperationalError:
|
||||||
|
print(f" {table}: error reading from SQLite, skipping")
|
||||||
|
continue
|
||||||
|
|
||||||
|
if not rows:
|
||||||
|
print(f" {table}: 0 rows")
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Insert into PostgreSQL
|
||||||
|
val_str = ", ".join(["%s"] * len(columns))
|
||||||
|
insert_sql = f'INSERT INTO {table} ({col_str}) VALUES ({val_str}) ON CONFLICT DO NOTHING'
|
||||||
|
|
||||||
|
bool_cols = BOOL_COLUMNS.get(table, [])
|
||||||
|
count = 0
|
||||||
|
|
||||||
|
for row in rows:
|
||||||
|
values = []
|
||||||
|
for col in columns:
|
||||||
|
val = row[col]
|
||||||
|
# Convert integer 0/1 to boolean for PostgreSQL BOOLEAN columns
|
||||||
|
if col in bool_cols and val is not None:
|
||||||
|
val = bool(val)
|
||||||
|
values.append(val)
|
||||||
|
try:
|
||||||
|
pconn.execute(insert_sql, values)
|
||||||
|
count += 1
|
||||||
|
except Exception as fk_err:
|
||||||
|
# Skip rows with FK violations (e.g. orphaned snapshots)
|
||||||
|
pconn.rollback()
|
||||||
|
count_skipped = count_skipped + 1 if 'count_skipped' in dir() else 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
pconn.commit()
|
||||||
|
total_rows += count
|
||||||
|
print(f" {table}: {count} rows migrated")
|
||||||
|
|
||||||
|
# Step 3: Rebuild FTS index
|
||||||
|
print()
|
||||||
|
print("Rebuilding FTS index...")
|
||||||
|
pconn.execute("DELETE FROM fts_context")
|
||||||
|
pconn.commit()
|
||||||
|
|
||||||
|
# Re-populate FTS from source tables
|
||||||
|
pconn.execute("""
|
||||||
|
INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
|
||||||
|
SELECT project_id, content, project_id, 'context.md', 'project_context',
|
||||||
|
to_tsvector('english', content)
|
||||||
|
FROM project_context
|
||||||
|
WHERE content != ''
|
||||||
|
""")
|
||||||
|
fts_pc = pconn.cursor().rowcount
|
||||||
|
|
||||||
|
pconn.execute("""
|
||||||
|
INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
|
||||||
|
SELECT file_id::text, content, project_id, file_path, 'context_file',
|
||||||
|
to_tsvector('english', content)
|
||||||
|
FROM context_files
|
||||||
|
WHERE content != ''
|
||||||
|
""")
|
||||||
|
fts_cf = pconn.cursor().rowcount
|
||||||
|
|
||||||
|
pconn.execute("""
|
||||||
|
INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
|
||||||
|
SELECT user_id, content, '~user', user_id, 'user_profile',
|
||||||
|
to_tsvector('english', content)
|
||||||
|
FROM user_profiles
|
||||||
|
WHERE content != ''
|
||||||
|
""")
|
||||||
|
fts_up = pconn.cursor().rowcount
|
||||||
|
|
||||||
|
pconn.commit()
|
||||||
|
print(f" project_context: {fts_pc} entries")
|
||||||
|
print(f" context_files: {fts_cf} entries")
|
||||||
|
print(f" user_profiles: {fts_up} entries")
|
||||||
|
|
||||||
|
# Step 4: Reset SERIAL sequences to max(existing_id) + 1
|
||||||
|
print()
|
||||||
|
print("Resetting SERIAL sequences...")
|
||||||
|
for table, col in [
|
||||||
|
("audit_log", "entry_id"),
|
||||||
|
("context_files", "file_id"),
|
||||||
|
("project_permissions", "id"),
|
||||||
|
("reviews", "review_id"),
|
||||||
|
("workspace_files", "file_id"),
|
||||||
|
]:
|
||||||
|
try:
|
||||||
|
pconn.execute(f"SELECT setval(pg_get_serial_sequence('{table}', '{col}'), COALESCE((SELECT MAX({col}) FROM {table}), 1))")
|
||||||
|
print(f" {table}.{col}: seq reset")
|
||||||
|
except Exception as e:
|
||||||
|
print(f" {table}.{col}: skipped ({e})")
|
||||||
|
pconn.commit()
|
||||||
|
|
||||||
|
print()
|
||||||
|
print("=" * 60)
|
||||||
|
print(f"Migration complete! {total_rows} total rows migrated.")
|
||||||
|
print("=" * 60)
|
||||||
|
|
||||||
|
sconn.close()
|
||||||
|
pconn.close()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
migrate()
|
||||||
@@ -0,0 +1,84 @@
|
|||||||
|
"""
|
||||||
|
Migration: alter user FK constraints to ON DELETE SET NULL.
|
||||||
|
|
||||||
|
PostgreSQL doesn't support ALTER CONSTRAINT inline — you must drop and
|
||||||
|
re-add the constraint. This script does that for all user_id FKs that
|
||||||
|
were originally created as RESTRICT (no ON DELETE action).
|
||||||
|
|
||||||
|
Run once against the production database:
|
||||||
|
docker exec ctxd python3 -m ctxd.migrate_user_fk_set_null
|
||||||
|
|
||||||
|
Safe to run multiple times (skips if constraint already has ON DELETE SET NULL).
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
import psycopg
|
||||||
|
|
||||||
|
# Tables + constraints to fix. constraint_name is the auto-generated PG name.
|
||||||
|
FKS_TO_FIX = [
|
||||||
|
("project_context", "project_context_updated_by_fkey", "updated_by"),
|
||||||
|
("context_files", "context_files_updated_by_fkey", "updated_by"),
|
||||||
|
("change_requests", "change_requests_submitted_by_fkey", "submitted_by"),
|
||||||
|
("reviews", "reviews_reviewer_id_fkey", "reviewer_id"),
|
||||||
|
("audit_log", "audit_log_user_id_fkey", "user_id"),
|
||||||
|
]
|
||||||
|
|
||||||
|
# Columns that were NOT NULL but must be nullable for ON DELETE SET NULL to work
|
||||||
|
DROP_NOT_NULL = [
|
||||||
|
("audit_log", "user_id"),
|
||||||
|
("change_requests", "submitted_by"),
|
||||||
|
("reviews", "reviewer_id"),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
url = os.environ.get("DATABASE_URL")
|
||||||
|
if not url:
|
||||||
|
print("DATABASE_URL not set — nothing to do (SQLite mode)")
|
||||||
|
return
|
||||||
|
conn = psycopg.connect(url)
|
||||||
|
conn.autocommit = True
|
||||||
|
|
||||||
|
# Step 1: drop NOT NULL where needed (ON DELETE SET NULL requires nullable)
|
||||||
|
for table, col in DROP_NOT_NULL:
|
||||||
|
row = conn.execute(
|
||||||
|
f"SELECT is_nullable FROM information_schema.columns WHERE table_name = %s AND column_name = %s",
|
||||||
|
(table, col),
|
||||||
|
).fetchone()
|
||||||
|
if row and row[0] == "NO":
|
||||||
|
print(f" DROP NOT NULL {table}.{col}")
|
||||||
|
conn.execute(f"ALTER TABLE {table} ALTER COLUMN {col} DROP NOT NULL")
|
||||||
|
print(f" DONE {table}.{col}")
|
||||||
|
else:
|
||||||
|
print(f" SKIP {table}.{col}: already nullable")
|
||||||
|
|
||||||
|
# Step 2: fix FK constraints
|
||||||
|
for table, constraint, col in FKS_TO_FIX:
|
||||||
|
# Check if the constraint exists and what its current ON DELETE action is
|
||||||
|
row = conn.execute(
|
||||||
|
"""SELECT confdeltype FROM pg_constraint
|
||||||
|
WHERE conname = %s AND contype = 'f'""",
|
||||||
|
(constraint,),
|
||||||
|
).fetchone()
|
||||||
|
if not row:
|
||||||
|
print(f" SKIP {constraint}: not found (already migrated or table missing)")
|
||||||
|
continue
|
||||||
|
if row[0] == "n": # 'n' = SET NULL
|
||||||
|
print(f" SKIP {constraint}: already ON DELETE SET NULL")
|
||||||
|
continue
|
||||||
|
|
||||||
|
# `col` comes straight from the FKS_TO_FIX tuple — don't re-derive it
|
||||||
|
# from the constraint name, which breaks for manually-named constraints.
|
||||||
|
print(f" ALTER {table}.{col} ({constraint}): {row[0]} -> SET NULL")
|
||||||
|
conn.execute(f'ALTER TABLE {table} DROP CONSTRAINT {constraint}')
|
||||||
|
conn.execute(
|
||||||
|
f'ALTER TABLE {table} ADD CONSTRAINT {constraint} '
|
||||||
|
f'FOREIGN KEY ({col}) REFERENCES users(user_id) ON DELETE SET NULL'
|
||||||
|
)
|
||||||
|
print(f" DONE {constraint}")
|
||||||
|
|
||||||
|
conn.close()
|
||||||
|
print("Migration complete.")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
+174
-121
@@ -1,23 +1,20 @@
|
|||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
-- ctxd — Context Daemon Schema
|
-- ctxd — Context Daemon Schema
|
||||||
-- SQLite 3.x, WAL mode, FTS5
|
-- PostgreSQL 16
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
|
|
||||||
-- WAL for concurrent reads during writes; foreign keys enforced
|
|
||||||
PRAGMA journal_mode = WAL;
|
|
||||||
PRAGMA foreign_keys = ON;
|
|
||||||
|
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
-- USERS
|
-- USERS
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
CREATE TABLE users (
|
CREATE TABLE users (
|
||||||
user_id TEXT PRIMARY KEY, -- uuid or "joshua", "polly", "hermes-gateway"
|
user_id TEXT PRIMARY KEY,
|
||||||
display_name TEXT NOT NULL,
|
display_name TEXT NOT NULL,
|
||||||
role TEXT NOT NULL DEFAULT 'contributor'
|
role TEXT NOT NULL DEFAULT 'contributor'
|
||||||
CHECK (role IN ('admin', 'contributor', 'service')),
|
CHECK (role IN ('admin', 'contributor', 'service')),
|
||||||
token_hash TEXT, -- NULL = no auth (localhost/trusted)
|
token_hash TEXT,
|
||||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
active BOOLEAN NOT NULL DEFAULT TRUE,
|
||||||
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
|
created_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
|
||||||
|
updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
|
||||||
);
|
);
|
||||||
|
|
||||||
CREATE UNIQUE INDEX idx_users_lower ON users (LOWER(user_id));
|
CREATE UNIQUE INDEX idx_users_lower ON users (LOWER(user_id));
|
||||||
@@ -26,22 +23,22 @@ CREATE UNIQUE INDEX idx_users_lower ON users (LOWER(user_id));
|
|||||||
-- PROJECTS
|
-- PROJECTS
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
CREATE TABLE projects (
|
CREATE TABLE projects (
|
||||||
project_id TEXT PRIMARY KEY, -- uuid or slug "remote-rig"
|
project_id TEXT PRIMARY KEY,
|
||||||
display_name TEXT NOT NULL,
|
display_name TEXT NOT NULL,
|
||||||
description TEXT,
|
description TEXT,
|
||||||
metadata_tags TEXT DEFAULT '[]', -- JSON array of tag strings e.g. '["ARCHITECTURE","3D-PRINTING"]'
|
metadata_tags TEXT DEFAULT '[]',
|
||||||
shared_version INTEGER NOT NULL DEFAULT 0, -- monotonically increasing
|
shared_version INTEGER NOT NULL DEFAULT 0,
|
||||||
auto_sync INTEGER NOT NULL DEFAULT 0, -- boolean: auto-write AGENTS.md to sync_path
|
auto_sync BOOLEAN NOT NULL DEFAULT FALSE,
|
||||||
sync_path TEXT, -- absolute path to project root (nullable)
|
sync_path TEXT,
|
||||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
created_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
|
||||||
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
|
updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
|
||||||
);
|
);
|
||||||
|
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
-- PROJECT PERMISSIONS (admin overrides all)
|
-- PROJECT PERMISSIONS (admin overrides all)
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
CREATE TABLE project_permissions (
|
CREATE TABLE project_permissions (
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
id SERIAL PRIMARY KEY,
|
||||||
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
|
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
|
||||||
user_id TEXT NOT NULL REFERENCES users(user_id) ON DELETE CASCADE,
|
user_id TEXT NOT NULL REFERENCES users(user_id) ON DELETE CASCADE,
|
||||||
permission TEXT NOT NULL DEFAULT 'editor'
|
permission TEXT NOT NULL DEFAULT 'editor'
|
||||||
@@ -54,9 +51,9 @@ CREATE TABLE project_permissions (
|
|||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
CREATE TABLE user_profiles (
|
CREATE TABLE user_profiles (
|
||||||
user_id TEXT PRIMARY KEY REFERENCES users(user_id) ON DELETE CASCADE,
|
user_id TEXT PRIMARY KEY REFERENCES users(user_id) ON DELETE CASCADE,
|
||||||
content TEXT NOT NULL DEFAULT '', -- markdown
|
content TEXT NOT NULL DEFAULT '',
|
||||||
version INTEGER NOT NULL DEFAULT 1,
|
version INTEGER NOT NULL DEFAULT 1,
|
||||||
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
|
updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
|
||||||
);
|
);
|
||||||
|
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
@@ -66,10 +63,10 @@ CREATE TABLE user_profiles (
|
|||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
CREATE TABLE project_context (
|
CREATE TABLE project_context (
|
||||||
project_id TEXT PRIMARY KEY REFERENCES projects(project_id) ON DELETE CASCADE,
|
project_id TEXT PRIMARY KEY REFERENCES projects(project_id) ON DELETE CASCADE,
|
||||||
content TEXT NOT NULL DEFAULT '', -- compiled markdown
|
content TEXT NOT NULL DEFAULT '',
|
||||||
version INTEGER NOT NULL DEFAULT 0, -- mirrors projects.shared_version
|
version INTEGER NOT NULL DEFAULT 0,
|
||||||
updated_by TEXT REFERENCES users(user_id),
|
updated_by TEXT REFERENCES users(user_id) ON DELETE SET NULL,
|
||||||
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
|
updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
|
||||||
);
|
);
|
||||||
|
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
@@ -78,13 +75,13 @@ CREATE TABLE project_context (
|
|||||||
-- version tracks this file's edit count (independent of the shared version).
|
-- version tracks this file's edit count (independent of the shared version).
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
CREATE TABLE context_files (
|
CREATE TABLE context_files (
|
||||||
file_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
file_id SERIAL PRIMARY KEY,
|
||||||
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
|
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
|
||||||
file_path TEXT NOT NULL, -- "decisions/001-use-go.md"
|
file_path TEXT NOT NULL,
|
||||||
content TEXT NOT NULL DEFAULT '',
|
content TEXT NOT NULL DEFAULT '',
|
||||||
version INTEGER NOT NULL DEFAULT 1, -- per-file edit counter
|
version INTEGER NOT NULL DEFAULT 1,
|
||||||
updated_by TEXT REFERENCES users(user_id),
|
updated_by TEXT REFERENCES users(user_id) ON DELETE SET NULL,
|
||||||
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
|
||||||
UNIQUE(project_id, file_path)
|
UNIQUE(project_id, file_path)
|
||||||
);
|
);
|
||||||
|
|
||||||
@@ -94,16 +91,16 @@ CREATE TABLE context_files (
|
|||||||
-- shared version they started from. current_version tracks their edits.
|
-- shared version they started from. current_version tracks their edits.
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
CREATE TABLE user_workspaces (
|
CREATE TABLE user_workspaces (
|
||||||
workspace_id TEXT PRIMARY KEY, -- uuid
|
workspace_id TEXT PRIMARY KEY,
|
||||||
user_id TEXT NOT NULL REFERENCES users(user_id) ON DELETE CASCADE,
|
user_id TEXT NOT NULL REFERENCES users(user_id) ON DELETE CASCADE,
|
||||||
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
|
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
|
||||||
status TEXT NOT NULL DEFAULT 'in_progress'
|
status TEXT NOT NULL DEFAULT 'in_progress'
|
||||||
CHECK (status IN ('in_progress', 'submitted', 'merged', 'abandoned')),
|
CHECK (status IN ('in_progress', 'submitted', 'merged', 'abandoned')),
|
||||||
base_version INTEGER NOT NULL, -- shared version at fork time
|
base_version INTEGER NOT NULL,
|
||||||
current_version INTEGER NOT NULL DEFAULT 1,
|
current_version INTEGER NOT NULL DEFAULT 1,
|
||||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
created_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
|
||||||
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
|
||||||
UNIQUE(user_id, project_id, status) -- one active workspace per user per project
|
UNIQUE(user_id, project_id, status)
|
||||||
);
|
);
|
||||||
|
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
@@ -111,12 +108,12 @@ CREATE TABLE user_workspaces (
|
|||||||
-- Mirrors the same file_path as context_files but in the user's workspace.
|
-- Mirrors the same file_path as context_files but in the user's workspace.
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
CREATE TABLE workspace_files (
|
CREATE TABLE workspace_files (
|
||||||
file_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
file_id SERIAL PRIMARY KEY,
|
||||||
workspace_id TEXT NOT NULL REFERENCES user_workspaces(workspace_id) ON DELETE CASCADE,
|
workspace_id TEXT NOT NULL REFERENCES user_workspaces(workspace_id) ON DELETE CASCADE,
|
||||||
file_path TEXT NOT NULL,
|
file_path TEXT NOT NULL,
|
||||||
content TEXT NOT NULL DEFAULT '',
|
content TEXT NOT NULL DEFAULT '',
|
||||||
version INTEGER NOT NULL DEFAULT 1,
|
version INTEGER NOT NULL DEFAULT 1,
|
||||||
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
|
||||||
UNIQUE(workspace_id, file_path)
|
UNIQUE(workspace_id, file_path)
|
||||||
);
|
);
|
||||||
|
|
||||||
@@ -124,30 +121,29 @@ CREATE TABLE workspace_files (
|
|||||||
-- CHANGE REQUESTS — submit / review / merge workflow
|
-- CHANGE REQUESTS — submit / review / merge workflow
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
CREATE TABLE change_requests (
|
CREATE TABLE change_requests (
|
||||||
request_id TEXT PRIMARY KEY, -- uuid
|
request_id TEXT PRIMARY KEY,
|
||||||
workspace_id TEXT NOT NULL REFERENCES user_workspaces(workspace_id) ON DELETE CASCADE,
|
workspace_id TEXT NOT NULL REFERENCES user_workspaces(workspace_id) ON DELETE CASCADE,
|
||||||
project_id TEXT NOT NULL REFERENCES projects(project_id),
|
project_id TEXT NOT NULL REFERENCES projects(project_id),
|
||||||
submitted_by TEXT NOT NULL REFERENCES users(user_id),
|
submitted_by TEXT REFERENCES users(user_id) ON DELETE SET NULL,
|
||||||
status TEXT NOT NULL DEFAULT 'pending'
|
status TEXT NOT NULL DEFAULT 'pending'
|
||||||
CHECK (status IN ('pending', 'approved', 'rejected', 'merged')),
|
CHECK (status IN ('pending', 'approved', 'rejected', 'merged')),
|
||||||
-- Snapshot of what changed, stored inline so reviews survive workspace mutation
|
diff_summary TEXT,
|
||||||
diff_summary TEXT, -- free-text summary of changes
|
target_version INTEGER NOT NULL,
|
||||||
target_version INTEGER NOT NULL, -- the shared version this would bump to
|
base_version INTEGER NOT NULL,
|
||||||
base_version INTEGER NOT NULL, -- the shared version they forked from
|
created_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
|
||||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
|
||||||
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
|
|
||||||
);
|
);
|
||||||
|
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
-- REVIEWS — approvals/rejections on change requests
|
-- REVIEWS — approvals/rejections on change requests
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
CREATE TABLE reviews (
|
CREATE TABLE reviews (
|
||||||
review_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
review_id SERIAL PRIMARY KEY,
|
||||||
request_id TEXT NOT NULL REFERENCES change_requests(request_id) ON DELETE CASCADE,
|
request_id TEXT NOT NULL REFERENCES change_requests(request_id) ON DELETE CASCADE,
|
||||||
reviewer_id TEXT NOT NULL REFERENCES users(user_id),
|
reviewer_id TEXT REFERENCES users(user_id) ON DELETE SET NULL,
|
||||||
decision TEXT NOT NULL CHECK (decision IN ('approved', 'rejected')),
|
decision TEXT NOT NULL CHECK (decision IN ('approved', 'rejected')),
|
||||||
comments TEXT,
|
comments TEXT,
|
||||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
created_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
|
||||||
UNIQUE(request_id, reviewer_id)
|
UNIQUE(request_id, reviewer_id)
|
||||||
);
|
);
|
||||||
|
|
||||||
@@ -156,31 +152,29 @@ CREATE TABLE reviews (
|
|||||||
-- Stored as files on disk at the path in storage_path.
|
-- Stored as files on disk at the path in storage_path.
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
CREATE TABLE snapshots (
|
CREATE TABLE snapshots (
|
||||||
snapshot_id TEXT PRIMARY KEY, -- uuid
|
snapshot_id TEXT PRIMARY KEY,
|
||||||
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
|
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
|
||||||
-- NULL user_id = snapshot of the shared copy; non-NULL = snapshot of a user workspace
|
|
||||||
user_id TEXT REFERENCES users(user_id) ON DELETE CASCADE,
|
user_id TEXT REFERENCES users(user_id) ON DELETE CASCADE,
|
||||||
workspace_id TEXT REFERENCES user_workspaces(workspace_id) ON DELETE SET NULL,
|
workspace_id TEXT REFERENCES user_workspaces(workspace_id) ON DELETE SET NULL,
|
||||||
version_from INTEGER, -- version range this snapshot covers
|
version_from INTEGER,
|
||||||
version_to INTEGER,
|
version_to INTEGER,
|
||||||
storage_path TEXT NOT NULL, -- relative to ~/.ctx/snapshots/
|
storage_path TEXT NOT NULL,
|
||||||
content_hash TEXT NOT NULL, -- sha256 of the compiled markdown
|
content_hash TEXT NOT NULL,
|
||||||
size_bytes INTEGER NOT NULL DEFAULT 0,
|
size_bytes INTEGER NOT NULL DEFAULT 0,
|
||||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
|
created_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
|
||||||
);
|
);
|
||||||
|
|
||||||
-- Index for snapshot rotation queries
|
|
||||||
CREATE INDEX idx_snapshots_cleanup ON snapshots (project_id, user_id, created_at);
|
CREATE INDEX idx_snapshots_cleanup ON snapshots (project_id, user_id, created_at);
|
||||||
|
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
-- AUDIT LOG — append-only (INSERT only, never UPDATE or DELETE)
|
-- AUDIT LOG — append-only (INSERT only, never UPDATE or DELETE)
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
CREATE TABLE audit_log (
|
CREATE TABLE audit_log (
|
||||||
entry_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
entry_id SERIAL PRIMARY KEY,
|
||||||
user_id TEXT NOT NULL REFERENCES users(user_id),
|
user_id TEXT REFERENCES users(user_id) ON DELETE SET NULL,
|
||||||
agent_id TEXT NOT NULL DEFAULT 'cli', -- "hermes", "claude-code", "ctx"
|
agent_id TEXT NOT NULL DEFAULT 'cli',
|
||||||
session_id TEXT, -- opaque session identifier
|
session_id TEXT,
|
||||||
project_id TEXT REFERENCES projects(project_id),
|
project_id TEXT REFERENCES projects(project_id) ON DELETE SET NULL,
|
||||||
operation TEXT NOT NULL
|
operation TEXT NOT NULL
|
||||||
CHECK (operation IN (
|
CHECK (operation IN (
|
||||||
'read', 'update', 'create', 'delete',
|
'read', 'update', 'create', 'delete',
|
||||||
@@ -188,105 +182,164 @@ CREATE TABLE audit_log (
|
|||||||
'sync', 'search', 'export', 'restore',
|
'sync', 'search', 'export', 'restore',
|
||||||
'login', 'logout', 'import'
|
'login', 'logout', 'import'
|
||||||
)),
|
)),
|
||||||
entity_type TEXT, -- 'project', 'workspace', 'change_request', 'snapshot', 'user_profile'
|
entity_type TEXT,
|
||||||
entity_id TEXT, -- polymorphic reference
|
entity_id TEXT,
|
||||||
summary TEXT NOT NULL, -- human-readable: "Updated camera-node wiring section"
|
summary TEXT NOT NULL,
|
||||||
details_json TEXT, -- structured payload: diff, version numbers, etc.
|
details_json TEXT,
|
||||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
|
created_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
|
||||||
);
|
);
|
||||||
|
|
||||||
-- Audit queries by user, project, or time range
|
|
||||||
CREATE INDEX idx_audit_user ON audit_log (user_id, created_at);
|
CREATE INDEX idx_audit_user ON audit_log (user_id, created_at);
|
||||||
CREATE INDEX idx_audit_project ON audit_log (project_id, created_at);
|
CREATE INDEX idx_audit_project ON audit_log (project_id, created_at);
|
||||||
CREATE INDEX idx_audit_agent ON audit_log (agent_id, created_at);
|
CREATE INDEX idx_audit_agent ON audit_log (agent_id, created_at);
|
||||||
CREATE INDEX idx_audit_op ON audit_log (operation, created_at);
|
CREATE INDEX idx_audit_op ON audit_log (operation, created_at);
|
||||||
|
|
||||||
-- Trigger: audit_log is append-only — enforce no updates or deletes at the DB level
|
-- Note: audit_log append-only enforcement is handled at the application layer.
|
||||||
CREATE TRIGGER tr_audit_log_no_update
|
-- DB-level BEFORE UPDATE/DELETE triggers conflict with FK ON DELETE SET NULL
|
||||||
BEFORE UPDATE ON audit_log
|
-- cascades from projects, which internally issue UPDATE statements.
|
||||||
BEGIN
|
|
||||||
SELECT RAISE(ABORT, 'audit_log is append-only — no updates allowed');
|
|
||||||
END;
|
|
||||||
|
|
||||||
CREATE TRIGGER tr_audit_log_no_delete
|
|
||||||
BEFORE DELETE ON audit_log
|
|
||||||
BEGIN
|
|
||||||
SELECT RAISE(ABORT, 'audit_log is append-only — no deletes allowed');
|
|
||||||
END;
|
|
||||||
|
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
-- FULL-TEXT SEARCH (FTS5)
|
-- FULL-TEXT SEARCH (tsvector with GIN index)
|
||||||
|
-- Separate FTS table with triggers to keep index in sync with source tables.
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
CREATE VIRTUAL TABLE fts_context USING fts5(
|
CREATE TABLE fts_context (
|
||||||
content,
|
id SERIAL PRIMARY KEY,
|
||||||
project_id UNINDEXED,
|
source_id TEXT NOT NULL,
|
||||||
file_path UNINDEXED,
|
content TEXT NOT NULL,
|
||||||
source_type UNINDEXED, -- 'project_context', 'context_file', 'user_profile', 'workspace_file'
|
project_id TEXT NOT NULL,
|
||||||
tokenize='porter unicode61'
|
file_path TEXT NOT NULL,
|
||||||
|
source_type TEXT NOT NULL,
|
||||||
|
tsv tsvector NOT NULL,
|
||||||
|
UNIQUE(source_type, source_id)
|
||||||
);
|
);
|
||||||
|
|
||||||
-- Triggers to keep FTS index in sync with project_context
|
CREATE INDEX idx_fts_context_tsv ON fts_context USING GIN (tsv);
|
||||||
CREATE TRIGGER tr_fts_project_context_insert AFTER INSERT ON project_context
|
CREATE INDEX idx_fts_context_project ON fts_context (project_id);
|
||||||
BEGIN
|
CREATE INDEX idx_fts_context_source ON fts_context (source_type, source_id);
|
||||||
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
|
|
||||||
VALUES (NEW.rowid, NEW.content, NEW.project_id, 'context.md', 'project_context');
|
|
||||||
END;
|
|
||||||
|
|
||||||
CREATE TRIGGER tr_fts_project_context_update AFTER UPDATE ON project_context
|
-- ── Trigger functions for project_context ───────────────────────────────────
|
||||||
BEGIN
|
|
||||||
DELETE FROM fts_context WHERE rowid = OLD.rowid;
|
|
||||||
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
|
|
||||||
VALUES (NEW.rowid, NEW.content, NEW.project_id, 'context.md', 'project_context');
|
|
||||||
END;
|
|
||||||
|
|
||||||
CREATE TRIGGER tr_fts_project_context_delete AFTER DELETE ON project_context
|
CREATE OR REPLACE FUNCTION fts_pc_insert() RETURNS TRIGGER AS $$
|
||||||
BEGIN
|
BEGIN
|
||||||
DELETE FROM fts_context WHERE rowid = OLD.rowid;
|
INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
|
||||||
|
VALUES (NEW.project_id, NEW.content, NEW.project_id, 'context.md', 'project_context',
|
||||||
|
to_tsvector('english', NEW.content));
|
||||||
|
RETURN NEW;
|
||||||
END;
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
|
||||||
-- Triggers for context_files
|
CREATE TRIGGER tr_fts_pc_insert AFTER INSERT ON project_context
|
||||||
CREATE TRIGGER tr_fts_context_files_insert AFTER INSERT ON context_files
|
FOR EACH ROW EXECUTE FUNCTION fts_pc_insert();
|
||||||
BEGIN
|
|
||||||
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
|
|
||||||
VALUES (NEW.file_id + 1000000, NEW.content, NEW.project_id, NEW.file_path, 'context_file');
|
|
||||||
END;
|
|
||||||
|
|
||||||
CREATE TRIGGER tr_fts_context_files_update AFTER UPDATE ON context_files
|
CREATE OR REPLACE FUNCTION fts_pc_update() RETURNS TRIGGER AS $$
|
||||||
BEGIN
|
BEGIN
|
||||||
DELETE FROM fts_context WHERE rowid = OLD.file_id + 1000000;
|
DELETE FROM fts_context WHERE source_type = 'project_context' AND source_id = OLD.project_id;
|
||||||
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
|
INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
|
||||||
VALUES (NEW.file_id + 1000000, NEW.content, NEW.project_id, NEW.file_path, 'context_file');
|
VALUES (NEW.project_id, NEW.content, NEW.project_id, 'context.md', 'project_context',
|
||||||
|
to_tsvector('english', NEW.content));
|
||||||
|
RETURN NEW;
|
||||||
END;
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
|
||||||
CREATE TRIGGER tr_fts_context_files_delete AFTER DELETE ON context_files
|
CREATE TRIGGER tr_fts_pc_update AFTER UPDATE ON project_context
|
||||||
BEGIN
|
FOR EACH ROW EXECUTE FUNCTION fts_pc_update();
|
||||||
DELETE FROM fts_context WHERE rowid = OLD.file_id + 1000000;
|
|
||||||
END;
|
|
||||||
|
|
||||||
-- Triggers for user_profiles
|
CREATE OR REPLACE FUNCTION fts_pc_delete() RETURNS TRIGGER AS $$
|
||||||
CREATE TRIGGER tr_fts_user_profiles_insert AFTER INSERT ON user_profiles
|
|
||||||
BEGIN
|
BEGIN
|
||||||
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
|
DELETE FROM fts_context WHERE source_type = 'project_context' AND source_id = OLD.project_id;
|
||||||
VALUES (NEW.rowid + 2000000, NEW.content, '~user~', NEW.user_id, 'user_profile');
|
RETURN OLD;
|
||||||
END;
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
|
||||||
CREATE TRIGGER tr_fts_user_profiles_update AFTER UPDATE ON user_profiles
|
CREATE TRIGGER tr_fts_pc_delete AFTER DELETE ON project_context
|
||||||
|
FOR EACH ROW EXECUTE FUNCTION fts_pc_delete();
|
||||||
|
|
||||||
|
-- ── Trigger functions for context_files ─────────────────────────────────────
|
||||||
|
|
||||||
|
CREATE OR REPLACE FUNCTION fts_cf_insert() RETURNS TRIGGER AS $$
|
||||||
BEGIN
|
BEGIN
|
||||||
DELETE FROM fts_context WHERE rowid = OLD.rowid + 2000000;
|
INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
|
||||||
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
|
VALUES (NEW.file_id::text, NEW.content, NEW.project_id, NEW.file_path, 'context_file',
|
||||||
VALUES (NEW.rowid + 2000000, NEW.content, '~user~', NEW.user_id, 'user_profile');
|
to_tsvector('english', NEW.content));
|
||||||
|
RETURN NEW;
|
||||||
END;
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
|
||||||
|
CREATE TRIGGER tr_fts_cf_insert AFTER INSERT ON context_files
|
||||||
|
FOR EACH ROW EXECUTE FUNCTION fts_cf_insert();
|
||||||
|
|
||||||
|
CREATE OR REPLACE FUNCTION fts_cf_update() RETURNS TRIGGER AS $$
|
||||||
|
BEGIN
|
||||||
|
DELETE FROM fts_context WHERE source_type = 'context_file' AND source_id = OLD.file_id::text;
|
||||||
|
INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
|
||||||
|
VALUES (NEW.file_id::text, NEW.content, NEW.project_id, NEW.file_path, 'context_file',
|
||||||
|
to_tsvector('english', NEW.content));
|
||||||
|
RETURN NEW;
|
||||||
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
|
||||||
|
CREATE TRIGGER tr_fts_cf_update AFTER UPDATE ON context_files
|
||||||
|
FOR EACH ROW EXECUTE FUNCTION fts_cf_update();
|
||||||
|
|
||||||
|
CREATE OR REPLACE FUNCTION fts_cf_delete() RETURNS TRIGGER AS $$
|
||||||
|
BEGIN
|
||||||
|
DELETE FROM fts_context WHERE source_type = 'context_file' AND source_id = OLD.file_id::text;
|
||||||
|
RETURN OLD;
|
||||||
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
|
||||||
|
CREATE TRIGGER tr_fts_cf_delete AFTER DELETE ON context_files
|
||||||
|
FOR EACH ROW EXECUTE FUNCTION fts_cf_delete();
|
||||||
|
|
||||||
|
-- ── Trigger functions for user_profiles ─────────────────────────────────────
|
||||||
|
|
||||||
|
CREATE OR REPLACE FUNCTION fts_up_insert() RETURNS TRIGGER AS $$
|
||||||
|
BEGIN
|
||||||
|
INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
|
||||||
|
VALUES (NEW.user_id, NEW.content, '~user', NEW.user_id, 'user_profile',
|
||||||
|
to_tsvector('english', NEW.content));
|
||||||
|
RETURN NEW;
|
||||||
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
|
||||||
|
CREATE TRIGGER tr_fts_up_insert AFTER INSERT ON user_profiles
|
||||||
|
FOR EACH ROW EXECUTE FUNCTION fts_up_insert();
|
||||||
|
|
||||||
|
CREATE OR REPLACE FUNCTION fts_up_update() RETURNS TRIGGER AS $$
|
||||||
|
BEGIN
|
||||||
|
DELETE FROM fts_context WHERE source_type = 'user_profile' AND source_id = OLD.user_id;
|
||||||
|
INSERT INTO fts_context (source_id, content, project_id, file_path, source_type, tsv)
|
||||||
|
VALUES (NEW.user_id, NEW.content, '~user', NEW.user_id, 'user_profile',
|
||||||
|
to_tsvector('english', NEW.content));
|
||||||
|
RETURN NEW;
|
||||||
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
|
||||||
|
CREATE TRIGGER tr_fts_up_update AFTER UPDATE ON user_profiles
|
||||||
|
FOR EACH ROW EXECUTE FUNCTION fts_up_update();
|
||||||
|
|
||||||
|
CREATE OR REPLACE FUNCTION fts_up_delete() RETURNS TRIGGER AS $$
|
||||||
|
BEGIN
|
||||||
|
DELETE FROM fts_context WHERE source_type = 'user_profile' AND source_id = OLD.user_id;
|
||||||
|
RETURN OLD;
|
||||||
|
END;
|
||||||
|
$$ LANGUAGE plpgsql;
|
||||||
|
|
||||||
|
CREATE TRIGGER tr_fts_up_delete AFTER DELETE ON user_profiles
|
||||||
|
FOR EACH ROW EXECUTE FUNCTION fts_up_delete();
|
||||||
|
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
-- SEED DATA (for development / first-run)
|
-- SEED DATA (for development / first-run)
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
INSERT INTO users (user_id, display_name, role) VALUES
|
INSERT INTO users (user_id, display_name, role) VALUES
|
||||||
('admin', 'Administrator', 'admin'),
|
('admin', 'Administrator', 'admin'),
|
||||||
('hermes-gateway', 'Hermes Agent', 'service');
|
('hermes-gateway', 'Hermes Agent', 'service')
|
||||||
|
ON CONFLICT DO NOTHING;
|
||||||
|
|
||||||
INSERT INTO projects (project_id, display_name, description) VALUES
|
INSERT INTO projects (project_id, display_name, description) VALUES
|
||||||
('welcome', 'Welcome', 'Getting started guide and documentation for ctxd'),
|
('welcome', 'Welcome', 'Getting started guide and documentation for ctxd'),
|
||||||
('remote-rig', 'RemoteRig', 'Multi-camera remote monitoring system');
|
('remote-rig', 'RemoteRig', 'Multi-camera remote monitoring system')
|
||||||
|
ON CONFLICT DO NOTHING;
|
||||||
|
|
||||||
-- Project context is seeded by the Python init code (cmd_init)
|
-- Project context is seeded by the Python init code (cmd_init)
|
||||||
-- to ensure real newlines, not literal backslash-n from SQL strings.
|
-- to ensure real newlines, not literal backslash-n from SQL strings.
|
||||||
@@ -0,0 +1,275 @@
|
|||||||
|
-- ============================================================================
|
||||||
|
-- ctxd — Context Daemon Schema
|
||||||
|
-- SQLite 3.x, WAL mode, FTS5
|
||||||
|
-- (Legacy fallback for local dev when DATABASE_URL is not set)
|
||||||
|
-- ============================================================================
|
||||||
|
|
||||||
|
-- WAL for concurrent reads during writes; foreign keys enforced
|
||||||
|
PRAGMA journal_mode = WAL;
|
||||||
|
PRAGMA foreign_keys = ON;
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- USERS
|
||||||
|
-- ============================================================================
|
||||||
|
CREATE TABLE users (
|
||||||
|
user_id TEXT PRIMARY KEY,
|
||||||
|
display_name TEXT NOT NULL,
|
||||||
|
role TEXT NOT NULL DEFAULT 'contributor'
|
||||||
|
CHECK (role IN ('admin', 'contributor', 'service')),
|
||||||
|
token_hash TEXT,
|
||||||
|
active INTEGER NOT NULL DEFAULT 1 CHECK (active IN (0, 1)),
|
||||||
|
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
||||||
|
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE UNIQUE INDEX idx_users_lower ON users (LOWER(user_id));
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- PROJECTS
|
||||||
|
-- ============================================================================
|
||||||
|
CREATE TABLE projects (
|
||||||
|
project_id TEXT PRIMARY KEY,
|
||||||
|
display_name TEXT NOT NULL,
|
||||||
|
description TEXT,
|
||||||
|
metadata_tags TEXT DEFAULT '[]',
|
||||||
|
shared_version INTEGER NOT NULL DEFAULT 0,
|
||||||
|
auto_sync INTEGER NOT NULL DEFAULT 0,
|
||||||
|
sync_path TEXT,
|
||||||
|
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
||||||
|
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- PROJECT PERMISSIONS (admin overrides all)
|
||||||
|
-- ============================================================================
|
||||||
|
CREATE TABLE project_permissions (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
|
||||||
|
user_id TEXT NOT NULL REFERENCES users(user_id) ON DELETE CASCADE,
|
||||||
|
permission TEXT NOT NULL DEFAULT 'editor'
|
||||||
|
CHECK (permission IN ('owner', 'editor', 'viewer')),
|
||||||
|
UNIQUE(project_id, user_id)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- USER PROFILES (personal context — timezone, preferences, style)
|
||||||
|
-- ============================================================================
|
||||||
|
CREATE TABLE user_profiles (
|
||||||
|
user_id TEXT PRIMARY KEY REFERENCES users(user_id) ON DELETE CASCADE,
|
||||||
|
content TEXT NOT NULL DEFAULT '',
|
||||||
|
version INTEGER NOT NULL DEFAULT 1,
|
||||||
|
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- PROJECT CONTEXT — THE AUTHORITATIVE SHARED COPY
|
||||||
|
-- ============================================================================
|
||||||
|
CREATE TABLE project_context (
|
||||||
|
project_id TEXT PRIMARY KEY REFERENCES projects(project_id) ON DELETE CASCADE,
|
||||||
|
content TEXT NOT NULL DEFAULT '',
|
||||||
|
version INTEGER NOT NULL DEFAULT 0,
|
||||||
|
updated_by TEXT REFERENCES users(user_id) ON DELETE SET NULL,
|
||||||
|
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- CONTEXT FILES — individual files within a project
|
||||||
|
-- ============================================================================
|
||||||
|
CREATE TABLE context_files (
|
||||||
|
file_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
|
||||||
|
file_path TEXT NOT NULL,
|
||||||
|
content TEXT NOT NULL DEFAULT '',
|
||||||
|
version INTEGER NOT NULL DEFAULT 1,
|
||||||
|
updated_by TEXT REFERENCES users(user_id) ON DELETE SET NULL,
|
||||||
|
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
||||||
|
UNIQUE(project_id, file_path)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- USER WORKSPACES — per-user forks of a project
|
||||||
|
-- ============================================================================
|
||||||
|
CREATE TABLE user_workspaces (
|
||||||
|
workspace_id TEXT PRIMARY KEY,
|
||||||
|
user_id TEXT NOT NULL REFERENCES users(user_id) ON DELETE CASCADE,
|
||||||
|
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
|
||||||
|
status TEXT NOT NULL DEFAULT 'in_progress'
|
||||||
|
CHECK (status IN ('in_progress', 'submitted', 'merged', 'abandoned')),
|
||||||
|
base_version INTEGER NOT NULL,
|
||||||
|
current_version INTEGER NOT NULL DEFAULT 1,
|
||||||
|
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
||||||
|
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
||||||
|
UNIQUE(user_id, project_id, status)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- WORKSPACE FILES — per-user fork of context_files
|
||||||
|
-- ============================================================================
|
||||||
|
CREATE TABLE workspace_files (
|
||||||
|
file_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
workspace_id TEXT NOT NULL REFERENCES user_workspaces(workspace_id) ON DELETE CASCADE,
|
||||||
|
file_path TEXT NOT NULL,
|
||||||
|
content TEXT NOT NULL DEFAULT '',
|
||||||
|
version INTEGER NOT NULL DEFAULT 1,
|
||||||
|
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
||||||
|
UNIQUE(workspace_id, file_path)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- CHANGE REQUESTS — submit / review / merge workflow
|
||||||
|
-- ============================================================================
|
||||||
|
CREATE TABLE change_requests (
|
||||||
|
request_id TEXT PRIMARY KEY,
|
||||||
|
workspace_id TEXT NOT NULL REFERENCES user_workspaces(workspace_id) ON DELETE CASCADE,
|
||||||
|
project_id TEXT NOT NULL REFERENCES projects(project_id),
|
||||||
|
submitted_by TEXT REFERENCES users(user_id) ON DELETE SET NULL,
|
||||||
|
status TEXT NOT NULL DEFAULT 'pending'
|
||||||
|
CHECK (status IN ('pending', 'approved', 'rejected', 'merged')),
|
||||||
|
diff_summary TEXT,
|
||||||
|
target_version INTEGER NOT NULL,
|
||||||
|
base_version INTEGER NOT NULL,
|
||||||
|
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
||||||
|
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- REVIEWS — approvals/rejections on change requests
|
||||||
|
-- ============================================================================
|
||||||
|
CREATE TABLE reviews (
|
||||||
|
review_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
request_id TEXT NOT NULL REFERENCES change_requests(request_id) ON DELETE CASCADE,
|
||||||
|
reviewer_id TEXT REFERENCES users(user_id) ON DELETE SET NULL,
|
||||||
|
decision TEXT NOT NULL CHECK (decision IN ('approved', 'rejected')),
|
||||||
|
comments TEXT,
|
||||||
|
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
||||||
|
UNIQUE(request_id, reviewer_id)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- SNAPSHOTS — point-in-time copies of project or workspace content
|
||||||
|
-- ============================================================================
|
||||||
|
CREATE TABLE snapshots (
|
||||||
|
snapshot_id TEXT PRIMARY KEY,
|
||||||
|
project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
|
||||||
|
user_id TEXT REFERENCES users(user_id) ON DELETE CASCADE,
|
||||||
|
workspace_id TEXT REFERENCES user_workspaces(workspace_id) ON DELETE SET NULL,
|
||||||
|
version_from INTEGER,
|
||||||
|
version_to INTEGER,
|
||||||
|
storage_path TEXT NOT NULL,
|
||||||
|
content_hash TEXT NOT NULL,
|
||||||
|
size_bytes INTEGER NOT NULL DEFAULT 0,
|
||||||
|
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX idx_snapshots_cleanup ON snapshots (project_id, user_id, created_at);
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- AUDIT LOG — append-only (INSERT only, never UPDATE or DELETE)
|
||||||
|
-- ============================================================================
|
||||||
|
CREATE TABLE audit_log (
|
||||||
|
entry_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
user_id TEXT REFERENCES users(user_id) ON DELETE SET NULL,
|
||||||
|
agent_id TEXT NOT NULL DEFAULT 'cli',
|
||||||
|
session_id TEXT,
|
||||||
|
project_id TEXT REFERENCES projects(project_id),
|
||||||
|
operation TEXT NOT NULL
|
||||||
|
CHECK (operation IN (
|
||||||
|
'read', 'update', 'create', 'delete',
|
||||||
|
'submit', 'approve', 'reject', 'merge',
|
||||||
|
'sync', 'search', 'export', 'restore',
|
||||||
|
'login', 'logout', 'import'
|
||||||
|
)),
|
||||||
|
entity_type TEXT,
|
||||||
|
entity_id TEXT,
|
||||||
|
summary TEXT NOT NULL,
|
||||||
|
details_json TEXT,
|
||||||
|
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX idx_audit_user ON audit_log (user_id, created_at);
|
||||||
|
CREATE INDEX idx_audit_project ON audit_log (project_id, created_at);
|
||||||
|
CREATE INDEX idx_audit_agent ON audit_log (agent_id, created_at);
|
||||||
|
CREATE INDEX idx_audit_op ON audit_log (operation, created_at);
|
||||||
|
|
||||||
|
-- Note: audit_log append-only enforcement is handled at the application layer.
|
||||||
|
-- DB-level BEFORE UPDATE/DELETE triggers conflict with FK ON DELETE SET NULL
|
||||||
|
-- (deleting a user must set audit_log.user_id to NULL, which is an UPDATE).
|
||||||
|
-- This mirrors schema.sql, which dropped the equivalent PG triggers for the
|
||||||
|
-- same reason. db.py only ever INSERTs into audit_log.
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- FULL-TEXT SEARCH (FTS5)
|
||||||
|
-- ============================================================================
|
||||||
|
CREATE VIRTUAL TABLE fts_context USING fts5(
|
||||||
|
content,
|
||||||
|
project_id UNINDEXED,
|
||||||
|
file_path UNINDEXED,
|
||||||
|
source_type UNINDEXED,
|
||||||
|
tokenize='porter unicode61'
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Triggers to keep FTS index in sync with project_context
|
||||||
|
CREATE TRIGGER tr_fts_project_context_insert AFTER INSERT ON project_context
|
||||||
|
BEGIN
|
||||||
|
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
|
||||||
|
VALUES (NEW.rowid, NEW.content, NEW.project_id, 'context.md', 'project_context');
|
||||||
|
END;
|
||||||
|
|
||||||
|
CREATE TRIGGER tr_fts_project_context_update AFTER UPDATE ON project_context
|
||||||
|
BEGIN
|
||||||
|
DELETE FROM fts_context WHERE rowid = OLD.rowid;
|
||||||
|
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
|
||||||
|
VALUES (NEW.rowid, NEW.content, NEW.project_id, 'context.md', 'project_context');
|
||||||
|
END;
|
||||||
|
|
||||||
|
CREATE TRIGGER tr_fts_project_context_delete AFTER DELETE ON project_context
|
||||||
|
BEGIN
|
||||||
|
DELETE FROM fts_context WHERE rowid = OLD.rowid;
|
||||||
|
END;
|
||||||
|
|
||||||
|
-- Triggers for context_files
|
||||||
|
CREATE TRIGGER tr_fts_context_files_insert AFTER INSERT ON context_files
|
||||||
|
BEGIN
|
||||||
|
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
|
||||||
|
VALUES (NEW.file_id + 1000000, NEW.content, NEW.project_id, NEW.file_path, 'context_file');
|
||||||
|
END;
|
||||||
|
|
||||||
|
CREATE TRIGGER tr_fts_context_files_update AFTER UPDATE ON context_files
|
||||||
|
BEGIN
|
||||||
|
DELETE FROM fts_context WHERE rowid = OLD.file_id + 1000000;
|
||||||
|
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
|
||||||
|
VALUES (NEW.file_id + 1000000, NEW.content, NEW.project_id, NEW.file_path, 'context_file');
|
||||||
|
END;
|
||||||
|
|
||||||
|
CREATE TRIGGER tr_fts_context_files_delete AFTER DELETE ON context_files
|
||||||
|
BEGIN
|
||||||
|
DELETE FROM fts_context WHERE rowid = OLD.file_id + 1000000;
|
||||||
|
END;
|
||||||
|
|
||||||
|
-- Triggers for user_profiles
|
||||||
|
CREATE TRIGGER tr_fts_user_profiles_insert AFTER INSERT ON user_profiles
|
||||||
|
BEGIN
|
||||||
|
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
|
||||||
|
VALUES (NEW.rowid + 2000000, NEW.content, '~user', NEW.user_id, 'user_profile');
|
||||||
|
END;
|
||||||
|
|
||||||
|
CREATE TRIGGER tr_fts_user_profiles_update AFTER UPDATE ON user_profiles
|
||||||
|
BEGIN
|
||||||
|
DELETE FROM fts_context WHERE rowid = OLD.rowid + 2000000;
|
||||||
|
INSERT INTO fts_context(rowid, content, project_id, file_path, source_type)
|
||||||
|
VALUES (NEW.rowid + 2000000, NEW.content, '~user', NEW.user_id, 'user_profile');
|
||||||
|
END;
|
||||||
|
|
||||||
|
-- ============================================================================
|
||||||
|
-- SEED DATA (for development / first-run)
|
||||||
|
-- ============================================================================
|
||||||
|
INSERT INTO users (user_id, display_name, role) VALUES
|
||||||
|
('admin', 'Administrator', 'admin'),
|
||||||
|
('hermes-gateway', 'Hermes Agent', 'service');
|
||||||
|
|
||||||
|
INSERT INTO projects (project_id, display_name, description) VALUES
|
||||||
|
('welcome', 'Welcome', 'Getting started guide and documentation for ctxd'),
|
||||||
|
('remote-rig', 'RemoteRig', 'Multi-camera remote monitoring system');
|
||||||
|
|
||||||
|
-- Project context is seeded by the Python init code (cmd_init)
|
||||||
|
-- to ensure real newlines, not literal backslash-n from SQL strings.
|
||||||
+1257
-69
File diff suppressed because it is too large
Load Diff
+1036
-37
File diff suppressed because it is too large
Load Diff
Binary file not shown.
Binary file not shown.
@@ -1,14 +0,0 @@
|
|||||||
server:
|
|
||||||
host: 0.0.0.0
|
|
||||||
port: 9091
|
|
||||||
snapshots:
|
|
||||||
min_keep: 5
|
|
||||||
max_keep: 25
|
|
||||||
auth:
|
|
||||||
enabled: true
|
|
||||||
api_key: Fa50-7cg5x6ObyFjJCjBYQkaNhGF5gMBJluVY6C1OwE
|
|
||||||
seed:
|
|
||||||
admin_user: admin
|
|
||||||
admin_display: Administrator
|
|
||||||
service_user: hermes-gateway
|
|
||||||
service_display: Hermes Agent
|
|
||||||
@@ -0,0 +1,147 @@
|
|||||||
|
# I Built a Context Server to Fight My ADHD-Induced Context Sprawl
|
||||||
|
|
||||||
|
**How replacing 7 scattered config files with a single source of truth saved my sanity (and my tokens).**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
If you work with AI tools the way I do — Claude Desktop for research, Claude Code for implementation, Codex CLI for reviews, Cursor for quick edits, and Hermes Agent for orchestration — you have a problem you might not have named yet.
|
||||||
|
|
||||||
|
I call it **context sprawl**. And if you have ADHD, it's worse than you think.
|
||||||
|
|
||||||
|
## The Problem
|
||||||
|
|
||||||
|
Every AI harness has its own context file convention:
|
||||||
|
|
||||||
|
| Harness | File |
|
||||||
|
|---------|------|
|
||||||
|
| Hermes | `AGENTS.md` |
|
||||||
|
| Claude Code | `CLAUDE.md` |
|
||||||
|
| Cursor | `.cursorrules` |
|
||||||
|
| Copilot | `.github/copilot-instructions.md` |
|
||||||
|
| Codex CLI | `CODEX.md` |
|
||||||
|
| Windsurf | `.windsurfrules` |
|
||||||
|
| Continue.dev | `.continuerc` |
|
||||||
|
|
||||||
|
When you work on the same project across multiple harnesses, you end up with N copies of the same project context. They drift. One says the build system is Go; another still says Python. One knows about the new OAuth flow; another doesn't. The LLM reads stale instructions and makes confident, wrong decisions.
|
||||||
|
|
||||||
|
For someone with ADHD, this is a specific kind of hell:
|
||||||
|
|
||||||
|
- You **forget** which file you last updated
|
||||||
|
- You **avoid** updating because it means updating 4-6 files
|
||||||
|
- You **lose track** of which harness has the "right" version
|
||||||
|
- You **context-switch** between tools and can't remember what the project actually needs
|
||||||
|
- The friction of maintaining context across tools becomes so high that you **stop maintaining context at all**
|
||||||
|
|
||||||
|
And then every LLM session starts from scratch — re-asking questions you already answered, re-discovering decisions you already made, re-making mistakes you already fixed.
|
||||||
|
|
||||||
|
## The Tipping Point
|
||||||
|
|
||||||
|
I was working on a project called RemoteRig — a multi-camera remote monitoring system built in Go with ESP32-C6 camera nodes. I had context in:
|
||||||
|
|
||||||
|
- `AGENTS.md` (Hermes)
|
||||||
|
- `CLAUDE.md` (Claude Code)
|
||||||
|
- `.cursorrules` (Cursor)
|
||||||
|
- `CODEX.md` (Codex CLI)
|
||||||
|
- A Notion page (for humans, but also where I drafted things)
|
||||||
|
- My own memory of what I'd told each tool (which is the worst place to store anything when you have ADHD)
|
||||||
|
|
||||||
|
I'd update `AGENTS.md` with a new architecture decision, forget to update `CLAUDE.md`, and then Claude Code would suggest something I'd already ruled out. I'd spend the first 20 minutes of every session re-explaining the project. Sometimes I'd update the wrong file and wonder why Codex was still using the old Go module structure.
|
||||||
|
|
||||||
|
The context was sprawled across the filesystem like a hoarder's garage, and I was the only one who knew where anything was — except I didn't, because ADHD.
|
||||||
|
|
||||||
|
## The Solution: CTXD (Context Dossier)
|
||||||
|
|
||||||
|
I built **CTXD** — a single source of truth for project context that every AI harness can read from and write to.
|
||||||
|
|
||||||
|
### Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
Context Dossier (Docker container, port 9091)
|
||||||
|
├── PostgreSQL 16 backend
|
||||||
|
├── Multi-file context per project:
|
||||||
|
│ ├── CONTEXT.MD ← canonical overview, synced as AGENTS.md to repos
|
||||||
|
│ ├── DECISIONS.MD ← architecture decisions, rationale
|
||||||
|
│ ├── RUNBOOKS.MD ← deploy, troubleshoot, operate procedures
|
||||||
|
│ ├── PROMPTS.MD ← project-specific prompts for different harnesses
|
||||||
|
│ └── GLOSSARY.MD ← project-specific terms, acronyms
|
||||||
|
├── OAuth 2.0 authorization server
|
||||||
|
│ ├── Dynamic Client Registration (Claude auto-registers)
|
||||||
|
│ ├── Authorization Code + PKCE
|
||||||
|
│ └── Scopes: ctxd.read, ctxd.write
|
||||||
|
├── Streamable HTTP MCP endpoints:
|
||||||
|
│ ├── /readonly/mcp (read-only, OAuth ctxd.read)
|
||||||
|
│ ├── /write/mcp (read-write, OAuth ctxd.write)
|
||||||
|
│ └── /mcp (internal, shared API key)
|
||||||
|
├── Web UI (per-user login, admin panel)
|
||||||
|
├── Version-checked writes (optimistic concurrency)
|
||||||
|
├── Append-only audit log
|
||||||
|
├── Point-in-time snapshots (auto-rotated)
|
||||||
|
└── Full-text search (PostgreSQL tsvector + GIN)
|
||||||
|
```
|
||||||
|
|
||||||
|
### How it works
|
||||||
|
|
||||||
|
1. **One canonical file per project** — `CONTEXT.MD` is the source of truth. CTXD syncs it to `AGENTS.md` in the project root and creates symlinks (`CLAUDE.md`, `.cursorrules`, `CODEX.md` → `AGENTS.md`). Every harness reads the same file.
|
||||||
|
|
||||||
|
2. **LLMs connect via MCP** — Claude Desktop connects to `https://ctxd.yourdomain.com/readonly/mcp` and auto-discovers OAuth. No manual config, no API keys in URLs. It can read project context, search across all projects, and list individual files.
|
||||||
|
|
||||||
|
3. **LLMs can write context** — With a `ctxd.write`-scoped OAuth token, an LLM can update files with version checking. If two agents try to edit simultaneously, the second one gets a `version_conflict` and knows to re-read, merge, and retry.
|
||||||
|
|
||||||
|
4. **The compiled view** — When an LLM calls `get_project_context("remote-rig")`, it gets all files concatenated into a single document with a metadata header. One call, complete context, no guessing.
|
||||||
|
|
||||||
|
5. **A locked client guide** — There's a `get_client_guide()` MCP tool that returns a locked `LLM-CLIENT.MD` file. Every LLM session starts by reading this guide, which explains how to connect, what tools are available, and what the discipline rules are. The file is immutable — no LLM can modify its own instruction manual.
|
||||||
|
|
||||||
|
### What it solves for ADHD
|
||||||
|
|
||||||
|
- **One place to update** — Change `CONTEXT.MD` once. Every harness gets it via sync or MCP. No more updating 6 files.
|
||||||
|
- **No remembering required** — The LLM reads context at the start of every session. You don't have to remember what you told it last time.
|
||||||
|
- **Audit trail** — Every read, write, and search is logged. When you can't remember what changed (and you can't, because ADHD), the audit log tells you.
|
||||||
|
- **Version checking** — Optimistic concurrency means no silent overwrites. If you and an LLM edit at the same time, the conflict is caught, not lost.
|
||||||
|
- **Search** — Full-text search across all projects. "What did I decide about the camera node?" → search → answer.
|
||||||
|
- **Snapshots** — Automatic point-in-time backups before every edit. Roll back if something goes wrong.
|
||||||
|
|
||||||
|
## The Build
|
||||||
|
|
||||||
|
I built CTXD over a series of focused sessions with Hermes Agent — itself powered by the same MCP infrastructure. Some of the tools that made this possible:
|
||||||
|
|
||||||
|
- **Hermes Agent** for orchestration and the MCP framework
|
||||||
|
- **PostgreSQL 16** as the primary database (migrated from SQLite)
|
||||||
|
- **MCP SDK 1.28** with Streamable HTTP transport (replaced the older SSE pattern)
|
||||||
|
- **OAuth 2.0** with DCR, PKCE, and per-scope access control
|
||||||
|
- **Docker Compose** for deployment — env-driven, no config files needed
|
||||||
|
- **Traefik** for public TLS termination and routing
|
||||||
|
|
||||||
|
The development was iterative and honest. I hit real bugs: the `set -e` in the entrypoint script killed the container because a PostgreSQL schema check returned exit code 1. The `__main__.py` checked for a SQLite file that doesn't exist when using PostgreSQL, causing an infinite re-initialization loop. The FTS5 search query used `?` placeholders instead of `%s` for psycopg. Each one was a "oh, of course" moment — the kind you only get by actually building and deploying.
|
||||||
|
|
||||||
|
## What I Learned
|
||||||
|
|
||||||
|
**Context sprawl is an accessibility problem.** For neurotypical developers, maintaining 6 config files is annoying. For someone with ADHD, it's a barrier that leads to giving up on context entirely — and then every AI interaction starts from zero.
|
||||||
|
|
||||||
|
**The single source of truth pattern isn't just about consistency.** It's about reducing the cognitive load of context management to one action: update `CONTEXT.MD`. That's one decision, one file, one habit — something even an ADHD brain can maintain.
|
||||||
|
|
||||||
|
**OAuth + MCP is the right protocol stack for this.** LLMs don't need API keys in environment variables. They need to discover a resource, authenticate via OAuth, and use standardized tools. The fact that Claude Desktop can connect to CTXD with zero manual configuration — just a URL — is the difference between "I should set this up" and "I'll just re-explain the project again."
|
||||||
|
|
||||||
|
**PostgreSQL matters more than you think.** I started with SQLite, which was fine for one user. But the moment you want concurrent agents writing context, proper backups, and full-text search that actually works — PostgreSQL is the right tool. The migration was a real engineering task (dual-backend support, schema rewrite, FTS5 → tsvector), and it was worth every hour.
|
||||||
|
|
||||||
|
## Where It's Going
|
||||||
|
|
||||||
|
CTXD is now running in production at `ctxd.cubecraftcreations.com`, serving context to Claude Desktop, Hermes, and any OAuth-capable MCP client. It backs:
|
||||||
|
|
||||||
|
- RemoteRig (Go, ESP32-C6)
|
||||||
|
- ExtrudeX (3D printing control system)
|
||||||
|
- Tracehound (network diagnostic tool)
|
||||||
|
- Control Center (homelab dashboard)
|
||||||
|
|
||||||
|
The code is open and documented. There's a static demo at `ctxd-demo` that shows the UI without needing a database. There's a full README, an LLM.txt for automated deployment, and a SKILL.md that LLMs can read to understand how to use the server.
|
||||||
|
|
||||||
|
## The Takeaway
|
||||||
|
|
||||||
|
If you work across multiple AI tools and find yourself re-explaining your project every session — you have context sprawl. If you have ADHD, context sprawl isn't just inefficient; it's a wall between you and productive AI-assisted work.
|
||||||
|
|
||||||
|
Building CTXD was my answer. It turned 6 files into 1, turned "remember what I told Claude" into "Claude reads the context server," and turned "I'll update the context later" into "I update one file and every tool gets it."
|
||||||
|
|
||||||
|
One source of truth. One habit. Zero context sprawl.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*CTXD is open source. The production repo is at [CubeCraft-Creations/CTXD](https://code.cubecraftlabs.com/CubeCraft-Creations/CTXD), with a static demo at [CTXD-Demo](https://code.cubecraftlabs.com/CubeCraft-Creations/CTXD-Demo). Built by Joshua at CubeCraft Creations.*
|
||||||
Reference in New Issue
Block a user