Compare commits
22 Commits
fc1a2f5103
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 3ef4f3e707 | |||
| bbb0215c98 | |||
| bdc984e5ff | |||
| 80635ce011 | |||
| 9bb89ee62f | |||
| bc43e9a8d1 | |||
| d2c6906c4f | |||
| 570b7d1dba | |||
| 451732c867 | |||
| 9e85c1b8ec | |||
| 59609f93c4 | |||
| 07cf223d16 | |||
| ce1c0a175f | |||
| 1c9d8f7648 | |||
| 87f02eb4d1 | |||
| 289c6b9300 | |||
| 12b60ee8c7 | |||
| e3567f649f | |||
| fe63ad350e | |||
| 364c7795d4 | |||
| b9f911994d | |||
| b91d03a6cd |
@@ -20,3 +20,6 @@ app/.env
|
|||||||
# Python cache
|
# Python cache
|
||||||
__pycache__/
|
__pycache__/
|
||||||
*.py[cod]
|
*.py[cod]
|
||||||
|
|
||||||
|
# Build artifacts (regenerated by `pip install -e`)
|
||||||
|
*.egg-info/
|
||||||
|
|||||||
@@ -0,0 +1,108 @@
|
|||||||
|
# CLAUDE.md
|
||||||
|
|
||||||
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||||
|
|
||||||
|
## What CTXD is
|
||||||
|
|
||||||
|
A single-process daemon that stores per-project "context dossiers" (multiple `.MD` files:
|
||||||
|
`CONTEXT.MD`, `DECISIONS.MD`, `RUNBOOKS.MD`, `PROMPTS.MD`, `GLOSSARY.MD`) and serves them to
|
||||||
|
LLM harnesses (Claude, ChatGPT, Hermes, Codex, Cursor) over **MCP via Streamable HTTP**, plus a
|
||||||
|
web UI and REST API. `CONTEXT.MD` can be synced to a repo as `AGENTS.md` with symlinks
|
||||||
|
(`CLAUDE.md`, `.cursorrules`, `CODEX.md`). The full user/operator guide is `README.md`; this file
|
||||||
|
is the orientation for editing the code.
|
||||||
|
|
||||||
|
All source lives in `app/src/ctxd/`. Run commands from `app/`.
|
||||||
|
|
||||||
|
## Architecture (the parts that need multiple files to grasp)
|
||||||
|
|
||||||
|
- **One ASGI app multiplexes three protocols.** `server.py` → `CombinedApp` (~line 1455) dispatches
|
||||||
|
every request to: REST + Web UI, the OAuth 2.0 authorization server, and the MCP endpoints. There
|
||||||
|
is no separate service per surface — "MCP is 502" and "web UI is 502" are the same process being
|
||||||
|
down. `serve_sync(cfg)` boots it under uvicorn.
|
||||||
|
- **Two MCP transports.** Streamable HTTP is the public one (paths in `MCP_STREAMABLE_PATHS` =
|
||||||
|
`/mcp`, `/readonly/mcp`, `/oauth/mcp`, all served by `CombinedApp`). `mcp_stdio.py` is a *separate*
|
||||||
|
stdin/stdout JSON-RPC server Hermes spawns directly — keep tool definitions in sync between the two
|
||||||
|
when adding tools.
|
||||||
|
- **MCP tools are built in two functions, gated by scope.** `make_mcp_server(cfg, readonly, oauth_scoped)`
|
||||||
|
exposes read tools (+ everything when API-key/LAN); `make_write_mcp_server(cfg)` exposes the write
|
||||||
|
set (`update_file`, `set_project_tags`, `sync_to_project`). OAuth `ctxd.read` vs `ctxd.write` scopes
|
||||||
|
decide what a token sees. To add an MCP tool, edit the `list_tools`/`call_tool` handlers inside these
|
||||||
|
builders.
|
||||||
|
- **`db.py` is plain functions over a `conn`, no ORM, dual-backend.** PostgreSQL is primary; SQLite
|
||||||
|
(`$CTXD_HOME/ctxd.db`) is the fallback when `DATABASE_URL` is empty (`cfg.use_postgres`). The same
|
||||||
|
query strings run on both via `_is_pg(conn)` and the placeholder helper `_ph(conn, n)`. **`schema.sql`
|
||||||
|
(PG) and `schema_sqlite.sql` must be kept in lockstep** — a table added to one must be added to the other.
|
||||||
|
- **OAuth state and web sessions are file-based JSON, not in the DB.** `OAuthStore` (`oauth_state.json`)
|
||||||
|
and `WebSessionStore` (`web_sessions.json`) live in `$CTXD_HOME` (`/data` in the container). They
|
||||||
|
survive DB swaps but are *not* covered by `pg_dump` — back them up separately.
|
||||||
|
- **Metadata headers are computed, never stored.** `build_metadata_header()` prepends the header on
|
||||||
|
read; `strip_metadata_header()` removes it before persisting. Don't store headers in the DB.
|
||||||
|
- **File paths are normalized** to uppercase with `.MD` (`normalize_file_path`). `CONTEXT.MD` is the
|
||||||
|
minimum file and cannot be deleted; in the `ctxd-docs` project, `CONTEXT.MD` and `LLM-CLIENT.MD` are
|
||||||
|
also locked against update/delete.
|
||||||
|
- **Writes are version-checked.** `file_update`/`context_update` take `base_version`; a mismatch is a
|
||||||
|
`409 conflict`. Every mutating op also writes an append-only `audit_log` row and may take a rotating
|
||||||
|
snapshot (`SNAPSHOT_MIN_KEEP`/`MAX_KEEP`).
|
||||||
|
|
||||||
|
### Latent / not wired up
|
||||||
|
|
||||||
|
`schema.sql` and `db.py` define a full collaborative-review flow — `user_workspaces` → `workspace_files`
|
||||||
|
→ `change_requests` → `reviews`, with `workspace_fork`, `workspace_submit`, `change_request_approve`,
|
||||||
|
etc. As of now this is **DB-layer only**: it is exposed through neither `server.py` (REST/MCP) nor
|
||||||
|
`cli.py`. Treat it as scaffolding, not a live feature.
|
||||||
|
|
||||||
|
## Entry points
|
||||||
|
|
||||||
|
`pyproject.toml` defines two console scripts (and `python -m ctxd` dispatches the same way via
|
||||||
|
`__main__.py`, choosing CLI vs daemon by the first arg):
|
||||||
|
|
||||||
|
- `dossier <command>` → CLI (`cli.py`, `cli_entry`). In production these run inside the container:
|
||||||
|
`docker exec ctxd dossier <command>`.
|
||||||
|
- `ctxd` → starts the daemon (`daemon_entry` → `serve_sync`).
|
||||||
|
|
||||||
|
## Build / run / deploy
|
||||||
|
|
||||||
|
Everything runs from `app/`. There is **no test runner or linter configured** — `scripts/test_*.py`
|
||||||
|
are standalone MCP smoke scripts, not a suite.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Local dev (SQLite, no Docker)
|
||||||
|
cd app
|
||||||
|
pip install -e ".[mcp]"
|
||||||
|
export CTXD_HOME=./dev-data
|
||||||
|
python -m ctxd init # initialize DB
|
||||||
|
python -m ctxd # serve → http://localhost:9091
|
||||||
|
|
||||||
|
# Production deploy (Docker + bundled PostgreSQL 16)
|
||||||
|
./scripts/deploy.sh # builds ctxd, starts postgres, waits for healthy, recreates ctxd, smoke-tests /status
|
||||||
|
|
||||||
|
# MCP smoke tests against a running server
|
||||||
|
python scripts/test_unified_mcp.py
|
||||||
|
python scripts/test_write_mcp.py
|
||||||
|
|
||||||
|
# Health
|
||||||
|
curl http://localhost:9091/status # → {"status":"ok", ...}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Deploy gotcha (the #1 source of public 502s):** `ctxd` depends on the `postgres` container being up
|
||||||
|
first. Always use `./scripts/deploy.sh` (or `docker compose up -d`, which starts both). **Never**
|
||||||
|
`docker compose up -d --no-deps ctxd` and **never** `docker restart ctxd` after a code change — the
|
||||||
|
former crash-loops without the DB, the latter runs the old image. After editing code you must rebuild
|
||||||
|
(`docker compose build ctxd`) before recreating.
|
||||||
|
|
||||||
|
## Config
|
||||||
|
|
||||||
|
All config is env-driven through `config.py` (`CtxConfig`), precedence **env var > `data/ctxd.yaml` >
|
||||||
|
default**. Key switches: `DATABASE_URL` (empty ⇒ SQLite fallback), `CTXD_HOME` (`/data` in container),
|
||||||
|
`CTXD_AUTH_ENABLED` + `CTXD_API_KEY` (shared key for LAN/Hermes = full MCP tools), `OAUTH_ENABLED` +
|
||||||
|
`OAUTH_ISSUER` (must be set together or the app won't start cleanly). The full table is in `README.md`.
|
||||||
|
|
||||||
|
## Conventions when changing code
|
||||||
|
|
||||||
|
- Adding a context-data operation: write the `conn`-taking function in `db.py` (handle both backends
|
||||||
|
via `_is_pg`/`_ph`), then expose it in the relevant surface(s) — `server.py` for REST/MCP, `cli.py`
|
||||||
|
for CLI, and `mcp_stdio.py` if Hermes needs it.
|
||||||
|
- Adding a column/table: edit **both** `schema.sql` and `schema_sqlite.sql`; for live PG instances add
|
||||||
|
a guarded migration (see `_migrate_pg` in `db.py` and the standalone `migrate_*.py` scripts).
|
||||||
|
- Don't persist metadata headers; don't bypass `base_version` checking on writes; keep every mutation
|
||||||
|
audit-logged.
|
||||||
@@ -0,0 +1,251 @@
|
|||||||
|
# CTXD — Context Dossier
|
||||||
|
# LLM Installation & Deployment Guide
|
||||||
|
# Feed this file to an LLM to assist with setup and deployment.
|
||||||
|
|
||||||
|
## What is CTXD?
|
||||||
|
|
||||||
|
CTXD (Context Dossier) is a single source of truth for multi-harness project context. It serves one canonical AGENTS.md per project to Claude, Hermes, Codex, Cursor, and any OAuth-capable MCP client via Streamable HTTP. It runs as two Docker containers: a PostgreSQL 16 database and a Python ASGI daemon.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- Docker 24+ and Docker Compose v2+
|
||||||
|
- A reverse proxy with TLS (Traefik recommended; Caddy or nginx also work)
|
||||||
|
- Linux host with ~1GB free RAM
|
||||||
|
- (Optional) Existing PostgreSQL 14+ instance if not using the bundled one
|
||||||
|
|
||||||
|
## Quick Deployment
|
||||||
|
|
||||||
|
1. Clone the repository:
|
||||||
|
git clone ssh://git@code.cubecraftlabs.com:2288/CubeCraft-Creations/CTXD.git
|
||||||
|
cd CTXD/app
|
||||||
|
|
||||||
|
2. Copy the environment template:
|
||||||
|
cp .env.example .env
|
||||||
|
|
||||||
|
3. Generate secrets and fill in .env:
|
||||||
|
python3 -c "import secrets; print(secrets.token_urlsafe(32))" # Use for CTXD_API_KEY
|
||||||
|
python3 -c "import secrets; print(secrets.token_urlsafe(32))" # Use for OAUTH_APPROVAL_KEY
|
||||||
|
|
||||||
|
4. Edit .env with your values. Required variables:
|
||||||
|
- DATABASE_URL — PostgreSQL connection string
|
||||||
|
- POSTGRES_PASSWORD — Password for the bundled PG container
|
||||||
|
- CTXD_API_KEY — Shared key for internal MCP and HTTP auth
|
||||||
|
- OAUTH_ENABLED=true
|
||||||
|
- OAUTH_ISSUER=https://your-domain.com
|
||||||
|
- OAUTH_APPROVAL_KEY — Key for approving OAuth authorizations
|
||||||
|
|
||||||
|
5. Build and start:
|
||||||
|
docker compose up -d
|
||||||
|
|
||||||
|
6. Wait for PostgreSQL to be healthy and the daemon to start:
|
||||||
|
docker compose ps
|
||||||
|
# Both ctxd-postgres and dossier should show "healthy" / "running"
|
||||||
|
|
||||||
|
7. Set the admin password:
|
||||||
|
docker exec ctxd dossier user-set-password admin -p "your-admin-password"
|
||||||
|
|
||||||
|
8. Verify:
|
||||||
|
curl http://localhost:9091/status
|
||||||
|
# Expected: {"status": "ok", "db": "/data/ctxd.db"}
|
||||||
|
|
||||||
|
9. Access the Web UI:
|
||||||
|
http://<server-ip>:9091/
|
||||||
|
Sign in with admin / your-admin-password
|
||||||
|
|
||||||
|
## Environment Variables Reference
|
||||||
|
|
||||||
|
All config is driven by environment variables in .env. Precedence: env var > ctxd.yaml > defaults.
|
||||||
|
|
||||||
|
### Database
|
||||||
|
- DATABASE_URL — PostgreSQL connection string. If empty, falls back to SQLite.
|
||||||
|
- POSTGRES_USER — PostgreSQL user for bundled container (default: ctxd)
|
||||||
|
- POSTGRES_PASSWORD — PostgreSQL password for bundled container
|
||||||
|
- POSTGRES_DB — PostgreSQL database name (default: ctxd)
|
||||||
|
|
||||||
|
### Server
|
||||||
|
- CTXD_HOST — Bind address (default: 0.0.0.0)
|
||||||
|
- CTXD_PORT — Listen port (default: 9091)
|
||||||
|
- CTXD_HOME — Data directory inside container (default: /data)
|
||||||
|
- LOG_LEVEL — Uvicorn log level: debug, info, warning, error (default: info)
|
||||||
|
|
||||||
|
### Auth
|
||||||
|
- CTXD_AUTH_ENABLED — Enable authentication globally (default: false)
|
||||||
|
- CTXD_API_KEY — Shared API key for Hermes/internal MCP and HTTP auth
|
||||||
|
- CTXD_EXTERNAL_READONLY_KEY — Legacy ?key= on read-only MCP (migration only)
|
||||||
|
|
||||||
|
### OAuth
|
||||||
|
- OAUTH_ENABLED — Enable OAuth authorization server (default: false)
|
||||||
|
- OAUTH_ISSUER — Public URL used in OAuth discovery metadata
|
||||||
|
- OAUTH_APPROVAL_KEY — Fallback key for /oauth/authorize approval
|
||||||
|
- OAUTH_APPROVAL_USER_ID — User ID for attributing approvals (default: admin)
|
||||||
|
- OAUTH_ACCESS_TOKEN_TTL — Access token lifetime in seconds (default: 3600)
|
||||||
|
- OAUTH_REFRESH_TOKEN_TTL — Refresh token lifetime in seconds (default: 2592000)
|
||||||
|
|
||||||
|
### Web Sessions
|
||||||
|
- WEB_SESSION_TTL — Session cookie lifetime in seconds (default: 604800)
|
||||||
|
|
||||||
|
### Snapshots
|
||||||
|
- SNAPSHOT_MIN_KEEP — Minimum snapshots per project (default: 5)
|
||||||
|
- SNAPSHOT_MAX_KEEP — Maximum snapshots before rotation (default: 25)
|
||||||
|
|
||||||
|
## MCP Endpoints
|
||||||
|
|
||||||
|
CTXD exposes three MCP surfaces via Streamable HTTP:
|
||||||
|
|
||||||
|
1. /readonly/mcp — Read-only tools (OAuth ctxd.read or API key)
|
||||||
|
Tools: list_projects, get_project_context, search_context, get_project_tags, list_files, get_file, get_client_guide
|
||||||
|
|
||||||
|
2. /write/mcp — Write tools (OAuth ctxd.write only)
|
||||||
|
Tools: update_file, set_project_tags, sync_to_project, get_client_guide
|
||||||
|
|
||||||
|
3. /mcp — Internal full MCP (shared API key only, not exposed publicly)
|
||||||
|
All read + write tools plus auto_generate_tags and get_user_profile
|
||||||
|
|
||||||
|
## Connecting LLM Clients
|
||||||
|
|
||||||
|
### Claude Desktop
|
||||||
|
Connector URL: https://your-domain.com/readonly/mcp
|
||||||
|
Claude auto-discovers OAuth metadata and registers via DCR.
|
||||||
|
For write access, use: https://your-domain.com/write/mcp
|
||||||
|
|
||||||
|
### Hermes Agent
|
||||||
|
Edit ~/.hermes/config.yaml:
|
||||||
|
mcp_servers:
|
||||||
|
dossier:
|
||||||
|
url: http://<server-ip>:9091/mcp
|
||||||
|
timeout: 30
|
||||||
|
headers:
|
||||||
|
Authorization: "Bearer your-api-key"
|
||||||
|
Restart Hermes after changing config.
|
||||||
|
|
||||||
|
### Other MCP Clients
|
||||||
|
1. Register an OAuth client: POST /oauth/register with your redirect URI
|
||||||
|
2. Authorize: GET /oauth/authorize (admin must approve)
|
||||||
|
3. Exchange code: POST /oauth/token
|
||||||
|
4. Connect to /readonly/mcp or /write/mcp with Bearer token
|
||||||
|
|
||||||
|
## Traefik Configuration
|
||||||
|
|
||||||
|
Route everything except the internal MCP endpoint:
|
||||||
|
|
||||||
|
rule: Host(`ctxd.yourdomain.com`) && !Path(`/mcp`)
|
||||||
|
|
||||||
|
This exposes: landing page, dashboard, REST API, OAuth, read-only MCP, write MCP.
|
||||||
|
Only /mcp (internal, API key) is blocked.
|
||||||
|
|
||||||
|
## CLI Commands
|
||||||
|
|
||||||
|
All commands run inside the container:
|
||||||
|
docker exec ctxd dossier <command>
|
||||||
|
|
||||||
|
Commands:
|
||||||
|
init Initialize database
|
||||||
|
project-create <id> [--display-name N] Create a project
|
||||||
|
project-list List projects
|
||||||
|
read <project_id> Print context to stdout
|
||||||
|
edit <project_id> Open in $EDITOR
|
||||||
|
file-list <project_id> List context files
|
||||||
|
file-read <project_id> <file_path> Read a single file
|
||||||
|
sync <project_id> [path] Sync AGENTS.md to project root
|
||||||
|
search "query" Full-text search
|
||||||
|
audit [--limit N] Show audit log
|
||||||
|
user-list List users
|
||||||
|
user-create <id> --display-name "Name" [--password "pw"]
|
||||||
|
user-set-password <id> -p "password"
|
||||||
|
oauth-client-create [-n "Name"] [--redirect-uri URI]
|
||||||
|
oauth-client-list List OAuth clients
|
||||||
|
oauth-client-revoke <client_id> Revoke client and invalidate tokens
|
||||||
|
import-vault <path> Import from Obsidian vault
|
||||||
|
|
||||||
|
## Admin UI
|
||||||
|
|
||||||
|
Sign in as admin, click "admin" in the masthead. Three tabs:
|
||||||
|
1. oauth clients — Create and revoke OAuth clients
|
||||||
|
2. users — List, create, edit, activate, inactivate, delete users
|
||||||
|
3. projects — List and delete projects (typed-name confirmation required)
|
||||||
|
|
||||||
|
## Project Files
|
||||||
|
|
||||||
|
Each project has multiple context files:
|
||||||
|
- CONTEXT.MD — Canonical overview (synced as AGENTS.md, cannot be deleted)
|
||||||
|
- DECISIONS.MD — Architecture decisions and rationale
|
||||||
|
- RUNBOOKS.MD — Deploy, troubleshoot, operate procedures
|
||||||
|
- PROMPTS.MD — Project-specific prompts for different harnesses
|
||||||
|
- GLOSSARY.MD — Project-specific terms and acronyms
|
||||||
|
|
||||||
|
The compiled view (get_project_context) concatenates all files with metadata header.
|
||||||
|
|
||||||
|
## Locked Files
|
||||||
|
|
||||||
|
- CONTEXT.MD — Cannot be deleted from any project (minimum required file)
|
||||||
|
- CONTEXT.MD and LLM-CLIENT.MD — Fully locked in the ctxd-docs project (cannot update or delete)
|
||||||
|
|
||||||
|
## Backups
|
||||||
|
|
||||||
|
PostgreSQL backup:
|
||||||
|
docker exec ctxd-postgres pg_dump -U ctxd ctxd > backup.sql
|
||||||
|
|
||||||
|
PostgreSQL restore:
|
||||||
|
cat backup.sql | docker exec -i ctxd-postgres psql -U ctxd ctxd
|
||||||
|
|
||||||
|
Snapshots are automatic (before each context update, rotated min 5 / max 25 per project).
|
||||||
|
|
||||||
|
## Migrating from SQLite to PostgreSQL
|
||||||
|
|
||||||
|
If you started with SQLite:
|
||||||
|
1. Start PostgreSQL: docker compose up -d postgres
|
||||||
|
2. Run migration: docker exec ctxd python3 -m ctxd.migrate_sqlite_to_pg
|
||||||
|
3. Set DATABASE_URL in .env and restart: docker compose up -d dossier
|
||||||
|
|
||||||
|
## Using an External PostgreSQL
|
||||||
|
|
||||||
|
1. Create database and user on your external PG instance
|
||||||
|
2. Set DATABASE_URL in .env to point to it
|
||||||
|
3. Start only the app: docker compose up -d dossier
|
||||||
|
(or scale postgres to 0: docker compose up -d --scale postgres=0)
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
Login fails: Reset password with "docker exec ctxd dossier user-set-password admin -p newpass"
|
||||||
|
Use double quotes if password contains single quotes. Use single quotes for $ ` ! characters.
|
||||||
|
|
||||||
|
MCP 404: Traefik is not routing /readonly/mcp. Check router rule includes it.
|
||||||
|
MCP 401: Auth is working. Check OAuth token scope and expiry.
|
||||||
|
|
||||||
|
PostgreSQL connection fails: Check DATABASE_URL password matches POSTGRES_PASSWORD.
|
||||||
|
If PG data volume was initialized with a different password, reset it:
|
||||||
|
docker exec ctxd-postgres psql -U ctxd -c "ALTER USER ctxd PASSWORD 'newpass'"
|
||||||
|
Then update .env.
|
||||||
|
|
||||||
|
Container keeps restarting: Check "docker logs ctxd --tail 30".
|
||||||
|
Common causes: wrong PG password, OAUTH_ENABLED=true but OAUTH_ISSUER empty, missing CTXD_API_KEY.
|
||||||
|
|
||||||
|
## File Structure
|
||||||
|
|
||||||
|
CTXD/
|
||||||
|
├── .env Production environment (gitignored, contains secrets)
|
||||||
|
├── .env.example Template with all documented variables
|
||||||
|
├── .gitignore
|
||||||
|
├── README.md Full documentation
|
||||||
|
├── SKILL.md LLM client guide (canonical source for LLM-CLIENT.MD)
|
||||||
|
├── data/ Runtime data (gitignored)
|
||||||
|
│ ├── ctxd.yaml Fallback config (env vars take precedence)
|
||||||
|
│ ├── oauth_state.json OAuth clients, codes, tokens
|
||||||
|
│ ├── web_sessions.json Web UI sessions
|
||||||
|
│ ├── snapshots/ Point-in-time context backups
|
||||||
|
│ └── pg/ PostgreSQL data volume
|
||||||
|
└── app/ Application source
|
||||||
|
├── docker-compose.yml
|
||||||
|
├── Dockerfile
|
||||||
|
├── pyproject.toml
|
||||||
|
└── src/ctxd/
|
||||||
|
├── config.py Env-driven config
|
||||||
|
├── db.py Database layer (PostgreSQL + SQLite)
|
||||||
|
├── schema.sql PostgreSQL schema
|
||||||
|
├── schema_sqlite.sql SQLite schema fallback
|
||||||
|
├── server.py ASGI: HTTP + MCP + OAuth
|
||||||
|
├── cli.py CLI commands
|
||||||
|
├── ui.html Web UI dashboard
|
||||||
|
├── landing.html Public landing page
|
||||||
|
├── auth_password.py PBKDF2 password hashing
|
||||||
|
└── migrate_sqlite_to_pg.py Migration script
|
||||||
@@ -22,17 +22,17 @@ CTXD provides:
|
|||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
```
|
```
|
||||||
Context Dossier (container: dossier, 0.0.0.0:9091)
|
Context Dossier (container: ctxd, 0.0.0.0:9091)
|
||||||
├── PostgreSQL 16 (container: ctxd-postgres) # Primary DB
|
├── PostgreSQL 16 (container: ctxd-postgres) # Primary DB
|
||||||
├── /data # Config, OAuth state, web sessions
|
├── /data # Config, OAuth state, web sessions
|
||||||
│ ├── ctxd.yaml # Fallback config (env vars take precedence)
|
│ ├── ctxd.yaml # Fallback config (env vars take precedence)
|
||||||
│ ├── oauth_state.json # OAuth clients, codes, tokens
|
│ ├── oauth_state.json # OAuth clients, codes, tokens
|
||||||
│ ├── web_sessions.json # Per-user web UI sessions
|
│ ├── web_sessions.json # Per-user web UI sessions
|
||||||
│ └── snapshots/ # Point-in-time context backups
|
│ └── snapshots/ # Point-in-time context backups
|
||||||
├── Streamable HTTP MCP endpoints:
|
├── Streamable HTTP MCP:
|
||||||
│ ├── /readonly/mcp (OAuth ctxd.read) # Read-only tools
|
│ ├── /mcp (OAuth ctxd.read + ctxd.write; API key on LAN = full tools)
|
||||||
│ ├── /write/mcp (OAuth ctxd.write) # Write tools
|
│ ├── /readonly/mcp (alias → same OAuth behavior)
|
||||||
│ └── /mcp (shared API key) # Internal full MCP
|
│ └── /oauth/mcp (alias)
|
||||||
├── OAuth Authorization Server:
|
├── OAuth Authorization Server:
|
||||||
│ ├── /.well-known/oauth-authorization-server # Discovery
|
│ ├── /.well-known/oauth-authorization-server # Discovery
|
||||||
│ ├── /.well-known/oauth-protected-resource # Resource metadata
|
│ ├── /.well-known/oauth-protected-resource # Resource metadata
|
||||||
@@ -84,12 +84,19 @@ OAUTH_APPROVAL_KEY=your-approval-key # Generate: python3 -c "import secrets;
|
|||||||
### 2. Build and start
|
### 2. Build and start
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# Recommended: build + postgres + recreate ctxd (avoids 502 when PG was never started)
|
||||||
|
chmod +x scripts/deploy.sh
|
||||||
|
./scripts/deploy.sh
|
||||||
|
|
||||||
|
# Or manual (always include postgres when DATABASE_URL uses host "postgres"):
|
||||||
docker compose up -d
|
docker compose up -d
|
||||||
```
|
```
|
||||||
|
|
||||||
This starts:
|
This starts:
|
||||||
- `ctxd-postgres` — PostgreSQL 16 (Alpine)
|
- `ctxd-postgres` — PostgreSQL 16 (Alpine)
|
||||||
- `dossier` — CTXD daemon (web UI + MCP + OAuth + REST API)
|
- `ctxd` — CTXD daemon (web UI + MCP + OAuth + REST API)
|
||||||
|
|
||||||
|
**After code changes**, use `./scripts/deploy.sh` or `docker compose up -d` — not `docker restart ctxd` alone (old image) and **not** `docker compose up -d --no-deps ctxd` (skips postgres → crash loop → public **502**).
|
||||||
|
|
||||||
### 3. Verify
|
### 3. Verify
|
||||||
|
|
||||||
@@ -105,7 +112,7 @@ curl http://localhost:9091/projects -H "Authorization: Bearer your-api-key"
|
|||||||
### 4. Set admin password
|
### 4. Set admin password
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker exec dossier dossier user-set-password admin -p "your-admin-password"
|
docker exec ctxd dossier user-set-password admin -p "your-admin-password"
|
||||||
```
|
```
|
||||||
|
|
||||||
> **Shell quoting:** Use double quotes (`"`) if the password contains single quotes (`'`). Use single quotes (`'`) if it contains `$`, backticks, or `!`. The failure mode is shell expansion, not PBKDF2.
|
> **Shell quoting:** Use double quotes (`"`) if the password contains single quotes (`'`). Use single quotes (`'`) if it contains `$`, backticks, or `!`. The failure mode is shell expansion, not PBKDF2.
|
||||||
@@ -148,6 +155,9 @@ All config is driven by environment variables. A `ctxd.yaml` file in `/data` can
|
|||||||
| `OAUTH_APPROVAL_USER_ID` | `admin` | Which user ID to attribute OAuth approvals to |
|
| `OAUTH_APPROVAL_USER_ID` | `admin` | Which user ID to attribute OAuth approvals to |
|
||||||
| `OAUTH_ACCESS_TOKEN_TTL` | `3600` | Access token lifetime in seconds |
|
| `OAUTH_ACCESS_TOKEN_TTL` | `3600` | Access token lifetime in seconds |
|
||||||
| `OAUTH_REFRESH_TOKEN_TTL` | `2592000` | Refresh token lifetime in seconds (30 days) |
|
| `OAUTH_REFRESH_TOKEN_TTL` | `2592000` | Refresh token lifetime in seconds (30 days) |
|
||||||
|
| **PostgreSQL (container)** | | |
|
||||||
|
| `CTXD_PG_WAIT_SECONDS` | `120` | Entrypoint: max wait for DB before exit (when `DATABASE_URL` set) |
|
||||||
|
| `CTXD_PG_WAIT_INTERVAL` | `2` | Seconds between connection attempts |
|
||||||
| **Web Sessions** | | |
|
| **Web Sessions** | | |
|
||||||
| `WEB_SESSION_TTL` | `604800` | Session cookie lifetime in seconds (7 days) |
|
| `WEB_SESSION_TTL` | `604800` | Session cookie lifetime in seconds (7 days) |
|
||||||
| **Snapshots** | | |
|
| **Snapshots** | | |
|
||||||
@@ -160,13 +170,9 @@ To use an external PostgreSQL instead of the bundled container:
|
|||||||
|
|
||||||
1. Create a database and user on your external PG instance
|
1. Create a database and user on your external PG instance
|
||||||
2. Set `DATABASE_URL` in `.env` to point to it
|
2. Set `DATABASE_URL` in `.env` to point to it
|
||||||
3. Start only the dossier service: `docker compose up -d dossier`
|
3. Start only the app (no bundled postgres): `docker compose up -d --scale postgres=0 ctxd`
|
||||||
|
|
||||||
The bundled `ctxd-postgres` container will start but is unused. To prevent it from starting, run it with zero scale:
|
Ensure `DATABASE_URL` points at your external host (not `postgres`). The entrypoint skips the compose-network wait when the URL is reachable.
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose up -d --scale postgres=0
|
|
||||||
```
|
|
||||||
|
|
||||||
### Fallback to SQLite
|
### Fallback to SQLite
|
||||||
|
|
||||||
@@ -174,22 +180,73 @@ If `DATABASE_URL` is empty or not set, CTXD falls back to SQLite at `$CTXD_HOME/
|
|||||||
|
|
||||||
## MCP Surfaces
|
## MCP Surfaces
|
||||||
|
|
||||||
CTXD exposes three MCP endpoints via Streamable HTTP:
|
CTXD exposes MCP via Streamable HTTP on **`/mcp`** (single public connector):
|
||||||
|
|
||||||
| Endpoint | Auth | Scope | Tools |
|
| Endpoint | Auth | Scope | Tools |
|
||||||
|----------|------|-------|-------|
|
|----------|------|-------|-------|
|
||||||
| `/readonly/mcp` | OAuth bearer or API key | `ctxd.read` | `list_projects`, `get_project_context`, `search_context`, `get_project_tags`, `list_files`, `get_file`, `get_client_guide` |
|
| `/mcp` | OAuth bearer | `ctxd.read` / `ctxd.write` | Scope-gated read + write tools |
|
||||||
| `/write/mcp` | OAuth bearer | `ctxd.write` | `update_file`, `set_project_tags`, `sync_to_project`, `get_client_guide` |
|
| `/mcp` | Shared API key (LAN/Hermes) | *(full)* | All tools including `get_user_profile`, `auto_generate_tags` |
|
||||||
| `/mcp` | Shared API key | *(full)* | All read + write tools + `sync_to_project`, `auto_generate_tags`, `get_user_profile` |
|
| `/readonly/mcp`, `/oauth/mcp` | OAuth (aliases) | same as `/mcp` | Backward-compatible URLs |
|
||||||
|
|
||||||
### Connecting an LLM Client
|
### Connecting an LLM Client
|
||||||
|
|
||||||
**Claude Desktop / Claude Web:**
|
**Claude Desktop / Claude Web:**
|
||||||
```
|
```
|
||||||
Connector URL: https://ctxd.yourdomain.com/readonly/mcp
|
Connector URL: https://ctxd.yourdomain.com/mcp
|
||||||
```
|
```
|
||||||
Claude auto-discovers OAuth metadata and registers via DCR. Request `scope=ctxd.read ctxd.write` for write access.
|
Claude auto-discovers OAuth metadata and registers via DCR. Request `scope=ctxd.read ctxd.write` for write access.
|
||||||
|
|
||||||
|
**ChatGPT (MCP connector — recommended; no Custom GPT required):**
|
||||||
|
|
||||||
|
ChatGPT can attach a **remote MCP server** in **Developer mode** (Plus/Pro and higher tiers for custom connectors). Use the same public URL as Claude — OAuth on `/mcp`, not REST Actions or a shared API key.
|
||||||
|
|
||||||
|
1. **Server** — deploy current CTXD and expose the public host (Traefik must route `/mcp` and `/oauth/*`; do not block `/mcp`):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd app
|
||||||
|
./scripts/deploy.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Smoke: `curl -sS https://ctxd.yourdomain.com/.well-known/oauth-authorization-server | head`
|
||||||
|
|
||||||
|
2. **ChatGPT** — Settings → **Connectors** → enable **Developer mode** → **Add connector** (MCP / custom remote) → **Server URL:**
|
||||||
|
|
||||||
|
```
|
||||||
|
https://ctxd.yourdomain.com/mcp
|
||||||
|
```
|
||||||
|
|
||||||
|
Start OAuth and copy ChatGPT’s **callback / redirect URL** exactly.
|
||||||
|
|
||||||
|
3. **CTXD admin** — Web UI → **admin** → **oauth clients**:
|
||||||
|
|
||||||
|
- After DCR, the client may appear automatically; otherwise **create client** with ChatGPT’s redirect URI.
|
||||||
|
- Set **allowed scopes** to **`ctxd.read`** and **`ctxd.write`** (create form or **scopes** → save on an existing row).
|
||||||
|
|
||||||
|
4. **Authorize** — complete the browser approval (sign in as CTXD admin in that browser, or use the OAuth approval key). Tokens are capped by the client’s allowed scopes.
|
||||||
|
|
||||||
|
5. **Chat** — enable the CTXD connector for the conversation. Tools include `list_projects`, `get_project_context`, `search_context`, `get_file`, `update_file`, etc. Call **`get_client_guide`** first in a new session.
|
||||||
|
|
||||||
|
**Optional CLI pre-register** (if ChatGPT asks for client credentials before DCR):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker exec ctxd dossier oauth-client-create -n "ChatGPT MCP" \
|
||||||
|
--redirect-uri 'PASTE_CHATGPT_CALLBACK_URL' \
|
||||||
|
--scope "ctxd.read ctxd.write"
|
||||||
|
```
|
||||||
|
|
||||||
|
| If connect fails | Check |
|
||||||
|
|------------------|--------|
|
||||||
|
| **“does not implement OAuth”** | Public host must return **200** JSON (not 502/404) for `/.well-known/oauth-protected-resource/mcp` and `/.well-known/oauth-authorization-server`. Traefik must proxy **all** paths to CTXD `:9091`. Set `OAUTH_ENABLED=true` and `OAUTH_ISSUER=https://ctxd.cubecraftlabs.com`. |
|
||||||
|
| 502 on well-known URLs | Backend down or wrong Traefik service — fix routing before OAuth can work |
|
||||||
|
| 404 on `/mcp` | Router not forwarding `/mcp` to CTXD (often a stale `!Path(/mcp)` rule) |
|
||||||
|
| 401 on `/mcp` | Re-authorize; access token may have expired |
|
||||||
|
| Redirect mismatch | Callback URL in admin must match ChatGPT’s string exactly |
|
||||||
|
| `invalid_request` on authorize | CTXD requires PKCE **S256** (`code_challenge` + `code_challenge_method=S256`) |
|
||||||
|
| Stale routes / old code | `./scripts/deploy.sh` or `docker compose up -d --force-recreate ctxd` after postgres is healthy (not `restart` alone; never `--no-deps` unless PG is already up) |
|
||||||
|
| No write tools | Client scopes include `ctxd.write`; re-authorize after changing scopes |
|
||||||
|
|
||||||
|
**Not required for ChatGPT MCP:** Custom GPT, OpenAPI Actions, or `CTXD_API_KEY` in ChatGPT (public path is OAuth). Hermes continues to use LAN `http://<server-ip>:9091/mcp` with the API key.
|
||||||
|
|
||||||
**Hermes Agent:**
|
**Hermes Agent:**
|
||||||
```yaml
|
```yaml
|
||||||
# ~/.hermes/config.yaml
|
# ~/.hermes/config.yaml
|
||||||
@@ -203,12 +260,12 @@ mcp_servers:
|
|||||||
|
|
||||||
**Other MCP clients (Codex, Cursor, etc.):**
|
**Other MCP clients (Codex, Cursor, etc.):**
|
||||||
- Register an OAuth client via `POST /oauth/register` with your redirect URI
|
- Register an OAuth client via `POST /oauth/register` with your redirect URI
|
||||||
- Connect to `/readonly/mcp` (read) or `/write/mcp` (read + write)
|
- Connect to **`/mcp`** with `scope=ctxd.read ctxd.write`
|
||||||
- Use the access token as `Authorization: Bearer <token>`
|
- Use the access token as `Authorization: Bearer <token>`
|
||||||
|
|
||||||
### MCP Tool Reference
|
### MCP Tool Reference
|
||||||
|
|
||||||
#### Read-only tools (`/readonly/mcp` and `/write/mcp`)
|
#### Read tools (require `ctxd.read` on `/mcp`)
|
||||||
|
|
||||||
| Tool | Args | Returns |
|
| Tool | Args | Returns |
|
||||||
|------|------|---------|
|
|------|------|---------|
|
||||||
@@ -220,7 +277,7 @@ mcp_servers:
|
|||||||
| `list_files` | `project_id` | All context files in a project |
|
| `list_files` | `project_id` | All context files in a project |
|
||||||
| `get_file` | `project_id`, `file_path` | Single file with metadata header |
|
| `get_file` | `project_id`, `file_path` | Single file with metadata header |
|
||||||
|
|
||||||
#### Write tools (`/write/mcp` only)
|
#### Write tools (require `ctxd.write` on `/mcp`)
|
||||||
|
|
||||||
| Tool | Args | Returns |
|
| Tool | Args | Returns |
|
||||||
|------|------|---------|
|
|------|------|---------|
|
||||||
@@ -252,6 +309,7 @@ Request both: `scope=ctxd.read ctxd.write`
|
|||||||
| Platform | Redirect URI |
|
| Platform | Redirect URI |
|
||||||
|----------|-------------|
|
|----------|-------------|
|
||||||
| Claude Desktop | `https://claude.ai/api/mcp/auth_callback` |
|
| Claude Desktop | `https://claude.ai/api/mcp/auth_callback` |
|
||||||
|
| ChatGPT (MCP) | Paste callback URL from ChatGPT connector OAuth UI (per connector) |
|
||||||
| Claude Code | `http://localhost:5555/oauth/callback` |
|
| Claude Code | `http://localhost:5555/oauth/callback` |
|
||||||
| Codex CLI | `http://localhost:7777/oauth/callback` |
|
| Codex CLI | `http://localhost:7777/oauth/callback` |
|
||||||
| Custom | Your app's documented OAuth callback |
|
| Custom | Your app's documented OAuth callback |
|
||||||
@@ -263,13 +321,13 @@ Request both: `scope=ctxd.read ctxd.write`
|
|||||||
**Via CLI:**
|
**Via CLI:**
|
||||||
```bash
|
```bash
|
||||||
# Create
|
# Create
|
||||||
docker exec dossier dossier oauth-client-create -n "Claude Desktop" --redirect-uri https://claude.ai/api/mcp/auth_callback
|
docker exec ctxd dossier oauth-client-create -n "Claude Desktop" --redirect-uri https://claude.ai/api/mcp/auth_callback
|
||||||
|
|
||||||
# List
|
# List
|
||||||
docker exec dossier dossier oauth-client-list
|
docker exec ctxd dossier oauth-client-list
|
||||||
|
|
||||||
# Revoke (invalidates all tokens for that client)
|
# Revoke (invalidates all tokens for that client)
|
||||||
docker exec dossier dossier oauth-client-revoke ctxd_xxxxxxxx
|
docker exec ctxd dossier oauth-client-revoke ctxd_xxxxxxxx
|
||||||
```
|
```
|
||||||
|
|
||||||
**Via API (admin session or API key):**
|
**Via API (admin session or API key):**
|
||||||
@@ -291,12 +349,10 @@ curl -X DELETE http://localhost:9091/oauth/clients/ctxd_xxxxxxxx -H "Authorizati
|
|||||||
|
|
||||||
### Router Rule
|
### Router Rule
|
||||||
|
|
||||||
Route everything except the internal MCP endpoint:
|
Route the public host to the backend (include `/mcp` — OAuth protects it):
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
rule: >
|
rule: Host(`ctxd.yourdomain.com`)
|
||||||
Host(`ctxd.yourdomain.com`) &&
|
|
||||||
!Path(`/mcp`)
|
|
||||||
```
|
```
|
||||||
|
|
||||||
This exposes:
|
This exposes:
|
||||||
@@ -304,10 +360,9 @@ This exposes:
|
|||||||
- Login (`POST /auth/login`, `GET /auth/me`)
|
- Login (`POST /auth/login`, `GET /auth/me`)
|
||||||
- Full Web UI dashboard (all REST API endpoints)
|
- Full Web UI dashboard (all REST API endpoints)
|
||||||
- OAuth (`/oauth/*`, `/.well-known/*`)
|
- OAuth (`/oauth/*`, `/.well-known/*`)
|
||||||
- Read-only MCP (`/readonly/mcp`)
|
- Public MCP (`/mcp` — OAuth read + write)
|
||||||
- Write MCP (`/write/mcp`)
|
|
||||||
|
|
||||||
Only blocked: `/mcp` (internal full MCP — shared API key only)
|
Hermes uses the same `/mcp` path on LAN with the shared API key (not exposed via public OAuth).
|
||||||
|
|
||||||
### Landing Page Behavior
|
### Landing Page Behavior
|
||||||
|
|
||||||
@@ -348,7 +403,7 @@ The compiled view (`get_project_context`) concatenates all files with `## FILENA
|
|||||||
All commands run inside the container:
|
All commands run inside the container:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker exec dossier dossier <command>
|
docker exec ctxd dossier <command>
|
||||||
```
|
```
|
||||||
|
|
||||||
### Commands
|
### Commands
|
||||||
@@ -461,7 +516,7 @@ If you started with SQLite and want to move to PostgreSQL:
|
|||||||
docker compose up -d postgres
|
docker compose up -d postgres
|
||||||
|
|
||||||
# 2. Run the migration script (reads from /data/ctxd.db, writes to DATABASE_URL)
|
# 2. Run the migration script (reads from /data/ctxd.db, writes to DATABASE_URL)
|
||||||
docker exec dossier python3 -m ctxd.migrate_sqlite_to_pg
|
docker exec ctxd python3 -m ctxd.migrate_sqlite_to_pg
|
||||||
|
|
||||||
# 3. Set DATABASE_URL in .env and restart
|
# 3. Set DATABASE_URL in .env and restart
|
||||||
docker compose up -d dossier
|
docker compose up -d dossier
|
||||||
@@ -526,7 +581,7 @@ docker compose build
|
|||||||
docker compose up -d --no-build
|
docker compose up -d --no-build
|
||||||
# Verify:
|
# Verify:
|
||||||
curl http://localhost:9091/status
|
curl http://localhost:9091/status
|
||||||
docker logs dossier --tail 20
|
docker logs ctxd --tail 20
|
||||||
```
|
```
|
||||||
|
|
||||||
### Key Conventions
|
### Key Conventions
|
||||||
@@ -544,7 +599,7 @@ docker logs dossier --tail 20
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Reset admin password
|
# Reset admin password
|
||||||
docker exec dossier dossier user-set-password admin -p "new-password"
|
docker exec ctxd dossier user-set-password admin -p "new-password"
|
||||||
```
|
```
|
||||||
|
|
||||||
If the password contains special characters, use the quoting that matches:
|
If the password contains special characters, use the quoting that matches:
|
||||||
@@ -569,7 +624,7 @@ The public host (`https://ctxd.yourdomain.com`) requires Traefik to route `/auth
|
|||||||
docker compose ps postgres
|
docker compose ps postgres
|
||||||
|
|
||||||
# Check connection
|
# Check connection
|
||||||
docker exec dossier python3 -c "
|
docker exec ctxd python3 -c "
|
||||||
import os
|
import os
|
||||||
import psycopg
|
import psycopg
|
||||||
conn = psycopg.connect(os.environ['DATABASE_URL'])
|
conn = psycopg.connect(os.environ['DATABASE_URL'])
|
||||||
@@ -581,13 +636,22 @@ docker exec ctxd-postgres psql -U ctxd -c "ALTER USER ctxd PASSWORD 'new-passwor
|
|||||||
# Then update .env with the new password
|
# Then update .env with the new password
|
||||||
```
|
```
|
||||||
|
|
||||||
### Container keeps restarting
|
### Container keeps restarting / public site 502
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker logs dossier --tail 30
|
docker compose ps -a
|
||||||
|
docker logs ctxd --tail 40
|
||||||
```
|
```
|
||||||
|
|
||||||
Common causes:
|
| Symptom | Cause | Fix |
|
||||||
|
|---------|--------|-----|
|
||||||
|
| `failed to resolve host 'postgres'` | `ctxd-postgres` not running (often after `--no-deps ctxd`) | `cd app && docker compose up -d postgres ctxd` or `./scripts/deploy.sh` |
|
||||||
|
| `Restarting (1)` on `ctxd` only | Same — app up without DB on compose network | Start postgres first; wait for **(healthy)** |
|
||||||
|
| Cloudflare **502** on public URL | Traefik/backend has no healthy upstream on `:9091` | Fix local `curl http://127.0.0.1:9091/status` first |
|
||||||
|
|
||||||
|
The container **entrypoint waits up to 120s** for PostgreSQL when `DATABASE_URL` is set (`CTXD_PG_WAIT_SECONDS` to override). If postgres never appears, logs print an explicit message instead of an immediate opaque crash.
|
||||||
|
|
||||||
|
Common other causes:
|
||||||
- `DATABASE_URL` password doesn't match what PG was initialized with
|
- `DATABASE_URL` password doesn't match what PG was initialized with
|
||||||
- `OAUTH_ENABLED=true` but `OAUTH_ISSUER` is empty
|
- `OAUTH_ENABLED=true` but `OAUTH_ISSUER` is empty
|
||||||
- Missing `CTXD_API_KEY` when `CTXD_AUTH_ENABLED=true`
|
- Missing `CTXD_API_KEY` when `CTXD_AUTH_ENABLED=true`
|
||||||
|
|||||||
@@ -22,28 +22,26 @@ metadata:
|
|||||||
|
|
||||||
# CTXD Client — LLM Agent Guide
|
# CTXD Client — LLM Agent Guide
|
||||||
|
|
||||||
Use when an LLM agent (Claude Desktop, Codex CLI, Hermes, custom harness) needs to read or update project context stored in a Context Dossier (CTXD) daemon.
|
Use when an LLM agent (Claude Desktop, Codex CLI, Hermes, custom harness) needs to read or update project context stored in Context Dossier (CTXD).
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
CTXD is a single source of truth for multi-harness project context. It exposes:
|
CTXD is a single source of truth for multi-harness project context. It exposes:
|
||||||
|
|
||||||
- **Read-only MCP** (`/readonly/mcp`) — OAuth bearer, `ctxd.read` scope, Streamable HTTP
|
- **Public MCP** (`/mcp`) — OAuth read + write (scope-gated) on the public host
|
||||||
- **Write MCP** (`/write/mcp`) — OAuth bearer, `ctxd.write` scope, Streamable HTTP
|
- **Hermes / automation** — same `http://<lan>:9091/mcp` with `CTXD_API_KEY` (full tool surface)
|
||||||
- **Web UI** (LAN `:9091` or public host) — per-user password login, admin panel
|
|
||||||
- **Internal full MCP** (`/mcp`) — shared API key (Hermes/automation only), Streamable HTTP
|
|
||||||
|
|
||||||
Public host: `https://ctxd.cubecraftcreations.com` (OAuth + MCP + landing page + dashboard).
|
Public host: `https://ctxd.cubecraftlabs.com` (OAuth + MCP + landing page + dashboard).
|
||||||
|
|
||||||
## Connection URLs
|
## Connection URLs
|
||||||
|
|
||||||
| Surface | URL | Auth |
|
| Surface | URL | Auth |
|
||||||
|---------|-----|------|
|
|---------|-----|------|
|
||||||
| Read-only MCP | `https://ctxd.cubecraftcreations.com/readonly/mcp` | OAuth `ctxd.read` |
|
| Public OAuth MCP (read + write) | `https://ctxd.cubecraftlabs.com/mcp` | OAuth `ctxd.read` and/or `ctxd.write` |
|
||||||
| Write MCP | `https://ctxd.cubecraftcreations.com/write/mcp` | OAuth `ctxd.write` |
|
| Legacy aliases | `/readonly/mcp`, `/oauth/mcp` | Same behavior as `/mcp` for OAuth |
|
||||||
| OAuth discovery | `https://ctxd.cubecraftcreations.com/.well-known/oauth-authorization-server` | Public |
|
| OAuth discovery | `https://ctxd.cubecraftlabs.com/.well-known/oauth-authorization-server` | Public |
|
||||||
| DCR registration | `POST https://ctxd.cubecraftcreations.com/oauth/register` | Public |
|
| DCR registration | `POST https://ctxd.cubecraftlabs.com/oauth/register` | Public |
|
||||||
| Landing page | `https://ctxd.cubecraftcreations.com/` | Public |
|
| Landing page | `https://ctxd.cubecraftlabs.com/` | Public |
|
||||||
|
|
||||||
## OAuth Flow
|
## OAuth Flow
|
||||||
|
|
||||||
@@ -51,7 +49,7 @@ Public host: `https://ctxd.cubecraftcreations.com` (OAuth + MCP + landing page +
|
|||||||
2. **Register** a client via `POST /oauth/register` (DCR) with your redirect URI
|
2. **Register** a client via `POST /oauth/register` (DCR) with your redirect URI
|
||||||
3. **Authorize** — open `/oauth/authorize` in a browser; an admin must approve
|
3. **Authorize** — open `/oauth/authorize` in a browser; an admin must approve
|
||||||
4. **Exchange** the authorization code for tokens at `POST /oauth/token`
|
4. **Exchange** the authorization code for tokens at `POST /oauth/token`
|
||||||
5. **Use** the access token as `Authorization: Bearer *** on MCP Streamable HTTP connections
|
5. **Use** the access token as `Authorization: Bearer <token>` on MCP connections (Streamable HTTP for read; SSE for write — see Connection URLs)
|
||||||
|
|
||||||
### Scopes
|
### Scopes
|
||||||
|
|
||||||
@@ -60,7 +58,9 @@ Public host: `https://ctxd.cubecraftcreations.com` (OAuth + MCP + landing page +
|
|||||||
| `ctxd.read` | `list_projects`, `get_project_context`, `search_context`, `get_project_tags`, `list_files`, `get_file`, `get_client_guide` |
|
| `ctxd.read` | `list_projects`, `get_project_context`, `search_context`, `get_project_tags`, `list_files`, `get_file`, `get_client_guide` |
|
||||||
| `ctxd.write` | `update_file`, `set_project_tags`, `sync_to_project`, `get_client_guide` |
|
| `ctxd.write` | `update_file`, `set_project_tags`, `sync_to_project`, `get_client_guide` |
|
||||||
|
|
||||||
Request both scopes for full read+write: `scope=ctxd.read ctxd.write`
|
Request both scopes for full read+write in **one connector**: `scope=ctxd.read ctxd.write`
|
||||||
|
|
||||||
|
**Admin:** In the Web UI → Admin → OAuth clients, set **allowed scopes** per client (create form or **scopes** on an existing row). Tokens cannot exceed what the client is allowed, even if the user requests more at authorize time.
|
||||||
|
|
||||||
### Redirect URIs by Platform
|
### Redirect URIs by Platform
|
||||||
|
|
||||||
@@ -85,7 +85,7 @@ Access tokens expire per server config (default ~1 hour). Refresh tokens are iss
|
|||||||
|
|
||||||
The guide lives in the `ctxd-docs` project as `LLM-CLIENT.MD`. It is **locked** — cannot be updated or deleted by any MCP/API client. If you need the guide updated, ask an admin to edit it via the Web UI.
|
The guide lives in the `ctxd-docs` project as `LLM-CLIENT.MD`. It is **locked** — cannot be updated or deleted by any MCP/API client. If you need the guide updated, ask an admin to edit it via the Web UI.
|
||||||
|
|
||||||
### Read-only (`/readonly/mcp`)
|
### Read-only tools (require `ctxd.read` on the same connector)
|
||||||
|
|
||||||
| Tool | Description |
|
| Tool | Description |
|
||||||
|------|-------------|
|
|------|-------------|
|
||||||
@@ -96,7 +96,7 @@ The guide lives in the `ctxd-docs` project as `LLM-CLIENT.MD`. It is **locked**
|
|||||||
| `list_files` | All context files in a project (multi-file mode) |
|
| `list_files` | All context files in a project (multi-file mode) |
|
||||||
| `get_file` | Single file with metadata header |
|
| `get_file` | Single file with metadata header |
|
||||||
|
|
||||||
### Write (`/write/mcp`)
|
### Write tools (require `ctxd.write` on the same connector)
|
||||||
|
|
||||||
| Tool | Description |
|
| Tool | Description |
|
||||||
|------|-------------|
|
|------|-------------|
|
||||||
@@ -164,7 +164,7 @@ Both return `403 cannot_update_locked` on PUT and `400 cannot_delete_context` on
|
|||||||
| `version_conflict` | Someone else updated the file | Re-read, merge, retry with new `base_version` |
|
| `version_conflict` | Someone else updated the file | Re-read, merge, retry with new `base_version` |
|
||||||
| `not_found` | Project or file doesn't exist | Check slug spelling; create project via Web UI |
|
| `not_found` | Project or file doesn't exist | Check slug spelling; create project via Web UI |
|
||||||
| `invalid_grant` | OAuth token expired or wrong scope | Refresh token or re-authorize with correct scope |
|
| `invalid_grant` | OAuth token expired or wrong scope | Refresh token or re-authorize with correct scope |
|
||||||
| `forbidden` | Wrong endpoint for the tool | Write tools only on `/write/mcp`; read tools on `/readonly/mcp` |
|
| `forbidden` | Wrong scope for the tool | Request `ctxd.write` for write tools; `ctxd.read` for read tools (same MCP URL) |
|
||||||
| `cannot_delete_context` | Tried to delete CONTEXT.MD or LLM-CLIENT.MD | Protected file — update content instead |
|
| `cannot_delete_context` | Tried to delete CONTEXT.MD or LLM-CLIENT.MD | Protected file — update content instead |
|
||||||
| `cannot_update_locked` | Tried to update LLM-CLIENT.MD | Locked guide — ask admin to update via Web UI |
|
| `cannot_update_locked` | Tried to update LLM-CLIENT.MD | Locked guide — ask admin to update via Web UI |
|
||||||
|
|
||||||
|
|||||||
@@ -2,6 +2,11 @@ name: ctxd
|
|||||||
|
|
||||||
# Docker Compose reads .env automatically for variable substitution.
|
# Docker Compose reads .env automatically for variable substitution.
|
||||||
# See .env.example for all available variables.
|
# See .env.example for all available variables.
|
||||||
|
#
|
||||||
|
# IMPORTANT: Production uses PostgreSQL (DATABASE_URL host "postgres").
|
||||||
|
# Always start BOTH services: docker compose up -d
|
||||||
|
# Or use: ./scripts/deploy.sh
|
||||||
|
# Do NOT use docker compose up -d --no-deps ctxd unless postgres is already running.
|
||||||
|
|
||||||
services:
|
services:
|
||||||
postgres:
|
postgres:
|
||||||
@@ -20,11 +25,11 @@ services:
|
|||||||
timeout: 5s
|
timeout: 5s
|
||||||
retries: 5
|
retries: 5
|
||||||
|
|
||||||
dossier:
|
ctxd:
|
||||||
build:
|
build:
|
||||||
context: .
|
context: .
|
||||||
dockerfile: Dockerfile
|
dockerfile: Dockerfile
|
||||||
container_name: dossier
|
container_name: ctxd
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
ports:
|
ports:
|
||||||
- "${CTXD_PORT:-9091}:${CTXD_PORT:-9091}"
|
- "${CTXD_PORT:-9091}:${CTXD_PORT:-9091}"
|
||||||
|
|||||||
+71
-5
@@ -1,11 +1,77 @@
|
|||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
set -e
|
set -e
|
||||||
|
|
||||||
# Initialize if database doesn't exist
|
wait_for_postgres() {
|
||||||
if [ ! -f "$CTXD_HOME/ctxd.db" ]; then
|
if [ -z "$DATABASE_URL" ]; then
|
||||||
echo "ctxd: initializing database at $CTXD_HOME"
|
return 0
|
||||||
python3 -m ctxd init --home "$CTXD_HOME"
|
fi
|
||||||
|
echo "ctxd: waiting for PostgreSQL (DATABASE_URL is set)..."
|
||||||
|
python3 <<'PY'
|
||||||
|
import os, sys, time
|
||||||
|
|
||||||
|
url = os.environ.get("DATABASE_URL", "")
|
||||||
|
max_wait = int(os.environ.get("CTXD_PG_WAIT_SECONDS", "120"))
|
||||||
|
interval = int(os.environ.get("CTXD_PG_WAIT_INTERVAL", "2"))
|
||||||
|
deadline = time.time() + max_wait
|
||||||
|
last_err = ""
|
||||||
|
attempt = 0
|
||||||
|
while time.time() < deadline:
|
||||||
|
attempt += 1
|
||||||
|
try:
|
||||||
|
import psycopg
|
||||||
|
conn = psycopg.connect(url, connect_timeout=5)
|
||||||
|
conn.close()
|
||||||
|
print("ctxd: PostgreSQL is ready")
|
||||||
|
sys.exit(0)
|
||||||
|
except Exception as e:
|
||||||
|
last_err = str(e)
|
||||||
|
if attempt == 1 or attempt % 10 == 0:
|
||||||
|
print(f"ctxd: postgres not ready (attempt {attempt}): {last_err}", file=sys.stderr)
|
||||||
|
time.sleep(interval)
|
||||||
|
|
||||||
|
print("ctxd: FATAL: PostgreSQL not reachable within wait window.", file=sys.stderr)
|
||||||
|
print(f"ctxd: last error: {last_err}", file=sys.stderr)
|
||||||
|
print("ctxd:", file=sys.stderr)
|
||||||
|
print("ctxd: Fix: start the full stack (postgres + ctxd), e.g.", file=sys.stderr)
|
||||||
|
print("ctxd: cd app && docker compose up -d postgres ctxd", file=sys.stderr)
|
||||||
|
print("ctxd:", file=sys.stderr)
|
||||||
|
print("ctxd: Avoid: docker compose up -d --no-deps ctxd (skips postgres)", file=sys.stderr)
|
||||||
|
print("ctxd: If postgres is on another host, fix DATABASE_URL / network.", file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
wait_for_postgres
|
||||||
|
|
||||||
|
# Initialize if needed
|
||||||
|
if [ -n "$DATABASE_URL" ]; then
|
||||||
|
# PostgreSQL mode — check if schema exists
|
||||||
|
NEEDS_INIT=$(python3 -c "
|
||||||
|
import psycopg, os, sys
|
||||||
|
try:
|
||||||
|
conn = psycopg.connect(os.environ['DATABASE_URL'])
|
||||||
|
cur = conn.execute(\"SELECT EXISTS (SELECT FROM pg_tables WHERE schemaname='public' AND tablename='users')\")
|
||||||
|
if not cur.fetchone()[0]:
|
||||||
|
print('yes')
|
||||||
|
else:
|
||||||
|
print('no')
|
||||||
|
except Exception:
|
||||||
|
print('yes')
|
||||||
|
" 2>/dev/null || echo "yes")
|
||||||
|
if [ "$NEEDS_INIT" = "yes" ]; then
|
||||||
|
echo "ctxd: initializing PostgreSQL database"
|
||||||
|
python3 -m ctxd init --home "$CTXD_HOME"
|
||||||
|
else
|
||||||
|
echo "ctxd: running pending migrations"
|
||||||
|
python3 -m ctxd.migrate_user_fk_set_null 2>&1 || echo "ctxd: migration warning (non-fatal)"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
# SQLite mode — check for db file
|
||||||
|
if [ ! -f "$CTXD_HOME/ctxd.db" ]; then
|
||||||
|
echo "ctxd: initializing database at $CTXD_HOME"
|
||||||
|
python3 -m ctxd init --home "$CTXD_HOME"
|
||||||
|
fi
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "ctxd: starting daemon on 0.0.0.0:9091"
|
echo "ctxd: starting daemon on ${CTXD_HOST:-0.0.0.0}:${CTXD_PORT:-9091}"
|
||||||
exec python3 -m ctxd
|
exec python3 -m ctxd
|
||||||
Executable
+46
@@ -0,0 +1,46 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Safe CTXD deploy: always ensures postgres is up before recreating ctxd.
|
||||||
|
# Usage: ./scripts/deploy.sh [--build-only | --no-build]
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||||
|
cd "$ROOT"
|
||||||
|
|
||||||
|
BUILD=1
|
||||||
|
if [[ "${1:-}" == "--no-build" ]]; then
|
||||||
|
BUILD=0
|
||||||
|
elif [[ "${1:-}" == "--build-only" ]]; then
|
||||||
|
docker compose build ctxd
|
||||||
|
echo "Built ctxd image only."
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$BUILD" -eq 1 ]]; then
|
||||||
|
echo "==> docker compose build ctxd"
|
||||||
|
docker compose build ctxd
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "==> docker compose up -d postgres"
|
||||||
|
docker compose up -d postgres
|
||||||
|
|
||||||
|
echo "==> waiting for postgres healthy..."
|
||||||
|
for i in $(seq 1 60); do
|
||||||
|
if docker compose ps postgres 2>/dev/null | grep -q '(healthy)'; then
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
sleep 2
|
||||||
|
done
|
||||||
|
if ! docker compose ps postgres 2>/dev/null | grep -q '(healthy)'; then
|
||||||
|
echo "ERROR: ctxd-postgres did not become healthy. Check: docker compose logs postgres" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "==> docker compose up -d --force-recreate ctxd"
|
||||||
|
docker compose up -d --force-recreate ctxd
|
||||||
|
|
||||||
|
PORT="${CTXD_PORT:-9091}"
|
||||||
|
echo "==> smoke: http://127.0.0.1:${PORT}/status"
|
||||||
|
sleep 3
|
||||||
|
curl -sf "http://127.0.0.1:${PORT}/status" | head -c 200
|
||||||
|
echo ""
|
||||||
|
docker compose ps postgres ctxd
|
||||||
@@ -0,0 +1,150 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Smoke test: one OAuth connector lists read+write tools on /mcp."""
|
||||||
|
import json, urllib.request, urllib.parse, hashlib, base64, secrets, http.client
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
BASE = "http://127.0.0.1:9091"
|
||||||
|
env_path = Path("/mnt/ai-storage/projects/ctxd/app/.env")
|
||||||
|
approval_key = ""
|
||||||
|
for line in env_path.read_text().splitlines():
|
||||||
|
if line.startswith("OAUTH_APPROVAL_KEY="):
|
||||||
|
approval_key = line.split("=", 1)[1]
|
||||||
|
break
|
||||||
|
if not approval_key:
|
||||||
|
raise SystemExit("no OAUTH_APPROVAL_KEY")
|
||||||
|
|
||||||
|
|
||||||
|
class NR(urllib.request.HTTPRedirectHandler):
|
||||||
|
def redirect_request(self, req, fp, code, msg, headers, newurl):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
opener = urllib.request.build_opener(NR)
|
||||||
|
r = opener.open(
|
||||||
|
urllib.request.Request(
|
||||||
|
f"{BASE}/oauth/register",
|
||||||
|
data=json.dumps(
|
||||||
|
{
|
||||||
|
"redirect_uris": ["http://localhost:9999/cb"],
|
||||||
|
"client_name": "unified-mcp-test",
|
||||||
|
"scope": "ctxd.read ctxd.write",
|
||||||
|
}
|
||||||
|
).encode(),
|
||||||
|
headers={"Content-Type": "application/json"},
|
||||||
|
method="POST",
|
||||||
|
),
|
||||||
|
timeout=15,
|
||||||
|
)
|
||||||
|
client = json.loads(r.read())
|
||||||
|
cid, secret = client["client_id"], client["client_secret"]
|
||||||
|
|
||||||
|
cv = secrets.token_urlsafe(32)
|
||||||
|
cc = base64.urlsafe_b64encode(hashlib.sha256(cv.encode()).digest()).rstrip(b"=").decode()
|
||||||
|
params = {
|
||||||
|
"response_type": "code",
|
||||||
|
"client_id": cid,
|
||||||
|
"redirect_uri": "http://localhost:9999/cb",
|
||||||
|
"code_challenge": cc,
|
||||||
|
"code_challenge_method": "S256",
|
||||||
|
"scope": "ctxd.read ctxd.write",
|
||||||
|
"state": "t",
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
opener.open(
|
||||||
|
urllib.request.Request(
|
||||||
|
f"{BASE}/oauth/authorize?" + urllib.parse.urlencode(params),
|
||||||
|
data=urllib.parse.urlencode({"approval_key": approval_key}).encode(),
|
||||||
|
headers={"Content-Type": "application/x-www-form-urlencoded"},
|
||||||
|
method="POST",
|
||||||
|
),
|
||||||
|
timeout=15,
|
||||||
|
)
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
loc = e.headers.get("Location", "")
|
||||||
|
code = dict(urllib.parse.parse_qsl(urllib.parse.urlparse(loc).query)).get("code", "")
|
||||||
|
|
||||||
|
r3 = opener.open(
|
||||||
|
urllib.request.Request(
|
||||||
|
f"{BASE}/oauth/token",
|
||||||
|
data=urllib.parse.urlencode(
|
||||||
|
{
|
||||||
|
"grant_type": "authorization_code",
|
||||||
|
"code": code,
|
||||||
|
"redirect_uri": "http://localhost:9999/cb",
|
||||||
|
"client_id": cid,
|
||||||
|
"client_secret": secret,
|
||||||
|
"code_verifier": cv,
|
||||||
|
}
|
||||||
|
).encode(),
|
||||||
|
headers={"Content-Type": "application/x-www-form-urlencoded"},
|
||||||
|
method="POST",
|
||||||
|
),
|
||||||
|
timeout=15,
|
||||||
|
)
|
||||||
|
token = json.loads(r3.read())["access_token"]
|
||||||
|
|
||||||
|
|
||||||
|
def mcp_post(path: str, body: dict, session_id: str | None = None) -> tuple[int, str, str | None]:
|
||||||
|
headers = {
|
||||||
|
"Authorization": f"Bearer {token}",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"Accept": "application/json, text/event-stream",
|
||||||
|
}
|
||||||
|
if session_id:
|
||||||
|
headers["mcp-session-id"] = session_id
|
||||||
|
conn = http.client.HTTPConnection("127.0.0.1", 9091, timeout=15)
|
||||||
|
conn.request("POST", path, body=json.dumps(body), headers=headers)
|
||||||
|
resp = conn.getresponse()
|
||||||
|
sid = resp.getheader("mcp-session-id")
|
||||||
|
data = resp.read(8000).decode(errors="replace")
|
||||||
|
conn.close()
|
||||||
|
return resp.status, data, sid
|
||||||
|
|
||||||
|
|
||||||
|
init_body = {
|
||||||
|
"jsonrpc": "2.0",
|
||||||
|
"id": 1,
|
||||||
|
"method": "initialize",
|
||||||
|
"params": {
|
||||||
|
"protocolVersion": "2025-03-26",
|
||||||
|
"capabilities": {},
|
||||||
|
"clientInfo": {"name": "unified-test", "version": "1"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
st, raw, sid = mcp_post("/mcp", init_body)
|
||||||
|
print("initialize:", st, "session:", sid)
|
||||||
|
if st != 200:
|
||||||
|
raise SystemExit(raw[:500])
|
||||||
|
|
||||||
|
if sid:
|
||||||
|
st2, raw2, _ = mcp_post(
|
||||||
|
"/mcp",
|
||||||
|
{"jsonrpc": "2.0", "id": 2, "method": "tools/list", "params": {}},
|
||||||
|
session_id=sid.split(",")[0].strip(),
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
st2, raw2, _ = mcp_post("/mcp", {"jsonrpc": "2.0", "id": 2, "method": "tools/list", "params": {}})
|
||||||
|
|
||||||
|
print("tools/list:", st2)
|
||||||
|
names = []
|
||||||
|
for line in raw2.splitlines():
|
||||||
|
if line.startswith("data:"):
|
||||||
|
try:
|
||||||
|
payload = json.loads(line[5:].strip())
|
||||||
|
for t in payload.get("result", {}).get("tools", []):
|
||||||
|
names.append(t.get("name"))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
if '"tools"' in raw2 and not names:
|
||||||
|
try:
|
||||||
|
payload = json.loads(raw2)
|
||||||
|
names = [t["name"] for t in payload.get("result", {}).get("tools", [])]
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
print("tools:", sorted(set(names)))
|
||||||
|
need = {"list_projects", "update_file", "sync_to_project"}
|
||||||
|
missing = need - set(names)
|
||||||
|
if missing:
|
||||||
|
raise SystemExit(f"missing tools: {missing}")
|
||||||
|
print("OK unified connector exposes read+write tools")
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""E2E: OAuth write token -> /write/sse and tools/list via SSE."""
|
||||||
|
import json, urllib.request, urllib.parse, hashlib, base64, secrets, http.client, time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
BASE = "http://127.0.0.1:9091"
|
||||||
|
env_path = Path("/mnt/ai-storage/projects/ctxd/app/.env")
|
||||||
|
approval_key = ""
|
||||||
|
for line in env_path.read_text().splitlines():
|
||||||
|
if line.startswith("OAUTH_APPROVAL_KEY="):
|
||||||
|
approval_key = line.split("=", 1)[1]
|
||||||
|
break
|
||||||
|
if not approval_key:
|
||||||
|
raise SystemExit("no OAUTH_APPROVAL_KEY")
|
||||||
|
|
||||||
|
class NR(urllib.request.HTTPRedirectHandler):
|
||||||
|
def redirect_request(self, *a): return None
|
||||||
|
opener = urllib.request.build_opener(NR)
|
||||||
|
|
||||||
|
r = opener.open(urllib.request.Request(
|
||||||
|
f"{BASE}/oauth/register",
|
||||||
|
data=json.dumps({"redirect_uris": ["http://localhost:9999/cb"], "client_name": "write-test", "scope": "ctxd.read ctxd.write"}).encode(),
|
||||||
|
headers={"Content-Type": "application/json"}, method="POST"), timeout=15)
|
||||||
|
client = json.loads(r.read())
|
||||||
|
cid, secret = client["client_id"], client["client_secret"]
|
||||||
|
|
||||||
|
cv = secrets.token_urlsafe(32)
|
||||||
|
cc = base64.urlsafe_b64encode(hashlib.sha256(cv.encode()).digest()).rstrip(b"=").decode()
|
||||||
|
params = {"response_type": "code", "client_id": cid, "redirect_uri": "http://localhost:9999/cb",
|
||||||
|
"code_challenge": cc, "code_challenge_method": "S256", "scope": "ctxd.read ctxd.write", "state": "t"}
|
||||||
|
try:
|
||||||
|
opener.open(urllib.request.Request(f"{BASE}/oauth/authorize?" + urllib.parse.urlencode(params),
|
||||||
|
data=urllib.parse.urlencode({"approval_key": approval_key}).encode(),
|
||||||
|
headers={"Content-Type": "application/x-www-form-urlencoded"}, method="POST"), timeout=15)
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
loc = e.headers.get("Location", "")
|
||||||
|
code = dict(urllib.parse.parse_qsl(urllib.parse.urlparse(loc).query)).get("code", "")
|
||||||
|
|
||||||
|
r3 = opener.open(urllib.request.Request(f"{BASE}/oauth/token",
|
||||||
|
data=urllib.parse.urlencode({"grant_type": "authorization_code", "code": code, "redirect_uri": "http://localhost:9999/cb",
|
||||||
|
"client_id": cid, "client_secret": secret, "code_verifier": cv}).encode(),
|
||||||
|
headers={"Content-Type": "application/x-www-form-urlencoded"}, method="POST"), timeout=15)
|
||||||
|
token = json.loads(r3.read())["access_token"]
|
||||||
|
|
||||||
|
conn = http.client.HTTPConnection("127.0.0.1", 9091, timeout=5)
|
||||||
|
conn.request("GET", "/write/sse", headers={"Authorization": f"Bearer {token}"})
|
||||||
|
resp = conn.getresponse()
|
||||||
|
print("write/sse status:", resp.status, "ctype:", resp.getheader("content-type"))
|
||||||
|
chunk = resp.read(500).decode(errors="replace")
|
||||||
|
print("body[:200]:", chunk[:200])
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
# POST initialize to /write/messages (needs session from SSE - simplified check)
|
||||||
|
conn2 = http.client.HTTPConnection("127.0.0.1", 9091, timeout=10)
|
||||||
|
body = json.dumps({"jsonrpc": "2.0", "id": 1, "method": "tools/list", "params": {}})
|
||||||
|
conn2.request("POST", "/write/messages", body=body,
|
||||||
|
headers={"Authorization": f"Bearer {token}", "Content-Type": "application/json"})
|
||||||
|
r2 = conn2.getresponse()
|
||||||
|
print("write/messages POST:", r2.status, r2.read(300).decode(errors="replace")[:200])
|
||||||
|
conn2.close()
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
Metadata-Version: 2.4
|
|
||||||
Name: ctxd
|
|
||||||
Version: 0.1.0
|
|
||||||
Summary: Context daemon — single source of truth for multi-harness project context
|
|
||||||
Requires-Python: >=3.11
|
|
||||||
@@ -1,13 +0,0 @@
|
|||||||
pyproject.toml
|
|
||||||
src/ctxd/__init__.py
|
|
||||||
src/ctxd/__main__.py
|
|
||||||
src/ctxd/cli.py
|
|
||||||
src/ctxd/config.py
|
|
||||||
src/ctxd/db.py
|
|
||||||
src/ctxd/mcp_stdio.py
|
|
||||||
src/ctxd/server.py
|
|
||||||
src/ctxd.egg-info/PKG-INFO
|
|
||||||
src/ctxd.egg-info/SOURCES.txt
|
|
||||||
src/ctxd.egg-info/dependency_links.txt
|
|
||||||
src/ctxd.egg-info/entry_points.txt
|
|
||||||
src/ctxd.egg-info/top_level.txt
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
|
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
[console_scripts]
|
|
||||||
ctx = ctxd:cli_entry
|
|
||||||
ctxd = ctxd:daemon_entry
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
ctxd
|
|
||||||
@@ -4,7 +4,10 @@ import sys
|
|||||||
|
|
||||||
if len(sys.argv) > 1 and sys.argv[1] in ('init', 'serve', 'project-list', 'project-create',
|
if len(sys.argv) > 1 and sys.argv[1] in ('init', 'serve', 'project-list', 'project-create',
|
||||||
'read', 'cat', 'edit', 'search', 'sync', 'audit',
|
'read', 'cat', 'edit', 'search', 'sync', 'audit',
|
||||||
'user-list', 'user-create', 'import-vault'):
|
'user-list', 'user-create', 'import-vault',
|
||||||
|
'user-set-password', 'oauth-client-create',
|
||||||
|
'oauth-client-list', 'oauth-client-revoke',
|
||||||
|
'file-list', 'file-read'):
|
||||||
from ctxd.cli import cli_entry
|
from ctxd.cli import cli_entry
|
||||||
cli_entry()
|
cli_entry()
|
||||||
else:
|
else:
|
||||||
@@ -12,7 +15,9 @@ else:
|
|||||||
from ctxd.server import serve_sync
|
from ctxd.server import serve_sync
|
||||||
|
|
||||||
cfg = CtxConfig.from_home()
|
cfg = CtxConfig.from_home()
|
||||||
if not cfg.db_path.exists():
|
# When using PostgreSQL, the DB is external — skip the file check.
|
||||||
|
# When using SQLite, verify the db file exists.
|
||||||
|
if not cfg.use_postgres and not cfg.db_path.exists():
|
||||||
print("Not initialized. Run 'ctx init' first.", file=sys.stderr)
|
print("Not initialized. Run 'ctx init' first.", file=sys.stderr)
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
serve_sync(cfg)
|
serve_sync(cfg)
|
||||||
+17
-10
@@ -64,14 +64,15 @@ Multi-camera remote monitoring system.
|
|||||||
|
|
||||||
def _seed_context(conn):
|
def _seed_context(conn):
|
||||||
"""Insert seed project context with real newlines."""
|
"""Insert seed project context with real newlines."""
|
||||||
conn.execute(
|
from ctxd.db import _is_pg, _ph
|
||||||
"INSERT OR IGNORE INTO project_context (project_id, content, version, updated_by) VALUES (?, ?, 0, 'system')",
|
ph = _ph(conn, 2)
|
||||||
('welcome', _WELCOME_CONTENT),
|
placeholders = ph.split(", ")
|
||||||
)
|
if _is_pg(conn):
|
||||||
conn.execute(
|
sql = f"INSERT INTO project_context (project_id, content, version, updated_by) VALUES ({placeholders[0]}, {placeholders[1]}, 0, 'admin') ON CONFLICT DO NOTHING"
|
||||||
"INSERT OR IGNORE INTO project_context (project_id, content, version, updated_by) VALUES (?, ?, 0, 'system')",
|
else:
|
||||||
('remote-rig', _REMOTE_RIG_CONTENT),
|
sql = f"INSERT OR IGNORE INTO project_context (project_id, content, version, updated_by) VALUES ({placeholders[0]}, {placeholders[1]}, 0, 'admin')"
|
||||||
)
|
conn.execute(sql, ('welcome', _WELCOME_CONTENT))
|
||||||
|
conn.execute(sql, ('remote-rig', _REMOTE_RIG_CONTENT))
|
||||||
|
|
||||||
|
|
||||||
def cmd_init(args):
|
def cmd_init(args):
|
||||||
@@ -350,6 +351,10 @@ def cmd_oauth_client_create(args):
|
|||||||
client = store.register_client({
|
client = store.register_client({
|
||||||
"client_name": args.name or "Claude MCP Client",
|
"client_name": args.name or "Claude MCP Client",
|
||||||
"redirect_uris": redirects,
|
"redirect_uris": redirects,
|
||||||
|
"scopes": getattr(args, "scopes", None) or (
|
||||||
|
[s for s in (args.scope or "").split() if s]
|
||||||
|
or ["ctxd.read", "ctxd.write"]
|
||||||
|
),
|
||||||
})
|
})
|
||||||
issuer = (cfg.oauth_issuer or "").rstrip("/") or "https://ctxd.cubecraftcreations.com"
|
issuer = (cfg.oauth_issuer or "").rstrip("/") or "https://ctxd.cubecraftcreations.com"
|
||||||
print(json.dumps({
|
print(json.dumps({
|
||||||
@@ -357,7 +362,8 @@ def cmd_oauth_client_create(args):
|
|||||||
"client_secret": client["client_secret"],
|
"client_secret": client["client_secret"],
|
||||||
"client_name": client.get("client_name"),
|
"client_name": client.get("client_name"),
|
||||||
"redirect_uris": client.get("redirect_uris"),
|
"redirect_uris": client.get("redirect_uris"),
|
||||||
"connector_url": f"{issuer}/readonly/sse",
|
"scope": client.get("scope"),
|
||||||
|
"connector_url": f"{issuer}/mcp",
|
||||||
"authorization_server": issuer,
|
"authorization_server": issuer,
|
||||||
"note": "Claude usually registers via POST /oauth/register automatically; save client_secret now — it is not shown again.",
|
"note": "Claude usually registers via POST /oauth/register automatically; save client_secret now — it is not shown again.",
|
||||||
}, indent=2))
|
}, indent=2))
|
||||||
@@ -370,7 +376,7 @@ def cmd_oauth_client_list(args):
|
|||||||
cfg = CtxConfig.from_home(args.home)
|
cfg = CtxConfig.from_home(args.home)
|
||||||
store = OAuthStore(cfg)
|
store = OAuthStore(cfg)
|
||||||
for c in store.list_clients_public():
|
for c in store.list_clients_public():
|
||||||
print(f"{c.get('client_id')} {c.get('client_name', '')} redirects={c.get('redirect_uris')}")
|
print(f"{c.get('client_id')} {c.get('client_name', '')} scope={c.get('scope', '')} redirects={c.get('redirect_uris')}")
|
||||||
|
|
||||||
|
|
||||||
def cmd_oauth_client_revoke(args):
|
def cmd_oauth_client_revoke(args):
|
||||||
@@ -558,6 +564,7 @@ def build_parser() -> argparse.ArgumentParser:
|
|||||||
sp.set_defaults(func=cmd_oauth_client_create)
|
sp.set_defaults(func=cmd_oauth_client_create)
|
||||||
sp.add_argument("--name", "-n", default="Claude MCP Client", help="Client display name")
|
sp.add_argument("--name", "-n", default="Claude MCP Client", help="Client display name")
|
||||||
sp.add_argument("--redirect-uri", action="append", dest="redirect_uri", help="Redirect URI (default: Claude MCP callback; repeat for multiple)")
|
sp.add_argument("--redirect-uri", action="append", dest="redirect_uri", help="Redirect URI (default: Claude MCP callback; repeat for multiple)")
|
||||||
|
sp.add_argument("--scope", default="ctxd.read ctxd.write", help="Allowed scopes (space-separated: ctxd.read ctxd.write)")
|
||||||
sp.add_argument("--home")
|
sp.add_argument("--home")
|
||||||
|
|
||||||
# oauth-client-list
|
# oauth-client-list
|
||||||
|
|||||||
@@ -130,6 +130,10 @@ class CtxConfig:
|
|||||||
def log_level(self) -> str:
|
def log_level(self) -> str:
|
||||||
return _env_str("LOG_LEVEL", "info")
|
return _env_str("LOG_LEVEL", "info")
|
||||||
|
|
||||||
|
@property
|
||||||
|
def demo_mode(self) -> bool:
|
||||||
|
return _env_bool("CTXD_DEMO_MODE", False)
|
||||||
|
|
||||||
# ── Snapshots ─────────────────────────────────────────────────
|
# ── Snapshots ─────────────────────────────────────────────────
|
||||||
@property
|
@property
|
||||||
def min_snapshots(self) -> int:
|
def min_snapshots(self) -> int:
|
||||||
|
|||||||
+17
-1
@@ -114,6 +114,22 @@ def _is_pg(conn) -> bool:
|
|||||||
return not isinstance(conn, sqlite3.Connection)
|
return not isinstance(conn, sqlite3.Connection)
|
||||||
|
|
||||||
|
|
||||||
|
def is_integrity_error(exc: BaseException) -> bool:
|
||||||
|
"""True if exc is a DB constraint violation (unique/primary-key, FK, check)
|
||||||
|
on either backend. Used to map duplicate-key errors to HTTP 409.
|
||||||
|
|
||||||
|
SQLite raises sqlite3.IntegrityError; psycopg raises subclasses of
|
||||||
|
psycopg.errors.IntegrityError (UniqueViolation, ForeignKeyViolation, ...).
|
||||||
|
"""
|
||||||
|
if isinstance(exc, sqlite3.IntegrityError):
|
||||||
|
return True
|
||||||
|
try:
|
||||||
|
import psycopg
|
||||||
|
except ImportError:
|
||||||
|
return False
|
||||||
|
return isinstance(exc, psycopg.errors.IntegrityError)
|
||||||
|
|
||||||
|
|
||||||
# ── Helpers ───────────────────────────────────────────────────────────────────
|
# ── Helpers ───────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
def _row_to_dict(row) -> dict | None:
|
def _row_to_dict(row) -> dict | None:
|
||||||
@@ -190,9 +206,9 @@ def user_delete(conn, user_id: str) -> dict:
|
|||||||
conn.execute(f"DELETE FROM users WHERE user_id = {ph}", (user_id,))
|
conn.execute(f"DELETE FROM users WHERE user_id = {ph}", (user_id,))
|
||||||
return {"ok": True}
|
return {"ok": True}
|
||||||
except (sqlite3.IntegrityError, Exception) as e:
|
except (sqlite3.IntegrityError, Exception) as e:
|
||||||
# Check if it's a foreign key violation
|
|
||||||
if _is_pg(conn):
|
if _is_pg(conn):
|
||||||
import psycopg
|
import psycopg
|
||||||
|
conn.rollback()
|
||||||
if isinstance(e, psycopg.errors.ForeignKeyViolation):
|
if isinstance(e, psycopg.errors.ForeignKeyViolation):
|
||||||
return {"ok": False, "error": "user_has_references", "hint": "Inactivate the user instead of deleting."}
|
return {"ok": False, "error": "user_has_references", "hint": "Inactivate the user instead of deleting."}
|
||||||
raise
|
raise
|
||||||
|
|||||||
@@ -179,6 +179,23 @@ def migrate():
|
|||||||
print(f" context_files: {fts_cf} entries")
|
print(f" context_files: {fts_cf} entries")
|
||||||
print(f" user_profiles: {fts_up} entries")
|
print(f" user_profiles: {fts_up} entries")
|
||||||
|
|
||||||
|
# Step 4: Reset SERIAL sequences to max(existing_id) + 1
|
||||||
|
print()
|
||||||
|
print("Resetting SERIAL sequences...")
|
||||||
|
for table, col in [
|
||||||
|
("audit_log", "entry_id"),
|
||||||
|
("context_files", "file_id"),
|
||||||
|
("project_permissions", "id"),
|
||||||
|
("reviews", "review_id"),
|
||||||
|
("workspace_files", "file_id"),
|
||||||
|
]:
|
||||||
|
try:
|
||||||
|
pconn.execute(f"SELECT setval(pg_get_serial_sequence('{table}', '{col}'), COALESCE((SELECT MAX({col}) FROM {table}), 1))")
|
||||||
|
print(f" {table}.{col}: seq reset")
|
||||||
|
except Exception as e:
|
||||||
|
print(f" {table}.{col}: skipped ({e})")
|
||||||
|
pconn.commit()
|
||||||
|
|
||||||
print()
|
print()
|
||||||
print("=" * 60)
|
print("=" * 60)
|
||||||
print(f"Migration complete! {total_rows} total rows migrated.")
|
print(f"Migration complete! {total_rows} total rows migrated.")
|
||||||
|
|||||||
@@ -0,0 +1,83 @@
|
|||||||
|
"""
|
||||||
|
Migration: alter user FK constraints to ON DELETE SET NULL.
|
||||||
|
|
||||||
|
PostgreSQL doesn't support ALTER CONSTRAINT inline — you must drop and
|
||||||
|
re-add the constraint. This script does that for all user_id FKs that
|
||||||
|
were originally created as RESTRICT (no ON DELETE action).
|
||||||
|
|
||||||
|
Run once against the production database:
|
||||||
|
docker exec ctxd python3 -m ctxd.migrate_user_fk_set_null
|
||||||
|
|
||||||
|
Safe to run multiple times (skips if constraint already has ON DELETE SET NULL).
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
import psycopg
|
||||||
|
|
||||||
|
# Tables + constraints to fix. constraint_name is the auto-generated PG name.
|
||||||
|
FKS_TO_FIX = [
|
||||||
|
("project_context", "project_context_updated_by_fkey", "updated_by"),
|
||||||
|
("context_files", "context_files_updated_by_fkey", "updated_by"),
|
||||||
|
("change_requests", "change_requests_submitted_by_fkey", "submitted_by"),
|
||||||
|
("reviews", "reviews_reviewer_id_fkey", "reviewer_id"),
|
||||||
|
("audit_log", "audit_log_user_id_fkey", "user_id"),
|
||||||
|
]
|
||||||
|
|
||||||
|
# Columns that were NOT NULL but must be nullable for ON DELETE SET NULL to work
|
||||||
|
DROP_NOT_NULL = [
|
||||||
|
("audit_log", "user_id"),
|
||||||
|
("change_requests", "submitted_by"),
|
||||||
|
("reviews", "reviewer_id"),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
url = os.environ.get("DATABASE_URL")
|
||||||
|
if not url:
|
||||||
|
print("DATABASE_URL not set — nothing to do (SQLite mode)")
|
||||||
|
return
|
||||||
|
conn = psycopg.connect(url)
|
||||||
|
conn.autocommit = True
|
||||||
|
|
||||||
|
# Step 1: drop NOT NULL where needed (ON DELETE SET NULL requires nullable)
|
||||||
|
for table, col in DROP_NOT_NULL:
|
||||||
|
row = conn.execute(
|
||||||
|
f"SELECT is_nullable FROM information_schema.columns WHERE table_name = %s AND column_name = %s",
|
||||||
|
(table, col),
|
||||||
|
).fetchone()
|
||||||
|
if row and row[0] == "NO":
|
||||||
|
print(f" DROP NOT NULL {table}.{col}")
|
||||||
|
conn.execute(f"ALTER TABLE {table} ALTER COLUMN {col} DROP NOT NULL")
|
||||||
|
print(f" DONE {table}.{col}")
|
||||||
|
else:
|
||||||
|
print(f" SKIP {table}.{col}: already nullable")
|
||||||
|
|
||||||
|
# Step 2: fix FK constraints
|
||||||
|
for table, constraint, col in FKS_TO_FIX:
|
||||||
|
# Check if the constraint exists and what its current ON DELETE action is
|
||||||
|
row = conn.execute(
|
||||||
|
"""SELECT confdeltype FROM pg_constraint
|
||||||
|
WHERE conname = %s AND contype = 'f'""",
|
||||||
|
(constraint,),
|
||||||
|
).fetchone()
|
||||||
|
if not row:
|
||||||
|
print(f" SKIP {constraint}: not found (already migrated or table missing)")
|
||||||
|
continue
|
||||||
|
if row[0] == "n": # 'n' = SET NULL
|
||||||
|
print(f" SKIP {constraint}: already ON DELETE SET NULL")
|
||||||
|
continue
|
||||||
|
|
||||||
|
col = constraint.replace(f"{table}_", "").replace("_fkey", "")
|
||||||
|
print(f" ALTER {table}.{col} ({constraint}): {row[0]} -> SET NULL")
|
||||||
|
conn.execute(f'ALTER TABLE {table} DROP CONSTRAINT {constraint}')
|
||||||
|
conn.execute(
|
||||||
|
f'ALTER TABLE {table} ADD CONSTRAINT {constraint} '
|
||||||
|
f'FOREIGN KEY ({col}) REFERENCES users(user_id) ON DELETE SET NULL'
|
||||||
|
)
|
||||||
|
print(f" DONE {constraint}")
|
||||||
|
|
||||||
|
conn.close()
|
||||||
|
print("Migration complete.")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -65,7 +65,7 @@ CREATE TABLE project_context (
|
|||||||
project_id TEXT PRIMARY KEY REFERENCES projects(project_id) ON DELETE CASCADE,
|
project_id TEXT PRIMARY KEY REFERENCES projects(project_id) ON DELETE CASCADE,
|
||||||
content TEXT NOT NULL DEFAULT '',
|
content TEXT NOT NULL DEFAULT '',
|
||||||
version INTEGER NOT NULL DEFAULT 0,
|
version INTEGER NOT NULL DEFAULT 0,
|
||||||
updated_by TEXT REFERENCES users(user_id),
|
updated_by TEXT REFERENCES users(user_id) ON DELETE SET NULL,
|
||||||
updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
|
updated_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')
|
||||||
);
|
);
|
||||||
|
|
||||||
@@ -124,7 +124,7 @@ CREATE TABLE change_requests (
|
|||||||
request_id TEXT PRIMARY KEY,
|
request_id TEXT PRIMARY KEY,
|
||||||
workspace_id TEXT NOT NULL REFERENCES user_workspaces(workspace_id) ON DELETE CASCADE,
|
workspace_id TEXT NOT NULL REFERENCES user_workspaces(workspace_id) ON DELETE CASCADE,
|
||||||
project_id TEXT NOT NULL REFERENCES projects(project_id),
|
project_id TEXT NOT NULL REFERENCES projects(project_id),
|
||||||
submitted_by TEXT NOT NULL REFERENCES users(user_id),
|
submitted_by TEXT REFERENCES users(user_id) ON DELETE SET NULL,
|
||||||
status TEXT NOT NULL DEFAULT 'pending'
|
status TEXT NOT NULL DEFAULT 'pending'
|
||||||
CHECK (status IN ('pending', 'approved', 'rejected', 'merged')),
|
CHECK (status IN ('pending', 'approved', 'rejected', 'merged')),
|
||||||
diff_summary TEXT,
|
diff_summary TEXT,
|
||||||
@@ -140,7 +140,7 @@ CREATE TABLE change_requests (
|
|||||||
CREATE TABLE reviews (
|
CREATE TABLE reviews (
|
||||||
review_id SERIAL PRIMARY KEY,
|
review_id SERIAL PRIMARY KEY,
|
||||||
request_id TEXT NOT NULL REFERENCES change_requests(request_id) ON DELETE CASCADE,
|
request_id TEXT NOT NULL REFERENCES change_requests(request_id) ON DELETE CASCADE,
|
||||||
reviewer_id TEXT NOT NULL REFERENCES users(user_id),
|
reviewer_id TEXT REFERENCES users(user_id) ON DELETE SET NULL,
|
||||||
decision TEXT NOT NULL CHECK (decision IN ('approved', 'rejected')),
|
decision TEXT NOT NULL CHECK (decision IN ('approved', 'rejected')),
|
||||||
comments TEXT,
|
comments TEXT,
|
||||||
created_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
|
created_at TEXT NOT NULL DEFAULT to_char(now() at time zone 'utc', 'YYYY-MM-DD"T"HH24:MI:SS"Z"'),
|
||||||
@@ -171,7 +171,7 @@ CREATE INDEX idx_snapshots_cleanup ON snapshots (project_id, user_id, created_at
|
|||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
CREATE TABLE audit_log (
|
CREATE TABLE audit_log (
|
||||||
entry_id SERIAL PRIMARY KEY,
|
entry_id SERIAL PRIMARY KEY,
|
||||||
user_id TEXT NOT NULL REFERENCES users(user_id),
|
user_id TEXT REFERENCES users(user_id) ON DELETE SET NULL,
|
||||||
agent_id TEXT NOT NULL DEFAULT 'cli',
|
agent_id TEXT NOT NULL DEFAULT 'cli',
|
||||||
session_id TEXT,
|
session_id TEXT,
|
||||||
project_id TEXT REFERENCES projects(project_id) ON DELETE SET NULL,
|
project_id TEXT REFERENCES projects(project_id) ON DELETE SET NULL,
|
||||||
|
|||||||
+368
-182
@@ -3,6 +3,7 @@ ctxd server — dual-protocol daemon serving context over MCP + HTTP.
|
|||||||
"""
|
"""
|
||||||
import asyncio
|
import asyncio
|
||||||
import base64
|
import base64
|
||||||
|
import contextvars
|
||||||
import hashlib
|
import hashlib
|
||||||
import html
|
import html
|
||||||
import json
|
import json
|
||||||
@@ -44,6 +45,15 @@ WRITE_MCP_TOOLS = {
|
|||||||
"sync_to_project",
|
"sync_to_project",
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Per-request OAuth scopes for the public MCP connector (/mcp).
|
||||||
|
_oauth_mcp_ctx: contextvars.ContextVar[dict] = contextvars.ContextVar(
|
||||||
|
"oauth_mcp_ctx",
|
||||||
|
default={"has_read": False, "has_write": False, "user_id": "oauth"},
|
||||||
|
)
|
||||||
|
|
||||||
|
# Streamable HTTP MCP paths. Canonical public URL is /mcp; aliases for older configs.
|
||||||
|
MCP_STREAMABLE_PATHS = frozenset({"/mcp", "/readonly/mcp", "/oauth/mcp"})
|
||||||
|
|
||||||
|
|
||||||
def _b64url_sha256(value: str) -> str:
|
def _b64url_sha256(value: str) -> str:
|
||||||
digest = hashlib.sha256(value.encode()).digest()
|
digest = hashlib.sha256(value.encode()).digest()
|
||||||
@@ -57,6 +67,61 @@ def _now() -> int:
|
|||||||
# Claude remote MCP OAuth redirect (Dynamic Client Registration default)
|
# Claude remote MCP OAuth redirect (Dynamic Client Registration default)
|
||||||
CLAUDE_MCP_REDIRECT_URI = "https://claude.ai/api/mcp/auth_callback"
|
CLAUDE_MCP_REDIRECT_URI = "https://claude.ai/api/mcp/auth_callback"
|
||||||
|
|
||||||
|
OAUTH_SCOPES_ALLOWED = frozenset({"ctxd.read", "ctxd.write"})
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_client_scope(payload: dict, *, default: str = "ctxd.read") -> str:
|
||||||
|
"""Return space-separated allowed scopes from payload scope or scopes[]."""
|
||||||
|
parts: list[str] = []
|
||||||
|
if isinstance(payload.get("scopes"), list):
|
||||||
|
parts = [str(s).strip() for s in payload["scopes"] if str(s).strip() in OAUTH_SCOPES_ALLOWED]
|
||||||
|
else:
|
||||||
|
raw = payload.get("scope", default)
|
||||||
|
if isinstance(raw, list):
|
||||||
|
parts = [str(s).strip() for s in raw if str(s).strip() in OAUTH_SCOPES_ALLOWED]
|
||||||
|
else:
|
||||||
|
parts = [s for s in str(raw).split() if s in OAUTH_SCOPES_ALLOWED]
|
||||||
|
if not parts:
|
||||||
|
raise ValueError("at least one scope required: ctxd.read, ctxd.write")
|
||||||
|
order = {"ctxd.read": 0, "ctxd.write": 1}
|
||||||
|
return " ".join(sorted(set(parts), key=lambda x: order.get(x, 9)))
|
||||||
|
|
||||||
|
|
||||||
|
def intersect_oauth_scopes(client: dict, requested: str) -> str:
|
||||||
|
"""Grant only scopes both requested and allowed on the client registration."""
|
||||||
|
allowed = {s for s in (client.get("scope") or "ctxd.read").split() if s in OAUTH_SCOPES_ALLOWED}
|
||||||
|
if not allowed:
|
||||||
|
allowed = {"ctxd.read"}
|
||||||
|
req = {s for s in (requested or "").split() if s in OAUTH_SCOPES_ALLOWED}
|
||||||
|
if not req:
|
||||||
|
req = allowed
|
||||||
|
granted = req & allowed
|
||||||
|
if not granted:
|
||||||
|
return ""
|
||||||
|
order = {"ctxd.read": 0, "ctxd.write": 1}
|
||||||
|
return " ".join(sorted(granted, key=lambda x: order.get(x, 9)))
|
||||||
|
|
||||||
|
|
||||||
|
def oauth_protected_resource_document(base: str, *, resource_path: str = "/mcp") -> dict:
|
||||||
|
"""RFC 9728 / MCP authorization discovery document."""
|
||||||
|
resource = f"{base.rstrip('/')}{resource_path}"
|
||||||
|
return {
|
||||||
|
"resource": resource,
|
||||||
|
"authorization_servers": [base.rstrip("/")],
|
||||||
|
"bearer_methods_supported": ["header"],
|
||||||
|
"scopes_supported": ["ctxd.read", "ctxd.write"],
|
||||||
|
"resource_documentation": f"{base.rstrip('/')}/",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def oauth_protected_resource_metadata_url(base: str, *, mcp_path: str = "/mcp") -> str:
|
||||||
|
"""Well-known URL for the MCP resource (RFC 9728 path insertion)."""
|
||||||
|
base = base.rstrip("/")
|
||||||
|
if mcp_path in ("", "/"):
|
||||||
|
return f"{base}/.well-known/oauth-protected-resource"
|
||||||
|
suffix = mcp_path.lstrip("/")
|
||||||
|
return f"{base}/.well-known/oauth-protected-resource/{suffix}"
|
||||||
|
|
||||||
|
|
||||||
def _is_public_oauth_as_route(path: str, method: str) -> bool:
|
def _is_public_oauth_as_route(path: str, method: str) -> bool:
|
||||||
"""OAuth authorization-server endpoints (public). Admin /oauth/clients stays on HTTPServer."""
|
"""OAuth authorization-server endpoints (public). Admin /oauth/clients stays on HTTPServer."""
|
||||||
@@ -110,8 +175,12 @@ class OAuthStore:
|
|||||||
redirect_uris = payload.get("redirect_uris") or []
|
redirect_uris = payload.get("redirect_uris") or []
|
||||||
if not isinstance(redirect_uris, list) or not redirect_uris:
|
if not isinstance(redirect_uris, list) or not redirect_uris:
|
||||||
raise ValueError("redirect_uris required")
|
raise ValueError("redirect_uris required")
|
||||||
|
reg = dict(payload)
|
||||||
|
if "scope" not in reg and "scopes" not in reg:
|
||||||
|
reg["scope"] = "ctxd.read ctxd.write"
|
||||||
client_id = "ctxd_" + secrets.token_urlsafe(24)
|
client_id = "ctxd_" + secrets.token_urlsafe(24)
|
||||||
client_secret = secrets.token_urlsafe(32)
|
client_secret = secrets.token_urlsafe(32)
|
||||||
|
scope_str = normalize_client_scope(reg)
|
||||||
client = {
|
client = {
|
||||||
"client_id": client_id,
|
"client_id": client_id,
|
||||||
"client_secret": client_secret,
|
"client_secret": client_secret,
|
||||||
@@ -122,7 +191,7 @@ class OAuthStore:
|
|||||||
"grant_types": payload.get("grant_types", ["authorization_code", "refresh_token"]),
|
"grant_types": payload.get("grant_types", ["authorization_code", "refresh_token"]),
|
||||||
"response_types": payload.get("response_types", ["code"]),
|
"response_types": payload.get("response_types", ["code"]),
|
||||||
"token_endpoint_auth_method": payload.get("token_endpoint_auth_method", "client_secret_post"),
|
"token_endpoint_auth_method": payload.get("token_endpoint_auth_method", "client_secret_post"),
|
||||||
"scope": payload.get("scope", "ctxd.read"),
|
"scope": scope_str,
|
||||||
}
|
}
|
||||||
self.state.setdefault("clients", {})[client_id] = client
|
self.state.setdefault("clients", {})[client_id] = client
|
||||||
self._save()
|
self._save()
|
||||||
@@ -159,6 +228,19 @@ class OAuthStore:
|
|||||||
self._save()
|
self._save()
|
||||||
return True
|
return True
|
||||||
|
|
||||||
|
def update_client(self, client_id: str, payload: dict) -> dict | None:
|
||||||
|
"""Update admin-editable client fields (scope, client_name)."""
|
||||||
|
clients = self.state.get("clients", {})
|
||||||
|
client = clients.get(client_id)
|
||||||
|
if not client:
|
||||||
|
return None
|
||||||
|
if "client_name" in payload and payload["client_name"]:
|
||||||
|
client["client_name"] = str(payload["client_name"]).strip()
|
||||||
|
if "scope" in payload or "scopes" in payload:
|
||||||
|
client["scope"] = normalize_client_scope(payload)
|
||||||
|
self._save()
|
||||||
|
return {k: v for k, v in client.items() if k != "client_secret"}
|
||||||
|
|
||||||
def create_code(self, *, client_id: str, redirect_uri: str, code_challenge: str, scope: str, resource: str | None, approved_by_user_id: str | None = None) -> str:
|
def create_code(self, *, client_id: str, redirect_uri: str, code_challenge: str, scope: str, resource: str | None, approved_by_user_id: str | None = None) -> str:
|
||||||
code = secrets.token_urlsafe(32)
|
code = secrets.token_urlsafe(32)
|
||||||
self.state.setdefault("codes", {})[code] = {
|
self.state.setdefault("codes", {})[code] = {
|
||||||
@@ -243,6 +325,20 @@ class OAuthStore:
|
|||||||
return False, None
|
return False, None
|
||||||
return True, record.get("approved_by_user_id")
|
return True, record.get("approved_by_user_id")
|
||||||
|
|
||||||
|
def resolve_public_oauth(self, token: str) -> tuple[bool, bool, bool, str | None]:
|
||||||
|
"""Valid OAuth access token for public MCP. Returns (ok, has_read, has_write, user_id)."""
|
||||||
|
if not token:
|
||||||
|
return False, False, False, None
|
||||||
|
record = self.state.get("access_tokens", {}).get(token)
|
||||||
|
if not record or record.get("expires_at", 0) <= _now():
|
||||||
|
return False, False, False, None
|
||||||
|
scope = record.get("scope", "")
|
||||||
|
has_read = "ctxd.read" in scope
|
||||||
|
has_write = "ctxd.write" in scope
|
||||||
|
if not has_read and not has_write:
|
||||||
|
return False, False, False, None
|
||||||
|
return True, has_read, has_write, record.get("approved_by_user_id")
|
||||||
|
|
||||||
|
|
||||||
class WebSessionStore:
|
class WebSessionStore:
|
||||||
"""Opaque bearer sessions for CTXD Web UI per-user login."""
|
"""Opaque bearer sessions for CTXD Web UI per-user login."""
|
||||||
@@ -331,9 +427,12 @@ def _public_user_row(user: dict) -> dict:
|
|||||||
|
|
||||||
# ── MCP Server ─────────────────────────────────────────────────────────────────
|
# ── MCP Server ─────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
def make_mcp_server(cfg: CtxConfig, readonly: bool = False):
|
def make_mcp_server(cfg: CtxConfig, readonly: bool = False, oauth_scoped: bool = False):
|
||||||
"""Create an MCP server instance wired to our database."""
|
"""Create an MCP server instance wired to our database."""
|
||||||
app = Server("ctxd-readonly" if readonly else "context-dossier")
|
if oauth_scoped:
|
||||||
|
app = Server("ctxd-oauth")
|
||||||
|
else:
|
||||||
|
app = Server("ctxd-readonly" if readonly else "context-dossier")
|
||||||
|
|
||||||
def _conn():
|
def _conn():
|
||||||
"""Short-lived connection per request (WAL allows concurrent reads)."""
|
"""Short-lived connection per request (WAL allows concurrent reads)."""
|
||||||
@@ -503,17 +602,50 @@ def make_mcp_server(cfg: CtxConfig, readonly: bool = False):
|
|||||||
},
|
},
|
||||||
),
|
),
|
||||||
]
|
]
|
||||||
|
if oauth_scoped:
|
||||||
|
ctx = _oauth_mcp_ctx.get()
|
||||||
|
allowed: set[str] = set()
|
||||||
|
if ctx.get("has_read"):
|
||||||
|
allowed |= READONLY_MCP_TOOLS
|
||||||
|
if ctx.get("has_write"):
|
||||||
|
allowed |= WRITE_MCP_TOOLS
|
||||||
|
if not allowed:
|
||||||
|
allowed = {"get_client_guide"}
|
||||||
|
return [tool for tool in tools if tool.name in allowed]
|
||||||
if readonly:
|
if readonly:
|
||||||
return [tool for tool in tools if tool.name in READONLY_MCP_TOOLS]
|
return [tool for tool in tools if tool.name in READONLY_MCP_TOOLS]
|
||||||
return tools
|
return tools
|
||||||
|
|
||||||
@app.call_tool()
|
@app.call_tool()
|
||||||
async def call_tool(name: str, arguments: dict):
|
async def call_tool(name: str, arguments: dict):
|
||||||
if readonly and name not in READONLY_MCP_TOOLS:
|
if oauth_scoped:
|
||||||
|
ctx = _oauth_mcp_ctx.get()
|
||||||
|
if name in WRITE_MCP_TOOLS and not ctx.get("has_write"):
|
||||||
|
return [types.TextContent(
|
||||||
|
type="text",
|
||||||
|
text=json.dumps({
|
||||||
|
"error": "forbidden",
|
||||||
|
"tool": name,
|
||||||
|
"message": "ctxd.write scope required",
|
||||||
|
}, indent=2),
|
||||||
|
)]
|
||||||
|
if name in READONLY_MCP_TOOLS and not ctx.get("has_read"):
|
||||||
|
return [types.TextContent(
|
||||||
|
type="text",
|
||||||
|
text=json.dumps({
|
||||||
|
"error": "forbidden",
|
||||||
|
"tool": name,
|
||||||
|
"message": "ctxd.read scope required",
|
||||||
|
}, indent=2),
|
||||||
|
)]
|
||||||
|
actor = ctx.get("user_id") or "oauth"
|
||||||
|
elif readonly and name not in READONLY_MCP_TOOLS:
|
||||||
return [types.TextContent(
|
return [types.TextContent(
|
||||||
type="text",
|
type="text",
|
||||||
text=json.dumps({"error": "forbidden", "tool": name, "message": "read-only endpoint"}, indent=2),
|
text=json.dumps({"error": "forbidden", "tool": name, "message": "read-only endpoint"}, indent=2),
|
||||||
)]
|
)]
|
||||||
|
else:
|
||||||
|
actor = "hermes-gateway"
|
||||||
|
|
||||||
conn = _conn()
|
conn = _conn()
|
||||||
try:
|
try:
|
||||||
@@ -536,7 +668,7 @@ def make_mcp_server(cfg: CtxConfig, readonly: bool = False):
|
|||||||
"updated_at": ctx.get("updated_at"),
|
"updated_at": ctx.get("updated_at"),
|
||||||
"content": ctx["content"],
|
"content": ctx["content"],
|
||||||
}
|
}
|
||||||
_db.audit_log(conn, "hermes-gateway", "read",
|
_db.audit_log(conn, actor, "read",
|
||||||
f"MCP read context for {pid}",
|
f"MCP read context for {pid}",
|
||||||
agent_id="hermes", project_id=pid,
|
agent_id="hermes", project_id=pid,
|
||||||
entity_type="project", entity_id=pid)
|
entity_type="project", entity_id=pid)
|
||||||
@@ -554,7 +686,7 @@ def make_mcp_server(cfg: CtxConfig, readonly: bool = False):
|
|||||||
query = arguments["query"]
|
query = arguments["query"]
|
||||||
limit = arguments.get("limit", 10)
|
limit = arguments.get("limit", 10)
|
||||||
results = _db.search(conn, query, limit=limit)
|
results = _db.search(conn, query, limit=limit)
|
||||||
_db.audit_log(conn, "hermes-gateway", "search",
|
_db.audit_log(conn, actor, "search",
|
||||||
f"MCP search: {query[:80]}",
|
f"MCP search: {query[:80]}",
|
||||||
agent_id="hermes")
|
agent_id="hermes")
|
||||||
conn.commit()
|
conn.commit()
|
||||||
@@ -584,7 +716,7 @@ def make_mcp_server(cfg: CtxConfig, readonly: bool = False):
|
|||||||
pid = arguments["project_id"]
|
pid = arguments["project_id"]
|
||||||
tags = [str(t).upper().replace(" ", "-") for t in arguments.get("tags", [])]
|
tags = [str(t).upper().replace(" ", "-") for t in arguments.get("tags", [])]
|
||||||
_db.project_set_tags(conn, pid, tags)
|
_db.project_set_tags(conn, pid, tags)
|
||||||
_db.audit_log(conn, "hermes-gateway", "set_tags",
|
_db.audit_log(conn, actor, "set_tags",
|
||||||
f"Set tags for {pid}: {', '.join(tags)}",
|
f"Set tags for {pid}: {', '.join(tags)}",
|
||||||
agent_id="hermes", project_id=pid,
|
agent_id="hermes", project_id=pid,
|
||||||
entity_type="project", entity_id=pid)
|
entity_type="project", entity_id=pid)
|
||||||
@@ -625,7 +757,7 @@ def make_mcp_server(cfg: CtxConfig, readonly: bool = False):
|
|||||||
result = _db.file_read(conn, pid, file_path)
|
result = _db.file_read(conn, pid, file_path)
|
||||||
if result is None:
|
if result is None:
|
||||||
return [types.TextContent(type="text", text=f"File '{file_path}' not found in project '{pid}'")]
|
return [types.TextContent(type="text", text=f"File '{file_path}' not found in project '{pid}'")]
|
||||||
_db.audit_log(conn, "hermes-gateway", "read",
|
_db.audit_log(conn, actor, "read",
|
||||||
f"MCP read file {file_path} for {pid}",
|
f"MCP read file {file_path} for {pid}",
|
||||||
agent_id="hermes", project_id=pid,
|
agent_id="hermes", project_id=pid,
|
||||||
entity_type="file", entity_id=file_path)
|
entity_type="file", entity_id=file_path)
|
||||||
@@ -649,7 +781,7 @@ def make_mcp_server(cfg: CtxConfig, readonly: bool = False):
|
|||||||
file_path = arguments["file_path"]
|
file_path = arguments["file_path"]
|
||||||
content = arguments["content"]
|
content = arguments["content"]
|
||||||
base_version = arguments["base_version"]
|
base_version = arguments["base_version"]
|
||||||
result = _db.file_update(conn, pid, file_path, content, "hermes-gateway", base_version)
|
result = _db.file_update(conn, pid, file_path, content, actor, base_version)
|
||||||
if result.get("ok"):
|
if result.get("ok"):
|
||||||
conn.commit()
|
conn.commit()
|
||||||
return [types.TextContent(
|
return [types.TextContent(
|
||||||
@@ -664,111 +796,6 @@ def make_mcp_server(cfg: CtxConfig, readonly: bool = False):
|
|||||||
return app
|
return app
|
||||||
|
|
||||||
|
|
||||||
def make_write_mcp_server(cfg: CtxConfig):
|
|
||||||
"""Create an MCP server that exposes only write tools (OAuth ctxd.write scope)."""
|
|
||||||
app = Server("ctxd-write")
|
|
||||||
|
|
||||||
def _conn():
|
|
||||||
return _db.init_db(cfg)
|
|
||||||
|
|
||||||
@app.list_tools()
|
|
||||||
async def list_tools():
|
|
||||||
tools = [
|
|
||||||
types.Tool(
|
|
||||||
name="update_file",
|
|
||||||
description="Update a single context file in a project with optimistic version checking",
|
|
||||||
inputSchema={
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"project_id": {"type": "string", "description": "Project slug"},
|
|
||||||
"file_path": {"type": "string", "description": "File name e.g. CONTEXT.md"},
|
|
||||||
"content": {"type": "string", "description": "New file content (markdown)"},
|
|
||||||
"base_version": {"type": "integer", "description": "Current version of the file (for conflict detection)"},
|
|
||||||
},
|
|
||||||
"required": ["project_id", "file_path", "content", "base_version"],
|
|
||||||
},
|
|
||||||
),
|
|
||||||
types.Tool(
|
|
||||||
name="set_project_tags",
|
|
||||||
description="Set metadata tags for a project (replaces all tags)",
|
|
||||||
inputSchema={
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"project_id": {"type": "string", "description": "Project slug"},
|
|
||||||
"tags": {"type": "array", "items": {"type": "string"}, "description": "Uppercase metadata tags"},
|
|
||||||
},
|
|
||||||
"required": ["project_id", "tags"],
|
|
||||||
},
|
|
||||||
),
|
|
||||||
types.Tool(
|
|
||||||
name="sync_to_project",
|
|
||||||
description="Write current shared context as AGENTS.md + symlinks to the project root",
|
|
||||||
inputSchema={
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"project_id": {"type": "string", "description": "Project slug"},
|
|
||||||
},
|
|
||||||
"required": ["project_id"],
|
|
||||||
},
|
|
||||||
),
|
|
||||||
types.Tool(
|
|
||||||
name="get_client_guide",
|
|
||||||
description="Return the CTXD client guide (LLM-CLIENT.MD) — always read this first. Covers OAuth, MCP tools, read/write endpoints, version-checked updates, and error handling.",
|
|
||||||
inputSchema={
|
|
||||||
"type": "object",
|
|
||||||
"properties": {},
|
|
||||||
},
|
|
||||||
),
|
|
||||||
]
|
|
||||||
return tools
|
|
||||||
|
|
||||||
@app.call_tool()
|
|
||||||
async def call_tool(name: str, arguments: dict):
|
|
||||||
conn = _conn()
|
|
||||||
try:
|
|
||||||
if name == "get_client_guide":
|
|
||||||
result = _db.file_read(conn, "ctxd-docs", "LLM-CLIENT.MD")
|
|
||||||
if result is None:
|
|
||||||
return [types.TextContent(type="text", text="Client guide not found — ctxd-docs/LLM-CLIENT.MD is missing")]
|
|
||||||
return [types.TextContent(
|
|
||||||
type="text",
|
|
||||||
text=json.dumps(result, indent=2),
|
|
||||||
)]
|
|
||||||
|
|
||||||
elif name == "update_file":
|
|
||||||
pid = arguments["project_id"]
|
|
||||||
file_path = arguments["file_path"]
|
|
||||||
content = arguments["content"]
|
|
||||||
base_version = arguments.get("base_version", 0)
|
|
||||||
result = _db.file_update(conn, pid, file_path, content, base_version, updated_by="oauth-write")
|
|
||||||
conn.commit()
|
|
||||||
return [types.TextContent(type="text", text=json.dumps(result, indent=2))]
|
|
||||||
|
|
||||||
elif name == "set_project_tags":
|
|
||||||
pid = arguments["project_id"]
|
|
||||||
tags = [str(t).upper().replace(" ", "-") for t in arguments.get("tags", [])]
|
|
||||||
_db.project_set_tags(conn, pid, tags)
|
|
||||||
_db.audit_log(conn, "oauth-write", "set_tags",
|
|
||||||
f"Set tags for {pid}: {', '.join(tags)}",
|
|
||||||
agent_id="oauth", project_id=pid,
|
|
||||||
entity_type="project", entity_id=pid)
|
|
||||||
conn.commit()
|
|
||||||
return [types.TextContent(type="text", text=json.dumps({"ok": True, "project_id": pid, "tags": tags}, indent=2))]
|
|
||||||
|
|
||||||
elif name == "sync_to_project":
|
|
||||||
pid = arguments["project_id"]
|
|
||||||
result = _db.sync_to_project(conn, pid)
|
|
||||||
conn.commit()
|
|
||||||
return [types.TextContent(type="text", text=json.dumps(result, indent=2))]
|
|
||||||
|
|
||||||
else:
|
|
||||||
return [types.TextContent(type="text", text=json.dumps({"error": "unknown tool", "tool": name}, indent=2))]
|
|
||||||
finally:
|
|
||||||
conn.close()
|
|
||||||
|
|
||||||
return app
|
|
||||||
|
|
||||||
|
|
||||||
# ── HTTP Server (stdlib-only, no dependencies) ────────────────────────────────
|
# ── HTTP Server (stdlib-only, no dependencies) ────────────────────────────────
|
||||||
|
|
||||||
class HTTPServer:
|
class HTTPServer:
|
||||||
@@ -785,6 +812,17 @@ class HTTPServer:
|
|||||||
try:
|
try:
|
||||||
return self._route(method, path, body, auth or {})
|
return self._route(method, path, body, auth or {})
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
# The shared PG connection is left in an aborted-transaction state by
|
||||||
|
# any failed statement; without this rollback every subsequent request
|
||||||
|
# 500s ("current transaction is aborted"). This is the single funnel
|
||||||
|
# that guarantees the connection is clean no matter which path failed.
|
||||||
|
try:
|
||||||
|
self._conn.rollback()
|
||||||
|
except Exception:
|
||||||
|
logger.exception("Rollback failed after request error")
|
||||||
|
if _db.is_integrity_error(e):
|
||||||
|
return (409, {"Content-Type": "application/json"},
|
||||||
|
json.dumps({"error": "conflict", "detail": str(e)}))
|
||||||
logger.exception("HTTP error")
|
logger.exception("HTTP error")
|
||||||
return (500, {"Content-Type": "text/plain"}, f"Internal error: {e}")
|
return (500, {"Content-Type": "text/plain"}, f"Internal error: {e}")
|
||||||
|
|
||||||
@@ -911,10 +949,9 @@ class HTTPServer:
|
|||||||
return (200, {"Content-Type": "application/json"}, json.dumps({"ok": True, "user_id": user_id}))
|
return (200, {"Content-Type": "application/json"}, json.dumps({"ok": True, "user_id": user_id}))
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
self._conn.rollback()
|
self._conn.rollback()
|
||||||
msg = str(e)
|
if _db.is_integrity_error(e):
|
||||||
if "UNIQUE" in msg or "idx_users_lower" in msg:
|
|
||||||
return (409, {"Content-Type": "application/json"}, json.dumps({"error": "user already exists"}))
|
return (409, {"Content-Type": "application/json"}, json.dumps({"error": "user already exists"}))
|
||||||
return (400, {"Content-Type": "application/json"}, json.dumps({"error": msg}))
|
return (400, {"Content-Type": "application/json"}, json.dumps({"error": str(e)}))
|
||||||
|
|
||||||
# POST /users/<id>/password — set or reset password (admin only)
|
# POST /users/<id>/password — set or reset password (admin only)
|
||||||
if method == "POST" and path.startswith("/users/") and path.endswith("/password"):
|
if method == "POST" and path.startswith("/users/") and path.endswith("/password"):
|
||||||
@@ -973,6 +1010,7 @@ class HTTPServer:
|
|||||||
return (400, {"Content-Type": "application/json"}, json.dumps({"error": "cannot delete your own account"}))
|
return (400, {"Content-Type": "application/json"}, json.dumps({"error": "cannot delete your own account"}))
|
||||||
result = _db.user_delete(self._conn, uid)
|
result = _db.user_delete(self._conn, uid)
|
||||||
if not result.get("ok"):
|
if not result.get("ok"):
|
||||||
|
self._conn.rollback()
|
||||||
status = 404 if result.get("error") == "not_found" else 409
|
status = 404 if result.get("error") == "not_found" else 409
|
||||||
return (status, {"Content-Type": "application/json"}, json.dumps(result))
|
return (status, {"Content-Type": "application/json"}, json.dumps(result))
|
||||||
actor = self._auth_user_id(auth)
|
actor = self._auth_user_id(auth)
|
||||||
@@ -1003,6 +1041,8 @@ class HTTPServer:
|
|||||||
return (200, {"Content-Type": "application/json"}, json.dumps({"ok": True, "project_id": pid}))
|
return (200, {"Content-Type": "application/json"}, json.dumps({"ok": True, "project_id": pid}))
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
self._conn.rollback()
|
self._conn.rollback()
|
||||||
|
if _db.is_integrity_error(e):
|
||||||
|
return (409, {"Content-Type": "application/json"}, json.dumps({"error": "project already exists"}))
|
||||||
return (400, {"Content-Type": "application/json"}, json.dumps({"error": str(e)}))
|
return (400, {"Content-Type": "application/json"}, json.dumps({"error": str(e)}))
|
||||||
|
|
||||||
# DELETE /projects/<id> — delete a project (admin only)
|
# DELETE /projects/<id> — delete a project (admin only)
|
||||||
@@ -1245,10 +1285,14 @@ class HTTPServer:
|
|||||||
if isinstance(redirect_uris, str):
|
if isinstance(redirect_uris, str):
|
||||||
redirect_uris = [redirect_uris]
|
redirect_uris = [redirect_uris]
|
||||||
try:
|
try:
|
||||||
|
reg_payload = dict(payload)
|
||||||
|
if "scope" not in reg_payload and "scopes" not in reg_payload:
|
||||||
|
reg_payload.setdefault("scope", "ctxd.read ctxd.write")
|
||||||
client = self.oauth_store.register_client({
|
client = self.oauth_store.register_client({
|
||||||
"client_name": payload.get("client_name") or "MCP connector",
|
"client_name": payload.get("client_name") or "MCP connector",
|
||||||
"redirect_uris": redirect_uris,
|
"redirect_uris": redirect_uris,
|
||||||
"scope": payload.get("scope") or "ctxd.read",
|
"scope": reg_payload.get("scope"),
|
||||||
|
"scopes": reg_payload.get("scopes"),
|
||||||
})
|
})
|
||||||
except ValueError as e:
|
except ValueError as e:
|
||||||
return (400, {"Content-Type": "application/json"}, json.dumps({"error": str(e)}))
|
return (400, {"Content-Type": "application/json"}, json.dumps({"error": str(e)}))
|
||||||
@@ -1261,7 +1305,7 @@ class HTTPServer:
|
|||||||
self._conn.commit()
|
self._conn.commit()
|
||||||
issuer = (self.cfg.oauth_issuer or "").rstrip("/") or "https://ctxd.cubecraftcreations.com"
|
issuer = (self.cfg.oauth_issuer or "").rstrip("/") or "https://ctxd.cubecraftcreations.com"
|
||||||
out = {k: v for k, v in client.items()}
|
out = {k: v for k, v in client.items()}
|
||||||
out["connector_url"] = f"{issuer}/readonly/mcp"
|
out["connector_url"] = f"{issuer}/mcp"
|
||||||
out["authorization_server"] = issuer
|
out["authorization_server"] = issuer
|
||||||
return (201, {"Content-Type": "application/json"}, json.dumps(out))
|
return (201, {"Content-Type": "application/json"}, json.dumps(out))
|
||||||
|
|
||||||
@@ -1285,6 +1329,31 @@ class HTTPServer:
|
|||||||
self._conn.commit()
|
self._conn.commit()
|
||||||
return (200, {"Content-Type": "application/json"}, json.dumps({"ok": True, "client_id": client_id}))
|
return (200, {"Content-Type": "application/json"}, json.dumps({"ok": True, "client_id": client_id}))
|
||||||
|
|
||||||
|
# PATCH /oauth/clients/<client_id> — update scopes / name (admin only)
|
||||||
|
if method == "PATCH" and path.startswith("/oauth/clients/"):
|
||||||
|
if auth.get("type") == "session" and not self._is_admin(auth):
|
||||||
|
return (403, {"Content-Type": "application/json"}, json.dumps({"error": "admin required"}))
|
||||||
|
if not self.oauth_store:
|
||||||
|
return (503, {"Content-Type": "application/json"}, json.dumps({"error": "oauth unavailable"}))
|
||||||
|
client_id = path[len("/oauth/clients/"):].strip()
|
||||||
|
if not client_id:
|
||||||
|
return (400, {"Content-Type": "application/json"}, json.dumps({"error": "client_id required"}))
|
||||||
|
payload = json.loads(body or b"{}")
|
||||||
|
try:
|
||||||
|
updated = self.oauth_store.update_client(client_id, payload)
|
||||||
|
except ValueError as e:
|
||||||
|
return (400, {"Content-Type": "application/json"}, json.dumps({"error": str(e)}))
|
||||||
|
if not updated:
|
||||||
|
return (404, {"Content-Type": "application/json"}, json.dumps({"error": "client not found"}))
|
||||||
|
actor = self._auth_user_id(auth)
|
||||||
|
_db.audit_log(
|
||||||
|
self._conn, actor, "update",
|
||||||
|
f"Updated OAuth client {client_id} scopes={updated.get('scope', '')}",
|
||||||
|
agent_id="oauth", entity_type="oauth_client", entity_id=client_id,
|
||||||
|
)
|
||||||
|
self._conn.commit()
|
||||||
|
return (200, {"Content-Type": "application/json"}, json.dumps(updated))
|
||||||
|
|
||||||
return (404, {"Content-Type": "text/plain"}, f"Not found: {method} {path}")
|
return (404, {"Content-Type": "text/plain"}, f"Not found: {method} {path}")
|
||||||
|
|
||||||
|
|
||||||
@@ -1300,11 +1369,11 @@ class CombinedApp:
|
|||||||
self.oauth_store = OAuthStore(cfg)
|
self.oauth_store = OAuthStore(cfg)
|
||||||
self.http_handler = HTTPServer(cfg, self.session_store, self.oauth_store)
|
self.http_handler = HTTPServer(cfg, self.session_store, self.oauth_store)
|
||||||
self.mcp_app = make_mcp_server(cfg)
|
self.mcp_app = make_mcp_server(cfg)
|
||||||
|
self.oauth_mcp_app = make_mcp_server(cfg, oauth_scoped=True)
|
||||||
self.readonly_mcp_app = make_mcp_server(cfg, readonly=True)
|
self.readonly_mcp_app = make_mcp_server(cfg, readonly=True)
|
||||||
self.write_mcp_app = make_write_mcp_server(cfg)
|
|
||||||
self._mcp_init_opts = self.mcp_app.create_initialization_options()
|
self._mcp_init_opts = self.mcp_app.create_initialization_options()
|
||||||
|
self._oauth_mcp_init_opts = self.oauth_mcp_app.create_initialization_options()
|
||||||
self._readonly_mcp_init_opts = self.readonly_mcp_app.create_initialization_options()
|
self._readonly_mcp_init_opts = self.readonly_mcp_app.create_initialization_options()
|
||||||
self._write_mcp_init_opts = self.write_mcp_app.create_initialization_options()
|
|
||||||
|
|
||||||
async def __call__(self, scope, receive, send):
|
async def __call__(self, scope, receive, send):
|
||||||
if scope["type"] == "http":
|
if scope["type"] == "http":
|
||||||
@@ -1371,12 +1440,7 @@ class CombinedApp:
|
|||||||
qs = urllib.parse.parse_qs(scope.get("query_string", b"").decode())
|
qs = urllib.parse.parse_qs(scope.get("query_string", b"").decode())
|
||||||
|
|
||||||
if method == "GET" and path.startswith("/.well-known/oauth-protected-resource"):
|
if method == "GET" and path.startswith("/.well-known/oauth-protected-resource"):
|
||||||
payload = {
|
payload = oauth_protected_resource_document(base)
|
||||||
"resource": f"{base}/readonly/mcp",
|
|
||||||
"authorization_servers": [issuer],
|
|
||||||
"bearer_methods_supported": ["header"],
|
|
||||||
"resource_documentation": f"{base}/",
|
|
||||||
}
|
|
||||||
await self._send_json(send, 200, payload)
|
await self._send_json(send, 200, payload)
|
||||||
return
|
return
|
||||||
|
|
||||||
@@ -1389,7 +1453,11 @@ class CombinedApp:
|
|||||||
"response_types_supported": ["code"],
|
"response_types_supported": ["code"],
|
||||||
"grant_types_supported": ["authorization_code", "refresh_token"],
|
"grant_types_supported": ["authorization_code", "refresh_token"],
|
||||||
"code_challenge_methods_supported": ["S256"],
|
"code_challenge_methods_supported": ["S256"],
|
||||||
"token_endpoint_auth_methods_supported": ["client_secret_post", "client_secret_basic", "none"],
|
"token_endpoint_auth_methods_supported": [
|
||||||
|
"client_secret_basic",
|
||||||
|
"client_secret_post",
|
||||||
|
"none",
|
||||||
|
],
|
||||||
"scopes_supported": ["ctxd.read", "ctxd.write"],
|
"scopes_supported": ["ctxd.read", "ctxd.write"],
|
||||||
}
|
}
|
||||||
await self._send_json(send, 200, payload)
|
await self._send_json(send, 200, payload)
|
||||||
@@ -1457,11 +1525,20 @@ class CombinedApp:
|
|||||||
if not ok_approve or not approver:
|
if not ok_approve or not approver:
|
||||||
await self._send_html(send, 403, "<h1>Forbidden</h1><p>Sign in to CTXD as an admin in this browser, or provide a valid OAuth approval key.</p>")
|
await self._send_html(send, 403, "<h1>Forbidden</h1><p>Sign in to CTXD as an admin in this browser, or provide a valid OAuth approval key.</p>")
|
||||||
return
|
return
|
||||||
|
granted_scope = intersect_oauth_scopes(client, scope_value)
|
||||||
|
if not granted_scope:
|
||||||
|
target = error_redirect("invalid_scope")
|
||||||
|
if target:
|
||||||
|
await send({"type": "http.response.start", "status": 302, "headers": [(b"location", target.encode())]})
|
||||||
|
await send({"type": "http.response.body", "body": b""})
|
||||||
|
else:
|
||||||
|
await self._send_json(send, 400, {"error": "invalid_scope"})
|
||||||
|
return
|
||||||
code = self.oauth_store.create_code(
|
code = self.oauth_store.create_code(
|
||||||
client_id=client_id,
|
client_id=client_id,
|
||||||
redirect_uri=redirect_uri,
|
redirect_uri=redirect_uri,
|
||||||
code_challenge=code_challenge,
|
code_challenge=code_challenge,
|
||||||
scope=scope_value,
|
scope=granted_scope,
|
||||||
resource=resource,
|
resource=resource,
|
||||||
approved_by_user_id=approver,
|
approved_by_user_id=approver,
|
||||||
)
|
)
|
||||||
@@ -1469,11 +1546,11 @@ class CombinedApp:
|
|||||||
self.http_handler._conn,
|
self.http_handler._conn,
|
||||||
approver,
|
approver,
|
||||||
"update",
|
"update",
|
||||||
f"Approved OAuth read-only access for {client.get('client_name', client_id)}",
|
f"Approved OAuth access for {client.get('client_name', client_id)}",
|
||||||
agent_id="oauth",
|
agent_id="oauth",
|
||||||
entity_type="oauth_client",
|
entity_type="oauth_client",
|
||||||
entity_id=client_id,
|
entity_id=client_id,
|
||||||
details={"redirect_uri": redirect_uri, "scope": scope_value},
|
details={"redirect_uri": redirect_uri, "scope": granted_scope},
|
||||||
)
|
)
|
||||||
self.http_handler._conn.commit()
|
self.http_handler._conn.commit()
|
||||||
q = {"code": code}
|
q = {"code": code}
|
||||||
@@ -1544,14 +1621,19 @@ class CombinedApp:
|
|||||||
break
|
break
|
||||||
return token
|
return token
|
||||||
|
|
||||||
async def _auth_error(status: int = 401, message: str = "unauthorized"):
|
async def _auth_error(status: int = 401, message: str = "unauthorized", *, mcp_resource: bool = False):
|
||||||
headers = [(b"content-type", b"application/json")]
|
headers = [(b"content-type", b"application/json")]
|
||||||
if self.cfg.oauth_enabled:
|
if self.cfg.oauth_enabled:
|
||||||
base = self._base_url(scope)
|
base = self._base_url(scope)
|
||||||
headers.append((
|
if mcp_resource:
|
||||||
b"www-authenticate",
|
meta_url = oauth_protected_resource_metadata_url(base, mcp_path="/mcp")
|
||||||
f'Bearer resource_metadata="{base}/.well-known/oauth-protected-resource"'.encode(),
|
else:
|
||||||
))
|
meta_url = oauth_protected_resource_metadata_url(base)
|
||||||
|
www = (
|
||||||
|
f'Bearer resource_metadata="{meta_url}", '
|
||||||
|
f'scope="ctxd.read ctxd.write"'
|
||||||
|
)
|
||||||
|
headers.append((b"www-authenticate", www.encode()))
|
||||||
await send({
|
await send({
|
||||||
"type": "http.response.start",
|
"type": "http.response.start",
|
||||||
"status": status,
|
"status": status,
|
||||||
@@ -1569,54 +1651,45 @@ class CombinedApp:
|
|||||||
await self._serve_oauth(scope, receive, send, oauth_body)
|
await self._serve_oauth(scope, receive, send, oauth_body)
|
||||||
return
|
return
|
||||||
|
|
||||||
# Public read-only MCP endpoint for Claude Desktop/web connectors.
|
# Streamable HTTP MCP — /mcp (OAuth read+write or API key full access on /mcp only).
|
||||||
# GET accepts OAuth bearer tokens. The legacy external_readonly_key query
|
if path in MCP_STREAMABLE_PATHS:
|
||||||
# fallback remains temporarily for migration away from ?key= URLs.
|
|
||||||
def _readonly_token_valid(token: str) -> bool:
|
|
||||||
if self.cfg.api_key and token == self.cfg.api_key:
|
|
||||||
return True
|
|
||||||
if self.cfg.external_readonly_key and token == self.cfg.external_readonly_key:
|
|
||||||
return True
|
|
||||||
if self.cfg.oauth_enabled and self.oauth_store.validate_access_token(token):
|
|
||||||
return True
|
|
||||||
return False
|
|
||||||
|
|
||||||
# Public read-only MCP (Streamable HTTP) — OAuth bearer or legacy readonly key.
|
|
||||||
if path == "/readonly/mcp":
|
|
||||||
token = _request_token()
|
token = _request_token()
|
||||||
if not _readonly_token_valid(token):
|
if path == "/mcp" and self.cfg.api_key and token == self.cfg.api_key:
|
||||||
await _auth_error()
|
await self._serve_streamable_mcp(
|
||||||
|
scope, receive, send,
|
||||||
|
self.mcp_app, self._mcp_init_opts,
|
||||||
|
surface_name="internal",
|
||||||
|
)
|
||||||
return
|
return
|
||||||
await self._serve_streamable_mcp(
|
ok, ctx = self._public_mcp_auth_context(token)
|
||||||
scope, receive, send,
|
if ok:
|
||||||
self.readonly_mcp_app, self._readonly_mcp_init_opts,
|
await self._serve_oauth_mcp_streamable(scope, receive, send, ctx)
|
||||||
)
|
return
|
||||||
|
await _auth_error(mcp_resource=True)
|
||||||
return
|
return
|
||||||
|
|
||||||
# Public write MCP (Streamable HTTP) — OAuth bearer with ctxd.write scope.
|
# Legacy SSE write path (prefer /mcp Streamable HTTP).
|
||||||
if path == "/write/mcp":
|
# Note: Using SSE instead of Streamable HTTP because MCP SDK 1.28's
|
||||||
|
# Streamable HTTP transport has a race condition where EventSourceResponse
|
||||||
|
# is killed by the task group before sending headers.
|
||||||
|
if method == "GET" and path == "/write/sse":
|
||||||
token = _request_token()
|
token = _request_token()
|
||||||
valid, _ = self.oauth_store.validate_write_token(token) if self.cfg.oauth_enabled else (False, None)
|
ok, ctx = self._public_mcp_auth_context(token)
|
||||||
if not valid:
|
if not ok:
|
||||||
await _auth_error()
|
await _auth_error()
|
||||||
return
|
return
|
||||||
await self._serve_streamable_mcp(
|
await self._serve_write_mcp_sse(scope, receive, send, ctx)
|
||||||
scope, receive, send,
|
|
||||||
self.write_mcp_app, self._write_mcp_init_opts,
|
|
||||||
)
|
|
||||||
return
|
return
|
||||||
|
|
||||||
# Internal full MCP (Streamable HTTP) — shared API key only.
|
if method == "POST" and path in ("/write/messages", "/write/messages/"):
|
||||||
if path == "/mcp":
|
|
||||||
token = _request_token()
|
token = _request_token()
|
||||||
if self.cfg.auth_enabled and token != self.cfg.api_key:
|
ok, ctx = self._public_mcp_auth_context(token)
|
||||||
|
if not ok:
|
||||||
await _auth_error()
|
await _auth_error()
|
||||||
return
|
return
|
||||||
await self._serve_streamable_mcp(
|
await self._serve_write_mcp_sse(scope, receive, send, ctx)
|
||||||
scope, receive, send,
|
|
||||||
self.mcp_app, self._mcp_init_opts,
|
|
||||||
)
|
|
||||||
return
|
return
|
||||||
|
|
||||||
token = _request_token()
|
token = _request_token()
|
||||||
|
|
||||||
def _resolve_http_auth(tok: str) -> tuple[bool, dict]:
|
def _resolve_http_auth(tok: str) -> tuple[bool, dict]:
|
||||||
@@ -1701,20 +1774,133 @@ class CombinedApp:
|
|||||||
"body": body_str.encode(),
|
"body": body_str.encode(),
|
||||||
})
|
})
|
||||||
|
|
||||||
async def _serve_streamable_mcp(self, scope, receive, send, mcp_app, init_opts):
|
def _public_mcp_auth_context(self, token: str) -> tuple[bool, dict]:
|
||||||
"""Streamable HTTP transport for MCP (single endpoint handles POST/GET/DELETE)."""
|
"""Resolve auth for the unified public MCP connector."""
|
||||||
import anyio
|
if self.cfg.api_key and token == self.cfg.api_key:
|
||||||
|
return True, {"has_read": True, "has_write": False, "user_id": "api-key"}
|
||||||
|
if self.cfg.external_readonly_key and token == self.cfg.external_readonly_key:
|
||||||
|
return True, {"has_read": True, "has_write": False, "user_id": "readonly-key"}
|
||||||
|
if self.cfg.oauth_enabled:
|
||||||
|
ok, has_read, has_write, uid = self.oauth_store.resolve_public_oauth(token)
|
||||||
|
if ok:
|
||||||
|
return True, {
|
||||||
|
"has_read": has_read,
|
||||||
|
"has_write": has_write,
|
||||||
|
"user_id": uid or "oauth",
|
||||||
|
}
|
||||||
|
return False, {}
|
||||||
|
|
||||||
|
async def _serve_oauth_mcp_streamable(self, scope, receive, send, ctx: dict):
|
||||||
|
reset = _oauth_mcp_ctx.set(ctx)
|
||||||
|
try:
|
||||||
|
await self._serve_streamable_mcp(
|
||||||
|
scope, receive, send,
|
||||||
|
self.oauth_mcp_app, self._oauth_mcp_init_opts,
|
||||||
|
surface_name="oauth",
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
_oauth_mcp_ctx.reset(reset)
|
||||||
|
|
||||||
|
async def _serve_write_mcp_sse(self, scope, receive, send, ctx: dict | None = None):
|
||||||
|
"""Write MCP over SSE transport (legacy; same app as /readonly/mcp)."""
|
||||||
|
from mcp.server.sse import SseServerTransport
|
||||||
|
|
||||||
|
if ctx is not None:
|
||||||
|
reset = _oauth_mcp_ctx.set(ctx)
|
||||||
|
else:
|
||||||
|
reset = None
|
||||||
|
|
||||||
|
try:
|
||||||
|
if not hasattr(self, "_write_sse_transport") or self._write_sse_transport is None:
|
||||||
|
self._write_sse_transport = SseServerTransport("/write/messages")
|
||||||
|
sse = self._write_sse_transport
|
||||||
|
|
||||||
|
method = scope.get("method", "GET")
|
||||||
|
path = scope.get("path", "/")
|
||||||
|
|
||||||
|
if method == "GET" and path == "/write/sse":
|
||||||
|
async with sse.connect_sse(scope, receive, send) as streams:
|
||||||
|
await self.oauth_mcp_app.run(
|
||||||
|
streams[0], streams[1], self._oauth_mcp_init_opts,
|
||||||
|
)
|
||||||
|
return
|
||||||
|
|
||||||
|
if method == "POST" and path in ("/write/messages", "/write/messages/"):
|
||||||
|
await sse.handle_post_message(scope, receive, send)
|
||||||
|
return
|
||||||
|
|
||||||
|
await send({
|
||||||
|
"type": "http.response.start",
|
||||||
|
"status": 404,
|
||||||
|
"headers": [(b"content-type", b"text/plain")],
|
||||||
|
})
|
||||||
|
await send({"type": "http.response.body", "body": b"Not found"})
|
||||||
|
finally:
|
||||||
|
if reset is not None:
|
||||||
|
_oauth_mcp_ctx.reset(reset)
|
||||||
|
|
||||||
|
async def _serve_streamable_mcp(self, scope, receive, send, mcp_app, init_opts, surface_name="mcp"):
|
||||||
|
"""Streamable HTTP transport for MCP with persistent session handling.
|
||||||
|
|
||||||
|
Transports are cached by session ID. The first request creates a
|
||||||
|
transport, starts the MCP server, and handles the request. Subsequent
|
||||||
|
requests with the same session ID reuse the transport.
|
||||||
|
"""
|
||||||
from mcp.server.streamable_http import StreamableHTTPServerTransport
|
from mcp.server.streamable_http import StreamableHTTPServerTransport
|
||||||
|
|
||||||
transport = StreamableHTTPServerTransport(mcp_session_id=None)
|
# Check for existing session ID from request headers
|
||||||
async with transport.connect() as (read_stream, write_stream):
|
req_headers = {k.decode().lower(): v.decode() for k, v in scope.get("headers", [])}
|
||||||
async def run_server():
|
session_id = req_headers.get("mcp-session-id", "")
|
||||||
await mcp_app.run(read_stream, write_stream, init_opts)
|
|
||||||
|
|
||||||
async with anyio.create_task_group() as tg:
|
# Get or create session cache for this surface
|
||||||
tg.start_soon(run_server)
|
cache_attr = f"_streamable_{surface_name}_sessions"
|
||||||
await transport.handle_request(scope, receive, send)
|
if not hasattr(self, cache_attr):
|
||||||
tg.cancel_scope.cancel()
|
setattr(self, cache_attr, {})
|
||||||
|
sessions = getattr(self, cache_attr)
|
||||||
|
|
||||||
|
# Reuse existing transport for this session
|
||||||
|
if session_id and session_id in sessions:
|
||||||
|
transport, server_task = sessions[session_id]
|
||||||
|
await transport.handle_request(scope, receive, send)
|
||||||
|
return
|
||||||
|
|
||||||
|
# New session — generate session ID, create transport, start server
|
||||||
|
import secrets as _secrets
|
||||||
|
import asyncio
|
||||||
|
new_session_id = _secrets.token_hex(16)
|
||||||
|
transport = StreamableHTTPServerTransport(
|
||||||
|
mcp_session_id=new_session_id,
|
||||||
|
is_json_response_enabled=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Connect the transport (creates read/write streams) — keep open for session lifetime
|
||||||
|
cm = transport.connect()
|
||||||
|
read_stream, write_stream = await cm.__aenter__()
|
||||||
|
|
||||||
|
async def run_server():
|
||||||
|
await mcp_app.run(read_stream, write_stream, init_opts)
|
||||||
|
|
||||||
|
server_task = asyncio.create_task(run_server())
|
||||||
|
|
||||||
|
# Wrap send to inject Mcp-Session-Id header
|
||||||
|
original_send = send
|
||||||
|
header_injected = False
|
||||||
|
|
||||||
|
async def send_with_session_id(message):
|
||||||
|
nonlocal header_injected
|
||||||
|
if message.get("type") == "http.response.start" and not header_injected:
|
||||||
|
existing = list(message.get("headers", []))
|
||||||
|
if not any(k.lower() == b"mcp-session-id" for k, _ in existing):
|
||||||
|
existing.append((b"mcp-session-id", new_session_id.encode()))
|
||||||
|
message["headers"] = existing
|
||||||
|
header_injected = True
|
||||||
|
await original_send(message)
|
||||||
|
|
||||||
|
# Handle the request (this blocks until the HTTP response is complete)
|
||||||
|
await transport.handle_request(scope, receive, send_with_session_id)
|
||||||
|
|
||||||
|
# Store transport and server task for reuse on subsequent requests
|
||||||
|
sessions[new_session_id] = (transport, server_task)
|
||||||
|
|
||||||
|
|
||||||
# ── Entry point ────────────────────────────────────────────────────────────────
|
# ── Entry point ────────────────────────────────────────────────────────────────
|
||||||
|
|||||||
+201
-11
@@ -831,6 +831,38 @@ body {
|
|||||||
letter-spacing: 0.06em;
|
letter-spacing: 0.06em;
|
||||||
color: var(--ink-dim);
|
color: var(--ink-dim);
|
||||||
}
|
}
|
||||||
|
.user-list-table .user-actions-cell {
|
||||||
|
white-space: nowrap;
|
||||||
|
display: flex;
|
||||||
|
gap: 0.3rem;
|
||||||
|
justify-content: flex-end;
|
||||||
|
}
|
||||||
|
.user-list-table .user-actions-cell button {
|
||||||
|
font-family: var(--font);
|
||||||
|
font-size: 0.55rem;
|
||||||
|
text-transform: uppercase;
|
||||||
|
letter-spacing: 0.06em;
|
||||||
|
padding: 0.2rem 0.4rem;
|
||||||
|
min-height: 1.7rem;
|
||||||
|
cursor: pointer;
|
||||||
|
border: 1px solid var(--border-light);
|
||||||
|
background: var(--input-bg);
|
||||||
|
color: var(--ink);
|
||||||
|
border-radius: 0;
|
||||||
|
-webkit-appearance: none;
|
||||||
|
}
|
||||||
|
.user-list-table .user-actions-cell button:hover {
|
||||||
|
background: var(--hover);
|
||||||
|
border-color: var(--accent);
|
||||||
|
color: var(--accent);
|
||||||
|
}
|
||||||
|
.user-list-table .user-actions-cell button.danger {
|
||||||
|
color: var(--danger);
|
||||||
|
}
|
||||||
|
.user-list-table .user-actions-cell button.danger:hover {
|
||||||
|
border-color: var(--danger);
|
||||||
|
color: var(--danger);
|
||||||
|
}
|
||||||
.user-state-active { color: var(--accent3); }
|
.user-state-active { color: var(--accent3); }
|
||||||
.user-state-inactive { color: var(--danger); }
|
.user-state-inactive { color: var(--danger); }
|
||||||
/* OAuth client list — compact, matches user table density */
|
/* OAuth client list — compact, matches user table density */
|
||||||
@@ -869,8 +901,42 @@ body {
|
|||||||
word-break: break-all;
|
word-break: break-all;
|
||||||
line-height: 1.35;
|
line-height: 1.35;
|
||||||
}
|
}
|
||||||
|
.admin-oauth-row .oauth-client-scopes {
|
||||||
|
color: var(--accent3);
|
||||||
|
font-size: 0.58rem;
|
||||||
|
margin-top: 0.2rem;
|
||||||
|
letter-spacing: 0.04em;
|
||||||
|
}
|
||||||
|
.oauth-scope-row {
|
||||||
|
display: flex;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
gap: 0.75rem 1.25rem;
|
||||||
|
margin: 0.35rem 0 0.5rem;
|
||||||
|
align-items: center;
|
||||||
|
}
|
||||||
|
.oauth-scope-row label {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.35rem;
|
||||||
|
font-size: 0.62rem;
|
||||||
|
text-transform: uppercase;
|
||||||
|
letter-spacing: 0.06em;
|
||||||
|
color: var(--ink);
|
||||||
|
margin: 0;
|
||||||
|
}
|
||||||
|
.admin-oauth-row .oauth-row-actions {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0.35rem;
|
||||||
|
align-items: stretch;
|
||||||
|
min-width: 5.5rem;
|
||||||
|
}
|
||||||
|
.admin-oauth-row .oauth-row-actions button {
|
||||||
|
width: 100%;
|
||||||
|
}
|
||||||
.dialog.dialog-admin button.oauth-revoke,
|
.dialog.dialog-admin button.oauth-revoke,
|
||||||
.dialog.dialog-admin .manage-user-actions button {
|
.dialog.dialog-admin .manage-user-actions button,
|
||||||
|
.dialog.dialog-admin .oauth-row-actions button {
|
||||||
font-family: var(--font);
|
font-family: var(--font);
|
||||||
font-size: 0.62rem;
|
font-size: 0.62rem;
|
||||||
text-transform: uppercase;
|
text-transform: uppercase;
|
||||||
@@ -884,6 +950,7 @@ body {
|
|||||||
border-radius: 0;
|
border-radius: 0;
|
||||||
-webkit-appearance: none;
|
-webkit-appearance: none;
|
||||||
}
|
}
|
||||||
|
.dialog.dialog-admin .oauth-row-actions button.oauth-scopes:hover,
|
||||||
.dialog.dialog-admin .manage-user-actions button:hover {
|
.dialog.dialog-admin .manage-user-actions button:hover {
|
||||||
background: var(--hover);
|
background: var(--hover);
|
||||||
border-color: var(--accent);
|
border-color: var(--accent);
|
||||||
@@ -899,6 +966,39 @@ body {
|
|||||||
border-color: #d4ae5c;
|
border-color: #d4ae5c;
|
||||||
color: var(--bg);
|
color: var(--bg);
|
||||||
}
|
}
|
||||||
|
.dialog.dialog-admin .oauth-row-actions button.oauth-scopes {
|
||||||
|
color: var(--ink);
|
||||||
|
background: var(--input-bg);
|
||||||
|
}
|
||||||
|
.dialog.dialog-admin .oauth-row-actions button.oauth-scopes.active {
|
||||||
|
border-color: var(--accent);
|
||||||
|
color: var(--accent);
|
||||||
|
background: var(--hover);
|
||||||
|
}
|
||||||
|
.oauth-scope-edit {
|
||||||
|
margin-top: 0.45rem;
|
||||||
|
padding: 0.45rem 0.5rem;
|
||||||
|
border: 1px solid var(--border);
|
||||||
|
background: var(--input-bg);
|
||||||
|
}
|
||||||
|
.dialog.dialog-admin .oauth-scope-edit button.primary {
|
||||||
|
font-family: var(--font);
|
||||||
|
font-size: 0.58rem;
|
||||||
|
text-transform: uppercase;
|
||||||
|
letter-spacing: 0.08em;
|
||||||
|
padding: 0.28rem 0.5rem;
|
||||||
|
min-height: 1.85rem;
|
||||||
|
cursor: pointer;
|
||||||
|
border: 1px solid var(--accent);
|
||||||
|
border-radius: 0;
|
||||||
|
background: var(--accent);
|
||||||
|
color: var(--bg);
|
||||||
|
-webkit-appearance: none;
|
||||||
|
}
|
||||||
|
.dialog.dialog-admin .oauth-scope-edit button.primary:hover {
|
||||||
|
background: #d4ae5c;
|
||||||
|
border-color: #d4ae5c;
|
||||||
|
}
|
||||||
.dialog.dialog-admin button.oauth-revoke {
|
.dialog.dialog-admin button.oauth-revoke {
|
||||||
border-color: var(--border-light);
|
border-color: var(--border-light);
|
||||||
color: var(--danger);
|
color: var(--danger);
|
||||||
@@ -1266,12 +1366,17 @@ body {
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
<div id="admin-oauth-sub-create" class="admin-oauth-subpanel">
|
<div id="admin-oauth-sub-create" class="admin-oauth-subpanel">
|
||||||
<p class="dialog-hint">Register OAuth clients for any MCP or OAuth consumer. Public connector URL: <code>/readonly/sse</code>. Many clients auto-register via <code>POST /oauth/register</code>; use this form when you need explicit <code>client_id</code> / <code>client_secret</code> (secret shown once).</p>
|
<p class="dialog-hint">Register OAuth clients for MCP connectors. Public URL: <code>/mcp</code>. Scopes cap what tokens may grant at authorize time.</p>
|
||||||
<label>client name</label>
|
<label>client name</label>
|
||||||
<input type="text" id="admin-oauth-name" value="MCP connector" autocomplete="off">
|
<input type="text" id="admin-oauth-name" value="MCP connector" autocomplete="off">
|
||||||
<label>redirect uri</label>
|
<label>redirect uri</label>
|
||||||
<input type="url" id="admin-oauth-redirect" placeholder="https://your-app.example/oauth/callback" autocomplete="off">
|
<input type="url" id="admin-oauth-redirect" placeholder="https://your-app.example/oauth/callback" autocomplete="off">
|
||||||
<p class="dialog-hint" style="margin-top:0.25rem">Use the callback URL your OAuth client documents.</p>
|
<p class="dialog-hint" style="margin-top:0.25rem">Use the callback URL your OAuth client documents.</p>
|
||||||
|
<label>allowed scopes</label>
|
||||||
|
<div class="oauth-scope-row">
|
||||||
|
<label><input type="checkbox" id="admin-oauth-scope-read" checked> ctxd.read</label>
|
||||||
|
<label><input type="checkbox" id="admin-oauth-scope-write" checked> ctxd.write</label>
|
||||||
|
</div>
|
||||||
<div class="actions" style="justify-content: flex-start; margin: 0.5rem 0;">
|
<div class="actions" style="justify-content: flex-start; margin: 0.5rem 0;">
|
||||||
<button class="primary" type="button" onclick="createOAuthClient()">generate client id / secret</button>
|
<button class="primary" type="button" onclick="createOAuthClient()">generate client id / secret</button>
|
||||||
</div>
|
</div>
|
||||||
@@ -1311,9 +1416,6 @@ body {
|
|||||||
<input type="password" id="admin-password" placeholder="required for new users; leave blank to keep unchanged" autocomplete="new-password">
|
<input type="password" id="admin-password" placeholder="required for new users; leave blank to keep unchanged" autocomplete="new-password">
|
||||||
<div class="manage-user-actions">
|
<div class="manage-user-actions">
|
||||||
<button class="primary" type="button" onclick="saveManagedUser()">save user</button>
|
<button class="primary" type="button" onclick="saveManagedUser()">save user</button>
|
||||||
<button type="button" onclick="setManagedUserActive(false)">inactivate</button>
|
|
||||||
<button type="button" onclick="setManagedUserActive(true)">activate</button>
|
|
||||||
<button type="button" onclick="deleteManagedUser()">delete</button>
|
|
||||||
<button type="button" onclick="clearManageUserForm()">clear</button>
|
<button type="button" onclick="clearManageUserForm()">clear</button>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
@@ -1641,14 +1743,21 @@ function renderUserList() {
|
|||||||
el.innerHTML = '<div class="empty-state">no users</div>';
|
el.innerHTML = '<div class="empty-state">no users</div>';
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
el.innerHTML = `<table class="user-list-table"><thead><tr><th>user id</th><th>name</th><th>role</th><th>state</th></tr></thead><tbody>${
|
el.innerHTML = `<table class="user-list-table"><thead><tr><th>user id</th><th>name</th><th>role</th><th>state</th><th style="text-align:right">actions</th></tr></thead><tbody>${
|
||||||
state.users.map(u => {
|
state.users.map(u => {
|
||||||
const active = u.active !== false;
|
const active = u.active !== false;
|
||||||
|
const uid = escapeHtml(u.user_id || '');
|
||||||
return `<tr>
|
return `<tr>
|
||||||
<td class="user-id">${escapeHtml(u.user_id || '')}</td>
|
<td class="user-id">${uid}</td>
|
||||||
<td>${escapeHtml(u.display_name || '')}</td>
|
<td>${escapeHtml(u.display_name || '')}</td>
|
||||||
<td>${escapeHtml(u.role || '')}</td>
|
<td>${escapeHtml(u.role || '')}</td>
|
||||||
<td class="${active ? 'user-state-active' : 'user-state-inactive'}">${active ? 'active' : 'inactive'}</td>
|
<td class="${active ? 'user-state-active' : 'user-state-inactive'}">${active ? 'active' : 'inactive'}</td>
|
||||||
|
<td class="user-actions-cell">
|
||||||
|
${active
|
||||||
|
? `<button type="button" onclick="userListAction('${uid}', 'inactivate')">inactivate</button>`
|
||||||
|
: `<button type="button" onclick="userListAction('${uid}', 'activate')">activate</button>`}
|
||||||
|
<button type="button" class="danger" onclick="userListAction('${uid}', 'delete')">delete</button>
|
||||||
|
</td>
|
||||||
</tr>`;
|
</tr>`;
|
||||||
}).join('')
|
}).join('')
|
||||||
}</tbody></table>`;
|
}</tbody></table>`;
|
||||||
@@ -1712,6 +1821,29 @@ async function setManagedUserActive(active) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function userListAction(userId, action) {
|
||||||
|
if (!userId) return;
|
||||||
|
if (action === 'delete') {
|
||||||
|
if (!confirm(`Delete user "${userId}"? If delete fails, inactivate instead.`)) return;
|
||||||
|
try {
|
||||||
|
await api('DELETE', `/users/${encodeURIComponent(userId)}`);
|
||||||
|
showToast(`user deleted · ${userId}`, 'success');
|
||||||
|
await loadUsers();
|
||||||
|
} catch (e) {
|
||||||
|
if (e.message !== 'unauthorized') showToast('delete failed: ' + e.message, 'error');
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const active = action === 'activate';
|
||||||
|
try {
|
||||||
|
await api('PATCH', `/users/${encodeURIComponent(userId)}`, { active });
|
||||||
|
showToast(active ? `activated · ${userId}` : `inactivated · ${userId}`, 'success');
|
||||||
|
await loadUsers();
|
||||||
|
} catch (e) {
|
||||||
|
if (e.message !== 'unauthorized') showToast('update failed: ' + e.message, 'error');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
async function deleteManagedUser() {
|
async function deleteManagedUser() {
|
||||||
const userId = document.getElementById('admin-manage-select').value || document.getElementById('admin-user-id').value.trim();
|
const userId = document.getElementById('admin-manage-select').value || document.getElementById('admin-user-id').value.trim();
|
||||||
if (!userId) {
|
if (!userId) {
|
||||||
@@ -1729,6 +1861,17 @@ async function deleteManagedUser() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function oauthScopesFromCheckboxes(readId, writeId) {
|
||||||
|
const scopes = [];
|
||||||
|
if (document.getElementById(readId).checked) scopes.push('ctxd.read');
|
||||||
|
if (document.getElementById(writeId).checked) scopes.push('ctxd.write');
|
||||||
|
return scopes;
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseClientScope(scopeStr) {
|
||||||
|
return new Set((scopeStr || '').split(/\s+/).filter(Boolean));
|
||||||
|
}
|
||||||
|
|
||||||
async function loadOAuthClients() {
|
async function loadOAuthClients() {
|
||||||
const el = document.getElementById('admin-oauth-list');
|
const el = document.getElementById('admin-oauth-list');
|
||||||
el.innerHTML = '<div class="loading">loading oauth clients…</div>';
|
el.innerHTML = '<div class="loading">loading oauth clients…</div>';
|
||||||
@@ -1740,14 +1883,25 @@ async function loadOAuthClients() {
|
|||||||
}
|
}
|
||||||
el.innerHTML = clients.map(c => {
|
el.innerHTML = clients.map(c => {
|
||||||
const cid = c.client_id || '';
|
const cid = c.client_id || '';
|
||||||
|
const scopeSet = parseClientScope(c.scope);
|
||||||
|
const esc = escapeHtml(cid);
|
||||||
return `
|
return `
|
||||||
<div class="admin-oauth-row">
|
<div class="admin-oauth-row" data-client-id="${esc}">
|
||||||
<div class="oauth-client-meta">
|
<div class="oauth-client-meta">
|
||||||
<div class="oauth-client-id">${escapeHtml(cid)}</div>
|
<div class="oauth-client-id">${esc}</div>
|
||||||
<div class="oauth-client-name">${escapeHtml(c.client_name || '')}</div>
|
<div class="oauth-client-name">${escapeHtml(c.client_name || '')}</div>
|
||||||
<div class="oauth-client-uri">${escapeHtml((c.redirect_uris || []).join(', '))}</div>
|
<div class="oauth-client-uri">${escapeHtml((c.redirect_uris || []).join(', '))}</div>
|
||||||
|
<div class="oauth-client-scopes">scopes: ${escapeHtml(c.scope || 'ctxd.read')}</div>
|
||||||
|
<div class="oauth-scope-row oauth-scope-edit" id="oauth-scope-edit-${esc}" style="display:none">
|
||||||
|
<label><input type="checkbox" class="oauth-edit-read" data-cid="${esc}" ${scopeSet.has('ctxd.read') ? 'checked' : ''}> ctxd.read</label>
|
||||||
|
<label><input type="checkbox" class="oauth-edit-write" data-cid="${esc}" ${scopeSet.has('ctxd.write') ? 'checked' : ''}> ctxd.write</label>
|
||||||
|
<button type="button" class="primary" data-save-scopes="${esc}" onclick="saveOAuthClientScopes(this.dataset.saveScopes)">save scopes</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="oauth-row-actions">
|
||||||
|
<button type="button" class="oauth-scopes" data-toggle-scopes="${esc}" onclick="toggleOAuthScopeEdit(this.dataset.toggleScopes, this)">scopes</button>
|
||||||
|
<button type="button" class="oauth-revoke" data-client-id="${esc}" onclick="revokeOAuthClient(this.dataset.clientId)">revoke</button>
|
||||||
</div>
|
</div>
|
||||||
<button type="button" class="oauth-revoke" data-client-id="${escapeHtml(cid)}" onclick="revokeOAuthClient(this.dataset.clientId)">revoke</button>
|
|
||||||
</div>`;
|
</div>`;
|
||||||
}).join('');
|
}).join('');
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
@@ -1757,6 +1911,36 @@ async function loadOAuthClients() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function toggleOAuthScopeEdit(clientId, btn) {
|
||||||
|
const panel = document.getElementById(`oauth-scope-edit-${clientId}`);
|
||||||
|
if (!panel) return;
|
||||||
|
const open = panel.style.display === 'none';
|
||||||
|
panel.style.display = open ? 'flex' : 'none';
|
||||||
|
const row = document.querySelector(`.admin-oauth-row[data-client-id="${CSS.escape(clientId)}"]`);
|
||||||
|
const toggleBtn = btn || row?.querySelector('button.oauth-scopes');
|
||||||
|
if (toggleBtn) toggleBtn.classList.toggle('active', open);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function saveOAuthClientScopes(clientId) {
|
||||||
|
if (!clientId) return;
|
||||||
|
const row = document.querySelector(`.admin-oauth-row[data-client-id="${CSS.escape(clientId)}"]`);
|
||||||
|
if (!row) return;
|
||||||
|
const scopes = [];
|
||||||
|
if (row.querySelector('.oauth-edit-read')?.checked) scopes.push('ctxd.read');
|
||||||
|
if (row.querySelector('.oauth-edit-write')?.checked) scopes.push('ctxd.write');
|
||||||
|
if (!scopes.length) {
|
||||||
|
showToast('select at least one scope', 'error');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
await api('PATCH', `/oauth/clients/${encodeURIComponent(clientId)}`, { scopes });
|
||||||
|
showToast(`scopes updated · ${clientId}`, 'success');
|
||||||
|
await loadOAuthClients();
|
||||||
|
} catch (e) {
|
||||||
|
if (e.message !== 'unauthorized') showToast('scope update failed: ' + e.message, 'error');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
async function revokeOAuthClient(clientId) {
|
async function revokeOAuthClient(clientId) {
|
||||||
if (!clientId) return;
|
if (!clientId) return;
|
||||||
if (!confirm(`Revoke OAuth client "${clientId}"?\n\nPending auth codes and access/refresh tokens for this client are invalidated.`)) return;
|
if (!confirm(`Revoke OAuth client "${clientId}"?\n\nPending auth codes and access/refresh tokens for this client are invalidated.`)) return;
|
||||||
@@ -1777,7 +1961,12 @@ async function createOAuthClient() {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
const client = await api('POST', '/oauth/clients', { client_name: name, redirect_uris: [redirect] });
|
const scopes = oauthScopesFromCheckboxes('admin-oauth-scope-read', 'admin-oauth-scope-write');
|
||||||
|
if (!scopes.length) {
|
||||||
|
showToast('select at least one scope', 'error');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const client = await api('POST', '/oauth/clients', { client_name: name, redirect_uris: [redirect], scopes });
|
||||||
const pre = document.getElementById('admin-oauth-result');
|
const pre = document.getElementById('admin-oauth-result');
|
||||||
pre.style.display = 'block';
|
pre.style.display = 'block';
|
||||||
pre.textContent = [
|
pre.textContent = [
|
||||||
@@ -1787,6 +1976,7 @@ async function createOAuthClient() {
|
|||||||
`authorization_server: ${client.authorization_server || ''}`,
|
`authorization_server: ${client.authorization_server || ''}`,
|
||||||
`client_id: ${client.client_id || ''}`,
|
`client_id: ${client.client_id || ''}`,
|
||||||
`client_secret: ${client.client_secret || ''}`,
|
`client_secret: ${client.client_secret || ''}`,
|
||||||
|
`scope: ${client.scope || scopes.join(' ')}`,
|
||||||
`redirect_uris: ${JSON.stringify(client.redirect_uris || [])}`,
|
`redirect_uris: ${JSON.stringify(client.redirect_uris || [])}`,
|
||||||
].join('\n');
|
].join('\n');
|
||||||
showToast('oauth client created — copy secret from admin panel', 'success');
|
showToast('oauth client created — copy secret from admin panel', 'success');
|
||||||
|
|||||||
@@ -0,0 +1,147 @@
|
|||||||
|
# I Built a Context Server to Fight My ADHD-Induced Context Sprawl
|
||||||
|
|
||||||
|
**How replacing 7 scattered config files with a single source of truth saved my sanity (and my tokens).**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
If you work with AI tools the way I do — Claude Desktop for research, Claude Code for implementation, Codex CLI for reviews, Cursor for quick edits, and Hermes Agent for orchestration — you have a problem you might not have named yet.
|
||||||
|
|
||||||
|
I call it **context sprawl**. And if you have ADHD, it's worse than you think.
|
||||||
|
|
||||||
|
## The Problem
|
||||||
|
|
||||||
|
Every AI harness has its own context file convention:
|
||||||
|
|
||||||
|
| Harness | File |
|
||||||
|
|---------|------|
|
||||||
|
| Hermes | `AGENTS.md` |
|
||||||
|
| Claude Code | `CLAUDE.md` |
|
||||||
|
| Cursor | `.cursorrules` |
|
||||||
|
| Copilot | `.github/copilot-instructions.md` |
|
||||||
|
| Codex CLI | `CODEX.md` |
|
||||||
|
| Windsurf | `.windsurfrules` |
|
||||||
|
| Continue.dev | `.continuerc` |
|
||||||
|
|
||||||
|
When you work on the same project across multiple harnesses, you end up with N copies of the same project context. They drift. One says the build system is Go; another still says Python. One knows about the new OAuth flow; another doesn't. The LLM reads stale instructions and makes confident, wrong decisions.
|
||||||
|
|
||||||
|
For someone with ADHD, this is a specific kind of hell:
|
||||||
|
|
||||||
|
- You **forget** which file you last updated
|
||||||
|
- You **avoid** updating because it means updating 4-6 files
|
||||||
|
- You **lose track** of which harness has the "right" version
|
||||||
|
- You **context-switch** between tools and can't remember what the project actually needs
|
||||||
|
- The friction of maintaining context across tools becomes so high that you **stop maintaining context at all**
|
||||||
|
|
||||||
|
And then every LLM session starts from scratch — re-asking questions you already answered, re-discovering decisions you already made, re-making mistakes you already fixed.
|
||||||
|
|
||||||
|
## The Tipping Point
|
||||||
|
|
||||||
|
I was working on a project called RemoteRig — a multi-camera remote monitoring system built in Go with ESP32-C6 camera nodes. I had context in:
|
||||||
|
|
||||||
|
- `AGENTS.md` (Hermes)
|
||||||
|
- `CLAUDE.md` (Claude Code)
|
||||||
|
- `.cursorrules` (Cursor)
|
||||||
|
- `CODEX.md` (Codex CLI)
|
||||||
|
- A Notion page (for humans, but also where I drafted things)
|
||||||
|
- My own memory of what I'd told each tool (which is the worst place to store anything when you have ADHD)
|
||||||
|
|
||||||
|
I'd update `AGENTS.md` with a new architecture decision, forget to update `CLAUDE.md`, and then Claude Code would suggest something I'd already ruled out. I'd spend the first 20 minutes of every session re-explaining the project. Sometimes I'd update the wrong file and wonder why Codex was still using the old Go module structure.
|
||||||
|
|
||||||
|
The context was sprawled across the filesystem like a hoarder's garage, and I was the only one who knew where anything was — except I didn't, because ADHD.
|
||||||
|
|
||||||
|
## The Solution: CTXD (Context Dossier)
|
||||||
|
|
||||||
|
I built **CTXD** — a single source of truth for project context that every AI harness can read from and write to.
|
||||||
|
|
||||||
|
### Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
Context Dossier (Docker container, port 9091)
|
||||||
|
├── PostgreSQL 16 backend
|
||||||
|
├── Multi-file context per project:
|
||||||
|
│ ├── CONTEXT.MD ← canonical overview, synced as AGENTS.md to repos
|
||||||
|
│ ├── DECISIONS.MD ← architecture decisions, rationale
|
||||||
|
│ ├── RUNBOOKS.MD ← deploy, troubleshoot, operate procedures
|
||||||
|
│ ├── PROMPTS.MD ← project-specific prompts for different harnesses
|
||||||
|
│ └── GLOSSARY.MD ← project-specific terms, acronyms
|
||||||
|
├── OAuth 2.0 authorization server
|
||||||
|
│ ├── Dynamic Client Registration (Claude auto-registers)
|
||||||
|
│ ├── Authorization Code + PKCE
|
||||||
|
│ └── Scopes: ctxd.read, ctxd.write
|
||||||
|
├── Streamable HTTP MCP endpoints:
|
||||||
|
│ ├── /readonly/mcp (read-only, OAuth ctxd.read)
|
||||||
|
│ ├── /write/mcp (read-write, OAuth ctxd.write)
|
||||||
|
│ └── /mcp (internal, shared API key)
|
||||||
|
├── Web UI (per-user login, admin panel)
|
||||||
|
├── Version-checked writes (optimistic concurrency)
|
||||||
|
├── Append-only audit log
|
||||||
|
├── Point-in-time snapshots (auto-rotated)
|
||||||
|
└── Full-text search (PostgreSQL tsvector + GIN)
|
||||||
|
```
|
||||||
|
|
||||||
|
### How it works
|
||||||
|
|
||||||
|
1. **One canonical file per project** — `CONTEXT.MD` is the source of truth. CTXD syncs it to `AGENTS.md` in the project root and creates symlinks (`CLAUDE.md`, `.cursorrules`, `CODEX.md` → `AGENTS.md`). Every harness reads the same file.
|
||||||
|
|
||||||
|
2. **LLMs connect via MCP** — Claude Desktop connects to `https://ctxd.yourdomain.com/readonly/mcp` and auto-discovers OAuth. No manual config, no API keys in URLs. It can read project context, search across all projects, and list individual files.
|
||||||
|
|
||||||
|
3. **LLMs can write context** — With a `ctxd.write`-scoped OAuth token, an LLM can update files with version checking. If two agents try to edit simultaneously, the second one gets a `version_conflict` and knows to re-read, merge, and retry.
|
||||||
|
|
||||||
|
4. **The compiled view** — When an LLM calls `get_project_context("remote-rig")`, it gets all files concatenated into a single document with a metadata header. One call, complete context, no guessing.
|
||||||
|
|
||||||
|
5. **A locked client guide** — There's a `get_client_guide()` MCP tool that returns a locked `LLM-CLIENT.MD` file. Every LLM session starts by reading this guide, which explains how to connect, what tools are available, and what the discipline rules are. The file is immutable — no LLM can modify its own instruction manual.
|
||||||
|
|
||||||
|
### What it solves for ADHD
|
||||||
|
|
||||||
|
- **One place to update** — Change `CONTEXT.MD` once. Every harness gets it via sync or MCP. No more updating 6 files.
|
||||||
|
- **No remembering required** — The LLM reads context at the start of every session. You don't have to remember what you told it last time.
|
||||||
|
- **Audit trail** — Every read, write, and search is logged. When you can't remember what changed (and you can't, because ADHD), the audit log tells you.
|
||||||
|
- **Version checking** — Optimistic concurrency means no silent overwrites. If you and an LLM edit at the same time, the conflict is caught, not lost.
|
||||||
|
- **Search** — Full-text search across all projects. "What did I decide about the camera node?" → search → answer.
|
||||||
|
- **Snapshots** — Automatic point-in-time backups before every edit. Roll back if something goes wrong.
|
||||||
|
|
||||||
|
## The Build
|
||||||
|
|
||||||
|
I built CTXD over a series of focused sessions with Hermes Agent — itself powered by the same MCP infrastructure. Some of the tools that made this possible:
|
||||||
|
|
||||||
|
- **Hermes Agent** for orchestration and the MCP framework
|
||||||
|
- **PostgreSQL 16** as the primary database (migrated from SQLite)
|
||||||
|
- **MCP SDK 1.28** with Streamable HTTP transport (replaced the older SSE pattern)
|
||||||
|
- **OAuth 2.0** with DCR, PKCE, and per-scope access control
|
||||||
|
- **Docker Compose** for deployment — env-driven, no config files needed
|
||||||
|
- **Traefik** for public TLS termination and routing
|
||||||
|
|
||||||
|
The development was iterative and honest. I hit real bugs: the `set -e` in the entrypoint script killed the container because a PostgreSQL schema check returned exit code 1. The `__main__.py` checked for a SQLite file that doesn't exist when using PostgreSQL, causing an infinite re-initialization loop. The FTS5 search query used `?` placeholders instead of `%s` for psycopg. Each one was a "oh, of course" moment — the kind you only get by actually building and deploying.
|
||||||
|
|
||||||
|
## What I Learned
|
||||||
|
|
||||||
|
**Context sprawl is an accessibility problem.** For neurotypical developers, maintaining 6 config files is annoying. For someone with ADHD, it's a barrier that leads to giving up on context entirely — and then every AI interaction starts from zero.
|
||||||
|
|
||||||
|
**The single source of truth pattern isn't just about consistency.** It's about reducing the cognitive load of context management to one action: update `CONTEXT.MD`. That's one decision, one file, one habit — something even an ADHD brain can maintain.
|
||||||
|
|
||||||
|
**OAuth + MCP is the right protocol stack for this.** LLMs don't need API keys in environment variables. They need to discover a resource, authenticate via OAuth, and use standardized tools. The fact that Claude Desktop can connect to CTXD with zero manual configuration — just a URL — is the difference between "I should set this up" and "I'll just re-explain the project again."
|
||||||
|
|
||||||
|
**PostgreSQL matters more than you think.** I started with SQLite, which was fine for one user. But the moment you want concurrent agents writing context, proper backups, and full-text search that actually works — PostgreSQL is the right tool. The migration was a real engineering task (dual-backend support, schema rewrite, FTS5 → tsvector), and it was worth every hour.
|
||||||
|
|
||||||
|
## Where It's Going
|
||||||
|
|
||||||
|
CTXD is now running in production at `ctxd.cubecraftcreations.com`, serving context to Claude Desktop, Hermes, and any OAuth-capable MCP client. It backs:
|
||||||
|
|
||||||
|
- RemoteRig (Go, ESP32-C6)
|
||||||
|
- ExtrudeX (3D printing control system)
|
||||||
|
- Tracehound (network diagnostic tool)
|
||||||
|
- Control Center (homelab dashboard)
|
||||||
|
|
||||||
|
The code is open and documented. There's a static demo at `ctxd-demo` that shows the UI without needing a database. There's a full README, an LLM.txt for automated deployment, and a SKILL.md that LLMs can read to understand how to use the server.
|
||||||
|
|
||||||
|
## The Takeaway
|
||||||
|
|
||||||
|
If you work across multiple AI tools and find yourself re-explaining your project every session — you have context sprawl. If you have ADHD, context sprawl isn't just inefficient; it's a wall between you and productive AI-assisted work.
|
||||||
|
|
||||||
|
Building CTXD was my answer. It turned 6 files into 1, turned "remember what I told Claude" into "Claude reads the context server," and turned "I'll update the context later" into "I update one file and every tool gets it."
|
||||||
|
|
||||||
|
One source of truth. One habit. Zero context sprawl.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*CTXD is open source. The production repo is at [CubeCraft-Creations/CTXD](https://code.cubecraftlabs.com/CubeCraft-Creations/CTXD), with a static demo at [CTXD-Demo](https://code.cubecraftlabs.com/CubeCraft-Creations/CTXD-Demo). Built by Joshua at CubeCraft Creations.*
|
||||||
Reference in New Issue
Block a user