Files
CTXD/README.md
T
overseer 9e85c1b8ec ops: prevent 502 from ctxd without postgres
- entrypoint: wait for DATABASE_URL (CTXD_PG_WAIT_SECONDS) with clear fatal message
- scripts/deploy.sh: postgres healthy then force-recreate ctxd
- compose + README: ban --no-deps ctxd as default; 502 troubleshooting
2026-06-25 13:59:51 +00:00

661 lines
26 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# CTXD — Context Dossier
A single source of truth for multi-harness project context. One canonical `AGENTS.md` per project, served to Claude, Hermes, Codex, Cursor, and any OAuth-capable MCP client via Streamable HTTP.
## Overview
CTXD solves context sprawl: when you work across multiple LLM harnesses (Claude Desktop, Claude Code, Codex CLI, Cursor, Copilot, Hermes), each has its own context file convention (`CLAUDE.md`, `.cursorrules`, `CODEX.md`, etc.). Without a canonical source, these drift independently.
CTXD provides:
- **Multi-file context per project** — `CONTEXT.MD`, `DECISIONS.MD`, `RUNBOOKS.MD`, `PROMPTS.MD`, `GLOSSARY.MD`
- **Compiled view** — all files concatenated with metadata header, served as a single document to agents
- **Sync to repos** — writes `CONTEXT.MD` as `AGENTS.md` + symlinks (`CLAUDE.md`, `.cursorrules`, `CODEX.md``AGENTS.md`)
- **Version-checked writes** — optimistic concurrency with `base_version` to prevent silent overwrites
- **OAuth 2.0 authorization server** — DCR, Authorization Code + PKCE, `ctxd.read` and `ctxd.write` scopes
- **Streamable HTTP MCP** — single-endpoint transport for read-only and write surfaces
- **Web UI** — per-user password login, admin panel, project/file management
- **PostgreSQL backend** — with SQLite fallback for local dev
- **Append-only audit log** — every read, write, create, delete, sync, and search is logged
- **Point-in-time snapshots** — automatic version snapshots with rotation (min 5, max 25 per project)
- **Full-text search** — PostgreSQL `tsvector` with GIN index (FTS5 on SQLite fallback)
## Architecture
```
Context Dossier (container: ctxd, 0.0.0.0:9091)
├── PostgreSQL 16 (container: ctxd-postgres) # Primary DB
├── /data # Config, OAuth state, web sessions
│ ├── ctxd.yaml # Fallback config (env vars take precedence)
│ ├── oauth_state.json # OAuth clients, codes, tokens
│ ├── web_sessions.json # Per-user web UI sessions
│ └── snapshots/ # Point-in-time context backups
├── Streamable HTTP MCP:
│ ├── /mcp (OAuth ctxd.read + ctxd.write; API key on LAN = full tools)
│ ├── /readonly/mcp (alias → same OAuth behavior)
│ └── /oauth/mcp (alias)
├── OAuth Authorization Server:
│ ├── /.well-known/oauth-authorization-server # Discovery
│ ├── /.well-known/oauth-protected-resource # Resource metadata
│ ├── /oauth/register # Dynamic Client Registration
│ ├── /oauth/authorize # Authorization + PKCE
│ └── /oauth/token # Token + refresh
├── Web UI + REST API (/) # Dashboard, admin, projects, files
└── Landing page (public host only) # Themed login form
```
## Quick Start
### Prerequisites
- Docker and Docker Compose
- A reverse proxy with TLS (Traefik, Caddy, nginx) for public exposure
- (Optional) An existing PostgreSQL 14+ instance if not using the bundled one
### 1. Clone and configure
```bash
cd /mnt/ai-storage/projects/ctxd/app
cp .env.example .env
```
Edit `.env` with your values:
```bash
# Database
DATABASE_URL=postgresql://ctxd:your-password@postgres:5432/ctxd
POSTGRES_USER=ctxd
POSTGRES_PASSWORD=your-strong-password
# Server
CTXD_HOST=0.0.0.0
CTXD_PORT=9091
CTXD_HOME=/data
# Auth
CTXD_AUTH_ENABLED=true
CTXD_API_KEY=your-api-key-here # Generate: python3 -c "import secrets; print(secrets.token_urlsafe(32))"
# OAuth
OAUTH_ENABLED=true
OAUTH_ISSUER=https://ctxd.yourdomain.com
OAUTH_APPROVAL_KEY=your-approval-key # Generate: python3 -c "import secrets; print(secrets.token_urlsafe(32))"
```
### 2. Build and start
```bash
# Recommended: build + postgres + recreate ctxd (avoids 502 when PG was never started)
chmod +x scripts/deploy.sh
./scripts/deploy.sh
# Or manual (always include postgres when DATABASE_URL uses host "postgres"):
docker compose up -d
```
This starts:
- `ctxd-postgres` — PostgreSQL 16 (Alpine)
- `ctxd` — CTXD daemon (web UI + MCP + OAuth + REST API)
**After code changes**, use `./scripts/deploy.sh` or `docker compose up -d` — not `docker restart ctxd` alone (old image) and **not** `docker compose up -d --no-deps ctxd` (skips postgres → crash loop → public **502**).
### 3. Verify
```bash
# Health check
curl http://localhost:9091/status
# → {"status": "ok", "db": "/data/ctxd.db"}
# List projects (requires API key)
curl http://localhost:9091/projects -H "Authorization: Bearer your-api-key"
```
### 4. Set admin password
```bash
docker exec ctxd dossier user-set-password admin -p "your-admin-password"
```
> **Shell quoting:** Use double quotes (`"`) if the password contains single quotes (`'`). Use single quotes (`'`) if it contains `$`, backticks, or `!`. The failure mode is shell expansion, not PBKDF2.
### 5. Access the Web UI
- **LAN:** `http://<server-ip>:9091/`
- **Public (via Traefik):** `https://ctxd.yourdomain.com/`
Sign in with `admin` / your password.
## Configuration
### Environment Variables
All config is driven by environment variables. A `ctxd.yaml` file in `/data` can override built-in defaults, but env vars always take precedence.
**Precedence:** env var > `ctxd.yaml` > built-in default
| Variable | Default | Description |
|----------|---------|-------------|
| **Database** | | |
| `DATABASE_URL` | *(empty)* | PostgreSQL connection string. If empty, falls back to SQLite at `$CTXD_HOME/ctxd.db` |
| `POSTGRES_USER` | `ctxd` | PostgreSQL user (for bundled PG container) |
| `POSTGRES_PASSWORD` | `ctxd_local_dev` | PostgreSQL password (for bundled PG container) |
| `POSTGRES_DB` | `ctxd` | PostgreSQL database name (for bundled PG container) |
| **Server** | | |
| `CTXD_HOST` | `0.0.0.0` | Bind address |
| `CTXD_PORT` | `9091` | Listen port |
| `CTXD_HOME` | `~/.ctx` | Data directory (inside container: `/data`) |
| `LOG_LEVEL` | `info` | Uvicorn log level (`debug`, `info`, `warning`, `error`) |
| **Auth** | | |
| `CTXD_AUTH_ENABLED` | `false` | Enable authentication globally |
| `CTXD_API_KEY` | *(empty)* | Shared API key for Hermes/internal MCP + HTTP auth |
| `CTXD_EXTERNAL_READONLY_KEY` | *(empty)* | Legacy `?key=` on read-only MCP (migration only) |
| **OAuth** | | |
| `OAUTH_ENABLED` | `false` | Enable OAuth authorization server |
| `OAUTH_ISSUER` | *(empty)* | Public URL (used in OAuth discovery metadata) |
| `OAUTH_APPROVAL_KEY` | *(empty)* | Fallback approval key for `/oauth/authorize` |
| `OAUTH_APPROVAL_USER_ID` | `admin` | Which user ID to attribute OAuth approvals to |
| `OAUTH_ACCESS_TOKEN_TTL` | `3600` | Access token lifetime in seconds |
| `OAUTH_REFRESH_TOKEN_TTL` | `2592000` | Refresh token lifetime in seconds (30 days) |
| **PostgreSQL (container)** | | |
| `CTXD_PG_WAIT_SECONDS` | `120` | Entrypoint: max wait for DB before exit (when `DATABASE_URL` set) |
| `CTXD_PG_WAIT_INTERVAL` | `2` | Seconds between connection attempts |
| **Web Sessions** | | |
| `WEB_SESSION_TTL` | `604800` | Session cookie lifetime in seconds (7 days) |
| **Snapshots** | | |
| `SNAPSHOT_MIN_KEEP` | `5` | Minimum snapshots retained per project |
| `SNAPSHOT_MAX_KEEP` | `25` | Maximum snapshots before rotation |
### Using an External PostgreSQL
To use an external PostgreSQL instead of the bundled container:
1. Create a database and user on your external PG instance
2. Set `DATABASE_URL` in `.env` to point to it
3. Start only the app (no bundled postgres): `docker compose up -d --scale postgres=0 ctxd`
Ensure `DATABASE_URL` points at your external host (not `postgres`). The entrypoint skips the compose-network wait when the URL is reachable.
### Fallback to SQLite
If `DATABASE_URL` is empty or not set, CTXD falls back to SQLite at `$CTXD_HOME/ctxd.db`. This is useful for local development or single-user deployments that don't need PostgreSQL features.
## MCP Surfaces
CTXD exposes MCP via Streamable HTTP on **`/mcp`** (single public connector):
| Endpoint | Auth | Scope | Tools |
|----------|------|-------|-------|
| `/mcp` | OAuth bearer | `ctxd.read` / `ctxd.write` | Scope-gated read + write tools |
| `/mcp` | Shared API key (LAN/Hermes) | *(full)* | All tools including `get_user_profile`, `auto_generate_tags` |
| `/readonly/mcp`, `/oauth/mcp` | OAuth (aliases) | same as `/mcp` | Backward-compatible URLs |
### Connecting an LLM Client
**Claude Desktop / Claude Web:**
```
Connector URL: https://ctxd.yourdomain.com/mcp
```
Claude auto-discovers OAuth metadata and registers via DCR. Request `scope=ctxd.read ctxd.write` for write access.
**ChatGPT (MCP connector — recommended; no Custom GPT required):**
ChatGPT can attach a **remote MCP server** in **Developer mode** (Plus/Pro and higher tiers for custom connectors). Use the same public URL as Claude — OAuth on `/mcp`, not REST Actions or a shared API key.
1. **Server** — deploy current CTXD and expose the public host (Traefik must route `/mcp` and `/oauth/*`; do not block `/mcp`):
```bash
cd app
./scripts/deploy.sh
```
Smoke: `curl -sS https://ctxd.yourdomain.com/.well-known/oauth-authorization-server | head`
2. **ChatGPT** — Settings → **Connectors** → enable **Developer mode** → **Add connector** (MCP / custom remote) → **Server URL:**
```
https://ctxd.yourdomain.com/mcp
```
Start OAuth and copy ChatGPTs **callback / redirect URL** exactly.
3. **CTXD admin** — Web UI → **admin** → **oauth clients**:
- After DCR, the client may appear automatically; otherwise **create client** with ChatGPTs redirect URI.
- Set **allowed scopes** to **`ctxd.read`** and **`ctxd.write`** (create form or **scopes** → save on an existing row).
4. **Authorize** — complete the browser approval (sign in as CTXD admin in that browser, or use the OAuth approval key). Tokens are capped by the clients allowed scopes.
5. **Chat** — enable the CTXD connector for the conversation. Tools include `list_projects`, `get_project_context`, `search_context`, `get_file`, `update_file`, etc. Call **`get_client_guide`** first in a new session.
**Optional CLI pre-register** (if ChatGPT asks for client credentials before DCR):
```bash
docker exec ctxd dossier oauth-client-create -n "ChatGPT MCP" \
--redirect-uri 'PASTE_CHATGPT_CALLBACK_URL' \
--scope "ctxd.read ctxd.write"
```
| If connect fails | Check |
|------------------|--------|
| **“does not implement OAuth”** | Public host must return **200** JSON (not 502/404) for `/.well-known/oauth-protected-resource/mcp` and `/.well-known/oauth-authorization-server`. Traefik must proxy **all** paths to CTXD `:9091`. Set `OAUTH_ENABLED=true` and `OAUTH_ISSUER=https://ctxd.cubecraftlabs.com`. |
| 502 on well-known URLs | Backend down or wrong Traefik service — fix routing before OAuth can work |
| 404 on `/mcp` | Router not forwarding `/mcp` to CTXD (often a stale `!Path(/mcp)` rule) |
| 401 on `/mcp` | Re-authorize; access token may have expired |
| Redirect mismatch | Callback URL in admin must match ChatGPTs string exactly |
| `invalid_request` on authorize | CTXD requires PKCE **S256** (`code_challenge` + `code_challenge_method=S256`) |
| Stale routes / old code | `./scripts/deploy.sh` or `docker compose up -d --force-recreate ctxd` after postgres is healthy (not `restart` alone; never `--no-deps` unless PG is already up) |
| No write tools | Client scopes include `ctxd.write`; re-authorize after changing scopes |
**Not required for ChatGPT MCP:** Custom GPT, OpenAPI Actions, or `CTXD_API_KEY` in ChatGPT (public path is OAuth). Hermes continues to use LAN `http://<server-ip>:9091/mcp` with the API key.
**Hermes Agent:**
```yaml
# ~/.hermes/config.yaml
mcp_servers:
dossier:
url: http://<server-ip>:9091/mcp
timeout: 30
headers:
Authorization: "Bearer your-api-key"
```
**Other MCP clients (Codex, Cursor, etc.):**
- Register an OAuth client via `POST /oauth/register` with your redirect URI
- Connect to **`/mcp`** with `scope=ctxd.read ctxd.write`
- Use the access token as `Authorization: Bearer <token>`
### MCP Tool Reference
#### Read tools (require `ctxd.read` on `/mcp`)
| Tool | Args | Returns |
|------|------|---------|
| `get_client_guide` | *(none)* | Locked `LLM-CLIENT.MD` guide — **call this first** |
| `list_projects` | *(none)* | All projects with version numbers |
| `get_project_context` | `project_id` | Compiled markdown of all context files |
| `search_context` | `query`, `limit?` | FTS results across all projects |
| `get_project_tags` | `project_id` | Metadata tags for a project |
| `list_files` | `project_id` | All context files in a project |
| `get_file` | `project_id`, `file_path` | Single file with metadata header |
#### Write tools (require `ctxd.write` on `/mcp`)
| Tool | Args | Returns |
|------|------|---------|
| `update_file` | `project_id`, `file_path`, `content`, `base_version` | `{"ok": true, "new_version": N}` or conflict error |
| `set_project_tags` | `project_id`, `tags[]` | `{"ok": true, "tags": [...]}` |
| `sync_to_project` | `project_id` | Writes `CONTEXT.MD` as `AGENTS.md` + symlinks to project root |
### Locked Files
| File | Scope | Protection |
|------|-------|------------|
| `CONTEXT.MD` | All projects | Cannot delete — minimum required file |
| `CONTEXT.MD` | `ctxd-docs` project only | Cannot update or delete |
| `LLM-CLIENT.MD` | `ctxd-docs` project only | Cannot update or delete |
## OAuth
### Scopes
| Scope | Grants |
|-------|--------|
| `ctxd.read` | Read-only MCP tools |
| `ctxd.write` | Write MCP tools (includes read) |
Request both: `scope=ctxd.read ctxd.write`
### Redirect URIs
| Platform | Redirect URI |
|----------|-------------|
| Claude Desktop | `https://claude.ai/api/mcp/auth_callback` |
| ChatGPT (MCP) | Paste callback URL from ChatGPT connector OAuth UI (per connector) |
| Claude Code | `http://localhost:5555/oauth/callback` |
| Codex CLI | `http://localhost:7777/oauth/callback` |
| Custom | Your app's documented OAuth callback |
### Managing OAuth Clients
**Via Admin UI:** `http://<server-ip>:9091/` → sign in as admin → **admin** → **oauth clients** tab
**Via CLI:**
```bash
# Create
docker exec ctxd dossier oauth-client-create -n "Claude Desktop" --redirect-uri https://claude.ai/api/mcp/auth_callback
# List
docker exec ctxd dossier oauth-client-list
# Revoke (invalidates all tokens for that client)
docker exec ctxd dossier oauth-client-revoke ctxd_xxxxxxxx
```
**Via API (admin session or API key):**
```bash
# Create
curl -X POST http://localhost:9091/oauth/clients \
-H "Authorization: Bearer your-api-key" \
-H "Content-Type: application/json" \
-d '{"client_name": "Claude Desktop", "redirect_uris": ["https://claude.ai/api/mcp/auth_callback"]}'
# List
curl http://localhost:9091/oauth/clients -H "Authorization: Bearer your-api-key"
# Revoke
curl -X DELETE http://localhost:9091/oauth/clients/ctxd_xxxxxxxx -H "Authorization: Bearer your-api-key"
```
## Public Exposure (Traefik)
### Router Rule
Route the public host to the backend (include `/mcp` — OAuth protects it):
```yaml
rule: Host(`ctxd.yourdomain.com`)
```
This exposes:
- Landing page (`GET /`)
- Login (`POST /auth/login`, `GET /auth/me`)
- Full Web UI dashboard (all REST API endpoints)
- OAuth (`/oauth/*`, `/.well-known/*`)
- Public MCP (`/mcp` — OAuth read + write)
Hermes uses the same `/mcp` path on LAN with the shared API key (not exposed via public OAuth).
### Landing Page Behavior
- **Not signed in** → themed landing page with login form
- **Signed in** (valid session cookie) → full dashboard
- Cookie: `ctxd_session` (SameSite=Lax, 7-day expiry)
- After login: cookie set + redirect to dashboard
A full Traefik template is at `app/templates/traefik-ctxd-readonly-mcp.yaml` in the `project-context-management` Hermes skill.
## Web UI
### Admin Panel
Sign in as admin → click **admin** in the masthead.
**Tabs:**
1. **oauth clients** — client list (revoke per row) + create client form
2. **users** — user list (id, name, role, active/inactive) + manage users (create, edit, activate, inactivate, delete)
3. **projects** — manage projects (list with remove button, typed-name confirmation for delete)
### Project Files
Each project has multiple context files:
| File | Purpose |
|------|---------|
| `CONTEXT.MD` | Canonical project overview (synced as `AGENTS.md` to repos) |
| `DECISIONS.MD` | Architecture decisions, rationale |
| `RUNBOOKS.MD` | Deploy, troubleshoot, operate procedures |
| `PROMPTS.MD` | Project-specific prompts for different harnesses |
| `GLOSSARY.MD` | Project-specific terms, acronyms |
The compiled view (`get_project_context`) concatenates all files with `## FILENAME` headers and a single metadata block at the top.
## CLI
All commands run inside the container:
```bash
docker exec ctxd dossier <command>
```
### Commands
```bash
# Initialize (auto-runs on first container start)
dossier init
# Projects
dossier project-create <project_id> [--display-name "Name"] [--description "Desc"]
dossier project-list
dossier read <project_id> # Print context to stdout
dossier edit <project_id> # Open in $EDITOR
# Context files
dossier file-list <project_id>
dossier file-read <project_id> <file_path>
# Sync
dossier sync <project_id> [path] # Set sync path and/or sync AGENTS.md
# Search
dossier search "query" # FTS across all projects
# Audit
dossier audit [--limit N]
# Users
dossier user-list
dossier user-create <user_id> --display-name "Name" [--password "pw"]
dossier user-set-password <user_id> -p "password"
# OAuth
dossier oauth-client-create [-n "Name"] [--redirect-uri URI]
dossier oauth-client-list
dossier oauth-client-revoke <client_id>
# Import
dossier import-vault <path> # Import from Obsidian vault
```
## REST API
All endpoints require `Authorization: Bearer <api_key>` or `Authorization: Bearer <session_token>` unless noted.
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/` | Web UI (LAN) or landing page (public host) |
| `GET` | `/status` | Health check (no auth) |
| `POST` | `/auth/login` | Web UI login → session token + cookie |
| `POST` | `/auth/logout` | Revoke session + clear cookie |
| `GET` | `/auth/me` | Current session identity |
| `GET` | `/users` | List users |
| `POST` | `/users` | Create user |
| `PATCH` | `/users/<id>` | Update user (admin) |
| `DELETE` | `/users/<id>` | Delete user (admin) |
| `POST` | `/users/<id>/password` | Set password (admin) |
| `GET` | `/oauth/clients` | List OAuth clients (admin) |
| `POST` | `/oauth/clients` | Register OAuth client (admin) |
| `DELETE` | `/oauth/clients/<id>` | Revoke OAuth client (admin) |
| `GET` | `/projects` | List all projects |
| `POST` | `/projects` | Create a project |
| `DELETE` | `/projects/<id>` | Delete a project (admin) |
| `GET` | `/projects/<id>/context` | Compiled context (all files) |
| `POST` | `/projects/<id>/context` | Update context (legacy single-file) |
| `GET` | `/projects/<id>/files` | List context files |
| `GET` | `/projects/<id>/files/<name>` | Read a single file |
| `POST` | `/projects/<id>/files` | Create a new file |
| `PUT` | `/projects/<id>/files/<name>` | Update a file (version-checked) |
| `DELETE` | `/projects/<id>/files/<name>` | Delete a file |
| `POST` | `/projects/<id>/migrate-files` | Migrate single-context to multi-file |
| `GET` | `/projects/<id>/snapshots` | List snapshots |
| `GET` | `/projects/<id>/tags` | Get metadata tags |
| `POST` | `/projects/<id>/tags` | Set metadata tags |
| `POST` | `/projects/<id>/sync` | Sync CONTEXT.MD as AGENTS.md |
| `POST` | `/projects/<id>/import` | Import raw text as context |
| `GET` | `/search?q=...` | Full-text search |
| `GET` | `/audit?limit=N` | Audit log |
| `GET` | `/.well-known/oauth-authorization-server` | OAuth discovery |
| `GET` | `/.well-known/oauth-protected-resource` | Resource metadata |
| `POST` | `/oauth/register` | Dynamic Client Registration |
| `GET/POST` | `/oauth/authorize` | Authorization endpoint |
| `POST` | `/oauth/token` | Token endpoint |
| `POST/GET/DELETE` | `/readonly/mcp` | Read-only MCP (Streamable HTTP) |
| `POST/GET/DELETE` | `/write/mcp` | Write MCP (Streamable HTTP) |
| `POST/GET/DELETE` | `/mcp` | Internal full MCP (API key only) |
## Backups
### PostgreSQL
```bash
# Backup
docker exec ctxd-postgres pg_dump -U ctxd ctxd > backup_$(date +%Y%m%d).sql
# Restore
cat backup_YYYYMMDD.sql | docker exec -i ctxd-postgres psql -U ctxd ctxd
```
### Snapshots
CTXD automatically takes point-in-time snapshots before each context update. Snapshots are stored as files in `/data/snapshots/<project_id>/` and rotated (min 5, max 25 per project).
## Migrating from SQLite to PostgreSQL
If you started with SQLite and want to move to PostgreSQL:
```bash
# 1. Start PostgreSQL
docker compose up -d postgres
# 2. Run the migration script (reads from /data/ctxd.db, writes to DATABASE_URL)
docker exec ctxd python3 -m ctxd.migrate_sqlite_to_pg
# 3. Set DATABASE_URL in .env and restart
docker compose up -d dossier
```
The migration handles all tables, rebuilds the FTS index, and skips orphaned rows with FK violations.
## Project Structure
```
/mnt/ai-storage/projects/ctxd/
├── .env # Production environment (gitignored)
├── .env.example # Template (committed)
├── .gitignore
├── SKILL.md # LLM client guide (canonical source)
├── README.md # This file
├── data/ # Runtime data (gitignored)
│ ├── ctxd.yaml # Fallback config (env vars take precedence)
│ ├── ctxd.db # SQLite DB (if no DATABASE_URL)
│ ├── pg/ # PostgreSQL data volume
│ ├── oauth_state.json # OAuth clients, codes, tokens
│ ├── web_sessions.json # Web UI sessions
│ └── snapshots/ # Point-in-time backups
└── app/ # Application source
├── docker-compose.yml
├── Dockerfile
├── pyproject.toml
├── .env # Same as root .env (symlinked or copied)
└── src/ctxd/
├── __init__.py
├── __main__.py # CLI/daemon entry point
├── config.py # Env-driven config with yaml fallback
├── db.py # Database layer (PostgreSQL + SQLite)
├── schema.sql # PostgreSQL schema
├── schema_sqlite.sql # SQLite schema (fallback)
├── server.py # ASGI app: HTTP + MCP + OAuth
├── cli.py # CLI commands
├── ui.html # Web UI dashboard
├── landing.html # Public landing page
├── auth_password.py # PBKDF2 password hashing
└── migrate_sqlite_to_pg.py # One-time migration script
```
## Development
### Local Development (SQLite, no Docker)
```bash
cd app
pip install -e ".[mcp]"
export CTXD_HOME=./dev-data
python -m ctxd init
python -m ctxd
# → http://localhost:9091
```
### Rebuilding After Code Changes
```bash
cd app
docker compose build
docker compose up -d --no-build
# Verify:
curl http://localhost:9091/status
docker logs ctxd --tail 20
```
### Key Conventions
- **Metadata headers** are dynamically generated on read, never stored in the DB
- **File paths** are normalized to uppercase with `.MD` extension
- **`CONTEXT.MD`** is the minimum required file — cannot be deleted from any project
- **Version checking** uses `base_version` parameter — mismatches return `409 conflict`
- **Audit log** is append-only at the application layer (every operation is logged)
- **OAuth state** (`oauth_state.json`) and **web sessions** (`web_sessions.json`) are file-based, not in PostgreSQL
## Troubleshooting
### Login fails with "invalid credentials"
```bash
# Reset admin password
docker exec ctxd dossier user-set-password admin -p "new-password"
```
If the password contains special characters, use the quoting that matches:
- `'` in password → use double quotes (`"`)
- `$`, `` ` ``, `!` in password → use single quotes (`'`)
### Login works on LAN but not on public host
The public host (`https://ctxd.yourdomain.com`) requires Traefik to route `/auth/login` and `/auth/me`. Check your Traefik router rule includes all paths (use `!Path(`/mcp`)` to block only the internal MCP).
### MCP connection fails
1. Check OAuth discovery: `curl https://ctxd.yourdomain.com/.well-known/oauth-authorization-server`
2. Check MCP endpoint: `curl -o /dev/null -w '%{http_code}' https://ctxd.yourdomain.com/readonly/mcp` → should be `401` (not `404`)
3. If `404`: Traefik isn't routing `/readonly/mcp` — update the router rule
4. If `401`: auth is working — check OAuth token scope and expiry
### PostgreSQL connection fails
```bash
# Check PG is running
docker compose ps postgres
# Check connection
docker exec ctxd python3 -c "
import os
import psycopg
conn = psycopg.connect(os.environ['DATABASE_URL'])
print('Connected:', conn.info.server_version)
"
# If password mismatch (PG data volume initialized with different password):
docker exec ctxd-postgres psql -U ctxd -c "ALTER USER ctxd PASSWORD 'new-password'"
# Then update .env with the new password
```
### Container keeps restarting / public site 502
```bash
docker compose ps -a
docker logs ctxd --tail 40
```
| Symptom | Cause | Fix |
|---------|--------|-----|
| `failed to resolve host 'postgres'` | `ctxd-postgres` not running (often after `--no-deps ctxd`) | `cd app && docker compose up -d postgres ctxd` or `./scripts/deploy.sh` |
| `Restarting (1)` on `ctxd` only | Same — app up without DB on compose network | Start postgres first; wait for **(healthy)** |
| Cloudflare **502** on public URL | Traefik/backend has no healthy upstream on `:9091` | Fix local `curl http://127.0.0.1:9091/status` first |
The container **entrypoint waits up to 120s** for PostgreSQL when `DATABASE_URL` is set (`CTXD_PG_WAIT_SECONDS` to override). If postgres never appears, logs print an explicit message instead of an immediate opaque crash.
Common other causes:
- `DATABASE_URL` password doesn't match what PG was initialized with
- `OAUTH_ENABLED=true` but `OAUTH_ISSUER` is empty
- Missing `CTXD_API_KEY` when `CTXD_AUTH_ENABLED=true`
## License
MIT