Development

For changing the code rather than running it: what to install, where things live, which gates to pass, and the two workflows that bite people.

Toolchain

Tool	Version	Used by
Docker + Compose	any with docker compose	the whole stack
just	any recent	every recipe below
uv	0.11.12 in the images	api/, ingestor/
pnpm	10.33.0, pinned via packageManager	dashboard/
Python	3.13 minimum; images and CI use 3.14	api/, ingestor/
Node	26 in CI and the dashboard image	dashboard/

uv fetches its own Python. Ruff and pyright target 3.13 in both pyproject.toml files, so code that only works on 3.14 is still linted against 3.13.

Repository Layout

Path	What it is
api/	Flask API, SQLAlchemy models, Alembic migrations, committed openapi.json
ingestor/	Tails Cowrie's JSON log into Postgres
dashboard/	Vue 3 SPA plus the nginx image that serves it
cowrie/	Honeypot persona: config, credentials, filesystem pickle, canned output, egress proxy
postgres/	Role-split init scripts and pg_hba.conf, used by docker-compose.prod.yml
proxy/	nginx config for the production compose file
k8s/	Manifests for one public instance. See Operating This Instance
scripts/	fetch-mmdb.sh and a Cowrie fixture regenerator
docs/	This site

justfile holds the top-level recipes; db.just is a module, reached as just db <recipe>.

Tests

The Python tests use a honeywatch_test database on the dev Postgres container. just db test-init creates it; just db test-reset recreates it.

Command	What it does
just test	The full local gate, below
just test-api	pytest in api/
just test-ingestor	pytest in ingestor/
just pnpm test	vitest run; add --coverage for the thresholds
just pnpm typecheck	vue-tsc --noEmit
just pnpm e2e	Playwright plus axe, previewing the built SPA on 4173
just check	just test then just openapi-check

just test runs just db test-init; ruff check, ruff format --check and pyright in both Python trees; both pytest suites; then pnpm install --frozen-lockfile, lint, build and test --coverage.

just test is not the whole of CI

It skips pnpm typecheck and pnpm e2e, and both are CI gates. Run them before you push. Trivy and CodeQL are CI-only too.

pnpm lint is eslint with zero warnings allowed plus an ASCII check that rejects any non-ASCII byte in src/**. Symbols go in as HTML entities or inline SVG. Generated client code is exempt.

The OpenAPI Drift Gate

api/openapi.json and the generated client in dashboard/src/api/generated/ are both committed and must match what the routes produce.

just api-openapi      # dump the spec
just openapi-regen    # the above, then regenerate the TS client
just openapi-check    # regen, then fail if either output drifted

The gate stages files

just openapi-check runs git add on those two paths and then git diff --cached --exit-code, so it can never pass while you have uncommitted changes in them, including changes you made on purpose. Commit the regenerated files first, then run the check.

CI runs the same sequence on the api leg of the Python matrix, so a spec mismatch reports as a failure of python (api). A separate spec-lint job runs Spectral against the spec.

The spec served at /api/v1/openapi.json is built once at boot and cached, so a running container keeps serving the old spec after a route change until you restart it.

Migrations

docker compose up runs api-migrate before the API and ingestor start, so the dev database is always at head. The recipes below are for working on migrations themselves.

Command	What it does
just db upgrade	alembic upgrade head, without recreating containers
just db downgrade	One revision back
just db heads	The head revision in api/alembic/versions/, not what the database has applied
just db revision "message"	Autogenerate from the model diff
just db revision-empty "message"	Empty revision, for data migrations and CONCURRENTLY work
just db shell	psql into the dev database

All of these run docker compose exec against the dev stack, so bring it up first. Read an autogenerated revision before applying it; Alembic emits drops for anything it does not recognise.

Index migrations wrap CREATE INDEX CONCURRENTLY in an autocommit block so the ingestor can keep writing. On a large table that looks like a hung migration. Let it finish.

Seeding Data

just seed        # synthetic data, WIPES every table first
just seed-live   # drips new sessions so the live feed and map move

Both run on the host through uv and connect to localhost:5433. just attack opens an SSH session against the local Cowrie instead; use a credential userdb.txt accepts, such as root / changeme, for a successful login.

Geolocation needs the MaxMind databases: just fetch-mmdb with MAXMIND_ACCOUNT_ID and MAXMIND_LICENSE_KEY in .env. To re-run enrichment over existing IPs, use the container:

docker compose exec ingestor python -m src.reclassify_geoip   # --ip <addr> for one

The host recipe just reclassify-geoip needs GEOIP_DATA_DIR=data (relative to ingestor/, where it runs), otherwise it looks for the files at the container path and finds nothing.

CI

ci.yml runs on pushes and pull requests to master, plus a weekly cron.

Job	What it runs
python (api), python (ingestor)	ruff check, ruff format --check, pyright, pytest against a Postgres service. The api leg also runs the OpenAPI gate
spec-lint	Spectral against api/openapi.json
dashboard	install --frozen-lockfile, lint, typecheck, build, test --coverage, Playwright e2e
image-scan	Builds each image and scans it with Trivy; exits 1 on any CRITICAL or HIGH

The weekly cron runs only image-scan, so a red build on a Sunday is usually a newly published CVE in a base image, not your code.

codeql.yml analyses Python and TypeScript. docs.yml builds this site with mkdocs build --strict when docs/** or mkdocs.yml change. A page missing from the nav is only an INFO message, so add new pages to mkdocs.yml yourself.

Releases are triggered by a v*.*.* tag. just bump-version <version> writes the version into both pyproject.toml files and package.json, re-locks, and opens a commit.