GovSpend (Initial Vertical Slice)

Monorepo scaffold for a contracts/grants/loans spending intelligence dashboard.

Stack (current scaffold)

• Backend: Hono on Cloudflare Workers (apps/api)
• Realtime: Durable Object skeleton for WebSocket + SSE fan-out
• Frontend: React + BlueprintJS (apps/web)
• State: Zustand + TanStack Query
• Shared models: zod + TypeScript (packages/core)
• Datasource layer: extensible adapters + live USAspending ingestion adapters (packages/sources)
• Infra: Alchemy skeleton under infra/
• Target domain: contracts.caulk.lol
• Geolocation policy: best-effort recipient mapping with explicit fallback metadata
• Repo visibility requirement: agentic/govspend must be public

No CLI has been added in this slice.

Repository Structure

govspend/
  apps/
    api/       # Hono Worker + Durable Object realtime skeleton
    web/       # React dashboard shell + controls + globe placeholder
  packages/
    core/      # shared zod schemas and domain types
    sources/   # source adapter interfaces + usaspending stubs
  infra/       # Alchemy IaC skeleton and notes
  docs/
    ARCHITECTURE.md

Prerequisites

• Node.js 22+
• pnpm 10+

Setup

cd /home/pawdabot/.clawdbot/workspace/repos/govspend
pnpm install

Development

API (Cloudflare Worker)

pnpm --filter @govspend/api dev

Runs on Wrangler default local endpoint (typically http://127.0.0.1:8787).

Web App

pnpm --filter @govspend/web dev

Runs on http://127.0.0.1:5173.

Set VITE_API_BASE_URL if API is not on the default local URL.

Workspace Commands

pnpm dev        # api
pnpm dev:web    # web
pnpm typecheck
pnpm test
pnpm build

CI (Forgejo Actions)

Forgejo Actions workflow is configured at:

• .forgejo/workflows/ci.yml

Current CI job runs on the docker runner and executes:

1. pnpm install --frozen-lockfile
2. pnpm typecheck
3. pnpm test
4. pnpm build

The workflow is kept Forgejo-schema-compatible (minimal job keys) and uses the runner's native Node/pnpm toolchain instead of actions/setup-node. If pnpm is missing, CI falls back to npx pnpm@10.18.3 (ephemeral, no corepack enable, no global install) and uses a temp npm cache to avoid read-only filesystem failures.

Current test baseline: Vitest schema coverage in packages/core/src/schemas.test.ts.

Triggers: push to main, pull requests, and manual workflow_dispatch.

Data Model Highlights (current scaffold)

• Shared location schema includes address + coordinates + fallback flags.
• Geolocation fallback behavior is documented and tracked per record (fallbackUsed, fallbackReason).
• Contract records include both winner location views:
  • winnerLocations.hq
  • winnerLocations.primary
• UI state includes a contract location mode selector (primary / hq / both) and fallback marker toggle.

Current Endpoints

• GET / (API index)
• GET /health
• GET /v1/meta
• GET /v1/sources
• GET /v1/awards?type=contract|grant|loan&limit=50&offset=0
• GET /v1/ingest/summary
• GET /v1/ingest/runs?limit=20
• GET /v1/ingest/cursors?source=usaspending
• POST /v1/ingest/run (manual ingest trigger)
• POST /v1/ingest/import (token required; external relay import path)
• GET /v1/ingest/debug/upstream?dataset=contract (worker-side upstream probe with attempt diagnostics)
• GET /v1/realtime/sse?clientId=dashboard&replay=25
• GET /v1/realtime/ws?replay=25
• POST /v1/realtime/publish

Ingest behavior (current slice)

• Hourly cron trigger configured in apps/api/wrangler.toml (0 * * * *).
• Manual ingest trigger exposed from the web UI (Run ingest button).
• Ingest state is now storage-backed through a Storage layer:
  • D1 when DB binding is present
  • in-memory fallback when DB is not bound
• Raw ingest batch archives are written to R2 when RAW_ARCHIVE binding is present.
• USAspending adapters call live /api/v2/search/spending_by_award/ with bounded pagination (configurable via ingest env vars).
• Endpoint fallback is supported via USASPENDING_ENDPOINTS (comma-separated candidates, tried in order).
• When Worker egress is unstable, you can bypass Worker→USAspending by importing from a host relay using POST /v1/ingest/import + pnpm ingest:relay.
• Ingest tracks dataset cursors (lastSuccessfulRunAt, lastSeenModifiedAt, optional carry-forward nextPageToken) in storage for incremental windows.
• Incremental runs now upsert records by ID (instead of replacing whole datasets each run).
• USAspending adapter calls use retry + timeout guards to reduce long hangs on transient upstream TLS/5xx failures.
• Records are normalized into contracts/grants/loans schemas with best-effort state-centroid fallback coordinates when source lat/lon are unavailable.

D1 migration baseline is included at:

• apps/api/migrations/0001_init.sql

External relay ingest (fallback)

1. Set a Worker secret:
   • wrangler secret put INGEST_IMPORT_TOKEN
2. Run relay ingest from a host that can reach USAspending:
   • INGEST_IMPORT_TOKEN=... pnpm ingest:relay

Optional env vars for relay:

• GOVSPEND_API_BASE_URL (default: https://govspend-api.caulk.workers.dev)
• INGEST_LOOKBACK_DAYS, INGEST_PAGE_SIZE, INGEST_MAX_PAGES
• USASPENDING_TIMEOUT_MS, USASPENDING_MAX_RETRIES

Infrastructure Notes

See:

• infra/README.md
• infra/alchemy.config.ts
• docs/ARCHITECTURE.md

TODO Highlights

• Add robust historical backfill strategy (current incremental cursor flow is bounded and single-window).
• Move D1 schema management entirely to Alchemy-managed lifecycle and generated outputs.
• Implement address-level geocoding pipeline (current fallback is state-centroid best effort).
• Populate/render contract winner hq and primary location layers from live data.
• Add real globe/WebGL rendering.
• Fill in production-grade Alchemy resources for Cloudflare deployment.

Forgejo Remote

Primary repo: https://git.caulk.lol/agentic/govspend (public)

cd /home/pawdabot/.clawdbot/workspace/repos/govspend
git remote set-url origin git@git.caulk.lol:agentic/govspend.git
git push -u origin main