Argus

Argus runs persistent agents that monitor changing public-web topics. Each agent searches on a schedule, retrieves and verifies evidence, updates a living database, and keeps a traceable history of what changed and why.

The product has three surfaces that use the same API:

• a Next.js workspace for creating pursuits, reviewing runs, browsing records, resolving review items, and approving external actions;
• a Hono API that owns authentication, pursuits, runs, retrieval, evidence, records, reviews, actions, budgets, and settings;
• a Node.js worker that runs the queues for discovery, retrieval, processing, finalization, reconciliation, and approved webhook delivery.

What Argus does

1. Create a pursuit with a topic, objective, cadence, source rules, and the shape of the records you want to maintain.
2. Argus proposes a structured contract for the pursuit. Review and activate it before the agent is allowed to run.
3. A scheduled or manual run searches for new public-web material.
4. The worker retrieves bounded source content, normalizes it, and stores the raw object privately for a limited time.
5. The model triages sources, extracts contract fields, and verifies every accepted value against source evidence.
6. Argus creates or updates records, preserves versions and evidence, and routes uncertain findings to review instead of silently publishing them.
7. The run finalizes with status, cost, retries, queue activity, and trace context. Approved webhook actions can execute afterward with a stable idempotency key.

Architecture

The full architecture is captured as an editable Excalidraw diagram:

Open the Argus technical pipeline diagram

The application is a small Compose-based system:

Next.js web workspace
        │
        ▼
Hono API ─────── PostgreSQL  durable product state
   │  │
   │  ├────────── Redis + BullMQ  reconstructable work
   │  └────────── MinIO/S3        bounded raw retrieval objects
   │
   └───────────── Node worker     discovery, processing, actions

API and worker ── OpenTelemetry ── self-hosted SigNoz

PostgreSQL is the source of truth for pursuits, runs, records, evidence, approvals, and actions. Redis carries jobs that can be reconstructed. Raw retrieval objects are private and expire after seven days; durable evidence stores only the excerpts, locators, hashes, and safe source metadata needed for the product history.

Why it is different

Most monitoring tools stop at search results or send a notification when a page changes. Argus keeps a durable, reviewable record of the entire research loop:

• what the pursuit asked for;
• which source and retrieval produced a finding;
• which evidence supports each field;
• whether the finding is new, an update, unchanged, a duplicate, rejected, or still a lead for review;
• how long the run took and what it cost;
• what external action was proposed, approved, rejected, or executed.

The result is a living database with an agent behind it, rather than a stream of disconnected alerts.

Main features

Agent fleet

The Fleet workspace shows active pursuits, run state, cadence, next wake-up, recent activity, budgets, and health. Pursuits can be created, configured, paused, resumed, archived, or run immediately.

Contract-driven setup

Pursuit setup turns a plain-language objective into an explicit record contract: fields, identity rules, source behavior, cadence, and action policy. The contract is reviewed before activation, which keeps later extraction and record updates bounded by an understandable schema.

Retrieval and evidence

Production retrieval uses Bright Data for search, public-page retrieval, browser retrieval, and configured social datasets. Retrieved content is treated as untrusted input, checked for supported types and size limits, and kept separate from model instructions.

Every agent-populated accepted value must point to evidence. HTML uses exact quotes, JSON uses pointers, PDFs use page locators, and feeds use item indexes.

Living records and review

Argus compares new candidates with existing identity fields and stores record versions instead of overwriting history. Conflicts, incomplete evidence, ambiguous identity, and meaningful changes can become review items. The Reviews workspace lets a human resolve those cases without losing the source trail.

Action gate

Webhook actions are proposed separately from agent reasoning. The Approvals workspace shows the exact payload, destination host, policy version, and retry state. An authenticated user must approve an action before delivery; endpoint versions are immutable and requests use signing plus idempotency keys.

Argus does not auto-approve consequential actions and does not include Slack, calendar, CRM, or email OAuth integrations in v1.

Agent monitoring with SigNoz

SigNoz is part of the operations product, not just infrastructure telemetry. Argus traces the path from discovery through retrieval, extraction, verification, persistence, scheduling, approval, and action execution. The repository includes dashboards for:

• agent fleet health, run rate, delays, queue depth, retries, budgets, and approval backlog;
• finding pipeline outcomes, evidence, verification, provider failures, duration, tokens, and cost.

The four tracked alert definitions cover missed schedules, repeated run or retrieval failures, budget limits, and a stalled queue backlog. SigNoz stays private by default. Set EXPOSE_SIGNOZ_LINKS=true only when the operator deployment is configured to expose trace links in the Argus UI.

Provider modes

PROVIDER_MODE=fixture is the default. It uses deterministic local data and does not spend provider credits, so the UI, queues, records, actions, and telemetry can be exercised safely.

PROVIDER_MODE=replay uses the sanitized checked-in capture for a repeatable end-to-end demonstration without network calls.

PROVIDER_MODE=live selects Bright Data retrieval and the configured OpenAI structured-model client. Live mode requires provider credentials and the corresponding Bright Data zones and dataset IDs. Demo and release commands require CONFIRM_LIVE_SPEND=yes before making live calls.

Repository layout

apps/
  web/             Next.js workspace, pages, components, and browser tests
  api/             Hono API, engine, providers, queues, workers, and tests

packages/
  contracts/       Shared Zod request and response contracts
  ui/              Shared Argus UI styles

config/
  provider-pricing.json  Provider cost catalog

infra/signoz/
  dashboards/      Agent fleet and finding pipeline dashboards
  alerts/          Alert definitions
  pours/           Pinned self-hosted SigNoz deployment

docs/
  architecture.*  Product architecture diagram and source
  plan/            Canonical product, API, schema, security, and test plans
  archive/         Historical proposals and background notes

compose.yml        PostgreSQL, Redis, MinIO, API, worker, and web services
Dockerfile         Node/Bun production-shaped images
scripts/           Local stack, database, demo, telemetry, and release tools

Local development

Prerequisites

• Node.js 24
• Bun 1.3.14
• Docker and Docker Compose with at least 4 GB available

The default local stack uses loopback ports 3000, 5432, 6379, 8787, 9000, 9001, and 8080 when SigNoz is enabled.

Install and start

cp .env.example .env
bun install --frozen-lockfile
scripts/stack.sh deps
bun run db:migrate
bun run dev

The web app runs at http://localhost:3000. The API runs at http://localhost:8787; its liveness and readiness endpoints are /health/live and /health/ready.

Fixture mode is safe for local work. Replace the blank values in .env only when using real providers or a shared deployment. Never commit .env or live credentials.

Run the Compose application stack

scripts/stack.sh app

This builds and starts the production-shaped API, worker, and web services alongside PostgreSQL, Redis, and MinIO. For a local observability deployment:

scripts/stack.sh observability

For the operator/demo mode, which publishes the SigNoz UI at http://localhost:8080:

scripts/stack.sh demo

Useful stack commands:

scripts/stack.sh status
scripts/stack.sh health
scripts/stack.sh logs api
scripts/stack.sh logs worker
scripts/stack.sh stop

Named volumes are retained when the stack stops.

PostgreSQL backup and restore

Backups and restore checks use a new database and never overwrite the active database:

mkdir -p ../argus-backups
scripts/postgres.sh backup ../argus-backups/argus.dump
scripts/postgres.sh restore ../argus-backups/argus.dump argus_restore_check

Environment configuration

Start with .env.example. The important configuration groups are:

• database and queues: DATABASE_URL, POSTGRES_*, REDIS_URL;
• object storage: OBJECT_STORAGE_*, MINIO_*;
• authentication and safety: PUBLIC_APP_URL, ENCRYPTION_KEY;
• providers: OPENAI_API_KEY, OPENAI_MODEL, BRIGHTDATA_*;
• provider behavior: PROVIDER_MODE, ENABLED_SOCIAL_PLATFORMS;
• cost controls: PRICING_CATALOG_PATH, MAX_PURSUIT_BUDGET_USD, DEPLOYMENT_DAILY_CEILING_USD;
• observability: OTEL_EXPORTER_OTLP_ENDPOINT, SIGNOZ_UI_URL, EXPOSE_SIGNOZ_LINKS;
• live rehearsal safety: CONFIRM_LIVE_SPEND and the LIVE_CANARY_* callback/webhook variables.

Configuration is validated at API startup. Invalid URLs, missing secrets, unsupported provider modes, malformed dataset IDs, and unsafe limits fail closed instead of producing a partially configured worker.

Backend API map

Core route groups include:

• GET /health/live and GET /health/ready for process and dependency health;
• /auth for signup, login, logout, and the current session;
• /v1/pursuits for pursuit setup, activation, cadence, pause/resume, and runs;
• /v1/records for living records, versions, evidence, and history;
• /v1/reviews for unresolved and resolved review items;
• /v1/actions for webhook configuration, previews, approvals, rejection, and execution history;
• /settings for authenticated deployment and provider settings;
• /v1/webhooks/brightdata/:token for asynchronous provider callbacks.

The browser uses same-origin cookie authentication. Mutating requests require the configured origin, and every protected lookup verifies ownership through its parent resource.

Testing and release

Web build and static checks:

bun run build
bun run typecheck
bun run lint

Backend, browser, and fixture checks:

bun run test
bun run test:integration
bun run test:e2e
bun run test:browser
bun run fixtures:validate

Operations and release checks:

bun run test:deployment
bun run test:signoz
bun run security:scan
bun run security:secrets
bun run gate:c
bun run demo:rehearse

bun run gate:c covers the Build C operations hard gate. It validates the foundation, deployment, SigNoz assets, and security checks. The full release gate adds the A/B/C gates, integration checks, repository and history scans, and the explicitly confirmed live canary:

bun run gate:release

Seed safe fixture telemetry when inspecting the dashboards manually:

bun run telemetry:seed

Demo flow

The repeatable demo uses fixture or replay mode and follows the real product path:

1. Start scripts/stack.sh demo.
2. Run scripts/stack.sh health.
3. Seed telemetry with bun run telemetry:seed.
4. Create and activate a pursuit in the web app.
5. Run it, inspect the record and evidence, and resolve any review item.
6. Approve one webhook action in Approvals.
7. Open both SigNoz dashboards and the linked trace.
8. Stop the stack with scripts/stack.sh stop.

bun run demo:rehearse performs the setup and prints the remaining UI steps. It reports the planned request count before a live rehearsal and refuses to continue without explicit confirmation.

Troubleshooting

• Run scripts/stack.sh status and scripts/stack.sh health first.
• Use scripts/stack.sh logs api or scripts/stack.sh logs worker for service errors.
• Confirm Node 24, Docker capacity, and the values copied from .env.example.

Documentation

The plan index is the canonical engineering reference. It links the API, PostgreSQL schema, retrieval, record lifecycle, workflow, actions, SigNoz, security, limits, and test contracts.

The repository intentionally does not include Kubernetes, a Python/FastAPI rewrite, LangGraph, pgvector, collaborative roles, automatic consequential actions, or separate microservices. Those are outside the v1 scope.